## lamindb.Storage | class lamindb.Storage(root: str, *, description: str | None = None, space: Space | None = None, host: str | None = None) | class lamindb.Storage(*db_args) Bases: "SQLRecord", "TracksRun", "TracksUpdates" Storage locations of artifacts such as local directories or S3 buckets. A storage location is either a directory (local or a folder in the cloud) or an entire S3/GCP bucket. Parameters: * **root** -- "str" The root path of the storage location, e.g., ""./mydir"", ""s3://my-bucket"", ""s3://my-bucket/myfolder"", ""gs://my-bucket/myfolder"", ""/nfs/shared/datasets/genomics"", ""/weka/shared/models/"", ... | * **description** -- "str | None = None" An optional | description. | * **space** -- "Space | None = None" A space to restrict access | permissions to the storage location. | * **host** -- "str | None = None" For local storage locations, a | globally unique identifier for the physical machine/server hosting the storage. This distinguishes storage locations that may have the same local path but exist on different servers, e.g. ""my-institute-cluster-1"", ""my-server-abcd"". See also: "lamindb.core.Settings.storage" Current default storage location of your compute session for writing artifacts. "StorageSettings" Storage settings. Keep artifacts local in a cloud instance Avoid storing artifacts in the cloud, but keep them on local infrastructure. # Examples When you create a LaminDB instance, you can **configure its default storage** location via "--storage": lamin init --storage /path/to/myfolder # or "s3://my-bucket/myfolder", "gs://my-bucket/myfolder", ... This is how you mount **new storage locations** in your LaminDB instance: # on AWS ln.Storage(root="s3://our-bucket/our-folder").save() # on Google Cloud ln.Storage(root="gs://our-bucket/our-folder").save() # on a local file system ln.Storage(root="/dir/our-shared-dir", host="our-server-123").save() -[ On MinIO or Cloudflare R2 ]- You can also mount storage locations on MinIO or Cloudflare R2: # on MinIO ln.Storage(root="s3://our-bucket/our-folder?endpoint_url=http://our-minio.example.com:9000").save() # on Cloudflare R2 ln.Storage(root="s3://our-bucket/our-folder?endpoint_url=https://our-account-id.r2.cloudflarestorage.com").save() **Query a storage location object** that you can pass to "Artifact()": storage = ln.Storage.get(root="s3://our-bucket/our-folder") View **all storage locations** used in your LaminDB instance: ln.Storage.to_dataframe() View the **default storage location** of your LaminDB instance: ln.setup.settings.storage Temporarily **switch to another storage location**: ln.settings.storage = "/dir/our-shared-dir" # or "s3://our-bucket/our-folder", "gs://our-bucket/our-folder", ... If you're operating in "keep-artifacts-local" mode (Keep artifacts local in a cloud instance ): ln.settings.local_storage = "/dir/our-other-shared-dir" # Notes A storage location is managed by at most one LaminDB instance: the location’s *managing instance*. Some locations are not managed with LaminDB and, hence, do not have a managing instance. -[ Managing access to storage locations ]- You can manage access through LaminHub's fine-grained access management or through AWS policies that you attach to your S3 bucket. To enable access management via LaminHub, head over to "lamin.ai/{account}/infrastructure" and click the green button that says "Connect S3 bucket". Read more: Access permissions. By default, a storage location inherits the access permissions of its managing instance. If you want to further restrict access to a storage location, you can move it into a space: space = ln.Space.get(name="my-space") storage = ln.Storage.get(root="s3://my-storage-location") storage.space = space storage.save() If you don't want to store data in the cloud, you can use local storage locations: Keep artifacts local in a cloud instance . -[ Writable vs. read-only storage locations ]- The "instance_uid" field of "Storage" defines its managing LaminDB instance. Only if a storage location's "instance_uid" matches your current instance's "uid" ("ln.settings.instance_uid"), you can write to it. All other storage locations are read-only in your current instance. Here is an example (source). [image] Some storage locations are not managed by any LaminDB instance, hence, their "instance_uid" is "None". -[ What is the ".lamindb/" directory inside a storage location? ]- It stores all artifacts that are ingested through "lamindb", indexed by the artifact "uid". This means you don't have to worry about renaming or moving files, as this all happens on the database level. Existing artifacts are typically stored in hierarchical structures with semantic folder names. Instead of copying such artifacts into ".lamindb/" upon calls of "Artifact("legacy_path").save()", LaminDB registers them with the semantic "key" representing the relative path within the storage location. These artifacts are marked with "artifact._key_is_virtual = False" and treated correspondingly. There is only a single ".lamindb/" directory per storage location. -[ What should I do if I want to bulk migrate all artifacts to another storage location? ]- Currently, you can only achieve this manually: 1. Copy or move artifacts into the desired new storage location 2. Adapt the corresponding record in the {class}`~lamindb.Storage` registry by setting the "root" field to the new location 3. If your storage location is connected to the hub, you also need to update the storage record on the hub | property host: str | None | Host identifier for local storage locations. Is "None" for locations with "type != "local"". A globally unique user-defined host identifier (cluster, server, laptop, etc.). property path: UPath Path. Uses the ".root" field and converts it into a "Path" or "UPath". uid: str Universal id, valid across DB instances. root: str Root path of storage (cloud or local path). | description: str | None | A description. type: StorageType Can be "local" vs. "s3" vs. "gs". Is auto-detected from the format of the "root" path. | region: str | None | Storage region for cloud storage locations. Host identifier for local storage locations. | instance_uid: str | None | The writing instance. Only the LaminDB instance with this "uid" can write to this storage location. This instance also governs the access permissions of the storage location unless the location is moved into a space. artifacts: Artifact Artifacts contained in this storage location. save(*args, **kwargs) Save the storage record. delete(permanent=None) Delete the storage location. This errors in case the storage location is not empty. Unlike other "SQLRecord"-based registries, this does *not* move the storage record into the trash. Parameters: | **permanent** ("bool" | "None", default: "None") -- "False" | raises an error, as soft delete is impossible. Return type: "None"