Set storage

Show code cell content Hide code cell content

!lamin close
!yes | lamin delete pgtest

import lamindb_setup as ln_setup

Set storage

Show code cell content Hide code cell content

import laminci

!docker stop pgtest && docker rm pgtest
pgurl = laminci.db.setup_local_test_postgres()

ln_setup.init(storage="./storage1", db=pgurl)

ln_setup.settings.storage.root

Local storage:

ln_setup.set.storage("./storage_2")

ln_setup.settings.storage.root

ln_setup.settings.storage.root_as_str

Show code cell content Hide code cell content

from pathlib import Path

assert ln_setup.settings.storage.root_as_str == f"{Path.cwd()}/storage_2"

Cloud storage:

ln_setup.set.storage("s3://lamindb-ci")

ln_setup.settings.storage.root

See an overview:

Show code cell content Hide code cell content

assert ln_setup.settings.storage.is_cloud
assert ln_setup.settings.storage.root_as_str == "s3://lamindb-ci"
assert ln_setup.settings.storage.region == "us-west-1"
# root.fs contains the underlying fsspec filesystem
assert (
    ln_setup.settings.storage.root.fs.cache_regions  # set by lamindb to True for s3 by default
)

You can set any additional fsspec filesystem arguments for cloud storage, such as profile or cache_regions (for s3 only), for example:

ln_setup.set.storage("s3://lamindb-ci", cache_regions=False)

Show code cell content Hide code cell content

# test cache_regions
assert not ln_setup.settings.storage.root.fs.cache_regions
# test setting storage not by owner
ln_setup.login("testuser2@lamin.ai", key="goeoNJKE61ygbz1vhaCVynGERaRrlviPBVQsjkhz")
ln_setup.set.storage("./storage_3")
assert ln_setup.settings.storage.root_as_str == f"{Path.cwd()}/storage_3"
!docker stop pgtest && docker rm pgtest

Currently not possible: setting storage for SQLite instance

If you try to set the storage for an sqlite instance, an error message is returned:

assert ln_setup.set.storage("mydata_storage_2") == "set-storage-failed"