Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify differences between Dataset.open() and Dataset.open_remote() #1216

Open
markbader opened this issue Nov 21, 2024 · 0 comments
Open
Labels

Comments

@markbader
Copy link
Contributor

Context

  • Affected library: webknossos

When accessing a Layer or a Mag with Layer._ensure_layer() or MagView._ensure_mag() the Dataset.open() method is always used. Even if the given path is a zarr path. This seems to work when the Dataset is public but leads to trouble when it isn't.

Expected Behavior

We should either ensure that Dataset.open() is not used for remote paths or check which kind of path it is and set the according authentication tokens of the webknossos context.

Current Behavior

Dataset.open_remote() currently only matches this pattern for remote paths: http(s)://<webknossos_url>/datasets/<orga_id>/<dataset_name>/view?token=<sharing_token>
Opening a Zarr streaming path does not work with open_remote.

Dataset.open() is able to open zarr streaming paths like: http(s)://<webknossos_url>/data/zarr/<orga_id>/<dataset_name>/... but it does not set the required x-auth-token for accessing not public zarr Datasets.

I think we should discuss which method should be capable of which kind of Dataset and add required assertions to avoid errors. Additionally, we might take a look at all places where UPaths are created from strings and add required authentication tokens etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant