dimension_separator and n5 key reversal

[d-v-b]

N5 requires that the dimension ordering of chunk keys be reversed. I.e., for a chunk key foo/bar/0.1.2, invert_chunk_coords will return foo/bar/2.1.0.

Pre-2.81, chunk keys used two delimiters: "/" delimits the hierarchy, and "." delimits the chunk indices. After 2.8.1, chunk keys can take the form foo/bar/0/1/2, and suddenly there is no distinction between hierarchy and chunk index delimiters. "/"-separated chunk keys can arise if the array's chunk store has a key_separator property: Array._chunk_key(). N5Store has a dimension_separator property but not a key_separator property, so for N5Store chunk keys will use the default "." separator, and invert_chunk_coords will work fine.

But FSStore uses the key_separator property for determining how to write keys to storage. Arrays created using FSStore will read the key_separator property of the store adapt chunk keys accordingly. Thus, attempts to extend FSStore to work for N5 (where key_separator is "/") will run into issues where chunk keys look like foo/bar/0/1/2 and it becomes impossible for invert_chunk_coords to parse this into (array_prefix, chunk_coordinate)

Zooming out, it seems like a leaky abstraction if the format of chunk keys changes as a function of the implementation details of the storage layer. Chunk keys should have a consistent, parseable format, and it is the job of the store to convert chunk keys into the names of objects in storage.

[d-v-b]

actually it looks like the key change was in this PR: zarr-developers/zarr-python#718

[joshmoore]

My best guess at the moment is that normalization is being called multiple times. I've certainly noticed the need to carefully set normalize_keys on the FSStore contructor elsewhere, but hadn't yet tracked down what was going on.

cc: @martindurant

[joshmoore]

@d-v-b : is this still an issue?

[d-v-b]

I transferred this issue from zarr-python to n5py.