Support for OME-ZARR input #5
Conversation
There was a problem hiding this comment.
I updated spark and some packages as they were in BigStitcher-Spark and replaced the assembly plugin with the shade plugin. With the assembly plugin the full jar didn't work properly
There was a problem hiding this comment.
Hi @cgoina, I do not think changes to this class are necessary. We simply need to use Views.hyperslice() in the SparkRSFISH class ...
There was a problem hiding this comment.
I had it like that initially but IMO these made the code in SparkRSFISH cleaner
There was a problem hiding this comment.
Why?
What I do not like is that statements like if ( d < 3 ) and return min.length > 3 ? Arrays.copyOf(min, 3) : min; do some "magic" depending on the input and are very fragile. What happens when I select a block size in c & t > 1?
I would like to keep it the way it is because a Block is a unit that computation is done on, which is either 2D or 3D.
| final JavaSparkContext sc = new JavaSparkContext( sparkConf ); | ||
|
|
||
| // only 2 pixel overlap necessary to find local max/min to start - we then anyways load the full underlying image for each block | ||
| final ArrayList< Block > blocks = Block.splitIntoBlocks( interval, blockSize ); |
There was a problem hiding this comment.
@cgoina I would simply add a loop over the dimensions >= 3 here (c,t), i.e. creating blocks for each c,t ... after using Views.hyperslice(img,c,t)
There was a problem hiding this comment.
the code also does not seem to work on 2d anymore (e.g. xyct)? We need to parse the OME-ZARR properties to do this properly. See here how to do that:
There was a problem hiding this comment.
I agree the correct way is to parse the OME attributes
There was a problem hiding this comment.
@StephanPreibisch and @krokicki I added support for reading attributes from OME-ZARR metadata and reverted all changes from aws subpackage - I think that can be deleted now because I N5Utils should handle the URI but I have not tested it with AWS or GC.
There was a problem hiding this comment.
Why?
What I do not like is that statements like if ( d < 3 ) and return min.length > 3 ? Arrays.copyOf(min, 3) : min; do some "magic" depending on the input and are very fragile. What happens when I select a block size in c & t > 1?
I would like to keep it the way it is because a Block is a unit that computation is done on, which is either 2D or 3D.
|
|
||
| public class SparkRSFISH implements Callable<Void> | ||
| { | ||
| public enum StorageType { N5, ZARR, HDF5 } |
There was a problem hiding this comment.
n5-universe already has a type for this so I am using that
| private int maxChannel = -1; | ||
|
|
||
| @Option(names = {"--excluded-channels"}, split = ",", description = "Comma-separated list of (0-based) channel values") | ||
| private Set<Integer> excludedChannels; |
There was a problem hiding this comment.
this option can help us with out EASI-FISH pipeline to exclude the DAPI channel from spot extraction. Before it was not an issue because the channels were subsets but now it could be helpful since channels are a dimension in the nd-array. Timepoints not an issue yet because so far we only have one timepoint
|
@krokicki It has never been merged. I cleaned/updated the branch. and i think it is ready to be merged |
4 participants
@StephanPreibisch Changes to be able to read and process 5D OME-ZARR inputs.