Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add utility methods to parallelize over blocks #53

Merged
merged 4 commits into from
Apr 21, 2018
Merged

Add utility methods to parallelize over blocks #53

merged 4 commits into from
Apr 21, 2018

Conversation

hanslovsky
Copy link
Member

This commits adds

  • ParallelizeOverBlocks.parallelize - parallelize a task over blocks and return futures
  • ParallelizeOverBlocks.parallelizeAndWait - parallelize a task over blocks and wait for the result
    with various signatures. The three signatures are distingished by the way the blocks for parallelization are specified:
  1. Interval interval, int[] blockSize
  2. long[] min, long[] max, int[] blockSize
  3. List< Interval > blocks

This is a common pattern that I have used multiple times and I think that imglib2-algorithm is the adequate package for such methods.

BlockOffsets.java contains helper methods for generating lists of blocks from intervals and block sizes.

@hanslovsky
Copy link
Member Author

#40 is also relevant for this, in particular for ParallelizeOverBlocks.parallelizeAndWait.

@hanslovsky
Copy link
Member Author

@imglib/admins please comment

@axtimwalde
Copy link
Member

Sounds reasonable but in practice I always needed more. We should have a look at this class
https://github.com/saalfeldlab/hot-knife/blob/master/src/main/java/org/janelia/saalfeldlab/hotknife/util/Grid.java
and add what's missing (i.e. cropping the grid intervals to input interval boundaries and related things).

 - Rename `BlockOffsets` to `Grids` to be closer to imglib2 namespace (`CellGrid` etc)
 - Add tests (detected a bug for min != 0)
 - new convenience methods for:
   - retrieving complete intervals (instead of just min)
   - retrieving complete intervals and position within cell grid
@hanslovsky
Copy link
Member Author

  • Rename BlockOffsets to Grids to be closer to imglib2 namespace (CellGrid etc)
  • Add tests (detected a bug for min != 0)
  • new convenience methods for:
    • retrieving complete intervals (instead of just min)
    • retrieving complete intervals and position within cell grid
      Unfortunately, it is not possible to use CellGrid here because we have to consider a non-zero min, unless we have specific implementations of CreateAndCropBlockToFitInterval and GetGridCoordinates for zero min and non-zero min, respectively. This does not sound reasonable to me, though.

We should still consider moving the Cells class into imglib2 core, though, as discussed in person.

@hanslovsky
Copy link
Member Author

I had a more common use case like this:

void forEachOffset( long[] min, long[] max, int[] step, Consumer< long[] > doAtOffset );

This can be generalized to allow for use with RandomAcess, for example. I will add this before the PR gets merged.

@hanslovsky
Copy link
Member Author

Any comments/objections against a merge? I would like to go ahead and merge this.

@hanslovsky
Copy link
Member Author

@imglib/admins if there are no further objections or comments, I will go ahead and merge this PR.

@hanslovsky hanslovsky merged commit 1dc40f8 into imglib:master Apr 21, 2018
@hanslovsky hanslovsky deleted the parallelize-over-blocks branch April 21, 2018 02:29
@hanslovsky hanslovsky restored the parallelize-over-blocks branch April 23, 2018 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants