Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Write efficient out-of-core 3D connected components labeling #37

Open
jamesavery opened this issue Apr 25, 2023 · 1 comment
Open

Write efficient out-of-core 3D connected components labeling #37

jamesavery opened this issue Apr 25, 2023 · 1 comment

Comments

@jamesavery
Copy link
Owner

ndimage label crashes on full resolution images due to lack of memory.
@jamesavery
Copy link
Owner Author

Bonus points for nice hierarchical algorithm.

carljohnsen added a commit that referenced this issue Mar 19, 2024
@carljohnsen
#37 Added out-of-core CPU parallel connected components
dd57486
carljohnsen added a commit that referenced this issue Mar 19, 2024
@carljohnsen
#37 unordered_set::contains() is a spicy new C++20 🌶️
a49067a
carljohnsen added a commit that referenced this issue Mar 19, 2024
@carljohnsen
#37 Bugfixes related to the relayed shapes
f07d308
carljohnsen added a commit that referenced this issue Mar 19, 2024
@carljohnsen
#37 Added functions for finding and filtering the largest component
c4173c2
carljohnsen added a commit that referenced this issue Mar 20, 2024
@carljohnsen
#37 apply_renamings also needed total_shape
b3fd971
carljohnsen added a commit that referenced this issue Mar 20, 2024
@carljohnsen
#37 Fixed bug with incorrect number of labels being sent down
79b045d
carljohnsen added a commit that referenced this issue Mar 20, 2024
@carljohnsen
#37 store and load partial now handles unaligned requests
94f1abd
carljohnsen added a commit that referenced this issue Mar 20, 2024
@carljohnsen
#37 Handle chunks with different sizes in apply_renamings
264ff1b
carljohnsen added a commit that referenced this issue Mar 20, 2024
@carljohnsen
#37 Handle chunks of different sizes in filter_largest
db4940c
carljohnsen added a commit that referenced this issue Mar 20, 2024
@carljohnsen
#37 Handle chunks of different sizes in largest_component
3ac4562
carljohnsen added a commit that referenced this issue Mar 20, 2024
@carljohnsen
#37 0600 now calls the combined largest_connected_components function
279b2f8
carljohnsen added a commit that referenced this issue Mar 22, 2024
@carljohnsen
#37 Restructured the I/O functions for connected components
2b17f38
carljohnsen added a commit that referenced this issue Mar 22, 2024
@carljohnsen
#37 bug fixes for the connected components I/O functions
f5314c9
carljohnsen added a commit that referenced this issue Mar 22, 2024
@carljohnsen
#37 Removed warnings
50ef8a5
carljohnsen added a commit that referenced this issue Mar 22, 2024
@carljohnsen
#37 Reworked renaming LUT synchronization
3aae551
carljohnsen added a commit that referenced this issue Mar 22, 2024
@carljohnsen
#37 Reintroduced omp pragmas
0ffda3b
carljohnsen added a commit that referenced this issue Apr 8, 2024
@carljohnsen
#37 Moved debug printing and verbose printing into if guards
633fdc8
carljohnsen added a commit that referenced this issue Apr 8, 2024
@carljohnsen
#37 Improved the performance of count_sizes
b40749f
carljohnsen added a commit that referenced this issue Apr 8, 2024
@carljohnsen
#37 count_sizes returned pure 0s
844df3e
carljohnsen added a commit that referenced this issue Apr 8, 2024
@carljohnsen
#37 filter_largest now runs faster
5e9c1c2
carljohnsen added a commit that referenced this issue Apr 8, 2024
@carljohnsen
#37 Update file loading function to use direct file reading
b2b516b
carljohnsen added a commit that referenced this issue Apr 8, 2024
@carljohnsen
#37 Small changes to 0600 to correctly call the right function
6ff319f
carljohnsen added a commit that referenced this issue Apr 18, 2024
@carljohnsen
#37 Changed 1500 to also use the new connected components to allow fo…
3e6cd5a 
…r a 1x blood mask
@carljohnsen carljohnsen mentioned this issue Apr 18, 2024
Closed
carljohnsen added a commit that referenced this issue Apr 25, 2024
@carljohnsen
#37 Changed connected_components to use ssd as the intermediate storage
c72878e
carljohnsen added a commit that referenced this issue Aug 7, 2024
@carljohnsen
#37 Removed if and assert from apply renaming
ed415e1
carljohnsen added a commit that referenced this issue Aug 7, 2024
@carljohnsen
#37 Made get_mappings sequential, as it's faster that way.
018b50b
carljohnsen added a commit that referenced this issue Aug 7, 2024
@carljohnsen
#37 Added in-memory merge_labeled_chunks function
ce9b285
carljohnsen added a commit that referenced this issue Aug 7, 2024
@carljohnsen
#37 Reduced the complexity of the relabel function, by writing a more…
270903b 
… efficient merge_labels function
carljohnsen added a commit that referenced this issue Aug 7, 2024
@carljohnsen
#37 Added pybind functions for the new merge_labeled_chunks function
1aaac04
carljohnsen added a commit that referenced this issue Aug 7, 2024
@carljohnsen
#37 Switched from multiprocessing.Pool to multiprocessing.ThreadPool
6164f51
carljohnsen added a commit that referenced this issue Aug 7, 2024
@carljohnsen
#37 Added the new merge_labeled_chunks function to the connected_comp…
1abcff7 
…onents header.
carljohnsen added a commit that referenced this issue Aug 8, 2024
@carljohnsen
#37 Changed the internal queue of labels to check in merge_labels fro…
dd935fa 
…m being a vector to a stack
carljohnsen added a commit that referenced this issue Aug 8, 2024
@carljohnsen
#37 Rewrote the relabel function, so it's faster. It now assembles th…
ddf5df8 
…e renaming lut as it traverses each set of connected planes.

Furthermore, rather than keeping a lut for each chunk at each level of the adjacency tree, it now only keeps one per chunk, which is updated accordingly.
carljohnsen added a commit that referenced this issue Sep 4, 2024
@carljohnsen
#37 Fixed relabel function to correctly traverse the connection graph.
1bd5da5 
Labels are also consecutive again.
carljohnsen added a commit that referenced this issue Sep 4, 2024
@carljohnsen
#37 Updated the recount labels function, which ensures that the label…
8823579 
…s in the LUT are consecutive.
carljohnsen added a commit that referenced this issue Sep 4, 2024
@carljohnsen
#37 Updated the in-memory version of connected components to use the …
8c4c93f 
…new helper functions.
carljohnsen added a commit that referenced this issue Sep 4, 2024
@carljohnsen
#37 Updated the on-disk version of connected components to match the …
1f5475d 
…in-memory version.
carljohnsen added a commit that referenced this issue Sep 4, 2024
@carljohnsen
#37 Updated the function signatures of the connected components funct…
62c4576 
…ions.
carljohnsen added a commit that referenced this issue Oct 8, 2024
@carljohnsen
#37 Fixed bug when running connected components where the last chunk …
3619457 
…was larger on the z axis than the rest.
carljohnsen added a commit that referenced this issue Oct 8, 2024
@carljohnsen
#37 Added merge_labeled_chunks overload, which handles unalignment th…
eb4f661 
…rough shape supplied by the caller.
carljohnsen added a commit that referenced this issue Oct 8, 2024
@carljohnsen
#37 Added python helper function for finding largest connected component
4370377
carljohnsen added a commit that referenced this issue Oct 8, 2024
@carljohnsen
#37 Updated 0600 to use the new largest_cc helper function
8b88b12
carljohnsen added a commit that referenced this issue Oct 8, 2024
@carljohnsen
#37 Updated 0750 to use the new largest_cc helper function
d039049
carljohnsen added a commit that referenced this issue Oct 8, 2024
@carljohnsen
#37 Fixed bug in largest_cc_of where the verbose argument for plottin…
afc1b69 
…g was positional, when it should be named.
carljohnsen added a commit that referenced this issue Oct 8, 2024
@carljohnsen
#37 mask name added to internal helper function plots
0c32387
carljohnsen added a commit that referenced this issue Oct 8, 2024
@carljohnsen
#37 Updated 1500 to use the new connected components helper functions.
31eb61c
@carljohnsen carljohnsen mentioned this issue Oct 8, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jamesavery