Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add flag to specify number of frames per-chunk in intermediate hdf5 data #367

Merged
merged 5 commits into from
Jul 25, 2024
Merged

Add flag to specify number of frames per-chunk in intermediate hdf5 data #367

merged 5 commits into from
Jul 25, 2024

Commits on Jul 25, 2024

  1. Add flag to write hdf5 intermediate data chunked

    @yousefmoazzam
    yousefmoazzam committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    d9b689d View commit details
    Browse the repository at this point in the history

  2. Change chunk intermediate flag to specify frames per-chunk 

    Setting the `--frames-per-chunk` flag to 0 will write the intermediate
data as contiguous, and any value > 0 will be the number of frames
(projections or sinograms) per-chunk.

Thus, this preserves the ability to turn chunking on/off when writing
intermediate data, but also adds the ability to choose the number of
frames per-chunk.
    @yousefmoazzam
    yousefmoazzam committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    5ac4ec3 View commit details
    Browse the repository at this point in the history

  3. Pass frames per chunk as param to data saving method 

    The method function now is standalone and has no reliance on the
framework's global variable `FRAMES_PER_CHUNK` to know the number of
frames per-chunk to write.

Instead, the associated method wrapper passes the framework's global
variable `FRAMES_PER_CHUNK` through to the method function.
    @yousefmoazzam
    yousefmoazzam committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    f2d41a5 View commit details
    Browse the repository at this point in the history

  4. Add tests for writing chunked intermediate data

    @yousefmoazzam
    yousefmoazzam committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    307ee62 View commit details
    Browse the repository at this point in the history

  5. Use 1 frame per chunk if flag value exceeds slicing dim length 

    If the value provided for the `--frames-per-chunk` flag is larger than
the length of the data's slicing dimension, then the number of frames
per-chunk will be set to 1 (and will be logged in the verbose logfile).

This is to avoid `h5py` errors with trying to create a chunk whose shape
exceeds the boundaries of the data being written.
    @yousefmoazzam
    yousefmoazzam committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    aef83c9 View commit details
    Browse the repository at this point in the history