Add parameter sweep functionality #380

Implements the simple property getters.

The motivation behind this change is that it would be useful to be able to use method wrapper objects (implementors of `MethodWrapper`) to execute methods in both: - the high throughput runner `TaskRunner` - the parameter sweep runner `ParamSweepRunner` The `DataSetBlock` class contains code that is mostly compatible with a parameter sweep run. However, one incompatibility is the `DataSetBlock.data.setter()` method, which places a constraint on the data involving the slicing dimension of the data. In parameter sweep runs there is only a single process being run, and so the concept of slicing dimension isn't relevant. Given that `DataSetBlock` contains a fair amount of logic that *is* usable in a parameter sweep run, after various iterations of testing out what could work well, the logic in `DataSetBlock` that is usable across both high throughput runs and parameter sweep runs has been extracted into a separate `BaseBlock` class. A high-level overview of the organisation of block-related functionality now is: - the various functionalites needed for a block type to be processable by implementors of `MethodWrapper` have been organised into three separate protocols: `BlockData`, `BlockTransfer`, and `BlockIndexing` - `BaseBlock` provides "typical implementations" for `BlockData` and `BlockTransfer`, which are reused by `DataSetBlock` via inheritance - thus, `BaseBlock` contains the code for "typical" implementations of methods that are common to blocks across both high throughput runs and parmeter sweep runs - `DataSetBlock` contains only the code specific to processing blocks in high throughput runs - in the future, the "typical implementations" contained in `BaseBlock` can be reused by a new block type that will be used in parameter sweep runs

… + no of sweep Before, the array to hold all sweep results was inferred from the `single_shape` value that was given when the writer object was first created. This had the implicit assumption that the shape of a single sweep result would be known at the time the writer object is created. This assumption holds when the method being executed in the sweep doesn't change the shape of the data during processing (most methods adhere to this). However, this assumption doesn't hold when the method executed in the sweep does change the shape of the data during processing (for example, reconstruction methods). Therefore, the writer now infers the shape of a single sweep result when the first block to be written has been given. This can then can be used to infer the shape of the array to hold all sweep results for both cases (when the method being executed in the sweep does and doesn't change the shape of the data).

Instead of creating multiple copies of the same wrapper with different values for the parameter to sweep over, use a single wrapper and update the parameters before each execution to reflect the new sweep value to be used.

Instead of requiring that the caller of the sweep runner creates the `Stages` object, have the sweep runner generate that itself from the pipeline object.

By default, the YAML loader used by the UI layer to load YAML pipeline files is now the modified version of `yaml.SafeLoader` that additionally handles the `!Sweep` and `!SweepRange` tags.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add parameter sweep functionality #380

Add parameter sweep functionality #380

Commits on Jul 18, 2024