Fixes for variable-based encoding in backends without step support

[franzpoeschel]

Close #1481

While working on the fix, I noticed a few other related issues. This PR contains workflow fixes for using variable-based encoding in backends that don't support IO steps:

• Throw an error when trying to create a second IO step in a non-supporting backend
• Use READ_LINEAR access in variableBasedSingleIteration test (READ_ONLY also works when there are no steps involved, but prints a warning in variable-based encoding)
• In READ_LINEAR mode, the constructor does not parse anything, parsing must be actively triggered. Series::readIterations() is a misleading name for this, so introduce Series::parseBase() as an alias.
• I noticed that Series::readIterations() returns a new iterator instance for each call. Since this iterator modifies the state of the Series, it only makes sense to have a single instance of it. Store that instance internally in the Series.
• file-based iteration encoding must not write snapshot attribute at Series::close() time, but at Iteration::close() instead
• Do not write the snapshot attribute during initialization of an iteration
• Ensure that each call to AbstractIOHandler::flush() sets m_lastFlushSuccessful (otherwise, a failed next iteration might leak into the output during the destructor run)

[guj]

Thanks Franz for the updates!

Just curious, what happens READ_LINEAR is applied to an h5/json file?

In terms of allowing any format to write one step and throw error on second steps when writing variable based. It is totally logical. From a client prospective, I would prefer openPMD-api to throw error when someone init series with h5/json + variable based, instead of wait until the second step.
The openPMD side of the code would be simpler too.

[franzpoeschel]

Just curious, what happens READ_LINEAR is applied to an h5/json file?

READ_LINEAR can be used with all backends, but it does not do any parsing in the Series constructor. The parsing has to be triggered by Series::readIterations(), or alternatively Series::parseBase() as introduced by this PR.
If the backend uses steps, only the first step will be parsed now (unlike READ_ONLY which will parse the entire file even when it uses steps), otherwise the entire file is parsed.

Especially, using READ_LINEAR and for(auto iteration : series.readIterations()) … is a workflow (the only one) that is supported by all backends.

In terms of allowing any format to write one step and throw error on second steps when writing variable based. It is totally logical. From a client prospective, I would prefer openPMD-api to throw error when someone init series with h5/json + variable based, instead of wait until the second step. The openPMD side of the code would be simpler too.

Writing a single step with variable-based encoding is a useful feature for data that has no iterations. We throw no error here because this is a workflow that we explicitly and intentionally support.

[guj]

Just curious, what happens READ_LINEAR is applied to an h5/json file?

READ_LINEAR can be used with all backends, but it does not do any parsing in the Series constructor. The parsing has to be triggered by Series::readIterations(), or alternatively Series::parseBase() as introduced by this PR. If the backend uses steps, only the first step will be parsed now (unlike READ_ONLY which will parse the entire file even when it uses steps), otherwise the entire file is parsed.

Especially, using READ_LINEAR and for(auto iteration : series.readIterations()) … is a workflow (the only one) that is supported by all backends.

In terms of allowing any format to write one step and throw error on second steps when writing variable based. It is totally logical. From a client prospective, I would prefer openPMD-api to throw error when someone init series with h5/json + variable based, instead of wait until the second step. The openPMD side of the code would be simpler too.

Writing a single step with variable-based encoding is a useful feature for data that has no iterations. We throw no error here because this is a workflow that we explicitly and intentionally support.

Thanks for the explanation. Maybe this behavior can be in the docs for future reference.

[franzpoeschel]

Maybe this behavior can be in the docs for future reference.

It is documented here

[ax3l]

Thank you for the report, updates and fixes!! 🚀 ✨

[ax3l]

Ah, we should have squashed these commits for easier backporting :) next time, hope I got it right :)