Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Opt-in support for unknown ADIOS2 engines #1652

Merged
merged 4 commits into from
Aug 1, 2024
Merged

Opt-in support for unknown ADIOS2 engines #1652

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d2349c8
Initial untested attempt at supporting unsupported engines
franzpoeschel Jul 18, 2024
8533aeb
Documentation
franzpoeschel Jul 31, 2024
61b53d3
Add an env var
franzpoeschel Aug 1, 2024
18f24ef
Small formatting issue in docs
ax3l Aug 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/source/backends/adios2.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,10 @@ Exceptions to this are the BP3 and SST engines which require their endings ``.bp

For file engines, we currently leverage the default ADIOS2 transport parameters, i.e. ``POSIX`` on Unix systems and ``FStream`` on Windows.

.. tip::

Use the ``adios2.engine.treat_unsupported_engine_as`` :ref:`JSON/TOML parameter <backendconfig-adios2>` for experimentally interacting with an unsupported ADIOS2 engine.
ax3l marked this conversation as resolved.
Show resolved Hide resolved

Steps
-----

Expand Down Expand Up @@ -81,6 +85,7 @@ environment variable default description
``OPENPMD_ADIOS2_HAVE_METADATA_FILE`` ``1`` Online creation of the adios journal file (``1``: yes, ``0``: no).
``OPENPMD_ADIOS2_NUM_SUBSTREAMS`` ``0`` Number of files to be created, 0 indicates maximum number possible.
``OPENPMD_ADIOS2_ENGINE`` ``File`` `ADIOS2 engine <https://adios2.readthedocs.io/en/latest/engines/engines.html>`_
``OPENPMD_ADIOS2_PRETEND_ENGINE`` *empty* Pretend that an (unknown) ADIOS2 engine is in fact another one (also see the ``adios2.pretend_engine`` :ref:`parameter <backendconfig-adios2>`).
``OPENPMD2_ADIOS2_USE_GROUP_TABLE`` ``0`` Use group table (see below)
``OPENPMD_ADIOS2_STATS_LEVEL`` ``0`` whether to generate statistics for variables in ADIOS2. (``1``: yes, ``0``: no).
``OPENPMD_ADIOS2_ASYNC_WRITE`` ``0`` ADIOS2 BP5 engine: 1 means setting "AsyncWrite" in ADIOS2 to "on". Flushes will go to the buffer by default (see ``preferred_flush_target``).
Expand Down
8 changes: 8 additions & 0 deletions docs/source/details/backendconfig.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,14 @@ Explanation of the single keys:

* ``adios2.engine.type``: A string that is passed directly to ``adios2::IO:::SetEngine`` for choosing the ADIOS2 engine to be used.
Please refer to the `official ADIOS2 documentation <https://adios2.readthedocs.io/en/latest/engines/engines.html>`_ for a list of available engines.
* ``adios2.engine.pretend_engine``: May be used for experimentally testing an ADIOS2 engine that is not explicitly supported by the openPMD-api.
Specify the actual engine via ``adios2.engine.type`` and use ``adios2.engine.pretend_engine`` to make the ADIOS2 backend pretend that it is in fact using another engine that it knows.
Some advanced engine-specific features will be turned off indiscriminately:

* The Span API will use a fallback implementation
* ``PerformDataWrite()`` will not be used, even when specifying ``adios2.engine.preferred_flush_target = "disk"``.
* Engine-specific parameters such as ``QueueLimit`` will not be set by default.
* No engine-specific filename extension handling will be executed, the extension specified by the user is taken "as is".
* ``adios2.engine.access_mode``: One of ``"Write", "Read", "Append", "ReadRandomAccess"``.
Only needed in specific use cases, the access mode is usually determined from the specified ``openPMD::Access``.
Useful for finetuning the backend-specific behavior of ADIOS2 when overwriting existing Iterations in file-based Append mode.
Expand Down
1 change: 1 addition & 0 deletions include/openPMD/IO/ADIOS/ADIOS2Auxiliary.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ namespace adios_defaults
using const_str = char const *const;
constexpr const_str str_engine = "engine";
constexpr const_str str_type = "type";
constexpr const_str str_treat_unsupported_engine_like = "pretend_engine";
constexpr const_str str_params = "parameters";
constexpr const_str str_usesteps = "usesteps";
constexpr const_str str_flushtarget = "preferred_flush_target";
Expand Down
4 changes: 0 additions & 4 deletions include/openPMD/IO/ADIOS/ADIOS2File.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -413,10 +413,6 @@ class ADIOS2File
private:
ADIOS2IOHandlerImpl *m_impl;
std::optional<adios2::Engine> m_engine; //! ADIOS engine
/**
* The ADIOS2 engine type, to be passed to adios2::IO::SetEngine
*/
std::string m_engineType;

/*
* Not all engines support the CurrentStep() call, so we have to
Expand Down
26 changes: 26 additions & 0 deletions include/openPMD/IO/ADIOS/ADIOS2IOHandler.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,32 @@ class ADIOS2IOHandlerImpl
* The ADIOS2 engine type, to be passed to adios2::IO::SetEngine
*/
std::string m_engineType;
std::optional<std::string> m_realEngineType;

inline std::string const &realEngineType() const
{
if (m_realEngineType.has_value())
{
return *m_realEngineType;
}
else
{
return m_engineType;
}
}
inline std::string &realEngineType()
{
return const_cast<std::string &>(
static_cast<ADIOS2IOHandlerImpl const *>(this)->realEngineType());
}
inline void pretendEngine(std::string facade_engine)
{
if (!m_realEngineType.has_value())
{
m_realEngineType = std::move(m_engineType);
}
m_engineType = std::move(facade_engine);
}
/*
* The filename extension specified by the user.
*/
Expand Down
26 changes: 13 additions & 13 deletions src/IO/ADIOS/ADIOS2File.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,6 @@ ADIOS2File::ADIOS2File(ADIOS2IOHandlerImpl &impl, InvalidatableFile file)
: m_file(impl.fullPath(std::move(file)))
, m_ADIOS(impl.m_ADIOS)
, m_impl(&impl)
, m_engineType(impl.m_engineType)
{
// Declaring these members in the constructor body to avoid
// initialization order hazards. Need the IO_ prefix since in some
Expand Down Expand Up @@ -324,7 +323,7 @@ namespace

size_t ADIOS2File::currentStep()
{
if (nonpersistentEngine(m_engineType))
if (nonpersistentEngine(m_impl->m_engineType))
{
return m_currentStep;
}
Expand All @@ -337,9 +336,9 @@ size_t ADIOS2File::currentStep()
void ADIOS2File::configure_IO_Read()
{
bool upfrontParsing = supportsUpfrontParsing(
m_impl->m_handler->m_backendAccess, m_engineType);
m_impl->m_handler->m_backendAccess, m_impl->m_engineType);
PerstepParsing perstepParsing = supportsPerstepParsing(
m_impl->m_handler->m_backendAccess, m_engineType);
m_impl->m_handler->m_backendAccess, m_impl->m_engineType);

switch (m_impl->m_handler->m_backendAccess)
{
Expand All @@ -355,7 +354,7 @@ void ADIOS2File::configure_IO_Read()
* In non-persistent (streaming) engines, per-step parsing is
* always fine and always required.
*/
streamStatus = nonpersistentEngine(m_engineType)
streamStatus = nonpersistentEngine(m_impl->m_engineType)
? StreamStatus::OutsideOfStep
: StreamStatus::Undecided;
parsePreference = ParsePreference::PerStep;
Expand All @@ -374,7 +373,7 @@ void ADIOS2File::configure_IO_Read()
* Prefer up-front parsing, but try to fallback to per-step parsing
* if possible.
*/
if (upfrontParsing == nonpersistentEngine(m_engineType))
if (upfrontParsing == nonpersistentEngine(m_impl->m_engineType))
{
throw error::Internal(
"Internal control flow error: With access types "
Expand Down Expand Up @@ -412,7 +411,7 @@ void ADIOS2File::configure_IO_Write()
// Also, it should only be done when truly streaming, not
// when using a disk-based engine that behaves like a
// streaming engine (otherwise attributes might vanish)
nonpersistentEngine(m_engineType);
nonpersistentEngine(m_impl->m_engineType);

streamStatus = StreamStatus::OutsideOfStep;
}
Expand Down Expand Up @@ -466,13 +465,13 @@ void ADIOS2File::configure_IO()

// set engine type
{
m_IO.SetEngine(m_engineType);
m_IO.SetEngine(m_impl->realEngineType());
}

if (!supportedEngine(m_engineType))
if (!supportedEngine(m_impl->m_engineType))
{
std::stringstream sstream;
sstream << "User-selected ADIOS2 engine '" << m_engineType
sstream << "User-selected ADIOS2 engine '" << m_impl->m_engineType
<< "' is not recognized by the openPMD-api. Select one of: '";
bool first_entry = true;
auto add_entries = [&first_entry, &sstream](auto &list) {
Expand Down Expand Up @@ -687,7 +686,7 @@ void ADIOS2File::configure_IO()
auxiliary::getEnvNum("OPENPMD_ADIOS2_STATS_LEVEL", 0);
m_IO.SetParameter("StatsLevel", std::to_string(stats_level));
}
if (m_engineType == "sst" && notYetConfigured("QueueLimit"))
if (m_impl->realEngineType() == "sst" && notYetConfigured("QueueLimit"))
{
/*
* By default, the SST engine of ADIOS2 does not set a limit on its
Expand Down Expand Up @@ -773,7 +772,8 @@ adios2::Engine &ADIOS2File::getEngine()
bool openedANewStep = false;
{
if (!supportsUpfrontParsing(
m_impl->m_handler->m_backendAccess, m_engineType))
m_impl->m_handler->m_backendAccess,
m_impl->m_engineType))
{
/*
* In BP5 with Linear read mode, we now need to
Expand Down Expand Up @@ -1048,7 +1048,7 @@ void ADIOS2File::flush_impl(ADIOS2FlushParams flushParams, bool writeLatePuts)
{
case FlushTarget::Disk:
case FlushTarget::Disk_Override:
if (m_engineType == "bp5" ||
if (m_impl->realEngineType() == "bp5" ||
/* this second check should be sufficient, but we leave the
first check in as a safeguard against renamings in
ADIOS2. Also do a lowerCase transform since the docstring
Expand Down
47 changes: 37 additions & 10 deletions src/IO/ADIOS/ADIOS2IOHandler.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,14 +192,17 @@ template <typename Callback>
void ADIOS2IOHandlerImpl::init(
json::TracingJSON cfg, Callback &&callbackWriteAttributesFromRank)
{
if (auto unsupported_engine_cfg =
auxiliary::getEnvString("OPENPMD_ADIOS2_PRETEND_ENGINE", "");
!unsupported_engine_cfg.empty())
{
auxiliary::lowerCase(unsupported_engine_cfg);
pretendEngine(std::move(unsupported_engine_cfg));
}
// allow overriding through environment variable
m_engineType =
realEngineType() =
auxiliary::getEnvString("OPENPMD_ADIOS2_ENGINE", m_engineType);
std::transform(
m_engineType.begin(),
m_engineType.end(),
m_engineType.begin(),
[](unsigned char c) { return std::tolower(c); });
auxiliary::lowerCase(realEngineType());

// environment-variable based configuration
if (int groupTableViaEnv =
Expand Down Expand Up @@ -255,7 +258,7 @@ void ADIOS2IOHandlerImpl::init(
if (maybeEngine.has_value())
{
// override engine type by JSON/TOML configuration
m_engineType = std::move(maybeEngine.value());
realEngineType() = std::move(maybeEngine.value());
}
else
{
Expand All @@ -264,6 +267,24 @@ void ADIOS2IOHandlerImpl::init(
"Must be convertible to string type.");
}
}

if (engineConfig.json().contains(
adios_defaults::str_treat_unsupported_engine_like))
{
auto maybeEngine = json::asLowerCaseStringDynamic(
engineConfig
[adios_defaults::str_treat_unsupported_engine_like]
.json());
if (!maybeEngine.has_value())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess a boolean flag would also do? "e.g. accept_unspported_engine=true"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If true, can always return ".bp" at realEngineType(). The actual value of the treat_supported_engine expected to be one of the known adios engines, and exactly which one is not relavent I think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess a boolean flag would also do? "e.g. accept_unspported_engine=true"?

That will not be enough because there is no "standard" way how an engine should be treated. Especially, most engines differ slightly in how steps should be treated (staging engines require steps, BP5 requires steps with normal Read mode, but cannot deal with steps in ReadRandomAccess, the Campaign engine fails when using steps in any way, other engines are mostly a mixture).

I don't think that providing one generic implementation for an unknown future engine will work, it seems better to use the existing implementations and let users try which works best.

If true, can always return ".bp" at realEngineType(). The actual value of the treat_supported_engine expected to be one of the known adios engines, and exactly which one is not relavent I think.

realEngineType() will return the Engine type that is actually passed on to ADIOS2. m_engineType determines the behavior of the ADIOS2 backend. Especially BP5 behaves slightly different from the other BP engines.
I don't see a reason to put extra work into restricting the options here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. Thanks

{
throw error::BackendConfigSchema(
{"adios2",
adios_defaults::str_engine,
adios_defaults::str_treat_unsupported_engine_like},
"Must be convertible to string type.");
}
pretendEngine(std::move(*maybeEngine));
}
}
auto operators = getOperators();
if (operators)
Expand Down Expand Up @@ -362,6 +383,12 @@ std::string ADIOS2IOHandlerImpl::fileSuffix(bool verbose) const
constexpr char const *const default_file_ending = ".bp4";
#endif

if (m_realEngineType.has_value())
{
// unknown engine type, use whatever ending the user specified
return m_userSpecifiedExtension;
}

static std::map<std::string, AcceptedEndingsForEngine> const endings{
{"sst", {{"", ""}, {".sst", ""}, {".%E", ""}}},
{"staging", {{"", ""}, {".sst", ""}, {".%E", ""}}},
Expand Down Expand Up @@ -656,7 +683,7 @@ void ADIOS2IOHandlerImpl::checkFile(

bool ADIOS2IOHandlerImpl::checkFile(std::string fullFilePath) const
{
if (m_engineType == "bp3")
if (realEngineType() == "bp3")
{
if (!auxiliary::ends_with(fullFilePath, ".bp"))
{
Expand All @@ -666,7 +693,7 @@ bool ADIOS2IOHandlerImpl::checkFile(std::string fullFilePath) const
fullFilePath += ".bp";
}
}
else if (m_engineType == "sst")
else if (realEngineType() == "sst")
{
/*
* SST will add this ending indiscriminately
Expand Down Expand Up @@ -1144,7 +1171,7 @@ void ADIOS2IOHandlerImpl::getBufferView(
begin(optInEngines),
end(optInEngines),
[this](std::string const &engine) {
return engine == this->m_engineType;
return engine == this->realEngineType();
}))
{
parameters.out->backendManagedBuffer = false;
Expand Down
Loading