loadable or executable individual sensor modules

[garyjg]

This is an old idea and a low priority but I didn't want to throw it out while cleaning up old notes

Rather than combining all the dynld modules into a single nidas_dynld DSO, consider compiling them into individual DSOs, allowing for simpler and more consistent dynamic plugin support for sensors and outputs.

Few nidas runtimes need all of the sensors, so loading the whole library is unnecessary. It may be memory-mapped anyway, but it's not clear that combining them all into one library serves any purpose, and it means the whole library may have external dependencies that only one module needs. Those more specialized plugins may also be better built outside the nidas source tree and installed into the NIDAS DSO search path. Since that works for some modules, it seems more consistent to do it for all of them.

The GCC web page about visibility is a relevant read, to use hidden visibility but export the right symbols from the DSOs (including plugin subclasses for typeinfo, exceptions, and the factory function). Boost has a DLL library that could help too.

• https://gcc.gnu.org/wiki/Visibility
• https://www.boost.org/doc/libs/release/libs/dll/

[garyjg]

There is also a related idea to run acquisition for each sensor in a separate subprocess. When the current sample stream handling in NIDAS is replaced with an actual messaging layer, then a sensor acquisition process only needs to read samples from the sensor device, timestamp them, and publish them. That can run as an entirely separate subprocess, so the dsm process becomes more of a controller process which can start and restart sensor processes. No one sensor can crash or deadlock the other sensors. Related to this issue of dynamically loaded runtime dependencies, the specific sensor executable could link whatever libraries it needs, and those libraries do not need to be loaded into the dsm process itself or the other sensors. Further, sensor executables do not need to be written in C++ or use the NIDAS libraries at all, as long as they can parse configuration from the NIDAS XML and publish NIDAS-compatible raw samples to the messaging layer. If one process is responsible for opening and reading one sensor, that also enables better parallelization and eliminates a lot of complicated threading in dsm. The current Raspberry Pi 3B DSM3 has 4 CPUs, but only one is used at a time since all sensor reads happen in one thread. Running sensor acquisition in parallel should also improve timestamp latency.

A first cut at this can add an argument to the dsm process which tells it to open and read a specific sensor. Eventually sensor subclasses could be built into their own executables for the dsm process to spawn and manage.

Finally, a controller process enables the option of stopping and restarting the entire sensor subprocess, such as to deploy a sensor configuration change.