Question on durability:persistence

[hawkejustin]

I know this isn't implemented yet, but was hoping someone had an example or maybe some direction on creating the 3rd party listener for DDS traffic tagged as "persistent" or "transient".
I assume it would be best to have one listener for all tagged messages on the domain.

[eboasson]

The basis of such a thing would be a subscription to the "DCPSPublication" built-in topic. You monitor that, and then you create readers for each topic you discover a transient/persistent writer on.

The QoS you use for those readers should be taken from the topic QoS, you can find out those by looking at "DCPSTopic", but it is best to have a fall-back to using the writer QoS if topic discovery is disabled (it brings some overhead and it is often not needed). Or you could decide that it must be enabled for this 🙂

To create a reader, you need a local version of the topic, you can arrange that using dds_find_topic if topic discovery is enabled, but you can also gather the type definition and create your own local topic definition.

All of the weird bits you need to do this are in principle in https://github.com/eclipse-cyclonedds/cyclonedds/blob/master/examples/dynsub/dynsub.c . You would have to do something different than that example does for partitions, and if someone publishes in, say, *, it gets really tricky. But as a starting point it should work.

I assume it would be best to have one listener for all tagged messages on the domain.

It wouldn't get you this, of course, there'd be many readers instead. What you can do is use the same code for handling data in each of them. For most things you'd want to do, you don't really need to know the type, and so a generic function operating samples of arbitrary types would work fine, as long as you keep track of which reader they can from.

[okellogg]

Could it be that the Roadmap needs updating?
In Short-term milestones - Cyclone 0.9 it says

Integration of Durability Service for Transient and Persistent data

but I cannot find related info in the release notes.

[eboasson]

Could it be that the Roadmap needs updating?

Yes ...

In Short-term milestones - Cyclone 0.9 it says

Integration of Durability Service for Transient and Persistent data

but I cannot find related info in the release notes.

It was a bit optimistic. My employer was sponsoring this, but the work got derailed by other obligations and I am also no longer sure what part is allowed to go into the open source. It is not abandoned, that's all I can say right now.

[hawkejustin]

I am revisiting this. I took your advice and am able to read the QoS of am endpoint, and create a reader if it's the durability I want.
First question I have is if the current implementation of transient/persistant would behave as transient local or volatile? Since they aren't implemented I'm sure it uses one of thier behaviors.
Next for transient, once I have a reader, can I just rebroadcast every message from that endpoint/topic? Would readers get 2 copies? That is obviously undesired. Otherwise I'll need to store messages and broadcast them if current writer goes down ( assuming they are behaving as transient local)
For persistence, I plan to just store raw message and required data I'm database and write them to dds bus on persistence app startup.