Always-on (or "session-free") recording

[emersonknapp]

Description

Support this end-to-end usage model: A rosbag2 recorder that is always on, from vehicle boot time until power down. This recording always places new split files into a single directory that acts as a "rolling buffer" that is bounded - by file count, duration, or disk utilization - with older files being automatically cleaned up.

Background

Currently, rosbag2's usage model is centered around the model of a "recording session", which targets a pattern of: intentionally start some job that you want to record, stop the recording when that job is done, and then retrieve and look at the session as a whole for that job.

However, in many types of more mature projects, the machine will always be "on the job" in some sense when the device is powered on. Incidents of interest may occur at any time, and may or may not be noticed or reported immediately. The desire is to hold as much data on-device as is reasonable so that when needed, the history can be accessed to review or debug.

This model of being always-on with log rotation is common concept when thinking of system applications rather than developer-attended prototypes. Think journald, etc.

Requirements

I see the following changes/features as being needed to smoothly support this use case:

• Allow writing files to an existing directory #1717, and either ignore or fully omit metadata.yaml.
  • There is no "session" concept in this model, each split bagfile is pretty much independent.
• Timestamp split bagfiles instead of serial numbering #1917
  • so that it's easy to pinpoint where a timestamp of interest can be found, and because sequential indexes lose meaning when the session is conceptually infinite
• Support --repeat-latched in ROS 2 (transient local) #1159
  • Because any given file might be consumed independently, the full messaging context as if subscribing to the system at that time is needed (think /tf_static and /robot_description as the most common culprit)
• Sequentially play multiple bagfiles without metadata.yaml #1918
  • Because you may download or specify some sequence of splits to cover a particular time period where an event of interest occurred
• Maintain the log buffer bounds by some criteria - by number of splits in directory, maximum disk usage of directory, or minimum containing disk free space.
  • I think that this feature can and should be implemented as a separate program that watches the log directory and cleans up old files as needed, so it may not even be part of rosbag2 repository, but I'm just calling it out here

Implementation

This is a meta-ticket that covers a series of independent features that add up to the workflow laid out here. Implementation and testing details should be covered in the tracking tickets for each sub-feature.