Additional metric showing successfully reloaded/restarted #94
Description
Currently it is quite hard to see whether or not the KMS has successfully reloaded it's state or whether that is still in progress. We are starting to do this in a roundabout way looking at number of messages written (by other services) to the KMS input/schedule topic vs the number of schedules the KMS processes (using the scheduler_messages_total metric).
It would be nice if the KMS output an additional metric that could basically just show whether or not it is currently reloading state.
Would suggest a gauge that is initialised to 0 on startup and then set to 1 when the future coming from the topic loader completes (here).
We should also document this and the other metrics that are output.
AC
- Metric is available which describes the state of reloading