No settlement-latency or poller-lag metric

[Darkdruce]

Summary

There is no measurement of how long payments take to settle or how far behind the poller's cursor is.

Location

• src/horizon.rs (poller/stream loops log but don't measure lag)

Details & impact

Silent degradation (e.g., stalled stream falling back to polling) is invisible until a merchant complains.

Suggested fix

Track cursor age and settlement latency as metrics; alert on thresholds.

Acceptance criteria

• Cursor age and settlement latency are observable.

[Markodiba]

Hi, I'd like to work on this issue.
I have backend experience building observability into payment and settlement systems, including work on Stellar/Soroban based platforms where tracking transaction state and latency was critical to catching silent failures early. I understand the risk here: if the poller's cursor falls behind or a stream silently degrades to polling, there's currently no signal until a merchant notices missed or delayed settlements.
My proposed approach:
Add a metric for cursor age, computed as the difference between the current time and the ledger close time of the last processed cursor position, exposed via whatever metrics stack the project already uses (Prometheus style counters/gauges would fit src/horizon.rs cleanly).
Add a settlement latency metric, measuring the time between when a payment is submitted and when it's confirmed settled, so degradation shows up as a distribution rather than a single failure.
Instrument the existing poller/stream loops in src/horizon.rs at the points where they already log, since those are the natural places to also record these metrics without restructuring the loop logic.
Add threshold based alerting hooks (or at least clearly documented thresholds) so cursor age and settlement latency breaches can trigger an alert rather than relying on logs being read after the fact.

[grantfox-oss]

🦊 GrantFox — @Markodiba has been assigned to this issue as part of the Official Campaign | FWC26 campaign!

Next steps:

1. Open a Pull Request referencing this issue (e.g., Closes #134)
2. Your PR will be reviewed by the StellarGateLabs maintainers

Good luck! Track your progress on GrantFox.

[grantfox-oss]

🎉 This issue has been marked as completed on GrantFox as part of the Official Campaign | FWC26 campaign!

@Markodiba's PR #210 was approved and merged by @Manuel1234477.

🏆 @Markodiba: You earned 35 FoxPoints for this contribution! Your current tier: Explorer (376 total points). Track your full progress on GrantFox.

👏 Great work, @Markodiba! Keep contributing to StellarGateLabs.