Improvements to the data stored in AuditDb

[t-bast]

We're missing a lot of data that would be really useful for auditing the economical performance of our peers. The goal of this issue is to gather the list of additional data we'd like to store. Feel free to comment with additional data that should be added.

• when a tx is published or confirms: store its feerate, input count and output count (and amounts?)
• channel relay: use incoming/outgoing node_id instead of channel_id?
• provide an API to evaluate routing fees earned for incoming/outgoing nodes
  • fees earned from relaying payments
  • fees paid in liquidity operations (channel creation / splice transactions)
  • what else?

[thomash-acinq]

When paying a wallet behind a LSP, we log the node id of the wallet but not the one of the LSP. When the recipient is not a public node and we need routing hints to reach it, we should log the public node ids from the routing hints instead of the useless node id of the wallet. It would enable us to more easily detect LSPs that we should open a channel to.

[DerEwige]

These are information that I currently store in my own database.
Some of it may not be well suited for the AuditDB

The local fee on sent payments.
So one can include it into the cost calculation if needed.
This would neatly fit in the sent section of the AuditDB as a new field

The duration of each payment (failed and success) including route taken and point of failure in case of failure.
I currently keep this information for 7 days and calculate performance metrics for each channel/node.
This allows me to identify slow channels/node and ban them as they pose much higher risk to stuck htlc and therefore force closes.

Failure logs for local constrains, like fee mismatch.
Afaik this does currently not get stored in the AuditDB, but could be very helpful.

[t-bast]

Failure logs for local constrains

What do you mean exactly? Whenever we try to relay a payment, the failure reason (if any)? We want to store very minimal information about relay failures, because that can otherwise be used to DoS (it doesn't cost anything to send tons of invalid payments that will just fail).

[DerEwige]

What do you mean exactly? Whenever we try to relay a payment, the failure reason (if any)? We want to store very minimal information about relay failures, because that can otherwise be used to DoS (it doesn't cost anything to send tons of invalid payments that will just fail).

Yes, ideally, I would like to know the exact failure reason for each failed relay.
I believe right now, there is not even a log with the details available nor an event raised.
I can see how it could bloat the database though.
But an event, I could sing up to from a plugin would be nice.

The ACINQ node is more of a gateway-/payment-node. Routing reputation is not that important especially as you discourage routing over your node with a 500 ppm fee wall.

For routing/rebalance heavy nodes, it is essential to minimize local failures.
Minimizing local failures attracts more traffic, as it show one is a reliable routing peer.

Detailed information about local failures helps to identify the route cause.

Examples:
If I often have insufficient funds and the requested amount is also above htlcMaximum_opt then I should update and broadcast my htlcMaximum more often.
If the requested amount is always below htlcMaximum_opt, then updating it faster won’t help.

If I often get FeeInsufficient after a fee update on a channel. I might want to increase enforcement-delay or increase the time between updates, or add more peers to faster spread the info.

[t-bast]

But an event, I could sing up to from a plugin would be nice.

Emitting an event on relay failures would make a lot of sense, this would indeed allow custom data collection from plugins (with the necessary protections against DoS risk based on what the node operator wishes to accomplish).