Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split composite DB index on htlc_infos table #2946

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

t-bast
Copy link
Member

@t-bast t-bast commented Nov 25, 2024

We were previously using a composite index on both the channel_id and the commitment_number in the htlc_infos table. This table is used to store historical HTLC information to be able to publish penalty txs when a malicious peer publishes a revoked commitment. This table is usually the largest DB table on nodes that relay a lot of payments, because it grows with the number of HTLCs received and sent (and only shrinks when channels are closed or spliced).

Using a composite index makes sense, but leads to increased memory usage compared to separate indices, thus reducing performance because this is a table on which we write a lot, but almost never read (only when we detect a revoked force-close). The read performance doesn't seem to be negatively impacted anyway when splitting the indices, and the memory usage is greatly improved.

The migration may take some time depending on the size of the table, but we cannot get around this.

Thanks to @DerEwige for the performance investigation that lead to this change (see #2932 for more details).

We were previously using a composite index on both the `channel_id` and
the `commitment_number` in the `htlc_infos` table. This table is used to
store historical HTLC information to be able to publish penalty txs when
a malicious peer publishes a revoked commitment. This table is usually
the largest DB table on nodes that relay a lot of payments, because it
grows with the number of HTLCs received and sent (and only shrinks when
channels are closed or spliced).

Using a composite index makes sense, but leads to increased memory usage
compared to separate indices, thus reducing performance because this is
a table on which we write a lot, but almost never read (only when we
detect a revoked force-close). The read performance doesn't seem to be
negatively impacted anyway when splitting the indices, and the memory
usage is greatly improved.

The migration may take some time depending on the size of the table,
but we cannot get around this.

Thanks to @DerEwige for the performance investigation that lead to
this change (see #2932 for more details).
@t-bast
Copy link
Member Author

t-bast commented Nov 25, 2024

Note that I didn't add any test because we already have migration tests from v2 to the latest version that fill the htlc_infos table and verify that we can still read its contents after the migration, so we shouldn't need any new test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant