Conversation
| #[derive(Clone)] | ||
| pub struct Db { | ||
| pool: deadpool_diesel::Pool<ConnectionManager, deadpool::managed::Object<ConnectionManager>>, | ||
| notify_cleanup_task: tokio::sync::mpsc::Sender<BlockNumber>, |
There was a problem hiding this comment.
Depending on what we do with the validator database (whether we re-use this or not), it would be good to make this task optional/disable-able
| // Strategy: For each vault_key, keep the latest entry (is_latest_update=true) and the | ||
| // most recent MAX_HISTORICAL_ENTRIES_PER_ACCOUNT historical entries, deleting the rest. | ||
| // | ||
| // Unfortunately, Diesel doesn't support window functions (ROW_NUMBER OVER) in its type-safe | ||
| // API, so we must use raw SQL for this complex deletion pattern. This is necessary because: | ||
| // 1. We need to partition by vault_key and order by block_num | ||
| // 2. We need to delete based on row ranking within each partition | ||
| // 3. The table is created WITHOUT ROWID, so we use the primary key (account_id, block_num, | ||
| // vault_key) |
There was a problem hiding this comment.
I'm not sure we need to keep MAX_HISTORICAL_ENTRIES_PER_ACCOUNT - we just need to make sure we have the data for the last
DELETE FROM account_vault_assets
WHERE
block_num < chain_tip - n
AND is_latest_update = falseTo make this query efficient, we may need to add indexes on block_num and is_latest_update fields.
This methodology should also apply to the account_storage_map_values table (and in the future to the accounts table).
drahnr
force-pushed
the
bernhard-cleanup-old-db-entries
branch
from
8d2ab2a to
1d45f47
Compare
|
I would probably prioritize cleanup of the |
|
This PR contains both cleanups, db and |
3 participants
Ref #1185 step 5.Closes #1355