Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Access] Add util command to backfill tx error messages db #6525

Open
wants to merge 29 commits into
base: master
Choose a base branch
from

Conversation

UlyanaAndrukhiv
Copy link
Contributor

@UlyanaAndrukhiv UlyanaAndrukhiv commented Oct 3, 2024

Closes: #6413

Context

This pull request introduces a utility command that allows ANs to backfill missing transaction error messages from start to end height and a set of ENs to their local databases. This ensures that historical ANs have the necessary data to serve requests moving forward.

The command accepts a list of EN IDs (execution-node-ids) , if not provided, it defaults to query any available EN. It optionally takes a start and end block height (start-height, end-height); if not provided, it defaults to the root block and latest sealed block.

Before execution, the command checks if AN indexing is enabled. For each failed transaction within the block range, it retrieves and stores the error messages locally. It queries ENs in order and stops after the first successful response.

The command uses GetTransactionErrorMessagesByBlockID for optimized batch retrieval.

Unit tests are included to ensure functionality.

@codecov-commenter
Copy link

codecov-commenter commented Oct 3, 2024

Codecov Report

Attention: Patch coverage is 21.42857% with 11 lines in your changes missing coverage. Please review.

Project coverage is 40.86%. Comparing base (899e12e) to head (258273c).

Files with missing lines Patch % Lines
cmd/access/node_builder/access_node_builder.go 0.00% 6 Missing ⚠️
...estion/tx_error_messages/tx_error_messages_core.go 42.85% 3 Missing and 1 partial ⚠️
utils/unittest/mocks/closer.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6525      +/-   ##
==========================================
- Coverage   41.20%   40.86%   -0.34%     
==========================================
  Files        2052     1988      -64     
  Lines      182191   176363    -5828     
==========================================
- Hits        75075    72078    -2997     
+ Misses     100824    98275    -2549     
+ Partials     6292     6010     -282     
Flag Coverage Δ
unittests 40.86% <21.42%> (-0.34%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@Guitarheroua Guitarheroua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, have a few comments!

admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
engine/access/rpc/backend/backend_events.go Outdated Show resolved Hide resolved
@UlyanaAndrukhiv UlyanaAndrukhiv marked this pull request as ready for review November 5, 2024 13:47
Copy link
Contributor

@peterargue peterargue left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a few comments about language in error messages, but otherwise looks good

admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
admin/commands/storage/backfill_tx_error_messages.go Outdated Show resolved Hide resolved
@UlyanaAndrukhiv
Copy link
Contributor Author

@Guitarheroua, @peterargue, I've tested the backfilling of transaction error messages and added an additional integration test TestBackfillTxErrorMessages based on TestAccessStoreTxErrorMessages. This approach made it easier to generate failed transaction results for testing. You can view the commit here: 258273c

// Verify that the error message retrieved matches the expected values.
s.Require().Equal(txID, errMsgResult.TransactionID)
s.Require().Equal(expectedTxResultErrorMessage, errMsgResult.ErrorMessage)
txErrorMessages := s.fetchTxErrorMessages(txResults, s.accessContainerName)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do you know that these weren't indexed regularly and were actually backfilled?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Access] Add util command to backfill tx error messages db
4 participants