Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/check tracked prefixes #20

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Feature/check tracked prefixes #20

wants to merge 2 commits into from

Commits on Jan 13, 2023

  1. Check tracked prefixes before adding to the database 

    Not all messages sent over cardinal streams are of interest to
Cardinal-EVM. For example logs, receipts, and transactions are needed
in Flume, but ignored by Cardinal-EVM.

With the Kafka Cardinal-Streams consumer, only tracked prefixes are
delivered, but with the Websockets consumer all prefixes are sent
across the wire and a part of the delivered PendingBatch. This means
that if we're syncing via websockets, we might be writing a lot of
additional information to disk that Cardinal-EVM will never use.

By checking keys against the set of tracked prefixes, we can ensure
that only useful information is recorded to the database.
    Austin Roberts committed Jan 13, 2023
    Configuration menu
    Copy the full SHA
    f2820fc View commit details
    Browse the repository at this point in the history

  2. Track whether contract storage was available 

    For data sent across Kafka, if we get a message we can assume that
it will have complete block information.

For data coming from websockets, we may only have headers, receipts,
and transactions, while state updates are missing.

Prior to this update, there was no way to tell whether state updates
were missing from a block. This adds a value that will be "0" if state
updates were unavailable and "1" if they were available, and makes
the Cardinal EVM refuse to add blocks that don't indicate availability.

This does mean that, even for Kafka, producer side updates must be
done before consumer side updates, as new consumers will reject messages
produced by old producers (but old consumers will still accept messages
from new producers).
    Austin Roberts committed Jan 13, 2023
    Configuration menu
    Copy the full SHA
    598389d View commit details
    Browse the repository at this point in the history