[ingress] introduce buffer queue #8
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
wlawt
force-pushed
the
ingress-queue
branch
from
84848ee to
93aa32e
Compare
wlawt
force-pushed
the
ingress-queue
branch
from
93aa32e to
194ab63
Compare
danyalprout
reviewed
Sep 22, 2025
Collaborator
danyalprout
left a comment
danyalprout
left a comment
There was a problem hiding this comment.
lgtm, lets also update the docker-compose file here to add a new topic (tips-ingress) or something
Comment on lines
+25
to
+29
| pub async fn enqueue_bundle( | ||
| &self, | ||
| bundle: &EthSendBundle, | ||
| sender: Address, | ||
| ) -> Result<(), Error> { |
Collaborator
There was a problem hiding this comment.
Should we add some retry logic with a backoff to this function? Maybe we can pull in this crate:
wlawt
commented
Sep 23, 2025
Comment on lines
+64
to
+66
| .with_min_delay(Duration::from_millis(100)) | ||
| .with_max_delay(Duration::from_secs(5)) | ||
| .with_max_times(3), |
Contributor
Author
There was a problem hiding this comment.
we could tune these params later?
wlawt
commented
Sep 23, 2025
| command: | | ||
| sh -c " | ||
| kafka-topics --create --if-not-exists --topic tips-audit --bootstrap-server kafka:29092 --partitions 3 --replication-factor 1 | ||
| kafka-topics --create --if-not-exists --topic tips-ingress --bootstrap-server kafka:29092 --partitions 3 --replication-factor 1 |
Contributor
Author
There was a problem hiding this comment.
should we add a retention policy (time or size based)?
wlawt
force-pushed
the
ingress-queue
branch
from
754cac1 to
b83a8ed
Compare
danyalprout
approved these changes
Sep 23, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
Add a queue to buffer transactions before writing to hot database.
This will allow the DB Writer to be a consumer of this queue and then be able to
insert_bundleto datastore.Tests
TODO