[EXPERIMENT] Emit all messages across sequence number gaps in messagesWithBatch (1.3.x branch)#518
Closed
pjfanning wants to merge 3 commits into
Closed
[EXPERIMENT] Emit all messages across sequence number gaps in messagesWithBatch (1.3.x branch)#518pjfanning wants to merge 3 commits into
pjfanning wants to merge 3 commits into
Conversation
…ache#516 * The query windowing introduced by apache#180 treated a windowed batch returning fewer messages than batchSize as the end of the journal: bounded streams (recovery, currentEventsByPersistenceId) completed early and silently dropped every message beyond a gap left by deleted messages, and live streams stalled forever polling an empty window. apache#195 fixed only the gap at the head of the journal. * Rework the unfoldAsync state from FlowControl into a QueryPlan state machine (QueryRemaining, QueryWindow, PollRemaining, Complete): a short windowed batch is no longer conclusive and falls back to one query over the full remaining range, which gaps cannot hide messages from. * Keep windowing as the dense fast path of apache#180: after a full batch shows the journal to be dense, queries stay bounded to [from, from + batchSize]. * Span the full remaining range on the first query, crossing a deleted journal head (snapshot cleanup) in a single round trip, subsuming the apache#195 special case. * Poll the full remaining range when tailing: a windowed poll can never reach messages appended beyond a trailing gap. * Specify the gap state machine in MessagesWithBatchTest (core, in-memory journal stub) and the database-coupled behavior in MessagesWithBatchDatabaseContractTest (H2 plus the five integration databases): hard and soft deletes, a prefix purge through the journal's delete, and serialization failures.
Oracle treats the empty string as NULL, so inserting the synthetic corrupt row with writer = "" violated the NOT NULL constraint on EVENT_JOURNAL.WRITER (ORA-01400) and failed only the Oracle integration tests. The production write path always carries a writer UUID, so this is a test-fixture-only fix.
These files contain no Akka-derived code, so they should carry the full ASF header (the headerLicense the build stamps on new files) rather than the shorter "derived from Akka" variant, per a maintainer review note.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Copies over #517 to see if it works ok in this branch.
Just an experiment at the moment.