Conversation
Previously, sessions whose messages spanned midnight were split across
two date buckets using per-message logical dates. In practice this
created thin tail slivers on the next day — a few morning messages from
a session that started the prior evening — that the LLM consistently
rejected as "no engineering work" because the slivers lacked context.
Skip-pattern analysis (2026-05-13, 8 recent skips) found that all
skipped groups had tiny day-scoped content (<2KB; smallest 389 chars
literal `/exit` boilerplate). The most recoverable false-negatives
were these midnight-split slivers where substantive work happened on
the start side of midnight but the tail got rejected in isolation.
Fix: each session is now atomic and attributed to its `started_at`
logical date (the date adjusted by `day_start_hour`). A session that
begins late one night and continues past midnight contributes its full
content to the start date.
`splitConversationByDay` is kept as an exported helper for external
tools that want per-message day attribution, but the orchestrator no
longer uses it.
Tests updated: the midnight-spanning suite now verifies atomic
attribution (one group, all 6 messages on Feb 20) and that
`filterDate("2026-02-21")` returns nothing for a session that started
on Feb 20 even when some messages are timestamped Feb 21.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two ingest/grouping defects cause journal entries to go stale or to be skipped as "no engineering work."
Changed session files are never re-ingested.
ingestSessionsskipped any session already present in the DB, so a session that Claude/Codex later appended to kept its truncated transcript forever. Any journal entry already generated for that (date, project) also stayed stale.Cross-midnight sessions were split into slivers.
groupSessionsByDateAndProjectused per-message logical dates, so a session started at 22:00 contributed a few morning messages to the next day as a separate group. Those tail slivers lack context and the model consistently rejects them. Skip-pattern analysis over 8 recent skips found every skipped group had tiny day-scoped content (<2KB, smallest was 389 chars of/exitboilerplate); the most recoverable false negatives were exactly these midnight slivers.Approach
Re-ingest on change. Sessions record source file size and mtime. Ingest re-parses a session when either differs, and invalidates journal entries for the affected (date, project) so the next
summarizeregenerates them. Unchanged files still short-circuit, so repeat ingests stay cheap.Sessions are atomic. Each session is attributed wholly to the logical date of its
started_at(adjusted byday_start_hour). A session that begins late and runs past midnight contributes its full content to the start date.splitConversationByDayremains exported for external tools that want per-message attribution; the orchestrator no longer calls it.How to test
New coverage: re-ingest when a source file grows, skip when size and mtime both match, journal invalidation on re-ingest, atomic attribution of a midnight-spanning session (one group, all six messages on the start date), and
filterDatematching the start date rather than the message date.Baseline on
mainis 92 passing; this branch is 95 passing, 0 failing, typecheck clean.Limitations
Change detection uses size and mtime, not a content hash. A file edited in place to exactly the same size within the same mtime granularity will not be detected. That does not occur with append-only agent transcripts.
Invalidation is per (date, project), so re-ingesting one session regenerates the whole day's entry for that project.