Skip to content

Integration branch: React frontend, OpenCode import, configurable model, explicit-date regeneration - #23

Closed
artcashin wants to merge 66 commits into
prime-radiant-inc:mainfrom
artcashin:feature/opencode-import-and-model-provider
Closed

artcashin wants to merge 66 commits into
prime-radiant-inc:mainfrom
artcashin:feature/opencode-import-and-model-provider

Conversation

@artcashin

Copy link
Copy Markdown

Single branch combining the session's work, for merging everything at once instead of landing four PRs in sequence.

This duplicates work already under review. Every change here is in an existing PR; nothing is unique to this branch. Reviewers wanting granular, independently-mergeable units should use those instead:

PR Scope
#19 React frontend + Claude Code subagent linkage, plus the CI fix below
#20 OpenCode session import + configurable summary provider
#22 Regenerate a journal entry when its date is named explicitly

Not included: #21 (session-title provider wiring), which remains a draft stacked on #19 and #20.

What's in it

  • OpenCode as a session source (opt-in). OpenCode keeps sessions in a single SQLite database rather than one file per session, so an adapter stages them as JSONL for the normal scanner. Enumeration reads OpenCode's database because opencode session list only reports the current directory's project; transcripts come from opencode export, which works from anywhere.
  • Configurable summary provider. Summaries can be written by any OpenAI-compatible endpoint instead of Claude Haiku. Absent config, behaviour is unchanged.
  • Explicit-date regeneration. A named --date regenerates its entry; --all still skips already-summarized dates, so the cost control from Summarize re-evaluates skipped groups on every run #10 is intact on the hot path.
  • CI fix. bunfig.toml scopes Bun's test runner to src/, and CI runs the web suite with vitest. Without this, a bare bun test — which both workflows run — collected React component tests that need a DOM and failed 18 of them.

Why the branch exists

It is the checkout a local launchd job runs from nightly, which needs OpenCode import and explicit-date regeneration together. That is an operational need, not a review unit — hence this PR being an integration convenience rather than the recommended way to review the work.

Testing

224 tests pass, typecheck clean, pre-commit hook green. Every change was written test-first. Exercised end-to-end against a live corpus: 1000+ sessions ingested, 96 OpenCode sessions across 16 projects with 61 subagents correctly flagged, summaries generated by a local Gemma via llama.cpp.

Two merge conflicts were resolved along the way, both keep-both: summarize.test.ts where two branches appended a describe block, and summarize.ts where the titling block met the changed upsertJournalEntry call.

🤖 Generated with Claude Code

artcashin and others added 30 commits July 19, 2026 07:58
Manual groups over coding sessions: a Groups nav tab, one group per
session, in-app create/rename/delete, membership in a separate table
that survives re-ingest.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7 TDD tasks: schema, helper module, nav, views, routes, session
control, verification. Corrects spec FK note (foreign_keys=ON means
session_id must have no FK to survive re-ingest).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
One-way import of Claude Desktop groups (dframe-group-scopes) into the
notebook: reader + reconcile (mirror by desktop_id, manual groups
untouched), Ungrouped view, and a block-while-Desktop-open guard behind
a single policy switch. Write-back deferred to Phase 2.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7 TDD tasks: desktop_id migration, classic-level reader (schema pinned
against real store), reconcile/mirror, Ungrouped view, block-while-open
guard, import route+button+CLI, verification.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Approach A (re-parse source JSONL on demand). Default text-only; Show
thinking/tools via query params re-parses source_path, warns if the
file is unavailable. No ingest/schema change. Precedes the group work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
4 TDD tasks: transcript parser, enriched session render + toggle
controls + CSS, route params, verification. Re-parses source_path on
demand; default view unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…es, param-preservation test

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Core match: collapsible tool calls (name + preview, result paired
inside), thinking bubbles + token estimate, teal accent + rounded
styling on the notebook's light palette. No markdown/diff (deferred),
no new deps.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3 TDD tasks: parser tool id/input enrichment, collapsible-tool +
thinking-bubble render rework + CSS, verification.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…t edge

Final-review cleanups: remove redundant style="display:block" on the
orphan tool_result div; update spec edge-case wording to match the
implemented (extras-dropped) behavior.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Umbrella doc: keep engineering-notebook backend, new React/Vite frontend
matching claude-session-viewer's session display, plus Desktop groups.
Recommends monorepo + a 5-phase roadmap (Phase 1 = session display
parity). Evidence-based session-chain findings (resume = same session;
subagents are the multi-file case). Each phase gets its own spec/plan.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
React/Vite app in web/, Hono JSON API (session list, structured
transcript, subagent), polished session viewer (collapsible tools,
thinking, subagent nesting, client-side hide/show toggles), Tailwind.
Subagent->Task mapping via first-prompt match (viewer's strategy).
Spikes deferred to plan: on-disk subagent layout, Vite<->Hono wiring.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
7 tasks: Vite/Hono scaffold+wiring, structured transcript endpoint,
subagent discovery via .meta.json + endpoints, session list, React API
layer + list view, session viewer (collapsible tools/thinking/subagents
+ toggles), verification. Additive-only; legacy views/DB untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- web/ Vite React-TS + Tailwind SPA; App pings /api
- createApiRouter mounted at /api; GET /api/ping
- createApp({react}) serves web/dist SPA (assets + index.html fallback),
  skipping legacy views; --react flag threaded through serve
- exclude web/ from backend tsconfig
Verified end-to-end: dev proxy + prod build; /api/ping, SPA, assets, and
client-route fallback all 200.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
artcashin and others added 28 commits July 19, 2026 12:49
Skip isMeta records (the <local-command-caveat> wrapper) and clean
command tags so a slash command reads like "/plugin marketplace add …"
instead of showing the caveat text. Surface isMeta on the structured
transcript; add title-extraction tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Filter isMeta (the <local-command-caveat> wrapper) and user messages
that are purely command scaffolding/output (<local-command-stdout>…) so
command output no longer shows as a "You" entry. Keep real prompts,
slash-command invocations, tool calls, and tool results.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the custom display filter and title command-cleaning that made
panel 3 diverge. Now identical to the viewer: transcript skips only
isMeta records (buildConversationThread); title is the first non-meta
user text sliced to 200 (extractFirstPrompt), no cleaning. Command/
stdout records render via the ported XmlView just as the viewer does.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…port)

Backend: /api/projects(+entries), /api/calendar, /api/groups(+/:id,
/ungrouped, /import-desktop). Adds groups.desktop_id migration, the
Desktop LocalStorage reader (classic-level) + importDesktopGroups
reconcile. Frontend: Projects & Groups 3-panel pages (session viewer in
panel 3), Calendar month grid linking days to the Journal; nav routes to
the real pages. Groups has a "Sync from Claude Desktop" button.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port claude-session-viewer's buildMessageTree + resolveActivePath: the
transcript endpoint now reconstructs the uuid/parentUuid tree (bridging
filtered meta/summary records) and returns the active branch instead of
every flat record. Header shows the active-path length. Codex/uuid-less
logs fall back to flat order.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude Code sometimes emits a thinking block with only a signature
(encrypted) and empty text. The viewer rendered these as blank boxes;
we dropped them. Now the parser keeps empty thinking blocks and
ThinkingBlock renders a "🔒 redacted thinking" marker, so it's visible
that reasoning occurred even when the text is encrypted. Same change
applied to the local claude-session-viewer's ThinkingBlock.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Render the transcript in bounded batches (80) and grow as the reader
scrolls (IntersectionObserver sentinel), so a 1000+ message session no
longer freezes on first paint or when toggling thinking/tools. Small
sessions render in one batch (jsdom tests unaffected).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lift the toggle state into a TranscriptTogglesProvider (context) so the
buttons live in the always-visible top bar, left of the search box, and
stay usable no matter where you are in the session scroll. SessionView
reads the toggles from context instead of owning them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- sessions.title / title_source columns
- import captures each Claude Desktop session's title (titleSource
  desktop|user) from local_*.json (same files the group import reads)
- generate a Desktop-style title via the Claude Agent SDK for Claude
  Code sessions without one; stored as title_source=generated
- `engineering-notebook title [--generate --limit N]` CLI + POST
  /api/sessions/:id/title on-demand endpoint
- API exposes titles (session list, group sessions, journal/project
  entry sessions); panel 2 and Groups list show the name instead of
  "Session N"

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…edits

- Panel 3 h1 shows the stored session title (Desktop/generated),
  falling back to the first prompt.
- summarize now generates a Desktop-style title for each summarized
  session that lacks one (applies Desktop titles first so it only
  LLM-generates where genuinely missing).
- User-edited Desktop titles are sticky (never overwritten by auto or
  generated); auto/generated titles refresh on re-import. Expose
  title_source in the session API.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A user record whose content is purely <local-command-stdout>…</…> is
machine output, not a user turn — it was rendering as a 'You' bubble.
Skip it in the tree build (bridging parentUuid through it), matching the
same fix applied to the local claude-session-viewer's buildMessageTree.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ingested subagent transcripts (<project>/<parentUUID>/subagents/agent-*.jsonl)
now record parent_session_id — taken from the records' own sessionId without
triggering the continuation prefix-skip (a separate continuationParentId drives
that). A new relinkSubagents() fills the link (from the column or recovered from
the path) and re-homes each subagent's project_id to its parent's project. It
runs at the end of ingest and is order-independent + idempotent, so it also
backfills. Adds idx_sessions_parent.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
/api/sessions/:id now returns the subagent backlink (parent_title, gated on
is_subagent so continuations aren't mislabeled), subtask_title, spawn_tool_use_id,
and the session's ingested subagents ordered chronologically. Journal, project,
and group rows carry their nested subagents via a shared ingestedSubagents()
helper (single batched query, filtered to ingested rows so there are no dead
links). Ungrouped and the flat /api/sessions list exclude subagents.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Panel 2 nests each session's subagents (revealed once the session is selected),
labelled by subtask title. Panel 3 gains: a compaction-proof SubagentIndex; a
"Subagent of …" backlink (shown only for subagents) that navigates to the parent
and scrolls/ring-highlights the exact spawn point, falling back to the index when
the spawn was compacted off the transcript; inline spawn links matched by
tool_use id (name-agnostic — Claude Code names the tool "Agent", not "Task"),
visible even with tools hidden; and subtask titles as the panel header. The
focus scroll fires once per target and the window grows-only, so reading past a
spawn no longer snaps back.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The date was interpolated raw into the muted/error summary cards; wrap it in
escapeHtml() like the surrounding fields.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ingestion was nested inside the remoteSources block, so a purely local setup
never scanned/ingested on sync. Hoist the scanSources/ingestSessions call so it
always runs, appending any remote cached paths when present.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… toggle

Compacted sessions keep their full pre-compaction messages in the same file, but
active-path resolution prunes them (e.g. 1,036 shown of 3,122). Panel 3 now
defaults to the uncompacted view: the transcript parser gains a `full` option
(returns every message in file order, dropping the injected isCompactSummary
blob) and a `compacted` flag; the endpoint honors ?full=1. A third top-bar
toggle — "View compacted" / "View uncompacted" — switches modes, defaulting to
uncompacted so pre-compaction detail (and its subagent spawn links) is shown.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… compacted

Plumb the transcript's `compacted` flag to the toggle bar. The "View compacted"
button is disabled (and greyed) unless the open session actually has a compacted
view to switch to. Non-compacted sessions are forced back to the default
uncompacted view so the disabled toggle never strands the reader in compacted
mode; the toggle re-disables when leaving the session.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Light green (matching the other toggles) when the session has a compacted view;
gray when it doesn't.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…uped to remove)

Adds POST /api/sessions/:id/group ({ groupId } assigns, { groupId: null }
unassigns; 404s on missing session/group). In the Groups view, panel-2 session
rows are draggable and panel-1 group rows (and Ungrouped) are drop targets with a
hover highlight; dropping assigns/unassigns the conversation and refreshes the
counts and current list. Subagent rows stay non-draggable.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two additions that were developed together because the first one made the
second one worth having.

OpenCode source adapter
-----------------------
OpenCode keeps sessions in a single SQLite database rather than one file per
session, so it cannot be scanned like the Claude Code and Codex sources. An
opt-in adapter exports each session into a staging directory of JSONL files
during `ingest`, which the normal scanner then picks up — leaving dedup,
--force, grouping, summarization and the web UI untouched.

Three things the implementation has to work around:

- `opencode session list` only reports sessions belonging to the current
  working directory's project, so no single cwd can enumerate them all.
  Sessions are enumerated from OpenCode's database; transcripts still come
  from `opencode export`, which works from anywhere.
- `opencode export` yields empty stdout over a pipe but is complete when
  redirected to a file, and a single export can run to megabytes. Both call
  sites go through one runToFile helper.
- is_subagent was derived purely from Claude Code's /subagents/ path
  convention, so flat-staged files always read as 0. Formats that state it
  outright now win; Claude Code keeps path detection, because its
  parentSessionId also covers continuations and would otherwise misflag them.

Projects are keyed off each session's working directory, so OpenCode and
Claude Code work in the same repo on the same day lands in one journal entry.

Configurable summary provider
-----------------------------
Summaries and session titles were hardcoded to Claude Haiku via the Agent SDK.
A summary_provider config block now selects any OpenAI-compatible endpoint,
including a local llama.cpp server. Absent config keeps the previous behaviour.

Both call sites route through the new provider, since leaving titles on
Anthropic would silently contradict a configured provider, and the Claude auth
preflight is skipped for non-Anthropic providers that authenticate with their
own key. Entries record the model that actually produced them. API keys are
referenced by environment variable name and never stored in config.

Reasoning models spend completion tokens before emitting content, so an
exhausted budget fails loudly rather than writing an empty summary.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`bun test` collected web/src/**/*.test.tsx and failed 18 of them with
"document is not defined". Those are React component tests that need a DOM,
which is configured for vitest in web/vite.config.ts — a file Bun's test
runner does not read. Nothing was wrong with the tests themselves: they pass
under their intended runner.

The failure was invisible locally because the pre-commit hook runs the scoped
`bun test ./src`, while both CI workflows run a bare `bun test`. main has no
web tests, so this branch would have turned CI red on merge.

bunfig.toml scopes Bun's runner to src/, leaving it responsible for the
backend suite alone. That also excludes web/src/session/adapt.test.ts, which
is DOM-free and did run under Bun, so CI now runs the web suite with vitest —
covering all 20 web tests, which CI never executed at all. web/ is not a
workspace of the root package, so its dependencies install separately.

Verified: bare `bun test` 182 pass / 0 fail, `bun run test:web` 20 pass,
typecheck clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A day is summarized long before it is over. Because group selection filtered
out every (date, project) that already had an entry — and the --date filter was
applied inside that guard — no flag could refresh the current day's entry.
Neither --all nor --date reached it; the only way was deleting the row by hand.

Naming a date is now treated as a deliberate request to regenerate it. --all is
unchanged and still only visits dates with no entry, so routine incremental runs
never rewrite existing work. Persistence was already an upsert, so a regenerated
entry replaces the old one in place.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Brings the explicit-date regeneration into the branch the local install runs
from, so a scheduled nightly job can refresh the previous day's entries.

Conflict was in summarize.test.ts, where both branches appended a describe
block; resolution keeps both.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@artcashin

Copy link
Copy Markdown
Author

Closing: the work here is already under review as #19, #20, #21 and #22, which are being merged sequentially to preserve their separate history. This branch remains in use locally as the checkout a nightly launchd job runs from; that automation is being submitted separately on its own branch.

@artcashin artcashin closed this Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant