Skip to content

End-to-end tests for the full capture-to-answer pipeline #8

Description

@grajat90

Summary

Priority. Build end-to-end tests that exercise the real system as a user would, from ingest through distillation, storage, and retrieval/answering, aligned with the project's testing philosophy in CLAUDE.md ("reproduce in an E2E setting as closely aligned with how an end user would experience it as possible").

Scope

  • Full pipeline E2E: from an audio fixture (test/fixtures/speech.wav) through the helper's transcribe path -> segmentation (internal/segment) -> consolidation (internal/consolidate) -> distillation (internal/distill) -> ingest/persist (internal/ingest, internal/store) -> retrieval/answer (internal/retrieve, internal/answer) -> CLI/RPC (internal/api, cmd/journal). Assert structure, fields, and grounding to real segment IDs, not exact LLM text.
  • CLI E2E: drive journal/journald over the real unix socket (build on cmd/journald/fileingest.go and existing ingest), covering ingest, search, ask, today, forget, pause/resume.
  • IPC E2E: exercise the machelper JSONL-over-socket contract with the Swift helper's transcribe-file/self-test where the SDK is present; keep it skippable/deterministic where it is not.
  • Invariant enforcement: the network guard must fail any local-only run that opens an unexpected socket; assert no transcript is persisted or logged; assert off-device egress only on opt-in.
  • CI: wire the deterministic E2E suite into make check/CI so it runs on every change; gate helper-dependent parts behind SDK availability.

Why priority

E2E coverage is the safety net that lets every other feature here (new STT models, multilingual, CLI/chat, web UI, MCP) land without regressing the privacy invariants or the pipeline.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestpriorityHigh-priority work

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions