test: add fast unit coverage for the composed --update-baseline summary text - #123
test: add fast unit coverage for the composed --update-baseline summary text#123Tgenz1213 wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
🟢 Approval recommended
No unresolved review issues remain.
Pull request overview
Adds CLI-level regression coverage for the --update-baseline summary, including skipped ADR checks.
Changes:
- Creates temporary Git repositories and ADR fixtures.
- Uses mocked embedding and chat providers.
- Asserts the exact composed summary text.
File summaries
| File | Description |
|---|---|
internal/cli/cli_test.go |
Adds direct runCheck coverage for baseline summary output. |
Review details
Suppressed comments (4)
internal/cli/cli_test.go:529
- This three-line comment exceeds the repository's two-line maximum for comments. The setup rationale can be stated in two lines without restating the implementation flow.
// Pre-build the index the way `archguard index` would -- runCheck's
// store.Load is a no-op (not a rebuild trigger) when indexFile doesn't
// exist yet.
internal/cli/cli_test.go:483
- This three-line comment exceeds the repository's two-line maximum for comments. Keep the rationale concise so the test remains readable without narrating the setup.
// One ADR whose chat call succeeds, one whose chat call fails -- the
// failure must be counted in SkippedADRChecks without blocking the
// other ADR's violation from being recorded.
internal/cli/cli_test.go:545
- This four-line comment exceeds the repository's two-line maximum and includes detailed timing/implementation narration. Keep only the short rationale for why the test invokes the retry path.
// runCheck hardcodes context.Background() internally (no injection
// point), and the issue forbids behavioral changes to add one, so this
// pays llm.AnalyzeDrift's real ~14s backoff (2s+4s+8s, 3 retries) for
// the one bad ADR's Chat failure -- a one-time cost, not a per-ADR one.
internal/cli/cli_test.go:513
- This failure path makes every
go test/CI run sleep for the full 2s+4s+8s retry backoff beforeSkippedADRCheckscan be asserted, adding a fixed ~14s to the suite. Since this test only needs a terminal analysis failure and must not change production behavior, returnbackoff.Permanent(errors.New("simulated LLM failure"))from the mock (as an intentionally non-retryable error) soAnalyzeDriftfails immediately; the existing engine-level test already cancels context to avoid this delay.
return "", errors.New("simulated LLM failure")
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…ry text Extracts the summary Printf calls into formatBaselineScanSummary and formatBaselineWrittenSummary so the exact wording is unit-testable in isolation, instead of indirectly through a full runCheck/engine/LLM integration run -- which would have duplicated existing coverage in internal/analysis and paid llm.AnalyzeDrift's real ~14s backoff on every test run for no additional signal. Closes #121 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
b804d96 to
b9b5e3f
Compare
There was a problem hiding this comment.
What is the point of a helper function for one print statement?
There was a problem hiding this comment.
Fair question. Without extracting it, the only way to test the exact composed string is to run the whole runCheck → Engine.Run → llm.AnalyzeDrift pipeline for real, which is what the first version of this PR did — and that's exactly what paid the 14s backoff tax and duplicated coverage internal/analysis already has. This isn't meant as a general-purpose abstraction, just a minimal seam so the Printf's exact wording is callable in isolation. Happy to revert to the integration-style test (eating the 14s) or drop this test angle entirely if you'd rather not have the extra functions — your call.
|
This is kind of silly. |
Summary
Adds unit coverage for the exact composed "Baseline scan complete: ..." and "Baseline written to ..." strings by extracting them into two small, pure functions (
formatBaselineScanSummary,formatBaselineWrittenSummary) and testing those directly with table-driven cases.Related issue
Closes #121
In scope
internal/cli/cli.go: extract the two--update-baselinesummaryPrintfcalls into named functions (output is byte-identical; pure refactor)internal/cli/cli_test.go:TestFormatBaselineScanSummary(table-driven: zero counts, nonzero ADR-check-skip count, all-distinct-nonzero counts) andTestFormatBaselineWrittenSummaryOut of scope
Architectural notes
None — no new invariant; the extraction is a pure refactor with identical output, verified by keeping the exact same format strings.
Follow-ups
None open. Note on process: this PR went through two design iterations before landing here, both driven by code review.
First iteration (original commit) called
runCheckdirectly through a full temp-git-repo + mocked-provider integration setup, per the issue's suggested cli_test.go-level approach. Review of that version surfaced:llm.AnalyzeDrift's real ~14s exponential backoff on every test run, sincerunCheckhardcodescontext.Background()with no injection point.internal/analysis/analysis_test.go'sTestRun_UpdateBaselineMode_ReportsSkippedADRCheckCount— the only genuinely new value was pinning the CLI's exact composed string, which didn't require re-exercising the whole engine/backoff pipeline."Baseline written to ...") was never asserted.Second iteration (current) replaces that integration test with the extraction described above, which resolves all four points at once: 0.00s instead of ~14s, no duplicated coverage, no oversized comments, and both summary lines now covered directly.
Checklist
feat:,fix:,docs:,refactor:,build(deps):, etc.)go test -race -cover ./...passesgolangci-lint run --timeout=5mis cleanCLAUDE.mdupdated if this changes build/test commands, adds or renames a top-level package, changes a cross-package interface, or adds a footgun (N/A)docs/arch/if this embodies an architecturally-significant decision (N/A)🤖 Generated with Claude Code