test(corpus): regression coverage for per-line JSONL loading in the auditor

[Nitjsefnie]

Closes #57 (assigned — thanks!)

As noted on the issue, the production fix already landed in 4e6fbbb (dup of #127), so this is the missing regression coverage: 7 tests in tests/unit/test_corpus/test_auditor.py locking the merged _load_jsonl_documents behavior, each verified against a live probe of the code before writing —

• N non-empty lines → N documents (via audit() totals)
• per-line document shape: content = stripped line, doc_id = {path}:L{n}, metadata extension/filename/line_number
• blank/whitespace lines skipped
• malformed JSON line skipped silently (the merged except json.JSONDecodeError: continue policy — asserted as a skip, no invented logging)
• .json files still load as a single document (routing regression guard)
• max_documents capped across JSONL lines (locked via the single-file path, which exercises the in-loop break)

Gates: uv run pytest tests/unit — 941 passed, 4 skipped; target file 19 passed (+7). Zero new ruff findings (the one import-ordering error in the file header is pre-existing on main, verified by diffing against pristine).

Generated by Claude Fable 5 (brief, review), Claude Opus 4.8 (implementation)

[himanshu231204]

/oc review pr

[opencode-agent]

Review: test/corpus-jsonl-coverage ✅ LGTM — ship it.

Summary

7 well-scoped regression tests that lock the merged _load_jsonl_documents behavior from 4e6fbbb. Every assertion maps 1:1 to the production code.

Verification

Test	Implementation	Status
test_jsonl_one_document_per_line	_load_jsonl_documents iterates splitlines() → N docs	✅
test_jsonl_document_shape	content=stripped, doc_id=f"{path}:L{n}", metadata keys	✅
test_jsonl_skips_empty_and_whitespace_lines	if not stripped: continue	✅
test_jsonl_skips_malformed_line_silently	except json.JSONDecodeError: continue	✅
test_json_file_loads_as_single_document	routing checks == ".jsonl" (.json misses → _load_single_file)	✅
test_json_file_document_extension	_load_single_file uses file_path.suffix → ".json"	✅
test_jsonl_respects_max_documents	if len(documents) >= self.max_documents: break	✅

Design choices worth noting (none blocking)

1. Sync vs async split — Tests that call audit() are async; tests that call _load_documents() directly are sync. Both correct, the latter avoids the async fixture machinery for unit-level assertions.

2. extension hardcoded as ".jsonl" in _load_jsonl_documents rather than derived from file_path.suffix. This is a normalization decision — matches the lowercase routing in _load_documents (path.suffix.lower() == ".jsonl"). Consistent.

3. No directory-level multi-JSONL test — The break in _load_documents only fires in the non-JSONL else branch, so a directory with multiple .jsonl files could load up to max_documents per file. This is a pre-existing design gap (not introduced by this PR) and out of scope.

Nitpicks

• test_jsonl_skips_malformed_line_silently could more explicitly assert the error didn't propagate with pytest.raises(SomeException) as a negative check (e.g., with pytest.raises(...): ... for the absence of a raise). The current approach (just verifying docs are returned) is fine and consistent with the rest of the test suite.

opencode session  |  github run

[himanshu231204]

@Nitjsefnie test_jsonl_skips_malformed_line_silently could more explicitly assert the error didn't propagate with pytest.raises(SomeException) as a negative check (e.g., with pytest.raises(...): ... for the absence of a raise). The current approach (just verifying docs are returned) is fine and consistent with the rest of the test suite.

[github-actions]

🎉 Congratulations @Nitjsefnie!

Your pull request has been successfully merged into main. 🚀

Thank you for contributing to OpenAgentHQ and helping improve the project.

We truly appreciate your contribution and hope to see you back with more amazing PRs!

Happy Open Sourcing! ❤️