feat(casework): fill missing_details from the description call - #441
feat(casework): fill missing_details from the description call#441gaurav-karki wants to merge 3 commits into
Conversation
`Case.missing_details` -- the "what we still don't hold" section on every case
page -- had no writer. This adds one, riding in the LLM call that already reads
the verdict.
Two parts make up the value. A DETERMINISTIC floor from what is bound (the
charge sheet, and whether an appeal was lodged), which is verifiable and never
guesses. Then up to 4 specific documents the model found the sources CITE but
our evidence lacks -- named with their dates, dispatch numbers and parties.
WHY IT RIDES IN THE DESCRIPTION CALL. `enrich_description` already bills 1+N
premium calls per case, where N is the verdict's 150,000-char chunk count.
Batch verdicts run to a 141,000-char median, so a standalone stage would
re-read the same फैसला from scratch and roughly double the batch's premium
spend. Derivation stays independent: the field comes from which materials are
bound plus what the sources cite, never from the finished narrative.
The gates differ -- `description` needs press release OR verdict,
`missing_details` needs the verdict -- so `STAGES["description"].provides` is
the one CONDITIONAL entry in that table. Read it as "can provide".
The model proposes and the rules dispose. `held_summary` prints our bound
evidence into the prompt so the model computes a difference rather than
guessing at absence, which also makes the grounding rule checkable in code:
a claimed-missing document we demonstrably hold is rejected, with the rule
named in the run log. Rejections are never silent -- a dropped item and a model
that found nothing need opposite follow-up.
Guards worth knowing about:
- This stage only writes into an EMPTY `missing_details`, and --force does not
override that. The importer's truncation guard puts `ACCUSED LIST
INCOMPLETE` in the same field and the 61 published values are hand-written;
the floor items cannot serve as a "we wrote this" signature because they
were copied verbatim FROM those hand-written cases.
- `patch_fields`, not two `patch_field` calls -- the second call's ETag would
already be stale, so a loop cannot stay conditional.
- `MAX_LLM_ITEMS` is the binding limit, not the char cap. A character cap
always cuts the LAST item, which is always the most specific one, because
specificity is long. Measured twice on real output before the cap was
demoted to a sanity guard.
- A partial fetch falls back to the floor. `has_verdict` reads bindings, so a
case whose court order 500s still reports True while the model never saw it.
Verified read-only against production on 2026-08-08 for three cases through a
GET-only proxy, no --apply: 078-CR-0111, 079-CR-0047, 078-CR-0118. Each
produced 6 items, ~$1.74 per case.
Two known limits are documented in the spec rather than papered over: nothing
detects "a sentence, not a document name", and `COURT_TYPES` cannot tell a
Supreme Court ruling from a Special Court verdict.
Design: docs/superpowers/specs/2026-08-07-missing-details-enricher-design.md
in the meta-repo.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code review is billed via overage credits. To resume reviews, an organization admin can raise the monthly limit at claude.ai/admin-settings/claude-code.
Once credits are available, push a new commit or reopen this pull request to trigger a review.
|
Warning Review limit reached
Next review available in: 27 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe description stage now generates ChangesMissing Details Generation
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant CaseworkPipeline
participant enrich_description
participant LLM
participant missing_details
participant CaseworkAPI
CaseworkPipeline->>enrich_description: run description stage
enrich_description->>LLM: request description and missing_documents
LLM-->>enrich_description: return parsed candidates
enrich_description->>missing_details: validate candidates and build output
missing_details-->>enrich_description: rendered missing_details or None
enrich_description->>CaseworkAPI: conditional patch_fields update
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
PR Reviewer Guide 🔍(Review updated until commit b819977)Here are some key observations to aid the review process:
🛠️ Relevant configurations:
[config] enable_ai_metadata: False
custom_model_max_tokens: 200000
git_provider: github
output_relevant_configurations: True
model: openai/cx/gpt-5.5
ENABLE_AUTO_APPROVAL: True
custom_reasoning_model: False
fallback_models: ['openai/cx/gpt-5.4-mini']
is_auto_command: True
publish_output: True
publish_output_progress: True
progress_gif_url:
progress_gif_width: 48
verbosity_level: 0
use_extra_bad_extensions: False
log_level: DEBUG
use_wiki_settings_file: True
use_repo_settings_file: True
use_global_settings_file: True
extra_config_url:
disable_auto_feedback: False
ai_timeout: 120
response_language: en-US
repo_context_files: ['AGENTS.md']
repo_context_from_default_branch: True
repo_context_max_lines: 500
max_description_tokens: 500
max_commits_tokens: 500
max_model_tokens: 32000
model_token_count_estimate_factor: 0.3
patch_extension_skip_types: ['.md', '.txt']
allow_dynamic_context: True
max_extra_lines_before_dynamic_context: 10
patch_extra_lines_before: 5
patch_extra_lines_after: 1
cli_mode: False
large_patch_policy: clip
duplicate_prompt_examples: False
seed: -1
temperature: 0.2
ignore_pr_title: ['^\\[Auto\\]', '^Auto', '^Bump ', '^chore\\(deps\\)']
ignore_pr_target_branches: []
ignore_pr_source_branches: []
ignore_pr_labels: []
ignore_pr_authors: []
ignore_repositories: []
ignore_language_framework: []
restricted_mode: False
reasoning_effort: medium
enable_claude_extended_thinking: False
extended_thinking_budget_tokens: 2048
extended_thinking_max_output_tokens: 4096
claude_extended_thinking_models_override: []
extract_issue_from_branch: True
branch_issue_regex:
enable_custom_labels: False
[pr_reviewer] require_ticket_analysis_review: False
require_score_review: False
require_tests_review: True
require_estimate_effort_to_review: True
require_can_be_split_review: False
require_security_review: True
require_estimate_contribution_time_cost: False
require_todo_scan: False
publish_output_no_suggestions: True
persistent_comment: True
extra_instructions: Focus on: logic errors and edge cases; security/authz regressions; missing error handling;
Django/DRF correctness (migrations, N+1 queries, transaction/atomicity, serializer & permission gaps).
Do NOT comment on formatting, import order, or naming — ruff handles those in CI.
num_max_findings: 3
final_update_message: True
enable_review_labels_security: True
enable_review_labels_effort: True
require_all_thresholds_for_incremental_review: False
minimal_commits_for_incremental_review: 0
minimal_minutes_for_incremental_review: 0
enable_intro_text: True
enable_help_text: False
|
PR Code Suggestions ✨Explore these optional code suggestions:
🛠️ Relevant configurations:
[config] enable_ai_metadata: False
custom_model_max_tokens: 200000
git_provider: github
output_relevant_configurations: True
model: openai/cx/gpt-5.5
ENABLE_AUTO_APPROVAL: True
custom_reasoning_model: False
fallback_models: ['openai/cx/gpt-5.4-mini']
is_auto_command: True
publish_output: True
publish_output_progress: True
progress_gif_url:
progress_gif_width: 48
verbosity_level: 0
use_extra_bad_extensions: False
log_level: DEBUG
use_wiki_settings_file: True
use_repo_settings_file: True
use_global_settings_file: True
extra_config_url:
disable_auto_feedback: False
ai_timeout: 120
response_language: en-US
repo_context_files: ['AGENTS.md']
repo_context_from_default_branch: True
repo_context_max_lines: 500
max_description_tokens: 500
max_commits_tokens: 500
max_model_tokens: 32000
model_token_count_estimate_factor: 0.3
patch_extension_skip_types: ['.md', '.txt']
allow_dynamic_context: True
max_extra_lines_before_dynamic_context: 10
patch_extra_lines_before: 5
patch_extra_lines_after: 1
cli_mode: False
large_patch_policy: clip
duplicate_prompt_examples: False
seed: -1
temperature: 0.2
ignore_pr_title: ['^\\[Auto\\]', '^Auto', '^Bump ', '^chore\\(deps\\)']
ignore_pr_target_branches: []
ignore_pr_source_branches: []
ignore_pr_labels: []
ignore_pr_authors: []
ignore_repositories: []
ignore_language_framework: []
restricted_mode: False
reasoning_effort: medium
enable_claude_extended_thinking: False
extended_thinking_budget_tokens: 2048
extended_thinking_max_output_tokens: 4096
claude_extended_thinking_models_override: []
extract_issue_from_branch: True
branch_issue_regex:
enable_custom_labels: False
[pr_code_suggestions] commitable_code_suggestions: False
dual_publishing_score_threshold: -1
focus_only_on_problems: True
extra_instructions: Prefer a few high-impact, project-specific suggestions over many generic ones.
Skip style/formatting (ruff-enforced) and changes under cases/migrations/.
enable_help_text: False
enable_chat_text: False
persistent_comment: True
max_history_len: 4
publish_output_no_suggestions: True
suggestions_score_threshold: 0
new_score_mechanism: True
new_score_mechanism_th_high: 9
new_score_mechanism_th_medium: 7
auto_extended_mode: True
num_code_suggestions_per_chunk: 3
max_number_of_calls: 3
parallel_calls: True
final_clip_factor: 0.8
decouple_hunks: False
demand_code_suggestions_self_review: False
code_suggestions_self_review_text: **Author self-review**: I have reviewed the PR code suggestions, and addressed the relevant ones.
approve_pr_on_self_review: False
fold_suggestions_on_self_review: True
num_code_suggestions: 4
|
PR Agent Walkthrough 🤖Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more. Here is a list of tools you can use to interact with the PR Agent:
(1) Note that each tool can be triggered automatically when a new PR is opened, or called manually by commenting on a PR. (2) Tools marked with [*] require additional parameters to be passed. For example, to invoke the |
|
Auto-approved PR |
![jawafdehi-pr-agent[bot]](https://avatars.githubusercontent.com/u/241969530?s=60&v=4){kind=small}
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
🧹 Nitpick comments (1)
tests/casework/test_enrich_description.py (1)
1299-1566: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd coverage for the “already, nothing left to write” path.
CASE_ALREADYhas a substantial description and a court-order evidence item, but it lacks a charge-sheet evidence item. Derive a case fromCASE_WITH_APPEALplus charge-sheet evidence, then assertstatus == "already",reason == "nothing left to write", and no conditionalPATCHrequest when both floor items are satisfied.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/casework/test_enrich_description.py` around lines 1299 - 1566, Add an end-to-end test near the existing missing_details apply-path tests that derives a case from CASE_WITH_APPEAL, adds charge-sheet evidence so both deterministic floor items are satisfied, and supplies the corresponding model response. Assert the report row has status "already" with reason "nothing left to write", and verify the stub API recorded no conditional PATCH request.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@tests/casework/test_enrich_description.py`:
- Around line 1299-1566: Add an end-to-end test near the existing
missing_details apply-path tests that derives a case from CASE_WITH_APPEAL, adds
charge-sheet evidence so both deterministic floor items are satisfied, and
supplies the corresponding model response. Assert the report row has status
"already" with reason "nothing left to write", and verify the stub API recorded
no conditional PATCH request.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: e5a1fb2d-aa08-4fd3-8e7c-5f9970445899
📒 Files selected for processing (7)
casework/common/missing_details.pycasework/common/pipeline.pycasework/enrich_allegations.pycasework/enrich_description.pytests/casework/test_enrich_allegations.pytests/casework/test_enrich_description.pytests/casework/test_missing_details.py
The design doc it referenced is not being kept. The rationale it held -- why this rides in the description call, why the char cap is a sanity guard, why the held-document rule matches a head noun rather than a length ratio, and the two limits that are not fixed -- lives in the pull request description instead. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Persistent review updated to latest commit 9fdfa6d |
The per-field idempotency gate admits an already-described case so its EMPTY `missing_details` can be filled. If BOTH floor items are already satisfied -- charge sheet bound AND a Supreme reference on file -- and the model finds nothing, there is nothing honest to say and no reason to rewrite the description, which leaves an empty patch. That must not become a request. The branch existed and was untested. Raised by CodeRabbit on #441. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Persistent review updated to latest commit b819977 |
|
|
@coderabbitai review |
|
1 participant
User description
casework: fill
missing_detailsfrom the description callCase.missing_details— the "what we still don't hold" section on every case page — had nowriter. Nothing in the pipeline filled it. This adds one, riding in the LLM call that
already reads the verdict.
What it produces
For a case holding a press release and a Special Court verdict:
क and ख are deterministic from what is bound. The rest are documents the verdict cites
that our evidence lacks — found by the model, then filtered.
Two fields, one call
enrich_descriptionalready bills 1 + N premium calls per case, where N is the verdict'schunk count:
summarize_verdictpays one request per 150,000 chars because the ठहर sits atthe end of a फैसला. Batch verdicts run to a 141,000-char median. A standalone stage would
re-read the same document from scratch, roughly doubling the batch's premium spend, so this
adds one output key instead.
Derivation stays independent —
missing_detailscomes from which materials are bound pluswhat the sources cite, never from the finished narrative.
casework/common/missing_details.pyholds that logic with no LLM and no API dependency.
The gates differ:
descriptionneeds press release or verdict,missing_detailsneeds theverdict specifically. So
STAGES["description"].providesis the one conditional entry inthat table — read it as "can provide".
The model proposes, the rules dispose
held_summaryprints our bound evidence into the prompt, so the model computes a differenceinstead of guessing at absence. That also makes one grounding rule checkable in code rather
than trusted: a claimed-missing document we demonstrably hold is rejected, with the rule
named in the run log.
Rejections are logged with their reason, never silently dropped — a dropped item and a model
that found nothing need opposite follow-up (prompt problem vs sourcing problem).
Verified against production, read-only
Three cases dry-run 2026-08-08 through a GET-only proxy (every write verb 405s), no
--apply:078-CR-0111079-CR-0047078-CR-0118~$1.74 per case, 2 calls each.
Guards worth knowing about
This stage only writes into an empty
missing_details, and--forcedoes not overridethat. The importer's truncation guard puts
ACCUSED LIST INCOMPLETEin the same fieldand the 61 published values are hand-written; the floor items cannot serve as a "we wrote
this" signature because they were copied verbatim from those hand-written cases.
patch_fields, not twopatch_fieldcalls — the second call's ETag would already bestale, so a loop cannot stay conditional.
MAX_LLM_ITEMSis the binding limit, not the char cap. A character cap always cuts thelast item, which is always the most specific one, because specificity is long. Measured
twice on real output before the cap was demoted to a sanity guard.
A partial fetch falls back to the floor.
has_verdictreads bindings, so a case whosecourt order returns a 500 still reports
Trueand the prompt still claims we hold theverdict. The model would then be diffing against an inventory it could not read, so its items
are dropped and only the deterministic floor is written.
The idempotency check is per field. A description-only check skipped every already-
described case before
missing_detailswas computed — so the ~188 production cases carryinga description could never get the new field, while
providesclaimed they were complete.Those cases are now processed, and only the empty field is patched: the description is never
rewritten without
--force.Two known limits, documented rather than papered over
Neither is fixable in this module. Both are in the spec.
अदालतले पर्याप्त प्रमाण मूल्याङ्कन गरेको छैन।, passes every code rule — 47 chars, no fillerphrase, head noun matches no held document. So a criticism of the court could publish.
Catching it means recognising Nepali verb endings, a different kind of rule than the rest.
COURT_TYPEShas one entry, so aSupreme Court ruling bound as
court_orderis indistinguishable from a Special Courtverdict. Needs a distinct material type upstream.
Review history
Four rounds of
/code-review(medium, medium, high, xhigh) found 32 findings; 30 are fixed,each with a test using the reviewer's own repro input. The two above are the exceptions.
Worth knowing what the later rounds caught, because the tests alone would not have:
len(word)/len(item) >= 0.5ratio. The promptdemands specificity, every qualifier drives that ratio down, so the rule stopped firing on
exactly the items it was written for —
विशेष अदालत काठमाडौंको फैसला (०८१-CR-००९१)passed at0.13 while the bare form was caught. Replaced with a head-noun test (Nepali is head-final).
अभियोगपत्रमा उल्लेखित संलग्न अनुसूची— an annex the charge sheet references — on 24 of 25cases.
--forceappending to this field grew it without bound across runs. Withdrawn entirely.Tests
1,391 pass in
tests/casework/.ruff,ty(repo-wide) andpre-commitclean.Design:
docs/superpowers/specs/2026-08-07-missing-details-enricher-design.mdin themeta-repo.
🤖 Generated with Claude Code
PR Type
Enhancement, Tests
Description
Add
missing_detailsgenerationShare description LLM call
Add deterministic document rules
Cover parsing, gating, writes
Diagram Walkthrough
File Walkthrough
2 files
Generate missing details with descriptionsAdd missing details rule engine1 files
Register conditional missing details provider1 files
Document allegations field ownership3 files
Cover dual-field description enrichmentTest missing details pure rulesClarify allegations missing details boundary🛠️ Relevant configurations:
These are the relevant configurations for this tool:
[config]
[pr_description]
Summary by CodeRabbit
New Features
Bug Fixes