What happened
On PR #2850, the review agent repeated 3 identical findings across 4 consecutive reviews (SHAs 443e737 → b1bbd14 → 0f47a19 → rebase re-review): (1) ADR 0062 factual-accuracy — repeated verbatim 4 times, (2) platform-nativeness.md broken-reference — repeated verbatim 4 times and posted as duplicate inline comments on Jul 7 and Jul 9, (3) missing-authorization — repeated verbatim 4 times. None of these findings' relevant code changed between reviews. The intervening commits added a single BYOA prose sentence and restored a lost line after rebase. Total: 12 finding instances for 3 unique findings (100% repetition rate). The duplicate inline comment on platform-nativeness.md:134 (same file, same line, same text posted twice) also demonstrates inline comment accumulation per #2794.
What could go better
The agent spent tokens re-reading the same files, re-analyzing unchanged content, and re-composing identical findings on each run. On a docs-only PR where the relevant code didn't change between pushes, 4 re-discoveries of the same 3 findings is pure waste. Pre-review context injection (parsing the sticky comment's prior findings before analysis) would let the agent skip re-discovery and only check whether prior findings still apply to the new diff. Confidence: high — the findings are textually identical and the relevant code did not change. This directly reinforces #1013.
Proposed change
This is additional evidence for #1013 and #2794. When implementing #1013's deduplication, the pre-review script should extract prior findings from the sticky comment and inject them as context for the agent. For each prior finding, the agent should determine whether the relevant code changed in the new commits — if not, carry forward the finding without re-analysis. For inline comments (#2794), the post-review script should check for existing inline comments on the same file/line with substantially similar text before posting new ones.
Validation criteria
On the next 5 PRs that receive 3+ review iterations with unchanged findings between runs, count how many findings are repeated verbatim across consecutive reviews. Baseline from PR #2850: 100% repetition rate (12 instances / 3 unique findings across 4 reviews). Target: less than 20% repetition rate — findings should only reappear if the relevant code changed. Inline comment count on any single file/line should not exceed 1 per unique finding.
Generated by retro agent from #2850
What happened
On PR #2850, the review agent repeated 3 identical findings across 4 consecutive reviews (SHAs 443e737 → b1bbd14 → 0f47a19 → rebase re-review): (1) ADR 0062 factual-accuracy — repeated verbatim 4 times, (2) platform-nativeness.md broken-reference — repeated verbatim 4 times and posted as duplicate inline comments on Jul 7 and Jul 9, (3) missing-authorization — repeated verbatim 4 times. None of these findings' relevant code changed between reviews. The intervening commits added a single BYOA prose sentence and restored a lost line after rebase. Total: 12 finding instances for 3 unique findings (100% repetition rate). The duplicate inline comment on platform-nativeness.md:134 (same file, same line, same text posted twice) also demonstrates inline comment accumulation per #2794.
What could go better
The agent spent tokens re-reading the same files, re-analyzing unchanged content, and re-composing identical findings on each run. On a docs-only PR where the relevant code didn't change between pushes, 4 re-discoveries of the same 3 findings is pure waste. Pre-review context injection (parsing the sticky comment's prior findings before analysis) would let the agent skip re-discovery and only check whether prior findings still apply to the new diff. Confidence: high — the findings are textually identical and the relevant code did not change. This directly reinforces #1013.
Proposed change
This is additional evidence for #1013 and #2794. When implementing #1013's deduplication, the pre-review script should extract prior findings from the sticky comment and inject them as context for the agent. For each prior finding, the agent should determine whether the relevant code changed in the new commits — if not, carry forward the finding without re-analysis. For inline comments (#2794), the post-review script should check for existing inline comments on the same file/line with substantially similar text before posting new ones.
Validation criteria
On the next 5 PRs that receive 3+ review iterations with unchanged findings between runs, count how many findings are repeated verbatim across consecutive reviews. Baseline from PR #2850: 100% repetition rate (12 instances / 3 unique findings across 4 reviews). Target: less than 20% repetition rate — findings should only reappear if the relevant code changed. Inline comment count on any single file/line should not exceed 1 per unique finding.
Generated by retro agent from #2850