What happened
PRs #989 and #1025 in konflux-ci/project-controller were near-identical Renovate bot PRs (1-line go.mod change swapping cenkalti/backoff major versions). Neither was ever merged. Combined over a 3-week period (Jun 19 - Jul 11), they accumulated: 16 review agent runs (11 on #989, 5 on #1025), 7 retro agent runs (4 on #989, 3 on #1025), 20 review-comment findings, and 0 human interactions. Every agent run reached the same conclusion. The retro on PR #1025 at Jul 9 explicitly noted the cross-PR pattern and non-deterministic review verdicts, yet the system dispatched another retro on Jul 11 (this run). Previous retro run on Jul 7 filed issue #3325 as evidence for #3177 and #3226, which was closed.
What could go better
The circuit breaker proposed in #4064 would have capped retro runs at 2 per PR, preventing 5 of the 7 retro runs. However, the cross-PR dimension reveals a gap even in the circuit breaker design: when Renovate autoclosed PR #989 and created PR #1025 with an identical pattern, the circuit breaker counter reset to zero for the new PR, allowing the waste cycle to restart. A stronger design would track not just per-PR retro count but also detect when a new PR matches a recently-retro'd pattern (same bot author, same dependency, same failure mode). The bot-PR skip proposals (#3177, #3226) would also help but are complementary rather than sufficient — the circuit breaker provides defense-in-depth for cases where bot detection has false negatives.
Proposed change
This is additional evidence for #4064 (retro per-PR circuit breaker) and secondarily for #3177 and #3226 (skip retro on bot dependency PRs closed without merge). The new data point strengthens the priority case: 23 agent runs on 2 PRs is the most extreme waste observed in this repo. Consider extending the circuit breaker design in #4064 to include a cross-PR deduplication check — before dispatching retro, query recent retro comments on other closed PRs from the same author with similar titles to detect pattern repetition.
Validation criteria
After implementing the circuit breaker from #4064: (1) No more than 2 retro runs should occur per PR regardless of close/reopen cycles. (2) When a Renovate PR is autoclosed and recreated with the same dependency pattern, total retro runs across both PRs should not exceed 3 (2 on the first, 1 on the second to capture any delta). (3) Over a 30-day window across enrolled repos, average retro runs per bot-authored dependency PR should drop from 3+ to 2 or fewer.
Generated by retro agent from konflux-ci/project-controller#1025
What happened
PRs #989 and #1025 in konflux-ci/project-controller were near-identical Renovate bot PRs (1-line go.mod change swapping cenkalti/backoff major versions). Neither was ever merged. Combined over a 3-week period (Jun 19 - Jul 11), they accumulated: 16 review agent runs (11 on #989, 5 on #1025), 7 retro agent runs (4 on #989, 3 on #1025), 20 review-comment findings, and 0 human interactions. Every agent run reached the same conclusion. The retro on PR #1025 at Jul 9 explicitly noted the cross-PR pattern and non-deterministic review verdicts, yet the system dispatched another retro on Jul 11 (this run). Previous retro run on Jul 7 filed issue #3325 as evidence for #3177 and #3226, which was closed.
What could go better
The circuit breaker proposed in #4064 would have capped retro runs at 2 per PR, preventing 5 of the 7 retro runs. However, the cross-PR dimension reveals a gap even in the circuit breaker design: when Renovate autoclosed PR #989 and created PR #1025 with an identical pattern, the circuit breaker counter reset to zero for the new PR, allowing the waste cycle to restart. A stronger design would track not just per-PR retro count but also detect when a new PR matches a recently-retro'd pattern (same bot author, same dependency, same failure mode). The bot-PR skip proposals (#3177, #3226) would also help but are complementary rather than sufficient — the circuit breaker provides defense-in-depth for cases where bot detection has false negatives.
Proposed change
This is additional evidence for #4064 (retro per-PR circuit breaker) and secondarily for #3177 and #3226 (skip retro on bot dependency PRs closed without merge). The new data point strengthens the priority case: 23 agent runs on 2 PRs is the most extreme waste observed in this repo. Consider extending the circuit breaker design in #4064 to include a cross-PR deduplication check — before dispatching retro, query recent retro comments on other closed PRs from the same author with similar titles to detect pattern repetition.
Validation criteria
After implementing the circuit breaker from #4064: (1) No more than 2 retro runs should occur per PR regardless of close/reopen cycles. (2) When a Renovate PR is autoclosed and recreated with the same dependency pattern, total retro runs across both PRs should not exceed 3 (2 on the first, 1 on the second to capture any delta). (3) Over a 30-day window across enrolled repos, average retro runs per bot-authored dependency PR should drop from 3+ to 2 or fewer.
Generated by retro agent from konflux-ci/project-controller#1025