[CLAUDE-OPUS-5]
Gap
"Scalability to thousands of samples" is a stated priority in PROJECT_REQUIREMENTS.md. No document budgets request quotas, and IMPLEMENTATION_PLAN.md §6 defers the whole topic to Phase 2 without numbers.
Evidence
The unauthenticated GitHub REST API allows 60 requests per hour per address. Task 6 alone costs roughly three calls per pull request across detail, reviews, and commits, so a single run of one task consumes about half the hourly budget. Three eval tasks target GitHub.
An authenticated token raises this to 5000 per hour, and a single GraphQL query retrieved all 30 contributor profiles that would otherwise have cost 30 REST calls. Both mitigations are available today and neither is specified.
Site-level throttling is a separate axis. Airbnb, X, and LinkedIn all rate-limit or challenge aggressive clients, and README.md:112 already concedes no anti-bot handling.
Scope
- Specify a per-domain request budget and a token bucket shared across concurrent workers.
- Specify credential-backed access where an API exists, and prefer batched queries over per-entity calls.
- Define behavior on quota exhaustion:
blocked with a retry-after, never a partial silent result.
- Record per-run request counts and quota headroom in the run result, so the Phase 1 batch runner in §5 can report cost per sample.
- State the concurrency ceiling per domain that the Phase 2 worker pool must respect.
Acceptance
- A per-domain budget table is committed and enforced in code.
- Quota exhaustion is a distinct, diagnosable status.
[CLAUDE-OPUS-5]
Gap
"Scalability to thousands of samples" is a stated priority in
PROJECT_REQUIREMENTS.md. No document budgets request quotas, andIMPLEMENTATION_PLAN.md§6 defers the whole topic to Phase 2 without numbers.Evidence
The unauthenticated GitHub REST API allows 60 requests per hour per address. Task 6 alone costs roughly three calls per pull request across detail, reviews, and commits, so a single run of one task consumes about half the hourly budget. Three eval tasks target GitHub.
An authenticated token raises this to 5000 per hour, and a single GraphQL query retrieved all 30 contributor profiles that would otherwise have cost 30 REST calls. Both mitigations are available today and neither is specified.
Site-level throttling is a separate axis. Airbnb, X, and LinkedIn all rate-limit or challenge aggressive clients, and
README.md:112already concedes no anti-bot handling.Scope
blockedwith a retry-after, never a partial silent result.Acceptance