What happened
The retro agent was triggered by PR #3174 on quay/quay-konflux-components but could not access any PR details (title, body, comments, reviews, timeline) because the GitHub API returned HTTP 403. The GH_TOKEN provided to the sandbox was invalid or lacked the necessary scopes. The agent could only analyze the static repository configuration, not the actual workflow that triggered the retro.
Without API access, the retro agent cannot: (1) reconstruct the timeline of agent interactions, (2) compare agent review vs human review, (3) measure rework rate, (4) assess token cost, or (5) evaluate autonomy readiness. All five optimization goals require visibility into the PR's lifecycle.
What could go better
The retro agent should have read access to the target repository's PRs, issues, and comments. This is a prerequisite for any meaningful retro analysis. Without it, the agent produces low-value output that wastes compute resources.
The harness should either: (a) inject a valid GH_TOKEN with read access into the retro sandbox, or (b) prefetch the PR data (comments, reviews, diff, timeline) during the pre-script and make it available as a file in the sandbox. Option (b) is more secure since it limits the token's exposure.
Confidence: High — this is directly observable from the 403 errors in this run.
Proposed change
In the retro agent's harness configuration (or the reusable-retro.yml workflow), ensure the sandbox has access to PR data. Two approaches:
Option A (preferred): In the pre-script for the retro agent, use the runner's GITHUB_TOKEN to fetch PR metadata and write it to a JSON file that gets mounted into the sandbox:
# pre-retro.sh
gh pr view "$PR_NUMBER" --repo "$REPO_FULL_NAME" --json title,body,state,author,comments,reviews,labels,reviewDecision > "$WORKSPACE/pr-data.json"
gh pr diff "$PR_NUMBER" --repo "$REPO_FULL_NAME" > "$WORKSPACE/pr-diff.patch"
Option B: Pass a read-only GH_TOKEN into the retro sandbox environment, similar to how the review harness passes GH_TOKEN.
The retro agent definition should document which data sources it expects and fail fast with a clear error if they're missing.
Validation criteria
- A retro agent run on any PR can successfully read the PR title, body, comments, reviews, and diff.
- The retro agent's output references specific events from the PR timeline (not just static repo analysis).
- No HTTP 403 errors appear in the retro agent's trace when accessing PR data.
Generated by retro agent from https://github.com/quay/quay-konflux-components/pull/3174
What happened
The retro agent was triggered by PR #3174 on quay/quay-konflux-components but could not access any PR details (title, body, comments, reviews, timeline) because the GitHub API returned HTTP 403. The GH_TOKEN provided to the sandbox was invalid or lacked the necessary scopes. The agent could only analyze the static repository configuration, not the actual workflow that triggered the retro.
Without API access, the retro agent cannot: (1) reconstruct the timeline of agent interactions, (2) compare agent review vs human review, (3) measure rework rate, (4) assess token cost, or (5) evaluate autonomy readiness. All five optimization goals require visibility into the PR's lifecycle.
What could go better
The retro agent should have read access to the target repository's PRs, issues, and comments. This is a prerequisite for any meaningful retro analysis. Without it, the agent produces low-value output that wastes compute resources.
The harness should either: (a) inject a valid GH_TOKEN with read access into the retro sandbox, or (b) prefetch the PR data (comments, reviews, diff, timeline) during the pre-script and make it available as a file in the sandbox. Option (b) is more secure since it limits the token's exposure.
Confidence: High — this is directly observable from the 403 errors in this run.
Proposed change
In the retro agent's harness configuration (or the reusable-retro.yml workflow), ensure the sandbox has access to PR data. Two approaches:
Option A (preferred): In the pre-script for the retro agent, use the runner's GITHUB_TOKEN to fetch PR metadata and write it to a JSON file that gets mounted into the sandbox:
Option B: Pass a read-only GH_TOKEN into the retro sandbox environment, similar to how the review harness passes GH_TOKEN.
The retro agent definition should document which data sources it expects and fail fast with a clear error if they're missing.
Validation criteria
Generated by retro agent from https://github.com/quay/quay-konflux-components/pull/3174