feat(#4026): support disabling agents via config.yaml enabled field#4049
feat(#4026): support disabling agents via config.yaml enabled field#4049fullsend-ai-coder[bot] wants to merge 1 commit into
Conversation
E2E tests did not runE2E tests run automatically for org/repo members and collaborators on pull requests. For other contributors, a maintainer must add the See E2E testing guide for details. |
Site previewPreview: https://024c9b6d-site.fullsend-ai.workers.dev Commit: |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
ascerra
left a comment
There was a problem hiding this comment.
Review: enabled: false does not stop agents from running
Problem
This PR filters disabled agents out of MergedAgents(), but that is not enough to disable them at runtime. Two independent gates still allow the agent to run.
Gap 1 — resolveAgentSource treats “missing” as “fall back”
In internal/cli/run.go:
agent := config.LookupMergedAgent(merged, agentName)
if agent == nil || !agent.IsConfig {
// tryAgentsRepoFallback → resolveHarnessPath
}Deleting an agent from MergedAgents makes Lookup return nil, which takes the same path as a normal scaffold agent. First-party names (triage, code, fix, review, retro, prioritize) are in defaultAgentsRepoKnownAgents and will still resolve via agents-repo fallback (or disk).
So for the stated use case (disable built-in scaffold agents), filtering in MergedAgents alone is a no-op at runtime. The PR claim that no run.go changes are needed is incorrect.
Gap 2 — dispatch ignores agents: entirely
Org dispatch.yml gates only on defaults.roles (yq '.defaults.roles[]'). It never reads agents[].enabled. With retro still in defaults.roles, dispatch still fires even if config says enabled: false.
What works today vs what this PR needs
| Goal | Works with this PR? |
|---|---|
| Disable custom (non-first-party) agent | Partially — may error on missing harness, not clean skip |
Disable built-in (retro, etc.) |
No — fallback + dispatch still run it |
Required fix (for another agent)
resolveAgentSource: Distinguish “explicitly disabled” from “not in merged set.” If config has{name: X, enabled: false}, return a clear skip/error — do not calltryAgentsRepoFallback/resolveHarnessPath.- Dispatch (scaffold
dispatch.yml+reusable-dispatch.yml): Honor disabled agents (or document that operators must also remove the role fromdefaults.roles— but that contradicts the issue goal of keeping workflow infra). - Tests: Add
resolveAgentSourcecoverage provingenabled: falseonretro/triagedoes not resolve a harness via fallback. Current tests only coverMergedAgentsin isolation.
Minimal repro config
defaults:
roles: [fullsend, triage, coder, review, retro, prioritize]
agents:
- name: retro
enabled: falseExpected after fix: retro does not dispatch / does not resolve a harness.
Actual with this PR: MergedAgents drops retro; dispatch + fallback still run it.
Refs: internal/cli/run.go (resolveAgentSource, defaultAgentsRepoKnownAgents), internal/scaffold/fullsend-repo/.github/workflows/dispatch.yml (Check role is enabled).
|
/fs-fix rebase and fix the issues raised in #4049 (review) and also #4049 (comment) |
|
🤖 Finished Fix · ✅ Success · Started 2:50 PM UTC · Completed 3:02 PM UTC |
🔧 Fix agent — iteration 1 (human-triggered)Addressed 4 of 5 review findings. Fixed resolveAgentSource to block disabled agents from resolving via fallback paths, added IsAgentExplicitlyDisabled helper, and added 11 tests. Disagreed with dispatch workflow changes (deferred to follow-up PR since resolveAgentSource now enforces the safety guarantee). Fixed (4):
Disagreed (1):
Tests: passed Decision points
Updated by fullsend fix agent |
Add IsAgentExplicitlyDisabled() to distinguish 'explicitly disabled in config' from 'not in config at all'. Update resolveAgentSource() to check this before falling through to agents-repo fallback or disk lookup — without this, setting enabled: false on a built-in agent (triage, retro, etc.) was a no-op because the fallback paths still resolved the harness. Add unit tests for IsAgentExplicitlyDisabled and integration tests for resolveAgentSource proving that enabled: false blocks all fallback resolution paths. Addresses review feedback on #4049
|
/fs-fix Rebase first. Add a dispatch-side check for agents[].enabled in internal/scaffold/fullsend-repo/.github/workflows/dispatch.yml. After the "Check role is enabled" step (line 340), add a new step "Check agent is enabled" that reads agents[].enabled from config.yaml for the matched stage. If the agent for that stage has enabled: false, skip dispatch (set skipped=true the same way the role check does). Use yq to check — something like: look up the agent entry whose name matches $STAGE and check if its enabled field is explicitly false. If there is no matching agent entry, or if enabled is omitted/true, proceed normally (default-enabled, backward compatible). Gate subsequent steps on this new step's skipped output the same way they gate on role-check.outputs.skipped. The resolveAgentSource check in run.go is correct and must stay — it's the safety net for direct fullsend run invocations. The dispatch check is the efficiency gate that prevents wasting a runner on an agent that will just fail at resolution time. |
|
🤖 Finished Fix · ❌ Failure · Started 3:49 PM UTC · Completed 3:59 PM UTC |
553b42f to
5fd307b
Compare
|
/fs-review |
|
🤖 Finished Review · ✅ Success · Started 5:32 PM UTC · Completed 5:49 PM UTC |
ReviewRe-review of same SHA This PR adds an No code changes since prior review. All prior findings remain applicable at their anchored severities. FindingsMedium
Low
Labels: PR modifies agent configuration (internal/config/), dispatch workflows (.github/workflows/, internal/scaffold/), and documentation (docs/) Previous runReviewRe-review of force-push from This PR adds an Improvement since prior review: The yq queries in both dispatch workflows now use FindingsMedium
Low
Labels: PR modifies agent configuration (internal/config/), dispatch workflows (.github/workflows/, internal/scaffold/), and documentation (docs/) Labels: PR modifies agent configuration (internal/config/) and dispatch workflows. Labels: PR modifies agent configuration (internal/config/) and dispatch workflows. Labels: PR modifies agent configuration (internal/config/) and dispatch workflows. Previous run (2)ReviewRe-review of force-push from FindingsMedium
Low
Labels: PR modifies agent configuration (internal/config/) and dispatch workflows (.github/workflows/, internal/scaffold/) Previous run (3)ReviewRe-review of force-push from This PR adds an No new issues since prior review. The force-push from FindingsMedium
Previous run (4)ReviewRe-review of force-push from This PR adds an No new issues since prior review. The force-push from FindingsMedium
Previous run (5)ReviewRe-review of force-push from This PR adds an Improvement since prior review: The force-push addressed the prior review's case-sensitivity finding — both dispatch workflows now use FindingsMedium
Previous runReviewRe-review of force-push from This PR adds an FindingsMedium
Low
Previous runReviewRe-review of force-push from This PR adds an Improvement since prior review: The force-push addressed four documentation findings from the prior review — ADR 0058 is now annotated with the FindingsMedium
Low
Previous runReviewFindingsMedium
Low
Previous run (6)ReviewRe-review of force-push from This PR adds an Improvement since prior review: The force-push addressed the prior review's case-sensitivity finding — both dispatch workflows now use FindingsMedium
Previous run (7)ReviewRe-review of force-push from This PR adds an FindingsMedium
Low
Previous runReviewRe-review of force-push from This PR adds an Improvement since prior review: The force-push addressed four documentation findings from the prior review — ADR 0058 is now annotated with the FindingsMedium
Low
Previous runReviewFindingsMedium
Low
Previous run (8)ReviewRe-review of force-push from This PR adds an FindingsMedium
Low
Previous run (9)ReviewRe-review of force-push from This PR adds an Improvement since prior review: The force-push addressed four documentation findings from the prior review — ADR 0058 is now annotated with the FindingsMedium
Low
Previous runReviewFindingsMedium
Low
Previous run (10)ReviewRe-review of force-push from This PR adds an Improvement since prior review: The force-push addressed four documentation findings from the prior review — ADR 0058 is now annotated with the FindingsMedium
Low
Previous run (11)ReviewFindingsMedium
Low
|
5fd307b to
99148e2
Compare
|
/fs-review |
|
🤖 Finished Review · ✅ Success · Started 6:02 PM UTC · Completed 6:17 PM UTC |
99148e2 to
43ec352
Compare
ascerra
left a comment
There was a problem hiding this comment.
approve as long as testing passes
|
/fs-review |
|
🤖 Finished Review · ✅ Success · Started 6:42 PM UTC · Completed 6:57 PM UTC |
|
/fs-review |
|
🤖 Finished Review · ✅ Success · Started 7:32 PM UTC · Completed 7:47 PM UTC |
waynesun09
left a comment
There was a problem hiding this comment.
Multi-agent review squad (Claude, Gemini, Codex reviewers) — posting the verified MEDIUM+ findings as inline comments. TL;DR: the Go layer is solid and well tested, but the new workflow agent-check gate is non-functional due to a jq-vs-yq builtin mixup (ascii_downcase → downcase), and the silent error-swallowing around the yq call is what masked it. One latent duplicate-entry bug in MergedAgents was also reproduced with a failing test. Details inline.
22d221a to
f72c816
Compare
|
/fs-review |
|
🤖 Finished Review · ✅ Success · Started 12:59 PM UTC · Completed 1:09 PM UTC |
waynesun09
left a comment
There was a problem hiding this comment.
Follow-up multi-agent review squad pass (4 agents dispatched: claude-coder, claude-researcher, gemini-code-review, cursor-code-review). Transparency note: the Gemini/Antigravity CLI leg failed outright (quota exhausted, ~46h reset) and the Codex/cursor-agent leg returned "no issues found" — in both cases the wrapping Claude agent did its own independent investigation instead of returning nothing. So this round is effectively 4 independent Claude review passes, not 3-model diversity.
TL;DR: the Go merge/validate/resolve logic and the dispatch-gating workflow step are correct for the core single-entry suppression case (the gaps from the earlier CHANGES_REQUESTED review and the ascii_downcase/fail-open/duplicate-entry issues above all look fixed). Four new findings below: one HIGH — validation actually rejects the disable-then-override-same-name pattern that this PR's own MergedAgents tests were written to support (related to, but distinct from, the duplicate-entries bug already fixed in the thread above) — plus three MEDIUM policy/docs gaps (an edit to the Decision section of the already-Accepted ADR 0058 without a PR-description callout, a coder-role vs code-agent-name mismatch that can make a disable entry silently no-op, and no pre-merge validation of config.yaml so a single bad enabled entry only surfaces as a whole-config failure at dispatch/run time). Details inline.
|
/fs-fix #4049 (review) |
|
🤖 Finished Fix · ✅ Success · Started 1:54 PM UTC · Completed 2:10 PM UTC |
🔧 Fix agent — iteration 1 (human-triggered)Fixed 2 of 4 review findings. (1) Fixed HIGH-severity ValidateAgentEntries bug: validation now accepts disable-then-enable and enable-then-disable same-name patterns, matching MergedAgents semantics. Added two new validation-layer tests. (2) Fixed MEDIUM-severity docs gap: added note about agent/harness name vs role name distinction. Disagreed with 2 findings: ADR 0058 edit callout is a PR-description process issue the fix agent cannot address; config.yaml CI validation is a new feature out of scope for this PR. Fixed (2):
Disagreed (2):
Tests: passed Updated by fullsend fix agent |
ValidateAgentEntries shared a single seen map that rejected any two entries with the same name, regardless of enabled state. This contradicted MergedAgents which supports disable-then-enable and enable-then-disable patterns (last-writer-wins). Change the seen map to track the disabled state of each entry so that same-name entries with different enabled states are accepted (matching MergedAgents semantics), while true duplicates (two entries with the same state) are still rejected. Also document that the disable name must match the agent/harness name (e.g. code), not the role name (e.g. coder), to prevent silent no-ops. Addresses review feedback on #4049
50eeb5d to
c716b69
Compare
|
/fs-review |
|
🤖 Finished Review · ❌ Failure · Started 8:48 PM UTC · Completed 9:05 PM UTC |
Add `Enabled *bool` to `AgentEntry` so operators can selectively disable agents (including scaffold defaults) without removing them from config. Nil defaults to enabled for backward compatibility. - Filter disabled agents in `MergedAgents()` and block fallback in `resolveAgentSource` - Gate dispatch on `agents[].enabled` in scaffold and reusable workflows with last-writer-wins semantics for enable-then-disable patterns - Add pre-dispatch config validation to catch malformed entries - Allow suppression-only entries (`enabled: false`, no source) to disable scaffold defaults by name - Allow disable-then-enable and enable-then-disable patterns for the same agent name in validation Closes #4026 Signed-off-by: Claude <noreply@anthropic.com> Signed-off-by: Greg Allen <gallen@redhat.com>
c716b69 to
df6d8b6
Compare
|
/fs-review |
|
🤖 Finished Review · ❌ Failure · Started 12:52 AM UTC · Completed 1:03 AM UTC |
|
/fs-review |
|
🤖 Finished Review · ✅ Success · Started 1:04 AM UTC · Completed 1:16 AM UTC |
Summary
Enabled *boolfield toAgentEntrystruct so operators can selectively disable agents (including built-in scaffold agents) without removing them from configurationMergedAgents()so they are excluded from the merged result setresolveAgentSourceagents[].enabledin both scaffold and reusable dispatch workflowsenabled: falsewith nosource) that exist solely to disable scaffold defaults by nameRelated Issue
Closes #4026
ADR Edits
This PR edits the Decision section of ADR 0058 (Agent Registration) (status: Accepted, 2026-06-29) to add an
enabled: falseexample to theAgentEntrystruct documentation. The edit extends the existing struct definition with the new field — it does not change the decision or its rationale.Changes
internal/config/config.goEnabled *boolfield withyaml:"enabled,omitempty"tag toAgentEntryIsEnabled()helper method that returnstruewhenEnabledis nil (default) or explicitlytrueValidateAgentEntries()to accept suppression-only entries (no source,enabled: false) as long as they have an explicit nameinternal/config/agents.goMergedAgents()to skip disabled config entries and remove any scaffold entries they overrideinOrderset to prevent duplicate entries when a disabled entry is followed by an enabled one with a different nameIsAgentExplicitlyDisabled()to distinguish "explicitly disabled" from "not in config"internal/cli/run.goresolveAgentSourcebefore agents-repo/disk fallback to prevent disabled agents from resolving via other sourcesinternal/scaffold/fullsend-repo/.github/workflows/dispatch.yml::warning::on failure instead of silently swallowing errors.github/workflows/reusable-dispatch.yml.fullsend/config.yamlpath and file-existence guarddocs/guides/user/customizing-agents.mdnamemust match agent/harness name (code,fix, etc.), not role name (coder)docs/ADRs/0058-agent-registration.mdenabled: falseexample to Decision section (see ADR Edits above)docs/plans/agent-registration.md,docs/plans/agent-extraction-to-agents-repo.mdEnabled *boolfield andenabled: falseexampleTests
internal/config/agents_test.go: 7 new tests — disabled scaffold/custom agents, suppression-only, enabled defaults, duplicate-name-rejected-by-validation,IsAgentExplicitlyDisabled(7 cases)internal/config/config_test.go: Suppression validation, disabled-with-source,IsEnabledunit tests, YAML round-trip testsinternal/cli/run_test.go: 4 tests — disabled/suppression blocks fallback, enabled still resolvesTesting
internal/config/...tests passinternal/cli/...resolveAgentSource tests passgo vet ./...cleango build ./...cleanChecklist
downcase(yq), notascii_downcase(jq)