agent-optimize: require merging disjoint-cluster accepted candidates before finalize - #482
Merged
Merged
Conversation
…before finalize merge_search.py already exists to combine disjoint-cluster survivors via integrate.py/funcmerge.py, but nothing required the optimizer's agent session to invoke it before sealing test — so a run's reported "best" could be whichever single-cluster fix scored highest, never a genuine combination of accepted candidates that each fixed a different subset of tasks. - SKILL.md / references/algorithm.md: make running merge_search.py on 2+ accepted disjoint-cluster candidates (then gating the merge like any candidate) an explicit, required step before any end-of-run measurement, not just an available tool. - merge_search.check_merge_compliance(): reads graph.jsonl for accepted candidates with disjoint target task ids and no edit_kind=="merge" node anywhere in the graph, for use as an audit signal. Never blocks or forces a decision. - measure.py: runs the check at finalize time and logs merge_compliance_warning to events.jsonl when tripped, so the omission is visible in the dashboard's activity log the same way agent_optimize_compliance already is for the screen ladder. Tests: core/tests/test_merge_compliance_check.py covers the positive case (2+ disjoint accepted clusters, no merge attempt), and the negative cases (a merge was attempted, fewer than 2 accepted candidates, non-disjoint clusters, unknown targets). Signed-off-by: Osher-Elhadad <Osher.Elhadad@ibm.com>
Contributor
|
❌ Automatic Labeling Failed An error occurred while trying to automatically label this pull request. Please check the workflow logs for details and add labels manually. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
merge_search.py(agent-optimize Phase 4: merge-as-graph-search across round survivors #438) already exists to combine disjoint-cluster survivors viaintegrate.py/funcmerge.py, but nothing required the optimizer's agent session to actually invoke it before sealing test — so a run's reported "best" could be whichever single-cluster fix happened to score highest, never a genuine combination of accepted candidates that each fixed a different subset of tasks.SKILL.md/references/algorithm.md: makes runningmerge_search.pyon 2+ accepted disjoint-cluster candidates (then gating the merge like any candidate) an explicit, required step before any end-of-run measurement, not just an available tool nobody reaches for under time pressure.merge_search.check_merge_compliance(): a new function that readsgraph.jsonlfor accepted candidates with pairwise-disjoint target task ids and checks whether anyedit_kind == "merge"node exists anywhere in the graph — the same kind of code-level audit signalround.py's existingagent_optimize_complianceevent provides for the screen ladder. It never blocks or forces a decision — this is an agent-driven framework wherehost.pyowns no algorithm decisions, so enforcement stays a prompt-level requirement; this is only the observability half.measure.pyruns the check at finalize time and logs amerge_compliance_warningevent toevents.jsonlwhen tripped, so the omission is visible in the dashboard's activity log (every event renders there regardless of kind) exactly like other compliance signals already are.Test plan
core/tests/test_merge_compliance_check.py— new unit tests: positive case (2+ accepted disjoint-cluster candidates, no merge attempt anywhere in the graph → flagged), and negative cases (a merge was already attempted; fewer than 2 accepted candidates; accepted clusters overlap; accepted candidates have no recorded target ids at all).python -m pytest core/tests/ -q— full suite, 1243 passed, 12 skipped, no regressions.python skills/_registry/lint_skills.py skills— no authoring violations;agent-optimize'sSKILL.mdbody stays under the 5000-token budget.