Skip to content

planner: shared registration files (tests/kunit/Kbuild) in every task scope serialize whole plan into one-task waves #20

Description

@rasatpetabit

Summary

During execution of the io-perf-caching run in OpenXCVR/openxcvr-kmod, the merged plan produced 12 tasks in 12 one-task waves — zero intra-wave parallelism for the entire run — even though the early subsystems (safety gate, probe governor, crack fairness, cache store) are genuinely independent. The execute workflow's per-wave parallel fan-out (one mp-implementer per task) never got more than one task to fan out.

Root cause

Same-wave tasks must be file-disjoint (enforced by mp validate-plan-index / the merge in mp merge-plan-fragments). Every one of the 12 tasks declared the shared KUnit registration file tests/kunit/Kbuild in its files scope (each task registers its new test there). One shared file in every scope ⇒ every task pair conflicts ⇒ the wave assignment degenerates into a fully serial chain. Secondary real overlaps (core/io.c in 3 tasks, core/port.c/core/cache.c in 2, Documentation/xcvr/index.rst in 2) reinforced it, but the Kbuild collision alone was sufficient to serialize everything.

This is a plan-authoring pattern the planner agents (mp-planner, mp-subsystem-planner) fall into naturally: "add a test" honestly implies "touch the shared registration file", and the disjointness rule then quietly costs all parallelism. Nothing surfaced this — the plan validated clean and executed correctly, just serially.

Suggestions for future planning

Any of these (roughly in order of leverage):

  1. Planner prompt guidance: teach mp-planner / mp-subsystem-planner to identify shared registration/wiring files (Kbuild, Makefile, Kconfig, doc index.rst, CI workflow yml) and either (a) route all wiring edits into a single final "wiring" task so feature tasks stay disjoint, or (b) use per-task fragment files (e.g. one Kbuild.d/<test>.inc per task, or per-test Kconfig stanzas) when the build system allows include-style composition.
  2. Merge-time diagnostic: mp merge-plan-fragments (or validate-plan-index) already computes the conflict graph — when the wave chain length is dominated by a single shared path (e.g. one file present in >N% of task scopes), emit a WARN naming the file and the wave-count delta vs. the plan with that file hoisted ("tests/kunit/Kbuild alone serializes 12 tasks into 12 waves; hoisting it would yield 5 waves"). Cheap, deterministic, and it turns an invisible cost into a REVISE-able finding at the plan gate.
  3. Plan-reviewer check: add "shared-file serialization" to mp-plan-reviewer's checklist so a fully-serial plan for a decomposable spec is at least a REVISE finding.

Option 2 is probably the sweet spot: it needs no planner-behavior change to be useful and makes the tradeoff visible at exactly the gate where a human approves the plan.

Where to find the evidence / transcript

  • Host: epyc2 (machine-id 413300f4287b443a98500854b33bddb9), user ras, date 2026-07-17.
  • Run bundle: /srv/dev/openxcvr/openxcvr-kmod/docs/masterplan/io-perf-caching/plan.index.json shows all 12 tasks with tests/kunit/Kbuild in scope, waves 0–11, one task each; state.yml, events.jsonl record the per-wave execution history.
  • Claude Code session transcript: session id 0ad11de4-383b-40f5-89f3-5e63daa2e9ad, project dir /home/ras/.claude/projects/-srv-dev-openxcvr-openxcvr-kmod/. The analysis exchange ("is there a reason this is only running a single task?") happens right after the wave-10 launch_workflow dispatch.
  • Wave-10 workflow run: run id wf_1a697b78-528, journal at /home/ras/.claude/projects/-srv-dev-openxcvr-openxcvr-kmod/0ad11de4-383b-40f5-89f3-5e63daa2e9ad/subagents/workflows/wf_1a697b78-528/journal.jsonl.
  • Masterplan version: plugin cache rasatpetabit-masterplan/masterplan/9.5.0.

Non-goal

Nothing malfunctioned: merge, validation, D6 scope enforcement, and the wave executor all behaved per spec. This is purely a planning-quality/throughput improvement — the disjointness rule is correct (parallel writers share one worktree), the plans just need to stop feeding it a universal conflict file.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions