diff --git a/projects/bravebot/docs/clustering.md b/projects/bravebot/docs/clustering.md new file mode 100644 index 0000000..5543685 --- /dev/null +++ b/projects/bravebot/docs/clustering.md @@ -0,0 +1,158 @@ +# bravebot: Clustering related issues + +Read when the prompt has a `Clustering:` paragraph. It appears only when this +profile's `clustering.sizeLimits` gives the lead story's `size/` label a ceiling +above 1 and the selector found candidates no larger than the lead. No paragraph +means a single story: ignore this file. + +This adapts bravebot's own `/implement-issues` skill +(`agents/skills/implement-issues/SKILL.md` in the target repo) to one loop +iteration. Where this file is silent, that skill's rules apply, and so do +`docs/workflow-pending.md`, `docs/workflow-committed.md` and the other files in +this profile's `docs/`. Where they disagree, this file wins for a clustered +iteration, because the loop owns claims, statuses and the iteration boundary and +the skill does not know about them. + +## The ceiling + +| Lead's `size/` label | Most issues in the cluster, lead included | +|---|---| +| `size/1` | 10 | +| `size/2` | 6 | +| `size/3` | 4 | +| `size/4`, `size/5`, or none | 1 | + +The ceiling is a maximum. A cluster of one is the common result and a good one. +A cluster is only larger when the issues are related, and padding it to the +ceiling is a failure: it makes a pull request nobody can review. The table lives +in the profile (`clustering.sizeLimits`) and is enforced by +`scripts/claim-companions.py`, which refuses a story past the ceiling whatever +you ask for. The selector offers only candidates whose `size/` is labelled and no +larger than the lead's, so the lead's size is the size of the biggest thing in +the cluster. + +## Related + +Issues are related when one of these holds, and only these: + +1. one root cause; +2. one is a prerequisite of another; +3. the same function, file or spec clause changes in both; +4. their changes would conflict when combined. + +The same area, the same crate, the same label or a similar title is not +relatedness. Similar-sounding issues with different causes are separate pull +requests. When unsure whether two are related, they are: a conflict between +pull requests costs more than a larger one. When unsure whether a candidate +belongs at all, it does not, and it stays in the queue for a later iteration. + +## Steps + +Do these in order, and stop at the first that leaves you with a cluster of one. + +1. **Work out the lead's change first.** Read the lead issue with the research + steps as usual, and find what it changes: the files, functions and spec + clauses. Do this before reading any candidate. +2. **Screen the candidates cheaply.** For each candidate read the issue with + `gh issue view --repo $ISSUE_REPO` and no more. Discard it at once unless + one of the four relations above holds against the lead's change. Do not open + a candidate's code until it has passed this. A candidate that a closed or + merged pull request already covers, or that has an open pull request, is + dropped, and you say which in `progress.txt`. +3. **Claim the survivors, once.** + + ```sh + $BOT_DIR/scripts/claim-companions.py --lead ... + ``` + + It prints `claimed`, `refused` (with the reason for each) and `limit`. Only + what is in `claimed` is yours. The claim also puts `bot/in-progress` on each + companion's issue and records `clusterCompanions` on the lead, so the + companions' statuses follow the lead's and `run.sh` hands every claim back + when the iteration ends. Call it once with the whole list; a second call is + for a candidate you found related only after starting, and counts against the + same ceiling. +4. **One worktree, one branch, one pull request.** The lead's worktree + (`../bravebot-`) and branch hold all of it. Make no second + worktree for a companion: the claim is for the pull request, and the shared + `.git` cannot check one branch out twice. If two claimed issues turn out to + conflict in the same lines so that both cannot survive, keep the lead's + behaviour, drop the companion from the pull request, and record it under + Decisions in the body. Skip it with + `update-prd-status.py skipped --reason "..."` only if it is + finished with; otherwise leave it pending, and say so in `progress.txt`. +5. **Commits follow issues, not rows.** One commit per issue, with its tests, the + spec clauses it makes wrong and the catalogue entries. Several issues may share + one commit where a single change fixes them. A further commit is only for a + dependency or for a refactor that changes no behaviour and that the fix is + built on. Every commit leaves the tree building. The commit that finishes an + issue says `Closes #`, one that finishes part of it `Part of #`. Do not + split by file, platform, call site or test. +6. **Reproduce each issue before changing anything for it,** and show it in the + body. Run the reproduction on `upstream/main`. If it no longer fails, the issue + is already fixed: add the test that pins it where none exists, and the pull + request still closes it, saying so under `## The problem`. +7. **Gate once, on the tree that is pushed.** The profile's presubmit sequence + runs once on the rebased branch, covering every issue in it: `make + check-affected` as [testing.md](./testing.md) gives it, and the Docker gates + only if an issue names another platform. Running it per issue is the + costliest mistake available here. For each new or changed test, name the wrong + implementation it rejects and say whether failure was demonstrated, as + [testing.md](./testing.md#choosing-the-tests) asks, per issue. +8. **Verify each issue against its own words.** Before pushing, write a short + checklist per issue (one row for each thing the issue asks for) and check the + final tree against it, not against your memory of the work. An issue with an + unmet row is `Part of`, never `Closes`. + +## The pull request + +The body follows [pr-descriptions.md](../../../docs/pr-descriptions.md) and +`scripts/check-pr-body.py`, with these changes for a cluster: + +- **A `Closes $ISSUE_REPO#` line for each issue, one per line, at the very + top.** The lead is first. Use `Part of` for one the pull request only + partly does, and then `Remaining:` names the piece. +- **`User impact:` is one line for the whole pull request.** +- **`## The problem`, `## Reproduce` and `## The fix` are per issue,** under a + `**# **` heading each, in the same order as the `Closes` lines. + Each reproduction is that issue's own, complete from a cold start. The fix + section for an issue that shares a root cause with another says so once and + points at it rather than repeating it. +- **`## Decisions to confirm`,** above `## The problem`, lists anything a + maintainer could reverse: a reading of an ambiguous issue, an issue dropped + from the cluster, one issue's behaviour given up for another's, a new flag or + setting, a permission widened. Each entry gives the issue, what was chosen, the + alternative, why, and what reversing it takes. Omit the heading if there are + none. +- **The title is one sentence naming what changes for a person,** no issue + numbers, no prefix. If it needs "and" between unrelated things, the cluster is + wrong. +- **Labels:** none are defined for this profile (`labels.pr` is empty). The + `bot/in-progress` labels the claim put on the companions' issues come off when + the iteration ends, with no action from you. + +Open the pull request once, with the `committed` then `pushed` flow in +`docs/workflow-committed.md` for the **lead**: + +```sh +python3 $BOT_DIR/scripts/update-prd-status.py committed --branch +python3 $BOT_DIR/scripts/update-prd-status.py pushed --pr-number +``` + +`update-prd-status.py` moves every claimed companion with the lead, to the same +branch and the same PR number, and prints a line for each. Do not run it for a +companion. Check the output names each one; a companion it did not name was not +claimed, and its issue is not in the cluster. Afterwards the PR belongs to all +of them: the syncs and the review loop key on the PR number, so a merge retires +all the stories together and a review fix round works the one branch. + +If the lead is skipped or invalid, its companions return to the queue as +pending. If you drop a companion mid-way and it is not finished with, leave it +pending and remove it from the body; its claim is released with the rest when +the iteration ends. + +## What to write in progress.txt + +The ordinary entry for the lead, plus one line per candidate: claimed and in the +pull request, claimed and dropped (why), or not claimed (why). A reviewer of the +loop reads that to learn whether the screening was too eager or too shy. diff --git a/projects/bravebot/profile.json b/projects/bravebot/profile.json index ecd5d6f..4757d85 100644 --- a/projects/bravebot/profile.json +++ b/projects/bravebot/profile.json @@ -68,6 +68,15 @@ "Report nothing that `make check` already decides: formatting, lints, clause numbering, lockfiles." ] }, + "clustering": { + "sizeLimits": { + "1": 10, + "2": 6, + "3": 4, + "4": 1, + "5": 1 + } + }, "uiPaths": [ "crates/tui/" ], diff --git a/run.sh b/run.sh index 0ed9964..5f04846 100755 --- a/run.sh +++ b/run.sh @@ -416,8 +416,12 @@ release_claim() { [ -n "$CURRENT_STORY_ID" ] || return 0 # stderr stays visible: a failed label release is otherwise a label that # blocks the issue on every machine with no trace of why. + # Release by slot, not by story: a clustered iteration holds companions beside + # its lead (scripts/claim-companions.py), and a claim left behind would keep + # them out of the queue for as long as this run lives. A slot holds nothing + # between iterations, so everything it holds is this iteration's. "$SCRIPT_DIR/scripts/claims.py" release \ - --story "$CURRENT_STORY_ID" --slot "$BOT_RUN_SLOT" >/dev/null || true + --slot "$BOT_RUN_SLOT" >/dev/null || true CURRENT_STORY_ID="" } @@ -850,6 +854,24 @@ $STORY_DETAILS Bot config (from config.json — do NOT read this file): $BOT_CONFIG" + # Only said when the profile clusters and the lead can take companions + # (select-task.py's "clustering" block). The ceiling is a maximum: the doc + # tells the agent to take only issues that are really related, and + # claim-companions.py enforces the ceiling whatever the agent asks for. + CLUSTER_LIMIT=$(echo "$TASK_JSON" | jq -r '.clustering.limit // empty') + if [ -n "$CLUSTER_LIMIT" ]; then + CLUSTER_POOL=$(echo "$TASK_JSON" | jq -r '.clustering.candidates | map("- \(.storyId): issue #\(.issueNumber), \"\(.title)\"" + (if (.triage | length) > 0 then " (" + (.triage | to_entries | map("\(.key) \(.value)") | join(", ")) + ")" else "" end)) | join("\n")') + CLUSTER_SIZE=$(echo "$TASK_JSON" | jq -r '.clustering.leadSize') + AGENT_PROMPT="$AGENT_PROMPT + +Clustering: $STORY_ID is size $CLUSTER_SIZE, so this iteration may fix up to $CLUSTER_LIMIT related issues in +one pull request, $STORY_ID included. That is a ceiling, not a target: take an issue only if it shares a root +cause, a file or function, or a spec clause with $STORY_ID, and take none if none is. Read +./$BOT_DIRNAME/projects/$BOT_PROFILE/docs/clustering.md before deciding, and follow it for the claim, the +worktree, the commits and the PR body. Candidates (each no larger than $STORY_ID): +$CLUSTER_POOL" + fi + # Without this the session reads requeuedFrom's reason as a live blocker and # skips the story again, which is the judgment the operator just overruled. REQUEUED_FROM=$(echo "$TASK_JSON" | jq -r '.requeuedFrom.status // empty') diff --git a/scripts/claim-companions.py b/scripts/claim-companions.py new file mode 100755 index 0000000..164c0ca --- /dev/null +++ b/scripts/claim-companions.py @@ -0,0 +1,187 @@ +#!/usr/bin/env python3 +"""Claim companion stories for a clustered iteration. + +select-task.py claims the lead story and, for a profile with a `clustering` +block, offers a pool of stories that could share the iteration (see +lib/cluster.py). The agent reads the issues, decides which are genuinely related +to the lead, and asks for them here. Related is a judgement about the issues' +text and code, so it stays with the agent; the ceiling, the claims and the +labels do not, so they are enforced here, under the PRD lock, where a model +cannot talk its way past them. + + ./scripts/claim-companions.py --lead US-012 US-015 US-031 + +A companion is claimed exactly like the lead: the same slot, pid and run, so it +is released with the lead when the iteration ends (run.sh releases by slot), and +a run elsewhere cannot select it. It is also recorded as checked this run. + +A story that cannot be taken is refused with its reason and the rest are still +claimed. Only a lead this slot does not hold fails the call. + +Prints {"claimed": [...], "refused": {id: reason}, "limit": N, "taken": N}. + +Exit codes: + 0 - the call was valid (check "claimed": it may be empty) + 1 - the lead is not claimed by this slot, or the profile does not cluster it + 2 - error +""" + +import argparse +import json +import os +import sys + +sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) +from lib import claims as claims_lib +from lib import cluster, issue_lock +from lib.issue_lock import issue_number +from lib.load_config import load_config, load_profile +from lib.prd_store import bot_dir_for, load_prd, prd_lock, save_prd + + +def _record_checked(run_state_path, story_ids): + """Add the companions to storiesCheckedThisRun, as selection does for the lead.""" + try: + with open(run_state_path) as f: + run_state = json.load(f) + except (OSError, json.JSONDecodeError): + return + checked = run_state.get("storiesCheckedThisRun", []) + for sid in story_ids: + if sid not in checked: + checked.append(sid) + run_state["storiesCheckedThisRun"] = checked + save_prd(run_state_path, run_state) + + +def claim_companions(bot_dir, prd_path, lead_id, wanted, slot, profile, in_progress): + """Claim what can be claimed. Returns (result, exit_code); under the PRD lock.""" + prd = load_prd(prd_path) + stories = prd.get("stories", []) + by_id = {s.get("id"): s for s in stories} + lead = by_id.get(lead_id) + + held = claims_lib.active(bot_dir, locked=True) + lead_claim = held.get(lead_id) + if lead is None or lead_claim is None or int(lead_claim.get("slot", -1)) != slot: + return {"error": f"{lead_id} is not claimed by slot {slot}"}, 1 + + limit = cluster.ceiling(profile, lead) + if limit <= 1: + return { + "error": f"the profile does not cluster {lead_id} " + f"(size {cluster.lead_size(lead)})" + }, 1 + + already = [sid for sid in lead.get("clusterCompanions", []) if sid in by_id] + claimed, refused = [], {} + # Never the labels this machine's own claims put on: those are not another + # machine's work. Same carve-out select-task.py makes. + blocked = set(in_progress) - { + issue_number(s) for s in stories if s.get("id") in held + } + + for sid in dict.fromkeys(wanted): + if sid == lead_id or sid in already: + refused[sid] = "already in this cluster" + continue + story = by_id.get(sid) + if story is None: + refused[sid] = "no such story" + continue + if 1 + len(already) + len(claimed) >= limit: + refused[sid] = ( + f"the ceiling for a size-{cluster.lead_size(lead)} lead is {limit}" + ) + continue + # One story, judged with the same rules the pool was built with. + if not cluster.companion_pool(lead, [story], limit, blocked): + refused[sid] = ( + "not pending, has a PR, names no issue, is unlabelled or larger " + "than the lead, or its issue is in progress elsewhere" + ) + continue + other = held.get(sid) + if other and int(other.get("slot", -1)) != slot: + refused[sid] = f"another run holds it (slot {other.get('slot')})" + continue + if not claims_lib.claim( + bot_dir, + sid, + slot, + lead_claim.get("pid", 0), + lead_claim.get("runId"), + locked=True, + ): + refused[sid] = "another run claimed it just now" + continue + if not issue_lock.acquire(issue_number(story), bot_dir=bot_dir): + claims_lib.release(bot_dir, story_id=sid, slot=slot, locked=True) + refused[sid] = "its issue was labelled in progress by another machine" + continue + claimed.append(sid) + + if claimed: + lead["clusterCompanions"] = already + claimed + for sid in claimed: + by_id[sid]["clusterLead"] = lead_id + save_prd(prd_path, prd) + + return { + "claimed": claimed, + "refused": refused, + "limit": limit, + "taken": 1 + len(already) + len(claimed), + }, 0 + + +def main(): + parser = argparse.ArgumentParser(description=__doc__.split("\n")[0]) + parser.add_argument("--prd", help="Path to prd.json") + parser.add_argument("--run-state", help="Path to run-state.json") + parser.add_argument("--lead", required=True, help="The story this iteration leads") + parser.add_argument( + "--slot", + type=int, + default=int(os.environ.get("BOT_RUN_SLOT", "1")), + ) + parser.add_argument("stories", nargs="+", help="Companion story ids") + args = parser.parse_args() + + script_dir = os.path.dirname(os.path.abspath(__file__)) + prd_path = args.prd or os.path.join(os.path.dirname(script_dir), "data", "prd.json") + bot_dir = bot_dir_for(prd_path, os.path.dirname(script_dir)) + run_state_path = ( + args.run_state + or os.environ.get("BOT_RUN_STATE_FILE") + or os.path.join(bot_dir, "data", "run-state.json") + ) + + profile = load_profile(load_config(), bot_dir) + wanted = [s.upper() for s in args.stories] + # Read before the lock, as select-task.py does: it is a network call. + in_progress = issue_lock.in_progress(bot_dir=bot_dir) + + try: + with prd_lock(prd_path): + result, code = claim_companions( + bot_dir, + prd_path, + args.lead.upper(), + wanted, + args.slot, + profile, + in_progress, + ) + except (OSError, ValueError, TimeoutError) as e: + print(f"Error: {e}", file=sys.stderr) + return 2 + + if result.get("claimed"): + _record_checked(run_state_path, result["claimed"]) + print(json.dumps(result)) + return code + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/lib/cluster.py b/scripts/lib/cluster.py new file mode 100644 index 0000000..ce540fa --- /dev/null +++ b/scripts/lib/cluster.py @@ -0,0 +1,119 @@ +"""Clustering: how many related issues one iteration may take together. + +A profile opts in with a `clustering` block in profile.json: + + "clustering": {"sizeLimits": {"1": 10, "2": 6, "3": 4, "4": 1, "5": 1}} + +The key is the `size` triage axis of the story the iteration is *led* by (see +lib/triage.py), the value the most stories that iteration may work in one pull +request, the lead included. It is a ceiling, never a target: whether two issues +are related is a judgement that needs the issues read, which belongs to the +agent (projects/bravebot/docs/clustering.md), and a lead with nothing related +is worked alone. + +This module decides only the two things a script can decide from the PRD: the +ceiling, and which other stories the agent may be offered as companions. A +profile with no `clustering` block gets a ceiling of 1 and no companions, so +nothing changes for it. +""" + +from . import triage +from .issue_lock import issue_number + +# How many companions are listed for the agent to choose from. The ceiling bounds +# how many it may take; the pool is wider than that so that a lead whose related +# issues are not the next ones in the queue still finds them. It is also a bound +# on what the prompt carries. +POOL_CAP = 40 + + +def size_limits(profile): + """The profile's {size: ceiling} as ints, or {} when it does not cluster. + + A malformed entry is dropped rather than raised: a bad number in a profile + must not stop a run, and dropping it falls back to working one story. + """ + raw = (profile.get("clustering") or {}).get("sizeLimits") or {} + limits = {} + for key, count in raw.items(): + try: + size, count = int(key), int(count) + except (TypeError, ValueError): + continue + if triage.MIN_VALUE <= size <= triage.MAX_VALUE and count >= 1: + limits[size] = count + return limits + + +def lead_size(story): + """The lead's size label as an int, or None when it has none. + + Unlike ordering, where a missing axis is the middle of the range, an + unlabelled size is treated as unknown here: how much work it is decides how + many can share an iteration, and guessing high would overload one. + """ + raw = (story.get("triage") or {}).get("size") + try: + size = int(raw) + except (TypeError, ValueError): + return None + if triage.MIN_VALUE <= size <= triage.MAX_VALUE: + return size + return None + + +def ceiling(profile, story): + """The most stories one iteration led by ``story`` may work, itself included.""" + size = lead_size(story) + if size is None: + return 1 + return size_limits(profile).get(size, 1) + + +def companion_pool(lead, ordered, limit, blocked_issues=()): + """Stories the agent may add to ``lead``, in the order they were given. + + ``ordered`` is the candidate list already filtered (not claimed elsewhere, not + checked this run, not terminal) and sorted the way selection sorts it. From it + this keeps what could share a pull request with the lead: + + - pending, and not quarantined (the caller passes the quarantine test); + - it names an issue, since a story with none cannot be closed by a pull request; + - its issue is not in progress on another machine; + - its size is labelled and no larger than the lead's, so the lead's size is the + size of the biggest thing in the cluster and the ceiling means what it says. + + Returns [] when the ceiling is 1. + """ + if limit <= 1: + return [] + size = lead_size(lead) + if size is None: + return [] + blocked = set(blocked_issues) + pool = [] + for story in ordered: + if story.get("id") == lead.get("id"): + continue + if story.get("status", "pending") != "pending": + continue + number = issue_number(story) + if not number or number in blocked: + continue + other = lead_size(story) + if other is None or other > size: + continue + pool.append(story) + if len(pool) >= POOL_CAP: + break + return pool + + +def pool_entry(story): + """One companion as the agent is shown it: enough to judge relatedness.""" + return { + "storyId": story.get("id"), + "issueNumber": issue_number(story), + "title": story.get("title", ""), + "triage": story.get("triage") or {}, + } diff --git a/scripts/select-task.py b/scripts/select-task.py index 706351c..981ab41 100755 --- a/scripts/select-task.py +++ b/scripts/select-task.py @@ -28,7 +28,7 @@ sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) from lib import claims as claims_lib -from lib import issue_lock, slots, triage +from lib import cluster, issue_lock, slots, triage from lib.issue_lock import issue_number from lib.load_config import build_research, load_config, load_profile from lib.named_target import describe_target, explicit_target @@ -116,6 +116,42 @@ def research_steps(bot_dir): return build_research(load_profile(config, bot_dir), config, bot_dir) +def clustering_offer(bot_dir, lead, candidates, blocked_issues, now): + """What the iteration may add to its lead story, or None for "work it alone". + + Only fresh pending work clusters: a story that already has a branch or a PR + is maintenance on that PR, and one PR per story is how it is tracked. The + pool is offered, never claimed here; the agent reads the issues, decides + which are related, and claims those with scripts/claim-companions.py, which + re-checks the ceiling under the PRD lock. + """ + if lead.get("status", "pending") != "pending" or lead.get("prNumber"): + return None + if pending_attempts(lead) >= MAX_PENDING_ATTEMPTS: + return None + config = load_config() + profile = load_profile(config, bot_dir) + limit = cluster.ceiling(profile, lead) + if limit <= 1: + return None + ordered = sorted( + ( + c + for c in candidates + if c.get("id") != lead.get("id") and assign_tier(c, now) == TIER_NORMAL + ), + key=lambda c: sort_key(c, now), + ) + pool = cluster.companion_pool(lead, ordered, limit, blocked_issues) + if not pool: + return None + return { + "limit": limit, + "leadSize": cluster.lead_size(lead), + "candidates": [cluster.pool_entry(s) for s in pool], + } + + def assign_tier(story, now=None): """Assign a priority tier to a story based on its status and lastActivityBy.""" if now is None: @@ -663,6 +699,12 @@ def _select_locked(args, prd_path, run_state_path, bot_dir, in_progress=None): "requeuedFrom": requeued, "candidateCount": len(candidates), "slot": args.slot, + # Absent unless the profile clusters and the lead is fresh pending work. + # A story named outright clusters too: naming one issue does not say the + # related ones should wait for a separate iteration. + "clustering": clustering_offer( + bot_dir, selected, candidates, blocked_issues, now + ), "storyDetails": selected, "research": research_steps(bot_dir), } diff --git a/scripts/update-prd-status.py b/scripts/update-prd-status.py index dd4e6fc..ab41650 100755 --- a/scripts/update-prd-status.py +++ b/scripts/update-prd-status.py @@ -77,6 +77,12 @@ def atomic_write_json(path, data): raise +# Subcommands whose result a clustered iteration's companions share: they ride +# the lead's branch and PR, so each moves to the same status with the same +# branch and PR number. Everything else (activity, pings) is the lead's alone. +CLUSTER_FOLLOWS = {"committed", "pushed"} + + def find_story(prd, story_id): """Find and return story dict by ID, or None.""" for story in prd.get("stories", []): @@ -259,6 +265,34 @@ def handle_fix_status(story, args): } +def apply_to_companions(prd, lead, subcommand, args): + """Move the lead's cluster companions along with it. + + Returns {companion id: changes}. A companion that is not in the status the + transition needs is left alone and not an error: the lead's own update has + already been validated, and a companion someone moved by hand is theirs. + + When the lead is skipped or invalid its companions are handed back to the + queue: they never had work done for them, so they are pending again. + """ + moved = {} + for sid in lead.get("clusterCompanions") or []: + companion = find_story(prd, sid) + if companion is None: + continue + if subcommand in CLUSTER_FOLLOWS: + if validate_transition(subcommand, companion): + continue + moved[sid] = HANDLER_MAP[subcommand](companion, args) + elif subcommand in ("skipped", "invalid"): + if companion.get("status", "pending") == "pending": + companion.pop("clusterLead", None) + moved[sid] = {"clusterLead": None} + if subcommand in ("skipped", "invalid") and moved: + lead.pop("clusterCompanions", None) + return moved + + def main(): # A comparison run repeats a story's work only so it can be compared with # the base run's. Moving the story on would make the base run's own status @@ -370,6 +404,7 @@ def main(): # Apply changes handler = HANDLER_MAP[args.subcommand] changes = handler(story, args) + companion_changes = apply_to_companions(prd, story, args.subcommand, args) # Write prd.json atomically atomic_write_json(prd_path, prd) @@ -397,6 +432,10 @@ def main(): print(f"Updated {args.story_id}:") for key, value in changes.items(): print(f" {key}: {json.dumps(value)}") + for sid, moved in companion_changes.items(): + print(f"Updated {sid} (clustered with {args.story_id}):") + for key, value in moved.items(): + print(f" {key}: {json.dumps(value)}") return 0 diff --git a/tests/test_cluster.py b/tests/test_cluster.py new file mode 100644 index 0000000..8062d58 --- /dev/null +++ b/tests/test_cluster.py @@ -0,0 +1,334 @@ +"""Clustering related issues into one iteration (lib/cluster.py and its callers).""" + +import argparse +import importlib.util +import json +import os +import sys + +import pytest + +SCRIPT_DIR = os.path.join(os.path.dirname(__file__), os.pardir, "scripts") +sys.path.insert(0, os.path.abspath(SCRIPT_DIR)) + +from lib import cluster # noqa: E402 + +BRAVEBOT_PROFILE = os.path.join( + os.path.dirname(__file__), os.pardir, "projects", "bravebot", "profile.json" +) + + +def story(sid, size=1, status="pending", issue=None, **extra): + """A story as add-backlog-to-prd builds it: the issue lives in the description.""" + s = {"id": sid, "title": f"Story {sid}", "status": status} + if issue is not False: + number = issue if issue is not None else int(sid.split("-")[1]) + s["description"] = f"Resolve issue #{number}: story {sid}" + if size is not None: + s["triage"] = {"size": size} + s.update(extra) + return s + + +def profile(limits=None): + return {"clustering": {"sizeLimits": limits}} if limits else {} + + +TABLE = {"1": 10, "2": 6, "3": 4, "4": 1, "5": 1} + + +class TestCeiling: + @pytest.mark.parametrize("size,expected", [(1, 10), (2, 6), (3, 4), (4, 1), (5, 1)]) + def test_the_requested_table(self, size, expected): + assert cluster.ceiling(profile(TABLE), story("US-001", size)) == expected + + def test_the_bravebot_profile_carries_that_table(self): + with open(BRAVEBOT_PROFILE) as f: + real = json.load(f) + for size, expected in ((1, 10), (2, 6), (3, 4), (4, 1), (5, 1)): + assert cluster.ceiling(real, story("US-001", size)) == expected + + def test_a_profile_without_the_block_never_clusters(self): + assert cluster.ceiling({}, story("US-001", 1)) == 1 + + def test_unlabelled_lead_is_worked_alone(self): + """Unlike ordering, an unknown size is not the middle one: guessing + wrong here puts ten issues in a pull request.""" + assert cluster.ceiling(profile(TABLE), story("US-001", None)) == 1 + + def test_malformed_entries_are_dropped_not_raised(self): + p = profile({"1": "many", "x": 3, "2": 0, "3": 4, "9": 5}) + assert cluster.size_limits(p) == {3: 4} + + +class TestPool: + def test_only_pending_stories_with_an_issue(self): + lead = story("US-001", 2) + ordered = [ + story("US-002", 1), + story("US-003", 1, status="pushed"), + story("US-004", 1, issue=False), + ] + pool = cluster.companion_pool(lead, ordered, 6) + assert [s["id"] for s in pool] == ["US-002"] + + def test_never_larger_than_the_lead(self): + lead = story("US-001", 2) + ordered = [story("US-002", 3), story("US-003", 2), story("US-004", 1)] + pool = cluster.companion_pool(lead, ordered, 6) + assert [s["id"] for s in pool] == ["US-003", "US-004"] + + def test_unlabelled_candidates_are_not_offered(self): + lead = story("US-001", 1) + assert cluster.companion_pool(lead, [story("US-002", None)], 10) == [] + + def test_lead_is_not_its_own_companion(self): + lead = story("US-001", 1) + assert cluster.companion_pool(lead, [lead], 10) == [] + + def test_issues_in_progress_elsewhere_are_not_offered(self): + lead = story("US-001", 1) + pool = cluster.companion_pool( + lead, [story("US-002", 1)], 10, blocked_issues={2} + ) + assert pool == [] + + def test_ceiling_of_one_offers_nothing(self): + lead = story("US-001", 4) + assert cluster.companion_pool(lead, [story("US-002", 1)], 1) == [] + + def test_pool_is_capped(self): + lead = story("US-001", 1) + ordered = [story(f"US-{n:03d}", 1) for n in range(2, 200)] + assert len(cluster.companion_pool(lead, ordered, 10)) == cluster.POOL_CAP + + def test_order_is_kept(self): + lead = story("US-001", 1) + ordered = [story("US-009", 1), story("US-003", 1)] + pool = cluster.companion_pool(lead, ordered, 10) + assert [s["id"] for s in pool] == ["US-009", "US-003"] + + +def load_script(name): + spec = importlib.util.spec_from_file_location( + name.replace("-", "_"), os.path.join(SCRIPT_DIR, name + ".py") + ) + mod = importlib.util.module_from_spec(spec) + spec.loader.exec_module(mod) + return mod + + +class TestSelectOffer: + @pytest.fixture + def select_task(self, monkeypatch): + mod = load_script("select-task") + monkeypatch.setattr(mod, "load_config", lambda *a, **k: {}) + monkeypatch.setattr(mod, "load_profile", lambda *a, **k: profile(TABLE)) + return mod + + def test_offers_a_pool_sized_by_the_lead(self, select_task): + lead = story("US-001", 2) + others = [story("US-002", 1), story("US-003", 2), story("US-004", 5)] + offer = select_task.clustering_offer("/bot", lead, [lead] + others, set(), None) + assert offer["limit"] == 6 + assert offer["leadSize"] == 2 + assert sorted(c["storyId"] for c in offer["candidates"]) == ["US-002", "US-003"] + + def test_nothing_for_a_large_lead(self, select_task): + lead = story("US-001", 4) + assert ( + select_task.clustering_offer( + "/bot", lead, [lead, story("US-002", 1)], set(), None + ) + is None + ) + + def test_nothing_for_a_story_that_already_has_a_pr(self, select_task): + lead = story("US-001", 1, status="pushed", prNumber=5) + assert ( + select_task.clustering_offer( + "/bot", lead, [lead, story("US-002", 1)], set(), None + ) + is None + ) + + def test_nothing_when_no_candidate_qualifies(self, select_task): + lead = story("US-001", 1) + assert select_task.clustering_offer("/bot", lead, [lead], set(), None) is None + + +class TestStatusFollowsLead: + @pytest.fixture + def ups(self, monkeypatch): + mod = load_script("update-prd-status") + monkeypatch.setattr( + mod, "load_config", lambda *a, **k: {"project": {"prRepository": "o/r"}} + ) + monkeypatch.setattr(mod, "require_config", lambda c, k: "o/r") + return mod + + def prd(self): + lead = story("US-001", 2, clusterCompanions=["US-002", "US-003"]) + return { + "stories": [ + lead, + story("US-002", 1, clusterLead="US-001"), + story("US-003", 1, clusterLead="US-001"), + story("US-004", 1), + ] + } + + def test_committed_and_pushed_move_the_companions(self, ups): + prd = self.prd() + lead = prd["stories"][0] + args = argparse.Namespace(branch="fix-a", pr_number=77) + ups.handle_committed(lead, args) + moved = ups.apply_to_companions(prd, lead, "committed", args) + assert sorted(moved) == ["US-002", "US-003"] + ups.handle_pushed(lead, args) + ups.apply_to_companions(prd, lead, "pushed", args) + for sid in ("US-002", "US-003"): + s = prd["stories"][int(sid[-1]) - 1] + assert (s["status"], s["branchName"], s["prNumber"]) == ( + "pushed", + "fix-a", + 77, + ) + assert prd["stories"][3]["status"] == "pending" + + def test_a_companion_in_another_status_is_left_alone(self, ups): + prd = self.prd() + prd["stories"][1]["status"] = "skipped" + lead = prd["stories"][0] + args = argparse.Namespace(branch="fix-a", pr_number=77) + ups.handle_committed(lead, args) + moved = ups.apply_to_companions(prd, lead, "committed", args) + assert list(moved) == ["US-003"] + assert prd["stories"][1]["status"] == "skipped" + + def test_skipping_the_lead_returns_companions_to_the_queue(self, ups): + prd = self.prd() + lead = prd["stories"][0] + args = argparse.Namespace(reason="blocked") + ups.handle_skipped(lead, args) + ups.apply_to_companions(prd, lead, "skipped", args) + assert "clusterLead" not in prd["stories"][1] + assert prd["stories"][1]["status"] == "pending" + assert "clusterCompanions" not in lead + + def test_a_story_without_companions_is_unaffected(self, ups): + prd = {"stories": [story("US-001", 1)]} + args = argparse.Namespace(branch="b") + assert ups.apply_to_companions(prd, prd["stories"][0], "committed", args) == {} + + +class TestClaimCompanions: + @pytest.fixture + def cc(self, monkeypatch, tmp_path): + mod = load_script("claim-companions") + self.claimed = {} + held = {"US-001": {"slot": 1, "pid": 99, "runId": "r"}} + monkeypatch.setattr(mod.claims_lib, "active", lambda *a, **k: held) + + def claim(bot_dir, sid, slot, pid, run_id=None, locked=False): + held[sid] = {"slot": slot, "pid": pid, "runId": run_id} + self.claimed[sid] = slot + return True + + monkeypatch.setattr(mod.claims_lib, "claim", claim) + monkeypatch.setattr(mod.claims_lib, "release", lambda *a, **k: []) + monkeypatch.setattr(mod.issue_lock, "acquire", lambda *a, **k: True) + self.held = held + self.tmp = tmp_path + return mod + + def write_prd(self, stories): + path = self.tmp / "prd.json" + path.write_text(json.dumps({"stories": stories})) + return str(path) + + def run(self, cc, stories, wanted, lead_size=2, **kw): + stories = [story("US-001", lead_size)] + stories + path = self.write_prd(stories) + result, code = cc.claim_companions( + str(self.tmp), + path, + "US-001", + wanted, + 1, + profile(TABLE), + kw.get("blocked", set()), + ) + return result, code, json.loads(open(path).read()) + + def test_claims_up_to_the_ceiling_and_refuses_the_rest(self, cc): + others = [story(f"US-{n:03d}", 1) for n in range(2, 10)] + wanted = [s["id"] for s in others] + result, code, prd = self.run(cc, others, wanted, lead_size=3) # ceiling 4 + assert code == 0 + assert result["claimed"] == ["US-002", "US-003", "US-004"] + assert result["taken"] == 4 + assert set(result["refused"]) == { + "US-005", + "US-006", + "US-007", + "US-008", + "US-009", + } + assert prd["stories"][0]["clusterCompanions"] == result["claimed"] + assert prd["stories"][1]["clusterLead"] == "US-001" + + def test_a_second_call_counts_against_the_same_ceiling(self, cc): + others = [story(f"US-{n:03d}", 1) for n in range(2, 8)] + first, _, _ = self.run(cc, others, ["US-002", "US-003", "US-004"], lead_size=3) + assert len(first["claimed"]) == 3 + # Lead now carries three; a fourth would be the fifth story. + path = str(self.tmp / "prd.json") + second, _ = cc.claim_companions( + str(self.tmp), path, "US-001", ["US-005"], 1, profile(TABLE), set() + ) + assert second["claimed"] == [] + assert "ceiling" in second["refused"]["US-005"] + + def test_a_large_lead_cannot_take_any(self, cc): + result, code, _ = self.run(cc, [story("US-002", 1)], ["US-002"], lead_size=4) + assert code == 1 + assert "does not cluster" in result["error"] + + def test_a_lead_this_slot_does_not_hold_fails(self, cc): + self.held.clear() + result, code, _ = self.run(cc, [story("US-002", 1)], ["US-002"]) + assert code == 1 + + def test_larger_candidate_is_refused(self, cc): + result, code, _ = self.run(cc, [story("US-002", 3)], ["US-002"], lead_size=2) + assert code == 0 + assert result["claimed"] == [] + assert "US-002" in result["refused"] + + def test_unknown_and_duplicate_ids_are_refused(self, cc): + result, _, _ = self.run( + cc, [story("US-002", 1)], ["US-002", "US-002", "US-404", "US-001"] + ) + assert result["claimed"] == ["US-002"] + assert set(result["refused"]) == {"US-404", "US-001"} | set() + + def test_a_story_another_run_holds_is_refused(self, cc): + self.held["US-002"] = {"slot": 2, "pid": 5, "runId": "x"} + result, _, _ = self.run(cc, [story("US-002", 1)], ["US-002"]) + assert result["claimed"] == [] + assert "another run" in result["refused"]["US-002"] + + def test_issue_in_progress_elsewhere_is_refused(self, cc): + result, _, _ = self.run(cc, [story("US-002", 1)], ["US-002"], blocked={2}) + assert result["claimed"] == [] + + def test_label_failure_hands_the_claim_back(self, cc, monkeypatch): + released = [] + monkeypatch.setattr(cc.issue_lock, "acquire", lambda *a, **k: False) + monkeypatch.setattr( + cc.claims_lib, "release", lambda *a, **k: released.append(k.get("story_id")) + ) + result, _, _ = self.run(cc, [story("US-002", 1)], ["US-002"]) + assert result["claimed"] == [] + assert released == ["US-002"]