Skip to content

trigger route/compare from a skill list + cost-ranked suggestions (CLI + viewer) #95

Description

@aktasbatuhan

Summary

The routing engine (route / compare, PR #85) is merged but manual and CLI-onlydaemon.py never invokes it, and there's no easy way to see which skills are worth the (token-expensive) compare. This issue closes that gap with two complementary pieces:

  1. List-driven triggering — let users pick skills from a discovered list and start route/compare, from both the CLI and the web viewer.
  2. Cost-ranked suggestions — watchmen ranks skills by frequency × cost and surfaces "these are worth routing," so the user knows where to spend a compare run.

Design principle: keep the human in the loop for the token spend and the repo edits. The daemon only suggests (cheap, pure SQL); it never auto-runs compare or auto-writes artifacts.

Part A — Trigger route/compare from a list

CLI

  • Add a watchmen skills <project> command to list buckets (none exists today; buckets are discoverable only as an error hint on a bad --bucket). Use available_skills() (util.py:264).
  • Add an interactive multi-select picker when route/compare is run without --bucket on a TTY — mirror the questionary pattern in commands/settings_menu.py (with its TTY-guard + non-interactive fallback). Run the picked buckets sequentially.

Web viewer

The viewer already has the long-running-job infra we need — POST /actions/runactions.start_run spawns a detached watchmen <action> <project> subprocess, logs to ~/.watchmen/web-runs/<id>.log + a sidecar, status polled via a 2s meta-refresh. It's whitelisted to {analyze, curate, learn} (viewer/actions.py:48).

  • Extend RUNNABLE_ACTIONS to include route / compare.
  • Generalize start_run to accept a regex-validated --bucket (route/compare need more argv than the current [action, project]); extend the /actions/run handler (viewer/server.py:627) to read a bucket form field. Reuse all existing run-tracking/log-tailing unchanged.
  • Add Route / Compare buttons on the skill-detail page (templates/skill.html) as POST forms, and a multi-select "pick skills" list (batch endpoint or client-side loop).

Part B — Suggest skills to route (frequency × cost)

  • New ranking helper computing per-bucket frequency (exact: SELECT skill_name, COUNT(*) FROM tool_calls ... GROUP BY skill_name, pattern in prune.py:_usage_for_bundle) × cost (accurate once the cost-tracking issue lands; Claude-Code-only). Flag buckets that are used a lot AND cost disproportionately more.
  • Daemon hook in cycle_once (after _run_analyst, ~daemon.py:248): compute the ranking per project and persist bundles/<project>/_route_suggestions.jsonmirror the prune-queue pattern (prune.py writes _prune_queue.json, viewer reads it). Pure SQL, zero token spend.
  • Surface through existing channels (minimal new plumbing):
    • Add a suggested_routes array to the curator state payload (curate.py ~:964) so the statusline 💡 and /watchmen:brief pick it up for free.
    • Add a kind:"route" branch in next_best_actions (viewer/actions.py) reading _route_suggestions.json → viewer "Next moves" banner + a dashboard "Suggested to route" card.
  • Toggle + dismissal: auto_suggest_routes per-project toggle (column via state.py:_migrate_project_columns, the PR feat(install): toggle auto_install from the settings TUI and viewer #90 pattern; + settings menu/viewer); dismissal persisted by mirroring _prune_dismissed.json.

Dependencies & sequencing

  • The cost dimension of the suggestion depends on accurate per-skill cost (see the cost-tracking issue). We can ship frequency-only suggestions first, then layer cost in once turns lands.
  • Part A (triggering) has no dependency and can ship independently.

Acceptance

  • watchmen skills <project> lists buckets; route/compare without --bucket shows a picker on a TTY (graceful fallback otherwise).
  • Viewer can start a route/compare run for a chosen bucket and track it like analyze/curate.
  • Daemon writes _route_suggestions.json; suggestions appear in statusline/brief + viewer without the user going looking.
  • Per-project auto_suggest_routes toggle + dismissal work.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions