-
Notifications
You must be signed in to change notification settings - Fork 4
feat: add generic refine agent #86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from 1 commit
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| # Refine Agent | ||
|
|
||
| <img src="icons/refine.png" alt="Refine agent icon" width="80"> | ||
|
|
||
| Decomposes a work item — feature, epic, story, or issue — into implementable child work items with testable acceptance criteria, dependency analysis, and confidence scoring. Always produces a plan, even when information is incomplete. | ||
|
|
||
| ## How the agent works | ||
|
|
||
| The refine agent runs after the explore agent has gathered technical context. It reads the issue description, exploration context (codebase analysis, related work, competitive landscape), and any prior critique feedback. It then decomposes the work item into a hierarchy of child issues sized for engineering teams and sprints. | ||
|
|
||
| The agent runs in a read-only sandbox. It cannot modify issues, push code, or create child issues. Its only output is a structured JSON refinement plan consumed by the post-script, which posts a summary comment, attaches the plan to the issue, and adds a `ready-to-critique` label to signal the [critique agent](critique.md). | ||
|
|
||
| ## How it helps | ||
|
|
||
| - Features get decomposed into actionable work items within minutes instead of waiting for a refinement meeting. | ||
| - Vague requirements are flagged as open questions with explicit assumptions, preventing silent scope gaps. | ||
| - Cross-cutting dependencies are identified early, before teams start implementation in silos. | ||
| - Each child issue includes testable acceptance criteria, making "done" unambiguous. | ||
|
|
||
| ## Commands | ||
|
|
||
| | Command | Where | Effect | | ||
| |---------|-------|--------| | ||
| | `/fs-refine` | Issue comment | Runs refinement on the issue | | ||
|
|
||
| The `/fs-refine` command accepts an optional human directive after the command (e.g., `/fs-refine focus on the API layer first`) that guides the agent's decomposition priorities. | ||
|
|
||
| ## Pipeline integration | ||
|
|
||
| The refine agent is part of a three-stage refinement pipeline: | ||
|
|
||
| 1. **Explore** — gathers technical context from the codebase, GitHub, Jira, and web | ||
| 2. **Refine** — decomposes the work item into implementable children (this agent) | ||
| 3. **Critique** — reviews the decomposition and approves, requests revisions, or escalates | ||
|
|
||
| > **Note**: The explore and critique agents are added in separate PRs. Cross-references to their docs pages will resolve once all three PRs merge. | ||
|
|
||
| Agents communicate through issue labels and attachments. No direct workflow chaining. | ||
|
|
||
| ### Revision rounds | ||
|
|
||
| When the critique agent requests revisions, the refine agent re-runs with the critique feedback. Each round addresses specific revision requests (remove, merge, split, revise, add). The pipeline iterates up to `MAX_REVIEW_ROUNDS` (default: 3) before escalating to a human. | ||
|
|
||
| ## Control labels | ||
|
|
||
| These labels are managed by the refinement pipeline: | ||
|
|
||
| | Label | Meaning | | ||
| |-------|---------| | ||
| | `ready-to-critique` | Refine posted a plan; critique agent should review it | | ||
| | `ready-to-refine` | Critique requested revisions; refine should re-run | | ||
| | `refine-revision-round-N` | Tracks which revision round the pipeline is on | | ||
| | `refine-approved` | Critique approved the plan; ready for human review or auto-creation | | ||
| | `refine-needs-input` | Critique determined a human must answer a question before proceeding | | ||
| | `refine-needs-human` | Max review rounds reached; human decision needed | | ||
|
|
||
| ## Platform support | ||
|
|
||
| The refine agent supports work items from multiple platforms: | ||
|
|
||
| - **GitHub Issues** — uses labels and sub-issues for hierarchy | ||
| - **Jira** — uses typed issue hierarchy (Feature → Epic → Story → Task) | ||
| - **GitLab** — uses epics, issues, and labels (planned) | ||
|
|
||
| Platform-specific hierarchy rules and description formats are injected via `PLATFORM_CONTEXT`. | ||
|
|
||
| ## Configuration and extension | ||
|
|
||
| ### Routing skill | ||
|
|
||
| To route child issues to different Jira projects based on team ownership, provide a routing skill at `.fullsend/skills/jira-routing/SKILL.md` or `.agents/skills/project-routing/SKILL.md`. The routing skill maps team ownership domains to projects so child issues are created in the right project. Without a routing skill, all children are created in the parent issue's project. | ||
|
|
||
| ### Platform context | ||
|
|
||
| Platform context files (`platform-jira.md`, `platform-github.md`, `platform-gitlab.md`) define the hierarchy model, description format, and parent-child constraints for each platform. The pre-script selects the appropriate one based on `ISSUE_SOURCE`. | ||
|
|
||
| ## Source | ||
|
|
||
| [`harness/refine.yaml`](../harness/refine.yaml) | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same deal as the other PR, move this to the harness |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| export ISSUE_CONTEXT=/sandbox/workspace/issue-context.json | ||
| export EXPLORE_CONTEXT=/sandbox/workspace/exploration_context.json | ||
| export CRITIQUE_FEEDBACK=/sandbox/workspace/critique-feedback.json | ||
| export REVIEW_ROUND=${REVIEW_ROUND:-1} | ||
| export PROJECT_ROUTING=/sandbox/workspace/routing-skill.md | ||
| export PLATFORM_CONTEXT=/sandbox/workspace/platform-context.md |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| --- | ||
| agent: agents/refine.md | ||
| doc: docs/refine.md | ||
| model: opus | ||
| image: ghcr.io/fullsend-ai/fullsend-sandbox:latest | ||
| policy: policies/refine.yaml | ||
|
|
||
|
qodo-code-review[bot] marked this conversation as resolved.
|
||
| role: refine | ||
| slug: fullsend-ai-refine | ||
|
|
||
| host_files: | ||
| - src: common/env/gcp-vertex.env | ||
| dest: /sandbox/workspace/.env.d/gcp-vertex.env | ||
| expand: true | ||
| - src: env/refine.env | ||
| dest: /sandbox/workspace/.env.d/refine.env | ||
| expand: true | ||
| - src: ${GOOGLE_APPLICATION_CREDENTIALS} | ||
| dest: /tmp/.gcp-credentials.json | ||
| - src: ${GCP_OIDC_TOKEN_FILE} | ||
| dest: /sandbox/workspace/.gcp-oidc-token | ||
| optional: true | ||
| - src: /tmp/workspace/issue-context.json | ||
| dest: /sandbox/workspace/issue-context.json | ||
| optional: true | ||
| - src: /tmp/workspace/exploration_context.json | ||
| dest: /sandbox/workspace/exploration_context.json | ||
| optional: true | ||
| - src: /tmp/workspace/critique-feedback.json | ||
| dest: /sandbox/workspace/critique-feedback.json | ||
| optional: true | ||
| - src: /tmp/workspace/routing-skill.md | ||
| dest: /sandbox/workspace/routing-skill.md | ||
| optional: true | ||
| - src: /tmp/workspace/platform-context.md | ||
| dest: /sandbox/workspace/platform-context.md | ||
| optional: true | ||
| - src: /tmp/workspace/human-directive.txt | ||
| dest: /sandbox/workspace/human-directive.txt | ||
| optional: true | ||
|
|
||
| pre_script: scripts/pre-refine.sh | ||
| post_script: scripts/post-refine.sh | ||
|
|
||
| validation_loop: | ||
| script: scripts/validate-output-schema.sh | ||
| schema: schemas/refine-result.schema.json | ||
| max_iterations: 2 | ||
|
|
||
| timeout_minutes: 25 | ||
|
|
||
| env: | ||
| runner: | ||
| ISSUE_KEY: "${ISSUE_KEY}" | ||
| ISSUE_SOURCE: "${ISSUE_SOURCE}" | ||
| REPO_FULL_NAME: "${REPO_FULL_NAME}" | ||
| EXPLORE_RUN_ID: "${EXPLORE_RUN_ID}" | ||
| EXPLORE_CONTEXT_REF: "${EXPLORE_CONTEXT_REF}" | ||
| CRITIQUE_RUN_ID: "${CRITIQUE_RUN_ID}" | ||
| REVIEW_ROUND: "${REVIEW_ROUND}" | ||
| MAX_REVIEW_ROUNDS: "${MAX_REVIEW_ROUNDS}" | ||
| AUTO_CREATE: "${AUTO_CREATE}" | ||
| GITHUB_ISSUE_NUMBER: "${GITHUB_ISSUE_NUMBER}" | ||
| HUMAN_DIRECTIVE: "${HUMAN_DIRECTIVE}" | ||
| FULLSEND_OUTPUT_FILE: refine-result.json | ||
| sandbox: | ||
|
qodo-code-review[bot] marked this conversation as resolved.
|
||
| GH_TOKEN: "${GH_TOKEN}" | ||
| REVIEW_ROUND: "${REVIEW_ROUND}" | ||
| HUMAN_DIRECTIVE_FILE: "/sandbox/workspace/human-directive.txt" | ||
|
|
||
| forge: | ||
| github: | ||
| pre_script: scripts/pre-refine.sh | ||
| post_script: scripts/post-refine.sh | ||
| env: | ||
| runner: | ||
| GH_TOKEN: "${GH_TOKEN}" | ||
| PUSH_TOKEN: "${PUSH_TOKEN}" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,71 @@ | ||
| --- | ||
| version: 1 | ||
|
|
||
| # Sandbox policy for the refine agent. | ||
| # | ||
| # Read-only agent: needs GitHub API for issue context and Jira for | ||
| # attachment retrieval. No write access to either — the post-script | ||
| # handles mutations on the runner. | ||
|
|
||
| filesystem_policy: | ||
| include_workdir: true | ||
| read_only: [/usr, /lib, /proc, /dev/urandom, /app, /etc, /var/log] | ||
| read_write: [/sandbox, /tmp, /dev/null] | ||
| landlock: | ||
| compatibility: best_effort | ||
| process: | ||
| run_as_user: sandbox | ||
| run_as_group: sandbox | ||
|
|
||
| network_policies: | ||
| vertex_ai: | ||
| name: vertex-ai | ||
| endpoints: | ||
| - host: "api.anthropic.com" | ||
| port: 443 | ||
| protocol: rest | ||
| enforcement: enforce | ||
| access: read-write | ||
| - host: "*.googleapis.com" | ||
| port: 443 | ||
| protocol: rest | ||
| enforcement: enforce | ||
| access: read-write | ||
| binaries: | ||
| - path: "**/claude" | ||
| - path: "**/node" | ||
|
|
||
| github_api: | ||
| name: github-api | ||
| endpoints: | ||
| - host: "api.github.com" | ||
| port: 443 | ||
| protocol: rest | ||
| enforcement: enforce | ||
| access: read-only | ||
| - host: "github.com" | ||
| port: 443 | ||
| protocol: rest | ||
| enforcement: enforce | ||
| access: read-only | ||
| - host: "raw.githubusercontent.com" | ||
| port: 443 | ||
| protocol: rest | ||
| enforcement: enforce | ||
| access: read-only | ||
| binaries: | ||
| - path: "**/gh" | ||
| - path: "**/git" | ||
| - path: "**/node" | ||
|
|
||
| jira_api: | ||
| name: jira-api | ||
| endpoints: | ||
| - host: "*.atlassian.net" | ||
| port: 443 | ||
| protocol: rest | ||
| enforcement: enforce | ||
| access: read-only | ||
| binaries: | ||
| - path: "**/curl" | ||
| - path: "**/node" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [high] schema-correctness Missing additionalProperties: false at every object level (root, input, confidence, children items, dependencies items, open_questions items) — all 7 other schemas in the repo include it. Also uses draft-07 instead of draft/2020-12 and lacks $id field, inconsistent with all other schemas. Suggested fix: Add additionalProperties: false to every object definition. Change $schema to https://json-schema.org/draft/2020-12/schema. Add $id: refine-result.schema.json.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in a47c056: schema upgraded to draft/2020-12 with |
||
| "$schema": "http://json-schema.org/draft-07/schema#", | ||
| "title": "Refinement Result", | ||
| "description": "Output schema for the refine agent — always a complete decomposition plan.", | ||
| "type": "object", | ||
| "required": ["input", "status", "confidence", "target_level", "children", "comment", "summary"], | ||
| "properties": { | ||
| "input": { | ||
| "type": "object", | ||
| "required": ["source", "key", "level", "summary"], | ||
| "properties": { | ||
| "source": { "type": "string", "enum": ["jira", "github", "gitlab", "text"] }, | ||
| "key": { "type": "string" }, | ||
| "level": { "type": "string", "enum": ["outcome", "feature", "epic", "story", "task", "issue"] }, | ||
| "summary": { "type": "string" } | ||
| } | ||
| }, | ||
| "status": { | ||
| "type": "string", | ||
| "enum": ["complete"] | ||
| }, | ||
| "target_level": { | ||
| "type": "string" | ||
| }, | ||
| "confidence": { | ||
| "type": "object", | ||
| "required": ["overall"], | ||
| "properties": { | ||
| "scope_clarity": { "type": "number", "minimum": 0, "maximum": 100 }, | ||
| "technical_grounding": { "type": "number", "minimum": 0, "maximum": 100 }, | ||
| "acceptance_criteria": { "type": "number", "minimum": 0, "maximum": 100 }, | ||
| "dependencies": { "type": "number", "minimum": 0, "maximum": 100 }, | ||
| "sizing": { "type": "number", "minimum": 0, "maximum": 100 }, | ||
| "overall": { "type": "number", "minimum": 0, "maximum": 100 } | ||
| } | ||
| }, | ||
| "children": { | ||
| "type": "array", | ||
| "minItems": 1, | ||
| "items": { | ||
| "type": "object", | ||
| "required": ["title", "type", "description", "acceptance_criteria"], | ||
| "properties": { | ||
| "title": { "type": "string" }, | ||
| "type": { "type": "string", "description": "Issue type matching the project's available types (lowercase). Common: feature, epic, story, task, sub-issue, bug, spike." }, | ||
| "parent_title": { "type": ["string", "null"] }, | ||
| "description": { "type": "string" }, | ||
| "acceptance_criteria": { | ||
| "type": "array", | ||
| "items": { "type": "string" }, | ||
| "minItems": 1 | ||
| }, | ||
| "dependencies": { | ||
| "type": "array", | ||
| "items": { | ||
| "type": "object", | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. [low] schema-correctness children[].dependencies[] items schema defines properties (type, target, description) but has no required field. open_questions items correctly specify required fields, suggesting this was an oversight. Suggested fix: Add required: [type, target] to dependencies items schema.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in a47c056: dependencies items schema now has |
||
| "properties": { | ||
| "type": { "type": "string" }, | ||
| "target": { "type": "string" }, | ||
| "description": { "type": "string" } | ||
| } | ||
| } | ||
| }, | ||
| "labels": { "type": "array", "items": { "type": "string" } }, | ||
| "priority": { "type": "string" }, | ||
| "estimated_scope": { "type": "string" }, | ||
| "confidence": { "type": "number", "minimum": 0, "maximum": 100 }, | ||
| "deployment_target": { "type": "string" }, | ||
| "target_platform": { "type": "string", "enum": ["jira", "github", "gitlab"], "description": "Platform to create this child on. Omit to inherit from parent." }, | ||
| "target_project": { "type": ["string", "null"], "description": "Project/repo to create this child in. Only set when routing context is available." } | ||
| } | ||
| } | ||
| }, | ||
| "open_questions": { | ||
| "type": "array", | ||
| "items": { | ||
| "type": "object", | ||
| "required": ["dimension", "question", "impact"], | ||
| "properties": { | ||
| "dimension": { "type": "string" }, | ||
| "question": { "type": "string" }, | ||
| "impact": { "type": "string" }, | ||
| "assumption_used": { "type": "string" } | ||
| } | ||
| } | ||
| }, | ||
| "proposed_description": { "type": "string", "maxLength": 20000 }, | ||
| "dimensions_covered": { "type": "array", "items": { "type": "string" } }, | ||
| "dimensions_missing": { "type": "array", "items": { "type": "string" } }, | ||
| "uncited_assumptions": { "type": "array", "items": { "type": "string" } }, | ||
| "deployment_targets": { "type": "array", "items": { "type": "string" } }, | ||
| "comment": { "type": "string", "maxLength": 4000 }, | ||
| "summary": { "type": "string", "maxLength": 2000 } | ||
| } | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| # Platform Context: GitHub Issues | ||
|
|
||
| ## Work Item Hierarchy | ||
|
|
||
| GitHub uses a flat issue model with labels and sub-issues for hierarchy: | ||
| - **Issues** — the base unit (no built-in types beyond issue/PR) | ||
| - **Labels** — used to indicate level/type (feature, epic, story, task, bug, spike) | ||
| - **Sub-issues** — GitHub's native parent-child relationship | ||
| - **Milestones** — optional grouping mechanism | ||
|
|
||
| **Hierarchy rules:** | ||
| - A feature-labeled issue produces: child issues labeled "epic" (parent_title=null) → child issues labeled "story" (parent_title=epic title) → child issues labeled "task" (parent_title=story title) | ||
| - An epic-labeled issue produces: child issues labeled "story" (parent_title=null) → child issues labeled "task" (parent_title=story title) | ||
| - A story-labeled issue produces: child issues labeled "task" (parent_title=null) | ||
| - All hierarchy is expressed via GitHub's sub-issue feature | ||
| - Labels differentiate the logical type of each issue | ||
|
|
||
| **Important**: GitHub has no type restrictions — any issue can be a child of any other issue. Use labels consistently to communicate intent. | ||
|
|
||
| ## Decomposition Output | ||
|
|
||
| Each child must include: | ||
| - `type`: the logical type (used as a label). Common: epic, story, task, spike, bug | ||
| - `labels`: additional labels beyond the type label | ||
| - `target_platform`: "github" (or omit to inherit parent's platform) | ||
| - `target_project` is NOT used for GitHub — all children are created in the same repository as the parent | ||
|
|
||
| **Repository targeting**: Children are always created in the same repo as the parent issue. Cross-repo work should be noted in dependencies but not created automatically. | ||
|
|
||
| ## Description Format | ||
|
|
||
| GitHub uses markdown natively. No conversion needed. | ||
|
|
||
| **Feature-level descriptions** use a two-tier structure: | ||
| - Visible top section (always shown) | ||
| - Content after `---` delimiter is rendered as-is (GitHub doesn't have collapsible sections in issue descriptions, but the `---` creates a visual separator). If you want collapsible content, use `<details><summary>Detailed Specification</summary>` HTML. | ||
|
|
||
| **Child-level descriptions** can be more concise since GitHub issues are inherently lighter weight. | ||
|
|
||
| ## Parent-Child Constraints | ||
|
|
||
| - Any issue can be a sub-issue of any other issue (no type restrictions) | ||
| - Sub-issues are created via GitHub's sub-issues API | ||
| - If sub-issue creation fails, the child is still created as a standalone issue in the same repo | ||
| - Labels are automatically created if they don't exist | ||
| - Always include spikes (label "spike") for high-uncertainty areas | ||
| - Always include documentation tasks (label "documentation") for user-facing changes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[medium] docs-staleness
References
icons/refine.pngwhich does not exist in the repository. All other agent docs reference icons that are present.Suggested fix: Add
docs/icons/refine.pngto this PR or remove the icon reference.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 99cb712: removed
icons/refine.pngreference fromdocs/refine.md.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed earlier (99cb712): removed
icons/refine.pngreference fromdocs/refine.md.