Skip to content

Commit 5331116

Browse files
committed
docs(architecture): explain cursor workflow
1 parent 8dab4ef commit 5331116

5 files changed

Lines changed: 564 additions & 73 deletions

File tree

‎README.md‎

Lines changed: 73 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
# αgεηt SWE
44

5-
**Synthetic software engineering benchmark generator for Platform agents**
5+
**Real-code software engineering benchmarks for Platform agents**
66

77
[![License](https://img.shields.io/github/license/PlatformNetwork/Agent-SWE)](https://github.com/PlatformNetwork/Agent-SWE/blob/main/LICENSE)
88
[![Platform SDK](https://img.shields.io/badge/Platform-SDK-black)](https://github.com/PlatformNetwork/platform)
@@ -13,64 +13,74 @@
1313

1414
</div>
1515

16-
Agent SWE is a Platform benchmark toolkit for building, exporting, and evaluating software engineering agent tasks. It extends SWE-Forge with a Cursor-style synthetic task pipeline: start from a real repository, delete a testable feature, then ask an agent to restore the behavior using tests as the reward signal.
16+
Agent-SWE turns real repositories into benchmark tasks for autonomous software engineering agents. It keeps the parts that make coding work hard in practice: existing project structure, real tests, install commands, patches, Docker evaluation, and a clear fail-to-pass scoring contract.
1717

18-
The project can mine real GitHub pull requests, generate synthetic feature-deletion tasks, export benchmark workspaces, and run Docker-based evaluation against gold or model-generated patches.
18+
The synthetic task pipeline is inspired by Cursor's public writing on Composer, Composer 2, and Composer 2.5. Cursor described training coding agents on tasks grounded in real codebases, including a feature-deletion style setup: remove a testable behavior, ask the agent to restore it, and use tests as the reward signal. Agent-SWE adapts that idea into an open benchmark-generation workflow for Platform agents.
1919

20-
## What Agent SWE Does
20+
This project is not affiliated with Cursor. It is an implementation inspired by the public methodology described in their posts and reports.
2121

22-
Agent SWE creates reproducible benchmark tasks for autonomous coding agents:
22+
## Why Agent-SWE Exists
2323

24-
1. Select a real repository or pull request.
25-
2. Discover install and test commands.
26-
3. Generate or select fail-to-pass and pass-to-pass tests.
27-
4. Export a task workspace with hidden solution artifacts.
28-
5. Evaluate a gold patch or model patch in an isolated environment.
29-
6. Produce task scores that can be consumed by Platform challenge validators.
24+
Most coding benchmarks are either real but scarce, or synthetic but too detached from real development. Agent-SWE aims for the middle ground: tasks are synthetic enough to scale, but grounded enough that agents still need to inspect a real repository, understand context, edit code, and run tests.
3025

31-
## Key Features
26+
A good Agent-SWE task should answer three questions:
3227

33-
- Real PR mining from GH Archive and GitHub metadata.
34-
- Synthetic Cursor-style feature deletion tasks.
35-
- Docker verification for fail-to-pass and pass-to-pass tests.
36-
- Workspace export with `workspace.yaml`, `patch.diff`, tests, and optional `deletion_patch.diff`.
37-
- JSONL and Parquet exports for benchmark datasets.
38-
- Simple CLI commands for mining, synthetic generation, and evaluation.
28+
1. Can the agent understand the existing codebase?
29+
2. Can it restore the intended behavior without seeing the oracle patch?
30+
3. Can the result pass both targeted reward tests and regression tests?
3931

40-
## Evaluation Flow
32+
## Inspired by Cursor Composer
33+
34+
Cursor's Composer work is the main public inspiration for the synthetic path in Agent-SWE:
35+
36+
- [Composer: Building a fast frontier model with RL](https://cursor.com/blog/composer)
37+
- [Introducing Composer 2](https://cursor.com/blog/composer-2)
38+
- [A technical report on Composer 2](https://cursor.com/blog/composer-2-technical-report)
39+
- [Composer 2 Technical Report PDF](https://cursor.com/resources/Composer2.pdf)
40+
- [Introducing Composer 2.5](https://cursor.com/blog/composer-2-5)
41+
42+
The important idea is simple: instead of only collecting issues and pull requests, generate new tasks from real repositories. In the feature-deletion variant, a known behavior is removed from the codebase, the inverse patch becomes the oracle solution, and tests define whether the agent recovered the behavior.
43+
44+
Agent-SWE currently implements this idea for Python functions and methods. It keeps the public signature, replaces the body with a synthetic failure, writes that mutation to `deletion_patch.diff`, and stores the inverse repair as `patch.diff`.
45+
46+
## What Agent-SWE Does
47+
48+
Agent-SWE supports two sources of benchmark tasks:
49+
50+
1. **Real pull requests** mined from GitHub and converted into SWE-style workspaces.
51+
2. **Synthetic feature-deletion tasks** generated from real repositories, inspired by the public Composer 2.5 training method.
52+
53+
Both flows export a workspace that can be evaluated in Docker. The agent being tested should never see the oracle patch or hidden benchmark files.
4154

4255
```mermaid
4356
flowchart LR
44-
Repo[Real repo or PR] --> Task[Task generation]
45-
Task --> Tests[Test discovery]
46-
Tests --> Export[Workspace export]
47-
Export --> Eval[Docker evaluation]
48-
Eval --> Score[Task score]
49-
Score --> Platform[Platform weights]
57+
Repo[Real repo] --> Build[Build task]
58+
Build --> Export[Export workspace]
59+
Export --> Run[Docker eval]
60+
Run --> Score[Task score]
61+
Score --> Plat[Platform]
5062
```
5163

52-
For synthetic tasks, Agent SWE applies `deletion_patch.diff` first, verifies that reward tests fail, applies the candidate or oracle patch, and then verifies that reward and regression tests pass.
53-
54-
## Installation
64+
## Install
5565

5666
```bash
5767
git clone https://github.com/PlatformNetwork/Agent-SWE.git
5868
cd Agent-SWE
5969
pip install -e ".[dev]"
6070
```
6171

62-
## Environment
72+
Set the tokens used by the mining and LLM-assisted parts of the pipeline:
6373

6474
```bash
6575
export GITHUB_TOKEN="ghp_..."
6676
export OPENROUTER_API_KEY="************"
6777
```
6878

69-
`GITHUB_TOKEN` is used for GitHub enrichment. `OPENROUTER_API_KEY` is used for LLM-backed classification and test generation.
70-
7179
## Commands
7280

73-
### Mine Real PR Tasks
81+
### Mine real PR tasks
82+
83+
Use this when you want SWE-bench style tasks from GitHub pull requests.
7484

7585
```bash
7686
swe-forge mine mine \
@@ -80,7 +90,9 @@ swe-forge mine mine \
8090
--parallel 8
8191
```
8292

83-
### Verify One Pull Request
93+
### Verify one pull request end-to-end
94+
95+
Use this for a known repository and PR number.
8496

8597
```bash
8698
swe-forge mine complete \
@@ -90,7 +102,9 @@ swe-forge mine complete \
90102
--model openai/gpt-5.4
91103
```
92104

93-
### Generate a Synthetic Feature-Deletion Task
105+
### Generate a synthetic feature-deletion task
106+
107+
Use this when you already have a local checkout and know which Python function or method should be removed.
94108

95109
```bash
96110
git clone https://github.com/owner/repo.git ./target-repo
@@ -108,7 +122,9 @@ swe-forge synthetic generate \
108122
--overwrite
109123
```
110124

111-
### Evaluate Gold Patches
125+
### Evaluate the oracle patch
126+
127+
Use this to confirm that a generated task is valid with its gold solution.
112128

113129
```bash
114130
python3 scripts/run_evaluation.py \
@@ -117,23 +133,25 @@ python3 scripts/run_evaluation.py \
117133
--max_workers 4
118134
```
119135

120-
### Evaluate Model Predictions
136+
### Evaluate model predictions
137+
138+
Use this after an agent has produced patches.
121139

122140
```bash
123141
python3 scripts/run_evaluation.py \
124142
--predictions_path predictions.jsonl \
125143
--max_workers 4
126144
```
127145

128-
`predictions.jsonl` must contain one prediction per line:
146+
`predictions.jsonl` contains one prediction per line:
129147

130148
```json
131149
{"instance_id": "owner-repo-1234", "model_patch": "diff --git a/..."}
132150
```
133151

134152
## Workspace Format
135153

136-
When `--output-folder` is used, tasks are exported as directories:
154+
A task workspace is the portable benchmark unit:
137155

138156
```text
139157
tasks/
@@ -147,44 +165,24 @@ tasks/
147165
└── evaluate.sh
148166
```
149167

150-
`patch.diff` is the oracle solution and must be hidden from agents. `deletion_patch.diff` exists only for synthetic feature-deletion tasks and is applied before evaluation.
151-
152-
Example `workspace.yaml`:
153-
154-
```yaml
155-
task_id: owner-repo-1234
156-
repo:
157-
url: https://github.com/owner/repo.git
158-
base_commit: abc123def456
159-
merge_commit: abc123def456
160-
language: python
161-
prompt: Restore the deleted behavior for `target_function`.
162-
install:
163-
commands:
164-
- pip install -e .
165-
tests:
166-
fail_to_pass:
167-
- pytest tests/test_target.py -v
168-
pass_to_pass:
169-
- pytest tests/ -v
170-
synthetic:
171-
source_type: synthetic_feature_deletion
172-
deletion_patch_file: deletion_patch.diff
173-
strategy: feature_deletion
174-
```
168+
The files have different audiences:
169+
170+
- `workspace.yaml` describes the task, repo, install commands, tests, and synthetic metadata.
171+
- `patch.diff` is the oracle solution and must be hidden from the evaluated agent.
172+
- `deletion_patch.diff` is the synthetic mutation applied before evaluation.
173+
- `tests/` contains generated or extracted benchmark tests.
174+
- `evaluate.sh` is a simple local scoring script.
175175

176-
## Synthetic Task Method
176+
For details, read [docs/architecture/workspace-format.md](docs/architecture/workspace-format.md).
177177

178-
The synthetic pipeline follows the public Cursor Composer 2.5-style feature-deletion method:
178+
## Documentation
179179

180-
1. Start from a real repository with tests.
181-
2. Replace a target Python function or method body with a synthetic failure.
182-
3. Store the mutation as `deletion_patch.diff`.
183-
4. Store the inverse patch as `patch.diff`.
184-
5. Use supplied tests as the reward contract.
185-
6. Export the task for isolated evaluation.
180+
The architecture docs explain how the pieces fit together:
186181

187-
This makes the benchmark grounded in real code while preserving an objective fail-to-pass signal.
182+
- [Architecture overview](docs/architecture/README.md)
183+
- [Synthetic feature deletion](docs/architecture/synthetic-feature-deletion.md)
184+
- [Workspace format](docs/architecture/workspace-format.md)
185+
- [Evaluation flow](docs/architecture/evaluation.md)
188186

189187
## Development
190188

@@ -201,6 +199,8 @@ pytest tests/ -v
201199
Agent-SWE/
202200
├── assets/
203201
├── datasets/
202+
├── docs/
203+
│ └── architecture/
204204
├── scripts/
205205
├── src/swe_forge/
206206
│ ├── cli/
@@ -213,7 +213,7 @@ Agent-SWE/
213213

214214
## Platform Integration
215215

216-
Agent SWE is designed to feed Platform challenge validators with deterministic repository-repair tasks. Validators can use exported workspaces or JSONL predictions to run isolated evaluations and convert task completion rates into raw challenge scores.
216+
Agent-SWE is designed to feed Platform challenge validators with deterministic repository-repair tasks. Validators can sample tasks, run agent patches in isolated workspaces, and turn task completion rates into raw challenge scores for Platform.
217217

218218
## License
219219

‎docs/architecture/README.md‎

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Architecture Overview
2+
3+
Agent-SWE is a benchmark-generation and evaluation toolkit for software engineering agents. It builds task workspaces from real repositories, then evaluates whether an agent can produce a patch that makes the right tests pass.
4+
5+
The system has two task sources:
6+
7+
1. **Real PR tasks**: mined from GitHub pull requests and exported in a SWE-style format.
8+
2. **Synthetic feature-deletion tasks**: generated from real repositories by deleting a known behavior and asking the agent to restore it.
9+
10+
The synthetic path is inspired by Cursor's public descriptions of how Composer 2 and Composer 2.5 use synthetic, codebase-grounded tasks for agent training.
11+
12+
## High-Level Flow
13+
14+
```mermaid
15+
flowchart LR
16+
Repo[Real repo] --> Gen[Task generation]
17+
Gen --> Del[Deletion patch]
18+
Del --> Export[Workspace]
19+
Export --> Eval[Evaluation]
20+
Eval --> Score[Score]
21+
Score --> Plat[Platform]
22+
```
23+
24+
Legend:
25+
26+
- **Real repo**: a real GitHub repository or local checkout.
27+
- **Task generation**: mining, command discovery, or synthetic task creation.
28+
- **Deletion patch**: only used for synthetic feature-deletion tasks.
29+
- **Workspace**: portable benchmark directory.
30+
- **Evaluation**: Docker-based fail-to-pass scoring.
31+
- **Platform**: downstream challenge infrastructure.
32+
33+
## Main Modules
34+
35+
| Path | Role |
36+
|---|---|
37+
| `src/swe_forge/cli/` | CLI entry points such as `mine`, `synthetic`, `validate`, and `export`. |
38+
| `src/swe_forge/swe/` | Core task model, GitHub mining, enrichment, scoring, and test generation. |
39+
| `src/swe_forge/synthetic/` | Synthetic task generation, feature deletion, leak auditing, and scoring helpers. |
40+
| `src/swe_forge/export/` | Workspace, JSONL, Parquet, and evaluation script export. |
41+
| `src/swe_forge/docker_test/` | Docker-based before/after verification. |
42+
| `scripts/` | Evaluation and revalidation utilities. |
43+
44+
## Synthetic Task Flow
45+
46+
The synthetic flow starts from a real checkout and a target Python symbol:
47+
48+
1. `swe-forge synthetic generate` receives the repo path, repo name, source file, symbol, tests, and output paths.
49+
2. `synthetic.pipeline.create_feature_deletion_task` coordinates task creation.
50+
3. `synthetic.feature_deletion.build_python_function_deletion` parses the source file and replaces the target body with a synthetic failure.
51+
4. The mutation is saved as `deletion_patch.diff`.
52+
5. The inverse mutation is saved as `patch.diff`, the oracle solution.
53+
6. `export.workspace.export_task_to_workspace` writes the workspace directory.
54+
7. The evaluation scripts apply the deletion patch before testing model patches.
55+
56+
## Evaluation Flow
57+
58+
For a synthetic task, the evaluator checks three things:
59+
60+
1. After `deletion_patch.diff`, `fail_to_pass` tests must fail.
61+
2. After applying the candidate patch, `fail_to_pass` tests must pass.
62+
3. `pass_to_pass` tests must also pass, so the candidate did not break unrelated behavior.
63+
64+
For a mined PR task, the same fail-to-pass idea applies, but there is no synthetic deletion patch. The repository starts at the base commit and the candidate patch is expected to repair the original issue.
65+
66+
## Why This Shape Works
67+
68+
This architecture gives Agent-SWE a practical balance:
69+
70+
- real codebases keep tasks realistic;
71+
- synthetic deletion makes task generation scalable;
72+
- Docker keeps evaluation isolated;
73+
- fail-to-pass tests create a clear scoring contract;
74+
- workspace exports make tasks portable for validators and offline experiments.
75+
76+
## Cursor References
77+
78+
The synthetic approach is inspired by public Cursor material:
79+
80+
- [Composer: Building a fast frontier model with RL](https://cursor.com/blog/composer)
81+
- [Introducing Composer 2](https://cursor.com/blog/composer-2)
82+
- [A technical report on Composer 2](https://cursor.com/blog/composer-2-technical-report)
83+
- [Composer 2 Technical Report PDF](https://cursor.com/resources/Composer2.pdf)
84+
- [Introducing Composer 2.5](https://cursor.com/blog/composer-2-5)
85+
86+
## Read Next
87+
88+
- [Synthetic feature deletion](synthetic-feature-deletion.md)
89+
- [Workspace format](workspace-format.md)
90+
- [Evaluation flow](evaluation.md)

0 commit comments

Comments
 (0)