|
| 1 | +# Contributing to ctxgraph |
| 2 | + |
| 3 | +ctxgraph welcomes contributions. The highest-impact way to help is submitting new benchmark episodes -- but code contributions, bug reports, and documentation improvements are all appreciated. |
| 4 | + |
| 5 | +## Benchmark Episode Submissions |
| 6 | + |
| 7 | +The extraction benchmark is the primary quality gate for ctxgraph. More episodes mean better coverage of real-world text patterns and harder edge cases. |
| 8 | + |
| 9 | +### JSON Schema |
| 10 | + |
| 11 | +Each episode in `crates/ctxgraph-extract/tests/fixtures/benchmark_episodes.json` follows this structure: |
| 12 | + |
| 13 | +```json |
| 14 | +{ |
| 15 | + "text": "2-6 sentences describing an architectural decision, incident, or migration.", |
| 16 | + "expected_entities": [ |
| 17 | + { "name": "Postgres", "entity_type": "Database", "span_start": 27, "span_end": 35 } |
| 18 | + ], |
| 19 | + "expected_relations": [ |
| 20 | + { "head": "AuthService", "relation": "depends_on", "tail": "Postgres" } |
| 21 | + ] |
| 22 | +} |
| 23 | +``` |
| 24 | + |
| 25 | +- `span_start` / `span_end` are character offsets into `text` (0-indexed, exclusive end). |
| 26 | +- `name` must match the exact substring at those offsets. |
| 27 | + |
| 28 | +### Valid Entity Types (10) |
| 29 | + |
| 30 | +| Type | Description | |
| 31 | +|------|-------------| |
| 32 | +| Person | Person or engineer | |
| 33 | +| Component | Software library or framework | |
| 34 | +| Service | Cloud service or API | |
| 35 | +| Language | Programming language | |
| 36 | +| Database | Database or data store | |
| 37 | +| Infrastructure | Server or cloud platform | |
| 38 | +| Decision | Architectural decision | |
| 39 | +| Constraint | Technical constraint | |
| 40 | +| Metric | Performance metric | |
| 41 | +| Pattern | Design pattern | |
| 42 | + |
| 43 | +### Valid Relation Types (9) |
| 44 | + |
| 45 | +| Relation | Meaning | |
| 46 | +|----------|---------| |
| 47 | +| chose | Chose or adopted a technology | |
| 48 | +| rejected | Rejected an alternative | |
| 49 | +| replaced | One thing replaced another | |
| 50 | +| depends_on | Dependency relationship | |
| 51 | +| fixed | Something fixed an issue | |
| 52 | +| introduced | Introduced or added a component | |
| 53 | +| deprecated | Deprecation action | |
| 54 | +| caused | Causal relationship | |
| 55 | +| constrained_by | Decision constrained by something | |
| 56 | + |
| 57 | +### Guidelines |
| 58 | + |
| 59 | +- **Text**: 2-6 sentences about real architectural decisions, incidents, migrations, or ADRs. |
| 60 | +- **Entities**: 2-6 per episode. Use canonical names (`"Postgres"`, not `"the primary Postgres cluster"`). Names must be exact substrings of the text. |
| 61 | +- **Relations**: 1-4 per episode. Head and tail must reference entity names defined in `expected_entities`. |
| 62 | +- **Adversarial episodes are welcome**: edge cases, ambiguous phrasing, unusual domain terminology, overlapping entity spans, entities that look like relations, etc. |
| 63 | + |
| 64 | +### How to Submit |
| 65 | + |
| 66 | +Open a PR that appends your episodes to `crates/ctxgraph-extract/tests/fixtures/benchmark_episodes.json`. Run the benchmark to see how the pipeline handles them: |
| 67 | + |
| 68 | +```bash |
| 69 | +CTXGRAPH_MODELS_DIR=~/.cache/ctxgraph/models \ |
| 70 | + cargo test --package ctxgraph-extract --test benchmark_test -- --ignored |
| 71 | +``` |
| 72 | + |
| 73 | +## Code Contributions |
| 74 | + |
| 75 | +- Rust edition 2024 |
| 76 | +- Run `cargo fmt` and `cargo clippy` before submitting a PR |
| 77 | +- Run tests: `cargo test --workspace` |
| 78 | +- Run the extraction benchmark (see above) if your changes touch the extraction pipeline |
| 79 | + |
| 80 | +## Bug Reports |
| 81 | + |
| 82 | +Use [GitHub Issues](../../issues). Include: |
| 83 | + |
| 84 | +- What you expected vs. what happened |
| 85 | +- Steps to reproduce |
| 86 | +- ctxgraph version (`cargo metadata --format-version 1 | jq '.packages[] | select(.name == "ctxgraph") | .version'` or check `Cargo.toml`) |
| 87 | +- OS and Rust version |
| 88 | + |
| 89 | +## License |
| 90 | + |
| 91 | +Contributions are licensed under MIT, matching the project license. |
0 commit comments