Background
We have unit tests for ConflictClassifier, but no benchmark to track classifier latency or accuracy
on a fixed corpus.
What needs to be done
Add ai-sync/benchmarks/classifier_benchmark.py that loads tests/fixtures/conflicts.jsonl, runs
classification, and prints accuracy + p50/p95 latency.
Definition of done
Background
We have unit tests for
ConflictClassifier, but no benchmark to track classifier latency or accuracyon a fixed corpus.
What needs to be done
Add
ai-sync/benchmarks/classifier_benchmark.pythat loadstests/fixtures/conflicts.jsonl, runsclassification, and prints accuracy + p50/p95 latency.
Definition of done
python -m ai_sync.benchmarks.classifier_benchmarkdocs/metrics-and-kpis.mdEstimated effort: 3–4 hours · Skills needed: Python, basic ML evaluation