28 lines (19 loc) · 592 Bytes

Usage

List scenarios:

cli-evals list --suite ./cli-agent-evals.config.mjs

Run canaries:

cli-evals run --suite ./cli-agent-evals.config.mjs --canary --agent claude --model sonnet

Run all scenarios:

cli-evals run --suite ./cli-agent-evals.config.mjs --all --agent claude --model sonnet --repeats 3

Run one scenario:

cli-evals run --suite ./cli-agent-evals.config.mjs --filter EV-013 --agent claude --model sonnet --keep

Live runs are opt-in. Use --canary before --all because real agents consume tokens, auth, and wall-clock time.