Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,7 @@ Available skills:
- `/crucible-tools:image-cleanup` — clean up local podman images (engine images, dangling images, local builds)
- `/crucible-tools:new-repo` — create a new repository in the GitHub organization with standard config
- `/crucible-tools:open-prs` — show all open PRs in the org (optionally filter by author)
- `/crucible-tools:pr-review` — structured multi-dimension code review for a PR
- `/crucible-tools:repo-status` — git status across all crucible repos
- `/crucible-tools:workflow-status` — show active CI workflow runs across crucible repos
- `ci-analyzer` agent — analyze GitHub Actions CI workflow runs to diagnose failures
Expand Down
10 changes: 7 additions & 3 deletions bin/_crucible_completions
Original file line number Diff line number Diff line change
Expand Up @@ -204,13 +204,17 @@ _complete_get_metric() {
COMPREPLY=($(compgen -W "yes no" -- "${cur}"))
return
;;
--aggregation)
COMPREPLY=($(compgen -W "sum avg max min" -- "${cur}"))
return
;;
--period|--source|--type|--begin|--end|--resolution|--breakout|--filter|--decimal-places|--timestamp-rows)
return
;;
esac

local remaining
remaining=$(_crucible_filter_used_flags 3 --run --period --source --type --begin --end --resolution --breakout --filter --output-format --date-format --decimal-places --output-content --horizontal-break --timestamp-rows)
remaining=$(_crucible_filter_used_flags 3 --run --period --source --type --begin --end --resolution --breakout --filter --output-format --date-format --decimal-places --output-content --horizontal-break --timestamp-rows --aggregation)
COMPREPLY=($(compgen -W "${remaining}" -- "${cur}"))
}

Expand Down Expand Up @@ -427,7 +431,7 @@ _complete_opensearch() {
add)
case "${prev}" in
--cdmver)
COMPREPLY=($(compgen -W "v7dev v8dev v9dev" -- "${cur}"))
COMPREPLY=($(compgen -W "v7dev v8dev v9dev v10dev" -- "${cur}"))
return
;;
--name|--host|--userpass)
Expand All @@ -452,7 +456,7 @@ _complete_opensearch() {
update)
case "${prev}" in
--cdmver)
COMPREPLY=($(compgen -W "v7dev v8dev v9dev" -- "${cur}"))
COMPREPLY=($(compgen -W "v7dev v8dev v9dev v10dev" -- "${cur}"))
return
;;
--name)
Expand Down
2 changes: 2 additions & 0 deletions bin/_help
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ function help() {
echo " | - Other arguments are optional:"
echo " | --breakout <list of one or more breakouts>"
echo " | (found from initial metric query, like cstype or id)"
echo " | --aggregation <sum|avg|max|min>"
echo " | (override the metric's default-aggregation for this query)"
echo " | --filter <gt|lt:number>"
echo " | (gt = only show metrics greater-then value, lt = onlt show metrics less than value"
echo "opensearch | Manage OpenSearch instance configuration (add, remove, update, info, query-opt, index-instance, etc.)"
Expand Down
8 changes: 7 additions & 1 deletion config/services.json
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,16 @@
"name": "local-v9",
"host": "localhost:9200",
"cdmver": "v9dev"
},
{
"name": "local-v10",
"host": "localhost:9200",
"cdmver": "v10dev"
}
],
"index-to": "local-v9",
"index-to": "local-v10",
"query-from": [
"local-v10",
"local-v9",
"local-v8"
]
Expand Down
33 changes: 27 additions & 6 deletions docs/how-cdm-works.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ run
| **period** | A time window within a sample | period UUID, name, begin/end timestamps |
| **param** | A benchmark parameter | arg name, value, role (client/server) |
| **tag** | Run metadata | name, value (e.g., kernel version, test purpose) |
| **metric_desc** | What a metric IS | source, type, class, breakout dimensions |
| **metric_desc** | What a metric IS | source, type, class, default-aggregation, breakout dimensions |
| **metric_data** | Actual values | begin, end, value, duration |

Each level adds context. A metric_data document inherits its
Expand All @@ -67,8 +67,23 @@ Defines what a metric measures:
`uperf`, `mpstat`, `procstat`)
- **type**: The specific metric name (e.g., `Gbps`,
`Busy-CPU`, `interrupts-sec`)
- **class**: The metric category — `throughput` (rate-based)
or `count` (accumulative)
- **class**: The metric category — `throughput` (rate-based),
`count` (accumulative), or `latency` (delay measurement)
- **default-aggregation**: How this metric should be
aggregated across breakout dimensions when producing a
single result value. One of:

| Type | Behavior | Use case |
|------|----------|----------|
| `sum` | Duration-weighted sum (default) | Throughput, IOPS, CPU-units-busy |
| `avg` | Duration-weighted average ÷ metric count | Pre-aggregated latency (TRex avg, iostat avg-service-time) |
| `max` | Maximum value | RT worst-case latency (cyclictest, oslat, ptp-latency) |
| `min` | Minimum value | Idle floor measurements |

Post-processors set this in the metric descriptor dict.
If absent, CDM falls back to `sum` (preserving current
behavior for existing metrics). Users can override the
aggregation at query time with `--aggregation`.
- **names**: Breakout dimensions that identify specific
instances of this metric (e.g., `hostname=worker-01`,
`cpu=3`, `device=sda`, `direction=rx`)
Expand Down Expand Up @@ -361,11 +376,15 @@ can be run manually to re-index or index imported results.
```bash
crucible get result --run <run-id>
crucible get metric --run <run-id> --source mpstat --type Busy-CPU
crucible get metric --run <run-id> --source cyclictest --type wakeup-latency-usec --aggregation avg
```

Query results from the command line. `get result` shows
the primary metric summary; `get metric` retrieves specific
metric data.
metric data. The `--aggregation` flag overrides the
metric's `default-aggregation` for that query — useful for
viewing the same data with different aggregation semantics
(e.g., `max` vs `avg` across engines).

### Managing results

Expand Down Expand Up @@ -394,7 +413,8 @@ CDM has evolved through several versions:
|---------|------------|
| v7dev | Original version, generic `id` fields |
| v8dev | Document-specific UUID fields (`iteration-uuid`, `period-uuid`, etc.) |
| v9dev | Per-month index naming (`cdm-v9dev-metric_data@2026.06`), additional document types |
| v9dev | Per-month index naming (`cdm-v9dev-metric_data@2026.06`), `metric_def` document type |
| v10dev | `default-aggregation` field on metric_desc, per-metric aggregation control (sum/avg/max/min) |

Multiple CDM versions can coexist in the same OpenSearch
instance. The `services.json` OpenSearch configuration
Expand All @@ -403,4 +423,5 @@ specifies which CDM version each instance uses, and the
go and which versions are searchable.

This enables gradual migration — new results can be indexed
in v9dev while older v8dev results remain queryable.
in v10dev while older v9dev and v8dev results remain
queryable.
8 changes: 4 additions & 4 deletions docs/how-services-work.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,13 +223,13 @@ confirm validity.
"opensearch": {
"instances": [
{
"name": "local-v9",
"name": "local-v10",
"host": "localhost:9200",
"cdmver": "v9dev"
"cdmver": "v10dev"
}
],
"index-to": "local-v9",
"query-from": ["local-v9"]
"index-to": "local-v10",
"query-from": ["local-v10"]
},
"image-sourcing": {
"use": true,
Expand Down
2 changes: 1 addition & 1 deletion schema/services.json
Original file line number Diff line number Diff line change
Expand Up @@ -210,7 +210,7 @@
"minLength": 1
},
"cdmver": {
"description": "The Common Data Model version used by this instance (e.g., 'v8dev', 'v9dev')",
"description": "The Common Data Model version used by this instance (e.g., 'v8dev', 'v9dev', 'v10dev')",
"type": "string",
"minLength": 1
},
Expand Down
Loading