Background
When querying mpstat data via the CDM API, the Linux CPU number is exposed as the breakout dimension num. This works, but creates two usability gaps:
-
No node breakout — There is no way to directly query per-NUMA-node CPU utilization. Analysts must manually cross-reference num values against the host's CPU topology to determine which NUMA node a busy CPU belongs to. On large systems (e.g. 768-CPU AMD EPYC with 2 NUMA nodes), this is cumbersome.
-
Non-obvious dimension name — The dimension is called num rather than cpu. Users and agents familiar with the concept of "CPU number" naturally try cpu as a breakout name and get no results, then incorrectly conclude that per-CPU data is not available.
Requested Changes
1. Add cpu as an alias for num
Allow --breakout cpu to work identically to --breakout num — same data, just a more intuitive name. Keeping num for backwards compatibility.
2. Add node as a derived breakout
Derive the NUMA node for each CPU using sysfs topology (/sys/devices/system/node/nodeN/cpulist or /sys/devices/system/cpu/cpuN/node*) at collection time and store it as a breakout dimension. This allows queries like:
crucible get metric --period <ID> --source mpstat --type Busy-CPU --breakout hostname,node
to immediately show per-NUMA-node aggregate CPU utilization without manual CPU-to-node mapping.
Other potentially useful topology breakouts to consider
While addressing the above, the following sysfs-derivable dimensions would also be valuable for network performance analysis:
shared-l3-domain — grouping CPUs by last-level cache domain (useful for identifying cache-sharing patterns between application and interrupt-handling threads)
die — already available as a CDM breakout but worth confirming correct derivation on multi-die platforms
Context
This came up during a 400G TCP stream benchmark investigation where:
- The NIC's NUMA node was node 1 (CPUs 192-383, 576-767)
- IRQs were correctly pinned to CPUs 192-223
- We needed to verify NUMA locality of application threads vs. IRQ-handling CPUs
- Querying by
num and then manually checking which CPUs are in node 1 vs node 0 added friction
A node breakout would allow a one-line query to confirm NUMA locality rather than a multi-step manual cross-reference.
AI-assisted-by: Claude Sonnet 4.6
Background
When querying mpstat data via the CDM API, the Linux CPU number is exposed as the breakout dimension
num. This works, but creates two usability gaps:No
nodebreakout — There is no way to directly query per-NUMA-node CPU utilization. Analysts must manually cross-referencenumvalues against the host's CPU topology to determine which NUMA node a busy CPU belongs to. On large systems (e.g. 768-CPU AMD EPYC with 2 NUMA nodes), this is cumbersome.Non-obvious dimension name — The dimension is called
numrather thancpu. Users and agents familiar with the concept of "CPU number" naturally trycpuas a breakout name and get no results, then incorrectly conclude that per-CPU data is not available.Requested Changes
1. Add
cpuas an alias fornumAllow
--breakout cputo work identically to--breakout num— same data, just a more intuitive name. Keepingnumfor backwards compatibility.2. Add
nodeas a derived breakoutDerive the NUMA node for each CPU using sysfs topology (
/sys/devices/system/node/nodeN/cpulistor/sys/devices/system/cpu/cpuN/node*) at collection time and store it as a breakout dimension. This allows queries like:to immediately show per-NUMA-node aggregate CPU utilization without manual CPU-to-node mapping.
Other potentially useful topology breakouts to consider
While addressing the above, the following sysfs-derivable dimensions would also be valuable for network performance analysis:
shared-l3-domain— grouping CPUs by last-level cache domain (useful for identifying cache-sharing patterns between application and interrupt-handling threads)die— already available as a CDM breakout but worth confirming correct derivation on multi-die platformsContext
This came up during a 400G TCP stream benchmark investigation where:
numand then manually checking which CPUs are in node 1 vs node 0 added frictionA
nodebreakout would allow a one-line query to confirm NUMA locality rather than a multi-step manual cross-reference.AI-assisted-by: Claude Sonnet 4.6