You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
fix(perf)!: unify local and release performance workflows
- Retain local Markdown, CSV, and provenance artifacts while allowing same-version comparisons of tracked changes.
- Exclude untracked files explicitly and isolate the narrowed non-exact comparison bundle.
- Promote distinct-release artifacts through performance-doc and keep performance-release atomic and rollback-capable.
- Render one consolidated table per benchmark suite and reject invalid release publication states.
BREAKING CHANGE: rename performance-rerender to performance-doc and performance-local-vs-linalg to performance-local-non-exact, replace --rerender with --promote-artifacts, and remove performance-archive-published.
|`performance-doc`| No | Consumes retained inputs | Yes |
53
+
|`performance-release`| Yes | Yes | Yes |
54
+
55
+
For a distinct release pair with no intervening source or configuration changes,
56
+
running `performance-local` followed by `performance-doc` produces the same
57
+
report and committed documentation as `performance-release`.
58
+
`performance-release` exists as the safer one-step release operation: it keeps
59
+
fresh measurement, validated artifact publication, and rollback-capable document
60
+
promotion in one command.
45
61
46
62
## Benchmark Suites
47
63
48
64
`la-stack` has two Criterion benchmark suites.
49
65
66
+
Newly rendered reports use one table per selected suite. Dimension and
67
+
adversarial-input group appear in a `Case` column instead of creating a separate
68
+
table for every group. The `vs_linalg` table is the one wider variant because it
69
+
adds nalgebra and faer context columns where matching peer measurements exist.
70
+
50
71
**`vs_linalg`** (`benches/vs_linalg.rs`) compares `la-stack` against
51
72
`nalgebra` and `faer` across D=2-64 for LU, solve, determinant, dot, norm, and
52
73
SPD factorization operations. Use this suite to answer "why choose la-stack over
@@ -76,16 +97,20 @@ just performance-local
76
97
77
98
This creates isolated temporary worktrees and runs both library revisions on the
78
99
same machine with the current checkout's benchmark sources, manifests, lockfile,
79
-
benchmark-input tests, recipes, and Rust toolchain. Only the baseline library
80
-
implementation comes from the release tag. Before either timing run, the command
81
-
runs `just test-bench-inputs` against that revision under the shared current
82
-
fixture harness. This is a prerequisite correctness gate over the deterministic
83
-
fixtures and operations, not validation of each timed Criterion sample. It
84
-
writes `target/bench-reports/performance.md` and records both
85
-
commits, CPU, operating system, Rust toolchain, lockfile and harness digests,
86
-
Criterion selection/commands, and both correctness-gate results. The report
87
-
reader rejects malformed or mismatched provenance and incomplete selected-suite
88
-
coverage.
100
+
benchmark-input tests, recipes, and Rust toolchain. Staged and unstaged changes
101
+
to tracked files are applied to the current worktree. Untracked files are
102
+
excluded; stage a new file before running the command if it must participate in
103
+
the comparison. Only the baseline library implementation comes from the release
104
+
tag. Before either timing run, the command runs `just test-bench-inputs` against
105
+
that revision under the shared current fixture harness. This is a prerequisite
106
+
correctness gate over the deterministic fixtures and operations, not validation
107
+
of each timed Criterion sample. It writes
108
+
`target/bench-reports/performance.md` plus retained `performance.csv` and
109
+
`performance.provenance.json` comparison inputs. The report and sidecar embed
110
+
both commits, CPU, operating system, Rust toolchain, lockfile and harness
111
+
digests, Criterion selection/commands, and both correctness-gate results. The
112
+
report reader rejects malformed or mismatched provenance and incomplete
113
+
selected-suite coverage.
89
114
90
115
The shared harness carries an explicit v0.4.3-only API adapter for renamed or
91
116
ownership-adjusted calls (`det_sign_exact`, `Tolerance`, and vector dot
@@ -116,24 +141,33 @@ timing, current validation, and current timing, so a long comparison exposes
116
141
completed samples and its active phase instead of remaining silent until the
117
142
final report is rendered.
118
143
119
-
If the checkout's package version is identical to the latest published release,
120
-
the command now stops before creating worktrees or running benchmarks because a
121
-
release report requires two distinct identifiers. For repeated optimization
122
-
within one package version, use the named-baseline loop below instead.
144
+
The local report may compare a checkout whose package version is identical to
145
+
the latest published release. Commit/ref and source-state provenance distinguish
146
+
the modified checkout from the tagged baseline even though both display the same
147
+
package version. Release artifact publication remains stricter and requires two
148
+
distinct release identifiers.
123
149
124
150
### Compare Current Code With A Specific Release
125
151
126
152
For a narrower non-exact check against a known release pair, run:
127
153
128
154
```bash
129
-
just performance-local-vs-linalg v0.4.4 v0.4.3
155
+
just performance-local-non-exact v0.4.4 v0.4.3
130
156
```
131
157
132
158
This generates a local `v0.4.3``vs_linalg` baseline, measures the current
133
159
la-stack `vs_linalg` rows, and renders a `vs_linalg` report. The report includes
134
160
saved baseline nalgebra/faer timings as context where matching peer rows exist,
135
161
without rerunning current peer crates.
136
162
163
+
This narrowed peer-context view uses the same metrics and renderer but writes a
164
+
separate `performance-non-exact.*` scratch bundle so it cannot replace the
165
+
canonical full comparison inputs accidentally.
166
+
167
+
When tags are provided explicitly, the current tag must match the package
168
+
version in the `HEAD` checkout. A mismatch is rejected before tags are fetched,
169
+
worktrees are created, or benchmarks run.
170
+
137
171
### Iterate Against A Local Saved Baseline
138
172
139
173
Use local saved baselines when tuning one kernel and comparing several edits
@@ -240,13 +274,18 @@ To reproduce and promote the report without running Cargo or creating Git
240
274
worktrees, use:
241
275
242
276
```bash
243
-
just performance-rerender
277
+
just performance-doc
244
278
```
245
279
246
280
This command fails closed on a missing, partial, malformed, mismatched, or
247
-
unsupported artifact pair. Use it for presentation-only report corrections;
248
-
changes to benchmark inputs, code, toolchains, or measurement configuration
249
-
require a fresh `performance-release` run.
281
+
unsupported artifact pair. It consumes the default CSV/JSON pair retained by a
282
+
successful `performance-local` or `performance-release` run, rewrites the
283
+
scratch Markdown, promotes it to `docs/PERFORMANCE.md`, and archives the previous
284
+
committed report when the release pair changes. Promotion requires distinct
285
+
current and baseline package versions, so a same-version local comparison is
286
+
retained and reproducible but cannot become release documentation. Use promotion
287
+
for presentation-only report corrections; changes to benchmark inputs, code,
288
+
toolchains, or measurement configuration require a fresh local or release run.
250
289
251
290
### Compare Published Release Artifacts
252
291
@@ -276,14 +315,15 @@ shared-harness workflow before attributing a difference solely to library code.
276
315
| Path | Committed? | Producer | Purpose |
277
316
|------|------------|----------|---------|
278
317
|`target/criterion/`| No |`cargo bench`, `bench-save-*`| Local Criterion measurements and named baselines. |
279
-
|`target/bench-reports/performance.md`| No |`bench-compare`, `performance-local*`| Local comparison report. |
280
-
|`target/bench-reports/performance.csv`| No |`performance-local*`, `performance-release`| Validated tabular inputs for the release comparison. |
281
-
|`target/bench-reports/performance.provenance.json`| No |`performance-local*`, `performance-release`| Schema, release, source, command, toolchain, host, digest, and harness provenance. |
318
+
|`target/bench-reports/performance.md`| No |`bench-compare`, `performance-local`, `performance-release`, `performance-doc`| Canonical local comparison report. |
319
+
|`target/bench-reports/performance.csv`| No |`performance-local`, `performance-release`| Validated tabular inputs for the canonical comparison. |
320
+
|`target/bench-reports/performance.provenance.json`| No |`performance-local`, `performance-release`| Schema, package identifiers, source, command, toolchain, host, digest, and harness provenance. |
321
+
|`target/bench-reports/performance-non-exact.*`| No |`performance-local-non-exact`| Narrowed non-exact report and retained peer-context comparison inputs. |
282
322
|`target/bench-reports/github-assets-performance.md`| No |`performance-github-assets`| Local report from published release artifacts. |
283
323
|`target/bench-reports/github-assets-performance.csv`| No |`performance-github-assets`| Tabular inputs derived from published native archives. |
284
324
|`target/bench-reports/github-assets-performance.provenance.json`| No |`performance-github-assets`| Provenance for the published-asset report inputs. |
0 commit comments