Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/setup-just/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,6 @@ runs:
echo "version=$version" >> "$GITHUB_OUTPUT"

- name: Install just
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: just@${{ steps.resolve.outputs.version }}
17 changes: 16 additions & 1 deletion .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ updates:
- "github-actions"
groups:
github-actions:
applies-to: version-updates
patterns:
- "*"
github-actions-security:
applies-to: security-updates
patterns:
- "*"

Expand All @@ -33,9 +38,14 @@ updates:
labels:
- "dependencies"
- "rust"
# Group all updates together
# Group version and security updates separately
groups:
dependencies:
applies-to: version-updates
patterns:
- "*"
rust-security:
applies-to: security-updates
patterns:
- "*"

Expand All @@ -55,5 +65,10 @@ updates:
- "python"
groups:
dependencies:
applies-to: version-updates
patterns:
- "*"
python-security:
applies-to: security-updates
patterns:
- "*"
4 changes: 2 additions & 2 deletions .github/workflows/audit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
restore-keys: advisory-db-

- name: Install cargo-audit
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: cargo-audit@${{ env.CARGO_AUDIT_VERSION }}

Expand Down Expand Up @@ -87,7 +87,7 @@ jobs:
github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository
)
uses: github/codeql-action/upload-sarif@f205ea1c3313d32999d8d6a48b4f6530d4437b38 # v4.37.4
uses: github/codeql-action/upload-sarif@5595ccaf912efad79be6eef63a5619ff05969be3 # v4.37.6
with:
sarif_file: audit-results.sarif
category: cargo-audit
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ jobs:
echo "version=$version" >> "$GITHUB_OUTPUT"

- name: Install cargo-nextest
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: cargo-nextest@${{ steps.cargo_nextest_version.outputs.version }}

Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -123,19 +123,19 @@ jobs:
run: uv sync --locked --group dev

- name: Install dprint
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: dprint@${{ steps.tool_versions.outputs.DPRINT_VERSION }}

- name: Install rumdl
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: rumdl@${{ steps.tool_versions.outputs.RUMDL_VERSION }}

- name: Install taplo
id: install-taplo
continue-on-error: ${{ matrix.os == 'windows-latest' }}
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: taplo-cli@${{ steps.tool_versions.outputs.TAPLO_VERSION }}

Expand All @@ -147,22 +147,22 @@ jobs:
TAPLO_VERSION: ${{ steps.tool_versions.outputs.TAPLO_VERSION }}

- name: Install typos
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: typos-cli@${{ steps.tool_versions.outputs.TYPOS_VERSION }}

- name: Install zizmor
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: zizmor@${{ steps.tool_versions.outputs.ZIZMOR_VERSION }}

- name: Install cargo-machete
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: cargo-machete@${{ steps.tool_versions.outputs.CARGO_MACHETE_VERSION }}

- name: Install cargo-nextest
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: cargo-nextest@${{ steps.tool_versions.outputs.CARGO_NEXTEST_VERSION }}

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/codecov.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,12 +68,12 @@ jobs:
run: rustup component add llvm-tools-preview

- name: Install cargo-llvm-cov
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: cargo-llvm-cov@${{ steps.tool_versions.outputs.CARGO_LLVM_COV_VERSION }}

- name: Install cargo-nextest
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: cargo-nextest@${{ steps.tool_versions.outputs.CARGO_NEXTEST_VERSION }}

Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,20 +35,20 @@ jobs:

- name: Initialize CodeQL
if: matrix.language != 'rust'
uses: github/codeql-action/init@f205ea1c3313d32999d8d6a48b4f6530d4437b38 # v4.37.4
uses: github/codeql-action/init@5595ccaf912efad79be6eef63a5619ff05969be3 # v4.37.6
with:
languages: ${{ matrix.language }}
queries: security-extended

- name: Initialize CodeQL (Rust)
if: matrix.language == 'rust'
uses: github/codeql-action/init@f205ea1c3313d32999d8d6a48b4f6530d4437b38 # v4.37.4
uses: github/codeql-action/init@5595ccaf912efad79be6eef63a5619ff05969be3 # v4.37.6
with:
languages: ${{ matrix.language }}
build-mode: none
queries: security-extended

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@f205ea1c3313d32999d8d6a48b4f6530d4437b38 # v4.37.4
uses: github/codeql-action/analyze@5595ccaf912efad79be6eef63a5619ff05969be3 # v4.37.6
with:
category: "/language:${{ matrix.language }}"
2 changes: 1 addition & 1 deletion .github/workflows/release-benchmarks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
echo "version=$version" >> "$GITHUB_OUTPUT"

- name: Install cargo-nextest
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: cargo-nextest@${{ steps.cargo_nextest_version.outputs.version }}

Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/rust-clippy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -74,12 +74,12 @@ jobs:
} >> "$GITHUB_OUTPUT"

- name: Install clippy-sarif
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: clippy-sarif@${{ steps.tool_versions.outputs.CLIPPY_SARIF_VERSION }}

- name: Install sarif-fmt
uses: taiki-e/cache-cargo-install-action@417450f3c33ee20393705369577571770643d4c7 # v3.0.7
uses: taiki-e/cache-cargo-install-action@9ee83daaa7b96a6fab930949ecf1122bba04a389 # v3.0.8
with:
tool: sarif-fmt@${{ steps.tool_versions.outputs.SARIF_FMT_VERSION }}

Expand All @@ -104,7 +104,7 @@ jobs:
github.event_name != 'pull_request'
|| github.event.pull_request.head.repo.full_name == github.repository
)
uses: github/codeql-action/upload-sarif@f205ea1c3313d32999d8d6a48b4f6530d4437b38 # v4.37.4
uses: github/codeql-action/upload-sarif@5595ccaf912efad79be6eef63a5619ff05969be3 # v4.37.6
with:
sarif_file: rust-clippy-results.sarif
category: "clippy"
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/semgrep-sarif.yml
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ jobs:
github.event_name != 'pull_request' ||
github.event.pull_request.head.repo.full_name == github.repository
)
uses: github/codeql-action/upload-sarif@f205ea1c3313d32999d8d6a48b4f6530d4437b38 # v4.37.4
uses: github/codeql-action/upload-sarif@5595ccaf912efad79be6eef63a5619ff05969be3 # v4.37.6
with:
sarif_file: semgrep-results.sarif
category: semgrep-repository-rules
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Install Rust 1.97.1 through [rustup](https://rustup.rs/), Git, Python 3.14,
pinned `just` version from its locked dependency graph:

```bash
cargo install --locked just --version 1.57.0
cargo install --locked just --version 1.58.0
```

Set up the remaining development tools and validate the checkout:
Expand Down
18 changes: 9 additions & 9 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ include = [
[dependencies]
# All runtime deps are optional; see [features] below.
# Must stay in sync with num-rational
num-bigint = { version = "0.4.6", optional = true }
num-bigint = { version = "0.4.8", optional = true }
num-rational = { version = "0.4.2", features = [ "num-bigint-std" ], optional = true }
num-traits = { version = "0.2.19", optional = true }

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -575,7 +575,7 @@ Install Rust 1.97.1 through [rustup](https://rustup.rs/), Git, Python 3.14,
`just` release from its locked dependency graph:

```bash
cargo install --locked just --version 1.57.0
cargo install --locked just --version 1.58.0
just setup # install/verify dev tools + sync Python deps
just check # lint/validate (non-mutating)
just fix # apply auto-fixes (mutating)
Expand Down
45 changes: 41 additions & 4 deletions docs/BENCHMARKING.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ the commands measure and where their outputs go.
| Full crate comparison | `just bench-vs-linalg` |
| README table and plot | `just plot-vs-linalg-readme` |
| Release report | `just performance-release v0.4.4 v0.4.3` |
| Re-render retained release inputs | `just performance-rerender` |
| Published-asset comparison | `just performance-github-assets v0.4.4 v0.4.3` |

Rule of thumb:
Expand All @@ -39,6 +40,8 @@ Rule of thumb:
- Use `bench-vs-linalg` plus plotting when updating README crate-to-crate
comparisons.
- Use `performance-release` only when preparing committed release artifacts.
- Use `performance-rerender` for report-format changes after a valid release
dataset has already been retained.

## Benchmark Suites

Expand Down Expand Up @@ -200,11 +203,33 @@ With no arguments, `just performance-release` infers the current release tag
from `Cargo.toml` and discovers the previous stable published release. During
release preparation, passing both tags explicitly removes ambiguity.

This command creates temporary worktrees, generates the comparison, writes
`docs/PERFORMANCE.md`, and archives the previous committed report under
This command creates temporary worktrees, validates the complete comparison,
and writes the exact report inputs to
`target/bench-reports/performance.csv` with adjacent
`performance.provenance.json`. The CSV records deterministic benchmark keys,
coverage status and notes, and baseline/current median estimates with complete
confidence intervals in nanoseconds. The JSON sidecar binds the CSV digest and
row count to the release pair, source states, commands, toolchain, Criterion
version, harness/configuration digests, host, and schema version.

The pair is validated and published before the temporary worktree is removed.
`docs/PERFORMANCE.md` is then rendered from a validated reload of that retained
pair, and the previous committed report is archived under
`docs/archive/performance/`. Archive filenames are release-pair names such as
`v0.4.2-vs-v0.4.1.md`. Publication fails rather than emitting a partial report
when a selected suite or required dimension is absent.
`v0.4.2-vs-v0.4.1.md`. Serialization, validation, rendering, coverage, or
promotion failures preserve the previous valid report and artifact pair.

To reproduce and promote the report without running Cargo or creating Git
worktrees, use:

```bash
just performance-rerender
```

This command fails closed on a missing, partial, malformed, mismatched, or
unsupported artifact pair. Use it for presentation-only report corrections;
changes to benchmark inputs, code, toolchains, or measurement configuration
require a fresh `performance-release` run.

### Compare Published Release Artifacts

Expand Down Expand Up @@ -235,7 +260,11 @@ shared-harness workflow before attributing a difference solely to library code.
|------|------------|----------|---------|
| `target/criterion/` | No | `cargo bench`, `bench-save-*` | Local Criterion measurements and named baselines. |
| `target/bench-reports/performance.md` | No | `bench-compare`, `performance-local*` | Local comparison report. |
| `target/bench-reports/performance.csv` | No | `performance-local*`, `performance-release` | Validated tabular inputs for the release comparison. |
| `target/bench-reports/performance.provenance.json` | No | `performance-local*`, `performance-release` | Schema, release, source, command, toolchain, host, digest, and harness provenance. |
| `target/bench-reports/github-assets-performance.md` | No | `performance-github-assets` | Local report from published release artifacts. |
| `target/bench-reports/github-assets-performance.csv` | No | `performance-github-assets` | Tabular inputs derived from published native archives. |
| `target/bench-reports/github-assets-performance.provenance.json` | No | `performance-github-assets` | Provenance for the published-asset report inputs. |
| `docs/PERFORMANCE.md` | Yes | `performance-release` | Latest curated release-to-release comparison. |
| `docs/archive/performance/` | Yes | `performance-release` | Older curated release-to-release comparisons. |
| `docs/assets/bench/` | Yes | `plot-vs-linalg-readme` | README benchmark CSV/SVG assets and JSON provenance. |
Expand All @@ -244,6 +273,14 @@ shared-harness workflow before attributing a difference solely to library code.
Published baseline assets use the filename
`la-stack-$TAG-criterion-baseline.tar.gz`.

Everything under `target/bench-reports/` is reproducible local scratch owned by
the performance-report workflows. It survives temporary-worktree cleanup but
may be removed by `just clean` or `cargo clean`; retain or copy the CSV/JSON pair
while reviewing or rerendering a release PR. The compact CSV is the analysis
and Markdown-reproduction layer. It does not replace the full native Criterion
`.tar.gz` archive attached to each GitHub Release, which remains the durable raw
baseline for post-release comparisons.

## `vs_linalg` Methodology

`vs_linalg` is a per-kernel comparison, not a single aggregate score. Each row
Expand Down
Loading
Loading