Skip to content

Commit 89f3720

Browse files
committed
feat(api)!: make exact f64 conversions strict
- Make Matrix and Vector the finite-by-construction public types for exact arithmetic. - Add rounded exact-to-f64 APIs for determinant and solve callers that want explicit lossy conversion. - Return typed Unrepresentable reasons when strict exact-to-f64 conversion would round or become non-finite. - Specialize D4 exact determinants and keep determinant/error-bound zero coefficients from evaluating overflowing absent terms. - Update exact benchmark comparison reporting to compare strict and rounded APIs against legacy v0.4.2 rows. BREAKING CHANGE: strict exact-to-f64 APIs now return LaError::Unrepresentable instead of silently rounding, public Matrix and Vector construction is fallible, and the previous finite proof wrapper APIs are removed.
1 parent 8e33f1a commit 89f3720

9 files changed

Lines changed: 548 additions & 127 deletions

File tree

CHANGELOG.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10+
### ⚠️ Breaking Changes
11+
12+
- Make exact f64 conversions strict
13+
1014
### Added
1115

1216
- Guard README dependency snippets [`7137fee`](https://github.com/acgetchell/la-stack/commit/7137fee16ab33e08f4dc6a60e02417e3e7c4e020)
@@ -32,6 +36,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
3236
- Keep the regular benchmark workflow focused on PR and main-branch comparison runs.
3337
- Document how to restore archived release baselines for future performance comparisons.
3438
- Feat!(api): make Matrix and Vector finite by construction [`1fa2f55`](https://github.com/acgetchell/la-stack/commit/1fa2f55cfac6f249a7e2bf30922901539e580dd8)
39+
- [**breaking**] Make exact f64 conversions strict [`8e33f1a`](https://github.com/acgetchell/la-stack/commit/8e33f1a8ec291bfcb6312375969efce076421e96)
40+
- Add explicit rounded exact-to-f64 APIs for determinant and solve results
41+
- Report exact conversion failures with typed Unrepresentable reasons
42+
- Remove finite proof wrapper APIs now that Matrix and Vector carry finiteness directly
43+
- Move error and tolerance contracts into first-class modules with prelude exports
44+
- Update exact benchmarks to distinguish strict Result paths from rounded f64 paths
45+
- Document and exercise the rounded fallback pattern for RequiresRounding errors
3546

3647
### Changed
3748

docs/BENCHMARKING.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -193,6 +193,23 @@ local. The report includes per-dimension tables showing median times,
193193
percent change, speedup, and last-release nalgebra/faer context where a
194194
matching `vs_linalg` peer exists.
195195

196+
For exact-arithmetic comparisons against v0.4.2 or older baselines, rows such
197+
as `det_exact_rounded_f64 (vs det_exact_f64)` mean the current rounded API is
198+
being compared to the historical lossy `*_exact_f64` benchmark. Rows such as
199+
`det_exact_f64_result (vs det_exact_f64)` intentionally show the overhead of
200+
the new strict conversion contract against that same historical baseline.
201+
202+
The default `release-signal` scope reports exact-arithmetic rows whose inputs
203+
are fixed across versions: deterministic D=2..=5 cases plus adversarial fixed
204+
matrices. Random percentile groups are exploratory tail probes; each benchmark
205+
run selects p50/p95/p99 input sets by timing the implementation under test, so
206+
those rows can measure different corpus subsets across versions. Include them
207+
when investigating tails with:
208+
209+
```bash
210+
uv run bench-compare v0.4.2 --suite exact --scope all-benches
211+
```
212+
196213
To generate a current snapshot without a saved baseline:
197214

198215
```bash

scripts/bench_compare.py

Lines changed: 91 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -43,16 +43,35 @@
4343
#
4444
# Mirrors the structure of `benches/exact.rs`: general-case per-dimension
4545
# groups (`exact_d{2..5}`), fixed-seed random percentile groups, plus
46-
# adversarial/extreme-input groups that share a fixed four-bench layout
47-
# (`det_sign_exact`, `det_exact`, `solve_exact`, `solve_exact_f64`).
48-
_EXTREME_BENCHES: list[str] = ["det_sign_exact", "det_exact", "solve_exact", "solve_exact_f64"]
46+
# adversarial/extreme-input groups that share a fixed five-bench layout
47+
# (`det_sign_exact`, `det_exact`, `solve_exact`, `solve_exact_f64_result`,
48+
# `solve_exact_rounded_f64`).
49+
_EXTREME_BENCHES: list[str] = [
50+
"det_sign_exact",
51+
"det_exact",
52+
"solve_exact",
53+
"solve_exact_f64_result",
54+
"solve_exact_rounded_f64",
55+
]
4956
_RANDOM_PERCENTILE_BENCHES: list[str] = [f"{operation}_{percentile}" for operation in _EXTREME_BENCHES for percentile in ("p50", "p95", "p99")]
5057

58+
_EXACT_DIMENSION_BENCHES: list[str] = [
59+
"det",
60+
"det_direct",
61+
"det_exact",
62+
"det_exact_f64_result",
63+
"det_exact_rounded_f64",
64+
"det_sign_exact",
65+
"solve_exact",
66+
"solve_exact_f64_result",
67+
"solve_exact_rounded_f64",
68+
]
69+
5170
EXACT_GROUPS: dict[str, list[str]] = {
52-
"exact_d2": ["det", "det_direct", "det_exact", "det_exact_f64", "det_sign_exact", "solve_exact", "solve_exact_f64"],
53-
"exact_d3": ["det", "det_direct", "det_exact", "det_exact_f64", "det_sign_exact", "solve_exact", "solve_exact_f64"],
54-
"exact_d4": ["det", "det_direct", "det_exact", "det_exact_f64", "det_sign_exact", "solve_exact", "solve_exact_f64"],
55-
"exact_d5": ["det", "det_direct", "det_exact", "det_exact_f64", "det_sign_exact", "solve_exact", "solve_exact_f64"],
71+
"exact_d2": _EXACT_DIMENSION_BENCHES,
72+
"exact_d3": _EXACT_DIMENSION_BENCHES,
73+
"exact_d4": _EXACT_DIMENSION_BENCHES,
74+
"exact_d5": _EXACT_DIMENSION_BENCHES,
5675
"exact_random_percentile_d2": _RANDOM_PERCENTILE_BENCHES,
5776
"exact_random_percentile_d3": _RANDOM_PERCENTILE_BENCHES,
5877
"exact_random_percentile_d4": _RANDOM_PERCENTILE_BENCHES,
@@ -63,6 +82,25 @@
6382
"exact_hilbert_5x5": _EXTREME_BENCHES,
6483
}
6584

85+
EXACT_RELEASE_SIGNAL_GROUPS: frozenset[str] = frozenset(group for group in EXACT_GROUPS if not group.startswith("exact_random_percentile_d"))
86+
87+
# v0.4.2 and earlier named the lossy exact-to-f64 benches after the public
88+
# `*_exact_f64` API. Current benches split that behavior into strict `*_result`
89+
# and lossy `*_rounded_f64` variants. Use the old baseline when present so
90+
# release reports show both compatibility-successor performance and strict
91+
# conversion overhead instead of silently dropping the new rows.
92+
EXACT_LEGACY_BASELINE_BENCHES: dict[str, str] = {
93+
"det_exact_f64_result": "det_exact_f64",
94+
"det_exact_rounded_f64": "det_exact_f64",
95+
"solve_exact_f64_result": "solve_exact_f64",
96+
"solve_exact_rounded_f64": "solve_exact_f64",
97+
}
98+
99+
_EXACT_LEGACY_PREFIX_BASELINE_BENCHES: tuple[tuple[str, str], ...] = (
100+
("solve_exact_f64_result_", "solve_exact_f64_"),
101+
("solve_exact_rounded_f64_", "solve_exact_f64_"),
102+
)
103+
66104
VS_LINALG_LA_STACK_ONLY_BENCHES_BY_METRIC: dict[str, list[str]] = {
67105
"det_via_lu": ["la_stack_det"],
68106
}
@@ -123,6 +161,7 @@ class Comparison:
123161
current_ns: float
124162
speedup: float # baseline / current (>1 = faster)
125163
pct_change: float # signed percent change (negative = faster)
164+
baseline_bench: str | None = None
126165
baseline_nalgebra_ns: float | None = None
127166
baseline_faer_ns: float | None = None
128167

@@ -231,6 +270,36 @@ def _collect_exact_results(criterion_dir: Path, sample: str, stat: str) -> list[
231270
return results
232271

233272

273+
def _legacy_exact_baseline_bench(bench: str) -> str | None:
274+
"""Return the legacy exact benchmark name for renamed rows."""
275+
legacy_bench = EXACT_LEGACY_BASELINE_BENCHES.get(bench)
276+
if legacy_bench is not None:
277+
return legacy_bench
278+
279+
for current_prefix, legacy_prefix in _EXACT_LEGACY_PREFIX_BASELINE_BENCHES:
280+
if bench.startswith(current_prefix):
281+
return f"{legacy_prefix}{bench.removeprefix(current_prefix)}"
282+
283+
return None
284+
285+
286+
def _exact_baseline_path(group_dir: Path, bench: str, baseline_name: str) -> tuple[str, Path]:
287+
"""Return the exact benchmark baseline path, falling back to legacy names."""
288+
base_path = group_dir / bench / baseline_name / "estimates.json"
289+
if base_path.exists():
290+
return (bench, base_path)
291+
292+
legacy_bench = _legacy_exact_baseline_bench(bench)
293+
if legacy_bench is None:
294+
return (bench, base_path)
295+
296+
legacy_path = group_dir / legacy_bench / baseline_name / "estimates.json"
297+
if legacy_path.exists():
298+
return (legacy_bench, legacy_path)
299+
300+
return (bench, base_path)
301+
302+
234303
def _ordered_vs_linalg_benches(group_dir: Path, sample: str) -> list[str]:
235304
"""Return present vs_linalg benches in a stable, metric-aware order."""
236305
present = {child.name for child in group_dir.iterdir() if child.is_dir() and (child / sample / "estimates.json").exists()}
@@ -275,18 +344,22 @@ def _collect_exact_comparisons(
275344
criterion_dir: Path,
276345
baseline_name: str,
277346
stat: str,
347+
scope: str,
278348
) -> list[Comparison]:
279349
"""Compare current exact-arithmetic results against a named baseline."""
280350
comparisons: list[Comparison] = []
281351

282352
for group, benches in EXACT_GROUPS.items():
353+
if scope == "release-signal" and group not in EXACT_RELEASE_SIGNAL_GROUPS:
354+
continue
355+
283356
group_dir = criterion_dir / group
284357
if not group_dir.is_dir():
285358
continue
286359

287360
for bench in benches:
288361
new_path = group_dir / bench / "new" / "estimates.json"
289-
base_path = group_dir / bench / baseline_name / "estimates.json"
362+
baseline_bench, base_path = _exact_baseline_path(group_dir, bench, baseline_name)
290363

291364
if not new_path.exists() or not base_path.exists():
292365
continue
@@ -306,6 +379,7 @@ def _collect_exact_comparisons(
306379
current_ns=new_point,
307380
speedup=speedup,
308381
pct_change=pct_change,
382+
baseline_bench=baseline_bench if baseline_bench != bench else None,
309383
)
310384
)
311385

@@ -346,6 +420,13 @@ def _baseline_peer_times(group_dir: Path, bench: str, baseline_name: str, stat:
346420
return (nalgebra_ns, faer_ns)
347421

348422

423+
def _comparison_bench_label(comparison: Comparison) -> str:
424+
"""Return the display label for a comparison table row."""
425+
if comparison.baseline_bench is None:
426+
return comparison.bench
427+
return f"{comparison.bench} (vs {comparison.baseline_bench})"
428+
429+
349430
def _collect_vs_linalg_comparisons(
350431
criterion_dir: Path,
351432
baseline_name: str,
@@ -403,7 +484,7 @@ def _collect_comparisons(
403484
"""Compare current (new) results against a named baseline."""
404485
comparisons: list[Comparison] = []
405486
if suite in ("all", "exact"):
406-
comparisons.extend(_collect_exact_comparisons(criterion_dir, baseline_name, stat))
487+
comparisons.extend(_collect_exact_comparisons(criterion_dir, baseline_name, stat, scope))
407488
if suite in ("all", "vs_linalg"):
408489
comparisons.extend(_collect_vs_linalg_comparisons(criterion_dir, baseline_name, stat, scope))
409490
return comparisons
@@ -542,7 +623,7 @@ def _comparison_tables(comparisons: list[Comparison], baseline_name: str) -> str
542623
)
543624
for c in items:
544625
cells = [
545-
c.bench,
626+
_comparison_bench_label(c),
546627
_format_time(c.baseline_ns),
547628
_format_time(c.current_ns),
548629
_format_pct(c.pct_change),

scripts/tests/test_bench_compare.py

Lines changed: 40 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -34,12 +34,20 @@ def _build_criterion_tree(criterion_dir: Path, stat: str = "median") -> None:
3434
group = criterion_dir / f"exact_d{d}"
3535
_write_estimates(group / "det" / "new" / "estimates.json", stat, det)
3636
_write_estimates(group / "det_exact" / "new" / "estimates.json", stat, det_exact)
37+
_write_estimates(group / "det_exact_f64_result" / "new" / "estimates.json", stat, det_exact * 1.1)
38+
_write_estimates(group / "det_exact_rounded_f64" / "new" / "estimates.json", stat, det_exact * 1.2)
39+
_write_estimates(group / "solve_exact_f64_result" / "new" / "estimates.json", stat, det_exact * 1.3)
40+
_write_estimates(group / "solve_exact_rounded_f64" / "new" / "estimates.json", stat, det_exact * 1.4)
3741

3842
ns_group = criterion_dir / "exact_near_singular_3x3"
3943
_write_estimates(ns_group / "det_sign_exact" / "new" / "estimates.json", stat, 12000.0)
44+
_write_estimates(ns_group / "solve_exact_f64_result" / "new" / "estimates.json", stat, 51000.0)
45+
_write_estimates(ns_group / "solve_exact_rounded_f64" / "new" / "estimates.json", stat, 52000.0)
4046

4147
random_group = criterion_dir / "exact_random_percentile_d3"
4248
_write_estimates(random_group / "det_exact_p95" / "new" / "estimates.json", stat, 33000.0)
49+
_write_estimates(random_group / "solve_exact_f64_result_p95" / "new" / "estimates.json", stat, 54000.0)
50+
_write_estimates(random_group / "solve_exact_rounded_f64_p95" / "new" / "estimates.json", stat, 55000.0)
4351

4452

4553
def _build_vs_linalg_tree(criterion_dir: Path, stat: str = "median") -> None:
@@ -196,7 +204,7 @@ def test_read_estimate_non_numeric_ci_bound_names_field(tmp_path: Path) -> None:
196204
def test_collect_results(tmp_path: Path) -> None:
197205
_build_criterion_tree(tmp_path)
198206
results = bench_compare._collect_results(tmp_path, "new", "median")
199-
assert len(results) == 6 # 2 benches x 2 dims + 1 near-singular + 1 random percentile
207+
assert len(results) == 18 # 6 benches x 2 dims + 3 near-singular + 3 random percentile
200208
groups = {r.group for r in results}
201209
assert "exact_d2" in groups
202210
assert "exact_d3" in groups
@@ -216,11 +224,37 @@ def test_collect_comparisons(tmp_path: Path) -> None:
216224
group = tmp_path / f"exact_d{d}"
217225
_write_estimates(group / "det" / "v0.3.0" / "estimates.json", "median", det)
218226
_write_estimates(group / "det_exact" / "v0.3.0" / "estimates.json", "median", det_exact)
227+
_write_estimates(group / "det_exact_f64" / "v0.3.0" / "estimates.json", "median", det_exact * 1.1)
228+
_write_estimates(group / "solve_exact_f64" / "v0.3.0" / "estimates.json", "median", det_exact * 1.3)
229+
random_group = tmp_path / "exact_random_percentile_d3"
230+
_write_estimates(random_group / "det_exact_p95" / "v0.3.0" / "estimates.json", "median", 66000.0)
231+
_write_estimates(random_group / "solve_exact_f64_p95" / "v0.3.0" / "estimates.json", "median", 108000.0)
219232

220233
comparisons = bench_compare._collect_comparisons(tmp_path, "v0.3.0", "median")
221-
assert len(comparisons) == 4 # 2 benches x 2 dims (near-singular has no baseline)
234+
assert len(comparisons) == 12 # 6 benches x 2 dims (near-singular has no baseline)
235+
assert {c.group for c in comparisons} == {"exact_d2", "exact_d3"}
222236
for c in comparisons:
223237
assert c.speedup == pytest.approx(c.baseline_ns / c.current_ns)
238+
assert {(c.bench, c.baseline_bench) for c in comparisons if c.baseline_bench is not None} == {
239+
("det_exact_f64_result", "det_exact_f64"),
240+
("det_exact_rounded_f64", "det_exact_f64"),
241+
("solve_exact_f64_result", "solve_exact_f64"),
242+
("solve_exact_rounded_f64", "solve_exact_f64"),
243+
}
244+
245+
all_comparisons = bench_compare._collect_comparisons(
246+
tmp_path,
247+
"v0.3.0",
248+
"median",
249+
scope="all-benches",
250+
)
251+
assert len(all_comparisons) == 15
252+
random_comparisons = [c for c in all_comparisons if c.group == "exact_random_percentile_d3"]
253+
assert {(c.bench, c.baseline_bench) for c in random_comparisons} == {
254+
("det_exact_p95", None),
255+
("solve_exact_f64_result_p95", "solve_exact_f64_p95"),
256+
("solve_exact_rounded_f64_p95", "solve_exact_f64_p95"),
257+
}
224258

225259

226260
def test_collect_comparisons_zero_current(tmp_path: Path) -> None:
@@ -301,12 +335,16 @@ def test_comparison_tables_per_dimension(tmp_path: Path) -> None:
301335
group = tmp_path / f"exact_d{d}"
302336
_write_estimates(group / "det" / "v0.3.0" / "estimates.json", "median", det)
303337
_write_estimates(group / "det_exact" / "v0.3.0" / "estimates.json", "median", det_exact)
338+
_write_estimates(group / "det_exact_f64" / "v0.3.0" / "estimates.json", "median", det_exact * 1.1)
339+
_write_estimates(group / "solve_exact_f64" / "v0.3.0" / "estimates.json", "median", det_exact * 1.3)
304340

305341
comparisons = bench_compare._collect_comparisons(tmp_path, "v0.3.0", "median")
306342
tables = bench_compare._comparison_tables(comparisons, "v0.3.0")
307343
assert "### D=2" in tables
308344
assert "### D=3" in tables
309345
assert "| Benchmark | v0.3.0 | Latest | Change | Speedup |" in tables
346+
assert "det_exact_rounded_f64 (vs det_exact_f64)" in tables
347+
assert "solve_exact_f64_result (vs solve_exact_f64)" in tables
310348

311349

312350
def test_comparison_tables_include_vs_linalg_peer_context(tmp_path: Path) -> None:

0 commit comments

Comments
 (0)