shakedown: wave 6: performance benchmarks (race + SSD)#3
Merged
Conversation
race-bench.sh runs xz -9 -T<P-cores> against a 200 MB incompressible random blob. Unlike Phase 4 SHA-256, LZMA does not benefit from SHA-NI or the Apple crypto coprocessor, so the wall-clock seconds and throughput numbers are comparable across chassis families. Generates fresh urandom per chunk to avoid LZMA detecting repetition (a single repeated chunk compresses ~10x and breaks the test). ssd-test.sh writes SIZE_GB (default 2) of incompressible random data, drops the page cache via sudo purge, reads it back, reports MB/s for write and read. Random data sidesteps APFS zero-compression. Free-space safety: skipped if df shows less than 2x test size. In --no-sudo mode the phase still runs with ALLOW_NO_PURGE=1; page_cache_dropped: false flags the inflated read number. Both scripts emit verdict: "info" since v0.2 has no calibration thresholds for these. Numbers feed the corpus for v0.3 thresholds.
run-shakedown.sh: invoke race-bench (Phase 10) and ssd-test (Phase 11) after Phase 2 battery and before Phase 4 cpu-variance, so both run on a cold chassis (boost headroom unburned, NVMe controllers not heat-soaked). SSD test passes through ALLOW_NO_PURGE=1 when --no-sudo is set. Aggregator loads the two new JSONs, adds 10_race_bench and 11_ssd_test phase blocks to the report dict in run order. Verdict "info" doesn't contribute to overall PASS/FAIL (only "fail" and "warn" do). SCHEMA.md: bumped to v1.1. Documents Phase 10 and Phase 11 field shapes. Notes that v0.2 has no calibration thresholds; numbers are informational until v0.3. Adds a footer note on phase-key sort order (lex sorts 10 before 2, but JSON insertion order preserves run order for human reading). lint.yml submission audit: 10_race_bench and 11_ssd_test added to REQUIRED_PHASES. Pre-wave-6 submissions need to be re-run to pass. README: phase table extended, runtime hint updated (~20-47 min), Roadmap NVMe item updated to reflect "wave 6 ships measurement, v0.3 ships thresholds." CHANGELOG: wave 6 entry under [Unreleased] with all the above plus the design rationale (why xz, why fresh urandom per chunk, why incompressible random for SSD).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two new automated phases land cold, before Phase 4 heats the chassis:
race-bench.sh):xz -9 -T<P-cores>of a 200 MB incompressible random blob. Unlike Phase 4 SHA-256 (which hardware-accelerates on Apple Silicon), LZMA is fair across chassis families since no crypto coprocessor benefits it. Wall seconds + MB/s.ssd-test.sh): 2 GB incompressible random write,sudo purgeto drop the page cache, sequential read back. Reports write_mb_per_s and read_mb_per_s. Random data sidesteps APFS zero-compression. In--no-sudomode it still runs withALLOW_NO_PURGE=1, flaggingpage_cache_dropped: falseso the inflated read number is honest about what it measured.Both verdicts are
"info"in v0.2 since there are no chassis-family thresholds yet. Numbers contribute to the calibration corpus; v0.3 sets pass/fail bands once submissions accumulate.Schema bumped to v1.1 (additive only, backward compatible with 1.0 reports). CI submission audit now requires the two new phase keys.
Test plan
shellcheck -x Verification/scripts/*.sh)BLOB_MB=10 race-bench.shproduces compression ratio ~1.0 (incompressible) and realistic CPU-bound wall timeSIZE_GB=1 ALLOW_NO_PURGE=1 ssd-test.shwrites/reads withdata_quality: "few_samples"and a clear note about cache not being dropped./runend-to-end on Intel MBP to confirm orchestrator wires up the new phases correctly./runend-to-end on Apple Silicon (pending; first AS submission of the project)