Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions .github/ISSUE_TEMPLATE/defect-report.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name: Defect report
description: Share a defect Shakedown found on your Mac
title: "Defect report [chip/model]"
title: "Defect report: [chip/model]"
labels: ["defect-report"]
body:
- type: markdown
Expand Down Expand Up @@ -41,16 +41,16 @@ body:
attributes:
label: Which phase failed
options:
- "0 Pre-flight"
- "1 Inventory"
- "2 Battery"
- "3 Sensors"
- "4 CPU variance"
- "5 Thermal load"
- "6 Display"
- "7 Physical inspection"
- "8 Apple Diagnostics"
- "9 Idle drain"
- "0: Pre-flight"
- "1: Inventory"
- "2: Battery"
- "3: Sensors"
- "4: CPU variance"
- "5: Thermal load"
- "6: Display"
- "7: Physical inspection"
- "8: Apple Diagnostics"
- "9: Idle drain"
- Multiple
validations:
required: true
Expand All @@ -76,7 +76,7 @@ body:
id: notes
attributes:
label: Anything else
description: Context that might matter store, ambient temp, repro pattern, etc.
description: Context that might matter: store, ambient temp, repro pattern, etc.
validations:
required: false

Expand Down
16 changes: 9 additions & 7 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:

jobs:
lint:
# Static checks only scripts target macOS at runtime, but shellcheck,
# Static checks only, scripts target macOS at runtime, but shellcheck,
# python ast.parse, and json.tool are all platform-agnostic.
runs-on: ubuntu-latest
steps:
Expand All @@ -29,7 +29,7 @@ jobs:

- name: Syntax-check Python heredocs
# Each script may contain one or more `python3 - <<'PYEOF' … PYEOF` blocks.
# Extract them with awk and run ast.parse catches Python syntax errors
# Extract them with awk and run ast.parse, catches Python syntax errors
# without executing macOS-only side effects (system_profiler, ioreg, etc.).
run: |
set -euo pipefail
Expand Down Expand Up @@ -68,9 +68,11 @@ jobs:
"target", "unit", "phases", "result", "result_reason",
"submission_safe"]
REQUIRED_PHASES = ["0_preflight", "1_inventory", "2_battery", "3_sensors",
"4_cpu_variance", "5_thermal_load", "6_display",
"7_physical", "8_apple_diagnostics", "9_idle_drain",
"10_race_bench", "11_ssd_test"]
"4_cpu_variance", "4b_cpu_variance_noaccel",
"5_thermal_load", "6_display", "7_physical",
"8_apple_diagnostics", "9_idle_drain",
"10_race_bench", "11_ssd_test", "12_memory_bandwidth",
"13_gpu_variance"]
FILENAME_RE = re.compile(r"^\d{4}-\d{2}-\d{2}-[a-z0-9-]+-[0-9a-f]{4}\.json$")

def find_keys(obj, predicate, path=""):
Expand Down Expand Up @@ -136,7 +138,7 @@ jobs:
for e in errors:
print(f" - {e}", file=sys.stderr)
sys.exit(1)
print(f"audited {len(files)} submission file(s) all clean")
print(f"audited {len(files)} submission file(s), all clean")
PYEOF

- name: Verify markdown links resolve to existing files
Expand All @@ -147,7 +149,7 @@ jobs:
#
# Implemented in Python instead of bash because the previous bash
# version used `${path//%/\\x}` parameter-expansion escapes that
# behave differently in zsh false positives for contributors
# behave differently in zsh, false positives for contributors
# running this check locally on default-shell macOS.
run: |
python3 <<'PYEOF'
Expand Down
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,18 @@ All notable changes to this project are documented here. Format follows [Keep a

## [Unreleased]

### Added: roadmap completion (wave 7)

- **Phase 4b non-accelerated CPU variance** (`cpu-variance.sh WORKLOAD=blake2b`, opt-in `./run --noaccel`). Reruns the Phase 4 variance methodology with BLAKE2b, which has no dedicated CPU instruction and so exercises the integer pipelines and memory instead of the SHA engine. Catches batch defects that SHA-NI / the Apple crypto coprocessor hide. Same script and verdict logic as Phase 4 (no duplicated thresholds): real `pass`/`warn`/`fail` when run, `skipped` placeholder otherwise. Runs on the already-hot chassis after Phase 4 with a short re-warm and no cold burst. Adds ~6 min when enabled.
- **Phase 12 memory bandwidth** (`memory-bandwidth.sh`). Multi-threaded sequential `ctypes.memmove` copy, one worker per P-core, over buffers sized to exceed the system-level cache (DRAM-bound, not cache-bound). Reports copy bandwidth, touched (read+write) bandwidth, and run-to-run spread. Pure Python stdlib with no native kernel, so it's an honest lower bound on a full STREAM triad (a single thread can't saturate Apple Silicon's multi-channel controllers, hence multi-threaded the way STREAM uses OpenMP). RAM-guarded: the working set is capped at 40% of physical memory, and it skips if it can't size cache-busting buffers. `info` verdict in v0.2. Runs cold, ~15 s.
- **Phase 13 GPU compute variance** (`gpu-variance.sh`, opt-in `./run --gpu`). Compiles a small Metal compute kernel with `swiftc` at runtime (no binary shipped) and runs sustained FMA work, measuring per-iteration GFLOP/s and variance. The GPU is the bigger thermal contributor on Apple Silicon. This is the one phase that isn't pure bash + Python stdlib; the Metal source is inline in the script heredoc. Degrades to `skipped` (never fails the run) when `swiftc`, the Metal compiler, or a Metal device is unavailable (headless / SSH session, or an Intel Mac without a usable GPU context). `info` verdict in v0.2. The default `./run` is unchanged and stays pure-script; see [SECURITY.md](SECURITY.md).
- **SSD conservative floor.** Phase 11 (`ssd-test.sh`) now emits `warn` (advisory, never fail) when read or write is below 500 MB/s with the page cache dropped, recorded as `ssd_floor_mb_per_s`. Every NVMe Mac since ~2016 clears > 1000 MB/s, so this flags the documented 256 GB single-NAND-die regression (M2 Air / base 13" MBP), a failing drive, or pre-NVMe storage. Calibrated chassis-family bands still land in v0.3.
- **14" vs 16" thermal sub-classes.** Split `active-cooled-pro` into `active-cooled-pro-14` (looser steady-state and cliff bands; the 14" M5 Max throttles by design) and `active-cooled-pro-16` (the previous strict thresholds). The bare `active-cooled-pro` stays valid as an alias for the 16" table, so existing presets and prior submissions stay comparable. Auto-detect can't distinguish 14" from 16" (system_profiler doesn't expose screen size on Apple Silicon), so the precise sub-class comes only from a `--target` preset; `mbp-14-m5-pro-24` and `mbp-16-m5-max-64` now set it explicitly.
- **Per-core pinning investigation** (`Verification/per-core-pinning.md`). Documents why macOS has no public per-core affinity API (`THREAD_AFFINITY_POLICY` is ignored on Apple Silicon, QoS only steers between the P and E clusters), why a single defective core gets averaged across workers, and why `worker_imbalance_pct_per_iter` is the chosen partial mitigation.
- **Generation calibrations: M1-M4 and Intel/T2 era.** Added `examples/m1-2020/`, `examples/m2-2022/`, `examples/m3-2023/`, `examples/m4-2024/`, and `examples/intel-2016/`, each a documented, sourced defect landscape in the `examples/m5-2026/` format (overview, per-issue notes, sources). Highlights: M1 24" iMac display-cable lines and 14"/16" speaker crackle; the M2 256 GB single-die SSD regression; M3's mostly-clean generation (8 GB RAM and M3 Pro bandwidth are spec choices, not defects); M4's OS-gated JIT kernel panic; and the Intel/T2 era's butterfly keyboard, Flexgate, T2 audio/panics, 2018 i9 throttling, and 2015 battery recall. Each issue maps to the Shakedown phase that surfaces it. Added representative target presets (`macbook-air-m1`, `macbook-air-m2`, `mbp-14-m3-pro`, `mac-mini-m4`) and repointed `mbp-16-intel-2019` at the new `examples/intel-2016/` calibration.
- **Schema bump to 1.2.** New phase keys (`12_memory_bandwidth`, `4b_cpu_variance_noaccel`, `13_gpu_variance`) added to the CI submission-audit required-phase list; the orchestrator always emits them (opt-in phases as `skipped` placeholders), so default-run submissions still pass. Backward compatible: pre-1.2 reports validate against the 1.1 shape.
- **Comparability impact.** The new `info` phases don't change existing verdicts. The SSD floor can turn a previously-`info` SSD result into `warn` (a PASS-with-warns) on genuinely slow drives. The `active-cooled-pro` → `-16` aliasing preserves prior thermal verdicts exactly; a 14" run with `--target mbp-14-...` is now judged against looser bands than before (it previously used the strict 16" table).

### Added: performance benchmarks (wave 6)

- **Phase 10 race benchmark (`race-bench.sh`).** Compresses a 200 MB incompressible random blob with `xz -9 -T<P-cores>` and reports wall-clock seconds plus MB/s throughput. Unlike Phase 4 (SHA-256, hardware-accelerated on Apple Silicon), LZMA is a general-purpose CPU workload that does not benefit from SHA-NI or Apple's crypto coprocessor, so the number is comparable across chassis families. Runs cold (before Phase 4) so boost headroom isn't already burned. Skipped gracefully if `xz` isn't on PATH.
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ A target preset lets users run `./run --target <name>` to assert the unit matche
| `memory_gb` | Asserted against `sysctl hw.memsize` |
| `model_must_include` | Substring asserted against `machine_model` (catches chassis size) |
| `calibration_dir` | Path to the `examples/<generation>/` notes for failure-analysis cross-reference |
| `thermal_chassis_class` | One of `fanless`, `active-cooled-pro`, `desktop`, `intel-laptop`, `intel-desktop`. Sets thermal thresholds and Phase 4 warmup duration. See [`targets/README.md`](targets/README.md) for the per-class definition. |
| `thermal_chassis_class` | One of `fanless`, `active-cooled-pro` (with `active-cooled-pro-14` / `active-cooled-pro-16` sub-classes for the 14" / 16" MacBook Pro), `desktop`, `intel-laptop`, `intel-desktop`. Sets thermal thresholds and Phase 4 warmup duration. See [`targets/README.md`](targets/README.md) for the per-class definition. |

Open a PR adding the JSON. Match an existing one for style.

Expand All @@ -52,7 +52,7 @@ The verification scripts and runbook **don't need to change**. They're calibrati

## Improving a script

The 5 scripts in `Verification/scripts/` are deliberately self-contained: bash plus Python heredocs, no external dependencies beyond what ships with macOS. Keep it that way. If you need to add a third-party benchmark (Geekbench, sysbench), make it optional with a fallback to the built-in equivalent.
The scripts in `Verification/scripts/` are deliberately self-contained: bash plus Python heredocs, no external dependencies beyond what ships with macOS. Keep it that way. If you need to add a third-party benchmark (Geekbench, sysbench), or something that genuinely needs more than the stdlib (the opt-in `--gpu` phase compiles a Metal kernel with `swiftc` at runtime), make it optional and have it skip cleanly when the dependency is absent, so the default `./run` stays pure bash plus Python stdlib.

### Validating changes to a script

Expand Down
Loading