Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ All notable changes to this project are documented here. Format follows [Keep a
- **`repeatability.sh` (Measurement System Analysis).** Runs the variance phase N times back to back on one unit and reports the gage's own run-to-run noise as a fraction of the fail tolerance (6-sigma of `spread_pct` over the runs, against the AIAG Gage R&R rule: under 10% capable, 10-30% marginal, over 30% the gage rivals the limit). Answers whether a metric is even capable of a pass/fail before we trust its limits. The repeatability leg of the production-QA path; reproducibility across stations and a golden-unit characterization run are still owed.
- **`--strict` precondition gating.** Production / calibration mode: the AC-power and quiet-system checks that are only warnings by default become hard preconditions, checked after inventory but before the load phases. A run not on AC or on a busy system aborts immediately (exit 3) instead of burning 45 minutes on a number that can't be compared to a baseline. Default runs are unchanged; the report records `strict_mode` in Phase 0. Ambient and chassis-soak gating still need fixturing the project lacks.
- **Preset auto-selection.** `./run` with no `--target` now matches the detected hardware (chip, memory, model) against the presets in `targets/` and uses the one that fits, so the right chassis class, inventory asserts, and golden baseline all apply with nothing to type. The match uses the same sources as the inventory asserts (`chip_type` / `cpu_type`, `hw.memsize`), so an auto-selected preset always passes its own asserts; exactly one match wins, and zero or several falls back to chassis auto-detection with asserts skipped, as before. `--target` is now an optional override for asserting an expected SKU or forcing a preset.
- **Guided picker.** With no flags on a terminal, `./run` now opens a short picker (verify a new Mac vs quick check, then strict and the AI load for the thorough profile), echoes the command it built, and runs it, so you don't have to remember flags. It detects the SKU for the header and sets the same variables the flags would. Any explicit flag, a non-interactive stdin (pipe / CI), or `SHAKEDOWN_YES=1` skips it entirely, so scripted and flag-driven use is unchanged.
- **Screen-size detection + Apple Silicon MacBook Pro coverage.** Screen size now comes from the built-in display panel's native resolution (`inventory.sh` records `unit.screen_inches`), because Apple Silicon does not put it in the model identifier. This makes auto-selection and the 14"/16" thermal sub-class work on Apple Silicon MacBook Pros, which previously matched only by the screen-size digit in Intel model IDs (`MacBookPro16,1`) and so never auto-selected (an M5 Max reports `Mac17,1`). Presets now match on `screen_inches` and a `"MacBook Pro"` family string instead of a bare `"14"`/`"16"`, the inventory asserts gain `screen_inches_matched`, and the chassis fallback picks the 14"/16" sub-class even when no preset matches. Added presets for the M1/M2 Pro+Max, M3 Max, M4 Pro+Max MacBook Pros and the M3/M4 Airs; other screen/chip combos fall back to the (now-correct) auto-detected chassis.
- **Store Day Checklist** reframed to a single in-store flow centered on the one command (`./run --store`) and the verdict banner.

Expand Down
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,9 @@ Shakedown is the procedure for catching those.
./run
```

`./run` auto-selects the matching preset from your hardware (chip, memory, model), so it bins against the right thresholds and baseline with nothing to type. Pass `--target mbp-16-m5-max-64` (see `targets/`) to assert an expected SKU, or to override the match; if no preset matches, it auto-detects chassis class from `system_profiler` and skips the inventory asserts.
On a terminal with no flags, `./run` opens a short guided picker (verify a new Mac vs quick check, plus strict and the AI load), builds the command, and runs it, so you don't have to remember flags. Pass any flag (or `SHAKEDOWN_YES=1`, or pipe it) and the picker is skipped, the run proceeds directly.

Either way, `./run` auto-selects the matching preset from your hardware (chip, memory, model, screen size), so it bins against the right thresholds and baseline with nothing to type. Pass `--target mbp-16-m5-max-64` (see `targets/`) to assert an expected SKU, or to override the match; if no preset matches, it auto-detects chassis class from `system_profiler` and skips the inventory asserts.

The orchestrator runs the automated phases (preflight → inventory → battery → race benchmark → SSD test → memory bandwidth → CPU variance → thermal load) end-to-end, asks for sudo once upfront (Phase 5 and the SSD page-cache drop need it), and writes a SCHEMA-compliant report to `Reports/local/` plus a sanitized PR-able copy to `Reports/submissions/`. Opt-in flags add heavier passes: `--noaccel` (a non-accelerated BLAKE2b variance pass), `--gpu` (a Metal GPU compute pass), and `--llama` (clones and builds llama.cpp for a combined CPU+GPU+memory AI load). `--store` bundles the thorough profile for verifying a new unit. Runtime ~20 min on Intel, ~27 min on Air, ~47 min on MacBook Pro.

Expand Down
9 changes: 5 additions & 4 deletions Verification/Store Day Checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,15 @@ Apple's QA catches the obvious build stuff (speakers, keys, screen). What it has
xcode-select --install
git clone https://github.com/ugglr/mac-shakedown ~/mac-shakedown
cd ~/mac-shakedown
./run --store
./run
```

- `xcode-select --install` is a one-time GUI dialog (~10 min); the rest is one paste.
- No SKU to type: `./run` auto-selects the matching preset from the machine's own hardware (it prints which one). Add `--target <name>` (see `targets/`) only if you want to assert an expected SKU or force a specific preset.
- `./run` asks a couple of questions. Pick **1) Verify a new Mac**. It detects the SKU itself (no flags to type) and builds the thorough run for you. (Same as `./run --store` if you'd rather type it.)
- Say **yes** to "refuse to score unless on AC and idle" so a bad-conditions run can't give you a false pass.
- Enter your login password when it asks for sudo (the thermal phase needs it).
- `--store` is the thorough profile: ~45 min of CPU variance (accelerated **and** non-accelerated), a 10-minute sustained thermal load, the memory triad, and the GPU pass. It takes over; walk around while it runs.
- Want the exact workload the M5 defect was reported on? Add `--llama` (it builds llama.cpp and runs an LLM load; extra time).
- The thorough profile is ~45 min: CPU variance (accelerated **and** non-accelerated), a 10-minute sustained thermal load, the memory triad, and the GPU pass. It takes over; walk around while it runs.
- Want the exact workload the M5 defect was reported on? Say **yes** to the AI/LLM load (it builds llama.cpp and runs an LLM load; extra time).

## Read the verdict

Expand Down
70 changes: 70 additions & 0 deletions Verification/scripts/run-shakedown.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ STORE=0
RUN_LLAMA=0
STRICT=0

ARGC=$#
while [[ $# -gt 0 ]]; do
case "$1" in
--target)
Expand Down Expand Up @@ -110,6 +111,75 @@ HELP
esac
done

# Interactive picker: with no flags on a terminal, ask a couple of questions and
# build the run instead of making the user remember flags. Any explicit flag, a
# non-interactive stdin (pipe / CI), or SHAKEDOWN_YES=1 skips it, so scripted and
# flag-driven use is unchanged. Sets the same variables the flags would.
interactive_picker() {
local sku choice ans built
sku=$(python3 - <<'PYEOF' 2>/dev/null
import json
import subprocess
try:
hw = json.loads(subprocess.check_output(
["system_profiler", "-json", "SPHardwareDataType"],
stderr=subprocess.DEVNULL))["SPHardwareDataType"][0]
chip = hw.get("chip_type") or hw.get("cpu_type") or "Mac"
except Exception:
chip = "this Mac"
try:
mem = round(int(subprocess.check_output(["sysctl", "-n", "hw.memsize"]).decode()) / (1024 ** 3))
print(f"{chip}, {mem} GB")
except Exception:
print(chip)
PYEOF
)
{
echo ""
echo "shakedown: detected ${sku:-this Mac}"
echo ""
echo " What are you doing?"
echo " 1) Verify a new Mac thorough, ~45 min"
echo " 2) Quick check ~20 min"
echo " 3) Custom / show flags"
} >&2
read -r -p " > " choice || true
case "$choice" in
1)
STORE=1
read -r -p " Refuse to score unless on AC and idle? [Y/n] " ans || true
if [[ ! "$ans" =~ ^[Nn] ]]; then STRICT=1; fi
read -r -p " Also run the AI/LLM load? (builds llama.cpp, needs network, adds time) [y/N] " ans || true
if [[ "$ans" =~ ^[Yy] ]]; then RUN_LLAMA=1; fi
;;
2)
: # default profile, no extra flags
;;
3)
echo " Run '$(basename "$0") --help' for all flags, then re-run with the ones you want." >&2
exit 0
;;
*)
echo " Unrecognized choice; running the quick check." >&2
;;
esac
built="./run"
if [[ "$STORE" -eq 1 ]]; then built+=" --store"; fi
if [[ "$STRICT" -eq 1 ]]; then built+=" --strict"; fi
if [[ "$RUN_LLAMA" -eq 1 ]]; then built+=" --llama"; fi
{
echo ""
echo " -> $built"
echo " Starting. Fans will get loud; it asks for your login password once."
} >&2
# The picker is the confirmation, so skip the second "Proceed?" prompt below.
SHAKEDOWN_YES=1
}

if [[ "$ARGC" -eq 0 && -t 0 && "${SHAKEDOWN_YES:-0}" != "1" ]]; then
interactive_picker
fi

export STRICT

# --store: thorough profile for verifying a new unit (especially M5). Run both CPU
Expand Down