shakedown: wave 5 — submission workflow, agent removal, methodology fixes#2
Merged
Conversation
Friends-can-try milestone: one-command runner that produces a predictable,
PR-able JSON report. No hosted backend — submissions land via PR, CI guards
against PII leakage, reviewer merges.
- run-shakedown.sh orchestrator wraps inventory + battery + cpu-variance +
thermal-load end-to-end. Prompts sudo upfront, aggregates into a
SCHEMA-compliant JSON, writes Reports/local/<name>.json (full, gitignored)
+ Reports/submissions/<name>.json (sanitized, committable). Filename:
{YYYY-MM-DD}-{preset}-{hash4}.json.
- .gitignore: Reports/local/ ignored, Reports/submissions/ tracked.
- CI submission audit (lint.yml): rejects PRs adding submission JSON that
contains _raw_* fields, raw_log_path, plaintext serial_number, malformed
filename, missing SCHEMA fields, or submission_safe != true. Skips
silently when no submission files exist.
- README "Submit a calibration report" section, CONTRIBUTING submission
flow, PR-template checkbox.
- Soft-failed the model_must_include inventory assert: Apple Silicon's
machine_model (e.g. Mac17,1) doesn't reliably carry screen-size info, so
a mismatch now warns instead of failing. chip_pattern and memory_gb
remain hard asserts.
Adapt the orchestrator to Macs that don't have a preset yet, or existing
units being self-tested rather than verified as new.
- Without --target: chassis class is auto-detected from system_profiler
(machine_name + chip_type → fanless / active-cooled-pro / desktop /
intel-laptop / intel-desktop). Overridable with CHASSIS_CLASS env var.
- Inventory phase: target asserts skipped, records actuals only.
- Battery phase: factory-fresh checks (cycle_count, ≥99% capacity)
skipped — those make sense for a new-purchase verification but false-fail
any used Mac. Real-degradation checks (<95% capacity, abnormal condition,
bad SMART) still apply.
- Filename slug becomes {chassis}-{memory}gb when no preset is given.
Adds a "this will run ~18 min of 100% CPU load, proceed? [y/N]" prompt before sudo is requested, so the user has a clear bail-out point before committing to the run. Defaults to abort — only proceeds on explicit y/Y. SHAKEDOWN_YES=1 bypasses the prompt for scripted / unattended runs. Reassurance text emphasizes that macOS handles thermal protection, so the risk is noisy fans, not a damaged machine.
UX polish based on the first real-hardware run. - ./run shim at the repo root: thin exec wrapper so `cd ~/mac-shakedown && ./run` works instead of the longer `./Verification/scripts/run-shakedown.sh`. - Flame ASCII banner on startup (red gradient, bold SHAKEDOWN block letters, dim subtitle). Suppressed when stderr isn't a TTY so CI / piped runs stay clean. - Sudo keep-alive: background `sudo -n true` every 60s while the orchestrator runs. Fixes the double-prompt on Intel where Phase 4 (~8 min) outlasts macOS's default 5-min sudo timestamp. - --no-sudo (alias --skip-thermal): skip Phase 5, the only sudo-requiring phase. Phase 4 variance still runs as the headline test. ~half the runtime with no password prompt at all.
The "max-min < 200 RPM" ramp check was warning on runs where fans were already at high RPM throughout — either because the chassis was heat-soaked from Phase 4, or because the user has fan curves manually pinned (TG Pro, Macs Fan Control, repair shop tooling). Both cases mean cooling is engaged; the lack of *delta* doesn't indicate a defect. If min(fan_avgs) > 3000 RPM, the ramp check is skipped and a data-quality note records that fans were pinned high. Verdict isn't affected.
The Sonoma format differs substantially from older Intel macOS:
Sonoma+: CPU Average frequency as fraction of nominal: 72.40% (1882.29 Mhz)
Intel energy model derived package power (CPUs+GT+SA): 6.14W
Legacy: CPU 0 frequency: 3192 MHz
Package Power: 12.5 W
The old regex expected the legacy format only, so on Sonoma p_freqs ended up
empty, data_quality dropped to "few_samples", and the thermal verdict
couldn't compute steady-state or cliff metrics. Carl's earlier Intel run
hit exactly this — "no P-cluster frequency data" in the report.
Added Sonoma-format regexes alongside the legacy ones; both run on every
line, whichever matches wins. Also case-insensitive on "Mhz" since Sonoma
uses lowercase 'h'.
Made the data-quality message generic ("no CPU frequency data" instead of
"no P-cluster") since the script handles both architectures.
Pre-phase build-up animation using Unicode block-flame characters in red: ▁ → ▂ → ▃ → ▄ ▁ → ▅ ▂ → … → █ ▇ ▅ ▃, then the phase label in bold red prefixed with ██. ~700 ms total — enough to feel like a transition, short enough not to pad the run. Falls back to a plain echo when stderr isn't a TTY so CI / piped logs stay clean. No emojis in the script.
Background ticker that prints a varied spark line (yellow/orange/red gradient, sparse ASCII glyphs: * · ✦) every 10–17 seconds during Phase 4 and Phase 5. Phase 4 warmup is 180s with no sub-script output, and Phase 5 is a full 10-min sustained load with no progress prints — the heartbeat keeps the screen alive without garbling the sub-script's own output (no \r overwrites; each spark gets its own line). Killed cleanly via stop_heartbeat() after each phase; the EXIT trap also catches any leak on abort or unexpected exit. Suppressed when stderr isn't a TTY.
The orchestrator runs end-to-end without an agent — sudo is interactive and the y/N prompt blocks, so agents can't actually drive ./run anyway. - Tagline drops "Claude-Code-driven" and "Apple Silicon" (we run on Intel too). - Quick start leads with ./run, removes the Claude Code install step. - Specifying your target switches the example commands from claude "…" to ./run. - "Optional: agent assistance" section removed entirely. - "Manual usage (without Claude Code)" renamed to "Running individual scripts (advanced)" and reframed as the rerun-one-phase escape hatch, not the fallback for users without an agent. - "Submit a calibration report" trimmed — Quick start now covers the orchestrator + filename + dual-copy split, so the section just describes the PR flow. - Repo layout adds ./run, run-shakedown.sh, mbp-16-intel-2019.json, Reports/local/, and Reports/submissions/. AGENTS.md and CLAUDE.md still exist (cross-tool convention, harmless) but are no longer promoted.
You (the agent that built this) can't actually run ./run end-to-end — sudo is interactive and the y/N prompt blocks. Same goes for every other agent. The "Claude-Code-driven harness" framing was historical; the script does the actual work, the runbook covers the manual checks, and there's no load-bearing role left for an LLM. Deleted: - AGENTS.md - CLAUDE.md Rewrote agent references to point at ./run, the orchestrator, or "step through these checks": - Verification/Runbook.md (intro framing, Phase 6/7/8 walkthroughs) - targets/README.md, CONTRIBUTING.md (claude "..." → ./run) - Reports/SCHEMA.md, SECURITY.md (agent strips → orchestrator strips) - examples/m5-2026/README.md, Shakedown Brain.md - targets/mbp-16-intel-2019.json (_note field) - .github/ISSUE_TEMPLATE/bug-report.yml (component dropdown) - Verification/scripts/cpu-variance.sh (comments) - examples/sample-report-illustrative/*.md (footer) - README.md repo-layout tree (drop the deleted entries) CHANGELOG entries about AGENTS.md from past waves are left in place — those describe history, not current state.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
./runorchestrator wraps the four script-driven phases, prompts sudo once upfront, aggregates into a SCHEMA-compliant JSON, and writes both a full local copy and a sanitized PR-able submission copy. CI submission-audit job catches PII leakage (_raw_*, plaintext serials, off-pattern filenames)../runshim at the repo root, ASCII flame banner on startup, ignite animation before each phase, heartbeat sparks during the silent stretches, sudo keep-alive (fixes double-prompt mid-run),--no-sudomode skips Phase 5 entirely, confirmation prompt before kicking off sustained load.--targetis now optional with chassis-class auto-detect fromsystem_profiler; battery cycle-count check softens when no preset is given (existing-Mac use case); fan-ramp warn skips when fans are already pinned high (manual fan control or heat-soaked chassis); Intelpowermetricsparsing now handles Sonoma's new format (CPU Average frequency as fraction of nominal: X% (Y Mhz)+Intel energy model derived package power (CPUs+GT+SA): Z.W).AGENTS.mdandCLAUDE.md; rewrote every agent reference across README, Runbook, CONTRIBUTING, SECURITY, SCHEMA, targets/README, examples/m5-2026/README, Shakedown Brain, bug-report template, and cpu-variance.sh comments to point at./run, the orchestrator, or "step through these checks".Test plan
shellcheck -x -e SC1091 Verification/scripts/*.sh runcleanast.parseclean