feat(ops): observability — Workers Logs, health-check alert cron, weekly ops digest (0.0.8-rc.8)

[unforced]

What / why

The cloud must not fail silently. Tonight's wrangler tail probe captured zero events — the account had no persistent Workers Logs enabled, so the structured failure lines we already emit (event=magic_link_send_failed, identity worker) went into the void, and nothing monitored health at all. This PR adds the three missing legs:

1. Workers Logs on all four workers

[observability] enabled = true + head_sampling_rate = 1 in both wrangler.tomls, at the top level (production) and [env.staging.observability] — observability is NOT inherited by named envs (per current wrangler docs), so it's stated explicitly in all four worker configs. Invocation logs + console.* are now persistently queryable (dashboard, logs API, wrangler tail).

2. Health-check + alert cron (identity worker, src/ops.ts)

[triggers] crons = ["*/10 * * * *", "0 14 * * 1"] on the identity worker (no third worker); routeCron(controller.cron) dispatches, with unknown patterns degrading to the health check (never a silent no-op).

Every 10 minutes:

• identity liveness = cheap D1 self-check (SELECT 1) — a self-HTTP-fetch would be near-tautological (the worker obviously runs if the cron fired) and Workers can't fetch their own routes anyway;
• vault liveness = GET <VAULT_ORIGIN>/health (10s timeout) — the router-level endpoint that never wakes a DO (already existed; identity gained a new public GET /health for monitors + the smoke scripts).

Alerting: failures email OPERATOR_ALERT_EMAIL (ag@unforced.org) via the existing send_email binding. Dedupe: 1 hour per check, persisted in the new D1 ops_alerts table; the mark is written only after a successful send (a failed email retries next tick), and the dedupe read fails open — if D1 itself is down we alert every 10 minutes rather than suppress the one email that matters. Every failure also logs a structured event=health_check_failed line independent of email. Staging runs the same crons, but has no send_email binding by design → the devlog sender writes the full alert/digest email into the worker log — deterministic for tests, zero real email.

3. Weekly ops digest (same handler, Mon 14:00 UTC ≈ Monday morning MT)

Counts-only from D1: users total/new-7d, vaults total/new-7d, magic links sent/failed-7d. The send counts come from the new magic_link_events per-day counter table, bumped in the auth handler on every send outcome (the failure path that previously only logged now also counts). PII-free by design: rows are (day, event, count) — never an address, domain, or IP; the digest email is asserted in tests to contain no addresses or vault names. A broken counter can never break sign-in (bumpMagicLinkEvent never throws).

Migration 0005_ops.sql (ops_alerts + magic_link_events). EmailSender grows a generic sendOps (binding + devlog impls).

Gates (literal)

• workers/identity: tsc --noEmit clean; vitest 111 passed (was 98 — +12 new ops.test.ts: cron routing/dispatch, dedupe window incl. failed-send retry, digest query shapes + PII-free assertion, /health shape, real worker.scheduled wiring via createScheduledController + fetchMock; +1 counter test in auth.test.ts)
• workers/vault: tsc --noEmit clean; vitest 99 passed | 1 todo (was 98+1 — +1 router /health shape test via SELF)
• root: bun test src 123 pass, 0 fail (362 expect() calls)
• CI (run 28636219414, head 3f0e2db): all three jobs pass — identity worker (typecheck + vitest/workerd), vault worker (typecheck + vitest/workerd), control plane (root bun test).

Live verification

• Staging (deploy-staging.sh): both crons registered (schedule: */10 * * * *, schedule: 0 14 * * 1 in deploy output); GET /health → 200 {"status":"ok","service":"identity"} (identity) and {"status":"ok"} (vault); full smoke-staging.ts: 38 pass, 0 fail.

• Production (deploy-prod.sh): custom domains retained, crons registered, migration applied; smoke-prod.ts: 13 pass, 0 fail (now includes the identity /health check).

• Tail proof (the blind-spot fix): wrangler tail parachute-identity now captures events —

  outcome=ok version=2c5d49e9 GET https://cloud.parachute.computer/health?probe=1 logs=0
  outcome=ok version=2c5d49e9 GET https://cloud.parachute.computer/health?probe=2 logs=0
  outcome=ok version=2c5d49e9 GET https://cloud.parachute.computer/health?probe=3 logs=0
  

  (version 2c5d49e9 = this deploy). Tonight's identical probe captured nothing.

• Cron firing: a deployed cron can't be forced (wrangler dev --test-scheduled + curl /__scheduled?cron=... is the local path; vitest drives the real worker.scheduled export via createScheduledController). A passive production tail across a */10 boundary is attached in a PR comment if captured.

🤖 Generated with Claude Code

https://claude.ai/code/session_01XLZtmuSs1RirWGMGyCB1QB

[unforced]

Live cron proof (production): a passive wrangler tail parachute-identity across the 03:20 UTC boundary caught the deployed cron actually firing:

SCHEDULED cron='*/10 * * * *' scheduledTime=2026-07-03T03:20:25+00:00 outcome=ok version=2c5d49e9

outcome=ok with zero logs/exceptions = the health check ran and found both legs healthy (no event=health_check_failed, no alert email) — exactly the quiet-when-green behavior designed. Leg 2 is verified end-to-end in production, not just in vitest.

Review folds in 3f0e2db (reviewer verdict was LGTM/no blockers): stale VAULT_ORIGIN docstring updated (the health check now depends on it in every env), the observability-inheritance comment softened (docs guidance varies; explicit-per-env either way), and an exact dedupe-boundary assertion added (now - last == ALERT_DEDUPE_MS rings; one tick inside the fresh window stays quiet). Identity suite still 111 passed, typecheck clean.