Windows: hook capture 500s on non-Latin-1 payloads, plus two related bugs

[natebased14]

watchmen — three bugs found on Windows 11 (v0.6.7)

Found while onboarding a fresh install on Windows 11 / Python 3.12 / PowerShell 5.1.
Two are one-line fixes; the third is why the first two were invisible.

Repo: https://github.com/firstbatchxyz/watchmen

---

1. Hook capture returns HTTP 500 on any non-Latin-1 payload (Windows)

File: src/watchmen/hook_server.py:77

with JSONL_PATH.open("a") as f:
    f.write(line + "\n")

Path.open() with no encoding uses locale.getpreferredencoding(), which on
Windows is the ANSI codepage (cp1252 on a US-English install), not UTF-8.

Lines 65 and 75 deliberately preserve non-ASCII:

payload_str = json.dumps(payload, ensure_ascii=False)
line = json.dumps({"received_at": received_at, **payload}, ensure_ascii=False)

So any hook payload containing a character outside cp1252 raises
UnicodeEncodeError and the endpoint returns 500.

File "src/watchmen/hook_server.py", line 78, in receive_hook
    f.write(line + "\n")
File "...\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input, self.errors, encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character 'Γ' in position 775

Impact: on Windows, hook capture fails for the majority of real events —
Claude Code payloads routinely contain box-drawing characters, arrows, and
non-Latin text. The SQLite insert on line 68 succeeds, so the DB and the JSONL
silently diverge. watchmen_observe.ps1 swallows the error and exits 0 by
design, so nothing surfaces to the user; the only trace is a "500" line in
~/.watchmen/logs/hooks.log.

Fix:

with JSONL_PATH.open("a", encoding="utf-8") as f:

Repro: Windows, watchmen hooks install, start watchmen.hook_server, then
run any Claude Code session whose tool output contains a non-cp1252 character.

---

2. _publish_watchmen_state never runs (bare import state)

File: src/watchmen/curate.py:992

import state as _state  # local import to avoid pulling state into module imports

This is a top-level absolute import of a module that only exists as
watchmen.state. It resolves only when the process cwd happens to be
src/watchmen/, which puts that directory on sys.path[0].

Normal invocations fail:

[4/4] writing _index.md...
      _publish_watchmen_state failed (non-fatal): ModuleNotFoundError: No module named 'state'

It is masked in two ways: the call site catches broadly and labels it
"non-fatal", and onboard.py runs curate subprocesses with
cwd=str(ROOT) where ROOT = Path(__file__).parent — i.e. src/watchmen/ —
so the import happens to succeed under the onboarding wizard and fails
everywhere else.

Impact: ~/.watchmen/state/<project>.json and ~/.watchmen/projects.json
are never written outside the wizard path. Per the docstring these back the
plugin statusLine indicator and the /watchmen:brief skill body, so both
silently render stale or empty.

Fix:

from watchmen import state as _state

---

3. Onboarding discards subprocess stderr, hiding every failure

File: src/watchmen/onboard.py:460 and :478

result["curator_last"] = out[-1] if out else ((r.stderr or "").strip()[:120])

stderr is only consulted when stdout is empty. Both watchmen.analyze and
watchmen.curate print a provider banner to stdout at startup
(agent.provider_banner()), so stdout is never empty and stderr is
unconditionally thrown away.

When a curator subprocess fails, the user sees the startup banner presented as
though it were the error:

✓ [3/38] exhibit-b-workflow analyst in 130s
  [3/38] exhibit-b-workflow: curator started
✗ [3/38] exhibit-b-workflow curator in 0s
     provider=chatgpt · ChatGPT subscription · model=gpt-5.4-mini · endpoint=chatgpt.com/backend-api/codex/responses

Every one of 38 projects failed this way with no diagnostic whatsoever. The
banner is misleading as a failure detail — it names the provider, which invites
misdiagnosis of a provider or credential problem.

Suggested fix: show stderr on failure regardless of stdout, e.g.

if result["curator_ok"]:
    result["curator_last"] = out[-1] if out else ""
else:
    err = (r.stderr or "").strip()
    result["curator_last"] = err[-400:] if err else (out[-1] if out else "")

Truncating from the tail keeps the exception rather than the traceback header.

---

Note on bulk onboarding concurrency

Unconfirmed, but worth a look: with the chatgpt subscription provider,
every curator across a 38-project run failed at 0s while the same projects
curated successfully when run individually or in pairs. The docstring at
onboard.py:491 notes concurrency was sized for OpenRouter/deepseek rate
limits (3 projects x 4 skill workers x ~2 critics = ~24 calls in flight).
That budget may be well over what a ChatGPT subscription tolerates, and a 429
would produce exactly the observed instant failure. Fix #3 would make this
diagnosable.