Use structured JSON output for local model classification

[jdcodes1]

Summary

• Use WebLLM's response_format: { type: 'json_object' } to force local models to output valid JSON instead of freeform text
• Update parseLocalModelResponse() to try JSON parsing first, falling back to the existing regex-based parsing for backward compatibility
• Simplify LOCAL_SYSTEM_PROMPT from 21 lines to 4 — format constraints are now enforced by the engine, not by prompt instructions
• Pass responseFormat as an optional parameter to generate() so non-classification callers (e.g. suggestAnnoyingReasons) still get freeform text

Problem: Local model responses are parsed with regex looking for "Matches X" or "No match" in freeform text. When the model phrases things differently, the regex misses and the post silently gets classified as "no match" — a false negative.

Fix: Constrained decoding guarantees valid JSON output. The model can only produce {"reasoning": "...", "match": "..."}. Zero parsing ambiguity. The vendored WebLLM (0.2.82-custom) already supports json_object response format — it just wasn't being used.

Changes

File	What
src/background/local-model.ts	Add CLASSIFICATION_RESPONSE_FORMAT constant, make responseFormat optional on generate(), pass it from callLocalInference, guard against "null" string match
src/shared/prompts.ts	Simplify LOCAL_SYSTEM_PROMPT (21 lines → 4 lines)
tests/background/local-model.test.ts	6 new tests: JSON match, no-match, empty string, "null" string, regex fallback, malformed JSON fallback

Test plan

• 6 new tests for JSON parsing + edge cases
• All 205 tests pass (existing regex-based tests still pass via fallback path)
• response_format only applied to classification calls, not suggestAnnoyingReasons
• Backward compatible — non-JSON model output still parsed via regex fallback

[imperion402]

what is this backward compatibility you speak of?

[imperion402]

what evals did you perform to validate this achieves similar metrics? What's the F1 score, accuracy, precision, etc...?

[gnguralnick]

this is the key point - we experimented previously with structured output like this and found that this simple of a prompt actually leads to far worse classification performance, which is why local models now receive approximately the same prompt as API ones. I'd love to be able to use a much shorter prompt if it would actually work as well since it would certainly process much faster.