Add hybrid analyzer combining heuristic and LLM judgment

README.md

@@ -35,9 +35,10 @@ Gateways like LiteLLM or Bifrost unify provider APIs and load-balance — but th
 | Analyzer | Cost | How it works |
 |---|---|---|
 | `heuristic` (default) | free, <1ms | Deterministic signal scoring: code blocks, requirement density, reasoning cues, question fan-out, context depth. Fully explainable — per-signal scores are returned with every decision. |
-| `llm` (optional) | free, ~50–200ms | Uses a small local model via Ollama (e.g. `qwen2.5:0.5b`) to classify category and complexity. Smarter on ambiguous prompts, still 100% local and private. Falls back to `heuristic` if Ollama is unavailable. |
+| `llm` (optional) | free, ~50–200ms | Uses a small local model via Ollama (e.g. `qwen2.5:0.5b`) to classify category and complexity with few-shot prompting. Smarter on ambiguous prompts, still 100% local and private. Falls back to `heuristic` if Ollama is unavailable. |
+| `hybrid` (optional) | free, <1ms + LLM latency | Blends heuristic signals with LLM judgment: heuristic always runs (cheap, never fails), LLM refines complexity via weighted blend and overrides category only when heuristic had low confidence. Combines explainability with LLM's strength on ambiguous prompts. |
 
-A third mode — analyzers trained on real routing outcomes — ships with [Route42 Pro](https://route42.app).
+A fourth mode — analyzers trained on real routing outcomes — ships with [Route42 Pro](https://route42.app).
 
 ### Providers & models
 - **Cloud providers:** OpenAI, Anthropic, Google Gemini, Mistral, Groq, DeepSeek, Alibaba, Moonshot, NVIDIA, OpenRouter.
@@ -146,10 +147,11 @@ Every response includes the selected model and the analyzer's signal breakdown,
 
 ```yaml
 analyzer:
-  mode: heuristic          # heuristic | llm
+  mode: heuristic          # heuristic | llm | hybrid
   llm:
-    model: qwen2.5:0.5b    # any small Ollama model
+    model: qwen2.5:0.5b    # any small Ollama model (required for llm and hybrid)
     timeout_ms: 1500       # falls back to heuristic on timeout
+    hybrid_weight: 0.5     # 0..1, blend weight for hybrid mode (default 0.5)
 ```
 
 ## Integration

docs/analyzer.md

@@ -19,7 +19,7 @@ type AnalysisResult struct {
     Complexity float64            // 0..1
     Category   string             // chat | code | math | analysis | general
     Signals    map[string]float64 // per-signal contributions
-    Analyzer   string             // "heuristic" | "llm"
+    Analyzer   string             // "heuristic" | "llm" | "hybrid"
 }
 
 type PromptAnalyzer interface {
@@ -32,14 +32,16 @@ type PromptAnalyzer interface {
 | Analyzer | Cost | How it works |
 |---|---|---|
 | `heuristic` (default) | free, <1ms | Deterministic signal scoring: code blocks, requirement density, reasoning cues, question fan-out, context depth. Fully explainable — per-signal scores are returned with every decision. |
-| `llm` (optional) | free, ~50–200ms | Uses a small local model via Ollama (e.g. `qwen2.5:0.5b`) to classify category and complexity. Smarter on ambiguous prompts, still 100% local and private. Falls back to `heuristic` if Ollama is unavailable. |
+| `llm` (optional) | free, ~50–200ms | Uses a small local model via Ollama (e.g. `qwen2.5:0.5b`) to classify category and complexity. Few-shot prompting with worked examples spreads scores across the full complexity range. Smarter on ambiguous prompts, still 100% local and private. Falls back to `heuristic` if Ollama is unavailable. |
+| `hybrid` (optional) | free, <1ms + LLM latency | Blends heuristic signals with LLM judgment: heuristic runs unconditionally (cheap, never fails), LLM refines complexity via weighted blend and overrides category only when heuristic had low confidence. Combines explainability of heuristic with LLM's strength on ambiguous prompts. |
 
 ```yaml
 analyzer:
-  mode: heuristic          # heuristic | llm
+  mode: heuristic          # heuristic | llm | hybrid
   llm:
-    model: qwen2.5:0.5b    # any small Ollama model
+    model: qwen2.5:0.5b    # any small Ollama model (required for llm and hybrid modes)
     timeout_ms: 1500       # falls back to heuristic on timeout
+    hybrid_weight: 0.5     # 0..1, weight given to LLM score in hybrid mode (default 0.5)
 ```
 
 You can inspect an analyzer's output without running the gateway:
@@ -112,19 +114,42 @@ dominates the score.
 Uses a small local model via Ollama. 100% local, $0, on-brand.
 
 - Default model: `qwen2.5:0.5b` (configurable; anything Ollama-served works).
-- Prompt (single-shot, JSON-forced):
-  ```
-  Classify this user request. Respond with ONLY JSON:
-  {"category":"chat|code|math|analysis|general","complexity":0.0-1.0}
-  complexity: 0=trivial one-liner, 0.5=typical task, 1=multi-constraint expert task.
-  Request: <last user message, truncated to 1500 chars>
-  ```
+- Prompt (few-shot, JSON-forced): 6 worked examples span all categories
+  and non-round complexity values (0.05, 0.12, 0.22, 0.38, 0.64, 0.93) to
+  calibrate the LLM's scoring across the full range and avoid the
+  round-number clustering problem of zero-shot prompts.
 - Guardrails: 1500ms timeout, strict JSON parse, clamp complexity to
   `[0,1]`, category whitelist. **Any failure → fall back to the
   heuristic analyzer** (routing must never be blocked by analysis errors).
 - Cache: LRU on `hash(last user message)` to avoid re-analyzing
   retries/regenerations.
 
+## Hybrid analyzer (optional)
+
+Combines the explainability of heuristic signals with LLM judgment on
+ambiguous prompts. The heuristic always runs first (cheap, never fails),
+and the LLM refines the result only when available.
+
+- **Complexity**: weighted blend `complexity = w * llm_score + (1-w) * heuristic_score`,
+  where `w` (default 0.5) is `analyzer.llm.hybrid_weight`.
+- **Category**: uses heuristic's category if it was confident (detected
+  anything other than `general`), otherwise uses LLM's category. This
+  preserves determinism for the common case and only defers to LLM on
+  genuinely ambiguous prompts.
+- **Fallback**: on any LLM error (timeout, parse, connection), returns
+  the heuristic result unchanged. Hybrid mode is a strict superset of
+  heuristic mode's reliability.
+- **Signals**: heuristic's per-signal breakdown is always present, with
+  two extra entries for auditability:
+  - `llm.complexity`: the raw LLM score before blending
+  - (LLM category not recorded in signals, but visible in logs if needed)
+- **Cost**: heuristic's <1ms plus LLM latency (50–200ms) when available.
+
+Hybrid mode directly fixes the two key weaknesses of pure LLM mode:
+- Round-number clustering (finding #3 in testing) → eliminated by blending
+  with heuristic's continuous scores
+- Loss of explainability (finding #4) → heuristic signals always present
+
 > **Route42 Pro** loads an ML-trained analyzer (`mode: ml`) when the
 > commercial model bundle is installed. It slots into the same interface,
 > so routing behavior is identical — Pro just has a more accurate

docs/config.md

@@ -20,10 +20,11 @@ server:
   api_token: ""             # optional bearer token for /api/* (empty = no auth)
 
 analyzer:
-  mode: heuristic           # heuristic | llm
+  mode: heuristic           # heuristic | llm | hybrid
   llm:
-    model: qwen2.5:0.5b     # any small Ollama model (required when mode: llm)
+    model: qwen2.5:0.5b     # any small Ollama model (required when mode: llm or hybrid)
     timeout_ms: 1500        # >0; falls back to heuristic on timeout
+    hybrid_weight: 0.5      # 0..1; weight for LLM score in hybrid mode (default 0.5)
 
 ollama:
   base_url: http://localhost:11434   # must not be empty
@@ -59,9 +60,10 @@ prefs:                      # initial routing preferences (first-run seed)
 
 | Field | Default | Validation | Notes |
 |---|---|---|---|
-| `mode` | `heuristic` | `heuristic` \| `llm` | Selects the prompt analyzer. See [`analyzer.md`](analyzer.md). |
-| `llm.model` | `qwen2.5:0.5b` | required when `mode: llm` | Any Ollama-served model. |
+| `mode` | `heuristic` | `heuristic` \| `llm` \| `hybrid` | Selects the prompt analyzer. See [`analyzer.md`](analyzer.md). |
+| `llm.model` | `qwen2.5:0.5b` | required when `mode: llm` or `mode: hybrid` | Any Ollama-served model. |
 | `llm.timeout_ms` | `1500` | >0 | On timeout the request falls back to the heuristic analyzer. |
+| `llm.hybrid_weight` | `0.5` | 0..1 | Weight given to LLM score in hybrid mode. `0` = pure heuristic, `1` = pure LLM. |
 
 ### `ollama`
 
@@ -117,6 +119,7 @@ All `ROUTE42_*` variables override file values. Provider keys use
 | `ROUTE42_ANALYZER_MODE` | `analyzer.mode` |
 | `ROUTE42_ANALYZER_LLM_MODEL` | `analyzer.llm.model` |
 | `ROUTE42_ANALYZER_LLM_TIMEOUT_MS` | `analyzer.llm.timeout_ms` |
+| `ROUTE42_ANALYZER_LLM_HYBRID_WEIGHT` | `analyzer.llm.hybrid_weight` |
 | `ROUTE42_OLLAMA_BASE_URL` | `ollama.base_url` |
 | `ROUTE42_DB_PATH` | `db.path` |
 | `ROUTE42_<PROVIDER>_API_KEY` | `providers.<provider>.api_key` |

internal/analyzer/factory.go

@@ -17,6 +17,11 @@ func New(cfg *config.Config) (PromptAnalyzer, error) {
 	case config.ModeLLM:
 		timeout := time.Duration(cfg.Analyzer.LLM.TimeoutMs) * time.Millisecond
 		return NewLLM(cfg.Ollama.BaseURL, cfg.Analyzer.LLM.Model, timeout, NewHeuristic()), nil
+	case config.ModeHybrid:
+		timeout := time.Duration(cfg.Analyzer.LLM.TimeoutMs) * time.Millisecond
+		h := NewHeuristic()
+		llm := NewLLM(cfg.Ollama.BaseURL, cfg.Analyzer.LLM.Model, timeout, h)
+		return NewHybrid(h, llm, cfg.Analyzer.LLM.HybridWeight), nil
 	default:
 		return nil, fmt.Errorf("analyzer mode %q is not supported", cfg.Analyzer.Mode)
 	}

internal/analyzer/hybrid.go

@@ -0,0 +1,59 @@
+package analyzer
+
+import "context"
+
+// Classifier is the interface for classifying prompts. LLMAnalyzer implements
+// this via its Classify method.
+type Classifier interface {
+	Classify(ctx context.Context, prompt string) (AnalysisResult, error)
+}
+
+// HybridAnalyzer blends the deterministic HeuristicAnalyzer with an LLM
+// analyzer's judgment: heuristic runs unconditionally (cheap, never fails),
+// and LLM output — when available — refines complexity via a weighted
+// blend and overrides category only when the heuristic had low confidence
+// (fell back to CategoryGeneral). On any LLM failure, the heuristic result
+// is returned unchanged, so hybrid mode is a strict superset of heuristic
+// mode's reliability guarantees.
+type HybridAnalyzer struct {
+	heuristic *HeuristicAnalyzer
+	llm       Classifier
+	weight    float64 // 0..1, weight given to the LLM's complexity score
+}
+
+// NewHybrid returns a hybrid analyzer that blends heuristic signals with
+// LLM judgment. weight should be in [0,1]; weights <= 0 default to 0.5.
+func NewHybrid(h *HeuristicAnalyzer, c Classifier, weight float64) *HybridAnalyzer {
+	if weight <= 0 {
+		weight = 0.5
+	}
+	return &HybridAnalyzer{heuristic: h, llm: c, weight: weight}
+}
+
+// Analyze implements PromptAnalyzer. It always runs the heuristic analyzer
+// (cheap, never fails) and attempts to refine it with the LLM analyzer's
+// judgment. On any LLM error, the heuristic result is returned unchanged.
+func (a *HybridAnalyzer) Analyze(ctx context.Context, messages []Message) (AnalysisResult, error) {
+	base, _ := a.heuristic.Analyze(ctx, messages) // heuristic never errors
+
+	prompt := lastUserMessage(messages)
+	llmRes, err := a.llm.Classify(ctx, prompt)
+	if err != nil {
+		// LLM failed; return heuristic result but tag as hybrid for observability.
+		base.Analyzer = NameHybrid
+		return base, nil
+	}
+
+	// Blend complexity: weight given to LLM, (1-weight) to heuristic.
+	base.Complexity = clamp01(a.weight*llmRes.Complexity + (1-a.weight)*base.Complexity)
+
+	// Override category only if heuristic had low confidence (fell back to general).
+	if base.Category == CategoryGeneral && llmRes.Category != "" {
+		base.Category = llmRes.Category
+	}
+
+	// Record the raw LLM complexity for auditability.
+	base.Signals["llm.complexity"] = llmRes.Complexity
+	base.Analyzer = NameHybrid
+	return base, nil
+}