fix: session isolation, Gemini tool-call correlation, compression failure hardening

.changeset/great-wolves-shake.md

@@ -0,0 +1,11 @@
+---
+"@context-chef/ai-sdk-middleware": minor
+---
+
+Session isolation, lossless-er tool output flattening, and tighter peer ranges.
+
+- **Session isolation**: compression state (fed token usage, compression suppression, the failure circuit breaker) is now tracked per session instead of per middleware instance. Pass `providerOptions: { contextChef: { sessionId } }` on each call; calls without a `sessionId` share one default session (prior behavior). New `maxSessions` option caps concurrently tracked sessions (default 256, LRU-evicted). Previously a middleware created at module scope leaked Janitor state across every conversation it served.
+- `stringifyToolOutput` no longer silently drops non-text content parts: file/media parts flatten to a `[tool result file: <mediaType>]` placeholder so compression and truncation see that they exist. The truncator's duplicate text extraction now delegates to the same implementation.
+- Tighten peerDependency ranges from `>=4` / `>=7` to `^4.0.0` / `^7.0.0`. The middleware depends on `LanguageModelV4*` type shapes from `@ai-sdk/provider` v4 — an unbounded range would let a future major install silently and fail at runtime.
+- Compression flattening now reuses `flattenForCompression` from `@context-chef/core` (behavior unchanged).
+- An invalid `sessionId` (empty string, non-string, or a malformed `contextChef` namespace) now warns once and routes to the default session instead of failing silently. The Janitor missing-compression-config nag fires once per middleware instead of once per session.

.changeset/lucky-pianos-guess.md

@@ -0,0 +1,7 @@
+---
+"@context-chef/tanstack-ai": minor
+---
+
+Conversation isolation: compression state (fed token usage, compression suppression, the failure circuit breaker) is now tracked per `ctx.conversationId` instead of per middleware instance. Calls without a `conversationId` share one default slot (prior behavior). New `maxSessions` option caps concurrently tracked conversations (default 256, LRU-evicted). Previously a middleware instance reused across chat() calls leaked Janitor state across every conversation it served. Compression flattening now reuses `flattenForCompression` from `@context-chef/core` (behavior unchanged).
+
+An empty-string `conversationId` now warns once and routes to the default slot (previously it was silently treated as a distinct conversation, diverging from the AI SDK middleware's semantics). The Janitor missing-compression-config nag fires once per middleware instead of once per conversation.

.changeset/tidy-moons-search.md

@@ -0,0 +1,14 @@
+---
+"@context-chef/core": minor
+---
+
+Fix Gemini tool call correlation, harden Janitor compression failure paths, and add shared middleware infrastructure.
+
+- `fromGemini` now synthesizes tool call IDs with per-name counters spanning the whole conversation and correlates each `functionResponse` to the oldest unconsumed call of the same name (FIFO). Previously the correlation state was scoped to a single message, so responses arriving in a later content entry always fell back to the `-0` suffix — repeat calls of the same tool produced duplicate IDs and misattributed results.
+- When the compression model throws, the raw error is no longer appended to the LLM-bound fallback summary. It now goes to the configured `logger` (default `console`) instead, keeping stack traces out of model context.
+- A throwing/rejecting `onCompress` hook no longer aborts `compile()`. It is caught and logged via `logger`; the compression result is kept. Hooks that follow the documented "must not throw" contract see no behavior change.
+- `onBeforeCompress` (and the deprecated `onBudgetExceeded`) now degrade the same way: a throwing hook is caught, logged via `logger`, and treated as if it returned `null` — default compression proceeds. Both hooks now share one failure stance: a broken hook never fails `compile()`.
+- The text-placeholder vocabulary is centralized in `Prompts`: new `getAttachmentPlaceholder` (previously janitor-internal), `getToolResultFilePlaceholder`, and `getToolResultPartPlaceholder`. Formats are unchanged — this gives each convention a single source instead of scattered string literals.
+- New export `SessionPool<T>` — a keyed instance pool with LRU eviction, used by the middleware packages to hold one Janitor per conversation. `maxSize` is validated (throws `RangeError` unless a positive integer — a non-positive cap would silently evict every entry on insert, disabling pooling). Companion exports `DEFAULT_SESSION_KEY`, `normalizeSessionKey`, and `dedupeConstructionWarnings` back the middlewares' shared session-key normalization and construction-nag dedupe.
+- New export `flattenForCompression(messages)` — the canonical role-flattening implementation required by the `summarizeHistory` compress-callback contract (tool results → user messages, tool calls → assistant text).
+- `OpenAIAdapter.compile` replaces its per-message `JSON.parse(JSON.stringify(...))` deep clone with a direct clone that skips `undefined` properties — same output, no string serialization detour (noticeable with base64 attachments). The clone honors `toJSON`, so a `Date` still lands as its ISO string.

packages/ai-sdk-middleware/README.md

@@ -126,6 +126,26 @@ Tools not listed fall back to the top-level defaults. The lookup key is `tool-re
 
 The middleware automatically extracts token usage from `generateText` and `streamText` responses and feeds it back to the compression engine. No manual `reportTokenUsage()` calls needed.
 
+### Session Isolation (Multi-User Servers)
+
+A wrapped model is usually created once at module scope and reused across requests. Compression state (fed token usage, compression suppression, the failure circuit breaker) is tracked per **session** — pass a `sessionId` through `providerOptions` on each call so concurrent conversations never share state:
+
+```typescript
+const model = withContextChef(openai('gpt-4o'), {
+  contextWindow: 128_000,
+  compress: { model: openai('gpt-4o-mini') },
+});
+
+// Per request:
+const result = await generateText({
+  model,
+  messages,
+  providerOptions: { contextChef: { sessionId: conversationId } },
+});
+```
+
+Calls without a `sessionId` share one default session — fine for single-conversation processes (a CLI, a notebook, one agent loop), wrong for multi-user servers: one user's over-budget conversation would trigger or suppress compression for everyone else. Up to `maxSessions` sessions are tracked concurrently (default 256, LRU-evicted); an evicted session is transparently recreated on next access, losing only its fed token usage.
+
 ### Compact (Mechanical Pruning)
 
 Zero-LLM-cost message pruning via AI SDK's `pruneMessages` — removes reasoning, tool calls, and empty messages:

packages/ai-sdk-middleware/package.json

@@ -47,8 +47,8 @@
     "@context-chef/core": "workspace:*"
   },
   "peerDependencies": {
-    "@ai-sdk/provider": ">=4",
-    "ai": ">=7"
+    "@ai-sdk/provider": "^4.0.0",
+    "ai": "^7.0.0"
   },
   "devDependencies": {
     "@ai-sdk/provider": "^4.0.0",

packages/ai-sdk-middleware/src/adapter.ts

@@ -10,6 +10,7 @@ import {
   type Attachment,
   ensureValidHistory,
   type Message,
+  Prompts,
   type ToolCall,
 } from '@context-chef/core';
 
@@ -314,7 +315,16 @@ export function stringifyToolOutput(output: LanguageModelV4ToolResultOutput): st
       return JSON.stringify(output.value);
     case 'content':
       return output.value
-        .map((v) => (v.type === 'text' ? v.text : ''))
+        .map((v) => {
+          if (v.type === 'text') return v.text;
+          // Non-text parts (files/media) must leave a trace in the flattened
+          // text — silently dropping them would break the round-trip claim
+          // and hide the part from compression/truncation entirely.
+          if (v.type === 'file') {
+            return Prompts.getToolResultFilePlaceholder(v.mediaType, v.filename);
+          }
+          return Prompts.getToolResultPartPlaceholder((v as { type: string }).type);
+        })
         .filter(Boolean)
         .join('\n');
     default:

packages/ai-sdk-middleware/src/middleware.ts

@@ -8,9 +8,14 @@ import {
   type ChefLogger,
   type CompressionDetails,
   compactMessages,
+  DEFAULT_SESSION_KEY,
+  dedupeConstructionWarnings,
+  flattenForCompression,
   Janitor,
   type Message,
+  normalizeSessionKey,
   Prompts,
+  SessionPool,
   type SummarizeHistoryOptions,
   summarizeHistory,
   XmlGenerator,
@@ -28,8 +33,6 @@ import { fromModelMessages } from './modelMessageAdapter';
 import { truncateToolResults } from './truncator';
 import type { ContextChefOptions, DynamicStateConfig } from './types';
 
-type CompressRole = 'system' | 'user' | 'assistant';
-
 /**
  * After this many compressions fire without an `onCompress` persistence hook,
  * warn once. The middleware compresses in-flight only — it never mutates the
@@ -92,10 +95,50 @@ export function createMiddleware(options: ContextChefOptions): LanguageModelMidd
     );
   };
 
-  const janitor = budgeting
-    ? createJanitor(options, options.contextWindow as number, logger, onCompressionFired)
+  // One Janitor per session. A middleware instance is usually created once at
+  // module scope (`const model = withContextChef(...)`) but serves many
+  // conversations — sharing one Janitor would leak token-usage feeds,
+  // compression suppression, and circuit-breaker counts across them. Callers
+  // opt in per call via `providerOptions: { contextChef: { sessionId } }`;
+  // calls without a sessionId share the default session (prior behavior).
+  // Construction-time config nags are deduped across sessions — the config
+  // is identical for every pooled Janitor, so once is enough.
+  const janitors = budgeting
+    ? new SessionPool(
+        dedupeConstructionWarnings(logger, (constructionLogger) =>
+          createJanitor(
+            options,
+            options.contextWindow as number,
+            constructionLogger,
+            onCompressionFired,
+          ),
+        ),
+        { maxSize: options.maxSessions },
+      )
     : null;
 
+  let invalidSessionKeyWarned = false;
+  const flagInvalidSessionKey = (raw: unknown) => {
+    if (invalidSessionKeyWarned) return;
+    invalidSessionKeyWarned = true;
+    logger.warn(
+      '[context-chef] Invalid providerOptions.contextChef sessionId (expected a non-empty ' +
+        `string, got ${raw === '' ? 'empty string' : typeof raw}); routing to the default session.`,
+    );
+  };
+
+  const janitorFor = (params: { providerOptions?: Record<string, unknown> }): Janitor | null => {
+    if (!janitors) return null;
+    const ns = params.providerOptions?.contextChef;
+    if (ns != null && typeof ns !== 'object') {
+      // Malformed namespace (e.g. contextChef: 'abc') — never a session key.
+      flagInvalidSessionKey(ns);
+      return janitors.get(DEFAULT_SESSION_KEY);
+    }
+    const raw = ns ? (ns as Record<string, unknown>).sessionId : undefined;
+    return janitors.get(normalizeSessionKey(raw, flagInvalidSessionKey));
+  };
+
   const clearsToolResults = !!options.clear?.some(
     (t) => t === 'tool-result' || (typeof t === 'object' && t.target === 'tool-result'),
   );
@@ -116,6 +159,7 @@ export function createMiddleware(options: ContextChefOptions): LanguageModelMidd
     specificationVersion: 'v4',
 
     transformParams: async ({ params }) => {
+      const janitor = janitorFor(params);
       let { prompt } = params;
 
       // 1. Truncate large tool results
@@ -176,9 +220,10 @@ export function createMiddleware(options: ContextChefOptions): LanguageModelMidd
       return { ...params, prompt };
     },
 
-    wrapGenerate: async ({ doGenerate }) => {
+    wrapGenerate: async ({ doGenerate, params }) => {
       const result = await doGenerate();
 
+      const janitor = janitorFor(params);
       if (!janitor) return result;
 
       if (result.usage?.inputTokens?.total != null) {
@@ -195,7 +240,8 @@ export function createMiddleware(options: ContextChefOptions): LanguageModelMidd
       return result;
     },
 
-    wrapStream: async ({ doStream }) => {
+    wrapStream: async ({ doStream, params }) => {
+      const janitor = janitorFor(params);
       if (!janitor) return doStream();
 
       const { stream, ...rest } = await doStream();
@@ -371,15 +417,6 @@ async function injectDynamicState(
   return result;
 }
 
-/**
- * Maps an IR role to a role accepted by generateText.
- * Tool messages are handled separately before this is called.
- */
-function toCompressRole(role: string): CompressRole {
-  if (role === 'system' || role === 'user' || role === 'assistant') return role;
-  return 'user';
-}
-
 /**
  * Adapts an AI SDK LanguageModelV4 into the compressionModel callback
  * that Janitor expects: (messages: Message[]) => Promise<string>
@@ -391,31 +428,9 @@ export function createCompressionAdapter(
   model: LanguageModel,
 ): (messages: Message[]) => Promise<string> {
   return async (messages: Message[]): Promise<string> => {
-    const formatted = messages.map((m): { role: CompressRole; content: string } => {
-      if (m.role === 'tool') {
-        return {
-          role: 'user' satisfies CompressRole,
-          content: `[Tool result${m.tool_call_id ? ` (${m.tool_call_id})` : ''}: ${m.content}]`,
-        };
-      }
-      if (m.role === 'assistant' && m.tool_calls?.length) {
-        const toolCallsDesc = m.tool_calls
-          .map((tc) => `[Called tool: ${tc.function.name}(${tc.function.arguments})]`)
-          .join('\n');
-        return {
-          role: 'assistant' satisfies CompressRole,
-          content: m.content ? `${m.content}\n${toolCallsDesc}` : toolCallsDesc,
-        };
-      }
-      return {
-        role: toCompressRole(m.role),
-        content: m.content,
-      };
-    });
-
     const { text } = await generateText({
       model,
-      messages: formatted,
+      messages: flattenForCompression(messages),
       maxOutputTokens: 2048,
     });
 

packages/ai-sdk-middleware/src/truncator.ts

@@ -4,6 +4,7 @@ import type {
   LanguageModelV4ToolResultPart,
 } from '@ai-sdk/provider';
 import { type ChefLogger, Offloader } from '@context-chef/core';
+import { stringifyToolOutput } from './adapter';
 import type { TruncateOptions } from './types';
 
 /**
@@ -141,16 +142,13 @@ function extractText(output: LanguageModelV4ToolResultOutput): string {
   switch (output.type) {
     case 'text':
     case 'error-text':
-      return output.value;
     case 'json':
     case 'error-json':
-      return JSON.stringify(output.value);
     case 'content':
-      return output.value
-        .map((v: { type: string; text?: string }) => (v.type === 'text' ? (v.text ?? '') : ''))
-        .filter(Boolean)
-        .join('\n');
+      return stringifyToolOutput(output);
     default:
+      // Marker outputs (e.g. execution-denied) measure as empty — they must
+      // never be rewritten by truncation.
       return '';
   }
 }

packages/ai-sdk-middleware/src/types.ts

@@ -257,4 +257,14 @@ export interface ContextChefOptions {
    * underlying Janitor and Offloader.
    */
   logger?: ChefLogger;
+  /**
+   * Cap on concurrently tracked sessions when session isolation is used
+   * (see the `providerOptions.contextChef.sessionId` docs on the package
+   * README). Each session gets its own Janitor so token-usage feeds and
+   * compression state never leak across conversations. Least-recently-used
+   * sessions beyond the cap are dropped and transparently recreated on next
+   * access. Must be a positive integer — the pool throws a RangeError
+   * otherwise. Default: 256.
+   */
+  maxSessions?: number;
 }

packages/ai-sdk-middleware/tests/adapter.test.ts

@@ -1,7 +1,31 @@
-import type { LanguageModelV4Prompt } from '@ai-sdk/provider';
+import type { LanguageModelV4Prompt, LanguageModelV4ToolResultOutput } from '@ai-sdk/provider';
 import type { Message } from '@context-chef/core';
 import { describe, expect, it } from 'vitest';
-import { fromAISDK, toAISDK } from '../src/adapter';
+import { fromAISDK, stringifyToolOutput, toAISDK } from '../src/adapter';
+
+describe('stringifyToolOutput', () => {
+  it('keeps a typed placeholder for non-text content parts instead of dropping them', () => {
+    const output: LanguageModelV4ToolResultOutput = {
+      type: 'content',
+      value: [
+        { type: 'text', text: 'Chart generated.' },
+        {
+          type: 'file',
+          data: { type: 'url', url: new URL('https://example.com/chart.png') },
+          mediaType: 'image/png',
+          filename: 'chart.png',
+        },
+      ],
+    };
+
+    const text = stringifyToolOutput(output);
+
+    expect(text).toContain('Chart generated.');
+    // The file part must leave a trace — not vanish silently.
+    expect(text).toContain('image/png');
+    expect(text).toContain('chart.png');
+  });
+});
 
 describe('fromAISDK', () => {
   it('converts system messages', () => {