diff --git a/.changeset/ai-sdk-v7-support.md b/.changeset/ai-sdk-v7-support.md new file mode 100644 index 0000000..d75b84b --- /dev/null +++ b/.changeset/ai-sdk-v7-support.md @@ -0,0 +1,31 @@ +--- +"@context-chef/ai-sdk-middleware": major +--- + +Support AI SDK v7 (provider spec V4). + +The middleware now targets `ai@>=7` / `@ai-sdk/provider@>=4`: it implements the V4 +language-model middleware spec (`specificationVersion: 'v4'`) and all public types +move from `LanguageModelV3*` to `LanguageModelV4*`. AI SDK v7's `wrapLanguageModel` +rejects a v3-spec middleware, so this is a breaking change that requires AI SDK v7. + +**Migration** + +- On AI SDK v7 (`ai@7`): upgrade to `@context-chef/ai-sdk-middleware@2`. +- Still on AI SDK v6 (`ai@6`): stay on `@context-chef/ai-sdk-middleware@1` — the 1.x + line continues to support the v3 spec. No code change is forced on you. + +**Removed** (deprecated APIs that were slated for removal in the next major): + +- `planCompaction`, `compactHistory`, and the `CompactionPlan` type (the provider-prompt + altitude variants) — use `planCompactionModelMessages` / `compactModelMessages` / + `CompactionPlanModelMessages` at the `ModelMessage` altitude instead. +- `onBudgetExceeded` on `ContextChefOptions` — use `onBeforeCompress` instead. + +Runtime behavior is unchanged. The only V4 nuance: provider-level `FilePart.data` +became a tagged union (`SharedV4FileData`); the prompt adapter handles it +transparently and the binary/URL payload still round-trips losslessly. + +On v7, durable in-loop compaction via `compactModelMessages` inside a +`ToolLoopAgent` `prepareStep` now persists across steps (AI SDK v7 carries +`prepareStep`-returned messages forward into later steps — v6 did not). diff --git a/README.md b/README.md index 05e4fcc..919ff69 100644 --- a/README.md +++ b/README.md @@ -940,7 +940,7 @@ Install only what you need: # Core library (OpenAI / Anthropic / Gemini direct SDK usage) npx skills add MyPrototypeWhat/context-chef --skill context-chef-core -# AI SDK middleware (Vercel AI SDK v6+) +# AI SDK middleware (Vercel AI SDK v7+) npx skills add MyPrototypeWhat/context-chef --skill context-chef-middleware # TanStack AI middleware (TanStack AI v0.10+) diff --git a/packages/ai-sdk-middleware/README.md b/packages/ai-sdk-middleware/README.md index 96d9399..51e3878 100644 --- a/packages/ai-sdk-middleware/README.md +++ b/packages/ai-sdk-middleware/README.md @@ -4,7 +4,7 @@ [![npm downloads](https://img.shields.io/npm/dm/@context-chef/ai-sdk-middleware.svg)](https://www.npmjs.com/package/@context-chef/ai-sdk-middleware) [![License](https://img.shields.io/npm/l/@context-chef/ai-sdk-middleware.svg)](https://github.com/MyPrototypeWhat/context-chef/blob/main/LICENSE) [![TypeScript](https://img.shields.io/badge/TypeScript-5.9-blue.svg)](https://www.typescriptlang.org/) -[![AI SDK](https://img.shields.io/badge/AI%20SDK-v6-black.svg)](https://ai-sdk.dev) +[![AI SDK](https://img.shields.io/badge/AI%20SDK-v7-black.svg)](https://ai-sdk.dev) [Vercel AI SDK](https://ai-sdk.dev) middleware powered by [context-chef](https://github.com/MyPrototypeWhat/context-chef). Transparent history compression, tool result truncation, and token budget management — zero code changes required. @@ -16,6 +16,9 @@ npm install @context-chef/ai-sdk-middleware ai ``` +> **AI SDK version.** `2.x` targets **AI SDK v7** (`ai@>=7`). Still on AI SDK v6? +> Use `@context-chef/ai-sdk-middleware@1` — the `1.x` line supports `ai@6`. + ## Quick Start ```typescript @@ -309,54 +312,6 @@ if (toSummarize.length > 0) { ModelMessage-altitude sibling of [`summarizeMessages`](#summarizemessagesprompt-model-options): summarize a `ModelMessage[]` slice into a single summary string via the same pipeline (role-flattening + core `summarizeHistory`). System messages are dropped. An empty slice returns `''` without a model call; it throws if the model call fails. Use it with `planCompactionModelMessages` when you want to drive summarization yourself instead of the one-shot `compactModelMessages`. -### `compactHistory(prompt, model, options)` - -> **Deprecated.** `compactHistory` / `planCompaction` take and return -> `LanguageModelV3Prompt` — the provider-protocol altitude, which nobody -> persists. Use [`compactModelMessages`](#compactmodelmessagesmessages-model-options) -> / `planCompactionModelMessages` instead. Still exported and working; removed in the next major. - -The V3-prompt variant of `compactModelMessages`. It splits the history on turn boundaries, summarizes the old slice, and returns a new prompt ready to persist — `[...system, , ...recent turns]`: - -```typescript -import { compactHistory } from '@context-chef/ai-sdk-middleware'; - -// Between model calls, when you own `messages`: -messages = await compactHistory(messages, summarizerModel, { - keepRecentTurns: 4, // keep the last 4 atomic turns verbatim - toolResultStubThreshold: 5000, -}); -// Persist the result — history actually shrinks and stays shrunk. -``` - -- The cut lands only on **turn boundaries** (an assistant + its tool results stay together), so it never orphans a tool result or splits a multi-block assistant message. -- System messages are preserved verbatim and never summarized. -- Returns the prompt **unchanged** when there is nothing old enough to compact (no more turns than `keepRecentTurns`) or the summarizer yields no text — safe to call unconditionally. Throws only if the model call throws. -- Accepts the same `SummarizeMessagesOptions` (`customCompressionInstructions`, `toolResultStubThreshold`) as `summarizeMessages`. - -### `planCompaction(prompt, options)` - -> **Deprecated.** Use [`planCompactionModelMessages`](#plancompactionmodelmessagesmessages-options). -> This V3-prompt variant is the provider-protocol altitude — a type you never -> persist. Still exported and working; removed in the next major. - -The synchronous split behind `compactHistory`, for when you want the boundary without summarizing (persist your own marker, or use a different summarizer). Returns `{ system, toSummarize, toKeep }` (all `LanguageModelV3Prompt`), cut on turn boundaries: - -```typescript -import { planCompaction, summarizeMessages } from '@context-chef/ai-sdk-middleware'; -import { Prompts } from '@context-chef/core'; - -const { system, toSummarize, toKeep } = planCompaction(messages, { keepRecentTurns: 4 }); -if (toSummarize.length > 0) { - const summary = await summarizeMessages(toSummarize, model); - messages = [ - ...system, - { role: 'user', content: [{ type: 'text', text: Prompts.getCompactSummaryWrapper(summary) }] }, - ...toKeep, - ]; -} -``` - ## How It Works ``` diff --git a/packages/ai-sdk-middleware/package.json b/packages/ai-sdk-middleware/package.json index f210829..89aad6a 100644 --- a/packages/ai-sdk-middleware/package.json +++ b/packages/ai-sdk-middleware/package.json @@ -47,13 +47,13 @@ "@context-chef/core": "workspace:*" }, "peerDependencies": { - "@ai-sdk/provider": ">=3", - "ai": ">=6" + "@ai-sdk/provider": ">=4", + "ai": ">=7" }, "devDependencies": { - "@ai-sdk/provider": "^3.0.8", + "@ai-sdk/provider": "^4.0.0", "@types/node": "^25.3.0", - "ai": "^6.0.140", + "ai": "^7.0.0", "tsdown": "^0.20.3", "typescript": "^5.9.3", "vitest": "^4.0.18" diff --git a/packages/ai-sdk-middleware/src/adapter.ts b/packages/ai-sdk-middleware/src/adapter.ts index 80b29b8..b35c7fc 100644 --- a/packages/ai-sdk-middleware/src/adapter.ts +++ b/packages/ai-sdk-middleware/src/adapter.ts @@ -1,9 +1,10 @@ import type { - LanguageModelV3Message, - LanguageModelV3Prompt, - LanguageModelV3ToolResultOutput, - LanguageModelV3ToolResultPart, - SharedV3ProviderOptions, + LanguageModelV4Message, + LanguageModelV4Prompt, + LanguageModelV4ToolResultOutput, + LanguageModelV4ToolResultPart, + SharedV4FileData, + SharedV4ProviderOptions, } from '@ai-sdk/provider'; import { type Attachment, @@ -12,10 +13,24 @@ import { type ToolCall, } from '@context-chef/core'; +/** + * Extracts the Attachment presence/metadata signal from a V4 file part's `data`. + * + * V4 restructured `FilePart.data` from a bare value into a tagged union + * (`SharedV4FileData`). Only inline string data (`{ type: 'data', data: string }`) + * yields a recorded signal — bytes, URLs, and provider references become `''`, + * exactly as the pre-v4 adapter only recorded already-string `data`. The real + * payload always round-trips losslessly through `_userContent`/`_assistantContent`; + * this value is only a presence signal Janitor reads via `attachments?.length`. + */ +function fileDataSignal(data: SharedV4FileData): string { + return data.type === 'data' && typeof data.data === 'string' ? data.data : ''; +} + /** Content types for each AI SDK message role */ -type UserContent = Extract['content']; -type AssistantContent = Extract['content']; -type ToolContent = Extract['content']; +type UserContent = Extract['content']; +type AssistantContent = Extract['content']; +type ToolContent = Extract['content']; /** * Extended IR message with typed pass-through fields for lossless AI SDK round-trip. @@ -26,12 +41,12 @@ export interface AISDKMessage extends Message { _assistantContent?: AssistantContent; _toolContent?: ToolContent; _originalText?: string; - _providerOptions?: SharedV3ProviderOptions; + _providerOptions?: SharedV4ProviderOptions; _toolName?: string; } /** - * Converts an AI SDK V3 prompt to context-chef IR messages. + * Converts an AI SDK V4 prompt to context-chef IR messages. * * Original AI SDK content is stored in per-role fields for lossless round-trip. * `_originalText` caches the extracted text so `toAISDK` can detect Janitor modifications. @@ -42,7 +57,7 @@ export interface AISDKMessage extends Message { * non-system message is a user message. This is a system boundary — IR * downstream is trusted to satisfy invariants. */ -export function fromAISDK(prompt: LanguageModelV3Prompt): AISDKMessage[] { +export function fromAISDK(prompt: LanguageModelV4Prompt): AISDKMessage[] { const messages: AISDKMessage[] = []; for (const msg of prompt) { @@ -73,7 +88,7 @@ export function fromAISDK(prompt: LanguageModelV3Prompt): AISDKMessage[] { // so we never invent a fake encoding for non-string inputs. attachments.push({ mediaType: part.mediaType, - data: typeof part.data === 'string' ? part.data : '', + data: fileDataSignal(part.data), ...(part.filename ? { filename: part.filename } : {}), }); } @@ -127,7 +142,7 @@ export function fromAISDK(prompt: LanguageModelV3Prompt): AISDKMessage[] { // _assistantContent carries the actual payload through round-trip. attachments.push({ mediaType: part.mediaType, - data: typeof part.data === 'string' ? part.data : '', + data: fileDataSignal(part.data), ...(part.filename ? { filename: part.filename } : {}), }); } @@ -189,14 +204,14 @@ function asAISDK(msg: Message): AISDKMessage { } /** - * Converts context-chef IR messages back to AI SDK V3 prompt format. + * Converts context-chef IR messages back to AI SDK V4 prompt format. * * Uses per-role original content when unmodified (detected via `_originalText`). * Falls back to constructing from IR fields when content was modified by Janitor * (e.g. compact() cleared tool results) or for new messages (e.g. compression summaries). */ -export function toAISDK(messages: Message[]): LanguageModelV3Prompt { - const prompt: LanguageModelV3Prompt = []; +export function toAISDK(messages: Message[]): LanguageModelV4Prompt { + const prompt: LanguageModelV4Prompt = []; let i = 0; while (i < messages.length) { @@ -240,11 +255,11 @@ export function toAISDK(messages: Message[]): LanguageModelV3Prompt { } if (msg.role === 'tool') { - const toolResults: LanguageModelV3ToolResultPart[] = []; + const toolResults: LanguageModelV4ToolResultPart[] = []; // Re-attach the message-level providerOptions captured on the first IR // message of the original tool turn (see fromAISDK). Take the first // non-undefined across the coalesced group. - let providerOptions: SharedV3ProviderOptions | undefined; + let providerOptions: SharedV4ProviderOptions | undefined; while (i < messages.length && messages[i].role === 'tool') { const toolMsg = asAISDK(messages[i]); const toolModified = @@ -289,7 +304,7 @@ export function toAISDK(messages: Message[]): LanguageModelV3Prompt { return prompt; } -export function stringifyToolOutput(output: LanguageModelV3ToolResultOutput): string { +export function stringifyToolOutput(output: LanguageModelV4ToolResultOutput): string { switch (output.type) { case 'text': case 'error-text': diff --git a/packages/ai-sdk-middleware/src/compaction.ts b/packages/ai-sdk-middleware/src/compaction.ts index 7da2bb0..05ab4c2 100644 --- a/packages/ai-sdk-middleware/src/compaction.ts +++ b/packages/ai-sdk-middleware/src/compaction.ts @@ -1,4 +1,3 @@ -import type { LanguageModelV3, LanguageModelV3Prompt } from '@ai-sdk/provider'; import { compactHistory as coreCompactHistory, planCompaction as corePlanCompaction, @@ -6,101 +5,11 @@ import { } from '@context-chef/core'; import type { LanguageModel, ModelMessage } from 'ai'; -import { fromAISDK, toAISDK } from './adapter'; import { createCompressionAdapter, type SummarizeMessagesOptions } from './middleware'; import { fromModelMessages, toModelMessages } from './modelMessageAdapter'; export type { PlanCompactionOptions } from '@context-chef/core'; -export interface CompactionPlan { - /** System messages, preserved verbatim — standing instructions are never summarized. */ - system: LanguageModelV3Prompt; - /** - * The old conversation slice to summarize (system excluded). Feed this to - * `summarizeMessages`. Empty when there is nothing old enough to compact. - */ - toSummarize: LanguageModelV3Prompt; - /** The recent conversation turns to keep verbatim. */ - toKeep: LanguageModelV3Prompt; -} - -/** - * @deprecated Use {@link planCompactionModelMessages}. This V3-prompt variant is - * the provider-protocol altitude — a type you never persist. Removed in the next - * major. - * - * Splits an AI SDK prompt into `{ system, toSummarize, toKeep }` on **turn - * boundaries**, for durable (caller-owned) compaction. - * - * Unlike the in-flight middleware `compress` — which only rewrites the outgoing - * request and is discarded each call — this is a pure, synchronous split you run - * against your *own* message store. Summarize `toSummarize`, then persist - * `[...system, , ...toKeep]` back to your store so the history actually - * shrinks. See {@link compactHistory} for the one-shot version. - * - * The AI-SDK-typed wrapper around core's provider-agnostic `planCompaction`: - * converts the prompt to IR via {@link fromAISDK}, splits on turn boundaries - * (assistant + its tool results stay together), and converts each slice back via - * {@link toAISDK}. - */ -export function planCompaction( - prompt: LanguageModelV3Prompt, - options: PlanCompactionOptions, -): CompactionPlan { - const plan = corePlanCompaction(fromAISDK(prompt), options); - return { - system: toAISDK(plan.system), - toSummarize: toAISDK(plan.toSummarize), - toKeep: toAISDK(plan.toKeep), - }; -} - -/** - * @deprecated Use {@link compactModelMessages}. `LanguageModelV3Prompt` is the - * provider-protocol altitude (ephemeral, never persisted); durable compaction - * belongs at the ModelMessage altitude. Removed in the next major. - * - * One-shot durable compaction: plan a turn-safe split, summarize the old slice, - * and return a new prompt ready to persist — `[...system, , ...toKeep]`. - * - * This is the recommended way to keep a long conversation lean when you own the - * message store (a long agent loop, or a chat past the budget). Run it between - * model calls and replace your stored messages with the result; the summary is - * a real `user` message wrapped with the "continued conversation" framing. - * - * Returns the prompt **unchanged** (same reference) when there is nothing old - * enough to compact (no more turns than `keepRecentTurns`) or when the summarizer - * yields no text — so it is safe to call unconditionally, and callers can skip - * persistence on a no-op via `result === prompt`. Throws only if the model call - * throws. - * - * The AI-SDK-typed wrapper around core's `compactHistory`: it binds `model` into - * a compression callback via {@link createCompressionAdapter} (core never calls a - * model directly). Do NOT also configure middleware `compress` (with a `model`) - * on the same path — that compresses twice. Use this OR in-flight `compress`, - * not both. - * - * @example - * ```ts - * // In your loop / between turns, when you own `messages`: - * messages = await compactHistory(messages, summarizerModel, { - * keepRecentTurns: 4, - * toolResultStubThreshold: 5000, - * }); - * ``` - */ -export async function compactHistory( - prompt: LanguageModelV3Prompt, - model: LanguageModelV3, - options: PlanCompactionOptions & SummarizeMessagesOptions, -): Promise { - const ir = fromAISDK(prompt); - const result = await coreCompactHistory(ir, createCompressionAdapter(model), options); - // core returns the input IR reference on a no-op — preserve the original - // prompt reference so callers can skip persistence via `result === prompt`. - return result === ir ? prompt : toAISDK(result); -} - export interface CompactionPlanModelMessages { /** System messages, preserved verbatim — standing instructions are never summarized. */ system: ModelMessage[]; @@ -143,7 +52,7 @@ export function planCompactionModelMessages( * loop, or inside a `ToolLoopAgent` `prepareStep` (`return { messages: await * compactModelMessages(messages, model, opts) }`). * - * `model` is `ai`'s `LanguageModel` (string id | V3 | V2) — exactly what + * `model` is `ai`'s `LanguageModel` (string id | V4) — exactly what * `prepareStep`/`generateText` give you. Reuses core's `compactHistory` + * `createCompressionAdapter` (tool-role flattening); no model is called directly. * diff --git a/packages/ai-sdk-middleware/src/index.ts b/packages/ai-sdk-middleware/src/index.ts index 4b23b7a..22e94e5 100644 --- a/packages/ai-sdk-middleware/src/index.ts +++ b/packages/ai-sdk-middleware/src/index.ts @@ -1,4 +1,4 @@ -import type { LanguageModelV3 } from '@ai-sdk/provider'; +import type { LanguageModelV4 } from '@ai-sdk/provider'; import { wrapLanguageModel } from 'ai'; import { createMiddleware } from './middleware'; @@ -7,12 +7,9 @@ import type { ContextChefOptions } from './types'; export type { ClearTarget } from '@context-chef/core'; export { type AISDKMessage, fromAISDK, toAISDK } from './adapter'; export { - type CompactionPlan, type CompactionPlanModelMessages, - compactHistory, compactModelMessages, type PlanCompactionOptions, - planCompaction, planCompactionModelMessages, } from './compaction'; export { @@ -50,9 +47,9 @@ export type { * ``` */ export function withContextChef( - model: LanguageModelV3, + model: LanguageModelV4, options: ContextChefOptions, -): LanguageModelV3 { +): LanguageModelV4 { const middleware = createMiddleware(options); return wrapLanguageModel({ model, middleware }); } diff --git a/packages/ai-sdk-middleware/src/middleware.ts b/packages/ai-sdk-middleware/src/middleware.ts index 1dc34ba..8d75220 100644 --- a/packages/ai-sdk-middleware/src/middleware.ts +++ b/packages/ai-sdk-middleware/src/middleware.ts @@ -1,8 +1,8 @@ import type { - LanguageModelV3, - LanguageModelV3Message, - LanguageModelV3Prompt, - LanguageModelV3StreamPart, + LanguageModelV4, + LanguageModelV4Message, + LanguageModelV4Prompt, + LanguageModelV4StreamPart, } from '@ai-sdk/provider'; import { type ChefLogger, @@ -56,14 +56,12 @@ export function createMiddleware(options: ContextChefOptions): LanguageModelMidd // `contextWindow`. Truncate/compact/skill/dynamicState-only // configurations get no Janitor at all: no budget checks, no token-usage // capture, and none of the Janitor's missing-tokenizer warnings. - const budgeting = Boolean( - options.compress || options.onCompress || options.onBeforeCompress || options.onBudgetExceeded, - ); + const budgeting = Boolean(options.compress || options.onCompress || options.onBeforeCompress); if (budgeting && options.contextWindow == null) { throw new Error( '[context-chef] `contextWindow` is required when a compression option (`compress`, ' + - '`onCompress`, `onBeforeCompress`, `onBudgetExceeded`) is configured — the budget ' + + '`onCompress`, `onBeforeCompress`) is configured — the budget ' + 'check has nothing to compare against without it.', ); } @@ -115,7 +113,7 @@ export function createMiddleware(options: ContextChefOptions): LanguageModelMidd } return { - specificationVersion: 'v3', + specificationVersion: 'v4', transformParams: async ({ params }) => { let { prompt } = params; @@ -202,7 +200,7 @@ export function createMiddleware(options: ContextChefOptions): LanguageModelMidd const { stream, ...rest } = await doStream(); - const transform = new TransformStream({ + const transform = new TransformStream({ transform(chunk, controller) { if (chunk.type === 'finish') { if (chunk.usage?.inputTokens?.total != null) { @@ -254,7 +252,7 @@ function createJanitor( compressedMessages: toAISDK(details.compressedMessages), }); }, - onBeforeCompress: options.onBeforeCompress ?? options.onBudgetExceeded, + onBeforeCompress: options.onBeforeCompress, logger, }; @@ -283,18 +281,18 @@ function createJanitor( } /** - * Prunes a LanguageModelV3Prompt via AI SDK's pruneMessages. + * Prunes a LanguageModelV4Prompt via AI SDK's pruneMessages. * - * LanguageModelV3Message (from @ai-sdk/provider) and ModelMessage + * LanguageModelV4Message (from @ai-sdk/provider) and ModelMessage * (from @ai-sdk/provider-utils) share identical runtime structure but * differ at the TypeScript level (e.g. ImagePart, FilePart.data). * Since pruneMessages only filters — never transforms — every content - * part in the output is an original V3 part, making the casts safe. + * part in the output is an original V4 part, making the casts safe. */ function compactPrompt( - prompt: LanguageModelV3Prompt, + prompt: LanguageModelV4Prompt, config: Omit[0], 'messages'>, -): LanguageModelV3Prompt { +): LanguageModelV4Prompt { const messages = prompt.map( (msg) => ({ @@ -310,7 +308,7 @@ function compactPrompt( role: msg.role, content: msg.content, providerOptions: msg.providerOptions, - }) as LanguageModelV3Message, + }) as LanguageModelV4Message, ); } @@ -339,9 +337,9 @@ async function resolveSkillMessages(skill: ContextChefOptions['skill']): Promise * - `system`: Adds as a standalone system message at the end. */ async function injectDynamicState( - prompt: LanguageModelV3Prompt, + prompt: LanguageModelV4Prompt, config: DynamicStateConfig, -): Promise { +): Promise { const state = await config.getState(); const xml = XmlGenerator.objectToXml(state, 'dynamic_state'); const placement = config.placement ?? 'last_user'; @@ -383,7 +381,7 @@ function toCompressRole(role: string): CompressRole { } /** - * Adapts an AI SDK LanguageModelV3 into the compressionModel callback + * Adapts an AI SDK LanguageModelV4 into the compressionModel callback * that Janitor expects: (messages: Message[]) => Promise * * Tool messages are converted to user messages describing the tool interaction, @@ -450,8 +448,8 @@ export type SummarizeMessagesOptions = SummarizeHistoryOptions; * `truncate`, `clear`, and `dynamicState`, remain safe to use alongside. */ export async function summarizeMessages( - prompt: LanguageModelV3Prompt, - model: LanguageModelV3, + prompt: LanguageModelV4Prompt, + model: LanguageModelV4, opts: SummarizeMessagesOptions = {}, ): Promise { const ir = fromAISDK(prompt).filter((m) => m.role !== 'system'); diff --git a/packages/ai-sdk-middleware/src/modelMessageAdapter.ts b/packages/ai-sdk-middleware/src/modelMessageAdapter.ts index 3b5f39f..bfb114f 100644 --- a/packages/ai-sdk-middleware/src/modelMessageAdapter.ts +++ b/packages/ai-sdk-middleware/src/modelMessageAdapter.ts @@ -1,4 +1,4 @@ -import type { LanguageModelV3ToolResultOutput } from '@ai-sdk/provider'; +import type { LanguageModelV4ToolResultOutput } from '@ai-sdk/provider'; import { type Attachment, ensureValidHistory, @@ -9,10 +9,11 @@ import type { ModelMessage } from 'ai'; import { stringifyToolOutput } from './adapter'; -// NOTE: This adapter intentionally parallels src/adapter.ts (the V3 adapter). The -// user/assistant/file extraction logic is duplicated by design; keep the two in -// sync. The deltas here are deliberate: string-shorthand content, ImagePart, and -// approval parts — none of which exist at the V3 (LanguageModelV3Prompt) altitude. +// NOTE: This adapter intentionally parallels src/adapter.ts (the provider-prompt +// adapter). The user/assistant/file extraction logic is duplicated by design; keep +// the two in sync. The deltas here are deliberate: string-shorthand content, +// ImagePart, and approval parts — none of which exist at the provider-prompt +// (LanguageModelV4Prompt) altitude. // Content/part types derived from ModelMessage — no part-type imports needed // (provider-utils does not export them all stably). Same trick as adapter.ts. @@ -20,10 +21,34 @@ type UserContent = Extract['content']; type AssistantContent = Extract['content']; type ToolContent = Extract['content']; type ProviderOptions = Extract['providerOptions']; +type MMFilePart = Extract[number], { type: 'file' }>; + +/** + * Extracts the Attachment presence/metadata signal from an app-altitude file + * part's `data`. The app `ModelMessage` accepts both the bare shorthand + * (`string | Uint8Array | URL`) and V4's tagged shapes (`{ type: 'data', data }`, + * `{ type: 'url', url }`, …). Only inline string data yields a signal — bytes, + * URLs, and references become `''`, mirroring the provider-prompt adapter's + * `fileDataSignal`. The real payload round-trips losslessly via `_mm*Content`; + * this value is only read by Janitor via `attachments?.length`. + */ +function fileDataSignal(data: MMFilePart['data']): string { + if (typeof data === 'string') return data; + if ( + typeof data === 'object' && + data !== null && + 'type' in data && + data.type === 'data' && + typeof data.data === 'string' + ) { + return data.data; + } + return ''; +} /** * IR message carrying the original ModelMessage content for lossless round-trip. - * Parallel to AISDKMessage (the V3 adapter's carrier) but typed to the + * Parallel to AISDKMessage (the provider-prompt adapter's carrier) but typed to the * application-layer ModelMessage shapes, and on distinct `_mm*` fields so the two * adapters can never read each other's pass-through by accident. */ @@ -79,7 +104,7 @@ export function fromModelMessages(messages: ModelMessage[]): ModelMessageIR[] { if (part.type === 'file') { attachments.push({ mediaType: part.mediaType, - data: typeof part.data === 'string' ? part.data : '', + data: fileDataSignal(part.data), ...(part.filename ? { filename: part.filename } : {}), }); } else if (part.type === 'image') { @@ -141,7 +166,7 @@ export function fromModelMessages(messages: ModelMessage[]): ModelMessageIR[] { } else if (part.type === 'file') { attachments.push({ mediaType: part.mediaType, - data: typeof part.data === 'string' ? part.data : '', + data: fileDataSignal(part.data), ...(part.filename ? { filename: part.filename } : {}), }); } @@ -174,8 +199,8 @@ export function fromModelMessages(messages: ModelMessage[]): ModelMessageIR[] { let firstOfMessage = true; for (const part of msg.content) { if (part.type === 'tool-result') { - // ModelMessage's ToolResultOutput is a structural superset of V3's (extra content[] members); stringifyToolOutput only reads .type/.value, so the cast is safe and matches the V3 adapter's projection. - const text = stringifyToolOutput(part.output as LanguageModelV3ToolResultOutput); + // ModelMessage's ToolResultOutput is a structural superset of the provider output's (extra content[] members); stringifyToolOutput only reads .type/.value, so the cast is safe and matches the provider-prompt adapter's projection. + const text = stringifyToolOutput(part.output as LanguageModelV4ToolResultOutput); anchor = { role: 'tool', content: text, diff --git a/packages/ai-sdk-middleware/src/truncator.ts b/packages/ai-sdk-middleware/src/truncator.ts index fdaf125..d0c0218 100644 --- a/packages/ai-sdk-middleware/src/truncator.ts +++ b/packages/ai-sdk-middleware/src/truncator.ts @@ -1,7 +1,7 @@ import type { - LanguageModelV3Prompt, - LanguageModelV3ToolResultOutput, - LanguageModelV3ToolResultPart, + LanguageModelV4Prompt, + LanguageModelV4ToolResultOutput, + LanguageModelV4ToolResultPart, } from '@ai-sdk/provider'; import { type ChefLogger, Offloader } from '@context-chef/core'; import type { TruncateOptions } from './types'; @@ -11,10 +11,10 @@ import type { TruncateOptions } from './types'; * When a storage adapter is provided, original content is persisted and a URI is included in the output. */ export async function truncateToolResults( - prompt: LanguageModelV3Prompt, + prompt: LanguageModelV4Prompt, options: TruncateOptions, logger: ChefLogger = console, -): Promise { +): Promise { const { threshold, headChars = 0, tailChars = 1000, storage } = options; const offloader = storage @@ -22,7 +22,7 @@ export async function truncateToolResults( : null; const policy = buildPolicyMap(options.perTool); - const result: LanguageModelV3Prompt = []; + const result: LanguageModelV4Prompt = []; for (const msg of prompt) { if (msg.role !== 'tool') { @@ -68,8 +68,8 @@ export async function truncateToolResults( output: { type: 'text', value: vfsResult.content, - } satisfies LanguageModelV3ToolResultOutput, - } satisfies LanguageModelV3ToolResultPart); + } satisfies LanguageModelV4ToolResultOutput, + } satisfies LanguageModelV4ToolResultPart); continue; } catch (error) { logger.warn( @@ -96,8 +96,8 @@ export async function truncateToolResults( newContent.push({ ...part, - output: { type: 'text', value: truncated } satisfies LanguageModelV3ToolResultOutput, - } satisfies LanguageModelV3ToolResultPart); + output: { type: 'text', value: truncated } satisfies LanguageModelV4ToolResultOutput, + } satisfies LanguageModelV4ToolResultPart); } result.push({ ...msg, content: newContent }); @@ -137,7 +137,7 @@ function buildPolicyMap(perTool: TruncateOptions['perTool']): Map void; /** * Called when token budget is exceeded, before LLM compression. @@ -244,20 +244,13 @@ export interface ContextChefOptions { history: Message[], tokenInfo: { currentTokens: number; limit: number }, ) => Message[] | null | undefined | Promise; - /** - * @deprecated Use `onBeforeCompress` instead. Will be removed in the next major version. - */ - onBudgetExceeded?: ( - history: Message[], - tokenInfo: { currentTokens: number; limit: number }, - ) => Message[] | null | undefined | Promise; /** * Transform the AI SDK prompt after compression, before sending to the model. * Use for custom prompt manipulation, RAG injection, etc. */ transformContext?: ( - prompt: LanguageModelV3Prompt, - ) => LanguageModelV3Prompt | Promise; + prompt: LanguageModelV4Prompt, + ) => LanguageModelV4Prompt | Promise; /** * Sink for degradation warnings (storage write failures, missing usage * data, misconfiguration). Defaults to `console`. Forwarded to the diff --git a/packages/ai-sdk-middleware/tests/adapter.test.ts b/packages/ai-sdk-middleware/tests/adapter.test.ts index 5604902..d762eff 100644 --- a/packages/ai-sdk-middleware/tests/adapter.test.ts +++ b/packages/ai-sdk-middleware/tests/adapter.test.ts @@ -1,17 +1,17 @@ -import type { LanguageModelV3Prompt } from '@ai-sdk/provider'; +import type { LanguageModelV4Prompt } from '@ai-sdk/provider'; import type { Message } from '@context-chef/core'; import { describe, expect, it } from 'vitest'; import { fromAISDK, toAISDK } from '../src/adapter'; describe('fromAISDK', () => { it('converts system messages', () => { - const prompt: LanguageModelV3Prompt = [{ role: 'system', content: 'You are helpful.' }]; + const prompt: LanguageModelV4Prompt = [{ role: 'system', content: 'You are helpful.' }]; const result = fromAISDK(prompt); expect(result).toEqual([{ role: 'system', content: 'You are helpful.' }]); }); it('converts user messages with text parts', () => { - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [ @@ -28,11 +28,11 @@ describe('fromAISDK', () => { it('stores original content including file parts', () => { const filePart = { type: 'file' as const, - data: 'base64data', + data: { type: 'data' as const, data: 'base64data' }, mediaType: 'image/png', }; const content = [{ type: 'text' as const, text: 'Look at this' }, filePart]; - const prompt: LanguageModelV3Prompt = [{ role: 'user', content }]; + const prompt: LanguageModelV4Prompt = [{ role: 'user', content }]; const result = fromAISDK(prompt); expect(result[0].content).toBe('Look at this'); expect(result[0]._userContent).toEqual(content); @@ -40,13 +40,18 @@ describe('fromAISDK', () => { }); it('maps multiple file parts to attachments', () => { - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [ { type: 'text', text: 'Check these' }, - { type: 'file', data: 'img1', mediaType: 'image/png' }, - { type: 'file', data: 'doc1', mediaType: 'application/pdf', filename: 'report.pdf' }, + { type: 'file', data: { type: 'data', data: 'img1' }, mediaType: 'image/png' }, + { + type: 'file', + data: { type: 'data', data: 'doc1' }, + mediaType: 'application/pdf', + filename: 'report.pdf', + }, ], }, ]; @@ -58,7 +63,7 @@ describe('fromAISDK', () => { }); it('does not set attachments when no file parts exist', () => { - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Just text' }] }, ]; const result = fromAISDK(prompt); @@ -67,12 +72,12 @@ describe('fromAISDK', () => { it('preserves Uint8Array file data verbatim through _userContent (no encoding into Attachment.data)', () => { const bytes = new Uint8Array([72, 101, 108, 108, 111]); // "Hello" - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [ { type: 'text', text: 'Binary please' }, - { type: 'file', data: bytes, mediaType: 'image/png' }, + { type: 'file', data: { type: 'data', data: bytes }, mediaType: 'image/png' }, ], }, ]; @@ -81,34 +86,34 @@ describe('fromAISDK', () => { // The real binary lives on _userContent for the AI SDK round-trip. expect(result[0].attachments).toEqual([{ mediaType: 'image/png', data: '' }]); const filePart = result[0]._userContent?.find((p) => p.type === 'file'); - expect(filePart?.data).toBe(bytes); // same reference, not a copy + expect((filePart?.data as { data: unknown }).data).toBe(bytes); // same reference, not a copy }); it('preserves URL file data verbatim through _userContent (no toString into Attachment.data)', () => { const url = new URL('https://example.com/img.png'); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [ { type: 'text', text: 'Remote image' }, - { type: 'file', data: url, mediaType: 'image/png' }, + { type: 'file', data: { type: 'url', url }, mediaType: 'image/png' }, ], }, ]; const result = fromAISDK(prompt); expect(result[0].attachments).toEqual([{ mediaType: 'image/png', data: '' }]); const filePart = result[0]._userContent?.find((p) => p.type === 'file'); - expect(filePart?.data).toBe(url); // same URL instance, not toString'd + expect((filePart?.data as { url: unknown }).url).toBe(url); // same URL instance, not toString'd }); it('toAISDK round-trips Uint8Array binary back to the AI SDK provider verbatim', () => { const bytes = new Uint8Array([1, 2, 3, 4, 5]); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [ { type: 'text', text: 'binary' }, - { type: 'file', data: bytes, mediaType: 'image/png' }, + { type: 'file', data: { type: 'data', data: bytes }, mediaType: 'image/png' }, ], }, ]; @@ -119,11 +124,11 @@ describe('fromAISDK', () => { const filePart = (userMsg?.content as Array<{ type: string; data?: unknown }>).find( (p) => p.type === 'file', ); - expect(filePart?.data).toBe(bytes); // same reference + expect((filePart?.data as { data: unknown }).data).toBe(bytes); // same reference }); it('converts assistant messages with text + tool calls', () => { - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'weather?' }] }, { role: 'assistant', @@ -162,7 +167,7 @@ describe('fromAISDK', () => { }); it('converts assistant reasoning to thinking', () => { - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'reason about this' }] }, { role: 'assistant', @@ -178,13 +183,13 @@ describe('fromAISDK', () => { }); it('maps assistant file parts to attachments', () => { - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'generate an image' }] }, { role: 'assistant', content: [ { type: 'text', text: 'Here is the image.' }, - { type: 'file', data: 'generated_img', mediaType: 'image/png' }, + { type: 'file', data: { type: 'data', data: 'generated_img' }, mediaType: 'image/png' }, ], }, ]; @@ -194,7 +199,7 @@ describe('fromAISDK', () => { }); it('converts tool messages to individual IR messages', () => { - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'q' }] }, { role: 'assistant', @@ -248,7 +253,7 @@ describe('fromAISDK', () => { }); it('handles json tool result output', () => { - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'query' }] }, { role: 'assistant', @@ -280,7 +285,7 @@ describe('fromAISDK', () => { // ─── Boundary sanitization ─── it('injects placeholder for missing tool result at boundary', () => { - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'do it' }] }, { role: 'assistant', @@ -307,7 +312,7 @@ describe('fromAISDK', () => { // ─── FIX #3: tool-call with input:undefined yields a string '{}' ─── it('serializes a tool-call with undefined input to "{}" (a string)', () => { - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'go' }] }, { role: 'assistant', @@ -342,7 +347,7 @@ describe('fromAISDK', () => { it('round-trips sanitized placeholder with original toolName (not "unknown")', () => { // Regression guard for the boundary-sanitize bug where injected placeholders // round-tripped as `toolName: 'unknown'` because toAISDK only read _toolName. - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'do it' }] }, { role: 'assistant', @@ -464,7 +469,7 @@ describe('toAISDK', () => { describe('round-trip', () => { it('preserves a full conversation through fromAISDK → toAISDK', () => { - const original: LanguageModelV3Prompt = [ + const original: LanguageModelV4Prompt = [ { role: 'system', content: 'You are a helpful assistant.' }, { role: 'user', content: [{ type: 'text', text: 'What is 2+2?' }] }, { role: 'assistant', content: [{ type: 'text', text: '4' }] }, @@ -477,7 +482,7 @@ describe('round-trip', () => { }); it('preserves tool call + result through round-trip', () => { - const original: LanguageModelV3Prompt = [ + const original: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'find something' }] }, { role: 'assistant', @@ -510,12 +515,12 @@ describe('round-trip', () => { }); it('preserves file parts through round-trip', () => { - const original: LanguageModelV3Prompt = [ + const original: LanguageModelV4Prompt = [ { role: 'user', content: [ { type: 'text', text: 'Analyze this' }, - { type: 'file', data: 'base64data', mediaType: 'image/png' }, + { type: 'file', data: { type: 'data', data: 'base64data' }, mediaType: 'image/png' }, ], }, ]; @@ -526,7 +531,7 @@ describe('round-trip', () => { }); it('uses modified content when Janitor changes it (e.g. compact)', () => { - const original: LanguageModelV3Prompt = [ + const original: LanguageModelV4Prompt = [ { role: 'tool', content: [ @@ -555,7 +560,7 @@ describe('round-trip', () => { }); it('preserves providerOptions through round-trip', () => { - const original: LanguageModelV3Prompt = [ + const original: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hello' }], @@ -570,7 +575,7 @@ describe('round-trip', () => { // ─── FIX #1: inline (provider-executed) tool-result must not trigger a placeholder ─── it('does not inject a placeholder for an inline (provider-executed) tool-result', () => { - const original: LanguageModelV3Prompt = [ + const original: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'search the web' }] }, { role: 'assistant', @@ -604,7 +609,7 @@ describe('round-trip', () => { }); it('pairs a normal tool-call and skips an inline one in the same assistant message', () => { - const original: LanguageModelV3Prompt = [ + const original: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'do two things' }] }, { role: 'assistant', @@ -651,7 +656,7 @@ describe('round-trip', () => { // ─── FIX #2: tool-message-level providerOptions survives round-trip ─── it('preserves tool-message-level providerOptions through round-trip', () => { - const original: LanguageModelV3Prompt = [ + const original: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'run it' }] }, { role: 'assistant', diff --git a/packages/ai-sdk-middleware/tests/compaction.test.ts b/packages/ai-sdk-middleware/tests/compaction.test.ts deleted file mode 100644 index fd52e5b..0000000 --- a/packages/ai-sdk-middleware/tests/compaction.test.ts +++ /dev/null @@ -1,216 +0,0 @@ -import type { - LanguageModelV3, - LanguageModelV3CallOptions, - LanguageModelV3Content, - LanguageModelV3FinishReason, - LanguageModelV3GenerateResult, - LanguageModelV3Prompt, -} from '@ai-sdk/provider'; -import { describe, expect, it } from 'vitest'; -import { compactHistory, planCompaction } from '../src/compaction'; - -// These are the AI-SDK boundary functions — thin wrappers over core's -// provider-agnostic `planCompaction` / `compactHistory`. The turn-split algorithm -// itself is exhaustively tested in core (durableCompaction.test.ts); the tests -// here cover what the boundary adds: fromAISDK/toAISDK round-tripping, the -// no-op reference short-circuit, and the summary's AI SDK message shape. - -/** Minimal V3 model whose summarization call returns a fixed string. */ -function createSummarizerModel(summaryText = 'SUMMARY'): LanguageModelV3 { - return { - specificationVersion: 'v3', - provider: 'test', - modelId: 'test-model', - supportedUrls: {}, - async doGenerate(_opts: LanguageModelV3CallOptions): Promise { - const content: LanguageModelV3Content[] = [{ type: 'text', text: summaryText }]; - const finishReason: LanguageModelV3FinishReason = { unified: 'stop', raw: undefined }; - return { - content, - finishReason, - warnings: [], - usage: { - inputTokens: { - total: 50, - noCache: undefined, - cacheRead: undefined, - cacheWrite: undefined, - }, - outputTokens: { total: 10, text: undefined, reasoning: undefined }, - }, - response: { id: 'id', timestamp: new Date(), modelId: 'test-model' }, - }; - }, - async doStream() { - throw new Error('not used'); - }, - }; -} - -/** N plain user/assistant turns, optionally with a leading system message. */ -function plainTurns(n: number, withSystem = true): LanguageModelV3Prompt { - const prompt: LanguageModelV3Prompt = withSystem - ? [{ role: 'system', content: 'You are helpful.' }] - : []; - for (let i = 0; i < n; i++) { - prompt.push({ role: 'user', content: [{ type: 'text', text: `q${i}` }] }); - prompt.push({ role: 'assistant', content: [{ type: 'text', text: `a${i}` }] }); - } - return prompt; -} - -describe('planCompaction', () => { - it('delegates the split to core and round-trips each slice through the adapter', () => { - // 3 user/assistant pairs = 6 turns (each message is its own turn). - const plan = planCompaction(plainTurns(3), { keepRecentTurns: 2 }); - - expect(plan.system.map((m) => m.role)).toEqual(['system']); - // 6 turns, keep 2 → summarize the first 4 turns, keep the last 2. - expect(plan.toSummarize).toHaveLength(4); - expect(plan.toKeep).toHaveLength(2); - // Kept tail is the most recent turn's messages. - const lastUser = plan.toKeep[0]; - expect(lastUser.role).toBe('user'); - }); - - it('round-trips a prompt with no system message', () => { - const plan = planCompaction(plainTurns(3, false), { keepRecentTurns: 2 }); - - expect(plan.system).toEqual([]); - expect(plan.toSummarize).toHaveLength(4); - expect(plan.toKeep).toHaveLength(2); - }); - - it('never splits an assistant tool-call from its tool result', () => { - // turns: [user q1] [assistant+tool_call, tool_result] [user q2] [assistant a2] - const prompt: LanguageModelV3Prompt = [ - { role: 'user', content: [{ type: 'text', text: 'q1' }] }, - { - role: 'assistant', - content: [{ type: 'tool-call', toolCallId: 'c1', toolName: 'foo', input: { a: 1 } }], - }, - { - role: 'tool', - content: [ - { - type: 'tool-result', - toolCallId: 'c1', - toolName: 'foo', - output: { type: 'text', value: 'ok' }, - }, - ], - }, - { role: 'user', content: [{ type: 'text', text: 'q2' }] }, - { role: 'assistant', content: [{ type: 'text', text: 'a2' }] }, - ]; - - // keepRecentTurns: 3 → split at the start of the tool turn. A naive - // "keep last 3 messages" cut would orphan the tool result; turn-based does not. - const plan = planCompaction(prompt, { keepRecentTurns: 3 }); - - expect(plan.toSummarize.map((m) => m.role)).toEqual(['user']); // just q1 - // The assistant (tool-call) and its tool result stay together at the head of toKeep. - expect(plan.toKeep.map((m) => m.role)).toEqual(['assistant', 'tool', 'user', 'assistant']); - }); - - it('preserves a multi-block assistant message verbatim in toKeep', () => { - const assistantContent = [ - { type: 'text' as const, text: 'let me check' }, - { type: 'reasoning' as const, text: 'because reasons' }, - { type: 'tool-call' as const, toolCallId: 'c1', toolName: 'foo', input: { a: 1 } }, - ]; - const prompt: LanguageModelV3Prompt = [ - { role: 'user', content: [{ type: 'text', text: 'q' }] }, - { role: 'assistant', content: assistantContent }, - { - role: 'tool', - content: [ - { - type: 'tool-result', - toolCallId: 'c1', - toolName: 'foo', - output: { type: 'text', value: 'ok' }, - }, - ], - }, - ]; - - const plan = planCompaction(prompt, { keepRecentTurns: 99 }); - const keptAssistant = plan.toKeep.find((m) => m.role === 'assistant'); - expect(keptAssistant?.content).toEqual(assistantContent); - }); -}); - -describe('compactHistory', () => { - it('returns [...system, summary, ...toKeep] with the summary as a wrapped user message', async () => { - const prompt = plainTurns(4); // system + 8 messages (8 turns) - const result = await compactHistory(prompt, createSummarizerModel('Hello'), { - keepRecentTurns: 2, - }); - - expect(result[0]).toEqual(prompt[0]); // system preserved at front - const summaryMsg = result[1]; - expect(summaryMsg.role).toBe('user'); - const text = - summaryMsg.role === 'user' && summaryMsg.content[0].type === 'text' - ? summaryMsg.content[0].text - : ''; - expect(text).toContain('Hello'); // summary survived - expect(text).toContain('continued from a previous conversation'); // wrapper framing - // Compacted: system + summary + last 2 turns, shorter than the original. - expect(result.length).toBeLessThan(prompt.length); - expect(result.length).toBe(1 /* system */ + 1 /* summary */ + 2 /* kept turns */); - }); - - it('returns the prompt unchanged when nothing is old enough to compact', async () => { - const prompt = plainTurns(2); - const result = await compactHistory(prompt, createSummarizerModel(), { keepRecentTurns: 99 }); - expect(result).toBe(prompt); // same reference — untouched, no model call - }); - - it('returns the prompt unchanged when the summarizer yields no text', async () => { - const prompt = plainTurns(4); - const result = await compactHistory(prompt, createSummarizerModel(' '), { - keepRecentTurns: 1, - }); - expect(result).toBe(prompt); - }); - - it('produces a valid prompt when the kept tail starts with a tool turn', async () => { - // turns: [user q1] [assistant+tool_call, tool_result] [user q2] [assistant a2] - const prompt: LanguageModelV3Prompt = [ - { role: 'system', content: 'sys' }, - { role: 'user', content: [{ type: 'text', text: 'q1' }] }, - { - role: 'assistant', - content: [{ type: 'tool-call', toolCallId: 'c1', toolName: 'foo', input: { a: 1 } }], - }, - { - role: 'tool', - content: [ - { - type: 'tool-result', - toolCallId: 'c1', - toolName: 'foo', - output: { type: 'text', value: 'ok' }, - }, - ], - }, - { role: 'user', content: [{ type: 'text', text: 'q2' }] }, - { role: 'assistant', content: [{ type: 'text', text: 'a2' }] }, - ]; - - // 4 conversation turns; keep 3 → summarize just [user q1], keep from the tool - // turn onward. The summary user message must be followed by the assistant - // tool-call and its result, in order — a valid, non-orphaned prompt. - const result = await compactHistory(prompt, createSummarizerModel('S'), { keepRecentTurns: 3 }); - expect(result.map((m) => m.role)).toEqual([ - 'system', - 'user', - 'assistant', - 'tool', - 'user', - 'assistant', - ]); - }); -}); diff --git a/packages/ai-sdk-middleware/tests/compactionModelMessages.test.ts b/packages/ai-sdk-middleware/tests/compactionModelMessages.test.ts index 441eb51..ab9f394 100644 --- a/packages/ai-sdk-middleware/tests/compactionModelMessages.test.ts +++ b/packages/ai-sdk-middleware/tests/compactionModelMessages.test.ts @@ -1,9 +1,9 @@ import type { - LanguageModelV3, - LanguageModelV3CallOptions, - LanguageModelV3Content, - LanguageModelV3FinishReason, - LanguageModelV3GenerateResult, + LanguageModelV4, + LanguageModelV4CallOptions, + LanguageModelV4Content, + LanguageModelV4FinishReason, + LanguageModelV4GenerateResult, } from '@ai-sdk/provider'; import type { ModelMessage } from 'ai'; import { describe, expect, it } from 'vitest'; @@ -11,15 +11,15 @@ import { compactModelMessages, planCompactionModelMessages } from '../src/compac /** Minimal V3 model whose summarization call returns a fixed string. A V3 model * is a valid `LanguageModel`, so it exercises the widened model param too. */ -function createSummarizerModel(summaryText = 'SUMMARY'): LanguageModelV3 { +function createSummarizerModel(summaryText = 'SUMMARY'): LanguageModelV4 { return { - specificationVersion: 'v3', + specificationVersion: 'v4', provider: 'test', modelId: 'test-model', supportedUrls: {}, - async doGenerate(_opts: LanguageModelV3CallOptions): Promise { - const content: LanguageModelV3Content[] = [{ type: 'text', text: summaryText }]; - const finishReason: LanguageModelV3FinishReason = { unified: 'stop', raw: undefined }; + async doGenerate(_opts: LanguageModelV4CallOptions): Promise { + const content: LanguageModelV4Content[] = [{ type: 'text', text: summaryText }]; + const finishReason: LanguageModelV4FinishReason = { unified: 'stop', raw: undefined }; return { content, finishReason, diff --git a/packages/ai-sdk-middleware/tests/middleware.test.ts b/packages/ai-sdk-middleware/tests/middleware.test.ts index b65d8a3..05d69fa 100644 --- a/packages/ai-sdk-middleware/tests/middleware.test.ts +++ b/packages/ai-sdk-middleware/tests/middleware.test.ts @@ -1,31 +1,31 @@ import type { - LanguageModelV3, - LanguageModelV3CallOptions, - LanguageModelV3Content, - LanguageModelV3FinishReason, - LanguageModelV3GenerateResult, - LanguageModelV3Prompt, - LanguageModelV3StreamPart, - LanguageModelV3StreamResult, + LanguageModelV4, + LanguageModelV4CallOptions, + LanguageModelV4Content, + LanguageModelV4FinishReason, + LanguageModelV4GenerateResult, + LanguageModelV4Prompt, + LanguageModelV4StreamPart, + LanguageModelV4StreamResult, } from '@ai-sdk/provider'; import type { Skill } from '@context-chef/core'; import { describe, expect, it, vi } from 'vitest'; import { withContextChef } from '../src/index'; import { createMiddleware } from '../src/middleware'; -function createMockModel(options?: { inputTokens?: number; outputText?: string }): LanguageModelV3 { +function createMockModel(options?: { inputTokens?: number; outputText?: string }): LanguageModelV4 { const inputTokens = options?.inputTokens ?? 100; const outputText = options?.outputText ?? 'Hello'; - const model: LanguageModelV3 = { - specificationVersion: 'v3', + const model: LanguageModelV4 = { + specificationVersion: 'v4', provider: 'test', modelId: 'test-model', supportedUrls: {}, - async doGenerate(_opts: LanguageModelV3CallOptions): Promise { - const content: LanguageModelV3Content[] = [{ type: 'text', text: outputText }]; - const finishReason: LanguageModelV3FinishReason = { unified: 'stop', raw: undefined }; + async doGenerate(_opts: LanguageModelV4CallOptions): Promise { + const content: LanguageModelV4Content[] = [{ type: 'text', text: outputText }]; + const finishReason: LanguageModelV4FinishReason = { unified: 'stop', raw: undefined }; return { content, finishReason, @@ -47,8 +47,8 @@ function createMockModel(options?: { inputTokens?: number; outputText?: string } }; }, - async doStream(_opts: LanguageModelV3CallOptions) { - const parts: LanguageModelV3StreamPart[] = [ + async doStream(_opts: LanguageModelV4CallOptions) { + const parts: LanguageModelV4StreamPart[] = [ { type: 'text-start', id: '1' }, { type: 'text-delta', id: '1', delta: outputText }, { type: 'text-end', id: '1' }, @@ -67,7 +67,7 @@ function createMockModel(options?: { inputTokens?: number; outputText?: string } }, ]; - const stream = new ReadableStream({ + const stream = new ReadableStream({ start(controller) { for (const part of parts) { controller.enqueue(part); @@ -82,8 +82,8 @@ function createMockModel(options?: { inputTokens?: number; outputText?: string } return model; } -function makeConversation(messageCount: number): LanguageModelV3Prompt { - const prompt: LanguageModelV3Prompt = [{ role: 'system', content: 'You are helpful.' }]; +function makeConversation(messageCount: number): LanguageModelV4Prompt { + const prompt: LanguageModelV4Prompt = [{ role: 'system', content: 'You are helpful.' }]; for (let i = 0; i < messageCount; i++) { prompt.push({ role: 'user', @@ -109,11 +109,11 @@ describe('createMiddleware', () => { contextWindow: 1_000_000, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hello' }] }, ]; - const params: LanguageModelV3CallOptions = { prompt }; + const params: LanguageModelV4CallOptions = { prompt }; const result = await assertDefined( middleware.transformParams, 'transformParams', @@ -133,7 +133,7 @@ describe('createMiddleware', () => { }); const longOutput = 'x'.repeat(200); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Run command' }] }, { role: 'assistant', @@ -159,7 +159,7 @@ describe('createMiddleware', () => { }, ]; - const params: LanguageModelV3CallOptions = { prompt }; + const params: LanguageModelV4CallOptions = { prompt }; const result = await assertDefined( middleware.transformParams, 'transformParams', @@ -189,9 +189,9 @@ describe('createMiddleware', () => { const model = createMockModel({ inputTokens: 200 }); - const doGenerate = (): PromiseLike => + const doGenerate = (): PromiseLike => model.doGenerate({ prompt: [] }); - const doStream = (): PromiseLike => model.doStream({ prompt: [] }); + const doStream = (): PromiseLike => model.doStream({ prompt: [] }); const result = await assertDefined( middleware.wrapGenerate, @@ -215,9 +215,9 @@ describe('createMiddleware', () => { const model = createMockModel({ inputTokens: 300 }); - const doGenerate = (): PromiseLike => + const doGenerate = (): PromiseLike => model.doGenerate({ prompt: [] }); - const doStream = (): PromiseLike => model.doStream({ prompt: [] }); + const doStream = (): PromiseLike => model.doStream({ prompt: [] }); const streamResult = await assertDefined( middleware.wrapStream, @@ -230,7 +230,7 @@ describe('createMiddleware', () => { }); const reader = streamResult.stream.getReader(); - const chunks: LanguageModelV3StreamPart[] = []; + const chunks: LanguageModelV4StreamPart[] = []; while (true) { const { done, value } = await reader.read(); if (done) break; @@ -253,9 +253,9 @@ describe('createMiddleware', () => { const model = createMockModel({ inputTokens: 200 }); - const doGenerate = (): PromiseLike => + const doGenerate = (): PromiseLike => model.doGenerate({ prompt: [] }); - const doStream = (): PromiseLike => model.doStream({ prompt: [] }); + const doStream = (): PromiseLike => model.doStream({ prompt: [] }); await assertDefined( middleware.wrapGenerate, @@ -289,9 +289,9 @@ describe('createMiddleware', () => { const model = createMockModel({ inputTokens: 200 }); - const doGenerate = (): PromiseLike => + const doGenerate = (): PromiseLike => model.doGenerate({ prompt: [] }); - const doStream = (): PromiseLike => model.doStream({ prompt: [] }); + const doStream = (): PromiseLike => model.doStream({ prompt: [] }); // Feed token usage over the budget so compression fires on next transformParams await assertDefined( @@ -333,7 +333,7 @@ describe('compress opt-in (no budgeting configured)', () => { truncate: { threshold: 50, tailChars: 10 }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hello' }] }, ]; const result = await assertDefined( @@ -356,9 +356,9 @@ describe('compress opt-in (no budgeting configured)', () => { const middleware = createMiddleware({}); const model = createMockModel({ inputTokens: 42 }); - const doGenerate = (): PromiseLike => + const doGenerate = (): PromiseLike => model.doGenerate({ prompt: [] }); - const doStream = (): PromiseLike => model.doStream({ prompt: [] }); + const doStream = (): PromiseLike => model.doStream({ prompt: [] }); const result = await assertDefined( middleware.wrapGenerate, @@ -395,11 +395,11 @@ describe('compress opt-in (no budgeting configured)', () => { }); describe('withContextChef wrapper', () => { - it('returns a LanguageModelV3', () => { + it('returns a LanguageModelV4', () => { const model = createMockModel(); const wrapped = withContextChef(model, { contextWindow: 128_000 }); - expect(wrapped.specificationVersion).toBe('v3'); + expect(wrapped.specificationVersion).toBe('v4'); expect(wrapped.provider).toBeDefined(); expect(wrapped.modelId).toBeDefined(); expect(typeof wrapped.doGenerate).toBe('function'); @@ -417,7 +417,7 @@ describe('withContextChef wrapper', () => { prompt: [{ role: 'user', content: [{ type: 'text', text: 'Hi' }] }], }); - const textContent = result.content.find((c: LanguageModelV3Content) => c.type === 'text'); + const textContent = result.content.find((c: LanguageModelV4Content) => c.type === 'text'); expect(textContent).toBeDefined(); if (textContent?.type === 'text') { expect(textContent.text).toBe('Hello from wrapped model'); @@ -432,7 +432,7 @@ describe('compact', () => { compact: { toolCalls: 'all' }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Run command' }] }, { role: 'assistant', @@ -488,7 +488,7 @@ describe('compact', () => { compact: { reasoning: 'all' }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'assistant', content: [ @@ -516,20 +516,20 @@ describe('compact', () => { }); }); -describe('onBudgetExceeded', () => { +describe('onBeforeCompress', () => { it('calls hook when budget is exceeded', async () => { - const onBudgetExceeded = vi.fn().mockReturnValue(null); + const onBeforeCompress = vi.fn().mockReturnValue(null); const middleware = createMiddleware({ contextWindow: 100, - onBudgetExceeded, + onBeforeCompress, }); const model = createMockModel({ inputTokens: 200 }); // Feed high token usage to trigger budget exceeded - const doGenerate = (): PromiseLike => + const doGenerate = (): PromiseLike => model.doGenerate({ prompt: [] }); - const doStream = (): PromiseLike => model.doStream({ prompt: [] }); + const doStream = (): PromiseLike => model.doStream({ prompt: [] }); await assertDefined( middleware.wrapGenerate, 'wrapGenerate', @@ -550,7 +550,7 @@ describe('onBudgetExceeded', () => { model, }); - expect(onBudgetExceeded).toHaveBeenCalled(); + expect(onBeforeCompress).toHaveBeenCalled(); }); }); @@ -564,7 +564,7 @@ describe('dynamicState', () => { }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'What next?' }] }, ]; @@ -599,7 +599,7 @@ describe('dynamicState', () => { }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -628,7 +628,7 @@ describe('dynamicState', () => { }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hello' }] }, ]; @@ -664,7 +664,7 @@ describe('dynamicState', () => { }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -698,7 +698,7 @@ describe('dynamicState', () => { }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -727,7 +727,7 @@ describe('transformContext', () => { }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hello' }] }, ]; @@ -754,7 +754,7 @@ describe('transformContext', () => { }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hello' }] }, ]; @@ -775,7 +775,7 @@ describe('transformContext', () => { }); it('runs after dynamicState injection', async () => { - const transformContext = vi.fn((prompt: LanguageModelV3Prompt) => prompt); + const transformContext = vi.fn((prompt: LanguageModelV4Prompt) => prompt); const middleware = createMiddleware({ contextWindow: 1_000_000, dynamicState: { @@ -785,7 +785,7 @@ describe('transformContext', () => { transformContext, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -801,9 +801,9 @@ describe('transformContext', () => { expect(transformContext).toHaveBeenCalledTimes(1); const received = transformContext.mock.calls[0][0]; // transformContext should see the dynamic state system message - const systemMsgs = received.filter((m: LanguageModelV3Prompt[number]) => m.role === 'system'); + const systemMsgs = received.filter((m: LanguageModelV4Prompt[number]) => m.role === 'system'); const hasState = systemMsgs.some( - (m: LanguageModelV3Prompt[number]) => + (m: LanguageModelV4Prompt[number]) => m.role === 'system' && m.content.includes('true'), ); expect(hasState).toBe(true); @@ -823,7 +823,7 @@ describe('skill', () => { skill: planningSkill, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'system', content: 'You are helpful.' }, { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, { role: 'assistant', content: [{ type: 'text', text: 'Hello!' }] }, @@ -865,7 +865,7 @@ describe('skill', () => { skill: skillFn, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -888,7 +888,7 @@ describe('skill', () => { skill: () => null, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'system', content: 'You are helpful.' }, { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -909,7 +909,7 @@ describe('skill', () => { skill: () => active, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'system', content: 'You are helpful.' }, { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -934,7 +934,7 @@ describe('skill', () => { skill: async () => planningSkill, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -955,7 +955,7 @@ describe('skill', () => { contextWindow: 1_000_000, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'system', content: 'You are helpful.' }, { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -974,7 +974,7 @@ describe('skill', () => { skill: { name: 'noop', description: 'noop', instructions: '' }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'system', content: 'You are helpful.' }, { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -994,7 +994,7 @@ describe('skill', () => { skill: { name: 'blank', description: 'blank', instructions: ' \n\t ' }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'system', content: 'You are helpful.' }, { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -1017,7 +1017,7 @@ describe('skill', () => { }, }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'system', content: 'You are helpful.' }, { role: 'user', content: [{ type: 'text', text: 'Hi' }] }, ]; @@ -1049,7 +1049,7 @@ describe('skill', () => { clear: [{ target: 'tool-result', keepRecent: 1 }], }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'system', content: 'You are helpful.' }, { role: 'user', content: [{ type: 'text', text: 'Run both commands.' }] }, { @@ -1108,9 +1108,9 @@ describe('skill', () => { }); const model = createMockModel({ inputTokens: 200 }); - const doGenerate = (): PromiseLike => + const doGenerate = (): PromiseLike => model.doGenerate({ prompt: [] }); - const doStream = (): PromiseLike => model.doStream({ prompt: [] }); + const doStream = (): PromiseLike => model.doStream({ prompt: [] }); // Push token usage so the next transform triggers compression. await assertDefined( @@ -1135,7 +1135,7 @@ describe('skill', () => { /** Build a prompt with two tool-call/result exchanges plus a trailing user message. * Starts with a user message so ensureValidHistory doesn't insert a placeholder. */ -function makeToolPrompt(): LanguageModelV3Prompt { +function makeToolPrompt(): LanguageModelV4Prompt { return [ { role: 'system', content: 'You are helpful.' }, { role: 'user', content: [{ type: 'text', text: 'Run both commands.' }] }, @@ -1229,7 +1229,7 @@ describe('clear', () => { it('clear without tool-result targets does not inject the explainer', async () => { const middleware = createMiddleware({ clear: ['thinking'] }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'system', content: 'You are helpful.' }, { role: 'assistant', @@ -1258,7 +1258,7 @@ describe('clear', () => { expect(logger.warn).toHaveBeenCalledTimes(1); expect(logger.warn.mock.calls[0][0]).toContain("clear: ['thinking']"); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'assistant', content: [ @@ -1333,8 +1333,8 @@ describe('compress persistence warning', () => { // write-back) — the scenario the warning is meant to surface. async function runCycle( middleware: ReturnType, - model: LanguageModelV3, - prompt: LanguageModelV3Prompt, + model: LanguageModelV4, + prompt: LanguageModelV4Prompt, ): Promise { await assertDefined( middleware.wrapGenerate, diff --git a/packages/ai-sdk-middleware/tests/modelMessageAdapter.test.ts b/packages/ai-sdk-middleware/tests/modelMessageAdapter.test.ts index f1fbbd4..af83f77 100644 --- a/packages/ai-sdk-middleware/tests/modelMessageAdapter.test.ts +++ b/packages/ai-sdk-middleware/tests/modelMessageAdapter.test.ts @@ -29,6 +29,23 @@ describe('fromModelMessages', () => { ]); }); + it('records V4 tagged file data ({ type: "data" }) as the attachment signal', () => { + const messages: ModelMessage[] = [ + { + role: 'user', + content: [ + { type: 'text', text: 'tagged' }, + { type: 'file', data: { type: 'data', data: 'base64str' }, mediaType: 'image/png' }, + ], + }, + ]; + const ir = fromModelMessages(messages); + // Tagged inline string data yields the same presence signal as the bare shorthand. + expect(ir[0].attachments).toEqual([{ mediaType: 'image/png', data: 'base64str' }]); + // The real (tagged) payload round-trips verbatim via the pass-through field. + expect(toModelMessages(ir)).toEqual(messages); + }); + it('extracts assistant tool calls and reasoning', () => { const messages: ModelMessage[] = [ { role: 'user', content: 'use a tool' }, diff --git a/packages/ai-sdk-middleware/tests/summarizeMessages.test.ts b/packages/ai-sdk-middleware/tests/summarizeMessages.test.ts index e1b56f6..80b9502 100644 --- a/packages/ai-sdk-middleware/tests/summarizeMessages.test.ts +++ b/packages/ai-sdk-middleware/tests/summarizeMessages.test.ts @@ -1,11 +1,11 @@ import type { - LanguageModelV3, - LanguageModelV3CallOptions, - LanguageModelV3Content, - LanguageModelV3FinishReason, - LanguageModelV3GenerateResult, - LanguageModelV3Prompt, - LanguageModelV3StreamPart, + LanguageModelV4, + LanguageModelV4CallOptions, + LanguageModelV4Content, + LanguageModelV4FinishReason, + LanguageModelV4GenerateResult, + LanguageModelV4Prompt, + LanguageModelV4StreamPart, } from '@ai-sdk/provider'; import type { ModelMessage } from 'ai'; import { describe, expect, it, vi } from 'vitest'; @@ -15,15 +15,15 @@ import { summarizeModelMessages } from '../src/middleware'; /** Minimal V3 model whose summarization call returns a fixed string. A V3 model * is a valid `LanguageModel`, so it exercises the widened model param too. */ -function createSummarizerModel(summaryText = 'SUMMARY'): LanguageModelV3 { +function createSummarizerModel(summaryText = 'SUMMARY'): LanguageModelV4 { return { - specificationVersion: 'v3', + specificationVersion: 'v4', provider: 'test', modelId: 'test-model', supportedUrls: {}, - async doGenerate(_opts: LanguageModelV3CallOptions): Promise { - const content: LanguageModelV3Content[] = [{ type: 'text', text: summaryText }]; - const finishReason: LanguageModelV3FinishReason = { unified: 'stop', raw: undefined }; + async doGenerate(_opts: LanguageModelV4CallOptions): Promise { + const content: LanguageModelV4Content[] = [{ type: 'text', text: summaryText }]; + const finishReason: LanguageModelV4FinishReason = { unified: 'stop', raw: undefined }; return { content, finishReason, @@ -46,17 +46,17 @@ function createSummarizerModel(summaryText = 'SUMMARY'): LanguageModelV3 { }; } -function mockModel(text: string): LanguageModelV3 { - const model: LanguageModelV3 = { - specificationVersion: 'v3', +function mockModel(text: string): LanguageModelV4 { + const model: LanguageModelV4 = { + specificationVersion: 'v4', provider: 'test', modelId: 'test-model', supportedUrls: {}, doGenerate: vi.fn( - async (_opts: LanguageModelV3CallOptions): Promise => { - const content: LanguageModelV3Content[] = [{ type: 'text', text }]; - const finishReason: LanguageModelV3FinishReason = { unified: 'stop', raw: undefined }; + async (_opts: LanguageModelV4CallOptions): Promise => { + const content: LanguageModelV4Content[] = [{ type: 'text', text }]; + const finishReason: LanguageModelV4FinishReason = { unified: 'stop', raw: undefined }; return { content, finishReason, @@ -79,8 +79,8 @@ function mockModel(text: string): LanguageModelV3 { }, ), - async doStream(_opts: LanguageModelV3CallOptions) { - const parts: LanguageModelV3StreamPart[] = [ + async doStream(_opts: LanguageModelV4CallOptions) { + const parts: LanguageModelV4StreamPart[] = [ { type: 'text-start', id: '1' }, { type: 'text-delta', id: '1', delta: text }, { type: 'text-end', id: '1' }, @@ -99,7 +99,7 @@ function mockModel(text: string): LanguageModelV3 { }, ]; - const stream = new ReadableStream({ + const stream = new ReadableStream({ start(controller) { for (const part of parts) { controller.enqueue(part); @@ -148,7 +148,7 @@ describe('summarizeMessages', () => { // - tool message becomes a user message with "[Tool result(call_1): RAW_TOOL_OUTPUT]" // - assistant tool-call becomes "[Called tool: search({"q":"x"})]" text // - no raw role:'tool' message should reach the model - const toolPrompt: LanguageModelV3Prompt = [ + const toolPrompt: LanguageModelV4Prompt = [ { role: 'system', content: 'SYSTEM_MARKER_DROP_ME' }, { role: 'user', content: [{ type: 'text', text: 'do the thing' }] }, { diff --git a/packages/ai-sdk-middleware/tests/truncator.test.ts b/packages/ai-sdk-middleware/tests/truncator.test.ts index 82dc04c..150d50b 100644 --- a/packages/ai-sdk-middleware/tests/truncator.test.ts +++ b/packages/ai-sdk-middleware/tests/truncator.test.ts @@ -1,10 +1,10 @@ -import type { LanguageModelV3Prompt } from '@ai-sdk/provider'; +import type { LanguageModelV4Prompt } from '@ai-sdk/provider'; import type { VFSStorageAdapter } from '@context-chef/core'; import { describe, expect, it, vi } from 'vitest'; import { truncateToolResults } from '../src/truncator'; describe('truncateToolResults', () => { - const makeToolPrompt = (output: string): LanguageModelV3Prompt => [ + const makeToolPrompt = (output: string): LanguageModelV4Prompt => [ { role: 'tool', content: [ @@ -54,7 +54,7 @@ describe('truncateToolResults', () => { }); it('does not affect non-tool messages', async () => { - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'system', content: 'x'.repeat(200) }, { role: 'user', content: [{ type: 'text', text: 'x'.repeat(200) }] }, ]; @@ -64,7 +64,7 @@ describe('truncateToolResults', () => { it('handles json tool output', async () => { const bigJson = JSON.stringify({ data: 'x'.repeat(500) }); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'tool', content: [ @@ -252,7 +252,7 @@ describe('truncateToolResults', () => { it('filters per-part: preserves one tool while truncating another in the same message', async () => { const keepOutput = 'k'.repeat(500); const truncOutput = 't'.repeat(500); - const prompt: LanguageModelV3Prompt = [ + const prompt: LanguageModelV4Prompt = [ { role: 'tool', content: [ diff --git a/packages/core/README.md b/packages/core/README.md index 7b8572f..7ab3ef7 100644 --- a/packages/core/README.md +++ b/packages/core/README.md @@ -340,7 +340,7 @@ Contracts: `history` is a flat `Message[]` with any **system messages inline** ( `keepRecentTurns: 0` is **full compaction** (Claude Code style): the whole conversation collapses into `[...system, ]` with no verbatim tail. This is the simplest result to persist — there is no kept tail to reconcile against your store's unit boundaries. The trade-off is that no verbatim recent context survives, so steer the summary toward a structured handoff via `customCompressionInstructions`. Use a small `keepRecentTurns` instead when the in-flight turn should stay verbatim. -> **ai-sdk users:** prefer `compactHistory` / `planCompaction` from [`@context-chef/ai-sdk-middleware`](https://www.npmjs.com/package/@context-chef/ai-sdk-middleware) — they take an `AI SDK` prompt and a model directly, wiring the adapter and role-flattening for you. +> **ai-sdk users:** prefer `compactModelMessages` / `planCompactionModelMessages` from [`@context-chef/ai-sdk-middleware`](https://www.npmjs.com/package/@context-chef/ai-sdk-middleware) — they take an `AI SDK` prompt and a model directly, wiring the adapter and role-flattening for you. #### Compression circuit breaker diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 3a3ba14..e4c0ebd 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -25,14 +25,14 @@ importers: version: link:../core devDependencies: '@ai-sdk/provider': - specifier: ^3.0.8 - version: 3.0.8 + specifier: ^4.0.0 + version: 4.0.0 '@types/node': specifier: ^25.3.0 version: 25.3.0 ai: - specifier: ^6.0.140 - version: 6.0.140(zod@4.3.6) + specifier: ^7.0.0 + version: 7.0.0(zod@4.3.6) tsdown: specifier: ^0.20.3 version: 0.20.3(synckit@0.11.12)(typescript@5.9.3) @@ -98,21 +98,21 @@ importers: packages: - '@ai-sdk/gateway@3.0.82': - resolution: {integrity: sha512-ddB9FrkHZank1zyx13vypU0RrPjsXWj3NvlsrJ4yFQnrpR+xh48W4wO9ijndUueBsaADjlvMz2Ghv8yq5tZGCQ==} - engines: {node: '>=18'} + '@ai-sdk/gateway@4.0.0': + resolution: {integrity: sha512-rcKukspbM4h511ot2E8TsPl7rXjRK1zHKrMCP7w4+XF55UKqQHaDzo2kKbGv5rp8Bjb1yQatIHJZE1E2yrOOMw==} + engines: {node: '>=22'} peerDependencies: zod: ^3.25.76 || ^4.1.8 - '@ai-sdk/provider-utils@4.0.21': - resolution: {integrity: sha512-MtFUYI1/8mgDvRmaBDjbLJPFFrMG777AvSgyIFQtZHIMzm88R/12vYBBpnk7pfiWLFE1DSZzY4WDYzGbKAcmiw==} - engines: {node: '>=18'} + '@ai-sdk/provider-utils@5.0.0': + resolution: {integrity: sha512-zj66M02jc6ASYwIgWZowsooDUwaVngeNZQ3H10GwcPMZ+KR6gHMhcUuKl6tkai+JPXTKDyHY1pnszuxRtw2D4A==} + engines: {node: '>=22'} peerDependencies: zod: ^3.25.76 || ^4.1.8 - '@ai-sdk/provider@3.0.8': - resolution: {integrity: sha512-oGMAgGoQdBXbZqNG0Ze56CHjDZ1IDYOwGYxYjO5KLSlz5HiNQ9udIXsPZ61VWaHGZ5XW/jyjmr6t2xz2jGVwbQ==} - engines: {node: '>=18'} + '@ai-sdk/provider@4.0.0': + resolution: {integrity: sha512-fr9Gs89prDWiuox/T+kCA+i2cJkHpxU5S+tr4megjTzRC27ZsvFhwjU/+XrqqMbvBUlfmXxTOYWy8ng45dsjIg==} + engines: {node: '>=22'} '@anthropic-ai/sdk@0.78.0': resolution: {integrity: sha512-PzQhR715td/m1UaaN5hHXjYB8Gl2lF9UVhrrGrZeysiF6Rb74Wc9GCB8hzLdzmQtBd1qe89F9OptgB9Za1Ib5w==} @@ -170,28 +170,24 @@ packages: engines: {node: '>=14.21.3'} cpu: [arm64] os: [linux] - libc: [musl] '@biomejs/cli-linux-arm64@2.4.15': resolution: {integrity: sha512-owaAMZD/T4LrD0ELNCk0Km3qrRHuM0X6EAyVE1FSqGY0rbLoiDLrO4Us2tllm6cAeB2Ioa9C2C08NZPdr8+0Ug==} engines: {node: '>=14.21.3'} cpu: [arm64] os: [linux] - libc: [glibc] '@biomejs/cli-linux-x64-musl@2.4.15': resolution: {integrity: sha512-CNq/9W38SYSH023lfcQ4KKU8K0YX8T//FZUhcgtMMRABDojx5XsMV7jlweAvGSl389wJQB29Qo6Zb/a+jdvt+w==} engines: {node: '>=14.21.3'} cpu: [x64] os: [linux] - libc: [musl] '@biomejs/cli-linux-x64@2.4.15': resolution: {integrity: sha512-0jj7THz12GbUOLmMibktK6DZjqz2zV64KFxyBtcFTKPiiOIY0a7vns1elpO1dERvxpsZ5ik0oFfz0oGwFde1+g==} engines: {node: '>=14.21.3'} cpu: [x64] os: [linux] - libc: [glibc] '@biomejs/cli-win32-arm64@2.4.15': resolution: {integrity: sha512-ouhkYdlhp/1GghEJPdWwD/Vi3gQ1nFxuSpMolWsbq3Lsq3QUR4jl6UdhhscdCugKU5vOEuMiJhvKj66O0OCq+w==} @@ -534,28 +530,24 @@ packages: engines: {node: ^20.19.0 || >=22.12.0} cpu: [arm64] os: [linux] - libc: [glibc] '@rolldown/binding-linux-arm64-musl@1.0.0-rc.3': resolution: {integrity: sha512-Z03/wrqau9Bicfgb3Dbs6SYTHliELk2PM2LpG2nFd+cGupTMF5kanLEcj2vuuJLLhptNyS61rtk7SOZ+lPsTUA==} engines: {node: ^20.19.0 || >=22.12.0} cpu: [arm64] os: [linux] - libc: [musl] '@rolldown/binding-linux-x64-gnu@1.0.0-rc.3': resolution: {integrity: sha512-iSXXZsQp08CSilff/DCTFZHSVEpEwdicV3W8idHyrByrcsRDVh9sGC3sev6d8BygSGj3vt8GvUKBPCoyMA4tgQ==} engines: {node: ^20.19.0 || >=22.12.0} cpu: [x64] os: [linux] - libc: [glibc] '@rolldown/binding-linux-x64-musl@1.0.0-rc.3': resolution: {integrity: sha512-qaj+MFudtdCv9xZo9znFvkgoajLdc+vwf0Kz5N44g+LU5XMe+IsACgn3UG7uTRlCCvhMAGXm1XlpEA5bZBrOcw==} engines: {node: ^20.19.0 || >=22.12.0} cpu: [x64] os: [linux] - libc: [musl] '@rolldown/binding-openharmony-arm64@1.0.0-rc.3': resolution: {integrity: sha512-U662UnMETyjT65gFmG9ma+XziENrs7BBnENi/27swZPYagubfHRirXHG2oMl+pEax2WvO7Kb9gHZmMakpYqBHQ==} @@ -617,79 +609,66 @@ packages: resolution: {integrity: sha512-t4ONHboXi/3E0rT6OZl1pKbl2Vgxf9vJfWgmUoCEVQVxhW6Cw/c8I6hbbu7DAvgp82RKiH7TpLwxnJeKv2pbsw==} cpu: [arm] os: [linux] - libc: [glibc] '@rollup/rollup-linux-arm-musleabihf@4.59.0': resolution: {integrity: sha512-CikFT7aYPA2ufMD086cVORBYGHffBo4K8MQ4uPS/ZnY54GKj36i196u8U+aDVT2LX4eSMbyHtyOh7D7Zvk2VvA==} cpu: [arm] os: [linux] - libc: [musl] '@rollup/rollup-linux-arm64-gnu@4.59.0': resolution: {integrity: sha512-jYgUGk5aLd1nUb1CtQ8E+t5JhLc9x5WdBKew9ZgAXg7DBk0ZHErLHdXM24rfX+bKrFe+Xp5YuJo54I5HFjGDAA==} cpu: [arm64] os: [linux] - libc: [glibc] '@rollup/rollup-linux-arm64-musl@4.59.0': resolution: {integrity: sha512-peZRVEdnFWZ5Bh2KeumKG9ty7aCXzzEsHShOZEFiCQlDEepP1dpUl/SrUNXNg13UmZl+gzVDPsiCwnV1uI0RUA==} cpu: [arm64] os: [linux] - libc: [musl] '@rollup/rollup-linux-loong64-gnu@4.59.0': resolution: {integrity: sha512-gbUSW/97f7+r4gHy3Jlup8zDG190AuodsWnNiXErp9mT90iCy9NKKU0Xwx5k8VlRAIV2uU9CsMnEFg/xXaOfXg==} cpu: [loong64] os: [linux] - libc: [glibc] '@rollup/rollup-linux-loong64-musl@4.59.0': resolution: {integrity: sha512-yTRONe79E+o0FWFijasoTjtzG9EBedFXJMl888NBEDCDV9I2wGbFFfJQQe63OijbFCUZqxpHz1GzpbtSFikJ4Q==} cpu: [loong64] os: [linux] - libc: [musl] '@rollup/rollup-linux-ppc64-gnu@4.59.0': resolution: {integrity: sha512-sw1o3tfyk12k3OEpRddF68a1unZ5VCN7zoTNtSn2KndUE+ea3m3ROOKRCZxEpmT9nsGnogpFP9x6mnLTCaoLkA==} cpu: [ppc64] os: [linux] - libc: [glibc] '@rollup/rollup-linux-ppc64-musl@4.59.0': resolution: {integrity: sha512-+2kLtQ4xT3AiIxkzFVFXfsmlZiG5FXYW7ZyIIvGA7Bdeuh9Z0aN4hVyXS/G1E9bTP/vqszNIN/pUKCk/BTHsKA==} cpu: [ppc64] os: [linux] - libc: [musl] '@rollup/rollup-linux-riscv64-gnu@4.59.0': resolution: {integrity: sha512-NDYMpsXYJJaj+I7UdwIuHHNxXZ/b/N2hR15NyH3m2qAtb/hHPA4g4SuuvrdxetTdndfj9b1WOmy73kcPRoERUg==} cpu: [riscv64] os: [linux] - libc: [glibc] '@rollup/rollup-linux-riscv64-musl@4.59.0': resolution: {integrity: sha512-nLckB8WOqHIf1bhymk+oHxvM9D3tyPndZH8i8+35p/1YiVoVswPid2yLzgX7ZJP0KQvnkhM4H6QZ5m0LzbyIAg==} cpu: [riscv64] os: [linux] - libc: [musl] '@rollup/rollup-linux-s390x-gnu@4.59.0': resolution: {integrity: sha512-oF87Ie3uAIvORFBpwnCvUzdeYUqi2wY6jRFWJAy1qus/udHFYIkplYRW+wo+GRUP4sKzYdmE1Y3+rY5Gc4ZO+w==} cpu: [s390x] os: [linux] - libc: [glibc] '@rollup/rollup-linux-x64-gnu@4.59.0': resolution: {integrity: sha512-3AHmtQq/ppNuUspKAlvA8HtLybkDflkMuLK4DPo77DfthRb71V84/c4MlWJXixZz4uruIH4uaa07IqoAkG64fg==} cpu: [x64] os: [linux] - libc: [glibc] '@rollup/rollup-linux-x64-musl@4.59.0': resolution: {integrity: sha512-2UdiwS/9cTAx7qIUZB/fWtToJwvt0Vbo0zmnYt7ED35KPg13Q0ym1g442THLC7VyI6JfYTP4PiSOWyoMdV2/xg==} cpu: [x64] os: [linux] - libc: [musl] '@rollup/rollup-openbsd-x64@4.59.0': resolution: {integrity: sha512-M3bLRAVk6GOwFlPTIxVBSYKUaqfLrn8l0psKinkCFxl4lQvOSz8ZrKDz2gxcBwHFpci0B6rttydI4IpS4IS/jQ==} @@ -771,8 +750,8 @@ packages: '@types/node@25.3.0': resolution: {integrity: sha512-4K3bqJpXpqfg2XKGK9bpDTc6xO/xoUP/RBWS7AtRMug6zZFaRekiLzjVtAoZMquxoAbzBvy5nxQ7veS5eYzf8A==} - '@vercel/oidc@3.1.0': - resolution: {integrity: sha512-Fw28YZpRnA3cAHHDlkt7xQHiJ0fcL+NRcIqsocZQUSmbzeIKRpwttJjik5ZGanXP+vlA4SbTg+AbA3bP363l+w==} + '@vercel/oidc@3.2.0': + resolution: {integrity: sha512-UycprH3T6n3jH0k44NHMa7pnFHGu/N05MjojYr+Mc6I7obkoLIJujSWwin1pCvdy/eOxrI/l3uDLQsmcrOb4ug==} engines: {node: '>= 20'} '@vitest/expect@4.0.18': @@ -804,6 +783,9 @@ packages: '@vitest/utils@4.0.18': resolution: {integrity: sha512-msMRKLMVLWygpK3u2Hybgi4MNjcYJvwTb0Ru09+fOyCXIgT5raYP041DRRdiJiI3k/2U6SEbAETB3YtBrUkCFA==} + '@workflow/serde@4.1.0': + resolution: {integrity: sha512-pav4F2BoirECWR7Nf1TKt+2eETcBj7jj4cBefQ8VXQCA6NPkaKeLfj/zMgi+3zYV5ZIBT4GuUiphsj0/b9hPQQ==} + acorn-walk@8.3.5: resolution: {integrity: sha512-HEHNfbars9v4pgpW6SO1KSPkfoS0xVOM/9UzkJltjlsHZmJasxg8aXkuZa7SMf8vKGIBhpUsPluQSqhJFCqebw==} engines: {node: '>=0.4.0'} @@ -813,9 +795,9 @@ packages: engines: {node: '>=0.4.0'} hasBin: true - ai@6.0.140: - resolution: {integrity: sha512-+jf6fQDPZe+gQlzPoP5mzy+DdfYOpE0cgUm99U8OxVTIPv19gCDuNzlKTZSntcQzLbo6LFXyNhJdV7XTWQ+5vA==} - engines: {node: '>=18'} + ai@7.0.0: + resolution: {integrity: sha512-hncs+jamJh8r36K6G8xky7oF4Ai/RLU5TF85FMzI2vElyMJGGnLoHihpdmuDiuY2BsktDWHevKaJM1l0VcRLGw==} + engines: {node: '>=22'} peerDependencies: zod: ^3.25.76 || ^4.1.8 @@ -936,8 +918,8 @@ packages: estree-walker@3.0.3: resolution: {integrity: sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==} - eventsource-parser@3.0.6: - resolution: {integrity: sha512-Vo1ab+QXPzZ4tCa8SwIHJFaSzy4R6SHf7BY79rFBDf0idraZWAkYrDjDj8uWaSm3S2TK+hJ7/t1CEmZ7jXw+pg==} + eventsource-parser@3.1.0: + resolution: {integrity: sha512-kJezFj9YFAMLeORyi7aCLxLbD5/qWMQnoMVlVPyHIll7lgRJCc3JVln9Vgl9nwQi0YkMnhdGTMNn7CkRRAptMg==} engines: {node: '>=18.0.0'} expect-type@1.3.0: @@ -1498,21 +1480,22 @@ packages: snapshots: - '@ai-sdk/gateway@3.0.82(zod@4.3.6)': + '@ai-sdk/gateway@4.0.0(zod@4.3.6)': dependencies: - '@ai-sdk/provider': 3.0.8 - '@ai-sdk/provider-utils': 4.0.21(zod@4.3.6) - '@vercel/oidc': 3.1.0 + '@ai-sdk/provider': 4.0.0 + '@ai-sdk/provider-utils': 5.0.0(zod@4.3.6) + '@vercel/oidc': 3.2.0 zod: 4.3.6 - '@ai-sdk/provider-utils@4.0.21(zod@4.3.6)': + '@ai-sdk/provider-utils@5.0.0(zod@4.3.6)': dependencies: - '@ai-sdk/provider': 3.0.8 + '@ai-sdk/provider': 4.0.0 '@standard-schema/spec': 1.1.0 - eventsource-parser: 3.0.6 + '@workflow/serde': 4.1.0 + eventsource-parser: 3.1.0 zod: 4.3.6 - '@ai-sdk/provider@3.0.8': + '@ai-sdk/provider@4.0.0': dependencies: json-schema: 0.4.0 @@ -1900,7 +1883,8 @@ snapshots: '@nodelib/fs.scandir': 2.1.5 fastq: 1.20.1 - '@opentelemetry/api@1.9.0': {} + '@opentelemetry/api@1.9.0': + optional: true '@oxc-project/types@0.112.0': {} @@ -2073,7 +2057,7 @@ snapshots: dependencies: undici-types: 7.18.2 - '@vercel/oidc@3.1.0': {} + '@vercel/oidc@3.2.0': {} '@vitest/expect@4.0.18': dependencies: @@ -2114,18 +2098,19 @@ snapshots: '@vitest/pretty-format': 4.0.18 tinyrainbow: 3.0.3 + '@workflow/serde@4.1.0': {} + acorn-walk@8.3.5: dependencies: acorn: 8.16.0 acorn@8.16.0: {} - ai@6.0.140(zod@4.3.6): + ai@7.0.0(zod@4.3.6): dependencies: - '@ai-sdk/gateway': 3.0.82(zod@4.3.6) - '@ai-sdk/provider': 3.0.8 - '@ai-sdk/provider-utils': 4.0.21(zod@4.3.6) - '@opentelemetry/api': 1.9.0 + '@ai-sdk/gateway': 4.0.0(zod@4.3.6) + '@ai-sdk/provider': 4.0.0 + '@ai-sdk/provider-utils': 5.0.0(zod@4.3.6) zod: 4.3.6 ansi-colors@4.1.3: {} @@ -2236,7 +2221,7 @@ snapshots: dependencies: '@types/estree': 1.0.8 - eventsource-parser@3.0.6: {} + eventsource-parser@3.1.0: {} expect-type@1.3.0: {} diff --git a/skills/context-chef-middleware/SKILL.md b/skills/context-chef-middleware/SKILL.md index 6bedb37..9a67ed8 100644 --- a/skills/context-chef-middleware/SKILL.md +++ b/skills/context-chef-middleware/SKILL.md @@ -1,6 +1,6 @@ --- name: context-chef-middleware -description: "Helps developers integrate @context-chef/ai-sdk-middleware into their Vercel AI SDK (v6+) projects. Use this skill when the user wants to add transparent context management to an AI SDK app, wrap a model with automatic history compression, truncate large tool results, manage token budgets, or inject dynamic state into AI SDK prompts. Also trigger when the user mentions 'context-chef middleware', 'AI SDK middleware', 'ai-sdk context', or asks about compressing history / truncating tool results / managing tokens in a Vercel AI SDK project." +description: "Helps developers integrate @context-chef/ai-sdk-middleware into their Vercel AI SDK (v7+) projects. Use this skill when the user wants to add transparent context management to an AI SDK app, wrap a model with automatic history compression, truncate large tool results, manage token budgets, or inject dynamic state into AI SDK prompts. Also trigger when the user mentions 'context-chef middleware', 'AI SDK middleware', 'ai-sdk context', or asks about compressing history / truncating tool results / managing tokens in a Vercel AI SDK project." argument-hint: "[feature-focus]" allowed-tools: Read, Grep, Glob, Bash, Write, Edit --- @@ -16,7 +16,7 @@ The key selling point: **zero code changes** to existing `generateText` / `strea Before asking questions, silently inspect the project: ``` -1. package.json → confirm they use `ai` (v6+) and an AI SDK provider (@ai-sdk/openai, @ai-sdk/anthropic, @ai-sdk/google, etc.) +1. package.json → confirm they use `ai` (v7+) and an AI SDK provider (@ai-sdk/openai, @ai-sdk/anthropic, @ai-sdk/google, etc.) 2. Lock file → detect package manager (pnpm-lock.yaml / yarn.lock / package-lock.json / bun.lockb) 3. tsconfig.json → TypeScript or JavaScript? 4. Existing AI SDK usage → look for patterns like: @@ -40,7 +40,7 @@ Based on what you found, present a brief summary of their setup and ask which fe | Need to inject task state for LLM attention | Dynamic state as XML in prompt | `dynamicState` | | Want custom prompt manipulation (RAG, metadata) | Post-compression transform hook | `transformContext` | | Want to know when compression happens | Compression callback | `onCompress` | -| Need control over what happens at budget limit | Budget exceeded hook | `onBudgetExceeded` | +| Need control over what happens at budget limit | Budget exceeded hook | `onBeforeCompress` | If the developer is unsure, recommend starting with: **compression + truncation** — these solve the most common problems with minimal setup. @@ -55,7 +55,7 @@ yarn add @context-chef/ai-sdk-middleware bun add @context-chef/ai-sdk-middleware ``` -The middleware depends on `ai` (v6+) and `@ai-sdk/provider` (v3+) as peer dependencies — the developer should already have these. +The middleware depends on `ai` (v7+) and `@ai-sdk/provider` (v4+) as peer dependencies — the developer should already have these. ## Step 4: Generate integration code @@ -154,9 +154,9 @@ transformContext: (prompt) => { } ``` -**Budget exceeded hook (`onBudgetExceeded`):** +**Budget exceeded hook (`onBeforeCompress`):** ```typescript -onBudgetExceeded: (history, { currentTokens, limit }) => { +onBeforeCompress: (history, { currentTokens, limit }) => { // Return modified messages, or null to let default compression handle it console.log(`Budget exceeded: ${currentTokens}/${limit} tokens`); return null; @@ -240,7 +240,7 @@ const aiSdkPrompt = toAISDK(irMessages); After generating the code: -1. Verify `ai` (v6+) is in their dependencies — the middleware requires AI SDK v6 +1. Verify `ai` (v7+) is in their dependencies — the middleware requires AI SDK v7 2. Verify they have at least one `@ai-sdk/*` provider installed 3. Explain the processing pipeline briefly: - Truncate large tool results (if configured) @@ -255,7 +255,7 @@ After generating the code: - Don't create a new wrapped model per LLM call — reuse it across the conversation (it tracks token usage) - Don't manually call `reportTokenUsage()` — the middleware extracts it automatically from `generateText` / `streamText` responses - The `compress.model` should be a cheap, fast model (e.g. `gpt-4o-mini`, `claude-haiku`) — it's used for summarization, not the main task -- `withContextChef()` returns a standard `LanguageModelV3` — it works anywhere the original model works +- `withContextChef()` returns a standard `LanguageModelV4` — it works anywhere the original model works - If using `compact` with `truncate` together, `truncate` runs first (on the raw AI SDK prompt), then `compact` runs on the IR (after conversion) ## When to recommend @context-chef/core instead diff --git a/skills/context-chef-middleware/references/api-reference.md b/skills/context-chef-middleware/references/api-reference.md index 9136585..ac4b035 100644 --- a/skills/context-chef-middleware/references/api-reference.md +++ b/skills/context-chef-middleware/references/api-reference.md @@ -2,9 +2,9 @@ ## Exports -### `withContextChef(model, options): LanguageModelV3` +### `withContextChef(model, options): LanguageModelV4` -Wraps an AI SDK language model with context-chef middleware. Returns a standard `LanguageModelV3` that can be used anywhere the original model was used. +Wraps an AI SDK language model with context-chef middleware. Returns a standard `LanguageModelV4` that can be used anywhere the original model was used. ```typescript import { withContextChef } from '@context-chef/ai-sdk-middleware'; @@ -24,11 +24,11 @@ const model = wrapLanguageModel({ model: openai('gpt-4o'), middleware }); ### `fromAISDK(prompt): AISDKMessage[]` -Converts an AI SDK `LanguageModelV3Prompt` to context-chef `Message[]` IR. Original AI SDK content is stored in per-role fields for lossless round-trip. +Converts an AI SDK `LanguageModelV4Prompt` to context-chef `Message[]` IR. Original AI SDK content is stored in per-role fields for lossless round-trip. -### `toAISDK(messages): LanguageModelV3Prompt` +### `toAISDK(messages): LanguageModelV4Prompt` -Converts context-chef `Message[]` IR back to AI SDK `LanguageModelV3Prompt`. Uses original content when unmodified; falls back to constructing from IR fields when content was modified by Janitor. +Converts context-chef `Message[]` IR back to AI SDK `LanguageModelV4Prompt`. Uses original content when unmodified; falls back to constructing from IR fields when content was modified by Janitor. ### `summarizeMessages(prompt, model, opts?): Promise` @@ -54,9 +54,9 @@ The main configuration object passed to `withContextChef()` or `createMiddleware | `compact` | `CompactConfig` | No | Mechanical compaction before LLM compression | | `dynamicState` | `DynamicStateConfig` | No | Dynamic state injection into prompt | | `tokenizer` | `(msgs: unknown[]) => number` | No | Custom tokenizer for precise token counting | -| `onCompress` | `(summary: string, count: number, details: { compressedMessages: LanguageModelV3Prompt }) => void` | No | Hook called after compression occurs. `details.compressedMessages` is the compressed slice in AI SDK format. | -| `onBudgetExceeded` | `(history, tokenInfo) => Message[] \| null \| Promise<...>` | No | Hook called when token budget is exceeded | -| `transformContext` | `(prompt) => LanguageModelV3Prompt \| Promise<...>` | No | Transform prompt after compression, before model | +| `onCompress` | `(summary: string, count: number, details: { compressedMessages: LanguageModelV4Prompt }) => void` | No | Hook called after compression occurs. `details.compressedMessages` is the compressed slice in AI SDK format. | +| `onBeforeCompress` | `(history, tokenInfo) => Message[] \| null \| Promise<...>` | No | Hook called when token budget is exceeded | +| `transformContext` | `(prompt) => LanguageModelV4Prompt \| Promise<...>` | No | Transform prompt after compression, before model | | `logger` | `ChefLogger` | No | Sink for degradation warnings (storage/compaction); defaults to `console`. `ChefLogger = { warn(message: string, ...args: unknown[]): void }` | | `clear` | `ClearTarget[]` | No | Placeholder-style tool-result clearing, runs after compression, auto-injects an explainer. Only `'tool-result'` takes effect; `'thinking'` is a no-op that warns — use `compact` for reasoning. | @@ -66,7 +66,7 @@ The main configuration object passed to `withContextChef()` or `createMiddleware | Field | Type | Required | Description | |---|---|---|---| -| `model` | `LanguageModelV3` | Yes | A cheap model for summarization (e.g. `openai('gpt-4o-mini')`) | +| `model` | `LanguageModelV4` | Yes | A cheap model for summarization (e.g. `openai('gpt-4o-mini')`) | | `preserveRatio` | `number` | No | Ratio of context window to preserve for recent messages. Default: `0.8` | | `toolResultStubThreshold` | `number` | No | Replace tool-result content longer than this many chars with a one-line metadata stub (`[Tool name returned N chars; omitted before summarization]`) before the to-be-summarized history is sent to the compression model. Recent (preserved) tool results are untouched. Default: undefined (disabled). |