Skip to content

Conversation

@christian-bromann
Copy link
Member

Port of langchain-ai/langchain#34607

Problem

Fixes token inflation in SummarizationMiddleware that caused context window overflow during summarization.

Root cause: When formatting messages for the summary prompt, JSON.stringify(messages) was being used, which includes all metadata fields (usage_metadata, response_metadata, additional_kwargs, etc.). This caused the stringified representation to use ~2.5x more tokens than countTokensApproximately estimates.

Symptoms:

  • Summarization triggers at expected threshold based on countTokensApproximately
  • But JSON.stringify(messages) in the prompt uses significantly more tokens
  • Can result in context length exceeded errors

Solution

Use getBufferString() to format messages, which produces compact output:

Human: What's the weather?
AI: Let me check...[tool_calls]
Tool: 72°F and sunny

Instead of verbose JSON representation:

[
  {
    "type": "human",
    "content": "What's the weather?",
    "additional_kwargs": {},
    "response_metadata": {}
  },
  ...
]

Changes

  • @langchain/core: Updated getBufferString() to use message's text property for compact content extraction, and added tool_calls/function_call support for AI messages
  • langchain: Updated createSummary() in summarization middleware to use getBufferString() instead of JSON.stringify()
  • Updated tests to reflect new expected format

@changeset-bot
Copy link

changeset-bot bot commented Jan 6, 2026

🦋 Changeset detected

Latest commit: 894a78a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 11 packages
Name Type
@langchain/core Patch
langchain Patch
@langchain/standard-tests Patch
@langchain/anthropic Patch
@langchain/google-cloud-sql-pg Patch
@langchain/google-common Patch
@langchain/google-genai Patch
@langchain/google-webauth Patch
@langchain/google-vertexai-web Patch
@langchain/google-vertexai Patch
@langchain/google-gauth Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@christian-bromann christian-bromann force-pushed the cb/use-get-buffer-string branch from 4bf9ff4 to 914c136 Compare January 6, 2026 01:19
@christian-bromann christian-bromann merged commit c28d24a into main Jan 6, 2026
36 checks passed
@christian-bromann christian-bromann deleted the cb/use-get-buffer-string branch January 6, 2026 01:35
@github-actions github-actions bot mentioned this pull request Jan 6, 2026
FilipZmijewski pushed a commit to FilipZmijewski/langchainjs that referenced this pull request Jan 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants