Skip to content

Conversation

JamesMustafa
Copy link

@JamesMustafa JamesMustafa commented Sep 20, 2025

Description of the feature

  • Updated ChatOllamaInput - thinking property with the recent additions in Ollama Thinking. More specifically, thinking intensity (low,medium,high) was added. This change is already present in LangChain python library.

  • Separated thinking content from the actual answer content. This was creating a big confusion for me initially, as I couldn't distinguish between thoughts/actual response. Ollama API provides message.content and message.thinking attributes which were already consumed by LangChain JS. However they were combined into one token, which lead to the unified final content.

const token = this.think
  ? responseMessage.thinking ?? responseMessage.content ?? ""
  : responseMessage.content ?? "";

In this PR, thinking_content attribute is added into additional_kwargs which isolates the thinking content only. This attribute is available everytime thinking is enabled. The content attribute now holds the actual answer only.

Results

Attaching an example AIMessage output payload I got while testing with local Ollama gpt-oss:20b model:

{
  "AIMessage": {
    "content": "There are **3** occurrences of the letter “r” in the word *strawberry*.",
    "additional_kwargs": {
      "thinking_content": "The user asks: \"how many r in the word strawberry?\" They want the count of letter 'r' in the word \"strawberry\". The word \"strawberry\" has letters: s t r a w b e r y. Count r's: there are 3 r's? Let's check: s(1), t(2), r(3), a(4), w(5), b(6), e(7), r(8), r(9), y(10). So r appears at positions 3, 8, 9. That's 3 r's. So answer: 3. But maybe they want explanation. The user just asked \"how many r in the word strawberry?\" So answer: 3. Provide a short answer."
    },
    "response_metadata": {
      "model": "gpt-oss:20b",
      "created_at": "2025-09-20T10:11:59.4606692Z",
      "done": true,
      "done_reason": "stop",
      "total_duration": 5971396100,
      "load_duration": 4384213800,
      "prompt_eval_count": 121,
      "prompt_eval_duration": 234847800,
      "eval_count": 191,
      "eval_duration": 1329256500
    },
    "tool_calls": [],
    "invalid_tool_calls": [],
    "usage_metadata": {
      "input_tokens": 121,
      "output_tokens": 191,
      "total_tokens": 312
    }
  }
}

Tests

Comprehensive integration tests for this specific feature/s added

Docs

TODO...

Copy link

changeset-bot bot commented Sep 20, 2025

⚠️ No Changeset found

Latest commit: 5bbc81e

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link

vercel bot commented Sep 20, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
langchainjs-docs Ready Ready Preview Sep 20, 2025 11:23am
1 Skipped Deployment
Project Deployment Preview Comments Updated (UTC)
langchainjs-api-refs Ignored Ignored Sep 20, 2025 11:23am

Comment on lines +95 to +102
// Quick test that string values work in practice
const res = await ollamaHigh.invoke([
new HumanMessage({ content: "How many r in the word strawberry?" })
]);

expect(res).toBeDefined();
expect(typeof res.content).toBe("string");
expect(res.content.length).toBeGreaterThan(0);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No reason to use the model if we just do type checks. In v1 branch we have migrated to Vitest which allows to create type tests.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @christian-bromann and thanks a lot for your review and feedback! :))

Can you please clarify what would be the best approach here in your opinion? As I already do test model invocation in other test (test string thinking parameter '$thinkLevel'), I removed it from this one, as you suggested.

I read a bit about Vitest, and I guess we can do something like that here:

test("test type safety for thinking parameter values", () => {
  expectTypeOf<ChatOllamaInput['think']>().toEqualTypeOf<boolean | ThinkingIntensity | undefined>()

  expectTypeOf<'high'>().toExtend<ThinkingIntensity>()
  expectTypeOf<'medium'>().toExtend<ThinkingIntensity>()
  expectTypeOf<'low'>().toExtend<ThinkingIntensity>()

  expectTypeOf<{ think: ThinkingIntensity }>().toExtend<Partial<ChatOllamaInput>>()
  expectTypeOf<{ think: boolean }>().toExtend<Partial<ChatOllamaInput>>()
});

However, Vitest is still not present inside the package.json file of langchain-ollama package. I see that it's already used in some of your internal libraries and also you plan to migrate to it, but what would be the best option currently? On one hand, I am not sure if adding vitest in the package.json would do any harm, on the other hand, if I try to do some workaround for type checking with Jest, this will create a technical debt for you in the future. What do you suggest?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants