Skip to content

Conversation

@amychisholm03
Copy link
Contributor

@amychisholm03 amychisholm03 commented Dec 4, 2025

Description

Adds tests for the @langchain/aws package, specifically testing the classes related to AWS Bedrock: ChatBedrockConverse and BedrockEmbeddings.

Fixes a bug where we were not properly handing AWS Bedrock streams when it was piped from another source and where the stream chunk type may differ.

How to Test

npm run versioned:major langchain
# to make sure we didn't break normal AWS Bedrock use
npm run versioned:major aws-sdk-v3

Related Issues

Closes #3489

@amychisholm03 amychisholm03 force-pushed the langchain-bedrock-tests branch 2 times, most recently from ec06c49 to a37f420 Compare December 5, 2025 17:56
@codecov
Copy link

codecov bot commented Dec 5, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.68%. Comparing base (e7bcff4) to head (ec14dc7).
⚠️ Report is 8 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff             @@
##             main    #3563       +/-   ##
===========================================
+ Coverage   81.30%   97.68%   +16.38%     
===========================================
  Files         420      434       +14     
  Lines       55460    56838     +1378     
  Branches        1        1               
===========================================
+ Hits        45090    55524    +10434     
+ Misses      10370     1314     -9056     
Flag Coverage Δ
integration-tests-cjs-20.x 73.58% <0.00%> (?)
integration-tests-cjs-22.x 73.62% <0.00%> (?)
integration-tests-cjs-24.x 74.35% <0.00%> (?)
integration-tests-esm-20.x 52.55% <0.00%> (?)
integration-tests-esm-22.x 52.60% <0.00%> (?)
integration-tests-esm-24.x 53.76% <0.00%> (?)
unit-tests-20.x 88.58% <0.00%> (?)
unit-tests-22.x 88.61% <0.00%> (?)
unit-tests-24.x 88.62% <0.00%> (?)
versioned-tests-20.x 80.99% <100.00%> (-0.31%) ⬇️
versioned-tests-22.x 80.99% <100.00%> (?)
versioned-tests-24.x 80.93% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@amychisholm03 amychisholm03 marked this pull request as ready for review December 5, 2025 19:01
Copy link
Member

@bizob2828 bizob2828 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haven't reviewed it all but this approach needs updated

@amychisholm03 amychisholm03 force-pushed the langchain-bedrock-tests branch 4 times, most recently from 81e9ed3 to 7e5ff78 Compare December 8, 2025 18:57
Copy link
Member

@bizob2828 bizob2828 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests are still failing for me:

stderr
node:fs:560
  return binding.open(
                 ^

Error: ENOENT: no such file or directory, open '../../langchain/node_modules/@aws-sdk/client-bedrock-runtime/package.json'
    at Object.openSync (node:fs:560:18)
    at Object.readFileSync (node:fs:444:35)
    at getAiResponseServer (/Users/revans/code/release/node-newrelic/test/versioned/aws-sdk-v3/common.js:206:8)
    at Object.<anonymous> (/Users/revans/code/release/node-newrelic/test/versioned/langchain/bedrock/runnables.test.js:30:32)
    at Module._compile (node:internal/modules/cjs/loader:1760:14)
    at Object..js (node:internal/modules/cjs/loader:1893:10)
    at Module.load (node:internal/modules/cjs/loader:1480:32)
    at Module._load (node:internal/modules/cjs/loader:1299:12)
    at TracingChannel.traceSync (node:diagnostics_channel:328:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:244:24) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '../../langchain/node_modules/@aws-sdk/client-bedrock-runtime/package.json'
}

you cannot pass in

@amychisholm03 amychisholm03 force-pushed the langchain-bedrock-tests branch from c3600b2 to 5083f1f Compare December 9, 2025 00:52
@bizob2828 bizob2828 changed the title test: @langchain/aws bedrock tests test: Add versioned tests for @langchain/aws Dec 9, 2025
bizob2828
bizob2828 previously approved these changes Dec 9, 2025
Copy link
Member

@bizob2828 bizob2828 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@amychisholm03 amychisholm03 force-pushed the langchain-bedrock-tests branch from 5083f1f to 0cbe0f3 Compare December 9, 2025 21:12
@amychisholm03 amychisholm03 force-pushed the langchain-bedrock-tests branch 2 times, most recently from ea960b4 to 313aadc Compare December 10, 2025 21:01
@bizob2828 bizob2828 changed the title test: Add versioned tests for @langchain/aws feat: Added support for @langchain/aws Dec 10, 2025
@amychisholm03 amychisholm03 force-pushed the langchain-bedrock-tests branch from 33e1e54 to 5bff6f2 Compare December 10, 2025 21:44
Copy link
Member

@bizob2828 bizob2828 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very close. the 2 runnables tests needs to templatize the prompt and pass in valid template when calling invoke or stream. also a todo is still there with invalid assertion description

Copy link
Member

@bizob2828 bizob2828 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good work on getting these tests in here and fixing our instrumentation to handle findings

@bizob2828 bizob2828 merged commit 56c4a78 into newrelic:main Dec 11, 2025
25 checks passed
@github-project-automation github-project-automation bot moved this from Needs PR Review to Done: Issues recently completed in Node.js Engineering Board Dec 11, 2025
@github-actions github-actions bot mentioned this pull request Dec 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done: Issues recently completed

Development

Successfully merging this pull request may close these issues.

Add versioned tests for @langchain/aws

2 participants