-
Notifications
You must be signed in to change notification settings - Fork 312
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
(chore)LLMObs: instrument bedrock runtime invoke model #5116
(chore)LLMObs: instrument bedrock runtime invoke model #5116
Conversation
Overall package sizeSelf size: 8.53 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.4.0 | 29.44 MB | 29.44 MB | | @datadog/native-appsec | 8.4.0 | 19.25 MB | 19.26 MB | | @datadog/native-iast-taint-tracking | 3.2.0 | 13.9 MB | 13.91 MB | | @datadog/pprof | 5.4.1 | 9.76 MB | 10.13 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.6.1 | 2.59 MB | 2.73 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.1.0 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | source-map | 0.7.4 | 226 kB | 226 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | ttl-set | 1.0.0 | 4.61 kB | 9.69 kB | | path-to-regexp | 0.1.12 | 6.6 kB | 6.6 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
BenchmarksBenchmark execution time: 2025-01-24 14:24:50 Comparing candidate commit 9107d4d in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 912 metrics, 21 unstable metrics. |
…-support' into yahya/llmobs-add-bedrock-runtime-support
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #5116 +/- ##
===========================================
- Coverage 95.02% 72.56% -22.47%
===========================================
Files 108 150 +42
Lines 3475 5277 +1802
===========================================
+ Hits 3302 3829 +527
- Misses 173 1448 +1275 ☔ View full report in Codecov by Sentry. |
…dex.js Co-authored-by: Sam Brenner <[email protected]>
…acing.js Co-authored-by: Sam Brenner <[email protected]>
…acing.js Co-authored-by: Sam Brenner <[email protected]>
…ils.js Co-authored-by: Sam Brenner <[email protected]>
Co-authored-by: Sam Brenner <[email protected]>
…ntime.spec.js Co-authored-by: Sam Brenner <[email protected]>
…-support' into yahya/llmobs-add-bedrock-runtime-support # Conflicts: # packages/datadog-instrumentations/src/aws-sdk.js
* Update file structure * join spaces * add top_k * add index for plugins * add llmobs index * refactor apm tracing * add llmobs instrumentation for bedrock * add llmobs instrumentation for bedrock * rename * Update file structure * conflict * conflict * add index for plugins * add llmobs index * add llmobs instrumentation for bedrock * lint change * Update packages/datadog-plugin-aws-sdk/src/services/bedrockruntime/index.js Co-authored-by: Sam Brenner <[email protected]> * Update packages/datadog-plugin-aws-sdk/src/services/bedrockruntime/tracing.js Co-authored-by: Sam Brenner <[email protected]> * Update packages/datadog-plugin-aws-sdk/src/services/bedrockruntime/tracing.js Co-authored-by: Sam Brenner <[email protected]> * Update packages/datadog-plugin-aws-sdk/src/services/bedrockruntime/utils.js Co-authored-by: Sam Brenner <[email protected]> * CODEOWNERS * CODEOWNERS * remove shouldSetChoiceId override * remove shouldSetChoiceId override * remove shouldSetChoiceId override * remove shouldSetChoiceId override * lint * Update packages/dd-trace/src/llmobs/plugins/bedrockruntime.js Co-authored-by: Sam Brenner <[email protected]> * Update packages/dd-trace/test/llmobs/plugins/bedrockruntime/bedrockruntime.spec.js Co-authored-by: Sam Brenner <[email protected]> * resolving review comments * rename span * Add test fixtures and reduce redundant code * Add test to ci workflow * Update file structure * conflict * conflict * add index for plugins * add llmobs index * add llmobs instrumentation for bedrock * lint change * Update packages/dd-trace/src/llmobs/plugins/bedrockruntime.js Co-authored-by: Sam Brenner <[email protected]> * Update packages/dd-trace/test/llmobs/plugins/bedrockruntime/bedrockruntime.spec.js Co-authored-by: Sam Brenner <[email protected]> * resolving review comments * rename span * Add test fixtures and reduce redundant code * Add test to ci workflow * rename plugin name to aws sdk * update ci yml * add refactor for split --------- Co-authored-by: Sam Brenner <[email protected]>
What does this PR do?
Adds LLM Obs support for Bedrock InvokeModel method (only llm calls and not the embed ones)
Supports invocation for the following model providers :
Disclaimer : This PR doesn't cover the token counts for the bedrock model calls
Motivation
Plugin Checklist
Additional Notes