-
Notifications
You must be signed in to change notification settings - Fork 798
Add OpenAI embeddings instrumentation #3461
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thank you!!!
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Outdated
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Outdated
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Outdated
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Outdated
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Outdated
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Outdated
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Outdated
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Outdated
Show resolved
Hide resolved
instrumentation-genai/opentelemetry-instrumentation-openai-v2/CHANGELOG.md
Outdated
Show resolved
Hide resolved
instrumentation-genai/opentelemetry-instrumentation-openai-v2/examples/embeddings/README.rst
Outdated
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/utils.py
Outdated
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Outdated
Show resolved
Hide resolved
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_async_embeddings.py
Outdated
Show resolved
Hide resolved
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_async_embeddings.py
Outdated
Show resolved
Hide resolved
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_embeddings.py
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/utils.py
Outdated
Show resolved
Hide resolved
Hey - just wondering what's the status of this PR as interested in having embeddings instrumented here. If there's anything I can do to help on it, let me know. |
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Outdated
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Show resolved
Hide resolved
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_async_embeddings.py
Outdated
Show resolved
Hide resolved
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_async_embeddings.py
Outdated
Show resolved
Hide resolved
...mentation-genai/opentelemetry-instrumentation-openai-v2/examples/embeddings/requirements.txt
Outdated
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Show resolved
Hide resolved
@pmcollins @xrmx @lmolkova Thank You! I'm working on it and wil have updates soon. |
@xrmx can you take another pass? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds comprehensive instrumentation for OpenAI's embeddings API to the existing OpenTelemetry instrumentation suite. The implementation provides automatic tracing and metrics collection for both synchronous and asynchronous embedding operations following OpenTelemetry semantic conventions.
- Adds support for tracing OpenAI embeddings API calls with proper span attributes and metrics
- Introduces new custom attributes for embedding-specific metadata (dimensions and encoding format)
- Includes comprehensive test coverage with both unit and integration tests
Reviewed Changes
Copilot reviewed 26 out of 28 changed files in this pull request and generated 4 comments.
Show a summary per file
File | Description |
---|---|
src/opentelemetry/instrumentation/openai_v2/__init__.py |
Adds instrumentation hooks for embeddings endpoints |
src/opentelemetry/instrumentation/openai_v2/patch.py |
Implements tracing logic for sync/async embeddings operations |
src/opentelemetry/instrumentation/openai_v2/utils.py |
Adds embeddings-specific request attribute handling |
tests/test_embeddings.py |
Comprehensive test suite for synchronous embeddings API |
tests/test_async_embeddings.py |
Test coverage for asynchronous embeddings functionality |
tests/test_utils.py |
Shared test utilities for consistent assertion patterns |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/utils.py
Show resolved
Hide resolved
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_embeddings.py
Show resolved
Hide resolved
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/test_async_embeddings.py
Show resolved
Hide resolved
...opentelemetry-instrumentation-openai-v2/src/opentelemetry/instrumentation/openai_v2/patch.py
Show resolved
Hide resolved
opentelemetry-sdk~=1.36.0 | ||
opentelemetry-exporter-otlp-proto-grpc~=1.36.0 | ||
opentelemetry-distro~=0.57b0 | ||
opentelemetry-instrumentation-openai-v2~=2.1b0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
opentelemetry-instrumentation-openai-v2~=2.1b0 | |
opentelemetry-instrumentation-openai-v2~=2.2b0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple of comments but LGTM.
attributes["gen_ai.embeddings.dimension.count"] = dimensions | ||
|
||
# Add encoding format if specified | ||
if "encoding_format" in kwargs: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to guard against instances of openai.NotGiven
(available since 1.13.4) and so add one test for sync and one for async checking that for the encoding_format
parameter . We have another PR adding guards for chat completions #3760
] = response_format | ||
|
||
service_tier = kwargs.get("service_tier") | ||
attributes[GenAIAttributes.GEN_AI_OPENAI_RESPONSE_SERVICE_TIER] = ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess this should be GenAIAttributes.GEN_AI_OPENAI_REQUEST_SERVICE_TIER
, care to fix while you are at it? Don't want to create conflicts on another PR :) With a separate entry in the changelog though. If you don't want to do that in this PR we can do it later, no worries.
Also I would ignore copilot comments. |
Description
This PR adds instrumentation for OpenAI's embeddings API in the GenAI instrumentation suite. The implementation follows the OpenTelemetry semantic conventions for generative AI systems and provides automatic instrumentation for the OpenAI Python client when using embeddings functionality.
The implementation captures important metadata about embedding operations including model, dimensions, and relevant timing information while respecting sensitive data handling practices.
ai.embedding.dimensions
- Number of dimensions in the embedding vectorsai.embedding.encoding_format
- The encoding format of the embedding vectors response (base64 or float)embeddings
Type of change
How Has This Been Tested?
Does This PR Require a Core Repo Change?
Checklist:
See contributing.md for styleguide, changelog guidelines, and more.