Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(weave): OpenAI Agents SDK integration #3882

Merged
merged 16 commits into from
Mar 20, 2025
Merged

feat(weave): OpenAI Agents SDK integration #3882

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
718066c
test
andrewtruong Mar 12, 2025
50e7b9b
test
andrewtruong Mar 17, 2025
b188b06
test
andrewtruong Mar 17, 2025
20097fb
test
andrewtruong Mar 17, 2025
0fee87a
test
andrewtruong Mar 17, 2025
c3834ed
test
andrewtruong Mar 17, 2025
e67c287
test
andrewtruong Mar 17, 2025
afda46f
test
andrewtruong Mar 17, 2025
6bae11b
test
andrewtruong Mar 18, 2025
4f83679
test
andrewtruong Mar 18, 2025
8264eff
test
andrewtruong Mar 18, 2025
05dfde8
test
andrewtruong Mar 18, 2025
7ae6629
test
andrewtruong Mar 18, 2025
4b1c850
test
andrewtruong Mar 19, 2025
8c9289a
test
andrewtruong Mar 20, 2025
be56dbe
Merge branch 'master' into andrew/oai3
andrewtruong Mar 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 13 additions & 7 deletions noxfile.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
import os

import nox

nox.options.default_venv_backend = "uv"
Expand Down Expand Up @@ -50,6 +52,7 @@ def lint(session):
"mistral1",
"notdiamond",
"openai",
"openai_agents",
"vertexai",
"bedrock",
"scorers",
Expand Down Expand Up @@ -77,20 +80,23 @@ def tests(session, shard):
}
# Add the GOOGLE_API_KEY environment variable for the "google" shard
if shard == "google_ai_studio":
env["GOOGLE_API_KEY"] = session.env.get("GOOGLE_API_KEY")
env["GOOGLE_API_KEY"] = os.getenv("GOOGLE_API_KEY", "MISSING")

# Add the NVIDIA_API_KEY environment variable for the "langchain_nvidia_ai_endpoints" shard
if shard == "langchain_nvidia_ai_endpoints":
env["NVIDIA_API_KEY"] = session.env.get("NVIDIA_API_KEY")
env["NVIDIA_API_KEY"] = os.getenv("NVIDIA_API_KEY", "MISSING")

# we are doing some integration test in test_llm_integrations.py that requires
# setting some environment variables for the LLM providers
if shard == "scorers":
env["GOOGLE_API_KEY"] = session.env.get("GOOGLE_API_KEY")
env["GEMINI_API_KEY"] = session.env.get("GEMINI_API_KEY")
env["ANTHROPIC_API_KEY"] = session.env.get("ANTHROPIC_API_KEY")
env["MISTRAL_API_KEY"] = session.env.get("MISTRAL_API_KEY")
env["OPENAI_API_KEY"] = session.env.get("OPENAI_API_KEY")
env["GOOGLE_API_KEY"] = os.getenv("GOOGLE_API_KEY", "MISSING")
env["GEMINI_API_KEY"] = os.getenv("GEMINI_API_KEY", "MISSING")
env["ANTHROPIC_API_KEY"] = os.getenv("ANTHROPIC_API_KEY", "MISSING")
env["MISTRAL_API_KEY"] = os.getenv("MISTRAL_API_KEY", "MISSING")
env["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY", "MISSING")

if shard == "openai_agents":
env["OPENAI_API_KEY"] = os.getenv("OPENAI_API_KEY", "MISSING")

default_test_dirs = [f"integrations/{shard}/"]
test_dirs_dict = {
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ scorers = [
]
notdiamond = ["notdiamond>=0.3.21", "litellm<=1.49.1"]
openai = ["openai>=1.0.0"]
openai_agents = ["openai-agents>=0.0.4"]
pandas-test = ["pandas>=2.2.3"]
presidio = ["presidio-analyzer==2.2.357", "presidio-anonymizer==2.2.357"]
modal = ["modal", "python-dotenv"]
Expand Down
Empty file added tests/integrations/openai_agents/__init__.py
View file
Open in desktop
Empty file.
94 changes: 94 additions & 0 deletions ...ntegrations/openai_agents/cassettes/openai_agents_test/test_openai_agents_quickstart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
interactions:
- request:
body: '{"input":[{"content":"Write a haiku about recursion in programming.","role":"user"}],"model":"gpt-4o","include":[],"instructions":"You
are a helpful assistant","stream":false,"tools":[]}'
headers:
accept:
- application/json
accept-encoding:
- gzip, deflate, zstd
connection:
- keep-alive
content-length:
- '186'
content-type:
- application/json
host:
- api.openai.com
user-agent:
- Agents/Python 0.0.0
x-stainless-arch:
- arm64
x-stainless-async:
- async:asyncio
x-stainless-lang:
- python
x-stainless-os:
- MacOS
x-stainless-package-version:
- 1.66.3
x-stainless-read-timeout:
- '600'
x-stainless-retry-count:
- '0'
x-stainless-runtime:
- CPython
x-stainless-runtime-version:
- 3.12.9
method: POST
uri: https://api.openai.com/v1/responses
response:
body:
string: !!binary |
H4sIAAAAAAAAA3RUwW7bMAy95ysInZvCdhzbyXWn3XYdusJgbDpVJ4uGRA0tivz7YNlx4q29BBGf
3iP5KPpjA6B0q46gHPmhLsq2orZLduk+r9JDVp3yKj/sT1le7rNsVyZlUuT7MjkUHVGbkXoYBfj0
So1cRdj6Od44QqG2xhFLyzzLisNhV0TMC0rwI6fhfjAk1E4kco6dOoINxsSAttcbdUuC2vg16sWF
RjTbqPaTA6AjQHghM3TBAHqvvaCVSb/Ht5qDDEFq4d9kV2o9t2RGmfMg25y3WZLl26TaJsXcaSSq
IzxtAAA+4i+AkveBRlpP3uN5aj8Ck7e9Py/W4h6r0dpTmaRpkVHRpdV+t6NPrY0aX1kVQccmZl53
GaGGrZC9VXtf8arqqx30dmNPF8bIEdQ3bgkaNMaDMGjxZLoHgF/2u+201UJgmAcP2oLXhmxDEf2B
IuSsB+rJnQkaQ+ge1ynQWha8ju/pecEu878pMp6e4wgGdGgMmVqYTR2LUkcQF2iCHf3RHHx9fYp1
HMEyYUfo2Wp7VsfZDEVdx07uLgGoM1lyKFT70Pfo3md0A3CZny87us8r1A8jI8Rw+pjM0WjgnKhj
1+PtfDeBeG9pdEox9ffCupkGHITVAkxmzcehHu5zumCb6OhIa7XHk7kuV4ivcylI29Ua5LuH/+N3
S7eU3WDzQu2NmEylz+x/tystPwM+012G85W0sKC5gUWyuBU8rb4aPQm2KDjKXzaXvwAAAP//AwBA
W3tF6QQAAA==
headers:
CF-RAY:
- 9221c6bccef336b3-YYZ
Connection:
- keep-alive
Content-Encoding:
- gzip
Content-Type:
- application/json
Date:
- Tue, 18 Mar 2025 03:52:16 GMT
Server:
- cloudflare
Set-Cookie:
- __cf_bm=Tqnkbc7A4PgF5SM.gzWaijlDAAnDOk9Zieo6zRFmpd4-1742269936-1.0.1.1-9U42TycTKwZ6d4ISt9Qix_e2j6Latc3s8rPzobIIvSxaXqbrj.2XcpXF2GDI88LQyM0sRkP63Fw5_2JqEPAcJqt3OQPH5IESe3_.Iu0IofY;
path=/; expires=Tue, 18-Mar-25 04:22:16 GMT; domain=.api.openai.com; HttpOnly;
Secure; SameSite=None
- _cfuvid=KqFFy64fH2k1v0MoM0hYELHA_M66ltXaaJFWGlP2Kng-1742269936949-0.0.1.1-604800000;
path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None
Transfer-Encoding:
- chunked
X-Content-Type-Options:
- nosniff
alt-svc:
- h3=":443"; ma=86400
cf-cache-status:
- DYNAMIC
openai-organization:
- wandb
openai-processing-ms:
- '728'
openai-version:
- '2020-10-01'
strict-transport-security:
- max-age=31536000; includeSubDomains; preload
x-request-id:
- req_f9ec95d382e76a8e0ef8eb42aa207c3b
status:
code: 200
message: OK
version: 1
Comment on lines +1 to +94
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

VCR cassette contains sensitive information about the request structure.

The YAML cassette for the test_openai_agents_quickstart test contains detailed information about the OpenAI API interaction. While this is necessary for testing, there are some considerations:

  1. The file contains user-agent strings and package version information (line 19, 29)
  2. The file shows runtime version information (line 37)
  3. The file includes organization identifiers (line 82)

Consider adding more header filtering in the VCR configuration to avoid leaking potentially sensitive information.

🏁 Script executed:

#!/bin/bash
# Check if the VCR configuration in the test file has appropriate header filtering

grep -A5 "mark.vcr" tests/integrations/openai_agents/openai_agents_test.py

Length of output: 535

Sensitive API Request Information Exposed in Test Cassette

The VCR cassette for test_openai_agents_quickstart still contains sensitive details recorded from the actual API request. While it's expected to capture some request data for testing, the current VCR configuration only filters the "authorization" header. This leaves other sensitive headers—such as the user-agent string, package version (lines 19 and 29), runtime version (line 37), and organization identifiers (line 82)—unfiltered and exposed in the cassette.

Actionable Recommendations:

  • Extend Header Filtering: Update the VCR configuration (in tests/integrations/openai_agents/openai_agents_test.py) to include additional headers, such as:
    • "user-agent"
    • "x-stainless-package-version"
    • "x-stainless-runtime"
    • Headers carrying organization identifiers (or any other sensitive metadata)
  • Double-check Captured Data: Ensure that the cassette does not inadvertently capture other sensitive request details that are not required for test reproducibility.

Loading