-
Notifications
You must be signed in to change notification settings - Fork 43
Open
Description
Problem
When using @langwatch/scenario programmatically from a server (e.g., running scenarios on-demand from an API), the SDK forces callers to use environment variables for LangWatch configuration:
// Current: Must mutate process.env before each run
process.env.LANGWATCH_API_KEY = projectApiKey;
process.env.LANGWATCH_ENDPOINT = endpoint;
await ScenarioRunner.run({ ... });This creates race conditions when running multiple scenarios concurrently for different projects, forcing the caller to implement a mutex:
Root Cause
run()injavascript/src/runner/run.tsreads fromgetEnv()which parsesprocess.envEventBusis instantiated with these env-derived valuessetup.tsinitializes observability at module load time using env vars
Proposed Solution
Add a langwatch field to ScenarioConfig that takes precedence over environment variables:
interface LangWatchConfig {
apiKey?: string;
endpoint?: string;
}
interface ScenarioConfig {
// ... existing fields
/**
* LangWatch reporting configuration.
* Takes precedence over LANGWATCH_API_KEY and LANGWATCH_ENDPOINT env vars.
*/
langwatch?: LangWatchConfig;
}Implementation Steps
-
Add
langwatchfield toScenarioConfig(javascript/src/domain/scenarios/index.ts) -
Modify
run()(javascript/src/runner/run.ts):const envConfig = getEnv(); const langwatchConfig = { endpoint: cfg.langwatch?.endpoint ?? envConfig.LANGWATCH_ENDPOINT, apiKey: cfg.langwatch?.apiKey ?? envConfig.LANGWATCH_API_KEY, }; eventBus = new EventBus(langwatchConfig);
-
Handle observability setup (
javascript/src/tracing/setup.ts):- Option A: Lazy initialization per run (breaking change)
- Option B: Accept that observability uses env vars at load time, but event reporting uses per-run config (pragmatic)
- Option C: Create a
configureObservability()function that can be called beforerun()
Consumer Example (After Fix)
// No mutex needed - each run is self-contained
await ScenarioRunner.run({
name: 'Test',
description: 'Scenario',
agents: [...],
langwatch: {
apiKey: project.apiKey,
endpoint: 'https://app.langwatch.ai',
},
});Context
This is blocking proper multi-tenant scenario execution in the LangWatch platform:
Labels
- enhancement
- sdk-js
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
Backlog