Skip to content

feat: Add programmatic langwatch config to ScenarioConfig (remove env var dependency) #203

@drewdrewthis

Description

@drewdrewthis

Problem

When using @langwatch/scenario programmatically from a server (e.g., running scenarios on-demand from an API), the SDK forces callers to use environment variables for LangWatch configuration:

// Current: Must mutate process.env before each run
process.env.LANGWATCH_API_KEY = projectApiKey;
process.env.LANGWATCH_ENDPOINT = endpoint;

await ScenarioRunner.run({ ... });

This creates race conditions when running multiple scenarios concurrently for different projects, forcing the caller to implement a mutex:

https://github.com/langwatch/langwatch/blob/main/langwatch/src/server/scenarios/simulation-runner.service.ts#L21-L47

Root Cause

  1. run() in javascript/src/runner/run.ts reads from getEnv() which parses process.env
  2. EventBus is instantiated with these env-derived values
  3. setup.ts initializes observability at module load time using env vars

Proposed Solution

Add a langwatch field to ScenarioConfig that takes precedence over environment variables:

interface LangWatchConfig {
  apiKey?: string;
  endpoint?: string;
}

interface ScenarioConfig {
  // ... existing fields
  
  /**
   * LangWatch reporting configuration.
   * Takes precedence over LANGWATCH_API_KEY and LANGWATCH_ENDPOINT env vars.
   */
  langwatch?: LangWatchConfig;
}

Implementation Steps

  1. Add langwatch field to ScenarioConfig (javascript/src/domain/scenarios/index.ts)

  2. Modify run() (javascript/src/runner/run.ts):

    const envConfig = getEnv();
    const langwatchConfig = {
      endpoint: cfg.langwatch?.endpoint ?? envConfig.LANGWATCH_ENDPOINT,
      apiKey: cfg.langwatch?.apiKey ?? envConfig.LANGWATCH_API_KEY,
    };
    eventBus = new EventBus(langwatchConfig);
  3. Handle observability setup (javascript/src/tracing/setup.ts):

    • Option A: Lazy initialization per run (breaking change)
    • Option B: Accept that observability uses env vars at load time, but event reporting uses per-run config (pragmatic)
    • Option C: Create a configureObservability() function that can be called before run()

Consumer Example (After Fix)

// No mutex needed - each run is self-contained
await ScenarioRunner.run({
  name: 'Test',
  description: 'Scenario',
  agents: [...],
  langwatch: {
    apiKey: project.apiKey,
    endpoint: 'https://app.langwatch.ai',
  },
});

Context

This is blocking proper multi-tenant scenario execution in the LangWatch platform:

Labels

  • enhancement
  • sdk-js

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

Status

Backlog

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions