[Bug]: E2E test shows "timeout must be an integer" SDK warning causing retries

[sjnims]

Bug Description

During E2E test execution, the Anthropic SDK receives a non-integer timeout value, triggering multiple retry attempts with the warning:

[WARN] Retry attempt 1: timeout must be an integer (waiting 1084ms)

This warning appears repeatedly (up to 3 retry attempts) before the system falls back to alternative behavior.

Important context: All unit tests pass. This issue only manifests during E2E test execution with real API calls.

Steps to Reproduce

1. Run RUN_E2E_TESTS=true npm test -- tests/e2e/
2. Observe the test output during scenario generation or execution
3. See repeated "timeout must be an integer" warnings with retry attempts

Expected Behavior

API calls should succeed on the first attempt without timeout-related warnings. The timeout parameter should be passed as an integer to the Anthropic SDK.

Actual Behavior

Multiple retry attempts occur due to timeout type issues:

[WARN] Retry attempt 1: timeout must be an integer (waiting 1084ms)
[WARN] Retry attempt 1: timeout must be an integer (waiting 1044ms)
[WARN] Retry attempt 2: timeout must be an integer (waiting 2056ms)
[WARN] Retry attempt 2: timeout must be an integer (waiting 2004ms)
[WARN] Retry attempt 3: timeout must be an integer (waiting 4253ms)
[WARN] Retry attempt 3: timeout must be an integer (waiting 4044ms)
[WARN] LLM generation failed, using fallback scenarios

The system eventually falls back to alternative scenarios, so tests can proceed, but this indicates a parameter typing issue.

Environment

• OS: macOS
• Node.js: v25.x
• Test runner: Vitest
• Anthropic SDK: (check package.json for version)

Error Logs / CLI Output

[WARN] Retry attempt 1: timeout must be an integer (waiting 1022ms)
[WARN] Retry attempt 1: timeout must be an integer (waiting 1015ms)
[WARN] Retry attempt 2: timeout must be an integer (waiting 2040ms)
[WARN] Retry attempt 2: timeout must be an integer (waiting 2168ms)
[WARN] Retry attempt 3: timeout must be an integer (waiting 4220ms)
[WARN] Retry attempt 3: timeout must be an integer (waiting 4042ms)
[WARN] LLM generation failed, using fallback scenarios

Additional Context

• Unit tests pass without this warning
• The retry mechanism handles the error gracefully (fallback works)
• This suggests either:
  1. E2E test configuration passes a non-integer timeout
  2. A timeout value calculation somewhere produces a float instead of int
  3. A type coercion issue in the API client setup for E2E tests

Investigation should check:

• Where timeout values are set for E2E tests
• Whether Math.floor() or similar is needed somewhere
• The Anthropic SDK's timeout parameter requirements

Related to #289 - both are E2E-specific issues where unit tests pass.

---

🤖 Created with Claude Code

[claude]

Analysis: E2E Test SDK Timeout Warning

1. Problem Assessment

Problem is clearly defined: Yes. During E2E test execution, the Anthropic SDK receives a non-integer timeout value, causing retry warnings: "timeout must be an integer". This only occurs in E2E tests, not unit tests.

Root cause: The E2E test helper createE2EConfig() in tests/e2e/helpers.ts creates configuration objects without including api_timeout_ms fields, which means these fields are undefined in the returned config object. The config is NOT validated through Zod schemas (which would apply defaults), so these undefined values propagate through the system.

When undefined timeout values reach the LLM utility functions (src/utils/llm.ts:77 and 134), they pass undefined to the SDK, triggering SDK-level validation errors.

Affected stages: Stage 2 (Generation) and Stage 4 (Evaluation) during E2E tests.

[claude]

2. Proposed Solution(s)

Solution: Add missing timeout fields to E2E config

Add api_timeout_ms and temperature fields to generation and evaluation configs in tests/e2e/helpers.ts:110-148.

Tradeoffs:

• Pros: Simple fix, matches schema defaults, makes E2E configs explicit
• Cons: Adds duplication between E2E helpers and schema defaults
• Risks: None

3. Alternative Approaches

Option A (Recommended): Validate E2E configs through Zod

Pass the config through validateConfig() to apply schema defaults automatically. Import validateConfig from src/config/loader.js and call it at the end of createE2EConfig().

Pros:

• Single source of truth (schema defaults)
• Catches other missing fields automatically
• Type safety at runtime

Cons:

• Adds validation overhead to test setup
• May catch unrelated validation errors

Option B: Add defensive checks in LLM utilities

Not recommended - treats symptom rather than root cause.

[claude]

4. Implementation Considerations

Files affected: tests/e2e/helpers.ts:110-148

Testing: Run RUN_E2E_TESTS=true npm test -- tests/e2e/ to verify warnings disappear

No breaking changes or state impact

Additional findings: The E2E config also omits temperature fields which have schema defaults (generation: 0.3, evaluation: 0.1).

5. Recommendation

Recommended approach: Option A (Validate E2E configs through Zod)

Why:

1. Single source of truth - schema defaults already defined and tested
2. Future-proof - automatically catches other missing fields
3. Type safety - runtime validation catches config issues early
4. Consistency - E2E tests should mirror production config flow

Implementation steps:

1. Import validateConfig in tests/e2e/helpers.ts
2. Call validateConfig(rawConfig) at end of createE2EConfig()
3. Run E2E tests to verify warnings disappear
4. Check for any validation errors from other missing fields

If validation overhead is a concern: Explicitly add all schema-defaulted fields instead.

---

File references:

• E2E helper: tests/e2e/helpers.ts:110-148
• LLM utility: src/utils/llm.ts:77, 134
• Schema defaults: src/config/schema.ts:130, 187
• Config validation: src/config/loader.ts:94-108

[sjnims]

PR submitted: #312