Skip to content

[Enhancement] Add comprehensive failure testing infrastructure #10

Description

@EricSanchezok

Summary

AWCP needs comprehensive failure testing. Current tests cover happy paths but lack systematic coverage of failure scenarios.


Current Coverage Gap

Tested: State machine transitions, admission control, basic SSE retry

Not Tested: Network failures, transport failures, SSE interruption, race conditions, data integrity


Proposed Test Structure

packages/sdk/test/
├── unit/           # Existing tests
├── component/      # New: isolated failure injection
│   ├── sse-failures.test.ts
│   ├── transport-failures.test.ts
│   └── network-failures.test.ts
└── integration/    # New: end-to-end with faults
    └── recovery-scenarios.test.ts

Key Test Cases (Priority Order)

P0 - Critical

  • SSE connection drop + retry exhausted
  • SSE timeout (no events received)
  • Transport checksum mismatch
  • Network timeout on INVITE/START
  • Executor unreachable (ECONNREFUSED)

P1 - Important

  • HTTP error responses (401, 429, 500, 502, 503)
  • Transport setup failures (mount, extraction)
  • Cancel during task execution
  • Recovery via GET /tasks/:id/result

P2 - Nice to Have

  • Malformed JSON responses
  • State machine invalid transitions
  • Partial upload/download failures

Recommended Dependencies

{
  "devDependencies": {
    "nock": "^13.5.0",
    "get-port": "^7.0.0"
  }
}

Example: SSE Failure Test

it('should retry on connection drop', async () => {
  let attempts = 0;
  server = createServer((req, res) => {
    attempts++;
    res.writeHead(200, { 'Content-Type': 'text/event-stream' });
    res.destroy(); // Drop connection
  }).listen(port);

  const client = new ExecutorClient({ sseMaxRetries: 3, sseRetryDelayMs: 10 });
  
  await expect(async () => {
    for await (const _ of client.subscribeTask(`http://localhost:${port}`, 'test')) {}
  }).rejects.toThrow();

  expect(attempts).toBe(4); // 1 + 3 retries
});

Checklist

  • Add nock, get-port dependencies
  • Create component test directory structure
  • Implement P0 test cases
  • Implement P1 test cases
  • Add to CI pipeline

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions