Skip to content

Hibernation API can result in out-of-order frames when combined with direct .send() calls #236

Description

@thoughtpolice

Basics

While testing out some WebSocket ideas, Claude found a pretty serious issue which is that a hibernated websocket seem to return frames out-of-order to the client when that same WebSocket object is used directly. The reproduction test (see below) actually is slightly more complicated than this (to test hibernation vs non hibernation), but I am 99% sure the below snippet is basically all that's needed for this to happen:

export class B extends DurableObject {
  fetch(request) {
    const [client, server] = [...new WebSocketPair()];
    this.ctx.acceptWebSocket(server);
    return new Response(null, { status: 101, webSocket: client });
  }
  webSocketMessage(ws) {
    ws.send(this.frame("socket event"));
  }
  rpc() {
    for (const ws of [...this.ctx.getWebSockets(), ...this.standard]) {
      ws.send(this.frame("rpc"));
    }
  }
}

Now, imagine you begin sending numbered messages to this object, which just sends them back immediately, 1, 2, 3, ... — you will get out of order frames with a rather small chance if you interleave rpc() calls. I think this is a bug because conceptually a Durable Object is (in essence) a fully sequential, linearized register. And so, if a durable object B has methods invoked in sequential order B.m1(X), B.m2(Y), ..., and each of those .methodN(...) performs some observable action, I would expect the resulting actions to be observed in the same order.

Ordinary, non-hibernated WebSockets do not display this behavior.

Reproduction

This MWE was written by Claude Opus 5.5 after minimizing it. It is slightly more elaborate because it shows the difference between the two modes.

git clone https://gist.github.com/thoughtpolice/17605c0819fef2805331f0e5ab91ffc6
cd 17605c0819fef2805331f0e5ab91ffc6
chmod +x run.sh

The client.ts is a simple Deno program, index.ts is the worker. The test harness does multiple runs (to ensure it is triggered) and assumes you provide the path to the celld and deno binaries:

$ ./run.sh /path/to/celld /path/to/deno # or just 'celld deno' off PATH
celld 0.6.0
standard: 200 frames, in order
standard: 200 frames, in order
standard: 200 frames, in order
hibernatable: 200 frames, 39 out of order
  n=602 (rpc) before n=601 (socket event)
  n=608 (rpc) before n=607 (socket event)
  n=610 (rpc) before n=609 (socket event)
hibernatable: 200 frames, 23 out of order
  n=830 (rpc) before n=829 (socket event)
  n=832 (rpc) before n=831 (socket event)
  n=834 (rpc) before n=833 (socket event)
hibernatable: 200 frames, 26 out of order
  n=1012 (rpc) before n=1011 (socket event)
  n=1016 (rpc) before n=1015 (socket event)
  n=1018 (rpc) before n=1017 (socket event)

In my case, when I discovered the problem originally, it was under higher load, and so the amount of disorder was much more severe (dozens or hundreds of misordered frames).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions