Batch sending messages from EndpointDriver to connection tasks

[rklaehn]

There is a significant performance difference between running the bulk bench with a current thread runtime and a multithreaded runtime.

Current thread runtime is faster despite all work (en/decryption, waiting on syscalls) happening on the same thread. This points to either the communication via channels or moving memory from one core to another being a bottleneck.

To try this out, use #280

We are very far from a speed where moving memory between cores can be a bottleneck, so what remains is the channel overhead.

We can reduce the channel overhead by batching. E.g. currently if we get a bunch of messages from recvmsg_x on mac fast or recvmsg with GRO on linux, we send them to the channels one by one.

I implemented this on mac, but you don't see a big difference.

[matheus23]

I implemented this on mac, but you don't see a big difference.

Does this mean you've tried to implement what this issue suggests, but haven't seen a performance improvement?

[rklaehn]

Yes. As I mentioned in another issue, on mac even with sendmsg_x / recvmsg_x we are very close to the limits due to syscalls. The bulk bench gets to ~300 + x MiB/s (this is with ACKs flowing back!), the raw low level bench in quinn-udp only gets to 400 MiB/s.

So no matter what you do you are not going to get much above ~300 MiB/s on mac. I guess I need a fast linux box for testing this. I won't even try to merge it unless I see a clear improvement, don't worry.

[rklaehn]

Branch: https://github.com/n0-computer/quinn/tree/batch-endpoint-to-connection