Trouble with TCP vs UDP

TCP

With TCP, delta must be high and some adjustments must be made to the OS to cope with the number of messages received. Otherwise the ARP becomes full and we run out of file descriptors for the tcp sockets

UDP

UDP solves the above problems. We run into other problems though. We can't pass objects with size more than the Datagram size (65507Bytes). We also have to think about the MTU in a churn environment.

Solving UDP Problems

We can try to:

• Compress using GZipInput/Outputstream
• Don't send an object but the events directly
• Send multiple messages comprised of fractions of the ball instead of the ball directly

What was done

• Drop GZip as the overhead was bigger than the gains from the compressions (1Byte on average)
• Sending events fields as Int and Long instead of events
• Split balls bigger than 5k Bytes