-
Notifications
You must be signed in to change notification settings - Fork 4
EtherateMT PACKET_MMAP Mode
EtherateMT uses a ring buffer in PACKET_MMAP
mode to send and receive packets. For Tx, one can tune the size of the ring buffer, how many frames will be stored in it, how many are sent in each “burst”. For Rx similar options also exist. Fiddle with these options and the frame size option to test the performance of different frame sizes and buffer sizes (page aligning buffers and frames for example can improve performance). See the CLI help option (-h
) to see all available options.
Note, the following are conditions that are checked in when using PACKET_MMAP
mode:
- The ring block size (
tp_block_size
) must be a multiple ofPAGE_SIZE
- The ring block size (
tp_block_size
) should be chosen to be a power of two or there will be a waste of memory. - The frame size (
tp_frame_size
) must be greater thanTPACKET_HDRLEN
- The frame size (
tp_frame_size
) must be a multiple ofTPACKET_ALIGNMENT
- The frame number (
tp_frame_nr
) must be exactlyframes_per_block*tp_block_nr
Don’t forget! In Tx mode with TPACKET v3 should support variny frame sizes in the TX_RING just like in the RX_RING however, this is untested as EtherateMT fills the TX_RING with frames of the same size (due to this commit: https://github.com/torvalds/linux/commit/7f953ab2ba46e8649537942c0a64668ca2ce5cc5#diff-81f235b603c5ae4c53d7fd607db6a55f).PACKET_MMAP
using TPACKET version 2; a ring is made from blocks and there can only be one frame per block. If the frames are 1500 bytes for example, each block is 1500 bytes + TPACKET_HDRLEN
. The ring size is then simply then the number of frames * block size since it's one frame per block (and the frame size is fixed for the entire Tx ring). In Rx mode using TPACKET v3 a ring is made of blocks and each block can hold multiple frames (as many as will fit without fragmenting them, also the frame sizes can vary as the block size is static). This means in an Rx ring the ring size is number of blocks * block size (instead of frame size, as a block can hold one or multiple frames, and as with the Tx ring must hold the frame data + TPACKET_HDRLEN
per frame), and entire blocks can be poll()
’ed and read()
meaning batches of frames can be read to a block in Rx mode with TPACKET v3.
- Introduction
- References
- EtherateMT Design Notes and Usage:
- EtherateMT Design Overview
- EtherateMT Socket Overview
- EtherateMT PACKET_MMAP Mode
- EtherateMT Transmit Overview
- EtherateMT Transmission - AF_PACKET Deep Dive:
- EtherateMT Transmission - AF_XDP