Skip to content

EtherateMT PACKET_MMAP Mode

James Bensley edited this page Dec 14, 2018 · 5 revisions

EtherateMT uses a ring buffer in PACKET_MMAP mode to send and receive packets. For Tx, one can tune the size of the ring buffer, how many frames will be stored in it, how many are sent in each “burst”. For Rx similar options also exist. Fiddle with these options and the frame size option to test the performance of different frame sizes and buffer sizes (page aligning buffers and frames for example can improve performance). See the CLI help option (-h) to see all available options.

Note, the following are conditions that are checked in when using PACKET_MMAP mode:

  • The ring block size (tp_block_size) must be a multiple of PAGE_SIZE
  • The ring block size (tp_block_size) should be chosen to be a power of two or there will be a waste of memory.
  • The frame size (tp_frame_size) must be greater than TPACKET_HDRLEN
  • The frame size (tp_frame_size) must be a multiple of TPACKET_ALIGNMENT
  • The frame number (tp_frame_nr) must be exactly frames_per_block*tp_block_nr

Don’t forget! In Tx mode with PACKET_MMAP using TPACKET version 2; a ring is made from blocks and there can only be one frame per block. If the frames are 1500 bytes for example, each block is 1500 bytes + TPACKET_HDRLEN. The ring size is then simply then the number of frames * block size since it's one frame per block (and the frame size is fixed for the entire Tx ring). In Rx mode using TPACKET v3 a ring is made of blocks and each block can hold multiple frames (as many as will fit without fragmenting them, also the frame sizes can vary as the block size is static). This means in an Rx ring the ring size is number of blocks * block size (instead of frame size, as a block can hold one or multiple frames, and as with the Tx ring must hold the frame data + TPACKET_HDRLEN per frame), and entire blocks can be poll()’ed and read() meaning batches of frames can be read to a block in Rx mode with TPACKET v3. TPACKET v3 should support variny frame sizes in the TX_RING just like in the RX_RING however, this is untested as EtherateMT fills the TX_RING with frames of the same size (due to this commit: https://github.com/torvalds/linux/commit/7f953ab2ba46e8649537942c0a64668ca2ce5cc5#diff-81f235b603c5ae4c53d7fd607db6a55f).