Can we use fixed buffers for UringSocket?

[armanbilge]

https://unixism.net/loti/tutorial/fixed_buffers.html

• I've seen examples for file I/O, but its unclear if they are possible / useful for network I/O
• buffers have to be registered ahead-of-time, which seems tricky to architect

[mox692]

I created a simple C program and experimented it, and it seems possible to read and write sockets using fixed_buffer apis such as io_uring_prep_write_fixed and io_uring_prep_read_fixed.

If my understanding is correct, fixed buffers are associated with io_uring, which is used throughout the program, so if we want to use fixed buffers we will need to add some buffer management logic somewhere, but I don't think it's difficult to do.

[armanbilge]

@mox692 nice job, that's great! Are there _fixed versions for send and recv too, which is what we are currently using for sockets?

If my understanding is correct, fixed buffers are associated with io_uring, which is used throughout the program, so if we want to use fixed buffers we will need to add some buffer management logic somewhere, but I don't think it's difficult to do.

Yes, I believe that's right. But I think we need to decide how many buffers we want and how big they should be so that we can pre-allocate them. Maybe that's okay, but I felt a bit unsure about it, particularly for services that expects to have lots of connections.

[mox692]

Are there _fixed versions for send and recv too

Ah, I missed that point. The program I used for the experiment was using the _fixed version of read/write, not recv/send. (my very rough example here )

But, now I look at the liburing implementation, it seems that io_uring_prep_read_fixed and io_uring_prep_readv are based on the same function io_uring_prep_rw, so if we done properly, it seems possible to perform socket read/write with flags set like recv/send.

But I think we need to decide how many buffers we want and how big they should be so that we can pre-allocate them

yeah, I also don't have a clear answer for this part..

• How to determine the size of the fixed buffer? (as you mentioned, this depends heavily on the application)
• What should happen when the fixed buffer becomes full?

I will consider these topics a bit more.

[armanbilge]

Related discussion, I think?

• Does TCP benefit from fixed writes? axboe/liburing#870