How does request caching works with HTTP 3?

[akostadinov]

Hi, it turned out that uploading bigger files than 64k exceeds the default limit of rpxy.

I was wondering, are such requests cached by rpxy? How will increasing the limit affect memory usage? Can caching be disabled so that the backend application handle it as it please?

[junkurihara]

Hi,

Yes, the maximum capable size of request body is 64k (65536 bytes) for http/3 by default. You can increase this by configuring request_max_body_size in [experimental.h3] section.

rust-rpxy/config-example.toml

Lines 120 to 126 in 3af6ac9

	[experimental.h3]
	alt_svc_max_age = 3600 # sec
	request_max_body_size = 65536 # bytes
	max_concurrent_connections = 10000
	max_concurrent_bidistream = 100
	max_concurrent_unistream = 100
	max_idle_timeout = 10 # secs. 0 represents an infinite timeout.

rpxy converts incoming http/3 requests to http/1.1 or http/2, and then forwards to backend apps. In this procedure, since requests are processed on the heap memory, the memory usage increases as you increase request_max_body_size. So please carefully configure the parameter.

Also, in the same sense, requests are not cached at rpxy since the request caching could easily exhaust the memory. (I may be misunderstand what you mean by 'request caching') Once it is processed and forwarded, it is discarded from the memory.

[akostadinov]

It seems like the term is request buffering.

I don't know how HTTP3 actually works. But I wonder, is it possible that partial requests are sent to the client whenever that maximum buffer size is reached? I assume it won't be trivial but sounds like a useful thing to avoid limits imposed on the proxy side.

[junkurihara]

I see. I got what you mean. In fact, I have already introduced such the feature. The following is the current implementation of handling request body for HTTP/3.

rust-rpxy/rpxy-lib/src/proxy/proxy_h3.rs

Lines 108 to 138 in e2e1c2e

	// generate streamed body with trailers using channel
	let (body_sender, req_body) = IncomingLike::channel();
	// Buffering and sending body through channel for protocol conversion like h3 -> h2/http1.1
	// The underling buffering, i.e., buffer given by the API recv_data.await?, is handled by quinn.
	let max_body_size = self.globals.proxy_config.h3_request_max_body_size;
	self.globals.runtime_handle.spawn(async move {
	let mut sender = body_sender;
	let mut size = 0usize;
	while let Some(mut body) = recv_stream.recv_data().await? {
	debug!("HTTP/3 incoming request body: remaining {}", body.remaining());
	size += body.remaining();
	if size > max_body_size {
	error!(
	"Exceeds max request body size for HTTP/3: received {}, maximum_allowed {}",
	size, max_body_size
	);
	return Err(RpxyError::H3TooLargeBody);
	}
	// create stream body to save memory, shallow copy (increment of ref-count) to Bytes using copy_to_bytes
	sender.send_data(body.copy_to_bytes(body.remaining())).await?;
	}
	// trailers: use inner for work around. (directly get trailer)
	let trailers = recv_stream.as_mut().recv_trailers().await?;
	if trailers.is_some() {
	debug!("HTTP/3 incoming request trailers");
	sender.send_trailers(trailers.unwrap()).await?;
	}
	Ok(()) as RpxyResult<()>
	});

As you can see, we handle and forward a part of body as it arrives. This means that the whole part of the body is not buffered on the memory, and that the memory consumption is not linearly increasing for the h3_request_max_body_size.

Here, we note that the buffering is controlled by the underling QUIC library (quinn or s2n-quic). I think that in most cases, QUIC libraries probably require a large portion of memory for buffering due to the speed of the channel, and in some case require the change of UDP buffer size (c.f., https://github.com/quinn-rs/quinn?tab=readme-ov-file#buffers for quinn) So, since rpxy doesn't handles the size of the buffer, I installed the limit of max body size for the safety just in case. Honestly, I have not confirmed if this is reasonable actually, and also think the limit may not be needed.

At this point I am unsure that we can manage the size of buffer of underlying quic libraries.

[xkr47]

I will sometimes need to upload gigabytes worth of data through rpxy so whatever buffering is done for the request body needs to be contained within some sane limits or it will become unusable. I'll need to do some tests.

[akostadinov]

The thing is that I have a hard time testing. My requests are split between h2 and h3, dunno why. With both - firefox and chrome. I need to somehow catch a phase where they decide fully on using h3 🤔