Precise semantics around 0-length reads/writes

[alexcrichton]

I had some discussion with @dicej today about some work in the WASI implementation of various implementations of streams. The main example here is tcp-socket.listen which returns a stream<tcp-socket>.

At the OS level Wasmtime is built on Tokio right now which provides this API for accepting sockets. Notably this does not provide a "readiness" style API and instead only provides an API of "do the accept right now and tell me the result".

This is somewhat at odds with a 0-length read of the the stream on the guest side where this is understood to semantically mean "please resolve this async read when the socket is ready to accept, but don't actually give me what's being accepted". In Wasmtime we can always perform an accept and buffer the result of the accept on the host, and that's what today's implementation does. My question and our discussion arose from this behavior, however, where I don't think this is necessarily what we want.

The main hazard of buffering a result is that if this were a shared resource of some kind (e.g. shared at the OS level, not the component model level) then buffering may not be what's desired here. For example if the same TCP listener on the host were loaned out to multiple wasm instances then buffering in one instance would prevent any other instance from accepting the socket. To make matters worse the guest which performed a 0-length read may then trap or close the listener altogether, permanently losing the buffered socket.

Basically the question I'd like to ask here is: can we relax the requirement about 0-length reads to be a "may block until ready" rather than "guaranteed to block until ready"? I'd like to update the Wasmtime implementation here to always say "0-length read? yep we're ready" and immediately return. That's a lie in the sense that it's not actually ready but it's the best we can do in Wasmtime without Tokio.

Now an important part here though is that we still actually implement this for other resources/streams. For example when reading a TCP socket we'd still handle the 0-length case specially and avoid actually reading any data, instead just waiting for readiness. My hope here is that we could enable these sorts of use cases when the stars align but at the same time not require every primitive everywhere to implement some sort of required buffering and handling of the buffered values.

[dicej]

The challenge here is that if e.g. a file stream just always says "yes, I'm ready" to a zero-length read, a wasi-libc implementation of select or poll which includes that file descriptor will always return immediately, turning the application's loop into a busy wait.

I do agree that the "guaranteed to block until ready" requirement can be impossible to implement on top of a completion-oriented API without buffering, and that buffering has its own hazards.

[alexcrichton]

Effectively, "yes", I agree with those consequences. For files in particular AFAIK that's already how native Linux/macOS work, they just say a file is always readable/writable and there's no notion of readiness.

To me we'll need to implement the "non busy loop path" for primitives where we can (e.g. TCP/UDP) but otherwise I don't think we should require this just for wasi-libc to work in 100% of cases where is already sort of doesn't.

[lukewagner]

Great point; this probably needs to be clarified in the text.

So first, in terms of CM-ensured invariants that a guest toolchain can rely on: I believe it is already the case that, in addition to writes, after a 0-length read succeeds, there is no guarantee that the next non-zero-length read will succeed without blocking. Even in the special case of component-to-component streaming: when a zero-length read rendezvous with a non-zero-length write, if the notified reader task somehow suspends (e.g., it calls yield), the writer component can execute again and cancel its read such that when the reader component gets resumed, it blocks performing a non-zero-length read.

A subtler question is: are 0-length reads allowed to repeatedly succeed (not as a result of a race but immediately every time, like Alex was saying for stream<tcp-socket>)? If the answer is "no", then wasi-libc can always just return EWOULDBLOCK and let the client select() again and hopefully it'll work next time, but if "yes", then wasi-libc needs to do buffering to avoid an iloop. For writes, due to the zero-length-rendezvous-wakes-the-writer semantics of component-to-component streaming, we've already answered "yes". But I'd think for reads the answer should probably also be "yes" (b/c if the answer were "no", then that's an important invariant, but not one we can easily specify or enforce). So then I think wasi-libc would need to same "fall back to buffering" behavior as writes.

That all being said, it's yet a different question of whether the default host impl of stream<tcp-socket> should actually exercise this allowance and make zero-length-reads trivially succeed. I get the general principle that implicitly buffering host resources is bad, but for the specific case of sockets+accept(), iiuc, even before the accept() call, the kernel has already allocated non-trivial socket resources for the incoming socket and SYN/ACKed the SYN and so the accept() is more of a backpressure mechanism than anything else. Thus, if the wasi-http host impl of a zero-length stream<tcp-socket>.read eagerly accept()s a socket, as long as there are a bounded number of such eagerly-accept()ed sockets, overall OS resource usage would not be meaningfully affected.

[dicej]

I get the general principle that implicitly buffering host resources is bad, but for the specific case of sockets+accept(), iiuc, even before the accept() call, the kernel has already allocated non-trivial socket resources for the incoming socket and SYN/ACKed the SYN and so the accept() is more of a backpressure mechanism than anything else. Thus, if the wasi-http host impl of a zero-length stream<tcp-socket>.read eagerly accept()s a socket, as long as there are a bounded number of such eagerly-accept()ed sockets, overall OS resource usage would not be meaningfully affected.

I think Alex's concern is that, without a lot of extra work in the host implementation, the incoming socket would just be dropped without anybody receiving it. I.e. if it's buffered as part of that stream during a zero-length read, and then the stream is dropped without a follow-up, non-zero-length read, the buffered value will also be dropped. The only way to avoid that would be to have some sort of global buffer shared among all streams (and instances? and host processes?) for the port being listened on. I'm not sure how that would work.

[alexcrichton]

after a 0-length read succeeds, there is no guarantee that the next non-zero-length read will succeed without blocking

agreed!

are 0-length reads allowed to repeatedly succeed

My take on this issue is, "yes". I don't think that necessarily means wasi-libc is absolutely required to buffer, but it does mean that if you use the wrong primitives in select or the wrong host implementation you'll hit an infinite loop, yes.

overall OS resource usage would not be meaningfully affected

To append to Joel's elaboration, one thing I'm worried about is:

• A guest performs a 0-length read
• The host accepts a socket
• The host puts the socket in a host-defined table (not CM-related necessarily) and then uses this a a Resource<TcpSocket> handle to give to the runtime.
• The runtime buffers this Resource<TcpSocket> and returns to the guest "I'm ready"
• The guest closes the stream and never actually reads the socket

In this case the host-defined table modified in the 3rd step is effectively a resource leak. We would have to arrange some sort of destructor on a stream to say "ok here you can destroy your unread buffered values too". While this is possible, I personally think it would be better to clarify "0-length reads may not work for readiness" because that's already the case for so many other primitives (e.g. files and stdio)

[lukewagner]

Gotcha; avoiding the need (at least in the short term) to avoid machinery for releasing buffered OS resources makes sense. So then if zero-length stream<tcp-socket>.read takes advantage of its freedom to complete immediately, to avoid the iloop, would wasi-libc (always or after seeing non-zero-length read return BLOCKED) do buffering on its side (having select() waitable-set.wait on a non-zero-length read), and drop any buffered resource handles if the socket file descriptor was close()ed before accept()?

[alexcrichton]

Sort of, yes, but personally I wouldn't consider that satisfactory because it's basically just pointing the same finger of blame at the guest instead of the host (as it's still doing the same thing of buffering).

More generally what I'm trying to get at is that while we can explicitly specify what happens in guest<->guest interactions I don't think we can have nearly as many guarantees when it comes to WASI and guest<->host interactions. Trying to take the mess that is the real world and specify precise semantics that works in all cases I feel is either going to have a very large implementation burden or corner us into design decisions which make certain use cases impossible.

Personally I find it really hard to contemplate/design in this space as everything is based on hypotheticals and attempting to loosely correlate some native API we're familiar with to component model lookalikes. For example the concerns I'm bringing up here are super vague and generally not that actionable ("what if a resource is shared across instances and buffering is visible?"). I have no idea if this is a relevant concern or if anyone will ever actually run into it. An example of this: I'm advocating for "just let wasi-libc infinite loop". I have no idea if that will actually affect anyone in practice. Sure I can create a program that is affected by it, but does that matter? If it takes a decade for anyone to run into it does that matter because in the interim it may have been fixed or otherwise have an obvious solution later?

Another example: for this specific issue, is it better to buffer something and leak it in niche cases or to let wasi-libc infinite loop? I have no idea as to which of these is more relevant myself. I feel like I'm just not in a position where I can weigh all these hypotheticals and determine which aspects need to have more weight on them than others. But at the same time I'm not sure any of us are in the position to necessarily do that so I don't know how we can resolve questions like this