[wasi] `async`/`await` on WASIp2

[dicej]

I've been working to support async/await in C# for doing concurrent I/O when targeting WASIp2. A key piece of that is the wasi:io/poll#poll function, which accepts an arbitrary number of pollable handles representing e.g. socket readiness, HTTP protocol events, timer events, etc. It's analogous to the traditional POSIX poll(2) function. My goal is to provide a System.Threading.Tasks-based abstraction on top of wasi:io/poll that supports idiomatic use of async/await, including the various Task combinators such as WhenAny, WhenAll, WhenEach. I've done similar work for Python (using a custom asyncio event loop) and Rust (using a custom async runtime), and am hoping to do the same for .NET.

So far, I've managed to write a custom TaskScheduler which supports simple cases by maintaining a list of Tasks and a list of the Pollables those tasks are awaiting. It has a Run method which, in a loop, alternates between the Task list and the Pollable list, executing the former and calling wasi:io/poll#poll on the latter. That works well for simple cases.

However, I've had trouble building more sophisticated examples using e.g. the new Task.WhenEach combinator due to the somewhat pervasive use of ThreadPool as a deferred execution mechanism throughout the System.Threading.Tasks library code. Given that WASI does not support multithreading and won't for the foreseeable future, ThreadPool's methods currently throw System.PlatformNotSupportedException, which makes it something of a landmine. For example, even though there's nothing inherently multithreaded about Task.WhenEach, the current implementation relies on a ManualResetValueTaskSourceCore with RunContinuationsAsynchronously set to true, which means it always queues continuations using ThreadPool.UnsafeQueueUserWorkItem.

Given that significant pieces of .NET's async/await infrastructure currently relies on multithreading to function, I'm wondering what our options are for WASI. A few come to mind:

• Give up on async/await and use callbacks for concurrency in WASIp2. Besides being un-ergonomic, it would significantly restrict the set of both standard and third-party library features available for use on that platform, given that anything that deals with I/O is presumably either synchronous and multithreaded or asynchronous based on async/await.
• Support async/await, but disallow use of features such as Task.WhenEach which require ThreadPool as an implementation detail, and possibly provide alternative implementations of those features which are single-thread-friendly.
• Refactor the parts of System.Threading.Tasks which are not inherently multithreaded (but currently use ThreadPool) to support an alternative, single-threaded deferred execution mechanism on platforms that do not support multithreading.
• Provide a WASI-specific ThreadPool implementation which simply queues work items without executing them, and provide some public API for running them from the main (and only) thread, e.g. as part of an application event loop.

Thoughts?

I should note that I'm quite new to the .NET ecosystem, so I'm happy to be corrected if I've misunderstood anything.

See also #98957, for which I would consider this issue a prerequisite.

[dotnet-policy-service]

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

[lewing]

cc @agocke

[pavelsavara]

In Mono/Browser when single-threaded, we have thread-less thread pool implemented via emscripten Module.safeSetTimeout which is just wrapper around JavaScript setTimeout.

runtime/src/mono/System.Private.CoreLib/src/System/Threading/ThreadPool.Browser.Mono.cs

Lines 82 to 88 in 9fca0c3

	internal static unsafe void RequestWorkerThread()
	{
	if (_callbackQueued)
	return;
	_callbackQueued = true;
	MainThreadScheduleBackgroundJob((void*)(delegate* unmanaged[Cdecl]<void>)&BackgroundJobHandler);
	}

runtime/src/mono/mono/utils/mono-threads-wasm.c

Lines 348 to 386 in 9fca0c3

	// when this is called from ThreadPool, the cb would be System.Threading.ThreadPool.BackgroundJobHandler
	// when this is called from sgen it would be wrapper of sgen_perform_collection_inner
	// when this is called from gc, it would be mono_runtime_do_background_work
	#ifdef DISABLE_THREADS
	void
	mono_main_thread_schedule_background_job (background_job_cb cb)
	{
	g_assert (cb);
	THREADS_DEBUG ("mono_main_thread_schedule_background_job: thread %p queued job %p to current thread\n", (gpointer)pthread_self(), (gpointer) cb);
	if (!jobs)
	schedule_background_exec ();
	if (!g_slist_find (jobs, (gconstpointer)cb))
	jobs = g_slist_prepend (jobs, (gpointer)cb);
	}
	GSList *jobs;
	G_EXTERN_C
	EMSCRIPTEN_KEEPALIVE void
	mono_background_exec (void)
	{
	MONO_ENTER_GC_UNSAFE;
	GSList *j = jobs, *cur;
	jobs = NULL;
	for (cur = j; cur; cur = cur->next) {
	background_job_cb cb = (background_job_cb)cur->data;
	g_assert (cb);
	THREADS_DEBUG ("mono_background_exec on thread %p running job %p \n", (gpointer)pthread_self(), (gpointer)cb);
	cb ();
	THREADS_DEBUG ("mono_background_exec on thread %p done job %p \n", (gpointer)pthread_self(), (gpointer)cb);
	}
	g_slist_free (j);
	MONO_EXIT_GC_UNSAFE;
	}
	#else /*DISABLE_THREADS*/

runtime/src/mono/browser/runtime/scheduling.ts

Lines 47 to 67 in 9fca0c3

	function mono_background_exec_until_done () {
	if (WasmEnableThreads) return;
	Module.maybeExit();
	try {
	while (pump_count > 0) {
	--pump_count;
	if (!loaderHelpers.is_runtime_running()) {
	return;
	}
	cwraps.mono_background_exec();
	}
	} catch (ex) {
	loaderHelpers.mono_exit(1, ex);
	}
	}
	export function schedule_background_exec (): void {
	if (WasmEnableThreads) return;
	++pump_count;
	Module.safeSetTimeout(mono_background_exec_until_done, 0);
	}

runtime/src/mono/System.Private.CoreLib/src/System/Threading/ThreadPool.Browser.Mono.cs

Lines 124 to 136 in 9fca0c3

	// this callback will arrive on the bound thread, called from mono_background_exec
	private static void BackgroundJobHandler()
	{
	try
	{
	_callbackQueued = false;
	ThreadPoolWorkQueue.Dispatch();
	}
	catch (Exception e)
	{
	Environment.FailFast("ThreadPool.BackgroundJobHandler failed", e);
	}
	}

This has benefit of yielding to the browser event loop.
Yielding is necessary so that JS Promises could get resolved, for example fetch.

Hope this helps.

I would like to learn more on how yielding out of the current (dotnet) component need to look like for WASI p2/p3.

[pavelsavara]

(Just thinking aloud)

The list of "jobs" we have in the thread-less thread pool solves all non-external Tasks and their dependencies.
Also note that any locks are NOOP in our single-threaded build.
And the browser (host) is calling back to us when there is setTimeout tick or when JS promise gets resolved.

Do I understand it right that in WASI instead of being called by host on "event" we are expected to have loop in some main() and do poll() on all external promises.

Meaning that WASI promises will become resolved by the WASI host even without us yielding ?

Would that poll loop still work in preview 3 too ?
Would that poll loop still work in multi-threaded too ?
Does it need to be re-entrant ? (dotnet calls another component synchronously, which calls back to our async export)
I'm concerned about WASM stack.

Could wasi-clocks play the role of setTimeout ? Would it help to anything ?

How would that look like if we are WASI reactor which exposes async API ?
Should all async (component) exported functions implement such poll loop ?

[pavelsavara]

To be able to exit that poll() loop we need to know if there is no more work to do.
Probably it means that all Tasks are resolved/rejected/canceled.
And unresolved Tasks could depend on external promises, which is OK.

(For context: In browser we marshal JS Promise as C# Task, when the JS calls us .then(()=>{ ... callCSharp() }) we call TaskCompletionSource.SetResult for the marshaled Task)

I'm not sure what would be the answer if there are multiple-threads in WASI. Should we have multiple poll() loops ?
Do WASI promises have thread affinity ? (in JS they do)

Does WASI support cancellation ? In preview 3 ?

[pavelsavara]

Is pollable concept going to survive preview3 ?

Is pollable equivalent of a promise which could resove just once ?
Or it's a wait handle/stream/semaphore, which could be lifted multiple times ?

If multiple times, are we going to create Task instance for each promised "event" of the pollable ?

[agocke]

To simplify the discussion slightly, I don't think there is any strong dependency on multi-threading with the .NET async model. It is possible that you will need to use a different default scheduler in your app model, but I don't know of any requirement that multi-threading is required for .NET async.

@stephentoub would probably be the best person to chime in on specific technical decisions that would be recommended for this area.

[dicej]

For reference, the Discord discussion about this starts here: https://discord.com/channels/732297728826277939/732297825731215521/1245818492322844673

[dicej]

Do I understand it right that in WASI instead of being called by host on "event" we are expected to have loop in some main() and do poll() on all external promises.

(@pavelsavara discussed this on Discord, but I'll answer here also for the record). Yes, WASIp2 requires the guest to run an event loop (maybe in main, but more generally any export, e.g. wasi:http/incoming-handler). WASIp3 will change that by using Component Model support for async imports and exports, at which point the event loop will move to the host.

Meaning that WASI promises will become resolved by the WASI host even without us yielding ?

Yes, the host can make progress on the promises regardless of if or when the guest yields.

Would that poll loop still work in preview 3 too ? Would that poll loop still work in multi-threaded too ? Does it need to be re-entrant ? (dotnet calls another component synchronously, which calls back to our async export) I'm concerned about WASM stack.

Quoting my Discord comments:

WASIp3 will be based on a major update to the Component Model which supports async imports and exports. The async exports may either be "stackless" such that the guest returns a task handle to the host and exports a callback for progress notifications, or "stackful", meaning the guest blocks on a poll-style intrinsic and supports concurrent calls on other fibers.

We expect languages that have stackless, async/await style coroutines such as JS, Python, Rust, and C# will want to use the callback approach, in which case the application developer will export async functions and wit-bindgen will handle the details of assigning a unique handle to each Promise/Task/Future and maintaining the lookup table @pavel Savara described. I've already implemented this for Rust: https://github.com/dicej/component-async-demo/. This mechanism also allows application code to "spawn" tasks which may continue to run after the export function has returned a result (e.g. https://github.com/dicej/component-async-demo/blob/aba5ebf363d5830cbab20bf9a453f927c47e0605/http-echo/src/lib.rs#L48-L59). The host runs the top level event loop, dispatching events to the appropriate callback exports with the appropriate handles, which the guest will translate to Promises/Tasks/Futures and resume them.

Could wasi-clocks play the role of setTimeout ? Would it help to anything ?

Yeah, I was thinking the same thing, but I don't think we need to involve the host with deferred execution at all -- it can all be taken care of in some combination of the .NET runtime and wit-bindgen-generated code.

How would that look like if we are WASI reactor which exposes async API ? Should all async (component) exported functions implement such poll loop ?

All exported functions that want to use async/await will need to run an event loop of some kind, but we could make that an implementation detail in wit-bindgen so the app developer only needs to write an async function and let the generated code add the event loop implicitly.