Initial implementation of new task system

[eddyashton]

Just implementation and tests, not used in any real system yet.

[eddyashton]

A design point from an in-person discussion, recorded here for posterity + future consideration:

The current OrderedTasks structure assumes a single order for both receiving request and sending responses, and stalling to produce an async response thus introduces a bubble where all future receiving is also paused. For maximum parallelism, where it's safe to do so, we'd like the option to fan-out multiple requests from a single session for concurrent execution. This implies separate queues - the responses must still be sent to match request order, but execution order may not match this. This needs a richer API than simply pausing an OrderedTasks, since multiple outstanding requests could produce async delayed responses. Each response thus needs some ordered identifier, fulfilling a ticket, and only contiguous blocks of tickets are scheduled for execution.

For example, we only allow executions like the following currently:

# [thread] action
[0] read A
[0] response A
[0] read B
[0] response B

[1] read C
[1] response C
[1] read D
[1] pause

[2] resume
[2] response D

We might instead want to support something like this:

[0] read A
[1] read B
[2] read C
[3] read D

[0] response A
[2] pause C
[1] response B (deferred)
[3] pause D

[0] resume C
[0] response C (undeferring response B)

[1] resume D
[1] response D

I think this should be simple enough to add, as another container-type on top of the existing task system, but I'll aim to derisk that before merging this.

[eddyashton]

I think this should be simple enough to add, as another container-type on top of the existing task system, but I'll aim to derisk that before merging this.

I've added a FanInTasks (ignore the naming!) container type today, that I think is the building block we need to support this pattern. There are a handful of open questions around exactly where this hits the implementation (what work can we safely fan-out, while retaining sensible single-client HTTP semantics?), but I think this is sufficient derisking (in particular - I think that the building blocks are present so that we can adapt to, or at least experiment with, such use cases).