having default implementations

[c-cube]

Currently Picos has a default implementation of the whole interface for both OCaml 4 and OCaml 5. The linecount for that is 756 lines (as per sloccount).

I think the .mli interface is great and what it provides is a good basis for a common language between libraries. However trying to implement these on OCaml 4 with things like await spawning a new thread is not going to be useful in practice¹. I think it'd be a lot better if Picos provided only this:

• type definitions for the basic data structures (trigger, computation, fibers)
• standard effects for operations like await and spawn and cancel_after, implementation being left to the actual scheduler that will provide them

and there could be a side library (say picos.basic or picos-basic or picos-default), totally optional, that would provide a basic implementation of all that with some simple scheduler that handles all the effects.

This makes picos a lighter, less fussy dependency, so it's an easier choice to depend on it.

Footnotes

1. I like threads, but while spawning a thread per request is mostly fine for some real use cases, spawning a thread per await when writing highly concurrent code based on fibers is going to have a really high overhead. ↩

[polytypic]

Thank you for the feedback!

I hope that the discussion here will prove valuable. Briefly, I believe there might be some misunderstandings here and also I believe it might be possible to improve the Picos (default) implementation for the purpose(s) that it serves.

Currently Picos has a default implementation of the whole interface for both OCaml 4 and OCaml 5. The linecount for that is 756 lines (as per sloccount).

Yes, the sloccount lib of the whole Picos library (at the time of writing this) is 756 lines.

I now see that we understand the concept of "default" implementation differently. What I consider the "default implementation" (at the time of writing this) is basically just the default.ml file. That file implements default versions of await, cancel_after, current, spawn, and yield in OCaml 4 compatible manner.

On OCaml 4 those default implementations are (at the time of writing this) just used rather directly to implement the Picos Trigger, Fiber, and Computation modules.

On OCaml 5 those default implementation are (at the time of writing this) used in case the corresponding effects Await, Cancel_after, Current, Spawn, and Yield are unhandled.

Aside from the default.ml file, the rest of the code is what I consider (more or less) essential implementation of the Picos interface that allows concurrent programming libraries to link to it and use it to interact with schedulers via the await, cancel_after, current, spawn, and yield entry points.

I think it'd be a lot better if Picos provided only this:

• type definitions for the basic data structures (trigger, computation, fibers)
• standard effects for operations like await and spawn and cancel_after [...]

I'm not entirely sure what you mean by "provided only [...] type definitions [...] standard effects". Could you elaborate on that? What exactly would be provided? A plain module type Picos = sig ... end?

Aside from the type signature, Picos proposal intentionally includes essential implementations of the core concepts (Trigger, Computation, Fiber) that work in (basically) fully specified manner such that code using them can rely on their behavior and implement correct concurrent abstractions using them (that can handle parallel cancelation correctly and, at the same time, allow cancelation to be determined as late as possible).

I'll continue after having lunch.

[polytypic]

implement these on OCaml 4 with things like await spawning a new thread

Hmm... There seems to be some misunderstanding here. The default implementation of await does not spawn threads.

The fundamental idea behind the OCaml 4 compatible default implementations is to make it so that a fiber is equivalent to a (sys)thread (or, on OCaml 5, when threads are not available, to a domain). So, for example, the default spawn implementation creates or spawns a (sys)thread for each new fiber.

The default implementation then stores information for the thread (or fiber) in thread-local storage. The per-thread information includes a mutex, a condition, and the fiber object.

What the default await implementation does is that it obtains the per-thread information (mutex, condition, and fiber) and depending on whether the fiber has been canceled (i.e. fiber both allows cancelation and the computation associated with the fiber has been canceled) either returns the cancelation exception or suspends the thread by locking the mutex and waiting on the condition.

At no point does await spawn a new thread.

---

The cancel_after feature of Picos is perhaps the most difficult one to implement, because the OCaml Stdlib doesn't offer such a feature (unlike e.g. the setTimeout feature of JavaScript available both in browsers and in Node.js). The default implementation of cancel_after uses a background (sys)thread per domain. The reason for using a background thread per domain is to avoid contention. On OCaml 4 there will, of course, only be a single background thread as there is only one domain (not considering Unix.fork).

The default implementation of cancel_after uses a background (sys)thread that runs a Unix.select loop. (As you know) Unix.select is pretty much the only reasonable way to implement a timeout mechanism in pure OCaml. Unix.select allows one to both specify a timeout (as a float value in seconds) and allows the (blocking) call to be forced to return before the timeout by e.g. writing through a file descriptor given to it. Domain-local storage is used to store per-domain information for the mechanism. The information essentially includes a priority queue of timeouts and a file descriptor that can be written to force the Unix.select to return to re-examine the priority queue.

---

Addition: Note that the per-domain state is only initialized and the background thread is only started in case the default cancel_after is actually used. IOW, when running on a scheduler that handles the Cancel_aftereffect, no background thread will be created. Similarly, if a scheduler handles all the other effects (like it should), then the per-thread state of the default implementation won't be initialized. BTW, it is reasonable for a scheduler to not handle the Cancel_after effect and rely on the default implementation. Note to self: Need to document the concept of levels/layers of Picos support and what makes an acceptable Picos compatible scheduler.

---

Addition: Ended up adding a section on Default behaviors to the documentation.

[polytypic]

Now, I agree that the default implementation (using (sys)thread per fiber) is not what most serious multicore applications in OCaml 5 should be using. (Note that at this point I'm talking about the default implementation which is essentially in the default.ml file.) That is because a proper scheduler (by handling the effects) can provide fibers more efficiently. The intention behind the default behaviors is to allow libraries implemented in Picos to work out-of-the-box without a scheduler.

Consider the following REPL interaction using Kcas (implemented in Picos):

utop # open Kcas ;;
utop # open Kcas_data ;;
utop # open Picos ;;
utop # let fiber = Loc.make None ;;
val fiber : '_weak1 option Loc.t = <abstr>
utop # let queue = Queue.create () ;;
val queue : '_weak2 Kcas_data.Queue.t = <abstr>
utop # let domain = Domain.spawn (fun () ->
  Loc.set fiber (Some (Fiber.current () :> Fiber.as_async));
  Queue.take_blocking queue) ;;
val domain : '_weak2 Domain.t = <abstr>
utop # Option.get (Loc.get fiber) |> Fiber.computation |> fun (Computation.Packed computation) ->
Computation.cancel computation (Exn_bt.get_callstack 0 (Failure "I got you!")) ;;
- : unit = ()
utop # Domain.join domain ;;
Exception: Failure "I got you!".

It all works! (The blocking take from a queue blocked the domain and we could even cancel the domain via Picos — normally such cancelation would happen indirectly through something else implemented in Picos like a structured concurrency mechanism.)

The primary audience for this is casual users. Users who are not yet intimately familiar with concepts such as schedulers or even experienced users that just want to quickly try something. Without some sort of default implementation it becomes much more involved to use anything implemented in Picos. Something along the lines of:

1. You first need to understand why your attempt to use a library using Picos fails with an Effect.Unhandled exception.

2. Then you need to understand what a scheduler is.

3. Then you need to find and choose a scheduler, install it, require it...

4. Then you need to run your code inside the scheduler (even in a REPL).

I believe requiring the above will create a barrier to entry and adoption of libraries implemented in Picos.

The default implementation avoids all that hassle at a fairly small cost: relatively small amount of code and tiny bit of overhead per effect invocation to handle the Unhandled effects.

Now, I don't personally consider the amount of code the default implementation adds as significant too much, but I guess there are cases where it might be a concern. The tiny bit of overhead it adds is also mostly negligible, but I agree that in serious applications it is an unnecessary overhead.

Perhaps there is some other way to provide default implementations in a manner that requires opt-in, but allows that opt-in to be done conveniently even in a REPL environment.