build: add packages to dune-project + re-generate opam files

[maxRN]

This PR adds all dream packages to the dune-project file and also re-generates the opam files. There should be no actual functional differences introduced by this PR.
This prepares dream to more easily switch to a dune-only workflow in the future.

Also fixed a typo in the project's description (trivailly -> trivially)

The opam files mostly changed in the order of the fields, since they are now being generated by dune during the build step.

Open questions

I wasn't sure how to understand this dependency restriction ("conf-libev" {os != "win32"} | "ocaml" {os = "win32"}).
I would expect we need to depend on OCaml on OS and not just Windows, but for now I've replicated this in the dune-project file as

"conf-libev" {os != "win32"}
"ocaml" {os = "win32"}

The dune dependency specification isn't as expressive as opam's in this case.

---

I formatted the dune-project file with dune format-dune-file and manually copy pasted the long descriptions back in, since multi-line strings aren't well supported there at the moment.

[aantron]

Thanks! I'm skeptical of changing to a Dune-only workflow for now. I don't see the advantage of it, but there is the disadvantage that it's not opam's native format and it doesn't support everything that opam supports. In particular,

I wasn't sure how to understand this dependency restriction ("conf-libev" {os != "win32"} | "ocaml" {os = "win32"}).
I would expect we need to depend on OCaml on OS and not just Windows, but for now I've replicated this in the dune-project file as

This is a way to have a conf-libev dependency only on non-Windows. I don't remember what happens if you don't write the line this way, but the last part introduces a fake redundant dependency on OCaml on Windows to work around some problem we were observing.

The multiline strings in Dune are also regrettable. Overall, the dune-project format is pretty difficult to read.

I'm not inherently against Dune-only, but I don't think it's ready. Given its drawbacks, I don't think it's worth adding a generation step for the opam file and complicating the workflow in this repo in this way.

I do see you've fixed at least that one typo, though, in the word "trivially," and that's very welcome for sure :)

[maxRN]

Thanks! I'm skeptical of changing to a Dune-only workflow for now. I don't see the advantage of it, but there is the disadvantage that it's not opam's native format and it doesn't support everything that opam supports. In particular,

My reasoning was that since dune is planned to be the OCaml toll going forward, dream will probably switch to using it more heavily sooner or later. Specifically I'm thinking of the package management features that are being worked on in dune.
Switching to a more dune-focused workflow earlier would allow for an easier adoption of new features down the line.

---

The multiline strings in Dune are also regrettable. Overall, the dune-project format is pretty difficult to read.

Nothing I can say against that really... I also wish the multiline strings had a nicer syntax.

---

I'm not inherently against Dune-only, but I don't think it's ready. Given its drawbacks, I don't think it's worth adding a generation step for the opam file and complicating the workflow in this repo in this way.

I don't see how this would change or complicate the workflow. Instead of editing the individual opam files, one would now edit the information in the dune-project file. On dune build the opam files are always automatically re-generated.

[aantron]

The outcome of such plans is not pre-determined, but depends on the actual ability of Dune to convince the community at large to use it in this way, for part of which Dune also needs feedback such as from the conversation in this PR. As a maintainer of a relatively large and well-known project I don't think I could suggest Dune as a reasonable option to Dream's contributors. This is not because I am against it in principle, but because it is an extra moving part, something opaque, something that needs more work, something that, at present, appears slightly crufty in an unnecessary way. This isn't a hard no -- if Dune improves, or if the whole rest of the community is clearly moving towards Dune-only, then we will also go there, but I don't see it as a clear "yes" right now. It's not very clearly better on its own merits (but perhaps I'm not informed about some advantage of it), nor is it the factual community default. Rather, it's an aspirational project, largely about reorganizing and tidiness, but with clear trade-offs in such, and whose future is still undecided.

[aantron]

As an example, I'd like to avoid having to do things like this. I'm sure Dune-only will get there eventually, but at the moment, practical work with Dune-only for a library project that gets published to opam (as opposed to an app, especially one that is not open-source):

1. Have up to three files, instead of dream.opam, dream.opam, dune-project, dream.opam.template, and potentially likewise for all other .opam files.
2. We still publish .opam files to opam-repository. They get updated there, downstream, as the community discovers new constraints during package releases. I then take these .opam files and diff against them and use copy-paste+diff to discover what has changed since the last release, so I can upstream any contraints I haven't been informed about into the .opam file in this repo. This is all doable with mentally translating back to dune-project, but it's extra work, especially if dune-project starts reordering dependency lists, especially if it's according to changes in the version of Dune that is generating the files (though I'm sure they would tend to avoid that), if it ever creates fake blame because the file is generated and it's decided that it's not important to keep its history tidy.

[aantron]

There are more objections, I don't remember everything immediately, but I do stumble upon them in practical work, and the answer in all cases would have been to use opam's native format for opam, while we still use opam, and while Dune's opam generation is still developing, given that the opam format is not, in the end, awful.

[aantron]

Or to put it another way, once (and if) we are actually moving towards a world in which we can have only dune-project in each repo, losing nothing that way, then I think Dream is ready to adopt this, even if it has some remaining drawbacks that might be individually serious. But in the current tentative and transitional state, it's worse to adopt this, than not. It's more suitable for private or semi-private projects, apps (leaves in the dependency graph), etc.

[maxRN]

Thanks for sharing your perspective. This part especially was new to me as a non-library developer

We still publish .opam files to opam-repository. They get updated there, downstream, as the community discovers new constraints during package releases. I then take these .opam files and diff against them and use copy-paste+diff to discover what has changed since the last release, so I can upstream any contraints I haven't been informed about into the .opam file in this repo. This is all doable with mentally translating back to dune-project, but it's extra work, especially if dune-project starts reordering dependency lists, especially if it's according to changes in the version of Dune that is generating the files (though I'm sure they would tend to avoid that), if it ever creates fake blame because the file is generated and it's decided that it's not important to keep its history tidy.

---

The outcome of such plans is not pre-determined, but depends on the actual ability of Dune to convince the community at large to use it in this way

From my perspective it seems like it's very much decided that Dune is going to be the preferred tool for this in the future. I also think there is value in adopting a tool earlier, even if it's not perfect, and dealing with the papercuts, helping bring it's development forward. However I now see that from a library author's perspective this is not the most desirable outcome.

[aantron]

I'll take an updated look locally at what the Dune-only workflow is like these days.