Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

init dream2nix integration templates + tests #8

Closed
wants to merge 2 commits into from

Conversation

DavHau
Copy link

@DavHau DavHau commented Dec 16, 2022

This PR initializes templates for integration with dream2nix and adds a test that can be executed via:
nix run .#dream2nix-integration-test.

I do not intend to complete this PR, but it can serve as a starting point for whoever wants to take on the work, and can be used for discussions.
The templates for the translator and builder are largely just copied from dream2nix/src/templates/..., and contain only minor modifications.

Usually we commit the modules for ecosystem support like these, directly to the dream2nix source itself, but as I understood you, @cdepillabout, the intention is to keep the stacklock2nix project separate. Therefore the PR adds the dream2nix modules to the stacklock2nix project itself and exposes them via the flake outputs. Feel free anytime to move this whole project inside dream2nix and maintain it from there, if you feel comfortable with that.

Some comments in the templates contain references to other files, like examples or specifications. Those references are relative to the source of the upstream dream2nix project.

In general, some more detailed guide on contributing ecosystem support to dream2nix can be found here

Short Summary

The purpose of the translator is to extract data from any given project source (by parsing the stack.yaml.lock for example) and return the extracted data via the dream-lock format, which is JSON serializable.
The specification of the dream-lock format can be found at dream2nix/src/specifications
A successful termination of the .#dream2nix-integration-test app implies that the dream-lock format produced by the translator is correct.

The purpose of the builder is to read the data represented by the dream-lock and generate nix packages and devShells from it.
The builder accesses the dream-lock indirectly through getter functions like getDependencies etc. which are provided by dream2nix (see builder template below).

Now there is a clean way and a dirty way of integrating stacklock2nix into dream2nix.
The dream-lock format contains fields like dependencies and sources, for which the format is strict. If those fields are used, it will:

  • allow dream2nix to take care of fetching dependencies
  • allow dream2nix to provides interfaces like inject and sourceOverrides allowing the user to manipulate the the dependency tree and sources
  • ensure compatibility between different translators and builders for haskell in dream2nix

Besides these strict fields, the dream-lock also offers the _subsystem field, which can be used as a free-form field. So the quick and dirty way would be to ignore some fields of the dream-lock and instead put data into _subsystem. That would work around some of the requirements of dream2nix but kill some features and general compatibility with other haskell modules.

It would be great to do it the clean way, so we can re-use the stacklock2nix builder for example with other translators like cabal-plan etc. and we won't have to do the work twice.

Feel free to comment if you have any more questions.

@DavHau DavHau marked this pull request as draft December 16, 2022 07:12
@cdepillabout
Copy link
Owner

cdepillabout commented Jan 17, 2023

Thanks a lot for sending this PR! Sorry I've taken a while to get back to you.

My original plan for the dream2nix / stacklock2nix integration was to have stacklock2nix setup as a low-level library that dream2nix could call into. So the dream2nix/builder.nix and dream2nix/translator.nix files could be maintained in the dream2nix repo, and stacklock2nix would be added as a flake input to dream2nix (or alternatively it could just be copied directly to dream2nix, if you want to minimize flake inputs in dream2nix).

stacklock2nix would provide all the low-level functionality that would be necessary for the dream2nix builder and translator. I'm not sure exactly what would be needed here, but I'm guessing some way to translate Haskell's stack.yaml and stack.yaml.lock files into JSON, so IFD could be avoided? And then some way to take those JSON files and build a full Haskell project? (Of course, if you wanted to fully eliminate IFD, you'd need some way to also convert the build plans for individual Haskell libraries into JSON, but I think that already exists in dream2nix?)

To be concrete, if you pass the stacklock2nix function a Haskell project directory, it outputs a few things that are intended to be used by the end-user:

{ inherit
stackYamlResolverOverlay
stackYamlExtraDepsOverlay
stackYamlLocalPkgsOverlay
suggestedOverlay
combinedOverlay
localPkgsSelector
pkgSet
devShell
;
# These are a bunch of internal attributes, used for testing.
# End-users should not rely on these. Treat these similar to
# `.Internal` modules in Haskell.
_internal = {
inherit
snapshotInfo
resolverParsed
;
};
}

I'm suggesting to add to this list a value like stackYamlAsJSON (which would effectively be stack.yaml, stack.yaml.lock, and all dependencies translated to JSON format) and buildFromJSON (which would take stackYamlAsJSON and produce a full package set, or whatever is needed by dream2nix). Then the builder and translator in dream2nix could directly use these two new functions.

This way, stacklock2nix could be easily(?) used in dream2nix, and stacklock2nix could continue to be used by end-users directly (for people that want to minimize dependencies).

Does this explanation make sense?

@cdepillabout
Copy link
Owner

I'm going to go ahead and close this, since I likely won't work on this in the near term, but I'd welcome anyone to pick this up and work towards what needs to be done here.

In #41 (comment) I talked a little about my vague plans for writing a cabalsolver2nix lib (similar to stacklock2nix, but using cabal.project as input instead of stack.yaml). I'll like to architect it so that it is much easier to use with dream2nix, and then use that architecture here in stacklock2nix as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants