chore: package as nix flake

[blaggacao]

RELNOTE: make kube-score discoverable as nix flake

You see a comment for the v1.7.2 tag, unless you'd not like to add it and force push this tag with that vendor sha, then let's call it supported from now on.

How to test it:

• check out this pr
• have nix installed curl -L https://nixos.org/nix/install | sh
• install nix flakes pre release: nix-env -iA nixpkgs.nixFlakes
• nix --experimental-features 'nix-command flakes' shell -c kube-score version
• nix --experimental-features 'nix-command flakes' shell -c which kube-score

Also interesting to check (if not specially configured, include the experimental flag from above):

• nix flake show github:blaggacao/kube-score
• nix shell github:blaggacao/kube-score (as a temporary solution for the inclined visitor out of the nix community until this is - maybe - merged. Be warned I'll not rebase or update my fork.)
• in the new shell where you are dropped in by the above command: which kube-score

$ nix flake show github:blaggacao/kube-score
github:blaggacao/kube-score/1a719d0597dcdb8eb5d57051eeed799e876e96bb
├───apps
│   ├───aarch64-linux
│   │   └───kube-score: app
│   ├───i686-linux
│   │   └───kube-score: app
│   ├───x86_64-darwin
│   │   └───kube-score: app
│   └───x86_64-linux
│       └───kube-score: app
├───defaultApp
│   ├───aarch64-linux: app
│   ├───i686-linux: app
│   ├───x86_64-darwin: app
│   └───x86_64-linux: app
├───defaultPackage
│   ├───aarch64-linux: package 'kube-score'
│   ├───i686-linux: package 'kube-score'
│   ├───x86_64-darwin: package 'kube-score'
│   └───x86_64-linux: package 'kube-score'
└───packages
    ├───aarch64-linux
    │   └───kube-score: package 'kube-score'
    ├───i686-linux
    │   └───kube-score: package 'kube-score'
    ├───x86_64-darwin
    │   └───kube-score: package 'kube-score'
    └───x86_64-linux
        └───kube-score: package 'kube-score'

$ kube-score version
kube-score version: development, commit: N/A, built: N/A

[blaggacao]

What goflags to be set to reference commit and built? kube-score version: development, commit: N/A, built: N/A

[zegl]

What goflags to be set to reference commit and built? kube-score version: development, commit: N/A, built: N/A

The pre built binaries are created with goreleaser, which sets the following ldflags during gobuild:

-s -w -X main.version={{.Version}} -X main.commit={{.Commit}} -X main.date={{.Date}} -X main.builtBy=goreleaser

Is there a way to build specific tags/refs with nix, or will it always try to build from the latest master?

[zegl]

As kube-score is using automated tooling for dependency management, especially for upgrades, I’d prefer if we didn’t have to specify this checksum. Is there a way to avoid it?

If it’s there, I’d assume that it will be out of date and that builds will start to fail as soon as go.mod is updated, or am I wrong here?

[blaggacao]

Your assumptions are both right. vendorSha256 = null does disable the paranoic check. However, paranoic checks are considered a good thing looking at it on the aggregate level. And, when I tried building it without it, there seems to be a bug somwhere in the enviornment when resolving some kubeapiserver packages to .. I can have another go to this, but it looked rather obscure. Not sure if this is a bug on nix tooling's side. Maybe not.

[blaggacao]

Unfortunately it looks like nixpkgs.buildGoPackage is built in a way so that it only can "trust" dependencies if they are vendored and checked into source.
See:
https://github.com/NixOS/nixpkgs/blob/0a2cff8e74baacfa8da9bad843699fe71a8c3053/pkgs/development/go-modules/generic/default.nix#L18-L20

So I'm afraid it looks like this is not possible, since I assume you neither want to vendor sources.

[blaggacao]

I think the root cause its that nix doesn't want to offload source of truth to another tooling. Even though Go arguably does a good job in reproducability and validation with go.sum. It's just that since those resources are fetched over the network, nix want's to make sure (on it's own) that it's in the box what it pretends to be.

[edolstra]

Maybe it's possible to do something similar to what we do with Rust, namely get the dependencies from Cargo.lock. That way you don't need a vendorSha256 in your flake.nix that could get out of date. See https://github.com/edolstra/import-cargo.

[blaggacao]

That sounds like it would be an excellent leverage for the nix community and would naturally lowers barrier of adoption.

[zegl]

I’m in favor of supporting more build systems and release channels to make it easier to get started with kube-score. So I’m supportive of this change, but I have some smaller questions (see the review).

[blaggacao]

Is there a way to build specific tags/refs with nix, or will it always try to build from the latest master?

Yes, on the caller side, use
github:zegl/kube-score/v1.7.3 instead of github:zegl/kube-score.

I have to warn that nix flakes are experimental, but I'd say this very basic syntax of input / output is unlikely to change. It's more likely that it gets more strongly typed and additional conventions arise which are not so relevant for simple tool packaging.

Note that - I think - github:zegl/kube-score/v1.7.2 wouldn't work since it doesn't yet contain a flake file.

[blaggacao]

This is as far as I could get:

Running from git:

$ nix shell github:blaggacao/kube-score
$ kube-score version
kube-score version: rolling, commit: aae617bb307a44f357298f24aebbcd207104a4d6, built: 20200719230910

Running locally:

$ nix shell --show-trace
$ kube-score version
kube-score version: rolling, commit: aae617bb307a44f357298f24aebbcd207104a4d6, built: 20200719230910

Running locally on dirty worktree:

$ nix shell --show-trace
$ kube-score version
kube-score version: rolling, commit: dirty, built: 20200718051326

Actually built more precisely directly references the time of the last commit, since between committing and building - thanks to nix flakes hermetic builds - we have absolute guaranty that no entropy is introduced. And if the built time would be actually eligible, it would, itself, be entropy to the build and break the whole essence of reproducible builds.

I'm trying to get advise on how to make this maintainable.

[zegl]

Thanks, I've been reading your advice thread and reading up on the status of Go in nix, and it seems like the community hasn't really made up its mind on how to handle dependencies for Go packages? (Recommend buildGoPackage instead of buildGoModule in the nixpkgs manual).

From this I've not been able to figure out how flakes for Go are supposed to be maintained in the long term, which is a shame.

We could merge this, but unless there's some simple way to update the pinned checksum (even if seems to be discouraged) with for example GitHub Actions the likelihood that this would keep working long term seems to be low. I've checked out the vgo2nix project which seems like it's able to do exactly this, but for buildGoPackage.

Do you have any suggestions for what to do?

[blaggacao]

I haven't disclaimed that I was trying to push boundaries and exploring new lands, for myself & for the nix community. And I am very greatful for your correspondance and interest!

You digged up some very interesting threads which I just finished to devoure.

I wouldn't be all too strict on fixed-output derivations since the problem at hand is scoped to "getting out that software to the nix community". Fixed-output derviations do the job.

The creator of nix argues:

Such impurities are bad for reproducibility because the dependencies on external files are completely implicit

That is true when you need discoverability and content adressability for your dependency graph. Especially the content adressability of the source graph avoids runaway storage requirements on caching build systems.

However, I would argue for the sole purpose of getting a build artifact distributed (the binary) in the nix package manager to end users (which conveniently pay the electricity bill for build resources themselves), ensuring the output integrity with fixed-output derivations fits the task.

I might add, that flakes are designed to decentrally cache those artifacts (similar to cachix). So say, in an organization setup, you would build it "at home" once and then distribute it through flakes caching to "the entires house".

This makes me think that those discussions are focused on two use cases: purity and (massive) build systems.

Flakes go a fundamentally different approach by encouraging de-centralization all over the place. This leaves us with "purity". But, since people know Go, they should be empowered to decide if they trust the go checksum mechanism enough. Assuming people put trust in Go's mechanism and by denying the massive build system use case, "purity" looses it's practical relevance for the average, distributed package maintainer.

I can faintly perceive on the horizon were this might be heading to: fixed-output hashes become part of the inputs of the flake, and since then are explicit and parseable part of the dependency graph. Although they still wouldn't convery the details a massive build system would wish to access for deduplication, they'd become an independent node on the dependency graph to reason about.

Let's not merge this. Let's keep it as a reference for the kind of problems that nix flakes ought to resolve to be a viable option in the wild for package authors, like you.

I dare to ping @edolstra since they are spear-heading flake's intellectual patronship.