[small, flakes] How to make a flake with a python library, supporting arbitrary Python versions?

[t184256]

For a python application, it's easy to find examples of flakes exporting a package.

But what if I'm packaging a library and I want it to support a range of python versions? What am I supposed to export (a function taking pythonPackages and producing my library) and which flake output should this function be exposed under?

[con-f-use]

Since this has only been briefly covered in NH29 without getting into flakes at all: @t184256 if you end up with a satisfying complete solution, would you care to post it here (or make a PR to include it in a 29-folder in this repo)?

I imagine a minimal flake.nix would look something like this:

{

  description = "Minimal example for Python Libraries as overlay";

  outputs = { self, nixpkgs }:  # implicitly gets `nixpkgs` from flake registry
  let
    system = "x86_64-linux";
    myPyLibOverlay = final: prev: {
      myPyLib = final.callPackage ./packages/myPyLib.nix { };
    };
  in
  {

    overlays = {
      customPyLibs = final: prev: {
        # pythonPackagesExtensions (PR #91850) merged Aug 6, 2022
        pythonPackagesExtensions = prev.pythonPackagesExtensions ++ [
          myPyLibOverlay
          #...
        ];
      };
    };

    nixosConfigurations = {
      someMachine = nixpkgs.lib.nixosSystem {
        inherit system;
        modules = [
          { nixpkgs.overlays = [ self.overlays.customPyLibs ]; }
          ./machines/some/configuration.nix 
        ];
      };
    };

  };

}

Please correct me if I'm wrong. I don't fully grok flakes and flake best practices yet.

[con-f-use]

Or using it in a flake package/app? Again, not sure if the import nixpkgs { overlays = ...; } part is evil or there's a better way.

[t184256]

I don't think the

import nixpkgs { overlays = ...; } part is evil or there's a better way.

I was asking that for a library I'm writing in the open, so I can share the resulting commit: t184256/asyncdbview@f0acee7

As I understood it, the minimal example would be something like this:

{
  description = "mylib/flake.nix example providing an overlay";
  outputs = { self, nixpkgs, flake-utils }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; overlays = [ overlay ]; };
        overlay = final: prev: {
          pythonPackagesExtensions =
            prev.pythonPackagesExtensions ++ [ mylib-pyextension ];
        };
        mylib-pyextension = pyFinal: pyPrev: {
          mylib = pkgs.callPackage mylib-package { python3Packages = pyFinal; };
        };
        mylib-package = {python3Packages}:
          python3Packages.buildPythonPackage {
            pname = "mylib";
            version = "0.0.1";
            src = ./.;
          };
      in
      {
        overlays.default = overlay;
      }
    );
}

{
  description = "myapp/flake.nix example using the overlay";
  inputs.mylib.url = "...";
  outputs = { self, nixpkgs, flake-utils, mylib }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs {
          inherit system;
          overlays = [ mylib.overlays.${system}.default ];
        };
        python3Packages = pkgs.python312Packages;  # <- non-standard
        myapp = python3Packages.buildPythonPackage {
          pname = "myapp";
          version = "0.0.1";
          src = ./.;
          propagatedBuildInputs = with python3Packages; [ mylib ];  # <-
        };
      in
      {
        packages.default = myapp;
      }
    );
}

[con-f-use]

Okay, so I played around a little bit, and I have a good bunch of questions related to this, that I think would be a good fit for a Nix Hour. @infinisil would you be willing to make a follow up on custom Python libraries and flakes? @t184256 would you participate in that? I could make it to any of the next two Nix Hours on 23rd ant 30th of May.

For a little heads-up, here is my experiment repo.

Since we haven't talked much about flakes on the Nix Hour it could also serve as a nice practical introduction to talk about it. I'd be willing to give a quick walk-through on air.

Concrete things, I have problems with:

• How do I write tests for flakes?
• How do I build nixosConfigurations (without nixos-rebuild)?
• How do I use nixosModules in nixosTest?
• Could nix install the lib and package editable when in nix develop mode?
• Can I specify a nixosConfiguration that I can be either a qemu-vm or target bare-metal hardware?

Some of these would nicely tie into topics you've covered on the Nix Hour. Also, I might be doing a few general beginners mistakes, that you people could be helpful in correcting.

[t184256]

I've since noticed that overlays shouldn't go under flake-utils.lib.eachDefaultSystem. Please refer to my template repository https://github.com/t184256/cookiecutter-python-nix for a hopefully better and more complete example of exporting an overlay with a python library (there's a reference output directory if you want to take a look at an example output).