typst: add initial support for typst packages

[cherrypiejam]

This PR adds initial support to build Typst documents with Typst packages. In particular, it adds a Typst package builder and exposes it at the top level to ease the process of defining new Typst packages. It further adds a new function to the Typst derivation for creating a new Typst environment in which the Typst compiler understands a specific set of packages, namely typst.withPackages. Moreover, three Typst packages are defined and collected as a single attribution set as an initial demonstration.

Only a few Typst packages are included in this PR for a reason. The Typst Universe (the official site for Typst packages) maintains a single git repo that contains all Typst packages. Each package entry does not necessarily map to the same version specified in their respective git repo published by their authors (if there is). This is because the package authors need to manually move their package and copy the source, instead of a pointer, to the Typst Universe repo. Breaking the Typst Universe repo down to per package requires separate places to store each of them, and due to the aforementioned reason, the exact source may disagree with the original git repo ¹. This is why, in my opinion, we might want to collect our own set of Typst packages instead of directly pulling from the Typst Universe repo. For gradual adaption, typst.withPackages also provides a method to override the predefined set of Typst packages, namely typst.withPackages.override, making it easier for end users overriding the existing set of Typst packages and integrating new packages into their project.

All packages from the Typst Universe are included in this PR, along with a maintenance script. The script simply generates a set of nix expressions for all packages from the Typst Universe. Each individual package is now fetched as a tarball from the Typst Universe instead of their individual repository.

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

Footnotes

1. Assuming the author(s) also publish their package in another repo outside of the Typst Universe repo. ↩

[SigmaSquadron]

Welcome to Nixpkgs! This is truly amazing; I've wanted to build Typst packages for a very long time but never got around to it.

Note that the commits should be split as much as possible. Each new typstPackage should be added in a separate commit. The changes to the base typst package to wrap it should also be a separate commit.

[SigmaSquadron]

Regarding the actual package set, even though we can't import from Universe directly, it might still be a good idea to scrape it for information and build packages automatically. typst.toml gives us all of the necessary information to build each package.

[drupol]

Hello,

Thanks for tackling this, I have a few comments, let me know if you have questions.

@jtojnar : I know you're busy these days, but if you could have a look at this, it would be great.

Thanks!

[cherrypiejam]

Regarding the actual package set, even though we can't import from Universe directly, it might still be a good idea to scrape it for information and build packages automatically. typst.toml gives us all of the necessary information to build each package.

Ah that is great. Didn't know they already have that information included. I'll work on the script when I got time, but it will have to be waited for another week or so.

[SigmaSquadron]

Looks good to me, just one final open question for anyone who knows about packagesFromDirectoryRecursive.

[drupol]

Would it be a good idea to have some documentation for this ?

[cherrypiejam]

Would it be a good idea to have some documentation for this ?

I can write it, but where should I put it?

(Module updates) Added a release notes entry if the change is significant

Not sure if this change should be considered as significant

[cherrypiejam]

All packages from the Typst Universe is here, and I added a maintenance script to generate the attribution set for them. I guess right it's just the style and comments --- e.g., do I need to add some comments at the beginning of the generated list? I am up for suggestions.

Addressed I think.

[ShamrockLee]

We can preserve the flexibility to override programmatically fetched packages and add manually defined ones without modifying the auto-generated files. We could place the generated file and the manual package definitions inside pkgs/development/typst-packages, reserve pkgs/top-level/typst-packages.nix to register the manual packages, and add some logic to prepend the auto-generated ones.

For the auto-generated files, serialization formats like JSON or TOML would be more structural and contain less boilerplate code. (TOML would be more git diff friendly than JSON as it doesn't have the trailing-comma-forbidding issue).

[ShamrockLee]

We should consider applying lib.recurseIntoAttrs to pkgs.typstPackages to make it nix search-able and allow the CI and nixpkgs-review to check them. (The current CI shows no package addition, which is incorrect.)

There should also be a way to override/extend them (e.g., typstPackages.extend), as packages inside depend on each other.

Tother with #369283 (comment) , we could have a pkgs/by-name/ty/typst/typst-packages-combined.nix that looks like:

{
  lib,
  config,
  pkgs,
  typst ? pkgs.typst,
}:

# TODO: Add splicing support
lib.makeExtensible (
  final:
  let
    typstPackagesFromUniverse = lib.mapAttrs (
      name: packageSpec:
      final.buildTypstPackage {
        # Package definitions based on packageSpec
      }
    ) (lib.importTOML ./typs-packages-from-universe.toml);
  in
  lib.recurseIntoAttrs {

    inherit typst;

    buildTypstPackage = pkgs.callPackage ../../../build-support/build-typst-package.nix { };

    callPackage = lib.callPackageWith (pkgs // final);
    callPackages = lib.callPackagesWith (pkgs // final);

    inherit typstPackagesFromUniverse;
  }
  // typstPackagesFromUniverse
  // import ../../../top-level/typst-packages.nix { inherit lib config pkgs; } final
)

and pkgs/top-level/typst-packages.nix like:

{
  lib,
  config,
  pkgs,
}:
final: {
  # Manually defined/overridden Typst packages
}

[cherrypiejam]

This makes sense, but I am wondering the need for manually defined/overridden Typst packages in nixpkgs. It is possible for users to override some of packages from the Typst Universe due to outdated versions, or because they want to patch the package for some specific features. Thus it makes sense to make it extensible.

However, I don't see the reason to have a part for manually defined Typst packages in nixpkgs --- as we are not going to add new packages because we are assigning the Typst packages into the preview namespace in our typst environment and preview simply is for packages from the Typst Universe. Also currently there is no need to patch a single Typst package, as they are directly pulled from the official mirror site and they should work as they are. Because of aforementioned points, I say we leave that part behind and add it back when needed.

[cherrypiejam]

We should consider applying lib.recurseIntoAttrs to pkgs.typstPackages to make it nix search-able and allow the CI and nixpkgs-review to check them. (The current CI shows no package addition, which is incorrect.)

There should also be a way to override/extend them (e.g., typstPackages.extend), as packages inside depend on each other.

Tother with #369283 (comment) , we could have a pkgs/by-name/ty/typst/typst-packages-combined.nix that looks like[...]

Updated the PR accordingly. Did not include a customized typst package set simply because there is no need of doing so for now (even if we do, it's going to be an empty set). However, the package-combined is there so it would be easy to extend the package set when necessary.

[ShamrockLee]

Considering there's no need for separate registration of overriding, we could further simplify the typstPackages expression into

{
  lib,
  callPackage,
}:

lib.makeExtensible (
  final:
  lib.recurseIntoAttrs (
    lib.mapAttrs (
      name: packageSpec:
      callPackage (
        {
          lib,
          buildTypstPackage,
          fetchurl,
        }:
        buildTypstPackage {
          inherit (packageSpec) pname version;

          src = fetchurl {
            inherit (packageSpec) url hash;
          };

          sourceRoot = ".";

          typstDeps = builtins.filter (x: x != null) (
            lib.map (d: (lib.attrsets.attrByPath [ d ] null final)) packageSpec.typstDeps
          );

          meta = {
            inherit (packageSpec) description;
            maintainers = with lib.maintainers; [ cherrypiejam ];
            license = lib.map (lib.getAttr lib.licensesSpdx) packageSpec.license;
          } // (if packageSpec ? "homepage" then { inherit (packageSpec) homepage; } else { });
        }
      ) { }
    ) (lib.importTOML ./typst-packages-from-universe.toml)
  )
)

From your previous auto-generated Nix expression, I saw that you expect the Typst packages to take typstPackages directly, so there's no need to customize callPackage and callPackages like pythonPackages does. I added an inner callPackage to give each package its own <pkg>.override.

In my previous suggestion, the typstPackagesFromUniverse attribute allows registering an overridden package that could refer to the auto-generated, previous self as typstPackagesFromUniverse.<name>. I removed it here since we no longer need it.

Some minor changes include:

• builtins.map -> lib.map, as builtins.map is essentially an implementation of lib.map (faster than the ones implemented with the Nix Language).
• lib.getLicenseFromSpdxId -> lib.licensesSpdx: lib.getLicensesFromSpdxId falls back to { shortName = "<input SPDX ID>"; } with a warning message, which is uncatchable without turning all warnings into errors. lib.getLicenseFromSpdxOr lets the user decide on the default behavior. After that, we have lib.spdxLicenses, which directly exposes the restructured licenses set with SPDX ID as its keys. Previous discussions can be found in lib.meta.licensesSpdx: mapping from SPDX ID to compatible licenses #333953. I am sorry for the immature design decisions I made.
  The above expression throws an attribute-missing error when the SPDX ID isn't found. If that's too strict, you may want to change it.

The file can now be called typst-packages.nix, as it doesn't "combine" the overridden packages from the top-level registration.

[cherrypiejam]

Thanks for the suggestions! Have patched them accordingly

[ShamrockLee]

Now, we have an example of a license not found by its SPDX ID: packages licensed under EUPL-1.2+ couldn't find a suitable license representation under lib.licenses. Even though I think the EUPL-1.2+ case should be fixed from the lib.licenses side, typstPackages should also offer a way to handle such an exception.

[cherrypiejam]

Now, we have an example of a license not found by its SPDX ID: packages licensed under EUPL-1.2+ couldn't find a suitable license representation under lib.licenses. Even though I think the EUPL-1.2+ case should be fixed from the lib.licenses side, typstPackages should also offer a way to handle such an exception.

Right the actual case is a bit more complicated. Apparently SPDX could be expressions, which means that it has its own little grammar to construct a logic license through a set of SPDX IDs (specifically, named licenses). Typst packages allow expression strings while it is not possible to perfectly translate the logic expression to Nix, as licenses in Nixpkgs are a list of named licenses (not clear how to diff AND/OR/WITH relations). "+" is also a valid token to represent a single named license id, while the helper functions in nixpkgs do not support parse any of them.

The solution for now is simply adhoc --- unparsable SPDX IDs will be replaced to one or more parsable IDs (based on its semantics). This is hardcoded in the maintaining script. The logical relations are simply ignored due to the aforementioned reason in nixpkgs.

[cherrypiejam]

I see the WITH relation is encoded as an independent license in nixpkgs, but the same argument still apply for AND/OR and "+"