Skip to content

About

NixOS flake for Lerd, the Herd-like local PHP dev environment. Packages the binary and ships the configuration.nix blocks the stack needs.

Topics

Resources

Stars

4 stars

Watchers

1 watching

Forks

Latest commit

 

History

42 Commits

Folders and files

Repository files navigation

lerd-nixos

Run Lerd, the Herd-like local PHP dev environment, declaratively on NixOS. One flake packages the binary and ships the configuration.nix blocks the stack needs.

Part of Lerd Docs License: MIT

Lerd runs PHP-FPM, nginx, a DNS resolver, and services (MySQL, Redis, and more) as rootless Podman containers managed by systemd user services (quadlets), plus two helpers (lerd-ui, lerd-watcher) that run the host lerd binary. That design assumes a Debian/Ubuntu-style host, so it needs a handful of NixOS-specific settings to work cleanly. This README is a complete, NixOS-only runbook from a fresh install to a working Laravel site.

Try it without installing

nix run github:lerd-env/lerd-nixos -- --help

Add it to your flake

The flake exposes a package (packages.x86_64-linux.lerd, also default) and an overlay (overlays.default).

1. Add the input

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";

    lerd = {
      url = "github:lerd-env/lerd-nixos";
      # Build lerd against your own nixpkgs instead of the one it pins,
      # so you don't download a second copy of nixpkgs:
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { self, nixpkgs, lerd, ... }: {
    nixosConfigurations.nixos = nixpkgs.lib.nixosSystem {
      system = "x86_64-linux";
      modules = [
        ./configuration.nix
        { environment.systemPackages = [ lerd.packages.x86_64-linux.default ]; }
      ];
    };
  };
}

(If your environment.systemPackages lives inside a module file instead of inline, pass lerd through specialArgs = { inherit lerd; }; and reference lerd.packages.${pkgs.system}.default there.)

Or use the overlay

{ pkgs, lerd, ... }:
{
  nixpkgs.overlays = [ lerd.overlays.default ];
  environment.systemPackages = [ pkgs.lerd ];
}

NixOS system configuration

Add the following to configuration.nix. Each block is explained below. Most of it is required; a couple of items are conditional and called out as such. Replace youruser with your username throughout.

{ config, lib, pkgs, ... }:

let
  # See block 9. Store paths, so the user unit does not need a login PATH.
  waitForDefaultRoute = pkgs.writeShellScript "wait-for-default-route" ''
    set -eu
    ip=${pkgs.iproute2}/bin/ip
    grep=${pkgs.gnugrep}/bin/grep
    sleep=${pkgs.coreutils}/bin/sleep

    until "$ip" -o route show default | "$grep" -q . \
       || "$ip" -o -6 route show default | "$grep" -q .; do
      "$sleep" 0.5
    done
  '';
in
{
  # 1. Rootless Podman, lerd runs everything in containers.
  virtualisation.podman.enable = true;
  virtualisation.containers.enable = true;

  # 2. Move Podman's default subnet pool off 10.x.
  #    REQUIRED ONLY IF a route on your machine claims 10.0.0.0/8 (common with
  #    corporate VPNs, check `ip route`). Podman's default pool lives in 10.x,
  #    and an overlapping route makes network creation fail with
  #    "could not find free subnet from subnet pools". Harmless to keep even
  #    without a VPN.
  virtualisation.containers.containersConf.settings.network.default_subnet_pools = [
    { base = "172.20.0.0/16"; size = 24; }
  ];

  # 3. Let rootless nginx bind 80/443. Without this, lerd asks for sudo to set
  #    the sysctl at runtime on every install; declaring it makes it permanent.
  boot.kernel.sysctl."net.ipv4.ip_unprivileged_port_start" = 80;

  # 4. Keep lerd's user containers alive outside an active graphical session
  #    (otherwise systemd-logind tears them down on lock/logout).
  users.users.youruser.linger = true;

  # 5. DNS for *.test, owned by NixOS, NOT by lerd (see notes below).
  #    Routes ONLY *.test to lerd's DNS container on 127.0.0.1:5300; everything
  #    else stays on your normal resolver, so a stopped lerd-dns can only ever
  #    break .test, never the whole internet.
  services.resolved.enable = true;
  services.resolved.settings.Resolve = {
    DNS = "127.0.0.1:5300";
    Domains = [ "~test" ];
  };
  networking.networkmanager.dns = "systemd-resolved";  # if you use NetworkManager

  # 6. Trust lerd's mkcert root CA system-wide (curl, PHP, Node, and more).
  #    The file doesn't exist yet on a fresh install, add this line AFTER the
  #    "First-time lerd setup" step below generates and copies it in.
  security.pki.certificateFiles = [ ./certs/lerd-rootCA.pem ];

  # 7. Point lerd's host-binary services at the Nix profile.
  #    lerd-ui and lerd-watcher exec `~/.local/bin/lerd`, where lerd would
  #    self-install on other distros. On NixOS the binary comes from the Nix
  #    profile, so without this symlink both fail with status=203/EXEC.
  #    /run/current-system/sw/bin/lerd tracks the current generation, so this
  #    survives nixos-rebuild and lerd updates.
  systemd.user.tmpfiles.rules = [
    "L %h/.local/bin/lerd - - - - /run/current-system/sw/bin/lerd"
  ];

  # 8. Host tools. PHP and Composer are optional (editor tooling and
  #    `lerd new`'s scaffold); lerd serves your app's PHP from containers
  #    regardless, pin the per-project version with `lerd isolate <ver>`.
  #    nssTools provides certutil, which mkcert uses to install the CA into
  #    Firefox and Chromium. The system store in block 6 does not cover them.
  environment.systemPackages = with pkgs; [
    nssTools
    php84
    php84Packages.composer
  ];

  # 9. Wait for a real default route before rootless containers start.
  #    Every rootless quadlet is ordered after
  #    podman-user-wait-network-online.service, whose stock command only
  #    polls network-online.target. On NixOS that target is reached by
  #    NetworkManager-wait-online (`nm-online -s`), which returns when
  #    NetworkManager has finished starting. On Wi-Fi that is often before
  #    a default route exists. Pasta then builds the rootless netns with no
  #    gateway and keeps it until the last container using it stops, so
  #    containers fail DNS (`getaddrinfo ... Try again`) until
  #    `lerd stop && lerd start`. A host that already has a route passes
  #    immediately. The unit's own 90s timeout still applies, so an offline
  #    boot is not held forever.
  #    https://github.com/containers/podman/issues/27976
  systemd.user.services.podman-user-wait-network-online = {
    overrideStrategy = "asDropin";
    serviceConfig.ExecStart = lib.mkForce [ "" waitForDefaultRoute ];
  };

  # 10. lerd 1.37 writes this into every PHP-FPM quadlet:
  #       ExecStartPre=/bin/mkdir -p -m 0700 %t/lerd/env
  #     NixOS's /bin contains only sh, so systemd reports
  #     "Unable to locate executable '/bin/mkdir'" (203/EXEC) and lerd
  #     reports "PHP x.y FPM quadlet: start lerd-phpXX-fpm failed". FPM
  #     never starts. /bin is a real directory; activation re-applies the
  #     symlink on every switch, after the script that creates /bin/sh.
  #     Drop this once lerd stops hardcoding /bin/mkdir
  #     (internal/podman/provided_env.go).
  system.activationScripts.lerd-bin-mkdir = {
    deps = [ "binsh" ];
    text = ''
      ln -sfn ${pkgs.coreutils}/bin/mkdir /bin/mkdir
    '';
  };
}

Why DNS is configured this way

This is the part that bites people. lerd is built for Debian/Ubuntu/Arch and, by default, imperatively rewrites your system resolver (a systemd-resolved drop-in and/or a NetworkManager dispatcher script) to point all DNS at its container. On those distros it works; on NixOS it fights the declarative config and, if the lerd-dns container isn't up, can take down all name resolution (you'll see Could not resolve host: cache.nixos.org even during nixos-rebuild).

The config above sidesteps that: NixOS owns the resolver and routes only ~test to lerd-dns. Because that routing lives in the main resolved.conf (which lerd never edits), it's the stable anchor. The consequence is that you should decline lerd's offer to configure DNS, see the next section.

First-time lerd setup

Do this once, in order. (You need working internet DNS first. If it's already broken from an earlier attempt, see Troubleshooting.)

  1. Apply the system config (without the cert line #6 yet, since the file doesn't exist):

    sudo nixos-rebuild switch
  2. Run the installer. Use --no-ipv6, it keeps the lerd network IPv4-only, which avoids a second class of subnet-pool errors on the IPv6 dual-stack bridge:

    lerd install --no-ipv6

    This generates the mkcert root CA at ~/.local/share/mkcert/rootCA.pem, writes the container quadlets, and starts dns/nginx/php-fpm.

    When it prints "Configuring NetworkManager dispatcher for .test DNS resolution" and asks for sudo, press Ctrl+C to decline. Your NixOS config (block #5) already resolves .test, so lerd doesn't need to touch DNS. Declining keeps it out of your resolver permanently.

  3. Trust the CA. Copy it into your config repo and enable line #6:

    mkdir -p /etc/nixos/certs   # or wherever your flake lives
    cp ~/.local/share/mkcert/rootCA.pem /etc/nixos/certs/lerd-rootCA.pem

    Uncomment/add security.pki.certificateFiles = [ ./certs/lerd-rootCA.pem ];, then:

    sudo nixos-rebuild switch

    (The CA cert is public and safe to commit; the matching private key stays in ~/.local/share/mkcert/, do not copy or commit it.)

  4. Verify the whole stack:

    resolvectl query app.test                       # → 127.0.0.1
    getent hosts cache.nixos.org                     # internet still resolves
    curl -sI https://lerd.test                       # TLS verifies against system CA
    systemctl --user is-active lerd-ui lerd-watcher  # → active (thanks to block #7)

Firefox and Chromium keep their own certificate databases. mkcert updates those with certutil from nssTools (block #8). Without that package the install leaves the browsers untrusting, even when curl accepts the system store from block #6. If a browser still warns, set security.enterprise_roots.enabled to true in about:config (Chromium then reads the system store), or import certs/lerd-rootCA.pem under Settings → Certificates → Authorities.

Create a Laravel project

lerd new myapp        # composer create-project laravel/laravel ./myapp
cd myapp
lerd isolate 8.4      # pin PHP version for this project (optional; 8.5 also available)
lerd link             # register ./myapp → myapp.test
lerd setup            # composer/npm install, .env, key, migrate, assets
lerd open             # open https://myapp.test

Handy follow-ups: lerd which (resolved PHP/docroot), lerd php artisan …, lerd db:create, lerd db:shell, lerd logs, lerd tui.

Troubleshooting

"could not find free subnet from subnet pools"

A route on your machine (usually a VPN) overlaps Podman's default 10.x pool. Confirm with ip route (look for a 10.0.0.0/8 line), then apply block #2 above and rebuild. If a half-created network is left over: podman network rm lerd before re-running lerd install --no-ipv6.

DNS is completely broken

Symptom: nothing resolves (Could not resolve host: cache.nixos.org), often right after a lerd command or reboot, and possibly blocking nixos-rebuild. This is lerd's imperative DNS setup. Recover at runtime:

# Remove lerd's resolver hooks
sudo rm -f /etc/systemd/resolved.conf.d/lerd.conf \
           /etc/NetworkManager/conf.d/lerd.conf \
           /etc/NetworkManager/dnsmasq.d/lerd.conf \
           /etc/NetworkManager/dispatcher.d/99-lerd-dns

# Reset and restart whichever resolver you run
sudo resolvectl revert <iface>          # e.g. enp6s0; ignore errors if resolved is off
sudo systemctl restart systemd-resolved || sudo systemctl restart NetworkManager

getent hosts cache.nixos.org

If it's still down and you need to rebuild, force a resolver temporarily:

sudo rm -f /etc/resolv.conf
printf 'nameserver 192.168.0.1\nnameserver 8.8.8.8\n' | sudo tee /etc/resolv.conf

The permanent fix is the NixOS DNS config (block #5) plus declining lerd's DNS prompt. Once that's in place this shouldn't recur. The only things that re-touch DNS are lerd install/lerd start (decline the prompt) and the lerd watcher (which only acts when .test is already broken).

"could not find ... lerd-nginx" on lerd link

lerd link only reloads nginx; the container is created by lerd install. If an earlier install failed partway, re-run lerd install --no-ipv6 (idempotent), then lerd start.

lerd-ui / lerd-watcher fail with status=203/EXEC

Their unit templates hardcode ExecStart=%h/.local/bin/lerd …, which doesn't exist on NixOS. Block #7 fixes it declaratively. To apply without a reboot:

mkdir -p ~/.local/bin
ln -sf /run/current-system/sw/bin/lerd ~/.local/bin/lerd
systemctl --user reset-failed lerd-ui lerd-watcher   # clears the start-limit lockout
systemctl --user restart lerd-ui lerd-watcher
systemctl --user is-active lerd-ui lerd-watcher       # → active

Editing the unit files instead won't survive: lerd install regenerates them from embedded templates, so the symlink is the durable fix.

A container won't start (start lerd-… failed)

Check the user service directly:

systemctl --user status lerd-<name> --no-pager
journalctl --user -xeu lerd-<name>.service --no-pager -n 50

Most "failed to start" cases on a fresh install trace back to missing linger (block #4), the unprivileged port sysctl (block #3), a missing /bin/mkdir on a PHP-FPM quadlet (block #10), or containers that started before the host had a default route (block #9).

PHP-FPM fails with Unable to locate executable '/bin/mkdir'

lerd 1.37 puts ExecStartPre=/bin/mkdir … on every PHP-FPM quadlet. NixOS does not ship /bin/mkdir, so the unit dies with status 203/EXEC and lerd reports PHP x.y FPM quadlet: start lerd-phpXX-fpm failed. Block #10 is the fix. After rebuilding:

systemctl --user reset-failed 'lerd-php*-fpm.service'
lerd start

Containers cannot resolve names after boot

Symptom: sites and lerd commands fail with getaddrinfo … Try again until lerd stop && lerd start (or a reboot that happens to win the race). Rootless Podman built its netns while NetworkManager was up but no default route existed yet, and it keeps that netns until the last container stops. Block #9 makes podman-user-wait-network-online.service wait for a route. It passes immediately when a route is already there.

lerd lan:share works on this machine and times out from another

NixOS drops inbound connections the firewall does not allow. lerd lan:share listens on port 9100 for the first shared site, then 9101, and so on. The proxy runs on the host, so the site loads locally either way.

networking.firewall.allowedTCPPortRanges = [
  { from = 9100; to = 9199; }
];

lerd lan:expose publishes nginx on 80 and 443 instead; allow those TCP ports the same way when you use it. The remote development page lists the remaining ports (DNS 5300, dashboard 7073) for ufw and firewalld. On NixOS those are networking.firewall.allowedTCPPorts and allowedUDPPorts. Leave the range closed when you are not sharing.


The DNS/cert/systemd integration notes above reflect getting lerd to coexist with NixOS's declarative model; they are not endorsed by upstream. Built and tested on x86_64-linux.

About

NixOS flake for Lerd, the Herd-like local PHP dev environment. Packages the binary and ships the configuration.nix blocks the stack needs.

Topics

Resources

Stars

4 stars

Watchers

1 watching

Forks

Contributors

Languages