server: UPS monitoring via NUT + PeaNUT dashboard

.gitignore

@@ -45,3 +45,6 @@ linux-server/backup/status/
 
 # Forgejo runner status JSON + last-state written at runtime (homepage card)
 linux-server/forgejo/runner-status/
+
+# PeaNUT runtime settings written by its UI
+linux-server/ups/peanut-config/

linux-server/HTTPS.md

@@ -224,6 +224,7 @@ side of the `ports:` mapping (`host:container`), not the host side.
 | filebrowser       | 80    | ✅ done       | none — works at root; proxy to container :80, **not** the old 8080 host map |
 | syncthing         | 8384  | ✅ done       | set `STGUIADDRESS=127.0.0.1:8384` (disables Syncthing's Host-header check, else `Host check error`); publish sync `:22000`/`:21027` on the **sidecar** (raw TCP/UDP, not via serve) |
 | glances           | 61208 | ✅ done       | **host-networked variant** — keep `network_mode: host`, sidecar proxies via `host.docker.internal`; widget url stays localhost |
+| peanut (UPS)      | 8097  | ✅ done       | host-networked variant like glances — PeaNUT must reach the loopback-only `upsd:3493`, sidecar proxies via `host.docker.internal`; widget url stays localhost |
 | adguard           | 80    | ✅ done       | UI at container :80 (not the 8083 host map); publish DNS `:53` tcp+udp on the **sidecar** (raw DNS, not via serve); no :443 so no DoH/serve conflict |
 | atvloadly         | 80    | ✅ done       | no `hostname:` on the app container — conflicts with `network_mode: service:...`; Apple TV discovery is unaffected by the shared netns since it goes through the host's avahi-daemon via bind-mounted sockets, not this container's own network |
 | nginx-proxy-mgr   | 81    | ✅ done       | host edge (binds `:80/:443/:81`); its **admin UI** is fronted by a host-gateway sidecar at `npm.<tailnet>`, while NPM itself stays the non-tailnet trusted-cert edge (see section below) |

linux-server/README.md

@@ -339,18 +339,32 @@ In NPM admin (`http://<server-ip>:81`):
        ```
     3. The status card on Homepage shows last-run time, status, and repo size; failures push to ntfy
 
-19. pi-hole | [GitHub](https://github.com/pi-hole/pi-hole) | [Docs](https://docs.pi-hole.net)
+19. UPS | [NUT](https://networkupstools.org/) | [Docs](https://networkupstools.org/docs/man/)
+    1. Battery-backup monitoring for the CyberPower PR1500LCDRT2U over USB — ntfy alerts on power events, clean shutdown on low battery, auto-restart when wall power returns. Full runbook in [`ups/README.md`](ups/README.md)
+    2. Deploy:
+       ```sh
+       sudo apt install nut   # or rerun the root setup.sh --profile server
+       cd linux-server/ups
+       cp .env.example .env   # set UPSMON_PASSWORD (openssl rand -hex 16) + ntfy
+       sudo bash setup.sh
+       upsc cyberpower ups.status   # expect: OL
+       docker compose up -d         # PeaNUT dashboard + homepage widget
+       ```
+    3. Dashboard (charge/load/runtime graphs) at `https://peanut.<tailnet>.ts.net/` (set `TS_AUTHKEY` in `ups/.env` for the sidecar); the homepage **ups** card reads it via the `peanut` widget on localhost
+    4. Set BIOS **Restore on AC Power Loss → Power On** so the server boots unattended after an outage
+
+20. pi-hole | [GitHub](https://github.com/pi-hole/pi-hole) | [Docs](https://docs.pi-hole.net)
     1. Network-wide DNS ad blocker — alternative to AdGuard Home
     2. Not yet configured
 
-20. Immich | [GitHub](https://github.com/immich-app/immich) | [Docs](https://immich.app/docs)
+21. Immich | [GitHub](https://github.com/immich-app/immich) | [Docs](https://immich.app/docs)
     1. Self-hosted photo and video backup — Google Photos alternative with mobile apps, face recognition, and timeline view
     2. Not yet configured
 
-21. Jellyfin | [GitHub](https://github.com/jellyfin/jellyfin) | [Docs](https://jellyfin.org/docs/)
+22. Jellyfin | [GitHub](https://github.com/jellyfin/jellyfin) | [Docs](https://jellyfin.org/docs/)
     1. Self-hosted media server — stream your own movies, TV shows, and music to any device
     2. Not yet configured
 
-22. Home Assistant | [GitHub](https://github.com/home-assistant/core) | [Docs](https://www.home-assistant.io/docs/)
+23. Home Assistant | [GitHub](https://github.com/home-assistant/core) | [Docs](https://www.home-assistant.io/docs/)
     1. Open source smart home hub — integrates with thousands of devices and services
     2. Not yet configured

linux-server/apt_packages.md

@@ -64,7 +64,10 @@ Tested on Ubuntu Server LTS. Most packages work on other Debian-based distros.
     1. Web-based server admin UI — system metrics, journal logs, network config, storage, and service management
     2. Enabled automatically via systemd socket activation on install; access at `https://<server-ip>:9090`
     3. Login with your Linux username and password
-19. sshpass | [apt](https://packages.ubuntu.com/search?keywords=sshpass) | [Homepage](https://sourceforge.net/projects/sshpass/)
+19. nut | [apt](https://packages.ubuntu.com/search?keywords=nut) | [Homepage](https://networkupstools.org/)
+    1. Network UPS Tools — monitors the CyberPower PR1500LCDRT2U over USB; ntfy alerts + clean shutdown on low battery
+    2. Needs one-time config after install: `sudo bash linux-server/ups/setup.sh` (see [`ups/README.md`](ups/README.md))
+20. sshpass | [apt](https://packages.ubuntu.com/search?keywords=sshpass) | [Homepage](https://sourceforge.net/projects/sshpass/)
     1. Non-interactive SSH password authentication — used by `SSH_and_GPG/add_remote_host.sh`
 
 ## Optional

linux-server/homepage/.env.example

@@ -45,6 +45,9 @@ HOMEPAGE_VAR_SYNCTHING_DOMAIN=syncthing.<tailnet>.ts.net
 # Glances — its own Tailscale sidecar name (glances.<tailnet>.ts.net), per HTTPS.md
 HOMEPAGE_VAR_GLANCES_DOMAIN=glances.<tailnet>.ts.net
 
+# PeaNUT (UPS dashboard) — its own Tailscale sidecar name (peanut.<tailnet>.ts.net), per HTTPS.md
+HOMEPAGE_VAR_PEANUT_DOMAIN=peanut.<tailnet>.ts.net
+
 # AdGuard Home — its own Tailscale sidecar name (adguard.<tailnet>.ts.net), per HTTPS.md
 HOMEPAGE_VAR_ADGUARD_DOMAIN=adguard.<tailnet>.ts.net
 

linux-server/homepage/config/services.yaml

@@ -53,6 +53,18 @@
             - field: repo_size_bytes
               label: Repo size
               format: bytes
+    - ups:
+        icon: mdi-battery-charging-high
+        href: https://{{HOMEPAGE_VAR_PEANUT_DOMAIN}}/
+        description: NUT · CyberPower 1500VA
+        server: my-docker
+        container: peanut
+        widget:
+          type: peanut
+          # peanut is host-networked (upsd is loopback-only); homepage is too,
+          # so localhost reaches it.
+          url: http://localhost:8097
+          key: cyberpower
     - glances:
         icon: si-glances
         href: https://{{HOMEPAGE_VAR_GLANCES_DOMAIN}}/

linux-server/post-install.md

@@ -252,6 +252,24 @@ git remote set-url origin ssh://git@forgejo.<tailnet>.ts.net:22/<username>/<repo
   ```
 - [ ] Get API key: Actions → Settings → API Key — add to `homepage/.env` as `HOMEPAGE_VAR_SYNCTHING_KEY`
 
+### UPS (NUT) — host service, no UI
+
+Monitors the CyberPower UPS over USB; alerts via ntfy and shuts the server down
+cleanly on low battery. Full runbook in [`ups/README.md`](ups/README.md).
+
+- [ ] Configure and deploy:
+  ```sh
+  cd linux-server/ups
+  cp .env.example .env   # set UPSMON_PASSWORD (openssl rand -hex 16) + ntfy
+  sudo bash setup.sh
+  ```
+- [ ] Verify: `upsc cyberpower ups.status` prints `OL`
+- [ ] Start the PeaNUT dashboard (`docker compose up -d`, needs `TS_AUTHKEY` in
+      `.env`) — graphs at `https://peanut.<tailnet>.ts.net/`, and the homepage
+      **ups** card goes live
+- [ ] Subscribe to the `server-ups` ntfy topic on your phone
+- [ ] Set BIOS **Restore on AC Power Loss → Power On**
+
 ---
 
 ## 8. Service reference
@@ -270,6 +288,7 @@ git remote set-url origin ssh://git@forgejo.<tailnet>.ts.net:22/<username>/<repo
 | Syncthing | http://\<server-ip\>:8384 | |
 | AdGuard Home | http://\<server-ip\>:8083 | Run setup wizard at :3003 first |
 | Cockpit | https://\<server-ip\>:9090 | |
+| PeaNUT (UPS) | https://peanut.\<tailnet\>.ts.net/ | Tailscale sidecar; homepage ups card reads it via localhost :8097 |
 | Tailscale Web UI | http://localhost:8088 | After `tailscale up` |
 | Tailscale proxy | http://localhost:8089 | Internal — used by Homepage widget |
 | Forgejo | https://forgejo.\<tailnet\>.ts.net/ | Tailscale sidecar (HTTPS via serve); Git over SSH on port 22 |

linux-server/ups/.env.example

@@ -0,0 +1,17 @@
+# Copy this file to .env and set the values. The .env is gitignored.
+
+# Local NUT password for the upsmon user (upsd listens on loopback only, but
+# keep it out of git anyway). Generate with: openssl rand -hex 16
+# Alphanumeric only — setup.sh renders it into /etc/nut configs with sed.
+UPSMON_PASSWORD=
+
+# ntfy alerts — base URL of your ntfy server (tailnet domain works from the
+# host) plus the topic. NTFY_TOKEN only needed if the topic requires auth.
+# Leave NTFY_URL empty to disable notifications.
+NTFY_URL=https://ntfy.<tailnet>.ts.net
+NTFY_TOPIC=server-ups
+NTFY_TOKEN=
+
+# Tailscale auth for the PeaNUT sidecar (peanut.<tailnet>.ts.net) — same OAuth
+# client secret as the other sidecars (Auth Keys scope + tag:container in the ACL).
+TS_AUTHKEY=

linux-server/ups/README.md

@@ -0,0 +1,92 @@
+# UPS (NUT)
+
+[Network UPS Tools](https://networkupstools.org/) monitoring for the CyberPower
+PR1500LCDRT2U connected over USB (`usbhid-ups` driver, standalone mode). On
+power loss it pushes ntfy alerts; when the battery runs low it shuts the server
+down cleanly and tells the UPS to cut its outlets so everything restarts when
+wall power returns.
+
+## Setup
+
+1. Install NUT (the root `setup.sh --profile server` installs it via
+   `packages.json`, or):
+   ```sh
+   sudo apt install nut
+   ```
+2. Configure:
+   ```sh
+   cd linux-server/ups
+   cp .env.example .env
+   # set UPSMON_PASSWORD (openssl rand -hex 16) + the ntfy URL/topic
+   ```
+3. Deploy configs to `/etc/nut` and enable the services:
+   ```sh
+   sudo bash setup.sh          # add --dry-run to preview
+   ```
+4. Verify:
+   ```sh
+   upsc cyberpower ups.status   # expect: OL (on line power)
+   upsc cyberpower               # full variable dump — charge, runtime, load
+   journalctl -u nut-monitor -f
+   ```
+5. Test a notification (safe — no shutdown involved):
+   ```sh
+   sudo -u nut NOTIFYTYPE=ONBATT /etc/nut/ups-notify.sh "test event"
+   ```
+6. In BIOS, set **Restore on AC Power Loss → Power On** so the machine boots
+   when the UPS re-energizes its outlets after an outage.
+
+## What happens during an outage
+
+1. Wall power drops → upsmon logs `ONBATT` and pushes an urgent ntfy alert.
+2. Power returns before the battery runs low → `ONLINE` alert, nothing else.
+3. Battery hits the low threshold → upsmon runs `SHUTDOWNCMD`
+   (`shutdown -h +0`), sets `/etc/killpower`, and the Debian shutdown hook
+   tells the UPS to power off its outlets after the OS halts.
+4. Wall power returns → the UPS re-energizes, and with the BIOS set to
+   power-on the server boots unattended.
+
+## Tuning
+
+- **Low-battery threshold**: CyberPower firmware fires LOWBATT late (~10%
+  charge). For more headroom, uncomment in `ups.conf` and re-run `setup.sh`:
+  ```
+  override.battery.charge.low = 25      # shut down at 25% charge
+  override.battery.runtime.low = 300    # or at 5 minutes runtime left
+  ```
+- Poll/timing knobs (`POLLFREQ`, `HOSTSYNC`, `DEADTIME`, `FINALDELAY`) are at
+  conventional values in `upsmon.conf.template`.
+- Brief blips send an ONBATT + ONLINE alert pair. If that gets noisy,
+  `upssched` can debounce (only alert after N seconds on battery) — not wired
+  up; layer it in later if needed.
+
+## Dashboard (PeaNUT)
+
+[PeaNUT](https://github.com/Brandawg93/PeaNUT) serves a web dashboard with
+charge/load/runtime graphs at `https://peanut.<tailnet>.ts.net/` (Tailscale
+sidecar per [`../HTTPS.md`](../HTTPS.md), host-networked variant like glances —
+PeaNUT itself must stay on the host network to reach the loopback-only `upsd`)
+and feeds the homepage **ups** card (`type: peanut` widget via localhost).
+Auth is disabled (read-only stats on a trusted network, same posture as
+glances). Its runtime settings dir (`peanut-config/`) is gitignored.
+
+```sh
+cd linux-server/ups
+# set TS_AUTHKEY in .env (same OAuth client secret as the other sidecars)
+docker compose up -d
+```
+
+## Files
+
+| Repo file | Deployed to | Purpose |
+|---|---|---|
+| `nut.conf` | `/etc/nut/nut.conf` | `MODE=standalone` (turns NUT on) |
+| `ups.conf` | `/etc/nut/ups.conf` | `usbhid-ups` driver for the CyberPower |
+| `upsd.conf` | `/etc/nut/upsd.conf` | upsd listens on loopback only |
+| `upsd.users.template` | `/etc/nut/upsd.users` | upsmon user (password from `.env`) |
+| `upsmon.conf.template` | `/etc/nut/upsmon.conf` | shutdown + notification policy |
+| `ups-notify.sh` | `/etc/nut/ups-notify.sh` | NOTIFYCMD → ntfy |
+| (rendered from `.env`) | `/etc/nut/ups-notify.env` | ntfy settings for the hook |
+
+Everything in `/etc/nut` is `root:nut 640` (the notify script `750`); secrets
+live only in the gitignored `.env` and the rendered `/etc/nut` files.

linux-server/ups/docker-compose.yml

@@ -0,0 +1,46 @@
+services:
+  # Tailscale sidecar — joins the tailnet as `peanut`, terminating HTTPS for
+  # https://peanut.<tailnet>.ts.net. Host-networked variant per ../HTTPS.md:
+  # peanut stays on the host network (it must reach the loopback-only upsd),
+  # so the sidecar runs in its own netns and proxies back to the host's :8097
+  # via host.docker.internal.
+  peanut-ts:
+    image: tailscale/tailscale:latest
+    container_name: peanut-ts
+    hostname: peanut
+    environment:
+      - TS_AUTHKEY=${TS_AUTHKEY}
+      - TS_STATE_DIR=/var/lib/tailscale
+      - TS_SERVE_CONFIG=/config/serve.json
+      # Requires TS_AUTHKEY to be allowed to apply tag:container (OAuth client, or
+      # a tag-scoped auth key). Drop this line if using an untagged personal key.
+      - TS_EXTRA_ARGS=--advertise-tags=tag:container
+    volumes:
+      - ./ts-state:/var/lib/tailscale
+      - ./ts-serve.json:/config/serve.json:ro
+    devices:
+      - /dev/net/tun:/dev/net/tun
+    cap_add:
+      - NET_ADMIN
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    restart: unless-stopped
+
+  # NUT web dashboard (charge/load/runtime graphs) — also feeds the homepage
+  # "ups" card's peanut widget. Host-networked because upsd listens on
+  # loopback only; the web UI binds 0.0.0.0:8097 so both the sidecar (via the
+  # docker bridge gateway) and the LAN can reach it.
+  peanut:
+    image: brandawg93/peanut:latest
+    container_name: peanut
+    restart: unless-stopped
+    network_mode: host
+    environment:
+      - NUT_HOST=127.0.0.1
+      - NUT_PORT=3493
+      - WEB_HOST=0.0.0.0
+      - WEB_PORT=8097
+      # Read-only UPS stats on a trusted LAN — same posture as glances.
+      - AUTH_DISABLED=true
+    volumes:
+      - ./peanut-config:/config

linux-server/ups/nut.conf

@@ -0,0 +1 @@
+MODE=standalone

linux-server/ups/setup.sh

@@ -0,0 +1,95 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Deploys NUT (Network UPS Tools) config for the CyberPower PR1500LCDRT2U to
+# /etc/nut and enables the services. Idempotent — re-runs are no-ops unless a
+# config changed. Run as root: sudo bash setup.sh [--dry-run]
+
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
+
+DRY_RUN=false
+[[ "${1:-}" == "--dry-run" ]] && DRY_RUN=true
+
+log() { printf '==> %s\n' "$*"; }
+die() { printf 'error: %s\n' "$*" >&2; exit 1; }
+run() {
+  if [[ "$DRY_RUN" == true ]]; then
+    printf '[dry-run] %s\n' "$*"
+  else
+    "$@"
+  fi
+}
+
+if [[ "$DRY_RUN" == false && $EUID -ne 0 ]]; then
+  die "must run as root: sudo bash $0"
+fi
+
+[[ -f "$SCRIPT_DIR/.env" ]] || die "no .env — cp .env.example .env, then set UPSMON_PASSWORD (openssl rand -hex 16)"
+set -a
+# shellcheck disable=SC1091
+source "$SCRIPT_DIR/.env"
+set +a
+
+[[ -n "${UPSMON_PASSWORD:-}" ]] || die "UPSMON_PASSWORD is empty in .env — generate with: openssl rand -hex 16"
+[[ "$UPSMON_PASSWORD" =~ ^[[:alnum:]]+$ ]] || die "UPSMON_PASSWORD must be alphanumeric (it is rendered into configs with sed)"
+
+if ! command -v upsdrvctl >/dev/null; then
+  if [[ "$DRY_RUN" == true ]]; then
+    printf 'warning: NUT is not installed yet — run: sudo apt install nut\n' >&2
+  else
+    die "NUT is not installed — run: sudo apt install nut (or rerun the root setup.sh)"
+  fi
+fi
+
+tmp="$(mktemp -d)"
+trap 'rm -rf "$tmp"' EXIT
+
+cp "$SCRIPT_DIR/nut.conf" "$SCRIPT_DIR/ups.conf" "$SCRIPT_DIR/upsd.conf" "$SCRIPT_DIR/ups-notify.sh" "$tmp/"
+sed "s|@UPSMON_PASSWORD@|$UPSMON_PASSWORD|" "$SCRIPT_DIR/upsd.users.template" > "$tmp/upsd.users"
+sed "s|@UPSMON_PASSWORD@|$UPSMON_PASSWORD|" "$SCRIPT_DIR/upsmon.conf.template" > "$tmp/upsmon.conf"
+{
+  printf 'NTFY_URL=%s\n' "${NTFY_URL:-}"
+  printf 'NTFY_TOPIC=%s\n' "${NTFY_TOPIC:-server-ups}"
+  printf 'NTFY_TOKEN=%s\n' "${NTFY_TOKEN:-}"
+} > "$tmp/ups-notify.env"
+
+changed=false
+deploy() {
+  local src="$1" dest="$2" mode="$3"
+  if [[ -f "$dest" ]] && cmp -s "$src" "$dest"; then
+    printf '  ✓ %s\n' "$dest"
+    return 0
+  fi
+  run install -o root -g nut -m "$mode" "$src" "$dest"
+  changed=true
+  printf '  installed %s\n' "$dest"
+}
+
+log "Deploying NUT configs to /etc/nut..."
+deploy "$tmp/nut.conf"        /etc/nut/nut.conf        640
+deploy "$tmp/ups.conf"        /etc/nut/ups.conf        640
+deploy "$tmp/upsd.conf"       /etc/nut/upsd.conf       640
+deploy "$tmp/upsd.users"      /etc/nut/upsd.users      640
+deploy "$tmp/upsmon.conf"     /etc/nut/upsmon.conf     640
+deploy "$tmp/ups-notify.env"  /etc/nut/ups-notify.env  640
+deploy "$tmp/ups-notify.sh"   /etc/nut/ups-notify.sh   750
+
+log "Enabling NUT services..."
+run systemctl enable --now nut-server.service nut-monitor.service
+
+if [[ "$changed" == true ]]; then
+  log "Configs changed — restarting NUT..."
+  run systemctl restart nut-server.service nut-monitor.service
+fi
+
+if [[ "$DRY_RUN" == false ]]; then
+  log "Verifying (driver can take a few seconds to settle)..."
+  sleep 3
+  if status="$(upsc cyberpower@localhost ups.status 2>/dev/null)"; then
+    log "ups.status: $status (OL = on line power)"
+  else
+    printf 'warning: upsc could not reach the UPS yet — check: journalctl -u nut-driver@cyberpower -u nut-server\n' >&2
+  fi
+  log "Test a notification with:"
+  printf '    sudo -u nut NOTIFYTYPE=ONBATT /etc/nut/ups-notify.sh "test event"\n'
+fi

linux-server/ups/ts-serve.json

@@ -0,0 +1,8 @@
+{
+  "TCP": { "443": { "HTTPS": true } },
+  "Web": {
+    "${TS_CERT_DOMAIN}:443": {
+      "Handlers": { "/": { "Proxy": "http://host.docker.internal:8097" } }
+    }
+  }
+}