diff --git a/.claude/hooks/guard-taboos-test.sh b/.claude/hooks/guard-taboos-test.sh index 1fb6769..ca0f185 100755 --- a/.claude/hooks/guard-taboos-test.sh +++ b/.claude/hooks/guard-taboos-test.sh @@ -192,6 +192,12 @@ check deny "python3 -c \"open('/dev/sda','wb').write(b'0'*4096)\"" check deny "perl -e 'open(D,\">\",\"/dev/nvme0n1\"); print D chr(0)'" check deny "node -e \"require('fs').writeFileSync('/dev/vda','')\"" +# --- SSH connection sharing (rules/ssh-connections.md) --------- +# heinzel's control socket must not read as key material, or +# every remote rm/mv/chmod sent with the standard options is +# denied (see the accepted false positives in the guard). +check pass 'ssh -o ControlMaster=auto -o ControlPath=~/.cache/heinzel/ssh-%C root@h "rm -f /var/tmp/old.log; chmod 644 /etc/motd"' + # --- must pass ------------------------------------------------- check pass 'fdisk -l' check pass 'sfdisk -l /dev/sda' diff --git a/.claude/hooks/guard-taboos.sh b/.claude/hooks/guard-taboos.sh index 9b9cdae..797ca25 100755 --- a/.claude/hooks/guard-taboos.sh +++ b/.claude/hooks/guard-taboos.sh @@ -77,6 +77,10 @@ # - `cp /etc/ssh/sshd_config /tmp/` is blocked although it only # reads the file — copy out via `cat /etc/ssh/sshd_config > # /tmp/copy` instead. +# - An ssh ControlPath under .ssh/ makes any rm, mv or chmod in +# the same command look like a key operation. heinzel keeps +# its sockets in ~/.cache/heinzel for that reason +# (rules/ssh-connections.md). # # Being blocked is EXPECTED behavior. Explain it to the user. # Never rephrase, re-quote, or otherwise obfuscate a command to diff --git a/.claude/settings.json b/.claude/settings.json index 3f4877c..5b61ea1 100644 --- a/.claude/settings.json +++ b/.claude/settings.json @@ -19,6 +19,15 @@ "timeout": 15 } ] + }, + { + "hooks": [ + { + "type": "command", + "command": "mkdir -p -m 700 \"$HOME/.cache/heinzel\"", + "timeout": 5 + } + ] } ], "PreToolUse": [ diff --git a/.claude/skills/heinzel-fleet-audit/SKILL.md b/.claude/skills/heinzel-fleet-audit/SKILL.md index d5d97a6..dc20978 100644 --- a/.claude/skills/heinzel-fleet-audit/SKILL.md +++ b/.claude/skills/heinzel-fleet-audit/SKILL.md @@ -55,7 +55,8 @@ servers" — that maps to single-host housekeeping. 3. **Probe in parallel.** For each in-scope host, run the probes from `references/probes.md` in a single batched - SSH command. Use `ssh -o BatchMode=yes -o ConnectTimeout=5`. + SSH command, with the standard options from `CLAUDE.md` → + SSH Options. Hosts that time out or refuse the connection go on a "skipped: unreachable" list. diff --git a/.claude/skills/heinzel-fleet-audit/references/probes.md b/.claude/skills/heinzel-fleet-audit/references/probes.md index 5bc32f4..4654caa 100644 --- a/.claude/skills/heinzel-fleet-audit/references/probes.md +++ b/.claude/skills/heinzel-fleet-audit/references/probes.md @@ -5,7 +5,7 @@ read-only. Group them into a single SSH invocation per host to minimise round-trips: ```bash -ssh -o BatchMode=yes -o ConnectTimeout=5 USER@HOST ' +ssh USER@HOST ' echo "###ua###"; echo "###sshd###"; echo "###fw###"; diff --git a/CLAUDE.md b/CLAUDE.md index c7a3b32..013fc4f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -61,9 +61,33 @@ user frames as quick. ### SSH Options Always use these options on every SSH and -SCP/rsync-over-SSH command: +SCP/rsync-over-SSH command (for rsync inside +`-e "ssh …"`): - ssh -o BatchMode=yes -o ConnectTimeout=5 … + ssh -o BatchMode=yes -o ConnectTimeout=5 \ + -o ControlMaster=auto \ + -o ControlPath=~/.cache/heinzel/ssh-%C \ + -o ControlPersist=10m \ + -o ServerAliveInterval=15 -o ServerAliveCountMax=3 … + +They share one connection per host and remote user +across calls. A SessionStart hook creates the socket +directory; where hooks do not run (OpenCode), run +`mkdir -p -m 700 ~/.cache/heinzel` first. + +**Fresh-login options** — for access tests and the +single retry after a hanging call: + + ssh -o BatchMode=yes -o ConnectTimeout=5 \ + -o ControlMaster=no -o ControlPath=none … + +Use them **instead of** the standard options, never +appended to them: for a repeated option, SSH keeps +the first value it sees. + +Rate limits count connections, not commands: read +`rules/ssh-connections.md`. When SSH stops +answering, read `rules/ssh-unreachable.md`. ## Access Control (Blacklist & Read-Only) diff --git a/README.md b/README.md index 3714299..36e6a05 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,12 @@ changes — no changes until you say go. This is not needed for local administration (localhost / your own machine). + Heinzel shares one SSH connection per host and + keeps it open for 10 minutes after the last call. + The sockets live in `~/.cache/heinzel` (mode 0700), + so any process of your local user can use an open + connection without asking for the key again. + Quick setup: generate a key with `ssh-keygen`, copy it to the server with `ssh-copy-id user@host`, and test with `ssh user@host`. See the @@ -959,6 +965,10 @@ rules/ — Upstream rule files (git-tracked) privilege-escalation.md — Sudo, root SSH, unprivileged mode os-detection.md — OS detection procedure ssh-user.md — SSH username & language management + ssh-connections.md — Bundled, shared SSH connections + (rate limits count connections) + ssh-unreachable.md — No retry loops; blocked path vs + broken host server-memory.md — Server memory file format changelog.md — Session logging procedure activity-check.md — Recent-activity summary on connect diff --git a/rules/privilege-escalation.md b/rules/privilege-escalation.md index 96721f5..6ae89cc 100644 --- a/rules/privilege-escalation.md +++ b/rules/privilege-escalation.md @@ -34,10 +34,14 @@ the sudo flag. ## Root SSH Fallback When sudo is unusable and a privileged action is -needed, probe root SSH access once: +needed, probe root SSH access once, with the +fresh-login options (`CLAUDE.md` → SSH Options) — a +shared root connection opened earlier would answer +even if root login has been disabled since: ``` ssh -o BatchMode=yes -o ConnectTimeout=5 \ + -o ControlMaster=no -o ControlPath=none \ root@hostname "id" 2>&1 ``` diff --git a/rules/ssh-connections.md b/rules/ssh-connections.md new file mode 100644 index 0000000..9398278 --- /dev/null +++ b/rules/ssh-connections.md @@ -0,0 +1,118 @@ +# SSH Connections: Few and Shared + +Firewall rules that rate-limit new connections to +port 22 (`ufw limit`: 6 in 30 seconds; iptables +`recent` or `hashlimit`) and network IPS signatures +for SSH scans count **TCP connections, not +commands**, successful logins included. A busy +heinzel session can trip them and lock itself out, +and the block looks like a broken host. If that has +already happened, see `rules/ssh-unreachable.md`. + +fail2ban, sshguard and sshd's own +`PerSourcePenalties` (OpenSSH 9.8+) count failed and +aborted logins, not successful ones. Sharing changes +nothing for them; only avoiding failed logins does. + +## 1. Bundle commands + +One call per logical step, not one per command: + + ssh … host 'sh -s' <<'EOS' + cmd1 + cmd2 + EOS + +- Send several files in one `scp`/`rsync`. +- Do not poll a host every few seconds. Run a long + job on the host (`nohup`, `systemd-run`, `daemon`) + and read its log at an interval of minutes. +- `ProxyJump` costs a connection to the jump host + **and** one to the target. + +## 2. Share connections + +The standard options in `CLAUDE.md` → SSH Options +turn on OpenSSH connection sharing (`ControlMaster`): +repeated calls ride **one TCP connection per local +user, remote user, host and port**. Later calls are +several times faster, open no new connection, and +key agents that confirm each use ask only once. + +It does not help with the first connection per host +and remote user, with the first one after +`ControlPersist` expires, or with **failed logins**, +which are never shared. + +### Why these values + +Keep them as they are: + +- **Command line, not `~/.ssh/config`:** works on + every machine without setup and overrides a + `ControlMaster` block the user keeps for + themselves. +- **`~/.cache/heinzel/`, not `~/.ssh/` or `/tmp`:** + the taboo guard reads any path under `.ssh/` as key + material and would block every remote `rm`, `mv` or + `chmod`; `/tmp` is writable by other users. +- **`%C`:** a fixed-length hash. Readable names can + pass the 104-byte socket path limit on macOS, and + SSH then fails the call (`ControlPath too long`). +- **`ServerAliveInterval=15`, + `ServerAliveCountMax=3`:** retire a master with a + dead network path after 45 seconds, whatever + `~/.ssh/config` says. `ConnectTimeout` does not + cover a call over an existing connection. + +### Fresh-login options + +The second option set in `CLAUDE.md` → SSH Options. +Use it for: + +- **Access tests.** Anything that answers "can this + login still succeed?" — after changing + `authorized_keys`, an account, its shell or groups, + PAM, host keys, or firewall rules on the SSH port, + and the root SSH probe in + `rules/privilege-escalation.md`. A shared master + answers from the login **before** the change, so a + broken login reads as working. Make all related + changes first, then run one fresh-login call that + also prints what you need (`id; groups`). +- **A call that hangs or fails while sharing:** a + stale master. Follow `rules/ssh-unreachable.md`. + +`Session open refused by peer` is different: the +master is fine but full (sshd `MaxSessions`, default +10 per connection). Let your own parallel calls to +that host finish, then repeat the call as usual. + +### Caveats + +- **The socket directory must exist.** If it is + missing, every call fails *after* the login with + `unix_listener: cannot bind to path … No such file + or directory` and exit 255. The host is fine: + create the directory + (`mkdir -p -m 700 ~/.cache/heinzel`) and repeat + the call. +- **`scp` never starts a master.** It passes + `-oControlMaster=no` ahead of your options, so it + only reuses one. Open it with an `ssh` call first; + the onboarding already does. +- **Never close a master you did not start.** Other + heinzel sessions and scripts on the same local + account use the same socket path, and + `ssh -O exit`/`-O stop` ends their sessions too. + For experiments, use a complete option set with its + own path (e.g. `~/.cache/heinzel/test-%C`) and close + only that one. +- **Login records understate activity.** `last`, + `who` and the sshd log show one login for many + calls. Use `rules/activity-check.md` to judge + whether someone else is working on the host. +- **Security.** While a master is open, any process + of the same local user can use it without key + approval. Keep `~/.cache/heinzel` at `0700`; on a + shared account, shorten `ControlPersist`. diff --git a/rules/ssh-unreachable.md b/rules/ssh-unreachable.md new file mode 100644 index 0000000..37adb62 --- /dev/null +++ b/rules/ssh-unreachable.md @@ -0,0 +1,65 @@ +# When SSH Stops Answering + +A host that suddenly stops accepting SSH is often +fine: a filter on the way blocks the client, +frequently because of heinzel's own connections (see +`rules/ssh-connections.md`). + +## Do not retry in a loop + +Reconnecting on failure is exactly the pattern rate +limits and IPS rules punish, and it keeps an existing +block alive. + +1. Retry **once** with the fresh-login options + (`CLAUDE.md` → SSH Options) plus `-v`. A stale + shared connection is the cheap explanation, and + `-v` shows every address tried (see Dual-stack). +2. If that fails too, stop. Wait several minutes + before the next attempt, and never wrap SSH in an + automatic retry. + +## Target or path? + +Signs that something **on the way** blocks, not the +host: the address answers ping but the SSH port +times out (no `Connection refused`); `Connection +timed out during banner exchange`; it worked a few +times, then stopped, and recovers by itself after +minutes; another service on the same address still +answers: + + curl -sS -o /dev/null -w '%{http_code}\n' \ + https:/// + +Then the host is up and a filter on the client's +path blocks SSH. Wait, or tell the user. Do not +chase routing, NAT or MTU. A check from inside the +target's network sees a clean path and proves +nothing about the client's. Firewall and IPS +changes: `CLAUDE.md` → Firewall & network. + +## Dual-stack + +OpenSSH tries a host's addresses one after another, +usually IPv6 first, and moves on only when an +address fails or `ConnectTimeout` runs out. A filter +that drops one family therefore shows as a delay of +`ConnectTimeout` on every new connection, and as a +failure once both families are blocked. + +The `-v` retry shows it without an extra connection: + + debug1: Connecting to [
] port 22. + debug1: connect to address
port 22: + +One family timing out while the other connects +points to a per-family filter. A firewall or IPS +exception covers only the family written in it: +after adding one, test both with a fresh login +(`ssh -4 …`, `ssh -6 …`). + +Do not probe with `nc -z` or `ssh-keyscan`: fail2ban +(modes `ddos` and `aggressive`) and sshd's +`PerSourcePenalties` count a connection that never +logs in. A successful login counts for neither.