Skip to content

Commit 79b2aeb

Browse files
committed
docs: add inventory file for multi-server-devnet skill
1 parent 8206d16 commit 79b2aeb

5 files changed

Lines changed: 232 additions & 13 deletions

File tree

.claude/skills/multi-server-devnet/SKILL.md

Lines changed: 27 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -50,12 +50,31 @@ Nothing about the servers is hardcoded. Establish these from the operator:
5050
Put those values in `scripts/devnet.env` (copy `scripts/devnet.env.example`;
5151
gitignored) instead of retyping them: the operator-side scripts source it via
5252
`scripts/devnet-env.sh`, and an env var exported in the shell still wins over the
53-
file. It is the one place a deployment's hosts, urls, and Grafana ids live.
54-
Per-devnet `NODES`/`SUBNETS` are recorded there too, but as the operator's
55-
inventory — the scripts take them as positional args, and the authority on a
56-
running devnet is always its own `genesis/config.yaml`
57-
(`ATTESTATION_COMMITTEE_COUNT`), which `start-devnet.sh` and `convert.sh` check
58-
against.
53+
file. It is the one place a deployment's urls and Grafana ids live.
54+
55+
**The servers themselves live in `scripts/devnet.inventory`** (copy
56+
`scripts/devnet.inventory.example`; gitignored) — name, ip, tags, and per-devnet
57+
`NODES`/`SUBNETS`, one row per host. Query it with `scripts/inventory.sh` rather
58+
than reading it by hand:
59+
60+
```bash
61+
inventory.sh --tag devnet-ab --field ip # ips, one per line, for a loop
62+
inventory.sh --tag devnet-ab --tag aggregator # AND across tags
63+
inventory.sh --tag validator # derived, see below
64+
SERVERS=$(inventory.sh --field name | tr '\n' ' ')
65+
```
66+
67+
Tag conventions: a `devnet-*` tag names the chain a host's nodes belong to (two
68+
hosts sharing one means the split-chain model, *not* two like-named devnets), and
69+
`aggregator` is a whole-server role. `validator` is **derived** — "has a `devnet-*`
70+
tag and is not tagged `aggregator`" — so it can never disagree with the aggregator
71+
tag. An unknown tag exits 2 with the known-tag list rather than returning nothing,
72+
because a typo that yields an empty loop reports success while doing nothing.
73+
74+
`NODES`/`SUBNETS` there are still the operator's inventory: the scripts take them
75+
as positional args, and the authority on a running devnet is always its own
76+
`genesis/config.yaml` (`ATTESTATION_COMMITTEE_COUNT`), which `start-devnet.sh` and
77+
`convert.sh` check against.
5978

6079
Per devnet, node `n` (0 ≤ n < NODES) on its host:
6180

@@ -473,7 +492,8 @@ Swap (persistent): `fallocate -l 16G /swapfile && chmod 600 && mkswap && swapon`
473492
| `promtail-config.sh` | operator | `NETWORK HOST_IP LOKI_PUSH_URL [N:client ...]` | Emit a per-host promtail.yml (docker_sd → central Loki, labels mirror prometheus, backlog guard) |
474493
| `sweep.sh` | operator | `CENTRAL_PROM_URL` | Cross-devnet audit: head/justified/finalized + client mix from the central Prometheus |
475494
| `deploy-finality-alert.sh` | operator | `[WEBHOOK_FILE]`; `METRICS_HOST`, `GRAFANA_*`, `PROM_DS_UID`; `DRY_RUN` | Render + ship the "lost finality" Slack alert to the central Grafana |
476-
| `devnet-env.sh` / `devnet.env.example` | operator | `$DEVNET_ENV`, `./devnet.env`, script dir | Load this deployment's hosts/urls/Grafana ids as defaults; exported vars win. Copy the example to `devnet.env` (gitignored) once |
495+
| `devnet-env.sh` / `devnet.env.example` | operator | `$DEVNET_ENV`, `./devnet.env`, script dir | Load this deployment's urls/Grafana ids as defaults; exported vars win. Copy the example to `devnet.env` (gitignored) once |
496+
| `inventory.sh` / `devnet.inventory.example` | operator | `[--tag T]... [--not-tag T]... [--field F] [--file P] [--count]`; `$DEVNET_INVENTORY`, `./devnet.inventory`, script dir | Select servers by tag from the fleet inventory (name, ip, tags, NODES, SUBNETS). `devnet-*` tags name the chain, `aggregator` is a server role, `validator` is derived (`devnet-*` and not `aggregator`). Unknown tag → exit 2 + known-tag list, so a typo can't masquerade as an empty fleet. Copy the example to `devnet.inventory` (gitignored) once |
477497

478498
**Grafana dashboards** (copy into the central Grafana's dashboards dir —
479499
`GRAFANA_DASHBOARDS_DIR`, *not* the provisioning tree; they auto-load in ~30s and

.claude/skills/multi-server-devnet/scripts/devnet.env.example

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -8,12 +8,6 @@
88
SERVERS="host-a host-b" # ssh targets, any count/names
99
SSH_USER=user # login user; docker is invoked with sudo
1010

11-
# Per-devnet size, as INVENTORY: the scripts take NODES/SUBNETS as positional
12-
# args, and the authority for a running devnet is its own genesis/config.yaml
13-
# (ATTESTATION_COMMITTEE_COUNT). Recorded here so a restart doesn't have to guess.
14-
# host-a: NODES=32 SUBNETS=4
15-
# host-b: NODES=16 SUBNETS=2
16-
1711
# --- central metrics / logs stack -------------------------------------------
1812
# sweep.sh reads CENTRAL_PROM_URL; the other two are args you pass to
1913
# prometheus-config.sh / promtail-config.sh.
Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
# The deployment's servers: what each one is called, where it is, what it belongs
2+
# to, and how big its devnet is. Copy to devnet.inventory (gitignored) and fill in.
3+
#
4+
# This is the operator's inventory, in the same sense devnet.env's values are: the
5+
# authority on a RUNNING devnet is always its own genesis/config.yaml
6+
# (ATTESTATION_COMMITTEE_COUNT), and the scripts still take NODES/SUBNETS as
7+
# positional args. Recorded here so a restart doesn't have to guess, and so
8+
# "which hosts are in devnet-c" has one answer instead of living in someone's head.
9+
#
10+
# Read it with inventory.sh rather than by hand:
11+
#
12+
# inventory.sh --tag devnet-ab --field ip # ips, one per line, for a loop
13+
# inventory.sh --tag devnet-ab --tag aggregator # AND: both tags must be present
14+
# inventory.sh --tag validator # derived, see TAGS below
15+
# SERVERS=$(inventory.sh --field name | tr '\n' ' ')
16+
#
17+
# FORMAT — whitespace-separated columns, aligned however you like. Blank lines and
18+
# lines whose first non-space character is '#' are ignored. Five columns:
19+
#
20+
# name ssh target / display name. Need not equal the machine's hostname,
21+
# and often doesn't after a host is repurposed -- inventory.sh never
22+
# resolves it, so a stale name here misroutes every ssh that uses it.
23+
# ip address. The one field worth re-deriving from the fleet rather than
24+
# trusting after a rebuild.
25+
# tags COMMA-separated, NO spaces (a space would start the next column).
26+
# nodes NODES on this server, or '-' if it runs no devnet.
27+
# subnets ATTESTATION_COMMITTEE_COUNT, or '-'.
28+
#
29+
# TAGS — free-form, with two conventions inventory.sh knows about:
30+
#
31+
# devnet-* names the chain this server's nodes belong to. Two servers sharing
32+
# one tag means they share one genesis, i.e. the split-chain model
33+
# (nodes 0..N on one, N+1.. on the other) -- NOT two devnets that
34+
# happen to be named alike. Also the `network` Prometheus label.
35+
# aggregator this server runs the aggregator duty. A whole-server role, not a
36+
# per-node one.
37+
#
38+
# `validator` is DERIVED, never written here: it means "has a devnet-* tag and
39+
# is not tagged aggregator". Writing it as a literal tag would let it disagree
40+
# with the aggregator column, so inventory.sh computes it instead.
41+
#
42+
# The example below shows both supported topologies at once: host-a + host-b share
43+
# the chain `devnet-ab` (64 nodes total, 32 each), host-c runs its own independent
44+
# `devnet-c`, and metrics-1 carries the Grafana/Prometheus/Loki stack and no nodes.
45+
46+
# name ip tags nodes subnets
47+
host-a 10.0.0.1 devnet-ab,aggregator 32 8
48+
host-b 10.0.0.2 devnet-ab 32 8
49+
host-c 10.0.0.3 devnet-c 16 2
50+
metrics-1 10.0.0.4 tooling - -
Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
#!/bin/bash
2+
# Select servers from the deployment's inventory by tag.
3+
#
4+
# inventory.sh [--tag T]... [--not-tag T]... [--field F] [--file P] [--count]
5+
#
6+
# Lookup order, first hit wins: --file, $DEVNET_INVENTORY, ./devnet.inventory,
7+
# <scripts dir>/devnet.inventory. Copy devnet.inventory.example to
8+
# devnet.inventory (gitignored) and fill it in; that file documents the format.
9+
#
10+
# --tag T keep rows carrying T. Repeatable, and a single argument may hold
11+
# several whitespace-separated tags. All of them must match (AND).
12+
# --not-tag T drop rows carrying T. Same repeat/multi-value rules.
13+
# --field F name | ip | tags | nodes | subnets | all. Default prints name +
14+
# ip aligned; `all` prints every column aligned.
15+
# --count print how many rows matched, nothing else.
16+
#
17+
# `validator` is a DERIVED tag: "has a devnet-* tag and is not tagged aggregator".
18+
# It is computed, not read, so it can never disagree with the aggregator tag.
19+
# `--tag validator` is exactly `--tag <that devnet> --not-tag aggregator`.
20+
#
21+
# An unknown tag is an ERROR, not an empty result. A typo ('devnet5' for
22+
# 'devnet-5') that quietly returns no hosts turns `for h in $(inventory.sh ...)`
23+
# into a loop that does nothing and reports success, which is the same class of
24+
# failure devnet-env.sh guards against: acting on the wrong deployment, silently.
25+
# A tag that IS known but matches nothing is likewise exit 1 -- a real fleet has
26+
# no empty groups, so an empty match means the inventory is stale.
27+
set -u
28+
29+
usage() { sed -n '2,26p' "$0" | sed 's/^# \{0,1\}//'; exit "${1:-0}"; }
30+
31+
file=""; field="default"; count=0
32+
tags=(); nottags=()
33+
34+
while [ $# -gt 0 ]; do
35+
case $1 in
36+
--tag) [ $# -ge 2 ] || { echo "$1 needs a value" >&2; exit 2; }
37+
# Split on whitespace so --tag 'devnet-5 aggregator' is two tags.
38+
for t in $2; do tags+=("$t"); done; shift 2 ;;
39+
--not-tag) [ $# -ge 2 ] || { echo "$1 needs a value" >&2; exit 2; }
40+
for t in $2; do nottags+=("$t"); done; shift 2 ;;
41+
--field) [ $# -ge 2 ] || { echo "$1 needs a value" >&2; exit 2; }
42+
field=$2; shift 2 ;;
43+
--file) [ $# -ge 2 ] || { echo "$1 needs a value" >&2; exit 2; }
44+
file=$2; shift 2 ;;
45+
--count) count=1; shift ;;
46+
-h|--help) usage 0 ;;
47+
*) echo "unknown argument: $1" >&2; usage 2 ;;
48+
esac
49+
done
50+
51+
case $field in
52+
name|ip|tags|nodes|subnets|all|default) ;;
53+
*) echo "--field $field: expected name|ip|tags|nodes|subnets|all" >&2; exit 2 ;;
54+
esac
55+
56+
# An explicit path that doesn't exist is a typo, not a reason to fall back to some
57+
# other inventory and act on the wrong fleet -- same rule devnet-env.sh applies.
58+
if [ -n "$file" ]; then
59+
[ -f "$file" ] || { echo "--file $file does not exist" >&2; exit 2; }
60+
elif [ -n "${DEVNET_INVENTORY:-}" ]; then
61+
file=$DEVNET_INVENTORY
62+
[ -f "$file" ] || { echo "DEVNET_INVENTORY=$file does not exist" >&2; exit 2; }
63+
else
64+
dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
65+
for candidate in "./devnet.inventory" "$dir/devnet.inventory"; do
66+
[ -f "$candidate" ] && { file=$candidate; break; }
67+
done
68+
[ -n "$file" ] || {
69+
echo "no inventory file found (tried \$DEVNET_INVENTORY, ./devnet.inventory, $dir/devnet.inventory)" >&2
70+
echo "copy $dir/devnet.inventory.example to $dir/devnet.inventory and fill it in" >&2
71+
exit 2
72+
}
73+
fi
74+
75+
awk -v want="${tags[*]-}" -v nowant="${nottags[*]-}" \
76+
-v field="$field" -v docount="$count" -v src="$file" '
77+
# Tags are matched against ",a,b," so "agg" never matches "aggregator".
78+
function hastag(t, x) {
79+
if (x == "validator")
80+
return (index(t, ",devnet-") > 0 && index(t, ",aggregator,") == 0)
81+
return index(t, "," x ",") > 0
82+
}
83+
BEGIN { nw = split(want, W, " "); nn = split(nowant, NW, " ") }
84+
85+
{ sub(/^[[:space:]]+/, "") }
86+
/^#/ || /^$/ { next }
87+
88+
{
89+
if (NF < 3) {
90+
# Named rather than skipped in silence: a row the parser drops is a host
91+
# that vanishes from every tag it belonged to.
92+
printf "%s:%d: ignoring '\''%s'\'', need at least name/ip/tags\n", src, FNR, $0 > "/dev/stderr"
93+
next
94+
}
95+
name = $1; ip = $2; tags = $3
96+
nodes = (NF >= 4 ? $4 : "-")
97+
subnets = (NF >= 5 ? $5 : "-")
98+
t = "," tags ","
99+
100+
# Every literal tag in the file, so a typo can be told from a real absence.
101+
n = split(tags, TT, ",")
102+
for (i = 1; i <= n; i++) if (TT[i] != "") seen[TT[i]] = 1
103+
104+
for (i = 1; i <= nw; i++) if (!hastag(t, W[i])) next
105+
for (i = 1; i <= nn; i++) if ( hastag(t, NW[i])) next
106+
107+
m++
108+
N[m] = name; I[m] = ip; T[m] = tags; O[m] = nodes; S[m] = subnets
109+
if (length(name) > wN) wN = length(name)
110+
if (length(ip) > wI) wI = length(ip)
111+
if (length(tags) > wT) wT = length(tags)
112+
if (length(nodes) > wO) wO = length(nodes)
113+
}
114+
115+
END {
116+
bad = 0
117+
for (i = 1; i <= nw; i++)
118+
if (W[i] != "validator" && !(W[i] in seen)) {
119+
printf "no such tag in %s: %s\n", src, W[i] > "/dev/stderr"; bad = 1
120+
}
121+
for (i = 1; i <= nn; i++)
122+
if (NW[i] != "validator" && !(NW[i] in seen)) {
123+
printf "no such tag in %s: %s\n", src, NW[i] > "/dev/stderr"; bad = 1
124+
}
125+
if (bad) {
126+
printf "known tags: " > "/dev/stderr"
127+
for (k in seen) printf "%s ", k > "/dev/stderr"
128+
printf "(+ derived: validator)\n" > "/dev/stderr"
129+
exit 2
130+
}
131+
132+
if (docount) { print m + 0; exit (m ? 0 : 1) }
133+
134+
for (i = 1; i <= m; i++) {
135+
if (field == "name") print N[i]
136+
else if (field == "ip") print I[i]
137+
else if (field == "tags") print T[i]
138+
else if (field == "nodes") print O[i]
139+
else if (field == "subnets") print S[i]
140+
else if (field == "all")
141+
printf "%-*s %-*s %-*s %-*s %s\n", wN, N[i], wI, I[i], wT, T[i], wO, O[i], S[i]
142+
else
143+
printf "%-*s %s\n", wN, N[i], I[i]
144+
}
145+
146+
if (m == 0) {
147+
printf "no host in %s matches%s%s\n", src, \
148+
(nw ? " tags: " want : ""), (nn ? " not: " nowant : "") > "/dev/stderr"
149+
exit 1
150+
}
151+
}
152+
' "$file"

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,3 +46,6 @@ finality-alerts.yaml
4646

4747
# Filled-in copy of the multi-server-devnet deployment env (names hosts/paths)
4848
devnet.env
49+
50+
# Filled-in copy of the multi-server-devnet server inventory (names hosts/ips)
51+
devnet.inventory

0 commit comments

Comments
 (0)