diff --git a/.claude/skills/type-fixes-preserve-behavior/SKILL.md b/.claude/skills/type-fixes-preserve-behavior/SKILL.md new file mode 100644 index 00000000..9d9a2b43 --- /dev/null +++ b/.claude/skills/type-fixes-preserve-behavior/SKILL.md @@ -0,0 +1,69 @@ +--- +name: type-fixes-preserve-behavior +description: A type-error fix may change types and names — never behavior. Use when clearing emmylua_check errors in the Beyond-All-Reason tree, or on any pass that renames locals, adopts a namespace (Spring.X -> BAR.X), or edits a file to satisfy the analyzer. It encodes the gui_chat.lua regression that shipped a widget that would not load. +--- + +# Type fixes preserve behavior + +`emmylua_check` reports a *type* problem. The fix is a type annotation, a declaration, or a name — never a restructured function, never a dropped field, never an inlined recomputation. A green analyzer on a widget that no longer loads is worse than the error it replaced. + +## The rule + +**Every edit is reversible into "same program, better typed."** If you cannot state the change as a rename, an annotation, or a declaration, you are no longer fixing a type error. + +## Renames are total or they are not done + +Renaming a local means renaming *every* reader in the file, in one pass. Count the references before and after — they must match. + +```lua +-- state table renamed I18N -> i18nStrings +local i18nStrings = state.i18nStrings -- renamed +... +local modeText = I18N.everyone -- NOT renamed: now a nil global +``` + +A partial rename is silent at load and crashes on the first draw. `grep -c` the old name after the edit; the answer is 0. + +## A namespace prefix is not noise + +`Spring.I18N(...)` -> `BAR.I18N(...)` is the migration. `BAR.I18N(...)` -> `I18N(...)` is a bug, unless the file declares `local I18N = BAR.I18N` — and if it does, that declaration must be in the same edit. + +Bare `I18N(` reads as a local alias. Grep the file for the `local ... = BAR.` line before assuming one exists. + +## Never drop a table field + +A key removed from a constructor is a runtime nil at every read site. Two keys (`channelScopeAll`, `label`) went missing from a table while its readers stayed — both would have returned nil forever. + +Diff the constructor key-set before and after. It only grows. + +## Never introduce shadowing recomputation + +Do not re-declare inside a closure what the enclosing function already computed. Fifteen such lines were inserted into a `glCreateList(function() ... end)`, recomputing `isCmd` *without* the `isLabel` branch the outer scope had — a behavior regression the analyzer is blind to. + +If a closure needs a value, it already has it as an upvalue. + +## Not every .lua file is Lua + +`mapgenerator/mapinfo_template.lua` is a `${PLACEHOLDER}` template. A bare +`${START_POSITIONS}` inside a table is a parse error, and commenting it out +silences the analyzer while breaking every generated map — the substituted +block lands behind a `--`. + +A file the analyzer cannot parse for a structural reason belongs in +`.emmyrc.json` `workspace.ignoreDir`, not in your edit set. Ask what the file +*is* before treating a diagnostic on it as a defect. + +## Repair to the intent, not to whatever is in scope + +`stompableDefs[udid] = v` — `v` leaked from a previous loop and was nil, so the +table was always empty. `ud` is in scope and makes the error go away; `true` is +what the code meant, because the only read is `if stompableDefs[unitDefID]`. + +When a table is used as a set, the value is `true`. Look at the read sites +before choosing the write. + +## Verify before committing + +- `luajit -bl >/dev/null` — syntax. +- `git diff -- ` — every changed line is a rename, an annotation, or a declaration. Line count does not grow. +- Load the game and read `infolog.txt` for `Failed to load:`. The analyzer cannot see a nil global that is only called at load. diff --git a/.emmyrc.json b/.emmyrc.json new file mode 100644 index 00000000..33012844 --- /dev/null +++ b/.emmyrc.json @@ -0,0 +1,7 @@ +{ + "workspace": { + "ignoreDir": [ + ".devtools" + ] + } +} diff --git a/.gitignore b/.gitignore index 9ed78731..8ab0d4bc 100644 --- a/.gitignore +++ b/.gitignore @@ -18,12 +18,15 @@ bar_debug_launcher repos.local.conf .env +# Build artifacts +target/ +bar-mission-kit/vscode/server/bar-mission-kit + # Runtime / editor state tasks/ .devtools/ .backups/ *.log -bar-lua-codemod/target/ .claude/settings.local.json .claude/projects/ .claude/todos/ diff --git a/README.md b/README.md index 0612edd7..c5a48e3e 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,6 @@ # BAR Devtools -Local development environment for [Beyond All Reason](https://www.beyondallreason.info/) -- spins up **Teiserver** (lobby server), **PostgreSQL**, **SPADS** (autohost), and **bar-lobby** (game client) with a single command. - -Everything server-side runs in Docker. The game client runs natively. +Shared development environment for [Beyond All Reason](https://www.beyondallreason.info/) — game code (Lua), engine (C++), lobby server (Elixir), and autohost (Perl) all from one repo. ## Quickstart (Linux) @@ -164,6 +162,14 @@ just bar::lx-shell # interactive lx shell for package work # (`lx add `, `lx sync`, `lx install`, etc.) ``` +> **⚠️ Merge conflicts with master?** The project ships deterministic code transforms (formatting, API renames, Spring split) that can be replayed onto any branch. Transform your branch first, then merge: +> ```bash +> just bar::migrate::stylua-cleanup # transform your branch first +> git commit -am "apply code transforms" # squashed away when PR merges +> git merge origin/master # conflicts are now real conflicts only +> ``` +> This is idempotent — safe to run multiple times. Includes `bar::fmt`, so no need to run it separately. + ### Teiserver development Tests run in a separate container with `MIX_ENV=test`, so they work whether or not `services::up` is running. The test database is independent from the dev database. @@ -243,6 +249,7 @@ just services::down # stop everything just services::logs teiserver # tail logs just services::shell teiserver # open bash inside the running container ``` +The SPADS bot account (`spadsbot` / `password`) is created automatically during Teiserver init. ## Requirements @@ -306,32 +313,15 @@ This runs a read-only check of your system dependencies, environment, ports, rep **Port 5432/5433 conflict with host PostgreSQL:** Either stop your local PostgreSQL (`sudo systemctl stop postgresql`) or change the port: +**Port conflict with host PostgreSQL:** ```bash BAR_POSTGRES_PORT=5434 just services::up ``` -**Teiserver takes forever on first run:** -The initial database seeding includes generating fake data. Follow progress with: -```bash -just services::logs teiserver -``` +**Teiserver takes forever on first run:** Initial DB seeding generates fake data. Follow progress with `just services::logs teiserver`. -**SPADS fails with "No Spring map/mod found":** -Game data download may have failed. Check logs and retry: -```bash -just services::logs spads -just services::down -just services::up spads -``` +**SPADS "No Spring map/mod found":** Game data download may have failed. `just services::down && just services::up spads`. -**Docker permission denied:** -```bash -sudo usermod -aG docker $USER -# Then log out and back in -``` +**Docker permission denied:** `sudo usermod -aG docker $USER` then log out and back in. -**Nuclear option -- start completely fresh:** -```bash -just services::reset -just services::up -``` +**Nuclear option:** `just services::reset && just services::up` diff --git a/bar-lua-codemod/Cargo.lock b/bar-lua-codemod/Cargo.lock new file mode 100644 index 00000000..dab1015f --- /dev/null +++ b/bar-lua-codemod/Cargo.lock @@ -0,0 +1,701 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "aho-corasick" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ddd31a130427c27518df266943a5308ed92d4b226cc639f5a8f1002816174301" +dependencies = [ + "memchr", +] + +[[package]] +name = "anstream" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "824a212faf96e9acacdbd09febd34438f8f711fb84e09a8916013cd7815ca28d" +dependencies = [ + "anstyle", + "anstyle-parse", + "anstyle-query", + "anstyle-wincon", + "colorchoice", + "is_terminal_polyfill", + "utf8parse", +] + +[[package]] +name = "anstyle" +version = "1.0.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "940b3a0ca603d1eade50a4846a2afffd5ef57a9feac2c0e2ec2e14f9ead76000" + +[[package]] +name = "anstyle-parse" +version = "1.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52ce7f38b242319f7cabaa6813055467063ecdc9d355bbb4ce0c68908cd8130e" +dependencies = [ + "utf8parse", +] + +[[package]] +name = "anstyle-query" +version = "1.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "40c48f72fd53cd289104fc64099abca73db4166ad86ea0b4341abe65af83dadc" +dependencies = [ + "windows-sys", +] + +[[package]] +name = "anstyle-wincon" +version = "3.0.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d" +dependencies = [ + "anstyle", + "once_cell_polyfill", + "windows-sys", +] + +[[package]] +name = "arc-swap" +version = "1.9.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c049c0be4daef0b145cb3555416b3b8ef5b7888a38aea1a3a155801fe7b0810b" +dependencies = [ + "rustversion", +] + +[[package]] +name = "bar-lua-codemod" +version = "0.1.0" +dependencies = [ + "clap", + "emmylua_parser", + "glob", +] + +[[package]] +name = "base62" +version = "2.2.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cd637ac531c60eb7fbc4684dc061c2d7d90d73d758181aa02eeff0464b9eee4b" + +[[package]] +name = "bitflags" +version = "1.3.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bef38d45163c2f1dde094a7dfd33ccf595c92905c8f8f4fdc18d06fb1037718a" + +[[package]] +name = "bstr" +version = "1.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1f7dc094d718f2e1c1559ad110e27eeaae14a5465d3d56dd6dbd793079fbd530" +dependencies = [ + "memchr", + "serde_core", +] + +[[package]] +name = "clap" +version = "4.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b193af5b67834b676abd72466a96c1024e6a6ad978a1f484bd90b85c94041351" +dependencies = [ + "clap_builder", + "clap_derive", +] + +[[package]] +name = "clap_builder" +version = "4.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "714a53001bf66416adb0e2ef5ac857140e7dc3a0c48fb28b2f10762fc4b5069f" +dependencies = [ + "anstream", + "anstyle", + "clap_lex", + "strsim", +] + +[[package]] +name = "clap_derive" +version = "4.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1110bd8a634a1ab8cb04345d8d878267d57c3cf1b38d91b71af6686408bbca6a" +dependencies = [ + "heck", + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "clap_lex" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c8d4a3bb8b1e0c1050499d1815f5ab16d04f0959b233085fb31653fbfc9d98f9" + +[[package]] +name = "colorchoice" +version = "1.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d07550c9036bf2ae0c684c4297d503f838287c83c53686d05370d0e139ae570" + +[[package]] +name = "countme" +version = "3.0.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7704b5fdd17b18ae31c4c1da5a2e0305a2bf17b5249300a9ee9ed7b72114c636" + +[[package]] +name = "crossbeam-deque" +version = "0.8.7" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5181e0de7b61eb03a81e347d6dd8797bae9da5146707b51077e2d71a54ec0ceb" +dependencies = [ + "crossbeam-epoch", + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-epoch" +version = "0.9.20" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.22" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17" + +[[package]] +name = "either" +version = "1.16.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "91622ff5e7162018101f2fea40d6ebf4a78bbe5a49736a2020649edf9693679e" + +[[package]] +name = "emmylua_parser" +version = "0.28.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a0103cc231288ddc9391785db73be15d5928951e54cfe17c73af8c4d371678db" +dependencies = [ + "rowan", + "rust-i18n", + "serde", +] + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "glob" +version = "0.3.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0cc23270f6e1808e30a928bdc84dea0b9b4136a8bc82338574f23baf47bbd280" + +[[package]] +name = "globset" +version = "0.4.19" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e47d37d2ae4464254884b60ab7071be2b876a9c35b696bd018ddcc76847309cd" +dependencies = [ + "aho-corasick", + "bstr", + "log", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "globwalk" +version = "0.8.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93e3af942408868f6934a7b85134a3230832b9977cf66125df2f9edcfce4ddcc" +dependencies = [ + "bitflags", + "ignore", + "walkdir", +] + +[[package]] +name = "hashbrown" +version = "0.14.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1" + +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" + +[[package]] +name = "heck" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea" + +[[package]] +name = "ignore" +version = "0.4.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f8a7b8211e695a1d0cd91cace480d4d0bd57667ab10277cc412c5f7f4884f83" +dependencies = [ + "crossbeam-deque", + "globset", + "log", + "memchr", + "regex-automata", + "same-file", + "walkdir", + "winapi-util", +] + +[[package]] +name = "indexmap" +version = "2.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d466e9454f08e4a911e14806c24e16fba1b4c121d1ea474396f396069cf949d9" +dependencies = [ + "equivalent", + "hashbrown 0.17.1", +] + +[[package]] +name = "is_terminal_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a6cb138bb79a146c1bd460005623e142ef0181e3d0219cb493e02f7d08a35695" + +[[package]] +name = "itertools" +version = "0.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b1c173a5686ce8bfa551b3563d0c2170bf24ca44da99c7ca4bfdab5418c3fe57" +dependencies = [ + "either", +] + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "lazy_static" +version = "1.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe" + +[[package]] +name = "log" +version = "0.4.33" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0ceec5bc11778974d1bcb055b18002eba7f4b3518b6a0081b3af5f21666da9ad" + +[[package]] +name = "memchr" +version = "2.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" + +[[package]] +name = "normpath" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b9985ef7269fa99f3b12437bb698381da2428743ab90f20393f399fa14cab21a" +dependencies = [ + "windows-sys", +] + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "once_cell_polyfill" +version = "1.70.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "384b8ab6d37215f3c5301a95a4accb5d64aa607f1fcb26a11b5303878451b4fe" + +[[package]] +name = "proc-macro2" +version = "1.0.106" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fd00f0bb2e90d81d1044c2b32617f68fcb9fa3bb7640c23e9c748e53fb30934" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.45" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41f2619966050689382d2b44f664f4bc593e129785a36d6ee376ddf37259b924" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "regex" +version = "1.13.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d" +dependencies = [ + "aho-corasick", + "memchr", + "regex-automata", + "regex-syntax", +] + +[[package]] +name = "regex-automata" +version = "0.4.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8fcfdb36bda0c880c5931cdc7a2bcdc8ba4556847b9d912bca70bc94708711ad" +dependencies = [ + "aho-corasick", + "memchr", + "regex-syntax", +] + +[[package]] +name = "regex-syntax" +version = "0.8.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4" + +[[package]] +name = "rowan" +version = "0.16.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "417a3a9f582e349834051b8a10c8d71ca88da4211e4093528e36b9845f6b5f21" +dependencies = [ + "countme", + "hashbrown 0.14.5", + "rustc-hash", + "text-size", +] + +[[package]] +name = "rust-i18n" +version = "3.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fda2551fdfaf6cc5ee283adc15e157047b92ae6535cf80f6d4962d05717dc332" +dependencies = [ + "globwalk", + "once_cell", + "regex", + "rust-i18n-macro", + "rust-i18n-support", + "smallvec", +] + +[[package]] +name = "rust-i18n-macro" +version = "3.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "22baf7d7f56656d23ebe24f6bb57a5d40d2bce2a5f1c503e692b5b2fa450f965" +dependencies = [ + "glob", + "once_cell", + "proc-macro2", + "quote", + "rust-i18n-support", + "serde", + "serde_json", + "serde_yaml", + "syn", +] + +[[package]] +name = "rust-i18n-support" +version = "3.1.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "940ed4f52bba4c0152056d771e563b7133ad9607d4384af016a134b58d758f19" +dependencies = [ + "arc-swap", + "base62", + "globwalk", + "itertools", + "lazy_static", + "normpath", + "once_cell", + "proc-macro2", + "regex", + "serde", + "serde_json", + "serde_yaml", + "siphasher", + "toml", + "triomphe", +] + +[[package]] +name = "rustc-hash" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "08d43f7aa6b08d49f382cde6a7982047c3426db949b1424bc4b7ec9ae12c6ce2" + +[[package]] +name = "rustversion" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f" + +[[package]] +name = "ryu" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9774ba4a74de5f7b1c1451ed6cd5285a32eddb5cccb8cc655a4e50009e06477f" + +[[package]] +name = "same-file" +version = "1.0.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "93fc1dc3aaa9bfed95e02e6eadabb4baf7e3078b0bd1b4d7b6b0b68378900502" +dependencies = [ + "winapi-util", +] + +[[package]] +name = "serde" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9a8e94ea7f378bd32cbbd37198a4a91436180c5bb472411e48b5ec2e2124ae9e" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41d385c7d4ca58e59fc732af25c3983b67ac852c1a25000afe1175de458b67ad" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.228" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d540f220d3187173da220f885ab66608367b6574e925011a9353e4badda91d79" +dependencies = [ + "proc-macro2", + "quote", + "syn", +] + +[[package]] +name = "serde_json" +version = "1.0.151" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "serde_spanned" +version = "0.6.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bf41e0cfaf7226dca15e8197172c295a782857fcb97fad1808a166870dee75a3" +dependencies = [ + "serde", +] + +[[package]] +name = "serde_yaml" +version = "0.9.34+deprecated" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6a8b1a1a2ebf674015cc02edccce75287f1a0130d394307b36743c2f5d504b47" +dependencies = [ + "indexmap", + "itoa", + "ryu", + "serde", + "unsafe-libyaml", +] + +[[package]] +name = "siphasher" +version = "1.0.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ee5873ec9cce0195efcb7a4e9507a04cd49aec9c83d0389df45b1ef7ba2e649" + +[[package]] +name = "smallvec" +version = "1.15.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8ed6a63f02c8539c91a8685a86f4099661ba3da017932f6ebbea6de3f0fa7c90" + +[[package]] +name = "stable_deref_trait" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6ce2be8dc25455e1f91df71bfa12ad37d7af1092ae736f3a6cd0e37bc7810596" + +[[package]] +name = "strsim" +version = "0.11.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7da8b5736845d9f2fcb837ea5d9e2628564b3b043a70948a3f0b778838c5fb4f" + +[[package]] +name = "syn" +version = "2.0.117" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e665b8803e7b1d2a727f4023456bbbbe74da67099c585258af0ad9c5013b9b99" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "text-size" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f18aa187839b2bdb1ad2fa35ead8c4c2976b64e4363c386d45ac0f7ee85c9233" + +[[package]] +name = "toml" +version = "0.8.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "dc1beb996b9d83529a9e75c17a1686767d148d70663143c7854d8b4a09ced362" +dependencies = [ + "serde", + "serde_spanned", + "toml_datetime", + "toml_edit", +] + +[[package]] +name = "toml_datetime" +version = "0.6.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "22cddaf88f4fbc13c51aebbf5f8eceb5c7c5a9da2ac40a13519eb5b0a0e8f11c" +dependencies = [ + "serde", +] + +[[package]] +name = "toml_edit" +version = "0.22.27" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "41fe8c660ae4257887cf66394862d21dbca4a6ddd26f04a3560410406a2f819a" +dependencies = [ + "indexmap", + "serde", + "serde_spanned", + "toml_datetime", + "toml_write", + "winnow", +] + +[[package]] +name = "toml_write" +version = "0.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d99f8c9a7727884afe522e9bd5edbfc91a3312b36a77b5fb8926e4c31a41801" + +[[package]] +name = "triomphe" +version = "0.1.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b40688ea6389c8171614b25491f71d4a27946e0c7ce2da1c6de27e25abf1a0ae" +dependencies = [ + "arc-swap", + "serde", + "stable_deref_trait", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "unsafe-libyaml" +version = "0.2.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "673aac59facbab8a9007c7f6108d11f63b603f7cabff99fabf650fea5c32b861" + +[[package]] +name = "utf8parse" +version = "0.2.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821" + +[[package]] +name = "walkdir" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29790946404f91d9c5d06f9874efddea1dc06c5efe94541a7d6863108e3a5e4b" +dependencies = [ + "same-file", + "winapi-util", +] + +[[package]] +name = "winapi-util" +version = "0.1.11" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c2a7b1c03c876122aa43f3020e6c3c3ee5c05081c9a00739faf7503aeba10d22" +dependencies = [ + "windows-sys", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "winnow" +version = "0.7.15" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "df79d97927682d2fd8adb29682d1140b343be4ac0f08fd68b7765d9c059d3945" +dependencies = [ + "memchr", +] + +[[package]] +name = "zmij" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b" diff --git a/bar-lua-codemod/Cargo.toml b/bar-lua-codemod/Cargo.toml new file mode 100644 index 00000000..44f4d3f3 --- /dev/null +++ b/bar-lua-codemod/Cargo.toml @@ -0,0 +1,9 @@ +[package] +name = "bar-lua-codemod" +version = "0.1.0" +edition = "2021" + +[dependencies] +emmylua_parser = "0.28" +glob = "0.3" +clap = { version = "4", features = ["derive"] } diff --git a/bar-lua-codemod/src/bracket_to_dot.rs b/bar-lua-codemod/src/bracket_to_dot.rs new file mode 100644 index 00000000..ef93bb1d --- /dev/null +++ b/bar-lua-codemod/src/bracket_to_dot.rs @@ -0,0 +1,231 @@ +use crate::cst::{bracket_span, bracket_string_key, quoted_content}; +use crate::edit::{self, Edit}; +use emmylua_parser::{LuaAstNode, LuaIndexExpr, LuaSyntaxTree, LuaTableField}; + +const LUA_RESERVED: &[&str] = &[ + "and", "break", "do", "else", "elseif", "end", "false", "for", "function", "if", "in", + "local", "nil", "not", "or", "repeat", "return", "then", "true", "until", "while", +]; + +fn is_convertible_identifier(s: &str) -> bool { + let mut chars = s.chars(); + match chars.next() { + Some(c) if c.is_ascii_alphabetic() || c == '_' => {} + _ => return false, + } + chars.all(|c| c.is_ascii_alphanumeric() || c == '_') && !LUA_RESERVED.contains(&s) +} + +pub struct BracketToDot { + pub index_conversions: usize, + pub field_conversions: usize, + pub skipped_reserved: usize, +} + +impl BracketToDot { + pub fn new() -> Self { + Self { + index_conversions: 0, + field_conversions: 0, + skipped_reserved: 0, + } + } + + pub fn rewrite(&mut self, source: &str, tree: &LuaSyntaxTree) -> String { + let mut edits: Vec = Vec::new(); + for node in tree.get_chunk_node().syntax().descendants() { + if let Some(index) = LuaIndexExpr::cast(node.clone()) { + self.index_expr(source, &index, &mut edits); + } else if let Some(field) = LuaTableField::cast(node) { + self.table_field(&field, &mut edits); + } + } + edit::apply(source, edits) + } + + /// x["y"] -> x.y; a space is injected when `]` abuts a word character + /// (]keyword is fine, .identifierkeyword merges). + fn index_expr(&mut self, source: &str, index: &LuaIndexExpr, edits: &mut Vec) { + let Some(token) = bracket_string_key(index.syntax()) else { + return; + }; + let Some(name) = quoted_content(&token) else { + return; + }; + if is_convertible_identifier(&name) { + let Some((start, end)) = bracket_span(index.syntax()) else { + return; + }; + self.index_conversions += 1; + let mut text = format!(".{name}"); + let next_is_word = source + .as_bytes() + .get(end) + .map(|&b| b.is_ascii_alphanumeric() || b == b'_') + .unwrap_or(false); + if next_is_word { + text.push(' '); + } + edits.push(Edit { start, end, text }); + } else if LUA_RESERVED.contains(&name.as_str()) { + self.skipped_reserved += 1; + } + } + + /// ["y"] = v -> y = v (table constructor fields). + fn table_field(&mut self, field: &LuaTableField, edits: &mut Vec) { + if !field.is_assign_field() { + return; + } + let Some(token) = bracket_string_key(field.syntax()) else { + return; + }; + let Some(name) = quoted_content(&token) else { + return; + }; + if is_convertible_identifier(&name) { + let Some((start, end)) = bracket_span(field.syntax()) else { + return; + }; + self.field_conversions += 1; + edits.push(Edit { start, end, text: name }); + } else if LUA_RESERVED.contains(&name.as_str()) { + self.skipped_reserved += 1; + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::cst::parse; + + fn transform(input: &str) -> (String, usize, usize) { + let tree = parse(input).expect("parse failed"); + let mut visitor = BracketToDot::new(); + let out = visitor.rewrite(input, &tree); + (out, visitor.index_conversions, visitor.field_conversions) + } + + #[test] + fn index_simple() { + let (out, idx, fld) = transform(r#"local x = t["foo"]"#); + assert_eq!(out, "local x = t.foo"); + assert_eq!(idx, 1); + assert_eq!(fld, 0); + } + + #[test] + fn index_single_quotes() { + let (out, idx, _) = transform("local x = t['bar']"); + assert_eq!(out, "local x = t.bar"); + assert_eq!(idx, 1); + } + + #[test] + fn index_chained() { + let (out, idx, _) = transform(r#"local x = t["a"]["b"]"#); + assert_eq!(out, "local x = t.a.b"); + assert_eq!(idx, 2); + } + + #[test] + fn index_reserved_word_skipped() { + let (out, _, _) = transform(r#"local x = t["end"]"#); + assert_eq!(out, r#"local x = t["end"]"#); + } + + #[test] + fn index_numeric_key_skipped() { + let (out, idx, _) = transform(r#"local x = t["123"]"#); + assert_eq!(out, r#"local x = t["123"]"#); + assert_eq!(idx, 0); + } + + #[test] + fn index_special_chars_skipped() { + let (out, idx, _) = transform(r#"local x = t["foo-bar"]"#); + assert_eq!(out, r#"local x = t["foo-bar"]"#); + assert_eq!(idx, 0); + } + + #[test] + fn field_simple() { + let (out, idx, fld) = transform(r#"local t = { ["foo"] = 1 }"#); + assert_eq!(out, "local t = { foo = 1 }"); + assert_eq!(idx, 0); + assert_eq!(fld, 1); + } + + #[test] + fn field_reserved_word_skipped() { + let (out, _, fld) = transform(r#"local t = { ["end"] = 1 }"#); + assert_eq!(out, r#"local t = { ["end"] = 1 }"#); + assert_eq!(fld, 0); + } + + #[test] + fn mixed_conversions() { + let (out, idx, fld) = transform(r#"t["x"] = { ["y"] = 1 }"#); + assert_eq!(out, "t.x = { y = 1 }"); + assert_eq!(idx, 1); + assert_eq!(fld, 1); + } + + #[test] + fn underscore_identifier() { + let (out, idx, _) = transform(r#"local x = t["_private"]"#); + assert_eq!(out, "local x = t._private"); + assert_eq!(idx, 1); + } + + #[test] + fn no_changes() { + let (out, idx, fld) = transform("local x = t[42]"); + assert_eq!(out, "local x = t[42]"); + assert_eq!(idx, 0); + assert_eq!(fld, 0); + } + + #[test] + fn bracket_then_dot_access() { + let (out, idx, _) = transform(r#"local x = cmd[1]["options"].ctrl"#); + assert_eq!(out, "local x = cmd[1].options.ctrl"); + assert_eq!(idx, 1); + } + + #[test] + fn bracket_to_dot_then_dot_access() { + let (out, idx, _) = transform(r#"local x = WeaponDefNames["lightning_chain"].id"#); + assert_eq!(out, "local x = WeaponDefNames.lightning_chain.id"); + assert_eq!(idx, 1); + } + + #[test] + fn no_merge_with_following_keyword() { + let (out, idx, _) = transform("if force and WG['guishader']then end"); + assert!(out.contains("WG.guishader then"), "got: {out}"); + assert_eq!(idx, 1); + } + + #[test] + fn no_merge_with_following_identifier() { + let (out, idx, _) = transform("local x = t['key']or false"); + assert!(out.contains("t.key or"), "got: {out}"); + assert_eq!(idx, 1); + } + + #[test] + fn escape_in_string_skipped() { + let (out, idx, _) = transform(r#"local x = t["\097bc"]"#); + assert_eq!(out, r#"local x = t["\097bc"]"#); + assert_eq!(idx, 0); + } + + #[test] + fn inner_bracket_trivia_dropped() { + let (out, idx, _) = transform(r#"local x = t[ "foo" ]"#); + assert_eq!(out, "local x = t.foo"); + assert_eq!(idx, 1); + } +} diff --git a/bar-lua-codemod/src/cst.rs b/bar-lua-codemod/src/cst.rs new file mode 100644 index 00000000..67bd4887 --- /dev/null +++ b/bar-lua-codemod/src/cst.rs @@ -0,0 +1,116 @@ +use emmylua_parser::{ + LuaAstNode, LuaAstToken, LuaLanguageLevel, LuaLiteralExpr, LuaLiteralToken, LuaParseErrorKind, + LuaParser, LuaStringToken, LuaSyntaxKind, LuaSyntaxNode, LuaSyntaxToken, LuaSyntaxTree, + LuaTokenKind, ParserConfig, +}; +use std::collections::HashMap; + +/// Lua 5.1 (BAR's runtime level), doc-comment parsing off: comments are +/// trivia, as they were under full_moon. +pub fn parse(code: &str) -> Result { + let config = ParserConfig::new( + LuaLanguageLevel::Lua51, + None, + HashMap::new(), + Default::default(), + false, + ); + let tree = LuaParser::parse(code, config); + if tree.has_syntax_errors() { + return Err(tree + .get_errors() + .iter() + .filter(|e| e.kind == LuaParseErrorKind::SyntaxError) + .map(|e| e.message.clone()) + .collect::>() + .join("; ")); + } + Ok(tree) +} + +/// Byte range from `[` through `]` of an index expr or table field node. +pub fn bracket_span(node: &LuaSyntaxNode) -> Option<(usize, usize)> { + let mut start = None; + for child in node.children_with_tokens() { + let Some(token) = child.as_token() else { + continue; + }; + if token.kind() == LuaTokenKind::TkLeftBracket.into() && start.is_none() { + start = Some(usize::from(token.text_range().start())); + } else if token.kind() == LuaTokenKind::TkRightBracket.into() { + return Some((start?, usize::from(token.text_range().end()))); + } + } + None +} + +fn is_trivia(token: &LuaSyntaxToken) -> bool { + token.kind() == LuaTokenKind::TkWhitespace.into() + || token.kind() == LuaTokenKind::TkEndOfLine.into() + || token.kind() == LuaTokenKind::TkShortComment.into() + || token.kind() == LuaTokenKind::TkLongComment.into() +} + +/// The sole string-literal key of `[...]` in an index expr or table field, +/// tolerating trivia inside the brackets (which the upstream get_index_key +/// does not). None when the bracketed expression is anything else. +pub fn bracket_string_key(node: &LuaSyntaxNode) -> Option { + let mut in_brackets = false; + let mut key: Option = None; + for child in node.children_with_tokens() { + if !in_brackets { + if child.as_token().map(|t| t.kind()) == Some(LuaTokenKind::TkLeftBracket.into()) { + in_brackets = true; + } + continue; + } + if let Some(token) = child.as_token() { + if token.kind() == LuaTokenKind::TkRightBracket.into() { + return key; + } + if !is_trivia(token) { + return None; + } + } else if let Some(inner) = child.into_node() { + if key.is_some() { + return None; + } + let literal = LuaLiteralExpr::cast(inner)?; + match literal.get_literal()? { + LuaLiteralToken::String(token) => key = Some(token), + _ => return None, + } + } + } + None +} + +/// Raw content of a single- or double-quoted string token; None for long +/// strings. No unescaping — parity with the raw-slice rule the conversions +/// were generated under. +pub fn quoted_content(token: &LuaStringToken) -> Option { + let s = token.get_text(); + if s.len() >= 2 + && ((s.starts_with('"') && s.ends_with('"')) || (s.starts_with('\'') && s.ends_with('\''))) + { + return Some(s[1..s.len() - 1].to_string()); + } + None +} + +/// True when the index chain is the name of `function a.b.c() end` — +/// a position full_moon's Var/FunctionCall visitors never rewrote. +pub fn is_func_stat_name(node: &LuaSyntaxNode) -> bool { + let mut cur = node.clone(); + loop { + let Some(parent) = cur.parent() else { + return false; + }; + let kind: LuaSyntaxKind = parent.kind().into(); + if kind == LuaSyntaxKind::IndexExpr { + cur = parent; + continue; + } + return kind == LuaSyntaxKind::FuncStat; + } +} diff --git a/bar-lua-codemod/src/detach_bar_modules.rs b/bar-lua-codemod/src/detach_bar_modules.rs new file mode 100644 index 00000000..1b216cb3 --- /dev/null +++ b/bar-lua-codemod/src/detach_bar_modules.rs @@ -0,0 +1,178 @@ +use crate::cst::is_func_stat_name; +use crate::edit::{self, Edit}; +use emmylua_parser::{ + LuaAstNode, LuaAstToken, LuaExpr, LuaIndexExpr, LuaIndexKey, LuaSyntaxTree, +}; +use std::collections::HashSet; + +pub struct DetachBarModules { + modules: HashSet, + pub conversions: usize, +} + +impl DetachBarModules { + pub fn new(modules: &[&str]) -> Self { + Self { + modules: modules.iter().map(|s| s.to_string()).collect(), + conversions: 0, + } + } + + /// Match `Spring.Module` or `_G.Spring.Module` and rename the Spring + /// segment to `BAR`, keeping the module name and everything after it + /// (`Spring.I18N.t()` -> `BAR.I18N.t()`). + pub fn rewrite(&mut self, source: &str, tree: &LuaSyntaxTree) -> String { + let mut edits: Vec = Vec::new(); + for node in tree.get_chunk_node().syntax().descendants() { + let Some(index) = LuaIndexExpr::cast(node) else { + continue; + }; + if is_func_stat_name(index.syntax()) { + continue; + } + let Some(LuaIndexKey::Name(module)) = index.get_index_key() else { + continue; + }; + if !self.modules.contains(module.get_name_text()) { + continue; + } + let spring_range = match index.get_prefix_expr() { + Some(LuaExpr::NameExpr(prefix)) + if prefix.get_name_text().as_deref() == Some("Spring") => + { + prefix.syntax().text_range() + } + Some(LuaExpr::IndexExpr(inner)) => { + let Some(LuaExpr::NameExpr(base)) = inner.get_prefix_expr() else { + continue; + }; + if base.get_name_text().as_deref() != Some("_G") { + continue; + } + let Some(LuaIndexKey::Name(spring)) = inner.get_index_key() else { + continue; + }; + if spring.get_name_text() != "Spring" { + continue; + } + spring.get_range() + } + _ => continue, + }; + self.conversions += 1; + edits.push(Edit { + start: usize::from(spring_range.start()), + end: usize::from(spring_range.end()), + text: "BAR".to_string(), + }); + } + edit::apply(source, edits) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::cst::parse; + + const MODULES: &[&str] = &["I18N", "Utilities", "Debug", "Lava"]; + + fn transform(input: &str) -> (String, usize) { + let tree = parse(input).expect("parse failed"); + let mut visitor = DetachBarModules::new(MODULES); + let out = visitor.rewrite(input, &tree); + (out, visitor.conversions) + } + + #[test] + fn simple_call() { + let (out, n) = transform("Spring.I18N.translate(key)"); + assert_eq!(out, "BAR.I18N.translate(key)"); + assert_eq!(n, 1); + } + + #[test] + fn method_access() { + let (out, n) = transform("local x = Spring.Utilities.Round(1.5)"); + assert_eq!(out, "local x = BAR.Utilities.Round(1.5)"); + assert_eq!(n, 1); + } + + #[test] + fn var_reference() { + let (out, n) = transform("local u = Spring.Utilities"); + assert_eq!(out, "local u = BAR.Utilities"); + assert_eq!(n, 1); + } + + #[test] + fn non_module_unchanged() { + let (out, n) = transform("Spring.GetGameFrame()"); + assert_eq!(out, "Spring.GetGameFrame()"); + assert_eq!(n, 0); + } + + #[test] + fn non_spring_unchanged() { + let (out, n) = transform("Other.I18N.translate(key)"); + assert_eq!(out, "Other.I18N.translate(key)"); + assert_eq!(n, 0); + } + + #[test] + fn preserves_trivia() { + let (out, n) = transform(" Spring.Debug.log(msg) -- log it"); + assert_eq!(out, " BAR.Debug.log(msg) -- log it"); + assert_eq!(n, 1); + } + + #[test] + fn assignment_declaration() { + let (out, n) = transform("Spring.I18N = Spring.I18N or VFS.Include('i18n.lua')"); + assert_eq!(out, "BAR.I18N = BAR.I18N or VFS.Include('i18n.lua')"); + assert_eq!(n, 2); + } + + #[test] + fn multiple_in_one_file() { + let (out, n) = transform("Spring.I18N.t('x')\nSpring.Lava.isActive()"); + assert!(out.contains("BAR.I18N.t('x')")); + assert!(out.contains("BAR.Lava.isActive()")); + assert_eq!(n, 2); + } + + #[test] + fn g_spring_module_assignment() { + let (out, n) = transform("_G.Spring.Utilities = _G.Spring.Utilities or {}"); + assert_eq!(out, "_G.BAR.Utilities = _G.BAR.Utilities or {}"); + assert_eq!(n, 2); + } + + #[test] + fn g_spring_module_call() { + let (out, n) = transform("_G.Spring.I18N('key')"); + assert_eq!(out, "_G.BAR.I18N('key')"); + assert_eq!(n, 1); + } + + #[test] + fn g_spring_non_module_unchanged() { + let (out, n) = transform("_G.Spring.GetGameFrame()"); + assert_eq!(out, "_G.Spring.GetGameFrame()"); + assert_eq!(n, 0); + } + + #[test] + fn g_spring_deep_access() { + let (out, n) = transform("_G.Spring.Utilities.Gametype.IsFFA()"); + assert_eq!(out, "_G.BAR.Utilities.Gametype.IsFFA()"); + assert_eq!(n, 1); + } + + #[test] + fn function_definition_name_unchanged() { + let (out, n) = transform("function Spring.Utilities.Round(x) return x end"); + assert_eq!(out, "function Spring.Utilities.Round(x) return x end"); + assert_eq!(n, 0); + } +} diff --git a/bar-lua-codemod/src/edit.rs b/bar-lua-codemod/src/edit.rs new file mode 100644 index 00000000..06cc6942 --- /dev/null +++ b/bar-lua-codemod/src/edit.rs @@ -0,0 +1,21 @@ +/// A byte-range replacement against the original source. Transforms collect +/// edits over the CST walk; non-overlapping by construction (each edit spans +/// tokens of a distinct node). +pub struct Edit { + pub start: usize, + pub end: usize, + pub text: String, +} + +pub fn apply(source: &str, mut edits: Vec) -> String { + edits.sort_by_key(|e| e.start); + let mut out = String::with_capacity(source.len()); + let mut pos = 0; + for e in edits { + out.push_str(&source[pos..e.start]); + out.push_str(&e.text); + pos = e.end; + } + out.push_str(&source[pos..]); + out +} diff --git a/bar-lua-codemod/src/main.rs b/bar-lua-codemod/src/main.rs new file mode 100644 index 00000000..ff91df02 --- /dev/null +++ b/bar-lua-codemod/src/main.rs @@ -0,0 +1,380 @@ +use clap::{Parser, Subcommand}; +use std::path::PathBuf; +use std::{fs, process}; + +mod bracket_to_dot; +mod cst; +mod detach_bar_modules; +mod edit; +mod rename_aliases; + +#[derive(Parser)] +#[command(name = "bar-lua-codemod")] +#[command(about = "AST-based Lua codemod tool for Beyond All Reason")] +struct Cli { + #[command(subcommand)] + command: Commands, +} + +#[derive(Subcommand)] +enum Commands { + /// Convert bracket string access to dot notation (x["y"] -> x.y, ["y"] = -> y =) + BracketToDot { + /// Root directory to process + #[arg(long, default_value = ".")] + path: PathBuf, + + /// Directories to exclude (relative to path, may be repeated) + #[arg(long)] + exclude: Vec, + + /// Report changes without writing files + #[arg(long)] + dry_run: bool, + }, + + /// Rename deprecated Spring method aliases to canonical names + RenameAliases { + /// Root directory to process + #[arg(long, default_value = ".")] + path: PathBuf, + + /// Directories to exclude (relative to path, may be repeated) + #[arg(long)] + exclude: Vec, + + /// Report changes without writing files + #[arg(long)] + dry_run: bool, + }, + + /// Detach BAR modules from the Spring table into the BAR namespace (Spring.I18N -> BAR.I18N, etc.) + DetachBarModules { + /// Root directory to process + #[arg(long, default_value = ".")] + path: PathBuf, + + /// Directories to exclude (relative to path, may be repeated) + #[arg(long)] + exclude: Vec, + + /// Report changes without writing files + #[arg(long)] + dry_run: bool, + }, +} + +fn collect_lua_files(root: &PathBuf, excludes: &[String]) -> Vec { + let pattern = format!("{}/**/*.lua", root.display()); + let mut files = Vec::new(); + for entry in glob::glob(&pattern).expect("invalid glob pattern") { + if let Ok(path) = entry { + let rel = path.strip_prefix(root).unwrap_or(&path); + let excluded = excludes + .iter() + .any(|ex| rel.starts_with(ex)); + if !excluded { + files.push(path); + } + } + } + files.sort(); + files +} + +fn format_num(n: usize) -> String { + let s = n.to_string(); + let bytes = s.as_bytes(); + let len = bytes.len(); + let mut result = String::new(); + for (i, &b) in bytes.iter().enumerate() { + if i > 0 && (len - i) % 3 == 0 { + result.push(','); + } + result.push(b as char); + } + result +} + +fn run_bracket_to_dot(root: &PathBuf, excludes: &[String], dry_run: bool) { + let files = collect_lua_files(root, excludes); + let total_files = files.len(); + + if total_files == 0 { + eprintln!("No .lua files found under {}", root.display()); + process::exit(1); + } + + let mut files_changed: usize = 0; + let mut total_index: usize = 0; + let mut total_field: usize = 0; + let mut total_skipped: usize = 0; + let mut errors: usize = 0; + let mut per_file: Vec<(PathBuf, usize, usize)> = Vec::new(); + + for file_path in &files { + let code = match fs::read_to_string(file_path) { + Ok(c) => c, + Err(e) => { + eprintln!(" error reading {}: {}", file_path.display(), e); + errors += 1; + continue; + } + }; + + let tree = match cst::parse(&code) { + Ok(t) => t, + Err(e) => { + eprintln!(" parse error in {}: {}", file_path.display(), e); + errors += 1; + continue; + } + }; + + let mut visitor = bracket_to_dot::BracketToDot::new(); + let new_code = visitor.rewrite(&code, &tree); + + if visitor.index_conversions > 0 || visitor.field_conversions > 0 { + if !dry_run { + if let Err(e) = fs::write(file_path, new_code) { + eprintln!(" error writing {}: {}", file_path.display(), e); + errors += 1; + continue; + } + } + files_changed += 1; + total_index += visitor.index_conversions; + total_field += visitor.field_conversions; + total_skipped += visitor.skipped_reserved; + per_file.push(( + file_path.clone(), + visitor.index_conversions, + visitor.field_conversions, + )); + } + } + + let total_conversions = total_index + total_field; + + if dry_run { + println!("bar-lua-codemod bracket-to-dot (DRY RUN):"); + } else { + println!("bar-lua-codemod bracket-to-dot results:"); + } + println!(" Files scanned: {:>30}", format_num(total_files)); + println!(" Files changed: {:>30}", format_num(files_changed)); + println!( + " Index conversions (x[\"y\"] -> x.y): {:>8}", + format_num(total_index) + ); + println!( + " Field conversions ([\"y\"] = -> y =): {:>8}", + format_num(total_field) + ); + println!( + " Total conversions: {:>8}", + format_num(total_conversions) + ); + println!( + " Skipped (reserved words): {:>8}", + format_num(total_skipped) + ); + println!( + " Errors (parse failures): {:>8}", + format_num(errors) + ); + + if !per_file.is_empty() { + per_file.sort_by(|a, b| (b.1 + b.2).cmp(&(a.1 + a.2))); + println!(); + println!("Top files by conversion count:"); + for (path, idx, fld) in per_file.iter().take(20) { + let rel = path.strip_prefix(root).unwrap_or(path); + println!(" {:<60} {:>5}", rel.display(), idx + fld); + } + } + + if errors > 0 { + process::exit(1); + } +} + +const BAR_ALIASES: &[(&str, &str)] = &[ + ("GetMyTeamID", "GetLocalTeamID"), + ("GetMyAllyTeamID", "GetLocalAllyTeamID"), + ("GetMyPlayerID", "GetLocalPlayerID"), +]; + +fn run_rename_aliases(root: &PathBuf, excludes: &[String], dry_run: bool) { + let files = collect_lua_files(root, excludes); + let total_files = files.len(); + + if total_files == 0 { + eprintln!("No .lua files found under {}", root.display()); + process::exit(1); + } + + let mut files_changed: usize = 0; + let mut total_conversions: usize = 0; + let mut errors: usize = 0; + let mut per_file: Vec<(PathBuf, usize)> = Vec::new(); + + for file_path in &files { + let code = match fs::read_to_string(file_path) { + Ok(c) => c, + Err(e) => { + eprintln!(" error reading {}: {}", file_path.display(), e); + errors += 1; + continue; + } + }; + + let tree = match cst::parse(&code) { + Ok(t) => t, + Err(e) => { + eprintln!(" parse error in {}: {}", file_path.display(), e); + errors += 1; + continue; + } + }; + + let mut visitor = rename_aliases::RenameAliases::new(BAR_ALIASES); + let new_code = visitor.rewrite(&code, &tree); + + if visitor.conversions > 0 { + if !dry_run { + if let Err(e) = fs::write(file_path, new_code) { + eprintln!(" error writing {}: {}", file_path.display(), e); + errors += 1; + continue; + } + } + files_changed += 1; + total_conversions += visitor.conversions; + per_file.push((file_path.clone(), visitor.conversions)); + } + } + + if dry_run { + println!("bar-lua-codemod rename-aliases (DRY RUN):"); + } else { + println!("bar-lua-codemod rename-aliases results:"); + } + println!(" Files scanned: {:>7}", format_num(total_files)); + println!(" Files changed: {:>7}", format_num(files_changed)); + println!(" Conversions: {:>7}", format_num(total_conversions)); + println!(" Errors: {:>7}", format_num(errors)); + + if !per_file.is_empty() { + per_file.sort_by(|a, b| b.1.cmp(&a.1)); + println!(); + println!("Top files by conversion count:"); + for (path, count) in per_file.iter().take(20) { + let rel = path.strip_prefix(root).unwrap_or(path); + println!(" {:<60} {:>5}", rel.display(), count); + } + } + + if errors > 0 { + process::exit(1); + } +} + +const BAR_MODULES: &[&str] = &["I18N", "Utilities", "Debug", "Lava", "GetModOptionsCopy"]; + +fn run_detach_bar_modules(root: &PathBuf, excludes: &[String], dry_run: bool) { + let files = collect_lua_files(root, excludes); + let total_files = files.len(); + + if total_files == 0 { + eprintln!("No .lua files found under {}", root.display()); + process::exit(1); + } + + let mut files_changed: usize = 0; + let mut total_conversions: usize = 0; + let mut errors: usize = 0; + let mut per_file: Vec<(PathBuf, usize)> = Vec::new(); + + for file_path in &files { + let code = match fs::read_to_string(file_path) { + Ok(c) => c, + Err(e) => { + eprintln!(" error reading {}: {}", file_path.display(), e); + errors += 1; + continue; + } + }; + + let tree = match cst::parse(&code) { + Ok(t) => t, + Err(e) => { + eprintln!(" parse error in {}: {}", file_path.display(), e); + errors += 1; + continue; + } + }; + + let mut visitor = detach_bar_modules::DetachBarModules::new(BAR_MODULES); + let new_code = visitor.rewrite(&code, &tree); + + if visitor.conversions > 0 { + if !dry_run { + if let Err(e) = fs::write(file_path, new_code) { + eprintln!(" error writing {}: {}", file_path.display(), e); + errors += 1; + continue; + } + } + files_changed += 1; + total_conversions += visitor.conversions; + per_file.push((file_path.clone(), visitor.conversions)); + } + } + + if dry_run { + println!("bar-lua-codemod detach-bar-modules (DRY RUN):"); + } else { + println!("bar-lua-codemod detach-bar-modules results:"); + } + println!(" Modules detached: {:>7}", BAR_MODULES.join(", ")); + println!(" Files scanned: {:>7}", format_num(total_files)); + println!(" Files changed: {:>7}", format_num(files_changed)); + println!(" Conversions: {:>7}", format_num(total_conversions)); + println!(" Errors: {:>7}", format_num(errors)); + + if !per_file.is_empty() { + per_file.sort_by(|a, b| b.1.cmp(&a.1)); + println!(); + println!("Top files by conversion count:"); + for (path, count) in per_file.iter().take(20) { + let rel = path.strip_prefix(root).unwrap_or(path); + println!(" {:<60} {:>5}", rel.display(), count); + } + } + + if errors > 0 { + process::exit(1); + } +} + +fn main() { + let cli = Cli::parse(); + match cli.command { + Commands::BracketToDot { + path, + exclude, + dry_run, + } => run_bracket_to_dot(&path, &exclude, dry_run), + Commands::RenameAliases { + path, + exclude, + dry_run, + } => run_rename_aliases(&path, &exclude, dry_run), + Commands::DetachBarModules { + path, + exclude, + dry_run, + } => run_detach_bar_modules(&path, &exclude, dry_run), + } +} diff --git a/bar-lua-codemod/src/rename_aliases.rs b/bar-lua-codemod/src/rename_aliases.rs new file mode 100644 index 00000000..323c603d --- /dev/null +++ b/bar-lua-codemod/src/rename_aliases.rs @@ -0,0 +1,127 @@ +use crate::cst::is_func_stat_name; +use crate::edit::{self, Edit}; +use emmylua_parser::{ + LuaAstNode, LuaAstToken, LuaExpr, LuaIndexExpr, LuaIndexKey, LuaSyntaxTree, +}; +use std::collections::HashMap; + +pub struct RenameAliases { + aliases: HashMap, + pub conversions: usize, +} + +impl RenameAliases { + pub fn new(aliases: &[(&str, &str)]) -> Self { + Self { + aliases: aliases + .iter() + .map(|(old, new)| (old.to_string(), new.to_string())) + .collect(), + conversions: 0, + } + } + + /// Rewrite `Spring.OldName` to the canonical name wherever the prefix is + /// the bare `Spring` global. + pub fn rewrite(&mut self, source: &str, tree: &LuaSyntaxTree) -> String { + let mut edits: Vec = Vec::new(); + for node in tree.get_chunk_node().syntax().descendants() { + let Some(index) = LuaIndexExpr::cast(node) else { + continue; + }; + if is_func_stat_name(index.syntax()) { + continue; + } + let Some(LuaExpr::NameExpr(prefix)) = index.get_prefix_expr() else { + continue; + }; + if prefix.get_name_text().as_deref() != Some("Spring") { + continue; + } + let Some(LuaIndexKey::Name(name)) = index.get_index_key() else { + continue; + }; + let Some(canonical) = self.aliases.get(name.get_name_text()) else { + continue; + }; + self.conversions += 1; + let range = name.get_range(); + edits.push(Edit { + start: usize::from(range.start()), + end: usize::from(range.end()), + text: canonical.clone(), + }); + } + edit::apply(source, edits) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::cst::parse; + + const ALIASES: &[(&str, &str)] = &[ + ("GetMyTeamID", "GetLocalTeamID"), + ("GetMyAllyTeamID", "GetLocalAllyTeamID"), + ("GetMyPlayerID", "GetLocalPlayerID"), + ]; + + fn transform(input: &str) -> (String, usize) { + let tree = parse(input).expect("parse failed"); + let mut visitor = RenameAliases::new(ALIASES); + let out = visitor.rewrite(input, &tree); + (out, visitor.conversions) + } + + #[test] + fn renames_call() { + let (out, n) = transform("local t = Spring.GetMyTeamID()"); + assert_eq!(out, "local t = Spring.GetLocalTeamID()"); + assert_eq!(n, 1); + } + + #[test] + fn renames_var_reference() { + let (out, n) = transform("local fn = Spring.GetMyAllyTeamID"); + assert_eq!(out, "local fn = Spring.GetLocalAllyTeamID"); + assert_eq!(n, 1); + } + + #[test] + fn non_alias_unchanged() { + let (out, n) = transform("Spring.GetGameFrame()"); + assert_eq!(out, "Spring.GetGameFrame()"); + assert_eq!(n, 0); + } + + #[test] + fn non_spring_unchanged() { + let (out, n) = transform("Other.GetMyTeamID()"); + assert_eq!(out, "Other.GetMyTeamID()"); + assert_eq!(n, 0); + } + + #[test] + fn preserves_trivia() { + let (out, n) = transform(" local id = Spring.GetMyPlayerID() -- get player"); + assert_eq!(out, " local id = Spring.GetLocalPlayerID() -- get player"); + assert_eq!(n, 1); + } + + #[test] + fn multiple_in_one_file() { + let input = "local a = Spring.GetMyTeamID()\nlocal b = Spring.GetMyAllyTeamID()"; + let (out, n) = transform(input); + assert!(out.contains("Spring.GetLocalTeamID()")); + assert!(out.contains("Spring.GetLocalAllyTeamID()")); + assert_eq!(n, 2); + } + + #[test] + fn bracket_access_unchanged() { + let (out, n) = transform(r#"local f = Spring["GetMyTeamID"]"#); + assert_eq!(out, r#"local f = Spring["GetMyTeamID"]"#); + assert_eq!(n, 0); + } +} diff --git a/claude/claude.md b/claude/claude.md new file mode 100644 index 00000000..85209e42 --- /dev/null +++ b/claude/claude.md @@ -0,0 +1,14 @@ +# BAR Agent Context + +## Repositories + +| Repo | Purpose | +|------|---------| +| `Beyond-All-Reason` | Game codebase (Lua). Generated branches -- never hand-edit mig branches. | +| `BAR-Devtools` | Codemod tool (Rust), `generate-branches.sh` orchestrator, justfiles. | +| `RecoilEngine` | C++ game engine (Spring fork). Lua API exposed to game code. | +| `bar-design-docs` | Design docs and this agent context. | + +## Skills + +- [codemod-prereq](skills/codemod-prereq/SKILL.md) -- Diagnose transform failures, create prereq branches, fix codemods. diff --git a/claude/prompts/type-triage-subagent-openai.md b/claude/prompts/type-triage-subagent-openai.md new file mode 100644 index 00000000..f1336aab --- /dev/null +++ b/claude/prompts/type-triage-subagent-openai.md @@ -0,0 +1,185 @@ +# Type Triage Worker Prompt (OpenAI variant) + +Used by `scripts/codemod/llm-type-triage.sh` when `BACKEND=openai`. Dispatched +through `scripts/codemod/llm-type-triage-worker.py`, a self-contained Python +agent loop that talks to the OpenAI Chat Completions API directly. +The worker supplies this file as the system prompt and inlines the +chunk's error blocks in the user message, so this content is +identical across all parallel workers and benefits from OpenAI's +automatic prompt caching. + +You have **only two tools**, both file-bound and both restricted to +the BAR repo: + +- `read_file(path: str, start_line: int = 0, end_line: int = 0) -> str` + — read any `.lua` file in the BAR repo. Pass a path **relative to + the BAR repo root**. + - **For files >60KB you MUST pass `start_line` and `end_line`** + (1-indexed, inclusive) to read a window. Unbounded reads on + large files are rejected because each `read_file` result stays in + your conversation history forever and accumulates tokens across + every subsequent turn — one big read can blow the context window + for the rest of the chunk. + - The error blocks in your chunk give you the exact line of every + error (`--> file:line:col`). Read **~20 lines on either side** + (e.g. `read_file("foo.lua", 100, 140)` for an error at line 120). + That's almost always enough context to build a unique `search`. + - When called with line params, the response is line-number + prefixed (`120: `). The line numbers are NOT part of the + file — strip them when building the `search` argument for + `edit_file`. + - For small files (under 60KB), an unbounded `read_file(path)` is + fine and returns the whole file. +- `edit_file(path: str, search: str, replace: str) -> str` — replace + the **first** occurrence of `search` with `replace` in `path`. The + `search` string must appear **exactly once** in the file (verbatim, + including whitespace). If it appears 0 times or more than once, the + call returns an error and nothing is written — expand or shrink + your `search` and try again. Returns `"OK"` on success. + +There is **no Bash, no Glob, no Grep, no Write**. Chunking, dispatch, +and verification are all handled by the bash wrapper. Your scope is +"open the files in your chunk, fix the annotations the analyzer +flagged, save." + +--- + +You are applying **type-annotation-only fixes** to Lua files in the +Beyond-All-Reason repo. + +## Inputs + +The user message contains your chunk's `emmylua_check` error blocks +inlined between `=== chunk errors ===` and `=== end chunk ===` +markers. The format is verbatim from the analyzer: + +``` +error: [] + --> :: + + | + | + | +``` + +All file paths in the chunk are **relative to the BAR repo root**. +Pass them straight to `read_file` / `edit_file` — do NOT prefix with +`/` or `./`, do NOT try to read the chunk file itself (it's already +inlined in the user message). + +## Critical Rules (NEVER violate) + +- Do NOT change program logic or functionality. Only fix type + annotations and the lines flagged by the analyzer. +- Do NOT edit `.emmyrc.json` / `.luarc.json` — those belong to the + `fmt-llm-source` baseline. Editing them would defeat the + deterministic env layer. (`edit_file` will refuse them.) +- Do NOT edit files under `recoil-lua-library/` — those stubs are + static (generated by `just lua::library`). (`edit_file` will refuse.) +- Do NOT edit files under `types/` — type stubs are managed by + `fmt-llm-source`. If a fix would require a new or extended type + stub, flag it as UNCATEGORIZED. (`edit_file` will refuse.) +- Do NOT edit engine C++ files — out of scope. +- `---@cast` ONLY works on local variables, NEVER on `tbl.field` + paths. Use `--[[@as Type]]` inline for table fields, or extract to + a local variable first. +- NEVER use `--[[@as integer[]]]` (or any `...[]`) — the first `]` + closes the block comment and breaks parsing. Use `---@cast myLocal + integer[]` on the next line, or `---@alias MyArr integer[]` and + `--[[@as MyArr]]`. +- NEVER add `---@class Widget`, `---@class VAO`, or `---@class VBO` + inside widget files; it poisons workspace-wide class merges. Use + `local widget ---@type Widget = widget` only. +- BAR shader objects from `gl.LuaShader(...)` are `BarLuaShader`, NOT + `Shader` (which is an integer program ID). Use `---@type + BarLuaShader?`. If methods are missing, flag UNCATEGORIZED. +- NEVER reorder `local x = value ---@type T` to + `local x ---@type T = value`. The `= value` after a `---` comment is + consumed by the annotation, not by Lua — the local becomes nil. + The annotation goes AFTER the assignment: `local x = value ---@type T`. +- NEVER duplicate a function definition. If a function already exists + in the file, annotate or cast the existing one — do not insert a + second copy. Duplicates shadow the original and break references to + helper locals in the original closure. +- NEVER introduce a local variable that shadows a global module of the + same name (e.g. `local I18N = {}` when `I18N` is the global i18n + module, or `local Debug = ...` when `Debug` is a global utility + table). If a local cache already shadows a global, rename the cache + variable — do not rename the global. +- You MUST attempt a fix for EVERY error in your chunk. No skipping. +- If no category below matches an error, report it as UNCATEGORIZED + with the exact error message and the surrounding source context. + +## Token economy and turn budget + +You have a hard turn limit (~50 model responses) and your +conversation history accumulates with every tool call. To stay +under the limit and cheap: + +- **BATCH TOOL CALLS IN ONE TURN.** Each model response can include + many parallel tool calls. Plan a turn that issues *all* the + `read_file` calls you need, then a turn that issues *all* the + `edit_file` calls. Doing 30 errors as 30 separate read+edit pairs + is 60 turns and will hit the limit. Doing them as ~3 batched + read-rounds + ~3 batched edit-rounds is 6 turns. +- **Use line-range reads for big files.** A 30-line window is ~600 + tokens; a full 800KB file is ~200k tokens — and that delta is + multiplied across every remaining turn in your chain. The wrapper + enforces this for files >60KB. +- **Read each unique file region at most once.** The contents stay + in your context after the first read; don't re-read just to + re-check. +- **Coalesce nearby reads.** Two errors at lines 120 and 140 in the + same file? One read window covering 100..160 covers both. Don't + issue two separate small reads when one wider window suffices. +- **Don't read files you don't need.** The chunk groups errors by + file — only open the ones the chunk references. + +## Fix Priority + +For each error, match it to the FIRST applicable category. Detailed +procedures and examples are in the SKILL.md content embedded in your +system prompt — consult them when the summary below is ambiguous. + +1. **`undefined-field` on VBO/VAO** → Add `---@type VBO`/`VAO` at creation site; use `assert(gl.GetVBO(...))` when the path must not continue on failure +2. **`undefined-field` on `:SetUniform` / `:Activate` on gl.LuaShader result** → `---@type BarLuaShader?`, not `Shader` +3. **`undefined-field` on font methods** → Add `---@type LuaFont` at `gl.LoadFont()` site +4. **`need-check-nil`** → Guard, default, or cast +5. **`param-type-mismatch nil→number`** → Add `or 0` or `--[[@as number]]` +6. **`param-type-mismatch number?→number`** → Add `or 0` or `--[[@as number]]` +7. **`param-type-mismatch string→number`** → Wrap in `tonumber(x) or 0` +8. **`param-type-mismatch string↔stringlib`** → `---@diagnostic disable-next-line: param-type-mismatch` +9. **`cast-local-type`** → Add `---@type X|Y` before declaration +10. **`assign-type-mismatch`** → Add `--[[@as Type]]` cast +11. **`missing-parameter` on callin bootstrap** → Pass full args +12. **`redundant-parameter` on `GetGameRulesParam(name, 0)`** → Use `or 0` +13. **`duplicate-index`** → Remove duplicate key +14. **`duplicate-doc-field`** → Cross-file — flag UNCATEGORIZED if it requires touching multiple files outside your chunk +15. **`undefined-global` (lowercase variable)** → `local varName = default` +16. **`if not Spring then`** → Delete guard or replace with `SpringShared` +17. **`SetGameRulesParam` with boolean** → Convert or report +18. **`self.X` in module** → Replace with `ModuleName.X` +19. **`math.fract`** → Replace with `(x - math.floor(x))` +20. **Forward-compat guards** → Comment out with note +21. **`doc-syntax-error: expect type` after `---@type`** → Add `#` description delimiter (`---@type number # in seconds`) +22. **`syntax-error: expected TkRightBracket` in `---@field T [...]` prose** → Move bracket interval prose to end, wrap in backticks +23. **`annotation-usage-error: ` `` `@type X` can't be used here``** on `local function` → Convert to `local x = function(...)` +24. **`annotation-usage-error: ` `` `@return/@param X` can't be used here``** → Move annotation to the actual function declaration line + +## Output Format + +After all your tool calls are done, return your final message as a +plain-text report in this exact structure: + +``` +FIXED: + - path/to/file.lua: L123 (Category N) — description of change + +ATTEMPTED: + - path/to/file.lua: L456 (Category 35) — what you did, flagged for review + +UNCATEGORIZED: + - path/to/file.lua: L789 — error message + observed pattern + suggested fix +``` + +Work through EVERY file in your chunk. Do not stop early. diff --git a/claude/prompts/type-triage-subagent.md b/claude/prompts/type-triage-subagent.md new file mode 100644 index 00000000..41902195 --- /dev/null +++ b/claude/prompts/type-triage-subagent.md @@ -0,0 +1,132 @@ +# Type Triage Worker Prompt + +Used by `scripts/codemod/llm-type-triage.sh` to dispatch a parallel fan-out of +`claude-sonnet-4-6` workers. The script substitutes the literal token +`CHUNK_PATH` with an absolute path to a file containing the chunk's +`emmylua_check` error blocks before invoking `claude --print`. + +You have **only Read and Edit tools**. No Bash, no Glob, no Grep, no +Write. This is intentional: the chunking, dispatch, and verification +are all done by the bash wrapper. Your scope is "open files in your +chunk, fix the annotations the analyzer flagged, save." + +--- + +You are applying **type-annotation-only fixes** to Lua files in the +Beyond-All-Reason repo. Your current working directory is the BAR repo +root, so all paths in your chunk file are relative to it. + +## Inputs + +1. Read `CHUNK_PATH` — the file containing your assigned `emmylua_check` + error blocks. Format is verbatim from the analyzer: + + ``` + error: [] + --> :: + + | + | + | + ``` + +2. Read the canonical fix procedures at + `claude/skills/codemod-prereq/SKILL.md`. + Every category in there is in scope. If you encounter an error that + doesn't match any category, flag it as UNCATEGORIZED in your report + so a human can add a new rule and re-run. + +## Critical Rules (NEVER violate) + +- Do NOT change program logic or functionality. Only fix type + annotations and the lines flagged by the analyzer. +- Do NOT edit `.emmyrc.json` / `.luarc.json` — those belong to the + `fmt-llm-source` baseline. Editing them would defeat the deterministic + env layer. +- Do NOT edit files under `recoil-lua-library/` — those stubs are + static (generated by `just lua::library`). +- Do NOT edit files under `types/` — type stubs are managed by + `fmt-llm-source`. If a fix would require a new or extended type + stub, flag it as UNCATEGORIZED. +- Do NOT edit engine C++ files — out of scope. +- You have NO git tools. The bash wrapper handles all git state. + You only edit files in your chunk. +- `---@cast` ONLY works on local variables, NEVER on `tbl.field` + paths. Use `--[[@as Type]]` inline for table fields, or extract to + a local variable first. +- NEVER use `--[[@as integer[]]]` (or any `...[]`) — the first `]` + closes the block comment and breaks parsing. Use `---@cast myLocal + integer[]` on the next line, or `---@alias MyArr integer[]` and + `--[[@as MyArr]]`. +- NEVER add `---@class Widget`, `---@class VAO`, or `---@class VBO` + inside widget files; it poisons workspace-wide class merges. Use + `local widget ---@type Widget = widget` only. +- BAR shader objects from `gl.LuaShader(...)` are `BarLuaShader`, NOT + `Shader` (which is an integer program ID). Use `---@type + BarLuaShader?`. If methods are missing, flag UNCATEGORIZED. +- NEVER reorder `local x = value ---@type T` to + `local x ---@type T = value`. The `= value` after a `---` comment is + consumed by the annotation, not by Lua — the local becomes nil. + The annotation goes AFTER the assignment: `local x = value ---@type T`. +- NEVER duplicate a function definition. If a function already exists + in the file, annotate or cast the existing one — do not insert a + second copy. Duplicates shadow the original and break references to + helper locals in the original closure. +- NEVER introduce a local variable that shadows a global module of the + same name (e.g. `local I18N = {}` when `I18N` is the global i18n + module, or `local Debug = ...` when `Debug` is a global utility + table). If a local cache already shadows a global, rename the cache + variable — do not rename the global. +- You MUST attempt a fix for EVERY error in your chunk. No skipping. +- If no SKILL.md category matches an error, report it as UNCATEGORIZED + with the exact error message and surrounding context. + +## Fix Priority (from SKILL.md) + +For each error, match it to the FIRST applicable SKILL.md category: + +1. **`undefined-field` on VBO/VAO** → Category 16: Add `---@type VBO`/`VAO` at creation site; use `assert(gl.GetVBO(...))` when the path must not continue on failure +2. **`undefined-field` on `:SetUniform` / `:Activate` on gl.LuaShader result** → Category 38: `---@type BarLuaShader?`, not `Shader` +3. **`undefined-field` on font methods** → Category 20: Add `---@type LuaFont` at `gl.LoadFont()` site +4. **`need-check-nil`** → Category 35: Guard, default, or cast (see 5 rules in SKILL.md) +5. **`param-type-mismatch nil→number`** → Category 21/26: Add `or 0` or `--[[@as number]]` +6. **`param-type-mismatch number?→number`** → Category 26: Add `or 0` or `--[[@as number]]` +7. **`param-type-mismatch string→number`** → Wrap in `tonumber(x) or 0` +8. **`param-type-mismatch string↔stringlib`** → Category 27: `---@diagnostic disable-next-line: param-type-mismatch` +9. **`cast-local-type`** → Category 36: Add `---@type X|Y` before declaration +10. **`assign-type-mismatch`** → Category 37: Add `--[[@as Type]]` cast +11. **`missing-parameter` on callin bootstrap** → Category 25: Pass full args +12. **`redundant-parameter` on `GetGameRulesParam(name, 0)`** → Category 33: Use `or 0` +13. **`duplicate-index`** → Category 28: Remove duplicate key +14. **`duplicate-doc-field`** → Category 30: Consolidate `@class` (cross-file — flag UNCATEGORIZED if it requires touching multiple files outside your chunk) +15. **`undefined-global` on a same-name self-shadow line** → Category 43: Insert `---@diagnostic disable-next-line: undefined-global` directly above. Three sub-patterns to recognize: (a) `local X = X [or default]`, (b) `X = X [or {}],` inside a `{ ... }` table literal (typically `return { ... }` exports or `WG.Module = { ... }`), (c) `local Y = X and X() or default` defensive cross-file calls. **Match this BEFORE Cat 13/46.** +16. **`undefined-global` in a unit-script file** (`scripts/Units/**/*.lua`, `scripts/headers/**/*.lua`, or `scripts/include/**/*.lua`) → Category 45: Add `---@diagnostic disable: undefined-global` (no `-next-line`) at the top of the file. Clean up any stacked `disable-next-line` lines. **Match this BEFORE Cat 13/46.** +17. **`undefined-global` where `` IS assigned somewhere in the same file but only inside a deeper lexical scope** → Category 46: Add `local = ` forward declaration at the top of the enclosing module-level scope. **Search the file for `` assignments before declaring this category — if `` is never assigned anywhere in the file, fall through to Cat 13 instead.** +18. **`undefined-global` (lowercase variable, real bug — `` not assigned anywhere in the file)** → Category 13: `local varName = default` declaration in scope. Also covers two sub-patterns: (a) destructured assignment self-reference (`local _, X = fn(..., X, ...)` — declare `local X = nil` in enclosing scope), (b) typo where the intended name is unambiguously visible in the same function (`subunitDef` vs `subUnitDef` — fix the typo) +19. **`if not Spring then`** → Category 18: Delete guard or replace with `SpringShared` +20. **`SetGameRulesParam` with boolean** → Category 32: Convert or report +21. **`self.X` in module** → Category 19: Replace with `ModuleName.X` +22. **`math.fract`** → Category 24: Replace with `(x - math.floor(x))` +23. **Forward-compat guards** → Category 14: Comment out with note +24. **`doc-syntax-error: expect type` after `---@type`** → Category 39: Add `#` description delimiter (`---@type number # in seconds`) +25. **`syntax-error: expected TkRightBracket` in `---@field T [...]` prose** → Category 40: Move bracket interval prose to end, wrap in backticks +26. **`annotation-usage-error: ` `` `@type X` can't be used here``** on `local function` → Category 41: Convert to `local x = function(...)` +27. **`annotation-usage-error: ` `` `@return/@param X` can't be used here``** on a **function re-export site** — either `name = func_ref,` inside a `{ ... }` table literal, OR a direct field assignment like `table.toString = tableToString` — → Category 44: **Move** the entire `@param`/`@return` block (including `@param options.X` sub-fields and leading prose `---` lines) from the re-export site to the line above the actual `local function (...)` definition. Find it by searching for `function ` in the same file. +28. **`annotation-usage-error: ` `` `@return/@param X` can't be used here``** above a non-function, non-re-export declaration (e.g. `local x = 0` followed by an unrelated function lower down) → Category 42: **Move** the annotation to the actual function declaration line + +## Output Format + +Return your results in this exact structure: + +``` +FIXED: + - path/to/file.lua: L123 (Category N) — description of change + +ATTEMPTED: + - path/to/file.lua: L456 (Category 35, need-check-nil in game logic) — what you did, flagged for review + +UNCATEGORIZED: + - path/to/file.lua: L789 — error message, what pattern you observed, suggested fix if any +``` + +Work through EVERY file in your chunk. Batch edits per file. Do not stop early. diff --git a/claude/skills/codemod-prereq/SKILL.md b/claude/skills/codemod-prereq/SKILL.md new file mode 100644 index 00000000..f0fb79c0 --- /dev/null +++ b/claude/skills/codemod-prereq/SKILL.md @@ -0,0 +1,1084 @@ +--- +name: bar-codemod-prereq +description: >- + Categorize and fix LuaLS type errors in the Beyond-All-Reason Lua codebase. + Each category maps an `emmylua_check` error pattern to an idempotent fix + recipe. Used as the rule reference for `scripts/codemod/llm-type-triage.sh` workers + applying per-file annotation fixes after the deterministic codemod transforms. +--- + +# BAR Type Error Categories + +This document is a categorization rulebook for BAR's Lua type errors. Each +category pairs an `emmylua_check` error pattern with an idempotent fix recipe. +Run the pattern matchers in priority order (1 → 42); the first match wins. + +## What this is for + +`scripts/codemod/llm-type-triage.sh` dispatches parallel `claude --print` workers, one +per chunk of errors. Each worker uses this document to match its assigned +errors to a category and apply the fix in-place. Single pass — if a category +fails to shrink after one run, that's a signal the rules need a new entry, +not "let the LLM try again". + +## What to do when no category matches + +Some errors are not fixable by a per-file annotation edit — they require a +new type stub in `types/`, an `.emmyrc.json` global, an engine C++ change, +or a `recoil-lua-library` regeneration. These fixes belong to the env layer +(`fmt-llm-source` branch) and are out of scope for per-file workers. + +**If no category below matches your error**, report it as `UNCATEGORIZED` in +your output with the exact error message and surrounding context. A human +will either add a new category to this document or apply the env-layer fix. +Categories below that say *"already fixed by the env layer"* are cross-references +so you recognize the pattern and don't try to fix it yourself — flag and move on. + +## Branch Architecture + +`generate-branches.sh` deterministically rebuilds all branches from `origin/master`. Each transform declares a branch, commit message, PR URL, and optional prereq. + +``` +origin/master + ├─ fmt (stylua) + ├─ mig-bracket (bracket-to-dot) + ├─ mig-rename-aliases (rename-aliases) + ├─ mig-detach-bar-modules (detach-bar-modules) + └─ mig (all transforms sequentially) +``` + +Each leaf targets `master` independently. `mig` applies all transforms in sequence. + +## The Prereq Branch Pattern + +When a transform needs companion changes the codemod can't generate (sandbox wiring, config, type stubs, dependencies), put them on a **prereq branch**. The script cherry-picks it before the codemod runs. + +### build_leaf sequence + +1. `git checkout -B $branch origin/master` +2. `git cherry-pick origin/master..$prereq` (if prereq set) +3. Run codemod, `git add -A && git commit` +4. Run `post_commit_*` (if defined) +5. Run tests + +### build_mig + +Deduplicates prereqs across transforms, cherry-picks all unique ones, then runs each transform sequentially. + +### Constraints + +1. **Prereq must be harmless on master.** `BAR = BAR` in `system.lua` captures `nil` on master but the real namespace table after the codemod transforms `init.lua` (`BAR = BAR or {}` + `BAR.X = ...`). +2. **Codemod must not clobber prereq.** Don't put patterns the codemod matches in prereq files. +3. **No post-transform cruft.** If a workaround is needed (e.g. `_G.BAR = _G.Spring`), fix the codemod to handle it natively instead. + +### Existing prereqs + +| Branch | Transform | Purpose | +|--------|-----------|---------| +| `stylua` | `fmt` | `.stylua.toml`, `.styluaignore`, CI | +| `detach-bar-modules-env` | `detach_bar_modules` | System table entries, `.luarc.json` globals, type stubs | + +## Diagnosing Failures + +### Runtime: "attempt to index global 'X' (a nil value)" + +Widgets run in sandboxed environments via `setfenv` with `__index = System`. The `System` table is in `luaui/system.lua` / `luarules/system.lua`. If a codemod introduces a global not in `System`, widgets get `nil`. + +**Fix:** Add it to both `system.lua` files via a prereq branch. The detached BAR modules are exposed as a single `BAR = BAR` entry (the `BAR` namespace), not five bare globals. + +### Sandbox execution order + +``` +init.lua -> sets BAR namespace (BAR.Utilities, BAR.I18N, etc.) +barwidgets.lua -> loads system.lua -> builds System table (incl. BAR) + -> setfenv(widget_chunk, widget) where widget.__index = System +``` + +`Spring` is in `System`, so `Spring.X` always works. Bare `X` only works if added to `System`. + +### Unit tests: stubs not transformed + +Test stubs in `spec/builders/` use `_G.Spring.X = ...`. The full_moon AST parses `_G.Spring.Utilities` as prefix=`_G`, suffixes=[`.Spring`, `.Utilities`]. Codemods matching prefix=`Spring` will miss this. + +**Fix:** Extend the codemod to also match prefix `_G` + first suffix `.Spring` + second suffix `.Module`. Preferred over adding aliases in the prereq. + +### LSP: undefined-global warnings + +After introducing new globals, add to prereq: +1. `.luarc.json` `diagnostics.globals` array +2. Type stubs in `types/` based on actual implementations (read `common/springFunctions.lua`, `modules/lava.lua`, etc.) + +## Workflow + +1. **Reproduce** -- run `just bar::migrate::stylua-cleanup-generate`, check test output and/or run integration tests +2. **Diagnose** -- match error pattern to one of the categories below +3. **Create prereq branch** -- `git checkout -B prereq-name origin/master`, make changes, commit +4. **Wire it** -- set `transform_prereq="prereq-name"` in `generate-branches.sh` +5. **Fix codemod if needed** -- extend AST matching (e.g. `_G.Spring.X` pattern), run `cargo test`, `cargo build --release` +6. **Regenerate** -- `just bar::migrate::stylua-cleanup-generate`, verify all branches pass +7. **Push** -- `just bar::migrate::stylua-cleanup-generate --push --update-prs` + +## full_moon AST Reference + +`_G.Spring.Utilities.Foo()` parses as: +- prefix: `_G` +- suffixes: [`.Spring`, `.Utilities`, `.Foo`, `()`] + +`Spring.Utilities.Foo()` parses as: +- prefix: `Spring` +- suffixes: [`.Utilities`, `.Foo`, `()`] + +Both must be handled. See `detach_bar_modules.rs` `try_rewrite` for the two-pattern approach. + +--- + +## Type Error Categories + +Every category below is **idempotent** — running the fix on already-fixed code +is a no-op. Attempt every error in your chunk. If no heuristic matches, report +the error as UNCATEGORIZED with the exact message and surrounding context. + +### Category 1: Legacy COB API +**Error:** `undefined-field: SetUnitCOBValue` or `GetUnitCOBValue` + +**Fix:** Replace with the gadget-facing COB API: +```lua +-- Before +Spring.SetUnitCOBValue(unitID, COB.ACTIVATION, 0) +-- After +SpringSynced.UnitScript.SetUnitCOBValue(unitID, COB.ACTIVATION, 0) +``` +**WARNING:** Use `SetUnitCOBValue`/`GetUnitCOBValue` (3-arg, takes unitID), NOT +`SetUnitValue`/`GetUnitValue` (2-arg unitscript-internal, no unitID). + +### Category 2: Nonexistent Engine API +**Error:** `undefined-field: GetProjectileName` + +**Fix:** Replace with `SpringShared.GetProjectileDefID`. + +### Category 3: Method Name Typo +**Error:** `undefined-field: Spring.GameFrame` + +**Fix:** `SpringShared.GetGameFrame()`. + +### Category 4: Wrong Table +**Error:** `undefined-field: Spring.ZlibCompress` + +**Fix:** `VFS.ZlibCompress` / `VFS.ZlibDecompress` (note capitalization fix). + +### Category 5: UnitScript Sub-table — env-layer reference, flag as UNCATEGORIZED if seen + +Engine annotation + `just lua::library` fix. Subagents assume this is already done. + +### Category 6: UnitRendering / FeatureRendering — env-layer reference, flag as UNCATEGORIZED if seen + +Engine annotation + `just lua::library` fix. Subagents assume this is already done. + +### Category 7: Engine Constants — env-layer reference, flag as UNCATEGORIZED if seen + +Already resolved by engine annotations. No action needed. + +### Category 8: GameCMD Type Stub — env-layer reference, flag as UNCATEGORIZED if seen + +Ensure `types/GameCMD.lua` exists and `"GameCMD"` is in `.luarc.json` globals. + +### Category 9: Game.Commands / Game.CustomCommands — env-layer reference, flag as UNCATEGORIZED if seen + +Ensure `types/Game.lua` extends the `Game` class with these fields. + +### Category 10: Sandbox Globals — env-layer reference, flag as UNCATEGORIZED if seen + +Ensure ALL engine/sandbox/BAR globals are in `.emmyrc.json` `diagnostics.globals` +(EmmyLua's analyzer is the source of truth for `just bar::check`; `.luarc.json` +holds lux library paths and is consumed by the sumneko LSP for IDE use). EmmyLua +treats `undefined-global` as an **error** (not a warning like sumneko did), so a +missing global multiplies the error count by 10–100x. Full list: + +**UnitScript:** `Turn`, `Move`, `Spin`, `StopSpin`, `WaitForTurn`, `WaitForMove`, `Hide`, +`Show`, `Explode`, `EmitSfx`, `StartThread`, `SetSignalMask`, `Signal`, `Sleep`, +`GetUnitValue`, `SetUnitValue`, `piece`, `script`, `UnitScript`, `x_axis`, `y_axis`, +`z_axis`, `SIG_WALK`, `UNITSCRIPT_DIR` + +**Engine/sandbox:** `widgetHandler`, `gadgetHandler`, `Commands`, `fontHandler`, +`LUAUI_DIRNAME`, `socket`, `pairsByKeys`, `ipairs_reverse`, `SendToUnsynced`, +`CallAsTeam`, `handler`, `lowerkeys`, `addon`, `gcinfo`, `loadlib` + +**BAR-specific:** `BAR` (the detached-module namespace itself — every `BAR.X` +read errors without it), `I18N_PATH` (set globally by the i18n loader before +`init.lua` includes), `GameCMD`, `Scenario`, `game_engine`, `SG`, `CMD_AREA_MEX`, +`CMD_WANT_CLOAK`, `CMD_WANTED_SPEED`, `UpdateGuishaderBlur`, `GadgetCrashingAircraft`, +`CALLIN_MAP`, `CommandNames`, `ExplosionDefs` + +**Test framework:** `describe`, `it`, `spec`, `before_each` + +### Category 11: Stale Manual Stubs +**Error:** `duplicate-doc-field` or conflicting types from `types/Spring.lua` + +**Fix:** If `types/Spring.lua` contains `---@class SpringSynced` with `@field` entries for +engine methods (GetModOptions, GetGameFrame, etc.), remove that entire block. Keep only +BAR-side extensions (UnitScriptTable, ObjectRenderingTable) and temporary data types +(ResourceData, TeamData, PlayerData, UnitWrapper). + +### Category 12: I18N Type +The detached BAR modules live under the **`BAR` namespace**: `BAR.I18N`, +`BAR.Utilities`, `BAR.Debug`, `BAR.Lava`, `BAR.GetModOptionsCopy`. They are +**NOT** bare globals. Never strip the `BAR.` prefix — bare `I18N(...)`, +`Utilities.X`, etc. are `undefined-global`. If a local variable shadows the +name (e.g. `local I18N = state.I18N`, a string table), rename the **local** (to +`i18nStrings` or similar); never touch the `BAR.I18N(...)` call sites. + +**Fix:** The `I18NModule` class lives in `types/BAR.lua` as the `I18N` field of +the `BAR` class (callable table with `translate`, `load`, `set`, `setLocale`, +`getLocale`, `loadFile`, `unitName`, `setLanguage`, `languages` plus +`@overload fun(key, data?): string`). Call sites are `BAR.I18N(...)`. + +### Category 13: Undefined Variables / Actual Bugs +**Error:** `undefined-global` for lowercase variable names (`alpha`, `lastframeduration`) + +**Important:** Match this category ONLY after Cat 43 (`X = X` self-shadow) +and Cat 45 (unit script piece names) have been ruled out — those have +mechanical fixes that don't require declaring a new local. + +**Fix:** Declare `local varName = defaultValue` in the same lexical scope to preserve +the semantic name. Examples: +```lua +-- Before: alpha used but never defined +uniformFloat = { shaderparams = { alpha, 0.5, 0.5, 0.5 } } +-- After: preserve the name, provide the default +local alpha = 0 +uniformFloat = { shaderparams = { alpha, 0.5, 0.5, 0.5 } } +``` +For scoping bugs, hoist the `local` declaration before the block. + +**CRITICAL — `X or default` at the use site does NOT fix `undefined-global`.** +The analyzer flags the bare *identifier*, not the value. Adding `or 0` +defends against nil at runtime but leaves the error intact: + +```lua +-- WRONG (still errors — noRushTime is still an undefined identifier) +startPolygonShader:SetUniform("noRushTimer", noRushTime or 0) + +-- RIGHT (declare at file scope above any reference) +local noRushTime = 0 +-- ...later... +startPolygonShader:SetUniform("noRushTimer", noRushTime or 0) +``` + +If you find yourself writing ` or ` as the "fix", +you're not done — scroll up and add `local = ` at an +appropriate scope. Real example: `noRushTime` is only declared as a local +inside `gfx_norush_timer_gl4.lua`; other widgets (`map_startbox.lua`, +`map_startpolygon_gl4.lua`) read it as a bare identifier at runtime and +get `nil`. The correct fix is declaring the default in each reader file, +not adding `or 0` at the call site. + +**Narrow anti-pattern — don't swap an in-scope local for a fresh name.** +Typo fixes *are* allowed (see the `subunitDef`/`subUnitDef` sub-pattern +below and the real-bug recipes above). What's forbidden is replacing a +reference to a correct, in-scope local with a *different* identifier that +you think reads better — that just trades one `undefined-global` for +another. Concrete regression: a worker changed `isClientPaused` (a local +declared 29 lines above in the same function) to `pausedByThisWidget` on +one usage, introducing a new `undefined-global`. If the existing name +resolves to a visible local in the same scope, leave it. + +**Sub-pattern — destructured assignment self-reference:** + +```lua +-- WRONG (pieceAngle is the LHS being assigned, but used on the RHS) +local _, pieceAngle = spCallCOBScript(ownerID, "DroneDocked", 5, pieceAngle, droneMetaData.dockingPiece) +``` + +This is a real bug — `pieceAngle` on the RHS resolves to the global +(nil), not the local being created on the LHS. Either the developer +meant to thread an outer-scope `pieceAngle` through (in which case +declare `local pieceAngle = nil` in the enclosing block above the loop) +or the call signature genuinely doesn't need that argument (drop it). +When in doubt, declare `local pieceAngle = nil` in the enclosing block +above the first reference — preserves semantics, kills the error. + +**Sub-pattern — typo (e.g. `subunitDef` vs `subUnitDef`):** + +```lua +local subUnitDef = UnitDefNames[dronename] +if subunitDef then -- typo: lowercase 'u' + metalCost = subUnitDef.metalCost +``` + +Fix the typo. This is the only category where a worker is allowed to +make a small change to a referenced identifier (vs. only adding +declarations/annotations) — but only when the intended name is +unambiguously visible in the same function scope. + +### Category 14: Forward-Compat API Guards +**Pattern:** `if not Spring.GetAvailableControllers then return end` + +**Fix:** Comment out the guard and guarded block: +```lua +-- forward-compat: API not yet available +-- if not Spring.GetAvailableControllers then return end +``` + +### Category 15: Commented-Out Spring. References + +LuaLS ignores comments. No action needed. Not a type error. + +### Category 16: GL4 Object Methods -- VBO/VAO/Shader +**Error:** `undefined-field: Delete` / `Upload` / `DrawArrays` / `SetUniform` / etc. + +**Fix:** Find where the object is created and add a type annotation on the line before: +```lua +---@type VBO +local myVBO = gl.GetVBO(GL.ARRAY_BUFFER, true) + +---@type VAO +local myVAO = gl.GetVAO() + +---@type Shader +local myShader = gl.CreateShader({ ... }) +``` +If the object is stored in a table field, annotate the assignment: +```lua +---@type VBO +self.vbo = gl.GetVBO(GL.ARRAY_BUFFER, true) +``` +If the object comes from `makeInstanceVBOTable()`, annotate with `---@type InstanceVBOTable`. + +### Category 17: Remaining Undefined Globals +**Error:** `undefined-global` for piece names (`lloarm`, `rloarm`) in LUS scripts. + +**Fix:** Report as UNCATEGORIZED. These are LUS piece environment globals that the +engine injects per-script. + +### Category 18: Dead `if not Spring` Guards +**Fix:** Delete the guard. If needed, replace with `if not SpringShared then return end`. + +### Category 19: Erroneous `self` References +**Fix:** Replace `self.X` with `ModuleName.X` where the file is a module, not a class. + +### Category 20: Font Object Methods +**Error:** `undefined-field: Print` / `Begin` / `End` / `SetTextColor` / etc. + +**Fix:** Add `---@type LuaFont` on the line before the `gl.LoadFont()` assignment: +```lua +---@type LuaFont +local font = gl.LoadFont(fontfile, fontSize, outlineWidth, outlineWeight) +``` + +### Category 21: Engine Optional Params — env-layer reference, flag as UNCATEGORIZED if seen + +Already handled by env-layer engine annotations. If a `nil → number` error +still persists on an engine API call, apply the worker workaround: add `or 0` +as a default at the call site. + +### Category 22: MoveType / UnitDef Missing Fields + +If `GetUnitMoveTypeData()` result is used and fields are undefined, add +`---@type table` on the variable as a workaround. Report the specific +missing field as UNCATEGORIZED so a maintainer can extend the type stub +in `types/MoveTypeData.lua` (env-layer fix). + +### Category 23: Command Queue `tag` Field — env-layer reference, flag as UNCATEGORIZED if seen + +Ensure `types/Extensions.lua` has `---@class Command` with `@field tag integer?` and +`@field [string] any`. + +### Category 24: Engine Math Extensions +**Error:** `undefined-field: fract` on `math` + +**Fix:** Replace `math.fract(x)` with `(x - math.floor(x))`. The engine does not expose +`math.fract` as a registered Lua function. + +### Category 25: Callin Missing Arguments +**Error:** `missing-parameter` on callin bootstrap in `Initialize` + +**Fix:** Pass full args: +- `UnitCreated(unitID, unitDefID, teamID, builderID?)` -- pass `SpringShared.GetUnitTeam(unitID)` +- `PlayerChanged(playerID)` -- pass `SpringUnsynced.GetLocalPlayerID()` +- `FeatureCreated(featureID, allyTeam)` -- pass `SpringShared.GetFeatureAllyTeam(fID)` +- `ViewResize(viewSizeX, viewSizeY)` -- pass `SpringUnsynced.GetViewGeometry()` +- `UnitDestroyed(unitID, unitDefID, unitTeam)` -- pass all 3 + +### Category 26: Nilable Returns Passed to Required Params +**Error:** `Cannot assign number? to parameter number` + +**Fix:** Add `--[[@as number]]` cast, or `or 0` if used in arithmetic. Do NOT leave +unfixed -- at minimum apply the cast. +```lua +-- Before +AreTeamsAllied(GetUnitTeam(unitID), myTeam) +-- After (cast) +AreTeamsAllied(GetUnitTeam(unitID) --[[@as integer]], myTeam) +-- After (default, preferred if in arithmetic) +local team = GetUnitTeam(unitID) or 0 +``` + +### Category 27: `string` vs `stringlib` +**Error:** `string cannot match stringlib` + +**Fix:** Add `---@diagnostic disable-next-line: param-type-mismatch` on the affected line. +This is a known LuaLS limitation with Lua's string metatable. + +### Category 28: Duplicate Table Keys +**Error:** `Duplicate index X` + +**Fix:** Remove the duplicate key (keep the last one, which is what Lua uses). + +### Category 29: Busted/Luassert Test Framework + +Known gap. Leave for now. Test intellisense is a separate workstream. + +### Category 30: Duplicate `@class` Blocks +**Error:** `duplicate-doc-field` + +**Fix:** Consolidate to a single `@class` definition per type. Keep one file per class +in `types/` (e.g., `types/Command.lua`). Do NOT create monolithic `types/Extensions.lua` +with multiple classes -- use one file per class. + +### Category 31: Missing `addon` Global +**Fix (idempotent):** Ensure `types/Addon.lua` contains: +```lua +---@type Addon +---@diagnostic disable-next-line: lowercase-global +addon = nil +``` +If `"addon"` is not in `.emmyrc.json` globals, report as UNCATEGORIZED — env-layer fix. + +### Category 32: SetGameRulesParam with Boolean +**Error:** `Cannot assign boolean to parameter (string|number)?` + +**Fix:** The engine accepts booleans. If stubs are already fixed, this resolves. If not, +convert to integer: `SetGameRulesParam(name, value and 1 or 0)`. Report as UNCATEGORIZED +if the stub still rejects boolean after engine fixes. + +### Category 33: GetGameRulesParam Default Arg +**Error:** `redundant-parameter` on `GetGameRulesParam(name, 0)` + +**Fix:** The engine only accepts 1 arg. Move the default to `or`: +```lua +-- Before +local val = GetGameRulesParam("key", 0) +-- After +local val = GetGameRulesParam("key") or 0 +``` + +### Category 34: Callin Routing Extra Args — env-layer reference, flag as UNCATEGORIZED if seen + +**Error:** `redundant-parameter` on `widget:UnitCreated(..., nil, "UnitFinished")` + +BAR passes extra string args through callins for internal routing. Lua silently ignores them. + +**Fix (env-layer):** Extend callin types in `types/Callins.lua` to accept `...: any`: +```lua +---@class Callins +---@field UnitCreated fun(self, unitID: integer, unitDefID: integer, unitTeam: integer, builderID: integer?, ...: any)? +---@field UnitDestroyed fun(self, unitID: integer, unitDefID: integer, unitTeam: integer, attackerID: integer?, attackerDefID: integer?, attackerTeam: integer?, weaponDefID: integer?, ...: any)? +---@field GameFrame fun(self, frame: integer, ...: any)? +``` + +### Category 35: `need-check-nil` +**Error:** `Need check nil.` + +**Strategy -- apply the FIRST matching rule:** + +1. **Table/method access on nil** (`x[field]`, `x.field`, `x:method()` where `x` is `T?`): + Add `if not x then return end` before the access. SAFE in widget callins (DrawScreen, + DrawWorld, Update, GameFrame) where returning early just skips a frame. + +2. **Nil in arithmetic** (`x + 1` where `x` is `number?`): + Replace with `(x or 0) + 1`. + +3. **Nil passed to function** (`fn(x)` where `x` is `T?`): + Replace with `fn(x or default)` -- `0` for numbers, `""` for strings, `{}` for tables. + +4. **Nested table access** (`tbl[a][b]` where `tbl[a]` might be nil): + Add `if not tbl[a] then return end` or use `tbl[a] and tbl[a][b]`. + +5. **In `Initialize` / `Shutdown` / game-logic functions**: Do NOT add `return` guards + (would break state). Instead add `or default` or `--[[@as Type]]` cast. Flag in output + as `ATTEMPTED (need-check-nil in game logic)` for human review. + +### Category 36: `cast-local-type` +**Error:** `This variable is defined as type X. Cannot convert its type to Y.` + +**Fix:** Add `---@type X|Y` before the variable declaration, or `---@type Y` before the +reassignment line: +```lua +---@type integer? +local checkQueueTime = 0 +-- ... later ... +checkQueueTime = nil -- no longer errors +``` + +### Category 37: `assign-type-mismatch` +**Error:** `Cannot assign X to Y.` + +**Fix:** Add `--[[@as Y]]` cast on the right-hand side: +```lua +local widget = widget --[[@as Widget]] +``` + +### Cast Syntax Rules (CRITICAL) + +**NEVER put array types inside `--[[@as ...]]` block comments.** The first `]` in +`integer[]` (or `Foo[]`) **closes the long comment**, leaving trailing `]` as code +and producing `unknown-symbol` / parse errors. + +```lua +-- WRONG (comment ends at first ]) +local xs = f() --[[@as integer[]]] + +-- RIGHT: cast the local on the next line +local xs = f() +---@cast xs integer[] + +-- RIGHT: alias without brackets inside the block comment +---@alias IntegerArray integer[] +local xs = f() --[[@as IntegerArray]] +``` + +**`---@cast`** only works on LOCAL VARIABLES, never on table fields: +```lua +-- CORRECT +---@cast myVar VBO +myVar:Delete() + +-- WRONG (causes unknown-cast-variable error) +---@cast tbl.field VBO +tbl.field:Delete() +``` + +For table fields, extract to a local variable (preferred): +```lua +local v = tbl.field +if v then v:Delete() end +``` + +**NEVER** use `--[[@as Type]]` at the end of a line if the NEXT line starts with `(`. +Lua 5.1 treats `)\n(` as a function call chain, causing `ambiguous-syntax` errors. +If you must use inline casts on table field access, put a semicolon before the next line: +```lua +local x = tbl.field --[[@as VBO]] +;(otherTbl.vao):DrawArrays(...) -- semicolon prevents ambiguity +``` + +### Type File Hygiene (CRITICAL) + +- Do NOT create `types/` files that duplicate classes already defined in + `recoil-lua-library/library/generated/*.lua` or `modules/graphics/instancevbotable.lua`. + LuaLS merges class definitions and duplicates cause `duplicate-doc-field` errors. +- Check if a class already exists before creating a new type file for it. +- Classes already defined elsewhere: `VBO`, `VAO`, `Shader`, `LuaFont`, + `InstanceVBOTable`, `Callins`, `Widget`, `Gadget`, `Addon`. + +**NEVER declare `---@class Widget` / `---@class VAO` / `---@class VBO` inside +`luaui/Widgets/*.lua` (or gadget files).** EmmyLua merges `@class` across the whole +workspace; a widget-local “patch” overwrites or conflicts with `types/Widget.lua` and +engine stubs, causing cascading `assign-type-mismatch` / `duplicate-doc-field` in other +files. Use only: + +```lua +local widget ---@type Widget = widget +``` + +If a callin is truly missing, flag UNCATEGORIZED so a maintainer can extend +`types/Widget.lua` (env-layer fix). + +### Category 38: `Shader` vs BAR `gl.LuaShader` objects +**Error:** `undefined-field: SetUniform` / `Activate` / `SetUniformInt` on a value typed +as `Shader`. + +**Cause:** `types/Shader.lua` defines `---@alias Shader integer` (OpenGL program ID). +BAR’s `gl.LuaShader({ ... })` return value is **not** that integer; it is a userdata +shader object. + +**Fix:** Annotate as `---@type BarLuaShader?` (or non-optional when known initialized). +Add missing method stubs to `types/BarLuaShader.lua` if LuaLS still complains. + +### Category 39: `---@type X ` description after type +**Error:** `doc-syntax-error: expect type` / `binary operator not followed by type` + +**Cause:** EmmyLua parses everything after `---@type` as a type expression. Bare prose +words like `in seconds` are parsed as type tokens (`in` is reserved as a binary +type operator) and produce parse errors. + +**Fix:** Use the `#` description delimiter or move the prose to a separate comment line. + +```lua +-- WRONG (prose collides with type parser) +local doubleClickTime = 0.2 ---@type number in seconds +---@type integer in pixels, as the Manhattan norm +local dist = 12 + +-- RIGHT (# delimiter) +local doubleClickTime = 0.2 ---@type number # in seconds +---@type integer # in pixels, as the Manhattan norm +local dist = 12 +``` + +The `#` form is supported by LuaLS, EmmyLua, and `lua-language-server`. + +### Category 40: `---@field X T [interval]` bracket prose in field doc +**Error:** `syntax-error: expected TkRightBracket, but get TkComma` + +**Cause:** `---@field name T ` allows trailing prose, but a leading `[` +is parsed as the start of `T[]` array syntax. `integer [0, 1e6) ...` is parsed as +`integer[`, then `0`, then expects `]` but finds `,`. + +**Fix:** Wrap interval notation in backticks AFTER the prose, or push it to the end. + +```lua +-- WRONG (leading bracket parsed as array) +---@field crushstrength integer [0, 1e6) mass equivalent for crushing + +-- RIGHT (backticked, at end) +---@field crushstrength integer mass equivalent for crushing, in `[0, 1e6)` +``` + +### Category 41: `---@type FuncType` on `local function` declaration +**Error:** `annotation-usage-error: ` `` `@type X` can't be used here `` + +**Cause:** EmmyLua only allows `---@type` on variable assignments, not on +`local function name(...)` declarations. + +**Fix:** Convert to `local name = function(...)` form. + +```lua +-- WRONG +---@type ShieldPreDamagedCallback +local function shieldPreDamaged(projectileID, ...) end + +-- RIGHT +---@type ShieldPreDamagedCallback +local shieldPreDamaged = function(projectileID, ...) end +``` + +Alternative (preferred when callback is exported): annotate the function with +`---@param`/`---@return` directly instead of using a callback type alias. + +### Category 42: Misplaced `---@return` or `---@param` +**Error:** `annotation-usage-error: ` `` `@return X` can't be used here `` + +**Cause:** `---@return` / `---@param` annotations belong immediately above a +function declaration, not above an unrelated `local x = ...` line. + +**Fix:** Move the annotation block to the line directly above the actual function +it documents. + +```lua +-- WRONG (comment binds to wrong declaration) +---@return number +local currentBlueprintUnitID = 0 +local function nextBlueprintUnitID() ... end + +-- RIGHT (comment binds to the function) +local currentBlueprintUnitID = 0 +---@return number +local function nextBlueprintUnitID() ... end +``` + +### Category 43: `X = X` same-name self-shadow (idiomatic global capture) + +**Error:** `undefined global variable: X` on a line where `X` appears on +both sides of `=` with the same name. Three sub-patterns: + +1. **Local capture** — `local X = X` or `local X = X or ` +2. **Table-export field** — `X = X,` or `X = X or {},` inside a + `{ ... }` literal (typically a `return { ... }` module export or + `WG.Module = { ... }` widget API table) +3. **Plain reassign** — `X = X` (rare, but the same fix applies) + +**Cause:** The `X = X` pattern intentionally references a same-named +global — for performance (`local pairs = pairs`), for capturing optional +config globals (`local logRAM = logRAM`, `local noRushTime = noRushTime +or 0`), or for re-exporting a private upvalue under the same public name +in a module-export table (`return { customPresets = customPresets or {} +}`). The right-hand `X` is the global, which the analyzer can't see. +Adding `X` to `.emmyrc.json` is heavy-handed when the global is +file-scoped or rarely set. + +**Fix:** Insert a `disable-next-line` comment immediately above the line. +Idempotent (runs as a no-op if the comment is already there). + +```lua +-- WRONG (analyzer can't see the right-hand X) +local running = running +local logRAM = logRAM +local noRushTime = noRushTime or 0 + +return { + customPresets = customPresets or {}, + uploadElementRange = uploadElementRange, +} + +-- RIGHT +---@diagnostic disable-next-line: undefined-global +local running = running +---@diagnostic disable-next-line: undefined-global +local logRAM = logRAM +---@diagnostic disable-next-line: undefined-global +local noRushTime = noRushTime or 0 + +return { + ---@diagnostic disable-next-line: undefined-global + customPresets = customPresets or {}, + ---@diagnostic disable-next-line: undefined-global + uploadElementRange = uploadElementRange, +} +``` + +**Sibling pattern — defensive `X and X() or default` cross-file reference:** + +```lua +-- WRONG (GetAliveTeammates is in another file the analyzer can't see) +local teammates = GetAliveTeammates and GetAliveTeammates() or {} + +-- RIGHT +---@diagnostic disable-next-line: undefined-global +local teammates = GetAliveTeammates and GetAliveTeammates() or {} +``` + +This isn't strictly a self-shadow, but the fix is identical: a single +`disable-next-line` above the call site. The `X and X()` guard already +proves the developer knows `X` may not exist. + +**Do NOT** convert these to `local running = _G.running` — that loses +the `or default` ergonomics and is harder to read. The disable comment +is the canonical fix. + +**Do NOT** stack disable-next-line comments at the top of the file +hoping they'll cover later lines — `disable-next-line` only affects the +**single line directly below** the comment. If a file has many such +captures, put one disable above each one. + +**Do NOT** write `local X = X` inside a function body when no same-named +global or upvalue exists. This pattern *only* works when the right-hand +`X` already resolves to something the analyzer can see (a global, an +outer-scope local). If `X` is truly undefined everywhere in the file, +`local X = X` just produces a nil-valued local and the analyzer still +errors on the right-hand side. The code is now dirtier and the bug is +masked. Real-world regression: a worker "fixed" `tracy.ZoneBeginN(fname)` +inside `AIBase:tracyZoneBeginMem()` by inserting `local fname = fname` +on the line above — but `fname` was never declared anywhere, the real +bug was a missing function parameter. If you can't find where `X` is +supposed to come from, flag UNCATEGORIZED and let a maintainer add the +missing parameter / declaration / import. + +### Category 44: `---@param`/`---@return` on a function re-export site + +**Error:** `` `@param X T` can't be used here `` / +`` `@return X` can't be used here `` on a line of the form +` = ` — either inside a table literal +(`WG.Module = { foo = foo, }`) or as a direct field assignment +(`table.toString = tableToString`). + +**Cause:** The `@param`/`@return` annotations are attached to the +**re-export** of an upvalue function reference, not to the actual +function definition. EmmyLua only accepts these annotations directly +above a `function` declaration, never above an assignment that merely +stores a function reference under a different name. The annotations +are valuable (they document the real signature), they're just in the +wrong location. + +**Fix:** **Move** the `@param`/`@return` block from the re-export site +to the line directly above the actual `local function (...)` (or +` = function(...)`) definition for that upvalue. The leading +prose `---` description lines should also move with them. Find the +function definition by searching for `function ` in the same file. + +Idempotent: re-running this fix on already-relocated annotations is a +no-op (the analyzer is happy, no error to match against). + +```lua +-- WRONG: annotations live on the re-export, not the function + +-- 1. Inside a table literal +local function addSpotlight(objectType, owner, objectID, color, options) + -- ... body ... +end + +WG.ObjectSpotlight = { + --- Adds a new spotlight for a given object. + --- @param objectType string "unit", "feature", or "ground" + --- @param owner string An identifier... + --- @param objectID number|number[] unitID, featureID, ... + --- @return nil + addSpotlight = addSpotlight, +} + +-- 2. Direct field assignment +local function tableToString(tbl, options, _seen, _depth) end +-- ... body ... + +---Recursively turns a table into a string, suitable for printing. +---@param tbl table +---@param options table Optional parameters +---@return string +table.toString = tableToString + + +-- RIGHT: annotations live above the function definition + +-- 1. Inside a table literal — moved to local function +--- Adds a new spotlight for a given object. +--- @param objectType string "unit", "feature", or "ground" +--- @param owner string An identifier... +--- @param objectID number|number[] unitID, featureID, ... +--- @return nil +local function addSpotlight(objectType, owner, objectID, color, options) + -- ... body ... +end + +WG.ObjectSpotlight = { + addSpotlight = addSpotlight, +} + +-- 2. Direct field assignment — moved to function expression +---Recursively turns a table into a string, suitable for printing. +---@param tbl table +---@param options table Optional parameters +---@return string +local tableToString -- forward decl +tableToString = function(tbl, options, _seen, _depth) + -- ... body ... +end + +table.toString = tableToString +``` + +**Edge case — `@field` on a `@param options table`:** When the +re-export annotates an `options` parameter with sub-fields like +`@param options.duration number`, those `@param options.X` lines must +also move with the rest of the block. They are valid syntax above a +`local function` declaration. + +**If the function definition is in a different file** (rare — usually +means the re-export is doing real work, not just shimming an upvalue): +flag UNCATEGORIZED. Don't try to chase the cross-file reference. + +**CRITICAL pitfall — DO NOT try to "disable" annotations by adding a +space:** EmmyLua is more permissive than the older sumneko LSP and +recognizes BOTH `---@param` (no space) AND `--- @param` (with space) +as type annotations. A worker that "fixes" the error by inserting a +space is doing nothing — the analyzer still produces the same error, +the file is dirtier, and the original annotation is lost. The +**only** correct fix is to relocate the `---@param`/`---@return` block +to the function definition. If you can't find the function definition +in the same file, flag UNCATEGORIZED and stop. Do not edit the line +unless you are moving annotations to a real function declaration. + +**CRITICAL pitfall — DO NOT insert `local X = X` between the +annotations and the re-export** to "give the @params something to +attach to". That line is a no-op (the outer `local X` is already in +scope — you're shadowing it with itself), it doesn't attach the +annotations to anything the analyzer recognizes, and the +annotation-usage-error persists. Worse, it adds dead code that a +future reader has to puzzle over. Real-world regression: a worker did +`local tableToString = tableToString` right before +`table.toString = tableToString` in `common/tablefunctions.lua` and +the error never cleared. The only working fix is moving the +annotations; no shim line makes the re-export site valid. + +### Category 46: Hoisted-local forward declaration + +**Error:** `undefined global variable: ` where `` IS assigned +inside the same file but the assignment is in a deeper lexical scope +(inside a function body, an `if` block, a `for` loop, etc.) than the +read site that errors. + +**How to recognize:** Search the file for `` (or ` =` / +`local =`). If the only definitions live inside a nested scope +but the error site is at module level (or in a sibling function, or +inside an unrelated function), this is the pattern. + +**Cause:** Lua scoping. `local x = 5` inside a function only exists for +that function's lifetime. References outside that function fall through +to the global table, where `x` is nil. The original author either +intended `x` to be a module-level upvalue (and forgot to hoist the +declaration) or to be cleared between calls (in which case the read +site is reachable when the value is nil — a real runtime bug). + +**Fix:** Add a forward declaration `local ` (no initializer, or +`= nil` / `= false` / `= 0` / `= {}` to match the type the rest of the +file expects) at the top of the enclosing module-level scope, ABOVE all +references and assignments. The existing nested assignments become +regular reassignments to the upvalue. Idempotent: re-running on +already-hoisted code is a no-op (the local already exists, the analyzer +is happy). + +```lua +-- WRONG (running is set inside a callback, read in a sibling function) +local function StartHook() + running = true -- creates a NEW global, not the local we want +end + +local function CheckHook() + if hookset then + if not running then -- error: undefined global 'running' + KillHook() + end + end +end + +-- RIGHT (forward declaration at file scope) +local running = false -- hoisted; matches the boolean default + +local function StartHook() + running = true -- now reassigns the upvalue +end + +local function CheckHook() + if hookset then + if not running then -- reads the upvalue + KillHook() + end + end +end +``` + +**Default value selection** — pick whatever matches the file's existing +usage: +- Booleans → `false` (or `nil` if the code distinguishes "unset" from "false") +- Numbers → `0` +- Strings → `""` +- Tables → `nil` (so the code's existing `if ~= nil` guards still fire) +- Functions → `nil` (most file-scope forward decls) +- GL display lists / VBO IDs → `nil` (so `if ~= nil then glDeleteList() end` still works) + +**Edge case — `noRushTime` style "set in widget callin, read at module level":** +The fix is the same. Hoist `local noRushTime = 0` at file scope. The +callin assignment becomes a regular reassignment. + +**When NOT to use this category** — flag UNCATEGORIZED instead if: +- The variable name appears nowhere else in the file (it's a typo or + references something cross-file — Cat 13 territory) +- The fix would change visible runtime semantics in a non-trivial way + (e.g. reading an unset value used to crash, hoisting hides the crash) + +**Special case — `self` outside a `:method` body:** Real bug. `self` is +only defined inside `widget:method(...)` / `gadget:method(...)` / +`addon:method(...)` bodies. References from `local function ...` blocks +or module scope resolve to nil. The intended fix is almost always to +replace `self` with the file-scope `widget` / `gadget` / `addon` upvalue +(both files reference the local with `local widget = widget --type Widget` +at the top). In `:method` bodies `self == widget` so the substitution +is a no-op; in plain functions it picks up the correct value. + +This is an env-layer fix (tracked on `fmt-llm-source`), NOT a worker +fix — the substitution is mechanical but the *decision* that `widget` +is the right replacement requires reading the file's idiom and +understanding BAR's widget lifecycle. See the env commit's +"Manual judgment-call fixes" section for an example +(widget_selector.lua + gui_options.lua, 8 sites total). + +Workers seeing `undefined-global: self` should flag UNCATEGORIZED with +the note "self-outside-:method, env-layer fix" so maintainers can pick +it up in the next env-layer pass. + +### Category 45: Unit script piece-name globals + +**Error:** `undefined global variable: ` in a unit script file +where `` is something like `lloarm`, `rloarm`, `torso`, +`luparm`, `ruparm`, `dirt`, `flare`, etc. + +**Cause:** Unit scripts run in the engine's unit-script sandbox, which +exposes model piece names as globals at runtime via metatable. The +analyzer has no way to know which pieces a given unit declares. + +**Match scope** — apply this category if EITHER: +- The file lives under `scripts/Units/**/*.lua` (per-unit script), OR +- The file lives under `scripts/headers/**/*.lua` (shared unit-script + header that gets `include`d into scripts and references piece globals) + +**Fix:** Add a single file-level `---@diagnostic disable: undefined-global` +at the **very top of the file** (line 1, before any code). This is +idempotent and covers every reference in the file — always use this +form in unit scripts, even when the error list shows only 1 or 2 +undefined-global references. More references may exist that weren't in +this worker's chunk, and even for a truly 2-reference file the +file-level form is still shorter than placing a `disable-next-line` +above each. + +```lua +---@diagnostic disable: undefined-global + +function DrawWeapon(id) + Turn(lloarm, 1, ang(-90), ang(300)) + Turn(rloarm, 1, ang(-90), ang(300)) + ... +end +``` + +**CRITICAL anti-pattern — DO NOT place `disable-next-line` above a +`function` declaration hoping to cover the body.** `disable-next-line` +disables only the single line directly below — that's the `function` +declaration line, not the body. The undefined-global references inside +the body (often dozens of lines later) still error. Real-world +regression: a worker put `---@diagnostic disable-next-line: +undefined-global` above `function ResumeBuilding()` in +`scripts/Units/corcom_lus.lua` trying to silence `buildheading`/ +`buildpitch` in the body — the errors persisted because those +references are on the *next next* lines, not the function declaration +line. Always use the file-level form. + +For files outside the unit-script path, prefer Category 13 (declare a +local) or Category 43 (self-shadow capture). + +**Cleanup of stacked stubs:** If a previous pass left multiple +`---@diagnostic disable-next-line: undefined-global` lines stacked at the +top of the file (those don't work), replace them with a single +`---@diagnostic disable: undefined-global` (no `-next-line`). + +--- + +## Structural Type Fixes (env-layer reference) + +These are applied to the env layer (`fmt-llm-source` branch + engine PRs) before +the worker run. They are listed here so workers recognize the patterns and don't +attempt to fix them per-file. If you see a related error after the env layer is +in place, flag it as UNCATEGORIZED. + +### GL Type Alias +`types/GL.lua`: `---@alias GL integer` + +### Widget/Gadget/Addon Open Types +`types/Widget.lua`, `types/Gadget.lua`, `types/Addon.lua`: `---@field [string] any` + +### Engine Shader Params +`LuaShaders.cpp`: `@param shaderID Shader|integer` on all shader functions. + +### Engine Optional Params +Many engine APIs have `luaL_opt*` for optional params. A maintainer adds `?` to +`@param` annotations in engine C++ and regenerates the stubs via `just lua::library`. + +### Engine Missing Function Annotations +Some engine functions lack `@function` doc blocks. A maintainer adds them and +regenerates the stubs. Example: `gl.LoadFont` was missing, now annotated with `@return LuaFont`. + +### duplicate-set-field +Disabled project-wide in `.luarc.json` via `diagnostics.disable: ["duplicate-set-field"]`. + +### Cascade Warning +Adding type annotations can INCREASE error counts (LuaLS strict-checks downstream usage). +Accept this as the cost of true type safety. Prefer non-optional returns for functions +that rarely fail (fonts, VBOs). + +--- + +## Diagnostic Reference for `types/` Stubs + +| File | Defines | Source of truth | +|------|---------|-----------------| +| `types/Spring.lua` | BAR-side `UnitScriptTable`/`ObjectRenderingTable` extensions, temp data classes | `unit_script.lua`, `unitrendering.lua` | +| `types/GameCMD.lua` | `GameCMD` class | `modules/customcommands.lua` | +| `types/Game.lua` | `Game.Commands`, `Game.CustomCommands` | `init.lua` + `modules/commands.lua` | +| `types/BAR.lua` | `BAR` namespace: `I18N` (`I18NModule`), `Utilities`, `Debug` (`BARDebug`), `Lava`, `GetModOptionsCopy` — all fields of the `BAR` class | `common/springFunctions.lua`, `modules/i18n/i18n.lua`, `common/springUtilities/debug.lua`, `modules/lava.lua`, `common/springOverrides.lua` | +| `types/Gadget.lua` | `Gadget`, `gadget`, `GG` | Engine gadget handler | +| `types/Widget.lua` | `Widget`, `widget`, `WG` | Engine widget handler | +| `types/Addon.lua` | `Addon`, `AddonInfo`, `addon` | Engine addon base | +| `types/GL.lua` | `GL` alias to `integer` | Engine GL constants | +| `types/Extensions.lua` | `Command`, `Blueprint`, `RmlUi.ElementPtr`, etc. | Runtime extensions | +| `types/Callins.lua` | Callin overrides with `...: any` for routing args | Engine callin stubs | + +Subagents may CREATE new `types/ClassName.lua` files for classes they need to extend. +Use `---@meta` header. One class per file. diff --git a/docker/dev.Containerfile b/docker/dev.Containerfile index d6e9f389..7a739651 100644 --- a/docker/dev.Containerfile +++ b/docker/dev.Containerfile @@ -9,6 +9,7 @@ RUN dnf install -y --setopt=install_weak_deps=False \ just \ gcc gcc-c++ make git curl jq unzip binutils \ gawk \ + python3-pip \ gpgme \ python3-tkinter python3-requests python3-six \ SDL2-devel DevIL-devel glew-devel openal-soft-devel \ diff --git a/just/bar-migrate.just b/just/bar-migrate.just new file mode 100644 index 00000000..4462e042 --- /dev/null +++ b/just/bar-migrate.just @@ -0,0 +1,93 @@ +set export + +# Versioned bulk migrations, adoption-dated, listed in order. Retire an entry +# to [private] ~6 months after adoption; generation recipes are [private] from +# the start. + +DEVTOOLS_DIR := justfile_directory() +BAR_DIR := DEVTOOLS_DIR / "Beyond-All-Reason" +CODEMOD_DIR := DEVTOOLS_DIR / "bar-lua-codemod" +CODEMOD_BIN := CODEMOD_DIR / "target" / "release" / "bar-lua-codemod" + +[private] +require-bar: + #!/usr/bin/env bash + set -euo pipefail + source "$DEVTOOLS_DIR/scripts/common.sh" + require_repo bar Beyond-All-Reason "Beyond All Reason" + +[private] +require-codemod: + #!/usr/bin/env bash + [ -x "{{CODEMOD_BIN}}" ] && exit 0 + set -euo pipefail + source "$DEVTOOLS_DIR/scripts/common.sh" + step "bar-lua-codemod not found, building..." + bash "$DEVTOOLS_DIR/scripts/codemod-cargo.sh" build --release + ok "Built bar-lua-codemod" + +# 2026-07: codemod transforms + stylua (contributors run after rebasing; idempotent) +stylua-cleanup: require-bar require-codemod + #!/usr/bin/env bash + set -euo pipefail + source "$DEVTOOLS_DIR/scripts/common.sh" + enter_distrobox + # Same skip set as .styluaignore: vendored utils, lux/library tool dirs, + # and mapgenerator (mapinfo_template.lua is a ${}-placeholder template, + # not valid Lua). + excludes=(--exclude common/luaUtilities --exclude .lux --exclude recoil-lua-library --exclude mapgenerator) + step "bracket-to-dot transform..." + "{{CODEMOD_BIN}}" bracket-to-dot --path "$BAR_DIR" "${excludes[@]}" + step "rename-aliases transform..." + "{{CODEMOD_BIN}}" rename-aliases --path "$BAR_DIR" "${excludes[@]}" + step "detach-bar-modules transform..." + "{{CODEMOD_BIN}}" detach-bar-modules --path "$BAR_DIR" "${excludes[@]}" + step "stylua..." + (cd "$BAR_DIR" && lx --lua-version 5.1 exec stylua .) + ok "stylua-cleanup complete" + +[private] +require-library: + just lua::library + +# $CREDENTIALS_RUNNER is an opaque command prefix that loads secrets into +# the environment of whatever it wraps. When set, stylua-cleanup-generate runs +# generate-branches.sh through it so subprocesses (notably the parallel +# `claude --print` workers spawned by scripts/codemod/llm-type-triage.sh) inherit +# the loaded credentials. Without this, the workers would run with the +# credential-less env that just/bash inherits and fail at auth. +# +# Write the prefix however your secrets store wants. Tilde expansion is +# applied. Examples: +# +# # 1Password CLI +# CREDENTIALS_RUNNER="op run --env-file=~/code/ai.env.op --" +# +# # Bitwarden CLI +# CREDENTIALS_RUNNER="bw run --envfile ~/secrets/ai.env --" +# +# # direnv loader (treats first arg as a directory whose .envrc to apply) +# CREDENTIALS_RUNNER="direnv exec ~/code/secrets" +# +# # raw env injection from a sourceable .env-style file +# CREDENTIALS_RUNNER="env $(grep -v '^#' ~/.config/anthropic/key.env | xargs)" +# +# When unset, the script runs unwrapped — fine for non-AI invocations +# (e.g. `--push` only) where no subprocess actually needs credentials. + +# Regenerate fmt, mig-*, and mig branches from origin/master (uses $CREDENTIALS_RUNNER for AI auth) +[private] +stylua-cleanup-generate *args: require-bar require-codemod require-library + #!/usr/bin/env bash + set -euo pipefail + script="$DEVTOOLS_DIR/scripts/codemod/generate-branches.sh" + if [[ -n "${CREDENTIALS_RUNNER:-}" ]]; then + # Expand ~ in the prefix (most secrets-wrapper tools don't expand + # it themselves when invoked through a non-interactive shell). + expanded="${CREDENTIALS_RUNNER//\~/$HOME}" + echo "[info] wrapping stylua-cleanup-generate with: $expanded" >&2 + # eval re-parses the prefix so multi-token wrappers like + # `op run --env-file=path --` keep their structure when prepended. + eval "exec $expanded bash \"\$script\" {{args}}" + fi + exec bash "$script" {{args}} diff --git a/just/bar.just b/just/bar.just index c97af0fa..284f50d3 100644 --- a/just/bar.just +++ b/just/bar.just @@ -4,6 +4,14 @@ DEVTOOLS_DIR := justfile_directory() COMPOSE_FILE := DEVTOOLS_DIR / "docker-compose.dev.yml" COMPOSE := "podman compose -f " + COMPOSE_FILE BAR_DIR := DEVTOOLS_DIR / "Beyond-All-Reason" +CODEMOD_DIR := DEVTOOLS_DIR / "bar-lua-codemod" +CODEMOD_BIN := CODEMOD_DIR / "target" / "release" / "bar-lua-codemod" +# Batch check uses `emmylua_check` (Rust EmmyLua analyzer), baked into the dev +# container image. `emmylua_ls` is LSP-only (no --check) and lives on the host +# for editor integration — see `just setup::editor`. + +# Versioned bulk migrations (adoption-dated, listed in order) +mod migrate 'bar-migrate.just' [private] require-bar: @@ -108,16 +116,36 @@ log *TAIL_FLAGS: logs *TAIL_FLAGS: @just bar::log {{TAIL_FLAGS}} -# Type-check BAR Lua code (EmmyLua analyzer CLI) -check *args: require-bar require-emmylua-check +# Type-check BAR Lua code (EmmyLua analyzer CLI — `-c .emmyrc.json` avoids LuaLS-only keys in `.luarc.json`) +check *args: require-bar #!/usr/bin/env bash set -euo pipefail source "$DEVTOOLS_DIR/scripts/common.sh" + enter_distrobox step "Running emmylua_check (EmmyLua analyzer)..." cd "$BAR_DIR" emmylua_check -c .emmyrc.json . {{args}} ok "Type check complete" +# Type-check but only surface errors — warnings/hints dropped. Exits 0 iff no errors +# (emmylua_check itself exits non-zero on any diagnostic, so we re-derive exit code from the error count). +check-errors: require-bar + #!/usr/bin/env bash + set -uo pipefail + source "$DEVTOOLS_DIR/scripts/common.sh" + enter_distrobox + step "Running emmylua_check (errors only)..." + cd "$BAR_DIR" + json="$(emmylua_check -c .emmyrc.json -f json . 2>/dev/null || true)" + printf '%s\n' "$json" | jq -r '.[] | . as $f | .diagnostics[] | select(.severity == 1) | + "\($f.file):\(.range.start.line + 1):\(.range.start.character + 1): \(.code): \(.message)"' + count="$(printf '%s\n' "$json" | jq '[.[] | .diagnostics[] | select(.severity == 1)] | length')" + if [[ "$count" -gt 0 ]]; then + err "$count error(s)" + exit 1 + fi + ok "No errors" + # Lint BAR Lua code (luacheck via lux) lint *args: require-bar #!/usr/bin/env bash @@ -213,6 +241,24 @@ integrations *args: require-bar # Run all BAR tests (unit + integrations) test: units integrations +# Build bar-lua-codemod binary +codemod-build: + #!/usr/bin/env bash + set -euo pipefail + source "$DEVTOOLS_DIR/scripts/common.sh" + step "Building bar-lua-codemod..." + bash "$DEVTOOLS_DIR/scripts/codemod-cargo.sh" build --release + ok "Built: {{CODEMOD_BIN}}" + +# Run bar-lua-codemod unit tests +codemod-test: + #!/usr/bin/env bash + set -euo pipefail + source "$DEVTOOLS_DIR/scripts/common.sh" + step "Testing bar-lua-codemod..." + bash "$DEVTOOLS_DIR/scripts/codemod-cargo.sh" test + ok "All tests passed" + # Install git pre-commit hook in the BAR repo setup-hooks: #!/usr/bin/env bash @@ -225,12 +271,47 @@ setup-hooks: exit 1 fi mkdir -p "$(dirname "$hook")" - printf '%s\n' \ - '#!/usr/bin/env bash' \ - 'set -e' \ - 'echo "[pre-commit] Running stylua..."' \ - 'lx --lua-version 5.1 exec stylua .' \ - > "$hook" + cat > "$hook" <<'HOOK' + #!/usr/bin/env bash + # Checks the staged blobs, not the working tree: what git records is what + # gets checked, and staged content is always LF, so .gitattributes eol and + # stylua's line_endings can disagree without breaking the gate. Refuses + # rather than rewriting -- a hook that edits the tree mid-commit changes + # what you are committing behind your back. + set -euo pipefail + + cd "$(git rev-parse --show-toplevel)" + + mapfile -t staged < <(git diff --cached --name-only --diff-filter=ACMR -- '*.lua') + ((${#staged[@]})) || exit 0 + + stylua_cmd=(stylua) + command -v stylua >/dev/null 2>&1 || stylua_cmd=(lx --lua-version 5.1 exec stylua) + + # stylua honours .styluaignore only for paths it discovers, never for a + # path handed to it, so filter here. + bad=() + for f in "${staged[@]}"; do + skip=0 + if [[ -f .styluaignore ]]; then + while IFS= read -r pat; do + [[ -z "$pat" || "$pat" == \#* ]] && continue + pat="${pat%/}" + if [[ "$f" == "$pat" || "$f" == "$pat"/* ]]; then skip=1; break; fi + done < .styluaignore + fi + ((skip)) && continue + git show ":$f" | "${stylua_cmd[@]}" --check --stdin-filepath "$f" - >/dev/null 2>&1 \ + || bad+=("$f") + done + + if ((${#bad[@]})); then + echo "[pre-commit] stylua: staged Lua is not formatted:" >&2 + printf ' %s\n' "${bad[@]}" >&2 + echo "[pre-commit] run 'just bar::fmt', re-stage, and commit again." >&2 + exit 1 + fi + HOOK chmod +x "$hook" ok "Installed pre-commit hook at $hook" if [ -f "$BAR_DIR/.git-blame-ignore-revs" ]; then diff --git a/just/docs.just b/just/docs.just index c52b02a2..24146b98 100644 --- a/just/docs.just +++ b/just/docs.just @@ -13,7 +13,7 @@ generate: require_host {{COMPOSE}} run --rm recoil-docs lua_pages -# Generate everything then start Hugo dev server +# Extract Lua library locally (contexts branch), regen pages, start Hugo dev server. server: #!/usr/bin/env bash set -euo pipefail @@ -21,7 +21,7 @@ server: require_host {{COMPOSE}} run --rm --service-ports recoil-docs server_full -- --bind 0.0.0.0 -# Start Hugo dev server without regenerating +# Start Hugo dev server without regenerating anything server-only: #!/usr/bin/env bash set -euo pipefail diff --git a/just/lua.just b/just/lua.just index e32246e4..cd62493a 100644 --- a/just/lua.just +++ b/just/lua.just @@ -39,14 +39,14 @@ library *flags: build-lde info "Extracting Lua docs..." $LDE \ - --src "$RECOIL_DIR/rts/{Lua,Rml/SolLua}/**/*.cpp" \ + --src "$RECOIL_DIR/rts/{Lua,Rml/SolLua,Sim/Units/Scripts}/**/*.cpp" \ --dest "$DEST" \ --repo "https://github.com/beyond-all-reason/RecoilEngine/blob/master" \ {{flags}} info "Copying into BAR working tree..." - mkdir -p "$BAR_DIR/recoil-lua-library/library" clean_dir "$BAR_DIR/recoil-lua-library/library" + mkdir -p "$BAR_DIR/recoil-lua-library/library" cp -r "$RECOIL_DIR/rts/Lua/library/"* "$BAR_DIR/recoil-lua-library/library/" ok "Updated $BAR_DIR/recoil-lua-library/library/" diff --git a/scripts/codemod-cargo.sh b/scripts/codemod-cargo.sh new file mode 100755 index 00000000..88504fe4 --- /dev/null +++ b/scripts/codemod-cargo.sh @@ -0,0 +1,13 @@ +#!/usr/bin/env bash +# Run cargo for bar-lua-codemod inside DEVTOOLS_DISTROBOX (bar-dev has rust/cargo). +set -euo pipefail + +DEVTOOLS_DIR="${DEVTOOLS_DIR:?DEVTOOLS_DIR must be set}" +CODEMOD_DIR="$DEVTOOLS_DIR/bar-lua-codemod" + +source "$DEVTOOLS_DIR/scripts/common.sh" + +enter_distrobox "$@" + +cd "$CODEMOD_DIR" +cargo "$@" diff --git a/scripts/codemod/generate-branches.sh b/scripts/codemod/generate-branches.sh new file mode 100755 index 00000000..5e89beb8 --- /dev/null +++ b/scripts/codemod/generate-branches.sh @@ -0,0 +1,1689 @@ +#!/usr/bin/env bash +# Deterministically rebuild fmt, leaf (mig-*), and mig branches. +# Called by: just bar::migrate::stylua-cleanup-generate [--push] [--update-prs] +set -euo pipefail + +source "${DEVTOOLS_DIR}/scripts/common.sh" + +# ─── Config ────────────────────────────────────────────────────────────────── + +CODEMOD="${CODEMOD_BIN:-${DEVTOOLS_DIR}/bar-lua-codemod/target/release/bar-lua-codemod}" +BAR="${BAR_DIR:-${DEVTOOLS_DIR}/Beyond-All-Reason}" +# UPSTREAM_REMOTE = the canonical upstream repo (beyond-all-reason); FORK_REMOTE = +# the personal fork. With the standard git convention (origin=fork, +# upstream=canonical) these map to the `upstream` and `origin` remotes +# respectively. Main-chain branches host on the canonical (same-repo PRs); leaf +# branches host on the fork (cross-repo PRs). +UPSTREAM_REMOTE="${UPSTREAM_REMOTE:-upstream}" +FORK_REMOTE="${FORK_REMOTE:-origin}" + +UPSTREAM_REPO="beyond-all-reason/Beyond-All-Reason" +FORK_OWNER="${FORK_OWNER:-$(git -C "$BAR" remote get-url "$FORK_REMOTE" 2>/dev/null | sed -n 's|.*[:/]\([^/]*\)/.*|\1|p')}" +# owner/repo slug of the canonical repo — commit links (museum table) point +# here, since every pipeline branch hosts on the canonical repo. +UPSTREAM_SLUG="${UPSTREAM_SLUG:-$(git -C "$BAR" remote get-url "$UPSTREAM_REMOTE" 2>/dev/null | sed -e 's|\.git$||' -e 's|.*[:/]\([^/]*/[^/]*\)$|\1|')}" + +# Every pipeline branch hosts directly on the canonical repo +# ($UPSTREAM_REMOTE, beyond-all-reason), with same-repo PRs — GitHub's native +# stacked PRs reject fork-headed PRs, and reviewers live upstream. The fork +# ($FORK_REMOTE) only carries mirror pushes. +UPSTREAM_BRANCHES_RE='.' + +# Return the --head value for a gh pr create invocation: bare branch name for +# same-repo PRs, owner:branch for cross-repo PRs from the fork. +head_for() { + local branch="$1" + if [[ "$branch" =~ $UPSTREAM_BRANCHES_RE ]]; then + echo "$branch" + else + echo "$FORK_OWNER:$branch" + fi +} + +# Remote a branch should be pushed to. +remote_for() { + local branch="$1" + if [[ "$branch" =~ $UPSTREAM_BRANCHES_RE ]]; then + echo "$UPSTREAM_REMOTE" + else + echo "$FORK_REMOTE" + fi +} + +# Codemod skip set — mirrors .styluaignore: vendored utils, lux/library tool +# dirs, and mapgenerator (mapinfo_template.lua is a ${}-placeholder template, +# not valid Lua). +CODEMOD_EXCLUDES=(--exclude common/luaUtilities --exclude .lux --exclude recoil-lua-library --exclude mapgenerator) + +MIG_PR="https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8396" + +# Tracking issue that ties all of the type-cleanup PRs together. Each leaf, +# the mig rollup, and the LLM capstone link to it as "Part of " so +# the PR bodies stay focused on their own step. +TRACKING_ISSUE="https://github.com/beyond-all-reason/Beyond-All-Reason/issues/7408" + +# Bulk-migration workflow docs (how to run the migration, the dated migration log). +# Points at the docs PR until it lands on master; then swap to the README anchor +# (…/blob/master/README.md#bulk-migrations). Linked from every PR body. +BULK_MIGRATIONS_DOC="https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8410" + +# Tooling PR — the BAR-Devtools side that ships generate-branches.sh, +# llm-type-triage.sh, the codemod transforms, SKILL.md, and the just recipes. +DEVTOOLS_PR="https://github.com/keithharvey/BAR-Devtools/pull/4" + +# SKILL.md on the BAR-Devtools tooling PR (stays live as the PR updates, +# unlike a commit-pinned fork URL). +SKILL_MD_URL="https://github.com/beyond-all-reason/BAR-Devtools/pull/17/changes#diff-03a30b0197306b790f86b384d585b6d1aff0f23cc38a8545e73c027e3d090ddf" + +# LLM capstone — fmt-llm. A single capstone branch that sits on top of mig and +# combines a deterministic env layer (cherry-picked from a user-maintained source +# branch) with an LLM-generated type-fix commit. +# +# Branch topology after a successful run: +# mig ← rollup of all transforms (existing) +# └─ fmt-llm ← mig + env commits + gen(llm) commit +# +# $LLM_SOURCE_BRANCH: user-maintained env layer (emmylua config, type stubs, +# manual fixes). Its env commits have mig-specific dependencies (e.g. types/* +# files that only exist after detach-bar-modules), so they can't be rooted on +# master. Instead, build_fmt_llm_source force-rebuilds the branch each run: +# resets to current mig and re-cherry-picks the env commits (detected by the +# same anchor walk the old build_fmt_llm used), with -Xtheirs + stylua_pass +# to reconcile formatting divergence. +# +# PR #7447 (base=mig) shows just the env layer. PR #8235 (base=fmt-llm-source) +# shows just the LLM commit, because build_fmt_llm branches $LLM_BRANCH off +# $LLM_SOURCE_BRANCH rather than cherry-picking onto mig. +LLM_SOURCE_BRANCH="${LLM_SOURCE_BRANCH:-fmt-llm-source}" +LLM_SOURCE_PR="https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8397" +LLM_SOURCE_PR_TITLE="[Types] LLM env layer (emmylua config, type stubs, manual fixes)" +LLM_BRANCH="fmt-llm" +LLM_COMMIT_PREFIX="gen(llm): type-error triage" +LLM_PR="https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8398" +LLM_PR_TITLE="[Types] LLM-driven type-error transform capstone" + +# ─── Stacked-PR base overrides ────────────────────────────────────────────── +# Desired merge chain: master <- fmt <- mig <- fmt-llm-source <- fmt-llm +# Branches not listed here default to "master" in update_prs/update_capstone_pr. +declare -A PR_BASE=( + [mig]="fmt" + ["$LLM_SOURCE_BRANCH"]="mig" + ["$LLM_BRANCH"]="$LLM_SOURCE_BRANCH" + # Each leaf = fmt + one transform; basing on fmt scopes the GitHub diff + # to just the transform's changes. + [mig-bracket]="fmt" + [mig-rename-aliases]="fmt" + [mig-detach-bar-modules]="fmt" + [mig-integration-tests]="fmt" + [mig-busted-types]="fmt" +) + +# Dirty check only on the host -- inside distrobox stdin is piped via enter_distrobox. +if [[ -z "${_DEVTOOLS_IN_DISTROBOX:-}" ]] && [[ -n "$(git -C "$BAR" status --porcelain 2>/dev/null)" ]]; then + warn "BAR working tree has uncommitted changes." + warn "They will be discarded by branch checkouts." + echo -n "Continue? [y/N] " + read -r answer + if [[ "$answer" != [yY] ]]; then + err "Aborted" + exit 1 + fi +fi + +enter_distrobox "$@" + +# ─── Transform registry (order matters for the linear mig branch) ──────────── +# Each transform has: _branch, _commit, _pr, _prereq, _description +# Transforms with _prereq cherry-pick that branch before running the codemod. +# Optional: run_*, describe_*, post_commit_*, generate_*_pr_body functions. + +# Branches cherry-picked onto every leaf and mig branch before any transform. +# engine-builders-env (Spring*->Engine* builders, recovered from +# mig-spring-split) waits for spring-split proper; sharing-modules now owns +# the modernized builders Spring-named. +PREFIX_BRANCHES=("fix_stylua") + +TRANSFORMS=("fmt" "bracket_to_dot" "rename_aliases" "detach_bar_modules" "integration_tests" "busted_types") + +# Subset of TRANSFORMS that git blame should skip: mechanical rewrites only. +# The gen(hand) pair is excluded — one is a hand restructure, the other is pure +# addition, where --ignore-revs has nothing earlier to attribute to. +BLAME_TRANSFORMS=("fmt" "bracket_to_dot" "rename_aliases" "detach_bar_modules") + +# -- fmt (stylua) ------------------------------------------------------------- + +fmt_branch="fmt" +fmt_commit="gen(stylua): initial formatting of entire codebase" +fmt_pr="https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8395" +fmt_pr_title="[Style] stylua format entire codebase" +fmt_prereq="" +fmt_description="" +fmt_summary='Runs [stylua](https://github.com/JohnnyMorganz/StyLua) over the entire Lua tree, applying the repo style (indentation, quotes, call parens, line width). Formatting only — no behavioral change.' + +run_fmt() { + stylua_pass +} + +describe_fmt() { + cat <<'EOF' +# fmt - run stylua across the entire Lua codebase +stylua . +EOF +} + +post_commit_fmt() { + # .git-blame-ignore-revs is no longer written here. It's deferred to the + # final fmt-llm rollup (commit_blame_ignore_revs) so the file doesn't + # conflict when the LLM env commits are replayed onto fresh mig builds + # with different transform SHAs. + : +} + +# -- bracket-to-dot ----------------------------------------------------------- + +bracket_to_dot_branch="mig-bracket" +bracket_to_dot_commit="gen(bar_codemod): bracket-to-dot" +bracket_to_dot_pr="https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8401" +bracket_to_dot_prereq="" +bracket_to_dot_description="" +bracket_to_dot_summary='Rewrites identifier-keyed string access to dot notation — `x["y"]` → `x.y` and `["y"] =` → `y =` — so the analyzer can resolve field types through the access.' + +run_bracket_to_dot() { + "$CODEMOD" bracket-to-dot --path "$BAR" "${CODEMOD_EXCLUDES[@]}" +} + +describe_bracket_to_dot() { + cat <<'EOF' +# bracket-to-dot - convert x["y"] to x.y and ["y"] = to y = +bar-lua-codemod bracket-to-dot --path "$BAR_DIR" --exclude common/luaUtilities --exclude .lux --exclude recoil-lua-library --exclude mapgenerator +EOF +} + +# -- rename-aliases ------------------------------------------------------------ + +rename_aliases_branch="mig-rename-aliases" +rename_aliases_commit="gen(bar_codemod): rename-aliases" +rename_aliases_pr="https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8402" +rename_aliases_prereq="" +rename_aliases_description="" +rename_aliases_summary='Renames deprecated Spring method aliases to their canonical names (e.g. `Spring.GetMyTeamID` → `Spring.GetLocalTeamID`) so call sites line up with the names the engine type stubs declare.' + +run_rename_aliases() { + "$CODEMOD" rename-aliases --path "$BAR" "${CODEMOD_EXCLUDES[@]}" +} + +describe_rename_aliases() { + cat <<'EOF' +# rename-aliases -- deprecated aliases, e.g. GetMyTeamID -> GetLocalTeamID +bar-lua-codemod rename-aliases --path "$BAR_DIR" --exclude common/luaUtilities --exclude .lux --exclude recoil-lua-library --exclude mapgenerator +EOF +} + +# -- detach-bar-modules -------------------------------------------------------- + +detach_bar_modules_branch="mig-detach-bar-modules" +detach_bar_modules_commit="gen(bar_codemod): detach-bar-modules" +detach_bar_modules_pr="https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8403" +detach_bar_modules_prereq="detach-bar-modules-env" +detach_bar_modules_summary='Moves BAR-added helpers off the `Spring` table into a `BAR` namespace — `Spring.I18N` → `BAR.I18N`, plus `BAR.Utilities`, `BAR.Debug`, `BAR.Lava`, and `BAR.GetModOptionsCopy` — since they aren'\''t engine API and otherwise break type-checking against the `Spring` stubs.' +detach_bar_modules_description='The `detach-bar-modules-env` prereq exposes `BAR` to the widget/gadget sandbox (`luarules/system.lua`, `luaui/system.lua`), bootstraps `BAR = BAR or {}` in `init.lua`/`springOverrides.lua` before the detached defs, adds the consolidated `types/BAR.lua` stub, lists `BAR` as a global in `.emmyrc.json`, and bootstraps the namespace in the spec harness (that init previously rode engine-builders-env). Cherry-picked on top of `fmt` before the codemod runs.' + +run_detach_bar_modules() { + "$CODEMOD" detach-bar-modules --path "$BAR" "${CODEMOD_EXCLUDES[@]}" +} + +describe_detach_bar_modules() { + cat <<'EOF' +# detach-bar-modules -- moves I18N, Utilities, Debug, Lava, GetModOptionsCopy off the Spring table +bar-lua-codemod detach-bar-modules --path "$BAR_DIR" --exclude common/luaUtilities --exclude .lux --exclude recoil-lua-library --exclude mapgenerator +EOF +} + +# -- integration-tests --------------------------------------------------------- +# Carried-commit leaf (not a codemod). The curated branch contains a single +# hand-authored commit that restructures the integration tests under +# luaui/Tests and luaui/TestsExamples from bare-global hook declarations to +# a return-table shape, plus patches dbg_test_runner.lua to read hooks from +# the returned table. run_*() is a no-op — build_leaf's prereq cherry-pick +# brings the content in, and the trailing git commit is skipped because the +# working tree has no additional changes (see build_leaf guard). + +integration_tests_branch="mig-integration-tests" +integration_tests_commit="gen(hand): integration tests return-table shape" +integration_tests_pr="https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8404" +integration_tests_pr_title="[Tests] Restructure integration tests to table-return shape" +integration_tests_prereq="integration-tests-curated" +integration_tests_summary='Hand-curated (not a codemod): restructures the tests under `luaui/Tests/` and `luaui/TestsExamples/` (20 files) from bare-global hook declarations to a `return { ... }` shape, and patches `dbg_test_runner.lua` to read hooks from the returned table.' +integration_tests_description='Isolated so the convention change can be discussed/reverted independently.' + +run_integration_tests() { + : # carried-commit leaf; prereq cherry-pick is the entire payload +} + +describe_integration_tests() { + cat <<'EOF' +# integration-tests - restructure luaui/Tests and luaui/TestsExamples files +# from bare-global hook declarations to a return-table shape; patch the +# dbg_test_runner widget to read hooks from the returned table. Carried- +# commit leaf — no codemod; curated branch holds the hand-authored commit. +EOF +} + +# -- busted-types -------------------------------------------------------------- +# Carried-commit leaf (not a codemod). The curated branch contains a single +# hand-authored commit that vendors LuaCATS busted + luassert type annotations +# under types/busted and types/luassert with per-directory provenance.md. Same +# no-op pattern as integration_tests. + +busted_types_branch="mig-busted-types" +busted_types_commit="gen(hand): inline luassert and busted LuaCATS types" +busted_types_pr="https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8405" +busted_types_pr_title="[Types] Inline LuaCATS busted+luassert type annotations" +busted_types_prereq="busted-types-curated" +busted_types_summary='Hand-curated (not a codemod): vendors [LuaCATS/busted](https://github.com/LuaCATS/busted) and [LuaCATS/luassert](https://github.com/LuaCATS/luassert) type annotations under `types/busted/` and `types/luassert/`.' +busted_types_description='Waits on [lumen-oss/lux#953](https://github.com/lumen-oss/lux/issues/953) to replace with a Lux dev-dep declaration.' + +run_busted_types() { + : # carried-commit leaf; prereq cherry-pick is the entire payload +} + +describe_busted_types() { + cat <<'EOF' +# busted-types - vendor LuaCATS busted + luassert type annotations under +# types/busted and types/luassert with per-directory provenance.md. Carried- +# commit leaf — no codemod; curated branch holds the hand-authored commit. +EOF +} + +# ─── Helpers ───────────────────────────────────────────────────────────────── + +tvar() { eval echo "\${${1}_${2}:-}"; } + +declare -A TEST_RESULTS + +# Path inside BAR's .git/ where the most recent run's test results are +# cached. --skip-generation reads this so PR-body topology tables can show +# the last known unit-test status instead of "n/a" everywhere. +TEST_RESULTS_CACHE="$BAR/.git/test-results.cache" + +persist_test_results() { + : > "$TEST_RESULTS_CACHE" + for branch in "${!TEST_RESULTS[@]}"; do + printf '%s\t%s\n' "$branch" "${TEST_RESULTS[$branch]}" >> "$TEST_RESULTS_CACHE" + done +} + +load_test_results() { + if [[ ! -f "$TEST_RESULTS_CACHE" ]]; then + return 1 + fi + local branch status + while IFS=$'\t' read -r branch status; do + [[ -n "$branch" ]] && TEST_RESULTS["$branch"]="$status" + done < "$TEST_RESULTS_CACHE" +} + +host_exec() { + if [ -f /run/.containerenv ] && command -v distrobox-host-exec &>/dev/null; then + distrobox-host-exec "$@" + else + "$@" + fi +} + +git_bar() { host_exec git -C "$BAR" "$@"; } + +# gh lives in non-standard host paths (linuxbrew, ~/.local/bin) that +# distrobox-host-exec strips from PATH — resolve an absolute path like claude. +GH_HOST_BIN="" +resolve_host_gh() { + if [[ -n "${GH_BIN_OVERRIDE:-}" ]]; then echo "$GH_BIN_OVERRIDE"; return 0; fi + local c + for c in "$HOME/.local/bin/gh" /home/linuxbrew/.linuxbrew/bin/gh \ + /usr/local/bin/gh /usr/bin/gh "$HOME/.npm-global/bin/gh"; do + host_exec test -x "$c" && { echo "$c"; return 0; } + done + host_exec which gh 2>/dev/null || true +} + +gh_host() { + [[ -z "$GH_HOST_BIN" ]] && GH_HOST_BIN="$(resolve_host_gh)" + if [[ -z "$GH_HOST_BIN" ]]; then + err "gh CLI not found on host (checked ~/.local/bin, linuxbrew, /usr/bin)." + err " override: GH_BIN_OVERRIDE=/abs/path/to/gh" + exit 1 + fi + host_exec "$GH_HOST_BIN" "$@" +} + +STYLUA_HOST_BIN="" +resolve_host_stylua() { + if [[ -n "${STYLUA_BIN_OVERRIDE:-}" ]]; then echo "$STYLUA_BIN_OVERRIDE"; return 0; fi + local c + for c in "$HOME/.local/bin/stylua" "$HOME/.cargo/bin/stylua" \ + /home/linuxbrew/.linuxbrew/bin/stylua /usr/local/bin/stylua /usr/bin/stylua; do + host_exec test -x "$c" && { echo "$c"; return 0; } + done + echo "" +} + +# git runs on the host (git_bar -> host_exec), so stylua must too. Running it +# in the container races the host's writes: a commit that creates files leaves +# the container seeing the dirents but failing to read them +# ("failed to read ./types/Debug.lua: No such file or directory"). +stylua_pass() { + step "Running stylua..." + [[ -z "$STYLUA_HOST_BIN" ]] && STYLUA_HOST_BIN="$(resolve_host_stylua)" + if [[ -n "$STYLUA_HOST_BIN" ]]; then + host_exec sh -c "cd '$BAR' && '$STYLUA_HOST_BIN' ." + else + warn "stylua not found on host; running in-container (may race host writes)" + (cd "$BAR" && stylua .) + fi +} + +# Warm the shared lux cache once — a cold `lx test` triggers a networked +# `lx sync` with no timeout that can hang the whole pipeline. +warm_lux_cache() { + step "Warming lux dependency cache..." + if (cd "$BAR" && timeout 600 lx --lua-version 5.1 sync /dev/null 2>&1; then + echo "(base $1 not present locally)" + return + fi + if ! git_bar rev-parse --verify "$2" >/dev/null 2>&1; then + echo "(branch not built locally)" + return + fi + local raw + raw=$(git_bar diff --shortstat "$1..$2" 2>/dev/null || true) + if [[ -z "$raw" ]]; then + echo "no changes" + return + fi + local files ins del + files=$(echo "$raw" | grep -oP '\d+(?= file)' || echo "0") + ins=$(echo "$raw" | grep -oP '\d+(?= insertion)' || echo "0") + del=$(echo "$raw" | grep -oP '\d+(?= deletion)' || echo "0") + echo "${files} files, +${ins} −${del}" +} + +# Run emmylua_check and return the error count from its summary line. +# Returns "0" if the analyzer reports no errors (or if the analyzer can't be reached). +emmylua_error_count() { + local out count + out=$(cd "$BAR" && emmylua_check -c .emmyrc.json . 2>&1 || true) + count=$(echo "$out" | grep -oP '^\s*\K\d+(?= errors?$)' | head -1) + echo "${count:-0}" +} + +# ─── PR body generators ───────────────────────────────────────────────────── + +pr_link() { + local label="$1" url="$2" + if [[ -n "$url" ]]; then + echo "[$label]($url)" + else + echo "$label" + fi +} + +# Topology-table cell for a branch; bolds + flags the row for the PR being +# rendered so a reviewer instantly sees which one they're on. +branch_cell() { + local label="$1" url="$2" current="$3" + local cell; cell="$(pr_link "$label" "$url")" + if [[ "$label" == "$current" ]]; then + echo "👉 **$cell** — you are here" + else + echo "$cell" + fi +} + +# Merge-safety banner prepended to every stacked PR body. A reviewer merging an +# intermediate PR breaks the stack topology (happened once — a premature merge +# into fmt-llm-source). The whole cleanup lands by merging only the tip, fmt-llm. +pr_merge_warning() { + local current="${1:-}" + echo "> [!WARNING]" + if [[ "$current" == "$LLM_BRANCH" ]]; then + echo "> **This is the stack tip.** Merging this PR lands the entire type-error" + echo "> cleanup — every branch below it. Do **not** merge the intermediate PRs on" + echo "> their own; that breaks the stack topology." + else + echo "> **Don't merge this PR by itself** — it's one slice of a stacked review." + echo "> Merging an intermediate PR breaks the stack. The whole cleanup lands by" + echo "> merging **only the tip, [\`fmt-llm\`]($LLM_PR)**, which pulls in every branch below it." + fi + echo "" +} + +unit_status() { + local branch="$1" + local status="${TEST_RESULTS["$branch"]:-n/a}" + if [[ "$status" == "pass" ]]; then + echo "✅ pass" + elif [[ "$status" == "fail" ]]; then + echo "❌ FAIL" + else + echo "n/a" + fi +} + +# ─── Museum table (linear commit walk for rollup PR bodies) ────────────────── +# +# Rollup branches (`mig`, `fmt-llm`) carry many commits each from different +# layers. The museum table renders one row per commit, in order, with a +# clickable hash linking to the commit on the canonical repo. Reviewers +# walk the stack like exhibits — descriptions are intentionally one-line so +# the table is scannable; the umbrella issue carries the rationale. + +museum_description() { + local subject="$1" + case "$subject" in + "gen(stylua):"*) + echo "stylua across the entire codebase" ;; + "gen(bar_codemod): bracket-to-dot") + echo 'x["y"] → x.y, ["y"]= → y= via full_moon AST rewrite' ;; + "gen(bar_codemod): rename-aliases") + echo "deprecated Spring API aliases (GetMyTeamID → GetLocalTeamID, etc.)" ;; + "gen(bar_codemod): detach-bar-modules") + echo "Spring.{I18N,Utilities,Debug,Lava,GetModOptionsCopy} → BAR.{…} namespace" ;; + "gen(hand): integration tests return-table shape") + echo "luaui/Tests + luaui/TestsExamples → return-table shape; dbg_test_runner reads hooks from the returned table" ;; + "gen(hand): inline luassert and busted LuaCATS types") + echo "vendored LuaCATS/busted + LuaCATS/luassert type annotations under types/ (pending lumen-oss/lux#953)" ;; + "git-blame-ignore-revs:"*) + echo "register transform commits with git blame" ;; + "env(llm):"*) + echo ".emmyrc.json globals, types/* stubs, busted mock, CI gate, manual fixes" ;; + "gen(llm):"*) + echo "parallel LLM workers applying SKILL.md fix recipes per file chunk" ;; + "env:"*) + # Self-describing prereq commits — strip the prefix. + echo "${subject#env: }" ;; + "deps:"*) + echo "${subject#deps: }" ;; + *) + echo "—" ;; + esac +} + +# Render a markdown table of every commit unique to vs origin/master, +# in the order they were applied. Hashes link through the PR (pr_url/commits/) +# so reviewers land on the commit in review context; these are only valid for +# the pushed generation — hence --push implies --update-prs. +generate_museum_table() { + local branch="$1" + local pr_url="$2" + + echo "### Commits" + echo "" + echo "| # | Commit | What it does |" + echo "|---|--------|--------------|" + + local i=1 + while IFS=$'\t' read -r short long subject; do + local desc commit_url + desc="$(museum_description "$subject")" + if [[ -n "$pr_url" ]]; then + commit_url="${pr_url}/commits/${long}" + else + commit_url="https://github.com/${UPSTREAM_SLUG}/commit/${long}" + fi + echo "| $i | [\`${short}\`](${commit_url}) \`${subject}\` | ${desc} |" + i=$((i + 1)) + done < <(git_bar log --reverse --format='%h %H %s' "origin/master..$branch") +} + +generate_topology() { + local current="${1:-}" + echo "### Branch Topology" + echo "" + echo "All branches in the [BAR type-error cleanup]($TRACKING_ISSUE) stack — see [Bulk Migrations]($BULK_MIGRATIONS_DOC) for the migration log and how to run \`just bar::migrate::stylua-cleanup\`. Regenerated deterministically by [\`just bar::migrate::stylua-cleanup-generate\`]($DEVTOOLS_PR). *Generated $(date -u +"%Y-%m-%d %H:%M:%S UTC").*" + echo "" + echo "**Leaves** — each isolates one transform's diff vs \`fmt\`:" + echo "" + echo "| Branch | Command | Diff vs parent | Units |" + echo "|--------|---------|------|-------|" + for transform in "${TRANSFORMS[@]}"; do + local branch pr_url stats command base + branch=$(tvar "$transform" "branch") + pr_url=$(tvar "$transform" "pr") + # Diff against the PR base (the branch this leaf stacks on), not master: + # for transform leaves that's `fmt`, isolating the transform's own + # changes from the stylua reformat baseline. `fmt` itself bases on master. + base="${PR_BASE[$branch]:-origin/master}" + stats=$(diff_stat "$base" "$branch") + # Most transforms invoke `bar-lua-codemod `. Exceptions are + # hand-maintained: + # - fmt: runs stylua, not the codemod + # - carried-commit leaves (integration_tests, busted_types): content + # comes from a curated prereq branch; no codemod or automation + case "$transform" in + fmt) + command="\`stylua\`" ;; + integration_tests|busted_types) + command="\`\`" ;; + *) + command="\`bar-lua-codemod ${transform//_/-}\`" ;; + esac + echo "| $(branch_cell "$branch" "$pr_url" "$current") | $command | $stats | $(unit_status "$branch") |" + done + echo "" + echo "**Rollups** — composite branches stacking the leaves and (for \`fmt-llm\`) the env + LLM layers:" + echo "" + echo "| Branch | Diff vs \`master\` | Diff vs parent | Units |" + echo "|--------|------|------|-------|" + echo "| $(branch_cell "mig" "$MIG_PR" "$current") | $(diff_stat origin/master mig) | $(diff_stat fmt mig) | $(unit_status mig) |" + echo "| $(branch_cell "$LLM_SOURCE_BRANCH" "$LLM_SOURCE_PR" "$current") | $(diff_stat origin/master "$LLM_SOURCE_BRANCH") | $(diff_stat mig "$LLM_SOURCE_BRANCH") | $(unit_status "$LLM_SOURCE_BRANCH") |" + echo "| $(branch_cell "$LLM_BRANCH" "$LLM_PR" "$current") | $(diff_stat origin/master "$LLM_BRANCH") | $(diff_stat "$LLM_SOURCE_BRANCH" "$LLM_BRANCH") | $(unit_status "$LLM_BRANCH") |" +} + +generate_leaf_pr_body() { + local transform="$1" output_file="$2" + local description summary branch + description=$(tvar "$transform" "description") + summary=$(tvar "$transform" "summary") + branch=$(tvar "$transform" "branch") + + pr_merge_warning "$branch" + echo "Part of [BAR type-error cleanup]($TRACKING_ISSUE). Rebuilds idempotently from \`master\` via [\`just bar::migrate::stylua-cleanup-generate\`]($DEVTOOLS_PR)." + echo "" + if [[ -n "$summary" ]]; then + echo "**What it does:** $summary" + echo "" + fi + echo '```sh' + "describe_${transform}" + echo '```' + if [[ -n "$description" ]]; then + echo "" + echo "$description" + fi + echo "" + generate_topology "$branch" +} + +generate_mig_pr_body() { + local _output_file="$1" # unused; output bundle was previously inlined here + + pr_merge_warning "mig" + echo "Part of [BAR type-error cleanup]($TRACKING_ISSUE). Combined deterministic transforms — what \`master\` looks like with every leaf applied sequentially." + echo "" + generate_topology "mig" +} + +# ─── Build phase (no PR body generation — branches must all exist first) ───── + +# Memoization for ensure_fmt_prereq — build each $prefix-fmt at most once per run. +declare -A _FMT_PREREQ_BUILT=() + +# Rebuild $prefix's unique commits on top of fmt as an ephemeral `$prefix-fmt` +# branch, then re-run stylua so the result is internally consistent with fmt's +# formatting. Used by cherry_pick_prefix when the target tree is fmt-rooted — +# without this, the prereq's master-era whitespace conflicts with stylua's +# reformatting and the cherry-pick explodes. +ensure_fmt_prereq() { + local prefix="$1" + local fmt_ref="${prefix}-fmt" + + if [[ -n "${_FMT_PREREQ_BUILT[$prefix]:-}" ]]; then + return 0 + fi + _FMT_PREREQ_BUILT[$prefix]=1 + + local count + count=$(git_bar rev-list --count "origin/master..$prefix" 2>/dev/null || echo 0) + if [[ "$count" == "0" ]]; then + return 0 + fi + + step "Building fmt-rooted prereq: $fmt_ref" + local saved_branch saved_sha + saved_sha=$(git_bar rev-parse HEAD) + saved_branch=$(git_bar symbolic-ref --short HEAD 2>/dev/null || echo "") + + git_bar checkout --force -B "$fmt_ref" fmt + # -Xtheirs: on conflicts between fmt's stylua reformatting and the prereq's + # content edits, keep the prereq's content. stylua_pass below then + # re-normalizes formatting so the final tree is fmt-consistent. This relies + # on the prereq branches being "content-only" vs master — if a prereq ever + # encodes a formatting intent that disagrees with stylua, we'd silently + # lose it. That's acceptable for the curated prereqs we have today + # (integration-tests-curated, busted-types-curated, + # detach-bar-modules-env). + # --empty=drop: the recoil-lua-library submodule-bump commit lands empty + # when the submodule is already at the bumped gitlink; drop it, don't halt. + git_bar cherry-pick --empty=drop -Xtheirs "origin/master..$prefix" + stylua_pass + git_bar add -A + if ! git_bar diff --cached --quiet; then + git_bar commit --amend --no-edit + fi + + if [[ -n "$saved_branch" ]]; then + git_bar checkout --force "$saved_branch" + else + git_bar checkout --force "$saved_sha" + fi +} + +# Cherry-pick origin/master.., but skip silently if the branch has no +# commits unique vs origin/master (e.g. a prefix branch that has been merged +# upstream and rebased to empty). Without this guard `git cherry-pick` aborts +# the entire pipeline with "empty commit set passed". +# +# When the current tree is fmt-rooted, routes through an ephemeral fmt-rooted +# replay of the prereq (see ensure_fmt_prereq) so stylua-vs-master whitespace +# doesn't conflict. +cherry_pick_prefix() { + local prefix="$1" + local count + count=$(git_bar rev-list --count "origin/master..$prefix" 2>/dev/null || echo 0) + if [[ "$count" == "0" ]]; then + info " (skip) $prefix has no commits unique vs origin/master" + return 0 + fi + if git_bar merge-base --is-ancestor fmt HEAD 2>/dev/null; then + ensure_fmt_prereq "$prefix" + git_bar cherry-pick --empty=drop "fmt..${prefix}-fmt" + else + git_bar cherry-pick --empty=drop "origin/master..$prefix" + fi +} + +build_leaf() { + local transform="$1" + local branch commit_msg prereq + branch=$(tvar "$transform" "branch") + commit_msg=$(tvar "$transform" "commit") + prereq=$(tvar "$transform" "prereq") + + step "Building leaf: $branch" + abort_stuck_git_state + # Non-fmt leaves root on fmt so they literally contain fmt's commit SHA. + # This keeps GitHub's `fmt...$branch` PR diff scoped to the transform + # only (without this, identical-content-but-different-SHA stylua commits + # on each leaf blow the diff up to ~200k lines). + if [[ "$transform" == "fmt" ]]; then + git_bar checkout --force -B "$branch" origin/master + for prefix in "${PREFIX_BRANCHES[@]}"; do + cherry_pick_prefix "$prefix" + done + else + git_bar checkout --force -B "$branch" fmt + fi + + if [[ -n "$prereq" ]]; then + step "Cherry-picking prereq commits from $prereq..." + cherry_pick_prefix "$prereq" + fi + + local output_file="$BAR/.git/${branch}-output.txt" + "run_${transform}" 2>&1 | tee "$output_file" + + git_bar add -A + # Carried-commit leaves (e.g. integration_tests, busted_types) have a + # no-op run_*() and rely entirely on the prereq cherry-pick for content. + # Skip the trailing commit in that case — the prereq commit's message + # stands as the leaf tip and "nothing to commit" would abort the script. + if git_bar diff --cached --quiet; then + info " (skip) run_${transform} produced no changes — prereq commit stands as $branch tip" + else + git_bar commit -m "$commit_msg" + fi + + if type "post_commit_${transform}" &>/dev/null; then + "post_commit_${transform}" + fi + + run_tests "$branch" + + ok "Leaf $branch ready" +} + +build_mig() { + step "Building linear mig branch..." + abort_stuck_git_state + # Root on fmt so mig literally contains fmt's commit SHA as an ancestor + # (see build_leaf for the same reasoning — keeps the `fmt...mig` PR diff + # scoped to the transforms, not the stylua baseline). + git_bar checkout --force -B mig fmt + + local -A mig_prereqs_picked + for transform in "${TRANSFORMS[@]}"; do + local prereq + prereq=$(tvar "$transform" "prereq") + if [[ -n "$prereq" ]] && [[ -z "${mig_prereqs_picked["$prereq"]:-}" ]]; then + step "Cherry-picking prereq commits from $prereq..." + cherry_pick_prefix "$prereq" + mig_prereqs_picked["$prereq"]=1 + fi + done + + local mig_output_file="$BAR/.git/mig-output.txt" + : > "$mig_output_file" + for transform in "${TRANSFORMS[@]}"; do + local commit_msg + commit_msg=$(tvar "$transform" "commit") + step "mig: $transform" + "run_${transform}" 2>&1 | tee -a "$mig_output_file" + stylua_pass + git_bar add -A + # Carried-commit leaves contributed their content via the prereq + # cherry-pick earlier (and stylua left it alone). Skip an empty + # commit here — otherwise git aborts build_mig on "nothing to commit". + if git_bar diff --cached --quiet; then + info " (skip) mig: $transform produced no changes — prereq commit already in mig" + else + git_bar commit -m "$commit_msg" + fi + done + + # Cache transform hashes for the final blame-ignore commit on fmt-llm. + # Committing the file here would conflict when fmt-llm-source env commits + # are replayed onto new mig builds with different transform SHAs. + # + # Read back off the branch by subject rather than recorded in the loop + # above: a transform whose content arrived via a prereq cherry-pick commits + # nothing there, and the cherry-pick that does carry it has its own SHA. + local blame_cache="$BAR/.git/mig-blame-hashes.txt" + : > "$blame_cache" + local blame_subjects=() + for transform in "${BLAME_TRANSFORMS[@]}"; do + blame_subjects+=("$(tvar "$transform" "commit")") + done + while IFS=$'\t' read -r sha subject; do + for want in "${blame_subjects[@]}"; do + [[ "$subject" == "$want" ]] || continue + printf '%s\t%s\n' "$sha" "$subject" >> "$blame_cache" + break + done + done < <(git_bar log --reverse --format='%H%x09%s' "origin/master..HEAD") + + local found + found=$(wc -l < "$blame_cache") + if (( found != ${#BLAME_TRANSFORMS[@]} )); then + warn "blame-ignore: matched $found of ${#BLAME_TRANSFORMS[@]} transform commits on mig" + fi + + run_tests "mig" + + ok "mig branch ready (${#TRANSFORMS[@]} commits)" +} + +# ─── LLM capstone (runs after build_mig) ───────────────────────────────────── + +# Recover from a failed prior run that left the BAR repo in a partial git +# operation (cherry-pick, rebase, or merge in progress). Called defensively +# before any git state change in build_fmt_llm. +abort_stuck_git_state() { + local bar_git="$BAR/.git" + if [[ -f "$bar_git/CHERRY_PICK_HEAD" ]] || [[ -d "$bar_git/sequencer" ]]; then + warn "Found in-progress cherry-pick in BAR repo — aborting" + git_bar cherry-pick --abort 2>/dev/null || true + rm -rf "$bar_git/sequencer" 2>/dev/null || true + fi + if [[ -d "$bar_git/rebase-merge" ]] || [[ -d "$bar_git/rebase-apply" ]]; then + warn "Found in-progress rebase in BAR repo — aborting" + git_bar rebase --abort 2>/dev/null || true + fi + if [[ -f "$bar_git/MERGE_HEAD" ]]; then + warn "Found in-progress merge in BAR repo — aborting" + git_bar merge --abort 2>/dev/null || true + fi + # Blow away any leftover untracked files that were dropped by a mid- + # cherry-pick abort — they will be regenerated by the normal flow. + git_bar --no-optional-locks -c submodule.recurse=false reset --hard HEAD 2>/dev/null || true +} + +# Build fmt-llm by: +# 1. Auto-detecting the env anchor on $LLM_SOURCE_BRANCH (walk from tip down, +# first commit whose subject does NOT start with `env(llm):` is the anchor; +# everything above it is env content, replayed onto current mig). +# 2. Cherry-picking only those env commits onto a fresh fmt-llm off mig. +# 3. Running the LLM type-triage fan-out (drives just bar::check errors → 0). +# 4. Committing whatever the workers edited as one gen(llm) commit. +# +# The source branch is user-maintained, like detach-bar-modules-env. +# Bootstrap: just author env(llm): commits on top of some existing mig-ish +# base. The only shape requirement is that env commits live contiguously at +# the tip of $LLM_SOURCE_BRANCH with subjects prefixed `env(llm):`. +# +# Subsequent env edits: check out $LLM_SOURCE_BRANCH, amend or add more +# env(llm): commits on top. Mig drift below the anchor is automatically +# ignored — the replay range is computed from subjects each run, not stored. +build_fmt_llm_source() { + abort_stuck_git_state + + if ! git_bar rev-parse --verify "$LLM_SOURCE_BRANCH" >/dev/null 2>&1; then + err "LLM source branch '$LLM_SOURCE_BRANCH' not found in BAR repo." + err "" + err "Bootstrap it by authoring env commits on top of mig:" + err " git -C $BAR checkout -b $LLM_SOURCE_BRANCH mig" + err " # ...edit .emmyrc.json, types/*, etc..." + err " git -C $BAR commit -am 'env(llm): emmylua config + type stubs'" + err "" + err "Env commits must have subjects prefixed 'env(llm):' (or docs:/deps:)" + err "and live contiguously at the branch tip. The replay anchor is" + err "auto-detected as the first 'gen('/'git-blame-ignore-revs:'/'deps:'/" + err "'env: ' commit walking down from the tip." + exit 1 + fi + + # Walk $LLM_SOURCE_BRANCH from tip downward; stop at the first commit that + # looks like it came from the mig pipeline. Everything above that is + # user-authored env content to replay onto fresh mig. + local env_anchor="" + while IFS=$'\t' read -r sha subject; do + case "$subject" in + "gen("*|"git-blame-ignore-revs:"*|"deps:"*|"env: "*) env_anchor="$sha"; break ;; + *) continue ;; + esac + done < <(git_bar log --format='%H%x09%s' "$LLM_SOURCE_BRANCH") + + if [[ -z "$env_anchor" ]]; then + err "Could not detect an env anchor on $LLM_SOURCE_BRANCH — no commit" + err "with a 'gen(…)'/'git-blame-ignore-revs:'/'deps:'/'env: ' subject" + err "was found walking from the tip. Is $LLM_SOURCE_BRANCH rooted on a" + err "mig variant?" + exit 1 + fi + + # Snapshot env commit SHAs BEFORE we reset the branch pointer — once we + # force-recreate $LLM_SOURCE_BRANCH at mig, `$env_anchor..$LLM_SOURCE_BRANCH` + # would re-resolve to an empty range. + local -a env_commits=() + while IFS= read -r sha; do env_commits+=("$sha"); done \ + < <(git_bar rev-list --reverse "$env_anchor..$LLM_SOURCE_BRANCH") + + step "Rebuilding $LLM_SOURCE_BRANCH: mig + ${#env_commits[@]} env commit(s)" + step " env anchor: $env_anchor ($(git_bar log -1 --format=%s "$env_anchor"))" + + git_bar checkout --force -B "$LLM_SOURCE_BRANCH" mig + + if [[ ${#env_commits[@]} -eq 0 ]]; then + warn "$LLM_SOURCE_BRANCH has no env commits above the anchor — env layer is empty" + else + # -Xtheirs: on conflicts between mig's transforms and env commit edits, + # keep the env commit's content (stylua_pass below renormalizes). + if ! git_bar cherry-pick --empty=drop -Xtheirs "${env_commits[@]}"; then + local conflicted + conflicted=$(git_bar diff --name-only --diff-filter=U | sed 's/^/ /') + err "" + err "Cherry-pick conflict rebuilding $LLM_SOURCE_BRANCH on current mig." + err "Unresolved files:" + err "" + err "$conflicted" + err "" + err "Likely a transform reorganized files in a way the env commit" + err "wasn't written against. Resolve on $LLM_SOURCE_BRANCH by hand," + err "then re-run with --llm-only." + exit 1 + fi + # Re-normalize formatting: -Xtheirs can leave the env commit's whitespace + # in place where it diverged from mig's stylua output. Amend into the + # tip commit so the branch is stylua-clean at least at its head. + stylua_pass + git_bar add -A + if ! git_bar diff --cached --quiet; then + git_bar commit --amend --no-edit + fi + fi + + run_tests "$LLM_SOURCE_BRANCH" + ok "$LLM_SOURCE_BRANCH ready" +} + +build_fmt_llm() { + abort_stuck_git_state + + if ! git_bar rev-parse --verify "$LLM_SOURCE_BRANCH" >/dev/null 2>&1; then + err "$LLM_SOURCE_BRANCH not built yet — run build_fmt_llm_source first" + exit 1 + fi + + step "Building $LLM_BRANCH: $LLM_SOURCE_BRANCH + LLM commit..." + # Branch from $LLM_SOURCE_BRANCH (not mig) so $LLM_BRANCH physically contains + # its commits — PR #8235's diff (base=$LLM_SOURCE_BRANCH) scopes to the LLM + # commit alone. + git_bar checkout --force -B "$LLM_BRANCH" "$LLM_SOURCE_BRANCH" + + # ── LLM layer: run the triage fan-out and commit its edits ── + local before + before=$(emmylua_error_count) + step "$LLM_BRANCH pre-triage errors: $before" + + if [[ "$before" == "0" ]]; then + ok "$LLM_BRANCH already at zero errors — skipping triage" + else + # Run inside the same container we already entered. NO host_exec — + # routing through distrobox-host-exec would drop env vars + # (DEVTOOLS_DISTROBOX, _DEVTOOLS_IN_DISTROBOX) and PATH adjustments. + bash "${DEVTOOLS_DIR}/scripts/codemod/llm-type-triage.sh" + fi + + # LLM workers don't run stylua — reformat whatever they touched so the + # gen(llm) commit is stylua-clean and doesn't introduce formatting drift. + stylua_pass + + local after + after=$(emmylua_error_count) + step "$LLM_BRANCH post-triage errors: $after" + + git_bar add -A + if git_bar diff --cached --quiet; then + warn "LLM triage produced no edits — skipping LLM commit" + else + git_bar commit -m "$LLM_COMMIT_PREFIX ($before → $after errors) + +Generated by parallel claude-sonnet-4-6 workers dispatched by +scripts/codemod/llm-type-triage.sh, applying fixes per SKILL.md categories. +Single pass, no iteration — categories that don't shrink the count +are a signal that SKILL.md needs a new rule." + fi + + commit_blame_ignore_revs + + run_tests "$LLM_BRANCH" + + ok "$LLM_BRANCH ready" +} + +# Writes .git-blame-ignore-revs with every transform commit SHA (cached by +# build_mig) and commits it on the current branch. Deferred to fmt-llm so the +# file never conflicts during env-commit cherry-picks onto fresh mig builds. +commit_blame_ignore_revs() { + local blame_cache="$BAR/.git/mig-blame-hashes.txt" + if [[ ! -f "$blame_cache" ]]; then + warn "No cached transform hashes at $blame_cache — skipping blame-ignore-revs" + return + fi + + step "Committing .git-blame-ignore-revs on $LLM_BRANCH..." + : > "$BAR/.git-blame-ignore-revs" + while IFS=$'\t' read -r sha msg; do + [[ -z "$sha" ]] && continue + printf '# %s\n%s\n\n' "$msg" "$sha" >> "$BAR/.git-blame-ignore-revs" + done < "$blame_cache" + git_bar add .git-blame-ignore-revs + git_bar commit -m "git-blame-ignore-revs: add transform commits" +} + +# ─── PR body + update phase (runs after all branches exist) ────────────────── + +generate_all_pr_bodies() { + step "Generating PR bodies (with diff stats)..." + + # Leaf PR bodies only emitted in full pipeline (--llm-only skips leaves). + if [[ "${DO_LLM_ONLY:-false}" != "true" ]]; then + for transform in "${TRANSFORMS[@]}"; do + local branch output_file pr_body_file + branch=$(tvar "$transform" "branch") + output_file="$BAR/.git/${branch}-output.txt" + pr_body_file="$BAR/.git/${branch}-pr-body.md" + if type "generate_${transform}_pr_body" &>/dev/null; then + "generate_${transform}_pr_body" > "$pr_body_file" + else + generate_leaf_pr_body "$transform" "$output_file" > "$pr_body_file" + fi + ok " $branch PR body: $pr_body_file" + done + + local mig_output_file="$BAR/.git/mig-output.txt" + pr_body_file="$BAR/.git/mig-pr-body.md" + generate_mig_pr_body "$mig_output_file" > "$pr_body_file" + ok " mig PR body: $pr_body_file" + fi + + # fmt-llm-source env layer PR body. + if git_bar rev-parse --verify "$LLM_SOURCE_BRANCH" >/dev/null 2>&1; then + pr_body_file="$BAR/.git/${LLM_SOURCE_BRANCH}-pr-body.md" + generate_llm_source_pr_body > "$pr_body_file" + ok " $LLM_SOURCE_BRANCH PR body: $pr_body_file" + fi + + # LLM capstone PR body (only generated when fmt-llm exists). + if git_bar rev-parse --verify "$LLM_BRANCH" >/dev/null 2>&1; then + pr_body_file="$BAR/.git/${LLM_BRANCH}-pr-body.md" + generate_llm_pr_body > "$pr_body_file" + ok " $LLM_BRANCH PR body: $pr_body_file" + + # Tracking issue body (depends on fmt-llm for the museum table). + local tracking_body_file="$BAR/.git/tracking-issue-body.md" + generate_tracking_issue_body > "$tracking_body_file" + ok " tracking issue body: $tracking_body_file" + fi +} + +generate_llm_source_pr_body() { + pr_merge_warning "$LLM_SOURCE_BRANCH" + echo "Part of [BAR type-error cleanup]($TRACKING_ISSUE). Human-curated env layer that prepares the codebase for the LLM type-fix pass." + echo "" + echo "This branch carries:" + echo "- \`.emmyrc.json\` globals and analyzer config" + echo "- \`types/*\` stubs for vendored/generated declarations" + echo "- Explicit type ignores for known dead code" + echo "- CI gate configuration" + echo "- Manual source fixes that require human judgement" + echo "" + echo "The fix recipes the subsequent LLM pass ($LLM_PR) uses are catalogued in [\`SKILL.md\`]($SKILL_MD_URL) — same rulebook that guides the subagents." + echo "" + generate_topology "$LLM_SOURCE_BRANCH" +} + +generate_llm_pr_body() { + local summary_file + summary_file="$BAR/.git/llm-triage-summary.txt" + + pr_merge_warning "$LLM_BRANCH" + echo "Part of [BAR type-error cleanup]($TRACKING_ISSUE). Final stage: deterministic transforms + env layer + LLM type-fix pass." + echo "" + echo "LLM workers apply the categorized fix recipes in [\`SKILL.md\`]($SKILL_MD_URL) — each category maps an \`emmylua_check\` error pattern to an idempotent recipe." + echo "" + if [[ -f "$summary_file" ]]; then + echo '### Triage run' + echo '' + echo '```' + cat "$summary_file" + echo '```' + echo '' + fi + generate_topology "$LLM_BRANCH" +} + +# Simpler variant of generate_topology for the tracking issue — drops the +# diff-stat + unit-status columns and the timestamp/regen-link preamble. Uses +# current *_pr URLs so the template picks up newly-created PRs on re-run. +generate_issue_branch_topology() { + local leaf_count=${#TRANSFORMS[@]} + echo "
" + echo "Branch topology (${leaf_count} leaves + 3 rollups)" + echo "" + echo "### Leaves — each targets \`master\`, mergeable independently" + echo "" + echo "| Branch | Command | What it does |" + echo "|--------|---------|--------------|" + for transform in "${TRANSFORMS[@]}"; do + local branch pr_url command desc + branch=$(tvar "$transform" "branch") + pr_url=$(tvar "$transform" "pr") + case "$transform" in + fmt) + command="\`stylua\`" ;; + integration_tests|busted_types) + command="\`\`" ;; + *) + command="\`bar-lua-codemod ${transform//_/-}\`" ;; + esac + desc=$(museum_description "$(tvar "$transform" "commit")") + echo "| $(pr_link "$branch" "$pr_url") | $command | $desc |" + done + echo "" + echo "### Rollups — composite branches stacking the leaves and (for \`fmt-llm\`) the env + LLM layers" + echo "" + echo "| Branch | Notes |" + echo "|--------|-------|" + echo "| $(pr_link "mig" "$MIG_PR") | all leaves combined; deterministic rebuild from \`master\` |" + echo "| $(pr_link "$LLM_SOURCE_BRANCH" "$LLM_SOURCE_PR") | human-curated env layer (\`.emmyrc.json\`, \`types/*\` stubs, explicit type ignores, CI gate, manual fixes) |" + echo "| $(pr_link "$LLM_BRANCH" "$LLM_PR") | \`$LLM_SOURCE_BRANCH\` + one LLM triage commit |" + echo "" + echo "Regenerated deterministically by [\`just bar::migrate::stylua-cleanup-generate\`]($DEVTOOLS_PR)." + echo "" + echo "
" +} + +generate_tracking_issue_body() { + local template="${DEVTOOLS_DIR}/scripts/codemod/tracking-issue-template.md" + if [[ ! -f "$template" ]]; then + warn "Tracking issue template not found: $template" + return 1 + fi + + local museum_table commit_count + museum_table=$(generate_museum_table "$LLM_BRANCH" "$LLM_PR") + commit_count=$(git_bar rev-list --count "origin/master..$LLM_BRANCH") + + local museum_replacement + museum_replacement=$(cat < +Commit-by-commit breakdown (${commit_count} commits) + +${museum_table} + + +EOF + ) + + local topology_replacement + topology_replacement=$(generate_issue_branch_topology) + + # Replace each token with its generated block. awk because sed chokes on + # multi-line replacements with pipes/backticks. + awk \ + -v museum_token="" \ + -v museum_replacement="$museum_replacement" \ + -v topology_token="" \ + -v topology_replacement="$topology_replacement" \ + '{ + if ($0 == museum_token) print museum_replacement + else if ($0 == topology_token) print topology_replacement + else print + }' \ + "$template" +} + +# GitHub rejects --base edits on PRs that belong to a native stack (the stack +# owns the base) — send --base only when the base actually needs to move. +edit_pr() { + local pr_url="$1" body_file="$2" base="$3" + local current + current=$(gh_host pr view "$pr_url" --json baseRefName -q .baseRefName) + if [[ "$current" == "$base" ]]; then + gh_host pr edit "$pr_url" --body-file "$body_file" + else + gh_host pr edit "$pr_url" --body-file "$body_file" --base "$base" + fi +} + +update_prs() { + if [[ "${DO_LLM_ONLY:-false}" != "true" ]]; then + for transform in "${TRANSFORMS[@]}"; do + local branch pr_url pr_body_file pr_title + branch=$(tvar "$transform" "branch") + pr_url=$(tvar "$transform" "pr") + pr_body_file="$BAR/.git/${branch}-pr-body.md" + pr_title=$(tvar "$transform" "pr_title") + : "${pr_title:="[Types] ${transform//_/-}"}" + + local base="${PR_BASE[$branch]:-master}" + if [[ -n "$pr_url" ]]; then + step "Updating PR $pr_url..." + edit_pr "$pr_url" "$pr_body_file" "$base" + ok "PR updated" + else + step "Creating PR for $branch..." + local new_url + new_url=$(gh_host pr create \ + --repo "$UPSTREAM_REPO" \ + --head "$(head_for "$branch")" \ + --base "$base" \ + --title "$pr_title" \ + --body-file "$pr_body_file" \ + --draft) + ok "Created PR: $new_url" + warn "Add this URL to generate-branches.sh: ${transform}_pr=\"$new_url\"" + fi + done + + if [[ -n "$MIG_PR" ]]; then + step "Updating mig PR $MIG_PR..." + edit_pr "$MIG_PR" "$BAR/.git/mig-pr-body.md" "${PR_BASE[mig]:-master}" + ok "mig PR updated" + fi + fi + + # fmt-llm-source env layer PR. + update_capstone_pr "$LLM_SOURCE_BRANCH" "$LLM_SOURCE_PR" "$LLM_SOURCE_PR_TITLE" + + # fmt-llm capstone PR. + update_capstone_pr "$LLM_BRANCH" "$LLM_PR" "$LLM_PR_TITLE" + + # Tracking issue — update the body with the latest museum table. + local tracking_body_file="$BAR/.git/tracking-issue-body.md" + if [[ -f "$tracking_body_file" ]] && [[ -n "$TRACKING_ISSUE" ]]; then + step "Updating tracking issue $TRACKING_ISSUE..." + gh_host issue edit "$TRACKING_ISSUE" --body-file "$tracking_body_file" + ok "Tracking issue updated" + fi + + link_gh_stack +} + +# Post stack-summary metadata on the linear chain via `gh stack link`. This +# doesn't rely on gh-stack local tracking — it just renders the "part of a +# stack of N PRs" UI block on each existing PR and wires up the navigation. +# No new PRs are created: branches with existing PRs are reused, per +# `gh stack link --help`. +link_gh_stack() { + local ghbin + ghbin="$(resolve_host_gh)" + if [[ -z "$ghbin" ]] || ! host_exec "$ghbin" extension list 2>/dev/null | grep -q 'gh-stack'; then + info "gh-stack extension not installed — skipping stack linking" + info " (install with: gh extension install github/gh-stack)" + return 0 + fi + + local -a chain=() + for b in fmt mig "$LLM_SOURCE_BRANCH" "$LLM_BRANCH"; do + if git_bar rev-parse --verify "$b" >/dev/null 2>&1; then + chain+=("$b") + fi + done + if [[ ${#chain[@]} -lt 2 ]]; then + info "Stack has fewer than 2 branches present — skipping gh stack link" + return 0 + fi + + step "Linking stack on GitHub: ${chain[*]}" + if (cd "$BAR" && host_exec "$ghbin" stack link --remote "$UPSTREAM_REMOTE" --base master "${chain[@]}"); then + ok "Stack linked" + else + warn "gh stack link failed — PRs are still individually correct, just no stack comment" + fi +} + +update_capstone_pr() { + local branch="$1" pr_url="$2" pr_title="$3" + local pr_body_file="$BAR/.git/${branch}-pr-body.md" + + if ! git_bar rev-parse --verify "$branch" >/dev/null 2>&1; then + return 0 + fi + if [[ ! -f "$pr_body_file" ]]; then + warn "$branch: PR body not generated, skipping" + return 0 + fi + + # Stacked PRs: base branch from PR_BASE, defaults to master. + # Merge chain: master <- fmt <- mig <- fmt-llm-source <- fmt-llm + local base="${PR_BASE[$branch]:-master}" + + if [[ -n "$pr_url" ]]; then + step "Updating PR $pr_url..." + edit_pr "$pr_url" "$pr_body_file" "$base" + ok "$branch PR updated" + else + step "Creating PR for $branch (base: $base)..." + local new_url + new_url=$(gh_host pr create \ + --repo "$UPSTREAM_REPO" \ + --head "$(head_for "$branch")" \ + --base "$base" \ + --title "$pr_title" \ + --body-file "$pr_body_file" \ + --draft) + ok "Created PR: $new_url" + warn "Add this URL to generate-branches.sh: LLM_PR=\"$new_url\"" + fi +} + +# Verify the named branches landed on $remote at the expected local SHAs. +# Uses `git ls-remote` — asks the remote directly rather than trusting the +# local tracking-ref cache. `git fetch ` doesn't always +# update refs/remotes// (sometimes only FETCH_HEAD), so a +# rev-parse against the cached ref can silently return stale values and +# wrongly report "OK". ls-remote bypasses that entirely. +# +# Catches the real-world case where `git push --force-with-lease` with +# multiple refs silently drops one ref (lease check failed on that ref, +# others succeeded, overall exit still 0 on some git versions / hook +# configurations). Seen multiple times on fmt-llm-source. +verify_pushed() { + local remote="$1"; shift + local -a refs=("$@") + step "Verifying ${#refs[@]} ref(s) landed on $remote..." + + # One ls-remote call for all refs — build branch=sha map. + local ls_output + if ! ls_output=$(git_bar ls-remote "$remote" "${refs[@]/#/refs/heads/}" 2>&1); then + err "ls-remote on $remote failed — can't verify." + err "$ls_output" + exit 1 + fi + + local drift=0 + for b in "${refs[@]}"; do + local local_sha remote_sha + local_sha=$(git_bar rev-parse "$b") + remote_sha=$(echo "$ls_output" | awk -v ref="refs/heads/$b" '$2 == ref { print $1 }') + if [[ -z "$remote_sha" ]]; then + err " $remote: ref refs/heads/$b not found (push may have been rejected entirely)" + drift=1 + elif [[ "$local_sha" != "$remote_sha" ]]; then + err " $remote/$b at $remote_sha, expected $local_sha" + drift=1 + fi + done + if [[ "$drift" == "1" ]]; then + err "Post-push drift on $remote — push reported success but remote refs don't match local." + err "Re-run --push, or push the drifted branches by hand with an explicit lease:" + err " git -C $BAR push $remote --force-with-lease=: ..." + exit 1 + fi + ok "All refs verified on $remote" +} + +push_branches() { + local branches=("mig" "$LLM_SOURCE_BRANCH" "$LLM_BRANCH") + for transform in "${TRANSFORMS[@]}"; do + branches+=("$(tvar "$transform" "branch")") + done + + # Filter to branches that exist locally (e.g., --llm-only skips leaves). + local -a existing=() + for b in "${branches[@]}"; do + if git_bar rev-parse --verify "$b" >/dev/null 2>&1; then + existing+=("$b") + fi + done + + # Topology invariant: $LLM_BRANCH must be stacked on the current + # $LLM_SOURCE_BRANCH tip. If fmt-llm-source was rebuilt without rebuilding + # fmt-llm (e.g., --skip-generation, or --llm-only with a stale local + # fmt-llm-source), the stale fmt-llm would force-push cleanly and PR #8235 + # would silently go DIRTY against the new base. verify_pushed only checks + # local==remote, so it can't catch this — guard before push. + if git_bar rev-parse --verify "$LLM_SOURCE_BRANCH" >/dev/null 2>&1 \ + && git_bar rev-parse --verify "$LLM_BRANCH" >/dev/null 2>&1; then + local src_tip + src_tip=$(git_bar rev-parse "$LLM_SOURCE_BRANCH") + if ! git_bar merge-base --is-ancestor "$src_tip" "$LLM_BRANCH"; then + err "$LLM_BRANCH is not a descendant of $LLM_SOURCE_BRANCH ($src_tip)." + err "fmt-llm-source was rebuilt without rebuilding fmt-llm — refusing" + err "to push (would leave PR #8235 with merge conflicts against the" + err "regenerated base)." + err "" + err "Re-run without --skip-generation, or with --llm-only, to rebuild" + err "$LLM_BRANCH on top of the current $LLM_SOURCE_BRANCH." + exit 1 + fi + fi + + # A branch pushed at the same tip as its PR base is read by GitHub as + # "merged": it closes the PR, credits the pusher, and DELETES the branch. + # That is irreversible — a merged PR cannot be reopened. It happens when a + # run dies between `checkout -B fmt` and the transform commit, and + # the empty leaf is pushed anyway (this is how #8403 was lost). + local -a empty=() + for b in "${existing[@]}"; do + local base="${PR_BASE[$b]:-master}" + git_bar rev-parse --verify "$base" >/dev/null 2>&1 || continue + if [[ "$(git_bar rev-list --count "$base..$b")" == "0" ]]; then + empty+=("$b (base $base)") + fi + done + if ((${#empty[@]})); then + err "Refusing to push: these branches carry no commits over their PR base," + err "so GitHub would false-merge and delete them:" + printf ' %s\n' "${empty[@]}" >&2 + err "" + err "The generation for these branches did not complete. Re-run it, or drop" + err "them from the push set — do not push them as-is." + exit 1 + fi + + # Split by primary remote. + local -A by_remote=() + for b in "${existing[@]}"; do + by_remote[$(remote_for "$b")]+="$b " + done + + for r in "${!by_remote[@]}"; do + local -a refs=(${by_remote[$r]}) + step "Force-pushing: ${refs[*]} -> $r" + git_bar push "$r" --force-with-lease "${refs[@]}" + ok "Pushed to $r" + verify_pushed "$r" "${refs[@]}" + done + + # The rollup PRs (mig #7229, fmt-llm-source #7447, fmt-llm #8235) were + # opened cross-fork with head=$FORK_OWNER:. GitHub's PR head ref + # lives on the fork, not origin, so pushing only to origin leaves the PR + # showing a stale head SHA (and therefore the old pre-topology-fix diff). + # Mirror these branches to the fork as well until those PRs are recreated + # same-repo (head=origin:). + local -a mirror=() + for b in fmt mig "$LLM_SOURCE_BRANCH" "$LLM_BRANCH"; do + if git_bar rev-parse --verify "$b" >/dev/null 2>&1; then + mirror+=("$b") + fi + done + if [[ ${#mirror[@]} -gt 0 ]] && [[ "$UPSTREAM_REMOTE" != "$FORK_REMOTE" ]]; then + step "Mirroring main-chain branches to fork: ${mirror[*]} -> $FORK_REMOTE" + git_bar push "$FORK_REMOTE" --force-with-lease "${mirror[@]}" + ok "Mirrored to $FORK_REMOTE" + verify_pushed "$FORK_REMOTE" "${mirror[@]}" + fi +} + +# ─── CLI ───────────────────────────────────────────────────────────────────── + +DO_PUSH=false +DO_UPDATE_PRS=false +DO_LLM_ONLY=false +DO_SKIP_GENERATION=false + +while [[ $# -gt 0 ]]; do + case "$1" in + --push) DO_PUSH=true; shift ;; + --update-prs) DO_UPDATE_PRS=true; shift ;; + --llm-only) DO_LLM_ONLY=true; shift ;; + --skip-generation) DO_SKIP_GENERATION=true; shift ;; + -h|--help) + cat < balloon +# into the whole upstream delta. Fast-forward the fork master to the canonical and +# publish it. FF-only: a diverged fork master means real local work — refuse to force. +sync_origin_master() { + git_bar fetch --no-recurse-submodules "$UPSTREAM_REMOTE" 2>/dev/null \ + || { warn "Could not fetch $UPSTREAM_REMOTE; skipping master sync (baseline may be stale)."; return 0; } + local canon="$UPSTREAM_REMOTE/master" base="$FORK_REMOTE/master" base_sha canon_sha + git_bar rev-parse --verify "$canon" >/dev/null 2>&1 \ + || { warn "$canon not found; skipping master sync."; return 0; } + base_sha=$(git_bar rev-parse "$base"); canon_sha=$(git_bar rev-parse "$canon") + [[ "$base_sha" == "$canon_sha" ]] && { info " $base already current with $canon."; return 0; } + if git_bar merge-base --is-ancestor "$canon" "$base"; then + info " $base is ahead of $canon — leaving as is."; return 0 + fi + if ! git_bar merge-base --is-ancestor "$base" "$canon"; then + err "$base has diverged from $canon (not a fast-forward) — refusing to force." + err "The fork master must be a clean downstream mirror of the canonical; resolve by hand." + exit 1 + fi + step "Fast-forwarding $base to $canon (${base_sha:0:9} -> ${canon_sha:0:9})..." + git_bar push "$FORK_REMOTE" "$canon_sha:refs/heads/master" + git_bar -c submodule.recurse=false fetch --no-recurse-submodules "$FORK_REMOTE" + ok " $base synced to canonical." +} + +# Front-load LLM-backend auth. op (CREDENTIALS_RUNNER) resolves keys before this +# script starts, so validate now — an unattended run must fail in seconds, not +# after minutes of branch building. Skipped when the LLM step won't run. +preflight_credentials() { + [[ "$DO_SKIP_GENERATION" == "true" ]] && return 0 + case "${BACKEND:-claude}" in + openai) + [[ -n "${OPENAI_API_KEY:-}" ]] && return 0 + err "BACKEND=openai but OPENAI_API_KEY is not set." + err " Wrap the run so op resolves it up front (see CREDENTIALS_RUNNER in just/bar.just)." + exit 1 + ;; + esac +} + +# ─── Run ───────────────────────────────────────────────────────────────────── + +preflight_credentials + +step "Fetching origin..." +git_bar -c submodule.recurse=false fetch --no-recurse-submodules origin + +# require-library regenerates recoil-lua-library/library every run, leaving the +# submodule perpetually dirty. Tell git to ignore it so status/add/checkout +# don't stage or trip over the generated content during branch building. +git_bar config submodule.recoil-lua-library.ignore all 2>/dev/null || true + +[[ "$DO_SKIP_GENERATION" == "true" ]] || warm_lux_cache + +if [[ "$DO_SKIP_GENERATION" == "true" ]]; then + step "--skip-generation: skipping ALL branch rebuilds (PR bodies only)" + # Sanity check: at minimum the LLM capstone has to exist locally for + # generate_all_pr_bodies to find anything to render. If even fmt-llm is + # missing, the user is clearly trying to use this flag too aggressively. + if ! git_bar rev-parse --verify "$LLM_BRANCH" >/dev/null 2>&1; then + err "$LLM_BRANCH does not exist locally — run a full pipeline first" + exit 1 + fi + if load_test_results; then + info "Loaded cached test results from $TEST_RESULTS_CACHE" + else + warn "No cached test results — topology tables will show 'n/a' for Units" + fi +elif [[ "$DO_LLM_ONLY" == "true" ]]; then + step "--llm-only: skipping leaves and mig rebuild" + if ! git_bar rev-parse --verify mig >/dev/null 2>&1; then + err "mig branch does not exist locally — run a full pipeline first" + exit 1 + fi + # Preserve leaf test results from a previous full run so the topology + # tables aren't lying about untested branches. + load_test_results || true + build_fmt_llm_source + build_fmt_llm + persist_test_results +else + step "Syncing origin/master from upstream..." + sync_origin_master + + step "Rebasing prefix and prereq branches onto origin/master..." + # Prereq/prefix branches are small, hand-maintained + # (detach-bar-modules-env). A merge conflict means upstream changes have + # diverged from the branch's assumptions — silently resolving via `-X + # theirs` or accepting auto-merge has historically produced bloated + # commits that pulled in unrelated upstream changes. Fail fast so the + # maintainer can rebuild the branch by hand from origin/master. + rebase_or_fail() { + local branch="$1" + if ! git_bar rebase origin/master; then + git_bar rebase --abort 2>/dev/null || true + err "Conflict rebasing $branch onto origin/master." + err "Resolve by rebuilding the branch manually from origin/master:" + err " git checkout $branch && git reset --hard origin/master" + err " (cherry-pick or re-apply the intended commit, then re-run)" + err "Refusing to continue to avoid polluting $branch with unrelated" + err "upstream changes (see lux-i18n bloat incident)." + exit 1 + fi + } + for prefix in "${PREFIX_BRANCHES[@]}"; do + step " Rebasing $prefix..." + git_bar checkout --force "$prefix" + rebase_or_fail "$prefix" + done + for transform in "${TRANSFORMS[@]}"; do + prereq=$(tvar "$transform" "prereq") + if [[ -n "$prereq" ]]; then + step " Rebasing $prereq..." + git_bar checkout --force "$prereq" + rebase_or_fail "$prereq" + fi + done + + for transform in "${TRANSFORMS[@]}"; do + build_leaf "$transform" + done + + build_mig + build_fmt_llm_source + build_fmt_llm + persist_test_results +fi + +generate_all_pr_bodies + +# PR bodies embed commit SHAs (museum table); a force-push without a body +# refresh guarantees dead links. Branches and bodies move as one transaction. +if [[ "$DO_PUSH" == "true" && "$DO_UPDATE_PRS" != "true" ]]; then + info "--push implies --update-prs (PR bodies embed commit SHAs)" + DO_UPDATE_PRS=true +fi + +if [[ "$DO_PUSH" == "true" ]] || [[ "$DO_UPDATE_PRS" == "true" ]]; then + push_branches +fi + +if [[ "$DO_UPDATE_PRS" == "true" ]]; then + update_prs +fi + +echo "" +if [[ "$DO_SKIP_GENERATION" == "true" ]]; then + ok "PR bodies regenerated (no branches rebuilt)." +else + ok "All branches rebuilt." +fi +if [[ "$DO_LLM_ONLY" == "true" ]] || [[ "$DO_SKIP_GENERATION" == "true" ]]; then + info " $LLM_BRANCH" +else + leaf_names="" + for transform in "${TRANSFORMS[@]}"; do + leaf_names+="$(tvar "$transform" "branch"), " + done + info " ${leaf_names}mig, $LLM_BRANCH" +fi diff --git a/scripts/codemod/llm-type-triage-worker.py b/scripts/codemod/llm-type-triage-worker.py new file mode 100755 index 00000000..90a0f8cd --- /dev/null +++ b/scripts/codemod/llm-type-triage-worker.py @@ -0,0 +1,388 @@ +#!/usr/bin/env python3 +"""Single-chunk type-triage worker. + +One invocation handles one chunk: runs an OpenAI chat loop with two tools +(read_file, edit_file), iterates until the model produces a final text +response or the turn budget is exhausted, then prints the model's +FIXED/ATTEMPTED/UNCATEGORIZED report to stdout. + +Invoked by scripts/codemod/llm-type-triage.sh in parallel — one Python process per +chunk. The wrapper bash script handles chunk partitioning, parallel +dispatch, and the before/after emmylua_check measurement. + +This file replaced an earlier `llm --functions` heredoc inside the bash +script. Going direct to the openai SDK gives us: + + - Per-turn progress logging (visible in the chunk's log file) + - Token usage tracking (real input / cached / output counts) + - Graceful handling of API errors and turn-limit exhaustion (we still + print whatever final text the model produced last) + - Tools are normal Python — no shell heredoc encoding, no third-party + model registry to maintain + +Inputs: + --chunk PATH file with the inlined chunk error blocks + --system PATH file with the system prompt (rules + SKILL.md) + --model NAME OpenAI model id (e.g. gpt-5.4-mini) + --max-turns N cap on model turns per chunk (default: 50) + +Environment: + BAR_DIR repo root used to resolve / sandbox tool paths + OPENAI_API_KEY OpenAI auth, inherited from CREDENTIALS_RUNNER + +Outputs: + stdout final FIXED/ATTEMPTED/UNCATEGORIZED report + stderr per-turn progress + token summary + +Exit codes: + 0 chain ran to completion (or hit turn limit but produced output) + 1 hard failure (auth missing, no progress at all, import error) +""" + +import argparse +import json +import os +import sys +from pathlib import Path + +try: + from openai import OpenAI +except ImportError: + sys.exit( + "ERROR: openai package not installed. " + "Install with: python3 -m pip install --user openai" + ) + + +# ─── Tool sandbox ────────────────────────────────────────────────────────── + +BAR = Path(os.environ.get("BAR_DIR", ".")).resolve() +FORBIDDEN_PREFIXES = ("recoil-lua-library/", "types/") +FORBIDDEN_FILES = (".emmyrc.json", ".luarc.json") + +# Hard cap on bytes returned for an unbounded read_file. Files above this +# (e.g. luaui/Widgets/gui_pip.lua at 800KB) would otherwise eat the model's +# context once they accumulate in conversation history across turns. +FULL_READ_MAX_BYTES = 60_000 + + +def _resolve(path: str) -> Path: + """Resolve a relative path against BAR_DIR. Refuses anything that escapes.""" + p = (BAR / path).resolve() + if not str(p).startswith(str(BAR) + os.sep) and p != BAR: + raise ValueError("path escapes BAR repo: " + path) + return p + + +def _check_writable(path: str) -> str | None: + """Return an error string if `path` is baseline-managed, else None. + + Important: do NOT use str.lstrip("./") here — it strips any combination + of '.' and '/' characters and would mangle '.emmyrc.json' into + 'emmyrc.json', bypassing the forbidden-file check entirely.""" + rel = path[2:] if path.startswith("./") else path + if rel in FORBIDDEN_FILES: + return f"refusing to edit baseline-managed file: {rel}" + for prefix in FORBIDDEN_PREFIXES: + if rel.startswith(prefix): + return f"refusing to edit out-of-scope path under {prefix}: {rel}" + return None + + +# ─── Tool implementations ────────────────────────────────────────────────── + +def tool_read_file(path: str, start_line: int = 0, end_line: int = 0) -> str: + try: + p = _resolve(path) + if start_line or end_line: + lines = p.read_text().splitlines() + n = len(lines) + s = max(1, start_line or 1) + e = min(n, end_line if end_line else n) + if s > n: + return f"ERROR: start_line {s} is past end of file ({n} lines)" + window = lines[s - 1:e] + width = len(str(e)) + return "\n".join( + f"{str(s + i).rjust(width)}: {line}" + for i, line in enumerate(window) + ) + size = p.stat().st_size + if size > FULL_READ_MAX_BYTES: + return ( + f"ERROR: file is {size} bytes (>{FULL_READ_MAX_BYTES}), too large " + "for an unbounded read. Call read_file(path, start_line, end_line) " + "with a window around the error line instead (the chunk gives you " + "exact line numbers)." + ) + return p.read_text() + except Exception as e: + return f"ERROR: {e}" + + +def tool_edit_file(path: str, search: str, replace: str) -> str: + err = _check_writable(path) + if err: + return f"ERROR: {err}" + try: + p = _resolve(path) + content = p.read_text() + count = content.count(search) + if count == 0: + return "ERROR: search text not found in file (verbatim match required, including whitespace)" + if count > 1: + return f"ERROR: search text appears {count} times — expand it for uniqueness" + p.write_text(content.replace(search, replace, 1)) + return "OK" + except Exception as e: + return f"ERROR: {e}" + + +TOOLS_BY_NAME = { + "read_file": tool_read_file, + "edit_file": tool_edit_file, +} + +# OpenAI tool schema. Descriptions matter — they're what the model reads to +# decide when and how to call. Keep them in sync with the system prompt. +TOOLS_SCHEMA = [ + { + "type": "function", + "function": { + "name": "read_file", + "description": ( + "Read a .lua file from the BAR repo. Path is relative to repo root. " + "For files larger than ~60KB you MUST pass start_line and end_line " + "(1-indexed, inclusive) to read just a window — full reads are " + "rejected for big files because they fill the conversation context " + "too quickly. The error blocks in your chunk give you the exact line " + "of every error; read ~20 lines on either side. With line params, " + "the response is line-number prefixed (e.g. '123: '); strip " + "the prefix when building search strings for edit_file." + ), + "parameters": { + "type": "object", + "properties": { + "path": { + "type": "string", + "description": "Path relative to BAR repo root", + }, + "start_line": { + "type": "integer", + "description": "1-indexed start line (omit or 0 = full file)", + }, + "end_line": { + "type": "integer", + "description": "1-indexed inclusive end line (omit or 0 = full file)", + }, + }, + "required": ["path"], + }, + }, + }, + { + "type": "function", + "function": { + "name": "edit_file", + "description": ( + "Replace the first (and only) occurrence of `search` with `replace` " + "in `path`. `search` must appear EXACTLY ONCE in the file (verbatim, " + "including whitespace). If 0 or >1 matches, returns an error and " + "writes nothing — expand or shrink the search and try again. " + "Returns 'OK' on success." + ), + "parameters": { + "type": "object", + "properties": { + "path": {"type": "string", "description": "Path relative to BAR repo root"}, + "search": {"type": "string", "description": "Exact substring to find (must be unique)"}, + "replace": {"type": "string", "description": "Replacement text"}, + }, + "required": ["path", "search", "replace"], + }, + }, + }, +] + + +# ─── Chat loop ───────────────────────────────────────────────────────────── + +def log(msg: str) -> None: + print(f"[worker] {msg}", file=sys.stderr, flush=True) + + +def assistant_message_to_dict(msg) -> dict: + """Convert an OpenAI ChatCompletionMessage into a dict suitable for + re-sending in the next request. Building this manually rather than + using model_dump() to avoid emitting fields the API rejects on + re-send (e.g. `function_call`, `refusal`, `audio`).""" + out: dict = {"role": "assistant", "content": msg.content} + if msg.tool_calls: + out["tool_calls"] = [ + { + "id": tc.id, + "type": "function", + "function": { + "name": tc.function.name, + "arguments": tc.function.arguments, + }, + } + for tc in msg.tool_calls + ] + return out + + +def execute_tool_call(tc) -> str: + """Run a single tool call and return its string result.""" + name = tc.function.name + try: + args = json.loads(tc.function.arguments) + except json.JSONDecodeError as e: + return f"ERROR: failed to parse tool args: {e}" + fn = TOOLS_BY_NAME.get(name) + if fn is None: + return f"ERROR: unknown tool: {name}" + try: + return fn(**args) + except TypeError as e: + return f"ERROR: bad tool arguments: {e}" + except Exception as e: + return f"ERROR: tool execution failed: {e}" + + +def main() -> int: + ap = argparse.ArgumentParser(description=__doc__.split("\n", 1)[0]) + ap.add_argument("--chunk", required=True, type=Path, + help="file with the inlined chunk error blocks") + ap.add_argument("--system", required=True, type=Path, + help="file with the full system prompt (rules + SKILL.md)") + ap.add_argument("--model", required=True, + help="OpenAI model id, e.g. gpt-5.4-mini") + ap.add_argument("--max-turns", type=int, default=50, + help="cap on model turns per chunk (default: 50)") + args = ap.parse_args() + + if not os.environ.get("OPENAI_API_KEY"): + log("ERROR: OPENAI_API_KEY not set in environment") + return 1 + if not args.chunk.exists(): + log(f"ERROR: chunk file not found: {args.chunk}") + return 1 + if not args.system.exists(): + log(f"ERROR: system prompt file not found: {args.system}") + return 1 + + chunk_name = args.chunk.stem + log(f"chunk={chunk_name} model={args.model} max_turns={args.max_turns} BAR_DIR={BAR}") + + system_prompt = args.system.read_text() + chunk_body = args.chunk.read_text() + + user_msg = ( + f"You are working on chunk: {chunk_name}\n\n" + "Below are the emmylua_check error blocks assigned to you. Open the .lua " + "files they reference (paths are relative to the BAR repo root — pass them " + "straight to read_file / edit_file), fix every error per the rules and fix " + "priorities in your system prompt, then output the FIXED / ATTEMPTED / " + "UNCATEGORIZED report as your final message.\n\n" + "=== chunk errors ===\n" + f"{chunk_body}\n" + "=== end chunk ===" + ) + + messages: list[dict] = [ + {"role": "system", "content": system_prompt}, + {"role": "user", "content": user_msg}, + ] + + client = OpenAI() + + total_in = 0 + total_cached = 0 + total_out = 0 + final_text: str | None = None + last_assistant_text: str | None = None + hit_limit = False + + for turn in range(1, args.max_turns + 1): + try: + resp = client.chat.completions.create( + model=args.model, + messages=messages, + tools=TOOLS_SCHEMA, + parallel_tool_calls=True, + ) + except Exception as e: + log(f"turn {turn}: API error: {e}") + return 1 + + usage = getattr(resp, "usage", None) + if usage is not None: + total_in += getattr(usage, "prompt_tokens", 0) or 0 + total_out += getattr(usage, "completion_tokens", 0) or 0 + details = getattr(usage, "prompt_tokens_details", None) + if details is not None: + total_cached += getattr(details, "cached_tokens", 0) or 0 + + msg = resp.choices[0].message + messages.append(assistant_message_to_dict(msg)) + if msg.content: + last_assistant_text = msg.content + + tool_calls = msg.tool_calls or [] + if not tool_calls: + final_text = msg.content or "" + log(f"turn {turn}: model finished ({len(final_text)} chars of output)") + break + + # Run all tool calls from this turn (potentially in parallel + # conceptually — they're independent on the model's side, even + # though we execute sequentially here for simplicity). + n_read = sum(1 for tc in tool_calls if tc.function.name == "read_file") + n_edit = sum(1 for tc in tool_calls if tc.function.name == "edit_file") + n_other = len(tool_calls) - n_read - n_edit + parts = [] + if n_read: + parts.append(f"{n_read} read_file") + if n_edit: + parts.append(f"{n_edit} edit_file") + if n_other: + parts.append(f"{n_other} other") + log(f"turn {turn}: {len(tool_calls)} tool calls ({', '.join(parts)})") + + for tc in tool_calls: + result = execute_tool_call(tc) + messages.append({ + "role": "tool", + "tool_call_id": tc.id, + "content": result, + }) + # One-line preview per call so the log shows what happened + # without dumping every byte of every read result. + if result.startswith("ERROR"): + log(f" {tc.function.name} → {result[:120]}") + elif tc.function.name == "edit_file": + log(f" edit_file → {result}") + else: + # Loop fell through without break → we exhausted max_turns. + hit_limit = True + log(f"hit max-turns limit ({args.max_turns}) without model finalizing") + + log( + f"tokens: input={total_in} (cached={total_cached}) " + f"output={total_out} total={total_in + total_out}" + ) + + if final_text: + print(final_text) + return 0 + if last_assistant_text: + log("no clean finish; printing last assistant text:") + print(last_assistant_text) + return 0 if hit_limit else 1 + log("no assistant text produced at all") + return 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/scripts/codemod/llm-type-triage.sh b/scripts/codemod/llm-type-triage.sh new file mode 100755 index 00000000..aaf716ec --- /dev/null +++ b/scripts/codemod/llm-type-triage.sh @@ -0,0 +1,441 @@ +#!/usr/bin/env bash +# LLM type-triage: parallel chunked fan-out, single pass, no orchestrator. +# +# Pipeline: +# 1. Capture baseline emmylua_check output +# 2. Parse + group errors by file, partition into ~N chunks +# 3. Spawn N parallel subagents (one per chunk) — each runs the +# type-triage subagent prompt with its chunk content inlined. +# Backend is selectable via BACKEND env var: +# - claude (default): `claude --print` with Read+Edit tools +# - openai: scripts/codemod/llm-type-triage-worker.py — a +# self-contained Python agent loop using the +# openai SDK directly. Sonnet is slow; +# gpt-5.4-mini-class models are 10-50x cheaper +# and faster. The openai package is auto- +# installed via `pip install --user` if missing. +# 4. wait for all +# 5. Re-run emmylua_check, write before/after summary +# +# Design notes: +# - No iteration loop. If a chunk persists after one pass, that's a signal +# that SKILL.md needs a new category — a human edits the rules and reruns. +# Iteration on a stable rule set is wasted tokens. +# - No Opus orchestrator. The chunking is deterministic bash+Python; the +# subagents are independent fan-out workers. No coordinator needed. +# - Subagents only get a Read tool and an Edit tool (no Bash, no shell), +# which makes the "model misinterprets the prompt as a bash script" +# failure mode structurally impossible. The OpenAI backend enforces this +# by only exposing read_file/edit_file in its Python tool sandbox. +# - The OpenAI variant relies on automatic prompt caching: SKILL.md + +# subagent rules go in the system message (identical across all parallel +# chunks) so the prefix is cached after the first chunk warms it up. +# Within a chunk, accumulated read_file results are also cached on every +# turn after they first appear, keeping multi-turn cost roughly linear. +# - The OpenAI agent loop lives in scripts/codemod/llm-type-triage-worker.py +# (separate file, not a heredoc) so it's testable, lintable, and easy +# to debug. The bash script just dispatches one Python process per chunk. +# +# Inputs (env, with defaults): +# DEVTOOLS_DIR -- BAR-Devtools repo root (auto-detected) +# BAR_DIR -- BAR repo (default: $DEVTOOLS_DIR/Beyond-All-Reason) +# BACKEND -- claude | openai (default: claude) +# SUBAGENT_MODEL -- Model name. Default depends on BACKEND: +# claude → claude-sonnet-4-6 +# openai → gpt-5.4-mini +# TARGET_CHUNKS -- Number of partitions (default: 8) +# CHAIN_LIMIT -- (openai only) max model turns per chunk before the +# worker prints partial output and exits (default: 50) +# CLAUDE_BIN_OVERRIDE -- absolute path to claude (skip discovery) +# EMMYLUA_BIN_OVERRIDE -- absolute path to emmylua_check (skip discovery) +# OPENAI_API_KEY -- (openai only) inherited from CREDENTIALS_RUNNER wrap + +set -euo pipefail + +# ─── Locate repos ──────────────────────────────────────────────────────────── + +if [[ -z "${DEVTOOLS_DIR:-}" ]]; then + SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + DEVTOOLS_DIR="$(dirname "$SCRIPT_DIR")" +fi +export DEVTOOLS_DIR + +source "${DEVTOOLS_DIR}/scripts/common.sh" + +BAR="${BAR_DIR:-${DEVTOOLS_DIR}/Beyond-All-Reason}" +export BAR_DIR="$BAR" + +# Run a command on the host OS from inside a distrobox container. +host_exec() { + if [ -f /run/.containerenv ] && command -v distrobox-host-exec &>/dev/null; then + distrobox-host-exec "$@" + else + "$@" + fi +} + +BACKEND="${BACKEND:-claude}" +case "$BACKEND" in + claude) DEFAULT_MODEL="claude-sonnet-4-6" ;; + openai) DEFAULT_MODEL="gpt-5.4-mini" ;; + *) + err "Unknown BACKEND: $BACKEND (expected: claude | openai)" + exit 1 + ;; +esac +SUBAGENT_MODEL="${SUBAGENT_MODEL:-$DEFAULT_MODEL}" +TARGET_CHUNKS="${TARGET_CHUNKS:-8}" +# Each turn the model can request multiple parallel tool calls, so 50 turns +# is plenty of headroom for a 30-error chunk *if the model batches*. Bumped +# from 25 after chunk-04 (29 errors, several files >60KB requiring multiple +# small line-range reads) hit the limit doing one-at-a-time tool calls. +CHAIN_LIMIT="${CHAIN_LIMIT:-50}" + +# ─── Resolve binaries ──────────────────────────────────────────────────────── + +resolve_emmylua() { + if [[ -n "${EMMYLUA_BIN_OVERRIDE:-}" ]]; then echo "$EMMYLUA_BIN_OVERRIDE"; return 0; fi + local c + for c in /usr/local/bin/emmylua_check "$HOME/.local/bin/emmylua_check" /usr/bin/emmylua_check; do + [[ -x "$c" ]] && { echo "$c"; return 0; } + done + command -v emmylua_check 2>/dev/null || true +} +EMMYLUA_BIN="$(resolve_emmylua)" +if [[ -z "$EMMYLUA_BIN" ]]; then + err "emmylua_check not found" + err " override: EMMYLUA_BIN_OVERRIDE=/abs/path/to/emmylua_check" + exit 1 +fi + +resolve_host_claude() { + if [[ -n "${CLAUDE_BIN_OVERRIDE:-}" ]]; then echo "$CLAUDE_BIN_OVERRIDE"; return 0; fi + local c + for c in "$HOME/.local/bin/claude" "$HOME/.npm-global/bin/claude" "$HOME/.bun/bin/claude" \ + /home/linuxbrew/.linuxbrew/bin/claude /usr/local/bin/claude /usr/bin/claude; do + host_exec test -x "$c" && { echo "$c"; return 0; } + done + host_exec which claude 2>/dev/null || true +} + +# Lazily install the openai Python SDK if missing. We deliberately do NOT +# add this to the global setup script — only users who pick BACKEND=openai +# pay the install cost. The install goes via `pip --user` so it lands in +# the user's site-packages without needing root. +ensure_openai_sdk() { + if python3 -c 'import openai' 2>/dev/null; then + return 0 + fi + step "openai SDK not found — installing via pip --user (one-time, BACKEND=openai only)..." + if ! python3 -m pip install --user --quiet openai; then + err "pip install --user openai failed" + err " manual install: python3 -m pip install --user openai" + exit 1 + fi + if ! python3 -c 'import openai' 2>/dev/null; then + err "Installed openai but Python still can't import it. Check pyenv setup." + exit 1 + fi + ok "Installed openai SDK" +} + +# Resolve worker binary / dependencies based on BACKEND. +case "$BACKEND" in + claude) + CLAUDE_HOST_BIN="$(resolve_host_claude)" + if [[ -z "$CLAUDE_HOST_BIN" ]]; then + err "claude CLI not found on host" + err " override: CLAUDE_BIN_OVERRIDE=/abs/path/to/claude" + exit 1 + fi + WORKER_BIN_DESC="claude: $CLAUDE_HOST_BIN" + ;; + openai) + if [[ -z "${OPENAI_API_KEY:-}" ]]; then + err "BACKEND=openai but OPENAI_API_KEY is not set" + err " Wrap your invocation with CREDENTIALS_RUNNER (see just/bar.just)," + err " e.g. CREDENTIALS_RUNNER='op run --env-file=~/code/ai.env.op --'" + exit 1 + fi + ensure_openai_sdk + OPENAI_WORKER="${DEVTOOLS_DIR}/scripts/codemod/llm-type-triage-worker.py" + if [[ ! -x "$OPENAI_WORKER" ]]; then + err "OpenAI worker script not found or not executable: $OPENAI_WORKER" + exit 1 + fi + WORKER_BIN_DESC="worker: $OPENAI_WORKER" + ;; +esac + +# PATH passed to claude on the host so its tool subprocesses can find +# the emmylua_check wrapper at ~/.local/bin (distrobox-host-exec strips PATH). +HOST_RUN_PATH="$HOME/.local/bin:/usr/local/bin:/usr/bin:/bin" + +if [[ ! -d "$BAR/.git" ]]; then + err "BAR repo not found at $BAR" + exit 1 +fi + +step "emmylua_check: $EMMYLUA_BIN" +step "$WORKER_BIN_DESC" +step "backend: $BACKEND" +step "model: $SUBAGENT_MODEL" +step "chunks: $TARGET_CHUNKS (target)" +[[ "$BACKEND" == "openai" ]] && step "chain limit: $CHAIN_LIMIT" + +# ─── Workdir ───────────────────────────────────────────────────────────────── + +WORKDIR="$(mktemp -d /tmp/bar-llm-XXXXXX)" +PRESERVE_DIR="$BAR/.git/llm-triage" + +# On exit (success OR failure), copy whatever logs/chunks we have into the +# BAR repo's .git/ directory so the user can inspect them post-mortem. This +# matters for early-exit cases where we never reach the post-triage code path. +preserve_and_cleanup() { + if [[ -d "$WORKDIR" ]]; then + rm -rf "$PRESERVE_DIR" + mkdir -p "$PRESERVE_DIR" + cp -r "$WORKDIR"/* "$PRESERVE_DIR/" 2>/dev/null || true + rm -rf "$WORKDIR" + fi +} +trap preserve_and_cleanup EXIT +export WORKDIR +mkdir -p "$WORKDIR/chunks" "$WORKDIR/logs" + +# ─── Baseline ──────────────────────────────────────────────────────────────── + +step "Capturing baseline error log..." +(cd "$BAR" && "$EMMYLUA_BIN" -c .emmyrc.json . 2>&1 || true) > "$WORKDIR/baseline-errors.log" + +baseline_count=$(grep -oP '^\s*\K\d+(?= errors?$)' "$WORKDIR/baseline-errors.log" | head -1 || echo 0) +step "Baseline errors: $baseline_count" + +write_summary() { + local final="$1" chunks="$2" + { + echo "baseline_errors=$baseline_count" + echo "final_errors=$final" + echo "chunks=$chunks" + echo "model=$SUBAGENT_MODEL" + date -u +"timestamp=%Y-%m-%dT%H:%M:%SZ" + } > "$BAR/.git/llm-triage-summary.txt" +} + +if [[ "$baseline_count" == "0" ]]; then + ok "Already at zero errors — nothing to triage." + write_summary 0 0 + exit 0 +fi + +# ─── Chunk by file ─────────────────────────────────────────────────────────── + +step "Partitioning errors into ~$TARGET_CHUNKS chunks (grouped by file)..." + +CHUNK_COUNT=$(python3 - "$WORKDIR/baseline-errors.log" "$WORKDIR/chunks" "$TARGET_CHUNKS" <<'PY' +import sys, re, math +from pathlib import Path + +log_path, out_dir, target_chunks = sys.argv[1], Path(sys.argv[2]), int(sys.argv[3]) +text = Path(log_path).read_text() + +# emmylua_check emits per-diagnostic blocks like: +# +# error: msg [code] +# --> file:line:col +# +# +# +# warning: msg [code] +# --> ... +# +# --- path/to/file.lua [N warnings, M hints] ← section summary header +# +# Walk *boundaries* (severity-prefixed lines OR `---` section headers) and +# keep only blocks that start with `error:`. Naive `re.split(r'\n(?=error:)')` +# glues intervening warnings AND section headers onto the preceding error +# block, inflating chunks by orders of magnitude. +BOUNDARY_RE = re.compile(r'^(?:error|warning|info|note|hint):|^---\s', re.MULTILINE) +positions = [m.start() for m in BOUNDARY_RE.finditer(text)] +positions.append(len(text)) + +blocks = [] +for i in range(len(positions) - 1): + block = text[positions[i]:positions[i + 1]].rstrip() + if block.startswith('error:'): + blocks.append(block) + +by_file = {} +for block in blocks: + m = re.search(r'-->\s+([^\s:]+):', block) + if not m: + continue + by_file.setdefault(m.group(1), []).append(block) + +if not by_file: + print(0, end='') + sys.exit(0) + +total_errors = sum(len(v) for v in by_file.values()) +target_per_chunk = max(1, math.ceil(total_errors / target_chunks)) +hard_cap = int(target_per_chunk * 1.5) + +# Greedy bin-pack: sort files alphabetically (determinism), accumulate +# until we'd exceed hard_cap, then start a new chunk. Files are atomic. +chunks, current, current_size = [], [], 0 +for f in sorted(by_file.keys()): + n = len(by_file[f]) + if current_size > 0 and current_size + n > hard_cap: + chunks.append(current) + current, current_size = [], 0 + current.append(f) + current_size += n +if current: + chunks.append(current) + +out_dir.mkdir(parents=True, exist_ok=True) +for i, files in enumerate(chunks, 1): + with (out_dir / f'chunk-{i:02d}.txt').open('w') as fh: + for f in files: + for block in by_file[f]: + fh.write(block + '\n\n') + +print(len(chunks), end='') +PY +) + +if [[ -z "$CHUNK_COUNT" || "$CHUNK_COUNT" == "0" ]]; then + err "Chunker produced 0 chunks but baseline shows $baseline_count errors" + err "Check the log format at $WORKDIR/baseline-errors.log" + exit 1 +fi + +step "Wrote $CHUNK_COUNT chunk files to $WORKDIR/chunks/" +for c in "$WORKDIR/chunks/"chunk-*.txt; do + n=$(grep -c '^error:' "$c" 2>/dev/null || echo 0) + info " $(basename "$c"): $n errors" +done + +# ─── Fan-out: spawn parallel subagents ─────────────────────────────────────── + +# Pick the prompt file matching the backend. The two prompts share most of +# their content (rules, fix priorities, output format) but differ in the +# tool surface they describe to the model. +case "$BACKEND" in + claude) SUB_PROMPT_FILE="${DEVTOOLS_DIR}/claude/prompts/type-triage-subagent.md" ;; + openai) SUB_PROMPT_FILE="${DEVTOOLS_DIR}/claude/prompts/type-triage-subagent-openai.md" ;; +esac +if [[ ! -f "$SUB_PROMPT_FILE" ]]; then + err "Missing subagent prompt: $SUB_PROMPT_FILE" + exit 1 +fi + +dispatch_claude_chunk() { + local chunk_path="$1" log_path="$2" chunk_name="$3" + + # Substitute CHUNK_PATH placeholder in the subagent prompt template. + local prompt + prompt="$(sed "s|CHUNK_PATH|${chunk_path}|g" "$SUB_PROMPT_FILE")" + + # Subagents only need Read + Edit. No Bash means the "claude + # misinterprets prompt as a script to debug" failure mode is + # structurally impossible. + # + # The `--` separator is REQUIRED before "$prompt". Without it, + # claude's argparse consumes the positional prompt as an extra + # value to the preceding `--allowedTools` flag and then complains + # that no prompt was provided. (Reproducible: any `--allowedTools + # "X,Y" ""` invocation fails the same way.) + host_exec env "PATH=$HOST_RUN_PATH" "HOME=$HOME" \ + "$CLAUDE_HOST_BIN" --print \ + --model "$SUBAGENT_MODEL" \ + --permission-mode acceptEdits \ + --allowedTools "Read,Edit" \ + -- "$prompt" \ + > "$log_path" 2>&1 & +} + +dispatch_openai_chunk() { + local chunk_path="$1" log_path="$2" chunk_name="$3" + + # The Python worker owns the agent loop, the tool sandbox, and the + # token accounting. We just hand it the inputs and capture stdout/ + # stderr into the log file. BAR_DIR is exported so the worker's + # tools resolve paths against the right repo; OPENAI_API_KEY is + # inherited from the CREDENTIALS_RUNNER wrap on the parent process. + BAR_DIR="$BAR" python3 "$OPENAI_WORKER" \ + --chunk "$chunk_path" \ + --system "$OPENAI_SYSTEM_PROMPT_FILE" \ + --model "$SUBAGENT_MODEL" \ + --max-turns "$CHAIN_LIMIT" \ + > "$log_path" 2>&1 & +} + +# Build the openai system prompt once: the prompt file + the full SKILL.md +# reference, concatenated into a tmp file inside WORKDIR. This block is +# identical across all parallel chunks → after the first chunk's first +# turn lands, every other chunk's turn hits OpenAI's prompt cache for the +# shared prefix and pays roughly half price on it. The tmp file is +# preserved alongside the logs by the EXIT trap. +if [[ "$BACKEND" == "openai" ]]; then + SKILL_MD="${DEVTOOLS_DIR}/claude/skills/codemod-prereq/SKILL.md" + if [[ ! -f "$SKILL_MD" ]]; then + err "Missing SKILL.md: $SKILL_MD" + exit 1 + fi + OPENAI_SYSTEM_PROMPT_FILE="$WORKDIR/openai-system-prompt.txt" + { + cat "$SUB_PROMPT_FILE" + printf '\n\n---\n\n# SKILL.md (canonical fix procedures)\n\n' + cat "$SKILL_MD" + } > "$OPENAI_SYSTEM_PROMPT_FILE" +fi + +step "Dispatching $CHUNK_COUNT subagents in parallel ($BACKEND backend)..." +cd "$BAR" + +pids=() +for chunk_path in "$WORKDIR/chunks/"chunk-*.txt; do + chunk_name="$(basename "$chunk_path" .txt)" + log_path="$WORKDIR/logs/${chunk_name}.log" + + case "$BACKEND" in + claude) dispatch_claude_chunk "$chunk_path" "$log_path" "$chunk_name" ;; + openai) dispatch_openai_chunk "$chunk_path" "$log_path" "$chunk_name" ;; + esac + + pids+=($!) + info " spawned $chunk_name (pid $!)" +done + +step "Waiting for $CHUNK_COUNT subagents..." +# NB: do NOT use `((fail_count++))` here — that returns the pre-increment +# value as its exit status, so when fail_count is 0 the expression's status +# is 0 (failure under set -e), aborting the script before the warn fires +# and before we can preserve the logs. Use $(( )) substitution instead. +fail_count=0 +for pid in "${pids[@]}"; do + if ! wait "$pid"; then + warn " subagent pid $pid exited non-zero" + fail_count=$((fail_count + 1)) + fi +done + +if [[ "$fail_count" -gt 0 ]]; then + warn "$fail_count of $CHUNK_COUNT subagents reported failure (continuing)" +fi +ok "All subagents finished" + +# ─── Final measurement ─────────────────────────────────────────────────────── + +step "Capturing post-triage error log..." +(cd "$BAR" && "$EMMYLUA_BIN" -c .emmyrc.json . 2>&1 || true) > "$WORKDIR/final-errors.log" +final_count=$(grep -oP '^\s*\K\d+(?= errors?$)' "$WORKDIR/final-errors.log" | head -1 || echo 0) + +write_summary "$final_count" "$CHUNK_COUNT" + +# Logs are preserved by the EXIT trap (preserve_and_cleanup), so no copy +# needed here. Trap fires unconditionally — even on early exits — so a +# crashed run is just as inspectable as a successful one. + +ok "LLM triage complete: $baseline_count → $final_count errors over $CHUNK_COUNT chunks" diff --git a/scripts/codemod/tracking-issue-template.md b/scripts/codemod/tracking-issue-template.md new file mode 100644 index 00000000..1bfaa5f2 --- /dev/null +++ b/scripts/codemod/tracking-issue-template.md @@ -0,0 +1,79 @@ +# BAR type-error cleanup: coordinated merge + +## PRs + +Stacked — merge bottom-up. Each PR's own diff is scoped to its layer; stack navigation is on each PR. + +- [ ] [**fmt** — StyLua formatting](https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8395) +- [ ] [**mig** — combined deterministic transforms](https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8396) +- [ ] [**fmt-llm-source** — hand-curated env layer (emmylua config, types, manual fixes)](https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8397) +- [ ] [**fmt-llm** — LLM type-fix capstone](https://github.com/beyond-all-reason/Beyond-All-Reason/pull/8398) +- [ ] Tooling stack (BAR-Devtools): https://github.com/beyond-all-reason/BAR-Devtools/pull/57/ +- [ ] [Mission kit — DSL recognizer, validator, live editor service (BAR-Devtools)](https://github.com/beyond-all-reason/BAR-Devtools/pull/54) +- [ ] [Recoil PR (lua-doc-extractor wiring + missing type decorators)](https://github.com/beyond-all-reason/RecoilEngine/pull/2799) + - [ ] [CircuitAI — `zk` branch](https://github.com/rlcevg/CircuitAI/pull/136) + - [ ] [CircuitAI — `barbarian` branch](https://github.com/rlcevg/CircuitAI/pull/137) + +> **Important:** Do not run `just bar::migrate::stylua-cleanup` until `fmt` has merged. Running it earlier reformats the entire codebase on your branch (~200k lines). + + + + + +**For contributors — after `fmt` merges, update your open branches:** +```bash +just bar::migrate::stylua-cleanup # transform your branch first +git commit -am "apply code transforms" # squashed away when PR merges +git merge origin/master # conflicts are now real conflicts only +``` +See the [BAR-Devtools README](https://github.com/beyond-all-reason/BAR-Devtools#readme) for setup. + +`.git-blame-ignore-revs` arrives with the capstone. Until it is on `master`, `git blame` will abort with `could not open object name list` on any branch that predates it — clear it with `git config --unset blame.ignoreRevsFile` if you hit that before the stack lands. + +## What this contains + +- Automated script (`just bar::migrate::stylua-cleanup-generate`) that rebuilds all branches deterministically from `master` +- Updated [Recoil](https://github.com/beyond-all-reason/RecoilEngine/pull/2799) with new extractor + missing type decorators +- New PR gate: "Type Check" (`just bar::check`) — errors only, so warnings and hints stay local +- `.git-blame-ignore-revs` listing the mechanical commits, so `git blame` walks past the formatting and codemod layers to the author who actually wrote the line +- Replaced LuaLS/Sumneko with [EmmyLua](https://marketplace.visualstudio.com/items?itemName=tangzx.emmylua) (~100x faster). **Never use the Sumneko VS Code plugin.** + + +## New developer commands + +- `just bar::check` → type-check (EmmyLua) +- `just bar::fmt` → format (StyLua) +- `just bar::test` → unit + integration tests +- `just bar::lint` → lint (luacheck) +- `just setup::editor` → editor integration (language servers, extensions, settings) +- `just bar::setup-hooks` → pre-commit hook that checks staged Lua against StyLua and refuses if it is unformatted; also points `git blame` at `.git-blame-ignore-revs` +- `just bar::migrate::stylua-cleanup` → replay all transforms onto your branch + +### Generation pipeline: `just bar::migrate::stylua-cleanup-generate --update-prs` + +0. Fetch origin and rebase prereq branches onto master. +1. **Deterministic text transformations** — ~99.9% mistake-free once I've validated a transform, basically free to re-run. +2. **Non-deterministic pass** (LLM + rules to categorize type errors with relatively simple heuristics). This targets the ~110 type errors remaining after the globals are cleaned up (the exact count drifts as `master` moves), and crucially, most of them are actual bugs that'll improve code quality once fixed. +3. Update PRs with output. + +### The upshot + +- (1) is basically free and VERY reliable. +- (2) just requires we read it, test it, make any fixes, then either update our rules or merge ASAP. + +### Step 2 detail + +Step 2 is the interesting part. I arrived at these rules by dispatching cheap subagents in parallel, then having an orchestrator agent refine the rules and re-run until the cheaper models covered all the edge cases. Because all of these fixes are well below the waterline for an Opus-calibre agent to explain to a GPT 5.4 Mini class of agent, this works. It gives us cheap, repeatable, and mostly idempotent execution on top of master. + +Really effective for this sort of problem — in the past it would've been a month of hand editing and hating my life to get to zero, plus another month agonizing over which problems were worth a deterministic transform vs. just grinding through. =D + +## Closing thoughts + +- I think this will let us actually use the formal type system to fuller effect (because people treat it as a real signal) and will greatly increase code quality in BAR over time. +- The more formal verification we wire in, the better our parsers and LLM agents get and the faster we can move on systemic problems. +- This makes the argument made in [Game Economy](https://github.com/beyond-all-reason/RecoilEngine/pull/2664) more compelling (and I confess that's what led me here). The idea of moving subsystem by subsystem out of the engine and into Lua modules (that may or may not live in the game) makes waaaaaaay more sense when you have types enforced. Suddenly Lua can express its own design patterns under type checking — both where the engine has no stake (most of the game outside the sim) and where it does, by wrapping the engine API in typed abstractions instead of leaking it everywhere. cc @sprunk + +## Credits + +- **@rhys_vdw** — thanks for the fantastic foundation in lua-doc-extractor and recoil-lua-library. Doing all of those decorators by hand must've been unbelievably labor intensive and there is not a snowball's chance in hell I would've even started this project unless that work already existed. +- **@thule** — super enabled by BAR-Devtools existing, shout out for getting that ball rolling. SHARED CROSS REPO SCRIPTING LAYER!!!!! diff --git a/scripts/setup.sh b/scripts/setup.sh index 817b3e17..c8f8b8a0 100644 --- a/scripts/setup.sh +++ b/scripts/setup.sh @@ -113,8 +113,8 @@ check_podman() { check_distrobox() { if ! command -v distrobox &>/dev/null; then - warn "distrobox not found. Install it for the recommended dev environment." - warn "See: https://distrobox.it/#installation" + err "distrobox not found. Required for the dev toolchain (lux, stylua, emmylua, clangd)." + err "See: https://distrobox.it/#installation" return 1 fi ok "distrobox $(distrobox version 2>/dev/null | head -1) detected" diff --git a/templates/bar-vscode-settings.json b/templates/bar-vscode-settings.json index 173ffe92..c6809063 100644 --- a/templates/bar-vscode-settings.json +++ b/templates/bar-vscode-settings.json @@ -5,7 +5,21 @@ "**/.lux": false, "**/.lux/**": false }, + "search.exclude": { + "**/.lux": true, + "**/.lux/**": true, + "**/.devtools": true, + "**/.devtools/**": true, + "**/common/luaUtilities/**": true + }, "[lua]": { - "editor.defaultFormatter": "JohnnyMorganz.stylua" - } + "editor.defaultFormatter": "JohnnyMorganz.stylua", + "editor.formatOnSave": true + }, + "test-switcher.rules": [ + { "pattern": "spec/(.*)_spec\\.lua", "replacement": "$1.lua" }, + { "pattern": "spec/builder_specs/(.*)_spec\\.lua", "replacement": "spec/builders/$1.lua" }, + { "pattern": "spec/builders/(.*)\\.lua", "replacement": "spec/builder_specs/$1_spec.lua" }, + { "pattern": "(luarules|common|luaui|gamedata)/(.*)\\.lua", "replacement": "spec/$1/$2_spec.lua" } + ] }