Decode and encode G17 instructions, verified by making the GPU compute a prediction

[AndreSlavescu]

The instruction set is undocumented, so the method is the substance. Reading bytes and
inferring field layouts produces plausible maps that are wrong, so every claim here was instead
put to the GPU: patch the machine code inside a binary archive, run it, and check the answer
against a number worked out beforehand.

metile/target/agx_execute.swift runs a kernel from an archive with failOnBinaryArchiveMiss,
which makes the archive the only permitted source of machine code. Without that flag the driver
quietly recompiles the MSL, a patched kernel and an unpatched one agree, and the experiment
proves nothing.

What that establishes

Edited code runs. The driver executes an edited __text and does not re-derive it from the
AIR it also carries.

Boundaries, found behaviourally. Nopping a byte range and running it locates instruction
starts. On four dependent a = fma(a, 2, 1) steps, only 0x5a, 0x62, 0x6a, 0x72 return 15
instead of 31 — exactly one fma removed at each. Four instructions, eight-byte stride. Other
alignments return 0, 1, 7 or 225: they run, and lie, which is why the test asserts the value
rather than that the kernel survived.

The immediate field is (e << 4) | (m << 1) | low, holding (1 + m/8) · 2^(e−11).

value	byte	value	byte
2.0	0xc1	1.0	0xb0
3.0	0xc9	3.0	0xc8
4.0	0xd1	5.0	0xd4
8.0	0xe1

And it encodes, not just decodes — the part that distinguishes understanding from a lookup
table. Predictions made before assembling the bytes, for constants no Metal compiler produced:

rewrite	predicted	measured
3 fmas → a*6+7	949	949
3 fmas → a*1.25+1.5	11.578125	11.578125
1 fma → a*6+7	103	103

Field map for the compact fma

Each byte established by patching it and running: byte 0 opcode (low nibble 9); byte 2 operand
mode plus a last-instruction flag — 0x03 there turns a*2+1 into a*a, giving 225 from an
accumulator holding 15; bit 0x20 of byte 6 disables the instruction; bytes 1 and 7 select
registers. Byte 4 was not probed, and says so.

Deliberately not claimed

No general disassembler — one form is mapped and decode reports the rest as unknown rather than
inventing mnemonics. And negative immediates have no known encoding in this field, so
encode_immediate refuses them instead of returning the nearest byte, which would corrupt a
kernel silently.

Two of my own errors, since both looked like the encoding failing

1. A confirmed boundary is not a known form. All four fmas sit on an eight-byte stride, but
   the compiler encodes the first differently, and reading the constant field there returned
   0.000488. Being a boundary is measured; being the mapped form is what the opcode nibble
   decides. The probe now separates the two.
2. The first prediction missed — 97 against a measured 103 — and the encoding was right. The
   patched instruction was the second fma of four, not the last. Predictions now replay the
   arithmetic from the actual instruction positions.

benchmarks/agx_isa_probe.py re-derives all four stages from scratch and exits non-zero if any
prediction misses, which is how to port this to different hardware or a newer toolchain.

624 pass. Lint and vulture clean.

🤖 Generated with Claude Code