Skip to content

Commit 0008582

Browse files
authored
examples/arduino: Make the generated Arduino library actually run models (#21546)
## Summary The Arduino library this directory generates could not compile, and had it compiled it could not have run a model. Kernels never reached the operator registry — ExecuTorch registers them through codegen and the build script never ran it, so every `Method::load` would have failed with `OperatorMissing`. CMSIS-NN was never vendored, so the Cortex-M ops shipped without the library they call. `schema/*.cpp` was never copied, the `*_aten.cpp` exclusion also deleted the portable-mode `tensor_parser_exec_aten.cpp`, and `__errno` was missing because the Zephyr core mixes picolibc with newlib's `libm_nano`. Both `minimal.cpp` and `zephyr.cpp` were vendored, so which `et_pal_*` backend you got depended on link order, and every `ET_LOG` was discarded either way. One backend now ships and its logs route to a hook the examples implement against `Serial`. The examples ship their models — previously none did, so every sketch `#error`ed when opened from the IDE menu. The README pointed at the Ethos-U `pte_to_header.py`, whose `network_model_sec` section no Arduino core defines, so following the docs produced a model that fails `Program::load`. Static link mode is mandatory and undocumented; the Dynamic default yields a silently dead board. Renamed to `ExecuTorch` because `arduino-lint` rejects "Arduino" in an Arduino library's name. Registering every portable kernel costs 1.58 MB against 786 KB of flash, so the op set is a curated default overridable via `ROOT_OPS`/`ALL_OPS`. Models and libraries must come from the same ExecuTorch commit — Cortex-M schemas change (`scratch` in #19636, #19825), and a mismatch loads fine, resolves every operator, then fails at `Method::execute`. The library now records and pins that commit. CI to enforce it follows separately. ## Test plan `arduino:zephyr:unoq`, board core 0.55.2, `link_mode=static`, flashed on hardware: ``` HelloExecuTorch Model loaded OK!, 1 method 60% flash, 20% RAM AddModel [1,2,3] + 1 = [2.00, 3.00, 4.00] 64% flash, 26% RAM KeywordSpotting 10/10 keywords correct 70% flash, 53% RAM ``` All ten MFCC inputs in one sketch, exercising the CMSIS-NN conv / depthwise / avgpool / linear kernels: ``` yes 8.95 no 4.78 up 4.63 down 9.72 left 7.87 right 7.41 on 12.03 off 8.02 stop 8.64 go 9.57 ``` `arduino-lint --library-manager submit`: no errors, no warnings, under both `specification` and `strict`. Clean regeneration is byte-identical across all 626 generated files. Only the Uno Q was tested; the other three boards remain marked Planned. Authored with Claude Code (Opus 5).
1 parent ad3a71f commit 0008582

15 files changed

Lines changed: 590 additions & 54 deletions

File tree

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,10 @@ tokenizer.json
3737
*.ptd
3838
!test_bpe_tokenizer.bin
3939
!test_tiktoken_tokenizer.model
40+
# Arduino examples ship a model, so build_arduino_library.sh has something to
41+
# turn into the model.h their sketches include. 1.1 KB for the add models,
42+
# 53 KB for the keyword spotting DS-CNN.
43+
!examples/arduino/examples/*/model.pte
4044

4145
# Editor temporaries
4246
*.idea

examples/arduino/README.md

Lines changed: 186 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -15,21 +15,30 @@ Arduino library. A build script vendors the runtime sources from this
1515
repository into a self-contained library that Arduino users install
1616
through the Library Manager or by copying into their libraries folder.
1717

18+
> **Who this is for.** This README is for maintainers of the packaging.
19+
> `build_arduino_library.sh` is a release tool, not something an Arduino
20+
> developer ever runs. Users install a prebuilt library from the Library
21+
> Manager and never see this directory; their documentation lives in
22+
> [meta-pytorch/executorch-arduino](https://github.com/meta-pytorch/executorch-arduino).
23+
>
24+
> If you are here to change ExecuTorch and want to know whether you broke
25+
> the Arduino library, read [Keeping this working](#keeping-this-working).
26+
1827
## How It Works
1928

2029
```
2130
PyTorch Model ──► torch.export ──► .pte file ──► model.h (C array)
2231
2332
Arduino Sketch (.ino)
24-
#include <ExecuTorchArduino.h>
33+
#include <ExecuTorch.h>
2534
#include "model.h"
2635
2736
arduino-cli compile ──► Upload ──► Runs on board
2837
```
2938

3039
### The three pieces
3140

32-
1. **The library** (`arduino_lib/ExecuTorchArduino/`) — the ExecuTorch
41+
1. **The library** (`arduino_lib/ExecuTorch/`) — the ExecuTorch
3342
runtime, CMSIS-NN kernels, and portable ops packaged for the Arduino
3443
build system. Generated by `build_arduino_library.sh`; not checked in.
3544

@@ -65,34 +74,34 @@ cd examples/arduino
6574
```
6675

6776
This copies the required ExecuTorch sources from the repository into
68-
`arduino_lib/ExecuTorchArduino/`, ready for Arduino.
77+
`arduino_lib/ExecuTorch/`, ready for Arduino.
6978

7079
### 2. Install the library
7180

7281
Copy the generated library into your Arduino libraries folder:
7382

7483
```bash
7584
# macOS:
76-
cp -r arduino_lib/ExecuTorchArduino ~/Documents/Arduino/libraries/
85+
cp -r arduino_lib/ExecuTorch ~/Documents/Arduino/libraries/
7786
# Linux:
78-
cp -r arduino_lib/ExecuTorchArduino ~/Arduino/libraries/
87+
cp -r arduino_lib/ExecuTorch ~/Arduino/libraries/
7988
```
8089

8190
Or with `arduino-cli`:
8291

8392
```bash
84-
cd arduino_lib && zip -r ExecuTorchArduino.zip ExecuTorchArduino && cd ..
85-
arduino-cli lib install --zip-path arduino_lib/ExecuTorchArduino.zip
93+
cd arduino_lib && zip -r ExecuTorch.zip ExecuTorch && cd ..
94+
arduino-cli lib install --zip-path arduino_lib/ExecuTorch.zip
8695
```
8796

8897
### 3. Export a model
8998

9099
Each sketch needs a `model.h` file — a `.pte` model converted to a C
91-
byte array. Use `pte_to_header.py` from the Arm examples to convert
100+
byte array. Use `pte_to_header.py` to convert
92101
any `.pte` file:
93102

94103
```bash
95-
python examples/arm/executor_runner/pte_to_header.py \
104+
python examples/arduino/pte_to_header.py \
96105
-p model.pte -d examples/arduino/examples/AddModel -o model.h
97106
```
98107

@@ -108,7 +117,7 @@ class Add(torch.nn.Module):
108117
et = to_edge(export(Add().eval(), (torch.tensor([1.,2.,3.]),))).to_executorch()
109118
with open('add.pte','wb') as f: f.write(bytes(et.buffer))"
110119

111-
python examples/arm/executor_runner/pte_to_header.py \
120+
python examples/arduino/pte_to_header.py \
112121
-p add.pte -d examples/arduino/examples/AddModel -o model.h
113122
```
114123

@@ -150,7 +159,7 @@ cmake build. If you haven't built ExecuTorch yet, run
150159
### 4. Write a sketch
151160

152161
```cpp
153-
#include <ExecuTorchArduino.h>
162+
#include <ExecuTorch.h>
154163
#include "model.h"
155164

156165
using executorch::extension::BufferDataLoader;
@@ -196,11 +205,74 @@ Arduino-specific abstractions.
196205
### 5. Compile and upload
197206

198207
```bash
199-
arduino-cli compile --fqbn arduino:zephyr:unoq MySketch
200-
arduino-cli upload --fqbn arduino:zephyr:unoq -p /dev/cu.usbmodem* MySketch
208+
arduino-cli compile --fqbn arduino:zephyr:unoq:link_mode=static MySketch
209+
arduino-cli upload --fqbn arduino:zephyr:unoq:link_mode=static -p /dev/cu.usbmodem* MySketch
201210
arduino-cli monitor -p /dev/cu.usbmodem* --config baudrate=115200
202211
```
203212

213+
## Keeping this working
214+
215+
The library is a *generated artifact*. Everything under the generated
216+
`src/` is copied out of this repository, and the example models are
217+
exported by this repository's Python. That gives one failure mode, and it
218+
has cost multiple days:
219+
220+
**The model and the library must come from the same ExecuTorch commit.**
221+
222+
Cortex-M operator schemas change. `scratch` was added to the conv operators
223+
on 2026-06-09 and to `avg_pool2d` later still. A `.pte` exported before a
224+
schema change passes `Program::load`, resolves every operator, and then
225+
fails inside `Method::execute` with `InvalidProgram (0x23)`, because the
226+
generated kernel wrapper expects one more argument than the model supplies.
227+
Nothing about that error names the real cause.
228+
229+
This bites hardest when the Python package and the C++ sources come from
230+
different places. `pip install executorch` gives a release wheel that can be
231+
months behind this checkout; the library you build here is current. Check
232+
which one you are exporting with:
233+
234+
```bash
235+
python -c "import executorch.backends.cortex_m.ops.operators as o; print(o.__file__)"
236+
```
237+
238+
If that prints a `site-packages` path rather than your checkout, run
239+
`./install_executorch.sh` first. Note that ExecuTorch refuses to build from a
240+
directory not named exactly `executorch` (pytorch/executorch#6475), which is
241+
a common reason people end up on a stale wheel without realising.
242+
243+
To check a model against a library without a board, decode the `.pte` and
244+
compare each `KernelCall`'s argument count against the `stack.size() == N`
245+
in the generated `src/executorch/codegen/RegisterCodegenUnboxedKernels*.cpp`.
246+
A mismatch there is the bug, found in seconds instead of hours.
247+
248+
### Things that are not obvious
249+
250+
- **`link_mode=static` is mandatory.** The Uno Q defaults to Dynamic, which
251+
builds the sketch as a Zephyr loadable extension. A library this size never
252+
starts that way: no serial output at all, so the board looks dead and offers
253+
nothing to diagnose. Dynamic also reports only the extension's size, roughly
254+
half the real figure.
255+
- **`ET_LOG` has to be routed somewhere.** `zephyr.cpp` logs through `fprintf`,
256+
and `platform_stubs.c` stubs `fprintf` out. The build script rewrites the
257+
logger to call a weak `et_arduino_log` hook, which the examples implement
258+
against `Serial`. Without it every runtime failure is a bare hex code.
259+
- **Only one platform backend may ship.** `minimal.cpp` and `zephyr.cpp` both
260+
define `et_pal_*`; shipping both leaves the choice to link order, and
261+
`minimal`'s logger is empty and its allocator returns `nullptr`.
262+
- **Compiling proves very little.** Every failure worth finding here compiled
263+
cleanly first. Flash a board.
264+
265+
### Error codes seen in practice
266+
267+
| Symptom | Cause |
268+
|---|---|
269+
| No serial output at all | Built in Dynamic link mode, or `Arduino_RouterBridge` missing |
270+
| `Program::load` -> `0x23` | Model header put the array in a section the linker discards; use `pte_to_header.py` from this directory, not the Ethos-U one |
271+
| `load_method` -> `0x14` | Operator not in the registered set; regenerate with `ROOT_OPS=` |
272+
| `load_method` -> `0x21` | `method_pool` too small; the log line gives the exact shortfall |
273+
| `execute` -> `0x23` | Model and library built from different ExecuTorch commits |
274+
275+
204276
## What is inside the library
205277

206278
The `build_arduino_library.sh` script assembles these components from
@@ -229,31 +301,66 @@ Arduino's build system:
229301

230302
2. **`cmake_macros.h` stub** — c10/torch headers expect a cmake-generated
231303
file. The build script generates a stub; `C10_USING_CUSTOM_GENERATED_MACROS`
232-
is defined in `ExecuTorchArduino.h` to skip the include.
304+
is defined in `ExecuTorch.h` to skip the include.
233305

234306
3. **`platform_stubs.c`** — provides weak stubs for `_Exit()`, `fprintf()`,
235307
and `__aeabi_f2lz` for the LLEXT environment on boards that lack them.
236308

237-
4. **Compile-time defines**`ExecuTorchArduino.h` sets
309+
4. **Compile-time defines**`ExecuTorch.h` sets
238310
`ET_ENABLE_DEPRECATED_CONSTANT_BUFFER=0` (requires models exported with
239311
current ExecuTorch) and `FLATBUFFERS_MAX_ALIGNMENT=1024`.
240312

241313
## Development
242314

243315
### Updating the library
244316

245-
After modifying ExecuTorch sources, regenerate the library:
246-
247317
```bash
248-
./build_arduino_library.sh # rebuild
318+
./build_arduino_library.sh # rebuild
249319
./build_arduino_library.sh --clean # remove generated output
320+
ROOT_OPS="aten::add.out,..." ./build_arduino_library.sh # pick the op set
321+
ALL_OPS=1 ./build_arduino_library.sh # every portable op
322+
```
323+
324+
The op set is a size decision. Registering every portable kernel costs about
325+
1.6 MB of text, twice the Uno Q's flash, because portable kernels are
326+
dtype-templated. The default registers the Cortex-M operators plus a small
327+
portable set, which lands around a quarter of flash.
328+
329+
### Re-exporting the example models
330+
331+
Each example ships a `model.pte` that the build script converts to the
332+
`model.h` its sketch includes. Regenerate them whenever an operator schema
333+
changes, or the models will fail at `execute` against the new runtime:
334+
335+
```bash
336+
# keyword spotting, from the checked-in checkpoint (no retraining)
337+
python export_model.py --checkpoint examples/KeywordSpotting/model.pth \
338+
--output /tmp/kws.h
250339
```
251340

341+
### Bumping the pin in executorch-arduino
342+
343+
The published library records the commit it was generated from in
344+
`extras/PROVENANCE.txt`, and pins that commit in `executorch_pin.txt`
345+
alongside it — the same one-SHA-per-file convention ExecuTorch uses in
346+
`.ci/docker/ci_commit_pins/`. To move it forward:
347+
348+
1. Update `executorch_pin.txt` to the new ExecuTorch commit
349+
2. Regenerate the library from a checkout at that commit
350+
3. Re-export the example models from the same checkout
351+
4. Confirm each model's `KernelCall` argument counts match the regenerated
352+
`RegisterCodegenUnboxedKernels*.cpp`
353+
5. Compile every example at `link_mode=static`, and flash at least one
354+
355+
Steps 2 and 3 have to happen together. Bumping the library without
356+
re-exporting the models is the mismatch described in
357+
[Keeping this working](#keeping-this-working).
358+
252359
### Testing
253360

254361
```bash
255-
arduino-cli compile --fqbn arduino:zephyr:unoq examples/HelloExecuTorch
256-
arduino-cli upload --fqbn arduino:zephyr:unoq -p /dev/cu.usbmodem* examples/HelloExecuTorch
362+
arduino-cli compile --fqbn arduino:zephyr:unoq:link_mode=static examples/HelloExecuTorch
363+
arduino-cli upload --fqbn arduino:zephyr:unoq:link_mode=static -p /dev/cu.usbmodem* examples/HelloExecuTorch
257364
arduino-cli monitor -p /dev/cu.usbmodem* --config baudrate=115200
258365
```
259366

@@ -325,20 +432,71 @@ Training and test audio from [Google Speech Commands v2](https://arxiv.org/abs/1
325432
people. Standard dataset used by the MLPerf Tiny benchmark. Download
326433
via `torchaudio.datasets.SPEECHCOMMANDS` (2.3 GB).
327434

435+
The dataset is © Google, released under
436+
[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), which asks for
437+
attribution. The keyword spotting weights checked in here
438+
(`examples/KeywordSpotting/model.pth` and the `.pte` generated from it) are
439+
trained on it and carry the same attribution.
440+
441+
Only the ten keyword classes are needed, so the full archive never has to
442+
land on disk:
443+
444+
```bash
445+
mkdir -p outputs/speech_commands/SpeechCommands/speech_commands_v0.02
446+
cd outputs/speech_commands/SpeechCommands/speech_commands_v0.02
447+
curl -sL http://download.tensorflow.org/data/speech_commands_v0.02.tar.gz \
448+
| tar xz ./yes ./no ./up ./down ./left ./right ./on ./off ./stop ./go
449+
```
450+
451+
That is 1.2 GB extracted instead of 2.3 GB downloaded plus 2.4 GB unpacked.
452+
453+
`download.tensorflow.org` serves no usable HTTPS -- its certificate does not
454+
cover that hostname -- which is why the URL is plain HTTP and why
455+
`torchaudio.datasets.SPEECHCOMMANDS` uses HTTP for it too. If transport
456+
integrity matters, download the archive first and check it against the SHA-256
457+
torchaudio pins for v0.02 before extracting:
458+
459+
```bash
460+
af14739ee7dc311471de98f5f9d2c9191b18aedfe957f4a6ff791c709868ff58
461+
```
462+
463+
You only need this to retrain. The exported model and its checkpoint are both
464+
checked in, so nothing here is required to build the library.
465+
328466
The DS-CNN KWS benchmark uses 12 output classes (silence, unknown, plus
329467
10 keywords). The Arduino export script trains the 10 keyword classes:
330468
yes, no, up, down, left, right, on, off, stop, go.
331469

332-
## LLEXT Memory Budget
470+
## Link Mode and Memory Budget
471+
472+
This applies to the Zephyr board core, which is the only core the library
473+
currently supports (`architectures=zephyr`). Other Arduino cores do not run
474+
Zephyr and have no link mode setting; they need a platform abstraction layer
475+
port before they can compile at all, and their memory behaviour is untested.
333476

334-
The Arduino Uno Q loads sketches as LLEXT (Loadable Extensions).
335-
Sizes reported by `arduino-cli compile` (Zephyr board core 0.55.2):
477+
On the Zephyr core, the Uno Q defaults to Dynamic link mode, which builds the
478+
sketch as a Zephyr loadable extension. Sketches this size never start that way: no serial output
479+
at all, so the board looks dead and offers nothing to diagnose. Build with
480+
`link_mode=static`. A 2 KB sketch runs fine under Dynamic, so the ceiling sits
481+
somewhere between that and these builds; it has not been pinned down.
336482

337-
| Build | Code | Data | Total | Status |
338-
|-------|------|------|-------|--------|
339-
| HelloExecuTorch (portable ops) | 62 KB | 27 KB | 89 KB ||
340-
| Add model (portable ops) | 88 KB | 35 KB | 123 KB ||
341-
| DS-CNN (selective CMSIS-NN) | 87 KB | 57 KB | 144 KB ||
483+
Dynamic also reports only the extension's own size, which reads far lower than
484+
what the board actually holds. Measured on an Arduino Uno Q, board core 0.55.2,
485+
against 786,432 bytes of flash and 131,072 bytes of RAM:
486+
487+
| Build | Flash (static) | RAM | Dynamic reported | On hardware |
488+
|-------|---------------|-----|------------------|-------------|
489+
| HelloExecuTorch | 472,728 (60%) | 3,060 (2%) | 27% | `Model loaded OK!`, 1 method |
490+
| AddModel | 507,664 (64%) | 11,252 (8%) | 30% | `[1,2,3] + 1 = [2.00, 3.00, 4.00]` |
491+
| KeywordSpotting (CMSIS-NN) | 557,520 (70%) | 46,068 (35%) | 30% | 10/10 keywords correct |
342492

343493
All CMSIS-NN sources are compiled, but the linker's
344494
`--gc-sections` discards unused functions from the final binary.
495+
496+
RAM is the binding constraint, not flash. Zephyr reserves 32 KB of main stack
497+
and a 32 KB heap out of 128 KB before the sketch gets any, and the arena the
498+
sketch hands to `MemoryManager` comes out of what remains. KeywordSpotting's
499+
DS-CNN plans 16 KB of buffers but needs considerably more for the method's own
500+
structures: a 28 KB arena fails `load_method` with `MemoryAllocationFailed`
501+
(0x21), and a 64 KB one loads but then fails `execute` with `InvalidProgram`
502+
(0x23), which is memory being overrun rather than a malformed program.

0 commit comments

Comments
 (0)