@bh/model-validation by barhanc · Pull Request #1324 · software-mansion/react-native-executorch

barhanc · 2026-07-21T02:58:47Z

Description

Summary Proof of Concept

Introduces a typed dimension model (Static / Dynamic / Constant) on the JS side and a companion-method parser on the C++ side so models can declare runtime-enforced dynamic shape constraints (min/max/step ranges or enumerated shape lists) that ExecuTorch's .pte metadata does not otherwise expose.

Changes

modelSchema.ts — Replaces loose number | string shape dims with a tagged union (StaticDim | DynamicDim | ConstantDim) and Static() / Dynamic() / Constant() constructors. SymbolicTensor normalizes plain numbers/strings to the typed form internally, so existing call sites keep working. Dynamic('L') additionally asserts the .pte exports a matching companion method.
model_companion_parser.{h,cpp} — Extracts parseDynamicInputShapes (formerly inlined in model.cpp) and adds parseEnumeratedInputShapes for the get_enumerated_dims_<method> variant. Same tensor-shape contract ([rank, 3] int32 for ranges; [num_shapes, rank] int32 for enums) — kept identical to avoid a breaking change to the Python export convention.
tensor_helpers.h — Introduces ShapeConstraint = variant<SymbolicShape, vector<SymbolicShape>> and a routing fromJs overload so a single constraints map can hold either range or enum shapes without leaking the distinction into Model.
model.{h,cpp} — Renames dynamicInputShapes_ → inputShapeConstraints_; rejects methods that define both companion flavors as mutually exclusive.
Extensions — textEmbedding and fsmnVoiceActivityDetection adopt Dynamic(...) to declare the symbolic dims that already required companion methods in practice.

Design rationale

Tagged union over plain primitives on the JS side makes the intent of a dimension (named wildcard vs. companion-method-backed dynamic vs. literal) plain at the call site, while toTypedShape keeps the public API source-compatible.
Companion-method approach (vs. parsing .pte metadata) is the only way to recover min/max/step ranges today — ExecuTorch serializes only the static upper bound. Reusing it for enumerated shapes keeps a single extension mechanism.
ShapeConstraint variant + routing overload centralizes range/enum dispatch in tensor_helpers rather than model.cpp, so Model stays agnostic of the constraint flavor and future constraint types stay pluggable.
Mutual-exclusion guard prevents ambiguous exports (both get_dynamic_dims_* and get_enumerated_dims_*) at load time rather than at validation time.

Load-time (JS) vs. runtime (C++)

This split is intentional because the .pte metadata only serializes the static upper bounds of dynamic dimensions — the actual [min, max, step] ranges and enumerated shape lists are only retrievable by executing the companion method, and the C++ layer has to run them anyway.

Load-time (validateModelSchema): pure metadata check run once when a schema is bound to a model in loadModel. Verifies tag counts, dtype, static shape signature, and — new in this branch — asserts that any input shape using Dynamic(...) has a corresponding companion method present in model.getMethodNames(). Surfaces misconfigured schemas (Dynamic without a companion, or non-matching shapes) as a hard error before any forward call instead of silently degrading to static-upper-bound-only behavior.
Runtime (Model ctor + per-call tensor::fromJs): the Model constructor executes each companion method once, caches the parsed ShapeConstraints into inputShapeConstraints_, and rejects methods that declare both get_dynamic_dims_* and get_enumerated_dims_*. On every forward, fromJs resolves the per-input constraint and validates the concrete incoming tensor against the actual dynamic range or enumerated list — this is where the real numeric bounds get enforced, not in JS.

Why validation isn't duplicated at load-time

The companion methods could be executed from JS at validate-time (the module is loaded by then, and the cost is negligible), but doing so duplicates work the runtime already does:

JS validation is declarative. validateModelSchema is intentionally a pure function of (modelMetadata, schema). Dynamic(...) is a promise the schema makes about the model export ("this dim is genuinely dynamic and a companion exists"), not a specification of numeric values the schema enforces. The schema layer only needs to confirm the promise is honored — which a methodNames.includes(...) check already does — and let the runtime enforce the actual numbers.
No duplicated work. C++ Model must parse the companion output into inputShapeConstraints_ regardless, since fromJs enforces per-input bounds on every forward. Executing and parsing the same companions again from JS would establish a second representation of the same constraint with no mechanism to keep them in sync across rebinds/reloads.
Clear runtime errors. C++ runtime validation already produces actionable error messages (range violations, dtype mismatch, shape mismatch) before the forward hits ExecuTorch. There is no debugging-information gap that running the companions at validate-time would close.

Known limitation: cross-tensor constraints

Per-tensor runtime validation is not complete — fromJs validates each input tensor against its own ShapeConstraint independently and cannot enforce consistency of a shared symbolic dim across multiple inputs. Concretely, passing tokens and mask of mismatched L to textEmbedding bypasses our validators and surfaces as an opaque ExecuTorch internal error deep in the forward.

Introduces a breaking change?

Yes
No

Type of change

Bug fix (change which fixes an issue)
New feature (change which adds functionality)
Documentation update (improves or adds clarity to existing documentation)
Other (chores, tests, code style improvements etc.)

Tested on

iOS
Android

Testing instructions

Build and run all example apps to check that there are no regressions.

Screenshots

Related issues

#1323

Checklist

I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have updated the documentation accordingly
My changes generate no new warnings

Additional notes

- Add DynamicDim type and Dynamic() constructor for dimensions that require companion methods - Add Static(), Constant() constructors and tagged union types (StaticDim, DynamicDim, ConstantDim) - Update SymbolicTensor to convert plain numbers/strings to typed dimensions internally - Add companion method validation in validateModelSchema (get_dynamic_dims_*, get_enumerated_dims_*) - Extract parseDynamicInputShapes and parseEnumeratedInputShapes to model_companion_parser.h/cpp - Use Dynamic('L') in textEmbedding and Dynamic('frames') in fsmnVoiceActivityDetection - Clean up index.ts exports to use export * for core modules

…JSDoc

msluszniak · 2026-07-21T13:24:32Z

I guess that cross-tensor validation is important and ultimately we want it, but for now it's ok to skip it. Just make sure we have some follow-up issue for this kind of check.

msluszniak

Only cpp side for now

msluszniak · 2026-07-21T13:47:02Z

+            if (maxDim > inputMeta.sizes()[axis]) {
+                throw std::runtime_error(std::format("{}output[{}], axis {} max dimension ({}) "
+                                                     "exceeds model metadata upper limit ({})",
+                                                     ctx, tensorIndex, axis, maxDim, inputMeta.sizes()[axis]));
+            }


How about the analogical check for minDim?

msluszniak · 2026-07-21T13:48:33Z

+    using executorch::aten::ScalarType;
+
+    const auto getEnumeratedShapesMethodName = std::format("get_enumerated_dims_{}", methodName);
+    const auto ctx = getEnumeratedShapesMethodName + ": ";
+
+    auto methodNames = unwrap(ctx + "failed to get method names", module.method_names());
+    if (methodName == getEnumeratedShapesMethodName ||
+        !methodNames.contains(getEnumeratedShapesMethodName)) {
+        return std::nullopt;
+    }
+
+    auto methodMeta = unwrap(std::format("{}failed to get meta for method '{}'", ctx, methodName),
+                             module.method_meta(methodName));
+
+    size_t expectedTensorInputs = 0;
+    for (size_t i = 0; i < methodMeta.num_inputs(); ++i) {
+        auto tag = unwrap(std::format("{}failed to get tag for input [{}]", ctx, i),
+                          methodMeta.input_tag(i));
+        if (tag == executorch::runtime::Tag::Tensor) {
+            expectedTensorInputs++;
+        }
+    }
+
+    auto result = unwrap(ctx + "failed to execute", module.execute(getEnumeratedShapesMethodName));
+    if (result.size() != expectedTensorInputs) {
+        throw std::runtime_error(std::format("{}number of outputs returned ({}) does not match the number of "
+                                             "tensor inputs declared by method '{}' ({})",
+                                             ctx, result.size(), methodName, expectedTensorInputs));
+    }


Maybe we should abstract common part of these two into function in anonymous helper?

msluszniak · 2026-07-21T14:54:10Z

+    for (size_t i = 0; i < expectedShape.size(); ++i) {
+        const auto &dim = expectedShape.at(i);
+
+        if (std::holds_alternative<int32_t>(dim)) {


Maybe use std::visit here instead of if else checks?

msluszniak · 2026-07-21T14:56:22Z

+inline std::shared_ptr<TensorHostObject>
+fromJs(jsi::Runtime &rt, const std::string &name, const jsi::Value &value,
+       std::optional<DType> expectedDtype, const ShapeConstraint &shapeConstraint) {
+    if (std::holds_alternative<SymbolicShape>(shapeConstraint)) {


Consider std::visit here, definitely not as useful as in tge previous case

barhanc · 2026-07-21T23:25:12Z

@msluszniak I'm going to close this PR because I came to a conclusion (based on exhaustive experiments with different API designs) that this is not an ideal way of solving the validation problem. I think we should actually export static method in .pte files which would return JSON string with full info about model schema. I would like to discuss the details whenever you're free.

msluszniak · 2026-07-22T09:05:44Z

Let's discuss it today! :))

barhanc added 2 commits July 21, 2026 03:56

small fix

d8dce6a

barhanc changed the title ~~Bh/validation~~ @bh/model-validation Jul 21, 2026

barhanc self-assigned this Jul 21, 2026

barhanc requested review from benITo47 and msluszniak July 21, 2026 03:01

docs: consolidate companion-method convention in validateModelSchema …

0f1a265

…JSDoc

msluszniak reviewed Jul 21, 2026

View reviewed changes

barhanc closed this Jul 21, 2026

Navigation Menu

Uh oh!

@bh/model-validation - #1324

@bh/model-validation#1324
barhanc wants to merge 3 commits into
rne-rewritefrom
bh/validation

barhanc commented Jul 21, 2026

Uh oh!

msluszniak commented Jul 21, 2026

Uh oh!

msluszniak left a comment

Uh oh!

msluszniak Jul 21, 2026

Uh oh!

msluszniak Jul 21, 2026

Uh oh!

msluszniak Jul 21, 2026

Uh oh!

msluszniak Jul 21, 2026

Uh oh!

barhanc commented Jul 21, 2026

Uh oh!

msluszniak commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

barhanc commented Jul 21, 2026

Description

Summary Proof of Concept

Changes

Design rationale

Load-time (JS) vs. runtime (C++)

Why validation isn't duplicated at load-time

Known limitation: cross-tensor constraints

Introduces a breaking change?

Type of change

Tested on

Testing instructions

Screenshots

Related issues

Checklist

Additional notes

Uh oh!

msluszniak commented Jul 21, 2026

Uh oh!

msluszniak left a comment

Choose a reason for hiding this comment

Uh oh!

msluszniak Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

msluszniak Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

msluszniak Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

msluszniak Jul 21, 2026

Choose a reason for hiding this comment

Uh oh!

barhanc commented Jul 21, 2026

Uh oh!

msluszniak commented Jul 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants