[RNE Rewrite] feat: add privacy filter pipeline by msluszniak · Pull Request #1321 · software-mansion/react-native-executorch

msluszniak · 2026-07-20T12:45:03Z

Description

Ports the privacy filter (token-level PII detection) to the new flow.

createPrivacyFilter — validates forward with validateSpec ([RNE Rewrite] refactor!: add better model schema contract and validation logic #1327) against the configured label space, pre-allocates the window tensors, and runs sliding windows with overlap.
privacyFilterUtils — BIOES grammar construction, constrained Viterbi decode, span extraction.
usePrivacyFilter hook, models.privacyFilter registry (openai/nemotron × XNNPACK/MLX), BIOES label constants.
Privacy filter demo screen in the NLP example app.

forward is declared as two spec variants: dynamic, matching the XNNPACK exports whose sequence dim binds to the exported range, and static, matching the MLX exports whose sequence dim binds to a single constant. The window size falls out of whichever variant matched. The sequence length shared by both inputs and the logits is declared as an equality runtime constraint, and the length buckets are snapped onto the exported range's grid.

Introduces a breaking change?

Yes
No

Type of change

Bug fix (change which fixes an issue)
New feature (change which adds functionality)
Documentation update (improves or adds clarity to existing documentation)
Other (chores, tests, code style improvements etc.)

Tested on

iOS
Android

Testing instructions

Needs the v0.10.0 XNNPACK models re-exported from @bh/new-schema on export-scripts, which swaps get_dynamic_dims_forward for a get_model_schema companion. The MLX models are unchanged.

cd apps/nlp && yarn ios (or yarn android).
Open the Privacy Filter screen.
Pick a model chip (OpenAI 8 types / Nemotron 55 types; MLX variants are iOS-only) and tap Detect PII.
Detected spans are highlighted inline and listed with label and token range.

Related issues

Closes #1246

Checklist

I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have updated the documentation accordingly
My changes generate no new warnings

Additional notes

Viterbi decode runs in O(numLabels + numEntities) per token instead of O(numLabels²), cutting a ~27% CPU overhead on Nemotron's large (221) label space.

msluszniak · 2026-07-23T15:07:12Z

This one needs to implement input shape validation on JS side, so it will wait till the PR with input validation. However, the core of the PR might be reviewed cc: @barhanc

Port the privacy filter (token-level PII detection) to the new TypeScript-orchestrated architecture. The current implementation runs windowing, constrained Viterbi decoding and span extraction in C++ (`models/privacy_filter/`). The transformer forward pass dominates the inference budget, so per the Amdahl's-law rule all of that moves to TypeScript, leaving the native layer untouched. - `createPrivacyFilter` task: validates the `forward` signature against the configured label space, pre-allocates the static window tensors, and runs sliding windows with 50% overlap (no truncation), keeping the more centred prediction near window boundaries. - `privacyFilterUtils`: BIOES grammar construction, constrained Viterbi decode, and span extraction. - `usePrivacyFilter` hook, `models.privacyFilter` registry entries (openai/nemotron, XNNPACK + MLX), and the BIOES label constants. - Privacy filter demo screen in the NLP example app. Closes #1246

On-device profiling showed the dense transition loop dominated the non-native cost for large label spaces: openai ( 33 labels): viterbi 6ms / 710ms total — ~1% of detect nemotron (221 labels): viterbi 264ms / 970ms total — ~27% of detect `model.execute` costs the same ~705ms for both models, so the entire gap was the decode: 44x more time for a 6.7x bigger label set, i.e. the expected N^2 blowup. Under BIOES the max over predecessors collapses into a few shared group maxima — every `O`/`B-`/`S-` target sees the same two candidates (best background state, best `E-`/`S-` state), and each `I-x`/`E-x` has exactly two predecessors (`B-x`, `I-x`). Storing the grammar as those groups rather than an N x N matrix makes each step O(numLabels + numEntities). Nemotron decode drops 264ms -> 5ms on device (~27% -> ~0.9% of detect, total 970ms -> 708ms), keeping the pipeline in TypeScript rather than pushing it into C++. Verified equivalent: identical-score paths vs the previous dense implementation across both label spaces, two bias sets and lengths 1-512, and still optimal + grammar-valid against a brute-force reference. Also declare the privacyFilter feature in the NLP example app so its native backend is provisioned.

The privacy filter ships MLX iOS variants (OPENAI.MLX_INT4, NEMOTRON.MLX_INT8) but the feature-to-backend map only requested xnnpack, so MLXBackend.xcframework was never downloaded or force-loaded and the MLX delegate failed at execute.

Point the privacy filter at v0.10.0, whose xnnpack exports carry a get_dynamic_dims_forward companion. When present, each sliding window is sized to the tokens it holds (bucketed to 32) rather than padded to the full window. The token-classification MoE runs every expert on every token, so inference is linear in sequence length and short inputs run ~4.6x faster on device. Static models without the companion keep the single full-window tensor unchanged.

…terminal - extractSpans now breaks at BIOES openers (B-/S-) so two adjacent same-type entities no longer merge into one span. - viterbiDecode gains `constrainEnd`; the final window is forced to close on a valid terminal (O/E/S) so a sequence cannot end on an open B-/I-. Interior windows stay unconstrained (their boundary tokens are discarded/re-decoded).

#1327 replaced `validateModelSchema` with `validateSpec` and dropped the `get_dynamic_dims_forward` companion in favour of `get_model_schema`. - Declare `forward` as two spec variants: `dynamic`, whose sequence dim binds to the exported range, and `static`, whose sequence dim binds to the single exported constant. The window size falls out of whichever variant matched, replacing both the `inputTensorMeta` shape read and the `getMethodNames()` probe for the old companion method. - Declare the sequence length shared by both inputs and the logits as an equality runtime constraint, so a length mismatch is rejected before `execute` instead of surfacing as an internal backend error. - Snap the length buckets onto the exported range's grid (and start them at its lower bound) rather than assuming a step of 1 from zero.

The XNNPACK exports declare a dynamic sequence dim (schema.json: range 2..256, plus the input/input/output equality constraint); the MLX exports have no dynamic_shapes and no companion, so they bind S to the constant 256. Say so instead of describing the split hypothetically.

barhanc

Mostly some small stuff and a question about the Viterbi algorithm. I will test the models and the example app on Monday. Rebase is also needed and the hook implementation should be changed following that.

barhanc · 2026-08-07T17:41:38Z

+ * @property {readonly string[]} labelNames - BIOES label list matching the
+ * model's `id2label` mapping exactly; index 0 must be `'O'`.
+ * @property {ViterbiBiases} [viterbiBiases] - Transition biases applied while
+ * decoding. Defaults to neutral (validity-only) Viterbi.
+ * @property {number} [padTokenId] - Token id used to pad the final window.
+ * Defaults to the o200k `<|endoftext|>` id.


We should use /** */ inlined comments for properties. The @property tag is for cases when there is no better place to include the comment (e.g. union type), see: https://typedoc.org/documents/Tags._property.html.

barhanc · 2026-08-07T17:46:34Z

+  /**
+   * Releases all allocated native resources.
+   */
+  dispose: () => void;
+  /**
+   * Asynchronously detects PII entity spans in the given text.
+   * @param input The text to scan for PII.
+   * @returns A promise resolving to the detected entity spans, in order.
+   */
+  detect: (input: string) => Promise<PiiEntity[]>;
+  /**
+   * Synchronous version of {@link detect} to be executed directly on the
+   * caller or worklet thread.
+   */
+  detectWorklet: (input: string) => PiiEntity[];


Suggested change

/**

* Releases all allocated native resources.

*/

dispose: () => void;

/**

* Asynchronously detects PII entity spans in the given text.

* @param input The text to scan for PII.

* @returns A promise resolving to the detected entity spans, in order.

*/

detect: (input: string) => Promise<PiiEntity[]>;

/**

* Synchronous version of {@link detect} to be executed directly on the

* caller or worklet thread.

*/

detectWorklet: (input: string) => PiiEntity[];

/**

* Releases all allocated native resources.

*/

dispose: () => void;

/**

* Asynchronously detects PII entity spans in the given text.

* @param input The text to scan for PII.

* @returns A promise resolving to the detected entity spans, in order.

*/

detect: (input: string) => Promise<PiiEntity[]>;

/**

* Synchronous version of {@link detect} to be executed directly on the

* caller or worklet thread.

*/

detectWorklet: (input: string) => PiiEntity[];

More readable this way :) Also maybe rename to detectPii so that it is more specific.

barhanc · 2026-08-07T17:50:57Z

+  }
+  bucketLengths.push(windowSize);
+
+  const bucketTensors = bucketLengths.map(


Why not dynamically allocated tensors as in other pipelines that use dynamic inputs (e.g. text embeddings, tts, stt)?. The bucketTensors probably use quite a lot of memory and the dynamic allocation should be orders of magnitude faster than inference cost.

barhanc · 2026-08-07T17:53:56Z

+      let slot = bucketLengths.length - 1;
+      for (let b = 0; b < bucketLengths.length; b++) {
+        if (bucketLengths[b]! >= validLen) {
+          slot = b;
+          break;
+        }
+      }


Suggested change

let slot = bucketLengths.length - 1;

for (let b = 0; b < bucketLengths.length; b++) {

if (bucketLengths[b]! >= validLen) {

slot = b;

break;

}

}

const bIdx = bucketLengths.findIndex(len => len >= validLen);

const slot = bIdx !== -1 ? bIdx : bucketLengths.length - 1;

barhanc · 2026-08-07T17:57:33Z

+      tAttentionMask.setData(maskData);
+
+      model.execute('forward', [tInputIds, tAttentionMask], [tLogits]);
+      tLogits.getData(logits);


Suggested change

tLogits.getData(logits);

const logits = tLogits.getData(new Float32Array(tLogits.numel);

barhanc · 2026-08-07T18:16:23Z

+
+/**
+ * Six Viterbi transition biases matching the openai/privacy-filter
+ * `viterbi_calibration.json` schema. Each value is added to the decoder score


A link to openai hf hosting the json would be helpful for API users.

barhanc · 2026-08-07T18:19:30Z

+// Privacy-filter-specific helpers: BIOES grammar construction and constrained
+// Viterbi decoding over per-token logits, plus span extraction. These live
+// under `utils/` (not the shared `ops.ts`) because they are only meaningful to
+// the privacy filter pipeline. The transformer forward pass dominates the
+// inference budget, so this decoding is written in pure TypeScript rather than
+// a native op (see the `add-native-extension` skill's Amdahl's-law rule).


This comment has some stuff that are development artefacts (like the comment about ops.ts) and some stuff that should genuinely stay like the comment about implementing viterbi decoding in pure TS. It should be rewritten slightly.

barhanc · 2026-08-07T18:19:43Z

+// inference budget, so this decoding is written in pure TypeScript rather than
+// a native op (see the `add-native-extension` skill's Amdahl's-law rule).
+
+const NEG_INF = -1e30;


There is Number.NEGATIVE_INFINITY.

barhanc · 2026-08-07T18:20:32Z

+ * @category Types
+ */
+export interface Grammar {
+  readonly numLabels: number;


For consistency let's add docs to all properties.

barhanc · 2026-08-07T18:27:32Z

+ * `false`.
+ * @returns The most likely label id per token.
+ */
+export function viterbiDecode(


Is this implementation based on some OpenAI (or other) code that was provided with the PII models or is it our custom implementation of Viterbi algorithm for PII?

msluszniak self-assigned this Jul 20, 2026

msluszniak added refactoring feature PRs that implement a new feature labels Jul 20, 2026

msluszniak linked an issue Jul 20, 2026 that may be closed by this pull request

[RNE Rewrite] Add Privacy Filter pipeline implementation #1246

Open

msluszniak commented Jul 20, 2026

View reviewed changes

msluszniak marked this pull request as ready for review July 23, 2026 15:06

msluszniak requested a review from barhanc July 23, 2026 15:06

msluszniak added 7 commits August 4, 2026 09:36

refactor(privacy-filter): drop redundant comments per review

2ad75bd

msluszniak force-pushed the @ms/rewrite-privacy-filter branch from e177430 to 2c0a322 Compare August 4, 2026 07:45

barhanc reviewed Aug 7, 2026

View reviewed changes

	tLogits.getData(logits);
	const logits = tLogits.getData(new Float32Array(tLogits.numel);

Uh oh!

Conversation

msluszniak commented Jul 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Introduces a breaking change?

Type of change

Tested on

Testing instructions

Related issues

Checklist

Additional notes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

msluszniak commented Jul 23, 2026

Uh oh!

barhanc left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

barhanc Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

msluszniak commented Jul 20, 2026 •

edited

Loading

barhanc Aug 7, 2026 •

edited

Loading