Performance: Avoid Set and Object.values overhead in hot loops

.jules/bolt.md

@@ -13,3 +13,7 @@ Action: Apply loop unrolling for max reductions in high-frequency typed array op
 ## 2024-11-20 - Softmax math.exp 8x unrolling with local var cache
 Learning: Unrolling the `Math.exp` accumulation loop to 8x and caching the multiplication `(tokenLogits[i] - maxLogit) * invTemp` into local variables before passing to `Math.exp` yields a measurable performance improvement (~4%) over the previous 4x unrolled implementation in the V8 engine, by reducing property access and allowing better instruction-level parallelism.
 Action: Utilize 8x loop unrolling paired with local variable caching for tight floating-point accumulation loops over TypedArrays.
+
+## 2024-11-20 - Set and Object.values overhead in hot loops
+Learning: In V8, iterating over objects with `Object.values()` creates intermediate array allocations that trigger garbage collection, and using a `Set` for tracking a very small number of unique items (e.g. 3-5 items) incurs higher initialization and hashing overhead compared to a plain array.
+Action: Replace `Object.values(obj)` with a `for...in` loop (checking `Object.hasOwn(obj, key)`) and replace `new Set()` with `[]` and `Array.includes()` for tracking small collections of items in high-frequency loops (like the inner DP decoder step) to improve performance.

src/parakeet.js

@@ -323,10 +323,12 @@ export class ParakeetModel {
     const logits = out['outputs'];
     const outputState1 = out['output_states_1'];
     const outputState2 = out['output_states_2'];
-    const seenOutputs = new Set();
-    for (const value of Object.values(out)) {
-      if (!value || typeof value.dispose !== 'function' || seenOutputs.has(value)) continue;
-      seenOutputs.add(value);
+    const seenOutputs = [];
+    for (const key in out) {
+      if (!Object.hasOwn(out, key)) continue;
+      const value = out[key];
+      if (!value || typeof value.dispose !== 'function' || seenOutputs.includes(value)) continue;
+      seenOutputs.push(value);
       if (value === logits || value === outputState1 || value === outputState2) continue;
       value.dispose();
     }
@@ -339,12 +341,12 @@ export class ParakeetModel {
     const failDecoderStep = (message) => {
       logits?.dispose?.();
 
-      const disposed = new Set();
+      const disposed = [];
       const disposeUniqueState = (state) => {
         if (!state) return;
         for (const tensor of [state.state1, state.state2]) {
-          if (!tensor || tensor === this._combState1 || tensor === this._combState2 || disposed.has(tensor)) continue;
-          disposed.add(tensor);
+          if (!tensor || tensor === this._combState1 || tensor === this._combState2 || disposed.includes(tensor)) continue;
+          disposed.push(tensor);
           tensor.dispose?.();
         }
       };