Skip to content

Performance: Avoid intermediate allocations with Object.values in hot paths #172

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ysdede wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Performance: Avoid intermediate allocations with Object.values in hot paths #172

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .jules/bolt.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,6 @@ Action: Apply loop unrolling for max reductions in high-frequency typed array op
## 2024-11-20 - Softmax math.exp 8x unrolling with local var cache
Learning: Unrolling the `Math.exp` accumulation loop to 8x and caching the multiplication `(tokenLogits[i] - maxLogit) * invTemp` into local variables before passing to `Math.exp` yields a measurable performance improvement (~4%) over the previous 4x unrolled implementation in the V8 engine, by reducing property access and allowing better instruction-level parallelism.
Action: Utilize 8x loop unrolling paired with local variable caching for tight floating-point accumulation loops over TypedArrays.
## 2024-11-20 - Avoid Object.values in hot loops
Learning: In hot paths with high frequency calls (like inner ML execution loops), tracking tensors via `Object.values()` combined with `Set.add()` causes excessive intermediate array allocations and hashing overhead. Replacing this with `for...in` and local array `.push()`/`.includes()` yields a ~3x to ~5x speedup for the specific operation.
Action: Avoid `Object.values` and small `Set` collections in extreme hot paths; prefer `for...in` and local arrays for small collections.
25 changes: 19 additions & 6 deletions src/parakeet.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -323,10 +323,11 @@ export class ParakeetModel {
const logits = out['outputs'];
const outputState1 = out['output_states_1'];
const outputState2 = out['output_states_2'];
const seenOutputs = new Set();
for (const value of Object.values(out)) {
if (!value || typeof value.dispose !== 'function' || seenOutputs.has(value)) continue;
seenOutputs.add(value);
const seenOutputs = [];
for (const key in out) {
const value = out[key];
Comment on lines +327 to +328
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot May 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using for...in iterates over inherited properties as well as own properties. Since Object.values() only considers own enumerable properties, this change could lead to unexpected behavior if the environment has modified Object.prototype. To maintain the same semantics while avoiding array allocations, consider adding an Object.hasOwn() check (or Object.prototype.hasOwnProperty.call() for older environments).

Suggested change
for (const key in out) {
const value = out[key];
for (const key in out) {
if (!Object.hasOwn(out, key)) continue;
const value = out[key];

if (!value || typeof value.dispose !== 'function' || seenOutputs.includes(value)) continue;
seenOutputs.push(value);
if (value === logits || value === outputState1 || value === outputState2) continue;
value.dispose();
}
Expand Down Expand Up @@ -683,10 +684,22 @@ export class ParakeetModel {
const s = performance.now();
const encOut = await this.encoderSession.run({ audio_signal: input, length: lenTensor });
tEncode = performance.now() - s;
enc = encOut['outputs'] ?? Object.values(encOut)[0];
enc = encOut['outputs'];
if (enc === undefined) {
for (const key in encOut) {
enc = encOut[key];
break;
}
Comment on lines +689 to +692
Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot May 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The logic for extracting the default output tensor is duplicated in both branches of the perfEnabled check. Also, to ensure that only own properties are considered (matching the previous behavior of Object.values()), add an Object.hasOwn() check inside the for...in loop.

          for (const key in encOut) {
            if (!Object.hasOwn(encOut, key)) continue;
            enc = encOut[key];
            break;
          }

}
} else {
const encOut = await this.encoderSession.run({ audio_signal: input, length: lenTensor });
enc = encOut['outputs'] ?? Object.values(encOut)[0];
enc = encOut['outputs'];
if (enc === undefined) {
for (const key in encOut) {
enc = encOut[key];
break;
}
}
}
} finally {
// Dispose per-call input tensors even when encoder execution fails.
Expand Down
Loading