Description:
Bug: The native embedding module (@lucid-memory/native) passes tokenized input directly to ONNX Runtime without capping sequence length at the model's 512-token maximum. When any text in a batch tokenizes to >512 tokens, the ONNX Add node fails because the model has a static input shape of [batch, 512] but receives [batch, N] where N > 512.
Error:
ONNX Runtime error: Non-zero status code returned while running Add node.
Name:'/embeddings/Add_1'
Attempting to broadcast an axis by a dimension other than 1. 512 by 655
To reproduce:
Store a memory with content exceeding ~2000 characters (tokenizes to >512 BPE tokens)
The background processPendingEmbeddings in retrieval.ts calls nativeEmbedding.embedBatch(texts) with that text
ONNX Runtime fails on the Add node
Affected functions:
embedBatch — pads all inputs to the longest sequence in the batch, but doesn't cap at 512
embed (single) — also affected, same ONNX constraint
Root cause: The ort session was exported with max_length=512 as a static input dimension. The native Rust NAPI bindings at packages/lucid-native/ don't truncate tokenized sequences to this limit before calling session.run().
Suggested fix: In the Rust tokenization/preprocessing step, truncate token IDs to min(actual_length, model_max_length) before creating the ONNX tensor. The model max length (512) should be read from the model config or passed as a parameter.
Workaround applied locally: Truncating text to 1300 characters (~325-465 tokens depending on token density) in the TypeScript layer before passing to embedBatch/embed, plus a fallback to single embedding with per-text error handling.
Environment:
@lucid-memory/native v0.6.5
Model: bge-base-en-v1.5-quantized.onnx
macOS 14 (arm64)
bun 1.3.13
Description:
Bug: The native embedding module (@lucid-memory/native) passes tokenized input directly to ONNX Runtime without capping sequence length at the model's 512-token maximum. When any text in a batch tokenizes to >512 tokens, the ONNX Add node fails because the model has a static input shape of [batch, 512] but receives [batch, N] where N > 512.
Error:
ONNX Runtime error: Non-zero status code returned while running Add node.
Name:'/embeddings/Add_1'
Attempting to broadcast an axis by a dimension other than 1. 512 by 655
To reproduce:
Store a memory with content exceeding ~2000 characters (tokenizes to >512 BPE tokens)
The background processPendingEmbeddings in retrieval.ts calls nativeEmbedding.embedBatch(texts) with that text
ONNX Runtime fails on the Add node
Affected functions:
embedBatch — pads all inputs to the longest sequence in the batch, but doesn't cap at 512
embed (single) — also affected, same ONNX constraint
Root cause: The ort session was exported with max_length=512 as a static input dimension. The native Rust NAPI bindings at packages/lucid-native/ don't truncate tokenized sequences to this limit before calling session.run().
Suggested fix: In the Rust tokenization/preprocessing step, truncate token IDs to min(actual_length, model_max_length) before creating the ONNX tensor. The model max length (512) should be read from the model config or passed as a parameter.
Workaround applied locally: Truncating text to 1300 characters (~325-465 tokens depending on token density) in the TypeScript layer before passing to embedBatch/embed, plus a fallback to single embedding with per-text error handling.
Environment:
@lucid-memory/native v0.6.5
Model: bge-base-en-v1.5-quantized.onnx
macOS 14 (arm64)
bun 1.3.13