-
Notifications
You must be signed in to change notification settings - Fork 1.4k
[Bug] recallScoreThreshold bypassed in pickMemoriesForInjection function #1106
Description
[Bug] recallScoreThreshold bypassed in pickMemoriesForInjection function
Description
The recallScoreThreshold configuration is being bypassed during memory recall, causing low-scoring memories (below threshold) to be injected into context.
Evidence
- Config:
recallScoreThreshold: 0.25 - Observed: Retrieved memories include scores as low as
0.11(below threshold) - Expected: All returned memories should have score ≥ 0.25
Root Cause Analysis
The issue is in memory-ranking.ts, specifically in the pickMemoriesForInjection function.
Flow:
-
postProcessMemoriescorrectly filters memories by threshold (line 49):if (clampScore(item.score) < options.scoreThreshold) { continue; // ✓ Correct filtering }
-
However,
pickMemoriesForInjection(line 221) bypasses this filtering when trying to fill up to the limit:const leaves = deduped.filter((item) => isLeafLikeMemory(item)); if (leaves.length >= limit) { return leaves.slice(0, limit); // ✗ No threshold check } const picked = [...leaves]; const used = new Set(leaves.map((item) => item.uri)); for (const item of deduped) { if (picked.length >= limit) { break; } if (used.has(item.uri)) { continue; } picked.push(item); // ✗ Adds items without threshold check! } return picked;
Problem:
When there aren't enough "leaf-like" memories (level === 2), the function supplements from the general deduped array without re-checking the score threshold. This allows low-scoring memories to pass through.
Proposed Fix
Add threshold validation in pickMemoriesForInjection:
for (const item of deduped) {
if (picked.length >= limit) {
break;
}
if (used.has(item.uri)) {
continue;
}
// ADD: Re-check threshold before adding
if (clampScore(item.score) < options.scoreThreshold) {
continue;
}
picked.push(item);
}Note: The function signature would need to accept scoreThreshold as a parameter, or use a default value.
Impact
- Low-quality/irrelevant memories are being injected into agent context
- Wastes token budget on low-scoring results
- Reduces overall recall quality
Environment
- OpenViking: v0.2.15
- Plugin installation: path-based (
/Users/neoshi/.openviking/plugins/openviking) - Config mode: remote
Additional Context
This was discovered during investigation of recall quality after disabling rerank. Statistics from 14 queries:
- Average score: 0.2827
- Score range: 0.1100 - 0.7222 (minimum below 0.25 threshold)
- Zero-result rate: 0%
Metadata
Metadata
Assignees
Labels
Type
Projects
Status