Use LRU eviction for evaluation cache instead of FIFO

[jdcodes1]

Summary

• Converts the 500-item evaluation cache from FIFO to LRU eviction
• On cache hit, deletes and re-inserts the entry to move it to the end of Map iteration order
• Eviction still removes the first key — but now that's the least recently used entry, not the oldest

Problem: Trending/viral posts appear repeatedly in feeds. Under FIFO, a post seen 50 times gets evicted just because 500 other posts came after it, forcing a redundant LLM call next time the user scrolls past it.

Fix: 3 lines — delete + set on cache hit promotes the entry. JavaScript Map preserves insertion order, so this is a clean O(1) LRU without any new data structures.

Test plan

• Added test verifying recently accessed entries survive eviction while true LRU entries are evicted
• All 200 existing tests pass
• No changes to cache key generation, persistence, or eviction trigger

[imperion402]

Problem: Trending/viral posts appear repeatedly in feeds. Under FIFO, a post seen 50 times gets evicted just because 500 other posts came after it, forcing a redundant LLM call next time the user scrolls past it.

Source? That literally never happens on X.com.

[jdcodes1]

Problem: Trending/viral posts appear repeatedly in feeds. Under FIFO, a post seen 50 times gets evicted just because 500 other posts came after it, forcing a redundant LLM call next time the user scrolls past it.

Source? That literally never happens on X.com.

variants of the same post, not exactly the same. happens frequently during large events- Iran, Anthropic Mythos

[imperion402]

Problem: Trending/viral posts appear repeatedly in feeds. Under FIFO, a post seen 50 times gets evicted just because 500 other posts came after it, forcing a redundant LLM call next time the user scrolls past it.

Source? That literally never happens on X.com.

variants of the same post, not exactly the same. happens frequently during large events- Iran, Anthropic Mythos

LRU Cache would have to be an exact match -> variant perf wouldn't be improved