Filing separately from #146048 because that's mostly about the index not being cached on ~first search (and instead getting incrementally loaded). This is about fully cached queries being slower.

Firefox profile of a trivial search (e) on nightly https://doc.rust-lang.org/nightly/std/?search=e: https://share.firefox.dev/4p1Va53 (1.225 seconds until "Load" event, which I think is a tiny bit before we see all of the search results).

Stable channel search (https://doc.rust-lang.org/stable/std/?search=e): https://share.firefox.dev/46bukjv takes 0.4 seconds or so, so about 2-3x faster.

On Chrome, I see loading finish faster, around 800ms on nightly, profile. But this is still about 2x slower than on stable, where results are available in 300ms.