Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance: Status Cache #4271

Open
bw-solana opened this issue Jan 3, 2025 · 2 comments · May be fixed by #3796
Open

Performance: Status Cache #4271

bw-solana opened this issue Jan 3, 2025 · 2 comments · May be fixed by #3796
Assignees

Comments

@bw-solana
Copy link

bw-solana commented Jan 3, 2025

Status cache is by far the biggest bottleneck right now if you ingest many txs regardless of CUs, because it's an early bottleneck

It's using a rwlock now. The short term fix is to use a concurrent hash table. Dashmap would work, but Alessandro has been benchmarking https://docs.rs/flurry/latest/flurry and it's much much faster.

However, the existing design doesn’t scale well to 1M TPS without pre-allocating a ton of memory.

@bw-solana bw-solana moved this to In progress in Agave Performance Jan 3, 2025
@bw-solana bw-solana linked a pull request Jan 3, 2025 that will close this issue
@4r33x
Copy link

4r33x commented Jan 5, 2025

I think it's worth to try scc or even lock-free papaya over dashmap\flurry

@alessandrod
Copy link

I've looked into both of those a while ago and they don't provide the consistency guarantees required for the status cache

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In progress
Development

Successfully merging a pull request may close this issue.

3 participants