[FEATURE] Overlapping decompression for shuffle read #2601
Description
Code of Conduct
- I agree to follow this project's Code of Conduct
Search before asking
- I have searched in the issues and found no similar issues.
Describe the feature
When applying the #2598 into the benchmark of terasort 100g, I found some bottleneck for the decompression time in the read phase. Maybe we could introduce the overlapping decompression for the shuffle read, I think it will be useful and valuable.
We could implement this based on the previous async read #2365 , the overlapping decompression could happen in the async read thread pool, that need to refactor partial code to support this mode.
Motivation
No response
Describe the solution
No response
Additional context
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!