[FEATURE] Dedicated faster serialization when shuffle writing/reading #2596
Description
Code of Conduct
- I agree to follow this project's Code of Conduct
Search before asking
- I have searched in the issues and found no similar issues.
Describe the feature
After overlapping compression is applied, the majority of shuffle write times is the seralization. Is it possible to replace the kyro serialization with the fast implementation like apache fory? It looks it has a impressive benchmark result .
Motivation
No response
Describe the solution
No response
Additional context
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!