Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Coordinator supports EstimateDiskSpaceQuotaManager #2257

Open
2 of 3 tasks
jerqi opened this issue Nov 19, 2024 · 0 comments
Open
2 of 3 tasks

[FEATURE] Coordinator supports EstimateDiskSpaceQuotaManager #2257

jerqi opened this issue Nov 19, 2024 · 0 comments

Comments

@jerqi
Copy link
Contributor

jerqi commented Nov 19, 2024

Code of Conduct

Search before asking

  • I have searched in the issues and found no similar issues.

Describe the feature

Spark Job configures its estimate disk space in its Spark configurations. Uniffle pass the estimate disk space to the coordinator. If the disk space exceeds the limitation of disk space, coordinator will reject the Spark Job.
More code you can see AbstractAccessChecker to add a new access checker in the coordinator code.
For Spark client, you can see DelegationRssShuffleManager.

Motivation

Avoid large jobs concurrently running.

Describe the solution

Add a new access checker class.

Additional context

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant