Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sparkExternalService? #648

Open
manuel opened this issue Sep 23, 2015 · 0 comments
Open

sparkExternalService? #648

manuel opened this issue Sep 23, 2015 · 0 comments
Assignees

Comments

@manuel
Copy link
Contributor

manuel commented Sep 23, 2015

Investigate ways to run long-running Spark streaming apps (not Spark batch actions) from Celos.

Maybe a new kind of external service is enough, maybe we also need a new schedule type (because a streaming app doesn't have slots).

First goal is to be able to monitor a long-running streaming app's status (running/failed, nothing more) in Celos UI.

Also consider deploying Spark apps from Celos CI. Maybe we need a new subdirectory spark or something in the deployment directory, and new entries in the target files or new cmd line options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants