Skip to content

nlesc-dirac/sagecal-spark-docker-swarm

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

25 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SageCal on Apache Spark (WIP)

About

The Docker swarm cluster with Apache Spark and HDFS filesystem to run SageCal. The setup uses Docker images to build scaleble cluster.

Included Software

Name Version
Hadoop 2.9
Spark 2.2.0
java OpenJDK-8
SageCal -

Note: The setup was only tested on Linux64 system.

Docker images

docker pull fdiblen/spark-worker-dirac docker pull fdiblen/spark-master-dirac docker pull fdiblen/hadoop

Instructions

Intallation instructions INSTALL.md

Instructions to submit a job JOBS.md

About

Apache Spark cluster in Docker Swarm

Resources

License

Stars

Watchers

Forks

Packages

No packages published