Starting

You can use Ruby Spark via interactive shell. Shell is using pry console.

$ ruby-spark shell

Adding spark to ruby project. Minimal steps.

require 'ruby-spark'
Spark.start

If you want configure Spark first. See configurations page for more details.

require 'ruby-spark'

Spark.load_lib(spark_home)
Spark.config do
  set_app_name 'RubySpark'
  set 'spark.ruby.serializer', 'oj'
  set 'spark.ruby.serializer.batch_size', 100
end
Spark.start

Context is available via.

Spark.context
Spark.sc

Stopping

Context is automatically stopped when program is ended but you can also use this. Than you can configure or start Spark again.

# Stop context, clear config and kill all workers
Spark.stop