README.md

Efficient, lightweight, and reliable distributed computation.

Documentation: https://finos.github.io/opengris-scaler/

Start there for installation options, command reference, worker manager guides, scaling policies, and examples.

What Is Scaler?

OpenGRIS Scaler is a distributed computing framework for running Python tasks across local machines or remote infrastructure.

It provides:

• A Python client API similar to multiprocessing patterns such as submit(), map(), and starmap().
• A centralized scheduler that dispatches work and balances load across workers.
• Worker managers for local execution and cloud-backed capacity.
• Support for graph/DAG execution, monitoring, and task recovery.

Architecture

• Clients submit tasks to a scheduler.
• The scheduler tracks state, applies scaling/allocation policies, and dispatches work.
• Worker managers provision workers locally or on external infrastructure.
• Workers execute tasks and return results.
• An object storage service stores task inputs and outputs used by the cluster.

Local Quickstart

Install the package:

pip install opengris-scaler

Create config.toml:

[object_storage_server]
bind_address = "tcp://127.0.0.1:8517"

[scheduler]
bind_address = "tcp://127.0.0.1:8516"
object_storage_address = "tcp://127.0.0.1:8517"

[[worker_manager]]
type = "baremetal_native"
scheduler_address = "tcp://127.0.0.1:8516"
worker_manager_id = "wm-native"

Start a fully local stack:

scaler config.toml

Submit tasks from Python:

from scaler import Client


def square(value: int) -> int:
    return value * value


with Client(address="tcp://127.0.0.1:8516") as client:
    results = client.map(square, range(10))

print(results)

Learn More

• Docs home: https://finos.github.io/opengris-scaler/
• Quickstart: https://finos.github.io/opengris-scaler/tutorials/quickstart.html
• Overview: https://finos.github.io/opengris-scaler/tutorials/overview.html
• Commands: https://finos.github.io/opengris-scaler/tutorials/commands.html