Safe Learning on Real Robots

A collection of algorithms and experiment tools for safe sim to real transfer and learning in robotics.

Features

• Three different CMDP solvers, CRPO, Saute-RL and primal-dual, compatible with (variants of) Brax's SAC, MBPO and PPO.
• Algorithm implementation is interchangeable between training in simulation to training on real robots via OnlineEpisodeOrchestrator. Check out rccar_experiments for a full example. Support for training online on any real robot supported by MuJoCo Playground, including Unitree Go1/2.
• Blazing fast training. Full compatibility with MuJoCo Playground. Reimplementation of OpenAI's Safety Gym in MJX and safety tasks from Real-World RL suite.

Requirements

• Python == 3.11.6
• venv or Poetry

Installation

Using pip

git clone https://github.com/yardenas/safe-learning
cd safe-learning
python3 -m venv venv
source venv/bin/activate
pip install -e .

Using Poetry

git clone https://github.com/yardenas/safe-learning
cd safe-learning
poetry install
poetry shell

Usage

Our code uses Hydra to configure experiments. Each experiment is defined as a yaml file in ss2r/configs/experiments. For example, to train a Unitree Go1 policy with a constraint on joint limit:

python train_brax.py +experiment=go1_sim_to_real

Docs

• Policies (in onnx format) used for the Unitree Go1 experiments can be found in ss2r/docs/policies.
• In ss2r/docs/videos you can find videos of 5 trials for each policy, marked by its policy id.