README_en.md

中文|EN

JoyRL-Offline

Install

Currently JoyRL support Python3.7 and Gym0.25.2

conda create -n joyrl python=3.7
conda activate joyrl
pip install -r requirements.txt

Torch:

# CPU
conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cpuonly -c pytorch
# GPU
conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cudatoolkit=11.3 -c pytorch -c conda-forge
# GPU with mirrors
pip install torch==1.10.0+cu113 torchvision==0.11.0+cu113 torchaudio==0.10.0 --extra-index-url https://download.pytorch.org/whl/cu113

Install Mujoco

pip install mujoco==2.3.2
pip install mujoco-py==2.1.2.14

click mujoco-releases and download mujoco 2.1.0, unzip and move to C:/[username]/.mujoco/, it should be like C:/[username]/.mujoco/mujoco210, then add C:/[username]/.mujoco/mujoco210/bin to SYSTEM PATH.

Note that if you are using Windows, you'd better to download visual-cpp-build-tools.

Usage

you can simply change the parameters (like env_name, algo_name) in config.config.GeneralConfig() and run:

python main.py

then it will a new folder named tasks to save results and models.

Or you can custom parameters with a yaml file as you can seen in config/custom_config_Train.yaml and run:

python main.py --yaml config/custom_config_Train.yaml

And there are presets yaml files in the presets folder and well trained results in the benchmarks folder.

Docs

please click docs

Environments

Please click envs to read environments instruments.

Algorithms

Name	Policy	Reference	Author	Notes
Monte Carlo		RL introduction	johnjim0816
Value Iteration		RL introduction	guoshicheng
Q-learning		RL introduction	johnjim0816
Sarsa		RL introduction	johnjim0816
DQN		DQN Paper	johnjim0816, guoshicheng (CNN)
DoubleDQN		DoubleDQN Paper	johnjim0816
PER_DQN		PER_DQN Paper	wangzhongren
NoisyDQN		NoisyDQN Paper	wangzhongren
DRQN		DRQN Paper	johnjim0816	understand LSTM
C51		C51 Paper		also called Categorical DQN
REINFORCE		REINFORCE Paper	johnjim0816
A2C		A2C blog	johnjim0816
A3C	On	A3C paper	johnjim0816, Ariel Chen
PPO		PPO Paper	johnjim0816, Wen Qiu	PPO-clip, PPO-kl
DDPG		DDPG Paper	johnjim0816
TD3		TD3 Paper	johnjim0816
SoftQ		SoftQ Paper	johnjim0816
GAIL		GAIL Paper	Yi Zhang
Dyna Q		Dyna Q Paper	guoshicheng