Release version v0.23

Algorithm

• Support Advantage-Weighted Regression (AWR)
  • https://arxiv.org/abs/1910.00177
• n_frames option is added to all algorithms
  • n_frames option controls frame stacking for image observation
• eval_results_ property is added to all algorithms
  • evaluation results can be retrieved from eval_results_ after training.

MDPDataset

• prev_transition and next_transition properties are added to d3rlpy.dataset.Transition.
  • these properties are used for frame stacking and Monte-Carlo returns calculation at AWR.

Document

• new tutorial page is added

Release version v0.22

Support ONNX export

Now, the trained policy can be exported as ONNX as well as TorchScript

cql.save_policy('policy.onnx', as_onnx=True)

Support more data augmentations

• data augmentations for vector obsrevation
• ColorJitter augmentation for image observation

Release version v0.2

• support model-based algorithm
  • Model-based Offline Policy Optimization
• support data augmentation (for image observation)
  • Data-reguralized Q-learning
• a lot of improvements
  • more dataset statistics
  • more options to customize neural network architecture
  • optimize default learning rates
  • etc

First release!

• online algorithms
  • Deep Q-Network (DQN)
  • Double DQN
  • Deep Deterministic Policy Gradients (DDPG)
  • Twin Delayed Deep Deterministic Policy Gradients (TD3)
  • Soft Actor-Critic (SAC)
• data-driven algorithms
  • Batch-Constrained Q-leearning (BCQ)
  • Bootstrapping Error Accumulation Reduction (BEAR)
  • Conservative Q-Learning (CQL)
• Q functions
  • mean
  • Quantile Regression
  • Implicit Quantile Network
  • Fully-parametrized Quantile Function (experimental)