Parl

New Features
- Support GPU cluster for XPARL Parallel training. For more details, please look for the latest [docs](https://parl.readthedocs.io/en/latest/parallel_training/gpu_cluster.html)

Framework
* add agent.train()/eval()
* fix some bugs of DDQN
* add ComptWrapper (Cpmpatible for different versions of gym and latest verion of mujoco)
Parallel Training
* support xparl in notebook
* add XPARL_PYTHON environment variable

Example
* add Paddle examples
* PPO
* MADDPG
* ES
* CQL
* IMPALA
* Baseline
* GridDispatching Competition
* Halite Competition
* add PPO、MADDPG、ES、CQL、IQL、Decision Transformer、MAPPO、MAML++ in benchmark

Tutorial
* add dygraph+parl2.0+paddle2.0 version of tutorials code for bilibili course
* add dependency version constraints to tutorials

Framework
- Support PaddlePaddle 2.0 (dynamic graph mode) by default
- Add integration testing for Windows

Parallel Training
- Refactor the heartbeat mechanism of the xparl module
- Use the synchronous xparl API for some distributed algorithms

Documentation
- Add Chinese documentation readthedocs

Example
- Paddle
- Policy Gradient
- DDPG
- DQN/Double DQN/ Dueling DQN
- SAC
- TD3
- OAC
- QMIX
- A2C
- AlphaZero
- Fluid
- QMIX
Application
- Self-driving system in CARLA simulator

Framework
- support the latest API of dynamic graph mode in PaddlePaddle
- support VisuaIDL visualization tool
- optimize compatibility under different systems

Parallel Training
- add monitoring page of the task output log
- support direct access and modification of attributes of remote objects
- support asynchronous function call in remote objects

Example
- add Prioritized DQN algorithm
- add AlphaZero solution for Kaggle Connect X competition
- add the champion models of both tracks of Neurips 2020 Learning-to-Run-a-Power-Network challenge
- add demonstration code of open class "World champion takes you to learn reinforcement learning from scratch"

New Features
- Add the first open-source industrial evolution strategy framework EvoKit
- Support Multi-Agent RL algorithms, including MADDPG
- Support multiple GPU training, provide a demonstration of DQN with multi GPU
- Add SOTA algorithms of continuous control problems, TD3 and SAC
- Add the champion model and training method of NeurIPS 2019 reinforcement learning competition
- Compatible with Windows

Parallel Training
1. Using a cluster to maintain the computation resource for parallel training.
2. Web UI for monitoring the cluster.
3. Support limiting the memory usage for each remote class.
4. Tutorial for the use of the cluster.

Example
1. Add the evolution strategies(ES) algorithm, using the PARL parallel module.
2. Append the A2C performance on a range of Atari games.
2. Append the IMPALA performance on a range of Atari games.

Tutorial
1. Add the official [documentation](https://parl.readthedocs.io/en/latest/) deployed at the readthedocs.
2. Add a [tutorial](https://parl.readthedocs.io/en/latest/new_alg.html) describing how to build a custom algorithm.
3. Add a [tutorial](https://parl.readthedocs.io/en/latest/parallel_training/setup.html) describing how to use the cluster for parallel computation.