Envpool

Note: we are still in the progress of adding more environment from dm_control suite and box2d.

New Environment

DeepMind Control Suite

1. Hopper (107, Trinkle23897)
2. Cheetah (110, yufansong)
3. Walker (114, yufansong)
4. Reacher (115, yufansong)
5. Finger (118, yufansong)
6. Pendulum (122, yufansong)
7. BallInCup (124, Benjamin-eecs)
8. PointMass (123, yufansong)
9. Acrobot (129, Benjamin-eecs)
10. Misc (106, 125, 130, Trinkle23897)

Box2D

1. LunarLander-v2, LunarLanderContinuous-v2 (111, Alicia1529)

Others

1. Pendulum-v1 (126, Trinkle23897)

Bug Fix

1. Fix episodic_life bug in atari env (121, Trinkle23897)

Enhancemt

1. Support dynamic shaped arrays (98, mavenlin)
2. Update Mujoco benchmark result (99, Trinkle23897)
3. Fix code linter (109, 116, 117, Trinkle23897)

This release is for a stable version for benchmarking. We will update the benchmark result for both Atari (Pong-v5) and Mujoco (Ant-v3) soon. We observe Ant-v3 can achieve 2M+ FPS with a 192-core machine.

Enhancement

1. Add `gym_reset_return_info` option for returning `(obs, info)` in `gym.Env.reset` (97)
2. Add clang-tidy check (85, 95, 100)

Bug Fix

1. Fix max_episode_steps for Pusher and Reacher (93)

Enhancement

1. add testing macro `ENVPOOL_TEST` (93, 94)
2. add `terminate_when_unhealthy` and `exclude_current_positions_from_observation` for most of mujoco envs (93)
3. eliminate `info["qpos0"]` and `info["qvel0"]` in mujoco env when generating wheel (93)
4. add instructions for building this project with mainland China users (92)

- Ant-v4 (with yufansong 74)
- HalfCheetah-v4 (75)
- Hopper-v4 (76)
- Humanoid-v4 (77)
- HumanoidStandup-v4 (78)
- InvertedDoublePendulum-v4 (Benjamin-eecs, 83)
- InvertedPendulum-v4 (79)
- Pusher-v4 (82)
- Reacher-v4 (81)
- Swimmer-v4 (80)
- Walker2d-v4 (Benjamin-eecs, 86)

We have successfully integrated Ant-v4 environment based on the newest [deepmind/mujoco](https://github.com/deepmind/mujoco/releases) package! (#74)
More environments are coming soon.

Other enhancement: support element-wise bound, fix classic_control action space (67)

Enhancement

1. Add ViZDoom single-player environment (47, 54)