Gym

Release notes

This is another very minor bug release.

Bugs Fixes
* As `reset` now returns `(obs, info)` then in the vector environments, this caused the final `step`'s info to be overwritten. Now, the final observation and info are contained within the info as "final_observation" and "final_info" pseudo-rnd-thoughts
* Adds warnings when trying to render without specifying the `render_mode` younik
* Updates Atari Preprocessing such that the wrapper can be pickled vermouth1992
* Github CI was hardened to such that the CI just has read permissions sashashura
* Clarify and fix typo in `GraphInstance` ekalosak

Release Notes

This is a very minor bug fix release for 0.26.0

Bug Fixes
* 3072 - Previously `mujoco` was a necessary module even if only `mujoco-py` was used. This has been fixed to allow only `mujoco-py` to be installed and used. YouJiacheng
* 3076 - `PixelObservationWrapper` raises an exception if the `env.render_mode` is not specified. vmoens
* 3080 - Fixed bug in `CarRacing` where the colour of the wheels were not correct foxik
* 3083 - Fixed BipedalWalker where if the agent moved backwards then the rendered arrays would be a different size. younik

Spelling
* Fixed truncation typo in readme API example rdnfn
* Updated pendulum observation space from angle to theta to make more consistent ikamensh

This release is aimed to be the last of the major API changes to the core API. All of the previously "turned off" changes of the base API (step termination / truncation, reset info, no seed function, render mode determined by initialization) are now expected by default. We still plan to make breaking changes to Gym itself, but to things that are very easy to upgrade (environments and wrappers), and things that aren't super commonly used (the vector API). Once those aspects are stabilized, we'll do a proper 1.0 release and follow semantic versioning. Additionally, unless something goes terribly wrong with this release and we have to release a patched version, this will be the last release of Gym for a while.

If you've been waiting for a "stable" release of Gym to upgrade your project given all the changes that have been going on, this is the one.

We also just wanted to say that we tremendously appreciate the communities patience with us as we've gone on this journey taking over the maintenance of Gym and making all of these huge changes to the core API. We appreciate your patience and support, but hopefully, all the changes from here on out will be much more minor.

Breaking backward compatibility
These changes are true of all **gym**'s internal wrappers and environments but for environments not updated, we provide the `EnvCompatibility ` wrapper for users to convert old gym v21 / 22 environments to the new core API. This wrapper can be easily applied in `gym.make` and `gym.register` through the `apply_api_compatibility ` parameters.

* `Step` Termination / truncation - The `Env.step` function returns 5 values instead of 4 previously (`observations, reward, termination, truncation, info`). A blog with more details will be released soon to explain this decision. arjun-kg
* Reset info - The `Env.reset` function returns two values (`obs` and `info`) with no `return_info` parameter for gym wrappers and environments. This is important for some environments that provided action masking information for each actions which was not possible for resets. balisujohn
* No `Seed` function - While `Env.seed` was a helpful function, this was almost solely used for the beginning of the episode and is added to `gym.reset(seed=...)`. In addition, for several environments like Atari that utilise external random number generators, it was not possible to set the seed at any time other than `reset`. Therefore, `seed` is no longer expected to function within gym environments and is removed from all gym environments balisujohn
* Rendering - It is normal to only use a single render mode and to help open and close the rendering window, we have changed `Env.render` to not take any arguments and so all render arguments can be part of the environment's constructor i.e., `gym.make("CartPole-v1", render_mode="human")`. For more detail on the new API, see [blog post](https://younis.dev/blog/render-api/) younik

Major changes
* Render modes - In `v25`, there was a change in the meaning of render modes, i.e. "rgb_array" returned a list of rendered frames with "single_rgb_array" returned a single frame. This has been **reverted** in this release with "rgb_array" having the same meaning as previously to return a single frame with a new mode "rgb_array_list" returning a list of RGB arrays. The capability to return a list of rendering observations achieved through a wrapper applied during `gym.make`. 3040 pseudo-rnd-thoughts younik
* Added `save_video` that uses `moviepy` to render a list of RGB frames and updated `RecordVideo` to use this function. This removes support for recording `ansi` outputs. 3016 younik
* `RandomNumberGenerator` functions: `rand`, `randn`, `randint`, `get_state`, `set_state`, `hash_seed`, `create_seed`, `_bigint_from_bytes` and `_int_list_from_bigint` have been removed. balisujohn
* Bump `ale-py` to `0.8.0` which is compatibility with the new core API
* Added `EnvAPICompatibility` wrapper RedTachyon

Minor changes
* Added improved `Sequence`, `Graph` and `Text` sample masking pseudo-rnd-thoughts
* Improved the gym `make` and `register` type hinting with `entry_point` being a necessary parameter of `register`. 3041 pseudo-rnd-thoughts
* Changed all URL to the new gym website https://www.gymlibrary.dev/ FieteO
* Fixed mujoco offscreen rendering with weight and height value > 500 3044 YouJiacheng
* Allowed toy_text environment to render on headless machines 3037 RedTachyon
* Renamed the motors in the mujoco swimmer envs 3036 lin826

This is a fairly minor bug fix release.

Bug Fixes
* Removes requirements for `_TimeLimit.truncated` in info for step compatibility functions. This makes the step compatible with Envpool arjun-kg
* As the ordering of `Dict` spaces matters when flattening spaces, updated the `__eq__` to account for the `.keys()` ordering. XuehaiPan
* Allows `CarRacing` environment to be pickled. Updated all gym environments to be correctly pickled. RedTachyon
* seeding `Dict` and `Tuple` spaces with integers can cause lower-specification computers to hang due to requiring 8Gb memory. Updated the seeding with integers to not require unique subseeds (subseed collisions are rare). For users that require unique subseeds for all subspaces, we recommend using a dictionary or tuple with the subseeds. olipinski
* Fixed the metaclass implementation for the new render api to allow custom environments to use metaclasses as well. YouJiacheng

Updates
* Simplifies the step compatibility functions to make them easier to debug. Time limit wrapper with the old step API favours terminated over truncated if both are true. This is as the old done step API can only encode 3 states (cannot encode `terminated=True` and `truncated=True`) therefore we must encode to only `terminated=True` or `truncated=True`. pseudo-rnd-thoughts
* Add Swig as a dependency kir0ul
* Add type annotation for `render_mode` and `metadata` bkrl

Release notes

* Added rendering for CliffWalking environment younik
* `PixelObservationWrapper` only supports the new render API due to difficulty in supporting both old and new APIs. A warning is raised if the user is using the old API vmoens

Bug fix
* Revert an incorrect edition on wrapper.FrameStack ZhiqingXiao
* Fix reset bounds for mountain car psc-g
* Removed skipped tests causing bugs not to be caught pseudo-rnd-thoughts
* Added backward compatibility for environments without metadata pseudo-rnd-thoughts
* Fixed `BipedalWalker` rendering for RGB arrays 1b15
* Fixed bug in `PixelObsWrapper` for using the new rendering younik

Typos
* Rephrase observations' definition in Lunar Lander Environment EvanMath
* Top-docstring in `gym/spaces/dict.py` Ice1187
* Several typos in `humanoidstandup_v4.py`, `mujoco_env.py`, and `vector_list_info.py` timgates42
* Typos in passive environment checker pseudo-rnd-thoughts
* Typos in Swimmer rotations lin826

Release notes

This release finally introduces all new API changes that have been planned for the past year or more, all of which will be turned on by default in a subsequent release. After this point, Gym development should get massively smoother. This release also fixes large bugs present in 0.24.0 and 0.24.1, and we highly discourage using those releases.

API Changes
* `Step` - A majority of deep reinforcement learning algorithm implementations are incorrect due to an important difference in theory and practice as `done` is not equivalent to `termination`. As a result, we have modified the `step` function to return five values, `obs, reward, termination, truncation, info`. The full theoretical and practical reason (along with example code changes) for these changes will be explained in a soon-to-be-released blog post. The aim for the change to be backward compatible (for now), for issues, please put report the issue on github or the discord. arjun-kg
* `Render` - The render API is changed such that the mode has to be specified during `gym.make` with the keyword `render_mode`, after which, the render mode is fixed. For further details see https://younis.dev/blog/2022/render-api/ and https://github.com/openai/gym/pull/2671. This has the additional changes
- with `render_mode="human"` you don't need to call .render(), rendering will happen automatically on `env.step()`
- with `render_mode="rgb_array"`, `.render()` pops the list of frames rendered since the last `.reset()`
- with `render_mode="single_rgb_array"`, `.render()` returns a single frame, like before.
* `Space.sample(mask=...)` allows a mask when sampling actions to enable/disable certain actions from being randomly sampled. We recommend developers add this to the `info` parameter returned by `reset(return_info=True)` and `step`. See https://github.com/openai/gym/pull/2906 for example implementations of the masks or the individual spaces. We have added an example version of this in the taxi environment. pseudo-rnd-thoughts
* Add `Graph` for environments that use graph style observation or action spaces. Currently, the node and edge spaces can only be `Box` or `Discrete` spaces. jjshoots
* Add `Text` space for Reinforcement Learning that involves communication between agents and have dynamic length messages (otherwise `MultiDiscrete` can be used). ryanrudes pseudo-rnd-thoughts

Bug fixes
* Fixed car racing termination where if the agent finishes the final lap, then the environment ends through truncation not termination. This added a version bump to Car racing to v2 and removed Car racing discrete in favour of `gym.make("CarRacing-v2", continuous=False)` araffin
* In `v0.24.0`, `opencv-python` was an accidental requirement for the project. This has been reverted. KexianShen pseudo-rnd-thoughts
* Updated `utils.play` such that if the environment specifies `keys_to_action`, the function will automatically use that data. Markus28
* When rendering the blackjack environment, fixed bug where rendering would change the dealers top car. balisujohn
* Updated mujoco docstring to reflect changes that were accidently overwritten. Markus28

Misc
* The whole project is partially type hinted using [pyright](https://github.com/microsoft/pyright) (none of the project files is ignored by the type hinter). RedTachyon pseudo-rnd-thoughts (Future work will add strict type hinting to the core API)
* Action masking added to the taxi environment (no version bump due to being backwards compatible) pseudo-rnd-thoughts
* The `Box` space shape inference is allows `high` and `low` scalars to be automatically set to `(1,)` shape. Minor changes to identifying scalars. pseudo-rnd-thoughts
* Added option support in classic control environment to modify the bounds on the initial random state of the environment psc-g
* The `RecordVideo` wrapper is becoming deprecated with no support for `TextEncoder` with the new render API. The plan is to replace `RecordVideo` with a single function that will receive a list of frames from an environment and automatically render them as a video using `MoviePy`. johnMinelli
* The gym `py.Dockerfile` is optimised from 2Gb to 1.5Gb through a number of optimisations TheDen