Gymnasium

Over the last few years, the volunteer team behind Gym and Gymnasium has worked to fix bugs, improve the documentation, add new features, and change the API where appropriate such that the benefits outweigh the costs. This is the first alpha release of `v1.0.0`, which aims to be the end of this road of changing the project's API along with containing many new features and improved documentation.

To install v1.0.0a1, you must use `pip install gymnasium==1.0.0a1` or `pip install --pre gymnasium` otherwise, `v0.29.1` will be installed. Similarly, the website will default to v0.29.1's documentation, which can be changed with the pop-up in the bottom right.

We are really interested in projects testing with these v1.0.0 alphas to find any bugs, missing documentation, or issues with the API changes before we release v1.0 in full.

Removing the plugin system
Within Gym v0.23+ and Gymnasium v0.26 to v0.29, an undocumented feature that has existed for registering external environments behind the scenes has been removed. For users of [Atari (ALE)](https://github.com/Farama-Foundation/Arcade-Learning-Environment), [Minigrid](https://github.com/farama-Foundation/minigrid) or [HighwayEnv](https://github.com/Farama-Foundation/HighwayEnv), then users could use the following code:
python
import gymnasium as gym

env = gym.make("ALE/Pong-v5")

such that despite Atari never being imported (i.e., `import ale_py`), users can still load an Atari environment. This feature has been removed in v1.0.0, which will require users to update to
python
import gymnasium as gym
import ale_py

gym.register_envs(ale_py) optional

env = gym.make("ALE/Pong-v5")

Alternatively, users can do the following where the `ale_py` within the environment id will import the module
python
import gymnasium as gym

env = gym.make("ale_py:ALE/Pong-v5") `module_name:env_id`


For users with IDEs (i.e., VSCode, PyCharm), then `import ale_py` can cause the IDE (and pre-commit isort / black / flake8) to believe that the import statement does nothing. Therefore, we have introduced `gymnasium.register_envs` as a no-op function (the function literally does nothing) to make the IDE believe that something is happening and the import statement is required.

Note: ALE-py, Minigrid, and HighwayEnv must be updated to work with Gymnasium v1.0.0, which we hope to complete for all projects affected by alpha 2.

Vector environments
To increase the sample speed of an environment, vectorizing is one of the easiest ways to sample multiple instances of the same environment simultaneously. Gym and Gymnasium provide the `VectorEnv` as a base class for this, but one of its issues has been that it inherited `Env`. This can cause particular issues with type checking (the return type of `step` is different for `Env` and `VectorEnv`), testing the environment type (`isinstance(env, Env)` can be true for vector environments despite the two actings differently) and finally wrappers (some Gym and Gymnasium wrappers supported Vector environments but there are no clear or consistent API for determining which did or didn’t). Therefore, we have separated out `Env` and `VectorEnv` to not inherit from each other.

In implementing the new separate `VectorEnv` class, we have tried to minimize the difference between code using `Env` and `VectorEnv` along with making it more generic in places. The class contains the same attributes and methods as `Env` along with `num_envs: int`, `single_action_space: gymnasium.Space` and `single_observation_space: gymnasium.Space`. Additionally, we have removed several functions from `VectorEnv` that are not needed for all vector implementations: `step_async`, `step_wait`, `reset_async`, `reset_wait`, `call_async` and `call_wait`. This change now allows users to write their own custom vector environments, v1.0.0a1 includes an example vector cartpole environment that runs thousands of times faster than using Gymnasium’s Sync vector environment.

To allow users to create vectorized environments easily, we provide `gymnasium.make_vec` as a vectorized equivalent of `gymnasium.make`. As there are multiple different vectorization options (“sync”, “async”, and a custom class referred to as “vector_entry_point”), the argument `vectorization_mode` selects how the environment is vectorized. This defaults to `None` such that if the environment has a vector entry point for a custom vector environment implementation, this will be utilized first (currently, Cartpole is the only environment with a vector entry point built into Gymnasium). Otherwise, the synchronous vectorizer is used (previously, the Gym and Gymnasium `vector.make` used asynchronous vectorizer as default). For more information, see the function [docstring](https://gymnasium.farama.org/main/api/registry/#gymnasium.make_vec).

python
​​env = gym.make("CartPole-v1")
env = gym.wrappers.ClipReward(env, min_reward=-1, max_reward=3)

envs = gym.make_vec("CartPole-v1", num_envs=3)
envs = gym.wrappers.vector.ClipReward(envs, min_reward=-1, max_reward=3)


Due to this split of `Env` and `VectorEnv`, there are now `Env` only wrappers and `VectorEnv` only wrappers in `gymnasium.wrappers` and `gymnasium.wrappers.vector` respectively. Furthermore, we updated the names of the base vector wrappers from `VectorEnvWrapper` to `VectorWrapper` and added `VectorObservationWrapper`, `VectorRewardWrapper` and `VectorActionWrapper` classes. See the [vector wrapper](https://gymnasium.farama.org/main/api/vector/wrappers/) page for new information.

To increase the efficiency of vector environment, autoreset is a common feature that allows sub-environments to reset without requiring all sub-environments to finish before resetting them all. Previously in Gym and Gymnasium, auto-resetting was done on the same step as the environment episode ends, such that the final observation and info would be stored in the step’s info, i.e., `info["final_observation"]` and `info[“final_info”]` and standard obs and info containing the sub-environment’s reset observation and info. This required similar general sampling for vectorized environments.

python
replay_buffer = []
obs, _ = envs.reset()
for _ in range(total_timesteps):
next_obs, rewards, terminations, truncations, infos = envs.step(envs.action_space.sample())

for j in range(envs.num_envs):
if not (terminations[j] or truncations[j]):
replay_buffer.append((
obs[j], rewards[j], terminations[j], truncations[j], next_obs[j]
))
else:
replay_buffer.append((
obs[j], rewards[j], terminations[j], truncations[j], infos["next_obs"][j]
))

obs = next_obs


However, over time, the development team has recognized the inefficiency of this approach (primarily due to the extensive use of a Python dictionary) and the annoyance of having to extract the final observation to train agents correctly, for [example](https://github.com/vwxyzjn/cleanrl/blob/master/cleanrl/dqn.py#L200). Therefore, in v1.0.0, we are modifying autoreset to align with specialized vector-only projects like [EnvPool](https://github.com/sail-sg/envpool) and [SampleFactory](https://github.com/alex-petrenko/sample-factory) such that the sub-environment’s doesn’t reset until the next step. As a result, this requires the following changes when sampling. For environments with more complex observation spaces (and action actions) then

python
replay_buffer = []
obs, _ = envs.reset()
autoreset = np.zeros(envs.num_envs)
for _ in range(total_timesteps):
next_obs, rewards, terminations, truncations, _ = envs.step(envs.action_space.sample())

for j in range(envs.num_envs):
if not autoreset[j]:
replay_buffer.append((
obs[j], rewards[j], terminations[j], truncations[j], next_obs[j]
))

obs = next_obs
autoreset = np.logical_or(terminations, truncations)
``

Finally, we have improved `AsyncVectorEnv.set_attr` and `SyncVectorEnv.set_attr` functions to use the `Wrapper.set_wrapper_attr` to allow users to set variables anywhere in the environment stack if it already exists. Previously, this was not possible and users could only modify the variable in the “top” wrapper on the environment stack, importantly not the actual environment its self.

Wrappers
Previously, some wrappers could support both environment and vector environments, however, this was not standardized, and was unclear which wrapper did and didn't support vector environments. For v1.0.0, with separating `Env` and `VectorEnv` to no longer inherit from each other (read more in the vector section), the wrappers in `gymnasium.wrappers` will only support standard environments and wrappers in `gymnasium.wrappers.vector` contains the provided specialized vector wrappers (most but not all wrappers are supported, please raise a feature request if you require it).

In v0.29, we deprecated the `Wrapper.__getattr__` function to be replaced by `Wrapper.get_wrapper_attr`, providing access to variables anywhere in the environment stack. In v1.0.0, we have added `Wrapper.set_wrapper_attr` as an equivalent function for setting a variable anywhere in the environment stack if it already exists; only the variable is set in the top wrapper (or environment).

Most significantly, we have removed, renamed, and added several wrappers listed below.
* Removed wrappers
- `monitoring.VideoRecorder` - The replacement wrapper is `RecordVideo`
- `StepAPICompatibility` - We expect all Gymnasium environments to use the terminated / truncated step API, therefore, user shouldn't need the `StepAPICompatibility` wrapper. [Shimmy](https://shimmy.farama.org/) includes a compatibility environments to convert gym-api environment's for gymnasium.
* Renamed wrappers (We wished to make wrappers consistent in naming. Therefore, we have removed "Wrapper" from all wrappers and included "Observation", "Action" and "Reward" within wrapper names where appropriate)
- `AutoResetWrapper` -> `Autoreset`
- `FrameStack` -> `FrameStackObservation`
- `PixelObservationWrapper` -> `AddRenderObservation`
* Moved wrappers (All vector wrappers are in `gymnasium.wrappers.vector`)
- `VectorListInfo` -> `vector.DictInfoToList`
* Added wrappers
- `DelayObservation` - Adds a delay to the next observation and reward
- `DtypeObservation` - Modifies the dtype of an environment’s observation space
- `MaxAndSkipObservation` - Will skip `n` observations and will max over the last 2 observations, inspired by the Atari environment heuristic for other environments
- `StickyAction` - Random repeats actions with a probability for a step returning the final observation and sum of rewards over steps. Inspired by Atari environment heuristics
- `JaxToNumpy` - Converts a Jax-based environment to use Numpy-based input and output data for `reset`, `step`, etc
- `JaxToTorch` - Converts a Jax-based environment to use PyTorch-based input and output data for `reset`, `step`, etc
- `NumpyToTorch` - Converts a Numpy-based environment to use PyTorch-based input and output data for `reset`, `step`, etc

For all wrappers, we have added example code documentation and a changelog to help future researchers understand any changes made. See the following [page](https://gymnasium.farama.org/main/api/wrappers/misc_wrappers/#gymnasium.wrappers.TimeLimit) for an example.

Functional environments
One of the substantial advantages of Gymnasium's `Env` is it generally requires minimal information about the underlying environment specifications however, this can make applying such environments to planning, search algorithms, and theoretical investigations more difficult. We are proposing `FuncEnv` as an alternative definition to `Env` which is closer to a Markov Decision Process definition, exposing more functions to the user, including the observation, reward, and termination functions along with the environment’s raw state as a single object.

python
from typing import Any
import gymnasium as gym
from gymnasium.functional import StateType, ObsType, ActType, RewardType, TerminalType, Params

class ExampleFuncEnv(gym.functional.FuncEnv):
def initial(rng: Any, params: Params | None = None) → StateType
…K
def transition(state: StateType, action: ActType, rng: Any, params: Params | None = None) → StateType
…
def observation(state: StateType, params: Params | None = None) → ObsType
…
def reward(
state: StateType, action: ActType, next_state: StateType, params: Params | None = None
) → RewardType
…
def terminal(state: StateType, params: Params | None = None) → TerminalType
…


`FuncEnv` requires that `initial` and `transition` functions to return a new state given its inputs as a partial implementation of `Env.step` and `Env.reset`. As a result, users can sample (and save) the next state for a range of inputs to use with planning, searching, etc. Given a state, `observation`, `reward`, and `terminal` provide users explicit definitions to understand how each can affect the environment's output.

Additional bug fixes
* Limit the cython version for `gymnasium[mujoco-py]` due to cython==3 issues by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/616)
* Fix `MuJoCo` environment type issues by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/612)
* Fix mujoco rendering with custom width values by logan-dunbar (https://github.com/Farama-Foundation/Gymnasium/pull/634)
* Fix environment checker to correctly report infinite bounds by chrisyeh96 (https://github.com/Farama-Foundation/Gymnasium/pull/708)
* Fix type hint for `register(kwargs)` from `**kwargs` to `kwargs: dict | None = None` by younik (https://github.com/Farama-Foundation/Gymnasium/pull/788)
* Fix `CartPoleVectorEnv` step counter to be set back to zero on `reset` by TimSchneider42 (https://github.com/Farama-Foundation/Gymnasium/pull/886)
* Fix registration for async vector environment for custom environments by RedTachyon (https://github.com/Farama-Foundation/Gymnasium/pull/810)

Additional new features
* New MuJoCo v5 environments (the changes and performance graphs will be included in a separate blog post) by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/572)
* Add support in MuJoCo human rendering to changing the size of the viewing window by logan-dunbar (https://github.com/Farama-Foundation/Gymnasium/pull/635)
* Add more control in MuJoCo rendering over offscreen dimensions and scene geometries by guyazran (https://github.com/Farama-Foundation/Gymnasium/pull/731)
* Add support to handle `NamedTuples` in `JaxToNumpy`, `JaxToTorch` and `NumpyToTorch` by RogerJL (https://github.com/Farama-Foundation/Gymnasium/pull/789) and pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/811)
* Add `padding_type` parameter to `FrameSkipObservation` to select the padding observation by jamartinh (https://github.com/Farama-Foundation/Gymnasium/pull/830)
* Add render check to `check_environments_match` by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/748)

Deprecation
* Remove unnecessary error classes in error.py by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/801)
* Stop exporting MuJoCo v2 environment classes from `gymnasium.envs.mujoco` by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/827)
* Remove deprecation warning from PlayPlot by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/800)

Documentation changes
* Updated the custom environment tutorial for v1.0.0 by kir0ul (https://github.com/Farama-Foundation/Gymnasium/pull/709)
* Add swig to installation instructions for Box2D by btjanaka (https://github.com/Farama-Foundation/Gymnasium/pull/683)
* Add tutorial Load custom quadruped robot environments using `Gymnasium/MuJoCo/Ant-v5` framework by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/838)
* Add third-party tutorial page to list tutorials write and hosted on other websites by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/867)
* Add more introductory pages by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/791)
* Add figures for each MuJoCo environments representing their action space by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/762)
* Fix the documentation on blackjack's starting state by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/893)
* Fix the documentation on Frozenlake and Cliffwalking's position by PierreCounathe (https://github.com/Farama-Foundation/Gymnasium/pull/695)
* Update the classic control environment's `__init__` and `reset` arguments by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/898)

**Full Changelog**: https://github.com/Farama-Foundation/Gymnasium/compare/v0.29.0...v1.0.0a1

A minimal release that fixes a warning produced by `Wrapper.__getattr__`.
In particular, this function will be removed in v1.0.0 however the reported solution for this was incorrect and the updated solution still caused the warning to show (due to technical python reasons).

Changes
* The `Wrapper.__getattr__` warning reports the incorrect new function, `get_attr` rather than `get_wrapper_attr`
* When using `get_wrapper_attr`, the `__getattr__` warning is still be raised due to `get_wrapper_attr` using `hasattr` which under the hood uses `__getattr__.` Therefore, updated to remove the unintended warning.
* Add warning to `VectorEnvWrapper.__getattr__` to specify that it also is deprecated in v1.0.0

**Full Changelog**: https://github.com/Farama-Foundation/Gymnasium/compare/v0.29.0...v0.29.1

We finally have a software citation for Gymnasium with the plan to release an associated paper after v1.0, thank you to all the contributors over the last 3 years who have made helped Gym and Gymnasium (https://github.com/Farama-Foundation/Gymnasium/pull/590)

misc{towers_gymnasium_2023,
title = {Gymnasium},
url = {https://zenodo.org/record/8127025},
abstract = {An API standard for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)},
urldate = {2023-07-08},
publisher = {Zenodo},
author = {Towers, Mark and Terry, Jordan K. and Kwiatkowski, Ariel and Balis, John U. and Cola, Gianluca de and Deleu, Tristan and Goulão, Manuel and Kallinteris, Andreas and KG, Arjun and Krimmel, Markus and Perez-Vicente, Rodrigo and Pierré, Andrea and Schulhoff, Sander and Tai, Jun Jet and Shen, Andrew Tan Jin and Younis, Omar G.},
month = mar,
year = {2023},
doi = {10.5281/zenodo.8127026},
}


Gymnasium has a [conda package](https://github.com/conda-forge/gymnasium-feedstock), `conda install gymnasium`. Thanks to ChristofKaufmann for completing this

Breaking Changes
* Drop support for Python 3.7 which has reached its end of life support by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/573
* Update MuJoCo Hopper & Walker2D models to work with MuJoCo >= 2.3.3 by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/589
* Add deprecation warnings to several features which will be removed in v1.0: `Wrapper.__get_attr__`, `gymnasium.make(..., autoreset=True)`, `gymnasium.make(..., apply_api_compatibility=True)`, `Env.reward_range` and `gymnasium.vector.make`. For their proposed replacement, see https://github.com/Farama-Foundation/Gymnasium/pull/535
* Raise error for `Box` bounds of `low > high`, `low == inf` and `high == -inf` by jjshoots in https://github.com/Farama-Foundation/Gymnasium/pull/495
* Add dtype testing for NumPy Arrays in `data_equivalence()` by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/515
* Remove [Jumpy](https://github.com/farama-Foundation/jumpy) from gymnasium wrappers as it was partially implemented with limited testing and usage by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/548
* Update project require for `jax>=0.4` by charraut in https://github.com/Farama-Foundation/Gymnasium/pull/373

New Features
* Remove the restrictions on pygame version, `pygame>=2.1.3` by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/558
* Adding `start` parameter to `MultiDiscrete` space, similar to the `Discrete(..., start)` parameter by Rayerdyne in https://github.com/Farama-Foundation/Gymnasium/pull/557
* Adds testing to `check_env` that closing a closed environment doesn't raise an error by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/564
* On initialisation `wrapper.RecordVideo` throws an error if the environment has an invalid render mode `(None, "human", "ansi")` by robertoschiavone in https://github.com/Farama-Foundation/Gymnasium/pull/580
* Add `MaxAndSkipObservation` wrapper by LucasAlegre in https://github.com/Farama-Foundation/Gymnasium/pull/561
* Add `check_environments_match` function for checking if two environments are identical by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/576
* Add performance debugging utilities, `utils/performance.py` by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/583
* Added Jax based cliff walking environment by balisujohn in https://github.com/Farama-Foundation/Gymnasium/pull/407
* MuJoCo
* Add support for relative paths with `xml_file` arguments by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/536
* Add support for environments to specify `info` in `reset` by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/540
* Remove requirement of environments defining `metadata["render_fps"]`, the value is determined on `__init__` using `dt` by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/525
* Experimental
* Add deprecated wrapper error in `gymnasium.experimental.wrappers` by charraut in https://github.com/Farama-Foundation/Gymnasium/pull/341
* Add `fps` argument to `RecordVideoV0` for custom fps value that overrides an environment's internal `render_fps` value by younik in https://github.com/Farama-Foundation/Gymnasium/pull/503
* Add experimental vector wrappers for lambda observation, action and reward wrappers by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/444

Bug Fixes
* Fix `spaces.Dict.keys()` as `key in keys` was False by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/608
* Updates the action space of `wrappers.RescaleAction` based on the bounds by mmcaulif in https://github.com/Farama-Foundation/Gymnasium/pull/569
* Remove warnings in the passive environment checker for infinite Box bounds by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/435
* Revert Lunar Lander Observation space change by alexdlukens in https://github.com/Farama-Foundation/Gymnasium/pull/512
* Fix URL links in `check_env` by robertoschiavone in https://github.com/Farama-Foundation/Gymnasium/pull/554
* Update `shimmy[gym]` to `shimmy[gym-v21]` or `shimmy[gym-v26]` by elliottower in https://github.com/Farama-Foundation/Gymnasium/pull/433
* Fix several issues within the experimental vector environment and wrappers by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/516
* Video recorder wrapper
* Fix `VideoRecorder` on `reset` to empty `recorded_frames` rather than `frames` by voidflight in https://github.com/Farama-Foundation/Gymnasium/pull/518
* Remove `Env.close` in `VideoRecorder.close` by qgallouedec in https://github.com/Farama-Foundation/Gymnasium/pull/533
* Fix `VideoRecorder` and `RecordVideoV0` to move `import moviepy` such that `__del__` doesn't raise `AttributeErrors` by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/553
* Mujoco
* Remove Hopper-v4's old render API func by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/588
* Fix TypeError when closing rendering by sonelu in (https://github.com/Farama-Foundation/Gymnasium/pull/440)
* Fix the wrong `nstep` in `_step_mujoco_simulation` function of `MujocoEnv` by xuanhien070594 in https://github.com/Farama-Foundation/Gymnasium/pull/424
* Allow a different number of actuator control from the action space by reginald-mclean in https://github.com/Farama-Foundation/Gymnasium/pull/604

Documentation Updates
* Allow users to view source code of referenced objects on the website by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/497
* Update website homepage by elliottower in https://github.com/Farama-Foundation/Gymnasium/pull/482
* Make atari documentation consistent by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/418 and add missing descriptions by dylwil3 in https://github.com/Farama-Foundation/Gymnasium/pull/510
* Add third party envs: safety gymnasium, pyflyt, Gym-Trading-Env, stable-retro, DACBench, gym-cellular-automata by elliottower, stefanbschneider, ClementPerroud, jjshoots, MatPoliquin, and robertoschiavone in 450, 451, 474, 487, 529, 538, 581
* Update MuJoCo documentation for all environments and base mujoco environment by Kallinteris-Andreas in 524, 522
* Update CartPole reward documentation to clarify different maximum rewards for v0 and v1 by robertoschiavone in https://github.com/Farama-Foundation/Gymnasium/pull/429
* Clarify Frozen lake time limit for `FrozenLake4x4` and `FrozenLake8x8` environments by yaniv-peretz in https://github.com/Farama-Foundation/Gymnasium/pull/459
* Typo in the documentation for single_observation_space by kvrban in https://github.com/Farama-Foundation/Gymnasium/pull/491
* Fix the rendering of warnings on the website by helpingstar in https://github.com/Farama-Foundation/Gymnasium/pull/520

**Full Changelog**: https://github.com/Farama-Foundation/Gymnasium/compare/v0.28.1...v0.29.0

Small emergency release to fix several issues

* Fixed `gymnasium.vector` as the `gymnasium/__init__.py` as it isn't imported https://github.com/Farama-Foundation/Gymnasium/pull/403
* Update third party envs to separate environments that support gymnasium and gym and have a consistent style https://github.com/Farama-Foundation/Gymnasium/pull/404
* Update the documentation for v0.28 as frontpage gif had the wrong link, experimental documentation was missing and add gym release notes https://github.com/Farama-Foundation/Gymnasium/pull/405

**Full Changelog**: https://github.com/Farama-Foundation/Gymnasium/compare/v0.28.0...v0.28.1

This release introduces improved support for the reproducibility of Gymnasium environments, particularly for offline reinforcement learning. `gym.make` can now create the entire environment stack, including wrappers, such that training libraries or offline datasets can specify all of the arguments and wrappers used for an environment. For a majority of standard usage (`gym.make(”EnvironmentName-v0”)`), this will be backwards compatible except for certain fairly uncommon cases (i.e. `env.spec` and `env.unwrapped.spec` return different specs) this is a breaking change. See the reproducibility details section for more info.
In v0.27, we added the `experimental` folder to allow us to develop several new features (wrappers and hardware accelerated environments). We’ve introduced a new experimental `VectorEnv` class. This class does not inherit from the standard `Env` class, and will allow for dramatically more efficient parallelization features. We plan to improve the implementation and add vector based wrappers in several minor releases over the next few months.
Additionally, we have optimized module loading so that PyTorch or Jax are only loaded when users import wrappers that require them, not on `import gymnasium`.
Reproducibility details
In previous versions, Gymnasium supported `gym.make(spec)` where the `spec` is an `EnvSpec` from `gym.spec(str)` or `env.spec` and worked identically to the string based `gym.make(“”)`. In both cases, there was no way to specify additional wrappers that should be applied to an environment. With this release, we added `additional_wrappers` to `EnvSpec` for specifying wrappers applied to the base environment (`TimeLimit`, `PassiveEnvChecker`, `Autoreset` and `ApiCompatibility` are not included as they are specify in other fields).
This additional field will allow users to accurately save or reproduce an environment used in training for a policy or to generate an offline RL dataset. We provide a json converter function (`EnvSpec.to_json`) for saving the `EnvSpec` to a “safe” file type however there are several cases (NumPy data, functions) which cannot be saved to json. In these cases, we recommend pickle but be warned that this can allow remote users to include malicious data in the spec.
python
import gymnasium as gym

env = gym.make("CartPole-v0")
env = gym.wrappers.TimeAwareObservation(env)
print(env)
<TimeAwareObservation<TimeLimit<OrderEnforcing<PassiveEnvChecker<CartPoleEnv<CartPole-v0>>>>>>
env_spec = env.spec
env_spec.pprint()
id=CartPole-v0
reward_threshold=195.0
max_episode_steps=200
additional_wrappers=[
name=TimeAwareObservation, kwargs={}
]

import json
import pickle

json_env_spec = json.loads(env_spec.to_json())
pickled_env_spec = pickle.loads(pickle.dumps(env_spec))
recreated_env = gym.make(json_env_spec)
print(recreated_env)
<TimeAwareObservation<TimeLimit<OrderEnforcing<PassiveEnvChecker<CartPoleEnv<CartPole-v0>>>>>>
Be aware that the `TimeAwareObservation` was included by `make`

To support this type of recreation, wrappers must inherit from `gym.utils.RecordConstructorUtils` to allow `gym.make` to know what arguments to create the wrapper with. Gymnasium has implemented this for all built-in wrappers but for external projects, should be added to each wrapper. To do this, call `gym.utils.RecordConstructorUtils.__init__(self, …)` in the first line of the wrapper’s constructor with identical l keyword arguments as passed to the wrapper’s constructor, except for `env`. As an example see the [Atari Preprocessing wrapper](https://github.com/Farama-Foundation/Gymnasium/blob/8c167b868dd386ac1eb4bbe2fb25da3da174c75d/gymnasium/experimental/wrappers/atari_preprocessing.py#L65)
For a more detailed discussion, see the original PRs - https://github.com/Farama-Foundation/Gymnasium/pull/292 and https://github.com/Farama-Foundation/Gymnasium/pull/355
Other Major Changes
* In Gymnasium v0.26, the `GymV22Compatibility` environment was added to support Gym-based environments in Gymnasium. However, the name was incorrect as the env supported Gym’s v0.21 API, not v0.22, therefore, we have updated it to `GymV21Compatibility` to accurately reflect the API supported. https://github.com/Farama-Foundation/Gymnasium/pull/282
* The `Sequence` space allows for a dynamic number of elements in an observation or action space sample. To make this more efficient, we added a `stack` argument which can support which can support a more efficient representation of an element than a `tuple`, which was what was previously supported. https://github.com/Farama-Foundation/Gymnasium/pull/284
* `Box.sample` previously would clip incorrectly for up-bounded spaces such that 0 could never be sampled if the dtype was discrete or boolean. This is fixed such that 0 can be sampled in these cases. https://github.com/Farama-Foundation/Gymnasium/pull/249
* If `jax` or `pytorch` was installed then on `import gymnasium` both of these modules would also be loaded causing significant slow downs in load time. This is now fixed such that `jax` and `torch` are only loaded when particular wrappers is loaded by the user. https://github.com/Farama-Foundation/Gymnasium/pull/323
* In v0.26, we added parameters for `Wrapper` to allow different observation and action types to be specified for the wrapper and its sub-environment. However, this raised type issues with pyright and mypy, this is now fixed through Wrapper having four generic arguments, `[ObsType, ActType, WrappedEnvObsType, WrappedEnvActType]`. https://github.com/Farama-Foundation/Gymnasium/pull/337
* In v0.25 and 0.v26 several new space types were introduced, `Text`, `Graph` and `Sequence` however the vector utility functions were not updated to support these spaces. Support for these spaces has been added to the experimental vector space utility functions: `batch_space`, `concatenate`, `iterate` and `create_empty_array`. https://github.com/Farama-Foundation/Gymnasium/pull/223
* Due to a lack of testing the experimental stateful observation wrappers (`FrameStackObservation`, `DelayObservation` and `TimeAwareObservation`) did not work as expected. These wrappers are now fixed and testing has been added. https://github.com/Farama-Foundation/Gymnasium/pull/224

Minor changes
* Allow the statistics of NormalizeX wrappers to be disabled and enabled for use during evaluation by raphajaner in https://github.com/Farama-Foundation/Gymnasium/pull/268
* Fix AttributeError in lunar_lander.py by DrRyanHuang in https://github.com/Farama-Foundation/Gymnasium/pull/278
* Add testing for docstrings (doctest) such that docstrings match implementations by valentin-cnt in https://github.com/Farama-Foundation/Gymnasium/pull/281
* Type hint fixes and added `__all__` dunder by howardh in https://github.com/Farama-Foundation/Gymnasium/pull/321
* Fix type hints errors in gymnasium/spaces by valentin-cnt in https://github.com/Farama-Foundation/Gymnasium/pull/327
* Update the experimental vector shared memory util functions by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/339
* Change Gymnasium Notices to Farama Notifications by jjshoots in https://github.com/Farama-Foundation/Gymnasium/pull/332
* Added Jax-based Blackjack environment by balisujohn in https://github.com/Farama-Foundation/Gymnasium/pull/338

Documentation changes
* Fix references of the MultiBinary and MultiDiscrete classes in documentation by Matyasch in https://github.com/Farama-Foundation/Gymnasium/pull/279
* Add Comet integration by nerdyespresso in https://github.com/Farama-Foundation/Gymnasium/pull/304
* Update atari documentation by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/330
* Document Box integer bounds by mihaic in https://github.com/Farama-Foundation/Gymnasium/pull/331
* Add docstring parser to remove duplicate in Gymnasium website by valentin-cnt in https://github.com/Farama-Foundation/Gymnasium/pull/329
* Fix a grammatical mistake in basic usage page by keyb0ardninja in https://github.com/Farama-Foundation/Gymnasium/pull/333
* Update docs/README.md to link to a new CONTRIBUTING.md for docs by mgoulao in https://github.com/Farama-Foundation/Gymnasium/pull/340
* `MuJoCo/Ant` clarify the lack of `use_contact_forces` on v3 (and older) by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/342

What's Changed

Thank you to our new contributors in this release: Matyasch, DrRyanHuang, nerdyespresso, khoda81, howardh, mihaic, and keyb0ardninja.

**Full Changelog**: https://github.com/Farama-Foundation/Gymnasium/compare/v0.27.1...v0.28.0

Release Notes

Bugs fixed

* Replace `np.bool8` with `np.bool_` for numpy 1.24 deprecation warning by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/221
* Remove shimmy as a core dependency by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/272
* Fix silent bug in ResizeObservation for 2-dimensional observations. by ianyfan in https://github.com/Farama-Foundation/Gymnasium/pull/230 and by RedTachyon in https://github.com/Farama-Foundation/Gymnasium/pull/254
* Change env checker assertation to warning by jjshoots in https://github.com/Farama-Foundation/Gymnasium/pull/215
* Revert `make` error when render mode is used without metadata render modes by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/216
* Update prompt messages for extra dependencies by XuehaiPan in https://github.com/Farama-Foundation/Gymnasium/pull/250
* Fix return type of `AsyncVectorEnv.reset` by younik in https://github.com/Farama-Foundation/Gymnasium/pull/252
* Update the jumpy error to specify the pip install is jax-jumpy by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/255
* Fix type annotations of `callable` to `Callable` by ianyfan in https://github.com/Farama-Foundation/Gymnasium/pull/259
* Fix experimental normalize reward wrapper by rafaelcp in https://github.com/Farama-Foundation/Gymnasium/pull/277

New features/improvements

* Improve LunarLander-v2 `step` performance by >1.5x by PaulMest in https://github.com/Farama-Foundation/Gymnasium/pull/235
* Added vector env support to StepAPICompatibility wrapper by nidhishs in https://github.com/Farama-Foundation/Gymnasium/pull/238
* Allow sequence to accept stacked np arrays if the feature space is Box by jjshoots in https://github.com/Farama-Foundation/Gymnasium/pull/241
* Improve the warning when an error is raised from a plugin by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/225
* Add changelog (release notes) to the website by mgoulao in https://github.com/Farama-Foundation/Gymnasium/pull/257
* Implement RecordVideoV0 by younik in https://github.com/Farama-Foundation/Gymnasium/pull/246
* Add explicit error messages when unflatten discrete and multidiscrete fail by PierreMardon in https://github.com/Farama-Foundation/Gymnasium/pull/267

Documentation updates
* Added doctest to CI and fixed all existing errors in docstrings by valentin-cnt in https://github.com/Farama-Foundation/Gymnasium/pull/274
* Add a tutorial for vectorized envs using A2C. by till2 in https://github.com/Farama-Foundation/Gymnasium/pull/234
* Fix `MuJoCo.Humanoid` action description by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/206
* `Ant` `use_contact_forces` obs and reward DOC by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/218
* `MuJoCo.Reacher-v4` doc fixes by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/219
* Mention truncation in the migration guide by RedTachyon in https://github.com/Farama-Foundation/Gymnasium/pull/105
* docs(tutorials): fixed environment creation link by lpizzinidev in https://github.com/Farama-Foundation/Gymnasium/pull/244
* `Mujoco/Hooper` doc minor typo fix by Kallinteris-Andreas in https://github.com/Farama-Foundation/Gymnasium/pull/247
* Add comment describing what convolve does in A2C tutorial by metric-space in https://github.com/Farama-Foundation/Gymnasium/pull/264
* Fix environment versioning in README.md by younik in https://github.com/Farama-Foundation/Gymnasium/pull/270
* Add Tutorials galleries by mgoulao in https://github.com/Farama-Foundation/Gymnasium/pull/258

Thanks to the new contributors to Gymnasium, if you want to get involved, join our discord server. Linked in the readme.
* PaulMest made their first contribution in https://github.com/Farama-Foundation/Gymnasium/pull/235
* nidhishs made their first contribution in https://github.com/Farama-Foundation/Gymnasium/pull/238
* lpizzinidev made their first contribution in https://github.com/Farama-Foundation/Gymnasium/pull/244
* ianyfan made their first contribution in https://github.com/Farama-Foundation/Gymnasium/pull/230
* metric-space made their first contribution in https://github.com/Farama-Foundation/Gymnasium/pull/264
* PierreMardon made their first contribution in https://github.com/Farama-Foundation/Gymnasium/pull/267
* valentin-cnt made their first contribution in https://github.com/Farama-Foundation/Gymnasium/pull/274
* rafaelcp made their first contribution in https://github.com/Farama-Foundation/Gymnasium/pull/277

**Full Changelog**: https://github.com/Farama-Foundation/Gymnasium/compare/v0.27.0...v0.27.1