Over the last few years, the volunteer team behind Gym and Gymnasium has worked to fix bugs, improve the documentation, add new features, and change the API where appropriate so that the benefits outweigh the costs. This is the complete release of `v1.0.0`, which will be the end of this road to change the project's central API (`Env`, `Space`, `VectorEnv`). In addition, the release has included over 200 PRs since `0.29.1`, with many bug fixes, new features, and improved documentation. So, thank you to all the volunteers for their hard work that has made this possible. For the rest of these release notes, we include sections of core API changes, ending with the additional new features, bug fixes, deprecation and documentation changes included.
Finally, we have published a paper on Gymnasium, discussing its overall design decisions and more at https://arxiv.org/abs/2407.17032, which can be cited using the following:
misc{towers2024gymnasium,
title={Gymnasium: A Standard Interface for Reinforcement Learning Environments},
author={Mark Towers and Ariel Kwiatkowski and Jordan Terry and John U. Balis and Gianluca De Cola and Tristan Deleu and Manuel Goulão and Andreas Kallinteris and Markus Krimmel and Arjun KG and Rodrigo Perez-Vicente and Andrea Pierré and Sander Schulhoff and Jun Jet Tai and Hannah Tan and Omar G. Younis},
year={2024},
eprint={2407.17032},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2407.17032},
}
Removing The Plugin System
Within Gym v0.23+ and Gymnasium v0.26 to v0.29, an undocumented feature for registering external environments behind the scenes has been removed. For users of [Atari (ALE)](https://github.com/Farama-Foundation/Arcade-Learning-Environment), [Minigrid](https://github.com/farama-Foundation/minigrid) or [HighwayEnv](https://github.com/Farama-Foundation/HighwayEnv), then users could previously use the following code:
python
import gymnasium as gym
env = gym.make("ALE/Pong-v5")
Despite Atari never being imported (i.e., `import ale_py`), users can still create an Atari environment. This feature has been removed in `v1.0.0`, which will require users to update to
python
import gymnasium as gym
import ale_py
gym.register_envs(ale_py) optional, helpful for IDEs or pre-commit
env = gym.make("ALE/Pong-v5")
Alternatively, users can use the following structure, `module_name:env_id, ' so that the module is imported first before the environment is created. e.g., `ale_py:ALE/Pong-v5`.
python
import gymnasium as gym
env = gym.make("ale_py:ALE/Pong-v5")
To help users with IDEs (e.g., VSCode, PyCharm), when importing modules to register environments (e.g., `import ale_py`) this can cause the IDE (and pre-commit isort / black / flake8) to believe that the import is pointless and should be removed. Therefore, we have introduced `gymnasium.register_envs` as a no-op function (the function literally does nothing) to make the IDE believe that something is happening and the import statement is required.
Vector Environments
To increase the sample speed of an environment, vectorizing is one of the easiest ways to sample multiple instances of the same environment simultaneously. Gym and Gymnasium provide the `VectorEnv` as a base class for this, but one of its issues has been that it inherited `Env`. This can cause particular issues with type checking (the return type of `step` is different for `Env` and `VectorEnv`), testing the environment type (`isinstance(env, Env)` can be true for vector environments despite the two acting differently) and finally wrappers (some Gym and Gymnasium wrappers supported Vector environments, but there are no clear or consistent API for determining which do or don't). Therefore, we have separated out `Env` and `VectorEnv` to not inherit from each other.
In implementing the new separate `VectorEnv` class, we have tried to minimize the difference between code using `Env` and `VectorEnv` along with making it more generic in places. The class contains the same attributes and methods as `Env` in addition to the attributes `num_envs: int`, `single_action_space: gymnasium.Space` and `single_observation_space: gymnasium.Space`. Further, we have removed several functions from `VectorEnv` that are not needed for all vector implementations: `step_async`, `step_wait`, `reset_async`, `reset_wait`, `call_async` and `call_wait`. This change now allows users to write their own custom vector environments, v1.0.0 includes an example vector cartpole environment that runs thousands of times faster written solely with NumPy than using Gymnasium's Sync vector environment.
To allow users to create vectorized environments easily, we provide `gymnasium.make_vec` as a vectorized equivalent of `gymnasium.make`. As there are multiple different vectorization options ("sync", "async", and a custom class referred to as "vector_entry_point"), the argument `vectorization_mode` selects how the environment is vectorized. This defaults to `None` such that if the environment has a vector entry point for a custom vector environment implementation, this will be utilized first (currently, Cartpole is the only environment with a vector entry point built into Gymnasium). Otherwise, the synchronous vectorizer is used (previously, the Gym and Gymnasium `vector.make` used asynchronous vectorizer as default). For more information, see the function [docstring](https://gymnasium.farama.org/main/api/registry/#gymnasium.make_vec). We are excited to see other projects utilize this option to make creating their environments easier.
python
env = gym.make("CartPole-v1")
env = gym.wrappers.ClipReward(env, min_reward=-1, max_reward=3)
envs = gym.make_vec("CartPole-v1", num_envs=3)
envs = gym.wrappers.vector.ClipReward(envs, min_reward=-1, max_reward=3)
Due to this split of `Env` and `VectorEnv`, there are now `Env` only wrappers and `VectorEnv` only wrappers in `gymnasium.wrappers` and `gymnasium.wrappers.vector` respectively. Furthermore, we updated the names of the base vector wrappers from `VectorEnvWrapper` to `VectorWrapper` and added `VectorObservationWrapper`, `VectorRewardWrapper` and `VectorActionWrapper` classes. See the [vector wrapper](https://gymnasium.farama.org/main/api/vector/wrappers/) page for new information.
To increase the efficiency of vector environments, autoreset is a common feature that allows sub-environments to reset without requiring all sub-environments to finish before resetting them all. Previously in Gym and Gymnasium, auto-resetting was done on the same step as the environment episode ends, such that the final observation and info would be stored in the step's info, i.e., `info["final_observation"]` and `info[“final_info”]` and standard obs and info containing the sub-environment's reset observation and info. Thus, accurately sampling observations from a vector environment required the following code (note the need to extract the `infos["next_obs"][j]` if the sub-environment was terminated or truncated). Additionally, for on-policy algorithms that use rollout would require an additional forward pass to compute the correct next observation (this is often not done as an optimization assuming that environments only terminate, not truncate).
python
replay_buffer = []
obs, _ = envs.reset()
for _ in range(total_timesteps):
next_obs, rewards, terminations, truncations, infos = envs.step(envs.action_space.sample())
for j in range(envs.num_envs):
if not (terminations[j] or truncations[j]):
replay_buffer.append((
obs[j], rewards[j], terminations[j], truncations[j], next_obs[j]
))
else:
replay_buffer.append((
obs[j], rewards[j], terminations[j], truncations[j], infos["next_obs"][j]
))
obs = next_obs
However, over time, the development team has recognized the inefficiency of this approach (primarily due to the extensive use of a Python dictionary) and the annoyance of having to extract the final observation to train agents correctly, for [example](https://github.com/vwxyzjn/cleanrl/blob/c37a3ec4ef8d33ab7c8a69d4d2714e3817739365/cleanrl/dqn.py#L174). Therefore, in v1.0.0, we are modifying autoreset to align with specialized vector-only projects like [EnvPool](https://github.com/sail-sg/envpool) and [SampleFactory](https://github.com/alex-petrenko/sample-factory) where the sub-environment's doesn't reset until the next step. As a result, the following changes are required when sampling:
python
replay_buffer = []
obs, _ = envs.reset()
autoreset = np.zeros(envs.num_envs)
for _ in range(total_timesteps):
next_obs, rewards, terminations, truncations, _ = envs.step(envs.action_space.sample())
for j in range(envs.num_envs):
if not autoreset[j]:
replay_buffer.append((
obs[j], rewards[j], terminations[j], truncations[j], next_obs[j]
))
obs = next_obs
autoreset = np.logical_or(terminations, truncations)
For on-policy rollout, to account for the autoreset requires masking the error for the first observation in a new episode (`done[t+1]`) to prevent computing the error between the last and first observations of episodes.
Finally, we have improved `AsyncVectorEnv.set_attr` and `SyncVectorEnv.set_attr` functions to use the `Wrapper.set_wrapper_attr` to allow users to set variables anywhere in the environment stack if it already exists. Previously, this was not possible and users could only modify the variable in the "top" wrapper on the environment stack, importantly not the actual environment itself.
Wrappers
Previously, some wrappers could support both environment and vector environments, however, this was not standardized, and was unclear which wrapper did and didn't support vector environments. For v1.0.0, with separating `Env` and `VectorEnv` to no longer inherit from each other (read more in the vector section), the wrappers in `gymnasium.wrappers` will only support standard environments and wrappers in `gymnasium.wrappers.vector` contains the provided specialized vector wrappers (most but not all wrappers are supported, please raise a feature request if you require it).
In v0.29, we deprecated the `Wrapper.__getattr__` function to be replaced by `Wrapper.get_wrapper_attr`, providing access to variables anywhere in the environment stack. In v1.0.0, we have added `Wrapper.set_wrapper_attr` as an equivalent function for setting a variable anywhere in the environment stack if it already exists; otherwise the variable is assigned to the top wrapper.
Most significantly, we have removed, renamed, and added several wrappers listed below.
* Removed wrappers
- `monitoring.VideoRecorder` - The replacement wrapper is `RecordVideo`
- `StepAPICompatibility` - We expect all Gymnasium environments to use the terminated / truncated step API, therefore, users shouldn't need the `StepAPICompatibility` wrapper. [Shimmy](https://shimmy.farama.org/) includes a compatibility environment to convert gym-api environments for gymnasium.
* Renamed wrappers (We wished to make wrappers consistent in naming. Therefore, we have removed "Wrapper" from all wrappers and included "Observation", "Action" and "Reward" within wrapper names where appropriate)
- `AutoResetWrapper` -> `Autoreset`
- `FrameStack` -> `FrameStackObservation`
- `PixelObservationWrapper` -> `AddRenderObservation`
* Moved wrappers (All vector wrappers are in `gymnasium.wrappers.vector`)
- `VectorListInfo` -> `vector.DictInfoToList`
* Added wrappers
- `DelayObservation` - Adds a delay to the next observation and reward
- `DtypeObservation` - Modifies the dtype of an environment's observation space
- `MaxAndSkipObservation` - Will skip `n` observations and will max over the last 2 observations, inspired by the Atari environment heuristic for other environments
- `StickyAction` - Random repeats actions with a probability for a step returning the final observation and sum of rewards over steps. Inspired by Atari environment heuristics
- `JaxToNumpy` - Converts a Jax-based environment to use Numpy-based input and output data for `reset`, `step`, etc
- `JaxToTorch` - Converts a Jax-based environment to use PyTorch-based input and output data for `reset`, `step`, etc
- `NumpyToTorch` - Converts a Numpy-based environment to use PyTorch-based input and output data for `reset`, `step`, etc
For all wrappers, we have added example code documentation and a changelog to help future researchers understand any changes made. See the following [page](https://gymnasium.farama.org/main/api/wrappers/misc_wrappers/#gymnasium.wrappers.TimeLimit) for an example.
Functional Environments
One of the substantial advantages of Gymnasium's `Env` is it generally requires minimal information about the underlying environment specifications; however, this can make applying such environments to planning, search algorithms, and theoretical investigations more difficult. We are proposing `FuncEnv` as an alternative definition to `Env` which is closer to a Markov Decision Process definition, exposing more functions to the user, including the observation, reward, and termination functions along with the environment's raw state as a single object.
python
from typing import Any
import gymnasium as gym
from gymnasium.functional import StateType, ObsType, ActType, RewardType, TerminalType, Params
class ExampleFuncEnv(gym.functional.FuncEnv):
def initial(self, rng: Any, params: Params | None = None) -> StateType:
...
def transition(self, state: StateType, action: ActType, rng: Any, params: Params | None = None) -> StateType:
...
def observation(self, state: StateType, rng: Any, params: Params | None = None) -> ObsType:
...
def reward(
self, state: StateType, action: ActType, next_state: StateType, rng: Any, params: Params | None = None
) -> RewardType:
...
def terminal(self, state: StateType, rng: Any, params: Params | None = None) -> TerminalType:
...
`FuncEnv` requires that `initial` and `transition` functions return a new state given its inputs as a partial implementation of `Env.step` and `Env.reset`. As a result, users can sample (and save) the next state for a range of inputs to use with planning, searching, etc. Given a state, `observation`, `reward`, and `terminal` provide users explicit definitions to understand how each can affect the environment's output.
Collecting Seeding Values
It was possible to seed with both environments and spaces with `None` to use a random initial seed value, however it wouldn't be possible to know what these initial seed values were. We have addressed this for `Space.seed` and `reset.seed` in https://github.com/Farama-Foundation/Gymnasium/pull/1033 and https://github.com/Farama-Foundation/Gymnasium/pull/889. Additionally, for `Space.seed`, we have changed the return type to be specialized for each space such that the following code will work for all spaces.
python
seeded_values = space.seed(None)
initial_samples = [space.sample() for _ in range(10)]
reseed_values = space.seed(seeded_values)
reseed_samples = [space.sample() for _ in range(10)]
assert seeded_values == reseed_values
assert initial_samples == reseed_samples
Additionally, for environments, we have added a new `np_random_seed` attribute that will store the most recent `np_random` seed value from `reset(seed=seed)`.
Environment Version Changes
* It was discovered recently that the MuJoCo-based Pusher was not compatible with `mujoco>= 3` as the model's density for the block that the agent had to push was lighter than air. This obviously began to cause issues for users with `mujoco>= 3` and Pusher. Therefore, we are disabled the `v4` environment with `mujoco>= 3` and updated to the model in MuJoCo `v5` that produces more expected behavior like `v4` and `mujoco< 3` (https://github.com/Farama-Foundation/Gymnasium/pull/1019).
* New v5 MuJoCo environments as a follow-up to v4 environments added two years ago, fixing consistencies, adding new features and updating the documentation (https://github.com/Farama-Foundation/Gymnasium/pull/572). Additionally, we have decided to mark the mujoco-py based (v2 and v3) environments as deprecated and plan to remove them from Gymnasium in future (https://github.com/Farama-Foundation/Gymnasium/pull/926).
* Lunar Lander version increased from v2 to v3 due to two bug fixes. The first fixes the determinism of the environment such that the world object was not completely destroyed on reset causing non-determinism in particular cases (https://github.com/Farama-Foundation/Gymnasium/pull/979). Second, the wind generation (by default turned off) was not randomly generated by each reset, therefore, we have updated this to gain statistical independence between episodes (https://github.com/Farama-Foundation/Gymnasium/pull/959).
* CarRacing version increased from v2 to v3 to change how the environment ends such that when the agent completes the track then the environment will terminate not truncate.
* We have remove `pip install "gymnasium[accept-rom-license]"` as `ale-py>=0.9` now comes packaged with the roms meaning that users don't need to install the atari roms separately with `autoroms`.
Additional Bug Fixes
* `spaces.Box` would allow low and high values outside the dtype's range, which could result in some very strange edge cases that were very difficult to detect by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/774)
* Limit the cython version for `gymnasium[mujoco-py]` due to `cython==3` issues by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/616)
* Fix mujoco rendering with custom width values by logan-dunbar (https://github.com/Farama-Foundation/Gymnasium/pull/634)
* Fix environment checker to correctly report infinite bounds by chrisyeh96 (https://github.com/Farama-Foundation/Gymnasium/pull/708)
* Fix type hint for `register(kwargs)` from `**kwargs` to `kwargs: dict | None = None` by younik (https://github.com/Farama-Foundation/Gymnasium/pull/788)
* Fix registration in `AsyncVectorEnv` for custom environments by RedTachyon (https://github.com/Farama-Foundation/Gymnasium/pull/810)
* Remove `mujoco-py` import error for v4+ MuJoCo environments by MischaPanch
(https://github.com/Farama-Foundation/Gymnasium/pull/934)
* Fix reading shared memory for `Tuple` and `Dict` spaces (https://github.com/Farama-Foundation/Gymnasium/pull/941)
* Fix `Multidiscrete.from_jsonable` on windows (https://github.com/Farama-Foundation/Gymnasium/pull/932)
* Remove `play` rendering normalization (https://github.com/Farama-Foundation/Gymnasium/pull/956)
* Fix non-used device argument in `to_torch` conversion by mantasu (https://github.com/Farama-Foundation/Gymnasium/pull/1107)
* Fix torch to numpy conversion when on GPU by mantasu (https://github.com/Farama-Foundation/Gymnasium/pull/1109)
Additional new features
* Added Python 3.12 and NumPy 2.0 support by RedTachyon in https://github.com/Farama-Foundation/Gymnasium/pull/1094
* Add support in MuJoCo human rendering to change the size of the viewing window by logan-dunbar (https://github.com/Farama-Foundation/Gymnasium/pull/635)
* Add more control in MuJoCo rendering over offscreen dimensions and scene geometries by guyazran (https://github.com/Farama-Foundation/Gymnasium/pull/731)
* Add stack trace reporting to `AsyncVectorEnv` by pseudo-rnd-thoughts in https://github.com/Farama-Foundation/Gymnasium/pull/1119
* Add support to handle `NamedTuples` in `JaxToNumpy`, `JaxToTorch` and `NumpyToTorch` by RogerJL (https://github.com/Farama-Foundation/Gymnasium/pull/789) and pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/811)
* Add `padding_type` parameter to `FrameSkipObservation` to select the padding observation by jamartinh (https://github.com/Farama-Foundation/Gymnasium/pull/830)
* Add render check to `check_environments_match` by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/748)
* Add a new `OneOf` space that provides exclusive unions of spaces by RedTachyon and pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/812)
* Update `Dict.sample` to use standard Python dicts rather than `OrderedDict` due to dropping Python 3.7 support by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/977)
* Jax environment return jax data rather than numpy data by RedTachyon and pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/817)
* Add `wrappers.vector.HumanRendering` and remove human rendering from `CartPoleVectorEnv` by pseudo-rnd-thoughts and TimSchneider42 (https://github.com/Farama-Foundation/Gymnasium/pull/1013)
* Add more helpful error messages if users use a mixture of Gym and Gymnasium by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/957)
* Add `sutton_barto_reward` argument for `CartPole` that changes the reward function to not return 1 on terminating states by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/958)
* Add `visual_options` rendering argument for MuJoCo environments by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/965)
* Add `exact` argument to `utlis.env_checker.data_equivilance` by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/924)
* Update `wrapper.NormalizeObservation` observation space and change observation to `float32` by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/978)
* Catch exception during `env.spec` if kwarg is unpickleable by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/982)
* Improving ImportError for Box2D by turbotimon (https://github.com/Farama-Foundation/Gymnasium/pull/1009)
* Add an option for a tuple of (int, int) screen-size in AtariPreprocessing wrapper by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/1105)
* Add `is_slippery` option for cliffwalking environment by CloseChoice (https://github.com/Farama-Foundation/Gymnasium/pull/1087)
* Update `RescaleAction` and `RescaleObservation` to support `np.inf` bounds by TimSchneider42 (https://github.com/Farama-Foundation/Gymnasium/pull/1095)
* Update determinism check for `env.reset(seed=42); env.reset()` by qgallouedec (https://github.com/Farama-Foundation/Gymnasium/pull/1086)
* Refactor mujoco to remove `BaseMujocoEnv` class by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/1075)
Deprecation
* Remove unnecessary error classes in error.py by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/801)
* Stop exporting MuJoCo v2 environment classes from `gymnasium.envs.mujoco` by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/827)
* Remove deprecation warning from PlayPlot by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/800)
Documentation changes
* Updated the custom environment tutorial for v1.0.0 by kir0ul (https://github.com/Farama-Foundation/Gymnasium/pull/709)
* Add swig to installation instructions for Box2D by btjanaka (https://github.com/Farama-Foundation/Gymnasium/pull/683)
* Add tutorial Load custom quadruped robot environments using `Gymnasium/MuJoCo/Ant-v5` framework by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/838)
* Add a third-party tutorial page to list tutorials written and hosted on other websites by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/867)
* Add more introductory pages by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/791)
* Add figures for each MuJoCo environment representing their action space by Kallinteris-Andreas (https://github.com/Farama-Foundation/Gymnasium/pull/762)
* Fix the documentation on blackjack's starting state by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/893)
* Update Taxi environment documentation to clarify starting state definition by britojr in https://github.com/Farama-Foundation/Gymnasium/pull/1120
* Fix the documentation on Frozenlake and Cliffwalking's position by PierreCounathe (https://github.com/Farama-Foundation/Gymnasium/pull/695)
* Update the classic control environment's `__init__` and `reset` arguments by pseudo-rnd-thoughts (https://github.com/Farama-Foundation/Gymnasium/pull/898)
**Full Changelog**: https://github.com/Farama-Foundation/Gymnasium/compare/v0.29.1...v1.0.0