Skrl

Changed
- Update the GitHub Actions workflows for testing and coverage
- Update minimum supported Python version to 3.8 and minimum dependencies versions

Fixed
- Fix environment wrapper issues with spaces utilities's keyword-only arguments (introduced in previous version)
- Fix noise device definition in runner implementations

Added
- Add Multi-Categorical model instantiator
- Add `one_hot_encoding` function to model instantiators to one-hot encode `Discrete` and `MultiDiscrete` tensorized spaces
- Allow `None` type spaces and samples/values in spaces utilities and define keyword-only arguments

Fixed
- Cast model instantiator's `initial_log_std` parameter to `float` in PyTorch
- Fix common property overwriting (e.g. `clip_actions`) in shared models composed of different mixin types

Fixed
- Force the use of the device local to process in distributed runs in JAX
- Update runner implementation to parse noises definitions for off-policy agents

Added
- Utilities to operate on Gymnasium spaces (`Box`, `Discrete`, `MultiDiscrete`, `Tuple` and `Dict`)
- `parse_device` static method in ML framework configuration (used in library components to set up the device)
- Model instantiator support for different shared model structures in PyTorch
- Support for automatic mixed precision training in PyTorch
- `init_state_dict` method to initialize model's lazy modules in PyTorch
- Model instantiators `fixed_log_std` parameter to define immutable log standard deviations
- Define the `stochastic_evaluation` trainer config to allow the use of the actions returned by the agent's model
as-is instead of deterministic actions (mean-actions in Gaussian-based models) during evaluation.
Make the return of deterministic actions the default behavior.

Changed
- Call agent's `pre_interaction` method during evaluation
- Use spaces utilities to process states, observations and actions for all the library components
- Update model instantiators definitions to process supported fundamental and composite Gymnasium spaces
- Make flattened tensor storage in memory the default option (revert changed introduced in version 1.3.0)
- Drop support for PyTorch versions prior to 1.10 (the previous supported version was 1.9)
- Update KL Adaptive learning rate scheduler implementation to match Optax's behavior in JAX
- Update AMP agent to use the environment's terminated and truncated data, and the KL Adaptive learning rate scheduler
- Update runner implementations to support definition of arbitrary agents and their models
- Speed up PyTorch implementation:
- Disable argument checking when instantiating distributions
- Replace PyTorch's `BatchSampler` by Python slice when sampling data from memory

Changed (breaking changes: style)
- Format code using Black code formatter (it's ugly, yes, but it does its job)

Fixed
- Move the batch sampling inside gradient step loop for DQN, DDQN, DDPG (RNN), TD3 (RNN), SAC and SAC (RNN)
- Model state dictionary initialization for composite Gymnasium spaces in JAX
- Add missing `reduction` parameter to Gaussian model instantiator
- Optax's learning rate schedulers integration in JAX implementation
- Isaac Lab wrapper's multi-agent state retrieval with gymnasium 1.0
- Treat truncation signal when computing 'done' (environment reset)

Removed
- Remove OpenAI Gym (`gym`) from dependencies and source code. **skrl** continues to support gym environments,
it is just not installed as part of the library. If it is needed, it needs to be installed manually.
Any gym-based environment wrapper must use the `convert_gym_space` space utility to operate

Added
- Distributed multi-GPU and multi-node learning (JAX implementation)
- Utilities to start multiple processes from a single program invocation for distributed learning using JAX
- Model instantiators `return_source` parameter to get the source class definition used to instantiate the models
- `Runner` utility to run training/evaluation workflows in a few lines of code
- Wrapper for Isaac Lab multi-agent environments
- Wrapper for Google Brax environments

Changed
- Move the KL reduction from the PyTorch `KLAdaptiveLR` class to each agent that uses it in distributed runs
- Move the PyTorch distributed initialization from the agent base class to the ML framework configuration
- Upgrade model instantiator implementations to support CNN layers and complex network definitions,
and implement them using dynamic execution of Python code
- Update Isaac Lab environment loader argument parser options to match Isaac Lab version
- Allow to store tensors/arrays with their original dimensions in memory and make it the default option

Changed (breaking changes)
- Decouple the observation and state spaces in single and multi-agent environment wrappers and add the `state`
method to get the state of the environment
- Simplify multi-agent environment wrapper API by removing shared space properties and methods

Fixed
- Catch TensorBoard summary iterator exceptions in `TensorboardFileIterator` postprocessing utils
- Fix automatic wrapper detection issue (introduced in previous version) for Isaac Gym (previews),
DeepMind and vectorized Gymnasium environments
- Fix vectorized/parallel environments `reset` method return values when called more than once
- Fix IPPO and MAPPO `act` method return values when JAX-NumPy backend is enabled

Added
- Define the `environment_info` trainer config to log environment info (PyTorch implementation)
- Add support to automatically compute the write and checkpoint intervals and make it the default option
- Single forward-pass in shared models
- Distributed multi-GPU and multi-node learning (PyTorch implementation)

Changed
- Update Orbit-related source code and docs to Isaac Lab

Fixed
- Move the batch sampling inside gradient step loop for DDPG and TD3
- Perform JAX computation on the selected device