Jiminy-py

pid_kd = [3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3]
[blocks_config.wrapper_kwargs]
augment_observation = true

Partially stack observation frames
[[blocks_config]]
wrapper_class = "gym_jiminy.common.wrappers.StackedJiminyEnv"
[blocks_config.wrapper_kwargs]
nested_fields_list = [
["t"],
["sensors", "ImuSensor"],
["targets"],
]
num_stack = 3


New features:
[gym] Add 'load_pipeline' utility to load pipeline from file.

Miscellaneous:
[gym] Use folder architecture for clarity and maintainability.

Small release that fixes several critical bugs related to quantity management and ray[RLlib] interface that were introduced recently. The overall performance of environment composition has been improved, especially for short-living episodes (<100 steps).

New features

* [gym_jiminy/common] Add mechanical power consumption reward. (873)
* [gym_jiminy/common] Add 'DriftTrackingBaseOdometryPoseReward'. (874)
* [gym_jiminy/toolbox|rllib] Implement task score at scheduler level rather than task settable wrapper level. (876)

Improvements

* [gym_jiminy/common] Add support of 'horizon=None' to MechanicalPowerConsumptionTermination. (870)
* [gym_jiminy/common] 'TrackingBaseHeightReward' now relies on relative base height rather than absolute. (874)
* [gym_jiminy/common] Rename 'ReferencePositionWithTrueOdometryPose' in 'ReferencePositionVector' and add option to disable odometry pose override. (874)
* [gym_jiminy/common] Add support of scalar concatenation to 'ConcatenatedQuantity'. (875)
* [gym_jiminy/common] Make sensor space bounds infinite to avoid out-of-bounds measurements due to noise/bias to trigger termination. (875)
* [gym_jiminy/common] Move PD deadband from PDController to PDAdapter to fix position/velocity inconsistency. (876)
* [gym_jiminy/common] Refactor 'DeltaQuantity' to improve performance for long horizon. (880)
* [gym_jiminy/common] Speedup dataset trajectory select. (882)
* [gym_jiminy/common] Speed up quantity reset by avoiding redundant state computations. (883)
* [gym_jiminy/toolbox] Do NOT raise exception if score is out-of-bounds. (871)
* [gym_jiminy/rllib] Add option to override env config when restoring runners. (872)
* [gym_jiminy/rllib] Restore curriculum state from checkpoints. (878)

Patches and bug fixes

* [python/dynamics] Fix trajectory interpolation mode 'clip'. (873)
* [python/dynamics] Fix bug making 'Trajectory.get' extremely inefficient. (881)
* [gym_jiminy/common] Fix 'MechanicalPowerConsumption' not exposed. (870)
* [gym_jiminy/common] Fix 'play_interactive'. (871)
* [gym_jiminy/common] Fix mechanical power termination condition. (873)
* [gym_jiminy/common] Fix 'BaseRelativeHeight' quantity ignoring mode. (873)
* [gym_jiminy/common] Fix deformation observer with backlash enabled. (876)-
* [gym_jiminy/common] Fix stack length computation from horizon. (880)
* [gym_jiminy/rllib] Fix eval runner connector stats not sync w/ eval. (870)
* [gym_jiminy/rllib] Fix eval runner connector updated during sample collection. (870)
* [gym_jiminy/rllib] Fix connector state not properly restored after loading checkpoint. (871)
* [gym_jiminy/rllib] Fix log paths not deleted during evaluation. (877)

Miscellaneous

* [misc] Fix Sphinx doc (github-pages) broken refs. (870)
* [misc] Fix RL tutorial notebook. (880)

This release mainly streamlines environment composition (i.e. plug-and-play rewards and termination conditions). Several API has been refactor to be more convenient and intuitive. Notably, the official gym wrapper API (gymnasium>=1.0) to get/set nested attributes has been adopted (i.e. `(get|set)_wrapper_attr`). Besides, a few additional quantities have been added. As usual, several bugs, this time mainly related to ray[RLLib], have been fixed, and task curriculum has been refactored to be implemented as a pipeline wrapper rather than a mixin class to integrate nicely with environment pipeline design.

New features

* [gym_jiminy/common] Add instantaneous mechanical power quantity. (862)
* [gym_jiminy/common] Add support of ellipsis to 'MaskedQuantity'. (862)
* [gym_jiminy/common] Add generic concatenation quantity. (862)
* [gym_jiminy/common] Add quantity mixing reference position vector with true odometry pose. (862)

Improvements

* [core|python] Ground profile now included in persistent log files and automatically loaded for replay. (867)
* [gym_jiminy/common] Flatten info hierarchy for reward components. (862)
* [gym_jiminy/common] Do not store individual termination condition states in info. (862)
* [gym_jiminy/common] Use asymmetric sigmoid normalization for stability reward. (862)
* [gym_jiminy/common] Trajectory time mode is not specific to each trajectory in the dataset. (863)
* [gym_jiminy/common] Trajectory time offset is now a global attribute of the dataset. (863)
* [gym_jiminy/common] 'QuantityManager' now has its own API instead of abusing mutable dict. (863)
* [gym_jiminy/common] Stop relying on confusing dark-magic '\_\_getattr\_\_' in favor of explicit delegation. (863)
* [gym_jiminy/common] Add option to allow registering variables to the telemetry in training mode. (862)
* [gym_jiminy/common] Move initial state sampling after reset all of the layers of the pipeline. (867)
* [gym_jiminy/toolbox|rllib] Task-settable capability is now provided by a wrapper. (864)

Patches and bug fixes

* [gym_jiminy/common] Fix hard-coded termination conditions not being recorded in 'info'. (862)
* [gym_jiminy/common] Fix 'log_path' not properly reset at setup. (862)
* [gym_jiminy/common] Fix 2D Euclidean space angle algebra for drift and shift termination conditions. (865)
* [gym_jiminy/rllib] Fix evaluation sampling. (863)

Miscellaneous

* [misc] Fix doc generation. (862)
* [misc] Fix broken Python bindings docstring substitution. (863)
* [misc] Add support of Python 3.13. (868)

This release mainly focuses on fixing a bugs across the codebase, including long lasting bugs related to impulse forces since inception. The latest version of `gymnasium` and `ray[rllib]` are now fully supported and has been validated on a production environment. Environment pipeline design should be ready for production. It is expected to be significantly slower (~20%) than "flat" environment due to the large overhead of nested function calls in python. Nevertheless, modular design enables to iterate faster and to leverage existing blocks that have been extensively validated and optimised already.

New features

* [python/tree] Add specialized 'deepcopy' operator for nested data structure. (852)
* [gym_jiminy/common] Add 'BodyObserver' to get orientation and angular velocity of IMUs' parent bodies. (844)
* [gym_jiminy/common] Add soft motor effort limit to 'MotorSafetyLimit' block. (846)
* [gym_jiminy/common] Add PD controller mode to force instantaneous state update. (847)
* [gym_jiminy/common] Add option to avoid clipping target motor accelerations of PD controller. (849)
* [gym_jiminy/common] Add target joint velocity deadband to PD controller. (849)
* [gym_jiminy/common] Add support of time offset to trajectory dataset. (855)
* [gym_jiminy/common] Add 'order' input argument to 'build_flatten'. Speed up 1D special case. (859)

Improvements

* [core] Improve error handling in 'Engine.read_log'. (859)
* [python/dynamics] 'Trajectory.get' systematically returns by value. (846)
* [gym_jiminy/common] Move leaky twist from mahony filter to body observer. (849)
* [gym_jiminy/common] Add estimated IMU angular velocity to Mahony filter observation. (844)
* [gym_jiminy/common] Share base engine measurement through all the layers of the pipeline. (845)
* [gym_jiminy/common] Stop relying on 'true' sensor measurements in computations. (845)
* [gym_jiminy/common] Stop relying on 'stepper_state' to get current time. (845)
* [gym_jiminy/common] Add aggregated 'terminated', 'truncated' info key for composite env. (847)
* [gym_jiminy/common] Do not overwrite 'BaseJiminyEnv.(observation|action)' if already defined. (849)
* [gym_jiminy/common] More robust random seed initialization. (849)
* [gym_jiminy/common] Support additional 'special' string when loading pipeline from toml. (849)
* [gym_jiminy/common] Plot variables in reverse registeration order. (852)
* [gym_jiminy/common] Only register block telemetry variables in debug or evaluation mode. (852)
* [gym_jiminy/common] Robust ZMP computation when robot is free falling. (854)
* [gym_jiminy/common] 'BaseJiminyEnv.evaluate' now returns both reward and info history. (856)
* [gym_jiminy/common] Refactor 'StackedQuantity' to improve performance for aggregated large stack. (857)
* [gym_jiminy/common] Refactor evaluation log file logics to allow for parallel evaluation. (859)
* [gym_jiminy/rllib] Log num steps instead of num iterations in abscisse. Robust eval to unavailable log file. (847)
* [gym_jiminy/rllib] Specify obs/act mirroring spec in args w/o relying on space. Only support vector obs/act. (859)

Patches and bug fixes

* [core] Fix rounding error when computing impulse force breakpoints. (852)
* [core] Fix wrong time 't' when computing derivative after successful integration. (852)
* [core] Fix joint jacobian not systematically updated when dealing with external forces. (852)
* [core] Fix impulse forces internal state not properly reset. (853)
* [python/simulator] Fix replay of extra log files and trajectories. (848)
* [python/robot] Fix default hardware generation. (852)
* [gym_jiminy/common] Fix wrong initial RPY observation for Mahony filter. (844)
* [gym_jiminy/common] Fix 'BaseJiminyEnv.play_interactive'. (845)
* [gym_jiminy/common] Fix sensor noise/bias for locomotion env. (846)
* [gym_jiminy/common] Fix plot of manually registered env variables. (848)
* [gym_jiminy/common] Fix 'ControlledJiminyEnv.action' not reset during setup. (849)
* [gym_jiminy/common] Fix Mahony filter angular velocity estimate. (850)
* [gym_jiminy/common] Fix partially broken deformation observer. (850)
* [gym_jiminy/common] Fix 'ShiftTrackingQuantityTermination' grace period shorter than horizon. (853)
* [gym_jiminy/common] Fix Capture Point quantity initialization messing up with global state. (854)
* [gym_jiminy/common] Fix 'evaluate' total reward computation. (856)
* [gym_jiminy/rllib] Fix extremely slow shallow batch copy. (859)
* [gym_jiminy/rllib] Fix true action mean computation and additional loss scalar reduction. (860)
* [gym_jiminy/rllib] Monkey patch RLlib to fix numerous bugs harming convergence. (860)

Miscellaneous

* [core] Rename 'positionLimit(Min|Max)' in 'positionLimit(Lower|Upper)' for consistency with Pinocchio. (849)
* [core] Move 'logInternalStepperSteps' from 'stepper' to 'telemetry' options for consistency. (852)
* [gym_jiminy/common] Use fortran order for flattening wrappers 'Flatten(Observation|Action)'. (859)
* [misc] Rename 'mesh_path_dir' in 'mesh_dir_path' for consistency. (848)
* [misc] Add support of 'gymnasium==1.0'. (859)
* [misc] Add support of 'ray[rllib]==2.40'. (859)

The environment pipeline design has been streamlined. It is not only more versatile and faster but also less buggy. It should be ready now to be tested for real-world applications.

New features

* [gym_jiminy/common] Add 'AdaptLayoutObservation' that generalizes 'FilterObservation'. (835)
* [gym_jiminy/common] Add 'QuantityObserver' block. (838)
* [gym_jiminy/common] Enable env composition to augment the observation space with trajectory reference. (838)
* [gym_jiminy/common] Add 'ScaleObservation', 'ScaleAction' env pipeline wrapper blocks. (839)

Improvements

* [python/dynamics] Take into account stride offset when trajectory time is wrapping. (838)
* [python|gym_jiminy] More natural figure layout to improve readability. (837)
* [gym_jiminy/common] Do not permanently alter original simulation options with enabling debug and/or evaluation modes. (832)
* [gym_jiminy/common] Rewrite binary log file automatically when calling 'BaseJiminyEnv.stop' in debug or evaluation mode. (832)
* [gym_jiminy/common] Add previous action as input argument for evaluation policy callback. (832)
* [gym_jiminy/common] Automatic environment pipeline update. (832)
* [gym_jiminy/common] Improve numerical stability and speed of 'swing_from_vector'. (834)
* [gym_jiminy/common] Add support of 'none' in pipeline config files. (834)
* [gym_jiminy/common] Add 'ignore_unbounded' optional argument to flatten obs/act wrappers. (834)
* [gym_jiminy/common] Add 'compute_rpy' optional argument to mahony observer. (837)
* [gym_jiminy/common] Add 'compute_rpy' optional argument to deformation estimator. (837)
* [gym_jiminy/common] Add locking mechanism to trajectory databaset. (838)
* [gym_jiminy/common] More robust pipeline registration mechanism. (838)
* [gym_jiminy/common] Support string representation of enums in pipeline config. (838)
* [gym_jiminy/common] Faster env pipeline. (841) (842)
* [gym_jiminy/toolbox] Add support of arbitrarily nested task-settable env. (832)
* [gym_jiminy/envs] Add mirror mat to obs/action spaces. (832)

Patches and bug fixes

* [core] Fix symbol visibility issues for some classes on windows. (836)
* [core] Fix bindings and serialization of convex geometry meshes. (836)
* [core] Fix 'array_copyto' helper. (841)
* [python/viewer] Fix 'add_camera'. (831)
* [python/viewer] Use firefox instead of chromium for offscreen rendering with meshcat to fix rendering on MacOS VM. (832)
* [python/viewer] Fix support of '-' in robot name. (834)
* [python/viewer] Fix conversion of 'hppfcl.Convex' to Panda3D. (836)
* [gym_jiminy/common] Disallow switch between evaluation and training mode when a simulation is running. (832)
* [gym_jiminy/common] Fix replay if no simulation is running. (832)
* [gym_jiminy/common] Fix composed reward computation. (832)
* [gym_jiminy/common] Use metaclass instead of inheritence for abstract classes. (832)
* [gym_jiminy/common] Enable typing of the obs and action spaces for 'gym.Env'. (832)
* [gym_jiminy/common] Fix nested gym space helpers. (832)
* [gym_jiminy/common] Fix missing final state when writing eval/debug log. (834)
* [gym_jiminy/common] Fix 'quat_multiply' to support in-place operation. (834)
* [gym_jiminy/common] Add twist estimate in internal mahony state if estimated. (834)
* [gym_jiminy/common] Fix mahony filter leaky integration for very small time constant. (834)
* [gym_jiminy/common] Check IMU and flex frames not empty when adding Mahony filter. (834)
* [gym_jiminy/common] Fix base 'BaseJiminyEnv.stop' not called in pipeline. (834)
* [gym_jiminy/common] Support pipelines without any layer. (834)
* [gym_jiminy/common] Make sure that the initial PD controller state is within bounds. (835)
* [gym_jiminy/common] Fix trajectory file never closed if loading fails. (838)
* [gym_jiminy/common] Add composition wrapper before observer-controller blocks. (838)
* [gym_jiminy/common] Fix 'AdditiveMixtureReward' for 'order=inf'. (838)
* [gym_jiminy/common] Fix partially broken 'spaces.build_reduce'. (839)
* [gym_jiminy/rllib] Full refactoring to support ray-rllib 2.38. (832)
* [misc] Drop support of Python<3.10. (831)

This release introduces composition of termination conditions via a first-class abstraction, in the same way as generic quantities and reward components. Apart from that, the viewer now supports adding extra camera to the scene, including depth map. In the later case, the implementation has been heavily optimised for speed. It takes about 400us on a laptop with intel integrated chipset to render a 128 x 128 depth map. One last new important feature is the support of smooth random ground generation through 2D Perlin processes. Querying the ground height at multiple locations in a LIDAR fashion has also been optimised for speed. It takes about 5us for 100 points. Finally, some critical components has been sped up, notably quantity management and evaluation is now one order of magnitude faster (10% to 25% overhead compare to physics evaluation for a full set of about 20 reward and termination conditions), and the viewer should be about twice faster for both onscreen and offscreen rendering with Panda3D.



New features

* [core] All random processes are now differentiable. (799)
* [core] Add Perlin ground generators. (799) (823) (825)
* [python/viewer] Add support of user-specified extra cameras (rgb and depth). (826)
* [gym/common] Add multi-ary operator quantity. (814)
* [gym/common] Add termination condition abstraction. (812)
* [gym/common] Add support of termination composition in pipeline environments. (812)
* [gym/common] Add quantity shift and drift tracking termination conditions. (812)
* [gym/common] Add base roll/pitch termination condition. (813)
* [gym/common] Add base relative height termination condition. (813)
* [gym/common] Add foot collision termination condition. (813)
* [gym/common] Add safety limits termination condition. (814)
* [gym/common] Add robot flying termination condition. (815)
* [gym/common] Add power consumption termination condition. (816)
* [gym/common] Add ground impact force termination condition. (816)
* [gym/common] Add base odometry pose drift tracking termination condition. (817)
* [gym/common] Add motor positions shift tracking termination condition. (817)
* [gym/common] Add relative foot odometry pose shift tracking termination conditions. (820)

Improvements

* [core] Make all Perlin processes faster and copy-able. (799) (824)
* [python/viewer] Significantly speed-up both offscreen and onscreen rendering for Panda3D. (826)
* [gym/common] More generic stacking quantity. (812)
* [gym/common] More generic actuated joint kinematic quantity. (814)
* [gym/common] Add 'order' option to 'AdditiveReward'. (821)
* [gym/common] Refactor quantity management to dramatically improve its performance. (821)

Patches and bug fixes

* [core] Fix robot serialization issue. (821)
* [core] Fix negative time support for all existing random processes. (799)
* [python/tree] Fix 'tree.unflatten_as' mixing up key order for 'gym.spaces.Dict'. (819)
* [python/simulator] Consistent keyword arguments between 'Simulator.build' and 'Simulator.add_robot'. (821)
* [python/viewer] Fix MacOS support. (822)
* [gym/common] Fix quantity hash collision issue in quantity manager. (821)