- Highlights
- New Features
- Improvements
- Critical Bug Fixes
- Other Bug Fixes
Highlights
While most surface-level functionality hasn't changed, the underlying infrastructure has been heavily reworked to reduce redundancy, improve standardization and ease-of-usage, and future-proof against expected expansions. Specifically, the following standards were pursued:
- Pretty much everything should have a name (no name = no reference in sim)
- All models should have a standardized interface (`MujocoModel`)
- Any manipulation-specific properties or methods should be abstracted away to a subclass to future-proof against novel robotic domains that might be added in the future.
- All associated attributes should try to be kept to a single object reference, to prevent silent errors from occurring due to partially modified objects. For example, instead of having `self.object` and `self.object_name`, just have `self.object`, since it already includes its own name reference in `self.object.name`.
New Features
This is not an exhaustive list, but includes the key features / changes in this PR most relevant to the common user that should greatly streamline environment prototyping and debugging.
Standardized Model Class Hierarchy
Now, all (robot, gripper, object) models inherit from the `MujocoModel` class, which defines many useful properties and methods, including references to the model joints, contact geoms, important sites, etc. This allows much more standardized usage of these models when designing environments.
Modularized Environment Class Hierarchy
We do not expect robosuite to remain solely manipulation-based. Therefore, all environment properties and methods common to manipulation-based domains were ported to `ManipulationEnv`, allowing future robot task domains to be added with little reworking. Similarly, common properties / methods common to Single or TwoArm environments were ported to `SingleArmEnv` and `TwoArmEnv`, respectively. This both (a) removes much redundant code between top-level env classes, and (b) frees users to focus exclusively on the environment prototyping unique to their use case without having to duplicate much boilerplate code. So, for example, `Lift` now has a class hierarchy of `MujocoEnv` --> `RobotEnv` --> `ManipulationEnv` --> `SingleArmEnv` --> `Lift`. Note that similar changes were made to the `Robot` and `RobotModel` base classes.
Standardized and Streamlined Object Classes
All object classes now are derived from `MujocoObject`, which itself is a subclass of `MujocoModel`. This standardizes the interface across all object source modalities (`Generated` vs. `XML` based), and provides the user with an expected set of properties that can be leveraged when prototyping custom environments. Additionally, complex, procedural object generation has been added with the `CompositeObject` class, of now which the `HammerObject` and `PotWithHandles` object are now subclasses of (as examples of how to design custom composite objects).
Greater Procedural Object Generation Support
`CompositeObject` and `CompositeBodyObject` classes have now been added. A `CompositeObject` is composed of multiple geoms, and a `CompositeBodyObject` is composed of multiple objects (bodies). Together, this allows for complex, procedural generation of arbitrary object shapes with potentially dynamic joint interactions. The `HammerObject` and `PotWithHandlesObject` are examples of the `CompositeObject` class, and `HingedBoxObject` is an example of the `CompositeBodyObject` class.
Standardized Geom Groups
All collision geoms now belong to group 0, while visual geoms belong to group 1. This means that methods can automatically check for the geom type by polling it's `group` attribute from its element or during sim. Moreover, all collision geoms are assigned solid rgba colors based on their semantic role (e.g.: robot vs. gripper vs. arena vs. objects). If rendering onscreen, you can easily toggle visualizing the visual and collision geoms by pressing `1` or `0`, respectively. This can be useful for debugging environments and making sure collision bodies are formed / interacting as expected.
High-Utility Methods for Environment Prototyping
Because of this improved structure, many methods can now take advantage of this standardization. Some especially relevant methods are discussed briefly below:
- `env.get_contacts(model)` (any env): This method will return the set of all geoms currently in contact with the inputted `model`. This is useful for debugging environments, or checking to see if certain conditions are met when designing rewards / interactions.
- `env._check_grasp(gripper, object_geoms)` (only manipulation envs): This method will return True if the inputted `gripper` is grasping the object specified by `object_geoms`, which could be a `MujocoModel` or simply a list of geoms that define the object. This makes it very easy to design environments that depend on certain grasping requirements being met.
- `env._gripper_to_target(gripper, target, ...)` and `env._visualize_gripper_to_target(gripper, target, ...)` (only manipulation envs): Methods to help streamline getting relevant distance info between a gripper `gripper` and `target`. `Target` can be a `MujocoModel` or any specific element (body, geom, site) name. The former calculates the distance, while the latter will set the gripper eef site sphere's color to be proportional to the distance to `target`. Both are useful for environment prototyping and debugging.
- `model.set_sites_visibility(sim, visible)` (any MujocoModel): This method will set all the sites belonging to `model` in the current `sim` to either be visible or not depending on the `visible` arg. This is useful for quick debugging or teleoperation, to aid the user in visualizing specific points of reference in sim.
Improvements
The following briefly describes other changes that improve on the pre-existing structure. Again, this is not an exhaustive list, but a highlighted list of changes.
- `MountModel` class added; pedestals used by robots are now assigned to this class and added to a `RobotModel` in a similar fashion to how the `GripperModel` is added. This allows abstraction of the robot model from its base mount model.
- Abstracted site visualizations to a wrapper (`VisualizationWrapper`). This wrapper provides fine-grained control over sites being visualized within the environment: can specify whether to visualize site groups belonging to the wrapped env. This is controlled via keywords provided by a given environment. For example, for `ManipulationEnv` classes, this includes `gripper`, `robot`, and `env` keys, each of which control its associated site visualization.
- Added openGL and openCV image convention option as a macro
- Added `macros.py` in `robosuite.utils` and single file to store all macros for our repo. This includes numba macros and now includes instance randomization and image convention macros. Users can modify these macros mid-script by importing the macros module and modifying the module-level vars directly.
- Placement samplers were no longer belong to `Task` class, but are separate. This is more intuitive, and allows for more modularity when designing future `Task` subclasses. Moreover, the placement sampler classes were refactored for more intuitive usage.
- Refactor all top-level environments in a standardized fashion
- Add functionality to modify cameras from `Arena` class; tuned cameras for Door, TwoArmHandover/PegInHole tasks
- Renamed / modified a bunch of stuff so it's more semantically accurate / intuitive
- Tuned Wipe environment with alternate compressed object observation space (this is enabled by default) and default environment parameters, such as table height / size and wipe marker sampling locations.
- Update `GymWrapper` class to be more robust to general usage -- now, automatically flattens image observations so that it is Gym-compatible and also extends from the Gym `Env` class directly.
- Add GPU device arg in environments for setups with multiple GPUs
- Add new papers (118)
- Improve documentation
Critical Bug Fixes
- Fixed grasping bug where a grasp is incorrectly inferred if a robot's two fingers are touching an object. This resulted in incorrect rewards being received which could negatively impact reward-based training. Grasps are now inferred correctly so robot cannot "cheat" a grasping-based reward.
- Fixed singular value problem with OSC controller (136). Control loop computations now utilizes `numpy.pinv` instead of our implementation of it.
- Fix absolute control and control limits setting for OSC controller.
- Fix model XML saving method. We now use `env.sim.model.get_model()` instead of `env.model.get_model()` so that we don't save a stale version of the current simulation snapshot.
Other Bug Fixes
- Fix wiping gripper mass (too high before, leading to bad force-torque sensor readings)
- Fix agentview cameras for `Door`, `TwoArmHandover`, and `TwoArmPegInHole` environments (123)
- Fix OSC controller bug that doesn't automatically re-update initial goal orientation upon reset