jiminy

It is now possible to load pipeline learning environment from file (json or toml). Here an example of configuration file:

# Choose the environment
[env_config]
env_class = "gym_jiminy.envs.ANYmalJiminyEnv"
[env_config.env_kwargs]
step_dt = 0.005

# Add Proportional-Derivative Controller block
[[blocks_config]]
block_class = "gym_jiminy.common.controllers.PDController"
[blocks_config.block_kwargs]
update_ratio = 2
pid_kp = [1500.0, 1500.0, 1500.0, 1500.0, 1500.0, 1500.0, 1500.0, 1500.0, 1500.0, 1500.0, 1500.0, 1500.0]
pid_kd = [3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3, 3.0e-3]
[blocks_config.wrapper_kwargs]
augment_observation = true

# Partially stack observation frames
[[blocks_config]]
wrapper_class = "gym_jiminy.common.wrappers.StackedJiminyEnv"
[blocks_config.wrapper_kwargs]
nested_fields_list = [
  ["t"],
  ["sensors", "ImuSensor"],
  ["targets"],
]
num_stack = 3

New features:

[gym] Add 'load_pipeline' utility to load pipeline from file.

Miscellaneous:

[gym] Use folder architecture for clarity and maintainability.

This release achieves the work of generic pipeline design by implementing missing core features to support it.

New features:

• [core] Enable to specify the initial acceleration optionally.

Improvements:

• [core] Log command instead of motors efforts at engine level. One must use effort sensors to log them.
• [core] Fetch telemetry right after updating command but before integrating the dynamics.
• [core] Log data at the min(controller_period, sensors_period) if > 0, every internal steps otherwise

Bug fixes:

• [core] Fixes hasDynamicsChanged not being set to false after recomputing the dynamics for FSAL stepper hypothesis.
• [gym] Fix argument forwarding with multiple inheritence.

Miscellaneous:

• [gym] Add pipeline design unit tests.

The sensors are now updated at the end of the integration steps instead of the beginning, so that the system state is now consistent with sensors data. Moreover, pipeline builder has been reworked more maximum flexibility.

New features:

• [python] Replace 'BaseJiminyController' controller by 'BaseJiminyObserverController' to add dedicated observer support.
• [gym] Add frame stacking pipeline block and wrapper.

Improvements:

• [core] Update sensors data at the end of 'step' for consistency with robot state.
• [gym] Controller/observer blocks must be fully configured at init. Block '_setup' is now called at 'reset'.
• [gym] Take advantage of new dedicated observer.
• [gym] 'get_observation' recursively shadow-copy observation before returning it to avoid altering original data structure.
• [gym] Rework pipeline builder to be more flexible.

Bug fixes:

• [gym] Fix observe/control update period computed at init instead of reset.
• [gym] Update '_action' buffer to match latest 'compute_command' call.

Miscellaneous:

• [core] Add 'PRINT_WARNING' macro, to print debug message when compiling in debug mode only.
• [gym] Rename '*DictRecursive' in '*DictNested' for clarity.

This release greatly improves the compliance with OpenAI Gym API by enforcing complete configuration of the environment at initialization. As a result, it is no longer possible to change environment properties such as the observation and action spaces, but it is still possible to update the load a different robot model as long as it is compatible with those properties. So it should not be limiting fr the vast majority of applications.

New features:

• [gym] Make sure the environment is completely initialized at 'init', which is expected by OpenAI Gym API.

Improvements:

• [core] Catch some seldom exceptions.
• [core] Enable to call 'initialize' multiple times for 'Engine*'.
• [core] 'isSimulationRunning' is set to True at the end of 'start' instead of the beginning.
• [gym] Force sending zero motor efforts at reset since the initial action is undefined.

Bug fixes:

• [core|python] Several critical bug fixes.
• [gym] Fix base env '_setup' method not called at reset.

Observers and controllers can now be mixed freely, with independent update period.

New features:

• [gym] Add 'PartialFrameStack' wrapper and 'build_wrapper' utility.
• [gym] Add dedicated 'observe_dt' attribute to observer interface.
• [gym] Rework observation block to support multiple update period and more flexible pipeline architecture.

Improvements:

• [core] Enable to access sensors group shared data matrices by reference without copy.
• [core/python] Do not cast-away constness when sharing memory between Eigen Matrix and np.ndarray.
• [python] Handle more C++ exceptions.
• [gym] Refactor 'reset' of gym environment to fix initial state inconsistency and make everything simpler.
• [gym] Factorize pipeline wrappers 'step' method.

Bug fixes:

• [python/viewer] Fix display in Spyder IDE.
• [gym] Fix set seed simulator. Fix cartpole 'compute_command'.
• [gym] Fix 'write_log' missing format.

Miscellaneous:

• [gym] Rename 'gym_jiminy_common' in 'gym_jiminy' for clarity.

This release consists only of bug fixes.

Bug fixes:

• [misc] Fix cmake Python packages installation not overwriting existing install in some cases.
• [misc] Improve re-run-ablility of Pypi wheels generation.
• [misc] Fix windows wheels deployement on Pypi.
• [python] Fix deprecated 'write_log' calls.

Every dependency but Boost.Python is now statically linked. Window wheel now bundles dynamic libraries in the same way Linux wheel does.

Miscellaneous:

• [misc] Use static linkage for every Boost components, except Boost Python.
• [misc] Add auditwheel-like processing to sanitize windows Python wheels.

It is now possible to export/import log files in HDF5 format. HDF5 is especially well suited for storing log data because it enables to compress and extract each variable individually and only when needed. It also enables to store metadata directly with data, which is very helpful to store information required to reproduce results.

New features:

• [core|python] Add import/export logs to HDF5 format.

Improvements:

• [misc] Improve re-run-ability of automatic build dependencies scripts.

Miscellaneous:

• [misc] Enable optimizations in build dependencies scripts.
• [misc] Disable completely build docs in build dependencies scripts.
• [misc] Disable dev warnings in build dependencies scripts.
• [misc] Quiet Git reset of submodules in build dependencies scripts.
• [misc] Add HDF5 SOUP dependency.
• [gym] Disable flaky Acrobot learning unit test.

This release enhances the new pipeline architecture design for Gym environments to be more flexible and easy-to-use. In addition, log variables are now stored in 64 bits instead of 32 bits.

New features:

• [core] Log variable in 64bits instead of 32bits (and drop compatibility with old proprietary parser).
• [gym] Add dedicated interface for control and observation handling.
• [gym] Add block interface for pipeline control design.
• [gym] Add unit tests.
• [gym] Add generic pipeline generator.

Improvements:

• [core] Do not raise an exception if qhull not available and warn only.
• [core/python] 'register_variable*' methods now checks that input array is valid.
• [python] Fix deprecation warnings.
• [python/meshcat] Fix irrelevant warning and error messages.
• [python|gym] Add flag to enable/disable verbose logging related to robot toml file handling.
• [python|gym] Various bug fixes. Improve support of Windows.
• [gym] Remove 'common.wrapper' submodule since they are deprecated for now.
• [gym] Replace 'set_zeros' by 'zeros' to support immutable spaces.
• [gym] Always register controller target in pipeline architecture design for internal computations.
• [gym] Goes back to float64 for consistency with Jiminy float type.
• [gym] add 'get_log' helper at 'BaseJiminyEnv' level.
• [gym] 'register_variables' now returns 'is_success' flag.
• [gym] Use 'meshcat' backend by default if rendering with mode 'rgb_array'.
• [gym] Rename 'controller_dt' in 'control_dt'. Take into account controller's rewards in pipeline wrapper.
• [gym] Fix 'control_dt' not updated for 'BaseControllerBlock'. Block now has wrapped env in arg instead of unwrapped one.
• [gym] Remove support of with 'observe_target' for 'ControlledJiminyEnv' because of no real usecase.

Miscellaneous:

• [misc] Remove old deprecated docstring style.
• [misc] Enable to re-run automatic build scripts.
• [misc] Install numba on CI instead of 'easy_install_ubuntu.sh' script.
• [misc] Add-back Pinocchio StdVec_* converters.
• [misc] Fix Pinocchio imported twise that was messing up with boost python converters on Windows.
• [misc] Update Meshcat to v0.0.19 and remove backported code.
• [misc] Remove extra layer of indirection for import of Python bindings.
• [misc] Fix Atlas collision sphere size to fix display in Meshcat.
• [misc] Remove text image from ANYmal to avoid Meshcat viewer failure to load meshes.
• [misc] Enable pylint Python linter and pypy static type checker for gym/common and gym/toolbox on CI.
• [misc] Fix compatibility with Python < 3.7 on Win.
• [misc] Set Meshcat backend recorder thread to high priority.
• [core] Add dedicated 'Macro.h' header file.
• [core] Add macro to print error message. Add filename and line number in error message.
• [core] rename constant 'TELEMETRY_MAX_BUFFER_SIZE' in 'TELEMETRY_MIN_BUFFER_SIZE' for consistency.
• [core] Consistent style over the whole codebase.
• [python/meshcat] Factorize 'Capsule' implementation.

Bug fixes:

• [core] Fix various edge case segfault.
• [python] Use 'tempfile.mkstemp' instead of 'tempfile.NamedTemporaryFile' to create temp files (Windows compatibility).
• [python/viewer] Fix edge case when calling 'is_alive' while parent process is dead.
• [gym] Various bug fixes, typing and style.

This release introduces a new pipeline control architecture design for Gym environments. A controller is an independent block, that can be connected directly to an environment or another controller. In addition, a new Gym toolbox sub-module has been created, which is intended to gather tools for learning based on Pytorch.

Bug fixes:

• [core] Fix edge case where the controller could be updated twise successively.
• [core/python] Fix controller functor q/v type to avoid creating Eigen::Ref for nothing.
• [core|python] Do not overwrite meshScale for convex, just ignore it when loading in viewer.
• [python/robot] Fix 'gcd' for float.
• [python/viewer] Properly initialize default backend viewer.
• [python/meshcat] Fix client opening.

Improvements:

• [core] Use collisionResults instead of distanceResults to compute collisions for efficiency. Use 5 contact points by default.
• [core] More robust normal and penetration depth handling for collisions.
• [python] Various minor style improvements.
• [python/meshcat] Enable specifying custom material type when load geometry.
• [python/meshcat] Enable to visualize FCL BVHModelOBBRSS meshes.
• [python/meshcat] Use group to enable/disable 'visible' property and to delete scene.
• [python|gym] Add new learning toolbox module.
• [gym] Add global option to enable/disable obs/action finite bounds.
• [gym] Add new 'BaseController' base controller and 'ControlledJiminyEnv' [env + controller] wrapper.
• [gym] Add 'PDController' block and use it in zoo envs.

Miscellaneous:

• [misc] Add copyright Pinocchio when necessary.
• [misc] Do not start URDF with comments since it is not properly supported.
• [misc] Cleanup unit test cmake config.
• [misc] Add support of ASCII art diagram in docs.
• [misc] More robust and versatile cmake pip install and wheel generation helpers.
• [misc] Improve CI scripts and remove warnings during build docs.
• [python|gym] Use Python namespace to enable installing components individually.
• [gym] Add 'controller_dt' attribute and 'compute_command' method. Rename '_setup_environment' in '_setup'.
• [gym] Use np.float32 for obs/action spaces.
• [gym] Update docs.

It is now possible to visualize all standard primitives shapes in Meshcat, and to enable/disable display of visual/collision geometries.

Bug fixes:

• [core] Fix issue with convex collision mesh scaling.

Improvements:

• [core] Use static cast instead of dynamics for collision geometry retrieval.
• [python/meshcat] Server now keeps track of object properties to synchronise them between clients (for example "visible").
• [python/meshcat] Achieve features parity with gepetto-gui: added support of Cone/Capsule/Convex shapes.

Miscellaneous:

• [misc] Remove compatibility code with Pinocchio < 2.4.4.

This release fixes several bugs and limitations of the automatic URDF import capability, and introduce Spotmicro robot.

Bug fixes:

• [core/model] Fix addFrame.
• [python/dynamics] Fix update_quantities frames poses not always updated.
• [python/robot] Fix geometry type not properly determine during collision bodies definition substitution.
• [python/simulator] Update internal buffer viewer_backend when creating viewer if necessary.
• [gym] Fix unexpected exception raising if '_fetch_obs' is not consistent with '_refresh_obsveration_space'.

Improvements:

• [core/model|gym] Add small tolerance for Lie algebra bounds to avoid rounding issues.
• [python/robot] Extract motor dynamics from URDF and enable motor friction by default.
• [python/robot] Link with collision geometry associated with fixed joint can now be considered as valid leaf.
• [python/robot] Take into account mesh scale while replacing mesh by minimum volume box vertices.
• [python/robot] Add option to disable automatic collision geometry replacement to avoid numerical instabilities.

Miscellaneous:

• [misc] Update doc of ANYmal URDF file.
• [misc] Add Spotmicro robot example.

Co-authored-by: Alexis Duburcq [email protected]

This release introduces new documentation built using Sphinx and using the official ReadTheDocs style template.

Bug fixes:

• [core/pywrap] Fix constraint attribute getter names.
• [gym] Add missing Tianshou acrobot example.
• [gym] Fix obs not contained in obs space because of '_sensors_data' buffer not being an actual 'dict' object.

Improvements:

• [core/pywrap] Improve bp::dict conversion efficiency by avoiding attribute lookup.
• [core/pywrap] Rename Controller 'register_variable*' for consistency.
• [gym] Do not rescale reward systematically anymore.
• [gym] Add method to get time/state/sensors spaces separately to ease definition of custom obs space.
• [gym] Use OrderedDict instead of dict for observation.
• [gym/locomotion] PID gains 0.0 by default.

Miscellaneous:

• [misc] Minor README + doxygen mainpage update.
• [misc] Drop support of Pinocchio < 2.3.0.
• [misc] Fix python wheel development status.
• [misc] Full PEP8 conformance.
• [misc] Copy C++ documentation in Boost Python bindings.
• [misc] Migrate to Sphinx RTD documentation instead of Doxygen.

Bug fixes:

• [core] Fix segfault when writing log while no data is available.
• [core] Refresh model proxies automatically when position/velocities bounds are modified.
• [python/simulator] Fix bug during initial state retrieval.

Improvements:

• [core] Add getter for max simulation duration.
• [gym] Improve internal buffer management. Add several assertion checks at gym env instantiation.
• [gym] Improvement documentation.
• [gym] Refactor Acrobot and Cartpole RL envs in accordance to official OpenAI Gym implementations.

Miscellaneous:

• [misc] Remove unused toys models.
• [core] Rename 'motor_inertia' in 'motors_inertias' for API consistency.

Minor patch release. Update Readme. Add ANYmal and Atlas robot learning environment.

This release introduces many much-wanted features such as geometry collisions and wheel constraint, and lays the foundation for learning locomotion tasks for any robot out-of-the-box without configuration. The accuracy of the integration has been greatly improved by implementing a Runge-Kutta stepper working on Lie-Group.

New features:

• [core] Add 'mechanicalReduction' motor option.
• [core] Add 'successiveIterFailedMax' stepper option.
• [core] Add meshPackageDirs optional argument to model.initialize method.
• [core] Add support of 'continuous' joint type.
• [core] Add support of encoders and motors of any 1-dof type.
• [core] Implement methods to add and remove frames from the robot model.
• [core] Enable to postpone Controller def after engine init, and to change it between simu.
• [core] Add support of analytical wheel constraint.
• [core] Add support of mesh collisions using HPP-FCL. Using convex hull instead of actual geometries for evaluating collisions.
• [core] Implement steppers working inherently on Lie-algebra, including Explicit Euler and Runge-Kutta steppers.
• [python] Enable to specify collision bodies, contact points, and ground model in TOML config file.
• [python] Add automatic default configuration generation based on URDF file.
• [python/robot] Automatically convert box primitive body collision in contact points for numerical robustness.
• [python/simulator] Refactor of engine/simulator classes to be more versatile yet simpler.
• [python/simulator] Add 'plot' method to simulator to display simulation data up to current time.
• [gym] Add base locomotion learning environments.
• [gym] Enable to set camera pose in env 'render' method.

Bug fixes:

• [core] Fix various segfault in case of exceptions.
• [core] More verbose error message when failing to load URDF model.
• [core] Fix systemData returning unusable robot and controller.
• [core] Fix exception catching addCouplingForce.
• [python/dynamics] Fix dynamics quantities not properly initialized at engine reset.
• [python/viewer] Fix play_trajectory 'wait_for_client' option not used.
• [gym] Fix terminal reward computation.

Improvements:

• [core] Expose sensor data individually. Add sensor repr. Expose EngineMultiRobot systems vector in Python.
• [core] Add ControllerFunctor default value 'None' for command and internal dynamics arguments.
• [core] More specific Utilities error messages.
• [core] Compute the centers of mass systematically.
• [core] Check that the initial state is not out of bounds.
• [Viewer] Remove deprecation warning of Pinocchio.
• [Viewer] Minor improvement of error handling.
• [python|gym] Improve PEP8 compliance and typing.
• [python/dynamics] 3 points in contact during freeflyer computationg instead of aligned with frame of closest contact point.
• [python/dynamics] Take into account collision body orientation for box primitive when computing freeflyer.
• [python/viewer] Refactor Viewer to make it simpler and clearer.
• [python/viewer] More robust handling of meshcat comms in Jupyter.
• [gym] Enable to refresh observation and action spaces separately.
• [gym] Properly handle step integration failure.
• [gym] Fix 'done' flag handling in 'step' method.
• [gym] Enable to define custom neutral configuration.
• [gym] Minor refactoring to enable getting normal env from goal env without being hacky nor code duplication.
• [gym] Remove 'is_success' info in base env.

Miscellaneous:

• [misc] Update dependency versions.
• [misc] Remove meshes and use primitives for toys models.
• [misc] Remove messy examples and fix the others.
• [misc] Multi-level data folder architecture.

Breaking changes:

• [misc] Remove Python 2 compatibility code.
• [core] Replace state x arguments by configuration q and velocity v for the input arguments of every engine methods .
• [python] Remove EngineAsynchronous and BasicSimulator, add Simulator gathering the features of both.
• [gym] Remove 'TorqueControl' from walker name since it is the default

This patch is the last one in the long series of patches to bring the viewer in Google Colaboratory. Sadly, it was impossible to make it works as flawlessly as in Jupyter because of technical limitations. It is mainly because Google Colab relies on old ipython/jupyter dependencies from 2018. However, it should work out-of-the-box once the team would have updated them.

Patches and bug fixes:

• [Viewer] Hide pinocchio deprecation warnings.
• [Viewer] Fix Google Colab support.

Patches and bug fixes:

• [Viewer] Do not rely on knowing the expected comm_id size in byte
• [Viewer] Handle already closed connections more reliably
• [Viewer] Add a warning to recommend increasing 'iopub_msg_rate_limit' of Jupyter (Done by default on Google Colab)

Patches and bug fixes:

• [Viewer] Do not apply the optimized patch for handling kernel shell messages selectively for ipykernel < 5.0
• [Viewer] Fix non-resizeable display window on notebook and wrong initial window size on Google Colab

Patches and bug fixes:

• [Viewer] No longer messing up with the queue of tasks in notebooks, so that it is now possible to schedule the execution of other cells and the replay of a simulation is running without causing kernel corruptions
• [Viewer] Do not examine Ipython ports while looking for running Meshcat server to avoid throwing a low-level kernel exception in notebooks, the later not being caught in Google Colab