Learning a Generative Transition Model for Uncertainty-Aware Robotic Manipulation
GPL-3.0 License
In this repository, we've published the code for our publication Learning a Generative Transition Model for Uncertainty-Aware Robotic Manipulation, accepted for the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021). As only parts of the code were specifically written for this publication, we introduce the code regarding the overall project idea.
Let a robot learn how to grasp and manipulate objects by itself. The industrial task of bin picking emphasizes some difficulties like dense clutter, partial observability and unknwon geometry of the target objects. Since teaching the robot grasping in a model-based way is infeasible, we let the robot try to learn grasping via try and error. It needs around 20000 grasp tries using active learning to grasp reliably. This repo includes a range of extensions to the default state-of-the-art planar grasps: Specific grasps, lateral grasps, reactive grasps, model predictions and shifting of objects.
The overall structure is as follows:
scripts/learning
directory. For the implementation of the BicycleGAN architecture, please head to scripts/learning/image_generators/bicycle.py
.This project is a ROS package with launch files and a package.xml. The ROS node /move_group is set to respawn=true. This enables to call rosnode kill /move_group to restart it.
For the robotic hardware, make sure to load launch/gripper-config.json
as the Franka end-effector configuration. Currently, following dependencies need to be installed:
And all requirements for Python 3.6 via Pip and python3.6 -m pip install -r requirements.txt
. Patching CvBridge for Python3 and CMake >= 3.12 is given by a snippet in GitLab. It is recommended to export to PYTHONPATH in .bashrc
: export PYTHONPATH=$PYTHONPATH:$HOME/Documents/bin_picking/scripts
.
For an easy start, run sh terminal-setup.sh
for a complete terminal setup. Start the mongodb daemon. Then run roslaunch bin_picking moveit.launch
, rosrun bin_picking grasping.py
and check the database server.
The robot learning database is a database, server and viewer for research around robotic grasping. It is based on MongoDB, Flask, Vue.js. It shows an overview of all episodes as well as live actions. It can also delete recorded episodes. The server can be started via python3.6 database/app.py
, afterwards open localhost in your browser.