End-to-end GPU-accelerated Learning and Control for Robotics with Isaac Gym

In the last few years, reinforcement learning (RL) has become one of the most promising research areas in machine learning and has demonstrated great potential for solving sophisticated decision-making problems. Deep reinforcement learning has achieved superhuman performance in board games like Go, Poker, Dota 2 and Starcraft, as well as showing impressive results in robotics domains, including legged locomotion and dexterous manipulation.

There are two critical bottlenecks in the wider accessibility of RL technology in robotics: 1) enormous computational requirements and 2) limited simulation speed. These problems are especially challenging when learning long-horizon behaviors for robots with high degrees of freedom. To address these bottlenecks, NVIDIA has developed Isaac Gym, a high-performance robotics simulator that is completely free of charge. Its fully GPU-accelerated simulation and training pipeline can help lower the barrier for research, enabling the solution of tasks with a single GPU that were previously only possible on massive CPU clusters. For example, we show that training a quadruped ANYmal robot to walk in simulation takes under 30 min on a single A100 GPU and achieves successful policy deployment on a real robot. Moreover, we reproduce an OpenAI training setup with the Shadow Hand from the "Learning Dexterity" paper and are able to train it in less than 2 hours on an RTX 3090 GPU, compared to 40 hours on a CPU cluster with 6144 cores.

In this tutorial, we introduce the end-to-end GPU accelerated training pipeline in Isaac Gym, which allows researchers to overcome these key limitations and achieve a 100x-1000x training speed-up in continuous control tasks. Afterward, we perform a deep dive into Isaac Gym’s tensor API, upon which the GPU-accelerated training pipeline is built. Next, we demonstrate applications to various robotics domains through case studies, including manipulation with the Shadow Hand, operational space control with the Franka arm, and deformable-object grasping with the Franka gripper. Several of these case studies will be presented by guest speakers from university labs spanning robotics, machine learning, and simulation. Finally, we will host breakout sessions where participants can freely ask questions to organizers in their specific research area. The only prerequisites for our tutorial will be basic familiarity with Python, as well as fundamental robotics and machine learning concepts.