What Is Isaac Sim?

NVIDIA Omniverse™ Isaac Sim is a robotics simulation toolkit for the NVIDIA Omniverse™ platform. Isaac Sim has essential features for building virtual robotic worlds and experiments. It provides researchers and practitioners with the tools and workflows they need to create robust, physically accurate simulations and synthetic datasets. Isaac Sim supports navigation and manipulation applications through ROS/ROS2. It simulates sensor data from sensors such as RGB-D, Lidar, and IMU for various computer vision techniques such as domain randomization, ground-truth labeling, segmentation, and bounding boxes.

System Architecture

_images/isaac_overview_system_diagram_0.png

Development Workflows

_images/isaac_overview_workflows_diagram_1.png

Omniverse Kit

Omniverse Isaac Sim uses the NVIDIA Omniverse™ Kit SDK, a toolkit for building native Omniverse applications and microservices. Omniverse Kit provides a wide variety of functionality through a set of light-weight plugins. Plugins are authored with C interfaces for persistent API compatibility; however, a Python interpreter is also provided for accessible scripting and customization.

The Python API can be used to write new extensions to Omniverse Kit or new experiences for Omniverse.

Tip

For a more in-depth look at developing in Kit, see the Kit Programming Manual.

Omniverse Nucleus

Omniverse Isaac Sim uses NVIDIA Omniverse™ Nucleus to access content such as USD files for environments and robots. Omniverse Nucleus services allow a variety of client applications, renderers, and microservices to share and modify representations of virtual worlds in Omniverse Isaac Sim.

Nucleus operates under a publish/subscribe model. Subject to access controls, Omniverse clients can publish modifications to digital assets and virtual worlds to the Nucleus Database (DB) or subscribe to their changes. Changes are transmitted in real-time between connected applications. Digital assets can include geometry, lights, materials, textures and other data that describe virtual worlds and their evolution through time.

This allows a variety of Omniverse-enabled client applications (Apps, Connectors, and others) to share and modify authoritative representations of virtual worlds.

Note

See Nucleus Overview for a more in-depth look at the Nucleus data model, architecture, and distribution platforms.

USD

Omniverse Isaac Sim uses the USD interchange file format to represent scenes. Universal Scene Description (USD) is an easily extensible, open-source 3D scene description and file format developed by Pixar for content creation and interchange among different tools. Because of its power and versatility, USD is being adopted widely, not only in the visual effects community, but also in architecture, design, robotics, manufacturing, and other disciplines.

For a more in-depth look at USD in Omniverse, see the NVIDIA USD primer What is USD?.
See the USD API docs for more details.
See the USD Glossary of Terms & Concepts for more details.
See the NVIDIA USD tutorials for a step-by-step introduction to USD.

Documentation Overview

The documentation is split into the following sections: Introduction, Installation Guide, API Documentation, Tutorials, Manuals, and Reference Materials.

Introduction

The Introduction section is a basic overview of Omniverse Isaac Sim and how it fits into the ecosystem of NVIDIA Omniverse™. It also outlines the architecture, workflow of Omniverse Isaac Sim, and the documentation layout.

Installation Guide

The Workstation Installation is the recommended installation process for all users.

Omniverse Isaac Sim also supports more advanced, headless, and remote installation options. Docker, Cloud, and Streaming-related setups are covered in the this page guide.

API Documentation

Python API Documentation for both Omniverse and Isaac Sim extensions can be found here.

Tutorials

The tutorials section contains step by step guide for using features from Isaac Sim in the categories listed below. We strongly recommend completing the Introductory Tutorials, GUI Tutorials, and Core API Tutorials in order to understand all the ways Omniverse Isaac Sim can be used, before moving on to the more focused tutorials.

Introductory: The fundamentals of working in Omniverse Kit and different workflows in Omniverse Isaac Sim.

GUI: The fundamental concepts of robotics in Omniverse Isaac Sim via GUI.

Development Tools: The tools and environments for debugging and development.

Core API: Building environments, robots, and tasks using Omniverse Isaac Sim Core Python APIs.

Scene setup: Importing and modifying robots and manipulators from external sources.

Motion Generation: Lula motion generation library.

ROS/ROS2: Examples of various ROS and ROS2 bridges and interfaces.

Mapping: Occupancy map generation.

Isaac Gym Tutorials: Reinforcement learning framework and Cloner APIs.

Replicator Tutorials: Synthetic data generation.

Isaac Cortex Tutorials: High level behavioral programming.

Features

The features section highlights Omniverse Isaac Sim extensions and tools for setting up environments, robot, and sensors, as well as physics toolkits, bridges, exporters, and debuggers.

Concepts

The concepts section contains explanations of glossaries and Omniverse Isaac Sim mechanics like world representation and environment.

How-To-Guides

The How-To-Guides contains short code snips and tips for specific Omniverse Isaac Sim operations.

Reference Material

The Reference Material section provides references, links, and resources to assist developers.

This section also provides a suite of useful examples for users to quickly try intermediate and advanced applications in Omniverse Isaac Sim. Each example includes well-commented source code that users grab and modify for their own use cases.

See the complete list of examples in the Isaac Examples Menu.

Isaac Sim USD Assets

The Isaac Sim USD Asset section provides a full list of supported environments, robots, props, sensors, and featured assets to assist developers.