Unitree H2 Plus Bimanual Pick, Handover & Place#

Use a PICO 4 Ultra headset and its handheld XR controllers to record demonstrations of a simulated Unitree H2 Plus picking up an apple, transferring it between hands, and placing it on a plate. It covers recording, conversion, training, and evaluation in Isaac Lab-Arena, with optional Isaac AutoData generation.

AVAILABLE

Unitree H2 Plus · NVIDIA Isaac Lab-Arena · AutoData · GR00T N1.7

H2 policy training is validated. Learned-policy evaluation is implemented but not yet validated.

Understand the Workflow#

Conceptual Overview

Understand how recording, optional data generation, conversion, training, and evaluation connect.

Conceptual Overview
Architecture and Data Requirements

Review the Isaac Lab-Arena setup, robot-control software, dataset requirements, and external asset ownership before running the stages.

H2 Architecture and Data Requirements

Before You Start#

Complete Installation to prepare the workstation. Recording requires a PICO 4 Ultra on the workstation’s LAN and its paired handheld XR controllers. Check the XR recording requirements and follow the H2 CloudXR setup before recording.

Choose a published stage input to begin with conversion, training, or evaluation. Published HDF5 conversion needs the GR00T runtime but no headset, CloudXR session, or simulation assets. Each stage page lists the setup required for its selected input.

Follow the Workflow Path#

To collect your own data, complete the six stages in order. After recording, continue directly to conversion or use optional data generation first. Each card opens a separate H2 stage page.

Recording uses Isaac Lab-Arena with Pink IK for the arms and AGILE for the lower body (h2_agile_pink), operated with handheld XR controllers. The recording guide selects this setup explicitly; the launcher’s default native G1 mode does not apply to H2.

Use the Environment Variable Reference to restore asset paths, runtime settings, and the existing run in a new terminal.

Asset Preparation

1

Prepare the specified Unitree H2 Plus simulation model, generate the URDF used by Pink IK, and download the Unitree H2 AGILE policy.

H2 Asset Preparation
Simulation Validation

2

Run the Unitree H2 simulation checks and practice driving the robot.

H2 Simulation Validation
Demonstration Recording

3

Record HDF5 demonstrations using the PICO 4 Ultra headset and handheld XR controllers. The task scores each episode automatically; replay the accepted episodes.

H2 Demonstration Recording
Dataset Conversion

4

Convert the selected episodes to LeRobot and check that their state, action, and video fields match the policy training requirements for Unitree H2.

H2 Dataset Conversion
Policy Training

5

Train GR00T N1.7 on the validated dataset of Unitree H2 demonstrations.

H2 Policy Training
Policy Evaluation

6

Run the learned 61-D joint policy, measure placement success, and review the handover sequence separately.

H2 Policy Evaluation

Tip

Use a published artifact to start at a later stage:

Workflow Artifacts#

The H2 guides keep each run in outputs/h2/<timestamp>/ under the repository root. H2_RUN_DIR identifies that directory. Keep its paths when continuing the run, and choose a new directory for another run.

Stage

Output in the Run Directory

Next Input

Recording and state replay

source.hdf5

Direct conversion or optional AutoData generation

Optional generation

generated.hdf5 and generation.json

Replay and conversion of the generated HDF5

Conversion and loader validation

lerobot/ and gr00t-loader.json

Validated LeRobot dataset for GR00T training

Training

training/checkpoint-<step>/

Checkpoint preflight and policy server

Evaluation

evaluation/<timestamp>/

Episode metrics, HTML report, and camera videos

For conversion, H2_TRAINING_HDF5 selects the recorded or generated file and H2_TRAINING_DEMOS holds its accepted episode count. The download instructions set these values for published inputs.

Hugging Face Dataset and Checkpoint#

The checkpoint and dataset revisions are pinned in runtime.toml. The evaluation guide accepts your own training output or the published inference checkpoint.

Dataset

DATASET ON HUGGING FACE

nvidia/GR00T-E2E-Reference-Workflow-SIM-H2-N1.7-AppleHandoverPlate-Dataset

Recorded HDF5, generated HDF5, and LeRobot demonstrations for the Unitree H2 Plus bimanual pick, handover, and place simulation workflow.

https://huggingface.co/datasets/nvidia/GR00T-E2E-Reference-Workflow-SIM-H2-N1.7-AppleHandoverPlate-Dataset
Checkpoint

MODEL ON HUGGING FACE

nvidia/GR00T-E2E-Reference-Workflow-SIM-H2-N1.7-AppleHandoverPlate-Model

Fine-tuned GR00T N1.7 checkpoint for the same Unitree H2 simulation task.

https://huggingface.co/nvidia/GR00T-E2E-Reference-Workflow-SIM-H2-N1.7-AppleHandoverPlate-Model

Supporting Guidance#

Data Generation (Optional)

Expand qualifying teleoperation demonstrations with Isaac AutoData when more data is needed. Retain only attempts that pass the shared task predicate.

H2 Data Generation With Isaac AutoData (Optional)