Unitree H2 Plus Bimanual Pick, Handover & Place#
Use a PICO 4 Ultra headset and its handheld XR controllers to record demonstrations of a simulated Unitree H2 Plus picking up an apple, transferring it between hands, and placing it on a plate. It covers recording, conversion, training, and evaluation in Isaac Lab-Arena, with optional Isaac AutoData generation.
AVAILABLE
Unitree H2 Plus · NVIDIA Isaac Lab-Arena · AutoData · GR00T N1.7
H2 policy training is validated. Learned-policy evaluation is implemented but not yet validated.
Understand the Workflow#
Understand how recording, optional data generation, conversion, training, and evaluation connect.
Review the Isaac Lab-Arena setup, robot-control software, dataset requirements, and external asset ownership before running the stages.
Before You Start#
Complete Installation to prepare the workstation. Recording requires a PICO 4 Ultra on the workstation’s LAN and its paired handheld XR controllers. Check the XR recording requirements and follow the H2 CloudXR setup before recording.
Choose a published stage input to begin with conversion, training, or evaluation. Published HDF5 conversion needs the GR00T runtime but no headset, CloudXR session, or simulation assets. Each stage page lists the setup required for its selected input.
Follow the Workflow Path#
To collect your own data, complete the six stages in order. After recording, continue directly to conversion or use optional data generation first. Each card opens a separate H2 stage page.
Recording uses Isaac Lab-Arena with Pink IK for the arms and AGILE for the
lower body (h2_agile_pink), operated with handheld XR controllers. The recording
guide selects this setup explicitly; the launcher’s default native G1 mode does not
apply to H2.
Use the Environment Variable Reference to restore asset paths, runtime settings, and the existing run in a new terminal.
1
Prepare the specified Unitree H2 Plus simulation model, generate the URDF used by Pink IK, and download the Unitree H2 AGILE policy.
2
Run the Unitree H2 simulation checks and practice driving the robot.
3
Record HDF5 demonstrations using the PICO 4 Ultra headset and handheld XR controllers. The task scores each episode automatically; replay the accepted episodes.
4
Convert the selected episodes to LeRobot and check that their state, action, and video fields match the policy training requirements for Unitree H2.
5
Train GR00T N1.7 on the validated dataset of Unitree H2 demonstrations.
6
Run the learned 61-D joint policy, measure placement success, and review the handover sequence separately.
Tip
Use a published artifact to start at a later stage:
Dataset conversion accepts downloaded recorded or generated HDF5 demonstrations.
Policy training accepts the provided LeRobot dataset.
Policy evaluation accepts the provided trained model.
Workflow Artifacts#
The H2 guides keep each run in outputs/h2/<timestamp>/ under the repository
root. H2_RUN_DIR identifies that directory. Keep its paths when continuing
the run, and choose a new directory for another run.
Stage |
Output in the Run Directory |
Next Input |
|---|---|---|
Recording and state replay |
|
Direct conversion or optional AutoData generation |
Optional generation |
|
Replay and conversion of the generated HDF5 |
Conversion and loader validation |
|
Validated LeRobot dataset for GR00T training |
Training |
|
Checkpoint preflight and policy server |
Evaluation |
|
Episode metrics, HTML report, and camera videos |
For conversion, H2_TRAINING_HDF5 selects the recorded or generated file and
H2_TRAINING_DEMOS holds its accepted episode count. The
download instructions
set these values for published inputs.
Hugging Face Dataset and Checkpoint#
The checkpoint and dataset revisions are pinned in runtime.toml.
The evaluation guide accepts your own training output or the published inference
checkpoint.
DATASET ON HUGGING FACE
nvidia/GR00T-E2E-Reference-Workflow-SIM-H2-N1.7-AppleHandoverPlate-Dataset
Recorded HDF5, generated HDF5, and LeRobot demonstrations for the Unitree H2 Plus bimanual pick, handover, and place simulation workflow.
MODEL ON HUGGING FACE
nvidia/GR00T-E2E-Reference-Workflow-SIM-H2-N1.7-AppleHandoverPlate-Model
Fine-tuned GR00T N1.7 checkpoint for the same Unitree H2 simulation task.
Supporting Guidance#
Expand qualifying teleoperation demonstrations with Isaac AutoData when more data is needed. Retain only attempts that pass the shared task predicate.