H2 Policy Training#
Fine-tune GR00T N1.7 on the converted dataset of Unitree H2 demonstrations to produce a checkpoint for simulation evaluation.
Prerequisites#
Use the published dataset below or complete H2 Dataset Conversion. Training uses the standalone GR00T environment prepared in H2 Asset Preparation. Open a fresh workstation terminal outside any distributed training job, then run the following commands from the workflow repository root.
The 1,000-demo H2 recipe was validated on one 48 GB NVIDIA RTX 5880 Ada. It freezes the visual backbone and LLM while training the projector and diffusion model. Other GPU capacities may require a smaller batch.
Use a Published Dataset#
We provide a dataset of H2 demonstrations so you can skip recording, data generation, and conversion, or check your work against our reference. Run the command below from the workflow repository root to download it.
To use your own converted dataset, skip this download and continue with policy training inputs.
Before downloading, complete Hugging Face access setup.
uv run --frozen --no-sync hf download nvidia/GR00T-E2E-Reference-Workflow-SIM-H2-N1.7-AppleHandoverPlate-Dataset \
--revision 72ad851a5d0b845d368f031c744876613fb0b3c7 \
--repo-type dataset \
--include 'generated/lerobot/**' \
--local-dir outputs/h2/downloads
The dataset is saved in outputs/h2/downloads/generated/lerobot. Continue to
Set the Training Inputs
and select Provided dataset. Use a new H2_RUN_DIR for training outputs.
This dataset contains 1,000 generated episodes; the 10 human seeds are not extra training episodes. It is
self-contained and does not need the raw HDF5 files. To run the
training-input validation,
set H2_TRAINING_DEMOS=1000 and keep the downloaded H2_LEROBOT path.
Set the Training Inputs#
Set the workspace and output paths in this terminal. For the provided dataset, choose a new
H2_RUN_DIRfor training outputs. For your own dataset, use the run directory containing your converted data:export E2E_ROOT="$PWD"
export ISAAC_GR00T_ROOT=/absolute/path/to/Isaac-GR00T
export H2_RUN_DIR=/absolute/path/to/outputs/h2/YYYYMMDD_HHMMSS
export H2_MODALITY="$E2E_ROOT/src/e2e_robot_workflows/gr00t/h2_static_apple_modality_config.py"
export TRAIN_OUTPUT="$H2_RUN_DIR/training"
export H2_SMOKE_OUTPUT="$H2_RUN_DIR/training-smoke"
Choose one dataset. Run only the matching block:
Provided dataset — downloaded above:
export H2_LEROBOT="$E2E_ROOT/outputs/h2/downloads/generated/lerobot"
Your own dataset — converted in your run directory:
export H2_LEROBOT="$H2_RUN_DIR/lerobot"
Both training commands below use your selected dataset.
Obtain access to both GR00T N1.7 and Cosmos-Reason2-2B, then authenticate and download the base model:
"$ISAAC_GR00T_ROOT/.venv/bin/hf" auth login
Verify the active account:
"$ISAAC_GR00T_ROOT/.venv/bin/hf" auth whoami
Download the base model:
"$ISAAC_GR00T_ROOT/.venv/bin/hf" download nvidia/GR00T-N1.7-3B
Check access to the processor configuration:
"$ISAAC_GR00T_ROOT/.venv/bin/hf" download nvidia/Cosmos-Reason2-2B config.json
The Cosmos download checks access before training loads its processor.
Run a Training Smoke Test#
In the same terminal, train for ten steps into a separate directory:
(cd "$ISAAC_GR00T_ROOT" && \ "$E2E_ROOT/tools/gr00t_runtime.sh" --gr00t-root "$ISAAC_GR00T_ROOT" -- \ python gr00t/experiment/launch_finetune.py \ --base-model-path nvidia/GR00T-N1.7-3B \ --dataset-path "$H2_LEROBOT" \ --output-dir "$H2_SMOKE_OUTPUT" \ --embodiment-tag new_embodiment \ --modality-config-path "$H2_MODALITY" \ --global-batch-size 32 \ --max-steps 10 \ --num-gpus 1 \ --save-steps 10 \ --save-total-limit 5 \ --no-tune-llm \ --no-tune-visual \ --tune-projector \ --tune-diffusion-model \ --dataloader-num-workers 4 \ --color-jitter-params \ brightness 0.3 \ contrast 0.4 \ saturation 0.5 \ hue 0.08)
Require finite loss and
$H2_SMOKE_OUTPUT/checkpoint-10. Use the evaluation preflight and server startup with that checkpoint to verify it loads. Shut down the server before returning to this training terminal.
Train the Full Checkpoint#
Start a new run from the base model using the same validated dataset and modality:
(cd "$ISAAC_GR00T_ROOT" && \ "$E2E_ROOT/tools/gr00t_runtime.sh" --gr00t-root "$ISAAC_GR00T_ROOT" -- \ python gr00t/experiment/launch_finetune.py \ --base-model-path nvidia/GR00T-N1.7-3B \ --dataset-path "$H2_LEROBOT" \ --output-dir "$TRAIN_OUTPUT" \ --embodiment-tag new_embodiment \ --modality-config-path "$H2_MODALITY" \ --global-batch-size 32 \ --max-steps 10000 \ --num-gpus 1 \ --save-steps 1000 \ --save-total-limit 5 \ --no-tune-llm \ --no-tune-visual \ --tune-projector \ --tune-diffusion-model \ --dataloader-num-workers 4 \ --color-jitter-params \ brightness 0.3 \ contrast 0.4 \ saturation 0.5 \ hue 0.08)
After training finishes, select the final checkpoint:
export H2_CHECKPOINT="$TRAIN_OUTPUT/checkpoint-10000"
Expected Result#
$H2_RUN_DIR/training/checkpoint-10000 contains the model, processor, statistics,
and training configuration needed by the H2 evaluation preflight. Checkpoints
are saved every 1,000 steps, retaining the most recent five.
Next Stage#
Continue to H2 Policy Evaluation.