H2 Policy Evaluation#
Run a trained Unitree H2 GR00T policy in Isaac Lab-Arena and measure placement success. Review the recorded motion separately to assess the full bimanual pick, handover, and place sequence.
Learned-policy evaluation is implemented but not yet validated. The commands below support integration checks and checkpoint assessment.
Use a Published Checkpoint#
We provide a trained policy for H2 so you can skip recording, data generation, conversion, and training, or check your work against our reference. Run the command below from the workflow repository root to download it, then follow H2 policy evaluation to run it in simulation.
If you want to train your own model, proceed to H2 policy training.
Before downloading, complete Hugging Face access setup.
uv run --frozen --no-sync hf download nvidia/GR00T-E2E-Reference-Workflow-SIM-H2-N1.7-AppleHandoverPlate-Model \
--revision 13397daebd2c34986fc18820374a4a2d33861482 \
--exclude optimizer.pt rng_state.pth scheduler.pt trainer_state.json training_args.bin \
--local-dir outputs/h2/published-checkpoint
The model is saved in outputs/h2/published-checkpoint. Select Provided model
when setting up policy evaluation for H2;
that choice sets the checkpoint path and source for both preflight and server startup.
Keep the .cache/huggingface/ directory created inside the download directory;
the launcher uses it to check that the model matches the pinned download.
The inference download does not need optimizer or trainer-resume state. Model files, weight shards, processor, normalization statistics, embodiment, and action-horizon checks still apply. Complete the simulator and GR00T setup on the evaluation page; no headset is needed.
Prerequisites#
Use the published checkpoint above or a checkpoint from H2 Policy Training, plus the assets from H2 Asset Preparation. Run commands from the repository root on the GPU workstation. Use one terminal for the policy server and another for simulation evaluation.
Preflight and Start the Policy Server#
In the server terminal, fetch the managed Arena checkout and restore the GR00T and H2 asset paths. Use the
pink_urdfvalue from the successful asset preparation JSON forE2E_H2_PINK_URDF. It ends inunitree_h2.urdf:export E2E_ROOT="$PWD"
export ARENA_ROOT="$(./tools/fetch_arena_checkout.sh)"
export ISAAC_GR00T_ROOT=/absolute/path/to/Isaac-GR00T
export E2E_H2_ASSET_CACHE="$HOME/.cache/e2e-reference-app/h2plus"
export E2E_H2_PINK_URDF="/absolute/path/to/unitree_h2.urdf"
export E2E_H2_AGILE_POLICY=/absolute/path/to/agile_velocity.onnx
See Prepare the Isaac Lab-Arena Checkout for checkout details. The standalone GR00T root is passed explicitly, so no Arena submodule is needed.
Choose one model. Run only the matching block in the same terminal:
Provided model — downloaded above:
export H2_CHECKPOINT="$E2E_ROOT/outputs/h2/published-checkpoint" export H2_CHECKPOINT_SOURCE=published
Your own model — produced by policy training for H2:
export H2_CHECKPOINT=/absolute/path/to/checkpoint-10000 export H2_CHECKPOINT_SOURCE=trained
Replace the path in the second block with your checkpoint directory. For the training smoke test, use
training-smoke/checkpoint-10instead. Both commands below use your selected model and source.Preflight the checkpoint:
./tools/launch_gr00t_server.sh \ --robot h2 \ --arena-root "$ARENA_ROOT" \ --gr00t-root "$ISAAC_GR00T_ROOT" \ --model-path "$H2_CHECKPOINT" \ --checkpoint-source "$H2_CHECKPOINT_SOURCE" \ --check
The check must report the H2 policy with 75 state values, 61 action values, and a 40-step horizon.
Start the policy server in the same terminal:
./tools/launch_gr00t_server.sh \ --robot h2 \ --arena-root "$ARENA_ROOT" \ --gr00t-root "$ISAAC_GR00T_ROOT" \ --model-path "$H2_CHECKPOINT" \ --checkpoint-source "$H2_CHECKPOINT_SOURCE" \ --device cuda \ --host 127.0.0.1 \ --port 5555
Wait for
Server is ready and listening on tcp://127.0.0.1:5555before launching evaluation. If checking only thatcheckpoint-10loads, follow Shut Down and return to training.
Run a Visual Smoke Test#
In the evaluation terminal, restore the runtime and output paths. Use the same
pink_urdfand AGILE policy paths as the server terminal:export ARENA_ROOT="$(./tools/fetch_arena_checkout.sh)"
export ISAAC_GR00T_ROOT=/absolute/path/to/Isaac-GR00T
export H2_RUN_DIR=/absolute/path/to/outputs/h2/YYYYMMDD_HHMMSS
export E2E_H2_ASSET_CACHE="$HOME/.cache/e2e-reference-app/h2plus"
export E2E_H2_PINK_URDF="/absolute/path/to/unitree_h2.urdf"
export E2E_H2_AGILE_POLICY=/absolute/path/to/agile_velocity.onnx
Run one episode:
OMNI_KIT_ACCEPT_EULA=YES ACCEPT_EULA=Y \ ./tools/launch_gr00t_eval.sh \ --robot h2 \ --arena-root "$ARENA_ROOT" \ --gr00t-root "$ISAAC_GR00T_ROOT" \ --remote_host 127.0.0.1 \ --remote_port 5555 \ --num_episodes 1 \ --device cpu \ --policy_device cpu \ --output_base_dir "$H2_RUN_DIR/evaluation-smoke" \ --viz kit \ -- --episode_length_s 30
Confirm the episode completes and writes the Arena metrics and report. A one-episode run checks integration; it does not qualify learned-policy quality.
Evaluate a Batch#
After the smoke test completes, run 100 episodes in the same evaluation terminal:
OMNI_KIT_ACCEPT_EULA=YES ACCEPT_EULA=Y \ ./tools/launch_gr00t_eval.sh \ --robot h2 \ --arena-root "$ARENA_ROOT" \ --gr00t-root "$ISAAC_GR00T_ROOT" \ --remote_host 127.0.0.1 \ --remote_port 5555 \ --num_episodes 100 \ --device cpu \ --policy_device cpu \ --record_camera_video \ --output_base_dir "$H2_RUN_DIR/evaluation" \ --viz none \ -- --episode_length_s 30
Both examples allow 30 seconds of simulation time per episode. This changes only the time budget. Inference runs on the server GPU; simulation and client tensors use the CPU. Use distinct output directories and seeds for independent evaluation batches.
Open
index.htmlin the timestamped directory under$H2_RUN_DIR/evaluation/. Check the completed episode count and review the per-episode results and camera videos:Output
Contents
episode_results_rank0.jsonlOne record per completed episode
index.htmlBrowsable evaluation report
robot-cam-env0-*-episode-*.mp4Per-episode policy-camera videos
The final terminal metrics line identifies the completed episode count and rates. For example, a ten-episode run with a locally trained checkpoint at step 10000, seed 43, and a 30-second episode budget printed:
[Rank 0/1] Metrics: {'num_episodes': 10, 'success_rate': 0.8, 'object_moved_rate': 1.0}That run reported eight placement successes and movement in all ten episodes. Use this example to recognize the output format; measure your selected checkpoint on the current task separately. Compare
num_episodeswith your requested count and the records inepisode_results_rank0.jsonl. An incomplete run must not be reported as a completed 100-episode evaluation.
Important
success_rate measures placement success: supported, settled contact between
apple and plate with both hands clear. It does not check pickup or handover
history. Review the recorded sequence separately before claiming complete
bimanual task success, and report that assessment apart from placement success.
Compare rates only for a checkpoint trained on the matching H2 scene, state and action formats, and camera stream. See H2 success criteria for how the criteria for accepting a recording differ from evaluation.
Shut Down#
After the evaluator exits, return to the server terminal and press Ctrl+C. Wait for
Shutting down server....Confirm port 5555 has no listener before starting another server:
ss -ltnp 'sport = :5555'
The output should contain no
LISTENentry. The H2 evaluator closes its client connection but leaves the separately launched policy server running.
Expected Result#
The timestamped evaluation directory contains the episode records, report, and camera videos. Report placement success with the completed episode count and checkpoint used. Full handover success remains a separate assessment.