Developer Guide#
The developer guide separates the everyday edit-and-build loop from the strict validation commands used before handoff.
Add a workflow path while preserving the repository’s environment, adapter, data, and validation contracts.
Development Environment#
The locked runtime uses Python 3.12. Install the complete simulator, development, and documentation environment from the repository root:
uv sync --frozen
For CPU-only static checks and dataset work:
uv sync --frozen --no-default-groups --extra dev --extra converter
uv.lock specifies the dependency set, runtime.toml records the tested external
software versions and model revisions, and tools/validate_repository.sh owns the validation
commands.
Simulation Assets#
The static-apple scenes load versioned NVIDIA Isaac Sim assets over public HTTPS.
Scene loading requires network access but no asset-server login. Unitree G1 and
Unitree H2 share the warehouse, table, apple, and plate assets. The G1 robot uses
its own public USD configured in configs/robots/g1.yaml.
Set E2E_ASSET_ROOT to use a collected library mirror instead of the public
scene assets. The mirror must retain these directories and entry points,
including their referenced resources:
background_library/warehouse/:warehouse.usdobject_library/table/:table.usdobject_library/apple/:apple.usdobject_library/clay_plate/:clay_plate.usd
These library-relative names are independent of the public filenames. An unset or empty variable selects the public defaults.
The robot is configured separately: E2E_ASSET_ROOT does not change the G1 USD.
To select a different robot USD, set E2E_G1_ROBOT_CONFIG to a complete G1 YAML
configuration with that USD in its usd field.
The public G1 USD uses its default white materials. Public transport URLs and
E2E_ASSET_ROOT are excluded from the environment contract digest. Library-relative
asset paths (usd_relpath) remain included. Matching contracts therefore do not
prove matching visual appearance. Keep the recording manifest’s
repository.revision and the selected robot and scene URLs with each dataset to
distinguish asset variants when preparing policy data.
Documentation Development Workflow#
The standard uv sync --frozen command installs the documentation tools. For
a smaller docs-only environment, run:
uv sync --frozen --no-default-groups --only-group docs
During normal editing, build the NVIDIA-themed HTML site directly:
uv run --no-sync sphinx-build -b html docs docs/_build/html
This command performs an incremental build. The generated site is written to
docs/_build/html/. Open docs/_build/html/index.html directly or serve it
locally:
uv run --no-sync python -m http.server \
--directory docs/_build/html \
8000
Open http://127.0.0.1:8000/ in a browser.
Documentation Style#
Use the guide to recording demonstrations on the physical G1 as the structural example for workflow stages:
State the task and expected result in a short opening paragraph.
Identify prerequisites, execution host, repository, environment, and terminal.
Divide the procedure into named sections with numbered actions. Use bullets for alternatives, and make optional stages explicit.
Show the expected output and carry its path into the next stage.
Use Title Case headings and parallel stage names. Write direct instructions,
keep UI labels and keyboard keys bold, and format commands, paths, variables,
and literal output as code. Use E2E in titles and end-to-end in ordinary prose.
Keep implementation explanations in reference or architecture pages.
Put each independent command in its own copyable code block. Group related environment exports or variable assignments that prepare one step. Keep multiline commands, pipelines, and heredocs intact. State when to switch terminals or environments; keep alternative commands and example output in separate blocks.
Callout |
Purpose |
|---|---|
Note |
Brief context or behavior distinction |
Tip |
Optional improvement or shortcut |
Important |
An easily missed requirement or limitation beside the affected step |
Warning |
A concrete risk, including data loss or invalid output, before the action |
Danger |
A serious physical hazard, preserving established safety requirements |
Dropdown |
Conditional troubleshooting or extended background |
See also |
Supporting references, separate from next-stage navigation |
Keep required actions in the main procedure. Keep callouts short and place them
before the action they affect. Use MyST colon-fenced directives such as
:::{note}; for a custom title, use :::{admonition} Title with :class: note
or the appropriate severity. Verify headings, callouts, cards, images, and
navigation in the rendered page at desktop and mobile widths.
Documentation Validation Workflow#
Before handing off a documentation change, run the documentation checks:
uv run --no-sync ./tools/validate_repository.sh docs
The command rebuilds every page in a fresh Sphinx environment. Warnings fail the build, and Sphinx checks internal and external links.
Complete the repository validation checks before handing off changes to documentation dependencies, tooling, or CI.
Repository Validation#
Run all static checks and CPU tests:
uv run --no-sync ./tools/validate_repository.sh host
The host validation command runs static checks and CPU tests without starting Isaac Sim.
On a compatible Linux x86_64 GPU host, validate the runtime and simulator behaviors:
uv run ./tools/validate_repository.sh runtime-preflight
uv run ./tools/validate_repository.sh sim-static-apple
uv run ./tools/validate_repository.sh sim-agile-locomotion
uv run ./tools/validate_repository.sh sim-recorder
The Unitree G1 Isaac Lab-Arena modes need a GPU and fetch the pinned Isaac Lab-Arena checkout themselves, as described under how Arena is obtained:
uv run --no-sync ./tools/validate_repository.sh sim-common-environment
uv run --no-sync ./tools/validate_repository.sh sim-arena-recording
uv run --no-sync ./tools/validate_repository.sh sim-arena-locomotion
The Unitree H2 Isaac Lab-Arena mode needs the same GPU runtime and additionally runs GR00T
statistics and its actual sharded training loader. It refuses to start until
the Unitree H2 robot model inputs and a clean, synced GR00T checkout at the runtime.toml
revision are supplied:
export E2E_H2_ASSET_CACHE=<path>/h2plus
export E2E_H2_PINK_URDF=<path>/unitree_h2.urdf
export E2E_H2_AGILE_POLICY=<path>/agile_velocity.onnx
export ISAAC_GR00T_ROOT=<path>/Isaac-GR00T
Then run the H2 check in the same terminal:
uv run --no-sync ./tools/validate_repository.sh sim-arena-h2
See Run the Unitree H2 Plus Arena workflow path.
Manual Simulator Checks#
The three built-in-task modes remain supported manual checks and provide the comparison path described in environment ownership.
Run sim-arena-h2 manually with the AGILE checkpoint and materialized Pink sidecar.
Environment Ownership#
Isaac Lab-Arena handles composition and evaluation. Recording and raw replay support both native Isaac Lab and Isaac Lab-Arena. The native implementation remains supported for existing users, and Arena is an explicit alternative. Their data uses a shared joint-space training format only after conversion.
G1StaticAppleEnvCfg therefore remains a supported data-production environment, not disposable
scaffolding. sim-common-environment checks that it and Isaac Lab-Arena still use the same task definition
without implying that their raw action schemas are interchangeable.
Entry point |
Current owner/status |
|---|---|
|
native by default; |
|
native by default; |
|
complete: the manifest routes native Unitree G1 28/32-D, Isaac Lab-Arena Unitree G1 23/50-D, and Isaac Lab-Arena Unitree H2 61-D raw actions to their owning environment |
|
native fixed-base behavior |
|
native cross-check alongside |
|
remains the native recorder/replay regression; Isaac Lab-Arena has its own recording checks |
Isaac Lab-Arena AutoData |
workflow-owned demonstration generation for Unitree G1 and Unitree H2 is implemented; see G1 data generation and H2 data generation for the run procedures |
Isaac Lab-Arena is reachable through its pinned launcher.
GR00T Modality Divergence#
The modality layout this repository registers is not Isaac Lab-Arena’s
isaaclab_arena_gr00t/embodiments/g1/modality.json. Two differences are intentional
and are the only ones:
Action horizon. 40 here, against 50 in Isaac Lab-Arena’s built-in Unitree G1 configuration. It is one parameter, appearing as
_ACTION_HORIZONinsrc/e2e_robot_workflows/gr00t/g1_static_apple_modality_config.pyand asaction_horizoninconfigs/stages/gr00t/closedloop_eval.yaml; training and evaluation must state the same value, and 40 is what GR00T-N1.7-3B’smax_action_horizonaccepts. A 50-horizon config crashes N1.7 in its data processor with a negative tensor dimension.Omitted action groups. Isaac Lab-Arena’s action layout includes
left_wrist_pose,right_wrist_pose, andtorso_orientation_rpy_command; this one does not. The wrist poses are kept as state, where they report the achieved pose, but not as actions: a commanded wrist pose is the input Pink IK solves, andjoint_pos_targetis what it solved to, so recording both would train two representations of one command. The torso-orientation command has no source at all on this task, whose AGILE action term publishes only the navigate and base-height commands.
Both are recorded in-repo rather than raised upstream: Isaac Lab-Arena’s layout is correct for the model and task it was tuned for, so this is a per-application choice rather than a defect.
Optional UI Smoke Tests#
Launch the finite recorder check:
OMNI_KIT_ACCEPT_EULA=YES ACCEPT_EULA=Y \
uv run --no-sync python scripts/record_demos.py \
--viz kit \
--teleop_device keyboard \
--sample_steps 300 \
--sample_action wrist_sine \
--sample_export \
--dataset_file /tmp/e2e-recorder-ui-smoke.hdf5
Launch the same scene through Isaac Lab-Arena:
OMNI_KIT_ACCEPT_EULA=YES ACCEPT_EULA=Y \
./tools/launch_arena_eval.sh \
--viz kit \
--rendering_mode quality \
--num_steps 600 \
--policy_type e2e_robot_workflows.arena.g1_smoke_policy.G1StandingSineSmokePolicy \
--output_base_dir /tmp/e2e-arena-ui \
--no_solve_relations
Both checks exit automatically. A zero smoke-test success rate is valid because these commands qualify loading, action flow, recording, metrics, and lifecycle, not checkpoint performance.
Exact runtime ownership and supported boundaries are collected in Reference.
Native AutoData Integration#
Native AutoData annotation and generation use AutoData’s isolated interpreter.
Arena generation uses this repository’s Arena runtime with the pinned AutoData
source overlaid. Both require a clean AutoData checkout at the revision in
runtime.toml.
The native bridge registers this repository’s environment ids before running
AutoData’s unmodified scripts through
python -m e2e_robot_workflows.autodata. The fixed and AGILE environments expose
separate action spaces. Their descriptors live in configs/stages/autodata/:
File |
Defines |
|---|---|
|
Fixed native 28-D actions |
|
Native 32-D actions with AGILE locomotion |
|
Right-hand transfer subtasks and boundary signal |
|
Left-hand transfer subtasks and boundary signal |
The generation environment supplies a success termination, randomized apple
pose, and subtask boundary signal. The recording environment instead delegates
episode admission to the task completion policy. AutoData’s environment factory
also restores this repository’s recorder terms, including joint_pos_target.
AGILE generation records navigate_cmd and base_height_cmd; fixed generation
omits them.
Important
A success termination must be callable. AutoData’s attribute-existence check
alone accepts None, which cannot score annotation or generation correctly.
After provisioning, validate both native modes and the left-hand descriptor against live environments:
./tools/launch_autodata.sh validate --wbc-mode fixed
./tools/launch_autodata.sh validate --wbc-mode agile
./tools/launch_autodata.sh validate --wbc-mode agile --manipulation-hand left
These checks compare the action width and observations with the descriptors. They exercise placement success and the hand-held-object boundary in both positive and negative cases. They require the AutoData simulator environment.
To check the installation with AutoData’s own pre-annotated demonstrations:
./tools/launch_autodata.sh example \
--viz none --device cpu --num_envs 5 --generation_num_trials 10
Validate the generated file after the example finishes:
./tools/launch_autodata.sh validate-dataset \
outputs/autodata/g1_example_generated.hdf5
This example uses AutoData’s task and data. It does not qualify this
repository’s recorded demonstrations. Use --viz kit to view the attempts.
Legacy Recording Migration#
Migration applies only to older workflow recordings without the root HDF5
attribute e2e_workflow_manifest. Current recordings already contain it.
A newly merged file missing its manifest is not evidence of a legacy recording;
retain and use its original manifested inputs for replay and AutoData.
Check for the attribute without modifying the file:
uv run --extra converter python - /absolute/path/to/recording.hdf5 <<'PY' import sys import h5py from e2e_robot_workflows.data.dataset_manifest import MANIFEST_ATTRIBUTE with h5py.File(sys.argv[1], "r") as dataset: print("Manifest present:", MANIFEST_ATTRIBUTE in dataset.attrs) PY
If present, use the original recording. Investigate a malformed or incompatible manifest instead of removing it to force migration.
If absent, identify the original recording method, action layout, and manipulating hand from the collection configuration. A filename or action width alone cannot identify those settings. For a confirmed native G1 AGILE recording using the right hand and 32-D layout:
uv run --extra converter python -m e2e_robot_workflows.data.dataset_manifest migrate-recording \ --input-file datasets/legacy.hdf5 \ --output-file datasets/legacy-migrated.hdf5 \ --lane native \ --layout native_g1_pink_agile_32d_v1 \ --manipulation-hand right \ --confirm-legacy-migration
Use values matching the original recording. List supported layouts with
uv run --extra converter python -m e2e_robot_workflows.data.dataset_manifest migrate-recording --help.
The output must be a new file. The original remains untouched. The copy is
marked legacy_unverified: annotation and generation accept it, but it remains
excluded from qualified workflow evidence. Return to
G1 data generation
with the migrated copy as the source.