CHORD

Paper arXiv soon Code soon
Scroll to Explore

Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration

Xinghao Zhu*, ‡Zixi Liu*Shalin Jain*Chenran Li†Milad Noori†Huihua ZhaoJohn Welsh
Michael Andres LinWei LiuTingwu WangXingye DaZhengyi LuoVishal KulkarniNaema Bhatti
Yuke ZhuLinxi FanBowen WenDanfei XuSoha PouyaYan Chang‡
*Equal Contribution†Core Contributor‡Project Lead & Corresponding Author

Overview

CHORD transfers human manipulation skills to dexterous robot policies through an object-centric contact-wrench representation — matching how contact moves an object, not just where it occurs.

Human Demonstration
Simulation
Real World

This video includes voiceover narration — unmute to listen.

Abstract

Dexterous robot manipulation can benefit from the abundance of human demonstrations, but transferring such demonstrations to robot policies remains challenging. We present CHORD, a framework for long-horizon manipulation of rigid and articulated objects with reinforcement learning. The key idea is object-centric contact wrench space guidance: we represent human and robot motions by the forces and torques they can induce on the object, enabling similarity to be measured by the induced instantaneous motions. This guidance makes reinforcement learning more scalable for contact-rich dexterous manipulation. We further introduce a large-scale simulation benchmark with 4,739 bimanual dexterous manipulation tasks, constructed from motion-capture datasets and reconstructed in-house videos. Evaluated on 1,831 benchmark tasks, CHORD achieves an average success rate of 82.12%, demonstrating strong scalability. CHORD also generalizes to whole-body manipulation from hand-only and third-person demonstrations, achieving a 90.77% success rate, and the learned policies transfer to the real world in both open-loop and closed-loop settings.

Contact Wrench Guidance

Position guidance targets where contact occurs; wrench guidance targets how contact affects object motion. Hover a panel to reveal the result.

Human hand demonstration at a box contact point
Human Demonstration
Robot hand using a position marker at the contact point
Position Guidance
Robot hand with contact wrench arrow guidance
Wrench Guidance

Hover Position or Wrench to see the resulting manipulation in full.

Position marker manipulation result

Position Guidance

The red marker is the human contact; the heat map shows nearby locations that look plausible under a position-matching objective. But locations close in space induce different contact normals and moment arms — matching where contact occurs can still produce unstable object motion.

Wrench guidance manipulation result

Wrench Guidance · Ours

CHORD represents each contact with a six-dimensional wrench combining force and torque, computed from the contact location, surface normal, and moment arm. Matching wrenches asks how the contact moves the object — reproducing the force-and-torque generation behind the demonstrated manipulation.

Turning demonstrated contacts into rewards

CHORD converts demonstrated contacts and contact wrenches into rewards for learning dexterous robot-hand policies. Hover a numbered region to explore each stage.

CHORD reward formulation from reference contact to contact wrench reward
1 · Extract Human Contacts From a demonstration, CHORD extracts contact events and estimates local friction constraints, shown as friction cones.
2 · Human Contact Wrench Per frame and hand, CHORD computes a human contact wrench. A wider manifold means richer force-and-torque authority over the object.
4 · Policy Contact Wrench Blue manifolds are produced by the learned policy — closely matching the demonstration across frames.
3 · Imitation + Contact Wrench CHORD combines imitation and contact-wrench rewards to train the dexterous robot-hand policy.

Dexterous Manipulation Benchmark with Human Demonstrations

Large-scale, long-horizon, contact-rich tasks paired with human demonstrations spanning rigid, articulated, and multi-object manipulation.

4,739 Simulation-ready tasks
Benchmark wrench distribution highlights
Task distribution with horizon, contact events, and Ferrari-Canny epsilon metrics.

Capabilities

A unified contact-wrench representation carries human manipulation skills across diverse manipulation behaviors, long-horizon tasks, whole-body embodiments, and real-world hardware.

Diverse Manipulation Behaviors

Contact-wrench guidance transfers across grasping, reorientation, handovers, and tool use.

{{ v.cap }}

Long-Horizon Manipulation

CHORD preserves contact intent over extended sequences with dense contact events.

{{ v.cap }}

Large-Scale Evaluation

We open-source 4,739 simulation-ready dexterous manipulation tasks paired with human demonstrations. Across 1,831 diverse evaluations, CHORD reaches an average success rate of 82.12%.

1,831 Benchmark evaluations
82.12% Average success rate

Hover a clip to preview

Whole-Body Loco-Manipulation

The contact-wrench representation is embodiment-agnostic: with a whole-body motion inpainting module, humanoid robots equipped with three-fingered hands can learn from hand-only and third-person-view demonstrations.

90.77% Whole-body success rate

Hand-Only Demonstrations

Third-Person-View Demonstrations

Real-World Deployment

Learned policies transfer from simulation to real dexterous hands.

3×
{{ v.cap }}

Quantitative Evaluation

CHORD sustains high success at scale and outperforms prior methods, succeeding across rigid, articulated, and multi-object manipulation.

Comparison With Prior Methods

Task Suite Metric Ref. Method Ref. Score Our Score
DM AUC DexMachina 0.232 ± 0.214 0.687 ± 0.358
MT MT-SR ManipTrans 0.428 0.639
SP SP-SR Spider 0.333 ± 0.488 0.359 ± 0.482
Ours-1 AUC DexMachina 0.211 ± 0.138 0.895 ± 0.052
Ours-1 SP-SR Spider 0.133 ± 0.327 0.999 ± 0.000
Ours-2 SP-SR Spider 0.533 ± 0.503 0.982 ± 0.022

Scores are comparable within each row; baselines use different task suites and evaluation protocols.

Large-Scale and Whole-Body Evaluation Results

Per-dataset success rates across benchmark tasks
Reward Correlation
Correlation between contact wrench reward and task success
Across 1,831 runs, normalized contact-wrench-support reward strongly correlates with task completion (Pearson r = 0.798; dataset-wise r = 0.76–0.89) — a reliable training signal and proxy metric for manipulation.
Tracking Horizon
Tracking accuracy across manipulation horizons
Under the DexMachina ADD-AUC metric, CHORD maintains high object-tracking accuracy across sequences up to roughly 48 seconds, while prior methods degrade as horizons grow.