Sirui Chen @eric_srchen

Excited to share what we've been working on! Huge kudos to Sirui for leading this! The single-policy approach for diverse scene interaction is just elegant. It's so rewarding to see it handle all those different motions with one unified solution. Honored to be a co-author on this journey. ❤️

Sirui Chen @eric_srchen

17 hours ago

Humanoids excel in free space but struggle with real-world contact. Meet SceneBot 🤖 the first unified RL framework for ALL free-space locomotion, terrain traversal and object interaction! By conditioning on per-link contact labels, it masters complex, interaction-rich tasks like

4 25 107 24K 58

Huge Congrats, Sirui! It’s been a pleasure collaborating with you. I really appreciate all the hard work you’ve put into making this project a reality. The single-policy paradigm for diverse scene interactions is incredibly exciting！I’m glad SuperOdometry could contribute to the system. Looking forward to seeing what the community builds on top of SceneBot!

Sirui Chen @eric_srchen

17 hours ago

SceneBot tracks both reference motions and body contact labels with global localization. Contact labels allow it to distinguish between placing two hands around an object. Global localization helps the robot to better align with the scene during training & inference time. This

1 0 2 605 0

How can robots learn dexterous manipulation from human demonstrations at scale? Excited to share CHORD: Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration. CHORD learns from human demos by focusing not only on where contact happens, but how that contact moves the object through force and torque guidance. This unified contact-wrench representation carries human manipulation skills across diverse behaviors, long-horizon tasks, whole-body embodiments, and real-world hardware. We evaluated CHORD on large-scale, long-horizon, contact-rich tasks paired with human demonstrations, spanning rigid, articulated, and multi-object manipulation. At scale: * 82.12% average success across 1,831 tasks * 90.77% whole-body manipulation success * 4,739 sim-ready dexterous manipulation benchmark * Transfer to real dexterous hands Project page: nvidia-isaac.github.io/video_to_data/… Tech report: nvidia-isaac.github.io/video_to_data/… Code will be released soon as part of Video to Data repo github.com/nvidia-isaac/v…, our end-to-end pipeline for converting human demonstration videos into simulation-ready assets and physics-grounded robot training data. Huge thanks to amazing contributors: @zhu_xinghao , Zixi Liu, Shalin Jain, Chenran Li, Milad Noori, Huihua Zhao, John Welsh, @michaelv03, Wei Liu, @TingwuWang , Xingye (Dennis) Da, @zhengyiluo, Vishal Kulkarni, @sNaema, @yukez, @DrJimFan, @bowenwen_me, @danfei_xu, @SohaPouya, @Dr_YanChang. #Robotics #PhysicalAI #DexterousManipulation #RobotLearning #NVIDIA

Humanoids excel in free space but struggle with real-world contact. Meet SceneBot 🤖 the first unified RL framework for ALL free-space locomotion, terrain traversal and object interaction! By conditioning on per-link contact labels, it masters complex, interaction-rich tasks like carrying a box upstairs. Code & data open-sourcing soon! 📦 🪜 Paper: arxiv.org/abs/2606.27581 Website: ericcsr.github.io/scenebot/

WBC is cool but it doesn't have real world functionality. So researchers thought, what if we combine both of them? This is how SceneBot was born. A whole-body controller that can actually make contact and pull off real-world tasks. The long-horizon clip says it best, carrying a box WHILE climbing stairs in one shot. Here is the problem : General motion trackers walk, run, dance, kick beautifully with a single policy. But the moment a robot touches a box or a stair, pure kinematics breaks down. So what changes here: 🔵 They add per-link contact labels on top of the reference motion. 🔵 The policy gets told which hand grabs the box and which foot lands on the stair. 🔵 ONE single policy now covers both free-space motion and heavy contact work. A dataset with motion plus full scene interaction barely exists. So they reconstruct the scene backwards from plain human motion. Given a retargeted motion, they infer where contacts happened and rebuild a plausible box or stair around it. The numbers back it up hard. On object tasks SONIC sits near 5% success while SceneBot hits 95%. We highly recommend trying the simulation demo on their project page. They basically packed almost everything they built into a SINGLE demo.

This work won't be possible without amazing collaborators: @ShiboZhaoSLAM, @zhenkirito123, and @jiaman01. Advisors: @GuanyaShi and @ckarenliu, as well as support from the Amazon FAR team( @wangyenjen, @xupei0610)!

We build an interactive demo with SceneBot policy tracking motion from a motion matching engine. Now playable in desktop website: ericcsr.github.io/scenebot/demo

The robotics community has collected enormous robot datasets on grippers, but what happens when the hardware changes? How do we make robot data outlive the hardware it was collected on? We introduce Cloak, a training recipe for zero-shot cross-embodiment transfer. We never collect ANY embodiment-specific data. 🧵

🪜 What if humanoids could climb ladders and work on them straight out of simulation? Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation. Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇

As dexterous hand become more human like , using human data also become easier than ever, we explore how to cotrain with both human and robot data and generalize to new task with only human data.

Ji Woong Kim @jwbkim

4 weeks ago

We show that robots can learn high-level task semantics, such as sorting rules, skill composition, and rule-based ordering, directly from human demos. This is useful because if your target task is a composition of the robot's existing skills, you could just collect human demos

8 32 113 23K 70

Humanoid robotics is hitting a data wall. Teleop and mocap took us far, but they don’t scale to every object, terrain, and behavior. We’re releasing GRAIL: research.nvidia.com/labs/dair/grai… — a fully digital pipeline for generating loco-manipulation data before the robot moves. 🧵(1/8)

@zhenkirito123 Congrats!

What is missing to bring real-time motion research into AAA games and real-world robotics? We present MotionBricks, a step toward bridging this gap with two key components: - a single generative latent motion backbone covering 350,000+ motion skills, running at 15,000 FPS with 2 ms latency and substantially improved quality and reliability. - a unified smart primitive interface for locomotion, object / scene interaction, with fine-grained control over generated behaviors. Webpage: nvlabs.github.io/motionbricks/ Code: github.com/NVlabs/GR00T-W… Paper: arxiv.org/abs/2604.24833 (ACM TOG / SIGGRAPH 2026)

The Movement Lab has a new website ✨ A look at what we've been working on: humanoid robots, physics-based animation, and robot learning. tml.stanford.edu

Ego centric data for navigation is the way to go!

Weizhuo(Ken) Wang @KenWangWeizhuo

3 months ago

A person walks around campus for 5 hours with cameras. That's it. That's the training data. The result? A humanoid robot that traverses unseen buildings, crowds, and glass walls — zero robot data, zero finetuning. EgoNav is here. egonav.weizhuowang.com None of these behaviors

15 67 303 36K 205

Happy Monday! More exciting SONIC releasing incoming.