Excited to share what we've been working on! Huge kudos to Sirui for leading this! The single-policy approach for diverse scene interaction is just elegant. It's so rewarding to see it handle all those different motions with one unified solution. Honored to be a co-author on this journey. ❤️
Humanoids excel in free space but struggle with real-world contact. Meet SceneBot 🤖 the first unified RL framework for ALL free-space locomotion, terrain traversal and object interaction!
By conditioning on per-link contact labels, it masters complex, interaction-rich tasks like
Huge Congrats, Sirui! It’s been a pleasure collaborating with you. I really appreciate all the hard work you’ve put into making this project a reality. The single-policy paradigm for diverse scene interactions is incredibly exciting!I’m glad SuperOdometry could contribute to the system. Looking forward to seeing what the community builds on top of SceneBot!
SceneBot tracks both reference motions and body contact labels with global localization. Contact labels allow it to distinguish between placing two hands around an object. Global localization helps the robot to better align with the scene during training & inference time. This
How can robots learn dexterous manipulation from human demonstrations at scale?
Excited to share CHORD: Learning Dexterous Manipulation Using Contact Wrench Guidance From Human Demonstration.
CHORD learns from human demos by focusing not only on where contact happens, but how that contact moves the object through force and torque guidance.
This unified contact-wrench representation carries human manipulation skills across diverse behaviors, long-horizon tasks, whole-body embodiments, and real-world hardware.
We evaluated CHORD on large-scale, long-horizon, contact-rich tasks paired with human demonstrations, spanning rigid, articulated, and multi-object manipulation.
At scale:
* 82.12% average success across 1,831 tasks
* 90.77% whole-body manipulation success
* 4,739 sim-ready dexterous manipulation benchmark
* Transfer to real dexterous hands
Project page: nvidia-isaac.github.io/video_to_data/…
Tech report: nvidia-isaac.github.io/video_to_data/…
Code will be released soon as part of Video to Data repo github.com/nvidia-isaac/v…, our end-to-end pipeline for converting human demonstration videos into simulation-ready assets and physics-grounded robot training data.
Huge thanks to amazing contributors: @zhu_xinghao , Zixi Liu, Shalin Jain, Chenran Li, Milad Noori, Huihua Zhao, John Welsh, @michaelv03, Wei Liu, @TingwuWang , Xingye (Dennis) Da, @zhengyiluo, Vishal Kulkarni, @sNaema, @yukez, @DrJimFan, @bowenwen_me, @danfei_xu, @SohaPouya, @Dr_YanChang.
#Robotics#PhysicalAI#DexterousManipulation#RobotLearning#NVIDIA
Humanoids excel in free space but struggle with real-world contact. Meet SceneBot 🤖 the first unified RL framework for ALL free-space locomotion, terrain traversal and object interaction!
By conditioning on per-link contact labels, it masters complex, interaction-rich tasks like carrying a box upstairs. Code & data open-sourcing soon! 📦 🪜
Paper: arxiv.org/abs/2606.27581
Website: ericcsr.github.io/scenebot/
WBC is cool but it doesn't have real world functionality.
So researchers thought, what if we combine both of them?
This is how SceneBot was born.
A whole-body controller that can actually make contact and pull off real-world tasks.
The long-horizon clip says it best, carrying a box WHILE climbing stairs in one shot.
Here is the problem :
General motion trackers walk, run, dance, kick beautifully with a single policy.
But the moment a robot touches a box or a stair, pure kinematics breaks down.
So what changes here:
🔵 They add per-link contact labels on top of the reference motion.
🔵 The policy gets told which hand grabs the box and which foot lands on the stair.
🔵 ONE single policy now covers both free-space motion and heavy contact work.
A dataset with motion plus full scene interaction barely exists.
So they reconstruct the scene backwards from plain human motion.
Given a retargeted motion, they infer where contacts happened and rebuild a plausible box or stair around it.
The numbers back it up hard.
On object tasks SONIC sits near 5% success while SceneBot hits 95%.
We highly recommend trying the simulation demo on their project page.
They basically packed almost everything they built into a SINGLE demo.
We build an interactive demo with SceneBot policy tracking motion from a motion matching engine. Now playable in desktop website: ericcsr.github.io/scenebot/demo
The robotics community has collected enormous robot datasets on grippers, but what happens when the hardware changes? How do we make robot data outlive the hardware it was collected on?
We introduce Cloak, a training recipe for zero-shot cross-embodiment transfer. We never collect ANY embodiment-specific data. 🧵
🪜 What if humanoids could climb ladders and work on them straight out of simulation?
Meet LadderMan: a perceptive system for zero-shot sim-to-real ladder climbing and on-ladder manipulation.
Watch the humanoid climb, stabilize, and manipulate—all in one system. 🤖👇
As dexterous hand become more human like , using human data also become easier than ever, we explore how to cotrain with both human and robot data and generalize to new task with only human data.
We show that robots can learn high-level task semantics, such as sorting rules, skill composition, and rule-based ordering, directly from human demos.
This is useful because if your target task is a composition of the robot's existing skills, you could just collect human demos
Humanoid robotics is hitting a data wall. Teleop and mocap took us far, but they don’t scale to every object, terrain, and behavior.
We’re releasing GRAIL: research.nvidia.com/labs/dair/grai… — a fully digital pipeline for generating loco-manipulation data before the robot moves. 🧵(1/8)
What is missing to bring real-time motion research into AAA games and real-world robotics?
We present MotionBricks, a step toward bridging this gap with two key components:
- a single generative latent motion backbone covering 350,000+ motion skills, running at 15,000 FPS with 2 ms latency and substantially improved quality and reliability.
- a unified smart primitive interface for locomotion, object / scene interaction, with fine-grained control over generated behaviors.
Webpage: nvlabs.github.io/motionbricks/
Code: github.com/NVlabs/GR00T-W…
Paper: arxiv.org/abs/2604.24833 (ACM TOG / SIGGRAPH 2026)
The Movement Lab has a new website ✨
A look at what we've been working on: humanoid robots, physics-based animation, and robot learning.
tml.stanford.edu
A person walks around campus for 5 hours with cameras.
That's it. That's the training data.
The result? A humanoid robot that traverses unseen buildings, crowds, and glass walls — zero robot data, zero finetuning.
EgoNav is here.
egonav.weizhuowang.com
None of these behaviors
2K Followers 3K FollowingContributing & exploring @arc ecosystem
Contribute @axisrobotics
Contribute @PrismaXai
Bringing the best things to @ConcreteXYZ
771 Followers 383 FollowingResearch Scientist at Meta MSL and previously FAIR
Ex: PhD @CMU_Robotics, BS @sjtu1896
Work on computer vision and robotics.
5K Followers 7K FollowingFounder & CEO Inference Analytics, building AI for sensitive data industries, AI entrepreneur. Alum: IBM (Big Data), Informatica, MicroStrategy, McKinsey.
52 Followers 686 FollowingRobotics PhD @umicharmlab @UMRobotics | generative planning + control for manipulation with robot hands.
I tell robots what to do. Sometimes they listen.
771 Followers 383 FollowingResearch Scientist at Meta MSL and previously FAIR
Ex: PhD @CMU_Robotics, BS @sjtu1896
Work on computer vision and robotics.
1K Followers 699 FollowingArtisanal Neuromancer @ UMA Robotics; x-Thinking & RL Gemini, x-Brain, x-DeepMind. Poking at neural networks to make them do something.
6K Followers 3 Following@chris_j_paxton, @micoolcho & @DJiafei geeking out weekly with authors of robotics AI papers. On YouTube / X / Spotify / Substack
1K Followers 728 FollowingSr. Research Scientist @NVIDIA | PhD from SJTU @sjtu1896 | Interested in 3D Computer Vision, Human Digitization | Views are my own
474 Followers 308 FollowingUndergraduate Student at PKU@PKU1898;
Intern at ShanghaiAILab@InternRobotics;
Humanoid Control/Perception/Interaction/Foundation Model
1K Followers 590 FollowingResearch scientist in robotics @ GEAR Nvidia. I obtained my PhD from University of Toronto @UofT, Vector Institute @VectorInst 😃
7K Followers 6K FollowingProduct Lead | Gemini Applied Research
Prev: Launched @aws Trainium, @Columbia Neuro
Tweets are my own. Retweets are not endorsements. Build anything.
3K Followers 78 FollowingThis is Wuji Technology official account.
Our is mission is to empower robot with human dexterity.
Founder & CEO: @Yunzhe_Pan
Email: [email protected]