@chris_j_paxton, @micoolcho & @DJiafei geeking out weekly with authors of robotics AI papers. On YouTube / X / Spotify / Substackyoutube.com/@RoboPapersJoined February 2025
There are few truly open models in the world, including both weights and data. However, these models are crucial for research and development of new systems — they help us learn which data is important and help develop new capabilities for deploying robots in the real world.
MolmoAct2 provides a foundation for open research into robotics. It is associated with its own open dataset, an open-data action tokenizer, and a reasoning variant which predicts depth tokens. And people have actually been using it across the community, running experiments in their own labs or homes.
@hq_fang and @DJiafei tell us more. Watch Episode 87 of RoboPapers, with @micoolcho and @chris_j_paxton, now!
Robot policies must be both reliable and highly capable to be useful; the best way to achieve this level of performance is with reinforcement learning. However, for reinforcement learning you are usually stuck between two difficult options: reinforcement in the real world is often risky and expensive, while reinforcement learning in a traditional simulator takes a lot of engineering work and has a persistent sim-to-real gap. What if instead you could train your robot purely in a world model?
RISE by @jiazhi_yang2024 et al. uses a compositional world model to predict the future and evaluate progress. This allows for a self-improving pipeline, which learns a world model from real data and then learns how the robot should perform different tasks. This pipeline results in a data-driven way to improve policy performance from real data but without real-world reinforcement learning.
Watch Episode #86 of RoboPapers, with @chris_j_paxton and @DJiafei, to learn more!
We’re excited to partner with BitRobot Network, Lightwheel AI, Singapore Institute of Technology and contributors like Jie Tan (Deepmind), Steve Xie (Lightwheel), Michael Cho (FrodoBots), etc.
Looking forward to push the boundary of humanoid loco-manipulation in this Humanoid
Collecting robot data at scale is key to deploying working manipulation policies, and the team from Tutor Intelligence @tutorintel is here to tell us about how to accomplish it. Their new announcement: a massive, 100-robot “data factory,” with a behind-the-scenes look at how to build a teleoperation platform and how to make robots and policies that are useful for their customers.
Tutor Intelligence is a full-stack robotics company: they build robot arms, they sell robot arms, they write the software and they train neural networks. @joshgruenstein, @JesseMMichel, @shirazkn, and
Joe McCalmon, and Joe McCalmon join us to tell us more about how they scale both teleop data and human interventions from their teleoperators in order to train the policies they need.
Watch Episode #85 of RoboPapers, with @chris_j_paxton and @DJiafei, to learn more!
Learning robust, general-purpose reward functions for robotics unlocks many potential applications, like on-robot reinforcement learning or dataset validation. However, there’s a question of how to actually train such reward functions. Training success/failure prediction leads to ambiguous signals partway through a demonstration — it’s hard to measure progress — making the method unsuitable for reinforcement learning, among other things. Predicting progress, on the other hand, does not give a good way of using failure data.
So why not do both? Robometer combines both progress and preference supervision, resulting in a stable, scalable, and highly general reward learning approach. @aliangdw@yigitkkorkmaz
and @Jesse_Y_Zhang join us to tell us more.
Watch Episode #84 of RoboPapers, with Chris Paxton and Jiafei Duan today!
60 Followers 87 FollowingRobot news, automation trends, and practical robotics insights, without the hype. Humans first, robots close behind. Tracking technology that serves people.
957 Followers 302 FollowingComputer vision, SSL, Robotics PhD student at AMI Labs and INRIA Paris (Willow team) - Supervision : Yann LeCun and Jean Ponce. Ex Meta FAIR
6K Followers 1K FollowingAssistant Professor at @NUScomputing| Robotics & AI PhD @uwcse| Host of @RoboPapers| Ex-@allen_ai, @NVIDIA
my opinion is my alone.