🎉Excited to share that 𝗣𝗮𝗻𝗼𝗚𝗲𝗻 has been accepted to #NeurIPS2023! We create diverse panorama environments for VLN via recursive image outpainting, improving SotA agents on R2R, R4R, CVDN unseen envs. Looking fwd to meeting you all in New Orleans!
cc @mohitban47 @uncnlp
🎉Excited to share that 𝗣𝗮𝗻𝗼𝗚𝗲𝗻 has been accepted to #NeurIPS2023! We create diverse panorama environments for VLN via recursive image outpainting, improving SotA agents on R2R, R4R, CVDN unseen envs. Looking fwd to meeting you all in New Orleans!
cc @mohitban47 @uncnlp
🚨Excited to share: “𝗣𝗮𝗻𝗼𝗚𝗲𝗻: Text-Conditioned Panoramic Environment Generation for Vision-and-Language Navigation”!🚨
PanoGen creates diverse 360-degree panorama via recursive outpainting, achieving SotA on R2R, R4R, CVDN.
pano-gen.github.io@mohitban47 @uncnlp
🧵
Excited to share our new #ICCV2023 paper: “Scaling Data Generation in Vision-and-Language Navigation”!
We reduce the long-lasting generalization gap between navigating in seen & unseen scenes to < 1%, & approaching human performance for 1st time! 🥳
arxiv.org/abs/2307.15644
🧵
Robot visual navigation in unseen homes is hard: end-to-end RL works well in sim but gets only 23% real-world success.
Today, in the first real-world empirical study of visual navigation, we show Modular Learning achieves 90% success in unseen homes!
theophilegervet.github.io/projects/real-…
1/N
When and how can models extrapolate to unseen domains?
Previous understanding is mostly limited to linear models or well-covered new domains. We make some first baby steps beyond these by considering *structured* domain shift and nonlinear models. arxiv.org/abs/2211.11719 1/n
Our project SayCan (say-can.github.io) wins Best Paper Award at the RSS Workshop on Scaling Robot Learning! Thanks to all who came to the poster session with great questions and feedback 😁
The future of "LLMs for Robotics and Robotics for LLMs" is brighter than ever!
Language-guided navigators shouldn’t be carted off to a new environment with each new instruction. Instead, they should persist & improve over time.
We present Iterative Vision-and-Language Navigation to evaluate exactly this!
arxiv.org/abs/2210.03087jacobkrantz.github.io/ivln
Do you want to learn to train and evaluate embodied AI solutions for 1000 household tasks in a realistic simulator? Join our BEHAVIOR Tutorial at #ECCV2022: Benchmarking Embodied AI Solutions in Natural Tasks!
Time: Monday, Oct 24th 14:00 local time (4:00 Pacific Time)
We are announcing Habitat-Matterport 3D Semantics Dataset!
216 real-world 3D spaces with dense semantic annotations available to download now:
aihabitat.org/datasets/hm3d-…
(1/3) Today we’re releasing the Habitat-Matterport 3D Semantics dataset, the largest public dataset of real-world 3D spaces with dense semantic annotations.
HM3D-Sem is free and available to use with FAIR's Habitat simulator: bit.ly/3EymX8x
2K Followers 2K FollowingResearch Scientist @ToyotaResearch | PhD in AI and DL @GeorgiaTech | Researching Large Behavioral Models | 3D Vision | Robotics
139 Followers 613 FollowingPh.D. @TheAIML, intern @Alibaba_Qwen, former intern @AdobeResearch, working on long horizon agent memory, Embodied Navigation, AIGC for world modeling.
78K Followers 943 Followingpretending to be semi-retired but secretly grinding prop trading firm founder. play a zero-sum game with positive-sum systems.
5K Followers 2K FollowingCo-Founder and Chief Strategy Officer at @WaldenRobotics . Adjunct Prof of CS at @Stanford, ex partner at @CalibrateVC & head of ML at @ToyotaResearch
2K Followers 2K FollowingResearch Scientist @ToyotaResearch | PhD in AI and DL @GeorgiaTech | Researching Large Behavioral Models | 3D Vision | Robotics
139 Followers 613 FollowingPh.D. @TheAIML, intern @Alibaba_Qwen, former intern @AdobeResearch, working on long horizon agent memory, Embodied Navigation, AIGC for world modeling.
286K Followers 190 FollowingCo-founder of Thinking Machines Lab @thinkymachines; Ex-VP, AI Safety & robotics, applied research @OpenAI; Author of Lil'Log
560K Followers 3K FollowingNVIDIA Director of Robotics & Distinguished Scientist. Co-Lead of GEAR lab. Solving Physical AGI, one motor at a time. Stanford Ph.D. OpenAI's 1st intern.