Bryan Chan @chanpyb
PhD student @rlai_lab. Prev: @GoogleDeepMind, @OcadoTechnology, @kindredai, @UofTCompSci chanb.github.io Edmonton Joined October 2020-
Tweets52
-
Followers196
-
Following539
-
Likes4K
@how_uhh Thanks for pointing out this paper! Yes this work is very similar in that they're using time-hierarchy theorem to motivate using recursive models that adds more computation to the model.
Why do we assume an RL agent can always compute the correct action immediately? Every policy is a program and thus resource bounded. In this blog, I argue why computation should be part of decision making, illustrated with some toy examples. 🔗chanb.github.io/blogs/necessit…
@danielwurgaft This loss/complexity tradeoff has started bothering me---I think we can be okay with (slightly) worse loss at the cost of better generalization, e.g. physics models will fail to predict noise but memorization can. Any thoughts about this, e.g. regularization, architecture, etc.?
@puneeshdeora @bhavya_vasudeva Great work! Our work arxiv.org/abs/2410.23042 shows that asymptotically ICL will choose the best prediction mode, but we didn't address anything about which is preferred when they're equally good---do you think it has to do with k=1 converging faster than k=3 in the MC case?
@Stone_Tao You can also look at the streaming setting: arxiv.org/pdf/2411.15370 and arxiv.org/pdf/2410.14606?
@Stone_Tao There are few but this is what I have on my mind: arxiv.org/pdf/1712.01275 I think with smaller buffer size the data gets closer to on policy data, and larger -> more off policy
I will be presenting this paper on how models trade-off in-context and in-weight learning at #ICLR2025 Drop by on Saturday and I’ll be happy to chat!
Excited to share that our work on understanding when ICL emerges has been accepted to #ICLR2025 ! Submission for preview: openreview.net/forum?id=aKJr5…
Meet the recipients of the 2024 ACM A.M. Turing Award, Andrew G. Barto and Richard S. Sutton! They are recognized for developing the conceptual and algorithmic foundations of reinforcement learning. Please join us in congratulating the two recipients! bit.ly/4hpdsbD
Excited to share that our work on understanding when ICL emerges has been accepted to #ICLR2025 ! Submission for preview: openreview.net/forum?id=aKJr5…
LLMs can leverage context information, i.e., in-context learning (ICL) or memorize solutions, i.e., in-weight learning (IWL) for prediction, but when do they happen? 1/N
Thanks @m_wulfmeier ! We were surprised to see that SAC-X is just very robust. Something that was interesting to us that we didn’t further investigate: Learning from examples ended up being more efficient than using reward. Let’s chat at #NeurIPS2024 if there’s a chance?
Here's a fascinating paper by @domo_mr_roboto's group linking hierarchical reinforcement learning and cheaply-obtainable auxiliary tasks arxiv.org/abs/2407.03311 Better exploration with minimal engineering effort remains a critical challenge (even for RLHF/AIF) - reminiscent of
@anianruoss One immediate observation I have is that there seems to be no boundary between two demonstration sequence (list. 1). Would it not be problematic because the model can’t tell they are different demonstrations without further training?
@daibond_alpha @iclr_conf 3. Both scores 3 and 5 are somewhat due to experimental results like significance and benchmarks. (3) is interesting because former provides no empirical insight, while latter provides some, arguably claiming "maybe" the theory applies. Which one is more important/contribution?
@daibond_alpha @iclr_conf Some interesting observations here: 1. It seems like the latter has "shorter reviews" and imo generally of lower quality than those of the former 2. The expectations seem to be different, maybe due to different primary areas? 3. ...
@m_wulfmeier @NeurIPSConf I’ll be at NeurIPS too! Let’s chat about robot learning!
Would you believe that deep RL can work without replay buffers, target networks, or batch updates? Our recent work gets deep RL agents to learn from a continuous stream of data one sample at a time without storing any sample. Joint work with @Gautham529 and @rupammahmood.
Our NeurIPS paper is now on arXiv: We introduce Action Value Gradient (AVG), a novel incremental deep RL method that learns in real-time, one sample at a time — no batch updates, target networks or a replay buffer! Co-authors @mhmd_elsaye @bellingerc @white_martha @rupammahmood
@c_voelcker @usmananwar391 What alternative are you using? I think I can see some limitations with the IQM approach but unsure what you think to address it
@c_voelcker @apsarathchandar What gets me is that they claim that AI model evals don’t include any uncertainty/statistics…
Hey all! We are thrilled to have @chanpyb from @UAlberta for this week's seminar! The talk is titled: "Why can't we use reinforcement learning for image-based robotic manipulation?". See you at 11:30AM ET! youtube.com/@MontrealRobot… #rl #manipulation, #imitationLearning
@c_voelcker Haha yes, mine is also a rant about the same thing.
Arbaaz Qureshi @arbaaz__qureshi
103 Followers 3K Following Applied Scientist @Amazon. Prev @MSFTResearch, @Google, @IBM & @Lowes. CS @UMassAmherst & @IITPAT. Interested in reinforcement learning theory and applications.
Earther @EartherAI
373 Followers 4K Following Researching Learning Systems | Deep Learning × Vision Mamba x Fusion Models
cammelia velvet @badnfluenz10238
26 Followers 3K Following fairy lights & racing thoughts ✨ 100% follow back
Aneesh Muppidi @aneeshers
777 Followers 752 Following Incoming PhD at @StanfordAILab. Rhodes Scholar @bold_lab_ai. prev @harvard
Kate Nave @KathrynNave
2K Followers 3K Following ° @LeverhulmeTrust fellow @EdinburghUni ° CogSci, Philosophy & Theoretical Biology ° Former sci/tech journalist ° Book: A Drive to Survive https://t.co/iF5YcWLVVj
Hoyeon Chang @hoyeon_chang
1K Followers 2K Following PhD student at @kaist_ai; I study compositionality and systematicity in NNs; Also a jazz pianist
Diogo Rezende @rezendediiogo
3 Followers 211 Following
Vertical Data @Vertical__Data
260 Followers 4K Following Turnkey AI infrastructure solutions for your compute needs. Hardware, GPU financing, colocation, and managed services. 📞+1 (702) 936-3715
GPU financing @GPUfinancing
231 Followers 4K Following Scale your AI infrastructure with GPU financing. Tailored, affordable solutions. 🚀
Weijia Fan @HoursWaking
10 Followers 136 Following I'm an MSc in Computer Science @ SZU and a Visiting Student @ KIT. I will be a PhD student at UAlberta and study Multimodal Learning and Computer Vision.
Charlotte @KYq0iAA2P45P3Jc
186 Followers 6K Following
Kris D. @M33pinator
495 Followers 185 Following Research Fellow @ https://t.co/7J7uozCcmA; @UAlbertaCS alum interested in AI, robotics, and penguins—dabbles w/ game dev, pixel art, speedrunning, and speedcubing.
Siddarth @siddarthv66
2K Followers 847 Following Post-training @MistralAI, PhD @Mila_Quebec | RLing LLMs
Bhavya Agrawalla @AgrawallaBhavya
192 Followers 421 Following Research Interests - Deep Reinforcement Learning. PhD student @CMU CS. Prev - Math and CS undergrad at MIT (2021-24), Silver @ IMO 2019
Sheila Schoepp @sheilaschoepp
29 Followers 282 Following
hr0nix @hr0nix
1K Followers 937 Following 🦾 Head of AI @TheHumanoidAI 💻 Ex @nebiusai, @Yandex, @MSFTResearch, @CHARMTherapeutx 🧠 Interested in (M|D|R)L, AGI, rev. Bayes 🤤 Opinions stupid but my own
Daniel Lawson @danielblawson9
188 Followers 549 Following Student, at Mila. Prev Purdue CS. Interested in ML, RL, and robotics. Veridis quo?
Farzane Aminmansour @FAminmansour
322 Followers 412 Following PhD student @UAlberta RIs: Reinforcement Learning, Computational Neuroscience, Machine Learning, Deep Learning
Jin Xie @jin_xie52910
2 Followers 511 Following
Daniel Wurgaft @danielwurgaft
440 Followers 302 Following PhD @Stanford working w @noahdgoodman & research fellow @GoodfireAI | Studying in-context learning and reasoning in humans and machines | Prev. @UofT CS & Psych
Puneesh Deora @puneeshdeora
160 Followers 395 Following PhD student at UBC. Working on theory of Deep Learning.
Alex Wang @AW125080
2 Followers 64 Following
Lucas Alegre @lnalegre
180 Followers 455 Following Professor at @INF_UFRGS. Interested in multi-task and multi-objective reinforcement learning.
Akib600 @Akib6001
24 Followers 1K Following
Jiaheng Hu @JiahengHu1
759 Followers 478 Following Ph.D. candidate @UTCompSci & research intern @Amazon FAR | Ex: @GoogleDeepMind Robotics, @allen_ai PRIOR
Daniel Plop @danrl0
53 Followers 401 Following
Denis Tarasov @ML_is_overhyped
223 Followers 190 Following PhD at ETH Zurich. Ex Meta, InstaDeep, JetBrains, Yandex, Tinkoff.
Harish Tayyar Madabus... @harish
2K Followers 2K Following Senior Lecturer (Associate Professor) in Artificial Intelligence. Work on NLP (#NLProc) and Large Language Models
Subhojeet Pramanik @subho_in
452 Followers 342 Following Researcher at @softmaxresearch working on scaling multi-agent reinforcement learning and organic alignment. MSc from @rlai_lab
Harish Ravichandar @h_ravichandar
891 Followers 1K Following Structural priors for frugal and self-sufficient robot learning. Robotics & AI Faculty @GeorgiaTech @ICatGT @GTrobotics @mlatgt
Tianle Cai @tianle_cai
9K Followers 5K Following Life-long learner, hacker, and thinker. Prev: PhD @Princeton, researcher @togethercompute @GoogleDeepMind @MSFTResearch @citsecurities.
Wang Ma @WangMa70190365
648 Followers 6K Following PhD Student @rpi | interning @Microsoft | previous @IBM | Bayesian Deep Learning | Uncertainty Quantification | UG @SUSTechSZ | Speedcuber
Dearfaw @dearfaw12111
35 Followers 2K Following There’s only one corner of the universe you can be sure of improving, and that’s your own self.
hualianmaozi @hualianmaozi
3 Followers 155 Following
Clémentine Dominé, ... @ClementineDomi6
3K Followers 2K Following PhD @GatsbyUCL MA Theoretical Physics @OfficialUoM @UCLA Prev. @ZuckermanBrain @SapienzaRoma @CERN @EPFL Creator @ArtlovesSci Organiser of @unireps
Raymond Chua @RaymondRChua
1K Followers 4K Following PhD @mcgillu and @Mila_Quebec. Into #AI 🤖 & #neuroscience 🧠. 🏊🏻🚴🏽♂️🏃🏻♂️🏕️ when away from 💻 .
Masoud Jafaripour @mjafaripoor110
375 Followers 2K Following Research Scientist Intern @EA, Researcher (CS @UAlbertaCS), Community @Cohere_Labs, focus on #LLMs #VLMs #NLProc #Reasoning, previously @SharifUni @UnivOfTehran
Sorsleau @SorsleaurEk
80 Followers 3K Following
Charan Sai Vaddi @CharanSaiVaddi
6 Followers 522 Following
Thoasmare @ThoasmareNMLxs
43 Followers 4K Following
Huy Nguyen @huynt07
33 Followers 895 Following Interested in building online agents that adapt, evolve, and discover in open-ended worlds.
Oak Lab @oaklab_ai
7K Followers 0 Following We are building learning algorithms that allow agents to achieve goals in big worlds.
Tim Roughgarden @Tim_Roughgarden
72K Followers 157 Following Prof @the_IAS. On leave @Columbia. Head of Research @a16zcrypto. Wrote Algorithms Illuminated, 20 Lectures on Algorithmic Game Theory, etc.
Ahmed Touati @ahtouati
752 Followers 1K Following Research Scientist, FAIR @MetaAI | PhD @Mila_Quebec
British Open-ended Le... @bold_lab_ai
963 Followers 31 Following
Ryan Williams @rrwill... @rrwilliams
8K Followers 477 Following Professor @MITEECS and @MIT_CSAIL. Complexity, algorithms, and related math. I'll let you know when P != NP (and when it's not)
Ben Recht @beenwrekt
35K Followers 324 Following optimization. machine learning. uc berkeley. I blog at https://t.co/fkJujOPsJb The world won't end.
Aneesh Muppidi @aneeshers
777 Followers 752 Following Incoming PhD at @StanfordAILab. Rhodes Scholar @bold_lab_ai. prev @harvard
Tiberiu Mușat @Tiberiu_Musat_
816 Followers 956 Following Trying to figure out how AI works 🔍🧠 Currently at @ETH Zurich, previously @EPFL 🇨🇭 LLMs, interpretability, emergence, grokking 🤖
Yash Sarrof @yashYRS
222 Followers 169 Following PhD student, Saarland University, advised by Michael Hahn, lab website : https://t.co/iSGrDEoxG6
Shenao Zhang @ShenaoZhang
895 Followers 504 Following SWE agents @reflection_ai, PhD student @NorthwesternU. RL for emergent intelligence.
Wojciech Czarnecki @wojczarnecki
3K Followers 304 Following 👨🏻🔬 AI Scientist (@IneffableLabs @ltmpredict @GoogleDeepMind @IsomorphicLabs @Samsung) 🧠 Building AIs (#gemini #alphastar #quake3) 🎮 Gamer 🎓 Math lover
Ineffable Intelligenc... @IneffableLabs
8K Followers 0 Following Making first contact with superintelligence.
Finding The Frame Wor... @RLFrameWorkshop
258 Followers 18 Following An @RL_conference workshop for examining conceptual frameworks in reinforcement learning.
Noah Golowich @GolowichNoah
1K Followers 109 Following Assistant Professor of CS at UT Austin. All posts in my individual capacity.
Hadi Vafaii @hadivafaii
1K Followers 174 Following Postdoc @UCBerkeley @Redwood_Neuro | 🧠+🤖 | 🎹 | 🎾 |
Will Dabney @wwdabney
1K Followers 79 Following Research scientist at DeepMind. On the critical path to AGI. Also, a persistent optimist.
Seijin Kobayashi @SeijinKobayashi
862 Followers 196 Following Research Scientist at Google | ETH Zurich | École Polytechnique
Oliver Groth @omgroth
2K Followers 542 Following Research Scientist @GoogleDeepMind. PhD from @oxfordrobots and @Oxford_VGG. AI Researcher, Robot Trainer, Video Game Nerd. All views are my own.
Kris D. @M33pinator
495 Followers 185 Following Research Fellow @ https://t.co/7J7uozCcmA; @UAlbertaCS alum interested in AI, robotics, and penguins—dabbles w/ game dev, pixel art, speedrunning, and speedcubing.
Aryeh Kontorovich @aryehazan
12K Followers 682 Following probability, statistics, metric spaces, Markov chains, freedom (social & academic), Israel, Jew stuff. opinions represent my employer & all other groups I'm in
Siddarth @siddarthv66
2K Followers 847 Following Post-training @MistralAI, PhD @Mila_Quebec | RLing LLMs
Bhavya Agrawalla @AgrawallaBhavya
192 Followers 421 Following Research Interests - Deep Reinforcement Learning. PhD student @CMU CS. Prev - Math and CS undergrad at MIT (2021-24), Silver @ IMO 2019
Humanoid @TheHumanoidAI
5K Followers 110 Following First AI and robotics company in the UK, creating the world’s leading, commercially scalable, and safe humanoid robots.
hr0nix @hr0nix
1K Followers 937 Following 🦾 Head of AI @TheHumanoidAI 💻 Ex @nebiusai, @Yandex, @MSFTResearch, @CHARMTherapeutx 🧠 Interested in (M|D|R)L, AGI, rev. Bayes 🤤 Opinions stupid but my own
RL & Agents Reading G... @rl_agents_rg
87 Followers 15 Following RL & Agents Reading Group @ University of Edinburgh We discuss recent papers in RL, MARL & related Sign up & join: https://t.co/i14FJcTX4D
Masoud Jafaripour @mjafaripoor110
375 Followers 2K Following Research Scientist Intern @EA, Researcher (CS @UAlbertaCS), Community @Cohere_Labs, focus on #LLMs #VLMs #NLProc #Reasoning, previously @SharifUni @UnivOfTehran
Rocky Duan @rocky_duan
3K Followers 97 Following Research Lead @ Amazon FAR (Frontier AI & Robotics). Previously @CovariantAI CTO, @OpenAI, @UCBerkeley PhD. 2024 Forbes 30 Under 30.
Andrew Saxe @SaxeLab
6K Followers 379 Following Prof at @GatsbyUCL and @SWC_Neuro, trying to figure out how we learn. Bluesky: @SaxeLab Mastodon: @[email protected]
Neuracore @Neuracore_AI
2K Followers 403 Following Robot Learning Infrastructure to Scale and Deploy Faster | Record asynchronous data, train VLA models, and ship solutions faster than competitors.
Jonathan Lee @jon_lee0
925 Followers 144 Following research @GoogleDeepMind. co-developed gemini deep think. co-led model training for IMO 🥇 | prev: RL PhD at @StanfordAILab
Johannes Oswald @oswaldjoh
2K Followers 708 Following Research Scientist, Paradigms of Intelligence Team, Google Zurich
Kimon Fountoulakis @kfountou
4K Followers 123 Following Associate Professor at CS UWaterloo Machine Learning and AI Lab: https://t.co/Yrb5b8lwrT
Russ Tedrake @RussTedrake
5K Followers 98 Following Co-Founder and CEO at Walden Robotics. Professor at MIT.
Arash Ahmadian @aahmadian_
2K Followers 739 Following Staff Research Scientist @GoogleDeepmind, RL & post-training, Gemini 3 onward. prev: @Cohere @CohereForAI
Jascha Sohl-Dickstein @jaschasd
30K Followers 822 Following Member of the technical staff @ Anthropic. Most (in)famous for inventing diffusion models. AI + physics + neuroscience + dynamics.
James @jmac_ai
776 Followers 580 Following Ask me about #ReinforcementLearning #AI research @SonyAI_global RL for games, robotics, and other real-world applications Views and tweets are my own.
Puneesh Deora @puneeshdeora
160 Followers 395 Following PhD student at UBC. Working on theory of Deep Learning.
Daniel Wurgaft @danielwurgaft
440 Followers 302 Following PhD @Stanford working w @noahdgoodman & research fellow @GoodfireAI | Studying in-context learning and reasoning in humans and machines | Prev. @UofT CS & Psych
Martin Hebart @martin_hebart
6K Followers 789 Following Proud dad, Prof. of Computational Cognitive Neuroscience, author of The Decoding Toolbox, founder of https://t.co/hWZCF7XuMU @ martinhebart.bsky. social
Laude Institute @LaudeInstitute
4K Followers 408 Following Laude Institute backs computer science researchers turning research into real-world impact. // @LaudeVenturesYou might like
