Daniel Plop @danrl0
Germany Joined November 2019-
Tweets56
-
Followers53
-
Following401
-
Likes364
it's gonna be a XOR moment, that will prove that today's neural nets can't learn continually in real-time from own experience alone
By failing to engage with SOTA RL-for-LLMs, the alt neolabs like Oak are putting themselves in a tough position; RL is being deployed massively at scale now, and by not understanding the issues with contemporary RL, they're slowing themselves down.
Agree speed matters, push HW & algos as hard as you can. But spend that speed _inside_ the constraint, not around it. Cloned envs & replay buffers buy tput by decoupling learning from the world's clock. Real bar: 1 agent on the live stream, learning faster than the world changes.
A little perspective: RL as a field spent 10 years making algorithms slower and slower. If you look at the original ALE, it actually can sim a few thousand frames per second per core. If you look at some of the last big env releases before a ton of people moved over to LLMs,
Concise and profound. The RL textbook is a good example.
I strive to make my writing unsummarizable, in the sense that it has so little fluff left in it that if you take any words out, as summaries by definition do, you lose a lot of interesting ideas.
"...A lab that’s focused on alternatives to transformers, where the distinction between training and inference doesn’t really exist,..." I haven't often seen that perspective out there :)
My MLSys keynote on AI writing systems code got more interest than I expected. The recording will take a while, so in the finest tradition of AI labs sharing blog posts, we’re starting the Core Automation Blog with this one coreauto.com/blog/when-ai-s…
@sageaurelius @deepseek_ai They will eventually launch a newer model and probably decommission this one?
@dileeplearning @carlkolon The most efficient data use is the data the agent generates for itself at the appropriate time as it interacts with the world.
The goal is to focus *only* on scaling with more compute once the algorithm has been found E.g. a Transformers-based algorithm doesn't meet this requirement bc it also requires human-curated data to scale, which doesn't scale as quickly as compute alone (Moore's Generalized Law).
The bitter lesson in 26 words: Don’t be distracted by human knowledge, as AI has been historically. Instead focus on methods for creating knowledge that scale with computation, like search and learning.
The strategy of explaining what humans can learn but machines can't yet has the risk of leading us astray in our pursuit of finding computational mechanisms for developing general intelligence. This is because we tend to map them onto one another.
@RichardSSutton I think it is worth studying human knowledge, particularly understanding the structure of its abstractions, as they can provide guidance about the kinds of things humans learn and machines do not (yet). I wouldn't call that "distraction".
That an algorithm trained on no human data whatsoever could reach the same goals that LLMs achieve by learning from it feels nearly impossible to imagine, and yet, for that to happen, the algorithm would need to learn *profoundly* & continually at every single timestep.
i might be very wrong here, but i don't think "no human data, no pre-training" is the right approach to get frontier models or scientific breakthroughs any time soon
@Shahules786 @adityak6798 @DavidAVenuto @GottliebEli @jackminong @varibandi @xdotli I think everyone agrees on 1). but I would like to understand better is 2) : why can simulating RL environments be more efficient than learning from grounded (noisy) human feedback?
@ID_AA_Carmack An age of (real-time) constraints together with fully embracing the Bitter lesson is a good invested time in developing truly general learning methods in these times of abundant economic pressure for creating useful AI.
@RichardSSutton Tell me you're talking about RL without talking about RL. I wonder whether he ever read the RL textbook carefully and really took the ideas seriously in there (e. g., general value functions!).
@mntruell The entire Chromium code is open source and likely in the training data of these foundation model.
@ahmetb It's simple. They have been trained on this kind of data, and learned to predict the most probable outcome from that distribution.
@yoavgo What should be changed to make this agent-env loop more general than it is already?
Great read! The sim2real problem is meaningless cuz, by design, it approximates the real problem of AI, and it can never take the agent's true point of view.
Reinforcement learning 🧠 on robots 🤖 can’t stay in simulation forever. My new post explores why direct, on-hardware learning matters and how we also need smarter mechanical design to enable it. kris.pengy.ca/designforlearn…
@KhurramJaved_96 Indeed, except for the sporadic recommendations of high-quality content by creators (sometimes) with as few as 100 subscribers, it's often better to discover content yourself through search.
@eruditenights @nntaleb Assuming one box contains >50-100 books, he has read over 7.5-15k books (and some of them 10+ times). Before I die, I want to have read so much and absorbed so much knowledge.
@KhurramJaved_96 @RichardSSutton This is just one of many gems from the book: "...when we shift our focus to the surface, using only the highest-level, most general problem statement, the internals must figure themselves out. This is how complexity works."
@KhurramJaved_96 There's a great book that covers this topic and more that I would recommend to you. I think @RichardSSutton would like it too. It's called "Discovered, Not Designed: Building Things in the Age of Complexity" by Sean McClure.
Fatima Davelouis @fatimadg22
58 Followers 215 Following Reinforcement Learning researcher at @rlai_lab, grad student at @UAlberta and @AmiiThinks. Math enthusiast, yoga fanatic, soccer player, salsa dancer.
Yoram Bachrach @yorambac
4K Followers 7K Following Research Scientist at Meta (prev Google DeepMind and Microsoft Research). Working on LLM Agents and Multi-Agent Systems.
Erfan Miahi @erfan_mhi
17K Followers 9K Following Collab with people from @googledeepmind & @rlai_lab (Rich Sutton's); Ex-founder; Post-training @pluralis! Reading #philosophy & doing extreme sports!
Marco Bagatella @mar_baga
274 Followers 177 Following ETH/CLS PhD candidate, interested in reinforcement learning and pizza making.
UriG @uri_gadot
68 Followers 288 Following Building real-time video models @decartAI | Ph.D. @TechnionLive | anything video models; Reinforcement Learning
Chen Tessler @ChenTessler
3K Followers 664 Following Research Scientist @NVIDIAAI ; Training humanoids 🤖 ; Reinforcement learning PhD @TechnionLive 🇮🇱. Views are my own.
Jihed Ncib @JihedNcib
1K Followers 3K Following Machine Learning Researcher (Post-doctoral) at the Connected Politics Lab and the School of Computer Science (University College Dublin).
CIMCYC @cimcyc
3K Followers 2K Following The Mind, Brain and Behavior Research Center at @CanalUGR, an international hub for psychological and neuroscience research. Maria de Maeztu Unit of Excellence
Kai Arulkumaran @kaixhin
8K Followers 6K Following Researcher, programmer, DJ, transhumanist. Now at @SakanaAILabs 👾🧠🦾🐟; previously Imperial College, DM, FAIR, Twitter Cortex, MSR, NNAISENSE, Araya
Kristina Stephan @KS_TechTalent
439 Followers 1K Following Tech Recruitment🕵🏻♀️| Talent Nerd | Aspiring DJ 🎧| 🇨🇦 FinTech
rebelmachina @rebel_machina
35 Followers 161 Following AI experimenter 🧪 building & bantering on 🐦 creator of IG: artv0yage 🪄
Aram Hăvărneanu @aramh
5K Followers 2K Following Mathematical engineer bringing type safety to the cloud. Previously worked on CUE @cue_lang. JSR PC, @(R6)+; I wrote the arm64, sparc64, and Solaris Go ports.
Gheorghe Lisca @gheorghelisca
38 Followers 325 Following Roboticist, Computer Scientist and Entrepreneur who leverages the power of AI in Biomechanics
Richie (Yat Long) Lo @chrysalis_ai
138 Followers 371 Following AI | RL | LLM | RL @ Orbital Industries, Co-founder @rooka_ai | Ex: Amazon AGI, @CompSciOxford, @rlai_lab, @Microsoft, @ZebraTechnology, @shell, @dyson
Ehsan Imani @EhsanImanii
1K Followers 181 Following RecSys/Post-Training @Spotify | RL/AI PhD @UAlberta
Zeyi Wang @wzy950618
19 Followers 130 Following
Dylan R. Ashley @oneDylanAshley
201 Followers 212 Following PhD student studying reinforcement learning with @SchmidhuberAI. MSc with Rich Sutton of @rlai_lab. Sometimes amateur photographer. Opinions are my own.
Anna Koop 😷🧡�... @annakoop
682 Followers 1K Following Machine learning scientist, crafter, general enthusiast (she/her). Avid SJW representing only myself. I vent joy and anger and hope and pain and love on here.
Veronica Chelu @VeronicaChelu
863 Followers 1K Following postdoc • computational neuroscience & psychedelics @mcgillu | Past: @GoogleDeepMind | PhD @rllabmcgill @MILAMontreal | @IVADO_Qc |🦋@veronicachelu.bsky.social
Manan Tomar @manan_tomar
574 Followers 585 Following Postdoc @MSFTResearch NYC. Previously @rlai_lab, @berkeley_ai, @AIatMeta, @iitmadras. Opinions, if you find any, are my dog’s.
Yi Wan @YiWan89352121
351 Followers 224 Following Pursuing understanding and creating intelligence. Founding research scientist @Pokee_AI | PhD at @UAlberta with @RichardSSutton.
Somjit Nath @Somjit77
357 Followers 736 Following PhD Student @mcgillu @Mila_Quebec; Previously @TCSResearch @UAlberta @AmiiThinks
Niko Yasui @yasuiniko
15 Followers 25 Following
Elie Aljalbout @EliJalbout
1K Followers 674 Following Research Scientist @NVIDIARobotics | Prev: @Meta @UZH_en @ETH @TU_Muenchen @VWGroup | Opinions are my own and do not represent my employer.
Alex Kearney @kongaloosh
1K Followers 935 Following Co-founder & Head of Agents @ArtificialAgncy • fmr @DeepMind, @twitter & @borealisAI • PhD @ualbertacs w/ @amiithinks • Really like the #indieweb; #YEG
Alan Chan @_achan96_
2K Followers 1K Following Research Fellow @GovAIOrg | AI policy | PhD from @Mila_quebec | 🇨🇦
GAO Yue (Anna) @GAOYue167
30 Followers 90 Following ML at @GPTZeroAI Alumni @rlai_lab @AmiiThinks supervised by @CsabaSzepesvari
Qingfeng Lan @qingfeng_lan
660 Followers 1K Following Researcher @Alibaba_Qwen, PhD @rlai_lab. Ideas are my own.
Per R.L. @PerLeik
18 Followers 128 Following Integrating Neuroscience, Cybernetics, and RL; aiming for real-time learning and improvisation for artificial agents.
Louis Kirsch @LouisKirschAI
3K Followers 803 Following Driving the automation of AI Research. Co-Founder @inherent_labs. Ex @GoogleDeepMind. PhD @SchmidhuberAI. @UCL, @HPI_DE alumnus.
Wes Chung @wes_chung
451 Followers 922 Following A (somewhat) general intelligence. Also, PhD candidate at @mcgillu/@MILAmontreal focused on reinforcement learning research
Abhishek Naik @anaik96
507 Followers 213 Following PhD graduate in reinforcement learning from the University of Alberta. Now, RL research in space! 🚀🛰️
Paniz Behboudian @PBehboudian
235 Followers 390 Following Data Scientist @Granify Former AI Research Engineer @Huawei_Canada | @AmiiThinks | Interested in #RL | MSc at @ualberta |
Vincent Liu @vl_dimension
213 Followers 575 Following ML researcher @UBC @UAlberta @AmiiThinks. Interested in reinforcement learning.
Hager Radi 🍉🇵�... @hager_radi
481 Followers 979 Following Applied Research Scientist @mila_quebec | previously @UAlbertaCS @rlai_lab @amiithinks.
Pouya @Pouyakhl
100 Followers 142 Following PhD in Hydrology @ualberta | Climate Data Scientist | Specializing in Water Resources Management, Water-Food-Environment Nexus & Climate Change.
Alex Lewandowski @axlewandowski
306 Followers 219 Following PhD candidate studying learning algorithms @uAlberta, @AmiiThinks, @rlai_lab
Matthew Schlegel @mattschleg
525 Followers 195 Following RL and ML in real world applications. PhD. Musician. Celiac. On the search for the perfect espresso. Currently working on hard things @ucalgary.
Katya Kudashkina @KatyaKuda
447 Followers 254 Following PhD Candidate @VectorInst, @uofg, @rlai_lab | Researcher, entrepreneur, athlete | founder & CEO @culturaAI | surf, create, learn, yoga, live |
Christian Dallago @sacdallago
2K Followers 586 Following 🏳️🌈 @NVIDIAEU & @TU_Muenchen. Was @allianz, @vant_ai. BioCS+ML dude. 🦋 @machine.learning.bio https://t.co/m2CTpQUd4u
Farzane Aminmansour @FAminmansour
322 Followers 412 Following PhD student @UAlberta RIs: Reinforcement Learning, Computational Neuroscience, Machine Learning, Deep Learning
Doug Casey @RealDougCasey
26K Followers 57 Following Best-selling author, world-renowned speculator, and libertarian philosopher.
Oak Lab @oaklab_ai
7K Followers 0 Following We are building learning algorithms that allow agents to achieve goals in big worlds.
Philip Kiely @philipkiely
10K Followers 881 Following Author of Inference Engineering | Early @baseten | Not an LLM (yet)
John Schulman @johnschulman2
78K Followers 2K Following Recently started @thinkymachines. Interested in reinforcement learning, alignment, birds, jazz music
Roberta Raileanu @robertarail
11K Followers 2K Following Open-Endedness Team Lead and Senior Staff Research Scientist @GoogleDeepMind. Honorary Associate Professor @bold_lab_ai. ex @Meta | @NYU | @Princeton.
Pluralis Research @Pluralis
13K Followers 45 Following Pluralis is a research lab focused on collectively-owned AI.
Dan Hollick @DanHollick
48K Followers 389 Following design engineer @cursor_ai. writing a book about software at https://t.co/dFGw9eLRQM prev: @tailwindcss, @raycast
Ahmad Beirami ✈️ ... @abeirami
12K Followers 3K Following stealth // ex Gemini RL+Inference @GoogleDeepMind // Chat AI @AIatMeta // RL Agents @EA // ML+Information Theory @MIT+@Harvard+@GeorgiaTech
Murray Rothbard @MurraySuggests
42K Followers 1K Following Austrian economics with a commitment to individual liberty. #bitcoin #antistate #antimedia
Yoram Bachrach @yorambac
4K Followers 7K Following Research Scientist at Meta (prev Google DeepMind and Microsoft Research). Working on LLM Agents and Multi-Agent Systems.
Fatima Davelouis @fatimadg22
58 Followers 215 Following Reinforcement Learning researcher at @rlai_lab, grad student at @UAlberta and @AmiiThinks. Math enthusiast, yoga fanatic, soccer player, salsa dancer.
Angelica 🌐⚛️�... @AngelicaOung
50K Followers 8K Following Manic Nuclear Scheme Girl. Rootless cosmopolitan. Watching China from Taipei. Taiwan is a part of the Sinosphere.
Sapient Intelligence @Sapient_Int
5K Followers 2 Following Building leaner architectures for deeper intelligence.
TK • 木下 @wordsofteekay
10K Followers 339 Following coding . ml . math . writing . books . https://t.co/SJXxlX5JeT
Nous Research @NousResearch
234K Followers 26 Following A bunch of nerds making progress toward open source AI https://t.co/vrD0aDJeto
Jiayi Weng @Trinkle23897
12K Followers 187 Following MTS @openai, author of the entire post-training RL infra, core contributor of ChatGPT/GPT4/GPT4o etc. 30U30
Hager Radi 🍉🇵�... @hager_radi
481 Followers 979 Following Applied Research Scientist @mila_quebec | previously @UAlbertaCS @rlai_lab @amiithinks.
Ineffable Intelligenc... @IneffableLabs
8K Followers 0 Following Making first contact with superintelligence.
RL in Big Worlds @rlc_bigworlds
616 Followers 148 Following A workshop to promote research in Big Worlds @RL_Conference 2026, where the environment 🌍 is much bigger than the agent 🤖
Patricia Marins @pati_marins64
402K Followers 772 Following . support my work here 👉🏻 PayPal: [email protected] pix: [email protected] join my Substack: @global21.substack.com
Irina Stanescu @thecaringtechie
2K Followers 827 Following Writing "The Caring Techie Newsletter" Teaching "Impact through Influence" on Maven Eng Leader / Leadership Coach ex-@Google, ex-@Uber.
Dr. Nicole LePera @Theholisticpsyc
1.2M Followers 321 Following Join my private healing community @selfhealerscirc 👇🏼
Zico Kolter @zicokolter
29K Followers 757 Following Professor and Head of Machine Learning Department at @CarnegieMellon. Board member @OpenAI and @Qualcomm. Chief Scientist @GraySwanAI.
Mullvad.net @mullvadnet
73K Followers 1 Following A fast, trustworthy, and easy-to-use VPN is a good first step toward reclaiming your privacy. Just €5/month. // Need help? Email [email protected]
Igor Babuschkin @ibab
110K Followers 885 Following CEO & Co-Founder @river_ai_inc. Previously @xAI, Research & Engineering
LaurieWired @lauriewired
158K Followers 294 Following researcher @google; serial complexity unpacker; https://t.co/Vl1seeNgYK ex @ msft & aerospace
Jaana Dogan ヤナ �... @rakyll
167K Followers 1K Following ML/AI Systems at Google. Simplicity and optimism. Personal opinions.
Ashvin Nair @ashvinair
3K Followers 3K Following RL foundations @cursor_ai. Prev: o1, o3, Code Interpreter @openai, 9 years learning to poke by poking at UC Berkeley
Subhojeet Pramanik @subho_in
452 Followers 342 Following Researcher at @softmaxresearch working on scaling multi-agent reinforcement learning and organic alignment. MSc from @rlai_lab
Vincent D. Warmerdam @fishnets88
12K Followers 2K Following Prefer common sense over hype. Employed @marimo_io, built https://t.co/KOwpfLMpuQ blog https://t.co/zREZ8H2IBZ.
Costa Huang @vwxyzjn
8K Followers 2K Following Exploiting physical rewards @periodiclabs. Creator of @cleanrl_lib. Prev: RL @allen_ai @huggingface.
Ivan Yashchuk @IvanYashchuk
654 Followers 806 Following I work on @PyTorch at @NVIDIA Born in Ukraine | Raised in Siberia Yugra | Living in 🇫🇮 and 🇨🇭
Mo Samsami @M_R_Samsami
2K Followers 672 Following Building Hyperreal @GoogleDeepMind. Omni, Genie & Robotics.
Art Gallery @X_ArtGallery
251K Followers 11K Following Classical art • Personal lens • Forgotten masters • Analysis with a voice of my own
Anya Parampil @anyaparampil
146K Followers 3K Following @TheGrayzoneNews. Author of “Corporate Coup: Venezuela and the End of US Empire”
Núria Armengol @NriaArmengol2
139 Followers 260 Following ETH/CLS PhD candidate focused on reinforcement learning and sports lover.
Alan Couzens @Alan_Couzens
93K Followers 998 Following Physiologist, #Developer, Health/Fitness/Performance Coach for endurance athletes. Using #data #science & #AI to facilitate Max Athletic Development 🇦🇺#INTP
Benjamin Parry @_benjaminparry
4K Followers 1K Following useful hands and a cheerful heart | @_torontosocietyYou might like
