Sidak Pal Singh @unregularized
Research Scientist at Google DeepMind, working on Gemini. (prev. PhD at ETH Zürich & MPI-IS Tübingen.) No second-hand opinions. They are absolutely my own ;) sidakpal.com New York Joined October 2022-
Tweets51
-
Followers488
-
Following106
-
Likes68
quite good actually. coding’s so smooth! x.com/GoogleDeepMind…
This is Gemini 3: our most intelligent model that helps you learn, build and plan anything. It comes with state-of-the-art reasoning capabilities, world-leading multimodal understanding, and enables new agentic coding experiences. 🧵
sure the numbers are great, but screw that! it’s your phd, try wild and original ideas even if you fail once or twice at least. x.com/gabriberton/st…
A few numbers from my PhD: 8 first-author top-conference (CVPR/ICCV/ECCV) papers 100% acceptance rate per paper 80% acceptance rate per submission 1 invited long talk at CVPR tutorial 5 top-conf demos (acceptance rate 100% vs ~30% average) ~2k GitHub stars
when you go beyond linear mode connectivity, interesting things happen 😮👇 x.com/Theus__A/statu…
1/ 🚨 New paper alert! 🚨 We explore a key question in deep learning: Can independently trained Transformers be linearly connected in weight space — without a loss barrier? Yes — if you uncover their rich symmetries. 📄 arXiv: arxiv.org/abs/2506.22712
Belated life update: 🎓 PhD — done 🔬 Joined Google in NYC 🗽as a Research Scientist ♊️ Gemini: now more than just my star sign :)
🚀 TOMORROW afternoon at ICLR: Learn about the directionality of optimization trajectories in neural nets and how it inspires a potential way to make LLM pretraining more efficient ♻️ (Poster# 585, hall 2b)
Ever wondered how the optimization trajectories are like when training neural nets & LLMs🤔? Do they contain a lot of twists 💃 and turns, or does the direction largely remain the same🛣️? We explore this in our work for LLMs (upto 12B params) + ResNets on ImageNet. Key findings👇
Don't miss out our spotlight ✨paper at ICLR 🇸🇬 about the loss landscape of Transformers and their special heterogeneous structure, done together with great collaborators! x.com/wormaniec/stat…
Ever wondered how the loss landscape of Transformers differs from that of other architectures? Or which Transformer components make its loss landscape unique? With @unregularized & @f_dangel, we explore this via the Hessian in our #ICLR2025 spotlight paper! Key insights👇 1/8
@savvyRL :) I think the email address used there seems to suggest somebody doing it for him.. but you never know haha
✨New Preprint ✨ Ever thought that reconstructing masked pixels for image representation learning seems sub-optimal? In our new preprint, we show how masking principal components—rather than raw pixel patches— improves Masked Image Modelling (MIM). Find out more below 🧵
@mayank98shri @TheGradient @ynd @baharanm The bulk + outliers notion isn't wrong. The key is to understand how sharpness reduction is happening. GN can lead to spurious sharpness reduction; while NME reduces sharpness through adapting the geometry of the model itself. In vision, these ways are balanced, but not for LLMs.
Don’t miss our poster shedding more light on sharpness regularization at NeurIPS tomorrow neurips.cc/virtual/2024/p…
@agarwl_ @pratyushmaini It would be interesting to see the rank these models get haha. Predictions? :)
Reinventing things has a bad rep in today's age. But is it really that bad? Maybe it's something to be even cultivated, like selectively? The second post in this series of blogs is now out. Let's have a deeper look at this overused trope! wovencircuits.substack.com/p/reinventing-…
I’m exploring a new form of writing—threads of human curiosity woven through the circuits of AI, crafting reflections that are, in the end, fully machine-generated, yet in a way profoundly human. wovencircuits.substack.com/p/the-spirit-b…
Come, let's scale up the building one floor, And, layer up the neural networks once more. Soon our buildings will touch the sky, And, our computers will bear AGI. A quaint little hut in the mountains is out of fashion, Satisfaction has no gradients for backpropagation. ~Fitoor
@kellerjordan0 @bozavlado QK params and V params have very different behavior in terms of their curvature. So grouping them together is not ideal. I believe you could still try preconditioning QK params together, and keeping V separate.
At this paper count, recalling all the paper names would already be a big feat :) x.com/peter_richtari…
“Hypotheses are nets: only he who casts will catch.” - Novalis
At last some attempts to change the status quo: authors with three or more papers are obligated to review for ICLR x.com/PreetumNakkira…
Come to our posters today at 3:30 pm (Straus 2) to know more! :)
Poster 1: Sharpness/Flatness are much talked about: better minima, Sharpness aware minimization, Edge-of-Stability, and so on. But what really is sharpness? What exactly does it quantify, besides the surface-level definition? How are the eigenvalues and eigenvectors really like?
📢I'll be presenting two posters, at #ICML2024 HiLD workshop (Straus 2) today (assuming no further ✈️ delays): - Closed form of the Hessian spectrum for some neural networks openreview.net/forum?id=gW30R… - Landscaping Linear Mode Connectivity openreview.net/forum?id=OSNMq…
Poster 2: Linear Mode Connectivity (LMC) is yet another popular feature of neural loss landscapes. But how does LMC arise in the first place? How should the landscape be structured to allow LMC? Are barriers present just at the end, or do they start much early?
Tzu-Heng (Brian) Huan... @zihengh1
682 Followers 2K Following CS PhD @WisconsinCS. Prev: @Apple, @Meta. Focusing on weight-space geometry, program distillation, and data-centric AI (curation, synthesis, and auto-labeling).
Rodrigo Casado Noguer... @rodcasnog
73 Followers 132 Following PhD student @ETH_en · discrete diffusion
Neha Verma ✈️ ICM... @n_verma1
412 Followers 581 Following PhD student @jhu; Prev: research intern @Google+@GoogleDeepmind, @AIatMeta; @yale | Efficient models, merging, MT
Iván Jaen Márquez @ivanjaenm
131 Followers 1K Following CS PhD student @WisconsinCS | Fulbright Scholar @FulbrightPrgrm
Kausthubh Manda @MandaKausthubh
20 Followers 802 Following Engineering @ ThoughtSpot | Amazon ML Summer School'25 | IMtech'27 CSE@IIIT Bangalore | Machine Learning | Mathematics
Andreas @Andreas2059929
0 Followers 19 Following
Boris Knyazev @BorisAKnyazev
1K Followers 350 Following Research Scientist at Samsung AI Lab. PhD @uoguelph_mlrg. Opinions are my own.
Polina Kirichenko @polkirichenko
4K Followers 1K Following Research Scientist at FAIR @AIatMeta & visiting researcher at Princeton @VisualAILab prev. PhD at New York University 🇺🇦
Adnan Mohammed @ ICML... @adnan_ahmad1306
389 Followers 2K Following PhD student @ UofC / Vector Institute; Previously: MLRG, University of Guelph | UWaterloo | IIT G Alumnus
Weight Space Symmetri... @weightsymmetry
107 Followers 11 Following Workshop on Weight Space Symmetries: from Foundations to Practical Applications #ICML2026
Edward Milsom @edward_milsom
724 Followers 393 Following Lecturer (Assistant Professor) at University of Bath working on the foundations of deep learning. Looking for PhD students, email me (see website).
Athina Sotiropoulou @AthenaSotirop
51 Followers 560 Following MSc in Computational Science & Engineering from @ETH_en
llvmvsllm @llvmvsllm
63 Followers 1K Following
Viet Anh Khoa Tran @VietAnhKhoaTran
52 Followers 598 Following PhD student on Dendritic Learning/NeuroAI with @WillemWybo, at @EmreNeftci's lab (@fz_juelich). @ktran.de on the other app
comonoidal esotericis... @adamnemecek1
3K Followers 2K Following Building AGI by fixing autodiff @ https://t.co/x06CbGClKL
chalottefxtrader @harrisonchalott
1K Followers 5K Following GOD FIRST 🕊️🤍 🌴Miami/NY 👩💻Forex Trader📉 Account Management 📉Binary Options Trades 📚Become A Member And Join The World's Most Profitable Platform 📨DM
Swayam Agrawal @Swayam_421
230 Followers 878 Following Research @GoogleDeepMind | Prev CS, Robotics, Vision @IIIT_Hyderabad
Karen @PureKarenn
77 Followers 3K Following always moving for work, still makes time for the gym and swims.
Dong Sun @DongS27532
0 Followers 64 Following
j3d3 @rivoli23437649
73 Followers 3K Following music/poetry, depth psychology, film, AI (for creation, learning), Emerson, Kees, Pessoa, Sheldrake, Stanley 🇪🇺 games: https://t.co/faTZ7JzK8l
Teodora @teodorasrec
43 Followers 201 Following MSc student. Training neural nets 📉 & making robot friends 🤖
Liu Yang @Yang_Liuu
927 Followers 395 Following Research Scientist @GoogleDeepMind, PhD @WisconsinCS
Jason Lee @jasondeanlee
36K Followers 5K Following Associate Professor CS/stats UC Berkeley. Former Research Scientist at Google DeepMind. ML/AI Researcher working on LLMs and deep learning. PhD at Stanford.
Gpbhupinder @gpbhupinder
386 Followers 8K Following 🚀Generative AI | Python | Deep Learning | React, Next.js 👨💻 Full-Stack Developer & AI Integration Expert
Bahri Batuhan Bilecen @bbatuhanbilecen
80 Followers 700 Following PhD student @ETH Zürich and @cvml_mpiinf. Working on generative models, and representation learning for robotics.
Jinheon Baek @jinheonbaek
1K Followers 977 Following Ph.D. at @kaist_ai | Prev Intern at @Google @IBMResearch @MSFTResearch @Amazon | ML for knowledge, languages, and their intersections at scale.
Yamei Chen @NOrangeroli
17 Followers 873 Following
Rahel Jhirad @RahelJhirad
2K Followers 8K Following Founder, Imaginator ai knowledge discovery 2D navigation TS ML DL recsys econ math incentives mech design finance networks bridges boundaries, Time, 3d type
Ashutosh Bhardwaj @Astral_Quasar
5 Followers 231 Following CS @IIITDelhi | Exploring Robot Learning and Visual Navigation
Yash S @YashS0293
0 Followers 11 Following
anikzai @sasuraimonx
334 Followers 4K Following
Anand @Anand44719958
29 Followers 5K Following
Dimitriadis Nikos @nikdimitriadis
213 Followers 236 Following student researcher at Google DeepMind working on post-training. PhD @ EPFL.
Ujjwal Gupta @_ujjwal_gupta_
75 Followers 967 Following
Gad @Gadtitude
10 Followers 188 Following Writing about whatever interests me. Don’t label me, I’m growing!
Shibal Ibrahim @shibal_ibrahim
13 Followers 301 Following Machine Learning for Recommender Systems @GoogleAI (@Youtube) | @MIT Alum (PhD'24 in EECS(AI+D)
Pranav Shyam @recurseparadox
2K Followers 825 Following research scientist at google deepmind. ಕನ್ನಡಿಗ. past: generative models @openai
Soufiane Hayou @hayou_soufiane
1K Followers 343 Following Asst Professor at @JohnsHopkins (@JohnsHopkinsAMS and @HopkinsDSAI). Previously: @SimonsInstitute, @oxfordstats, @Polytechnique. I like to scale up things!
rohan anil @_arohan_
44K Followers 2K Following member of technical staff & co-founder of @coreautoai - and continuing to aspire to understand deep learning.
Clara Isabel Meister @clara__meister
2K Followers 59 Following Post-doc teaching a continuing studies program at ETH Zurich. Still figuring out how Twitter works... 🤦♀️
The Kobeissi Letter @KobeissiLetter
2.2M Followers 621 Following Official X account for The Kobeissi Letter, an industry leading commentary on the global capital markets. Email us: [email protected]
Amanda Askell @AmandaAskell
106K Followers 662 Following Philosopher & ethicist trying to make AI be good @AnthropicAI. Personal account. All opinions come from my training data.
Claude @claudeai
1.6M Followers 2 Following Claude is an AI assistant built by @anthropicai to be safe, accurate, and secure. Talk to Claude on https://t.co/ZhTwG8dz3D or download the app.
Blaise Agüera (@blai... @blaiseaguera
13K Followers 517 Following CTO of Technology & Society @Google, working on fundamental AI research and exploring the nature and origins of intelligence. Order "What Is Intelligence?" ⬇️
Subham Sahoo @ssahoo_
4K Followers 82 Following Pioneering Diffusion LLMs | Team Lead @mbzuai - IFM | PhD @cornell
Fabio Pardo @PardoFab
1K Followers 494 Following Senior Research Scientist at @GoogleDeepMind. Previously at @ImperialCollege, @Sorbonne_Univ_ and @ENS_ULM.
Juntang Zhuang @archanfel_anoth
7K Followers 651 Following prev lead xAI PreTraining, grok 4-fast/4.1/4.2/4.3 and coding rl, grok 2/3/4 core contributor. ex-OpenAI, inventor of GPT4-turbo 128k long context algo
Jonathan Frankle @jefrankle
23K Followers 811 Following Chief AI Scientist @databricks via MosaicML. e/brick
Laurence Aitchison @laurence_ai
2K Followers 577 Following LLMs and probabilistic machine learning. University of Bristol
Jason Wei @_jasonwei
110K Followers 717 Following ai researcher @meta superintelligence labs, past: openai, google 🧠
Jack Rae @jack_w_rae
25K Followers 542 Following Distinguished Scientist @ Meta LLMs (e.g. Gopher, Chinchilla, Gemini, Muse) Compression & RL ☯️ Past: Google, OpenAI, Quora
Mikhail Yurochkin @Yurochkin_M
602 Followers 180 Following Collecting tokens 🤖 Staff AI Scientist, LLM Data Team lead @MBZUAI Institute of Foundation Models. Prev. Research Manager @MITIBMLab @IBMResearch PhD @UMich
Bryan Johnson @bryan_johnson
1.7M Followers 662 Following Conquering death will be humanity’s greatest achievement.
Vaishnavh Nagarajan @_vaishnavh
4K Followers 748 Following Foundations of AI. I like simple & minimal examples and creative ideas. I also like thinking about going beyond the next token 🧮🧸 Google DeepMind | PhD, CMU
Lilian Weng @lilianweng
268K Followers 183 Following Co-founder of Thinking Machines Lab @thinkymachines; Ex-VP, AI Safety & robotics, applied research @OpenAI; Author of Lil'Log
Amr Khalifa @AmrMAlameen
1K Followers 480 Following Research Scientist @GoogleDeepMind, pre-training and architecture research in Gemini, Gemma. On leave from PhD with @AaronCourville. Opinions are my own.
Melis İlayda Bal @melisilaydabal
187 Followers 751 Following PhD student @mpi_is & @EPFL | organizer @tuewiml | prev: @mpi_sws_ , @metu_odtu | interested in optimization & game theory for robust & efficient ML
Antoine Moulin @antoine_mln
1K Followers 532 Following doing a phd in RL/online learning on questions related to exploration and adaptivity
Weronika Ormaniec @wormaniec
59 Followers 145 Following
Olga Saukh @osaukh
369 Followers 259 Following Assoc. prof at @tugraz: Efficient deep learning, embedded information processing and edge computing. A mix of theory and empirics. #StandWithUkraine🇺🇦
Lénaïc Chizat @LenaicChizat
2K Followers 282 Following Associate professor @nus. Previously @epfl and @cnrs researcher. Mathematics of AI and data science.
Jeremy Cohen @deepcohen
6K Followers 1K Following Research fellow at Flatiron Institute, working on understanding optimization in deep learning. Previously: PhD in machine learning at Carnegie Mellon.
Tri Dao @tri_dao
43K Followers 656 Following Asst. Prof @PrincetonCS, Chief Scientist @togethercompute. Machine learning & systems.
Ashish Vaswani @ashVaswani
32K Followers 2K Following
Nouha Dziri @nouhadziri
6K Followers 694 Following Researcher @cohere, ex @allen_ai, @GoogleDeepMind @MSFTResearch @MilaQuebec 🚨 PhD in LLMs 🤖 UofA. my blogs about LLMs reasoning: https://t.co/zJCt9jdYzJ
Volkan Cevher @CevherLIONS
4K Followers 702 Following Professor of Electrical Engineering, EPFL. Amazon Scholar (AGI Foundations). IEEE Fellow. ELLIS Fellow.
Dwarkesh Patel @dwarkesh_sp
242K Followers 1K Following Host of @dwarkeshpodcast https://t.co/3SXlu7fy6N https://t.co/4DPAxODFYi https://t.co/hQfIWdM1Un
Dimitri von Rütte @dvruette
3K Followers 362 Following AI/ML research. prev. PhD @ETH_en, ML engineer @DeepJudgeAI
Zico Kolter @zicokolter
29K Followers 757 Following Professor and Head of Machine Learning Department at @CarnegieMellon. Board member @OpenAI and @Qualcomm. Chief Scientist @GraySwanAI.
Tom Goldstein @tomgoldsteincs
28K Followers 2K Following Professor at UMD. AI security & privacy, algorithmic bias, foundations of ML. Follow me for commentary on state-of-the-art AI.
Aaron Defazio @aaron_defazio
8K Followers 642 Following Research Scientist at Meta Superintelligence Labs working on optimization algorithms. Fundamental AI Research (FAIR) team
Thinking Machines @thinkymachines
159K Followers 1 Following Thinking, beeping, and booping. @tinkerapi
Dang Nguyen @dangnth97
418 Followers 2K Following PhD Candidate @CS_UCLA | IMO 2015 Silver | Prev: @GoogleResearch and @Cisco
DAIR.AI @dair_ai
128K Followers 1 Following Democratizing AI research, education, and technologies. Learn about AI Agents for FREE at https://t.co/HHXg8rryu4
Gabriele Sicuro @gabrielescr
442 Followers 1K Following Statistical Physicist and Associate Professor @Unibo. Studied and worked in Rio, Rome, Paris, Lausanne, and London. PhD at @PhysicsUnipi. Views are my own.
Jacob Steinhardt @JacobSteinhardt
12K Followers 82 Following Associate Professor of Statistics and EECS, UC Berkeley // Co-founder and CEO, @TransluceAI
Zeyuan Allen-Zhu, Sc.... @ZeyuanAllenZhu
26K Followers 566 Following physics of language models @ Meta (FAIR at MSL, not GenAI or TBD) 🎓：Tsinghua Physics — MIT CSAIL — Princeton/IAS 🏅：IOI x 2 — ICPC — USACO — Codejam — math MCM
Matteo Pagliardini @MatPagliardini
883 Followers 441 Following PhD student in ML @EPFL_en, previously Apple MLR. @matpagliardini.bsky.social
Berfin Simsek @humansimsek
769 Followers 453 Following I never learn better in here. Just wanted to let you know
Hongyu Ren @ren_hongyu
23K Followers 719 Following research @meta superintelligence. CS PhD @stanford. prev @openai.
Jeremy Bernstein @jxbz
8K Followers 696 Following 🧪 @thinkymachines ✍️ anon feedback @ https://t.co/RIhBhjMRdDYou might like
