Ben Walker @benjaminwalker
Founder & CEO, Continuous Intelligence. Building models that learn from the world as it unfolds in continuous time. Email: [email protected] continuousintelligence.co.uk University of Oxford Joined February 2022-
Tweets161
-
Followers187
-
Following186
-
Likes105
Unless I’ve missed something, there are still no technical details on how they make the approach subquadratic. Anyone know how they choose which previous tokens a query should attend to without first looking at all previous tokens, or is the subquadratic claim just marketing?
The transformer architecture used for ChatGPT, Gemini, and Claude has defined the last decade of AI. It also introduced a fundamental constraint: compute scales quadratically as context grows. Longer inputs, exponentially higher costs and accuracy that degrades well before the
Using the new GPT-Image-2 to help me express what it feels like to watch a one-off specific instruction survive Codex compaction, and then get passed down forever as legend through each successive compaction
Looks like codex and chatgpt are now down and this is the first time I have seen a foreign language in a response, is this somehow linked?
Codex felt it could only express how totally declarative the dataclass definition should be in Russian
@MingchenZhuge @tydsh @karpathy @DrJimFan @SchmidhuberAI @_akhaliq @hardmaru @zechunliu @YoungXiong1 @HaoZhe65347 @cai_zhipeng What distinguishes a neural computer from a world model? If it learns the dynamics of computation, memory, I/O, and state updates well enough to reproduce the machine’s behaviour, then it seems like a world model where the “world” happens to be a computer.
That explains why gpt-5.4-codex is at capacity
To celebrate 3 million weekly codex users, we are resetting usage limits. We will do this every million users up to 10 million. Happy building!
If only Yule had known about this when he invented autoregressive modelling in 1927, we could have had language models before computers!
Hate to break it to you, but the first LLM was created by Andrey Markov in 1913. he tallied up 20,000 letters from a famous novel and computed p(vowel | vowel) p(consonant | vowel) p(vowel | consonant) p(consonant | consonant) basically 'training' a bigram by hand
Most machine learning is about finding the right feature extractor for a linear readout. Even an LLM.
A visualisation of the idea behind rough path theory: a path is not fully described by its value Each curve has the same area: when the number of circles doubles, their radius is scaled by 2^{-1/2}. The path's value converges to the straight line, but the total area does not
@xlr8harder My chat keeps using a LaTeX macro it invented in a previous chat
ChatGPT found a song I’d been trying to find for ages, then built a playlist around it that was way better than Spotify’s suggestions. How is an LLM better at music recommendation than a direct recommendation system?
Fantastic OxYSS session with Emma Prevot (@OxfordStats) & @benjaminwalker (@OxUniMaths) on the intersection of Causal Inference and Continuous-Time ML. A vibrant discussion on the future of temporal modelling!
New ChatGPT tell: the sentence uses a colon. This is, however, not the only one.
Everything starts to look like a path once you stare at it long enough
@ryu0000000001 A natural next step is developing path-to-path models that can handle irregular or over-sampled inputs. Currently, we use the Log-ODE method, but because it outputs a sequence (the solution only at interval endpoints), the resulting models cannot be stacked.
Take the same underlying path and increase the number of samples. As the sequence length grows, RNNs become difficult to train and Transformers become too expensive. Our continuous-time models instead converge to a continuous hidden-state path.
Want to know more? Log-NCDEs: efficient continuous-time sequence models with strong empirical performance. arxiv.org/abs/2402.18512 SLiCEs: parallel-in-time continuous-time models that don't sacrifice expressivity. arxiv.org/abs/2505.17761
@HochreiterSepp Never liked the term 'Linear RNN', as their updates are nonlinear in (h_t, x_t). The real bottleneck is restrictive structure that prevents hidden-state interactions. Without that restriction, Linear RNNs already have the expressivity for world modelling: arxiv.org/abs/2505.17761
Continuous Intelligen... @Continu_Intel
10 Followers 29 Following Building models that learn from the world as it unfolds in continuous time.
0xc00000e @ ICML @x3n0graft
1 Followers 67 Following
Victor Robert @viktorobert
1K Followers 6K Following Humanist, Technologist, Free Thinker, 3X founder. Fusion, AI and robotics will usher in humanities golden age. We will rise to the ocean of stars
Pim de Witte @PimDeWitte
16K Followers 2K Following Building @gen_intuition: models for envs that require deep spatiotemporal reasoning. I like games, OSS, AI, and once built the world’s largest RuneScape server.
MuRo @M_Sumaraq
114 Followers 3K Following
Christoph Reich @ChristophR1996
376 Followers 830 Following @ELLISforEurope Ph.D. Student @tumcvg, @visinf & @Oxford_VGG | https://t.co/yaqdM5V4IW. from @etitdarmstadt & https://t.co/VHpc5R9Fs1. from @CS_TUDarmstadt | Prev. @NECLabsAmerica & @koeppl_lab
Louis @Louis9687221579
109 Followers 4K Following Mainline Economics | Idea page | ramblings of a schizo
Zeyuan Allen-Zhu, Sc.... @ZeyuanAllenZhu
26K Followers 566 Following physics of language models @ Meta (FAIR at MSL, not GenAI or TBD) 🎓：Tsinghua Physics — MIT CSAIL — Princeton/IAS 🏅：IOI x 2 — ICPC — USACO — Codejam — math MCM
Matthew Ford @fordmatt18
5 Followers 64 Following
Kobbitic @kobbitic
6 Followers 1K Following
Matteo Papini @papinimat
528 Followers 728 Following Tenure Track Assistant Professor at @LaStatale Bluesky: @papinimat LinkedIn: https://t.co/hRQmvJewMB
Thomas Lew @thomas__lew
224 Followers 194 Following Research Scientist at @ToyotaResearch. Optimal Control, Machine Learning, Robotics. PhD @Stanford. Previously intern at @Google, @NASAJPL.
Latios @Latios630519
2 Followers 610 Following
cty @cty
52 Followers 2K Following
Tran Dang Hoan @hoanhaiphong
31 Followers 5K Following
Alexis C. @4l3k6_C
126 Followers 3K Following
Alex Yanko 🇺🇦 @LeopolisDream
1K Followers 3K Following ML Team Lead, Former Head of Data and Analytics, working on SaaS projects. Data Science 🔭 Analytics 📈 Engineering.
Manas @trippyhat4
0 Followers 272 Following
Marcus Barnes @MarcusBarnes
1K Followers 6K Following PhD Researcher | LLM4SE & multi-agent AI systems | LLMs for mathematics & autoformalization | Open to research collaborations, industry roles & consulting
Jj @Jj33524298
150 Followers 7K Following
tensor @_1010sor
17 Followers 668 Following first year undergrad in animal husbandry + rhythmic gymnastics @uwaterloo. learning machines
Xero @chaos_xero
0 Followers 3K Following
deliciousSandwich @mrsirrisrm
132 Followers 6K Following
Amey Varhade @ameyvarhade
533 Followers 4K Following Research Fellow @MSFTResearch Previously @IITGuwahati @ibm_in
Raymond Ng @Raymondng_aisg
4 Followers 2K Following
Achilles_of_Myrmidons @AMyrmidons
498 Followers 3K Following
రామయ్య �... @generalusername
402 Followers 2K Following MechE • DatSci • Linguistics • CG Ⱄⰾⱁⰲⱁ Ⱀⰵⰶⰻⱅⱁ
Roger Frigola @RogerFrigola
1K Followers 5K Following Engineer for America's Cup and Formula 1 teams. Machine Learning PhD @Cambridge_Uni. From Barcelona. Flat-sixes and four-cylinder transaxles (sic).
Antibody News @antibodynewshq
257 Followers 5K Following
Joaquin Gajardo @JoaquinGajardoC
14 Followers 304 Following CS and AgTech PhD student @ETH Zürich. Working on ML and 3D vision for agriculture. My mantra: be humble, gentle, help people, and never stop learning :)
SunWukong @Kapishreshtha
75 Followers 4K Following
Kelvin 🦖🤓 @kelvinhan
92 Followers 2K Following #NLProc PhD-ed at @labo_Loria. Currently Research Fellow at @singaporetech. Questions generator. https://t.co/3mUSCnSHTf
Timothy Hitge 🇿�... @tim_hitge
76 Followers 1K Following Research Assistant @imperialcollege. @GoogleDeepMind Scholar. MSc in AI for Science 🧪 @AIMSacza. Prev intern @instadeepai
Zhaoyang Wang @zhaoyangwang_
773 Followers 7K Following Researcher | Agentic AI, Bayesian Optimization, reinforcement learning, and AutoML. Uni of Birmingham.
Akano Lordstrong @Grace_fuelled
217 Followers 6K Following
Corolla @StateOfCorolla
48 Followers 697 Following
Matthew Willetts @matthewjfw3
199 Followers 844 Following Correlation doesn’t imply causation unless I’m doing the regression. prev: Visiting Researcher @turinginst, Research Fellow @uclcs, ML PhD @oxcsml @oxfordstats.
Evelyn @tummycom
861 Followers 7K Following Open Source, Mountain Time. Give me or the universe anonymous feedback: https://t.co/ZaUAOdtsG9
Giosue Migliorini @joh_sweh
53 Followers 619 Following Ph.D. student in stats @UCIrvine | former AI research intern @FlagshipPioneer, @LosAlamosNatLab, @UniBocconi
Continuous Intelligen... @Continu_Intel
10 Followers 29 Following Building models that learn from the world as it unfolds in continuous time.
Pim de Witte @PimDeWitte
16K Followers 2K Following Building @gen_intuition: models for envs that require deep spatiotemporal reasoning. I like games, OSS, AI, and once built the world’s largest RuneScape server.
Jürgen Schmidhuber @SchmidhuberAI
207K Followers 0 Following OG of: P and T in ChatGPT, 100x deeper learning, meta learning and RSI, neural distillation, GAN/World Model... Co-authored most-cited AI paper of 20th century
Álvaro Cartea @AlvaroCartea
377 Followers 68 Following Professor of Mathematical Finance, and Director of the Oxford-Man Institute, University of Oxford. Book: Algorithmic and High-Frequency Trading.
Joaquin Gajardo @JoaquinGajardoC
14 Followers 304 Following CS and AgTech PhD student @ETH Zürich. Working on ML and 3D vision for agriculture. My mantra: be humble, gentle, help people, and never stop learning :)
Kelvin 🦖🤓 @kelvinhan
92 Followers 2K Following #NLProc PhD-ed at @labo_Loria. Currently Research Fellow at @singaporetech. Questions generator. https://t.co/3mUSCnSHTf
Timothy Hitge 🇿�... @tim_hitge
76 Followers 1K Following Research Assistant @imperialcollege. @GoogleDeepMind Scholar. MSc in AI for Science 🧪 @AIMSacza. Prev intern @instadeepai
Giosue Migliorini @joh_sweh
53 Followers 619 Following Ph.D. student in stats @UCIrvine | former AI research intern @FlagshipPioneer, @LosAlamosNatLab, @UniBocconi
Nora Hedwig Nordlinde... @HeNordlinder
160 Followers 755 Following Phylogenetics and deep learning @karolinskainst.
Jorge Bravo Abad @bravo_abad
12K Followers 9K Following Physicist @UAM_Madrid · Director, AI for Materials Lab · Building AI-driven loops turning scientific discovery into infrastructure · Two books on AI and science
Philip Schroeder @philip_mit
759 Followers 694 Following PhD student at MIT in Computer Science. @MIT_CSAIL @MITEECS @nlp_mit
Fernando Moreno-Pino @fermorenp
231 Followers 2K Following Assistant Professor in Machine Learning at @BristolUni. Research Associate at @Oxford_Man_Inst. Previously PostDoc at @UniofOxford and PhD at @uc3m.
Philipp Nazari @philna00
115 Followers 297 Following PhD Candidate at Max Planck ETH Center for Learning Systems
Teemu Sarapisto @Tsarpf
227 Followers 497 Following CS/ML PhD research in high-dim time-series @helsinkiuni Before: 5y of C++/JS/VR/AR/ML at Varjo, Yle, Reaktor, Automattic After dark: synthesizers & 3D gfx
Riccardo Grazzi @riccardograzzi
136 Followers 158 Following Researcher at MSR Cambridge, previously at IIT in Genoa, Italy. Working on principled, efficient optimization for machine learning and on LLMs' expressivity.
Paul Thompson @PTenigma
8K Followers 3K Following Neuroscientist, professor AI guided tour - https://t.co/yyVhX7Jwv2 ENIGMA guided tour - https://t.co/6oEtWdx7np
Leo Gao @nabla_theta
13K Followers 584 Following working on AGI alignment. prev: GPT-Neo, the Pile, LM evals, RL overoptimization, scaling SAEs to GPT-4, interp via circuit sparsity. EleutherAI cofounder.
EXO Labs @exolabs
52K Followers 2 Following Frontier AI on local hardware. EXO 1.0 is now open-source (Apache 2.0): https://t.co/SGGGK784Qp
Alex Cheema @alexocheema
52K Followers 3K Following building @exolabs | prev @UniOfOxford We're hiring: https://t.co/UlkApFndnH
Peter Potaptchik @PPotaptchik
364 Followers 448 Following DPhil student at Oxford https://t.co/JH0l4u7wHv
Sajad Movahedi @Sajad_Movahedi_
126 Followers 196 Following PhD student in machine learning at @ELLISInst_Tue and @MPI_IS with @orvieto_antonio.
Emilian Postolache @EmilianPostola1
601 Followers 660 Following Lead ML Researcher @irisaudiotech | PhD in CS @SapienzaRoma in @GladiaLab | Former @CaFoscari, @SonyCSL, @Dolby and @c4dm
Yuxin Wen @ywen99
635 Followers 877 Following AI Security @OpenAI | PhD @umdcs advised by @tomgoldsteincs
Fu-En (Fred) Yang @FuEnYang1
966 Followers 2K Following Research Scientist @NVIDIAAI | Ph.D. @NTU_TW | Prev. Research Intern @NVIDIAAI | Unifying World, Language & Action for Generalist Robotics
Wei Deng @dwgreyman
429 Followers 652 Following Research Interests: Sampling and Diffusion (Language) Models AI/ML researcher @ Morgan Stanley, 21-. Purdue PhD ’21. Ex-Baidu, 12–15.
Kasper Green Larsen @kasperglarsen
2K Followers 260 Following Professor and Head of Algorithms, Data Structures and Foundations of Machine Learning at Computer Science, Aarhus University
An Thái Lê @an_thai_le
482 Followers 661 Following Assistant Professor for #RobotLearning at VinUniversity | Director of Foundation AI at VinRobotics
Weiyan Shi @shi_weiyan
9K Followers 1K Following Prof @Northeastern | MIT TR-35 | #AI2050 Early Career Fellow | Prev @Columbia @StanfordNLP | Co-created CICERO | human-AI co-evolution + AI safety
Puze LIU @ RSS @liu_puze
369 Followers 294 Following Curiosity Driven, On Robot Learning. Associate Professor @Tongji_Uni Previous Deputy Head at @DFKI SAIROL, former w/@ias_tudarmstadt and @jan_peters
Neel Jain @neeljain1717
778 Followers 1K Following PhD candidate @umdcs @ml_umd advised by @tomgoldsteincs. Undergrad at Williams College in Math. My views are my own.
tingwu.wang @TingwuWang
1K Followers 593 Following Research scientist in robotics @ GEAR Nvidia. I obtained my PhD from University of Toronto @UofT, Vector Institute @VectorInst 😃
Avi Schwarzschild @A_v_i__S
922 Followers 254 Following Trying to learn about deep learning faster than deep learning can learn about me.
Percy Liang @percyliang
110K Followers 425 Following professor of computer science @Stanford @stanfordnlp, co-founder of @togethercompute, creator of https://t.co/7R5THVogW2, co-founder of @simile_ai, pianist
Samuel Albanie 🇬�... @SamuelAlbanie
8K Followers 1K Following gemini evals & post-training @GoogleDeepMind
Visual Inference Lab @visinf
855 Followers 393 Following Visual Inference Lab of @stefanroth at @TUDarmstadt. Research in Computer Vision and Machine Learning.
Shayne Longpre @ShayneRedford
6K Followers 1K Following Lead the Data Provenance Initiative. PhD @MIT. 🇨🇦 Prev: @Google Brain, Apple, Stanford. AI/ML/NLP
Christoph Reich @ChristophR1996
376 Followers 830 Following @ELLISforEurope Ph.D. Student @tumcvg, @visinf & @Oxford_VGG | https://t.co/yaqdM5V4IW. from @etitdarmstadt & https://t.co/VHpc5R9Fs1. from @CS_TUDarmstadt | Prev. @NECLabsAmerica & @koeppl_lab
Hossein Souri @HosseinSouri8
453 Followers 702 Following Senior AI Researcher at @Samsung_RA. CS PhD at @JohnsHopkins, MS at @UofMaryland.
Moran Mizrahi @moranmiz
376 Followers 311 Following PhD in CS @Csehuji (@HyadataLab); Passionate about NLP, Data Science, HCI and Computational Creativity.
Alvaro Arroyo @arroyo_alvr
295 Followers 235 Following PhD ML @UniofOxford ; Transformers & Graph Representation Learning; Previously at @imperialcollege
Jonas Geiping @jonasgeiping
5K Followers 898 Following Machine Learning Researcher in Tübingen at the ELLIS Institute & Max Planck for Intelligent Systems // Working on Safety & Efficiency of modern ML
Felix Sarnthein @__safelix__
123 Followers 344 Following PhD student in machine learning at @ELLISInst_Tue, @MPI_IS and @CSatETH with @orvieto_antonio. Prev: MSc in CS at @ETH
Yifan Zhang @yifanzhang_
15K Followers 4K Following PhD at @Princeton University, Princeton AI Lab Fellow. RL & LLM Reasoning, Pretraining & Language Modeling. Prev @ Seed @Tsinghua_Uni
Sunny Sanyal @SunnySanyal9
1K Followers 803 Following On Job market | PhD candidate @UTexasECE| Prev Intern @GoogleDeepMind (FAI), @LightningAI & @AmazonScience (Alexa)
Hannah Rose Kirk @hannahrosekirk
4K Followers 755 Following AI researcher trying to make sense of the cyberspace 🤖 Workstream Lead @AISecurityInst. Uni of Ox PhD @oiioxford & Prev @Cambridge_Uni.You might like
