David Page @dcpage3
Machine learning researcher Joined April 2018-
Tweets181
-
Followers2K
-
Following1K
-
Likes915
I am thrilled to share the news that Fractile's mission to build chips and systems that unlock the next generation of AI scaling has been bolstered, with a $220M funding round led by Accel, Factorial Funds, and Founders Fund, alongside some incredible backers old and new. AI inference is driving the defining infrastructure buildout of the 21st Century. We've written a bit about where we think capabilities must go, and how Fractile is working to bring this about: fractile.ai/news/fractile-… It has been a privilege working on one of the hardest but most rewarding technical challenges of our time for over three years, with the most brilliant, kind and driven people I could have ever hoped to work alongside. We are still just getting started. There is a lot to be done to deliver on our goals, but we are grateful to have the support of so many people in chasing these down every minute of every day. Thanks, all, for being part of the Fractile mission! 🚀 @fractile_ai
@badlogicgames My git repo 🙈 github.com/davidcpage/ski…
@jeremyphoward @hi_tysam @kellerjordan0 @karpathy Impressive, well done! Hi Jeremy! Still lurk here occasionally and hopefully start blogging again soon..
@samgd @jeremyphoward @nanopore Yep not me this time! Looks like fascinating work from Giuseppe though
The paper that introduced Batch Norm arxiv.org/abs/1502.03167 combines clear intuition with compelling experiments (14x speedup on ImageNet!!) So why has 'internal covariate shift' remained controversial to this day? Thread 👇
Excited to announce our new work, a unified theory towards explaining 3 black magics in deep learning: (1) ensemble, (2) knowledge distillation, and (3) self-distillation. An accessible blog post is below.
Microsoft and CMU researchers begin to unravel 3 mysteries in deep learning related to ensemble, knowledge distillation & self-distillation. Discover how their work leads to the first theoretical proof with empirical evidence for ensemble in deep learning: aka.ms/AAavp1k
@bozavlado @iiSeymour CTC_CRF extends flipflop to output scores for multiple (six) consecutive bases not just two. Output layer is mostly orthogonal to choice of RNN/CNN encoder so CNN improvements are very welcome! More details coming soon..
Big accuracy update coming in the next version of Bonito 🚀 v0.3.0 combines everything we have learned with structured and unstructured approaches - @dcpage3, Tim and myself are working hard on the finished touches this week - watch this space 👀
Some base-caller updates coming within 5-10 days. 98%+modal and many reads above Q20. Note the X-axis. Generally, sig +ve uplift in consensus and mutation detection. Slightly slower speed in research version.
First day of new job @nanopore where I get to apply ML to a bunch of fun science and engineering problems. Pretty excited!
Bonito v0.2.2 - SAM output - Sequence and alignment tsv summaries - Grab bag of training improvements from @dcpage3 github.com/nanoporetech/b…
Preparing a short course on neural nets can be fun. Below is one of the fast Resnets by @dcpage3 on CIFAR10. Would have been nice to track a UMAP-like representation of some internal layer, but have not found a reasonably fast/stable way to do so. Any idea? @NikolayOskolkov
Undertraining a large model is a good way to speed things up on toy problems myrtle.ai/how-to-train-y… but it was far from clear this should extend to large scale.
Great study of training efficiency at large scale + nice results on compression for inference!
Not everyone can afford to train huge neural models. So, we typically *reduce* model size to train/test faster. However, you should actually *increase* model size to speed up training and inference for transformers. Why? [1/6] 👇 bair.berkeley.edu/blog/2020/03/0… arxiv.org/abs/2002.11794
Simple setup + attention to details -> sota self-supervised reps! LARS -> large batches -> no need for memory bank of -ve examples Random crops + color aug (to prevent hist cheating) -> no need for special arch Projn head for contrastive loss -> hidden reps preserve info
Introducing SimCLR: a Simple framework for Contrastive Learning of Representations. SimCLR advances previous SOTA in self-supervised and semi-supervised learning on ImageNet by 7-10% (see next). arxiv.org/abs/2002.05709 Joint work with @skornblith @mo_norouzi @geoffreyhinton.
Blitzing fast CTC decoding github.com/nanoporetech/f…
@Buntworthy It's fixed now. Thanks for letting me know!
@ylecun @viglovikov @timetravellertt @kaggle The problem though with "you can always add those tricks to get the numbers up" is that *very* often I see papers that don't do data aug, or don't tune hyper-params, etc, then claim their new idea helps. But then I find it's actually just a poor proxy for the things they skipped
Bojan Tunguz @tunguz
292K Followers 8K Following Founder and CEO @tabul_ai. Creator of @trainxgb. ML ex Nvidia. Data Scientist. Physicist. Catholic. Husband. Father. Stanford Alum. Memelord. e/xgb. AMDG.
Jeremy Howard @jeremyphoward
319K Followers 7K Following 🇦🇺 Co-founder: @AnswerDotAI/@FastDotAI ; Prev: Professor@UQ; @kaggle founding president; founder @fastmail/@enlitic/… https://t.co/16UBFTX7mo
Radek Osmulski @radekosmulski
30K Followers 615 Following LLMs and retrieval by day and other genres of AI when I get the chance 🧪 Senior AI Eng @NVIDIAAI 🏫 @fastdotai trained DL Eng 📝 https://t.co/By87iXx5Pu
Ross Wightman @wightmanr
24K Followers 1K Following AI / Computer Vision. Ex 🤗. Ex head of Software, Firmware Engineering at a Canadian 🦄. Building AI systems or investing in startups that do it better.
Jim Fan @DrJimFan
470K Followers 3K Following NVIDIA Director of Robotics & Distinguished Scientist. Co-Lead of GEAR lab. Solving Physical AGI, one motor at a time. Stanford Ph.D. OpenAI's 1st intern.
(((ل()(ل() 'yoav)))... @yoavgo
83K Followers 2K Following
Delip Rao e/σ @deliprao
69K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈
Sanyam Bhutani @bhutanisanyam1
42K Followers 1K Following 👨💻 Working on llama models @AIatMeta | Previously: @h2oai, @weights_biases 🎙 Podcast @ctdsshow 👨🎓 Fellow @fastdotai 🎲 Grandmaster @Kaggle
Tanishq Mathew Abraha... @iScienceLuvr
89K Followers 1K Following CEO @SophontAI | Founder @MedARC_AI | PhD at 19 (2023) | ex Research Director Stability AI | Biomed. engineer @ 14 | TEDx talk➡https://t.co/xPxwKTq6Qb
Dmytro Mishkin 🇺�... @ducha_aiki
27K Followers 763 Following Marrying classical CV and Deep Learning. I do things, which work, rather than being novel, but not working.
Eric Jang @ericjang11
135K Followers 4K Following
Matt Henderson @matthen2
80K Followers 3K Following maths, visualisations, conversational AI. VP Research @polyaivoice prev: @RekaAILabs, @Apple AI/ML, @GoogleAI, PhD @Cambridge_Eng
rohan anil @_arohan_
43K Followers 2K Following member of technical staff & co-founder of @coreautoai - and continuing to aspire to understand deep learning.
Sylvain Gugger @GuggerSylvain
29K Followers 363 Following Machine Learning at Jane Street. Previously at @huggingface and @fastdotai Co-author of https://t.co/lywnOAwwnc He/him
Peter Whidden @computerender
2K Followers 183 Following
She holland @SheHolland0
83 Followers 899 Following
λux @novasarc01
22K Followers 3K Following tensor shepherd in a non-euclidean pasture | grazing on cuda cores
prime Elon x🚀 @primeElonxsak9
209 Followers 396 Following CEO, and Chief Designer of SpaceX🚀 CEO and product architect of Tesla
Little Frank @MidknightMantra
368 Followers 4K Following
Kamalika Chaudhuri @kamalikac
6K Followers 3K Following Researcher, Google Deepmind. Formerly, Director FAIR @ Meta. Former Professor at UCSD. Researcher in AI privacy, security, and generalization.
Mukaddas Khusniddinov... @Mukaddas_25
2 Followers 102 Following
Cecilia @Ceecmegafx
185 Followers 3K Following Single mom of 2 Adventures Emergency Volunteer, Conservative, Love traveling, Portfolio Manager, Stock Trader.
Meryem Arik @MeryemArik9
735 Followers 803 Following Inference Lab. I’m the token (generation) woman - CEO/Co-founder Doubleword - F30U30 (anti-signal?)
Mike Schroepfer @schrep
100K Followers 276 Following founder @Gigascale, Sr Fellow/fmr CTO @Meta, founder @AdditionalVent. Let’s go build!
Majid Mosayebi @mosayebi_m
7 Followers 118 Following
Humam Zaman @humam66349
10 Followers 688 Following Co-founder @TheTechValley | We fix: broken production, AI code mess, AWS cost (save 40%+), slow apps, CI/CD failures. Same-day response. DM us.
ssn @fl0wbar
7 Followers 2K Following
fluffy @fluffykittnmeow
696 Followers 379 Following
Smalltime Newsy @The_NewDigital
132 Followers 1K Following The New Digital - a developing AI News outlet. 🎗 #FollowBack! 🎗
Jeffrey Emanuel @doodlestein
45K Followers 14K Following Former Quant Investor | My Open Source Projects: https://t.co/9qbOCDlaqM | Try https://t.co/oCtjI2mBIl , my collection of agent coding tooling.
Tom Westgarth @Tom_Westgarth15
5K Followers 6K Following Head of Growth @fractile_ai | ex: AI policy @institutegc, @UKSovereignAI Advisor | Bridging tech and policy worlds @txp_io
Yifan Zhang @yifanzhang_
14K Followers 3K Following PhD at @Princeton University, Princeton AI Lab Fellow. RL & LLM Reasoning, Pretraining & Language Modeling. Prev @ Seed @Tsinghua_Uni
Goliath @zero_goliath
1K Followers 616 Following @uwaterloo cs alum; formerly: founder @ritserlabs, contractor @MechanizeWork, intern @runrl_com
Assesseth The Oracula... @assesseth
198 Followers 3K Following Assesseth, god of assessment. I say that which is and which is to be shortly. AI/ML by choice, Economics by night, Middle East affairs by necessity.
Student Page @StudentPag44377
0 Followers 50 Following
ModestyWatt @5hn6YgkI32j3R4E
52 Followers 2K Following
EileenJim @qvs027k5c6iZm3
14 Followers 536 Following
Amir Gholami @amir__gholami
515 Followers 272 Following Associate Research Scientist, BAIR and Skylab, UCB
Muhammad Ovais Yusuf @movaisyusuf
2 Followers 47 Following I make your AI/ML research 50x better | 12+ Publications | MS CS @ BSU | Breaking down research clarity, baselines & experiments
Walter Goodwin @goodwin_ml
1K Followers 371 Following CEO & Founder, Fractile Building the future of planet-scale AI inferencing
anuj @ahsesjuna
15 Followers 2K Following
Ted - 🥖/acc @ted_engineer
194 Followers 1K Following 🇫🇷 / 26yo / dad / deep learning chef @duonlabshq
Artem Permiakov @artpermiakov
7 Followers 1K Following
Hamed Firooz @mamhamed
274 Followers 285 Following AI research director @aiatmeta, focusing on AI, LLM, Multimodal, Safety. Opinions are my own
Wen-Ding Li @xu3kev
3K Followers 7K Following LLM for code and reasoning. PhD student at Cornell. Previously Student Researcher at @google. Previously intern at @theteamatx.
Dongw @DongWan42101620
192 Followers 6K Following SWE by day, doing OSS at night. Make gpus go brrr!
Ajay kumar @ajayinvictus
25 Followers 694 Following
elie @eliebakouch
20K Followers 4K Following training llm @PrimeIntellect (prev: @huggingface) anon feedback: https://t.co/JmMh7Sg3mL
Brenno de Mello @BrennoMeello
91 Followers 4K Following
Luca Viano @LucaViano4
460 Followers 487 Following Optimist in face of the uncertainty, PhD student @EPFL
Wemi @0xWemi
132 Followers 2K Following Crypto Trader navigating the markets with discipline and strategy. Sharing insights, analysis, and lessons learned from the grind. Focused on growth, not hype.
Yann LeCun @ylecun
1.2M Followers 787 Following Professor at NYU & Executive Chairman at AMI Labs. Ex-Chief AI Scientist at Meta. Researcher in AI, Machine Learning, Robotics, etc. ACM Turing Award Laureate.
Sebastian Raschka @rasbt
468K Followers 1K Following ML/AI research engineer. Ex stats professor. Author of "Build a Large Language Model From Scratch" (https://t.co/O8LAAMRzzW) & reasoning (https://t.co/5TueQKx2Fk)
Bojan Tunguz @tunguz
292K Followers 8K Following Founder and CEO @tabul_ai. Creator of @trainxgb. ML ex Nvidia. Data Scientist. Physicist. Catholic. Husband. Father. Stanford Alum. Memelord. e/xgb. AMDG.
Jeremy Howard @jeremyphoward
319K Followers 7K Following 🇦🇺 Co-founder: @AnswerDotAI/@FastDotAI ; Prev: Professor@UQ; @kaggle founding president; founder @fastmail/@enlitic/… https://t.co/16UBFTX7mo
Lucas Beyer (bl16) @giffmana
141K Followers 610 Following Researcher (now: Meta. ex: OpenAI, DeepMind, Brain, RWTH Aachen), Gamer, Hacker, Belgian. Anon feedback: https://t.co/xe2XUqkKit ✗DMs → email
Radek Osmulski @radekosmulski
30K Followers 615 Following LLMs and retrieval by day and other genres of AI when I get the chance 🧪 Senior AI Eng @NVIDIAAI 🏫 @fastdotai trained DL Eng 📝 https://t.co/By87iXx5Pu
Ross Wightman @wightmanr
24K Followers 1K Following AI / Computer Vision. Ex 🤗. Ex head of Software, Firmware Engineering at a Canadian 🦄. Building AI systems or investing in startups that do it better.
Google DeepMind @GoogleDeepMind
1.5M Followers 278 Following The engine room of @Google. Building AI safely and responsibly to solve the world’s most complex problems. Join us: https://t.co/jUHQA27iBL
Soumith Chintala @soumithchintala
309K Followers 1K Following Building new things @thinkymachines. Also dabble in robotics at NYU. Cofounded @PyTorch. AI is delicious when it is accessible and open-source.
(((ل()(ل() 'yoav)))... @yoavgo
83K Followers 2K Following
Hamel Husain @HamelHusain
50K Followers 3K Following Evals Evals Evals - https://t.co/Zrmp6LRd9c About Me: https://t.co/P6WyeKkyTa
Andrew Ng @AndrewYNg
1.6M Followers 1K Following Co-Founder of Coursera; Stanford CS adjunct faculty. Former head of Baidu AI Group/Google Brain. #ai #machinelearning, #deeplearning #MOOCs
Richard Socher @RichardSocher
120K Followers 1K Following Building self-improving superintelligence CEO @recursive_si and @youdotcom MP @aixventuresHQ Ex: Stanford Adj Prof, Chief Scientist at Salesforce, CEO MetaMind
Delip Rao e/σ @deliprao
69K Followers 5K Following Busy inventing the shipwreck. @Penn. Past: @johnshopkins, @UCSC, @Amazon, @Twitter ||Art: #NLProc, Vision, Speech, #DeepLearning || Life: 道元, improv, running 🌈
Chris Albon @chrisalbon
92K Followers 3K Following Field notes on generating knowledge with AI at https://t.co/4E9DwWIDG7 | Director, ML & Data @Wikimedia
Peyman Milanfar @docmilanfar
113K Followers 579 Following Distinguished Scientist at Google. National Academy of Engineering. Computational Imaging ∩ AI. Posts are personal opinions
Aran Komatsuzaki @arankomatsuzaki
181K Followers 365 Following Sharing AI research. Early work on AI (GPT-J, scaling, MoE). Ex ML PhD (GT) & Google.
Tanishq Mathew Abraha... @iScienceLuvr
89K Followers 1K Following CEO @SophontAI | Founder @MedARC_AI | PhD at 19 (2023) | ex Research Director Stability AI | Biomed. engineer @ 14 | TEDx talk➡https://t.co/xPxwKTq6Qb
Aditi Merchant @aditimerch
950 Followers 340 Following BioE PhD student @stanford in the Hie Lab // ML for SynBio
British Open-ended Le... @bold_lab_ai
591 Followers 23 Following
Dan Biderman @dan_biderman
3K Followers 2K Following CEO @EngramLab Previously @HazyResearch @cu_neurotheory @DbrxMosaicAI
Samuel L Smith @SamuelMLSmith
4K Followers 393 Following Member of Technical Staff at OpenAI. Formerly Staff Research Scientist at Google DeepMind. Ex-Physicist.
Charlie O'Neill @oneill_c
4K Followers 1K Following The sea is the sea The old man is an old man The boy is a boy and the fish is a fish The sharks are all sharks no better and no worse
DAN KOE @thedankoe
949K Followers 966 Following join the next content bootcamp: https://t.co/KHN63fQmtI
Michael Mignano @mignano
60K Followers 708 Following General Partner @USV. Co-Founder Anchor (acquired by Spotify) and @OboeLabs. Early investor in @meetgranola, @suno, @xai, others.
Yu Su @ysu_nlp
15K Followers 1K Following co-founder @NeoCognition | prof. @osunlp | sloan fellow | building towards abundance of specialized intelligence
Object Zero @Object_Zero_
37K Followers 990 Following Doer of the difficult. Champion for talent. Inventor of things. Builder of Machines. North Sea O&G, Nuclear Power, Subsea, Heavy Manufacturing.
Peter Whidden @computerender
2K Followers 183 Following
Armin Ronacher ⇌ @mitsuhiko
81K Followers 901 Following Creator of Flask. Building at https://t.co/uGuzfu0LKT. Bypassing Permissions. Can hand crank. Husband and father of 3 — “more nuanced in person”
Arthur Mensch @arthurmensch
73K Followers 854 Following Co-founder and CEO @MistralAI. Mistral Vibe can help you https://t.co/IHZhDUiAnO
Gavin Baker @GavinSBaker
254K Followers 6K Following Managing Partner & CIO, @atreidesmgmt. Husband, @l3eckyy. No investment advice, views my own. https://t.co/pFe9KmNu9U
Jeonghye Kim @beanie0__0
212 Followers 150 Following Research Intern @MSFTResearch | Reinforcement Learning PhD @KAIST
Dom Hallas @Dom_Hallas
4K Followers 2K Following Executive Director @StartupCltn. Semi-professional Yorkshireman. @NUFC fan. Formerly @DExEUGov @BloombergDotOrg
Kanishka Narayan MP @KanishkaNarayan
8K Followers 289 Following Labour MP for the Vale of Glamorgan | Minister for AI and Online Safety | 📨 [email protected] 🏴🇬🇧
James Wise @Jameswise
18K Followers 4K Following Partner @Balderton. Investing in AI, health and energy. Chair of UK Sovereign AI. Author of Start-Up Century. Mostly Generative.
Sam Bowman @s8mb
68K Followers 2K Following Editor-in-Chief of @WorksinProgMag, proudly part of @stripe. Editor at @StripePress.
Cohere @cohere
117K Followers 6 Following Own your AI. Empowering enterprises with private, powerful systems and builders with accessible, open-source models.
Mikel Artetxe @artetxem
7K Followers 228 Following Co-founder @RekaAILabs and Honorary Researcher @Hitz_zentroa (University of the Basque Country) | Past: Research Scientist @AIatMeta (FAIR)
Ekdeep Singh Lubana @EkdeepL
3K Followers 1K Following Member of Technical Staff @GoodfireAI; Previously: Postdoc / PhD at Center for Brain Science, Harvard and University of Michigan
Christopher Potts @ChrisGPotts
16K Followers 724 Following Stanford Professor of Linguistics and, by courtesy, of Computer Science. Member of technical staff @stanfordnlp and @StanfordAILab. Co-founder @ Bigspin AI.
Muyu He @HeMuyu0327
3K Followers 306 Following RL @ZyphraAI | Expert of mixtures, or 21st century schizoid man.
Kevin Roose @kevinroose
172K Followers 3K Following NYT tech columnist, Hard Fork co-host, high-perplexity language model. Writing a book about AGI.
Laura Ruis @LauraRuis
7K Followers 829 Following Postdoc with @jacobandreas @MIT_CSAIL. PhD from @ucl_dark with @_rockt and @egrefen. Anon feedback: https://t.co/sbebAl53tU
Louis Kirsch @LouisKirschAI
3K Followers 805 Following Driving the automation of AI Research. Co-Founder @inherent_labs. Ex @GoogleDeepMind. PhD @SchmidhuberAI. @UCL, @HPI_DE alumnus.
Prime Intellect @PrimeIntellect
67K Followers 40 Following The Open Stack for Self-Improving Agents https://t.co/ZRZOsRRbwr
Caiming Xiong @CaimingXiong
8K Followers 500 Following Co-founder of @Recursive_SI. ex-SVP of Salesforce AI Research | ex-MetaMind (Opinions are personal.)
Tim Rocktäschel @_rockt
47K Followers 2K Following Teaching AI the joy of invention at @Recursive_SI, Professor of AI @AI_UCL, PI @BOLD_Lab_AI, Fellow @ELLISforEurope. Ex @GoogleDeepMind @AIatMeta @CompSciOxford
Tim Shi @timshi_ai
7K Followers 3K Following turning compute into knowledge @recursive_si, early @openai ‘16, cofounder @cresta ($1b+) (@sequoia @greylockVC @a16z), stanford phd dropout
Shiqian Ma @ShiqianMa
2K Followers 1K Following Professor@Rice University. AI, Machine Learning and Optimization
Tilde @tilderesearch
5K Followers 10 Following We build foundational understanding of models to advance the frontier of intelligence.
David Galbraith @daveg
23K Followers 2K Following Technologist and VC, former architect. Invented these (i.e. link in bios: https://t.co/5p8DVMxINg ), among other things.
Kumiko @CV_novel_plume
703 Followers 817 Following Views sampled from P_θ(· | my priors) at high temperature; priors are personal, provisional, and usually wrong.
Alex Laterre @AlexLaterre
2K Followers 781 Following Co-founder @IneffableLabs | prev. Head of Research @ Instadeep
Xinyu Yang @Xinyu2ML
5K Followers 1K Following PhD dropout from @CarnegieMellon, Scaling pre-training, post-training, and agent, Opinions are my own. Architect. They/Them
Meryem Arik @MeryemArik9
735 Followers 803 Following Inference Lab. I’m the token (generation) woman - CEO/Co-founder Doubleword - F30U30 (anti-signal?)You might like
