-
Tweets23
-
Followers5
-
Following383
-
Likes8
humans are still in another league. Maybe intelligence is not who wins 4â1. Maybe it's who learns more from less.
AlphaGo beat Lee Sedol 4â1. But consider the training data: Lee Sedol: perhaps 10,000â30,000 games over a lifetime. AlphaGo: millions of self-play games. If intelligence is measured by performance alone, AlphaGo wins. If intelligence is measured by performance per unit experienc
4/4 What I found most interesting was how shifted window attention lets information gradually flow across the entire image. The model achieved better medical image segmentation performance than several CNN and hybrid Transformer-CNN approaches on CT datasets.
3/4 Interesting architectural ideas: ⢠Window Multi-Head Self Attention (W-MSA) ⢠Shifted Window Attention (SW-MSA) ⢠Patch Merging for downsampling ⢠Patch Expanding for upsampling ⢠Skip connections between encoder and decoder This creates a pure Transformer U-Net.
1/4 Today I read the paper âSwin-Unet: Unet-like Pure Transformer for Medical Image Segmentationâ. The paper tries to replace traditional CNN-based U-Net architectures with a fully Transformer-based segmentation model for medical imaging.
4/4 Another interesting systems idea in the paper was âContinuous Depth-wise Batchingâ. Since the same layers are reused recursively, they combine this with early exiting and theoretically achieve up to 2â3Ă inference throughput gains. Authors: @raymin0223 @adamjfisch @harhrayr
3/4 The results were surprisingly strong. Recursive Gemma 1B outperformed vanilla TinyLlama 1.1B and Pythia 1B models trained with similar parameter budgets. As LoRA rank increased, the relaxed recursive models nearly recovered full-size transformer performance.
1/4 Today I read the paper âRelaxed Recursive Transformersâ from ICLR 2025. Core idea: Instead of using N unique transformer layers, reuse K layers multiple times recursively to reduce parameter count and inference cost. Paper: openreview.net/forum?id=WwpYSâŚ
Prompts are software architecture and weights are hardware.
- "On that day, I want you and I to have that same conversation again. I will tell you exactly about today's conversation, about how your policy literally caused the United States to concede the second largest market in the world for no good reason at all."
Few of the most badass quotes from Jensen in his latest appearance on the Dwarkesh - "The input is electrons, the output is tokens. In the middle is Nvidia." - "The beautiful thing is you're talking to the expert."
- "Is Blackwell 50 times more advanced lithography than Hopper? Not even close. Blackwell is 50 times Hopper." - "Comparing AI to anything that you just mentioned is lunacy."
- "When Nvidia first started, there were 60 3D graphics companies. We are the only one that survived." - "I am the evidence." - "You're not talking to somebody who woke up a loser"
In most of the scenario a calculator is missing whether it is chemistry or chess or anything else
Can LLM be integrated with a traditional rl model suppose LLM able to understand/build the embedding of a chess board generated by RL. This will give LLM world to work with đ¤
How to summon the monster: Prompt Engineering â Agentic â SFT â RLHF â RL (synthetic data for self-play)
Does it matter if the things we use are handicrafts or mass-produced? If not, then it also won't matter if the movie we are seeing is AI-generated or not."
AI-ishere @AIishere_art
111 Followers 122 Following đŽ AI Artist | Vedic mysticism Ă Futuristic vision Surreal drops daily â Midjourney & Stable Diffusion đŠ Collabs open in DMs đ My universe â
Chetan @Bigboysayshello
34 Followers 81 Following Just a nerd coder who codes his life. IITian. Playing with APIs pushing in production. Do CP @codeforces and practice dsa @leetcode
Rory Bernier @RoryCrave
3K Followers 8K Following AI Red Teamer & Product Strategist | LLM & Agent Security ⢠Zero-Trust | Building verifiable AI systems | Clarity Foundry
đŚMysticUnicornđŤ... @LibertyNwvdvx
241 Followers 1K Following đThe world is so big, I want to see it âď¸đď¸
tianhang zhu @TianhangZhuzth
32K Followers 26 Following head of LLM training at @Fundamental. former head of reinforcement learning at @Alibaba_Qwen and 01ai.
Îťux @novasarc01
22K Followers 3K Following tensor shepherd in a non-euclidean pasture | grazing on cuda cores
Aishwarya Kamath @ashkamath20
8K Followers 633 Following Staff Research Scientist @GoogleDeepMind on the Gemini team. Vision Lead on Gemma 3 & Gemma 4. PhD at NYU with @ylecun Masters at UMass Amherst
Woosuk Kwon @woosuk_k
8K Followers 783 Following @inferact | @vllm_project | prev: PhD @Berkeley_EECS
Paper2comic @paper2comic
37 Followers 158 Following Comics that help non-experts vaguely understand research papers. Made by a non-expert.
Ani Kembhavi @anikembhavi
4K Followers 336 Following AI Research @meta Former Director @wayve_ai @allen_ai. Best/Outstanding papers at CVPR, Neurips, CoRL, IROS and ICRA.
Max Mynter @MaxMynter
6K Followers 2K Following python mines @mistralAI || rust oss when I have time || vim, nix, and haskell when i feel like it
Mimansa Jaiswal @MimansaJ
6K Followers 6K Following Currently RS @aiatmeta | LLMs/SLMs Post Training | Data, Evals, Rewards and Agentic System Orchestration https://t.co/Nv4aMHzhTC
basvanopheusden @basvanopheusden
3K Followers 265 Following Research at OpenAI, previously @imbue_ai and @cocosci_lab lab at Princeton. All opinions my own
Timon Willi @TimonWilli
371 Followers 70 Following RS @AIatMeta, DPhil w/ @j_foerst, @UniofOxford; Formerly: Research Intern @GoogleDeepMind / PhD @VectorInst / RS at @nnaisense / MSc w/ @SchmidhuberAI
Johannes Treutlein @j_treutlein
470 Followers 192 Following Research Scientist at Truthful AI, working on alignment in LLMs with @OwainEvans_UK. Formerly @AnthropicAI.
Wonbong Jang / Won @wbjang11
274 Followers 465 Following working at @AIatMeta, PhD in CS @ucl - interested in Computer Vision / Machine Learning
Silvia Sapora @silviasapora
2K Followers 446 Following PhD student at @UniofOxford with @FLAIR_Ox and @OxCSML @imperialcollege CompSci '15-'19
Julie Kallini âď¸ ... @JulieKallini
3K Followers 505 Following CS PhD @StanfordNLP đ˛ Previously: SWE @Meta, Class of '21 @PrincetonCS
Zhaofeng Wu @zhaofeng_wu
3K Followers 287 Following PhD student @MIT_CSAIL | Previously @allen_ai | MS'21 BS'19 BA'19 @uwnlp
elie @eliebakouch
20K Followers 4K Following training llm @PrimeIntellect (prev: @huggingface) anon feedback: https://t.co/JmMh7Sg3mL
AIM Intelligence @AIM_Intel
175 Followers 36 Following We AIM for Trustworthy and Safe AI https://t.co/AoRGr5Je89
FAR.AI @farairesearch
21K Followers 23 Following Frontier alignment research to ensure the safe development and deployment of advanced AI systems.
Sachin Goyal @goyalsachin007
2K Followers 729 Following Pretraining @ Anthropic | Past: PhD @ CMU MLD, intern at Meta, Google and MSR | UG: IIT Bombay
Maxime Labonne @maximelabonne
29K Followers 541 Following Head of Post-Training @liquidai đť GitHub: https://t.co/ElXDsjzGOn đ¤ HF: https://t.co/2ECS7Gjhwb đ Blog: https://t.co/Gz5bhbYuIy
Takahiro Miki @ki_ki_ki1
3K Followers 912 Following @GoogleDeepMind / prev ETH Zurich RSL. Reinforcement learning Locomotion. DARPA SubT Challenge winner.
Lucy Lai, PhD @drlucylai
7K Followers 2K Following cognitive scientist & asst teaching prof @ucsandiego âď¸ | previously: ph.d. @PiN_Harvard, b.a. @RiceUniversity, @OISTedu, @MPC_CompPsych, @BCM_Neurosci
Gary Marcus @GaryMarcus
228K Followers 7K Following OG GenAI Skeptic; spoke at US Senate. Warned about hallucinations in 2001. Advocating world models & neurosymbolic AI ever since. Author, Marcus on AI & 6 books
Yunyao Li @yunyao_li
6K Followers 712 Following Bring GenAI and Knowledge Graph to enterprise systems. | Director of ML @Adobe Experience Platform | Previously @Apple @IBMResearch. Tweets are all mine.
Peide Huang @peide_huang
2K Followers 327 Following Research scientist @GoogleDeepMind. Robotics & Physical AGI. Ex-Apple. Ph.D. @CarnegieMellon, M.S. @Stanford, B.E. @NTUsg. All opinions are my own.
Subham Sahoo @ssahoo_
4K Followers 81 Following Pioneering Diffusion LLMs | Team Lead @mbzuai - IFM | PhD @cornell
Georgia Channing @cgeorgiaw
7K Followers 429 Following AI4Science @ đ¤, PhD @OxfordTVG â science is a candle in the dark
Sylvie Shi @sylvieshi00
728 Followers 361 Following llm pretraining @cohere. previously led reranking at cohere
acquire.com @acquiredotcom
103K Followers 2K Following The largest startup acquisition marketplace. Buy and sell SaaS, ecommerce, agencies, content, newsletters, mobile apps and crypto businesses.
Jacob X. Li âď¸ IC... @jacobli99
3K Followers 167 Following PhD Student @MIT EECS & CSAIL, advised by @lateinteraction. Prev. @BrownUniversity
kache @yacineMTB
303K Followers 6K Following reinforcement learning, robots. prev eng @ x, stripe. 6'3 (height) tensorpunk subscribe to read my blog!
Jack Morris @jxmnop
52K Followers 1K Following research @engramlab // language models, information theory, science of AI // formerly @cornell
Nathan Lambert @natolambert
91K Followers 921 Following Open model research @ something new. Prev. co-led Olmo at Ai2. Contact via email. Writes @interconnectsai Wrote The RLHF Book, đď¸đââď¸
Elliot Arledge @elliotarledge
39K Followers 278 Following low taste kernel engineer | KernelBench-Hard | made the 12 hr CUDA course | https://t.co/2hzezpoDwQ | @shipfr8 alum
Seungjae (Jay) LEE @JayLEE_0301
229 Followers 94 Following @UofMaryland, @nvidia GEAR | Ex @SeoulNatlUni, @ToyotaResearch
Ryan Peterman @ryanlpeterman
28K Followers 434 Following Building the podcast & ergonomic keyboard I wish existed ⢠ex-software engineer @instagram, @meta ⢠See what I'm building here â
Xueyan Zou @xyz2maureen
2K Followers 184 Following Assistant Professor @Tsinghua_Uni prev. PostDoc đ¤ @UCSDJacobs; PhD đ @UWMadison; Intern @MSFTResearch @BytedanceTalk
Trends for United States
You might like
