Karpathy just wrote the manual for Claude + Obsidian as a real second brain.
Most vaults die the same way. A year of saved articles and highlights. None of it linked. The graph rots while it still looks impressive.
So he moved the upkeep to the model. You curate sources and ask questions. Claude files, links, and reconciles. You keep judgment. It keeps the books.
raw belongs to you and never gets edited. wiki belongs to Claude. It isn't RAG. Your sources compile once into linked pages and compound from there.
9 rules. Start with 10 sources, not 10,000.
Most people hoard notes. This turns them into a brain that maintains itself.
The Ultimate Step-By-Step LLM Engineering Projects Roadmap (2026 Edition)
- Build a tokenizer
- Learn embeddings
- Implement RoPE / ALiBi
- Hand-wire attention
- Build MHA
- Build a Transformer block
- Train a mini-former
- Compare objectives
- Build sampling
- Speculative decoding
- KV cache
- MQA / GQA / MLA
- Long context
- FlashAttention
- Hardware budgets
- Toy MoE
- Sparse model trade-offs
- State-space / linear attention
- Diffusion language models
- Data pipelines
- Synthetic data
- Scaling laws
- SFT / DPO / RLHF / GRPO
- Quantization
- Serving stacks
- Eval harnesses
- RAG
- Tool use / agents
- Vision-language adapters
- Interpretability
- Red-team suite
- Full capstone model system
One request: Choose an Opensource AI lab when you make it
Opensource is where humanity gets to keep the tools
DM me when you've made it ;)
I genuinely don't understand why everyone isn't using this yet
Andrej Karpathy, a co-founder of OpenAI, posted a simple idea that hit 16 million views: stop using AI to write code, use it to build a second brain.
You point Claude Code at a folder, drop in any source, an article, a transcript, a PDF, and Claude reads it, links it, and files it into a living wiki of everything you know. It compounds like interest, the more you feed it, the smarter it gets.
Here's the whole thing:
> Install Obsidian, create a vault, open it in Claude Code
> Paste Karpathy's wiki idea file and tell Claude to build it
> Claude makes three folders: raw for sources, wiki for its pages, a CLAUDE.md that runs it
> Drop any source into raw and say "ingest this"
> Ask questions across everything, forever
Five minutes to set up, and you never start from a blank chat again.
Full step-by-step guide with Claude and Obsidian, link below.
Bookmark this
this PhD student had 47 interviews and 4 offers before she was hired at OpenAI.
she practiced with her “notes on LLMs” and math and they’re a goldmine. super concise and organic and shared to everyone for free. you can use her notes or her topic list to study on your own.
I'm joining OpenAI next week!🥹 The job search turned out to be really challenging but also super rewarding, so I wrote a small blog to share what I learned along the way and hopefully make the process a little less mysterious for the next person. alisawuffles.github.io/blog/job-search
Keeping up with LLM systems research is messy when papers, reports, frameworks, and course links are scattered everywhere.
LLMSys-PaperList is a curated LLM systems reading list for AI/ML engineers, researchers, and builders tracking how large language models are trained, served, and optimized.
It helps you follow the field by organizing papers and resources into practical systems categories instead of one flat bookmark dump.
Key features:
• Training systems – pre-training, post-training/RLHF, fault tolerance, and straggler mitigation sections
• Serving systems – LLM serving, agent systems, edge serving, and efficiency optimization links
• Multi-modal coverage – separate training and serving sections for multi-modal systems
• Research context – industrial LLM technical reports, survey papers, benchmarks, leaderboards, and traces
• Learning path – frameworks, ML systems readings, MLSys courses, and conference-specific paper lists
Free public GitHub repo.
Link in the reply 👇
Google has published a paper that might end the transformer era.
For the last 7 years, every major AI, ChatGPT, Claude, Gemini, has been built on the exact same architecture: The Transformer.
But Transformers have a fatal flaw.
To remember context, they have to process every single word against every other word. It’s called quadratic complexity. As your prompt gets longer, the compute cost explodes.
The alternative is the old-school RNN (Recurrent Neural Network). RNNs are incredibly cheap and fast, but they have a fixed memory size. If you give them a long document, they get amnesia.
Until today.
Google researchers published Memory Caching: RNNs with Growing Memory.
And it fixes the biggest bottleneck in AI.
Instead of an RNN having a fixed, rigid memory that constantly overwrites itself, Google gave it a "save" button.
The technique allows the RNN to cache checkpoints of its hidden states as it reads.
The memory capacity of the RNN can now dynamically grow as the sequence gets longer.
They built four different variants, including sparse selective mechanisms where the AI actively chooses exactly which checkpoints matter most.
The results rewrite the rules of efficiency.
On long-context understanding and recall-intensive tasks, these new Memory-Cached RNNs closed the gap with Transformers.
They achieved competitive accuracy without the explosive, quadratic compute cost. It perfectly bridges the gap between the cheap efficiency of an RNN and the massive capability of a Transformer.
We have spent billions scaling Transformers because we thought they were the only way an AI could remember a long conversation.
But Google just proved we don't need to process the whole history every single time.
We just needed a smarter cache.
Think of yourself as an LLM.
Every social interaction, every meeting, burns your tokens.
Unless someone is a paid subscriber to your attention, you are under no obligation to answer low-quality prompts.
611 Followers 504 Following🇺🇸US First
🙅♀️No Gift Card Robot
Living in a coastal city, I love the sea. It is my longing for freedom🌊
Loves to travel fitness yoga dog lover
51 Followers 1K Following59yrs Richard Van Tuyl who works for a construction company in the Kansas City as won a $1000,000 and giving back to the society by paying credit card debts
337 Followers 7K FollowingPart of the innovation journey with @elonmusk| Pushing the boundaries of technology, committed to a sustainable future| Other account
563 Followers 7K FollowingGet Rich. Get Fit. Get Smarter. Learn what the schools won't teach you. Hit follow and I'll help you become the best you can be.
782 Followers 44 FollowingApplied AI | Gen AI Researcher @AaltoUniversity | Building @ToDataBeyond an AI Newsletter read by 27k+ AI developers & Data Scientists
673K Followers 123 Following#1 New York Times Bestselling Author of five books, including "The Subtle Art of Not Giving F*ck". Co-Founder of @Purpose_AI. Host of Solved Podcast.
11K Followers 1K FollowingUniversity Professor. Assoc Head Coach for @WT_Wrestling. I read old books, lift heavy weights, and mostly post about Tolstoy.
723K Followers 120 Following#1 NYT Bestselling Author: The 48 Laws of Power, The Art of Seduction, The 33 Strategies of War, The 50th Law, Mastery, The Laws of Human Nature, The Daily Laws
795K Followers 1K FollowingOfficial Twitter account of the World’s Favorite Genius. Managed by CMG Worldwide, authorized representative of the Albert Einstein Estate.
250K Followers 265 FollowingPublisher Skeptic Magazine https://t.co/i81i0rp1HP | Host The Michael Shermer Show | NEW BOOK “TRUTH”: https://t.co/0FvkBGFl26
457K Followers 220 FollowingTheoretical physicist. Best-selling author. Science & public policy advocate. President of @OriginsProject & Host of Origins Podcast, Board of FSU Canada
850K Followers 50 FollowingOfficial X of Theoretical Physicist, Futurist, Bestselling Author, On-Air Personality, Professor of Physics, and Public Speaker.
29K Followers 2K FollowingCEO + Founder of @riseatseven. We build category leaders - on every searchable platform. Drum dadi agency of the year Forbes #30under30! UK & NY US