boolean_filter @boolean_filter

"Transformers" by Daniel Jurafsky and James H. Martin is one of the clearest and most mathematically grounded introductions to the Transformer architecture I have ever read. Chapter 8 introduces the Transformer as the standard architecture behind modern large language models. What makes this chapter particularly interesting is its step-by-step presentation of the underlying mechanisms: contextual embeddings, self-attention, query, key and value vectors, scaled dot-product attention, multi-head attention, residual streams, feedforward layers, layer normalization, masking, and the parallel matrix formulation of attention. In particular, the treatment of attention as a weighted sum of contextual representations is especially valuable. The chapter first develops an intuitive, simplified view of attention and then gradually derives the full formulation using the Q, K, and V matrices. This approach makes it easier to understand what is actually happening inside the architecture from an algebraic and matrix-based perspective, rather than simply viewing the usual block diagrams. I think it is an excellent resource for anyone interested in understanding how Transformers work from linguistic, mathematical, and computational perspectives. web.stanford.edu/~jurafsky/slp3…

The University of Michigan put their entire robotics degree on GitHub. Not one course. The whole curriculum. ROB 101 — Computational Linear Algebra for Robotics ROB 311 — How to Build Robots and Make Them Move ROB 501 — Mathematics for Robotics ROB 530 — Mobile Robotics Every lecture video on YouTube. Every textbook on GitHub. Every problem set, every exam, every line of code. Professor Jessy Grizzle said it best when they launched it: "Linear algebra has become the language of computer vision, machine learning, robotics, and autonomy." So instead of making students wait four semesters of calculus before touching a robot... they built a curriculum that starts with the math that actually matters, applied to real robotics problems from day one. This is what open education looks like when a top-10 engineering school decides to mean it. Free. GitHub. YouTube. 📌 [github.com/michiganroboti…] Follow for more robotics resources! —— Weekly robotics and AI insights. Subscribe free: 22astronauts.com

Python is wild for time series. Introducing Orbit. A Python library for Bayesian probabilistic time series forecasting by Uber.

Jane Street pays $750K/year for quants who master matrix calculations holistically that can be used to get alpha from space satellites. 57-minutes. free. By MIT Applied Math team. "If you are hand-computing derivatives on complicated matrix factorizations, it is incredibly error-prone. Modern automatic differentiation is more computer science than calculus." here's the breakdown: • The math behind large-scale engineering simulations • How backpropagation relies entirely on reverse-mode adjoint differentiation • Thinking in matrices instead of scalar arrays Watch this now. This will change the way you optimize networks forever and then read below article.

venus @RitOnchain

a month ago

x.com/i/article/2064…

5 9 76 29K 108

An engineering professor who failed math her entire childhood spent years figuring out exactly what had been sabotaging her, and the answer was not low intelligence. It was a hidden mode her brain kept switching into that nobody had ever told her existed. Her name is Barbara Oakley. The book is called A Mind for Numbers. She failed math and science from grade school to the end of high school. Numbers felt like a language everyone else had been taught in secret. So she ran toward the thing she was good at. She enlisted in the Army right after graduation, and the Army paid her to learn Russian at the Defense Language Institute in Monterey. She got very good at Russian. Good enough to earn a degree in Slavic Languages, serve four years in Germany as a Signal Officer, and rise to Captain. Then the wall appeared. She watched her career options shrink because she could not handle the technical side of her own job. The people with math moved up and moved out. The people without it stayed stuck. So at 26 she did something that sounds insane. She left the Army and enrolled in engineering, starting from remedial math, sitting in classrooms with teenagers. In between, she worked as a Russian translator on Soviet trawlers in the Bering Sea and as a radio operator in Antarctica. Today she is a professor of engineering at Oakland University with a doctorate in systems engineering. The question that drove her for years was simple. What changed? She was the same brain that failed algebra. Why did it suddenly start working? The clue was hiding in the one subject she had mastered. She noticed she had never learned Russian by staring at it. She practiced a little every day, walked away, came back, and the language quietly assembled itself between sessions. Math she had attacked the opposite way. Lock eyes with the problem. Push harder. Refuse to look away until it cracks. It never cracked. And neuroscience explains why. Your brain has two modes. The focused mode is the one you know. Tight attention, prefrontal cortex engaged, grinding through familiar steps. The diffuse mode is the one nobody teaches you. It runs in the background when you relax. It is loose, wide, and wired for connecting ideas that sit far apart from each other. Oakley uses a pinball machine to explain the difference. In focused mode, the bumpers are packed tight. Your thought bounces in the same small circle, over the same ground, again and again. In diffuse mode, the bumpers spread out. The thought travels. It reaches parts of the brain the tight loop could never touch. The trap has a name. The Einstellung effect. The first approach that comes to mind blocks every better approach behind it. The harder you focus, the tighter the loop, the more locked in you become. The grinding feels virtuous. It is actually the cage. And every time her mind wandered off a math problem as a kid, she dragged it back, believing the wandering was laziness. The wandering was her brain trying to switch into the mode that solves things. She spent ten years fighting the half of her brain that wanted to help her. You cannot run both modes at once. The diffuse mode only takes over when you genuinely let go. Which is why answers ambush you in the shower, on a walk, at the edge of sleep. Salvador Dali knew this. He napped in a chair holding a key over a plate, and the instant he drifted off, the key dropped, woke him, and he carried the half-formed ideas straight back into focused work. Edison did the same trick with ball bearings. Two of the most inventive minds in history were deliberately farming the mode the rest of us treat as slacking off. The practical version fits in two sentences. Focus hard on the problem until you stall. Then stop completely, and let the other mode take the shift. The break is not a reward for the work. The break is the work. It is also why cramming fails and procrastination is fatal. Diffuse mode needs hours and nights between focused sessions to build anything, and procrastination burns that time before the first session even starts. Oakley failed math for ten years using one mode at full strength. She became an engineering professor the day she started using both.

I just spent months handwriting a 200 page guide on the entirety of ML foundations and math from scratch. The guide features: - Neural Nets (Backprop, Adam, SGD, Batch Norm) - ML Algorithms (SVM, Grad Boosting, K-means, PCA) - Hardware (Tensor Cores, Systolic Arrays, CUDA) - Transformers (Multi-Head Attn, KV Cache, LoRA) - Vision (ViT, Convolutions, MAE, IoU, NMS, VLM) - Agents (OpenClaw, ReAct, Memory, Orchestration) Everything I wish I had years ago, for free.

A DEVELOPER TAUGHT GIT WITH A BOX OF CHILDREN'S TOYS AND ENGINEERS WITH TEN YEARS IN SAY IT'S THE FIRST TIME THE THING EVER ACTUALLY MADE SENSE 90 minutes, one table, a pile of Tinkertoys. No wall of jargon -- he builds a real Git repo out of plastic rods right in front of you. -> The moment he snaps the first pieces together, Git stops being scary command-line magic and becomes what it really is: a chain of tiny objects pointing at each other. Branches, merges, rebase, the staging area -- every concept that's ever burned you at 2am -- he rebuilds with toys until a four year old could follow. He calls Git a two-trick pony. After this you'll see exactly why. Memorizing commands was never the skill -> holding the graph in your head is. And with an AI agent now committing and rebasing on your machine all day, that mental model is the only thing between you and a history you can't read. Scroll the comments and you'll see the same thing over and over: this is the talk that finally made Git click and made people the one their whole team comes to when it breaks. Bookmark & watch it today. It's the 1.5 hours that pays you back for the rest of your career ↓

slash1s @slash1sol

2 months ago

MIT HANDED ITS DEEP LEARNING COURSE TO A FRONTIER-LAB ENGINEER FOR 68 MINUTES BECAUSE 90% OF PEOPLE SHIPPING AI CODE CAN'T EXPLAIN HOW THE MODEL ACTUALLY WORKS This is Maxime Labonne. He runs post-training at Liquid AI and wrote the LLM Engineer's Handbook. MIT gave him the room

35 71 690 237K 1K

You don’t need 10 years of experience to stand out in tech. You just need to master the fundamentals that 90% of people skip. Here are 7 fundamentals that will level up your career 👇

in 1988 Jack Crenshaw wrote Let's Build a Compiler because most compiler books were impossible for beginners to follow in 2017 Nora Sandler did something similar for C while working through Abdulaziz Ghuloum's paper on incremental compiler construction she adapted the ideas from Scheme to C the series starts with a compiler that can do exactly one thing return an integer from main then adds one feature at a time until it becomes a real compiler generating x86 64 assembly the entire series is free online it eventually became a 750 page No Starch Press book in 2024 the original blog posts are still available and still work as a step by step guide

Terence Tao says the math behind today’s LLMs is actually simple. Training and running them mostly uses linear algebra, matrix multiplication, and a bit of calculus, material an undergraduate can handle. We understand how to build and operate these models. The real mystery is why they work so well on some tasks and fail on others, and why we cannot predict that in advance. We lack good rules for forecasting performance across tasks, so progress is largely empirical. A key reason is the nature of real-world data. Pure noise is well understood, perfectly structured data is well understood, but natural text sits in between, partly structured and partly random. Mathematics for that middle regime is thin, similar to how physics struggles at meso-scales between atoms and continua. Because of this gap, we can describe the mechanisms but cannot yet explain capability jumps or give reliable task-level predictions. That mismatch, simple machinery versus hard-to-predict behavior, is the core puzzle. ---- Video from 'Dr Brian Keating' YT Channel (Link in comment)

Hanson's C Interfaces and Implementations remains undefeated for learning proper abstraction in C

9. Most people underestimate the power of doing something for decades. Compounding is not just financial. Skills compound. Relationships compound. Reputation compounds. Confidence compounds. The person who stays obsessed longest usually wins. Stick to something for at least 10 years.

4. Fear is why people stay stuck. Not logic. Not circumstances. Not lack of talent. Fear. And fear disguises itself beautifully: • “I’m being realistic” • “Timing isn’t right” • “Maybe later” • “I should play it safe” Fear rarely sounds dramatic.

In 2023, Stanford professor Graham Weaver gave his last lecture on how to destroy fear & live a wildly ambitious life. His frameworks: - Suffering is inevitable - Signup for "10 years" test - "Not me" & "Not now" traps 13 lessons on how to build an asymmetric life:

someone asked Beej how sockets work in C. he got tired of explaining it. so in 1995 he put it all online. it's been the definitive socket programming guide for 30 years. it covers everything: TCP, UDP, IPv4, IPv6, non-blocking I/O, select(), poll(). graduate OS courses worldwide assign it. it's funnier than any technical book has a right to be. it's free and always will be.

You can't engineer luck. Cleanest phrasing of P vs NP I've heard. NP is the magical computer that always tells you which path to take. P is what current silicon can do. Tetris is NP-complete. Chess is EXP-complete. MIT 6.006 Introduction to Algorithms, Fall 2011.

A Hungarian psychologist raised three daughters to prove that any child could become a chess grandmaster through early specialization. He succeeded. Two of them became grandmasters. One became the greatest female chess player who ever lived. Then a sports scientist looked at the data and found something nobody wanted to hear. His name is David Epstein. The book is called "Range." The Polgar experiment is one of the most famous case studies in the history of deliberate practice. Laszlo Polgar wrote a book before his daughters were even born arguing that geniuses are made, not born. He homeschooled all three girls in chess from age four. By their teens, Susan, Sofia, and Judit were dominating tournaments against grown men. Judit became the youngest grandmaster in history at the time, breaking Bobby Fischer's record. The story became the gospel of early specialization. Pick a domain young, drill it hard, and you can manufacture excellence. Epstein opens his book by telling that story honestly and then quietly demolishing the conclusion most people drew from it. Chess works that way. Most things do not. Here is the distinction that took him four years of research to articulate, and that almost nobody who quotes the 10,000 hour rule has ever read. There are two kinds of environments in which humans develop expertise. Psychologists call them kind and wicked. A kind environment has clear rules, immediate feedback, and patterns that repeat reliably. Chess is the cleanest example. Every game ends with a winner and a loser. Every move is recorded. The board never changes shape. The pieces never invent new ways to move. A child who plays ten thousand games will see most of the patterns that exist in the game, and pattern recognition is exactly what chess mastery is built on. A wicked environment is the opposite. Feedback is delayed or misleading. Rules shift. The patterns that worked yesterday may be exactly the wrong patterns to apply tomorrow. Most of the real world looks like this. Medicine is wicked. Investing is wicked. Building a company is wicked. Scientific research is wicked. Almost every job that involves a complex changing system with humans in it is wicked. The Polgar sisters trained in the kindest environment any human can train in. Their success was real and the method was correct. The mistake was generalizing the method to fields where the underlying structure of the environment is completely different. Epstein's research is what made the implication impossible to ignore. He looked at the careers of elite athletes outside of chess and golf and found that the pattern was almost the inverse of what people assumed. The athletes who reached the very top of their sports were overwhelmingly people who had played multiple sports as children, specialized late, and often switched disciplines well into their teens. Roger Federer played squash, badminton, basketball, handball, tennis, table tennis, and soccer before tennis became his focus. The kids who specialized in tennis at age six and trained year-round for a decade mostly burned out, got injured, or topped out at lower levels of the sport. The same pattern showed up everywhere he looked outside of kind environments. Inventors with the most patents had worked in multiple unrelated fields before their breakthrough work. Comic book creators with the longest careers had drawn for the most different genres before settling. Scientists who won Nobel Prizes were dramatically more likely than their peers to be serious amateur musicians, painters, sculptors, or writers. The skill that mattered in wicked environments was not depth in one pattern. It was the ability to recognize when a pattern from one domain applied unexpectedly in another. That kind of thinking cannot be built by drilling a single subject. It can only be built by accumulating mental models from many subjects and learning to move between them. The deeper finding is the one that should change how you think about your own career. Specialists in wicked environments often get worse with experience, not better. Epstein cites studies of doctors, financial analysts, intelligence officers, and forecasters showing that years of experience in a narrow domain frequently produce more confident judgments without producing more accurate ones. The expert builds elaborate mental models that feel comprehensive and turn out to be increasingly disconnected from the actual structure of the problem. They stop noticing what does not fit their framework. They mistake fluency for understanding. Generalists do better in wicked domains for a reason that sounds almost mystical until you understand the mechanism. They have less invested in any single mental model, so they abandon broken models faster. They are used to being a beginner, so they are not threatened by the discomfort of not knowing. They have seen enough different domains that they can usually find an analogy from one field that unlocks a problem in another. The technical name for this is analogical thinking, and the research on it is one of the most underrated bodies of work in cognitive science. The single most useful sentence in the entire book is the one Epstein puts almost as a throwaway. Match quality matters more than head start. A person who tries six different fields in their twenties and finds the one that genuinely fits them will outperform a person who picked one field at fourteen and stuck to it on willpower alone. The lost years were not lost. They were the search process that produced the match. Every field they walked away from taught them something they later imported into the field they finally chose. The reason this is so hard to accept is cultural, not empirical. We tell children to pick a path early. We reward the prodigy who knew at six. We treat the late bloomer as someone who failed to launch on time, when the data suggests they were running an entirely different and often more effective optimization process underneath. The Polgar sisters were not wrong. The conclusion the world drew from them was. If your environment is genuinely kind, specialize early and drill hard. If it is wicked, and almost every interesting human problem is, then the people who win are the ones who refused to specialize until they had seen enough to know what was actually worth specializing in. You are not behind. You were running the right experiment all along.

Every x86 chip you can buy in 2026 still has a microcode ROM inside the legacy decoder doing Wilkes-style horizontal microprogramming. Foster wrote that chapter in 1970. It is still correct.

A professor at MIT spent his life studying uncertainty. Near the end, he compressed everything into a single one-hour lecture. No buzzwords. No heavy theory. Just a clear explanation of how prediction really works. Not long after, he was gone. This is that talk. The idea at its core is simple but powerful: prediction isn’t about being certain it’s about understanding probabilities Most people will scroll past it. A few will see it and start thinking differently. Save it.

A Stanford CS professor told his class something at the start of the semester that made half the students close their laptops. He said the skill that will separate the people who thrive in the next decade from the people who stall has almost nothing to do with coding. His name is Andrew Ng, and he has trained more machine learning engineers than almost anyone alive. Here is what he said, and why it changes how you should be learning right now. He said the bottleneck is no longer writing code. It is knowing which problems are worth solving in the first place. For thirty years, being a good engineer meant being able to build what someone else defined. In the world that is arriving, every engineer has infinite leverage to build almost anything, which means the person who picks the right thing to build now wins by orders of magnitude over the person who builds the wrong thing flawlessly. His framework for problem selection is deceptively simple. He calls it the three-question filter. The first question is whether the problem you are working on actually matters to someone who would pay for it or use it daily. Most students fail here. They work on projects that are interesting to them and nobody else, and then wonder why the portfolio produces no offers. The second question is whether the problem is still hard now that AI exists. If a single prompt to a hosted model solves it, the problem is no longer valuable to solve yourself. The interesting problems live in the gap between what AI can do alone and what it can do when combined with domain knowledge, careful system design, and data nobody else has access to. The third question is the one most people skip. Can you actually ship a working version in a week. Not a polished version. A crappy, embarrassing, actually-functional version. Ng said the number one predictor of which of his students ended up building something important was not talent. It was the willingness to ship something bad fast and then improve it in public. He said the students who kept tweaking in private for six months before showing anyone almost always produced worse final work than the students who shipped a broken version on week one and iterated based on real feedback. The people who are actually winning right now are not the ones with the best ideas. They are the ones who learned to pick problems that matter and ship solutions that barely work, before anyone else has even finished thinking about it.