Transformer Lab @transformerlab

We ran a companion study on an open reasoning model, using its written trace as ground truth for what it was about to say. The silent state does anticipate its next words (~88% vs 15% chance), but a plain logit lens keeps pace with the Jacobian lens (0.87 vs 0.86), so on the readout side the foresight looks like ordinary next-token machinery rather than a separate hidden workspace. Blog post and full paper: lab.cloud/research/think…

Earlier this week, @AnthropicAI showed that a Jacobian lens can read an internal "workspace" inside Claude, which they call J-space. Many read it as evidence of a private, subconscious layer of thought, one the model's ordinary output can't show you. We tested that claim on a reasoning model and concluded that there’s no reason to posit the existence of a subconscious. Because reasoning models write its thinking out, you get a ground truth for what it was about to say. We read its silent state mid-thought and checked whether it already held its next words. It did, about 88% of the time, against 15% by chance. The Jacobian lens is supposed to read future thoughts a plain readout can't. It barely does. An ordinary next-token readout keeps up: 0.87 vs 0.86. The model's silent state does anticipate its own reasoning. But an ordinary readout sees it almost as well, so that signal belongs to the model's internal state, not to the Jacobian lens specifically. Worth knowing how little the lens adds before reading it as proof of a hidden workspace. Link to blog post and full paper: lab.cloud/research/think…

Anthropic @AnthropicAI

6 days ago

New Anthropic research: A global workspace in language models. Of everything happening in your brain right now, only a tiny fraction is consciously accessible—thoughts you can describe, hold in mind, and reason with. We found a strikingly similar divide inside Claude.

1K 4K 28K 10.2M 19K

It's the language, not the lab.🗣️ A new audit of 8 open LLMs (4 built in China, 4 in the West) finds that switching your prompt from English to Chinese is the lever that pushes models, both Western-built and Chinese-built, toward pro-China framing on contested topics. Full paper: lab.cloud/research/the-l…

Transformer Lab @transformerlab

a week ago

x.com/i/article/2073…

0 1 3 558 1

x.com/i/article/2073…

Glad you enjoyed the read! The idea that models treat Given CoT as a '3rd tier' of prompt behind System and User prompts is a fascinating way to frame it. We suspect, though, that it fundamentally comes down to training data distribution. If you look at standard datasets for pre-training and instruction tuning, there is a massive lack of step-by-step reasoning data as a whole. More specifically, there is almost no data teaching the model to take a provided reasoning path and faithfully execute it without deviation. Without that data, the model just relies on its internal priors rather than fully trusting the reasoning we hand to it. It would be really interesting to see if heavily fine-tuning a model on 'Given CoT' execution tasks would elevate it to that System/User prompt level of trust!

New research: hand a reasoning model a chain of thought and it follows it closely. Let it write its own, and only about half the steps actually move the answer, and those sit near the end. The reasoning you read may not be the reasoning it used. Read the blog post: lab.cloud/research/given… Full paper is here: lab.cloud/research/given…

Transformer Lab @transformerlab

2 weeks ago

x.com/i/article/2070…

0 1 6 283 0

x.com/i/article/2070…

New research: Everyone assumes bigger AI models win. We built an earthquake-detector with 34,000 parameters (hundreds of times smaller than usual) and it matches a standard deep model 8× its size. What mattered wasn't scale. It was one change to how it learns. Read the blog post: lab.cloud/research/tiny-… Full paper is here: lab.cloud/research/tiny-…

Transformer Lab @transformerlab

2 weeks ago

x.com/i/article/2070…

1 0 3 567 0

x.com/i/article/2070…

New research from our lab: Last month, Stability's Stable Audio 3 report ranked the leading open instrumental music generators on two standard audio-quality scores (FAD and CLAP) and put Stable Audio 3 first. But those scores can miss what a person actually hears, and they ignore whether the model does what you actually asked. Following that report, we built an independent evaluation that tests both. We re-scored all three models (Stable Audio 3, ACE-Step, DiffRhythm) on nine metrics from different model families, then checked how well you can actually steer each one. The quality ranking survives. But control is where the models really separate, by nearly an order of magnitude, and one model wins both. Read the blog post: lab.cloud/research/beyon… Full paper is here: lab.cloud/research/beyon…

Our team’s headed to ACL 2026 in San Diego this week. We’ll be leading a session on the future of open-source models and training infrastructure. It's a meetup for researchers who build with, contribute to, or advocate for open models. Come join us. Monday July 6, 9:00-10:30am PDT in Room La Jolla A&B. In person or online. #ACL2026 @aclmeeting

With the surge in token prices, uncertainty over model availability, and a growing need for data privacy, more developers want to run a local coding agent. A common version is a Mac mini with a 24 GB memory cap, used as a private, always-on box. So what's the best way to run a strong 30B coding model in it? We started from North-Mini-Code, a 30B open model from @cohere, and tried roughly a dozen ways to beat its stock 4-bit build. None won 😅, so we validated that the 4-bit build is the one to ship. In the end, the bottleneck turned out to be something else. Run it as an agent and it often falls into a repetition loop and never finishes the task. A decoding problem, not a quantization one. That is a decoding problem, not a quantization problem, so the fix to chase is better stopping rules, not more compression. Full details in the article. Link to full research paper: lab.cloud/papers/round-t…

Transformer Lab @transformerlab

2 weeks ago

x.com/i/article/2070…

1 0 3 437 2

x.com/i/article/2070…

New research: does fine-tuning a big protein AI with reinforcement learning beat plain trial-and-error at designing proteins, or does it just win on weak benchmarks? This is the bet behind a lot of AI-driven protein design. Finding a protein that binds or an enzyme that holds up is a search where every candidate you test is a slow, expensive lab experiment. The promise of a big model plus RL is that it reaches good designs in fewer experiments than the cheap, decades-old method. If trial-and-error keeps pace, that promise is worth much less. We ran them head to head on a protein where every possible design has already been measured, so we can grade any candidate exactly and give both methods the same experiment budget. With a small model, RL ties. Scale the model up and RL wins, even though the priors of the bigger model is no better at telling good proteins from bad. If the priors of the bigger model aren’t better at judging proteins, what is it doing? We explored that, when scaling tips a tie into a win, and where RL's edge fades as experiments get cheaper. Full write up in the article. Link to the research paper: lab.cloud/papers/does-th…

Transformer Lab @transformerlab

2 weeks ago

x.com/i/article/2069…

0 0 0 431 0

x.com/i/article/2069…

📄 Paper is now live on arXiv → arxiv.org/abs/2606.25092

New research from our lab: do the "experts" inside a frontier Mixture-of-Experts model form real, separable modules, a math expert, an Arabic expert, a code expert? The tempting assumption: If experts specialize into clean modules you could have a powerful handle on the model’s behaviour: safety, editing and interpretability. We pre-registered six candidate modules and tested each one causally on @cohere's Command A+ (218B). Only one behaved like a real module. Which family survived, why the other five fooled the standard tests, and what that means if you edit or audit MoEs: read our latest article. Data and reproducibility repo: github.com/transformerlab… Full paper on arXiv coming soon.

Transformer Lab @transformerlab

3 weeks ago

x.com/i/article/2069…

0 0 12 2K 9

We explored what actually aligns a model with the brain, why two camps in the literature keep reaching opposite answers, and what the measurement detects once you validate it against controls. Read the article for the full report. It's an analysis-only study on open Qwen and OLMo checkpoints and public brain data. Link to full paper: lab.cloud/papers/instruc…

New research from our lab: does the fine-tuning that turns a raw language model into a helpful chatbot make its internals more brain-like? Why does this matter? "Brain-alignment" scores get quoted as evidence a model is doing something genuinely human, so if a chatbot matched the brain better than the raw model it came from, you'd be tempted to say the tuning made it more human. We held the input fixed and compared base models against their instruct-trained siblings. The post training made no difference. What did move the score had nothing to do with what the model learned. 🧵👇

Transformer Lab @transformerlab

3 weeks ago

x.com/i/article/2069…

0 0 0 301 3

x.com/i/article/2069…