Aidan Pak @AidanPak

@evisdrenova This is the case for intelligent routing between closed and oss models. @OpenRouter is great and I also think @databricks has a strong case to own routing for large enterprise

@n0w00j

It's fascinating to see how frontier capabilities seem to be first unlocked via the harness, but over time, that behavior gets inevitably baked into the weights of the model and the scaffolding around it evolves. For example, in the early days, agent frameworks wrapped LLMs in pre-defined DAGs with dedicated planning, context management, and tool-calling steps to overcome limitations in the base model such as hallucinating tool arguments, forgetting instructions, and context anxiety. These encoded assumptions quickly became stale as reasoning models were introduced and quickly improved, but importantly, that didn't mean the infrastructure around the model went away. What's super interesting about SPIRAL is that it's research on how to encode the current SOTA behavior of a harness (e.g. Claude Code dynamic workflows / loops) directly into the weights. Specifically, training a model to spin up parallel reasoning traces and then synthesize them into a final answer (which is one of the multi-agent architectures that is prevalent with dynamic workflows, e.g. fan-out-and-synthesize) but instilled end to end via RL rather than in the harness. I have been incredibly impressed by the capabilities unlocked by dynamics workflows. This is where Claude writes the orchestration as a JS program and then runs that code. The script holds the loop, the branches, and the intermediate results, fans out a pile of subagents in parallel, and a synthesizer folds them back into one answer. Other multi-agent patterns like tournaments, adversarial verification, and generate-and-filter are also prevalent. This behavior is currently a part of the harness but might soon be embedded in the weights.

Jubayer Ibn Hamid @jubayer_hamid

5 days ago

The most capable reasoning systems in AI scale inference compute along several axes: sequential compute to think longer, parallel compute to sample many independent attempts, and aggregative compute to synthesize prior traces into a new improved one. But during training, we only

15 89 413 169K 472

@StockSavvyShay RIP consumer electronics

Great post! I think the next step for "code review" looks less like “review” and more like code validation / dynamic verification. It will be less statically reading a diff / running tests, and more simulation where agents drive an environment, run a feature against its real dependencies, check how it behaves against other services, and adversarially try to break it. The problem today is that productizing this is incredibly hard: (1) dev environments are custom to each enterprise, and (2) the cost of compute to run them at scale is still too high. I believe @cognition is heading in this direction

@JakeEpstein1 Super cool stuff Jake! Offloading as much work as possible to small, local OSS models will be increasingly important. Would be great to add in a smart router / BYO keys to route between frontier / local models

@leylndd @GeoffreyHuntley VCs when Sequoia leads the A

Longer-running agents are creating enormous KV cache that has to be moved around: offloaded across storage tiers, shared between nodes, etc. The reason CPU demand is exploding isn't that CPUs are replacing GPUs (Matmul still belongs on the GPU) which is purpose-built for it. It's that CPUs are increasingly the ones moving all that KV and data around to keep the GPUs fed along with the actual computing required to execute tool calls, navigate a browser, etc.. It's why NVIDIA's NVL72 rack pairs the GPUs with Grace CPUs.

@n0w00j 🤣

@Jason I've heard from a source close to a Mag 7 CEO that the government is quietly pressuring large companies (Microsoft, Meta, etc.) to avoid mass layoffs. Many leaders obviously see an incredible opportunity to optimize their headcount given AI but being forced to slow it down

The utilization number you're referring to here is GPU MFU (model FLOPs utilization), which isn't a measure of whether the GPU has actually been rented out. There are no "dark" GPUs. You can see that in the price of renting the same chip (especially older generations like the A100) continuing to climb year after year. What MFU measures is how much of a GPU's theoretical compute the workload actually uses at runtime, and the answer is in that 11-45% range. Think of it as essentially the percent of time the GPU cores are doing real math versus sitting idle waiting for work, and in today's AI workloads, the core sit idle waiting on data to be moved between chips (networking bottleneck). This is especially brutal in training, where the whole job runs at the speed of the slowest GPU in the cluster. Every iteration in training requires whats called an "all-reduce," where the entire fleet has to exchange information before the next step can begin, so one straggler stalls every other chip until it catches up. A cluster pinned at 30% MFU can still be 100% leased. 

@alexch3n @n0w00j please setup all engineers w a full clutch + transmission to shift between worktrees.

@signulll I've also heard from a source close to a Mag 7 CEO that the government is quietly pressuring large vendors selling into federal (Microsoft, Meta, etc.) to avoid mass layoffs. Many leaders obviously see an incredible opportunity to optimize their headcount given AI.

Anthropic added more ARR in the past month than Snowflake, Palantir, and Databricks have added combined in their entire existence (three of the most influential companies of the SaaS era). Anthropic has roughly 1/5th the headcount of Databricks alone...

Agree that open models can probably subsume 80-90% of tasks over time (especially now with post-trained OSS >= closed-source models for a specialized task). However, I still believe 80-90% of total spend will go to the frontier as the economic value will continue to be wildly skewed toward the model that solves the hardest problems.

@rdd147 Uber blew through its 2026 token budget in 3 months as well. Incredibly bullish for OSS inference providers like @baseten to capture this internal spend

@djfarrelly Great post! The harness should ultimately sit outside the sandbox

@trq212 @NotionHQ and while we're at it @NotionHQ , can you also embed more voice features? Would be great for my car rides if I could just listen to a dictation of all my AI-generated, imported markdowns or have it generate a podcast for me using elevenlabs.

@trq212 @NotionHQ can you pls implement embeddable HTML blocks for cool, interactive visualizations? Even better if u can edit the the block directly in-app instead of needing to edit the raw HTML. 🙏