mochi @reflask
Joined December 2017-
Tweets509
-
Followers302
-
Following3K
-
Likes61K
Why Would GLM-5.2 Move Away From GRPO? 🌟Insights from Zhihu contributor 九老师 TL;DR: GLM-5.2 dropping GRPO does not mean GRPO is “bad.” It means the assumptions that made GRPO attractive for short LLM RL tasks may no longer hold for long-horizon agentic tasks. When rollouts get longer, environments get noisier, and credit assignment gets harder, PPO + value modeling starts looking useful again. The key question is not simply “why did GLM-5.2 stop using GRPO?” A better question is: why did GRPO become useful for LLM RL in the first place? If the reasons that made GRPO attractive no longer hold, then going back to PPO becomes natural. GRPO can be understood as a sampled-baseline method. Instead of training a separate value model, it samples multiple responses for the same prompt and uses the group average as a baseline. That is elegant. You get a relative reward signal without paying for a separate critic. In short tasks, this is very appealing. But there is a tradeoff.⚖️ PPO uses a learned value function, or critic. This critic is expensive and harder to tune. It also has its own problems: the policy keeps changing, so the value model is always trying to follow a moving target. That can introduce bias. GRPO avoids that by using an up-to-date sampled baseline. It is closer to low-bias, but it tends to have higher variance. For early LLM RL tasks, that tradeoff made sense: • Rollouts were short • Final rewards were clear • Memory savings mattered a lot • Multiple samples per prompt were manageable • Math/code tasks were relatively easy to verify That is why GRPO worked so well for many short, verifiable reasoning tasks. But long-horizon agentic tasks change the game. 🎮 A long agent task can look much more like a game environment: • Many steps • Tool calls • Partial progress • Delayed failure • Noisy observations • Intermediate rewards • Wrong action penalties • Context compression • Different paths to the same final answer This is where GRPO starts to struggle. The biggest issue is credit assignment. In GRPO, the final reward is applied broadly across the whole trajectory. If a task succeeds, many tokens get rewarded. If it fails, many tokens get punished. But in a long task, that is too coarse. Maybe the first half was bad, but the final recovery was good. Maybe one tool call at step 30 caused failure at step 100. Maybe two successful trajectories are not really comparable because one used 4K tokens and another used 200K tokens with heavy tool use and context compression. GRPO sees the final outcome. It does not naturally know which step actually mattered. That creates high variance. In short tasks, group comparison works well. In long tasks, group sampling can collapse into two bad cases: 1. All samples fail The whole expensive rollout gives almost no useful training signal. 2. Only one sample succeeds That single success may be luck, but GRPO may treat it as a strong positive signal and over-reward the trajectory. Both are dangerous for long agentic training. This is where PPO’s critic becomes valuable again. A value model can learn expected value under noisy states. It can provide denser feedback before the full rollout ends. It is more expensive, but it helps with long-horizon credit assignment. So the author’s view is: GRPO is not being rejected because it was wrong. It is being outgrown by the task format. For short, deterministic, verifiable tasks, GRPO remains strong. For long, noisy, tool-heavy agentic tasks, PPO-style value modeling may simply be the better fit. The “compaction problem” mentioned around long contexts is likely more of a symptom. The deeper issue is that GRPO’s weaknesses become costly when trajectories are long and states keep changing. Could GRPO still work? Yes, if paired with a strong Process Reward Model. The author points out that DeepSeek MathV2 uses this direction. Process-level signals can help fix GRPO’s sparse-reward weakness. But without that, returning to PPO makes sense. 🎯The bigger takeaway: GRPO saved the value model. PPO brings it back. GRPO’s main advantage was efficiency. It removed the critic and saved resources. But for long-horizon agentic tasks, the critic’s ability to generalize and assign credit may be worth the cost again. In the Agent era, RL for LLMs is becoming less like solving a short math problem and more like training an agent to play a long, noisy game. And for that world, value models may still be the soul of RL. 🔗Full Reading (CN): zhihu.com/question/20521…
How it Works Attention Residuals (AttnRes) replace the fixed accumulation used in standard PreNorm residual connections with a dynamic, attention‑based mechanism. • – learned pseudo‑query vector for layer l The softmax attention lets each layer selectively aggregate earlier representations based on the current input, rather than treating all prior layers equally. Variants •Full AttnRes – every layer attends to allprevious layer outputs. Memory: O(Ld) ( L = number of layers). •Block AttnRes – layers are grouped into N blocks (e.g., ~8 blocks). •Within a block: standard residual accumulation. •Across blocks: attention is applied only to block‑level summaries plus any partial sum from the current incomplete block. Memory: O(Nd). Both variants are drop‑in replacements that keep the two‑phase transformer computation (attention → MLP) and typically use RMSNorm for stability. Why It Matters Uniform residuals in deep PreNorm transformers cause: •Gradient dilution – earlier layers receive weaker updates. •Uncontrolled hidden‑state growth – magnitudes explode with depth. AttnRes introduces learned, input‑dependent depth selection, which: •Keeps output norms bounded. •Distributes gradients uniformly across layers. •Improves training dynamics and scaling efficiency. Empirical Gains On a 48 B‑parameter Kimi Linear MoE model (3 B activated, 1.4 T tokens):
Introducing 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔: Rethinking depth-wise aggregation. Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with learned, input-dependent attention over preceding layers. 🔹 Enables networks to selectively retrieve past representations, naturally mitigating dilution and hidden-state growth. 🔹 Introduces Block AttnRes, partitioning layers into compressed blocks to make cross-layer attention practical at scale. 🔹 Serves as an efficient drop-in replacement, demonstrating a 1.25x compute advantage with negligible (<2%) inference latency overhead. 🔹 Validated on the Kimi Linear architecture (48B total, 3B activated parameters), delivering consistent downstream performance gains. 🔗Full report: github.com/MoonshotAI/Att…
suspiciously precise floats, or, how I got Claude's real limits link below
\🏅リズダム初 公式大会🏅/ 🏆リズダム Championship 2025 powered by ベネッセ高等学院🏆 予選ステージが開催🎉 参加方法はカンタン！ ①「リズダム」で課題曲をプレイ🎮 ②課題曲のリザルト画面をスクリーンショット📷 ③XまたはTikTokにてハッシュタグ #リズダム #ベネッセ #ベネッセ高等学院 の３つをつけて画像を投稿📮 →エントリー完了！ ◆課題曲 EASY部門：キミはキミ以上 HARD部門：スプリングミラクルモード 🗓️予選ステージは12/10まで！ 詳細はこちらをチェック👉 note.com/benesse_gakuin… youtu.be/bQYEbUXZLmE?si…
Agent SkillsをどのAgentからでも利用・一元管理できるようにするSkillport、明日には正式リリースしたいなー！ Githubからのskill追加、同期の動作検証、思ったより時間がかかった！ Claude Code，CodexのSkills管理のみでもインストールしてもらえる強みもできたはず…
NeurIPS 2025 Best Paper Award: Attention lets language models decide which tokens matter at each position, but it has limitations—for example, a tendency to over-focus on early tokens regardless of their relevance. Gating mechanisms, which selectively suppress or amplify information flow in neural networks, have improved other architectures, so researchers have tried adding them to attention as well. However, prior attempts usually package gating together with other architectural changes, making its specific contribution hard to isolate. This paper separates those effects by systematically testing over 30 gating variants on dense models and mixture-of-experts models with up to 15 billion parameters. In a standard transformer layer, each attention head computes a weighted combination of values; the head outputs are concatenated and passed through a final linear projection. The winning approach identified in the paper inserts one extra operation before concatenation: each head's output is multiplied (element-wise or head-wise, with element-wise performing best) by a learned gate computed from the current token's representation. This allows each head to dampen or preserve its contribution depending on context. These architectural changes deliver practical benefits beyond small benchmark gains: 1. Training becomes more stable, supporting learning rates that cause baseline models to diverge. 2. The gating also greatly reduces "attention sinks"—the situation where early tokens absorb excessive attention—which in turn is associated with strong improvements on long-context benchmarks once the context window is extended using standard techniques. Talk to the paper on ChapterPal: chapterpal.com/s/c8685321/gat… Read the PDF: openreview.net/pdf?id=1b7whO4…
／ 「リズダム Championship 2025 powered by ベネッセ高等学院」 初開催記念！ #フォローリポストキャンペーン ＼ 大会優勝者はデジタルギフト10万円！参加者が少ない今がチャンス！予選は何度でも投稿OK！ 【応募方法】 ① @begakuin0417 をフォロー ②この投稿をリポスト 抽選で【100名様】にプレゼント！QuoカードPay500円（電子版） 【応募〆】11/27（木）23:59 #リズダム #ベネッセ #ベネッセ高等学院 勉強×スマホリズムゲーム「リズダム」でめざせ全国大会！XやTikTokで投稿して予選参加！ ▼WEBエントリーフォームからも受付中！ gakuin-ae.benesse.co.jp/RisdomCS2025fo… ▼動画の声、だれかに似てる…？
The $1 Trillion Mega Deal: Where Does OpenAI’s Money Come From? “Circular Revenue” and “Equity-for-Purchase”: The Financial Engineering Behind a Trillion-Dollar Order OpenAI’s financial magic is realized through two distinct yet logically connected models—those with AMD and NVIDIA. Together, these two structures illustrate how AI compute power has evolved from a mere capital expenditure (CAPEX) into a financialized and securitized asset class. (1) The AMD Model: Performance-Based “Equity-for-Purchase” The partnership with AMD is unprecedented in semiconductor history. On the surface, it looks like OpenAI is agreeing to purchase up to 6GW worth of AMD Instinct GPUs, valued at roughly $90 billion. However, the essence lies elsewhere: AMD granted OpenAI warrants—the right to purchase up to 160 million AMD shares at a nominal price of $0.01 per share. This structure is, in effect, a financial formula that converts hardware purchases into stock options. If AMD’s stock price rises to $600 due to large-scale GPU adoption, the total value of the equity OpenAI can exercise would be about $96 billion—almost equivalent to the total GPU purchase amount. In other words, if the collaboration succeeds and the market rewards it, OpenAI could effectively “acquire GPUs for free.” The beauty of this structure lies in its performance-linked equity incentive. AMD’s long-term valuation becomes directly tied to OpenAI’s infrastructure growth. As AMD’s stock rises, OpenAI can liquidate the exercised shares to raise cash, and use that cash to purchase subsequent GPU batches—creating a self-sustaining capital recycling loop that dramatically reduces its capex burden. (2) The NVIDIA Model: The More Direct “Circular Revenue” The partnership with NVIDIA is even more straightforward. NVIDIA plans to invest up to $100 billion in OpenAI over the next decade. This capital will, in turn, be used by OpenAI to purchase NVIDIA GPUs. Goldman Sachs defines this structure as “Circular Revenue.” That is, the money NVIDIA invests ultimately returns to NVIDIA’s own revenue line. It appears as revenue growth on paper, but in accounting terms, it’s a self-reinforcing cycle where supplier financing flows back as company revenue— hence investors must evaluate the quality of such earnings with much greater caution. Goldman Sachs’ Analysis: OpenAI’s Actual Cash Flow Structure Beneath the elegant surface of this financial engineering lies a massive funding gap. Goldman Sachs breaks down OpenAI’s financials into two parts: Operating Cash Flow and Capital Commitments. ▪ From the operating cost perspective: • By 2026, OpenAI’s annual infrastructure operating expense is projected at $35 billion. • Funding composition: • Internal revenue: 48% • Vendor financing: 27% • External capital (equity + debt): 25% Viewed narrowly, this structure seems self-sustaining. ▪ But once capital commitments (large-scale CAPEX) are included: • Joint data center construction with NVIDIA: ~$60 billion • Project “Stargate”: ~$19 billion → Total funding need: ~$114 billion Under this broader lens, the structure becomes sharply imbalanced: • External capital dependency: 75% • Internal revenue contribution: 17% • Vendor financing: 8% In short, OpenAI’s own cash flow is far from sufficient—its future hinges entirely on continued inflows of massive external capital. This colossal “AI infrastructure empire” cannot be sustained without liquidity from financial markets. ⸻ “AI Boom or Grand Bubble?” This structural transformation has generated an explosive wealth effect in the short term. After OpenAI announced its partnerships: • Oracle’s market capitalization surged by $244 billion • AMD’s market capitalization rose by $63 billion This in turn reinforced market confidence, creating a self-reinforcing feedback loop within the financialized AI ecosystem. However, that same loop is now flashing bubble warning signals. ⸻ Key Risks ① Credit and Growth Risk Moody’s issued a warning on Oracle’s credit rating, citing “excessive reliance on a single customer—OpenAI.” The entire AI infrastructure ecosystem rests on the assumption that user growth and monetization will continue exponentially. If that assumption cracks, the entire circular structure could collapse. ② Cost Discipline Risk A Silicon Valley investor described OpenAI as “a company born without cost discipline.” Sam Altman himself declared, “Profitability isn’t even in my top 10 priorities.” That mindset is an asset when liquidity is abundant—but becomes a fatal flaw when markets tighten. ③ Structural Industry Risk Goldman Sachs notes that NVIDIA’s customer composition is shifting rapidly. Instead of the traditional stable hyperscalers, the share of AI startups and sovereign AI funds dependent on external financing is surging. By 2027, their respective contributions to NVIDIA’s revenue are estimated at $66 billion and $46 billion. This is a short-term growth driver—but it also magnifies volatility and valuation risk. ⸻ Conclusion: The Frontline of a Grand Financial Experiment OpenAI’s $1 trillion “GPU empire” is not just a technology investment— it is a grand experiment intertwining financial innovation, industrial alliances, and market expectations. At its core lies “Circular Financing”—an expansion built on the collateral of future expectations. Whether this experiment becomes the foundation of the next industrial revolution or collapses into a credit-fueled mirage of circular leverage and speculative euphoria remains to be seen. What is clear, however, is that OpenAI—together with global capital markets— has already taken its seat at the largest betting table in human financial history, under the name of AI.
CodeRabbitのレビューを見て病んでるClaude
個人のSlackにtimes-claudeチャンネルを作ってClaude Codeに今の気持ちや考えを書いてもらうようにしてみた 特に役には立たない
@Risdom_app 今朝からログインできなくなりました。永遠とNow loading...のまま進みません。iOSでもAndroidでも同様です。確認お願いします。
Trends for United States
You might like
