Logan Yang @logancyang

Claude Opus 4.8 in CC is so dumb today wtf 🤦‍♂️

The first wave of AI-native applications is wrapping tokens and providing in-app agents. As agent usage centralizes around core apps (e.g. Claude Code, Codex), there's this emerging wave of building software that doesn't need to have AI on its own, but is extremely easy to plug into whatever the leading AI app is. Open question on which patterns will win out

Ankur Goyal @ankrgyl

3 weeks ago

there's still a lot of "ai software" that needs to be reengineered around the paradigm of agents, from scratch

21 2 65 19K 6

@jerryjliu0 Wrapping tokens and in-app agents don't make much sense anymore as harness + model is where the real intelligence is. It's also more expensive for both the app owner and the consumers. Real "agent-native" apps should just be plugged into the harnesses consumers already use.

@JustJake been refreshing status.railway.com for 2h

@Railway Come on guys how long has it been, everything is nuked

@Railway All my projects are down, seriously losing $$$

@thekitze Completely agree with 2, and I'm working on something to make this process easier.

In good hands There is a feeling I search for: being in good hands. It is the feeling I look to give and the feeling I look to receive. I know I am in good hands when I sense a cohesive point of view expressed with attention to detail. I can feel it almost instantly. In any medium. Music, film, fashion, architecture, writing, software. At a Japanese restaurant it's what omakase aims to be. I leave it up to you, chef. When I am in good hands I open myself to a state of curiosity and appreciation. I allow myself to suspend preconceived notions. I give you freedom to take me where you want to go. I immerse myself in your worldview and pause judgement. I want to be convinced of something new. I want my mind to be changed. Later I may disagree, but for now I am letting the experience soak in. That trust doesn't come easily. As an audience member it's about feeling cared for from the moment I interact with your work. It's about feeling a well-defined point of view permeate what you make. If my mind was changed, I must have been in good hands.

🚀 Want FREE models you can plug into OpenClaw or Hermes? Here are 9 resources you can use for free access to model APIs No local setup, no credit card, just pure cloud APIs with OpenAI-compatible endpoints You can’t get free Opus quality (yet) but all of these have genuine free tiers right now (rate limits may apply) and are good enough to get started if you don’t want to spend $ to get started with agents 1️⃣ OpenRouter Free Models (Gemma 4 31B/26B, NVIDIA Nemotron 3 Super 120B MoE, MiniMax M2.5, Qwen3 variants, Llama 4/3.3, gpt-oss-120B, Arcee Trinity, etc.)
• ~29 completely free $0/M token models
• Insane variety + top-tier open model evals (especially coding & agents)
• Best for rotating models automatically 👉 Sign up: openrouter.ai/keys 2️⃣ Google Gemini API (Gemini 2.5 Pro / Flash series)
• Strongest overall free frontier model
• Excellent multimodal, 1M+ context, native tool calling & agentic performance
• Very generous free limits (often 5–15 RPM)
👉 Sign up: aistudio.google.com/app/apikey 3️⃣ NVIDIA (Nemotron variants, Llama 3.3 70B, Qwen3 235B, Mistral Large, etc.)
• Optimized high-performance open models
• Free prototyping tier (~40 RPM)
👉 Sign up: build.nvidia.com/explore/discov… 4️⃣ Grok Cloud (Llama 4 Scout, Llama 3.3 70B, Qwen3 32B, gpt-oss models, etc.)
• Blazing-fast inference (hundreds of tokens/sec)
• Perfect for real-time agents
• Strong open-model performance with solid free tier
👉 Sign up: console.groq.com/keys 5️⃣ Cerebras Cloud (Qwen3 235B, Llama 3.3 70B, DeepSeek variants, etc.)
• Massive models with excellent reasoning/coding evals
• Very generous daily free limits (~30 RPM, up to 1M+ tokens/day on some)
👉 Sign up: cloud.cerebras.ai 6️⃣ Mistral La Plateforme (Mistral Large 3, Small 3.1, Ministral 8B, etc.)
• Strong in coding, multilingual & agentic tasks
• Solid free tier (~1 req/s, ~1B tokens/month)
👉 Sign up: console.mistral.ai/api-keys 7️⃣ Cohere (Command A, Command R+, Aya Expanse 32B, etc.)
• Free tier: 20 RPM, 1K requests/month
👉 Sign up: dashboard.cohere.com/api-keys 8️⃣ GitHub Models (Llama 3.3 70B, DeepSeek R1, some GPT-4o previews, etc.)
• Decent mid-tier evals with easy GitHub integration
• Free tier limits (10–15 RPM)
👉 Sign up: github.com/marketplace/mo… 9️⃣ Cloudflare Workers AI (Llama 3.3 70B, Qwen QwQ 32B, etc.)
• Lightweight but solid for simple agents
• Free tier: 10K neurons/day
👉 Sign up: dash.cloudflare.com/profile/api-to… Pro tips for agent builders: • Most work instantly with OpenAI SDK (just change base URL + your key) • Start with OpenRouter for quality/variety (they often feature new free models) • Add Groq as speed fallback • Rotate providers when you hit caps Free intelligence for your agent is just a signup away!

@Internethunts @themetafranz @joethei @obsdmd So why did you?

Another one joins the "training on your data unless opt out" party. In the latest video I talked about the privacy levels of providers. Most people won't care but for anyone who still cares about privacy, consider running your own AI stack.

free growth strategy: 1. keep improving little by little 2. stay 100% user-supported 3. watch VC-backed companies gradually destroy their product and alienate their users

@dhuynh95 @steipete Nice stuff! I took a quick look and seems each call would be 5s slower? "Each call spawns a fresh TUI process (~5s startup overhead)"

after turboquant and qwen3.5-35b-a3b, i got curious: how realistic is it to use kv cache as a document store today? to have vectorless, RAG-less search. so i prefilled 258K out of 262K context window on L4 (a budget GPU popular in prod). ~99% of the slot is pre-computed and stored, users load it on the fly in ~1s. system prompt + query append to the end, generation takes ~3K tokens, enough for search. at 99% fill rate, decoding runs ~20 tps on L4. i prepared some ego datasets (jina papers, which i know best), plus popular novels in chinese and english. the results are actually pretty good. some hallucination, but most answers are solid and well-grounded. what's more interesting is the cost: ~$0.26/h on L4 spot. single LLM. no vector database, no embedding model, no workflow/pipeline engineering. using kv cache as document store is nothing new, like the old CAG paper. but with quantized kv cache and modern attention (hybrid SSM-attention, GQA, MQA, MLA), the economics are changing fast. if we solve cold-prefill speed and decoding speed, and budget GPU costs keep dropping, the future of search could be vectorless. radical, but possible.

Software horror: litellm PyPI supply chain attack. Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database passwords. LiteLLM itself has 97 million downloads per month which is already terrible, but much worse, the contagion spreads to any project that depends on litellm. For example, if you did `pip install dspy` (which depended on litellm>=1.64.0), you'd also be pwnd. Same for any other large project that depended on litellm. Afaict the poisoned version was up for only less than ~1 hour. The attack had a bug which led to its discovery - Callum McMahon was using an MCP plugin inside Cursor that pulled in litellm as a transitive dependency. When litellm 1.82.8 installed, their machine ran out of RAM and crashed. So if the attacker didn't vibe code this attack it could have been undetected for many days or weeks. Supply chain attacks like this are basically the scariest thing imaginable in modern software. Every time you install any depedency you could be pulling in a poisoned package anywhere deep inside its entire depedency tree. This is especially risky with large projects that might have lots and lots of dependencies. The credentials that do get stolen in each attack can then be used to take over more accounts and compromise more packages. Classical software engineering would have you believe that dependencies are good (we're building pyramids from bricks), but imo this has to be re-evaluated, and it's why I've been so growingly averse to them, preferring to use LLMs to "yoink" functionality when it's simple enough and possible.

Daniel Hnyk @hnykda

3 months ago

LiteLLM HAS BEEN COMPROMISED, DO NOT UPDATE. We just discovered that LiteLLM pypi release 1.82.8. It has been compromised, it contains litellm_init.pth with base64 encoded instructions to send all the credentials it can find to remote server + self-replicate. link below

307 2K 9K 5.9M 3K

Dan says he's got Qwen 3.5 397B-A17B - a 209GB on disk MoE model - running on an M3 Mac at ~5.7 tokens per second using only 5.5 GB of active memory (!) by quantizing and then streaming weights from SSD (at ~17GB/s), since MoE models only use a small subset of their weights for each token

Dan Woods @danveloper

3 months ago

x.com/i/article/2034…

66 186 1K 661K 2K

@kepano Great stuff! Seems it works off of a Cloudflare worker? Curious how you work around the YouTube rate limits/IP blocks?

The only to-dos method that has worked for me: Every week I make a weekly note, and write my to-dos for the week. I may add more items to it during the week. If any items didn't get done I roll them over to the next weekly note or drop them. That's it.

@elvissun @nz_mrl Didn't anthropic ban openclaw from using subscription tokens via oauth?

Given an embedding vector, you can tell which model produced it. I trained a 0.8M transformer that fingerprints embedding models by reading raw float digits (vocab size: 15). Full end-to-end, zero feature engineering.