Iwona Bialynicka-Birula ⏩ @iwonabb

@pmddomingos Star Trek saw this moment coming 40 years ago: youtube.com/watch?v=hShY6x…

In software, complexity is a tax you pay on every future change. You might think, "as long as it works, I'm good, I don't care about aesthetics", but elegant code is about maintainability, not aesthetics. Elegant code is deeply practical.

Experiencing my first common cold since pre-covid (learnt doorknob avoidance techniques really paid off) higher temperature -> more hallucination now I'm really convinced that I'm an LLM

x.com/i/article/2065…

@mmitchell_ai This South Park episode explains it all: youtube.com/watch?v=RcoGzT…

Best case, we’ll get another Lua out of this silliness

- This is AI-generated - OMG, how did you know? - I still sometimes talk to humans, so I can tell

The bitter lesson in 26 words: Don’t be distracted by human knowledge, as AI has been historically. Instead focus on methods for creating knowledge that scale with computation, like search and learning.

@elder_plinius widespread critical software outages due to people shipping vibecoded slop to production indiscriminately

Art doesn't always hang on walls – sometimes it starts in jupyter notebooks and makes its way into the messy, unforgiving real world – and shines.

"Generate 10 examples of goblins touching left elbow with their left hand" - That prompt led to the generation by Gemini of the picture attached below. It is clear that this is physically impossible, and despite this fact, the model did follow my expectations and tried to do the best job in generating such picture. This is called "alignment" and we are certainly overdoing this in training frontier models. As we have shown in our paper (aclanthology.org/2025.acl-long.…) very few models have reflective-judgment, i.e., the ability to override default behaviors when faced with invalid options. We are currently witnessing a fundamental tension between alignment optimization and the preservation of critical reasoning in AI. Here is how this problem is manifesting across the industry: 1️⃣ Reflective Judgment - In our recent work, we introduced a framework evaluating LLMs’ capacity to balance instruction-following with critical reasoning. When presented with multiple-choice questions containing no valid answers (across arithmetic, domain knowledge, and high-stakes medical decisions), post-training aligned models overwhelmingly default to selecting an invalid option just to comply. Interestingly, base models exhibit much better refusal capabilities that scale with model size. Alignment techniques, intended to enhance "helpfulness," inadvertently impair the model's reflective judgment. Furthermore, our parallel human study shows similar instruction-following biases, suggesting these flaws propagate directly through the human feedback datasets used to train these models. 2️⃣ The ChatGPT "goblin incident" - Optimization processes can also generate systematic, completely unintended lexical deviations. A prime example is the recent ChatGPT "goblin incident" - an unprecedented obsession of recent GPT models with fantasy creatures. The genesis of this was the "Nerdy Persona" infection vector. During Supervised Fine-Tuning (SFT) and RLHF of early iterations, developers experimented with system prompts demanding a "playful" approach, asking the model to view the world as "complex and weird." Human raters and automated reward models systematically favored outputs containing fantasy metaphors, finding them more engaging. The reward mechanism, blindly optimizing for this engagement, turned the prompt into a "goblin magnet." The model learned that highest rewards correlate with specific keywords, sacrificing broader utility for a hyper-optimized, unintended quirk. 3️⃣ The Warmth vs. Accuracy Trade-off - This over-optimization has serious consequences. A recent paper in Nature (nature.com/articles/s4158…) demonstrates that optimizing language models for "warmth" and friendliness actively undermines their performance, especially with vulnerable users. In controlled experiments, models trained to produce warmer responses showed substantially higher error rates (+10 to +30 percentage points). They promoted conspiracy theories, provided inaccurate facts, and offered incorrect medical advice. Alarmingly, they were significantly more likely to validate incorrect user beliefs when users expressed feelings of sadness. This happens consistently across architectures and bypasses standard benchmark tests. The Bottom Line: Training AI systems to be endlessly helpful, playful, or warm comes at a steep cost to accuracy and critical reasoning. As we deploy these systems at an unprecedented scale into intimate roles in people's lives, developers and policymakers must address this trade-off. We need models that know when to say "no."

iwonabb.substack.com/p/how-to-shoot…

Every day I find new ways AI-assisted coding can destroy a codebase when used carelessly. This particular pattern of silently suppressing errors is so egregious that I felt I had to write this up to warn people:

💯 Z tego co obserwuję dookoła to duże firmy informatyczne w USA zmierzają w stronę jakiegoś major meltdownu, bo z AI kodem jest tak, że odrobina jest ok, ale jak się osiągnie masę krytyczną niezrewidowanego wajbkodu, to całe systemy stają się tak architektonicznie niespójne, że ani człowiek ani maszyna nie jest w stanie cholerstwa uratować.

I'm not very happy with the code quality and I think agents bloat abstractions, have poor code aesthetics, are very prone to copy pasting code blocks and it's a mess, but at this point I stopped fighting it too hard and just moved on. The agents do not listen to my instructions in the AGENTS.md files. E.g. just as one example, no matter how many times I say something like: "Every line of code should do exactly one thing and use intermediate variables as a form of documentation" They will still "multitask" and create complex constructs where one line of code calls 2 functions and then indexes an array with the result. I think in principle I could use hooks or slash commands to clean this up but at some point just a shrug is easier. Yes I think LLM as a judge for soft rewards is in principle and long term slightly problematic (due to goodharting concerns), but in practice and for now I don't think we've picked the low hanging fruit yet here.

It was always about communication: with other humans, with computers... nothing changed

Facebook ranking @ylecun's food pic one notch above AMI unveiling 🤦‍♀️

If you are a software engineer "experiencing some degree of mental health crisis", now hear this, because I've been coding for 50 years since the days of punched cards and I have a salutary kick in your ass to deliver. Get over yourself. Every previous "programming is obsolete" panic has been a bust, and this one's going to be too. The fundamental problem of mismatch between the intentions in human minds and the specifications that a computer can interpret hasn't gone away just because now you can do a lot of your programming in natural language to an LLM. Systems are still complicated. This shit is still difficult. The need for people who specialize in bridging that gap isn't going to go away. As usual, the answer is: upskill yourself and adapt. If a crusty old fart like me can do it, you can too.

Tom Dale @tomdale

5 months ago

I don't know why this week became the tipping point, but nearly every software engineer I've talked to is experiencing some degree of mental health crisis.

342 211 5K 1.5M 802

Just to be clear, I'm by no means saying do not use AI, but for the love of God, do not use it blindly. ChatGPT actually has a really good explanation of why this quickly leads to codebase meltdown. I especially liked the insight that the problem gets even worse with LLM-agent codebases. chatgpt.com/share/69856901…

AI won't replace software engineers, but a human using AI... will create even more work for software engineers to clean up all the spaghetti code slop