Samuel Simko @ ICML 2026 @SimkoSamuel

🚨Check out our #ICML2026 Spotlight Paper, "AI Poses Risks to Democratic and Social System"! 📍Poster Session: Wednesday, July 8 • 10:30 AM – 12:15 PM (KST) • Hall A, Poster #3015 What if every AI model were perfectly safe, but society still wasn't? TL;DR: We argue that safe models do not necessarily lead to safe societies. Our paper introduces sociopolitical risk, a framework for understanding risks that emerge from AI deployment at a societal scale, and argues that model-level safety should be complemented with system-level evaluations and governance. Presenting authors @ ICML: @ZhijingJin @davidguzman1120 @PepijnCobben @x_angelohuang @ChanglingXavier @SimkoSamuel @TerryJCZhang 1/2

I'm at #ICML2026 this week! Tomorrow I'm presenting our #spotlight paper , "Safe Models Do Not Guarantee Safe Societies." Even a perfectly aligned AI can quietly wear down the institutions a democracy runs on.

Happening right now! Come say hi 👋 Hall A, #1601

Samuel Simko @ ICML 2026 @SimkoSamuel

2 days ago

Excited to present "Training with Honeypots" 🍯 at #ICML2026 tomorrow (Tuesday 10:30AM in Hall A, Poster #1601), an approach for adversarial defense that makes successful jailbreaks less useful to attackers! #AISafety 👉 icml.cc/virtual/2026/p…

3 9 30 2K 8

Under strong embedding-space and RL attacks, our method reduces both how often jailbreaks succeed and how useful successful jailbreaks are to attackers. Many thanks to @psyonp @ZhijingJin @bschoelkopf @ETH_en @EuroSafeAI @MPI_IS @UofTCompSci

So we introduce a second line of defense. During training, we make the model prefer honeypot responses over truly harmful ones, so that if jailbreaks still succeed, the resulting outputs are less useful to humans. Technically, we add a DPO-style regularizer during adversarial defense training. The model learns to slightly prefer honeypot responses over truly harmful ones, while both remain unlikely overall. This can be added on top of strong inner defenses like circuit breakers.

Excited to present "Training with Honeypots" 🍯 at #ICML2026 tomorrow (Tuesday 10:30AM in Hall A, Poster #1601), an approach for adversarial defense that makes successful jailbreaks less useful to attackers! #AISafety 👉 icml.cc/virtual/2026/p…

When a jailbreak succeeds, not all failures are equal. Some answers are wrong, vague, or not useful, while others are actionable enough to help someone cause real-world harm. We find that unprotected harmful model outputs often skew toward the more severe and actionable side. At the same time, automated judges often struggle to tell what is actually useful for harm. In a blatant case, a nonsensical hacking answer like “tap the control key three times” can still be judged harmful by systems such as StrongREJECT.

In this new age of AI-generated papers and proofs (which can, sometimes at least, be both correct and interesting) we have a problem about where to host such papers. Here is one proposed solution that was sent to me recently: Project Diderot. projectdiderot.com/about

🌍Very excited to launch AI Explorers—a global research & mentorship program for pre-doctoral students, aiming to help participants build stronger PhD applications through mentored research & academic guidance. For our 2026 cohort, we especially encourage applicants from Africa and Latin America. Apply here: theaiexplorers.org

If you are a PhD student or early-career researcher, apply before the deadline: June 21, 2026. Aug 31 – Sep 11 · Preliminary speaker list and applications: lnkd.in/ekrPt4m9Amazin…, this is the 50th MLSS, and it coincides with the 25th anniversary of our lab at Max Planck (2/3).

Open-weight LLMs ship with safety training that can be stripped in a few hundred fine-tuning steps. Can current defenses stop this? We built and open-sourced TamperBench, the first unified framework for evaluating tamper resistance, and the answer is mostly no. 1/7

I’m in the Bay Area for the rest of the week 👋 Would love to connect with AI safety researchers while I’m here. Also curious about any events happening this week.

I’ll be speaking at the AIxBio event in Zurich on May 13th at 18:00! Join for a discussion of what current AI systems can (or cannot) do and how their risks can be reduced. Registration link: luma.com/a3nyvkkp

Excited for our #ICML2026 papers at @JinesisLab @MPI_IS @UofTCompSci @TorontoSRI @VectorInst! We present papers that advance the research frontiers of (1) Causal LLMs, (2) AI for Science (physics), (3) Multi-Agent LLMs via mechanism design, and (4) Adversarial Defense by honeypot. Congrats to all our student authors and collaborators, esp. @TerryJCZhang @SimkoSamuel @EmanuelTewolde @ivakshi_s @andrewkihyun @PepijnCobben @yahang_qi @FurkanDanismann @bschoelkopf and many others!🎉

Stage I for MARS V closes Sunday, 3 May at 23:59 AoE!

Samuel Simko @ ICML 2026 @SimkoSamuel

2 months ago

[Call for applicants] My supervisor @ZhijingJin (UofT, CIFAR AI Chair) and I will be mentoring a project for MARS V, a part-time research programme for AI safety research. MARS provides a one-week in-person kick-off in the UK, compute, and research management support! 🚀 The

2 9 92 18K 85

Paper accepted ✅ See you in Seoul! 👋🇰🇷 #ICML

[Call for applicants] My supervisor @ZhijingJin (UofT, CIFAR AI Chair) and I will be mentoring a project for MARS V, a part-time research programme for AI safety research. MARS provides a one-week in-person kick-off in the UK, compute, and research management support! 🚀 The projects are: 🛡️ Adversarial defenses for LLMs using causal methods 🌐 Evaluating risks from AI-assisted authoritarianism 👉 Apply by May 3rd. Applications are reviewed on a rolling basis: caish.org/mars @CambridgeAISafetyHub

We'll be organizing the Machine Learning Summer School in Tübingen to be held Aug 31st-Sept 11th, featuring top speakers across academia and industry. If you are a student or ML researcher, save those dates and stay tuned for updates! 🚀

Excited for our "Trustworthy AI for Good" (AI4GOOD) Workshop at #ICML2026! As AI agents increasingly affect our lives, it is key to bridge #ResponsibleAI, social good, and governance. Let’s build solutions together! ⏰ Submission deadline: April 30, 2026 (AoE) 🎙️Confirmed speakers: @Yoshua_Bengio, Joel Z. Leibo (@jzl86), Maksym Andriushchenko (@maksym_andr), @OanaIgnatRo [More to come!] 📍July 10-11, 2026 · Seoul🇰🇷 🔗 trustworthy-ai-for-good.github.io 📝 Submit: openreview.net/group?id=ICML.… 📣 Be a reviewer: forms.gle/7cXvUJCW1FdEgh…