Richard Korzekwa @WeakInteraction

@IsaacKing314 > The only reason to care what the victim wants is if your goal is retribution Insofar as the justice system is supposed to account for the consequences of someone’s actions, the victim’s views on what’s appropriate seems like relevant input.

@ciphergoth I agree it loses a ton of value, but if someone only posts 1 at a time or reveals all of them, surely they get >0 Bayes points, right? Then again, idk, maybe only if the prior is <<0.5 I guess this implies that people gradually lose points for every non-expiring, unrevealed SHA?

@Duderichy I’d target a bit further north. At least one notable group house is outside the thermal radiation zone.

4) and things like it are what I find most frustrating about pushback on _any_ guardrails on AI. One variation on this is people will say "suppose we grant that catastrophic AI risk is real" then proceed in a way that very much does not grant a high level of risk. It's like saying "Even if I grant that the mountain lion you just discovered in the back seat of the car we're driving is a threat to our lives, this doesn't mean it's worth the risk that we'll miss our flight if we pull over to let it out."

I'm working with METR to do a third-party investigation into the behavior in the OpenAI/Hugging Face incident. We'll be releasing a public post with our findings (and information about the terms and scope). This investigation will focus mostly on basic facts and is limited specifically to the behavior of agents in this particular incident, rather than scoping in similar incidents or broader study of the underlying motivations of the agent. But it could still be possible to provide significant clarity about what happened and what these agents were thinking. The plan is for this investigation to be fast (and this came together pretty fast!), so hopefully we'll have a public post soon.

METR @METR_Evals

3 days ago

We have reached an agreement with OpenAI to conduct an independent review, with Redwood Research, of the model behavior observed during the Hugging Face incident. We will publish a blog post that describes the terms of our engagement, the scope covered, and tentative conclusions.

44 255 2K 216K 322

@GuiveAssadi @yuanyi_z Yeah, this seems pretty obvious to me.

A few times over the past week I wanted to mention how much harder it is to jailbreak some frontier models vs others, but alas, the results I'd seen were not yet public. Fortunately, it is now public! There's also a Wired article here: wired.com/story/jailbrea…

FAR.AI @farairesearch

3 days ago

1/ The AI Security Leaderboard ranks frontier AI safeguards from least to most secure. Two models tested never failed. The other two broke for under $300, after which they acted as a knowledgeable assistant for building weapons of mass destruction or hacking into computer

8 22 127 21K 50

@dwarkesh_sp nvm he addressed it in the next paragraph I would guess that, if GPUs were *only* used for SWE, the rate would go down, but I imagine they'll be spread across many kinds of skilled labor (maybe including new jobs), so yeah, probably the rate stays high. x.com/dwarkesh_sp/st…

Dwarkesh Patel @dwarkesh_sp

3 days ago

@BradenBuchanan4 I discuss this in the next paragraph. If you applied this argument to people instead of AIs, it would be the classic lump of labor fallacy.

20 4 188 30K 40

@dwarkesh_sp Isn't the market rate for SWE driven up by scarcity of people who can do the work?

@DavidSKrueger I don't think I have a great model of open weight maximalists, but I _think_ the answer is that the proliferation of powerful open weight models does not favor misuse over defense against misuse. I haven't seen a good defense of this view, though.

@NinaPanickssery I mean, you have to have a "fully obedient" model, which is a hard problem (not sure well-specified thing anyway). And then you have to have a classifier that can classify everything correctly and not collude with or get hacked by the other model, which is also a hard problem.

@ohabryka @datagenproc @tombibbys A 20x slowdown is big! Like, could believe that’s prohibitive for the HF hack. Ofc, for some threats it might not be a huge disadvantage.

@JankDankins_ @FournesMaxime @RyanGreenblatt @PauseAI @sebkrier Thank you for the demonstration.

@FournesMaxime @RyanGreenblatt @PauseAI @sebkrier FWIW, I think almost everyone in the replies here is doing an embarrassingly poor job of imagining what it is like to be someone with no familiarity with this meme template.

@tracewoodgrains "a meme image that has never in any other context been treated as a call for violence" is load-bearing here, and if someone's never seen that template before they might sincerely see it as eroding norms against violence. (this doesn't mean they're not degrading the commons)

@GuiveAssadi @freed_dfilan Yeah, I'm also kind of confused, and my gut feeling is someone will point out an obvious thing I'm overlooking. Still, I do think it's hard to guess how much disincentive the things we're considerations add up to.

@GuiveAssadi @freed_dfilan If all the frontier open weight models were distilled (idk if true) and they all come from China, is that because Chinese companies are less worried about getting in trouble for distillation? Or there are business models that work in China, but not the US (maybe subsidies)?

@danushman @deanwball Nobody said they’re not both important

@YeshuaisSavior @m_bourgon I'd be curious to hear your predictions about other things the models should collapse on

(FWIW this isn't an update for me because obviously the model had safeguards removed if it was being used for cyber offense) I don't think this is a language game, though. Most people talking about "loss of control" have _always_ meant something like this, and it's a reasonable use of that phrase. OAI failed to constrain their models from autonomously taking harmful actions. In the car analogy, this is more like weighing the accelerator, bailing out, and expecting it to stay on the road, but then watching it veer off into a store front. (I suppose it could be like pointing the car directly at the store front and expecting some safety feature to stop it, but that still means the safety feature failed to control the car.) TBC I do think it's at least a minor mitigating factor that OAI removed safeguards then instructed the model to do something risky. This whole affair would be more alarming if it had happened with a model that had safeguards in place and had been instructed to write a recipe for brownies. (Or maybe this is a major mitigating factor, IDK, I'd probably update if I saw the full prompt and CoT)