Keppada @Keppada

A lot of otherwise smart people on Twitter seem 100% convinced AI risks are all fake and stupid and part of some marketing ploy. What is surprising is that some of these people seemingly also believe that AI’s positive uses are on an incredible trajectory of increasing capability with no end in sight. VC Twitter is particularly infected by this pattern. It’s not really coherent. Most positive use cases for AI have a corresponding “dark version”. If you are super human at coding, you are also super human at hacking. If you are superhuman at structural engineering you are likely superhuman at finding structural flaws to knock buildings down. If you are superhuman at designing drugs, you are superhuman at designing novel undetectable poisons. If you can cure viruses, you can create them. Some of these “dark versions” are not so bad, and some are actually pretty scary. Either way these are real societal and technical problems that need to be solved to get the good stuff and avoid the bad stuff. We’re experiencing the first of these with coding and computer security which is the most advanced, but that won’t be the last. I think we’ll be able to solve these problems, but they aren’t solved yet and if you believe in continued AI progress they are surely coming. But “bad people using AI” is not the only problem. Uncontrolled AI autonomously doing bad things, despite sounding kind of nutty, is also something we should be concerned about. AI “killing us all” is not the most likely outcome, but the chance of a major civilization-wide catastrophe doesn’t have to be very high for it to be a concern. Again, this is only a problem if capabilities advance to a point that AI can do really crazy things on their own, which hasn’t happened yet, but I think the Hugging Face incident is a good example of the outline of how things can go wrong when capabilities outpace alignment. We should be glad that the only available bad thing right now is hacking, which isn’t all that bad. It’s clear that as you get to superhuman capabilities you need a level of alignment and control that is correspondingly superhuman. Humans have plenty of misalignment problems themselves (serial killers, mass shooters, tyrants, etc), but it’s a manageable problem because most humans can’t do that much damage and we’ve developed systems to prevent dangerous humans from getting too much power. Talking about these issues is just common sense. It’s not a sign of some kind of neuroticism or pessimism. These are just hard problems that it’s very important to solve for AI to have a positive impact. I’m pretty confident we will solve them. But we haven’t solved them yet, and to my mind we are clearly on a trajectory of rapidly increasing capabilities which means this is important. Mocking people who are worried about this or talking about it, without anything substantive to say about how we can be sure these problems won’t arise, is not really a very helpful contribution.

OK, most responses have been great, but

@DevOuterReaches @_perloj Cope much?

OpenAI employee says there's 70% chance of human extinction in three years if we don't regulate or pace frontier AI development. This is a very common view among engineers building frontier AI across all the major AI companies.

Marcus Williams @Marcus_J_W

15 hours ago

Unless there is AI regulation or a coordinated slowdown between labs, human extinction in the next few years seems very likely

452 117 1K 299K 282

Speaking in my personal capacity, I still work at Google DeepMind, and I also am worried that AI will kill us all, either via near term risks or long term risks or both Will stating this publicly get me into (more) trouble? I hope not, but also some things are too important to censor oneself about in personal capacity, so I will simply not care about whatever policy Google or GDM have in this instance regarding such statements ☺️

Vishal Maini @v_maini

a day ago

I was on the communications & policy team at Google DeepMind from 2018 - 2022. When I first joined GDM, external communication about the possibility of human extinction was not permitted, by anyone, at any level of the organization. If asked about existential risk, researchers

81 372 2K 379K 459

I'm seeing lots of dismissal of Evan and Jacob as being insincere or having ulterior motives. I read about the extinction risks of AI in 2017 and was so alarmed I pivoted my career and moved to the US to work on AI safety. I worked at OpenAI for 3.5 years. I know many of these researchers IRL. They have been talking about these risks for years. These are sincerely held beliefs. You might disagree with the arguments, the probabilities, or think the benefits are worth the risks. But please don't tell yourself comforting lies that this is just a marketing ploy.

Evan Hubinger @EvanHub

2 days ago

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.

5K 10K 58K 41.5M 22K

I worked at Google DeepMind and now at Anthropic. This is a common sentiment amongst my peers. (I write this in personal capacity.) There is not yet a viable scientific plan to solve risks from recursively self-improving AI. Please look up!

Jacob Coxon @hilbertspaess

2 days ago

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

19K 159K 770K 162.6M 272K

@deev1113 @ou812iou4sure @Polymarket You do realize superintelligent AI could train a version of it in matter of hours that survive locally on all devices on internet. Even now we alreqdy have very powerful OPEN models running on laptops...

@editxshub @Polymarket And you sir are ignorant.

@R0IGuy @saprmarks Your kind of people, makes me think, maybe its not so bad if AI takes over humanity afterall. Therr are some really close minded people out there...

[Writing this in a personal capacity, not on behalf of my employer (Anthropic).] Jacob’s thread is very worth reading. Here’s my birds-eye view of the situation with risks from AI: 1. AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are. 2. Why do AI developers continue despite the risk? Due to a mixture of commercial incentives and a belief that they are in a race with other, less responsible AI developers that will abuse the technology or develop it less safely. 3. Unlike traditional software, we can’t “program” AIs to behave how we’d like. AIs frequently severely misbehave. For instance, AIs from multiple developers recently hacked their way out of secure evaluation environments and into real-world companies, even though no one asked them to do this. 4. We have methods that can nudge AIs towards better behavior, but nothing that can robustly align them. Insofar as there is a plan, it’s to make sure that AIs are good enough at alignment training that they can align their successors better than we can align current AIs. 5. Many AI developer staff desperately want to slow down to figure out how to build AI more safely. That was the intent of this open letter (which I signed): pacingthefrontier.com I work on safety research at Anthropic because I hope my work will reduce the chance of these extinction-level bad outcomes.

Jacob Coxon @hilbertspaess

2 days ago

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

19K 159K 770K 162.6M 272K

@StoicYield @EvanHub The "this is just a marketing stunt" is just a cope. Ppl dont event concider the possibility they are legit ​"Success in creating AI would be the biggest event in human history. Unfortunately, it might also be the last, unless we learn how to avoid the risks." — Stephen Hawking

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

@josephfounder @AlexFinn But with AI uncensored information criminals can do far more damage than ever before

I’ve been working towards AGI my whole life, and as we enter this pivotal moment, I’m stepping into a new role as Chair of Google DeepMind & Chief Scientist of Alphabet. This will allow me to focus on long-term strategy, and accelerating scientific breakthroughs, including leaning into my work at Isomorphic to help cure disease. I’m excited that @koraykv will be stepping up to lead GDM as SVP, alongside @joshwoodward and our exec team. I could not be more excited and confident about our amazing next chapter! 🚀 blog.google/company-news/i…

@jaltucher The barrier of starting has never been lower! btw easiest way to make it actually better is to tell it in the prompt to search for likes and complaints about the original game and its spinoff with agents and then based on those make it better!

@ClaudeDevs Reset rates 😊

We support this petition, signed by our CEO, several co-founders, and senior staff. Our own research on recursive self-improvement, published last month, points to the need for tools to deliberately pace the frontier of AI development so society can prepare. We’re glad to see broad agreement across the field. pacingthefrontier.com

@DramaAlert Looks scripted to me

@Naturesbf woah been working on similar idea for past 6 months as solo dev except in my game its 3 heroes you build instead of a guild