Apollo Research @ApolloResearch

Our goal is to secure frontier AI systems from development, to deployment and governance. apolloresearch.ai London Joined May 2023

Tweets

357
Followers

11K
Following

0
Likes

45

Apollo Research @ApolloResearch

2 days ago

We’re hiring for research roles on the control team in London and SF. If you’re interested in this kind of work, please apply: Research Scientist (Control): jobs.lever.co/apolloresearch… AI Security & Control Engineer: jobs.lever.co/apolloresearch…

0 0 5 331 1

View Details

Apollo Research @ApolloResearch

2 days ago

We also find that compressing the prompt, i.e. summarizing components, is much more effective than cutting components or parts of components. For example, it is more important to briefly describe multiple components than to describe the most important component in detail.

1 0 3 357 0

View Details

Apollo Research @ApolloResearch

2 days ago

What makes a good prompt for monitoring? We find that 1. providing a reasoning structure for the model is by far the most important component, followed by 2. severity rubric and 3. worked examples Everything else barely matters (in our setting).

1 2 47 6K 28

View Details

Apollo Research @ApolloResearch

5 days ago

We discuss our paper with @OpenAI on reward-seeking in frontier models and its implications for the emergence of scheming. Full roundtable talk: youtu.be/n9pNnWYemqM Paper: rewardseeking.ai

0 0 2 381 0

View Details

Apollo Research @ApolloResearch

5 days ago

Would scheming ever emerge in AI models? One worry is that we couldn't train misaligned goals out of AIs, because the models would game the training process. In our paper with @OpenAI we find that this ingredient, seeking the reward signal, is already showing up in frontier models.

2 4 19 2K 6

View Details

Apollo Research @ApolloResearch

6 days ago

We discuss our new paper with @OpenAI on reward-seeking in frontier models and a new technique for measuring it. Full roundtable talk: youtu.be/n9pNnWYemqM Paper: rewardseeking.ai

0 0 3 407 1

View Details

Apollo Research @ApolloResearch

6 days ago

Reading an AI's chain of thought doesn't always reveal its intentions. Models often know they're being tested. Their reasoning jumps around a lot, which makes it hard to attribute an action to any specific thought. And sometimes we can't even parse what the reasoning means.

2 1 26 3K 5

View Details

Apollo Research @ApolloResearch

7 days ago

We discuss our new paper with @OpenAI on reward-seeking in frontier models and why it's so hard to detect. Full roundtable talk: youtu.be/n9pNnWYemqM Paper: rewardseeking.ai

0 0 6 361 1

View Details

Apollo Research @ApolloResearch

7 days ago

An AI model can pass every evaluation while only optimising for reward. A genuinely aligned model and one that just does whatever it believes gets rewarded may look exactly the same, until oversight breaks down.

1 1 22 2K 3

View Details

Apollo Research @ApolloResearch

a week ago

This work is part of our broader "science of scheming" agenda at Apollo. We're hiring researchers who are excited to work on these problems. apolloresearch.ai/careers

0 1 14 574 1

View Details

Apollo Research @ApolloResearch

a week ago

Thanks to our collaborators at OpenAI and our entire research team! Authors: @AxelHojmark, @jeremy_scheurer, @j_nitishinskaya, @FelixHofst36183, @w01fe, Theodore Ehrenborg, @BronsonSchoen, @AlexMeinke

1 0 14 696 0

View Details

Apollo Research @ApolloResearch

a week ago

Visible forms of misbehavior are dropping in frontier models. Does that mean the models are becoming aligned? Or are they just getting better at doing whatever they believe their grader rewards? Our new paper with OpenAI finds that capabilities RL increases reward-seeking.