Jeremiah Milbauer @jerelev

PhD student @CarnegieMellon and @MozillaAI, making things for thinking with // once upon a time @GoogleAI @Waymo @BKCHarvard, cs+phil @UChicago jeremiah.milbauer.info nyc Joined October 2014

Tweets

62
Followers

187
Following

327
Likes

60

Jeremiah Milbauer @jerelev

a month ago

Very excited about information-based approaches to measuring AI capabilities. If removing vocabulary words makes "cultural" stories indistinguishable, there is still a lot of progress to be made before achieving deep cultural understanding.

Supriti Vijay @SupritiVijay

a month ago

Cultural competence evaluations must go beyond just salience of cultural markers and interrogate the mechanisms of localization in generated content. 📜Read more: arxiv.org/pdf/2606.14626 Work with @shaily99, @jerelev, @841io 🎊

1 0 5 361 0

0 0 2 200 0

View Details

Jeremiah Milbauer @jerelev

3 months ago

At the end of the day my great grandfather would unhook his horse from the cart in bushwick and his horse would find his own way back to the stable. Please do not do this with your citi bike.

David @blissposting

3 months ago

Silly level of drunk but my point is you gotta treat the citi bike as much like a horse as you can. The Citi bike has a soul. And you gotta respect that

16 341 8K 222K 366

0 0 0 92 0

View Details

Jeremiah Milbauer @jerelev

3 months ago

Now imagine chatbots deciding who you are, and what you deserve, from months of implicit micro-signals... aclanthology.org/2025.findings-…, with @anjali_ruban, Emma Strubell, @gneubig, @MaartenSap showed this already happens at small scale. But what about long user histories?

Nav Toor @heynavtoor

3 months ago

Imagine you live in a small village. English is not your first language. You did not go to a fancy school. You open Claude and ask it a simple question about the water cycle. Claude answers like this. "My friend, the water cycle, it never end, always repeating, yes. Like the

168 1K 4K 391K 2K

0 2 8 5K 5

View Details

Jeremiah Milbauer @jerelev

4 months ago

for one beautiful week a year AGI means something different

0 0 1 82 0

View Details

Jeremiah Milbauer @jerelev

6 months ago

Is this Glen Wheeler (mathematician) or Glen Wheeler (marine biologist)? Yes, both exist.

Steve McCormick @Quasilocal

6 months ago

Perfect title, no notes.

4 38 414 23K 51

1 0 3 108 0

View Details

Jeremiah Milbauer @jerelev

8 months ago

At CMU, most of us have 8+ GPUs each at all times, not counting ad-hoc availability, cloud compute, and individual lab servers. This data is so wrong it's got to have both numerator and denominator errors.

Francois Chaubard @FrancoisChauba1

8 months ago

Last night, @agupta and I hosted a great dinner with 14 professors at #NeurIPS2025 from leading academic labs across the US, and many cited compute in academia as "abhorrent". Out of curiosity I just pulled these stats. This is insane. To do meaningful AI research today you need

68 61 486 265K 207

0 0 6 397 1

View Details

Jeremiah Milbauer @jerelev

8 months ago

@rasbt @FrancoisChauba1 @agupta Why not just run jupyter as a slurm job?

0 0 0 66 0

View Details

Jeremiah Milbauer @jerelev

9 months ago

it's giving "we need to get good at cyber"

Ben Golub @ben_golub

9 months ago

From the people who renamed computation power as “compute” and model output as “inference”

58 24 785 310K 150

0 0 0 101 0

View Details

Chenhao Tan @ChenhaoTan

9 months ago

Recruiting PhDs & postdocs for: 🤖 agents "taking over" science (hypogenic.ai and 📌) 🧪 Real scientists ➡️AI (e.g., materials, chem, physics) 📜 Theory + incentives for H-AI collab & credit (e.g., formalizing tacit knowledge) new adventures for me, 🔄 if you can! 🙌

5 63 284 39K 202

View Details

Jeremiah Milbauer @jerelev

9 months ago

Arxiv predates the web. There are now many more options for uploading any old PDF (and making it discoverable) than there were in 1991. But on the other hand there are enormous benefits – trust, access, stability – to having everything in one place.

0 0 0 50 0

View Details

Jeremiah Milbauer @jerelev

9 months ago

I tend toward the belief that arxiv has been an incredible boon to science. But! I do sometimes worry about how easy it is to make any old ramble look & feel like "science" – just use the right overleaf template and upload to arxiv.

Thomas G. Dietterich @tdietterich

9 months ago

The Computer Science section of @arxiv is now requiring prior peer review for Literature Surveys and Position Papers. Details in a new blog post

42 80 523 366K 161

1 0 1 118 0

View Details

Jeremiah Milbauer @jerelev

10 months ago

'tis the season of getting cold emails from the boldest phd applicants. In the interest of fairness for those who did not know they could ask, please DM if you'd like an inside perspective on AI/NLP at CMU. (Or share with those you know who'd benefit!)

0 0 0 66 0

View Details

Jeremiah Milbauer @jerelev

10 months ago

one of the most important emerging issues in the LLM era of social science

Emily Byun @ ICML'26 @yewonbyun_

10 months ago

💡Can we trust synthetic data for statistical inference? We show that synthetic data (e.g. LLM simulations) can significantly improve the performance of inference tasks. The key intuition lies in the interactions between the moments of synthetic data and those of real data

2 35 144 33K 84

0 0 1 157 0

View Details

Chenghao Yang @chrome1996

a year ago

Have you noticed… 🔍 Aligned LLM generations feel less diverse? 🎯 Base models are decoding-sensitive? 🤔 Generations get more predictable as they progress? 🌲 Tree search fails mid-generation (esp. for reasoning)? We trace these mysteries to LLM probability concentration, and introduce Branching Factor (BF) — a simple measure that captures it all. Key Findings: — BF declines over time → generations become more deterministic — Alignment tuning slashes BF → shrinks the generative horizon — Low BF explains decoding sensitivity → fewer good options to prune — CoT stabilizes generation → shifts key info to late, low-BF regions — Avoid late branching → too many low-probability, low-quality continuations — Alignment surfaces low-entropy paths already latent in base models 📜 Paper: arxiv.org/abs/2506.17871 🌐 Website: yangalan123.github.io/branching_fact… 🎥 2-min explainer below. Joint work w/ @universeinanegg , thanks for the constructive feedback from @PeterWestTM @UChicagoCI @zhaoran_wang @_Hao_Zhu @ZhiyuanCS @TenghaoHuang45 @TuhinChakr

1 26 96 18K 36

View Details

Jeremiah Milbauer @jerelev

2 years ago

@lateinteraction Having contributed some qs to lastexam, I don't think the affordances of this style of benchmark actually measure "all knowing" -- requiring exact answers to complex/google-proof questions limits the scope to a narrow set of problem types

0 0 0 95 0

View Details

Jeremiah Milbauer @jerelev

2 years ago

@mariaKhalusova @lateinteraction QA has long been treated as a system. I think there were component-based evaluations for QA as early as 2006, if not before. Just not done in the highly-portable benchmark style that NLP research has now.

0 0 0 40 0

View Details

Jeremiah Milbauer @jerelev

2 years ago

@fabian_stephany Is there a way to localize the pageview data to individual states?

0 0 0 11 0

View Details

Jeremiah Milbauer @jerelev

2 years ago

@EmilySilcock1 x.com/antgoldbloom/s…

Anthony Goldbloom @antgoldbloom

2 years ago

The overlooked GenAI use case: cleaning, processing, and analyzing data. blog.sumble.com/the-overlooked… Job post data tell us what companies plan to do with GenAI. The most common use case is data analytics projects. Examples: - AstraZeneca: using LLMs on freeform documents to

22 112 641 165K 793

1 0 1 164 2

View Details

Jeremiah Milbauer @jerelev

2 years ago

We've also seen that AI-generated recommendations reinforce existing stereotypes; whether this leads to changes in behavior at scale & worse outcomes is an important question for the next 5-10 years. paper: arxiv.org/abs/2410.05613

Cory Doctorow NO LONGER ON TWIT TER @doctorow

2 years ago

I think it behooves us to be skeptical of stories about AI driving people to believe wrong things and commit ugly actions. Not that I like the AI slop that is filling up our social media, but when we look at the ways that AI is harming us, slop is pretty low on the list. 1/

3 67 191 69K 65

0 3 13 6K 7

View Details

Jeremiah Milbauer @jerelev

2 years ago

Glad to see OpenAI taking a look at this! It’s not just names – we showed that even implicit clues about your identity can lead to racial stereotypes in model responses: x.com/jerelev/status…