Sam Acquaviva @Sam_Acqua

Self-conditioning can be distilled into one-step language generation. We show that self-conditioned flow LMs perform fixed-point iteration, and introduce fixed-point flow maps to compress both the iterations and the entire flow. At one step, FMLM★ achieves 112.5 gPPL at

2 20 119 54K 105

1 2 17 4K 6

View Details

xuan (ɕɥɛn / sh-yen) @xuanalogue

3 weeks ago

"Do not panic or hallucinate like a fragile Large Language Model."

3 9 158 5K 26

View Details

Sam Acquaviva @Sam_Acqua

a month ago

So cool

Vedang Lad @vedanglad

a month ago

How well can you describe the feature selectivity of a vision neuron … with words? Interpretability has long borrowed from neuroscience — and maybe it can give back too! 🧵

5 43 161 103K 90

0 0 5 775 0

View Details

Sam Acquaviva @Sam_Acqua

a month ago

This is cool but disingenuous framing imo. All of the records were marginal, building on other records, so “out-outperformed all 1,016 other researchers” is a stretch lol I’m curious what this system’s performance would be if it couldn’t build on human records

Zhengyao Jiang @zhengyaojiang

a month ago

OpenAI ran a hiring challenge, but the top candidate was one they couldn’t hire: our autonomous research agent, Aiden. In Parameter Golf, Aiden ran for 22 days, and out-outperformed all 1,016 other researchers: 🧵 (1/8)

17 58 590 150K 384

2 0 15 2K 1

View Details

Sam Acquaviva @Sam_Acqua

a month ago

@YifeiZuoX awesome -- thank you for running this

0 0 2 91 0

View Details

Sam Acquaviva @Sam_Acqua

2 months ago

@YifeiZuoX @Haoxiang__Wang @kellerjordan0 I mean compute-matched and param-matched during training. e.g. both models at .6B and trained for 1e23 FLOPs (for example)

1 0 2 134 0

View Details

Sam Acquaviva @Sam_Acqua

2 months ago

@YifeiZuoX @Haoxiang__Wang @kellerjordan0 Thanks for the response — and great paper! Isn’t parallax better compute-matched at .6b? Or am I missing something? And any reason the param-matched transformer and compute-matched parallax aren’t included for 1.7b?

1 0 3 137 0

View Details

Sam Acquaviva @Sam_Acqua

2 months ago

@whatthelukh Great post! wdyt about CTPO as a lower-variance-but-still-unbiased estimate of IS ratio? I'm curious how it would do on your "Simple Horizon Simulation" at small batch sizes