Justin Johnson @jcjohnss

Cofounder @theworldlabs. Building Spatial Intelligence. Joined January 2014

Tweets

394
Followers

21K
Following

616
Likes

3K

Justin Johnson @jcjohnss

a week ago

@satishmreddy @jon_barron @theworldlabs We are doing it even better than we could have at FAIR :)

1 0 2 47 0

View Details

3D is an exciting area where we are still figuring out the right tasks, problem formulations, architectures, and the best ways to scale. We're sharing some of our ideas here in our first-ever papers from @theworldlabs, led by an awesome set of interns.

World Labs @theworldlabs

2 weeks ago

Today we are sharing three new research papers, each exploring a new way to generate 3D content by leveraging large-scale generative models and 2D priors. These projects were led by our incredible interns @HaoZhang623 @BDuisterhof @DrTunnels [1/4]

16 92 657 111K 446

8 19 163 39K 69

View Details

Justin Johnson @jcjohnss

2 weeks ago

Working with @HaoZhang623 , @BDuisterhof , and @DrTunnels on these projects over the past few months has been great - congrats to all of you!

1 0 5 1K 0

View Details

Fei-Fei Li @drfeifei

3 weeks ago

x.com/i/article/2062…

172 1K 5K 1.1M 6K

View Details

Kyle Sargent @KyleSargentAI

4 weeks ago

Today we released “GPIC: A Giant Permissive Image Corpus for Visual Generation.” It’s a 100M image dataset for visual generation, with text captions and 100% known+permissive licenses, hosted on HuggingFace. I’m excited to get this out! Check it out: gpic.stanford.edu

3 15 77 6K 14

View Details

Justin Johnson @jcjohnss

4 weeks ago

In recent years, academic and industry work in generative modeling has drifted so far apart that they are playing totally different games, and techniques that work in academia may not transfer to industry problems. The divide isn't just about scale -- the different tasks in academia vs industry lead to different fundamental challenges. Academic work focuses on class-conditional ImageNet generation. This has a very weak conditioning signal (single categorical label) and the problem is very data-constrained, with all SOTA methods training for hundreds of epochs. The main challenge in this regime is combatting overfitting. Industry work on image or video generation usually has a much richer conditioning signal (e.g. very long captions, input images, etc) and is almost always underfitting since data can be scaled to absurd degrees. Overfitting (at least for pretraining) isn't a concern; instead we want to fit the complex data distribution *as fast as possible*. We hope that GPIC is approachable on the academic budgets people are already expending on ImageNet, but will lead to problems more similar to the industry-scale challenges in generative modeling.

0 4 22 2K 5

View Details

Justin Johnson @jcjohnss

4 weeks ago

GPIC should be the new standard benchmark for generative modeling. Training 1 epoch on GPIC is the same cost as 100 epochs on ImageNet, but is a much better proxy for real-world problems. If you work in generative modeling, try GPIC for your next project!

Keshigeyan Chandrasegaran @keshigeyan

4 weeks ago

1/ Introducing GPIC: a Giant Permissive Image Corpus and benchmark for visual generation! 🚀100M VLM-captioned image-text pairs for training 📊1M image-text pairs for benchmarking 🖼️~28 trillion pixels 🤗Centrally Hosted ✅Fully permissive for research + commercial use Dataset,

15 83 372 147K 231

9 17 103 46K 35

View Details

Justin Johnson @jcjohnss

2 months ago

@jon_barron Thanks! I'm sure this will continue to improve with more / better data, but clear messaging is really important

0 0 10 395 0

View Details

Justin Johnson @jcjohnss

4 months ago

Latent Forcing lets us train strong pixel-space diffusion models that benefit from DINOv2 alignment like many recent latent diffusion models. But it's a lot more than that -- it's also a new way to approach coarse-to-fine generation and latent "chain-of-thought" reasoning for diffusion models. Super proud of this project led by @BaadeAlan!

Alan Baade @BaadeAlan

4 months ago

What's the right space to diffuse in: Raw Data or Latents? Why not both! In Latent Forcing, we order a joint diffusion trajectory to reveal Latents before Pixels, leading to improved convergence while being lossless at encoding and end-to-end at inference. w/ @drfeifei+... 1/n

8 72 542 121K 373

1 16 140 14K 107

View Details

World Labs @theworldlabs

5 months ago

RTFM (Real-Time Frame Model) explores real-time, interactive frame generation from images, producing new views on the fly without explicit 3D. It differs from Marble, which targets high-fidelity persistent worlds (3DGS). We’re interested in how ideas from both might connect 🌎

19 54 490 74K 174

View Details

Justin Johnson @jcjohnss

5 months ago

@tebayoso Here you go: platform.worldlabs.ai

0 0 8 993 2

View Details

Justin Johnson @jcjohnss

5 months ago

Creating 3D worlds is now just an API call away with the launch of our new World API -- I can't wait to see what you build with it.

World Labs @theworldlabs

5 months ago

The World API is live. Generate persistent, explorable 3D worlds from text, images, and video. Integrate them directly into your products.

107 264 3K 661K 3K

1 3 67 8K 19

View Details

Justin Johnson @jcjohnss

6 months ago

VGG is pretty unique in that it has no downsampling for the first few layers, then downsamples very slowly. VGG actually *upsamples* at the start since it increases channels from 3 to 64 in the first 2 layers at the original spatial resolution. Other early CNN architectures (AlexNet, Inception, ResNet) all used aggressive downsampling at the start -- e.g. ResNet does 7x7 conv stride 2, then 2x2 pooling. This makes them a lot cheaper, and throwing away resolution doesn't seem to matter for classification. I always hypothesized that the high-resolution and high-channel feature maps in VGG let it represent high-frequency texture information a lot better than other architectures.

1 0 4 69 0

View Details

Justin Johnson @jcjohnss

7 months ago

@KhanNdifor @TheoMediaAI Awesome, glad you got it working!

1 0 1 80 0

View Details

Justin Johnson @jcjohnss

7 months ago

The 3D scene is rendered on the user's device after generation, which is a lot more demanding than playing a video. Looks like that CPU is from 2015~2016, and is ~2-3x slower than most mobile CPUs of the last few years. If you have a recent smartphone (say iPhone 14 or later) it will probably run much smoother there.

1 0 0 90 0

View Details

Justin Johnson @jcjohnss

7 months ago

@KhanNdifor @TheoMediaAI What device are you using?

1 0 0 81 0

View Details

World Labs @theworldlabs

7 months ago

Today, we’re sharing a behind-the-scenes look at how we used Marble to create the worlds you see in our launch video.

24 75 372 130K 126

View Details

Andy Cheng @DrTunnels

7 months ago

Wait— you can make a whole music video (Spaghetti 🍝from @le_sserafim ) on Marble and have fun doing it? I just created 20 Spaghetti Worlds, all 100% Marble-made. Can you guess their flavors? Marble from @theworldlabs is the first AI that actually lets me build these wild worlds, explore them freely, and craft smooth camera paths— all while vibing to music in real time. It’s insanely fun. I’m dropping the link to my Spaghetti Worlds and recipe in the reply. And I’m issuing a challenge to everyone: Think you can beat my music video? You’ll have to out-cook me with your worlds and your camera moves. 🍝🎥🔥 Sound on 🔊

0 1 8 14K 5

View Details

Winnie Lin @winilinnnn

7 months ago

we've been having so much fun testing our own tech 😆 I was able to create this world from just 2 input images in 10 minutes (and spent another 20 minutes making the flythrough video). workflow description below ⬇️