Jay Fraser @loststudios

AI & Machine Learning Engineer | Software & Web Developer | UI/UX Designer | Emerging Technologies Ethics Specialist Buffalo, NY Joined March 2009

Tweets

1K
Followers

370
Following

5K
Likes

7K

Andrej Karpathy @karpathy

9 months ago

I quite like the new DeepSeek-OCR paper. It's a good OCR model (maybe a bit worse than dots), and yes data collection etc., but anyway it doesn't matter. The more interesting part for me (esp as a computer vision at heart who is temporarily masquerading as a natural language person) is whether pixels are better inputs to LLMs than text. Whether text tokens are wasteful and just terrible, at the input. Maybe it makes more sense that all inputs to LLMs should only ever be images. Even if you happen to have pure text input, maybe you'd prefer to render it and then feed that in: - more information compression (see paper) => shorter context windows, more efficiency - significantly more general information stream => not just text, but e.g. bold text, colored text, arbitrary images. - input can now be processed with bidirectional attention easily and as default, not autoregressive attention - a lot more powerful. - delete the tokenizer (at the input)!! I already ranted about how much I dislike the tokenizer. Tokenizers are ugly, separate, not end-to-end stage. It "imports" all the ugliness of Unicode, byte encodings, it inherits a lot of historical baggage, security/jailbreak risk (e.g. continuation bytes). It makes two characters that look identical to the eye look as two completely different tokens internally in the network. A smiling emoji looks like a weird token, not an... actual smiling face, pixels and all, and all the transfer learning that brings along. The tokenizer must go. OCR is just one of many useful vision -> text tasks. And text -> text tasks can be made to be vision ->text tasks. Not vice versa. So many the User message is images, but the decoder (the Assistant response) remains text. It's a lot less obvious how to output pixels realistically... or if you'd want to. Now I have to also fight the urge to side quest an image-input-only version of nanochat...

vLLM @vllm_project

9 months ago

🚀 DeepSeek-OCR — the new frontier of OCR from @deepseek_ai , exploring optical context compression for LLMs, is running blazingly fast on vLLM ⚡ (~2500 tokens/s on A100-40G) — powered by vllm==0.8.5 for day-0 model support. 🧠 Compresses visual contexts up to 20× while keeping

53 357 3K 1.5M 1K

557 2K 13K 3.3M 7K

View Details

Jay Fraser @loststudios

11 months ago

@karlmehta @PingThread unroll

0 0 1 14 0

View Details

Jay Fraser @loststudios

11 months ago

@karlmehta @threadreaderapp unroll

1 0 1 18 0

View Details

Jay Fraser @loststudios

12 months ago

@_mihir_sathe @sama ☝️ Absolutely agree!

0 0 9 129 0

View Details

Jay Fraser @loststudios

a year ago

@ukrainiansquad United States! Still supporting Ukraine! 🇺🇦 Our administration is an embarrassment!

0 0 0 10 0

View Details

Jay Fraser @loststudios

2 years ago

@sandra_doorley You are a absolute disgrace! RESIGN!

0 1 0 27 0

View Details

Jay Fraser @loststudios

3 years ago

I'm sold on HTMX. It's such a refreshing shift! htmx.org

Boot.dev @bootdotdev

3 years ago

What is HTMX? As explained by the creator... with @wagslane and @htmx_org !

11 13 115 22K 9

0 0 3 94 0

View Details

Jay Fraser @loststudios

3 years ago

@PicturesFoIder Alfie Solomons

0 0 0 10 0

View Details

Jay Fraser @loststudios

4 years ago

@flaviocopes Definitely check out lit.dev

0 0 1 0 0

View Details

Jay Fraser @loststudios

4 years ago

Super useful! 👇

Addy Osmani @addyosmani

4 years ago

DevTools now shows inactive CSS properties. Hovering explains why the rule has no visible effect. So nice.

43 535 4K 0 272

0 0 0 0 0

View Details

Jay Fraser @loststudios

4 years ago

@jaredcwhite Same way I'm feeling about CSS these days. 🔥

0 0 1 0 0

View Details

Jay Fraser @loststudios

4 years ago

Are you writing 'Defensive CSS'? It's a set of CSS practices that designers and developers can use to write CSS that is future-proof, resulting in fewer bugs in user interfaces. Check out defensivecss.dev created by @shadeed9 🔥

0 0 2 0 1

View Details

Jay Fraser @loststudios

4 years ago

Kevin's site: kevinpowell.co Kevin's YT channel: youtube.com/kepowob

0 0 0 0 0

View Details

Jay Fraser @loststudios

4 years ago

If you are a frontend web developer - whether beginner, intermediate or advanced - do yourself a favor and follow @KevinJPowell Kevin has made me fall in love with modern CSS. His tutorials are excellent and you become aware how powerful modern CSS has really become! 🔥

1 0 6 0 0

View Details

Jay Fraser @loststudios

4 years ago

@MentorWebDev Modern Vanilla CSS

0 0 1 0 0

View Details

Andrej Karpathy @karpathy

4 years ago

Reading through OpenAI Whisper paper github.com/openai/whisper some notes:

35 407 2K 0 798

View Details

Jay Fraser @loststudios

4 years ago

Loving this! Finally! Great work @zachleat 😎

0 0 0 0 0

View Details

Jared White 👋 @[email protected] @jaredcwhite

4 years ago

Something I'd wish I'd known about ARIA and a11y in web development much sooner is it doesn't just make everything better for certain demographics, it makes things better for _everybody_ — including the developers! Bake accessibility into the foundation of your frontend FTW!

1 2 12 0 0

View Details

Jay Fraser @loststudios

4 years ago

Don't sleep on the :has() CSS pseudo-class! 🔥 @KevinJPowell demonstrates how powerful a utility this is in his video below. Browser support is nearly here! youtube.com/watch?v=OGJvhp…

0 0 1 0 0

View Details

Jared White 👋 @[email protected] @jaredcwhite

4 years ago

I’m putting my money where my mouth is (so to speak). I’m building a tool to convert Tailwind utility class HTML soup to 100% vanilla HTML + CSS with design token variables. Output looks virtually identical. Copypasta from your favorite TW UI library, click a button, and boom!