Loïck BOURDOIS @BdsLoick

FAT5 (Flash Attention T5) boy @huggingface Fellow 🤗 lbourdois.github.io/blog/ France Joined October 2021

Tweets

346
Followers

278
Following

240
Likes

788

Loïck BOURDOIS @BdsLoick

2 months ago

@IBM @Microsoft @GoogleDeepMind @baai @qwen @orionweller @vllm @Google @Meta @huggingface @Nils_Reimers @jaseweston Big thanks to my HF Fellows bros for multilingual evaluation @tomaarsen, Bram Vanroy, @christopher, @w00jun_ @mrm8488, @prithivMLmods and to @AI_AlphaEdge for the time dedicated to this project 🙏 Links 👇 Blogpost: huggingface.co/blog/lbourdois… Models: huggingface.co/spaces/alphaed…

0 2 4 393 2

View Details

Loïck BOURDOIS @BdsLoick

2 months ago

@IBM @Microsoft @GoogleDeepMind @baai @qwen @orionweller @vllm @Google @Meta @huggingface @Nils_Reimers @jaseweston From these 16 families, we generated more than 5,500 monolingual models in 124 different languages.

1 0 4 96 0

View Details

Loïck BOURDOIS @BdsLoick

2 months ago

New blog post on @huggingface! An introdution to Trimming ✂️ a little-known but highly effective model reduction method. We achieved up to 87.24% size reduction while preserving performance 🧵

1 4 11 422 3

View Details

Introducing a revival of PapersWithCode! As @ilyasut said, we're back to the "age of research". Hence, it's important to share research and build on each other's work. > find SOTA per domain, not just LLMs > leaderboards > methods > all parsed at scale using AI agents.

33 90 607 79K 481

View Details

tomaarsen @tomaarsen

4 months ago

🌐 I've just released Sentence Transformers v5.4: we're going fully multimodal for embeddings & reranking! Also featuring a modular CrossEncoder, and automatic Flash Attention 2 input flattening. Highlights in 🧵

19 32 175 29K 58

View Details

みぃ🍵 @mithernet

4 months ago

著者です！ Attentionの「相対比較しかできない」という制約を外した、新しい機構を提案しました ①まずわかりやすい利点 ✅学習時より圧倒的に長い文でも性能維持＆正確な情報取得 ✅収束が非常に高速（LR=1でも学習可能） ✅モデルサイズ4割削減 ✅推論速度3倍超 (続く) arxiv.org/abs/2604.01178

15 132 802 90K 608

View Details

Loïck BOURDOIS @BdsLoick

5 months ago

CuTeDSL is really nice For those wishing to get into writing kernels in this language, github.com/b-albar/machete can be useful Boris ALBAR reimplemented Flash Attention, RoPE, RMSnorm, etc. Everything compatible with HF Transformers (tests on llama3, GLM4.7, Qwen3), TRL, PEFT/LoRA

maharshi @maharshii

5 months ago

CuTeDSL is my new favourite thing: I wrote a kernel for RMS norm after learning about layouts, tiling, copying tensors, reductions and so on, especially for inference and it is about 2.13x faster than a triton fused kernel for the given shape.

13 7 264 17K 78

1 0 0 173 2

View Details

Basile Terver @BasileTerv987

6 months ago

𝗜𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝗶𝗻𝗴 𝗘𝗕-𝗝𝗘𝗣𝗔 ⚡ An open-source library making JEPAs accessible, trainable on a single GPU in hours! 🚀 🔗 Paper: arxiv.org/abs/2602.03604 💻 Code: github.com/facebookresear…

12 95 657 92K 514

View Details

Loïck BOURDOIS @BdsLoick

6 months ago

@MaziyarPanahi @OpenMed_AI @huggingface huggingface_hub api is all you need to do it programmably 👀

1 0 1 16 0

View Details

Loïck BOURDOIS @BdsLoick

6 months ago

@gui_penedo I suppose all good things must come to an end. Thank you very much for the high-quality multilingual datasets

0 0 1 25 0

View Details

Loïck BOURDOIS @BdsLoick

7 months ago

@lhoestq @huggingface @mervenoyann @abhi1thakur If you rename the `datasets` library to `nlp` as in early 2020, I'll make sure it passes 700k before the end of the year 👀