-
Tweets274
-
Followers552
-
Following3
-
Likes614
Pulpie is open source and available on Hugging Face today. See our article to get started and for more details on how we built Pulpie. x.com/FeynAI/status/…
The gains are architectural. Today's best extractors are decoders limited by memory bandwidth. Pulpie is an encoder bound by compute. GPUs are starved for bandwidth, not compute. As a result, Pulpie is performant everywhere. Our testing shows Pulpie to be 7x faster on A100, and 20x faster on L4 GPUs.
Introducing Pulpie, models for cleaning the web. 70% of a typical HTML page is ads, navigation, and sidebars. Pulpie removes this noise and returns a clean document you can use in pre-training or as context. Previously, cleaning 1 billion pages with the best extractors cost $159,000. Pulpie brings that down to $7,900. A 20x decrease.
🔥🚀Chonkie 1.6.8 is out with PyEmscripten wheels! 🐍You can now run the full Python Chonkie library directly in your browser using Pyodide🦛 This is an early release that brings Chonkie’s core functionality to the web, but it's a big step towards 𝕓𝕣𝕠𝕨𝕤𝕖𝕣 𝕟𝕒𝕥𝕚𝕧𝕖 𝕡𝕪𝕥𝕙𝕠𝕟 𝕔𝕙𝕦𝕟𝕜𝕚𝕟𝕘. We’d love to hear your feedback! try it out and let us know what works, what doesn't, and what you'd like to see next! Happy coding! 🦛✨
Happy to share that @ChonkieAI just crossed 4K GitHub stars and 4 million PyPI downloads! 🎉 from a small side project to a fast growing library used by millions. Huge thanks to the community, so stay tuned as bigger things are coming 🔥
@TeraflopAI You can also check out this excellent blog post by @EnricoShippole where he breaks down what you can build on top of this: 🔗 teraflopai.com/blog/chonkie Highly recommended if you want to see the real possibilities 👀
🐦⬛+🦛 = 🔥 You can now leverage @TeraflopAI directly inside the Chonkie ecosystem. Seamless integration, zero friction. Just pure power. Check the post below for all the details 👇
We are happy to announce our official integration into @ChonkieAI. You can now use @TeraflopAI’s powerful text segmentation API directly within the Chonkie open-source library to chunk text seamlessly at scale.
As always, a huge shout out to all our contributors, they're the reason why open source thrives. Check out all of our releases on GitHub: ChonkieJS: github.com/chonkie-inc/ch… Chonkie Python: github.com/chonkie-inc/ch… Happy Chonking! 🦛✨
Chonkie Python continues to grow with our latest addition, the TeraFlopAI Chunker. With help from TeraflopAI's segmentation API, this chunker specializes in semantic splitting. We've found it particularly good for legal documents.
Introducing a refresh of Chonkie's JS library. 7 powerful chunkers available right out of the box, to help you build the best retrieval system in JS.
🚀 chunk v0.10.1 — multi-byte pattern chunking Split text at CJK punctuation, emoji, tokenizer markers, or any UTF-8 pattern. Compose with single-byte delimiters. Zero-copy. UTF-8 safe. SIMD-accelerated for 1-3 patterns, Aho-Corasick for 4+. 🔗 github.com/chonkie-inc/ch…
Our OSS engineer @itsclelia recently built 𝗹𝗶𝘁𝗲𝘀𝗲𝗮𝗿𝗰𝗵, a fully local document ingestion and retrieval CLI/TUI application powered by LiteParse ⚡ litesearch demonstrates how developers can assemble a high-performance, local-first retrieval pipeline using open tools from across the ecosystem: • Parsing: LiteParse, the fast and accurate document parser we recently open sourced • Chunking: @ChonkieAI • Embeddings: A local @nomic_ai model via @huggingface transformers.js • Vector storage: A local @qdrant_engine edge shard (custom-built in Rust and compiled as a native add-on) • Retrieval: Query stored files with optional path-based filtering and configurable relevance thresholds • Runtime: @bunjavascript for speed and versatility 💻 Check out the repository and try it yourself: github.com/AstraBert/lite… 📚 LiteParse docs: developers.llamaindex.ai/liteparse?utm_…
stop using MCPs to give your agents documentation. mandex gives your AI agent the same docs — but local, version-pinned, and 25ms instead of 800ms. One download. Works offline. No API keys. No rate limits. No monthly bill. 100+ packages. 1,500+ versions. Free forever. 🔗👇
Shoutout to all our contributors who chipped in big for this update. Check out our full release notes on GitHub. github.com/chonkie-inc/ch… Happy chonking! 🦛✨
⚡ Native async support is here. await chunker.achunk(text) Every chunking method now has an async equivalent. Chonkie no longer blocks your event loop.
🦛 Chonkie v1.6 just dropped. Our biggest release yet. → HTML table chunking → Self-hostable chunking API → Native async support The hippo is growing bigger and stronger. Here's what's new 🧵
Ishan Singh @CoolboiMeu
63 Followers 128 Following Issued in Public Interest. Resident Builder @_lagrangepoint
adam ghaida @adamghaida
2K Followers 751 Following engineering @orchid_hq // cs + philosophy @ carnegie mellon // prev @perplexity_ai
Ali 🍁 @ThatCdnAli
98 Followers 1K Following i made this cool chrome extension please try it @tabminddotco
Kishore @kish2011_
684 Followers 3K Following WooCommerce developer | Looking for a WordPress or PHP dev? I'm open to hire!! https://t.co/Ug1opoJK5e
Vishal Fulsundar @Vishal_Fulsunda
14 Followers 568 Following Aspiring data wizard managing page @wizard_data51
Abdelkrim Dakoaun @dakoaun34679
29 Followers 3K Following
Artem Murzin @artemmurzin
639 Followers 311 Following CEO @conareai | @xai grokathon winner | uni dropout | 19 🇺🇦
adam @theCTO
9K Followers 832 Following creator of email - i don't want you on my cap table posts are mostly satire
jeremy @Ununited11
191 Followers 2K Following
Read Only @ReadOnly1916717
2 Followers 828 Following
Jesse Waites @JesseWaites
2K Followers 3K Following Polyglot Software Developer. AI, Ruby on Rails, iOS, Mac Apps and more. Hiker, Cyclist, and Ice Climber based in the beautiful city of Boston, Massachusetts.
pokapo @pokapo5
19 Followers 2K Following
Shabah @GhostOfTuring
155 Followers 950 Following talking to the void about tech stuff and pet projects, fmr many things, im older than you think.
rahmat @thisis_rahmat
2K Followers 3K Following I’m just here to procrastinate. Opinions are mine and no you can’t have them.
Lishi @ikbear
2K Followers 7K Following Email verification API with 99.9% accuracy. Free 100+ daily checks. Protect your sender reputation.
Shojaei @realshojaei
4K Followers 5K Following AI Engineer | AI Applications, LLM Systems & Agentic AI
HappyQQ_AI @HappyQQ_AI
19K Followers 9K Following 为 AI 飞速发展助力，打造 AI 相关的好用的软件平台。 公众号：移动互联网 | 微信号：HappyQQ_AI。 AI开发交流Discord社区：https://t.co/yNQuB0qZRC
Jacob Würtz Lyngbye @jacobwl
219 Followers 865 Following Helping companies implement AI for better productivity and quality of life.
Kire @XbyKire
3K Followers 3K Following
rachmaninoffian @rachmaninoffian
140 Followers 6K Following battle hardened armchair general tfr (+ real estate) monocausalist
Andrea @__AndreaW__
132 Followers 3K Following Cerco di seguire persone in buona fede che abbiano opinioni diverse dalla mia. L'ignorante non si conosce mica dal lavoro che fa ma da come lo fa (C. Pavese)
yesvcan @yesvcan1
204 Followers 7K Following
JY Z @JunYuZzzzz
81 Followers 4K Following
Prakshan Vasuthan @PVasuthan
0 Followers 835 Following
Div @hurtbadly2
110 Followers 2K Following Scientific computing / Ostracizing with Julia Cryo-ET, HEP, HPC 🍎🟣🍏
AN @trailinga
46 Followers 6K Following shankaro shankarah sakshat | vyaso narayano hari | ubhayor madhya vivadhe | kim karothi kinkaramyaham
Vaibhaav @battarchicken
202 Followers 625 Following core contributor @aoagents | research @IITPAT | prev. @UofSC, ai*crypto stealth | ml, research | absorbing art in every shape it takes | que sera, sera
Nate Swanberg @nateswanberg
143 Followers 2K Following
abdalrahman.eweiwi @aeweiwi
62 Followers 1K Following
Gerardo Huerta @RoblenHuerta
185 Followers 2K Following Digital electronics, Embedded systems, NLP, Climate Change AI
Joaquin Bravo @jackbravo
1K Followers 1K Following Programador de Guadalajara. Me gustan linux, el software libre, el servicio social, el fucho, la bici y la guitarra.
Jesse Hirsh @jessehirsh
10K Followers 3K Following Futurist, farmer, researcher, public speaker, and punk.
Lain 🦛 @not_so_lain
2K Followers 1K Following Software Engineer @FeynAI | @huggingface fellow | y2k bothered
Bhavnick Minhas @minhash
1K Followers 974 Following Subagent Spawner @FeynAI (YC X25), Barista @better_auth, @IITGuwahati Alum, Ex community lead @Cohere_Labs 🩵
Shreyash Nigam @shreyash_nm
359 Followers 356 Following @FeynAI | prev. @ycombinator, @googleads, @UWTrends for United States
You might like
