GoodieHART @GoodieHART

Android was sold to us as an open platform. Now Google wants every developer to register and submit ID just to let you install their apps. Don't let them rug-pull 3 billion users. keepandroidopen.org @AlteredDeal #KeepAndroidOpen

The Angelos Network, a distributed network of workers running on Termux devices with the aim of splitting workloads of various tasks and mitigating rate limiting and Anti-Bot Systems @arc @circle @CanteenApp #Web3‌‌ #x402 #usdc #termux github.com/GoodieHART/ang…

Participating in the Canteen-Lepton Hackathon! 🔥

Test it out using curl! …t-llamaendpoint-fastapi-app.modal.run

You can test the models deployed on my endpoint, each at a different quantization levels. Why this matters is that you may notice incoherence for models with more aggressive quantization applied

If you come across a GGUF model file, here's how to decode its capabilities. Let's start with a filename from my deployed endpoint: llama-2-7b-chat.Q4_K_M.gguf

@davidokocha086 @X A blessed new year to you, many more successes coming your way in Jesus name 🙏

OPay Scholars 2026 just got stronger!🥳 We’re excited to welcome 3MTT - an initiative of the Federal Ministry of Communications, Innovation & Digital Economy - as a partner in OPay Scholars 2026. Together with OPay, Google, and 3MTT, we’re expanding opportunities for young innovators through the National Innovation Challenge. Imagine what this could mean for a student with a great idea. Application still ongoing. OPayScholars2026 3MTT NationalInnovationChallenge OPayBeyondBanking

@kenkenlewu For higher generations they are designed to be much faster and efficient, so it's an absolute yes to check what generation the CPU is!🫡 if you don't know how to verify which would be much better, I use this tool technical.city/en/cpu

I post info about my latest projects and experiments in full on my Telegram channel. It's a growing community, come along 🙌 t.me/bytestream_io

I really apologize, this thread got really messed up 🙄

GoodieHART @GoodieHART

2 weeks ago

When dealing with hosting AI models, there’s a very important concept you should keep in mind, especially if you use an inference framework like Llama.cpp, that is, how the AI models are packaged and shipped.

1 0 0 13 0

And the newer MXFP4 MXFP4 (Microscaling Block Floating Point, 2026): - 4-bit with block-level scaling, it's a very new quant scheme Don't worry if this list seems overwhelming, we'll break them down to understand when each one makes sense soon ✊🏼

These are some common quantization schemes you'll come across: K-Quants (the standard family, best quality-per-byte): - Q2_K, Q3_K_S, Q3_K_M, Q3_K_L - Q4_K_S, Q4_K_M ← the sweet spot - Q5_K_S, Q5_K_M, Q6_K, Q8_0

When dealing with hosting AI models, there’s a very important concept you should keep in mind, especially if you use an inference framework like Llama.cpp, that is, how the AI models are packaged and shipped.

These are some more quantization schemes I-Quants (importance-weighted, pushes compression further): - IQ1_S / IQ1_M - IQ2_XXS through IQ2_M - IQ3_XXS through IQ3_M T-Quants (ternary — weights are -1, 0, or +1): - TQ1_0, TQ2_0

3. llama.context_length / gemma4.context_length — Max context window (e.g. `4096`, `262144`) 4. tokenizer.ggml.model — Tokenizer type (e.g. `"llama"`, `"gpt2"`) 5. general.file_type — Quantization scheme encoded as an integer (e.g. `15 = Q4_K_M`, `10 = Q2_K`)

As for the metadata section of a GGUF file, it is self-describing. Some of the keys you'll find inside every GGUF file: 1. general.architecture — Model architecture (e.g. `"llama"`, `"gemma4"`) 2. general.name — Human-readable model name (e.g. `"Llama 2 7B Chat"`)