Ioannis Papapanagiotou @ipapapa

Engineering @Google | ex-industry: @SnowflakeDB @Netflix @IBM | ex-academia: @Purdue @NCState ipapapa.github.io California, USA Joined March 2010

Tweets

1K
Followers

610
Following

659
Likes

1K

Ruslan Meshenberg @rusmeshenberg

2 months ago

An overview of how we're integrating AI in the reliability tech and practices - sre.google/resources/prac… credit: @ipapapa

0 3 11 907 5

View Details

Ioannis Papapanagiotou @ipapapa

7 months ago

Benchmarked: H100 vs Gemini Flash. Result: H100 choked on prefill. Gemini flew. Without complex disaggregation, a single H100 is just an expensive heater. ioannisp.medium.com/the-real-cost-…

0 0 2 174 0

View Details

Ioannis Papapanagiotou @ipapapa

7 months ago

@karanjagtiani04 Holding up well! After the first query warms the cache, we're seeing ~98% cost reduction. The catch: your prefix (system prompt + context) needs to stay consistent. Change it, cache resets. Quality improved too - full context catches cross-references that chunked RAG misses.

0 0 0 14 0

View Details

Ioannis Papapanagiotou @ipapapa

7 months ago

I've been recommending RAG to save costs. Time to reconsider. Testing Gemini Flash 3 with 22 technical manuals + implicit caching (75% off repeated prefixes): Full documents as system prompt became: → Cost-competitive with RAG → Higher quality responses

2 0 1 125 0

View Details

Ioannis Papapanagiotou @ipapapa

7 months ago

📑 Markdown/Header: Splits by structure → Respects semantic boundaries → Uneven chunk sizes - which affects retrieval consistency and costs. All splitters have flaws - you're just choosing which flaw to accept.

0 0 0 75 0

View Details

Ioannis Papapanagiotou @ipapapa

7 months ago

Choosing a text splitter for RAG? 📏 Character: Splits by count (e.g. 2000 chars) → Simple, predictable → Cuts mid-sentence, mid-table 🔢 Token (TikToken): Splits by LLM tokens → Better for context planning → Still arbitrary cuts

1 0 0 107 0

View Details

Ioannis Papapanagiotou @ipapapa

7 months ago

For technical support chatbots with stable system prompts? Full Documents + caching might be the better architecture. Caching changes the ROI calculation completely.

0 0 0 70 0

View Details

Ioannis Papapanagiotou @ipapapa

7 months ago

The trade-off isn't cost vs quality anymore. It's: • Cold start cost (Full Docs higher) • Query volume (Full Docs wins at scale) • Context stability (Full Docs needs consistent prefix) RAG retrieves fragments. Full Docs preserves relationships.

1 0 0 76 0

View Details

Ioannis Papapanagiotou @ipapapa

8 months ago

Retrieval: If you are dealing with non-English technical RAG, don't struggle with generic local models. The specialized multilingual APIs are worth every millisecond of latency saved.⚡️ #AI #RAG

0 0 0 63 0

View Details

Ioannis Papapanagiotou @ipapapa

8 months ago

The new reranker is so effective it retrieved too many similar manuals. I countered this with a System Prompt "Phase Filter"—explicitly instructing the LLM to verify requirements against the retrieved specs.

1 0 0 62 0

View Details

Ioannis Papapanagiotou @ipapapa

8 months ago

Running a high-performance multilingual reranker on a 1 vCPU instance is a recipe for latency. The local ms-marco model was "good enough," but I needed "perfect" handling of foreign technical terminology without crashing the server.

1 0 0 72 0

View Details

Ioannis Papapanagiotou @ipapapa

8 months ago

Model: Gemini 3.0 Pro (Preview) Embeddings: gemini-embedding-001 Reranker: ms-marco-MiniLM (Local) The accuracy difference is night and day. If you aren't reranking, you aren't really searching. 🚀 #AI #RAG #Engineering #Gemini #LLMs

0 0 0 82 0

View Details

Ioannis Papapanagiotou @ipapapa

8 months ago

The LLM kept "lazy-loading" generic definitions from general category files instead of reading the specific manuals. I injected a "Critical Instruction" into the System Prompt. Result: It now ignores the easy answer and digs for the specific brands.