DeployQL @DeployQL

We're adding support for XTR in LintDB. We can score more documents with the same performance as ColBERT, and we're the only database that supports this. Customers benefit when your LLM can use the right data.

We're advancing responsible AI with a larger platform. Store, retrieve, and now interpret retrieved results. This gives our customers more trust in our results. Get in touch if your Gen AI applications require higher reliability and trustworthiness.

LintDB's latest prerelease adds collections. Collections will encode your text for you, store text metadata, and retrieve it during the search process. This massively simplifies the setup to operationalize ColBERT.

LintDB: github.com/DeployQL/LintDB

v0.2 of LintDB is out! 🚀Incorporates a new retrieval process found in "Efficient Multi-Vector Dense Retrieval with Bit Vectors" 🚀Linux artifacts are built against MKL for faster BLAS. Stay tuned for v0.3, where we'll add an ONNX server for CoLBERT models.

DeployQL is the only vector database that is purpose-built for late interaction embeddings. We provide a platform to bootstrap and fine tune retrievers, index data, retrieve it, and analyze overall performance. Fully managed model iteration. Better results in less time.

Retrieval is a hard topic within RAG, but state of the art is advancing quickly! 🚀 Late Interaction and multiple embeddings per doc performs the best on complex queries. 🖥️Synthetic data generation boosts relevancy for better performance. Here's how DeployQL can help you:

Decode, detect & diagnose issues in your #LLMs! Join us for a workshop on troubleshooting user behavior changes & performance drift in LLMs with text quality metrics like sentiment, toxicity, vocabulary choice & sentence structure! Register now: bit.ly/3HEIWuE

3. Generate query tasks Expand your training data coverage by inferring how your application is used. This helps evaluate where users may encounter problems and helps stakeholders gain confidence heading to production.

2. Generate positive and negative document pairs Create fully synthetic documents based on the queries alone. The query can even be synthetic, too! This has been shown to be valuable by itself. This is a great way to bootstrap a fine tuned embedding model from scratch!

Synthetic data is one way to increase the relevancy of results from RAG. Fine tuning retrievers on it offers a few real-world benefits: 1. 🚀Better user experiences. 2. 🧍Coverage of more user behavior. 3. 🔏Privacy protection for your data. Here are a few ways to use it:

Retri-evals is still evolving, and we want your opinion! We use MTEB to evaluate against a range of baseline datasets, and we're adding support for synthetic dataset building. Where are you struggling with RAG?

3. Everyone has experimented with different chunking, embedding models, prompts, and index types. What works best for you? Depending on your scale, reprocessing your data is going to cost money. Retri-evals wants to make benchmarking these options simpler so you move faster.

2. When moving to production, we want to know infrastructure metrics -- latency, index size, cost. Tradeoffs need to be made that will impact the quality of results. Retri-evals aims to surface these tradeoffs.

The repo: github.com/DeployQL/retri… 1. It's not clear how to evaluate RAG solutions. LLM-as-a-judge metrics is a great start, but how do we know if the LLM generated a bad response, or if retrieval isn't retrieving the right data? Retri-evals evaluates retrieval directly.

Working on RAG applications? We're building a way to bootstrap better retrieval and ranking algorithms for Gen AI. We've open sourced our repo to evaluate against MTEB and soon bootstrap your own dataset. Here's why this matters. 🧵

What is 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗺𝗼𝗱𝗲𝗹 𝗖𝗼𝗺𝗽𝗿𝗲𝘀𝘀𝗶𝗼𝗻 and why you might need it? When you deploy Machine Learning models to production you need to take into account several operational metrics that are in general not ML related. Today we talk about two of them: 👉 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗟𝗮𝘁𝗲𝗻𝗰𝘆: How long does it take for your Model to compute inference result and return it. 👉 𝗠𝗼𝗱𝗲𝗹 𝗦𝗶𝘇𝗲: How much memory does your model occupy when it’s loaded for serving inference results. Both of these are important when considering operational performance and feasibility of your model deployment in production. 👉 Large models might not fit on a device if you are considering edge deployments. 👉 Latency of retrieving inference results might make business case non feasible. E.g. Recommendation Engines require latencies in milliseconds as ranking has to be applied as the user browses your website or app in real time. 👉  … You can influence both latency and size by applying different Model Compression methods, some of them are: ➡️ 𝗣𝗿𝘂𝗻𝗶𝗻𝗴: this method is mostly used in tree-based and Neural Network algorithms. In tree-based ones we prune leaves or branches from decision trees. In Neural Networks we remove nodes and synapses (weights) while trying to retain ML performance metrics. ✅ In both cases the output is a reduction in the number of Model Parameters and model size. ➡️ 𝗞𝗻𝗼𝘄𝗹𝗲𝗱𝗴𝗲 𝗗𝗶𝘀𝘁𝗶𝗹𝗹𝗮𝘁𝗶𝗼𝗻: this type of compression is achieved by: 👉 Training an original large model which is called the Teacher model.  👉 Training a smaller model to mimic the Teacher model by transferring knowledge from it, this model is called the Student model. Knowledge in this context can be extracted from the outputs, internal hidden state (feature representations) or a combination of both. 👉 We then use the “Student” model in production. ➡️ 𝗤𝘂𝗮𝗻𝘁𝗶𝘇𝗮𝘁𝗶𝗼𝗻: a most commonly used method that doesn’t have much to do with Machine Learning. This approach uses fewer bits to represent model parameters. 👉 You can apply quantization techniques both during the training and after the models has been already trained. 👉 In regular Neural Networks what is quantized are Model Weights, Biases and Activation Functions. 👉 Most usual quantization is from float to integer (32 bits to 8 bits). ➡️ … [𝗜𝗺𝗽𝗼𝗿𝘁𝗮𝗻𝘁]: while the above methods do reduce the size of the models, allowing them to be deployed in production scenarios, there is almost always a reduction in accuracy so be careful and evaluate it accordingly. -------- Follow me to upskill in #MLOps, #MachineLearning, #DataEngineering, #DataScience and overall #Data space. Also hit 🔔to stay notified about new content. 𝗗𝗼𝗻’𝘁 𝗳𝗼𝗿𝗴𝗲𝘁 𝘁𝗼 𝗹𝗶𝗸𝗲 💙, 𝘀𝗵𝗮𝗿𝗲 𝗮𝗻𝗱 𝗰𝗼𝗺𝗺𝗲𝗻𝘁! Join a growing community of Data Professionals by subscribing to my 𝗡𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿: newsletter.swirlai.com

Combining DuckDB and PyArrow allows you to efficiently process datasets larger than memory on a single machine. In the following code, running DuckDB on a #PyArrow dataset is approximately 2906 times faster than running #DuckDB on a #pandas DataFrame.

@Aurimas_Gr Also worth calling out the speedup from batching requests at every step instead of many single requests. As systems evolve and grow, it's easy to miss code that accidentally loops over a network request instead of making only one request.