RAG is the Backbone of Enterprise AI @Weaviate

Uploaded June 2025 | Updated September 2026, 2 hours ago

“You don’t need RAG, context windows are huge now…”
Sounds great, right?

But the reality is: 𝗯𝗶𝗴𝗴𝗲𝗿 𝗶𝘀𝗻’𝘁 𝗮𝗹𝘄𝗮𝘆𝘀 𝗯𝗲𝘁𝘁𝗲𝗿, especially if it’s slower, costlier, and oblivious to updates.

Join Box’s CTO Ben Kus, along with Bob van Lujit and Connor Shorten, in the latest Weaviate Podcast to learn more about enterprise-scale AI solutions, Box's three-layer infrastructure puzzle, embeddings, retrieval, and much more!

➡️ youtube.com/watch?v=pPvSur8iEXY

Search Agents with Nandan Thakur - Weaviate Podcast #137!

Weaviate 1.29 Release Highlights & Outlook

RAGAS with Jithin James, Shahul Es, and Erika Cardenas - Weaviate Podcast #77!

Data Agents with Shreya Shankar - Weaviate Podcast #135!

$DSPy and ColBERT with Omar Khattab! - Weaviate Podcast #85 Hey everyone! I am beyond excited to present our interview with Omar Khattab from Stanford University! Omar is one of the worlds leading scientists on AI and NLP. I highly recommend you check out Omars remarkable list of publications linked below! This interview completely transformed my understanding of building RAG and LLM applications! I believe that DSPy will be one of the most impactful software project in LLM development because of the abstractions around *program optimization*. Here is my TLDR of this concept of LLM programs and program optimization with DSPy, I of course encourage you to view the podcast and listen to Omars explanation haha. RAG is one of the most popular LLM programs we have seen. RAG typically consists of two components of retrieve and then generate. Within the generate component we have a prompt like please ground your answer based on the search results {search_results}. DSPy gives us a framework to optimize this prompt, bootstrap few-shot examples, or even fine-tune the model if needed. This works by compiling the program based on some evaluation criteria we give DSPy. Now lets say we add a query re-writer that takes the query and writes a new query before sending it to the retrieval system, and a reranker that takes the search results and re-orders them before handing them to the answer generator. Now we have 4 components of query writer, retrieve, rerank, answer. The 3 components of query writer, rerank, and answer all have a prompt that can be optimized with DSPy to enhance the description of the task or add examples! This optimization is done with DSPys Teleprompters. There are a few other really interesting components to DSPy as well such as the formatting of prompts with the docstrings and Signature abstraction, which in my view is quite similar to instructor or LMQL. DSPy also comes with built-in prompts like Chain-of-Thought that offer a really quick way to add this reasoning step and follow a structured output format. I am having so much fun learning about DSPy and I highly recommend you join me in viewing the GitHub repository linked below (with new examples!!): Omar also discusses ColBERT and late interaction retrieval! Omar describes how this achieves the contextualized attention of cross encoders but in a much more scalable system with the maximum similarity between vectors! Stay tuned for more updates from Weaviate as we are diving into multi vector representations to hopefully support systems like this soon! Links: Omar Khattab: https://omarkhattab.com/ DSPy: https://github.com/stanfordnlp/dspy Demonstrate-Search-Predict: https://arxiv.org/abs/2212.14024 DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines: https://arxiv.org/pdf/2310.03714.pdf DSPy Assertions: Computational Constraints for Self-Refining Language Model Pipelines: https://arxiv.org/pdf/2312.13382.pdf Chapters 0:00 Weaviate at NeurIPS 2023! 0:38 Omar Khattab 0:57 What is the state of AI? 2:35 DSPy 10:37 LLM Pipelines 14:24 Prompt Tuning and Optimization 18:12 Models for Specific Tasks 21:44 LLM Compiler 23:32 Colbert or ColBERT? 24:02 ColBERT$

Advanced Chunking Techniques: Semantic & LLM-Based Chunking (Simply!) Explained

The Future of Search with Nils Reimers and Erika Cardenas - Weaviate Podcast #97!

Weaviate vector database |

RAG is the Backbone of Enterprise AI