Hi everyone,
The world of Retrieval-Augmented Generation (RAG) has moved far beyond the simple "embed and search" paradigm of 2023. The pace of innovation is staggering, but it has also created a confusing landscape of new terms and techniques: agentic RAG, late-interaction models, multi-modal retrieval, and more.
How do you know which of these advanced methods are just hype and which are essential for building production-ready systems?
To help answer that, we’ve compiled a new, free resource for the community.
I am excited to share "Beyond Naive RAG: Practical Advanced Methods": an open mini-book that condenses over 5 hours of expert instruction from leading researchers into an easy-to-read, annotated format.
Each chapter is based on a presentation from our AI Evals course, featuring world-class researchers who are pushing the boundaries of what’s possible with RAG systems.
We explore five key areas where the field is advancing:
The "RAG is Dead" Controversy: Ben Clavié sets the record straight on why retrieval is more important than ever.
Modern IR Evaluation: Nandan Thakur explains why traditional search metrics are failing and introduces new benchmarks like FreshStack that measure what really matters for RAG: diversity, grounding, and coverage.
Reasoning-Enhanced Retrieval: Orion Weller from Johns Hopkins University shows how to embed instruction-following and reasoning directly into the retrieval process with models like Promptriever and Rank1.
Late Interaction Models: Antoine Chaffin dives into the limitations of single-vector search and explains how models like ColBERT overcome the information loss problem for superior performance.
Multiple Representations: Bryan Bischof and Ayush Chaurasia provide a powerful framework for building flexible systems that use intelligent routing across multiple, diverse "maps" of your data to better serve user intent.
This mini-book is designed for practitioners. It combines theory with practical recipes and timestamped references to the original videos, allowing you to explore the topics that matter most to you.
Below is the table of contents of the book.
Table of Contents
1: I Don’t Use RAG, I Just Retrieve Documents
1.1 The Title
1.2 The “RAG is Dead” Controversy
1.3 What is RAG Really?
1.4 The Standard RAG Flow
1.5 The Problem with “2023 RAG”
1.6 Single-Vector Search Limitations
1.7 Why Long Context Doesn’t Replace RAG
1.8 Retrieval is Essential
1.9 Key Takeaways
1.10 Better RAG is the Solution
1.11 The Retrieval Landscape
2: Modern IR Evaluation for RAG
2.1 Introduction and Speaker Background
2.2 The History of Information Retrieval
2.3 The Cranfield Paradigm
2.4 The BEIR Benchmark
2.5 Problems with Current Benchmarks
2.6 The RAG Era Changes Everything
2.7 Different Users, Different Goals
2.8 The Evaluation Mismatch
2.9 Introducing FreshStack
2.10 FreshStack Data Sources
2.11 The FreshStack Pipeline
2.12 FreshStack Evaluation Metrics
3: Optimizing Retrieval with Reasoning Models
3.1 LLM Capabilities: Instruction Following and Reasoning
3.2 The Search Paradigm Hasn’t Changed
3.3 Evolution of Search Paradigms
3.4 Understanding Instructions in IR
3.5 Introducing Promptriever and Rank1
3.6 Promptriever: Instruction-Trained Retrieval
3.7 Promptriever Evaluation Results
3.8 Rank1: Reasoning-Based Reranking
3.9 Rank1 Performance Results
3.10 Finding Novel Relevant Documents
4: Late Interaction Models For RAG
4.1 Dense Vector Search Architecture
4.2 Why Dense Models Became Popular
4.3 The Benchmark Problem
4.4 Hidden Limitations
4.5 Long Context Performance
4.6 Complex Retrieval Tasks
4.7 BM25’s Surprising Strength
4.8 The Pooling Problem
4.9 Late Interaction Solution
4.10 Performance Advantages
4.11 Interpretability Benefits
4.12 Barriers to Adoption
4.13 PyLate: Making Late Interaction Accessible
4.14 Future Research Directions
5: RAG with Multiple Representations
5.1 The Map is Not the Territory
5.2 Deconstructing RAG Buzzwords
5.3 A First-Principles View of RAG
5.4 The Three Responsibilities of an IR Engineer
5.5 Practical Application: Curving Space
5.6 Agents as Routers
5.7 Dynamic Representations
5.8 Demo: Semantic Dot Art
5.9 System Architecture
5.10 Integration with Other Techniques
Conclusion
Key Takeaways
Looking Forward
Resources
You can get your free copy here:
We hope this helps you navigate the evolving landscape of RAG and build better AI systems. Know someone who might benefit from learning about advanced RAG methods? Send them this email!
Thanks,
Hamel
P.S. I want to let you know about this code to get 35% off our next cohort of our evals course. You'll get lifetime access to the material and recordings, and learn with students from OpenAI, Meta, Google, WalMart, Airbnb and more.