RAG: Retrieval-Augmented Generation Interview Prep

RAG vs KB-Routing — behavioral trade-offs for production system design

Sample interview questions

1. What is the primary difference between BM25 sparse retrieval and embedding-based dense retrieval?
   • A. BM25 requires a GPU; embedding-based retrieval runs on CPU only.
   • B. BM25 uses term frequency and inverse document frequency to score exact keyword matches; embedding-based retrieval maps queries and documents to a shared vector space to capture semantic similarity, enabling matches on paraphrases and conceptually related terms. ✓
   • C. BM25 is always faster; embedding-based retrieval is always more accurate.
   • D. BM25 works on structured data (tables, JSON); embedding-based works on unstructured text only.

   Option B is correct. BM25 is a probabilistic sparse retrieval algorithm built on TF-IDF principles: it scores documents by how frequently query terms appear (TF) normalized by how rare those terms are across the corpus (IDF), with a document-length penalty. It excels at exact-match queries and handles out-of-vocabulary terms well. Embedding-based (dense) retrieval encodes the query and each document into a fixed-dimensional vector using a neural encoder (Voyage AI, BGE, Nomic Embed, OpenAI text-embedding-3). Similarity is measured by cosine distance, capturing semantic relationships — a query about "cardiac arrest" matches a document about "heart attack" even with no word overlap. Option A is wrong. BM25 is a pure CPU-based algorithm; it does not require a GPU. The GPU/CPU distinction is not what differentiates the two methods. Option C is wrong. BM25 is faster (no neural inference required), and dense retrieval often outperforms BM25 on semantic queries — but BM25 outperforms dense retrieval on exact-match and rare-term queries. Neither is uniformly superior. Option D is wrong. Both BM25 and embedding-based retrieval target unstructured text. Structured data retrieval (SQL, etc.) is a separate layer. Production reality: the standard production pattern is hybrid retrieval combining both methods, fused with Reciprocal Rank Fusion (RRF), which consistently outperforms either method alone across diverse query types.

2. What is Reciprocal Rank Fusion (RRF) and why is it used in production RAG pipelines?
   • A. RRF is a reranking model that uses cross-encoders to rescore retrieval results.
   • B. RRF is a rank-combination formula that fuses ranked lists from multiple retrieval methods (e.g., BM25 and dense) by summing reciprocal ranks, producing a single merged list without requiring score normalization. ✓
   • C. RRF is a technique for reducing embedding dimensionality to improve retrieval speed.
   • D. RRF stands for Retrieval and Ranking Framework — it is an orchestration system for multi-stage retrieval pipelines.

   Option B is correct. Reciprocal Rank Fusion (RRF) is a formula for combining results from multiple retrieval systems with different score scales. For each document in the combined result pool, its RRF score is the sum of 1/(k + rank_i) across each retrieval system that returned it, where rank_i is its rank position and k is a smoothing constant (commonly 60). The key insight is that RRF is rank-based, not score-based — it does not require normalizing BM25 scores (unbounded) against cosine similarity scores (0–1). RRF consistently outperforms any single retrieval method across diverse query types. Option A is wrong. RRF is not a reranking model. Cross-encoder rerankers (like Cohere Rerank) are a separate, more computationally expensive step that re-scores individual query-document pairs with full context. RRF is a lightweight formula applied to ranked lists. Option C is wrong. Dimensionality reduction (e.g., PCA, product quantization) is a different technique for speeding up approximate nearest-neighbor search. RRF operates on ranked lists, not on embedding vectors. Option D is wrong. RRF is a specific mathematical formula (Cormack et al., 2009), not a general framework name. Production reality: hybrid-search support is provider-specific. Qdrant and Weaviate expose built-in fusion (RRF and others); pgvector is vector-similarity only, so a hybrid pipeline pairs it with Postgres full-text search and fuses the two result lists in application or SQL logic. Enabling it is rarely a single universal flag, but it usually produces an immediate retrieval-quality improvement.

3. Your RAG system has high retrieval recall but low answer faithfulness — the generated answers contain claims not in the retrieved passages. What is most likely broken?
   • A. The embedding model needs to be upgraded to a higher-dimensional model.
   • B. The generator LLM is adding claims from its parametric knowledge beyond the retrieved context. The grounding instruction in the system prompt needs to be strengthened, and a faithfulness scorer should be added to measure and enforce this. ✓
   • C. The top-k selection is too large — reduce to k=3 to force the model to focus.
   • D. The chunking strategy is wrong — larger chunks will fix the faithfulness issue.

   Option B is correct. When recall is high (relevant documents are being retrieved) but faithfulness is low (answers contain unsupported claims), the failure is in the generation phase: the model mixes retrieved information with parametric knowledge rather than staying within the retrieved context. The fix has two parts: (1) strengthen the grounding instruction — add an explicit instruction such as "Answer only using information from the provided context. If the context does not contain the answer, say so." Make this instruction prominent and specific; (2) add automated faithfulness evaluation using RAGAS faithfulness or HHEM-2.1 (Vectara) to measure and catch this failure continuously. Option A is wrong. The embedding model affects retrieval recall and ranking. Upgrading it does not change generation behavior — the problem statement says recall is already high. Option C is wrong. Reducing k would cut context, potentially eliminating relevant passages. This might accidentally improve faithfulness by giving the model less irrelevant context to mix with parametric knowledge, but the correct fix is generation constraint, not retrieval reduction. Option D is wrong. Chunk size affects whether relevant information fits in a single chunk, but does not directly address the model's tendency to add parametric knowledge beyond retrieved context. Production reality: RAGAS faithfulness scores below 0.7 on a sample of production queries indicate a grounding instruction problem. Test with oracle-injected context to confirm the fix before re-evaluating end-to-end.

4. A customer-support agent needs to handle 200 known FAQ topics and a long tail of open-ended policy questions. What is the strongest architecture?
   • A. Pure RAG over all content — retrieve the right FAQ or policy chunk and synthesize an answer for every query.
   • B. Hybrid: route high-confidence, known-intent queries to pre-defined templates (KB-Routing), and fall back to RAG for the long tail open-ended queries. ✓
   • C. Pure KB-Routing with 200+ templates — one per FAQ topic, and add more templates over time.
   • D. Fine-tune a small model on all 200 FAQ pairs and deploy it as a classifier.

   Option B is correct. This is the production-default hybrid pattern: a router classifies intent with a confidence threshold; high-confidence known-intent queries go to deterministic templates (cheap, fast, predictable output); low-confidence or open-ended queries fall through to RAG (flexible, handles the long tail). Templates cover the 80% repetitive volume with deterministic quality; RAG handles the 20% long tail. The key operational lever is the confidence threshold on the router — tune it so precision stays above ~95% before routing to a template. Option A is wrong. Pure RAG over all content would handle known FAQs by retrieving the same chunks and synthesizing slightly different wording each time — costing 10–100x more per query than serving a template, adding variable latency, and producing output that isn't consistent between identical queries. Option C is wrong. Starting with 200 templates is already at the upper end of manageable maintenance. As query patterns evolve, maintaining 300+ templates becomes a liability — stale templates silently return outdated answers. When template count exceeds your maintenance budget (typically 30–100), the long tail belongs in RAG. Option D is wrong. Fine-tuning a classifier doesn't produce answers; it only classifies. You still need an answer-generation mechanism for each class, which reintroduces the template-maintenance or RAG problem. Production reality: most production customer-support agents, banking assistants, and healthcare bots use the hybrid route-first pattern. The confidence threshold is the load-bearing knob — it determines whether a mis-routed query produces a confidently-wrong template response or correctly falls through to RAG.

5. How do you evaluate RAG quality when you have no human-labeled reference answers?
   • A. You cannot evaluate without human labels — pause deployment until labeled data is available.
   • B. Use reference-free evaluation: RAGAS context precision and faithfulness (no reference answer needed), LLM-as-judge scoring with a rubric, and user proxy signals (follow-up questions, click-through) as weak supervision. ✓
   • C. Evaluate on a public benchmark dataset for question answering — results on public benchmarks predict production quality.
   • D. Use BLEU or ROUGE scores against the retrieved passages as reference text.

   Option B is correct. Evaluating RAG without human-labeled reference answers is possible through reference-free methods: (1) RAGAS context precision measures whether each retrieved passage is relevant to the query — no reference answer needed; RAGAS faithfulness measures whether the generated answer is supported by retrieved context — also reference-free; (2) LLM-as-judge with a rubric (e.g., "Does this answer directly address the query? Is every claim supported by the provided context?") provides a reference-free quality signal; (3) user proxy signals — follow-up questions suggesting incomplete answers, or query rephrasing suggesting dissatisfaction — provide weak supervision from implicit feedback. Option A is wrong. Reference-free evaluation methods are widely used in production RAG systems. Pausing deployment is unnecessarily conservative. Option C is wrong. Public benchmarks (Natural Questions, TriviaQA, SQuAD) test general factual QA, not domain-specific RAG performance on your knowledge base. A system that scores 85% on NQ may perform poorly on your enterprise domain. Option D is wrong. BLEU and ROUGE measure n-gram overlap with a reference string. Using retrieved passages as the reference measures whether the answer copies from context, not whether it is correct or helpful. This rewards verbatim extraction. Production reality: RAGAS is a widely used framework for reference-free RAG evaluation (DeepEval, TruLens, Phoenix/Arize, and custom LLM judges are common alternatives). Run it on a sample of 100–200 production queries weekly to catch quality drift before users report it.

6. Your RAG knowledge base contains documents across legal, medical, and finance domains, each requiring different retrieval precision and faithfulness thresholds. What architecture best handles this?
   • A. Use a single RAG pipeline with a very large top-k to ensure all domain-relevant documents are retrieved.
   • B. Domain-stratified RAG: a domain classifier routes each query to a domain-specific RAG pipeline (legal-RAG, medical-RAG, finance-RAG), each with its own chunking, reranker, and faithfulness threshold. ✓
   • C. Fine-tune a separate embedding model for each domain and use them in a single shared retrieval index.
   • D. Use keyword-only BM25 retrieval, since exact-match keywords are the most reliable cross-domain retrieval signal.

   Option B is correct. Domain-stratified RAG addresses the problem of large, heterogeneous knowledge bases where different domains have different precision requirements, vocabulary distributions, and safety thresholds. A domain classifier on top routes each query to the appropriate domain-specific pipeline. Each domain gets: its own chunk corpus (reducing the retrieval search space 5–50x), its own reranker tuned for that domain's vocabulary, and its own faithfulness threshold (e.g., medical RAG may gate at 0.85; general RAG at 0.70). This also makes per-domain quality monitoring tractable — you can detect a regression in legal-RAG without it being buried in aggregate metrics. Option A is wrong. A very large top-k retrieves more documents but does not solve the precision problem — domain-irrelevant documents still compete with domain-relevant ones. Larger k also increases context-window costs and can dilute the relevant content. Option C is wrong. Fine-tuning separate embedding models per domain and merging them into a single index creates a different problem: the embedding spaces of different fine-tuned models are not comparable, so nearest-neighbor search across them is meaningless without domain-specific routing anyway. Option D is wrong. BM25-only retrieval would miss semantic paraphrases and domain-specific synonyms. In medical and legal domains where terminology is dense and precise, hybrid retrieval consistently outperforms BM25-only. Production reality: domain-stratified RAG is a common architecture in regulated industries (legal, healthcare, finance). The domain classifier is typically a lightweight embedding-based router (semantic-router or a small classifier) that adds minimal latency.

7. What is the most commonly overlooked tuning lever in RAG pipelines, and what are the correct parameters for standard text documents?
   • A. The LLM temperature setting — lower temperature improves faithfulness.
   • B. Chunking strategy — splitting documents into pieces before embedding. Standard parameters are 256–1024 tokens per chunk with 20–50 token overlap. ✓
   • C. The number of attention heads in the embedding model — more heads improve retrieval recall.
   • D. The reranker model size — always use the largest available cross-encoder.

   Option B is correct. Chunking is the most-overlooked tuning lever in RAG, per the canonical definition (GLOSSARY: "Splitting a document into pieces before embedding. The most-overlooked tuning lever in retrieval."). The failure modes of poor chunking are severe: chunks too short (50 tokens) lose context — the LLM sees fragments and confabulates around them; chunks too long (5000 tokens) bury the relevant sentence, reducing retrieval precision and faithfulness. Standard parameters for most text documents are 256–1024 tokens per chunk with 20–50 token overlap. Overlap prevents relevant sentences from being split across chunk boundaries. The right value depends on document structure — narrative text may tolerate larger chunks; FAQ-style documents benefit from smaller, self-contained chunks. Option A is wrong. LLM temperature affects output diversity but not faithfulness in the sense of grounding claims in retrieved passages. A temperature of 0 does not prevent parametric hallucination; a strong grounding instruction does. Option C is wrong. The number of attention heads is an architecture-level property of the embedding model, not a RAG tuning lever. Teams choose an embedding model from available options (Voyage AI, BGE, Nomic Embed, OpenAI text-embedding-3); they do not tune internal model architecture. Option D is wrong. Reranker model size involves a latency-quality trade-off, but "always use the largest" is operationally impractical and ignores per-query latency budgets. Chunking is the higher-leverage first tuning lever. Production reality: most RAG teams ship with default chunk sizes from their framework (often 512 tokens) and never revisit. Building a small evaluation set and measuring recall@5 at 256, 512, and 1024 token chunk sizes takes an afternoon and often yields significant improvements.

8. A RAG system produces well-written, fluent answers, but auditors cannot verify which specific documents support each claim. What is missing, and what does it require from the LLM?
   • A. The system is missing a reranker — adding Cohere Rerank will automatically attribute claims to sources.
   • B. The system is missing citation markers — the LLM must be instructed to emit explicit references (e.g., [chunk 2] or [doc-id]) for each claim, and downstream code should verify every claim has a citation marker. ✓
   • C. The system needs a larger context window to include all source documents simultaneously.
   • D. The system needs a knowledge graph overlay to track entity provenance automatically.

   Option B is correct. A citation marker is an explicit reference in a RAG answer pointing back to which retrieved chunk it used — required for verifiability (GLOSSARY: "Explicit reference in a RAG answer pointing back to which chunks it used. Required for verifiability."). Without citation markers, a RAG answer is unverifiable: auditors, compliance reviewers, and downstream citation-drift detectors cannot check whether claims are actually supported by retrieved documents. The implementation requires: (1) instructing the LLM to emit citations in a structured format (e.g., "[chunk 2]" or "[doc-id: xyz]") for each claim; (2) downstream code that checks citation-presence as a structural requirement — if a claim has no citation marker, the answer fails quality gating. An adversarial critic can additionally verify whether cited chunks actually support the stated claims. Option A is wrong. A reranker improves retrieval precision by re-scoring candidates, but it does not automatically generate attribution markers in the output text. Attribution requires explicit LLM instruction. Option C is wrong. A larger context window allows more retrieved documents to fit, but does not solve the attribution problem — the LLM still needs to be instructed to cite which portion of the context each claim comes from. Option D is wrong. A knowledge graph overlay tracks entity relationships at ingestion time and supports graph-based retrieval, but does not automatically generate in-text citation markers for LLM outputs. Production reality: citation-presence enforcement is a structural quality gate. Systems that skip this discover attribution failures only when an auditor asks "where did you get that?" — at which point the answer is unrecoverable from the log.

9. What distinguishes Self-RAG from standard single-pass RAG, and when does the added complexity pay off?
   • A. Self-RAG uses a larger embedding model and retrieves more documents per query — it is strictly better for all use cases.
   • B. Self-RAG uses special reflection tokens to decide whether retrieval is needed, assess retrieved document relevance, and evaluate the faithfulness of its own draft — all within a single inference pass. The complexity pays off when queries vary in whether retrieval is needed and when retrieval quality varies by query. ✓
   • C. Self-RAG stores retrieved documents in a self-updating cache to avoid re-retrieval on repeat queries.
   • D. Self-RAG is identical to standard RAG but uses a fine-tuned generator LLM.

   Option B is correct. Self-RAG (Asai et al., 2023) is an agentic RAG variant where the model emits special reflection tokens to decide: (a) whether retrieval is needed at all (if not, it answers from parametric knowledge); (b) whether each retrieved segment is relevant (it scores and filters retrieved content); (c) whether its own generated draft is faithful to the retrieved context. This all happens within a single inference pass — no separate orchestration loop required. The complexity pays off when: queries vary widely in whether retrieval is needed (some questions don't require external context); retrieval quality is heterogeneous (some retrieved docs are irrelevant and should be filtered); and reducing hallucination on open-domain queries with heterogeneous information needs is a priority. Self-RAG adds latency (the reflection tokens require more generation) but significantly reduces hallucination on these query types. Option A is wrong. Self-RAG does not simply retrieve more documents. Its defining feature is conditional retrieval and self-assessment, not scale. Option C is wrong. Self-RAG does not maintain a cache of retrieved documents. That describes a semantic caching pattern, which is distinct. Option D is wrong. Self-RAG requires a fine-tuned model that emits the special reflection tokens. It is not identical to standard RAG with a different generator; the training procedure is part of the pattern. Production reality: most teams should start with single-pass RAG and escalate to agentic patterns (Self-RAG, CRAG, or LangGraph agentic RAG) only when single-pass quality plateaus on measured failures — not in anticipation of them. Agentic RAG is slower and more expensive; the quality gain must be measured to justify the cost.

10. How do you choose an embedding model for a production RAG pipeline over enterprise technical documentation?
    • A. Always use the largest available embedding model for the best results.
    • B. Evaluate models on your domain specifically: test Voyage AI, BGE, Nomic Embed, and OpenAI text-embedding-3-small on a sample of your documents and queries, measure recall@k and latency, and select the best cost-recall-latency tradeoff for your volume. ✓
    • C. Use whichever embedding model is provided by your LLM provider for maximum compatibility.
    • D. Embedding model selection does not significantly affect RAG quality — any recent model will do.

    Option B is correct. Embedding model selection has a significant impact on retrieval quality, and the right choice depends on your domain and constraints. The evaluation process: (1) create a sample evaluation set of 50–200 query-document pairs from your domain; (2) measure recall@5 and recall@10 for each candidate model; (3) measure latency per embedding and cost per token; (4) compare top candidates — Voyage AI (strong on technical domains), BGE (strong MTEB performance, open weights), Nomic Embed (open weights, competitive quality), OpenAI text-embedding-3-small (good balance of quality and cost for general domains). The best model for enterprise technical documentation may not match public MTEB rankings because MTEB benchmarks use general-domain data. Option A is wrong. Larger embedding models are not always better. Nomic Embed (768d) often matches or exceeds larger models on domain-specific tasks at lower cost and latency. Always evaluate on your specific domain. Option C is wrong. Vendor compatibility (using OpenAI embeddings with GPT-4o) is operationally convenient but not a quality criterion. The embedding model and the generator LLM interact only through retrieved text — they do not need to be from the same vendor. Option D is wrong. Embedding model quality varies significantly, and the differences materialize clearly in recall metrics. A model with 10 percentage points lower recall@5 produces meaningfully worse end-to-end RAG quality. Production reality: Voyage AI's voyage-3 and voyage-code-3 models consistently outperform general-purpose models on technical and code documentation. If your knowledge base is primarily technical documentation or code, evaluate voyage-code-3 specifically.

Frequently asked questions

What is RAG (Retrieval-Augmented Generation)?

RAG is a pipeline pattern: retrieve content from a knowledge base, inject it into an LLM prompt as context, and ask the LLM to synthesize an answer grounded in the retrieved chunks. The LLM controls how retrieved facts are expressed; the knowledge base controls which facts the LLM has access to. Every answer is composed at runtime — no two answers to the same question are guaranteed to be identical. The two main quality concerns are faithfulness (does the answer actually follow from the retrieved chunks?) and completeness (did retrieval find the right chunks?). Production RAG pipelines combine: query embedding, hybrid retrieval (BM25 + dense with RRF), reranking (cross-encoder), context assembly, and LLM synthesis.

How do you build RAG over technical standards like 3GPP or IEEE, or over datasheets?

Index the published specification or datasheet corpus — 3GPP TS/TR documents, IEEE 802.11 editions and amendments, device datasheets and their errata — then have the assistant answer only from retrieved passages with a citation back to the exact clause or page, never from the model's parametric memory, which confidently invents clause numbers, register addresses, and parameter values. Three choices matter more than in general-purpose RAG: (1) chunk on document structure — clause and section boundaries, table rows, register-map entries — rather than fixed token windows, so a single normative requirement is not split across chunks; (2) preserve and index version and document metadata (3GPP release, IEEE 802.11 edition/amendment, datasheet revision), because standards are version-dependent and returning the wrong release is a wrong answer; (3) enforce citation markers so an engineer can jump to the source clause and confirm the normative text before acting on it. Hybrid retrieval (BM25 + dense fused with RRF) earns its place here because spec and datasheet text mixes exact identifiers — message names, information elements, register fields — that reward keyword match with conceptual questions that reward semantic match. This is the RAG pattern behind AI-assisted standards navigation for wireless and firmware engineers (see /topics/ai-native-tools-for-telecom).

How is RAG different from KB-Routing?

Both patterns retrieve from a knowledge base and end with an LLM call, but they produce fundamentally different systems. RAG retrieves content chunks and asks the LLM to synthesize a new answer — output is compositional, high-variability, and grounded in the retrieved text. KB-Routing retrieves a predetermined template or workflow and asks the LLM to follow it — output is low-variability and template-shaped, with the response structure controlled by the template, not composed at runtime. RAG is better for open Q&A, summarization, and knowledge synthesis. KB-Routing is better for customer support, regulated outputs, and structured workflows where predictability matters. Most production systems use both: route predictable intents to templates, fall back to RAG for the long tail.

What are the main components of a RAG system?

An embedding model converts text to dense vectors — OpenAI text-embedding-3-small/large, Voyage AI, BGE (BAAI), or Nomic Embed are common choices. A vector database stores and searches those embeddings (see /topics/vector-databases-and-embeddings) — pgvector (PostgreSQL extension), Qdrant, Weaviate, Pinecone, Chroma, and LanceDB are the main options. A retriever finds top-K documents by approximate nearest-neighbor search. An optional reranker (Cohere Rerank or a cross-encoder from Hugging Face) re-scores candidates by joint query-document similarity. The generator (LLM) reads the retrieved and reranked context and produces the final answer. Frameworks like LlamaIndex, LangChain, and Haystack orchestrate these components into a composable pipeline.

What is chunking and why does it matter so much?

Chunking is splitting documents into pieces before embedding — the most-overlooked tuning lever in RAG. Chunks too short (50 tokens) lose context: the LLM sees fragments and confabulates around them. Chunks too long (5000 tokens) bury the relevant sentence, reducing retrieval precision and faithfulness. Standard parameters are 256–1024 tokens per chunk with 20–50 token overlap. Overlap prevents relevant content from being split across chunk boundaries. The right value depends on document structure — narrative text tolerates larger chunks; FAQ-style documents benefit from smaller, self-contained chunks. Measuring recall@5 at multiple chunk sizes on a representative query set takes an afternoon and often yields significant improvements.

What are the eight most common RAG failure modes?

From most to least impactful: (1) Over-RAG when KB-Routing would do — predictable queries going through expensive RAG instead of cheap templates; (2) bad chunking — chunks too short lose context, chunks too long bury relevance; (3) missing hybrid retrieval — pure dense search misses exact-match keywords, pure BM25 misses semantic paraphrases; (4) missing reranker — top-K bi-encoder search has high recall but moderate precision, the 10th result is often noise; (5) faithfulness failure — model adds parametric knowledge beyond retrieved context; (6) stale KB-Routing templates — policy changed, template not updated, confidently-wrong answers ship at scale; (7) missing fallback path — router returns no confident match, system fails with "I don't know" instead of falling through to RAG; (8) faithfulness drift — model upgrade changes citation behavior, scores drop silently without monitoring.

What are citation markers and why are they required for verifiability?

A citation marker is an explicit reference in a RAG answer pointing back to which retrieved chunk it used. Without citation markers, a RAG answer is unverifiable: auditors and compliance reviewers cannot check whether claims are actually supported by source documents. The implementation requires: instructing the LLM to emit citations in a structured format (e.g., "[chunk 2]" or "[doc-id: xyz]") for every claim; and downstream code that checks citation-presence as a structural gate — if a claim has no citation marker, the answer fails quality requirements. An adversarial critic can additionally check whether cited chunks actually support the stated claims. Production RAG systems that skip citation enforcement discover attribution failures only when an auditor asks "where did you get that?" — at which point the log cannot recover the answer.

What is Reciprocal Rank Fusion (RRF)?

Reciprocal Rank Fusion is a formula that merges ranked lists from multiple retrievers without requiring calibrated similarity scores. For each document, RRF computes 1 / (k + rank) from each retriever that returned it, where k is a smoothing constant (typically 60). Scores are summed and documents re-ranked by total score. RRF is robust to outlier rankings — a document ranked #1 by one retriever but #100 by another still gets a strong merged score. It is the standard merging strategy when combining BM25 and dense embedding results in hybrid RAG pipelines. Support is provider-specific: Qdrant and Weaviate expose RRF-based hybrid fusion natively, while pgvector implements vector similarity only, so a hybrid pipeline pairs it with Postgres full-text search and applies RRF in application or SQL logic.

What is Self-RAG and when does the added complexity pay off?

Self-RAG (Asai et al., 2023) is an agentic RAG variant where the model emits special reflection tokens to: decide whether retrieval is needed at all, score each retrieved segment's relevance, and evaluate the faithfulness of its own draft output — all within a single inference pass. Unlike standard RAG (always retrieve), Self-RAG conditionally retrieves: if the model judges retrieval unnecessary, it answers from parametric knowledge. CRAG (Corrective RAG) extends this with a web-search fallback when the local vector store returns low-confidence results. Both add latency but reduce hallucination significantly on open-domain queries with heterogeneous information needs. Most teams should start with single-pass RAG and escalate to agentic patterns only on measured failures — not in anticipation of them.

What is domain-stratified RAG and when should you use it?

Domain-stratified RAG routes each query to a domain-specific RAG pipeline (legal-RAG, medical-RAG, finance-RAG), each with its own chunks, reranker, and faithfulness threshold. A domain classifier on top — typically a lightweight embedding-based router — fans queries out to the appropriate domain pipeline. This reduces the retrieval search space 5–50x per domain, makes per-domain rerankers feasible, and lets each domain tune its own quality thresholds (e.g., medical RAG gates at 0.85 faithfulness; general RAG at 0.70). Use it when: the knowledge base is large and stratified across domains with different vocabulary distributions; different domains require different precision or safety guarantees; or aggregate quality metrics are masking per-domain regressions.

How do you evaluate faithfulness without human labels?

RAGAS faithfulness runs an LLM judge that decomposes the generated answer into atomic claims, then checks each claim against the retrieved documents. A claim supported by at least one retrieved document counts as faithful; unsupported claims reduce the faithfulness score. HHEM-2.1 from Vectara is a purpose-trained cross-encoder that detects hallucination in (premise, hypothesis) pairs — pass (retrieved doc, generated claim) pairs to get a numerical hallucination probability. Both run in CI against a regression set and as online sampled metrics in production (see /topics/llm-evaluation-best-practices). A low faithfulness score signals a generation/grounding problem — the answer is not supported by the retrieved context (the model is adding parametric claims) — rather than a retrieval problem; a retrieval failure shows up instead as low context-precision/recall. ARES provides an alternative with fine-tuned classifier-based evaluation and statistical confidence intervals.

Related topics