Module 02: Retrieval & Search

Production RAG, Vector DBs & GraphRAG

Interactive technical curriculum synthesized from FAANG and Tier-1 AI startup interview rounds.

Enterprise RAG, High-Dimensional Vector Search & GraphRAG

Comprehensive coverage of QA-02, QA-15, QA-17, QA-23, and QA-37. Master multi-stage hybrid retrieval, Reciprocal Rank Fusion, 50M vector memory scaling, and Leiden community detection.

Interactive RAG Pipeline Flow Explainer

Click each pipeline phase to see the exact production behavior, algorithms, and latency budgets:

Query HyDE & Decomposition
Resolves ambiguous intent (<40ms)

Redis Semantic Vector Cache
Cosine similarity ≥ 0.96 (<15ms)

Two-Stage Hybrid Retrieval
Dense (HNSW) + Sparse (BM25)

Reciprocal Rank Fusion (RRF)
k=60 rank normalization

Cross-Encoder Reranker
BGE-Reranker-Large (Top-100 → Top-5)

Step 1: Query Transformation & HyDE

Why: Raw user queries are noisy, brief, and contain colloquialisms. Standard vector search matches keywords poorly when phrased casually.

Mechanism: A small 0.5B model generates a hypothetical candidate document answering the question. The hypothetical document embedding is then queried against the vector space.

# HyDE Query Transformation
hypothetical_doc = slm.generate(f"Write a technical passage answering: {query}")
query_vector = embedder.encode(hypothetical_doc)

QA-02 • System Design

Whiteboard: End-to-End Production RAG System

Source: Naresh Edagotti (PracticAI) • Track: AI Engineer • Difficulty: Hard

▼

Interview Scenario: "Draw and defend an end-to-end production RAG system serving 10M enterprise documents. How do you handle ambiguous user queries, prevent stale knowledge retrieval, and keep latency under 800ms?"

Key Architectural Breakdown to Deliver:

Query Transformation: Run HyDE and multi-query decomposition to resolve ambiguous slang.
Redis Semantic Cache: Check pre-computed vector embeddings. If similarity ≥ 0.96, return answer immediately (<15ms latency, $0 token cost).
Hybrid Dual-Search: Run dense vector similarity (capturing semantic intent) alongside BM25 sparse keyword search (capturing exact product SKUs, error codes, and IDs).
RRF (Reciprocal Rank Fusion, k=60): Merge both ranked lists via score sum(1 / (60 + rank)).
Cross-Encoder Reranker: Joint attention across top-100 candidates pruned to top-5 high-signal chunks.

Candidate Red Flag: Relying solely on vector embeddings without keyword BM25 search. Vector embeddings frequently fail on specific SKUs, serial numbers, error codes, and function names.

QA-23 • Memory Sizing

50M Vectors: HNSW vs IVF-PQ & Preventing Linux OOM Kills

Source: MLOps Community Debrief • Track: MLOps / Infra • Difficulty: Hard

▼

The Scenario: "Your team is indexing 50,000,000 1,536-dimensional vectors in a vector database. The server crashes with Linux OOM-killer during index build. How do you calculate memory requirements and choose between HNSW and IVF-PQ?"

Interactive Vector RAM Estimator

Vectors to Index (Millions): Dimensions (e.g. OpenAI 1536):

Raw FP32 Data:

307.2 GB

HNSW Index Footprint (2x Graph Overhead):

614.4 GB (OOM Panic!)

IVF-PQ Quantized Footprint:

18.0 GB (Production Safe)

Whiteboard Mathematical Proof:

Raw Vectors = 50M * 1,536 dimensions * 4 bytes (FP32) = 307.2 GB
HNSW Graph Overhead: Each vector stores M=32 to 64 bidirectional graph edges. The graph metadata adds an additional 1.0x to 1.5x footprint, pushing total memory to 614.4 GB RAM.
IVF-PQ Solution: Partitions vector space into 4,096 Voronoi clusters. Product quantization compresses 1,536 floats (6,144 bytes) to 64 bytes. Total memory drops from 614 GB to 18 GB, fitting easily into memory with >95% recall.

QA-37 • Advanced Architecture

GraphRAG vs Classic Vector RAG: Resolving Global Multi-Hop Queries

Source: AI Architect Loop • Track: Architect / Principal • Difficulty: Hard

▼

The Question: "Why does vector search fail when asked: 'What are the top 3 themes across 50,000 customer feedback tickets?' How does GraphRAG solve this?"

The Core Asymmetry: Vector distance captures local semantic affinity (finding paragraphs similar to a phrasing). It cannot aggregate or synthesize across documents that share no common phrasing.

GraphRAG Resolution:

Entity & Relation Ingestion: LLM parses documents into (Subject, Relation, Object) triples.
Leiden Community Detection: Graph is partitioned into hierarchical modular communities (C0, C1, C2).
Community Summaries: LLM pre-summarizes each cluster at ingest time.
Query Time: Global questions map to high-level community report vectors using parallel map-reduce synthesis.

QA-17 • Context Tradeoffs

"If Models Have 1M+ Tokens, Why Not Pass All Docs?"

Source: Syamala Jayanthi • Track: AI Engineer • Difficulty: Medium

▼

Three Critical Production Red Flags:

Latency (TTFT) Explosion: Processing 1M tokens in prefill takes 15 to 45 seconds before the first token is emitted, violating interactive SLAs.
Exponential Financial Burn: At $3.00/1M input tokens, 1,000 queries/day costs $90,000/month compared to $300/month with hybrid RAG.
Attention Dilution ("Lost in the Middle"): Self-attention weights dilute over irrelevant text, increasing hallucinations and missing fine-grained numerical constraints.