← Back to Index

System Design Principles: Building Production-Grade RAG

The architectural principles that separate a RAG demo from a production-grade retrieval system.


The Five Properties of a Production RAG System

Every production RAG system must optimize for five properties. Interviews test whether you understand the trade-offs between them.

1. Reliability

2. Latency

3. Freshness

4. Controllability

5. Observability


Pipeline vs. Agent Architecture: The Core Trade-off

These are the two high-level organizational patterns for any RAG system. Every architecture in the taxonomy is an instance of one or the other.

Pipeline Architecture

User Query
    │
    ├──► Embedding
    │     └──► Encoding
    │
    ├──► Retrieval (Fixed Strategy)
    │     └──► Top-k ANN Search
    │
    ├──► Reranking (Optional)
    │     └──► Cross-Encoder
    │
    └──► LLM Generation
          └──► Fixed Prompt Template
          
    Answer

Strengths:

Weaknesses:

When to choose: Most production systems start here. Naive, Advanced, Modular RAG are pipelines.


Agent Architecture

User Query
    │
    ├──► Agent Decides: [Retrieve? Reason? Retrieve Again?]
    │
    ├──► Retrieve (Agent-Chosen Strategy)
    │     └──► Re-evaluate: Is Context Sufficient?
    │
    ├──► Generate (Partial Answer)
    │
    ├──► Agent Decides: [Done? Need More Context?]
    │
    └──► Loop Until Done
    
    Answer

Strengths:

Weaknesses:

When to choose: When simple retrieval isn't enough. Agentic RAG, Self-RAG are agents.

Dimension Pipeline Agent
Latency <300ms (2–3 calls) >500ms (4–8 calls)
Determinism High Low
Debuggability High Low
Cost predictability High Low
Failure recovery Manual (circuits) Implicit (agent decides)
Routing complexity Hard-coded Learned (LLM decides)

Incremental Migration: Pipeline → Agent

Don't rewrite your whole system. Add agent logic incrementally:

# Start: Pure pipeline
def rag_v1(query: str) -> str:
    context = retrieve(query)
    return generate(query, context)

# v2: Add a validation step (Corrective RAG pattern)
def rag_v2(query: str) -> str:
    context = retrieve(query)
    if not is_sufficient(context):  # feedback
        context = retrieve(query, reranked=True)
    return generate(query, context)

# v3: Add agent logic (Agentic RAG pattern)
def rag_v3(query: str) -> str:
    agent_state = initialize_agent(query)
    while not agent_state.done:
        action = agent.decide(agent_state)
        if action == "retrieve":
            context = retrieve(query, agent_state.strategy)
            agent_state.update(context)
        elif action == "generate":
            answer = generate(query, agent_state.context)
            agent_state.mark_done(answer)
    return agent_state.answer

Scalability Patterns

As your corpus grows, different bottlenecks emerge. These patterns address them.

Pattern 1: Read-Path Optimization

Optimize the retrieval and ranking phases.

Query
  │
  ├──► Embedding Cache (check if embedding exists)
  │    └──► Miss → Embed online → Cache
  │
  ├──► Vector DB (partitioned index)
  │    └──► Partition 1  ┐
  │         Partition 2  ├──► Merge Top-k
  │         Partition 3  ┘
  │
  ├──► Reranker (GPU-accelerated, batched)
  │
  └──► Answer

Techniques:

Thresholds:


Pattern 2: Write-Path Optimization

Optimize indexing and updates.

New Document
  │
  ├──► Async Queue (batch documents)
  │
  ├──► Chunker (parallelize chunking)
  │
  ├──► Embedder (batch embed; share embedding service)
  │
  ├──► Vector DB Insert (batch insert)
  │
  └──► Index Ready (async; doesn't block user)

Techniques:

Thresholds:


Pattern 3: Horizontal Scaling

Scale retrieval across multiple machines.

Query
  │
  ├──► Load Balancer
  │
  ├──► Retrieval Node 1 ┐
  │    Retrieval Node 2 ├──► Merge Results
  │    Retrieval Node 3 ┘
  │
  └──► Answer

Techniques:

Thresholds:


Failure Modes and Circuit Breakers

Production systems fail. Design for graceful degradation.

Component Failure Mode Detection Signal Mitigation Circuit Breaker Pattern
Embedding Service Timeout or error; embeddings are slow Latency spike >500ms or error rate >5% Fall back to BM25 (sparse-only retrieval) If embedding fails 3x in 10s, skip embeddings for 30s
Vector DB Out of memory or corrupted index Query error; timeout Use cached embeddings; skip reranking If VectorDB fails 3x in 10s, return empty result + fallback
Reranker Overloaded; queue backs up Latency >5s per request Skip reranking; return top-k from dense retrieval If reranker latency >1s, disable reranking for 1 minute
LLM Service Rate-limited or down 429 or 503 errors Return retrieved context as-is; don't generate If LLM fails 5x, serve pre-generated summaries from cache
Chunker Malformed document causes parsing error Exception; incomplete chunks Skip document; log error; alert If chunker fails on >10% of batch, stop processing

Circuit Breaker Pattern in Python:

from datetime import datetime, timedelta

class CircuitBreaker:
    def __init__(self, failure_threshold=3, recovery_timeout=30):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.failure_count = 0
        self.last_failure_time = None
        self.state = "closed"  # "closed" = normal, "open" = failing, "half-open" = testing
    
    def call(self, func, *args, **kwargs):
        if self.state == "open":
            if datetime.now() - self.last_failure_time > timedelta(seconds=self.recovery_timeout):
                self.state = "half-open"
            else:
                raise Exception(f"Circuit breaker open. Failing fast.")
        
        try:
            result = func(*args, **kwargs)
            if self.state == "half-open":
                self.state = "closed"
                self.failure_count = 0
            return result
        except Exception as e:
            self.failure_count += 1
            self.last_failure_time = datetime.now()
            if self.failure_count >= self.failure_threshold:
                self.state = "open"
            raise e

# Usage
embedding_breaker = CircuitBreaker(failure_threshold=3, recovery_timeout=30)

def embed_with_fallback(query: str) -> np.ndarray:
    try:
        return embedding_breaker.call(embed_model.encode, query)
    except:
        # Fallback: use BM25 instead
        return sparse_retrieval(query)

Cost Architecture

Understanding cost drivers lets you optimize intelligently.

Three Cost Centers in RAG

  1. Embedding Cost (per-token, one-time at index-time + per-query at retrieval-time)

  2. Vector Storage Cost (per-GB-month)

  3. LLM Inference Cost (per-token, generation-time)

Cost Breakdown Example: 10M Docs, 10K QPS

Cost Center Cost per Day Cost per Month % of Total
Embedding (queries only) $10 $300 5%
Vector Storage $20 $600 10%
LLM Inference $160 $4800 85%
Total $190 $5700 100%

Optimization Techniques:

Technique Cost Component Estimated Savings Quality Trade-off
Semantic caching (don't re-compute for repeated queries) Embedding 20–30% Minimal
Quantization (8-bit embeddings instead of 32-bit) Vector storage 75% ~1% recall loss
Model tier routing (GPT-3.5 for simple queries, GPT-4 for complex) LLM 40–50% Latency variance
Batch generation (generate 10 answers at once) LLM 5–10% (amortize overhead) Latency increase
Smaller embedding model (BGE instead of OpenAI) Embedding 50% Domain-specific; test first

Observability Stack

What to instrument and why.

Retrieval Metrics

Track every retrieval call:

Query → [Embedding Time] → [VectorDB Latency] → [Top-k Results]
           ↓ Log                  ↓ Log              ↓ Log
        [Metric]              [Metric]           [Metric]

Key Metrics:

Generation Metrics

Track LLM output quality:

Offline (on a sample of queries):

Online (continuous):

System Metrics

Health of the system as a whole:

Dashboard Mockup

┌─────────────────────────────────────────────────────────┐
│ RAG System Dashboard                                    │
├─────────────────────────────────────────────────────────┤
│                                                         │
│  Latency (P95): 287ms  ↑ 15% from baseline            │
│  QPS: 9,847           ↑ Normal                         │
│  Error Rate: 0.08%    ✓ Healthy                        │
│                                                         │
│  Retrieval Metrics:                                     │
│    Recall@5: 0.78     ↓ Alert (target: 0.85)          │
│    Precision@5: 0.92  ✓ Normal                         │
│    Cache Hit: 34%     ✓ Normal                         │
│                                                         │
│  Generation Metrics (sample):                           │
│    Faithfulness: 0.81 ↓ Alert (target: 0.90)          │
│    Relevance: 0.88    ✓ Normal                         │
│    User Feedback: 87% positive                         │
│                                                         │
│  Alerts:                                                │
│    ⚠ Recall dropped. Check embedding model quality    │
│    ⚠ Faithfulness below threshold. Check reranker     │
│    ⚠ P95 latency spiked. Check VectorDB load         │
│                                                         │
└─────────────────────────────────────────────────────────┘

Instrumentation Example: OpenTelemetry

from opentelemetry import trace, metrics
from opentelemetry.exporter.jaeger import JaegerExporter

tracer = trace.get_tracer(__name__)
meter = metrics.get_meter(__name__)

def retrieve_with_tracing(query: str) -> list:
    with tracer.start_as_current_span("retrieve") as span:
        span.set_attribute("query", query)
        
        # Embedding span
        with tracer.start_as_current_span("embed_query"):
            embedding = embed_model.encode(query)
        
        # VectorDB span
        with tracer.start_as_current_span("vector_search"):
            results = vector_db.search(embedding, k=10)
        
        span.set_attribute("num_results", len(results))
        return results

# Meter for metrics
retrieval_latency = meter.create_histogram("retrieval.latency_ms")

def tracked_retrieval(query: str):
    start = time.time()
    results = retrieve_with_tracing(query)
    latency_ms = (time.time() - start) * 1000
    retrieval_latency.record(latency_ms)
    return results

The Canonical System Design Interview Answer Structure

When asked "Design a RAG system for...", use this template. Filling it in for any prompt gives you a complete, production-grade design.

1. Clarify Requirements

2. Data Flow Diagram

User Query
    ↓
[Which embedding model?]
    ↓
[Which vector database?]
    ↓
[Retrieval strategy: dense, hybrid, multi-hop?]
    ↓
[Reranking: yes or no?]
    ↓
[Which LLM? Which prompt template?]
    ↓
Answer

3. Component Selection

4. Scale Analysis

5. Failure Modes & Mitigation

6. Evaluation Plan

Requirements:

Data Flow:

User Query (natural language)
    ├──► Embedding (OpenAI text-embedding-3-small, 512 dims, $0.02/1M tokens)
    │
    ├──► Retrieval (Qdrant, HNSW index, partitioned by department)
    │    └──► Top-50 candidates
    │
    ├──► Reranking (BGE-reranker-large, cross-encoder)
    │    └──► Top-5 results
    │
    └──► LLM (GPT-4 Turbo, XML-delimited context to prevent injection)
         └──► Generated answer

Component Selection:

Scale Analysis:

Failure Modes:

Evaluation: