14 — Contextual RAG (Contextual Retrieval)

Answer:

Contextual Retrieval (introduced by Anthropic, 2024) addresses a fundamental weakness of standard chunking: when a document is split into chunks, individual chunks often lose the context needed to make them retrievable.

The problem:

Original document: "Q3 2024 Earnings Report — Acme Corp"
...
Chunk 47: "The growth rate improved to 23% compared to last quarter."

If a user asks "What was Acme Corp's Q3 2024 growth rate?", the chunk does not contain "Acme Corp", "Q3", or "2024" — it only says "The growth rate improved to 23%." The chunk will score poorly for the query and may not be retrieved.

The solution:

Before embedding, prepend an LLM-generated context sentence to each chunk that situates it within its document:

Contextual prefix: "This chunk is from Acme Corp's Q3 2024 Earnings Report.
It describes the company's quarter-over-quarter revenue growth."

Chunk text: "The growth rate improved to 23% compared to last quarter."

Combined for embedding: "[prefix] [chunk text]"

Anthropic's experiments show this reduces retrieval failures by 35% on average when combined with BM25 hybrid retrieval.

Answer:

The context prefix is generated by prompting an LLM with the full document and the specific chunk:

CONTEXT_PROMPT = """<document>
{full_document}
</document>

Here is the chunk we want to situate within the above document:
<chunk>
{chunk_text}
</chunk>

Please give a short succinct context (1-2 sentences) to situate this chunk
within the overall document for the purposes of improving search retrieval.
Answer only with the succinct context, no preamble."""

context = llm.invoke(CONTEXT_PROMPT.format(
    full_document=doc.page_content,
    chunk_text=chunk.page_content
))
contextualized_chunk = f"{context}\n\n{chunk.page_content}"

Cost without optimization:

• 1 LLM call per chunk
• Each call sends the full document + chunk (potentially thousands of tokens)
• For a 10K-chunk corpus: 10,000 LLM calls — expensive

Cost with prompt caching (critical optimization):

Claude's prompt caching caches the full document portion of the prompt across all chunks from the same document:

Document prompt prefix (cached after first chunk):
"<document>... 5,000 tokens of document ...</document>"
                ↑ cached — paid once per document

Per-chunk variable suffix:
"<chunk>... 200 tokens ...</chunk> Please give context..."
                ↑ paid per chunk, but small

For a document with 50 chunks:

• Without caching: 50 × 5,200 tokens = 260,000 tokens
• With caching: 5,000 tokens (first call) + 49 × 200 tokens = 14,800 tokens
• 94% token reduction for multi-chunk documents

Answer:

Contextual Retrieval and BM25 are complementary and their gains are additive:

Why they're additive:

• Contextual embeddings improve semantic retrieval by giving the embedding model the vocabulary it needs ("Acme Corp", "Q3 2024").
• BM25 improves exact keyword retrieval, catching cases where semantic similarity fails.
• Adding context to chunks also improves BM25 performance because the context prefix adds exact-match keywords (company names, dates, product names) that the raw chunk lacked.

Implementation pattern:

# Index: store both the contextualized text and original chunk
for chunk in chunks:
    context = generate_context(full_doc, chunk)
    contextualized_text = f"{context}\n\n{chunk.text}"
    
    # Embedding uses contextualized text
    embedding = embed_model.embed(contextualized_text)
    
    # BM25 index also uses contextualized text (keyword enriched)
    bm25_index.add(contextualized_text, doc_id=chunk.id)
    
    # Store in vector DB with metadata
    vectorstore.upsert(id=chunk.id, embedding=embedding,
                       metadata={"text": chunk.text,  # Original for display
                                 "context": context})

# Query: standard hybrid search (unchanged)
dense_results = vectorstore.search(query_embedding, k=20)
sparse_results = bm25_index.search(query, k=20)
final = reciprocal_rank_fusion(dense_results, sparse_results)

Answer:

When to choose:

• Contextual Retrieval: Quick win for any corpus. Low complexity, ~1 day to implement, proven 35–67% improvement.
• RAPTOR: When you need global/cross-document synthesis and have budget for the build complexity. Implement contextual retrieval first.

They compose: Apply contextual prefixes to leaf chunks, then build RAPTOR tree on top of the contextualized chunks. Each step adds independent value.

Answer:

Step 1 — Identify failure cases in your current system

The most valuable signal is a set of known retrieval failures:

• Queries where users said "you missed the answer"
• Queries where the retrieved chunks had low similarity scores
• Queries where the LLM said "I don't have enough information"

Step 2 — Build a retrieval eval set

Manually annotate 100–200 (query, expected_chunks) pairs. Include:

• Queries that use document-level vocabulary ("the Q3 report", "the merger agreement")
• Queries that use vocabulary not in any chunk ("Acme's performance last fall")

Step 3 — A/B test on eval set

# Measure Recall@5 and Recall@10
standard_recall_at_5  = evaluate(standard_retriever, eval_set, k=5)
contextual_recall_at_5 = evaluate(contextual_retriever, eval_set, k=5)

# Measure token cost
avg_context_tokens = mean(len(tokenize(ctx)) for ctx in generated_contexts)
total_index_cost = avg_context_tokens * len(chunks) * price_per_token

Step 4 — Threshold for deployment

Deploy contextual retrieval if:

• Recall@5 improves by ≥ 5 percentage points AND
• Faithfulness of context prefixes ≥ 0.95 (spot-check 50 prefixes) AND
• Index build cost fits within budget

Answer:

Failure modes:

1. Hallucinated context — LLM makes up document-level details not present in the document.

   • Example: chunk is from an internal policy doc; LLM writes "This is from the 2023 Annual Report" (wrong).
   • Mitigation: Constrain the prompt to only include information present in the document. Run faithfulness check on a sample.

2. Generic/useless context — LLM writes "This chunk discusses various topics." — no retrieval benefit.

   • Mitigation: Include 3–5 few-shot examples of good vs. bad context prefixes in the prompt.

3. Context contradicts the chunk — LLM summarizes the document incorrectly, creating a misleading context.

   • Example: Chunk says "Revenue fell 10%"; context says "This chunk from the strong-growth Q3 report..."
   • Mitigation: Validate generated context with NLI against the full document.

4. Too long context prefix — LLM generates a paragraph instead of 1–2 sentences, bloating the embedded text.

   • Mitigation: Set max_tokens=100 for context generation; trim to first 2 sentences.

5. Identical prefixes — When multiple chunks from the same document section get the same generic prefix, no retrieval benefit is gained.

   • Mitigation: Include the specific chunk in the prompt and ask for chunk-specific context, not just document-level.

Quality check pipeline:

for chunk, context in zip(chunks, contexts):
    # Check length
    assert len(context.split()) < 60, "Context too long"
    # Check it's not generic
    if context.strip().lower().startswith("this chunk discusses"):
        flag_for_review(chunk, context)
    # Faithfulness spot check (sample 5%)
    if random.random() < 0.05:
        score = faithfulness_check(context, full_doc)
        log_metric("context_faithfulness", score)

Answer:

Contextual Retrieval and reranking operate at different stages of the pipeline and are complementary:

Query
  │
  ▼
Contextual Hybrid Retrieval (dense + BM25 on contextualized chunks)
→ Candidate set: top-20 chunks
  │
  ▼
Cross-Encoder Reranker
→ Receives: (query, chunk_text) pairs
→ Returns: top-5 reranked chunks
  │
  ▼
LLM Generation

Key question: Should the reranker score the contextualized text or the original chunk?

• Score contextualized text: The reranker sees the same enriched text the embedding model used. More context for relevance judgment. Risk: reranker focuses on the context prefix rather than the content.
• Score original chunk text: The reranker evaluates only the content the user will see. Cleaner signal. May miss context-dependent relevance.

Recommendation: Use the original chunk text for the reranker's document input. The retrieval stage (benefiting from context) selects the right candidates; the reranker then precisely scores relevance against those candidates. Pass the context as metadata for the generation step if citations are needed.

Observed improvement stack (Anthropic's data):

• Standard embeddings only: baseline
• • Contextual embeddings: −35%
• • BM25: −67%
• • Reranking on top: additional −5 to −10% — marginal improvement since contextual + BM25 already surfaces good candidates.

Answer:

The Contextual Retrieval prompt requires the full document to be in context when generating each chunk's prefix — but documents can be 100K+ tokens, exceeding even Claude's 200K window.

Strategy 1 — Use a document summary instead of the full document

Pre-generate a 500-token summary of the document (with a separate LLM call), then use the summary as the "document" in the context generation prompt:

doc_summary = llm.invoke(f"Summarize this document in 500 tokens:\n{full_doc}")

# Use summary in context generation
context = llm.invoke(CONTEXT_PROMPT.format(
    full_document=doc_summary,  # Summary, not full doc
    chunk_text=chunk.text
))

Trade-off: context quality may be lower since the LLM only sees the summary, not the full document.

Strategy 2 — Section-level context generation

Split the document into logical sections (chapters, sections by H1/H2 headings). Generate context for each chunk using only its section's text:

Document: 200K tokens
Section: 5K tokens (fits in context)
Chunk: 500 tokens (within section)
→ Context generated from section text only

Strategy 3 — Hierarchical context

Generate two levels of context:

1. Document-level summary (generated from the full document in one call, cached)
2. Chunk-level context (generated from section text)

Combine both as the prefix: "[doc-level context]. [chunk-level context]. [chunk text]"

Strategy 4 — Long-context models (Claude 3.5 Sonnet, Gemini 1.5 Pro)

Models with 200K+ context windows can handle most enterprise documents. Use them specifically for context generation, even if the main generation model is cheaper/smaller.

Answer:

Assumptions:

• 100,000 documents
• Average document: 10 chunks × 500 tokens = 5,000 tokens
• Average chunks per corpus: 1,000,000
• Context generation: 100 output tokens per chunk
• Input tokens per chunk (with caching): full doc first chunk + incremental subsequent chunks

Without prompt caching:

Per chunk input: 5,000 (doc) + 500 (chunk) + 50 (prompt) = 5,550 tokens
Total input tokens: 1,000,000 chunks × 5,550 = 5.55B tokens
Output tokens: 1,000,000 × 100 = 100M tokens

Cost (Claude Haiku 3.5):
Input: 5.55B × $0.80/1M = $4,440
Output: 100M × $4/1M = $400
Total: ~$4,840

With prompt caching (Claude):

Per document: first chunk pays full input; 9 subsequent chunks pay only incremental
First chunk per doc: 5,550 tokens (written to cache)
Subsequent 9 chunks per doc: 550 tokens each (cache hit)

Input tokens (non-cached): 100,000 docs × 5,550 = 555M
Input tokens (cached reads): 100,000 docs × 9 chunks × 550 = 495M
Cache write: 555M × $1/1M = $555
Cache read: 495M × $0.08/1M = $39.60
Output: 100M × $4/1M = $400
Total: ~$995 (79% reduction)

Key insight: Prompt caching is not optional for Contextual Retrieval at scale — it's what makes the approach economically viable. Without it, the cost is ~5× higher.

Ongoing cost: This is a one-time index-build cost. Re-generation is only needed when documents change (incremental updates).

Answer:

INGESTION PIPELINE
──────────────────
New/Updated Document
  → Chunker (512 tokens, 50 overlap, structure-aware)
  → Context Generator (Claude Haiku 3.5, with prompt caching per document)
       Input: full_doc + chunk → Output: 1-2 sentence context prefix
       Quality gate: reject context if faithfulness < 0.9 or length > 100 tokens
  → Contextualized chunk = [prefix] + [chunk text]
  → Dual indexing:
       a. Embed contextualized_chunk → store in Qdrant (1536 dims)
       b. Add contextualized_chunk to BM25 index (Elasticsearch or Weaviate)
  → Store metadata: {chunk_id, doc_id, original_text, context_prefix, created_at}

QUERY PIPELINE
──────────────
User query
  → Embed query (text-embedding-3-small)
  → Parallel:
       Dense search: Qdrant top-20
       Sparse search: BM25 top-20
  → Reciprocal Rank Fusion → top-20 merged candidates
  → Cross-encoder reranker (ms-marco-MiniLM-L-12-v2) → top-5
  → Generation: pass original_text (not contextualized) to LLM to avoid
    prefix text leaking into the user-visible answer
  → Return answer + citations (chunk_id, doc_id, original_text excerpt)

MONITORING
──────────
Track:
- Context generation faithfulness (sampled 5% of new chunks)
- Retrieval Recall@5 on golden eval set (daily CI gate)
- Context generation cost per 1000 documents
- BM25 + dense agreement rate (low agreement → possible retrieval issue)

Incremental updates:

• On document update, re-generate contexts only for changed chunks (use content hash to detect changes).
• Re-index changed chunks in both dense and sparse indexes.
• Keep old chunk versions with a superseded_at timestamp; exclude from retrieval with a metadata filter.

Answer:

Contextual Retrieval introduces a data-handling concern in multi-tenant systems: the context generation step sends the full document to an LLM API.

Risk 1 — Data egress to LLM provider

The context generation prompt sends the full document to the LLM provider (e.g., Anthropic, OpenAI). For tenants with strict data residency or confidentiality requirements, this may violate compliance constraints.

Mitigations:

• Use a self-hosted open-source LLM (LLaMA 3, Mistral) for context generation. Slightly lower context quality but zero data egress.
• Use provider-side data processing agreements (Anthropic's commercial API DPA).
• For highly sensitive documents, skip contextual retrieval and rely on metadata filtering + BM25 instead.

Risk 2 — Cross-tenant context leakage

If a shared LLM API is used for context generation and prompt caching is enabled, verify that caching does not allow one tenant's document to influence another tenant's context generation. (With correctly scoped prompt caching keys, this is not a risk, but it should be explicitly verified.)

Risk 3 — Context prefix reveals document structure to index admins

The generated context prefix ("This chunk is from Tenant A's confidential HR policy document regarding termination procedures...") is stored as metadata. Ensure that metadata access is subject to the same ACL as the chunk content.

Best practice for multi-tenant: Generate context using a per-tenant isolated model invocation (separate API keys, separate prompt caching namespaces) and store context prefixes with the same access controls as the source document.

Answer:

In streaming ingestion, documents arrive continuously and must be indexed quickly. Contextual Retrieval's LLM call per chunk adds latency to the indexing path.

Architecture options:

Option 1 — Synchronous (simple, higher latency)

New document → Chunk → Generate contexts (blocking) → Embed → Index
Latency per document: chunk_time + (N_chunks × context_gen_time) + embed_time
For a 10-chunk document: ~5–10 seconds (depending on LLM latency)

Option 2 — Async two-phase indexing (recommended)

Phase 1 (immediate, < 1 second):
  New document → Chunk → Embed WITHOUT context → Index in "pending" state
  → Document is searchable immediately (lower quality)

Phase 2 (async, within minutes):
  Background worker → Generate contexts for pending chunks
  → Re-embed with context → Update vector store in-place
  → Update chunk state to "contextualized"

This gives immediate availability with progressive quality improvement.

Option 3 — Pre-index context as a separate field

Store the context prefix as a metadata field separate from the embedding. Use a background job to populate the context field. At query time, if the context field is populated, use the contextualized embedding; otherwise fall back to the raw embedding.

Prompt caching in streaming:

Prompt caching requires the full document to be in the prompt. In a streaming pipeline where documents are processed chunk-by-chunk, ensure all chunks from the same document are processed in a single batch to maximize cache reuse:

async def index_document(doc):
    chunks = chunker.split(doc)
    # Process all chunks from this document together
    # so the document prefix is cached across all chunk calls
    contexts = await asyncio.gather(*[
        generate_context(doc.full_text, chunk) for chunk in chunks
    ])
    await embed_and_index(chunks, contexts)

Answer:

Document → Chunk (standard fixed-size or semantic chunking)
                │
                ▼
For each chunk: LLM generates a short contextual prefix describing
what the chunk is about relative to the WHOLE document
                │
                ▼
Prefix + chunk, concatenated → Embedder → Vector Store
                │
(also indexed for BM25, Q3, using the same prefixed text)
                │
Query → Hybrid (BM25 + dense) retrieval over prefixed chunks → Generator

The single addition over standard chunking is the contextual-prefix generation step — everything else in the pipeline (chunking, embedding, hybrid retrieval, generation) is unchanged from Advanced RAG (#02). This is what makes Contextual Retrieval one of the cheapest architectural upgrades in this bank to adopt: it doesn't change the retrieval algorithm at all, only what text gets embedded and indexed in the first place, which is why Q9's cost analysis is entirely about the one-time (or per-update) prefix-generation cost rather than any change to query-time cost.

Answer:

Contextual Retrieval was introduced by Anthropic in a 2024 engineering blog post (Introducing Contextual Retrieval), building directly on Claude's prompt caching feature (#10 Q6) to make the per-chunk context-generation step economically practical — without caching the full document in context across many chunk-generation calls, generating a bespoke context prefix for every chunk of every document would be prohibitively expensive at corpus scale, since each call would otherwise re-process the whole source document from scratch.

Like Agentic Web RAG (#31 Q8) and LazyGraphRAG (#47 Q9), Contextual Retrieval is a product/engineering-blog-originated technique rather than a peer-reviewed paper's contribution — its reported results (a meaningful reduction in retrieval failure rate when combined with hybrid search and reranking, per Q3) are drawn from Anthropic's own internal benchmarking rather than an academic benchmark suite, which is worth noting explicitly when discussing "the paper" for this specific architecture in an interview setting.

Answer:

Both use an LLM to generate additional text that improves retrieval, but they intervene at different points and for different purposes. HyDE (#22) generates a hypothetical answer to the query, at query time, and embeds that instead of the raw query — it's a query-side technique addressing the query-document vocabulary gap for whatever chunks already exist in the index. Contextual Retrieval generates a context prefix for each chunk, at index time (once, not per query), addressing the problem of a chunk being ambiguous or under-specified in isolation — it's a document-side technique that changes what gets indexed in the first place, with no per-query LLM call at all.

The two are complementary and address genuinely different gaps: HyDE helps regardless of how well-formed the indexed chunks are, by improving what the query embeds as; Contextual Retrieval helps regardless of how the query is phrased, by improving what the chunks embed as. A production system could use both simultaneously — contextualized chunks in the index, queried via HyDE-generated hypothetical documents — since neither technique's mechanism interferes with the other.

Answer:

The distinctive mechanism is prepending an LLM-generated, whole-document-aware summary to each chunk before embedding, so a chunk's embedding reflects not just its own isolated content but also its role within the source document. Standard chunking embeds a chunk exactly as it appears — if a chunk reads "the company's revenue grew by that amount," with the specific company and quarter established only in an earlier, now-separated part of the document, the chunk's embedding has no way to represent that missing context, since the embedding model only ever sees the chunk's own text.

This single addition directly targets Naive RAG's (#01 Q1) chunking-artifact failure mode from a different angle than large retrieval units (LongRAG, #45) or hierarchical summarization (RAPTOR, #13, Q4 of this file) — rather than making chunks bigger or building a separate summary tree, Contextual Retrieval keeps chunks exactly the same size but enriches what gets embedded for each one, which is why it composes cleanly with hybrid search and reranking (Q3, Q7) without requiring any change to the retrieval algorithm itself.

Answer:

Prompt-caching strategy is the knob most specific to why Contextual Retrieval is economically practical at all (Q9, Q14) — without effectively caching the source document across a document's many chunk-generation calls, the per-chunk cost of context generation would scale with document length on every single chunk, rather than being amortized across the whole document's chunks via a cached context.

Answer:

Build a golden set specifically containing queries whose correct chunk is genuinely ambiguous in isolation (Q1's own motivating example — a chunk that only makes sense given surrounding document context) alongside queries whose correct chunk is already self-contained and unambiguous — a benchmark dominated by already-clear chunks won't show Contextual Retrieval's benefit, since there's nothing for the prefix to disambiguate. Compare retrieval recall@k with and without context prefixes on both segments, expecting a measurable gain specifically on the ambiguous-chunk segment and little to no difference (or possibly slight noise-driven regression) on the already-clear segment.

This segmented approach is what Q5's own "measure whether Contextual Retrieval improves your specific corpus" question is really asking beneath its general framing — a corpus dominated by short, self-contained FAQ-style chunks (little ambiguity to resolve) is a poor fit for this technique regardless of what Anthropic's own benchmarks reported, while a corpus of long, narrative documents with heavy cross-reference and pronoun usage across paragraphs is exactly where the measured gain should be largest, mirroring the same "measure on your corpus, don't trust published numbers to transfer" discipline used throughout this bank.

Answer:

Q6 already flags this general risk category; the specific failure mode worth isolating is a prefix that doesn't just add unhelpful noise but actively misrepresents what the chunk is about — for example, summarizing a chunk as being "about Q3 2024 results" when the chunk actually discusses Q3 2023 results, because the context-generation model made an off-by-one error reading the surrounding document. Since the prefix is concatenated with the chunk before embedding, a misleading prefix doesn't just fail to help — it can actively pull the chunk's embedding toward queries about the wrong topic (Q3 2024) while the chunk's actual content (Q3 2023) sits underneath, unseen by the embedding until a human reads the retrieved result directly.

Detection: for a sample of generated prefixes, verify factual claims in the prefix (specific numbers, dates, named entities) against the actual chunk content — an automated check comparing entities/numbers mentioned in the prefix against entities/numbers present in the chunk itself catches many of these cases without requiring full human review of every prefix. Mitigation: constrain the context-generation prompt to only reference information that's genuinely present in the document (an explicit "do not add facts not stated in the source" instruction, the same anti-hallucination framing used for summarization prompts elsewhere in this bank, e.g. MemoRAG's #44 Q2), and treat this automated fact-consistency check as a standard part of the indexing pipeline rather than a one-time quality spot-check.

Answer:

Current limitations: (1) benefit is corpus-dependent and can be near-zero for already-clear chunks (Q18) — this isn't a universal upgrade, and applying it indiscriminately to a corpus that doesn't need disambiguation wastes indexing cost for no retrieval gain; (2) hallucinated prefixes are a real, hard-to-fully-eliminate risk (Q6, Q19) — an automated fact-consistency check catches many but not necessarily all misrepresentations; (3) indexing cost and complexity increase (Q9) relative to standard chunking, even if made economical via prompt caching; (4) doesn't address every chunking-boundary failure — a genuinely fragmented mid-sentence split (Q1's original example) is helped by a context prefix, but a chunk that's simply too short to contain a complete unit of meaning may need larger chunks or hierarchical summarization (RAPTOR, #13) rather than more context about what it's missing.

Likely evolution: automated, corpus-aware decision tooling that estimates expected benefit before committing indexing cost (directly addressing limitation 1) — analyzing a corpus sample for chunk-level ambiguity rate and projecting likely retrieval-recall gain, similar in spirit to the decision-gate benchmarks used to justify other architectural investments throughout this bank; and tighter integration of the fact-consistency check (Q19) as a standard, automated pipeline stage rather than a manual spot-check, as prompt-caching costs continue to fall and make more thorough per-chunk validation economically routine.

Answer:

The situation implies a modest, fairly static corpus (maybe a few thousand contract chunks), no dedicated ML team, and a concrete pain point: associates searching for "the indemnification clause" or "the termination-for-convenience language" get chunks that read fine in isolation but don't say which contract, party, or clause type they belong to — exactly Q1's motivating failure.

The straightforward approach is to run the standard Contextual Retrieval pipeline (Q1, Q3) essentially as-is: chunk each contract, generate a one-to-two-sentence context prefix per chunk with a mid-tier model, and index the prefixed text in a hybrid dense+BM25 setup. At this scale, prompt caching (Q2) still helps but isn't make-or-break the way it is at enterprise scale (Q9) — a few thousand chunks is a one-time indexing job measured in dollars, not a recurring infrastructure concern.

Two trade-offs worth flagging to the firm: first, the small team has no one to run Q6's faithfulness-checking pipeline at scale, so a lightweight manual spot-check (reading 20-30 generated prefixes before trusting the rest) substitutes for automated validation. Second, contracts are re-indexed only when new documents arrive, not continuously, so freshness is a non-issue — the bigger risk is a hallucinated prefix quietly misattributing a clause to the wrong contract, which a small firm won't catch without that spot-check discipline.

Answer:

The hard constraints are a nightly refresh SLA across 40 jurisdictions and near-certain data-residency/regulatory rules that vary by jurisdiction — some filings likely cannot leave their home region to reach a shared third-party LLM API, which changes Q11's multi-tenant isolation question from a nice-to-have into a hard requirement.

The recommended approach: partition the pipeline by jurisdiction, with a per-jurisdiction context-generation invocation (its own API key/endpoint or, where residency rules demand it, a self-hosted open-source model deployed in-region) rather than one global batch job. Nightly, diff each jurisdiction's filings against the previous run (content hash, per Q12's pattern) and regenerate context prefixes only for changed or new chunks — full-corpus regeneration every night across 40 jurisdictions would blow both the time and cost budget. Prompt caching per document (Q2) still applies within each jurisdiction's job.

The real trade-off is between uniformity and compliance: a single global pipeline is operationally simpler but breaks residency rules for at least some jurisdictions, while 40 independently-configured pipelines (different models, different faithfulness thresholds, different regulatory review cadences) is far more resilient but meaningfully more to build and maintain. Given the regulatory stakes, compliance wins even at that operational cost.

Monitor per-jurisdiction: refresh completion time against the nightly SLA (with alerting on backlog), faithfulness spot-check pass rate on a sample of regenerated prefixes, and any cross-jurisdiction data-egress violations as a hard-fail alert, not a warning.