28 — Atlas

Answer:

Atlas (Izacard et al., Meta, 2022) is a retrieval-augmented language model designed for few-shot learning on knowledge-intensive tasks. It pairs a learned dense retriever (Contriever) with a Fusion-in-Decoder (FiD) reader and trains them jointly.

Headline result: Atlas with 11B parameters matches or outperforms a 540B-parameter PaLM on knowledge tasks (e.g., Natural Questions) using only 64 training examples — a ~50× smaller model winning in the few-shot setting.

The central thesis: retrieval is the key enabler of few-shot knowledge-intensive learning. Because knowledge lives in the retrievable corpus (not parameters), Atlas doesn't need millions of examples or huge parameter counts to "know" facts — it needs only to learn how to use retrieved evidence, which it can do from few examples.

Why it's a training-time RAG (this section): the retriever is learned and jointly optimized with the reader (unlike RETRO's frozen retriever), and Atlas studies how to train the retriever+reader system efficiently — retrieval is integral to training, not bolted on at inference.

Answer:

Atlas composes two learned components:

RETRIEVER — Contriever (dense, dual-encoder)
  - Embeds the query and documents; retrieves top-k by inner product.
  - Pre-trained with contrastive self-supervision (no labels needed).
  - LEARNED / fine-tuned jointly with the reader.

READER — Fusion-in-Decoder (FiD), based on T5 (seq2seq)
  - Encodes EACH retrieved passage independently (concatenated with the query).
  - The DECODER attends jointly over all encoded passages (fusion in decoder)
    to generate the answer.
  - Scales to many passages without quadratic attention blowup.

PIPELINE:
  query → Contriever retrieves top-k passages
        → FiD encodes each (query+passage) separately
        → decoder fuses them → generates answer

Why this pairing:

• Contriever gives a strong, trainable dense retriever that works well even with little supervised data (its contrastive pretraining is label-free).
• FiD (pattern 29) is the natural reader for retrieval — it handles many passages efficiently by encoding them separately and fusing in the decoder.
• Both are differentiable and jointly trained, so the retriever learns to fetch what the reader can use.

Atlas is essentially "learned Contriever + FiD reader, trained together, optimized for few-shot."

Answer:

The challenge: retrieval (a discrete top-k selection) isn't directly differentiable, so you need a way to give the retriever a training signal from the reader's loss. Atlas studied several objectives for this; the key ones:

1. Attention Distillation (ADist) — the favored approach.

• Use the reader's cross-attention over passages as a measure of which passages were useful for producing the answer.
• Distill that signal into the retriever: train the retriever so its retrieval scores match the reader's attention-derived importance.
• Intuition: "the passages the reader attended to most are the ones the retriever should rank highest."

2. Other objectives Atlas compared:

• EMDR² (Expectation-Maximization) — treat retrieved docs as latent variables, marginalize (REALM-like).
• Perplexity Distillation (PDist) — train the retriever so retrieved docs minimize the reader's perplexity on the answer.
• Leave-one-out (LOOP) — measure each doc's contribution by the change in loss when removed.

Common principle: all derive a signal of document usefulness from the reader and push it into the retriever, so the retriever is optimized for what helps the reader answer, not generic similarity.

Index refresh (the REALM problem again): as the retriever updates, document embeddings drift, so Atlas must refresh the index during training. It uses strategies like periodic re-encoding and an over-retrieve-then-refresh schedule — and notably explores query-side-only updates to reduce re-indexing cost.

Result: joint training is what makes Atlas few-shot efficient — the retriever quickly learns task-relevant retrieval from few examples because it's guided by the reader's signal.

Answer:

Atlas's distinct identity:

1. Few-shot is the explicit goal — Atlas is engineered and evaluated for learning from dozens of examples, where the others target pre-training (REALM/RETRO) or reader scaling (FiD).
2. It combines and refines the others' pieces: Contriever (learned retriever, the REALM lineage) + FiD reader (pattern 29) + a careful study of how to train them jointly (the ADist objective).
3. Generative reader + learned retriever, jointly trained — RETRO has a learned reader but frozen retriever; FiD has a generative reader but a separate retriever; REALM learns the retriever but with an extractive encoder. Atlas is the one that learns both, generatively, with few-shot data efficiency.

One-liner: Atlas = "jointly-trained Contriever + FiD, optimized so a small model learns knowledge tasks from very few examples."

Answer:

Atlas's core argument: retrieval decouples "knowing facts" from "learning the task," which is exactly what few-shot needs.

The reasoning:

1. Knowledge doesn't need to be learned from the few examples. In a closed-book model, the few-shot examples must teach both the task format and somehow surface the relevant knowledge (already in params). With retrieval, the facts come from the corpus, so the few examples only need to teach the model how to use retrieved evidence — a far smaller learning problem.

2. Fewer parameters needed. Since knowledge lives in the retrievable corpus, the model doesn't have to memorize the world in its weights. An 11B model suffices where a closed-book model needs 540B to store comparable knowledge — and a smaller model generalizes from few examples more readily for the narrow "use the evidence" skill.

3. The corpus is a huge "free" knowledge source. Few-shot is hard because labeled data is scarce; but the unlabeled retrieval corpus is enormous. Retrieval lets the model leverage that large knowledge source without needing labels for it.

4. Updatability without retraining. New facts → update the corpus, no new examples needed. Closed-book few-shot can't acquire new knowledge from 64 examples.

Empirical payoff: with 64 examples, Atlas-11B matches/beats PaLM-540B on NQ — because the only thing it had to learn from those 64 examples was how to read retrieved passages to answer, not the world's facts.

The principle: in low-data regimes, move knowledge out of parameters and into retrieval, so the limited supervision is spent on the task skill, not on knowledge acquisition.

Answer:

Benchmarks (knowledge-intensive NLP):

• Open-domain QA: Natural Questions (NQ), TriviaQA, WebQuestions — the headline few-shot results.
• KILT benchmark suite — a battery of knowledge-intensive tasks (QA, fact-checking like FEVER, entity linking, slot filling, dialogue). Atlas set strong results across KILT.
• MMLU — multitask knowledge (massive multitask language understanding).
• Fact checking (FEVER) and question answering under both few-shot and full-data (fine-tuned) regimes.

Evaluation settings (the key comparison):

Atlas-specific evaluation strengths it demonstrated:

1. Interpretability/updatability: you can inspect retrieved documents to see why it answered, and update the corpus to change knowledge — Atlas explicitly highlighted editing the index (e.g., to reflect new facts) without retraining.
2. Ablating training objectives (ADist vs EMDR² vs PDist) — measuring which joint-training signal works best.
3. Few-shot scaling — performance vs number of examples, the central claim.

Principle: evaluate in both few-shot and full-data regimes and against closed-book baselines to isolate retrieval's contribution, plus test knowledge updatability as a distinct capability.

Answer:

1. Joint-training complexity. Training the retriever + reader together with attention-distillation objectives plus index refresh is intricate and compute-heavy — far more involved than fine-tuning a reader on a frozen retriever. Reproducing it well is non-trivial.

2. Index refresh cost (the REALM problem). Because the retriever is learned, document embeddings drift and the index must be re-encoded during training. Atlas mitigates (query-side updates, periodic refresh) but doesn't eliminate this — it's a fundamental cost of end-to-end retriever training.

3. Inference latency/cost. FiD encodes each retrieved passage separately; with many passages, the encoder cost scales linearly with k, making inference heavier than a single-pass model (a known FiD cost, inherited).

4. Retriever cold-start / dependence on Contriever pretraining. Atlas leans on a strong contrastively-pretrained retriever; without good retriever initialization, joint training struggles.

5. Corpus dependence. Few-shot success assumes the answer is retrievable from the corpus. For knowledge absent from the corpus, retrieval can't help and few-shot reverts to hard.

6. Superseded scale. Like its peers, Atlas predates the largest modern LLMs; modern instruction-tuned LLMs + inference-time RAG achieve strong few-shot knowledge QA without joint retriever training — questioning whether the training complexity is needed today.

Net: Atlas is a landmark proof that joint retrieval training enables few-shot knowledge learning, but it's operationally complex (joint training + index refresh + linear FiD cost) and, like the other training-time methods, is often outcompeted in practice by simpler inference-time RAG on strong modern LLMs.

Answer:

SETUP: Atlas fine-tuned on just 64 (question, answer) examples for
       open-domain QA, with a Wikipedia-scale retrieval corpus indexed
       by the (jointly-trained) Contriever.

INFERENCE for a new question:
  Q: "Who designed the Eiffel Tower?"

  1. Contriever embeds Q → retrieves top-k passages from the corpus:
       [passage about Gustave Eiffel, passage about the tower's history, ...]

  2. FiD reader:
       - Encode EACH (Q + passage) pair independently:
           enc("Q ... | Gustave Eiffel was a French engineer ...")
           enc("Q ... | The Eiffel Tower was completed in 1889 ...")
           ...
       - Decoder cross-attends over ALL encoded passages jointly (fusion)
       - Generates: "Gustave Eiffel" (grounded in the retrieved passage)

  3. (Optional) Inspect retrieved passages → see WHY it answered (interpretability).

What the 64 examples actually taught it:

• Not the fact "Gustave Eiffel designed the tower" — that came from retrieval.
• Rather, how to read retrieved passages and produce a concise answer in the expected format. That narrow skill is learnable from few examples — which is the whole point of Atlas.

Updatability demonstration: if the corpus is edited (e.g., a fact changes), Atlas's answer changes without any retraining — because knowledge is in the corpus, not the 64 examples or the weights.

Answer:

GOAL: a small model that learns a knowledge-intensive task from few labels
      by leaning on retrieval — e.g., a domain QA system with scarce labels.

COMPONENTS
──────────
- Retriever: strong contrastively-pretrained dense retriever (Contriever-style)
  → good zero/few-shot retrieval before any task fine-tuning.
- Reader: FiD (seq2seq) — encode each passage separately, fuse in decoder.
- Corpus: domain documents, indexed (ANN/MIPS).

TRAINING (few-shot)
───────────────────
1. Initialize with a pretrained retriever + pretrained seq2seq reader.
2. Joint fine-tune on the few labeled examples using an attention-distillation
   objective (reader attention → retriever target), Atlas's favored signal.
3. Index refresh strategy:
     - cheapest: freeze document encoder, update only the query encoder
       (avoids re-indexing) — common pragmatic choice.
     - fuller: periodically re-encode the corpus as the retriever updates.

KEY DECISIONS
─────────────
- How much to train the retriever: query-side-only (cheap, no re-index) vs
  full joint (better relevance, costly re-indexing).
- k passages: more = better coverage but linear FiD encoder cost.
- Training objective: ADist (attention distillation) generally robust;
  PDist/EMDR² alternatives.

WHEN IT'S WORTH IT vs JUST INFERENCE-TIME RAG
─────────────────────────────────────────────
- Worth it: labels are scarce AND a frozen retriever clearly under-retrieves
  for your domain (joint training meaningfully helps).
- Often NOT worth it: a strong modern instruction-tuned LLM + off-the-shelf
  retriever + few-shot prompting achieves similar results with no joint
  training or index refresh. Try that baseline FIRST.

MONITORING
──────────
- Few-shot curve (performance vs #examples), retrieval recall@k,
  index-staleness gap, knowledge-update correctness, FiD inference latency.

The pivotal design decision is how end-to-end to train the retriever: full joint training buys task-aligned retrieval (Atlas's edge) but reintroduces index-refresh cost, so query-side-only updates are the common compromise — and for many modern use cases, inference-time RAG on a strong LLM is the simpler baseline to beat first.

Answer:

1. Retrieval enables few-shot knowledge learning — proven. Atlas's defining contribution is the demonstration that moving knowledge into retrieval lets a small model learn knowledge-intensive tasks from very few labels. This reframed retrieval from "an accuracy booster" to "the enabler of data-efficient knowledge tasks."

2. A recipe for jointly training retriever + reader. Atlas systematically studied how to give a retriever a training signal from the reader (ADist, PDist, EMDR², LOOP) and showed attention distillation works well. This is a reference for anyone training end-to-end retrieval.

3. Validated the Contriever + FiD combination as a strong, trainable RAG stack — a template echoed in later retrieval-augmented systems.

4. Index-refresh strategies (query-side-only updates, periodic re-encoding) — practical techniques for the recurring "training a retriever invalidates the index" problem (shared with REALM).

5. Updatability + interpretability as first-class properties. Atlas emphasized that you can edit the corpus to change knowledge and inspect retrieved docs to explain answers — now standard RAG selling points, but Atlas made them concrete and measured.

Relation to modern practice: today's strong LLMs + inference-time RAG often achieve few-shot knowledge QA without Atlas's joint training — so Atlas's implementation is less used directly. But its thesis — "retrieval is what makes knowledge-intensive few-shot learning feasible with small models" — is foundational and broadly accepted, and its joint-training recipes remain the reference when end-to-end retriever optimization is needed.

Answer:

TRAINING
  - Joint retriever+reader fine-tuning with attention-distillation.
  - INDEX REFRESH: re-encoding the corpus as the retriever updates
    (mitigated by query-side-only updates, but a real cost if fully joint).
  - Few-shot helps here: with only ~64 examples, the supervised phase is
    short — the expense is the joint-training machinery, not data volume.

INFERENCE (per query)
  - Contriever retrieval: query encode + ANN top-k (fast).
  - FiD reader: encode EACH of the k passages SEPARATELY → encoder cost
    scales LINEARLY in k; decoder fuses them.
    → the dominant inference cost, and it grows with k.

Main cost drivers:

• Training: the joint-training + index-refresh machinery (not example count, since it's few-shot).
• Inference: FiD's linear-in-k passage encoding — more passages = proportionally more encoder compute.

Optimizations:

1. Query-side-only retriever updates → no re-indexing during training (big saving).
2. Tune k — fewer passages cut FiD encoder cost (the main inference lever).
3. Smaller model — Atlas's whole point is parameter efficiency; an 11B reader is cheaper to serve than a 540B closed-book model of similar quality.
4. Cache retrievals for repeated queries; ANN (approximate) retrieval.
5. Passage reranking/pruning before FiD so only the most useful passages are encoded.

Framing: Atlas trades joint-training complexity for few-shot data efficiency and a small served model. Its notable inference cost is FiD's linear passage-encoding — the same trade-off discussed in pattern 29 (Fusion-in-Decoder).

Answer:

1. Corpus is the knowledge (and trust) boundary. Atlas's answers come from the retrieval corpus, so corpus poisoning directly corrupts outputs — and because the retriever is trained on the corpus distribution, adversarial documents can also bias what the retriever learns to fetch.

• Mitigation: vet/trust-score corpus sources; monitor over-retrieved documents; curate training corpora (training-time poisoning is harder to detect).

2. Freshness — a headline strength, with a caveat. Atlas explicitly supports updating knowledge by editing the corpus without retraining (re-embed new docs with the retriever). But a retriever trained on an old distribution may under-retrieve novel-domain content (train/serve skew).

• Mitigation: refresh the index on corpus changes; watch retrieval recall on new topics; periodically re-tune the query encoder if the domain drifts.

3. Stale index. A learned retriever's index not refreshed after corpus edits → outdated retrievals → outdated answers.

• Mitigation: re-index on change; version index vs corpus.

4. Retrieval gaps → hallucination. If the answer isn't retrievable (corpus gap or top-k miss), the FiD reader may generate ungrounded answers.

• Mitigation: tune k; calibrate on retrieval confidence; abstain when retrieval is weak; the interpretability of retrieved docs helps detect this.

5. Access control. A shared corpus/index can surface documents a user shouldn't see; FiD will read them into the answer.

• Mitigation: ACL-filtered retrieval / per-tenant index partitions.

6. PII / verbatim leakage. FiD can copy retrieved passage text into answers, leaking sensitive corpus content.

• Mitigation: PII scrubbing pre-indexing; output filtering.

Upside for security/trust: Atlas's interpretability (you can inspect the retrieved passages behind any answer) makes auditing and detecting poisoned/incorrect grounding easier than with a closed-book model — a genuine robustness advantage of the retrieval-augmented design.

Answer:

Pre-training/fine-tuning (joint):
Retriever (a Contriever-style dense retriever) + Reader (a Fusion-in-
Decoder-style, #29, generator) trained TOGETHER, with the retriever's
parameters updated based on how much each retrieved passage helped
the reader's output -- similar in spirit to REALM's (#26) latent-
variable training, but built on top of FiD's passage-fusion mechanism
rather than REALM's masked-LM objective

Few-shot fine-tuning (Q5):
Given only a handful of labeled examples for a new task, both
retriever and reader are further fine-tuned jointly -- retrieval
quality itself improves from just a few examples, not just the
reader's output formatting

Inference:
Query -> retriever fetches passages -> FiD-style reader fuses them
       -> answer

Atlas's position in this training-time-RAG family (alongside REALM, #26, and RETRO, #27) is combining REALM's joint-training philosophy with FiD's (#29) proven passage-fusion reading mechanism, specifically optimized for the few-shot regime (Q5) — where joint training's benefit is largest, since a reader with better-tuned retrieval needs fewer labeled examples to reach good task performance than a reader stuck with generic, unspecialized retrieval.

Answer:

Atlas (Izacard et al., Few-shot Learning with Retrieval Augmented Language Models, Meta AI, arXiv:2208.03299, 2022) specifically targeted the few-shot learning regime — demonstrating that a retrieval-augmented model with a jointly-trained retriever and reader could achieve strong performance on knowledge-intensive tasks using orders of magnitude fewer labeled examples than comparable closed-book (no retrieval) large language models required.

The paper's headline result (Q1) is Atlas matching or exceeding much larger closed-book models on knowledge-intensive few-shot benchmarks with a fraction of the parameters, directly extending REALM's (#26) and RETRO's (#27) shared thesis — that retrieval can substitute for parametric scale — specifically into the few-shot setting, where the paper argues retrieval is especially powerful (Q5) because a jointly-trained retriever can be adapted to a new task's information needs from just a few examples, while a closed-book model has no equivalent lever to quickly adapt what it "knows."

Answer:

Atlas's reader is architecturally a Fusion-in-Decoder-style model (#29) — encoding each retrieved passage independently, then fusing them in the decoder's cross-attention (#29 Q2) — so the two share their core passage-fusion mechanism directly. The difference is training: FiD (#29) is typically paired with a frozen retriever (its own comparison table, #29 Q4, treats retriever training as separate from FiD's own contribution), while Atlas trains its retriever jointly with the FiD-style reader, specifically to improve few-shot adaptation (Q5).

This makes Atlas best understood as "FiD's reading mechanism, plus REALM-style joint retriever training, specifically optimized for the few-shot regime" — it doesn't invent a new way of fusing retrieved passages, it takes FiD's already-effective fusion approach and adds the joint-training capability that lets the retriever itself adapt to a new task from limited labeled data, which a frozen retriever paired with FiD alone cannot do.

Answer:

Both jointly train a retriever with a generator, but Atlas targets few-shot task adaptation specifically, while REALM's joint training happens once, during general-purpose masked-language-model pre-training, with the retriever's specialization to any specific downstream task happening only via subsequent fine-tuning on that task's full training set. Atlas's joint training extends into the few-shot fine-tuning stage itself (Q13) — the retriever keeps adapting jointly with the reader even when only a handful of labeled examples exist for a brand-new task, which is a meaningfully different regime than REALM's large-pre-training-corpus joint optimization.

This is what makes Atlas's headline claim (Q14) specifically about few-shot performance rather than REALM's more general "retrieval substitutes for scale" framing — Atlas's contribution is demonstrating that joint retriever-reader training's benefit is not just about absolute task performance, but about how quickly (with how few labeled examples) that performance can be reached, which matters most exactly when labeled data is scarce.

Answer:

Few-shot example count is the knob most specific to Atlas's own contribution among this list — it's the parameter Atlas's headline claim (Q14) is actually about, making it worth the most deliberate empirical validation (Q19) rather than defaulting to a number borrowed from a different task's tuning.

Answer:

Build a few-shot evaluation protocol with a genuinely small number of labeled examples per task, and compare Atlas's joint retriever-reader fine-tuning against a baseline that fine-tunes only the FiD-style reader while keeping the retriever completely frozen (the natural ablation, isolating exactly the joint-training contribution Q16 identifies as Atlas's distinctive mechanism). Measure task accuracy as a function of example count specifically — Atlas's claimed advantage (Q14) should be largest at the smallest example counts, where a frozen, generic retriever has the least chance of happening to already suit the new task well, and should narrow as example count grows large enough that even a frozen retriever's reader-side fine-tuning captures most of the achievable performance.

This curve-shaped comparison, rather than a single-point accuracy comparison, is what actually validates Atlas's specific few-shot claim — a single comparison at one example count can't distinguish "joint training helps a little, uniformly" from "joint training helps a lot exactly in the low-data regime, which is the point," and only the latter finding would confirm Atlas's architecture is solving the problem it specifically targets.

Answer:

With only a handful of labeled examples, jointly updating both retriever and reader parameters simultaneously risks instability: a small number of examples provides a noisy gradient signal, and if the retriever's parameters shift too aggressively based on that noisy signal, retrieval quality can actually degrade relative to the original, more generically-trained retriever — the opposite of the intended few-shot adaptation benefit. This is a genuine risk specifically because joint training couples two components' optimization together; a reader-only fine-tuning approach doesn't have an equivalent way for a few noisy examples to corrupt the retrieval component at all, since the retriever simply isn't touched.

Detection: track retrieval quality (recall@k against a held-out validation set, separate from the few-shot training examples) before and after few-shot joint fine-tuning — a measurable drop in retrieval quality post-fine-tuning, even if reader-side task accuracy improved, is the signature of this failure, and is easy to miss if only end-to-end task accuracy is monitored. Mitigation: a conservative retriever learning rate (Q17) is the primary lever — constraining how far the retriever's parameters are allowed to move during few-shot fine-tuning limits the damage a noisy few-example gradient signal can do, trading some of the theoretically-available adaptation speed for stability, which is generally the right trade-off when the alternative is risking retrieval quality regression from an already-good starting point.

Answer:

Atlas's few-shot approach (Q14) requires actually updating model parameters (both retriever and reader) using the few available labeled examples — a genuine fine-tuning process, even if a lightweight one relative to full-dataset training. Modern in-context learning, by contrast, achieves few-shot task adaptation by simply including the few examples directly in the prompt of a frozen, already-capable LLM, with no parameter updates at all — the same fine-tune-vs-prompt trade-off pattern that recurs throughout this bank (WebGPT vs. Agentic Web RAG, #39/#31; RAFT vs. prompt engineering, #16 Q20).

Prompting superseded joint fine-tuning for most few-shot use cases for the same reason prompted tool use superseded WebGPT-style RLHF fine-tuning (#39 Q20): as base models became dramatically more capable at using in-context examples directly, the accuracy gap that justified Atlas's more elaborate joint-training approach largely closed, while prompting requires no training infrastructure at all and adapts instantly to a new task by simply changing what's in the prompt. Atlas's lasting contribution (Q10) is best understood the way REALM's is (#26 Q20) — establishing that retrieval-augmented few-shot learning was a viable, powerful approach, an insight modern in-context-learning-based RAG systems still rely on even though few of them reproduce Atlas's specific joint-training mechanism.

Answer:

Eighty labeled examples is squarely the regime Atlas was built for (Q1, Q5, Q14), so the core recipe applies directly: initialize with a pretrained Contriever retriever and a pretrained FiD-style reader, then jointly fine-tune both on the 80 examples using an attention-distillation objective so the reader's cross-attention signal shapes what the retriever learns to prioritize (Q3).

What the lab's constraints imply: a university lab typically has limited GPU budget and no dedicated infra team, so full joint retriever training with periodic full-corpus re-encoding (Q3's index-refresh cost) is likely too expensive relative to the benefit. The pragmatic choice is query-side-only retriever updates (Q9) — freeze the document encoder entirely and only update the query encoder, which avoids re-indexing the botany corpus while still letting retrieval adapt somewhat to the task.

Recommended approach: initialize from a strong off-the-shelf Contriever + FiD checkpoint, fine-tune with attention distillation and query-side-only updates, and keep k (retrieved passages) modest (5–10) to control FiD's linear encoding cost on a small compute budget.

Trade-offs to flag: (1) query-side-only updates trade away some of the retrieval-quality gain full joint training would give (Q9's "cheapest vs. fuller" spectrum); (2) the lab should first benchmark against simply prompting a strong modern LLM with the same 80 examples and retrieved passages (Q9's "try that baseline first") — if that's already competitive, the joint-training machinery may not be worth the engineering effort for a one-off academic benchmark.

Answer:

The hard constraint here is combinatorial: dozens of domains, each potentially needing its own retrieval adaptation, but full joint retriever+reader fine-tuning per domain (Q3, Q9) — with its index-refresh cost — doesn't scale economically if it has to be repeated dozens of times and again for every new domain added later.

Recommended architecture: maintain one shared, strong pretrained Contriever + FiD base, and adapt per domain via query-side-only updates plus domain-scoped indexes rather than full joint retraining per domain. Each domain gets its own corpus and index (so a materials-science query only retrieves from materials-science documents), but the retriever and reader backbone stay shared and mostly frozen, with only lightweight, domain-specific query-encoder adaptation from each domain's few-shot examples (Q9's cheapest-tier option).

Why not full joint training per domain: with dozens of domains, the index-refresh cost (Q3, Q12) that Atlas already flags as a limitation multiplies dozens of times over, and re-encoding a domain's corpus every time its query encoder updates is precisely the recurring tax the architecture should avoid at this scale.

Trade-offs: shared-backbone adaptation sacrifices some of the retrieval-quality ceiling full joint training would reach per domain (Q18's frozen-vs-joint comparison), but keeps onboarding a new domain cheap — a few-shot fine-tune of the query encoder against a new index, not a full retraining cycle.

What to monitor: per-domain few-shot accuracy curves (Q18), retrieval recall@k segmented by domain (to catch domains where the shared backbone under-serves specialized vocabulary), and index-staleness gap per domain as documents are added. If a specific domain's accuracy plateau is well below the others, that domain alone may justify a full joint fine-tune as an exception rather than changing the strategy for all.