01 — Azure AI Search for RAG

Answer:

Azure AI Search is the retrieval tier: it stores your chunked content (text, metadata, and embeddings) in an index and returns the best-matching chunks for a query. The LLM (typically Azure OpenAI) never talks to the index directly; your application, or an orchestration layer, calls the search service, takes the top results, and builds the grounded prompt.

Sources (Blob, SharePoint, SQL, Cosmos DB)
   │  indexer + skillset (crack, chunk, embed)
   ▼
Azure AI Search index  ◄── query: keyword + vector (+ semantic rerank) + filters
   │ top chunks + citations
   ▼
App / orchestrator ──► Azure OpenAI (generate grounded answer)

What it adds over a bare vector store: full-text BM25, hybrid fusion, a semantic reranker, filters/facets, indexers that pull from data sources, and enterprise security (Entra ID, RBAC, private endpoints). See the RAG overview on Microsoft Learn.

Answer:

Microsoft's own benchmark guidance is that hybrid plus semantic ranking gives the most relevant results (vector relevance and ranking); in an interview, say "start with hybrid + semantic ranker, then measure" and justify any simpler choice with data.

Answer:

Each attribute decides which on-disk structures are built for a field, so they trade storage and indexing cost for capability.

Practical tips: only mark fields you will actually filter or facet on, because each flag adds index overhead; keep an ID/parent key and a source URL field retrievable for citations; and remember a field with retrievable: false is not a security control. Reference: create a search index.

Answer:

• Exhaustive KNN scans every vector and returns the exact nearest neighbors. It builds no graph, so it does not consume vector index size quota, but cost grows linearly with corpus size.
• HNSW builds a hierarchical navigable small world graph and does approximate nearest neighbor search: high recall, low latency, but the graph must sit in memory and counts against vector index size quota.

Tunable HNSW parameters are m (neighbors per node), efConstruction (index-time candidate list), efSearch (query-time candidate list) and the metric (cosine, dotProduct, euclidean). Docs note a default efConstruction of 400 with a range of 100 to 1,000.

Choose HNSW for most production corpora. Choose exhaustive KNN for small or medium datasets, when exactness matters more than speed, or to build ground truth when measuring HNSW recall. A field indexed for HNSW can still be queried exhaustively with "exhaustive": true; the reverse is not possible. Source: vector relevance and ranking.

Answer:

The semantic ranker is a Microsoft-hosted language-model reranker (adapted from Bing) applied after initial retrieval. It takes the BM25 or RRF result set, re-scores it by meaning, and can return extractive captions (highlighted passages) and optionally answers. Scores in @search.rerankerScore range from 0 to 4.

Limits to know:

• It only reranks the top results of the first-stage ranking (docs say the top 50), so it cannot rescue a document that first-stage retrieval missed.
• Each document is summarized to a bounded input before scoring, so long chunks are truncated and field priority order in the semantic configuration matters.
• Captions and answers are verbatim text from your index; nothing is generated.
• It is a premium feature billed by usage, with a free monthly allowance.

Source: semantic ranking overview.

Answer:

Billing in the dedicated model is by search units, replicas multiplied by partitions, so cost scales roughly linearly as you add either. A replica cannot help if you have run out of storage or vector quota; a partition will not raise QPS much. For availability SLAs you need multiple replicas (check the current SLA page for exact counts).

Tiers (Free, Basic, S1 to S3, S3 HD, L1/L2 storage-optimized) set per-partition size and object limits. As of late 2026 there is also a serverless pricing model in preview that removes replica/partition management; check current docs before depending on it. Source: choose a tier and capacity planning.

Answer:

A hybrid request carries a search text string and one or more vectorQueries. The engine executes each as a separate query in parallel, producing independent ranked lists, then fuses them with RRF.

For every document in every list: score = 1 / (rank + k) with a small constant k (the docs cite 60). Scores for the same document are summed across lists and the fused list is sorted. Because RRF uses rank positions rather than raw scores, you never have to normalize BM25 (unbounded) against cosine-derived scores (about 0.33 to 1).

{
  "search": "reset MFA for contractors",
  "vectorQueries": [
    { "kind": "text", "text": "reset MFA for contractors",
      "fields": "contentVector", "k": 50, "weight": 2.0 }
  ],
  "select": "id, title, chunk, url",
  "top": 10
}

Gotchas: the RRF constant k is unrelated to the k of nearest neighbors; weight on a vector query scales that list's contribution; a text query plus two vector queries over five fields is 11 parallel executions. The kind: text query relies on a vectorizer configured on the field. Source: hybrid search scoring (RRF).

Answer:

Integrated vectorization moves chunking and embedding inside the indexing pipeline, and embedding of query text inside the query pipeline, so you do not run a separate ETL job.

Indexing side: a data source, an indexer (pulls, cracks documents, tracks changes), a skillset (a chunking skill such as Text Split, then an embedding skill such as the Azure OpenAI Embedding skill), and an index that receives the chunk rows (usually via index projections).

Query side: a vectorizer defined in the index and attached to a vector profile, so a kind: text vector query is embedded with the same model automatically.

Blob/SharePoint ─► indexer ─► skillset[ split ─► embed ] ─► index (chunks + vectors)
query text ─► vectorizer (same model) ─► vector query

Trade-offs: much less glue code and built-in change tracking, but less control over chunking logic, retries and throttling (retry/batching is built in and not configurable), and you share Azure OpenAI tokens-per-minute quota between indexing and queries. For custom parsing or exotic chunkers, many teams run their own pipeline and push documents. Source: integrated vectorization.

Answer:

Goal: match on small, precise chunks but give the LLM the surrounding context.

Two common patterns:

1. Index projections (one-to-many). The skillset splits each document into chunks and projects each chunk as its own row into a chunk index, with a parent_id field linking back to the source. Optionally a second index holds parent-level metadata. When a chunk matches, you look up the parent (or neighbors) by key.
2. Single index with context fields. Store chunk, chunk_order, parent_id and perhaps prev/next text or a larger parent_text field (not searchable, retrievable). After retrieval, fetch neighbors with a filter like parent_id eq 'x' and chunk_order ge 4 and chunk_order le 6.

Design notes: store a stable key per chunk, keep parent_id filterable, and project ACL fields to every chunk row or security trimming silently fails. Chunk size guidance is corpus-specific: measure retrieval recall at a few sizes and overlaps rather than copying a default. See chunk large documents and index projections.

Answer:

Rescoring re-ranks the compressed-search candidates using original (or, for binary, the binary dot product) vectors. Oversampling controls how many extra candidates are pulled before rescoring: effective candidates is roughly k times the oversampling factor, and only the top k are returned. Only HNSW supports it; exhaustive KNN scans everything anyway.

Process: create a new vector profile and field (you cannot change compression on an existing field), then measure recall/NDCG with and without oversampling on your own queries. Source: vector quantization.

Answer:

Interaction rules of thumb: set k at least as large as what you want after fusion (for example 30 to 50) so the vector side can compete with BM25; use top for the number of chunks sent to the LLM; raise efSearch or use oversampling only after you measure a recall gap against exhaustive results. Do not confuse k with the RRF constant. A relevance threshold on vector or hybrid queries exists as a preview feature at the time of writing. Source: vector query how-to.

Answer:

A scoring profile adjusts the BM25 relevance score of text queries using field weights and functions (freshness, magnitude, distance, tag). That makes it good for boosting recent documents, authoritative sources, or title matches in the keyword portion of retrieval.

The Learn relevance page states that vector queries do not have scoring profile or term-boosting features. So in a hybrid query, a profile influences only the text ranking, and RRF then fuses it with vector ranks, which dilutes its effect. A boost you need to be strong should be applied differently:

• Filter instead of boost for hard rules ("only current policy versions").
• Add a recency or authority signal after retrieval in your own re-rank step, or via semantic configuration field choice.
• Use vector weight to emphasize semantic versus keyword evidence.

Interview point: treat scoring profiles as a keyword-side tool and verify with debug subscores whether they actually change final order. Source: vector relevance and ranking and relevance overview.

Answer:

Filters use OData expressions over filterable fields (tenant eq 'a' and year ge 2024). With vectors there are two strategies:

• Pre-filter: restrict the candidate set first, then do nearest-neighbor search among matches. You get k results that all satisfy the filter, which is what RAG usually wants.
• Post-filter: find nearest neighbors first, then filter. If the filter is selective, you can end up with fewer than k results, or none.

Azure AI Search exposes this through a vector filter mode setting (pre/post variants; confirm names and defaults in the filters in vector queries article).

Failure modes to mention:

1. A very selective pre-filter on HNSW can reduce recall or latency predictability, so benchmark selective filters (per-tenant, per-user) explicitly.
2. Filters on non-filterable fields fail at query time.
3. High-cardinality Collection(Edm.String) fields used for ACLs are fine with search.in, but giant OR chains are slow.
4. Compare with Qdrant or Weaviate, which advertise filter-aware HNSW traversal; always test your own filter selectivity rather than assume parity.

Answer:

Evaluate retrieval separately from generation.

1. Golden set: 50 to 200 real queries with labeled relevant chunk or document IDs (mine from logs, have SMEs label, or bootstrap with an LLM and spot check).
2. Metrics per configuration: recall@k, MRR, NDCG@10 for each variant: BM25 only, vector only, hybrid, hybrid + semantic ranker, different chunk sizes, with and without quantization.
3. Ground truth for ANN: run the same vector query with "exhaustive": true and compare to HNSW results to measure recall loss from HNSW or compression.
4. Per-query diagnosis: pass the debug parameter (for example vector, semantic, or all) to see subscores and understand whether the text or vector side surfaced a document (hybrid scoring docs).
5. End-to-end: add faithfulness and answer relevance scoring (Azure AI Foundry evaluators, RAGAS, or similar) once retrieval is stable.
6. Production signals: log query, filters, returned IDs, reranker scores and thumbs up/down; alert on queries with a low top reranker score as a proxy for corpus gaps.

Change one variable at a time and keep the golden set fixed so regressions are attributable.

Answer:

Microsoft Learn describes four approaches (document-level access control):

Choosing: for custom identity systems or anything you need in production today, use security filters, with the group field filterable and retrievable: false. Use native trimming when the source is ADLS Gen2, SharePoint, or Purview-governed content and preview terms are acceptable.

Pitfalls: (1) the filter is only as trustworthy as your app; resolve group membership server-side from a validated token, never from client input. (2) Chunked indexes must carry the ACL on every chunk (index projections). (3) Permission changes in the source only appear after the next sync/indexer run, so there is an exposure window. (4) retrievable: false hides the field but is not an access control. Also consider separate indexes per tenant only when isolation requirements justify the object-count limits.

Answer:

Constraint: ethical walls make a leak a compliance incident, and permissions change faster than a nightly crawl.

Design:

1. Ingestion: SharePoint indexer (or a custom pipeline using Microsoft Graph) into a chunk index through a skillset (Text Split + embedding). Project matter ID and permission fields onto every chunk.
2. Authorization choice: evaluate the SharePoint ACL ingestion preview, which uses the caller's Entra token. If preview status is unacceptable, use the security filter pattern: store the staffing group/matter IDs in a filterable, non-retrievable collection field and filter every query with search.in.
3. Freshness of permissions: do not rely on the nightly full indexer for access changes. Make staffing the source of truth in a small table; resolve the attorney's current matter list at query time and pass it as the filter. Then adding or removing a person takes effect immediately, with no re-index. Reserve re-indexing for content changes.
4. Query path: hybrid + semantic ranker, pre-filtered by matter list; log the filter applied for audit.
5. Verification: automated tests that run attorney A's queries and assert no chunk with matter outside A's list ever returns, including adversarial prompts and queries crafted to match another matter.
6. Defense in depth: private endpoints, managed identity, Entra RBAC (Search Index Data Reader) for the app, no direct user access to the index.

Trade-off: query-time matter lists can be long; search.in handles hundreds to thousands of values with sub-second filters per the docs, but if lists grow very large, model access as a smaller set of group IDs instead.

Answer:

Diagnosis: dense embeddings capture meaning but tokenize opaque identifiers into subword pieces, so 0x80070005 is "close" to many similar codes. This is a lexical-match failure, not a model-quality failure. Confirm by running the failing queries as pure BM25 and checking that the right chunk ranks first; use debug subscores to see the two sides.

Fix:

1. Move to hybrid retrieval: keep the vector field, add search text over a searchable chunk field so BM25 catches the literal token. RRF merges both lists.
2. Check the analyzer on the text field. The standard analyzer may split 0x80070005 or part numbers with hyphens oddly; consider a custom analyzer or a keyword-style field for IDs, and mirror the ID into a separate searchable field you can boost with a scoring profile or searchFields.
3. Add the semantic ranker over the fused top results so a paraphrase-friendly model orders the final list.
4. Evaluate with a golden set containing both exact-code and paraphrase queries; tune vector weight if one side dominates.
5. Optionally normalize queries (extract code patterns with a regex and issue a filter/exact lookup first).

Trade-off: hybrid adds a text index and slightly more latency but is the standard remedy; do not fine-tune embeddings first.

Answer:

Agentic retrieval is a multi-query pipeline exposed through a knowledge base that references one or more knowledge sources (indexed content on your service or remote sources). Per the Learn overview, at low or medium reasoning effort an LLM plans focused subqueries from the question and chat history; the subqueries run in parallel (keyword, vector, or hybrid), each is semantically reranked, and results are merged into one grounding response with optional references and an activity log. At minimal effort the planning step is skipped.

Use it when: questions have several asks, depend on conversation context, contain typos, or span several sources. Stay classic when: queries are simple single-fact lookups, latency is tight, or you want full control and predictable cost.

Cost and status notes (verify in current docs): classic semantic ranking is billed per query, agentic retrieval is billed in tokens plus Azure OpenAI tokens for planning; LLM-based planning and answer synthesis were preview features and tied to preview REST versions, while minimal-effort retrieval with GA knowledge source types was available on the stable API. Foundry IQ builds on the same knowledge-base concept. Source: agentic retrieval overview.

Compare: this resembles query decomposition in LangChain/LlamaIndex, but hosted and with built-in reranking.

Answer:

How to decide: if the stack is already Azure, security/compliance matters, and you want fewer moving parts, Azure AI Search is usually the shortest path. Pick pgvector when the data already lives in Postgres and corpus size is moderate. Pick Elasticsearch/OpenSearch when you need heavy full-text analytics. Pick Qdrant or Weaviate for portability, cost control, or advanced filtering at scale. None of these is universally best: run your golden set against two candidates before committing. Capabilities change quickly, so check each project's current docs.

Answer:

Step 1: raw vector math. 1536 float32 dimensions is 6,144 bytes per vector. 50M vectors is about 307 GB before the HNSW graph, text index, and metadata. Quantization shrinks it: int8 scalar is about 4x (roughly 77 GB of vectors), binary far more, at some recall cost. Truncating dimensions on a Matryoshka-capable embedding model shrinks it further.

Step 2: what counts against quota. HNSW graphs live in memory and consume the vector index size quota; exhaustive KNN fields do not, but then you pay in query latency. Keeping full-precision originals for rescoring adds storage back. Look up the current vector quota per partition for your tier in service limits and vector index size; do not rely on remembered numbers.

Step 3: pick a tier and partitions. Candidates are S-series for balanced latency or storage-optimized L-series for large, rarely updated indexes (higher query latency). Partitions = storage need divided by per-partition vector quota, with headroom for updates.

Step 4: replicas from peak QPS and HA requirements; load test, since latency depends on filters, hybrid, and semantic ranker.

Step 5: other cost lines. One-time embedding of 50M chunks (Azure OpenAI tokens), semantic ranker per-query billing, agentic retrieval tokens if used, and re-embedding cost if you change models.

Levers: quantization with oversampling, retrievable: false on vectors, fewer filterable/facetable fields, dedupe chunks, tier choice. Compare the total against self-hosted Qdrant or OpenSearch including ops time, not just infrastructure.

Answer:

Treat ingestion like any ETL system: idempotent keys, monitoring of success counts vs source counts, and a replayable path. Source: integrated vectorization limitations and indexer overview.

Answer:

The semantic ranker does not read your whole chunk. Per the Learn overview, only the top 50 first-stage results are reranked, and each document is summarized to a bounded input (up to 2,048 tokens for the generated summary string, built from title, keywords and content fields in the semantic configuration, with the title and keyword fields themselves capped at 128 tokens). Anything beyond the limit is trimmed, and field order in the configuration sets trimming priority.

Consequences:

• A chunk much longer than the summarization budget can be ranked on a partial view, so a fact near the tail may not influence its reranker score.
• Very large chunks dilute embeddings too, so vector recall also suffers.
• Captions are extractive snippets for display; they are not guaranteed to contain the fact the LLM needs, so do not send only captions to the model.

Fixes: use smaller chunks (a few hundred tokens up to roughly a page, tested on your corpus) with overlap; put the most descriptive field first in the semantic configuration; use parent/child retrieval so the ranker scores precise chunks while the LLM gets expanded context (Q9); raise top or k so more candidates reach the LLM; and evaluate with a golden set that includes "fact deep in the document" cases. Limits and token budgets can change, so re-check semantic ranking overview.

Answer:

Correct: B. RRF ignores raw scores and uses each document's position in each ranked list, adding 1/(rank + k) contributions (docs cite k around 60) so documents ranked well in multiple lists rise. Option A describes score normalization, which RRF is designed to avoid because BM25 is unbounded while vector scores are not. Option C is an intersection strategy; RRF keeps documents that appear in only one list, just with a lower combined score. Option D is wrong because RRF is a fixed formula with no per-index training; the only tuning knobs are things like vector query weights.

Answer:

Correct: D. The semantic ranker is a second-stage (L2) reranker over the BM25 or RRF results (docs state the top 50), and its captions and answers are extracted verbatim from indexed content. Option A is wrong because it cannot rerun the query over the whole corpus; recall is bounded by first-stage retrieval. Option B is wrong because there is no generative model in this step, so it never composes new text. Option C is wrong because it depends on first-stage retrieval to supply candidates and cannot replace it.

Answer:

Correct: A. The documented security-filter pattern uses a string collection that is filterable so queries can apply group_ids/any(g: search.in(g, '...')), with retrievable set to false so the ACL list is not echoed in results. It must be present on every chunk row, otherwise chunks without it are not correctly trimmed. Option B makes access depend on full-text matching of a delimited string and exposes group lists in responses. Option C is wrong because facetable does not enable filtering, and trimming only the parent document leaves chunk rows unprotected. Option D is wrong because sortable has nothing to do with filtering and a separate per-file index does not constrain chunk results.

Answer:

Correct: C. Per Microsoft Learn, oversampling for scalar-quantized vectors requires the original full-precision vectors to rescore candidates; with discardOriginals they are gone, so rescoring and oversampling cannot be used. (Binary quantization is the exception, since it can rescore using the binary dot product.) Option A is backwards: only HNSW supports rescoring, because exhaustive KNN scans everything anyway. Option B is wrong because oversampling is not tied to a pricing tier. Option D is wrong because oversampling is an ordinary query parameter on vector queries against compressed fields when the original vectors are kept.