Retrieval Augmented Generation Tutorial for Developers

Retrieval-augmented generation started, for me, as a debugging problem. An LLM deployed inside an enterprise kept confidently answering questions about a product line the company had discontinued eighteen months prior. The model's training data said the product existed; the warehouse said it did not. The fix people kept proposing was a better model, a bigger model, a fine-tuned model. None of those proposals addressed the actual problem, which was that the model had no mechanism to consult reality at inference time. That experience clarified something that has shaped how I think about every LLM deployment since: hallucination, in most production contexts, is not a model quality problem. It is a structural problem. The model is pattern-matching from training weights rather than reasoning over current facts, because current facts were never made available to it.

RAG is the architectural response to that structural problem. It does not bake knowledge into weights. It retrieves external information at inference time and folds it into the generation step, so the model is reasoning over evidence rather than recalling impressions. The distinction from fine-tuning is meaningful: fine-tuning updates the model's parameters to reflect new knowledge, which is expensive, slow, and still produces a static artifact with its own cutoff date. The distinction from prompt stuffing, where relevant documents are manually inserted into context without any retrieval logic, is equally meaningful: prompt stuffing does not scale, does not generalize, and requires a human to decide what is relevant.

What RAG produces, when built correctly, are answers grounded in real, citable sources. The model can point to the document that justified its claim, and a downstream system or a user can verify that the claim is actually in that document.

This is no longer an experimental pattern. Amazon's Alexa for Shopping, Google AI Search, and ChatGPT Search all use retrieval augmentation as load-bearing infrastructure, not as a research prototype layered on top. Enterprise adoption is accelerating; analysts broadly characterize RAG as the dominant architectural pattern for deploying LLMs in business contexts, with the market projected to reach multibillion-dollar scale within the decade. For a developer evaluating where to invest deeply, that trajectory is a meaningful signal. RAG is the baseline expectation for production AI, not an advanced option.

The Three-Stage Pipeline Every RAG System Is Built On

Every RAG system, regardless of complexity, runs on the same three-stage pipeline: retrieval, augmentation, and generation. Understanding these stages as distinct, composable steps is more useful than any particular framework or library, because it gives you a mental model for diagnosing failures and targeting improvements.

Stage 1, Retrieval, is where the system finds relevant information. Text, whether a user query or a document in your corpus, is converted into a dense numerical vector by an embedding model. Semantic similarity between query and document vectors is then computed, usually via cosine similarity or dot product, to surface the most relevant chunks. Most production failures originate here. Wrong chunks, poor ranking, irrelevant context: these errors propagate forward and no amount of generation-layer sophistication recovers from them.

Stage 2, Augmentation, is where retrieved chunks are assembled into the prompt that gets sent to the LLM. This is not mere concatenation. The ordering of chunks, the selection among retrieved candidates, and the structure of the surrounding prompt template all materially affect what the model attends to and how faithfully it uses the retrieved context. Augmentation is where retrieval quality becomes visible to the LLM: a well-retrieved but poorly assembled prompt will still produce degraded output.

Stage 3, Generation, is where the LLM produces an answer grounded in the retrieved context, ideally with citations back to specific sources. Citation is a design requirement, not a courtesy feature. It allows users and downstream systems to verify provenance, which is the mechanism that makes RAG's outputs auditable in a way that pure generation is not. The generator, importantly, is only as good as what it receives. This deserves emphasis because teams consistently reach for a larger or more capable model when outputs are poor, when the actual deficiency is upstream in retrieval or augmentation.

Every technique in the sections that follow targets one of these three stages. Orienting your learning around which stage a given technique improves will make the progression legible.

Building the Retrieval Layer: Chunking, Embeddings, and Vector Stores

Chunking Strategy

How you split documents before embedding them is one of the most consequential decisions in a RAG build, and it is the one most often treated as a configuration detail rather than a first-order design choice.

Fixed-size chunking divides documents by character or token count with a defined overlap between chunks. It is simple, predictable, and reproducible. It is also frequently wrong at boundaries, splitting sentences mid-thought or separating a claim from the evidence that follows it. For structured data like logs or tabular records, fixed-size chunking is often appropriate because the unit of meaning is already well-defined.

Semantic chunking splits on meaning rather than count. The most common approach embeds rolling sentences or paragraphs and places chunk boundaries at points of high embedding distance, where the topic is shifting. The qualitative gain in recall is real: chunks that contain coherent, self-contained information surface for the right queries more reliably than chunks that happen to begin and end at the 512th token.

The practical guidance is simple: start with semantic chunking unless your corpus structure makes fixed-size obviously appropriate. The overhead is modest, and the retrieval improvement typically justifies it.

Embedding Model Selection

The embedding model converts both your documents and your queries into the vector space where similarity is computed. Three practical considerations govern the choice.

First, embed documents and queries with the same model. This is non-negotiable. Different models construct different vector spaces; similarity scores across models are not meaningful.

Second, understand the quality-latency-cost tradeoffs. Hosted models like OpenAI's text-embedding-3 series are high quality and easy to integrate. Voyage-3-large, in published benchmark comparisons across diverse retrieval tasks, outperforms both OpenAI and Cohere embeddings by a meaningful margin; for quality-sensitive applications it is worth evaluating. Open-source models like those from the Sentence-BERT family can be self-hosted for cost control at scale but require you to manage the infrastructure.

Third, consider domain specificity. General-purpose embedding models perform well on general-purpose text. Highly technical domains, such as biomedical literature or legal contracts, may warrant domain-tuned models or at minimum benchmarking your specific corpus against several candidates before committing.

Vector Store Selection

The vector store is where embedded chunks live, indexed for efficient similarity search. The consolidating shortlist in production settings includes Pinecone, Weaviate, Milvus, and Qdrant. Choosing among them involves a small number of concrete tradeoffs.

Managed services like Pinecone reduce operational overhead significantly: no infrastructure to run, no index management to perform manually. Self-hosted options like Qdrant (which runs locally with minimal setup, making it useful for development) and Milvus give you more control over data residency, cost structure, and configuration. For teams without dedicated MLOps capacity, a managed service is usually the right starting point.

Evaluate candidates on indexing speed, query latency at scale, filtering support (the ability to restrict retrieval to a subset of the corpus by metadata), and whether hybrid search is supported natively, which brings us to the next point.

Hybrid Search

Neither keyword search nor vector search alone is sufficient for production retrieval. Keyword search misses paraphrased queries; it fails when the user's vocabulary does not match the document's vocabulary. Vector search misses exact-term requirements; it finds semantically adjacent results but can lose precision on specific product codes, names, or technical identifiers. Hybrid search combines both, using BM25 or a similar sparse retrieval method alongside dense vector retrieval, then merging and re-ranking the result sets.

Hybrid search should be treated as the production default, not an advanced optimization. The cost of implementing it early is low. The cost of retrofitting it after deploying a vector-only system that consistently fails on exact-term queries is considerably higher.

RAG-Fusion and Reciprocal Rank Fusion

When your corpus spans multiple sources or heterogeneous document types, a single retrieval query may not surface all relevant information, because the framing that retrieves well from one source retrieves poorly from another. RAG-fusion addresses this by issuing multiple reformulated queries, retrieving separate result sets for each, and merging them using reciprocal rank fusion (RRF).

RRF assigns each document a score based on its rank in each result set rather than its raw similarity score, which makes the merged ranking robust to score-scale differences across retrieval runs. The fusion computation is straightforward:

def reciprocal_rank_fusion(result_sets, k=60):
    scores = {}
    for result_set in result_sets:
        for rank, doc_id in enumerate(result_set):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank + 1)
    return sorted(scores.items(), key=lambda x: x[1], reverse=True)

The k parameter dampens the influence of very high-ranked documents and is typically set between 40 and 80. Use this pattern when your corpus is multi-source or when a single query reformulation is unlikely to capture all relevant angles of a complex question.

Wiring Retrieval to an LLM: A Working End-to-End Implementation

Stack Selection

For a tutorial implementation that most developers can run without friction, the choices here are pragmatic rather than exhaustive.

LLM: OpenAI GPT-4o via API. Capable, well-documented, and familiar to the majority of the developer audience likely building their first RAG system.

Embedding model: Voyage-3-large for quality-sensitive use cases; OpenAI text-embedding-3-small for cost-sensitive ones. Pick one and use it consistently throughout.

Vector store: Qdrant running locally via Docker for development (zero-infrastructure, no account required); Pinecone for teams that want a managed environment from the start.

Orchestration: LangChain or LlamaIndex. The abstraction cost of a framework is real: you are adding a dependency and accepting its conventions. The benefit is that the framework handles prompt templating, chain management, and retriever interfaces in a way that lets you focus on the logic of your application rather than on plumbing. For a first implementation, the benefit outweighs the cost. For a high-performance production system, you may eventually want to reduce framework dependencies, but that is a later-stage optimization.

Step-by-Step Implementation

Step 1: Load and chunk the document corpus.

from llama_index.core import SimpleDirectoryReader
from llama_index.core.node_parser import SemanticSplitterNodeParser
from llama_index.embeddings.openai import OpenAIEmbedding

embed_model = OpenAIEmbedding(model="text-embedding-3-small")
documents = SimpleDirectoryReader("./corpus").load_data()

splitter = SemanticSplitterNodeParser(
    buffer_size=1,
    breakpoint_percentile_threshold=95,
    embed_model=embed_model
)
nodes = splitter.get_nodes_from_documents(documents)

The breakpointpercentilethreshold controls how sensitive the splitter is to topic shifts. A higher value produces fewer, larger chunks; a lower value produces more granular ones. Tune this against your corpus.

Step 2: Embed chunks and upsert to the vector store.

from qdrant_client import QdrantClient
from qdrant_client.models import Distance, VectorParams, PointStruct

client = QdrantClient(":memory:")  # Use URL for persistent instance
client.create_collection(
    collection_name="corpus",
    vectors_config=VectorParams(size=1536, distance=Distance.COSINE)
)

points = []
for i, node in enumerate(nodes):
    vector = embed_model.get_text_embedding(node.get_content())
    points.append(PointStruct(id=i, vector=vector, payload={"text": node.get_content(), "source": node.metadata.get("file_name")}))

client.upsert(collection_name="corpus", points=points)

Step 3: Embed an incoming query and retrieve top-k chunks.

def retrieve(query: str, top_k: int = 5):
    query_vector = embed_model.get_text_embedding(query)
    results = client.search(collection_name="corpus", query_vector=query_vector, limit=top_k)
    return [(r.payload["text"], r.payload["source"]) for r in results]

The top_k value is a lever with real consequences. Too high and you introduce irrelevant context that dilutes the signal; too low and you risk missing relevant chunks. Five is a reasonable starting point; evaluate against your specific corpus and query distribution.

Step 4: Build the augmented prompt.

def build_prompt(query: str, retrieved_chunks: list) -> str:
    context_block = "\n\n".join(
        f"[Source: {source}]\n{text}" for text, source in retrieved_chunks
    )
    return f"""You are a precise assistant. Answer the question using only the information provided in the context below. If the context does not contain sufficient information to answer, say so explicitly. Cite the source for each claim.

CONTEXT:
{context_block}

QUESTION:
{query}

ANSWER:"""

The instruction to cite sources and to acknowledge when context is insufficient are both load-bearing elements of this template. Remove either and the model's behavior degrades toward generation from weights rather than from retrieved evidence.

Step 5: Call the LLM and return a cited response.

from openai import OpenAI

openai_client = OpenAI()

def answer(query: str) -> str:
    chunks = retrieve(query)
    prompt = build_prompt(query, chunks)
    response = openai_client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}],
        temperature=0.2
    )
    return response.choices[0].message.content

Low temperature (0.2) is intentional. You want the model reasoning faithfully over retrieved context, not generating creative variations.

Common Failure Modes

Context dilution from high top-k. Retrieving fifteen chunks when five would suffice fills the context window with marginal or irrelevant material. The model attends to all of it, and answer quality degrades. Monitor retrieval precision, not just recall.

Chunks too large. If a chunk is several thousand tokens, the LLM may attend to the wrong portion of it. Smaller, coherent chunks produce more reliable attention.

Prompt template too loose. If the template does not explicitly instruct the model to stay within the retrieved context, a sufficiently capable model will often generate plausible-sounding answers from its training weights, especially when the retrieved context is sparse or ambiguous. The template above is explicit about this boundary; variations that remove that instruction tend to surface hallucinations.

What this system does not yet handle: questions about events that happened after your corpus was indexed, and questions complex enough to require iterative retrieval. Those are the next two sections.

Why Static Knowledge Bases Alone Break Down in Production

The failure mode is not subtle, and it tends to surface at the worst possible moments: a user asks about yesterday's regulatory update, last hour's price movement, or a product that shipped this morning, and the system returns a confident, well-cited answer that is simply out of date.

Indexing latency is irreducible. A pre-indexed corpus cannot answer questions about events that postdate its last ingestion run. The gap between when something happens and when a vector store knows about it is typically hours to days for teams running batch re-ingestion pipelines, and the maintenance of those pipelines is a cost most teams underestimate. Documents go stale, get superseded, get deleted. A static vector store without continuous re-ingestion is not a knowledge base; it is an archive.

The domains where this matters most are also the domains where AI deployment is most commercially attractive: financial markets, live inventory and e-commerce, breaking news, regulatory compliance, flight availability, clinical trial updates. In each of these, the cost of a stale answer is not an inconvenience; it is a business risk.

The deeper issue is structural. Static RAG is, by design, retrieval from a snapshot. It inherits the same fundamental constraint as a search engine operating on a crawl from last week: it knows what it knew when it last looked. But what if a better vector store or more frequent re-ingestion could close that gap entirely? It cannot. A better vector store does not solve this. A more frequent re-ingestion pipeline reduces the staleness window but does not eliminate it. The fix for applications that need to reason about the current state of the world is architectural: add a live retrieval source.

Adding Real-Time Web Retrieval to the Pipeline

The Two-Component Pattern

Web retrieval for RAG requires two capabilities: discovering relevant pages and extracting clean, LLM-ready text from those pages. These are distinct problems. A search API handles the first; a content extraction or reader API handles the second. Raw HTML is not a suitable input for an LLM. It is token-expensive, noisy with navigation elements and scripts, and poorly structured for reasoning. The extraction step is not a convenience; it is a quality gate. The target output format is clean Markdown: structured, stripped of noise, and token-efficient.

Web Search API Options in 2025

The developer tooling for AI-native web retrieval has matured considerably. Several distinct categories have emerged.

AI-native retrieval APIs are designed from the ground up for LLM consumption rather than for human browsing. Tavily returns answer-first outputs with source citations and straightforward pricing tiers; it is oriented toward retrieval tasks where you want a synthesized, backed result rather than a ranked list of links. Exa operates on an embedding-based semantic index, which means it retrieves based on conceptual similarity rather than keyword matching; results tend to be richer in context per result. Both return content formatted for AI pipelines rather than requiring downstream parsing logic.

Independent-index APIs like the Brave Search API maintain their own web index rather than operating as a wrapper around a major engine's results. The practical consequence is that Brave does not log query data by default, which is a meaningful advantage in privacy-sensitive verticals: healthcare, legal, and financial services all operate under regulatory frameworks where query logging to a third party is a compliance consideration.

SERP scraper APIs like SerpAPI and Serper offer lower cost per query but return raw structured data that requires more processing before it is LLM-ready, consuming tokens that would otherwise go to generation. It is also worth considering the legal risk dimension here: services that scrape search engine results pages have faced ongoing legal scrutiny from the platforms they scrape. Google's suit against Bright Data on these grounds is a public example. Teams building production systems on scraper-backed retrieval should factor platform terms of service and evolving legal exposure into the architecture decision.

You.com offers Web Search and Contents APIs oriented explicitly toward developer use cases, returning real-time web data formatted for LLM consumption. Its performance on DeepSearchQA benchmarks is worth noting for teams where retrieval accuracy is a primary concern.

MCP integration is an emerging consideration for teams building agentic workflows. Brave Search, Exa, and Firecrawl ship official Model Context Protocol servers; others have community-maintained options. If your architecture involves agents that invoke retrieval as a tool, MCP compatibility simplifies the integration pattern.

Integrating Web Retrieval into the Existing Pipeline

The architectural move here is to treat web search as a second retrieval source operating in parallel with the vector store, not as a replacement for it. Static corpus retrieval and live web retrieval serve different query types: the former for proprietary or domain-specific knowledge, the latter for current events and publicly available information. A routing layer decides which source to query, or whether to query both.

import asyncio

async def retrieve_from_vector_store(query: str) -> list:
    # Reuses the retrieve() function from earlier
    return retrieve(query)

async def retrieve_from_web(query: str, api_client) -> list:
    response = await api_client.search(query)
    return [(result["content"], result["url"]) for result in response.results]

async def parallel_retrieve(query: str, api_client) -> list:
    vector_results, web_results = await asyncio.gather(
        asyncio.to_thread(retrieve_from_vector_store, query),
        retrieve_from_web(query, api_client)
    )
    return deduplicate(vector_results + web_results)

Running both retrieval calls concurrently via asyncio.gather keeps end-to-end latency acceptable even with two I/O-bound operations. The deduplication step before augmentation prevents the LLM from receiving redundant context, which wastes tokens and can create conflicting citation signals.

The augmentation step downstream does not need to know which source produced which chunk; it receives a ranked list of (text, source) tuples either way. Keeping the interface uniform means the prompt template and the LLM call are unchanged. Clean extracted content from a reader API meaningfully reduces the token volume sent to the LLM compared to minimally processed web output, which is a real cost consideration at scale.

Extending the Pipeline to Agentic RAG for Multi-Step Reasoning Tasks

Where the Single-Pass Pipeline Hits Its Ceiling

A single retrieve-then-generate pass assumes that the right retrieval query is knowable before any retrieval happens, and that one round of retrieval is sufficient. Neither assumption holds for complex questions.

Consider: "How did the Federal Reserve's interest rate decisions in the past six months affect mortgage origination volumes, and how does that compare to the pattern from the 2018 tightening cycle?" Answering this requires retrieving current rate decision history, retrieving current mortgage data, retrieving historical data from 2018, and then synthesizing across all three. The retrieval query for the third step depends on what the first two returned. A static pipeline cannot express this dependency.

Multi-hop reasoning, where answer A depends on finding B which depends on finding C, is the canonical case. But even simpler tasks, like answering a question where the relevant information is distributed across sources that use different terminology, can exhaust a single-pass pipeline. That raises an important question: if a single pass is structurally insufficient, what does the alternative look like?

What Agentic RAG Adds

An agentic RAG system replaces the fixed pipeline with an LLM that plans, calls retrieval as a tool, reasons over intermediate results, and decides whether to retrieve again or generate a final answer. The loop is not predetermined; the agent controls it. "Retrieve once, generate once" becomes: decide what to retrieve, retrieve it, reason over the result, decide whether that is sufficient, retrieve again if not, and generate when the evidence base is adequate.

This is a meaningfully different architecture. The agent is not executing a script; it is exercising judgment about information sufficiency, which means the quality of the decision prompt and the termination condition are now first-class design concerns.

Framework Options

LangChain's LangGraph models the agent as an explicit state machine. Retrieval loops, decision nodes, and termination conditions are all represented as graph edges. This is verbose to set up but produces auditable systems where every state transition is inspectable, which matters in regulated environments.

LlamaIndex's AgentQueryEngine operates at a higher abstraction level and is faster to prototype. It trades auditability for convenience; appropriate for early-stage development, less so for production deployments where explainability is required.

Microsoft AutoGen enables multi-agent patterns where separate agents handle distinct roles, such as a retrieval agent and a generation agent coordinating through a shared message channel. Useful when retrieval and synthesis logic are complex enough to benefit from separation of concerns.

OpenAI function calling and tool use is the low-level primitive most frameworks build on. Defining retrieval as a callable tool and letting the model decide when and how to call it is the foundational pattern; understanding it directly makes framework choices more legible.

A Practical Implementation Sketch

The tool definitions and the decision prompt are where most tutorials stop short. Here is a minimal implementation that makes them explicit.

tools = [
    {
        "type": "function",
        "function": {
            "name": "search_web",
            "description": "Search the live web for current information on a topic.",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string", "description": "The search query"}
                },
                "required": ["query"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "query_vector_store",
            "description": "Retrieve relevant chunks from the internal document corpus.",
            "parameters": {
                "type": "object",
                "properties": {
                    "query": {"type": "string", "description": "The retrieval query"}
                },
                "required": ["query"]
            }
        }
    }
]

DECISION_PROMPT = """You are a research assistant with access to two retrieval tools: a vector store of internal documents and a live web search. 

Given the user's question, decide which tool to call, call it, reason over the results, and decide whether you have enough information to answer or need to retrieve again. When you have sufficient evidence, return a final answer with citations.

Do not generate an answer before you have retrieved supporting evidence."""

The termination condition is embedded in the tool design: the agent has no "answer" tool. It generates text only when it exits the tool-calling loop, which happens when the model judges its evidence base sufficient. In practice, you may want to add an explicit maximum-iteration safeguard to prevent runaway loops on malformed queries.

Enterprise demand for adaptive, iterative AI systems is a meaningful driver of agentic RAG adoption. Static pipelines are easier to debug; agentic systems are more capable on complex tasks. The tradeoff is real, and the right choice depends on the complexity distribution of your actual query workload.

Evaluating Retrieval Quality and Generation Faithfulness Before Going to Production

Evaluation cannot be deferred because non-determinism is not a theoretical concern; it is a practical one. The same system, under realistic load, can retrieve correctly for ninety percent of queries and catastrophically for ten percent, and that ten percent often includes the queries users care most about. Shipping without evaluation is not moving fast; it is moving blind.

Retrieval Metrics

Precision@k measures what fraction of the top-k retrieved chunks were actually relevant. High precision means the system is not polluting the context window with irrelevant material.

Recall@k measures what fraction of all relevant chunks in the corpus made it into the top-k results. High recall means the system is not missing evidence that should have been retrieved.

Mean Reciprocal Rank (MRR) captures how high the first relevant result appears in the ranked list. A system that consistently buries the most relevant chunk at position eight will underperform a system with lower overall recall but better ranking of its best results.

nDCG (Normalized Discounted Cumulative Gain) extends MRR by weighting highly-relevant results more than marginally-relevant ones, giving a richer picture of ranking quality across the full result set.

RAGAS context precision and context recall operationalize these concepts specifically for RAG evaluation pipelines. Context precision asks whether the retrieved context contains relevant information ranked appropriately; context recall asks whether all necessary information made it into the retrieved set. Both are computable without human annotation if you have a ground-truth question-answer dataset, which you should construct for your domain before production.

Generation Metrics

Faithfulness is the most important generation metric in a RAG system. It measures whether the generated answer stays within what the retrieved context supports, or whether the LLM introduces claims that are not grounded in the retrieved evidence. A high-faithfulness system with mediocre retrieval is preferable to a low-faithfulness system with excellent retrieval, because at least the former's errors are traceable.

Answer relevance measures whether the answer actually addresses the question asked, as distinct from being topically related to it. An answer can be faithful to retrieved context and still be irrelevant if the retrieval stage surfaced the wrong documents.

Citation coverage tracks whether claims in the generated answer are traceable to specific retrieved sources. This is the mechanism that makes the system auditable; gaps in citation coverage are early indicators of the model generating from weights rather than from evidence.

Hallucination rate, the share of generated statements not grounded in retrieved context, should drop substantially compared to ungrounded LLM output in a well-implemented RAG system. If it does not, the diagnostic usually points back to the prompt template (the model is being allowed to generate freely) or to retrieval quality (the model is not receiving sufficient grounding material).

Evaluation Tooling

RAGAS is the most widely adopted open-source framework for RAG evaluation. It implements the faithfulness, answer relevance, context precision, and context recall metrics described above, and it can operate without human-labeled ground truth by using an LLM as the judge on reference-free metrics.

LangSmith (from LangChain) provides tracing and evaluation tooling integrated with LangChain pipelines. It is particularly useful for evaluating agentic systems where the intermediate steps, not just the final output, need to be inspectable.

TruLens, developed by TruEra, frames evaluation around a "RAG triad" of context relevance, groundedness, and answer relevance, and offers both open-source and hosted evaluation options.

DeepEval provides a broader suite of LLM evaluation metrics, including several RAG-specific ones, and integrates with standard testing frameworks in a way that makes evaluation a part of a CI pipeline rather than a separate manual process.

The practical recommendation: instrument evaluation from the first prototype, not as a pre-production checkpoint. The metrics you collect on early builds inform chunking decisions, top-k settings, and prompt template design. Treating evaluation as a gate rather than a feedback loop means you are throwing away signal during the phase of development when it is most actionable.

---

Building a production RAG system is, in practice, a sequence of retrieval problems. The LLM is the easiest part to swap out; retrieval quality, augmentation design, and evaluation rigor are where the durable engineering work lives. The developers I have watched build the most reliable systems are the ones who instrument retrieval metrics before they worry about model selection, and who treat a hallucinated output as a retrieval failure to investigate rather than a model limitation to accept. That disposition, more than any particular tool choice, is what separates systems that hold up in production from ones that quietly erode user trust over weeks.