What is a Retrieval-Augmented Generation (RAG)?

An LLM's training data has a cutoff date and doesn't include an organization's private documents, so asking it a question about current events or internal company knowledge without help will produce either a stale answer or a fabricated one. RAG addresses this by running a retrieval step first — usually a vector similarity search over an embedded document store, though keyword search and hybrid approaches are also used — to pull the most relevant passages for the question, then feeding those passages into the model's prompt alongside the question itself. The model's job shifts from "recall an answer" to "synthesize an answer from the provided evidence," which sharply reduces hallucination when the retrieval step surfaces the right material. RAG's quality depends heavily on the retrieval half of the pipeline, not just the generation half: poor chunking, stale embeddings, or a retriever that misses the actually-relevant document will produce a confidently wrong answer even from a strong model, because the model is faithfully summarizing bad or incomplete evidence. Production RAG systems invest as much in retrieval quality — re-ranking, hybrid search, freshness — as in prompt design. RAG is distinct from fine-tuning: fine-tuning bakes knowledge into model weights permanently, while RAG keeps knowledge external and updatable, which is why it's the dominant pattern for keeping an AI system current against a changing knowledge base.