Quick answer: Retrieval augmented generation (RAG) is a technique that lets an AI model look up relevant documents before it answers, then write its response from what it found rather than from memory alone. It is how a chatbot can cite sources and answer questions about your company’s handbook or this week’s news.
AI chatbots quietly changed over the last couple of years, and you probably noticed without naming it. Little numbered citations appeared under the answers. Links to sources. Lines like “according to your company handbook.”
The model underneath did not suddenly learn more. Something was added in front of it: a search step that finds the relevant document and hands it to the generative AI before it writes a word. That technique is RAG.
This page explains RAG in plain language, follows one real question through a RAG system, names where you have already used it, and is honest about where it still gets things wrong.
No math, no code. If you can picture a student with an open textbook, you already have the hard part.
What Is RAG in Plain Language?
Picture two students in the same exam. One works from memory, closed book: fluent, confident, and occasionally inventing a date that sounds right. The other is handed the textbook, open to the right page, before writing a line.
A standard large language model is the first student. RAG is the second. Everything else is plumbing.
The name describes the sequence. Retrieval means finding text that relates to the question. Augmented means adding that text to the question. Generation means writing the answer out of it.
What does not change matters just as much: the model’s training stays exactly as it was. What changes is what it is looking at when it answers, which is why a RAG system can learn your company’s new refund policy as fast as someone uploads the PDF.
RAG is not a model you can download. It is a technique, a way of wiring a search step onto a language model you already have.
The term comes from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, University College London, and NYU. Lewis has since said the acronym was rushed: “We always planned to have a nicer sounding name.”
If you remember one thing from this page, remember the open book.
How Does RAG Work, Step by Step?
One worked example is enough to understand every RAG system you will ever meet. Here it is.
A customer types into a retailer’s support chat: “Can I return a jacket I bought six weeks ago?”
The question becomes a search. The chatbot does not answer yet. First it converts the question into a form it can match against stored documents, using the meaning of the words rather than the words themselves. More on how that matching works in the next section.
Retrieval. It searches the retailer’s own help-center articles and policy PDFs and pulls back the handful of passages that look most relevant, including the paragraph of the returns policy that covers seasonal outerwear.
Augmentation. The system pastes those passages into the prompt alongside the original question. This is the “augmented” in the name, and it is the step the customer never sees.
Generation. The model writes its answer out of the open book instead of out of memory, and it can point back at the policy page it used.
Without the retrieval step, the same chatbot would have produced something fluent and generic about 30-day windows and original packaging, assembled from the thousands of return policies in its training data and possibly matching this retailer’s policy by accident. In one academic comparison, a model with retrieval scored 4.12 out of 5 for correctness against 2.18 for the same model without it.
One answer cites the policy. The other sounds like it did.
What Is a Vector Embedding?
Search a music app for “songs for a rainy Sunday morning” and you get results containing none of those words. Nothing in the system knows what rain has to do with music. It is embeddings doing the work.
An embedding turns a piece of text into a long list of numbers that stands for its meaning, so a computer can measure how close two pieces of text are. Think of it as a map reference for meaning. “Cat,” “dog,” and “lion” land in one neighborhood. “Car,” “truck,” and “vehicle” land somewhere else.
The models that produce these numbers are usually transformer models, the same family behind the chatbots themselves.
This is why retrieval works on meaning rather than exact keywords, and why a customer asking about “sending a jacket back” finds a policy paragraph that only says “returns.”
That mental picture is all you need. The technical detail underneath is optional.
What Are Some Real Examples of RAG?
Explainers about RAG have a habit of stopping at the abstraction. Here are four places you have probably met one this month.
AI Search Engines That Show Their Sources
Perplexity runs a live web search for every query and attaches numbered citations mapped to the exact passage behind each claim. Microsoft Copilot grounds its answers in Bing’s index, so a page must be indexed and ranking there before Copilot can quote it. Those little source links under an AI answer are the visible evidence of a retrieval step.
Support Chatbots That Actually Know the Policy
DoorDash’s delivery-support assistant condenses the conversation so far, then retrieves knowledge-base articles and past support cases before it replies. LinkedIn built something similar on top of a knowledge graph of historical support tickets and cut median resolution time per issue by 28.6%. The difference between a helpful support bot and an infuriating one is usually retrieval quality, not the model.
Assistants That Answer From Your Employer’s Own Documents
“How many sick days do I get in Singapore?” is unanswerable from general training data and trivial once the right handbook section has been retrieved, with a link to the source PDF. Royal Bank of Canada’s internal system, Arcane, finds banking policy scattered across web platforms, PDFs, and spreadsheets, then answers with attribution back to the document it used.
Specialist and Media Tools
Vimeo turns video transcripts into a searchable index, so a question about a two-hour recording returns the relevant moment plus a written answer. In medicine the stakes are higher: one study found a model answering without retrieval failed to work out that the question was about a specific pancreatic cancer at all, while retrieval anchored it to the right condition first.
The pattern behind all four: RAG turns up wherever the useful information lives in a specific document rather than in the model’s general knowledge.
Why Is RAG Important?
Changing what a RAG system knows does not involve touching the AI at all. You add a document. Retraining a large language model costs money, time, and specialist staff. Dropping a new PDF into a knowledge base costs an afternoon.
That single property explains most of the appeal:
- Answers stay current. The model’s weights are fixed, but the knowledge base is not, so new information is usable the moment it is added.
- Claims become checkable. Because the answer is built from retrieved passages, the system can cite them, like a footnote you can follow.
- Sensitive data stays where it belongs. Proprietary information sits in an access-controlled database instead of being baked permanently into the model, so normal permissions still apply.
The augmentation step is prompt engineering happening automatically: the retrieved passages are written into the prompt before you ever see it.
How Is RAG Different From Fine-Tuning?
Fine-tuning changes how a model behaves. RAG changes what it has in front of it. Coaching the student is fine-tuning; handing them the open book is RAG. They are complements, not rivals, and plenty of systems use both. But if the model does not know your facts, more coaching will not fix it.
What Are the Limits of RAG?
The fix for AI making things up can produce a brand new kind of wrong answer. The open book can be open to the wrong page. If the retriever pulls an outdated or unrelated document, the model writes a fluent, authoritative answer out of it anyway, and nothing in the wording signals that anything went wrong. A medical chatbot retrieving a superseded study will recommend the superseded treatment, confidently, with a citation attached. RAG reduces hallucination. It does not abolish it.
- Retrieval is a guess, not a lookup. Similarity search finds passages that are likely relevant, a poor fit when you need an exact figure such as a price or a policy number.
- The common failure is silence, not invention. In production systems the document that would have answered the question often never reaches the model.
- Quality depends on unglamorous prep. How documents are split up and cleaned before indexing drives answer quality more than the choice of model.
So treat a citation as an invitation, not a verdict. If the answer matters, open the source it cites.
Frequently Asked Questions About RAG
Is RAG a type of AI?
No. RAG is a technique that pairs a search step with an existing language model, not a model in its own right. You cannot download it; you build it around a model you already have.
Does RAG replace search engines?
No, it usually depends on one. Many RAG systems expand your question into related queries, send them to a search index such as Bing or Google, and feed the results into the answer. The search engine does the finding; the model does the writing.
Why does RAG still give wrong answers sometimes?
Because its accuracy depends on what the retriever finds. An ambiguous question or a stale document in the knowledge base produces a confident answer built on the wrong source. See “What Are the Limits of RAG?” above for the full picture.
Do I need to understand RAG to use AI tools?
No. Knowing it exists is enough, and it explains why some AI answers arrive with source links and others do not, and why a work assistant can quote your handbook while a general chatbot cannot.
