What is RAG (Retrieval-Augmented Generation)?
Technique where the agent fetches relevant documents from a vector store or search index before answering, then grounds its response in them. Reduces hallucination and lets agents work over large corpora the LLM never saw during training.
Where the idea comes from
Retrieval-augmented generation was introduced in a 2020 paper by Patrick Lewis and colleagues, accepted at NeurIPS 2020. It pairs a pretrained language model, which holds knowledge in its parameters, with an external searchable memory: in the paper, a dense index of Wikipedia queried by a pretrained retriever.
What it does in practice
Instead of relying only on what the model memorised in training, the system fetches passages relevant to the question and gives them to the model while it writes the answer. The paper reports this improves performance on knowledge-intensive tasks. Today the usual recipe is to embed your documents, store the vectors, embed each question, fetch the nearest passages and include them in the prompt.
Limits to know
A model can only use what retrieval returns, so poor chunking, a weak embedding model or a missed document produces a confident answer from the wrong passages. Retrieved text also counts toward the context window and is untrusted input, so it is a route for prompt injection.
Example
A support agent embeds your help-centre articles. When a customer asks about refunds, the three closest articles are retrieved and placed in the prompt, and the model answers from them and cites which article it used.
Sources
See also
← Back to the full AI agent glossary.