What is RAG?
RAG, or retrieval-augmented generation, is a technique in which an AI system first retrieves relevant information from a trusted source, such as your documents, and then gives it to a language model to use when writing its answer.
Last reviewed .
How RAG works
A RAG system retrieves relevant material and adds it to the model's context. A common approach splits documents into passages and searches their embeddings in a vector database, but keyword search, SQL queries, and hybrid retrieval can also supply context.
The model then writes its answer using that material, and many systems can point to the sources they used.
Why teams use RAG
RAG lets a model answer using current or private information it was never trained on, without retraining it. It can also reduce hallucinations by grounding answers in approved content.
Updating the source documents can update the knowledge available to RAG without retraining the model, but retrieval indexes and caches may also need refreshing before answers reflect the changes.
What can go wrong
If retrieval pulls the wrong passages, or the source content is outdated, the answer will be poor. The model can also still misstate what it retrieved, so test the system and review its output.
Common questions
What does RAG stand for?
RAG stands for retrieval-augmented generation. It means the model's generated answer is supported by information retrieved just before it responds.
What's the difference between RAG and fine-tuning?
RAG supplies information to the model at the time of each request. Fine-tuning changes the model through extra training. RAG suits changing facts, and fine-tuning suits consistent style or behavior.
Does RAG eliminate hallucinations?
No. It lowers the risk by giving the model good source material, but errors can still occur, so important answers need checking.
What do you need to build a RAG system?
You need suitable source content, a retrieval method, a way to provide the retrieved information to a language model, and checks on the answer. Embeddings and a vector database are common options, but keyword, SQL, and hybrid retrieval can work too.
Want help putting this to work?
Tell us where your marketing stands today, and we'll show you what to do first.