Ask a vendor what RAG means and you'll get "retrieval-augmented generation," followed by a diagram with too many arrows. Here's the version that fits on a sticky note: RAG is an open-book exam.
A language model on its own is a very well-read stranger. It has absorbed most of the public internet, but it has never seen your price list, your policies, your contracts, or the way you like to phrase things. Ask it about your business and it does what confident strangers do: it guesses fluently.
RAG fixes that by handing the model the right pages of your book before it answers. That's the entire trick.
How it works, in four steps
- Index. Your documents get split into chunks: policies, manuals, proposals, past emails, whatever you choose to include.
- Embed. Each chunk is converted into a mathematical fingerprint that captures what it means, not just what words it contains.
- Retrieve. When someone asks a question, the system finds the chunks whose meaning is closest to it. Not keyword search — meaning search.
- Generate. The model writes an answer using those chunks, ideally citing which document each claim came from.
The result: an assistant that answers from your actual material and can show its receipts. When the answer isn't in the book, a well-built system says so instead of improvising. That single behavior is worth more than any demo.
Why not just retrain the model on our stuff?
Because your knowledge changes weekly and retraining is slow, expensive, and bad at facts. Fine-tuning teaches a model style and format; it's terrible at reliably storing knowledge you'll want to update next quarter. With RAG, updating the AI means updating the document. Save the file, done.
Where it earns money
- A staff assistant that knows every SOP, so new hires stop interrupting senior people ten times a day.
- A support bot that answers from your real warranty terms instead of vibes.
- A proposal helper that pulls from past winning quotes.
- A "what did we agree with this client?" search that actually works.
What separates good RAG from bad RAG
Three unglamorous things: how the documents are chunked (bad splits produce answers missing half the context), how fresh the index stays (stale index, stale answers), and whether permissions carry through (the intern's assistant should not quote the payroll file). None of this shows up in a sales demo. All of it shows up in month two.
That's RAG. A mouthful of a name for a simple, sturdy idea: stop asking the stranger to guess, and hand it the book.