Large language models know a lot, but they do not know your company's policies, products or latest data, and they may make things up. Retrieval-augmented generation (RAG) addresses both problems.
How RAG works
- Documents are split into small passages and converted into embeddings: numbers that capture their meaning.
- The embeddings are stored in a vector database.
- When a user asks a question, the system finds the most relevant passages.
- Those passages are given to the language model with the question, so the answer is based on real sources and can cite them.
Why companies use it
RAG is cheaper and faster to update than retraining a model, keeps answers grounded in approved content, and makes it possible to show sources.
What makes a good RAG system
- Clean, well-structured documents.
- Good passage splitting and search, often combining keyword and semantic search.
- Careful evaluation: are the answers correct and supported by the sources?
- Access controls, so people only see what they are allowed to see.
Building a small RAG assistant over your own notes is an excellent portfolio project for aspiring AI engineers.