What Is RAG and How Chatbots Use It

The Problem RAG Solves

AI models like GPT and Claude are trained on vast amounts of internet text, but they do not know anything specific about your business. They do not know your pricing, your return policy, your product specifications, or your operating hours. Without RAG, a chatbot would either make up answers based on general knowledge or simply say "I don't know" to every business-specific question.

RAG bridges this gap. You upload your own documents (product manuals, FAQ pages, policy documents, training materials), and the system makes them searchable. When a visitor asks a question, the chatbot retrieves the most relevant sections from your content and uses them as the factual basis for its response. The AI model's language ability generates a natural answer, while your documents provide the facts.

How RAG Works Step by Step

1. Document Ingestion

When you upload a document, the system breaks it into chunks of roughly 200 to 500 words each. This chunking is important because the AI model works best when given focused, relevant passages rather than entire documents. A 20-page manual might produce 40 to 80 chunks. See How to Chunk Documents for Better AI Understanding for details on how chunking affects quality.

2. Embedding Creation

Each chunk is converted into a vector embedding, a mathematical representation of the text's meaning. Two passages about the same topic will have similar embeddings even if they use different words. These embeddings are stored in a searchable database at a cost of 3 credits per chunk.

3. Query Processing

When a visitor sends a message, their question is also converted into an embedding using the same process. The system then compares the question's embedding against all stored chunk embeddings to find the most similar ones. This is called vector search or semantic search, and it finds relevant content based on meaning rather than exact keyword matching.

4. Context Assembly

The top matching chunks (typically 3 to 5) are retrieved and added to the prompt that gets sent to the AI model. The final prompt looks something like: system instructions + retrieved document chunks + conversation history + the user's question. The AI model reads all of this and generates a response that draws from your specific documents.

5. Response Generation

The AI model composes a natural-language answer using the retrieved information. Because it has the actual text from your documents, it can cite specific details, prices, steps, and policies rather than guessing. The response feels conversational because the AI model handles language fluency, while the accuracy comes from your actual content.

Why RAG Is Better Than Alternatives

RAG vs Putting Everything in the System Prompt

You could paste your entire FAQ into the chatbot's system prompt, but this hits limits fast. System prompts work for small amounts of static information (a few hundred words), but a serious knowledge base might contain thousands of pages. RAG lets you store unlimited content and retrieves only the relevant portions per question, keeping each request efficient and focused.

RAG vs Fine-Tuning

Fine-tuning modifies the AI model itself to know your information. It is expensive, time-consuming, and hard to update. When your pricing changes, you would need to retrain the model. With RAG, you just update the document and re-embed it, the chatbot starts using the new information immediately. For nearly all business chatbot use cases, RAG is the practical choice. See How Is Training Different From Fine-Tuning for a deeper comparison.

What Makes RAG Work Well

• Quality source documents: The chatbot can only be as good as the content you give it. Well-written, accurate, comprehensive documents produce better answers.
• Proper chunking: Chunks should be large enough to contain complete thoughts but small enough to be specific. A chunk that covers one topic retrieves better than one that covers three.
• Sufficient coverage: If a topic is not in your documents, the chatbot cannot answer questions about it. Upload everything you want the chatbot to know about.
• Regular updates: When information changes, update the corresponding documents so the chatbot stays current.

RAG on This Platform

The platform handles the entire RAG pipeline automatically. You upload documents through the admin panel or use the document upload feature, and the system handles chunking, embedding, storage, and retrieval behind the scenes. When a visitor asks a question, the chatbot searches your embeddings, retrieves the best matches, and generates a response. You do not need to configure vector databases, write search queries, or manage any infrastructure.

Embedding storage costs 3 credits per chunk (a one-time cost when you upload), and retrieval is included in the per-message cost of the chatbot response. A typical business knowledge base of 50 to 100 pages costs a few hundred credits to embed initially, then nothing additional for retrieval.