RAG vs fine-tuning: which should I use?

RAG (Retrieval-Augmented Generation) retrieves relevant documents at query time and includes them in the LLM's context. Fine-tuning modifies the LLM's weights by training on your specific data. They solve different problems.

Use RAG when: your knowledge base changes frequently, you need citations and source attribution, you want to avoid the cost and complexity of model training, or accuracy with specific documents is critical. RAG is the right choice 80% of the time for business applications.

Use fine-tuning when: you need the model to adopt a specific tone or style consistently, you're optimizing for a narrow task (classification, extraction), you need faster inference (no retrieval step), or RAG context window isn't large enough for your needs.

Cost comparison: RAG costs $0.01-$0.10 per query (embedding + retrieval + LLM call). Fine-tuning costs $50-$5,000 for training plus $0.003-$0.06 per query for inference. RAG has higher per-query cost but zero upfront investment.

Implementation complexity: RAG requires a vector database (Pinecone, Weaviate, pgvector), an embedding pipeline, and retrieval logic — typically 2-4 weeks to build. Fine-tuning requires a curated training dataset (hundreds to thousands of examples), training infrastructure, and evaluation — typically 4-8 weeks.

The practical answer: start with RAG. It's faster to implement, easier to update, and provides source attribution. Move to fine-tuning only when RAG demonstrably underperforms on your specific task.