Agent vs RAG

Plain RAG retrieves relevant context once, then generates an answer — a single, fixed pass. An agent can decide whether to retrieve, evaluate if what it found is good enough, and retrieve again — RAG becomes one possible tool an agent chooses to use, rather than an always-on step.

Simple RAG — Fixed Pipeline

Query → embed → vector search → retrieve top-k chunks → generate answer

This always runs the same way, every time, regardless of whether retrieval was actually necessary or sufficient. See RAG Pipeline.

Agentic RAG — Dynamic Decision-Making

User Query
 ↓
Agent decides: does this query actually need retrieval?
 ↓ (yes)
Search / retrieve
 ↓
Agent evaluates: is this result sufficient to answer confidently?
 ↓ (no — insufficient)
Reformulate query, search again
 ↓ (yes — sufficient)
Generate answer

This is covered in full depth in RAG Agents.

Why the Distinction Matters

Simple RAG can fail silently when: the query doesn't actually need retrieval (wasted latency/cost), or the first retrieval attempt returns weak results (poor answer, with no mechanism to try again). An agentic approach adds the judgment layer that decides when to retrieve, whether results are good enough, and when to try a different approach — at the cost of more LLM calls and added complexity.

Practical Use Case

"What's 15% of 200?" doesn't need document retrieval at all — a plain RAG pipeline would still search the vector database anyway (wasted work), while an agentic system can recognize the question is self-contained and skip retrieval entirely, answering directly.

Common Mistakes

• Assuming RAG is inherently agentic just because it involves an LLM and retrieval — plain, single-pass RAG is not agentic by the definition used in this hub
• Adding agentic decision-making to every RAG query regardless of need — extra LLM calls for simple, clearly-in-scope questions add cost and latency without meaningfully improving answer quality

Interview Relevance

"How would you improve a RAG system that sometimes retrieves irrelevant context?" — a strong answer can reference agentic retrieval: letting the system evaluate retrieved context and re-query if it's insufficient, rather than blindly generating from whatever the first search returned.

Practice Question

Design (in plain steps, not code) an agentic RAG flow for a legal-document assistant that should retrieve from three different document sets depending on the question's topic.

Related Notes

• RAG Agents
• RAG
• Query Routing