The prompt template used in the final generation step of RAG has a real, measurable impact on answer quality — clear instructions about using the context, handling missing information, and citation expectations meaningfully reduce hallucination and improve trustworthiness.
A Solid Baseline RAG Prompt
SYSTEM_PROMPT = """
You are a support assistant. Answer the user's question using
ONLY the context provided below. Do not use any outside
knowledge.
If the context does not contain enough information to answer
the question, respond exactly: "I don't have enough information
to answer that."
Cite the source of your answer using the [Source: X] labels
in the context.
"""
USER_PROMPT_TEMPLATE = """
Context:
{context}
Question: {question}
"""Why Each Piece Matters
| Instruction | What It Prevents |
|---|---|
| "Use ONLY the context provided" | The model blending in its own (possibly outdated or wrong) memorized knowledge instead of the grounded source |
| Explicit "don't know" response format | Fabricated answers when context doesn't actually cover the question — a major hallucination-reduction technique |
| Citation instruction | Answers with no traceable source, making trust and verification harder |
Before/After Example
Weak: "Here's some info: {context}. Answer: {question}"
→ vague, no explicit grounding requirement, no fallback
instruction for missing information
Better: the full template above
→ explicit grounding, explicit fallback, explicit citation
expectationPractical Use Case
This prompt-level discipline is often the single highest-leverage, lowest-effort improvement available for reducing RAG hallucination — before reaching for more complex fixes (fine-tuning, more sophisticated retrieval), a well-designed RAG prompt is worth getting right first.
Common Mistakes
- Not explicitly instructing the model to prioritize context over its own knowledge — without this, it may blend in outdated or incorrect memorized information
- No fallback instruction for insufficient context, leading to fabricated answers instead of an honest "I don't know"
- Requesting citations without providing labeled sources in the context for the model to actually cite
Interview Relevance
"What specific prompt instructions reduce hallucination in a RAG system?" — grounding instructions ("use only the context"), an explicit fallback for missing information, and citation requirements are the expected concrete answer.
Practice Question
Rewrite this weak RAG prompt to reduce hallucination risk: "Use this info to answer: {context}. Question: {question}"
