Getting reliable JSON out of an LLM starts with clear prompting — explicit format instructions, a shown example, and (where the provider supports it) a dedicated JSON output mode rather than relying on instructions alone.
Prompting for JSON — Before/After
Weak:
"Give me the order info as JSON."
Better:
"Return ONLY valid JSON matching this exact structure, with no
additional text before or after:
{
\"order_id\": string,
\"status\": string,
\"total\": number
}
Order: [order details]"Showing the exact expected structure (not just saying "as JSON") substantially improves output consistency — the model has a concrete pattern to match rather than inferring structure from a vague instruction.
A Common Failure: Extra Text Around the JSON
Model output (unwanted):
"Sure! Here's the JSON you requested:
{"order_id": "4521", "status": "shipped", "total": 1025}
Let me know if you need anything else!"Explicitly instructing "no additional text, only the JSON object" reduces this, though isn't always perfectly reliable through prompting alone — see Structured Generation for provider features that enforce this more strictly.
Parsing Defensively
import json
def safe_parse_json(raw_output):
# Strip common wrapping patterns before attempting to parse
cleaned = raw_output.strip()
if cleaned.startswith("```json"):
cleaned = cleaned.removeprefix("```json").removesuffix("```").strip()
try:
return json.loads(cleaned)
except json.JSONDecodeError:
return None # caller must handle this explicitly — never
# assume parsing always succeedsPractical Use Case
Any feature extracting structured data from unstructured text (support tickets, resumes, invoices) relies on this pattern — clear format instructions, a shown example, and defensive parsing that treats a malformed response as an expected, handled case rather than a rare exception.
Common Mistakes
- Assuming the model will always return clean JSON with no wrapping text or formatting artifacts, and not handling the case where it doesn't
- Not showing the exact expected structure, relying on a vague "return as JSON" instruction
- Using regex to try to extract JSON from free text instead of proper parsing with clear error handling
Interview Relevance
"How would you reliably get JSON output from an LLM in production?" — explicit structure in the prompt, defensive parsing, and validation are the three pillars of a solid answer.
Practice Question
Write a prompt (with shown structure) for extracting name, email, and requested action from a customer email, and the defensive parsing code to handle the response.
