Choosing which model to call — not just which provider — is a real, ongoing engineering decision, balancing capability, cost, and latency against what a specific task actually requires.
A Practical Selection Framework
| Question | Implication |
|---|---|
| How complex/ambiguous is the task? | Complex reasoning → favor a more capable model; simple/narrow tasks → a smaller model is often sufficient (see Model Size) |
| What's the request volume? | High-volume features are far more cost-sensitive per-request than low-volume ones |
| What's the latency requirement? | Real-time, interactive features need faster (often smaller) models; batch/offline processing can tolerate slower, larger ones |
| Does the task need a specific capability? | Some models support features others don't (e.g. certain structured-output modes, specific context window sizes, multimodal input) — check current provider docs |
Model Routing — Using Different Models for Different Requests
# Conceptual pattern
def select_model(task_type, complexity_estimate):
if task_type == "simple_classification":
return "small-fast-model"
elif complexity_estimate == "high":
return "flagship-model"
else:
return "balanced-model"
model = select_model(task_type="ticket_summary", complexity_estimate="low")
response = llm_client.generate(model=model, ...)Many production systems route different request types to different models rather than using one model for everything — a genuine cost/latency optimization once volume is significant.
Practical Use Case
A support platform might use a smaller model for initial ticket categorization (high volume, low complexity) and a larger model only for drafting the actual nuanced response (lower volume per ticket, higher quality bar) — matching model cost to task value.
Common Mistakes
- Defaulting to the newest/most capable model for every request type without testing whether a cheaper model performs adequately for that specific task
- Never revisiting model choice as new model versions/options become available — the right choice can change as the landscape evolves
- Switching models purely for cost without re-evaluating output quality against your actual use case first
Interview Relevance
"How would you decide which model to use for a new feature?" — task complexity, volume, latency requirements, and required capabilities, tested empirically rather than assumed, form a strong answer.
Practice Question
Design a simple model-routing rule (in plain logic) for an application with three request types: quick FAQ answers, detailed technical troubleshooting, and bulk email categorization.
