An observation is the result of an action, fed back to the LLM so it can decide what to do next. How observations are formatted and filtered directly affects both cost and decision quality.
Observation Flow
Action: get_order_status(order_id="4521")
Raw result: {"order_id": "4521", "status": "shipped",
"carrier": "BlueDart", "tracking": "BD9284...",
"items": [...], "warehouse_id": "WH-12",
"internal_notes": "customer called twice"}
Observation added to context (filtered/formatted):
"Order 4521 status: shipped via BlueDart, tracking BD9284..."Note what was left out: internal warehouse IDs and internal notes weren't relevant to the agent's task and don't need to consume context budget or risk being echoed back to an end user inappropriately.
Formatting Considerations
| Consideration | Why It Matters |
|---|---|
| Size | Large raw results (long documents, big JSON payloads) can consume excessive context — consider summarizing or extracting only relevant fields |
| Structure | Consistent, predictable formatting helps the model parse results reliably across many different tool types |
| Sensitive data | Internal-only fields shouldn't be passed into context that might get echoed into a user-facing response |
| Errors | Tool failures need to be observable too — see Tool Failures — silently dropping a failed call's result leaves the agent unaware anything went wrong |
Practical Use Case
When a search tool returns 20 results but only the top 3 are relevant, formatting the observation to include just those 3 (with a note that more exist) keeps context lean while still letting the agent request more if genuinely needed.
Common Mistakes
- Passing raw, unfiltered API responses directly into context — wastes context budget and can leak internal fields into a user-facing response
- Not surfacing tool errors as explicit observations — the agent needs to see that something failed to reason about a retry or a different approach
- Inconsistent observation formatting across different tools, making it harder for the model to reliably parse results
Interview Relevance
"How would you handle a tool that returns a very large result?" — summarization, field filtering, and pagination-style follow-up requests are the expected kinds of answers.
Practice Question
A weather-lookup tool returns 15 fields of raw data. Design a filtered observation format that includes only what's relevant to answering "should I carry an umbrella today?"
