How Tool Calling Works

A closer, more technical look at each step of the tool calling flow — what the model actually receives, what it produces, and what your application code is responsible for at each stage.

Step 1 — Your Application Describes Available Tools

tools = [
    {
        "name": "get_order_status",
        "description": "Get the current shipping status of an order",
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string", "description": "The order ID, e.g. '4521'"}
            },
            "required": ["order_id"]
        }
    }
]
# This schema is sent to the model ALONGSIDE the conversation —
# see Tool Schema for schema design details

Step 2 — The Model Decides Whether to Call a Tool

response = llm_client.generate(
    messages=[{"role": "user", "content": "Where's my order 4521?"}],
    tools=tools
)

if response.tool_calls:
    # the model decided it needs a tool
    tool_call = response.tool_calls[0]
    print(tool_call.name)       # "get_order_status"
    print(tool_call.arguments)  # {"order_id": "4521"}
else:
    # the model answered directly, no tool needed
    print(response.text)

Not every user message requires a tool call — a well-implemented model only requests one when genuinely needed for the task (see Tool Selection).

Step 3 — Your Application Validates and Executes

if tool_call.name == "get_order_status":
    order_id = tool_call.arguments.get("order_id")
    if not is_valid_order_id(order_id):
        result = {"error": "Invalid order ID format"}
    else:
        result = get_order_status(order_id)  # your real function

Step 4 — The Result Goes Back to the Model

messages.append({"role": "assistant", "tool_calls": [tool_call]})
messages.append({"role": "tool", "content": json.dumps(result)})

final_response = llm_client.generate(messages=messages, tools=tools)
print(final_response.text)  # the natural-language answer to the user

This is a second LLM call — the model needs to see the tool's result before it can generate a coherent final answer incorporating it.

Practical Use Case

Understanding this as (at least) two separate LLM calls — one to decide on the tool call, one to generate the final answer after seeing the result — matters for latency and cost estimation; a tool-calling interaction is never a single round trip.

Common Mistakes

• Forgetting the second LLM call after tool execution, and trying to construct the final user-facing response manually instead of letting the model incorporate the result naturally
• Not handling the case where the model requests a tool that doesn't exist or with malformed arguments

Interview Relevance

"How many LLM API calls does a single tool-calling interaction typically require?" — at least two: one where the model decides to call a tool, and a second after the tool result is returned, to generate the final response.

Practice Question

Write the message history structure (roles and content) for a complete tool-calling exchange: user question, model's tool call decision, tool result, final answer.

Related Notes

• Tool Schema
• Tool Result
• Tool Selection