Tool Calling

Definition

Tool Calling is the mechanism by which an LLM selects and invokes external functions with structured arguments during inference, based on JSON schemas provided in the API request. When tools are defined in the API call, the model can respond not with text but with a structured tool call object specifying which function to invoke and with what parameters. The calling application then executes the actual function and returns the result back to the model as a tool result message. Tool calling is the foundation of agentic behavior—it is the mechanism that connects language model reasoning to real-world computation.

Engineering Context

Tool calling (OpenAI's term; Anthropic calls it "tool use") is a core primitive for building agents. The model returns a structured tool call object (function name + arguments as JSON) rather than text. The application executes the function and returns results. Reliability requires: strict schema definition with clear descriptions (the model uses descriptions to decide when to call each tool), validation of model-generated arguments before execution, idempotency for side-effecting tools (so retries are safe), and structured error feedback so the model can handle failures gracefully. Parallel tool calling (multiple tool calls in a single model response) can significantly reduce latency for tasks that require multiple independent lookups.

Related Terms