What is tool calling (function calling)? When you use it, does the model run your functions itself?
Suggested answer
Tool calling is how an LLM gets to use data and actions outside itself. The most important point to make is that the model never runs any function itself: it only writes a request, and your code decides whether to run it.
One tool call goes through four steps:
- The application sends a request with a list of tools. Each tool has a name, a description (including when it should be used) and a JSON Schema for its input.
- If it needs a tool, the model returns a
tool_useblock (the tool name and arguments) withstop_reasonset totool_use, instead of an answer. - The application runs the real function, for example reading order 1041 from the database, and sends the result back in a
tool_resultblock with the sameid. - The model reads the result and writes the final answer.
It's worth adding a few points that show you've actually built one:
- Input written by the model is untrusted data, and you check it the way you'd check data a user submits (format, permissions). The schema only guarantees the types, not that the values make sense.
- Tool errors should go back to the model (for example
is_error: truewith "there is no order 1099") so it can ask the customer again, instead of crashing the whole application. - Every tool use costs an extra API call, and the tool definitions count as input tokens in every request.
- When a question needs several steps (find the order number by phone number, then look up its status), the application has to keep looping until the model answers in text. That's the agent loop.