AI Agent Tool Calling SOP: Teaching Large Models to Call Tools

Large models can write poetry, reason, and translate, but ask one "how many units of SKU-8821 are in stock right now" and it will simply invent a number. Real-time business data lives outside training data, and a model's knowledge stops on the day training ended. To make an agent actually do work--query databases, call APIs, read files, place orders--you don't need a bigger context window. You need a mechanism that lets the model "call tools": function calling (OpenAI's name) or tool use (Anthropic's name).

This SOP gives you a copyable implementation template: define tool schema -> model decides -> execute -> feed results back, a four-step loop. Code is verified against official docs, current as of 2026-08-09, with APIs subject to the official source. It complements the site's MCP Server Dev SOP (building the standardized tool layer) and MCP Clients Comparison (choosing a client): those two solve "how tools are exposed in a standard way," this one solves "how the model decides to call them."

1. What Is Tool Calling: The Four-Step Loop

The essence of tool calling is teaching the model to say "I don't know, but I know who does." The mechanism breaks into four steps:

Define tool schemas: You tell the model which tools exist and what arguments each takes, described with JSON Schema.
Model decides: The user sends an instruction; the model combines the instruction with the tool list and decides whether to call a tool, which one, and what arguments to pass. Note the model only produces a "call intent"--it does not execute anything.
Execute: Your code takes the call intent the model returned and actually queries the database, calls the API, reads the file, and gets real results.
Feed results back: You inject the execution result back into the conversation; the model uses it to generate a final answer (or decides to call another tool).

Key insight: the model does not execute code. It only produces a structured "call intent." Your application code is what actually runs. This division of labor is the bedrock of tool-calling safety--the model cannot silently rm your database unless you wrote a tool that does that and told it it could.

2. Defining Tool Schemas: JSON Schema Is the Common Language

Whether OpenAI or Anthropic, the kernel of a tool definition is JSON Schema. Here is a "check inventory" tool:

json

{
  "name": "check_inventory",
  "description": "Query the current stock quantity and warehouse for a SKU. Call when the user asks about inventory, stockouts, or shipping origin.",
  "parameters": {
    "type": "object",
    "properties": {
      "sku": { "type": "string", "description": "Product SKU code, e.g. SKU-8821" },
      "warehouse": {
        "type": "string",
        "enum": ["beijing", "shanghai", "guangzhou"],
        "description": "Warehouse code; if omitted, sums across all warehouses"
      }
    },
    "required": ["sku"]
  }
}

Three rules for writing schemas:

Write the description to cover "when to call": The model decides from the description, not the name. "Query inventory" is too weak; "Call when the user asks about inventory, stockouts, or shipping origin" tells the model the trigger. The description should also state the negative boundary--for instance, "do not use this tool for historical inventory, only current"--so the model doesn't force a call when it shouldn't.
Be explicit about required: Mark which params are mandatory and which are optional. Don't mark all required, and don't mark none. For optional params, explain the default behavior in the description (e.g., "if warehouse is omitted, sums across all warehouses").
Use enum for enumerations: Restrict warehouse codes with enum instead of a free-form string--the model might invent chongqing and your system has no such warehouse. By the same logic, boolean-like params are safer as enum ["true","false"] than a bare string.

The stricter your schema, the fewer parameter hallucinations. Section 6 expands on this. Two advanced techniques: first, set additionalProperties: false to lock the parameter set so the model can't sneak in fields you never defined; second, enable strict: true (OpenAI) to force the model to emit parameters that strictly conform to the schema--under structured outputs mode the model can't even add surplus fields, which is the single most effective lever against parameter hallucinations.

3. OpenAI Function Calling Implementation

OpenAI currently has two API paths: Chat Completions (classic, still in wide use) and Responses (newer, stateful). Both share the same loop logic with slightly different shapes. The main example below uses Chat Completions--the most universal and best-documented path.

Note: The early-2023 functions / function_call (singular) parameters are deprecated. The current unified API uses tools / tool_choice (plural). Don't copy old tutorials.

typescript

import OpenAI from "openai";
const openai = new OpenAI();

// 1. Define tool schema (Chat Completions wraps it in a function layer)
const tools: OpenAI.Chat.Completions.ChatCompletionTool[] = [
  {
    type: "function",
    function: {
      name: "check_inventory",
      description: "Query the current stock quantity and warehouse for a SKU.",
      parameters: {
        type: "object",
        properties: {
          sku: { type: "string", description: "Product SKU code" },
          warehouse: { type: "string", enum: ["beijing", "shanghai", "guangzhou"] }
        },
        required: ["sku"],
        additionalProperties: false
      }
    }
  }
];

async function checkInventory(sku: string, warehouse?: string) {
  // Your real query logic: hit a database / call an ERP API
  return { sku, qty: 142, warehouse: warehouse ?? "beijing" };
}

// 2. Model decides: send the request with tools attached
const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
  { role: "system", content: "You are an inventory assistant. When info is missing, call tools to check real inventory. Never fabricate numbers." },
  { role: "user", content: "How many units of SKU-8821 are left in the Beijing warehouse?" }
];

const resp = await openai.chat.completions.create({
  model: "gpt-5.6",
  messages,
  tools,
  tool_choice: "auto" // auto | none | required | {type:"function",function:{name:"check_inventory"}}
});

// 3. Execute: the model decides to call check_inventory
const msg = resp.choices[0].message;
if (msg.tool_calls && msg.tool_calls.length > 0) {
  messages.push(msg); // feed back the assistant's tool-call intent
  for (const call of msg.tool_calls) {
    const args = JSON.parse(call.function.arguments);
    const result = await checkInventory(args.sku, args.warehouse);
    // 4. Feed result back, correlating via tool_call_id
    messages.push({
      role: "tool",
      tool_call_id: call.id,
      content: JSON.stringify(result)
    });
  }
  // 5. Model takes the result and produces a natural-language answer
  const final = await openai.chat.completions.create({ model: "gpt-5.6", messages, tools });
  console.log(final.choices[0].message.content);
  // => "There are 142 units of SKU-8821 in the Beijing warehouse."
}

Key fields: tool_choice controls model behavior--"auto" (default, model decides), "none" (forbidden), "required" (must call at least one), or {type:"function",function:{name:"..."}} to force a specific function. call.id / tool_call_id is the correlation key between a call and its result--don't drop it. Adding strict: true forces the model to emit parameters that strictly conform to the schema (structured outputs), which suppresses parameter hallucinations significantly.

The Responses API equivalent: use openai.responses.create(), where the response output contains items of type:"function_call" (with name / arguments / call_id), and you feed results back with {type:"function_call_output", call_id, output}. It carries conversation state via previous_response_id, so multi-turn flows don't require you to accumulate messages yourself.

4. Anthropic Tool Use Implementation

Anthropic's tool use shares OpenAI's logic, but four API shape differences are worth comparing:

python

import anthropic, json
client = anthropic.Anthropic()

# 1. Define tool: note input_schema (not parameters), no function wrapper
tools = [{
    "name": "check_inventory",
    "description": "Query the current stock quantity and warehouse for a SKU.",
    "input_schema": {
        "type": "object",
        "properties": {
            "sku": {"type": "string", "description": "Product SKU code"},
            "warehouse": {"type": "string", "enum": ["beijing", "shanghai", "guangzhou"]}
        },
        "required": ["sku"]
    }
}]

def check_inventory(sku, warehouse=None):
    # Your real query logic
    return {"sku": sku, "qty": 142, "warehouse": warehouse or "beijing"}

messages = [{"role": "user", "content": "How many units of SKU-8821 are left in the Beijing warehouse?"}]

# 2. Model decides
response = client.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    tools=tools,
    tool_choice={"type": "auto"},  # auto | any | tool | add disable_parallel_tool_use
    messages=messages
)

# 3. Execute: find the tool_use block in response.content
if response.stop_reason == "tool_use":
    messages.append({"role": "assistant", "content": response.content})
    tool_use = next(b for b in response.content if b.type == "tool_use")
    result = check_inventory(**tool_use.input)  # input is already a dict, no JSON.parse
    # 4. Feed back: role is "user", content is a tool_result block
    messages.append({
        "role": "user",
        "content": [{"type": "tool_result", "tool_use_id": tool_use.id, "content": json.dumps(result)}]
    })
    # 5. Model takes the result and produces a final answer
    final = client.messages.create(
        model="claude-opus-5", max_tokens=1024, tools=tools, messages=messages
    )
    print(final.content[0].text)

Four differences: (1) the schema field is input_schema (OpenAI calls it parameters); (2) tools are flat, not wrapped in function; (3) the result is fed back with role "user" and a tool_result block (OpenAI uses role:"tool" + tool_call_id); (4) tool_use.input is already a parsed dict, no JSON.parse needed. Anthropic's {"type":"any"} is equivalent to OpenAI's "required", and {"type":"tool","name":"..."} forces a specific function. Which provider to pick? If your agent mainly calls tools you wrote yourself (your database, your APIs), both work--the call loop is identical, the differences are only in field naming and result-feedback shape. If you depend on server-side tools (like Anthropic's web_search / code_execution server tools), you must follow that platform's API. Another consideration: parallel tool calls. Anthropic natively supports returning multiple tool_use blocks per turn, controlled via disable_parallel_tool_use; OpenAI Chat Completions also supports multiple tool_calls, so your execution loop must iterate with a for rather than only taking the first one.

5. The Execute-and-Feed-Back Loop: With Error Handling

In production, tools fail--database timeouts, API 403s, nonexistent SKUs. The key principle: don't swallow exceptions, don't let the agent hang, and feed the error back to the model in structured form so it decides the next step itself.

python

def run_tool(tool_use):
    try:
        result = check_inventory(**tool_use.input)
        return {"content": json.dumps(result), "is_error": False}
    except InventoryNotFound:
        # Business error: SKU doesn't exist, feed back so model asks the user
        return {"content": f"SKU {tool_use.input['sku']} does not exist, please confirm the code", "is_error": True}
    except Exception as e:
        # System error: timeout / 403, feed back so model retries or switches path
        return {"content": f"Tool execution failed: {type(e).__name__}, you may retry once", "is_error": True}

# Feed back with is_error flagged; Claude adjusts strategy accordingly
messages.append({
    "role": "user",
    "content": [{"type": "tool_result",
                 "tool_use_id": tool_use.id, **run_tool(tool_use)}]
})

Add three guardrails: max loop count (cap at 5-10 to stop the model from retrying a failing tool forever); timeouts (10-30s per tool call so one slow query can't drag down the whole agent); error fallback (after N consecutive failures, have the model fall back to "Sorry, inventory lookup is temporarily unavailable" instead of banging its head against the wall). The logic behind these guardrails is "the model is not reliable": it will repeatedly call a failing tool, micro-adjust arguments and retry, and ignore prior error context. You must draw boundaries at the application layer, or a single slow query can burn hundreds of thousands of tokens and dozens of minutes. In practice, track "tool-call history + error count" in session state and force a cutoff once the cap is hit, giving the user a clear "cannot complete right now" instead of letting it spin forever.

6. Five Pitfalls

Pitfall 1: Loose schemas cause parameter hallucinations Vague descriptions, all-string params, no enum, and the model invents arguments. The symptom: a structurally valid but semantically wrong call--like warehouse: "chongqing" when you have no such warehouse. Fix: write a clear description for every parameter, use enum for enumerations, mark required explicitly, and add strict: true where possible.

Pitfall 2: The model fabricates parameter values The model fills "that product" from the user's casual mention as sku: "that-product". Models are good at structuring, bad at guessing real IDs. Fix: write "ask the user for the SKU if missing, do not guess" in the system prompt; or add a search_product fuzzy-search tool so the model searches first instead of inventing a SKU.

Pitfall 3: Unhandled tool-call failures hang the agent The model calls a tool, your code throws and doesn't feed back a result, and the next request is missing the tool_result, so the API errors out. This is the most common production incident. Fix: wrap all tool execution in try/except, and feed back an is_error: true result even on failure so the model decides whether to retry or pivot.

Pitfall 4: No timeouts or loop caps The model keeps calling a slow tool or gets stuck in a "fail-retry-fail" loop, burning tokens and time. Fix: add a timeout to every tool, and cap the agent loop at a max number of turns (5-10 recommended); on overflow, force a fallback answer.

Pitfall 5: Overly broad tools cause security issues Give the model an execute_sql(query) tool and it may craft DROP TABLE. Finer granularity is safer--use check_inventory(sku, warehouse) rather than run_any_sql(sql). For write operations (placing orders, refunds, deletes), always add permission checks and a confirmation step in the tool layer--don't let the model refund money on a single sentence.

7. FAQ

Q1: Are function calling and tool use the same thing? Yes. OpenAI originally called it function calling, later renamed it tool calling / tools; Anthropic has always called it tool use. The underlying logic is identical: the model produces a structured call intent, the app executes it and feeds the result back. The API field names differ, the loop is the same.

Q2: Does the model write and run code itself? No. The model only produces the "which tool, what arguments" intent (a JSON blob). Your application code is what actually runs. The model has no access to your database password and cannot run a shell on your machine--unless you explicitly provide a tool that does so.

Q3: Should I use OpenAI's Responses API or Chat Completions? Chat Completions is more universal, better documented, and broadly framework-compatible--suitable for most scenarios. The Responses API carries conversation state (previous_response_id) and natively supports multi-turn and background mode, suited to complex agent orchestration. New projects can prefer Responses; existing projects are fine continuing on Chat Completions. Practical decision factors: whether you need server-side conversation state (Responses supports it natively; Chat Completions requires you to accumulate messages yourself); whether your framework (LangChain / LlamaIndex, etc.) already supports Responses; and which API your team knows better. Don't migrate just because it's "new"--Chat Completions is not a deprecated API, and it remains the primary path for tool calling across the industry.

Q4: How do I make the model call more accurately with fewer hallucinations? Three techniques: (1) strict schemas--descriptions that specify the trigger, params with enum and descriptions, strict: true enabled; (2) system-prompt constraints--"ask the user if info is missing, don't guess parameters," "only call the necessary tools"; (3) less is more--don't pile dozens of tools into the list, as the more pressure to choose, the more wrong choices; use on-demand loading (OpenAI's tool_search) to surface tools in batches.

Q5: What's the relationship with MCP? They're complementary. MCP is the "standardized tool layer"--it defines how tools are discovered and transported so one tool can be called by any client (see MCP Server Dev SOP). Function calling / tool use is the "model-side decision mechanism"--it defines how the model decides which tool to call. An agent typically uses both: MCP exposes tools in a standard way, and the model's function-calling mechanism decides when and which to call. Model-side decision + standardized tool layer together make a complete agent.

Reusable System Prompt Template

text

You are {role}, and can call the following tools to complete tasks: {tool list}

Calling discipline:
1. When a required parameter is missing, ask the user first. Never guess or fabricate parameter values.
2. Call only the minimum set of tools the task requires. No redundant calls.
3. When a tool returns an error (is_error), explain the problem to the user and suggest a next step. Do not repeatedly retry the same call.
4. Tool-returned data is the single source of truth. Do not override tool results with training knowledge.
5. For write operations (order/refund/delete), confirm with the user before calling.

Available tools:
- check_inventory(sku, warehouse?): Check inventory
- search_product(keyword): Fuzzy search products
- create_order(sku, qty, address): Place order (requires user confirmation)

References

OpenAI Function Calling guide (tools parameter, tool_choice, tool_calls, strict mode): https://platform.openai.com/docs/guides/function-calling
OpenAI Developers API docs (Responses API: function_call / function_call_output / call_id): https://developers.openai.com/api/docs/guides/function-calling
Anthropic Tool Use docs (input_schema, tool_use block, tool_result feedback): https://docs.anthropic.com/en/docs/build-with-claude/tool-use/overview
Model Context Protocol official site (standardized tool layer, complementary to this article's model-side mechanism): https://modelcontextprotocol.io
AI Cake: MCP Server Dev SOP: https://aiwebcool.com/en/mcp-server-dev-sop
AI Cake: MCP Clients Comparison: https://aiwebcool.com/en/mcp-clients-comparison-review