Tool calling

Every tool-capable model on INFRO uses the same OpenAI tools and tool_choice schema. Define tools once; INFRO translates requests and responses to and from each provider's native format — Anthropic tool_use blocks, Gemini function declarations, DeepSeek's variant — so the shapes on this page work identically across the catalog.

Tool calling is a loop: send tool definitions with a chat completion, the model responds with one or more tool_calls instead of text, you execute them and return the results, and the model produces its final answer. This page walks the full loop, then covers tool_choice, parallel calls, and streaming.

Check model support

A model accepts the tools parameter when its capabilities array includes "tools". List the catalog with GET /v1/models:

curl https://api.infro.io/v1/models \
  -H "Authorization: Bearer $INFRO_API_KEY"

Most current-generation models support tools, including every model on this page. Check at integration time rather than assuming — capability sets differ between a vendor's flagship and its small models.

Define a tool

Tools go in the top-level tools array. Each entry describes one function the model may call; arguments are specified as JSON Schema.

typestringrequired

Always "function".

function.namestringrequired

The name the model uses to call the tool. Letters, numbers, underscores, and dashes; 64 characters max.

function.descriptionstring

What the tool does and when to use it. The model leans on this heavily when deciding whether to call — write it for the model, not for humans.

function.parametersobject

JSON Schema for the arguments. Keep it flat where you can, and use enum and per-property description fields — they measurably improve argument quality.

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get the current weather for a city. Use when the user asks about weather conditions.",
    "parameters": {
      "type": "object",
      "properties": {
        "city": {
          "type": "string",
          "description": "City name, e.g. \"Lisbon\""
        },
        "unit": {
          "type": "string",
          "enum": ["celsius", "fahrenheit"],
          "description": "Temperature unit. Defaults to celsius."
        }
      },
      "required": ["city"]
    }
  }
}

Tool definitions are part of the prompt prefix, so keep the tools array byte-stable across requests — system prompt first, then tools, then history — and provider-side prompt caching can keep hitting.

The tool-calling loop

When the model decides to call a tool, the response has finish_reason "tool_calls" and the assistant message carries a tool_calls array instead of (or alongside) text content. Each entry:

idstring

Unique id for this call. Echo it back as tool_call_id on the result message.

typestring

Always "function".

function.namestring

Which tool to run.

function.argumentsstring

JSON-encoded arguments — a string, not an object. It is model-generated: parse defensively and validate before executing anything with side effects.

Sending results back has three rules: append the assistant message exactly as received (it carries the tool_calls your results answer), append one "tool" role message per call with tool_call_id set to the call's id and content set to the result as a string, then request the next completion with the same tools array. The complete loop:

import os
import json
from openai import OpenAI

client = OpenAI(
    base_url="https://api.infro.io/v1",
    api_key=os.environ["INFRO_API_KEY"],
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["city"],
            },
        },
    }
]

def get_weather(city: str, unit: str = "celsius") -> str:
    # Replace with a real lookup
    return json.dumps({"city": city, "temp": 21, "unit": unit, "conditions": "clear"})

messages = [{"role": "user", "content": "What's the weather in Lisbon right now?"}]

response = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=messages,
    tools=tools,
)
message = response.choices[0].message

while message.tool_calls:
    # 1. Append the assistant turn that requested the calls
    messages.append(message)

    # 2. Execute every call, one tool message per tool_call_id
    for tool_call in message.tool_calls:
        if tool_call.function.name == "get_weather":
            args = json.loads(tool_call.function.arguments)
            result = get_weather(**args)
        messages.append({
            "role": "tool",
            "tool_call_id": tool_call.id,
            "content": result,
        })

    # 3. Ask for the next turn with the same tools
    response = client.chat.completions.create(
        model="anthropic/claude-sonnet-5",
        messages=messages,
        tools=tools,
    )
    message = response.choices[0].message

print(message.content)

The loop runs until the model answers in plain text (finish_reason "stop"); multi-step tasks commonly take two or three iterations — the model calls a tool, reads the result, and calls another. If you use fallbacks, pick fallback models that also have the tools capability — one that can't call tools answers in plain text instead — and read the response model field to see which model actually served each turn.

tool_choice

tool_choice controls whether the model may, must, or must not call tools. INFRO maps each value to the provider's equivalent, so all four forms work on any model with the tools capability.

Forcing a tool guarantees a call, not good arguments — a model pushed into a tool it wouldn't have chosen will invent parameters it doesn't have. Validate arguments before executing, especially with "required" or a forced function.

Parallel tool calls

Where the underlying model supports it, one assistant turn can carry several entries in tool_calls — independent lookups batched into a single round trip. Execute them all (concurrently if you like) and append one "tool" message per call. Order doesn't matter; results are matched by tool_call_id, not position. The loop above already handles this — its for loop over message.tool_calls returns every result before the next request.

Streaming tool calls

With stream: true, tool calls arrive incrementally in delta.tool_calls. Fragments are keyed by index within the turn — the first fragment of each call carries the id, type, and function.name; subsequent fragments append pieces of the function.arguments string. Concatenate arguments per index until the chunk with finish_reason "tool_calls", then parse.

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"id":"call_9f2b","type":"function","function":{"name":"get_weather","arguments":""}}]},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"{\"city\":\"Lis"}}]},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{"tool_calls":[{"index":0,"function":{"arguments":"bon\"}"}}]},"finish_reason":null}]}

data: {"choices":[{"index":0,"delta":{},"finish_reason":"tool_calls"}]}

data: [DONE]

Parallel calls interleave in the same stream, distinguished by index. The SSE format, usage accounting via stream_options, and mid-stream failure behavior are covered in streaming.

Which models to use

Tool-use quality varies more across models than plain-text quality does, and it shifts between versions — treat this as a starting point and benchmark on your own tools. In our experience, anthropic/claude-sonnet-5 and anthropic/claude-opus-5 are consistently strong on long multi-step chains and at recovering from bad tool results; openai/gpt-5.1 is similarly dependable with tight schema adherence; and moonshot/kimi-k2 stands out among open-weight models for agentic tool use at a much lower price.

For high-volume, single-call workloads — one lookup, one answer — openai/gpt-5-mini and google/gemini-2.5-flash are usually enough; compare per-token pricing on the models page. And if what you need is validated JSON output rather than executed actions, use structured outputs instead: tool schemas describe actions, response_format describes shapes.