Structured outputs

Structured outputs turn a model's reply into JSON you can parse directly, instead of prose you have to scrape. INFRO supports both standard response_format modes: json_object, which produces syntactically valid JSON, and json_schema, which constrains output to a schema you define.

response_format is part of the standard OpenAI request shape on POST /v1/chat/completions, so it works unchanged through the OpenAI SDKs — no extra_body needed. INFRO passes it through to the provider that serves the request and never rewrites what comes back.

JSON object mode

Set response_format to {"type": "json_object"} and the model emits a single JSON value instead of prose. Nothing enforces which keys appear, though — describe the shape you want in the prompt and validate the result.

{
  "model": "anthropic/claude-sonnet-5",
  "messages": [
    {"role": "system", "content": "Reply in JSON with keys sentiment and confidence."},
    {"role": "user", "content": "The checkout flow keeps timing out and support has not replied in two days."}
  ],
  "response_format": {"type": "json_object"}
}

JSON schema mode

json_schema mode constrains generation to a schema, so the response contains exactly the fields you declared. Pass a json_schema object inside response_format:

namestringrequired

An identifier for the schema, such as invoice_extraction. Letters, digits, underscores, and dashes.

strictboolean

Set true to enforce the schema during decoding. Without it, the schema is advisory and output can drift.

schemaobjectrequired

A standard JSON Schema object. For strict mode, set additionalProperties: false on every object and list every property in required — mark optional fields with a nullable type like ["string", "null"] instead of omitting them.

Strict mode works only on models whose capabilities array includes "json" in the catalog — see Models for the full response shape. Check before you send:

curl -s https://api.infro.io/v1/models \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  | jq -r '.data[] | select(.capabilities | index("json")) | .id'

If the request also carries fallbacks, make sure every model in the chain has the json capability. The response's model field reports which one actually served.

Example: extracting fields from text

The request below pulls four typed fields out of a free-text invoice line. With strict: true the response contains exactly those fields.

curl https://api.infro.io/v1/chat/completions \
  -H "Authorization: Bearer $INFRO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.1",
    "messages": [
      {"role": "system", "content": "Extract invoice fields from the text."},
      {"role": "user", "content": "Invoice #4821 from Acme Corp, due March 15, 2026, total $1,240.50."}
    ],
    "response_format": {
      "type": "json_schema",
      "json_schema": {
        "name": "invoice_extraction",
        "strict": true,
        "schema": {
          "type": "object",
          "properties": {
            "invoice_number": {"type": "string"},
            "vendor": {"type": "string"},
            "due_date": {"type": "string", "description": "ISO 8601 date"},
            "total": {"type": "number"}
          },
          "required": ["invoice_number", "vendor", "due_date", "total"],
          "additionalProperties": false
        }
      }
    }
  }'

The structured object arrives as a JSON string in choices[0].message.content, the same place as any other completion — parse it with json.loads or JSON.parse. It also works with stream: true: deltas arrive as ordinary content chunks and the JSON parses once the stream completes. See Streaming.

Prompting tips

• In json_object mode, say "JSON" explicitly and sketch the shape you want — a one-line example object in the system prompt beats a paragraph of description.
• Use description strings inside the schema. They reach the model and steer field content like prompt text, without bloating the conversation.
• Keep extraction requests at low temperature (0 to 0.3). You want determinism, not creativity.
• Budget max_tokens for the whole object. Truncated JSON never parses; finish_reason: "length" means the output was cut off mid-object.
• If you send the same schema on every request, keep the system prompt stable and put the varying input last so provider-side prompt caching can kick in.

Handling validation failures

INFRO does not rewrite or repair model output. If a model emits invalid JSON in json_object mode, or a provider's strict decoder fails, the raw text comes back in a normal 200 response — there is no special error status for malformed output.

Treat parsing as part of the request lifecycle: parse, validate against your schema client-side (pydantic, zod, or a plain JSON Schema validator), and retry on failure. A cheap, effective retry appends the parse error and the bad output to the conversation and asks the model to correct it. Check finish_reason first — if it is "length", the fix is a higher max_tokens, not a retry. Transport-level failures are a separate concern; see Errors.

Combining with tool calling

Tools and response_format solve different problems. Use tool calling when the model should decide to take an action or fetch data mid-conversation; use response_format when you always want the final answer in a fixed shape. Don't define a dummy tool just to get structured output — json_schema mode is simpler and skips a round trip.

The two combine cleanly: run the tool loop as usual, returning each result as a tool role message with its tool_call_id, and once the model stops requesting tools its final message conforms to response_format. While tools are being called, finish_reason is tool_calls and content may be empty; the structured payload arrives on the final stop message.