GPT-5.1 vs Claude vs Gemini vs DeepSeek: price-performance

Put GPT-5.1, Claude Opus 5, Gemini 3 Pro, and DeepSeek V3.2 on the same benchmark suite and you get four scores within a few points of each other — and an output-price spread of roughly 60x. That's the model-selection problem in 2026 in one sentence. Capability converged. Price didn't. Picking models by leaderboard position means optimizing a variable that stopped mattering a few generations ago while ignoring the one that shows up on your invoice.

This is a comparison for teams running production workloads: list prices, context windows, latency and tool-use behavior, and what each model costs per 1,000 requests across three common workload shapes. It ends with the only advice that generalizes — how to run a cheap eval on your own traffic — because the right answer depends on your tasks, not ours.

The contenders, at list price

All figures are list prices at the time of writing (August 2026); re-check them before you commit — providers reprice quietly, some bill a higher tier above certain context lengths, and cache-hit discounts change effective rates substantially. The shape of the table is stable, though: DeepSeek V3.2's output tokens cost roughly 24x less than GPT-5.1's and roughly 60x less than Opus 5's. No reading of any benchmark says Opus is 60x better. Whether it's better enough for your workload is the actual question.

Benchmarks saturated. Your workload didn't

The public benchmarks that drove model selection for years are now mostly ceiling-locked. When four models cluster within a few points, the deltas sit inside the noise floor of prompt phrasing, sampling temperature, and harness quirks — before you account for contamination, which nobody can fully rule out. More fundamentally, a leaderboard score is an average over someone else's task distribution. You don't ship someone else's task distribution.

That splits workloads into two regimes. Where failures are cheap and detectable — codegen gated by tests, extraction gated by schema validation, summaries a human skims anyway — the cheapest model that clears your quality bar wins, and retries are a rounding error. Where failures are expensive or invisible — an agent taking real actions, customer-facing answers nobody double-checks, legal or medical summarization — reliability compounds and premium models earn their price. Most teams have both kinds of traffic and pay for all of it as if it were the second kind.

What separates them in production

Latency

These numbers move with region, provider, time of day, and reasoning-effort settings, so treat them as directional. GPT-5.1 has good time-to-first-token at low reasoning effort but wide variance once effort goes up. Opus 5 is generally the slowest per token of the four at full reasoning — which matters less than it sounds in agentic loops, where wall-clock time is dominated by tool execution. Gemini 3 Pro has been notably quick at long-context prefill in our testing. DeepSeek V3.2 is the interesting case: it's open-weight and served by a dozen-plus providers, so its latency varies more between hosts than between it and its competitors. That's an argument for routing across providers rather than pinning one.

Tool-use reliability

Qualitative, based on production traffic — your schemas will surface different failure modes:

• Claude Opus 5 is the most dependable on long multi-step chains in our experience: fewer malformed calls, better recovery when a tool errors mid-sequence. This is where its price premium is easiest to defend.
• GPT-5.1 does excellent structured output and parallel tool calls; its main quirk is occasional overeagerness — calling a tool when a direct answer would do.
• Gemini 3 Pro is solid but has been the pickiest about schema edge cases; test your unions and optional fields.
• DeepSeek V3.2 improved markedly over V3 at function calling, but on deep chains we still see occasional argument drift. At its prices, strict validation plus a retry is a fine mitigation — three attempts cost well under half of one GPT-5.1 attempt.

Long-context behavior

Gemini 3 Pro's ~1M window is real, and its recall near the top of it is the best we've seen — with the usual hedge that needle-in-a-haystack retrieval is a solved trick for all four models. The harder test is synthesis across facts scattered through the context, and every model degrades there as contexts grow; evaluate it directly if it matters to you. GPT-5.1's ~400K behaves well. Opus 5 caps at 200K but is very solid inside it. DeepSeek's ~128K sounds limiting until you remember that most workloads that "need" huge context actually need better retrieval — stuffing 600K tokens into a prompt is usually the expensive way to avoid fixing your chunking.

Cost per 1,000 requests: three workload shapes

Illustrative math at list price — no caching, no retries. Three shapes that cover a lot of real traffic: a support-chat turn (~800 tokens in, ~250 out), RAG summarization (~12,000 in, ~500 out), and an agentic coding task (~60,000 cumulative input across turns, ~8,000 out).

Three things stand out. First, scale turns small-looking deltas into headcount: a chat product doing 10 million requests a month pays roughly $3,300 on DeepSeek V3.2 versus $102,500 on Opus 5 for identical traffic. Second, RAG is input-dominated, so input price is nearly everything — this is where DeepSeek's ~$0.28 rate is hardest to argue with. Third, agentic work inflates both dimensions, and it's where the caveats bite hardest: prompt caching can cut the input side by 50-90% on iterative loops, and success rate can legitimately flip the ranking. If Opus one-shots a task that a cheaper model fails twice and then hands to an engineer to untangle, the $0.50-versus-$0.02-per-attempt comparison was never the right one.

How to run your own eval

Everything above is context, not a conclusion. The conclusion comes from an afternoon of eval work on your own traffic:

1. Sample 100-500 real tasks from production logs. Keep the ugly ones — malformed inputs, angry users, edge-case documents. Models differentiate in the tail, not at the median.
2. Define a programmatic pass/fail per task. Unit tests for code, exact match or schema validation for extraction, an LLM judge for prose — hand-check 10% of judge verdicts before trusting them, and use a judge from a different model family than the candidates.
3. Run every candidate on identical prompts, 3+ runs per task. A single run measures luck. Don't hand-tune prompts per model yet; compare defaults first, then invest tuning effort in the top two.
4. Record four numbers per model: success rate, cost per solved task, p95 latency, and malformed-output rate. That last one predicts your on-call burden better than any benchmark.
5. Re-run monthly. Providers ship silent updates. The model you evaluated in March is not necessarily the model answering your requests in August.

The historical blocker was plumbing: four SDKs, four auth schemes, four sets of quirks. With an OpenAI-compatible gateway, the whole A/B harness is a loop over strings:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.infro.io/v1",
    api_key=os.environ["INFRO_API_KEY"],
)

MODELS = [
    "openai/gpt-5.1",
    "anthropic/claude-opus-5",
    "google/gemini-3-pro",
    "deepseek/deepseek-v3.2",
]

for model in MODELS:
    for case in eval_set:
        r = client.chat.completions.create(
            model=model,          # the only line that varies
            messages=case.messages,
            temperature=0,
        )
        record(model, case, r)    # pass/fail, cost, latency

That's the whole integration. Through INFRO the same loop reaches 120+ models — the full list is in the model catalog — and the usage dashboard already breaks out cost and latency per model, so half of step four is done for you.

Where each model wins

• DeepSeek V3.2 — the default for high-volume, verifiable work: classification, extraction, summarization, first-pass codegen. At roughly $0.28/$0.42, the burden of proof sits on every more expensive option.
• GPT-5.1 — the balanced pick when one model should do everything competently: strong structured output, good tool use, sane latency, mid-tier price.
• Gemini 3 Pro — genuinely long-context work: whole-codebase questions, long transcripts, multi-document synthesis where retrieval can't carve the problem down.
• Claude Opus 5 — long-horizon agents and work where a wrong answer costs more than the tokens: complex refactors, deep tool chains, outputs someone acts on without checking.

The honest answer for most teams is a blend, not a winner. Route the verifiable 80% of traffic to a cheap model and reserve the premium model for the 20% where reliability pays for itself. That used to be an architecture project. Now it's a routing rule.

Run the numbers yourself

To run this comparison without the plumbing: INFRO puts all four models — and about 120 others — behind one OpenAI-compatible endpoint, with automatic failover and pricing at or below list price on most models (current rates are in the pricing section). Change the base URL, loop over model strings, and let your own eval settle the argument. The quickstart takes about five minutes.