LLM Inference Cost Comparison 2026: API Pricing Guide

Last updated: July 2026

LLM pricing pages look simple until an application leaves the demo stage. A table may show input and output prices, but production traffic adds system prompts, retrieved documents, tool schemas, conversation history, cached prefixes, structured outputs, retries and sometimes hidden reasoning tokens. The result is a unit-economics problem: how much does one successful answer, document, support ticket or agent task cost when the whole workflow is included?

This LLM inference cost comparison is a practical way to answer that question. It uses a July 2026 snapshot of first-party prices from OpenAI, Anthropic and Google, then turns the prices into comparable examples. The values are a reference snapshot, not a permanent promise. Model names, preview status, tokenizer behavior, service tiers and regional premiums change, so re-check the provider’s pricing page before committing a budget.

What counts as LLM inference cost?

Input tokens are the tokens sent to the model: the user prompt, system instructions, conversation history, retrieved documents, tool definitions and sometimes previous tool results. A request that looks like “classify this ticket” may carry a large policy prompt and a full customer history. That context is billable even if the final answer is only one sentence.

Output tokens are generated tokens. They are usually more expensive than input tokens because generation consumes more compute and must run sequentially. A verbose answer, JSON object, code patch or reasoning trace can therefore dominate cost even when the prompt is short. Output limits are both a user-experience setting and a financial control.

Cached input applies when a provider recognizes a repeated prefix or cached context. It can be substantially cheaper than ordinary input, but cache writes, cache storage duration, minimum prefix lengths and invalidation rules vary. The right question is not “does this provider support caching?” but “what percentage of my input tokens will actually hit the cache, and how long will the prefix remain stable?”

Reasoning or intermediate tokens are provider-specific. Some APIs expose them in usage fields, some price them as output, and some describe them separately. Agent frameworks can also make multiple model calls in one user-visible task. If you calculate only the final answer, your estimate will be systematically low.

Tool and infrastructure fees include web search, grounding, image input, file search, embeddings, reranking, sandbox execution, storage, logging and egress. A model can have a cheap token rate while the application’s retrieval and tool layer becomes the larger bill. This is especially common in research agents and RAG systems with many retrieved chunks.

Reference API pricing in July 2026

The comparison below uses a small, recognizable set of first-party models rather than trying to list every provider. OpenAI’s current API pricing lists GPT-5.5 at $5 input and $30 output per million short-context tokens, with cached input at $0.50. Anthropic lists Claude Sonnet 4.6 at $3 input, $15 output, and $0.30 for cache hits. Google lists Gemini 3.1 Pro Preview at $2 input and $12 output for prompts up to 200,000 tokens; its price rises to $4/$18 above that threshold.

These are not quality scores. A cheaper token can be a false economy if the model needs more retries, produces less reliable structured output or requires a larger prompt to reach the same quality. Conversely, a premium model may be the rational choice when one accurate call replaces three weaker calls and a human correction loop.

Why the “cheapest” row changes with context length

Gemini 3.1 Pro Preview has a clear threshold in its published pricing: prompts above 200,000 tokens are charged at a higher rate. OpenAI also separates short and long context pricing for flagship models. Anthropic’s current pricing page describes a one-million-token context window for several newer models at standard pricing, but the number of tokens still matters for throughput, latency and cache behavior. A model’s context window is not the same thing as a free allowance.

Long context is also a product decision. Sending an entire repository, conversation or customer record on every turn may be simpler than retrieval, but it can increase cost and latency. Retrieval reduces the prompt only when chunking, filtering and ranking are good enough. A cheap model fed with irrelevant context can cost more than a stronger model fed with a compact, high-signal prompt.

The cost formula

For a single request, a useful first approximation is:

For a workflow, multiply that request cost by the number of model turns and add the expected cost of retries. If 2% of calls fail validation and are retried, the expected multiplier is roughly 1.02 before additional escalation or human review. Agent loops are more variable: a median of four turns can hide a 95th percentile of twelve turns when the task is ambiguous.

from dataclasses import dataclass

@dataclass
class Price:
    input_per_m: float
    cached_per_m: float
    output_per_m: float

def estimate(input_tokens, cached_tokens, output_tokens, price, tool_fees=0.0,
             retry_rate=0.0, turns=1):
    token_cost = (
        input_tokens * price.input_per_m
        + cached_tokens * price.cached_per_m
        + output_tokens * price.output_per_m
    ) / 1_000_000
    one_task = (token_cost + tool_fees) * turns
    return one_task * (1 + retry_rate)

prices = {
    "gpt-5.5": Price(5.00, 0.50, 30.00),
    "claude-sonnet-4.6": Price(3.00, 0.30, 15.00),
    "gemini-3.1-pro": Price(2.00, 0.20, 12.00),
}

for name, price in prices.items():
    cost = estimate(10_000, 7_000, 1_000, price,
                    tool_fees=0.002, retry_rate=0.03, turns=3)
    print(name, round(cost, 6))

The code deliberately keeps input_tokens and cached_tokens separate. Do not subtract cached tokens from total input unless the provider’s usage schema tells you to. Some APIs report cache reads alongside ordinary input; others expose a separate field. Build the estimator around the provider’s actual response usage object, then compare it with the invoice.

Worked example 1: a short classification request

Assume 2,000 ordinary input tokens and 500 output tokens, no cache, no tools and one call. At the reference prices, the token-only estimates are about $0.025 for GPT-5.5, $0.0135 for Claude Sonnet 4.6 and $0.010 for Gemini 3.1 Pro Preview. This is a small absolute difference, but at one million identical tasks it becomes roughly $25,000, $13,500 and $10,000 before retries and infrastructure.

The example can mislead if the classification prompt contains 100,000 tokens of policy text. The output may still be 500 tokens, but the input side becomes the dominant line item. It also ignores quality: if the cheapest model produces 8% invalid JSON and the other produces 1%, the retry cost and operational friction can reverse the conclusion.

Worked example 2: a long RAG answer

Assume 100,000 input tokens and 2,000 output tokens with no cache. The token-only cost is approximately $0.56 on GPT-5.5, $0.33 on Claude Sonnet 4.6 and $0.224 on Gemini 3.1 Pro Preview, assuming the Gemini prompt stays below the 200,000-token threshold. At 100,000 tasks per month, that is about $56,000, $33,000 and $22,400 respectively.

Now suppose 80,000 of the 100,000 input tokens are a stable policy prefix. If the application achieves a cache hit for that prefix, the estimate changes. GPT-5.5 charges about $0.04 for 80,000 cached tokens instead of $0.40 at ordinary input pricing; Claude’s cache-hit line is about $0.024 instead of $0.24; Gemini’s cache line is about $0.016 instead of $0.16. The savings are meaningful, but only if the prefix is actually stable and the cache lifetime matches the traffic pattern.

Worked example 3: an agentic workflow

An agent task is not one prompt. Imagine a research agent with three model turns: a planner, a tool-selection turn and a final synthesizer. Each turn carries some of the conversation and tool schema. If every turn uses 10,000 input tokens and 1,000 output tokens, the raw token bill is roughly three times the one-call estimate before search fees. If the agent retrieves web pages or runs code, the tool results become new input tokens on the next turn.

This is why an agent can cost much more than a chatbot even when the final answer is short. The agent’s visible response hides intermediate decisions. In a production system, log a trace with model name, input tokens, cached tokens, output tokens, tool calls, latency, validation result and retry count. Our guide to agentic workflows versus AI agents covers the architectural trade-off; the cost comparison here gives you the accounting layer.

Prompt caching: the highest-leverage discount with the most caveats

Prompt caching works best when many requests share a large, unchanged prefix: system policy, product catalog, codebase instructions or a long legal rubric. Put stable content before dynamic user content, keep the prefix above the provider’s minimum, and record cache reads separately. If the user prompt is unique and the system message is only 500 tokens, caching may not move the bill enough to justify the complexity.

Cache writes are not always free. Anthropic lists separate five-minute and one-hour cache-write prices, while cache hits are cheaper. Google lists cached-content and storage prices. OpenAI’s API table distinguishes cached input and cache writes for eligible models. The exact economics depend on the ratio of writes to reads. A low-volume service that creates a cache for every request may pay more than a service with a stable prefix and high reuse.

Cache correctness is also a security boundary. Never reuse a cache entry across tenants when it contains private customer data. Include a tenant or policy version in the cache key, and invalidate the prefix when permissions, pricing rules or product facts change. Saving tokens is not worth leaking context.

Batch, Flex and offline work

If a task does not need an immediate response, batch processing can change the economics. OpenAI’s published batch table halves the GPT-5.5 short-context input and output rates relative to standard pricing. Anthropic lists batch rates for Sonnet 4.6 at $1.50 input and $7.50 output per million tokens. Google lists Gemini 3.1 Pro Preview batch rates at $1/$6 under the 200,000-token threshold.

Batch is a product constraint, not just a discount. You trade interactive latency for queueing, and you need idempotency, partial-failure handling and a way to reconcile results. Good batch candidates include nightly classification, document extraction, evaluation runs, embedding refreshes and report generation. Bad candidates include live fraud decisions, interactive coding help and customer support where the user is waiting.

Flex or lower-priority tiers can also be attractive for workloads that tolerate variable latency. Measure the end-to-end value, not just the token discount. If a slower result causes an extra user action or a queue timeout, the application may lose more revenue than it saves in inference.

Why tokenizer differences matter

One million tokens is not a universal amount of text across providers. Tokenizers split the same code, Polish sentence, JSON document or multilingual transcript differently. Anthropic’s pricing documentation notes that newer Claude models use a tokenizer that produces approximately 30% more tokens for the same text, depending on workload shape. That does not mean Claude is automatically more expensive; it means a benchmark must measure provider-reported tokens on the actual corpus.

Do not estimate multilingual or code-heavy traffic with an English word-count shortcut. Sample real prompts, count them with each provider’s tokenizer where available, and use the resulting distribution. For budgeting, use p50 and p95 tokens, not just an average. The tail is where long documents, tool errors and unexpected conversation history show up.

OpenAI, Anthropic or Google: how to choose by workload

Choose OpenAI when your stack benefits from broad service coverage

OpenAI is a reasonable choice when one API family covers text, structured output, multimodal input, specialized reasoning, coding and batch workflows you already need. The important cost controls are cached input, batch or Flex where appropriate, output caps and regional processing premiums. If your application uses a direct model and then a second vendor for embeddings or search, compare the complete stack rather than assuming model price is the whole decision.

Choose Anthropic when prompt-heavy reasoning is the workload

Claude Sonnet 4.6 has a familiar input/output shape and published cache tiers. Anthropic’s documentation makes tool-use tokens and cache behavior visible, which helps teams build a more honest estimator. The platform can be attractive for long instruction sets and document workflows, but measure tokenizer expansion and output length on your own data.

Choose Google when context economics and batch matter

Gemini 3.1 Pro Preview offers a lower standard input price in the reference table and explicit pricing tiers around 200,000 tokens, plus standard batch and Flex modes. That can be compelling for large-volume workloads that fit the model’s quality envelope. Preview status, rate limits, grounding fees and model availability should be treated as part of the procurement risk.

For a wider provider decision, use our best inference APIs guide alongside this cost model. The two pages answer different questions: the buyer guide compares platform fit, while this page forces you to calculate the bill for your workload.

Cost controls that work in production

Model routing is often the highest-impact control. Start with a quality gate, not a price assumption. If a small model passes a structured-output validator and a semantic test, keep it. If it fails, escalate to a larger model. This makes cost a function of observed difficulty rather than a global guess.

Guard agent loops explicitly. Set a maximum number of turns, a maximum tool budget, a maximum output budget and a hard dollar budget per task. A kill switch should stop a run when the agent repeats the same tool call, grows its context unexpectedly or exceeds a p95 latency threshold. Our AI agent architecture guide covers the control layers; the budget is the economic version of the same boundary.

What belongs in a monthly LLM budget?

Build a budget from workload classes, not from a single “requests per month” number. For each class, record requests, tokens per request, cache hit rate, output length, turns, tools, retries, model mix and target quality. Then calculate p50, p95 and worst-case caps. Add a separate line for embeddings, reranking, web search, storage, observability and human review.

A useful dashboard has four views. The first is cost per request, which catches sudden prompt growth. The second is cost per valid or accepted result, which catches quality-driven retries. The third is cost by tenant or product feature, which supports pricing and quota decisions. The fourth is cost by model and service tier, which reveals whether a discount is actually reaching the workload.

Key takeaways

The headline price per million tokens is only the first layer of LLM economics. Input context, output verbosity, caching, long-context thresholds, reasoning turns, tool fees, retries and infrastructure can move the result by multiples. A short classification request and a research agent are not the same product just because both call a chat model.

Use the published provider prices as inputs to a small calculator, then replace assumptions with real usage telemetry. Compare cost per successful task, measure p95 behavior and keep a hard budget around agent loops. If you do those three things, price changes become an input to an operating model rather than a surprise on the invoice.