Cost per token is the wrong unit: how to measure AI cost per task

In brief

A cheaper per-token model can cost more per finished task. How to define a task, instrument it and compute cost per successful task, with worked math.

Steve Elliott

Steve Elliott · · 7 min read


On this page
  1. Why do per-token price comparisons mislead?
  2. What counts as a task?
  3. How do you instrument cost per task?
  4. Worked example: when the model that is twice the price costs less
  5. What does cost per successful task leave out?
  6. How should finance use the number?
  7. Sources

Cost per token tells you what a model charges, not what a piece of work costs. A model that is half the price per token can still cost more per successful task, because it may need more turns, more reasoning and more attempts to get the job done.

In our view, the unit that belongs in a business case or a model-selection decision is cost per successful task: everything spent on a unit of work, including failed runs, divided by the runs that met the quality bar. Per-token price is an input to that number, not a substitute for it.

Why do per-token price comparisons mislead?

A cost per token comparison assumes both models use the same number of tokens for the same job. Provider documentation and the FinOps Foundation point to six reasons they don't.

  • Tokenizers differ. Anthropic says Claude 4.7 and later models use a newer tokenizer that produces about 30% more tokens for the same text than Claude Sonnet 4.6 and earlier, depending on the content. A per-token price on one tokenizer is not comparable with a per-token price on another.
  • Thinking is billed but often not shown. OpenAI, Anthropic and Google all bill reasoning tokens at output rates, including thinking the response hides or only summarizes; we cover how in reasoning models and the cost of thinking tokens.
  • Context is re-sent every turn. The FinOps Foundation's paper on token costs notes that multi-turn conversations and agents resend their full history on every call, and says input volume usually dominates spend in conversational applications. An agent that takes ten steps pays for its early context ten times, at full or cached rates.
  • Caching changes the effective rate. On Anthropic's prompt caching prices, a cache hit on Claude Sonnet 5.5 costs $0.10 per million tokens against $2 for base input, while writing to the 5-minute cache costs $2.50. Google's Gemini 3.8 Flash charges $0.075 per million cached tokens plus $0.50 per million tokens per hour of storage through December 31, 2026, doubling to $0.15 and $1.00 from January 1, 2027. How much of a task hits the cache depends on how the application is built, not on the list price.
  • Tools carry their own charges. Anthropic charges $10 per 1,000 web searches on top of tokens, and search results count as input tokens in later turns. OpenAI lists web search at $10 per 1,000 calls, with search content tokens billed at model rates for most models. Tool definitions add input tokens to every request: Anthropic adds 286 system-prompt tokens for tool use on Claude Opus 5.5 and Sonnet 5.5, before the tool schemas themselves.
  • Failures and retries cost money. OpenAI warns that a response that hits its output limit means you "could incur costs for input and reasoning tokens without receiving a visible response." A model that fails a third of the time is billed for every one of those runs.

What counts as a task?

For cost per task, a task is a unit of work the business would recognize and could count without the AI: a resolved service-desk ticket, a processed invoice, a contract clause reviewed, a pull request merged. It is not an API call: one task may take one call or forty.

The FinOps Foundation paper, last updated October 28, 2026, lists cost per successful outcome among its unit metrics: the total cost of a multistep process, counting only runs whose output clears the quality floor for that workload. It adds a point that per-token comparisons ignore: "Failed and discarded runs are still cost."

We think a usable task definition has four parts:

  1. A start and an end. The trigger (ticket opened, document received) and the finish (ticket closed, record written).
  2. A success test. Automated where possible (schema valid, tests pass, answer matches a labeled set), sampled human review where not. Fix it before comparing models.
  3. A scope of cost. Model tokens, cache writes and reads, tool charges, retries and fallbacks to another model. Say explicitly what is left out.
  4. A volume the business plans with. Tickets per month or invoices per quarter, so the unit cost can be multiplied into a forecast.

It is the test we apply to IT unit costs in why most IT unit costs never change a decision: a unit the consumer recognizes, paired with a quality measure.

How do you instrument cost per task?

The work is attaching each response's usage data to a task.

  1. Tag every call with a task ID. Each model call, tool call and retry carries the ID of the task it serves, through your gateway or application logs.
  2. Capture the usage fields, not an estimate. Anthropic's usage object reports input tokens, cache-read and cache-creation tokens, output tokens and server tool requests such as web searches. OpenAI reports reasoning tokens inside its output token details. Gemini reports thinking tokens in a thoughtsTokenCount field.
  3. Price usage from a dated rate table. Keep the per-model rates you applied and the date, so a price change or a negotiated discount can be replayed against history.
  4. Record the outcome. Pass or fail against the success test, attached to the same task ID.
  5. Compute the ratio over a period. Total cost of all attempts, including failures and retries, divided by tasks that passed.

Don't borrow averages. The FinOps Foundation's indicative range runs from about 500 to 2,000 tokens per session for simple Q&A or retrieval to 10,000 to 200,000 or more for agentic workflows, and the paper says to validate it against your own data.

Worked example: when the model that is twice the price costs less

The token counts and success rates below are illustrative, made up to show the mechanics. The prices are Anthropic's published list prices as of November 2026. Both models use the same tokenizer, so the token counts are comparable.

Per million tokensClaude Sonnet 5.5Claude Opus 5.5
Base input$2.00$4.00
5-minute cache write$2.50$5.00
Cache hit$0.10$0.20
Output, including thinking$10.00$20.00

On every line, Opus 5.5 costs twice as much per token. Take an illustrative service-desk agent that resolves tickets using tools. Assume an 8,000-token prefix of instructions and tool definitions that is already cached and read on every turn (steady traffic keeps the 5-minute cache warm), and that each turn adds 2,000 new tokens of tool results and replies, written to the cache once and read on later turns.

Assume the cheaper model takes 7 turns and 1,100 output tokens per turn, and passes the success test on 65% of tickets. The dearer model takes 5 turns and 1,000 output tokens per turn, and passes on 90%.

Per attemptSonnet 5.5Opus 5.5
Cache reads98,000 tokens × $0.10 = $0.009860,000 tokens × $0.20 = $0.0120
Cache writes14,000 tokens × $2.50 = $0.035010,000 tokens × $5.00 = $0.0500
Output7,700 tokens × $10 = $0.07705,000 tokens × $20 = $0.1000
Cost per attempt$0.1218$0.1620
Success rate65%90%
Cost per successful task$0.187$0.180

Over 1,000 tickets, the cheaper model spends $121.80 for 650 resolved tickets; the dearer one spends $162.00 for 900. Twice the price per token, 33% more per attempt, about 4% less per successful task.

What the example shows:

  • Output is the biggest line. It is about 63% of the attempt cost for Sonnet 5.5 and 62% for Opus 5.5, and it includes thinking you may never see.
  • Caching matters more than the model choice here. Without caching, the same attempts would cost $0.301 and $0.380 at base input prices.
  • The result turns on a few points of success rate. Sonnet 5.5 wins on cost per successful task if it passes about 68% or more ($0.1218 divided by $0.180). That is why the comparison has to run on your own tasks, with your own success test.

What does cost per successful task leave out?

Two things, and both can be larger than the tokens.

The failed tasks still need doing. In the example, the cheaper model leaves 350 tickets in 1,000 for someone else, against 100. In our view, if people pick them up, the staff time will dwarf a token difference of under a cent per task. We covered why labor, not tokens, dominates most AI initiative costs in AI initiative costs: tokens vs labor.

The harness around the model. The FinOps Foundation paper says that in production RAG and agentic architectures, vector databases, embeddings, rerankers, orchestration, caches and observability often represent 40 to 60 percent of total feature spend, and that none of it appears on the provider invoice. It warns that cost-per-query figures that leave out the harness underreport cost.

In our view, the model-level figure is still worth computing: it moves when you change model, prompt, effort or caching, so engineers can act on it. It just isn't the full cost of the service.

How should finance use the number?

We suggest finance treat the number this way:

  • Compare models on cost per successful task, never on list price alone. Run both candidates on the same sample of real tasks, with the success test fixed in advance.
  • Recompute when prices or settings change. A change to reasoning effort, caching or tool use can move tokens per task more than a price cut does.
  • Report it beside the success rate and the volume. A falling cost per task with a falling success rate is not a saving. Treated this way, AI unit economics becomes a measure a budget owner can plan with.

In our view, a cost per task earns its keep only when it is read beside the service it supports, the owner who answers for it and the budget it draws on. That argues for holding them in one operating model.

Sources

Frequently asked questions

Why is cost per token misleading for comparing AI models?

Because models use different numbers of tokens for the same work. Tokenizers differ, reasoning tokens are billed as output but often hidden, conversation history is re-sent every turn, caching changes the effective rate, tools add their own charges, and failed runs are still billed. A model with a lower price per token can need enough extra turns, thinking and retries to cost more per finished task.

How do you calculate cost per successful task?

Add up the cost of every attempt at a task over a period, including failed runs, retries, cache writes and reads, and tool charges, priced from a dated rate table. Divide by the number of tasks that passed a success test you defined in advance. The FinOps Foundation describes the same idea as cost per successful outcome, counting only runs that clear the quality floor.

What success rate makes a pricier model cheaper per task?

Divide the cheaper model's cost per attempt by the pricier model's cost per successful task. If the cheaper model's success rate falls below that figure, it costs more per successful task. In our illustrative example on Anthropic's November 2026 prices, Sonnet 5.5 costs $0.1218 per attempt and Opus 5.5 $0.180 per success, so Sonnet 5.5 needs to pass about 68% of tasks.

Does cost per task include everything an AI feature costs?

No. Measured from provider usage data, it covers model tokens, caching and tool charges. It leaves out the harness around the model, which the FinOps Foundation says often accounts for 40 to 60 percent of feature spend in production RAG and agentic systems, and the staff time spent on tasks the model fails. In our view, report it as the model-level unit cost and show the rest separately.

Free playbook

Run the enterprise from any AI.

Read The Headless Enterprise
Steve Elliott

Steve Elliott

Published November 22, 2026

More on AI Economics

Tokens, inference cost, cost per task.