Cost per token is the wrong unit: how to measure AI cost per task
In brief
A cheaper per-token model can cost more per finished task. How to define a task, instrument it and compute cost per successful task, with worked math.
Steve Elliott · · 7 min read
On this page
Cost per token tells you what a model charges, not what a piece of work costs. A model that is half the price per token can still cost more per successful task, because it may need more turns, more reasoning and more attempts to get the job done.
In our view, the unit that belongs in a business case or a model-selection decision is cost per successful task: everything spent on a unit of work, including failed runs, divided by the runs that met the quality bar. Per-token price is an input to that number, not a substitute for it.
Why do per-token price comparisons mislead?
A cost per token comparison assumes both models use the same number of tokens for the same job. Provider documentation and the FinOps Foundation point to six reasons they don't.
- Tokenizers differ. Anthropic says Claude 4.7 and later models use a newer tokenizer that produces about 30% more tokens for the same text than Claude Sonnet 4.6 and earlier, depending on the content. A per-token price on one tokenizer is not comparable with a per-token price on another.
- Thinking is billed but often not shown. OpenAI, Anthropic and Google all bill reasoning tokens at output rates, including thinking the response hides or only summarizes; we cover how in reasoning models and the cost of thinking tokens.
- Context is re-sent every turn. The FinOps Foundation's paper on token costs notes that multi-turn conversations and agents resend their full history on every call, and says input volume usually dominates spend in conversational applications. An agent that takes ten steps pays for its early context ten times, at full or cached rates.
- Caching changes the effective rate. On Anthropic's prompt caching prices, a cache hit on Claude Sonnet 5.5 costs $0.10 per million tokens against $2 for base input, while writing to the 5-minute cache costs $2.50. Google's Gemini 3.8 Flash charges $0.075 per million cached tokens plus $0.50 per million tokens per hour of storage through December 31, 2026, doubling to $0.15 and $1.00 from January 1, 2027. How much of a task hits the cache depends on how the application is built, not on the list price.
- Tools carry their own charges. Anthropic charges $10 per 1,000 web searches on top of tokens, and search results count as input tokens in later turns. OpenAI lists web search at $10 per 1,000 calls, with search content tokens billed at model rates for most models. Tool definitions add input tokens to every request: Anthropic adds 286 system-prompt tokens for tool use on Claude Opus 5.5 and Sonnet 5.5, before the tool schemas themselves.
- Failures and retries cost money. OpenAI warns that a response that hits its output limit means you "could incur costs for input and reasoning tokens without receiving a visible response." A model that fails a third of the time is billed for every one of those runs.
What counts as a task?
For cost per task, a task is a unit of work the business would recognize and could count without the AI: a resolved service-desk ticket, a processed invoice, a contract clause reviewed, a pull request merged. It is not an API call: one task may take one call or forty.
The FinOps Foundation paper, last updated October 28, 2026, lists cost per successful outcome among its unit metrics: the total cost of a multistep process, counting only runs whose output clears the quality floor for that workload. It adds a point that per-token comparisons ignore: "Failed and discarded runs are still cost."
We think a usable task definition has four parts:
- A start and an end. The trigger (ticket opened, document received) and the finish (ticket closed, record written).
- A success test. Automated where possible (schema valid, tests pass, answer matches a labeled set), sampled human review where not. Fix it before comparing models.
- A scope of cost. Model tokens, cache writes and reads, tool charges, retries and fallbacks to another model. Say explicitly what is left out.
- A volume the business plans with. Tickets per month or invoices per quarter, so the unit cost can be multiplied into a forecast.
It is the test we apply to IT unit costs in why most IT unit costs never change a decision: a unit the consumer recognizes, paired with a quality measure.
How do you instrument cost per task?
The work is attaching each response's usage data to a task.
- Tag every call with a task ID. Each model call, tool call and retry carries the ID of the task it serves, through your gateway or application logs.
- Capture the usage fields, not an estimate. Anthropic's usage object reports input tokens, cache-read and cache-creation tokens, output tokens and server tool requests such as web searches. OpenAI reports reasoning tokens inside its output token details. Gemini reports thinking tokens in a thoughtsTokenCount field.
- Price usage from a dated rate table. Keep the per-model rates you applied and the date, so a price change or a negotiated discount can be replayed against history.
- Record the outcome. Pass or fail against the success test, attached to the same task ID.
- Compute the ratio over a period. Total cost of all attempts, including failures and retries, divided by tasks that passed.
Don't borrow averages. The FinOps Foundation's indicative range runs from about 500 to 2,000 tokens per session for simple Q&A or retrieval to 10,000 to 200,000 or more for agentic workflows, and the paper says to validate it against your own data.
Worked example: when the model that is twice the price costs less
The token counts and success rates below are illustrative, made up to show the mechanics. The prices are Anthropic's published list prices as of November 2026. Both models use the same tokenizer, so the token counts are comparable.
| Per million tokens | Claude Sonnet 5.5 | Claude Opus 5.5 |
|---|---|---|
| Base input | $2.00 | $4.00 |
| 5-minute cache write | $2.50 | $5.00 |
| Cache hit | $0.10 | $0.20 |
| Output, including thinking | $10.00 | $20.00 |
On every line, Opus 5.5 costs twice as much per token. Take an illustrative service-desk agent that resolves tickets using tools. Assume an 8,000-token prefix of instructions and tool definitions that is already cached and read on every turn (steady traffic keeps the 5-minute cache warm), and that each turn adds 2,000 new tokens of tool results and replies, written to the cache once and read on later turns.
Assume the cheaper model takes 7 turns and 1,100 output tokens per turn, and passes the success test on 65% of tickets. The dearer model takes 5 turns and 1,000 output tokens per turn, and passes on 90%.
| Per attempt | Sonnet 5.5 | Opus 5.5 |
|---|---|---|
| Cache reads | 98,000 tokens × $0.10 = $0.0098 | 60,000 tokens × $0.20 = $0.0120 |
| Cache writes | 14,000 tokens × $2.50 = $0.0350 | 10,000 tokens × $5.00 = $0.0500 |
| Output | 7,700 tokens × $10 = $0.0770 | 5,000 tokens × $20 = $0.1000 |
| Cost per attempt | $0.1218 | $0.1620 |
| Success rate | 65% | 90% |
| Cost per successful task | $0.187 | $0.180 |
Over 1,000 tickets, the cheaper model spends $121.80 for 650 resolved tickets; the dearer one spends $162.00 for 900. Twice the price per token, 33% more per attempt, about 4% less per successful task.
What the example shows:
- Output is the biggest line. It is about 63% of the attempt cost for Sonnet 5.5 and 62% for Opus 5.5, and it includes thinking you may never see.
- Caching matters more than the model choice here. Without caching, the same attempts would cost $0.301 and $0.380 at base input prices.
- The result turns on a few points of success rate. Sonnet 5.5 wins on cost per successful task if it passes about 68% or more ($0.1218 divided by $0.180). That is why the comparison has to run on your own tasks, with your own success test.
What does cost per successful task leave out?
Two things, and both can be larger than the tokens.
The failed tasks still need doing. In the example, the cheaper model leaves 350 tickets in 1,000 for someone else, against 100. In our view, if people pick them up, the staff time will dwarf a token difference of under a cent per task. We covered why labor, not tokens, dominates most AI initiative costs in AI initiative costs: tokens vs labor.
The harness around the model. The FinOps Foundation paper says that in production RAG and agentic architectures, vector databases, embeddings, rerankers, orchestration, caches and observability often represent 40 to 60 percent of total feature spend, and that none of it appears on the provider invoice. It warns that cost-per-query figures that leave out the harness underreport cost.
In our view, the model-level figure is still worth computing: it moves when you change model, prompt, effort or caching, so engineers can act on it. It just isn't the full cost of the service.
How should finance use the number?
We suggest finance treat the number this way:
- Compare models on cost per successful task, never on list price alone. Run both candidates on the same sample of real tasks, with the success test fixed in advance.
- Recompute when prices or settings change. A change to reasoning effort, caching or tool use can move tokens per task more than a price cut does.
- Report it beside the success rate and the volume. A falling cost per task with a falling success rate is not a saving. Treated this way, AI unit economics becomes a measure a budget owner can plan with.
In our view, a cost per task earns its keep only when it is read beside the service it supports, the owner who answers for it and the budget it draws on. That argues for holding them in one operating model.
Sources
- Anthropic, Pricing (model prices, prompt caching, tool use, web search, tokenizer note), accessed November 22, 2026
- Anthropic, Thinking, accessed November 22, 2026
- OpenAI, API pricing (tool pricing), accessed November 22, 2026
- OpenAI, Reasoning models, accessed November 22, 2026
- Google, Gemini Developer API pricing (Gemini 3.8 Flash), accessed November 22, 2026
- Google, Gemini thinking, accessed November 22, 2026
- FinOps Foundation, Managing AI Token Costs from Direct Model Provider APIs, last updated October 28, 2026
Frequently asked questions
Why is cost per token misleading for comparing AI models?
Because models use different numbers of tokens for the same work. Tokenizers differ, reasoning tokens are billed as output but often hidden, conversation history is re-sent every turn, caching changes the effective rate, tools add their own charges, and failed runs are still billed. A model with a lower price per token can need enough extra turns, thinking and retries to cost more per finished task.
How do you calculate cost per successful task?
Add up the cost of every attempt at a task over a period, including failed runs, retries, cache writes and reads, and tool charges, priced from a dated rate table. Divide by the number of tasks that passed a success test you defined in advance. The FinOps Foundation describes the same idea as cost per successful outcome, counting only runs that clear the quality floor.
What success rate makes a pricier model cheaper per task?
Divide the cheaper model's cost per attempt by the pricier model's cost per successful task. If the cheaper model's success rate falls below that figure, it costs more per successful task. In our illustrative example on Anthropic's November 2026 prices, Sonnet 5.5 costs $0.1218 per attempt and Opus 5.5 $0.180 per success, so Sonnet 5.5 needs to pass about 68% of tasks.
Does cost per task include everything an AI feature costs?
No. Measured from provider usage data, it covers model tokens, caching and tool charges. It leaves out the harness around the model, which the FinOps Foundation says often accounts for 40 to 60 percent of feature spend in production RAG and agentic systems, and the staff time spent on tasks the model fails. In our view, report it as the model-level unit cost and show the rest separately.
Steve Elliott
Published November 22, 2026
More on AI Economics
Tokens, inference cost, cost per task.
Keep reading
Self-hosting an open-weight model vs paying per token: where the break-even sits
At Google Cloud prices as of November 2026, self-hosting gpt-oss-120b struggles to beat the same model sold per token. The break-even math, and when it wins.
Finding shadow AI spend: where AI cost hides outside the IT budget
Unbudgeted AI spend sits on cards, in price uplifts, in separate API accounts and in metered add-ons. How to find each, and whether to consolidate or block.
Key Metrics to Measure Success
The right success metrics depend on your goals: revenue, retention, customer satisfaction, efficiency and growth. Learn how to choose and track them.
How to Measure Customer Retention Rate
Customer retention rate is the share of customers you keep over a period: customers at the end minus new customers, divided by customers at the start. Learn to track it.