Cost per Token
What is Cost per Token?
Cost per token is the price paid for each token a language model processes, usually quoted per million tokens with separate rates for input and output, and the basic unit price of AI services.
Updated November 2026 · 6 min read
Cost per token is the unit price of generative AI. Most AI model providers publish their prices as a dollar amount per million input tokens and per million output tokens, and every request is billed by multiplying the tokens used by those rates. Prices differ enormously between models: a small, fast model can cost a tiny fraction of what the most capable frontier model costs for the same number of tokens, and the gap is often more than tenfold.
Cost per token is also the figure most often used to compare models and providers, and the one most often misread. A model with a lower price per token can still cost more per task if it needs longer prompts, produces longer answers or takes more attempts to get the right result. For IT financial management, cost per token is the starting point for AI cost analysis, but decisions should rest on the cost of completed work, not the price of individual tokens.
Definition of Cost per Token
Cost per token is the amount charged for processing one AI token. Because individual token prices are tiny, providers quote them per million tokens, such as $3 per million input tokens and $15 per million output tokens.
The effective cost per token for an organization can differ from the list price once discounts, caching, batch processing and committed spending agreements are taken into account.
Components of Token Pricing
Most providers set separate prices for input and output tokens, with output typically priced several times higher.
Many offer reduced prices for cached input, which is content reused across requests, and for batch processing, where requests are completed within a longer time window instead of immediately.
Some charge more per token for very long requests, for priority processing or for features such as reasoning. Image, audio and video inputs may be converted to tokens at their own rates.
List Price and Effective Price
The list price is what a provider publishes. The effective price is what an organization actually pays per token after all discounts and pricing features are applied.
An application that caches most of its input and runs some work in batches may achieve an effective cost per token well below the list price.
Tracking effective cost per token over time shows whether optimization efforts and negotiated discounts are working. It is the AI equivalent of the effective savings rate used in cloud cost management.
Importance of Cost per Token in IT Financial Management
Cost per token determines how much each unit of AI usage costs, so it directly affects budgets, forecasts and the economics of AI-enabled products.
Token prices have fallen sharply for models of a given capability over time, while new, more capable models often launch at higher prices. Organizations need to revisit their model choices regularly to benefit from these changes.
Because so many AI costs are calculated as tokens multiplied by price, small changes in price have a large effect at scale. A reduction of a fraction of a cent per thousand tokens can be worth a great deal across billions of tokens a month.
Comparing Models
Cost per token is the simplest way to compare models, but it can mislead. A cheaper model that produces lower-quality answers may require retries, human correction or a second model call.
Models also differ in how many tokens they need for the same task. Some produce longer answers or more reasoning, and different tokenizers can count the same text differently.
The more useful comparison is cost per task at an acceptable level of quality, which captures both the price and the tokens each model uses.
Budgeting and Pricing
Token prices feed directly into AI budgets and forecasts, combined with expected request volumes and tokens per request.
For companies that sell AI-enabled products, cost per token shapes gross margin and pricing decisions, especially for features that customers can use without limit.
Price changes by providers, both increases and reductions, should prompt a review of forecasts and of which models are used for which tasks.
How to Manage Cost per Token
Managing cost per token involves choosing the right model for each task, using the pricing features providers offer and negotiating where volumes justify it.
The aim is the lowest effective price for the quality each task requires.
Lowering the Effective Price
Use prompt caching for instructions and content repeated across requests, since cached input is billed at a large discount.
Run work that does not need an immediate answer, such as overnight classification or report generation, through batch inference, which is often priced at about half the standard rate.
At higher volumes, consider committed spending agreements or provisioned capacity with providers, which can reduce unit prices in exchange for predictable spending.
Matching Models to Tasks
Route simple requests to smaller, cheaper models and reserve the most capable models for tasks that need them. An LLM router can make this choice automatically for each request.
Re-evaluate model choices when providers release new models or change prices, since a newer, cheaper model may now meet the same quality bar.
For steady, high-volume workloads, compare API prices with the cost of hosting an open-weight model, converted to an equivalent cost per token.
Examples of Cost per Token in IT Financial Management
The examples below show cost per token applied to real decisions.
Both use illustrative prices. The first lowers the effective cost per token for an existing workload, and the second shows why the cheapest token is not always the cheapest task.
In each, the analysis looked beyond list prices to what the organization actually paid for useful work.
Lowering an Effective Rate
A company spends $40,000 a month on a model priced at $3 per million input tokens and $15 per million output tokens. Input tokens account for $24,000 of the bill and output tokens for $16,000, and its effective cost is close to list price, since it uses no caching or batching.
It caches its long system prompt, which makes up 60% of input tokens and is then billed at a tenth of the normal rate, and moves nightly report generation, 30% of its volume, to batch processing at half price.
Monthly spending falls to about $23,000 with no change in the model or the work done, reducing its effective cost per token by more than 40%.
Cheap Tokens, Expensive Tasks
Another company compares two models for extracting data from invoices. Model A costs $0.50 per million input tokens and Model B $3, and each invoice uses about 5,000 input tokens. On price alone, Model A looks six times cheaper.
Testing shows Model A extracts invoices correctly 78% of the time, while Model B succeeds 97% of the time. Each failure costs about $2 in staff time to fix by hand.
Including corrections, the cost per invoice is about $0.44 with Model A and about $0.08 with Model B, so the company chooses the more expensive model per token.
Conclusion
Cost per token is the price paid for each token a model processes, usually quoted per million tokens with separate input and output rates. The effective price paid can be much lower than the list price once caching, batching and negotiated discounts are applied.
In IT financial management, cost per token is the starting point for budgeting, forecasting and comparing AI models, but sound decisions rest on the cost of completed work at an acceptable quality. Using caching and batch pricing, matching models to tasks, revisiting choices as prices change and comparing models on cost per task help organizations pay the right price for AI.
Related terms
AI Tokens
AI tokens are the small units of text, such as words or parts of words, that language models read and write, and the unit most AI services use to measure and charge for usage.
Input and Output Tokens
Input tokens are the tokens sent to a language model in a request, and output tokens are the tokens it generates in response. Most AI services price output tokens several times higher.
Cost per Task
Cost per task is the full cost of completing one unit of AI-assisted work, such as a resolved support request or a processed document, including every model call, tool and retry involved.
Prompt Caching
Prompt caching is a technique in which an AI provider stores the processed form of repeated prompt content, such as standard instructions, so later requests can reuse it at lower cost and latency.