AI Tokens

AI tokens are the basic units that large language models work with. Before a model can process a prompt, the text is split into tokens by a tokenizer, and the model then generates its answer one token at a time. A token is often a whole common word, but longer or rarer words are split into several tokens, and spaces, punctuation and numbers are tokens too. As a rough rule for English text, a token is about four characters, or around three quarters of a word.

Tokens matter far beyond the mechanics of the model, because they are also the unit of cost. Most AI services price their models per million tokens, with separate rates for the tokens sent to the model and the tokens it produces. Every prompt, document, conversation history and answer is counted in tokens. For IT financial management, tokens are to AI what compute hours are to cloud: the basic quantity that drives the bill, and the place to start when explaining or reducing AI costs.

Definition of AI Tokens

An AI token is a unit of text, or of other data such as image patches or audio segments, that a model processes as a single element. Each model family has its own tokenizer, so the same text can produce a different number of tokens on different models.

Usage is usually measured in two directions. Input and output tokens are counted separately, because generating output requires more computation than reading input and is priced higher.

How Text Becomes Tokens

Tokenizers break text into pieces drawn from a fixed vocabulary, typically tens of thousands to a few hundred thousand entries. Common words in widely used languages are usually single tokens, while unusual words, names and technical terms are split into several.

Languages other than English often need more tokens for the same meaning, because tokenizer vocabularies are weighted toward the languages most common in training data. The same request can therefore cost more in some languages than in others.

Code, numbers and structured data such as JSON can also be token-heavy, because symbols, spaces and long identifiers are each counted. This matters for applications that send large documents or data to a model.

Tokens Beyond Text

Models that accept images, audio or video convert these inputs into tokens as well. An image might be counted as hundreds or thousands of tokens depending on its size and the detail level requested.

Some models also generate internal reasoning tokens while working through a problem. These are usually not shown in the answer but are billed as output.

In each case the principle is the same: the more tokens a request involves, the more computation it needs and the more it costs.

Importance of AI Tokens in IT Financial Management

Tokens are the main driver of spending on AI services priced by usage. Understanding them is the first step in forecasting, allocating and optimizing that spending.

Token counts also explain why AI costs vary so widely between applications. A short chat reply might use a few hundred tokens, while an agent that reads long documents and makes several model calls might use hundreds of thousands for a single task.

Because each request is billed separately, small differences in tokens per request become large differences in monthly cost once usage reaches scale.

Tokens as a Cost Driver

The cost of a model call is roughly the number of input tokens multiplied by the input price, plus the number of output tokens multiplied by the output price. Cost per token varies widely between models, often by a factor of ten or more.

Total spending is therefore driven by four things: the number of requests, the input tokens per request, the output tokens per request and the model chosen. Each is a lever that can be measured and managed.

These drivers make tokens a natural basis for driver-based planning of AI budgets, in the same way that compute hours or storage volumes drive cloud plans.

Tokens and Allocation

Because tokens are counted for every request, they provide a precise basis for allocating AI costs to the teams, products and customers that generate them.

This requires tagging or metering requests with the information needed to allocate them, such as the application, feature or user. Without that metadata, a shared API account shows only a total.

Token-level data also supports showback, helping teams see how their design choices affect cost.