Tokenomics

Every answer from an AI model begins as physical work. Electricity powers a data centre. GPUs load and run a model. The provider measures the text that goes in and comes back out. The unit on that meter is a token.

Calling tokens the currency of AI is a useful shortcut. It captures the fact that they are the common unit used to buy model access. A token measures a piece of information that the model reads or writes. The electricity and computing power sit behind that meter, and different pieces can require very different amounts of work behind the scenes.

A token is a piece of text, not necessarily a word

Before a language model can process a sentence, a tokenizer breaks it into smaller units. A common word may stay whole. A longer or less familiar word may split into several pieces. Punctuation and spaces can count too. The exact split depends on the tokenizer used by that model.

For ordinary English, OpenAI gives a rough estimate of four characters, or about three quarters of a word, per token. That is useful for planning, but it is not a conversion rule. Languages, code, numbers and unusual formatting can produce a different token count, and two models may divide the same sentence differently.

Input and output are two different jobs

Input tokens include the prompt, system instructions, conversation history and any text retrieved from files or tools. The model processes that context in a stage called prefill. The work is highly parallel, so modern accelerators can handle many input tokens together.

Output tokens are the answer and, for some reasoning models, internal reasoning tokens that may not be shown. The model generates them sequentially. Each new token depends on everything that came before it, so a 500-token answer involves hundreds of linked decoding steps.

That sequential process is one reason output is usually priced more highly than input. Current standard price lists often charge roughly four to six times as much for output, although the ratio varies. Providers also price scarce low-latency capacity and long contexts differently, so the simple input and output rate is only the starting point.

The physical journey behind one response

1. 1.Your application sends text to the provider.
2. 2.A tokenizer converts the text into token IDs.
3. 3.Servers route the request to accelerators in a data centre.
4. 4.The accelerators process the input and build a working memory of the context.
5. 5.The model generates output tokens one at a time until it stops.
6. 6.The provider records input, cached input and output usage, then applies its price list.

Electricity is present throughout that journey. It powers the accelerators, CPUs, memory, networking and cooling. An NVIDIA DGX H100 system with eight H100 GPUs has a stated maximum system power use of about 10.2 kW. A production service may use many such systems, but it also shares them across many requests. This is why a universal figure for electricity per token would be misleading.

What makes one token more expensive to serve

The part of the model that runs

Larger dense models generally need more memory and computation for each token. Total parameter count is not enough to predict cost, because many modern models use a mixture-of-experts design. DeepSeek-V3, for example, has 671 billion parameters in total but activates 37 billion for each token. The inactive parameters still need to be stored and moved around the system, yet they do not all perform the calculation for every token.

The length of the context and answer

Long prompts require more prefill work and more memory for the model's running context. Long answers repeat the decoding step more times. Reasoning settings can add internal output tokens, while tool calls can bring new text back into the next prompt. A short visible answer can therefore sit at the end of a much larger token bill.

The hardware and serving software

Faster accelerators, lower-precision arithmetic, speculative decoding, better batching and efficient memory management can all reduce cost per token. Utilisation matters just as much. A costly GPU serving a full queue of well-batched work spreads its cost across more tokens than the same GPU waiting for occasional low-latency requests.

The service level

Immediate responses reserve capacity differently from background jobs. Google and Anthropic both offer discounted batch processing, while Google also offers cheaper flexible capacity and higher-priced priority serving. Prompt caching reduces repeated input work. These options change the price even when the model and token count stay the same.

Why model prices do not follow one neat rule

The provider's list price has to cover serving infrastructure, engineering, support and continued investment. It may also reflect available capacity, a temporary promotion, a push for market share, or a premium for speed and reliability. Training a frontier model is expensive, but the price of the next token is mainly connected to inference and the commercial service wrapped around it. Providers do not publish enough detail to separate those elements precisely.

A current price snapshot makes the point. Claude Fable 5 is listed at $10 per million input tokens and $50 per million output tokens. GPT-5.6 Luna is $0.20 and $1.20. DeepSeek V4 Flash has peak cache-miss rates of $0.44 and $1.32, with lower off-peak prices. These gaps are far larger than a single parameter-count explanation can support.

The scatter chart in the carousel plots input price against output price on logarithmic axes. Most models sit near a diagonal because output is consistently dearer, but the points do not form clean capability tiers. Price tells you what a provider charges for access. It does not tell you, on its own, how good the model is or what it costs the provider to run.

The useful number is cost per successful task

The cheapest token can still produce an expensive workflow if the model needs repeated attempts, writes too much, or fails the task. A more expensive model can cost less overall when it completes the job once with a shorter answer. The practical comparison is the cost of a representative task that meets your quality standard.

• Route routine work to the cheapest model that passes a real test.
• Set sensible limits for output length and reasoning effort.
• Cache instructions and reference material that repeat across requests.
• Use batch or flexible capacity for work that does not need an immediate answer.
• Measure retries, tool calls and successful outcomes alongside token totals.

Tokens make AI usage measurable, while the infrastructure and pricing behind them remain complex. Behind each one sits a chain of electricity, hardware, model architecture, serving software and commercial choices. Understanding that chain turns a vague API bill into something a business can design and control.