Token-based pricing explained: how AI costs arise
A token is the smallest unit of text into which a language model splits text, usually a word or part of a word. With token-based pricing you pay for every token the model reads and writes. Input and output are usually priced separately, so costs grow with usage.
What a token is
Language models do not calculate with letters or whole sentences but with tokens. A short, common word is often a single token, while a long compound technical term is split into several. How many tokens a text produces depends on the model and on the language. A word count is therefore not enough for a cost estimate. Only a measurement with the model you actually use will do.
How token-based pricing works
With every request the provider counts two quantities: the input tokens, meaning everything sent to the model, and the output tokens, meaning the answer it generates. Both are counted separately and are often priced differently. The monthly bill is requests times input tokens times input price, plus requests times output tokens times output price.
The underestimated item is context length. The input includes not only the question but everything the model needs to see for its answer: system instructions, the conversation so far and inserted documents. All of this is counted again with every request.
| Cost driver | Effect on the bill | Lever |
|---|---|---|
| Number of requests | grows linearly with usage | scope the use case narrowly |
| Context length of the input | is charged again with every request | send only relevant passages |
| Length of the output | counts per generated token | specify the answer format |
| Multi-step workflows | every intermediate step is a separate request | count the steps before you build |
| Model size | larger models usually cost more per token | choose the smallest suitable model |
How it differs from self hosting at fixed cost
With self hosting the model runs on your own or dedicated reserved hardware. There is no bill per token, but fixed costs for hardware, staff and operations, spread across all requests.
| Criterion | Token-based pricing | Self hosting at fixed cost |
|---|---|---|
| Cost structure | variable per token | fixed, independent of volume |
| Predictability | fluctuates with usage | known in advance |
| Suited to | low or highly fluctuating volume | high, stable volume |
| Where data stays | usually with the provider | in your own infrastructure |
| Lead time | immediate start | setup required |
How common both routes are is shown by the 2026 business survey of the ifo Institute, the Munich based economic research institute (ifo Konjunkturumfrage): 22.5 % of AI users rely exclusively on free systems, 18.7 % run their own AI systems, and the rest buy in from outside. Where the break even point lies cannot be stated in general terms. The detailed comparison is in our article on self hosting a language model versus renting one.
Practical example from our work
We see a typical pattern with knowledge assistants. An employee's question is short, yet several relevant passages from manuals are sent along with each question so that the answer is backed by evidence. The input is then often many times the size of the actual question. If usage grows from one department to several, the bill grows at the same rate. With AI in your own data centre this cost effect disappears, because the capacity is paid for at a fixed price.
