What is a token in an LLM? Cost and limits
A token is the unit of text a language model reads and writes: a short word, a fragment of a longer one, a comma, a period or a space. Text is converted into tokens before it enters the model, and the answer is produced one token at a time. Tokenization depends on the model, so the same sentence does not count the same number of tokens across vendors.
Definition written by Alex Dionisio, Senior Product Manager. Updated November 13, 2026.
Why it matters for a PM
Tokens are the unit of account for almost everything about an LLM. Vendors bill per token, usually pricing input and output separately. The context window is measured in tokens. Response time grows with the number of tokens generated. Estimating the tokens of a typical request lets a PM put a cost on a feature and spot the design choices that make it balloon.
Example
A team plans automatic summaries of meeting notes. It estimates the tokens in an average set of notes, the system prompt and the generated summary, then multiplies by the number of meetings per month. The math shows that also sending the full project history would multiply the cost by five, so the team sends a short project summary instead.
Key points
- In English, a token averages a bit less than a word; other languages, such as French, usually need more.
- Output tokens generally cost more than input tokens, and they take longer to produce.
- A request's token count includes the system prompt, the history, the documents and the tool results.
- Vendors provide ways to count a text's tokens before sending it.
Common mistakes
- Costing a feature by words or by requests instead of by tokens.
- Forgetting that the conversation history is sent again on every turn, which drives up the cost.
- Comparing two vendors' prices without accounting for their different tokenizers.
Go further with Module
The courses and lessons that cover this concept:
Guides on this topic
- RAG vs fine-tuning: how to choose
What each approach solves, the decision criteria and how to evaluate, with a decision grid.
- Evaluating an AI feature (evals)
Test cases, criteria, LLM-as-a-judge, release thresholds and monitoring, with an evals plan.