History, caching and heavy users
The cost of a first call looks reassuring, then the real bill comes as a surprise: conversations grow longer and a few customers consume far more than everyone else. This lesson helps you anticipate those effects before launch. You get a realistic cost per user, the one your profitability calculation actually needs.
Updated November 14, 2026.
In short
The per-user cost of a conversational feature depends less on the price of one call than on how usage repeats: messages piling up, very active users, retried requests. Some provider options lower it under specific conditions. For a PM, the monthly figure per active user is what goes into the profitability calculation, not the price of a single request.
Lesson objective
By the end of this lesson, you will be able to go from the cost of one use to the monthly cost per user, factoring in the history resent at each turn, prompt caching, batch processing and the share of heavy users.
Questions this lesson answers
Why does the cost of an LLM conversation grow with every message?
Model APIs keep no memory between calls: the application sends the earlier exchanges back with every message, and the provider bills them all over again.
How do you estimate the monthly cost of an AI feature per user?
Split users into usage profiles, work out what each profile spends based on its frequency, take the weighted average, then add a buffer for failed and repeated requests.
Topics covered
- chatbot cost per user
- LLM conversation history
- prompt caching
- AI heavy users
Where it fits
Estimate the cost per use
How much will each use of the feature cost, and each user per month, before it is even built?
Lessons in this module
- Estimate the cost of an LLM request before you build
- History, caching and heavy users (this lesson)
- The fixed and hidden costs of an AI feature
What you will learn in the course
This lesson is part of the course Build the business case for an AI feature
Related courses
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Design a RAG architecture that fits your product
3 hr 30 min
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Evaluate an AI feature: test sets, metrics and LLM judges
3 hr
- AI & ProductExpertAdvanced level: for deeper mastery and specialization.
Ship and monitor an AI feature in production
3 hr