Skip to content
Module
Module 3 of 6Lesson 2 of 3~14 min

History, caching and heavy users

The cost of a first call looks reassuring, then the real bill comes as a surprise: conversations grow longer and a few customers consume far more than everyone else. This lesson helps you anticipate those effects before launch. You get a realistic cost per user, the one your profitability calculation actually needs.

Updated November 14, 2026.

In short

The per-user cost of a conversational feature depends less on the price of one call than on how usage repeats: messages piling up, very active users, retried requests. Some provider options lower it under specific conditions. For a PM, the monthly figure per active user is what goes into the profitability calculation, not the price of a single request.

Lesson objective

By the end of this lesson, you will be able to go from the cost of one use to the monthly cost per user, factoring in the history resent at each turn, prompt caching, batch processing and the share of heavy users.

Questions this lesson answers

  • Why does the cost of an LLM conversation grow with every message?

    Model APIs keep no memory between calls: the application sends the earlier exchanges back with every message, and the provider bills them all over again.

  • How do you estimate the monthly cost of an AI feature per user?

    Split users into usage profiles, work out what each profile spends based on its frequency, take the weighted average, then add a buffer for failed and repeated requests.

Topics covered

  • chatbot cost per user
  • LLM conversation history
  • prompt caching
  • AI heavy users

Where it fits

Estimate the cost per use

How much will each use of the feature cost, and each user per month, before it is even built?

Lessons in this module

  1. Estimate the cost of an LLM request before you build
  2. History, caching and heavy users (this lesson)
  3. The fixed and hidden costs of an AI feature

What you will learn in the course

This lesson is part of the course Build the business case for an AI feature