Module 3 of 5Lesson 2 of 3~18 min
Manage cost per request and per outcome
The bill for an AI feature can climb without anyone noticing before the end of the month. This lesson teaches you to calculate what a request and a useful result really cost, identify the most effective levers for bringing that cost down and set budget alerts that warn you in time.
Lesson objective
By the end of this lesson, you will be able to calculate the cost per request and the cost per useful outcome of an AI feature, identify its main reduction levers (length, caching, model, unnecessary steps) and set budget alerts.
Topics covered
- LLM costs
- tokens
- cost per request
- AI budget
- cost optimization
Where it fits
Measure quality, cost and latency
How do you know, every week, whether the feature answers well, what it costs and whether it is fast enough?
Lessons in this module
- Track quality in production
- Manage cost per request and per outcome (this lesson)
- Control perceived latency
What you will learn in this module
This lesson is part of the course Ship and monitor an AI feature in production
- Set up quality signals in production (user feedback, sampled human review, online evaluations, drift) and connect them to the evaluation set.
- Manage cost per request and per useful outcome, and perceived latency (budgets, alerts, caching, streaming, response length, model choice).
Related courses
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Build an AI assistant for your product
3 hr
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Design a RAG architecture that fits your product
3 hr 30 min
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Evaluate an AI feature: test sets, metrics and LLM judges
3 hr