Module 4 of 5Lesson 3 of 3~18 min
Change models without regressions
Switching models, whether by choice or because the provider is retiring the old one, is a full release in its own right. This lesson helps you write the procedure that governs it, from regression testing to rollback, with a timeline aligned with the provider's deprecation dates.
Lesson objective
By the end of this lesson, you will be able to write the model change procedure for your AI feature: triggers, criterion-by-criterion regression evaluation, shadow comparison, staged rollout, rollback criteria and a timeline aligned with deprecation dates.
Topics covered
- model migration
- regression testing
- model deprecation
- shadow testing
In the glossary
Where it fits
Incidents, fallbacks and model changes
What do you do when quality, the provider or the model shifts under your feet?
Lessons in this module
- Recognize AI-specific incidents
- Plan the fallbacks
- Change models without regressions (this lesson)
What you will learn in this module
This lesson is part of the course Ship and monitor an AI feature in production
- Diagnose AI-specific incidents (drift, provider outage, deprecation, call loop, cost spike) and plan the right fallbacks.
- Run a model or prompt change with regression evaluations, shadow comparison, staged rollout and a rollback plan.
Related courses
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Build an AI assistant for your product
3 hr
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Design a RAG architecture that fits your product
3 hr 30 min
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Evaluate an AI feature: test sets, metrics and LLM judges
3 hr