Recognize AI-specific incidents
An AI feature can fail in ways that classic software never does, from a silent drop in quality to a model retired by the provider. This lesson helps you recognize these incidents, rank them by severity and write the runbook that tells the team exactly what to do for each one.
Lesson objective
By the end of this lesson, you will be able to recognize the incidents specific to an AI feature (quality drift, provider unavailability, model deprecation, call loop, cost spike, misuse), classify them by severity and write the runbook that says what to do for each.
Topics covered
- AI incidents
- runbook
- model deprecation
- incident management
Where it fits
Incidents, fallbacks and model changes
What do you do when quality, the provider or the model shifts under your feet?
Lessons in this module
- Recognize AI-specific incidents (this lesson)
- Plan the fallbacks
- Change models without regressions
What you will learn in this module
This lesson is part of the course Ship and monitor an AI feature in production
- Diagnose AI-specific incidents (drift, provider outage, deprecation, call loop, cost spike) and plan the right fallbacks.
- Run a model or prompt change with regression evaluations, shadow comparison, staged rollout and a rollback plan.
Related courses
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Build an AI assistant for your product
3 hr
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Design a RAG architecture that fits your product
3 hr 30 min
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Evaluate an AI feature: test sets, metrics and LLM judges
3 hr