Specify validation, retries and fallback

Lesson objective

By the end of this lesson, you will be able to specify the three levels of output validation (syntax, schema, business rules), a retry rule and a fallback path suited to the cost of an error.

Where it fits

When the output breaks the contract

What does the product check, and what does it do when the output is invalid, refused or cut off?

Lessons in this module

1. Specify validation, retries and fallback (this lesson)
2. Plan for the cases where the guarantee does not hold

What you will learn in the course

This lesson is part of the course Get reliable, product-ready AI outputs (JSON, schemas, function calling)

• Diagnose where free-text output breaks a feature and decide which outputs require a contract.
• Choose between instructions alone, JSON mode, structured outputs and function calling for a given need, and justify the choice.
• Design an output schema (types, enums, required fields, nullable fields, descriptions, "unknown" values) where every field serves a consumer.
• Specify validation (syntax, schema, business rules), retries, handling of refusals and truncation, and the fallback path.
• Version and evolve an output contract without breaking its consumers, accounting for provider differences and the split of roles between PM and developers.
• Build a set of 20 robustness cases and set acceptance thresholds before going to production.

Related courses

• Build an AI assistant for your productAdvanced · ~3 hr
• Design a RAG architecture that fits your productAdvanced · ~3 hr 30 min
• Evaluate an AI feature: test sets, metrics and LLM judgesAdvanced · ~3 hr