Plan for the cases where the guarantee does not hold

Lesson objective

By the end of this lesson, you will be able to describe the documented cases where a structured output does not follow the schema (refusal, truncation, enum casing), how each provider signals them, and the behaviour expected from the product.

Where it fits

When the output breaks the contract

What does the product check, and what does it do when the output is invalid, refused or cut off?

Lessons in this module

1. Specify validation, retries and fallback
2. Plan for the cases where the guarantee does not hold (this lesson)

What you will learn in the course

This lesson is part of the course Get reliable, product-ready AI outputs (JSON, schemas, function calling)

• Diagnose where free-text output breaks a feature and decide which outputs require a contract.
• Choose between instructions alone, JSON mode, structured outputs and function calling for a given need, and justify the choice.
• Design an output schema (types, enums, required fields, nullable fields, descriptions, "unknown" values) where every field serves a consumer.
• Specify validation (syntax, schema, business rules), retries, handling of refusals and truncation, and the fallback path.
• Version and evolve an output contract without breaking its consumers, accounting for provider differences and the split of roles between PM and developers.
• Build a set of 20 robustness cases and set acceptance thresholds before going to production.

Related courses

• Build an AI assistant for your productAdvanced · ~3 hr
• Design a RAG architecture that fits your productAdvanced · ~3 hr 30 min
• Evaluate an AI feature: test sets, metrics and LLM judgesAdvanced · ~3 hr