Hallucinations, sycophancy, variability, the three traps to anticipate

Lesson objective

By the end of this lesson, you will be able to spot the passages of an AI answer that are exposed to hallucinations, recognise sycophantic behaviour, take run-to-run variability into account, and choose for each case a check that does not rely on the model's own opinion.

Where it fits

How an LLM produces an answer

Where does an AI answer come from, and why can it be wrong while sounding right?

Lessons in this module

1. An LLM predicts how a text continues, and that explains almost everything
2. Hallucinations, sycophancy, variability, the three traps to anticipate (this lesson)

What you will learn in the course

This lesson is part of the course Understand what LLMs do well, and where they fail

• Explain how an LLM produces an answer (tokens, next-token prediction, pretraining then post-training) and why a fluent answer is not a verified answer.
• Anticipate hallucinations, sycophancy, poorly calibrated confidence and run-to-run variability, and choose the right check for each case.
• Locate what the model knows (training data, knowledge cutoff, uneven coverage) and what web search, provided documents and connectors change.
• Explain the context window as working memory (limit, degradation over long contexts, no memory across conversations without a dedicated feature) and derive working rules from it.
• Assess what reasoning (thinking) and reading images or documents bring, and what they do not guarantee.
• Sort the tasks of your job into "delegate / verify / keep" based on the cost of an error, how easy it is to verify, the model property involved and the value of doing it yourself.

Related courses

• Build reliable Claude Skills for your product workAdvanced · ~2 hr 30 min
• Choosing and using AI tools at workAll levels · ~1 hr 40 min
• Prompting for Product Managers: delegate, describe, verifyJunior · ~2 hr 45 min