Hallucinations, sycophancy, variability, the three traps to anticipate
An AI answer can be confidently wrong, agree with you by reflex or change from one try to the next. This lesson teaches you to recognize these three behaviors in your own use and to pick, for each one, a way to check that does not depend on the model judging itself.
Lesson objective
By the end of this lesson, you will be able to spot the passages of an AI answer that are exposed to hallucinations, recognize sycophantic behavior, take run-to-run variability into account, and choose for each case a check that does not rely on the model's own opinion.
Topics covered
- AI hallucinations
- LLM sycophancy
- answer variability
- checking an AI answer
In the glossary
Where it fits
How an LLM produces an answer
Where does an AI answer come from, and why can it be wrong while sounding right?
Lessons in this module
- An LLM predicts how a text continues, and that explains almost everything
- Hallucinations, sycophancy, variability, the three traps to anticipate (this lesson)
What you will learn in this module
This lesson is part of the course Understand what LLMs do well, and where they fail
- Explain how an LLM produces an answer (tokens, next-token prediction, pretraining then post-training) and why a fluent answer is not a verified answer.
- Anticipate hallucinations, sycophancy, poorly calibrated confidence and run-to-run variability, and choose the right check for each case.
Related courses
- AI & ProductivityAdvancedRegular practice: you already use the tools or work on this topic.
Build reliable Claude Skills for your product work
2 hr 30 min
- AI & FundamentalsAll levelsNo prerequisites: accessible to everyone.
Choosing and using AI tools at work
1 hr 40 min
- AI & FundamentalsJuniorSome basics are useful: introductory knowledge of the topic.
Prompting for Product Managers: delegate, describe, verify
2 hr 45 min