Module 2 of 4Lesson 2 of 5~13 min
Understanding what an LLM does when you prompt it
When an AI invents a figure or ignores an instruction, rerunning the request gets you nowhere until you know what caused the error. This lesson explains how a large language model works in terms a PM can use, so you can trace each failure to its cause and pick the right fix instead of rewording at random.
Lesson objective
By the end of this lesson, you will be able to trace an AI mistake back to its cause (prediction, knowledge, working memory, instruction following, sycophancy) and choose the matching fix instead of resending the same request.
Topics covered
- how LLMs work
- AI hallucinations
- context window
- LLM sycophancy
- AI errors
Where it fits
Delegate, describe, verify
How do you get a reliable result from generative AI, and know when not to trust it?
Lessons in this module
- Choosing what to hand over to AI
- Understanding what an LLM does when you prompt it (this lesson)
- Describing the task in a structured prompt
- Four techniques that change the result
- Checking and iterating with discernment
What you will learn in the course
This lesson is part of the course Prompting for Product Managers: delegate, describe, verify
- Decide, for a task in your product cycle, whether to hand it to AI, in which mode (automation, augmentation, agent), and what you keep.
- Explain how an LLM behaves (prediction, dated knowledge, working memory, instruction following) to anticipate its mistakes and pick the right fix.
- Write a prompt that describes the deliverable, the process and the quality bar, with context, delimited data and examples when needed.
- Evaluate an AI output (content, process, behavior) against explicit criteria, then improve it through targeted iterations.
- Produce PM deliverables with AI (feedback synthesis, persona, interview guide, user stories) from real data, and check that they are faithful to it.
- Configure standing instructions and a dedicated assistant for a recurring task, test it on real cases, and decide when to move to an agent.
- Apply diligence rules (shared data, transparency, accountability) before using or circulating work produced with AI.
Related courses
- Build reliable Claude Skills for your product workAdvanced · ~2 hr 30 min
- Choosing and using AI tools at workAll levels · ~1 hr 40 min
- Understand what LLMs do well, and where they failAll levels · ~2 hr