Module 2 of 4Lesson 5 of 5~12 min
Checking and iterating with discernment
A polished AI answer can still be wrong, and it is often the PM who pays for it in front of the team or a customer. This lesson gives you a way to judge an AI response and to fix your prompt in a targeted way, so each attempt improves on the last instead of going in circles.
Lesson objective
By the end of this lesson, you will be able to evaluate an AI output on three levels (content, process, behavior) against explicit criteria, and fix the prompt with a targeted change rather than a random rewording.
Topics covered
- evaluating AI output
- prompt iteration
- fact-checking AI
- critical thinking with AI
- AI discernment
Where it fits
Delegate, describe, verify
How do you get a reliable result from generative AI, and know when not to trust it?
Lessons in this module
- Choosing what to hand over to AI
- Understanding what an LLM does when you prompt it
- Describing the task in a structured prompt
- Four techniques that change the result
- Checking and iterating with discernment (this lesson)
What you will learn in the course
This lesson is part of the course Prompting for Product Managers: delegate, describe, verify
- Decide, for a task in your product cycle, whether to hand it to AI, in which mode (automation, augmentation, agent), and what you keep.
- Explain how an LLM behaves (prediction, dated knowledge, working memory, instruction following) to anticipate its mistakes and pick the right fix.
- Write a prompt that describes the deliverable, the process and the quality bar, with context, delimited data and examples when needed.
- Evaluate an AI output (content, process, behavior) against explicit criteria, then improve it through targeted iterations.
- Produce PM deliverables with AI (feedback synthesis, persona, interview guide, user stories) from real data, and check that they are faithful to it.
- Configure standing instructions and a dedicated assistant for a recurring task, test it on real cases, and decide when to move to an agent.
- Apply diligence rules (shared data, transparency, accountability) before using or circulating work produced with AI.
Related courses
- Build reliable Claude Skills for your product workAdvanced · ~2 hr 30 min
- Choosing and using AI tools at workAll levels · ~1 hr 40 min
- Understand what LLMs do well, and where they failAll levels · ~2 hr