Module 4 of 6Lesson 3 of 3~12 min
Specify a RAG system with your team
RAG looks technical, yet most of its settings are product decisions that belong to the PM. This lesson helps you understand each step of a RAG pipeline, identify what you need to decide with your team and anticipate the ways it can fail in production.
Lesson objective
By the end of this lesson, you will be able to describe the steps of a RAG pipeline (ingestion, chunking, indexing, retrieval, cited generation), the product decisions in each and its failure modes.
Topics covered
- RAG
- retrieval augmented generation
- RAG pipeline
- AI document search
- AI assistant
In the glossary
Where it fits
Give it the right knowledge
How does the assistant get the right information at the right time?
Lessons in this module
- Manage context and conversation memory
- Choose how the assistant gets its knowledge
- Specify a RAG system with your team (this lesson)
What you will learn in the course
This lesson is part of the course Build an AI assistant for your product
- Identify a use case that justifies an AI assistant and write its framing brief (problem, users, allowed actions, out of scope, success criteria).
- Design the conversational experience: entry point, tone, handling uncertainty, handoff to a human and response format.
- Choose and justify a knowledge strategy (instructions, injected context, RAG, fine-tuning) and a conversation memory strategy.
- Specify the assistant's tools and actions (data read, actions written, confirmation, permissions) and choose how to build it.
- Identify the risks (injection, leaks, excessive actions, costs) and design layered guardrails that go beyond the prompt.
- Design an evaluation plan with a reference dataset, criteria, grading methods, release thresholds and a regression rule.
- Define production metrics (product, quality, cost, latency), alert thresholds and the loop from user feedback to the evaluation set.
Related courses
- Design a RAG architecture that fits your productAdvanced · ~3 hr 30 min
- Evaluate an AI feature: test sets, metrics and LLM judgesAdvanced · ~3 hr
- Ship and monitor an AI feature in productionExpert · ~3 hr