Module 2 of 6Lesson 3 of 3~12 min
Understand the anatomy of an LLM assistant
Behind a simple chat window sit several building blocks, and each one carries product decisions a PM needs to understand. This lesson helps you explain to your team what happens when a user sends a message to an LLM assistant, and identify where quality, cost and risk are decided.
Lesson objective
By the end of this lesson, you will be able to explain to a team what happens on every message (application, system prompt, history, context, tools, model) and to locate where each product decision is made.
Topics covered
- how LLMs work
- AI assistant
- system prompt
- context window
- assistant architecture
Where it fits
Frame the assistant
Does my product need an assistant, and to do what exactly?
Lessons in this module
- Identify a use case worth an assistant
- Write the framing brief
- Understand the anatomy of an LLM assistant (this lesson)
What you will learn in the course
This lesson is part of the course Build an AI assistant for your product
- Identify a use case that justifies an AI assistant and write its framing brief (problem, users, allowed actions, out of scope, success criteria).
- Design the conversational experience: entry point, tone, handling uncertainty, handoff to a human and response format.
- Choose and justify a knowledge strategy (instructions, injected context, RAG, fine-tuning) and a conversation memory strategy.
- Specify the assistant's tools and actions (data read, actions written, confirmation, permissions) and choose how to build it.
- Identify the risks (injection, leaks, excessive actions, costs) and design layered guardrails that go beyond the prompt.
- Design an evaluation plan with a reference dataset, criteria, grading methods, release thresholds and a regression rule.
- Define production metrics (product, quality, cost, latency), alert thresholds and the loop from user feedback to the evaluation set.
Related courses
- Design a RAG architecture that fits your productAdvanced · ~3 hr 30 min
- Evaluate an AI feature: test sets, metrics and LLM judgesAdvanced · ~3 hr
- Ship and monitor an AI feature in productionExpert · ~3 hr