Module 2 of 5Lesson 2 of 2~18 min
Log and trace without exposing data
Without traces, you cannot understand why an answer went wrong, and with too many, you store data you should never have kept. This lesson helps you define what each model call should record and write a logging policy that respects personal data and states how long each field is kept.
Lesson objective
By the end of this lesson, you will be able to specify what a trace must contain for each model call, and to write a logging policy that states which fields to keep, mask or exclude, for what purpose and for how long.
Topics covered
- LLM tracing
- logging
- logging policy
- personal data
- observability
Where it fits
Launch without betting everything
How do you expose an AI feature to real users in stages, while being able to reconstruct every response?
Lessons in this module
- Launch in stages, with a kill switch
- Log and trace without exposing data (this lesson)
What you will learn in this module
This lesson is part of the course Ship and monitor an AI feature in production
- Design the staged launch of an AI feature (feature flag, shadow, canary, percentages) with quantified promotion criteria and a tested kill switch.
- Define what you log and trace for each model call, what you mask or exclude (personal data), how long you keep it, and with which tool.
Related courses
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Build an AI assistant for your product
3 hr
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Design a RAG architecture that fits your product
3 hr 30 min
- AI & ProductAdvancedRegular practice: you already use the tools or work on this topic.
Evaluate an AI feature: test sets, metrics and LLM judges
3 hr