Instrument a product and use data to make decisions
Instrument an existing product as a team (tracking plan, identity, quality, CNIL compliance), analyse funnels and cohorts, and bring a data-backed decision.
Instrument an existing product as a team, then use the data to settle a real product decision. You learn to choose what to measure (North Star, metrics tree, guardrails), write a tracking plan developers can implement (events, properties, naming, identity, accounts), QA and monitor data collection, comply with the GDPR and the French CNIL rules, read a funnel and retention cohorts and choose a tool. Then you turn an analysis into a reasoned decision without falling for vanity metrics, Simpson's paradox or confusing correlation with causation. You finish with a tracking plan, a dashboard and a decision memo for your product.
What you will be able to do
- Build a product's metrics tree (North Star, input metrics, guardrails) from the decisions to be made, and rule out vanity metrics.
- Design an implementable tracking plan (events, properties, naming convention, identity and accounts, emitting source, owners).
- QA and monitor event collection to get reliable data, and know what to do when it is not.
- Collect usage data in line with the GDPR and the CNIL rules (consent, audience measurement exemption, minimisation, retention period).
- Analyse a conversion funnel, activation and cohort retention, and interpret the resulting curves correctly.
- Choose a product analytics tool (PostHog, Amplitude, Mixpanel, GA4, data warehouse) based on needs, constraints and cost.
- Turn an analysis into a reasoned decision (dashboard, decision memo) while avoiding interpretation traps (vanity, Simpson, correlation, small counts).
Prerequisites
- Have already worked on a digital product in production, with an engineering team
- Be able to read a chart and compute a percentage; SQL helps but is not required
- Recommended course before this one: Model your product's data and query it with SQL, to fetch numbers from the data warehouse yourself.
- This course covers instrumenting an existing product as a team. Measuring a side project launch and designing an A/B test are covered in other Module courses.
Syllabus
What will I be able to do by the end of this course, and in what order?
- Course overview3 steps
Objective · By the end of this overview, you will know what you are going to produce (a tracking plan, a dashboard and a decision memo for your product), which decision Atelio will settle throughout the course and which five modules take you there.
Out of everything I could measure, what actually helps the team decide?
Objective · By the end of this lesson, you will be able to propose a product's North Star, break it down into a tree of three to five input metrics a team can act on, and choose the guardrail metrics that prevent it from being improved at customers' expense.
Objective · By the end of this lesson, you will be able to start from a decision to be made to write the questions and metrics that inform it, tell an actionable metric from a vanity metric, and propose a definition of activation that can be checked against the data.
Which events should I collect, how should I name them and who should they be attached to, so that the data answers my questions?
Objective · By the end of this lesson, you will be able to describe a flow as events and properties, choose the right level of detail, and write a naming convention the whole team applies.
Objective · By the end of this lesson, you will be able to specify when to identify a user, how to link their anonymous and logged-in events, how to attach events to a client account in B2B, and which events to send from the server rather than from the browser or the app.
Objective · By the end of this lesson, you will be able to write a complete tracking plan (one row per event with the question served, trigger, properties, source, identity, owner and status) and write acceptance criteria in tickets that developers and testers can check.
How do I know my numbers are right, and that I collect them by the rules?
Objective · By the end of this lesson, you will be able to QA an event before release, set up simple collection monitoring (volumes, ratios, reconciliation with the database) and diagnose a data anomaly before drawing a product conclusion from it.
Objective · By the end of this lesson, you will be able to say, for each tool and each event in a tracking plan, whether it needs the user's consent or may fall under the audience measurement exemption, remove unnecessary personal data and set a retention period, before validating everything with your DPO.
Where do users drop off, who comes back, and which tool shows it?
- Build and read a funnel4 steps
Objective · By the end of this lesson, you will be able to build a funnel (steps, order, unit counted, conversion window, entry population), spot the step with the largest drop-off and break it down by segment before concluding anything.
Objective · By the end of this lesson, you will be able to build and read a cohort retention table (activity definition, calculation mode, unit, period), say whether usage levels off, and compare cohorts or segments without mistaking their difference for a causal effect.
Objective · By the end of this lesson, you will be able to compare the main families of tools (product analytics, web audience measurement, warehouse-native analytics) and four common tools (PostHog, Amplitude, Mixpanel, GA4) with a criteria grid, and recommend a justified choice for a given product.
How do I go from a chart to a decision the team can follow, without drawing the wrong conclusion?
Objective · By the end of this lesson, you will be able to spot a Simpson's paradox, a confounding variable, a seasonal effect or a small-numbers effect in an analysis, rephrase the conclusion with appropriate caution, and recognise when only a controlled experiment allows a conclusion.
Objective · By the end of this lesson, you will be able to design a dashboard of five to seven charts tied to questions and definitions, and write a one-page decision memo that states the decision, the data, the alternatives, the limits and the re-evaluation criterion.
Objective · By the end of this lesson, you will have assembled, on your own product, a tracking plan, a dashboard and a decision memo, self-assessed them with a rubric, and you will leave with an exit kit and a 7-day and 30-day application plan.