Skip to content
Module
Module 6 of 6Lesson 1 of 3~15 min

Interpretation traps, from Simpson to causation

A convincing chart can lead to the wrong decision if nobody questions how the groups were formed. This lesson helps a PM recognize the most common interpretation biases, soften an overconfident conclusion, and know when only an experiment can settle the question. A valuable reflex in front of a hurried leadership committee.

Take this lesson: your first course is freeSee the full syllabus

Lesson objective

By the end of this lesson, you will be able to spot a Simpson's paradox, a confounding variable, a seasonal effect or a small-numbers effect in an analysis, rephrase the conclusion with appropriate caution, and recognize when only a controlled experiment allows a conclusion.

Questions this lesson answers

  • What is Simpson's paradox in product data analysis?
  • How do you tell correlation apart from causation in product data?
  • How do you avoid common interpretation traps in a product analysis?

Topics covered

  • Simpson's paradox
  • correlation vs causation
  • confounding variable
  • seasonality

Where it fits

Decide with data

How do I go from a chart to a decision the team can follow, without drawing the wrong conclusion?

Lessons in this module

  1. Interpretation traps, from Simpson to causation (this lesson)
  2. A dashboard that serves, a memo that decides
  3. Final workshop: tracking plan, dashboard and memo for your product

What you will learn in this module

This lesson is part of the course Instrument a product and use data to make decisions

  • Build a product's metrics tree (North Star, input metrics, guardrails) from the decisions to be made, and rule out vanity metrics.
  • Design an implementable tracking plan (events, properties, naming convention, identity and accounts, emitting source, owners).
  • Turn an analysis into a reasoned decision (dashboard, decision memo) while avoiding interpretation traps (vanity, Simpson, correlation, small counts).

Related courses