Skip to content
Module
Module 3 of 5Lesson 2 of 3~14 min

Choose the randomization unit

Randomizing by user, account or session affects a test's validity as much as its power. The lesson helps you weigh a coherent experience for every user, the risk of interference between groups and the number of units available. A critical choice in B2B and marketplaces, best discussed early with the engineering team.

Take this lesson: your first course is freeSee the full syllabus

Lesson objective

By the end of this lesson, you will be able to choose the unit to randomize (user, account, session, business object, cluster) by weighing consistency of experience, interference and number of units, and to spot the B2B and marketplace cases where groups influence each other.

Questions this lesson answers

  • How do you choose the randomization unit for an A/B test?
  • What is interference between groups in an A/B test?
  • When should you randomize by cluster instead of by user?

Topics covered

  • randomization unit
  • interference
  • B2B A/B testing
  • marketplace
  • cluster randomization

In the glossary

Full glossary

Where it fits

Write the protocol

What must be settled before launching the test so that its result is readable and credible?

Lessons in this module

  1. Hypothesis, primary metric and guardrails
  2. Choose the randomization unit (this lesson)
  3. Compute the sample size and the duration

What you will learn in this module

This lesson is part of the course Design and analyze an A/B test

  • Write a testable hypothesis, choose a sensitive, attributable primary metric, guardrails and the exposure event to log.
  • Choose the randomization unit and prevent interference between groups, including in B2B.
  • Compute a sample size and a duration from the baseline rate, the minimum detectable effect, the significance level and the power.

Related courses