Skip to content
Module
Tech

What is latency? Definition and impact on UX

Latency is the delay between the moment a request is sent and the moment the response starts arriving or is complete. It builds up along the whole path: network, server, database, third-party services and, for AI features, model generation. It is measured in milliseconds or seconds, and usually reported as percentiles such as p95 rather than as an average.

Definition written by Alex Dionisio, Senior Product Manager. Updated November 13, 2026.

Why it matters for a PM

Perceived speed is part of product quality: a slow search or an assistant that takes too long to answer makes people give up. Averages hide the slowest cases, which are exactly the ones users remember. The PM sets realistic latency targets for each journey and chooses UX levers such as progressive display when the delay itself cannot be cut.

Example

An assistant built into legal software answers in four seconds on average, but one user in twenty waits more than fifteen. The team streams the answer as it is generated, caches the most consulted documents and reserves the heaviest model for complex questions. The p95 drops sharply.

Key points

  • Percentiles (p50, p95, p99) describe the experience better than the average.
  • For an LLM, time to first token weighs heavily on perceived speed.
  • Total latency is the sum of its steps: you need to know which one dominates.
  • Caching, background processing and progressive display are common levers.

Common mistakes

  • Tracking only average latency.
  • Measuring from the office network rather than under users' real conditions.
  • Optimizing a step that accounts for only a small share of the total delay.

Go further with Module

The courses and lessons that cover this concept:

Guides on this topic