What is latency? Definition and impact on UX
Latency is the delay between the moment a request is sent and the moment the response starts arriving or is complete. It builds up along the whole path: network, server, database, third-party services and, for AI features, model generation. It is measured in milliseconds or seconds, and usually reported as percentiles such as p95 rather than as an average.
Definition written by Alex Dionisio, Senior Product Manager. Updated November 13, 2026.
Why it matters for a PM
Perceived speed is part of product quality: a slow search or an assistant that takes too long to answer makes people give up. Averages hide the slowest cases, which are exactly the ones users remember. The PM sets realistic latency targets for each journey and chooses UX levers such as progressive display when the delay itself cannot be cut.
Example
An assistant built into legal software answers in four seconds on average, but one user in twenty waits more than fifteen. The team streams the answer as it is generated, caches the most consulted documents and reserves the heaviest model for complex questions. The p95 drops sharply.
Key points
- Percentiles (p50, p95, p99) describe the experience better than the average.
- For an LLM, time to first token weighs heavily on perceived speed.
- Total latency is the sum of its steps: you need to know which one dominates.
- Caching, background processing and progressive display are common levers.
Common mistakes
- Tracking only average latency.
- Measuring from the office network rather than under users' real conditions.
- Optimizing a step that accounts for only a small share of the total delay.
Go further with Module
The courses and lessons that cover this concept:
Guides on this topic
- Write a spec for an AI coding agent
The sections of a spec a coding agent can execute without guessing, with a SPEC.md template.
- Claude Code for Product Managers
What a PM can do with Claude Code, the method and the guardrails, with a starter CLAUDE.md.
- Evaluating an AI feature (evals)
Test cases, criteria, LLM-as-a-judge, release thresholds and monitoring, with an evals plan.