LLM (large language model)
An LLM (large language model) is a neural network trained on a very large body of text to predict, word after word, what is most likely to come next in a text. It powers assistants such as ChatGPT, Claude and Gemini, and most generative AI features built into products. It does not look facts up in a database: it generates a plausible answer from what it learned and from what it is given in the conversation.
Definition written by Alex Dionisio, Senior Product Manager. Updated October 26, 2026.
Why it matters for a PM
Almost every AI feature a PM specifies today runs on an LLM: summarising, extracting, classifying, drafting, chat assistants, agents. Knowing what the model really does helps you pick the right use cases, anticipate failures and ask engineering the right questions: which model, what cost per request, what latency, which data goes to which vendor, and how quality will be measured before launch.
Example
A team wants to sort support tickets into five categories automatically. An LLM handles this well with a clear instruction and a few examples per category. The same team then wants it to state the exact delivery date of an order. The model does not have that information: it needs access to the order system through a tool, or it will invent a believable date.
Key points
- An LLM generates likely text, not verified facts, which is where hallucinations come from.
- Its knowledge stops at the end of its training data, unless it is given search tools or documents.
- It only sees what fits in its context window: the prompt, the conversation history and the documents passed in.
- Two identical calls can return two different answers, so quality is measured on a set of cases, not one try.
- Choosing a model is a trade-off between quality, cost, latency and the vendor's data terms.
Common mistakes
- Treating it as a search engine or a reliable database.
- Judging a model on a polished demo instead of testing it on real cases, hard ones included.
- Sending personal or confidential data to a vendor without checking the contract and data retention terms.
Go further with Module
The courses and lessons that cover this concept: