What the model knows, since when, and what tools add

Lesson objective

By the end of this lesson, you will be able to say, for a given question, whether the model can answer from memory, whether you need to turn on web search or whether you must provide a document, and to choose the matching check.

Where it fits

What the model knows, and what it sees

What does the model know on its own, what do tools add, and what does it retain from our conversation?

Lessons in this module

1. What the model knows, since when, and what tools add (this lesson)
2. The context window, a powerful but uneven working memory

What you will learn in the course

This lesson is part of the course Understand what LLMs do well, and where they fail

• Explain how an LLM produces an answer (tokens, next-token prediction, pretraining then post-training) and why a fluent answer is not a verified answer.
• Anticipate hallucinations, sycophancy, poorly calibrated confidence and run-to-run variability, and choose the right check for each case.
• Locate what the model knows (training data, knowledge cutoff, uneven coverage) and what web search, provided documents and connectors change.
• Explain the context window as working memory (limit, degradation over long contexts, no memory across conversations without a dedicated feature) and derive working rules from it.
• Assess what reasoning (thinking) and reading images or documents bring, and what they do not guarantee.
• Sort the tasks of your job into "delegate / verify / keep" based on the cost of an error, how easy it is to verify, the model property involved and the value of doing it yourself.

Related courses

• Build reliable Claude Skills for your product workAdvanced · ~2 hr 30 min
• Choosing and using AI tools at workAll levels · ~1 hr 40 min
• Prompting for Product Managers: delegate, describe, verifyJunior · ~2 hr 45 min