How LLMs Generate Text
A large language model generates text by repeatedly predicting a probability distribution over the next token, selecting one token under a decoding rule, appending it to the context, and running the loop again.
You will be able to: Trace one complete next-token prediction and explain how temperature and sampling change the output.
Quick start
Reach one useful result first
- Explain
- 15 min
- Visualize
- 10 min
- Practice
- 25 min
- Deep dive
- 25 min
Explanation
Start with the direct answer, follow one example, then check your understanding.
Choose your explanation depth
The mechanism stays the same; emphasis and evidence change.
An LLM writes one token at a time by repeatedly choosing a likely continuation.
Carried example
After “The capital of France is”, Paris receives a high score, but the model still predicts a distribution rather than retrieving a fact.
Focus on the observable input, change, and output of next-token generation.
Learning goals and prerequisites
After this lesson
- Trace one autoregressive generation step
- Interpret next-token probabilities without treating them as facts
- Define observable stop and verification behavior
Helpful before starting
- Everyday reading and percentages; no Python or advanced mathematics required
5 chapters · about 1 hr 3 min estimated study
0/5 chapters marked complete
Direct answer
An LLM generates text by predicting one token at a time. At each step it scores the possible next tokens, a decoding rule selects one, and that token is appended to the context before the process repeats. The model is not retrieving a finished paragraph from storage.
Follow the mechanism
- The current text is converted into token IDs.
- The model turns those IDs into a logit, or raw score, for every token in its vocabulary.
- Softmax converts the logits into relative probabilities, and the decoding policy selects one candidate.
- The selected token becomes part of the next input. Generation ends only when a stop token, length limit, completed schema, or application rule says to stop.
Running example
Start with “The opposite of hot is”. Suppose the visible candidates are cold: 72%, warm: 18%, and <stop>: 10%. Greedy decoding selects “cold”, so the next context is “The opposite of hot is cold”. The model now computes a new distribution; it does not reuse the old one. If <stop> receives 81% at the next step, the application can stop with a complete answer. Changing the original context to “A mild day feels” would change the scores because the model is now answering a different continuation problem.
What this does not mean
A 72% next-token probability does not mean there is a 72% chance that the completed statement is factually true. It means “cold” is relatively likely under this model, this context, and this decoding setup. Facts still require evidence, calculation, a trusted tool, or review.
Ideas to understand
- Context → logits → probabilities → selected token → updated context is the full generation loop.
- Every selected token changes the input used for the next prediction.
- Token likelihood describes continuation behavior; it does not verify a claim.
- Stopping and validation are application responsibilities, not hidden model guarantees.
Decision and failure guide
Use How LLMs Generate Text with evidence
A complete explanation includes the conditions that make the technique useful and the nearby ways it can fail.
Before you call it ready
- Expose candidate scores
- Record context changes
- Define stop reasons
- Require independent support for factual claims
Common failure → better decision
- Saying the model retrieves a complete answer
- Trace the repeated next-token selection loop.
- Treating probability as truth
- Verify claims with evidence or tools.
- Ignoring application controls
- Specify validation, stop, and failure behavior.
Was this lesson helpful?
Submit to the team when server feedback is available; otherwise this browser keeps a local copy and tells you so.
Concept connections
See prerequisites, applications, and nearby decisions.
Often compared with
Continue next
Get one next lesson each week
A focused path reminder, not a marketing newsletter. Unsubscribe in one click.