A large language model (LLM) is a transformer-based neural network trained on massive amounts of text to predict the next token in a sequence — and, through that single training objective at sufficient scale, it ends up able to converse, summarize, translate, write code, and reason to a useful degree.
What Makes It "Large"
"Large" refers to two things together: the size of the training data (typically hundreds of billions to trillions of tokens of text) and the number of learnable parameters in the model (see LLM Parameters) — commonly billions. There's no official threshold that separates a "language model" from a "large language model"; it's a relative, evolving term, not a precise technical boundary.
What an LLM Actually Learns
Purely from next-token prediction across huge amounts of text, an LLM implicitly picks up:
- Grammar and fluent writing style
- Factual knowledge present in its training data (as of its training cutoff — see LLM Limitations)
- Patterns of reasoning and problem-solving seen in text (code, math explanations, arguments)
- Formatting conventions (lists, code blocks, tables) it saw during training
None of this is explicitly programmed — it emerges from the scale of the data and the training objective. This is also why LLM behavior can be surprising or inconsistent: it's a statistical pattern-matcher over its training data, not a hand-coded rule engine.
LLM vs Earlier NLP Models
| Earlier NLP Models (pre-2018) | LLMs | |
|---|---|---|
| Typical size | Millions of parameters | Billions of parameters |
| Task scope | Trained for one specific task (e.g. sentiment classification) | One model handles many tasks via prompting |
| Architecture | RNNs, LSTMs, or small transformers | Large transformer, usually decoder-only (see Decoder-Only Transformers) |
Practical Use Case
The same base LLM, given different prompts, can summarize a document, draft an email, explain a legal clause, or generate SQL — one general-purpose model instead of a separate purpose-built model per task, which is the core practical reason LLMs became so widely adopted so quickly.
Common Mistakes
- Assuming "large" implies a specific, fixed parameter count — it's a relative term that has shifted over time as models have grown
- Treating an LLM as a search engine or database — it generates plausible text based on patterns, and can be confidently wrong (see LLM Hallucinations)
Interview Relevance
Q: "What is an LLM, in one sentence?" — a strong answer mentions the transformer architecture, the scale of training data/parameters, and the next-token-prediction training objective, not just "an AI that writes text."
Practice Question
Explain, without using the word "AI," what an LLM is trained to do and how that single objective leads to abilities like summarization and translation.
