Almost every modern LLM is a decoder-only transformer — a stack of identical processing blocks, each combining self-attention and a feed-forward network, that transforms input tokens into a prediction for the next token.
The Pipeline, Top to Bottom
Input text: "The cat sat on the"
↓ tokenization
Token IDs: [1, 47, 892, 12, 5]
↓ embedding layer
Vectors (dense numeric representations of each token)
↓ + positional encoding (so the model knows word order)
↓ N stacked transformer blocks, each containing:
- self-attention (tokens weigh relevance of other tokens)
- feed-forward network (further transforms each token's representation)
↓ output layer
Probability distribution over the entire vocabulary for the next token
↓
Sampled next token: "mat" (or "floor", "chair", ranked by probability)This note covers how the pieces fit together at the LLM level. The attention mechanism itself — the part doing the real conceptual work — gets a full deep dive in Transformer Architecture and Self-Attention.
"Stacked Blocks" — What Scale Actually Means
A bigger model generally means more transformer blocks stacked (depth) and/or larger vectors within each block (width) — both increase the number of parameters (see LLM Parameters) and, generally, the model's capability, at the cost of more compute for both training and inference.
Why Decoder-Only (Not Encoder-Decoder)?
The original transformer (2017) used an encoder-decoder design for translation. Most modern general-purpose LLMs use a decoder-only design instead — simpler, and well suited to open-ended text generation where the model just keeps predicting the next token, rather than encoding one fixed input before decoding a separate output. See Encoder-Decoder and Decoder-Only Transformers for the full comparison.
Common Mistakes
- Assuming every generative AI model uses this exact architecture — image-generation diffusion models are structured very differently
- Confusing "bigger model" with strictly "smarter model" — architecture choices, training data quality, and training technique all matter alongside raw parameter count
Interview Relevance
"Describe the high-level architecture of a modern LLM" is a standard warm-up question — the pipeline above (tokenize → embed → transformer blocks → output distribution) is the expected shape of the answer.
Practice Question
Explain, in your own words, what changes about the pipeline above between processing the 1st token of a prompt and generating the 50th output token.
