The original 2017 transformer used two separate stacks — an encoder that processes the full input at once, and a decoder that generates output step by step, attending back to the encoder's representation. This architecture is still used today for specific tasks, though most modern general-purpose LLMs use a simpler decoder-only design instead.
How It Was Originally Used: Translation
Input (French): "Le chat est noir"
↓
Encoder: processes the entire input sentence at once,
building a rich representation of its full meaning
Decoder: generates the output one token at a time, attending
to (a) what it's generated so far, and (b) the
encoder's representation of the full input
Output (English): "The" → "cat" → "is" → "black"Two Kinds of Attention at Work
| Attention Type | Where | What It Does |
|---|---|---|
| Self-attention (encoder) | Within the encoder | Lets the input sentence relate its own tokens to each other |
| Self-attention (decoder) | Within the decoder | Lets the output-so-far relate to itself (causal/masked, see Self-Attention) |
| Cross-attention | Decoder attending to encoder | Lets each output token look back at the full input representation |
Where Encoder-Decoder Models Still Matter
Tasks with a clear, fixed source and a genuinely different target sequence — translation, and some summarization/structured-transformation tasks — still use encoder-decoder architectures. Encoder-only models (like BERT-style architectures) are separately used for understanding-focused tasks (classification, embeddings) where no generation is needed at all.
Why Most Modern Chat/Generation LLMs Don't Use This
General-purpose conversational and generation-focused LLMs mostly use decoder-only architectures instead — simpler (one stack, not two), and well suited to open-ended generation where there isn't a clean "fixed input, then generate output" split the way translation has. See Decoder-Only Transformers for why this became the dominant approach for general-purpose LLMs.
Common Mistakes
- Assuming all transformers are encoder-decoder — most modern general-purpose LLMs are decoder-only, a meaningfully simpler architecture
- Confusing cross-attention (decoder attending to encoder) with self-attention (a sequence attending to itself) — they serve different structural roles
Interview Relevance
"When would you still use an encoder-decoder architecture today?" — a good answer: tasks with a clear, distinct source-to-target transformation, like translation, where the encoder's job (understand the whole input) is genuinely different from the decoder's job (generate a related but distinct output).
Practice Question
Explain why a translation task benefits from cross-attention (decoder attending to the full encoder output), in terms of what the decoder needs to "see" while generating each output word.
