Query, Key, and Value are the three vectors self-attention computes for every token — the actual mathematical mechanism behind the "which tokens matter to which" weighting described conceptually in Attention.
An Analogy Before the Math
Think of a library search: your Query is what you're looking for. Every book has a Key — a short descriptor of what it's about. You compare your query against every book's key to find good matches, then actually read the Value — the book's real content — weighted by how good each match was. Self-attention does exactly this, but for every token comparing against every other token, simultaneously.
How Q, K, V Are Produced
For each token's input representation (a vector, x):
Query = x × W_Q (a learned weight matrix)
Key = x × W_K (a different learned weight matrix)
Value = x × W_V (a third learned weight matrix)
W_Q, W_K, W_V are learned during training — the model
discovers what makes a useful "query," "key," and "value"
representation purely from optimizing prediction accuracy.The Attention Calculation
For a given token's Query, against every token's Key:
score = Query · Key (dot product — measures similarity)
scaled_score = score / sqrt(key_dimension) (keeps values stable)
attention_weight = softmax(scaled_scores) (normalizes to sum to 1)
output = sum(attention_weight × corresponding Value, for all tokens)This is the "scaled dot-product attention" formula from the original transformer paper. The softmax step is what turns raw similarity scores into a clean probability-like distribution — tokens with higher relevance get proportionally more weight in the final blended output.
Why the Scaling Step Matters
Without dividing by sqrt(key_dimension), dot-product scores can grow very large as vector dimensions increase, pushing softmax into a regime where it produces extremely peaked (nearly one-hot) outputs — which can hurt training stability. The scaling keeps the values in a more numerically well-behaved range.
Practical Use Case
Understanding Q/K/V is what's needed to actually read transformer research papers or implement attention from scratch — for most application-building work (prompting, RAG, fine-tuning), this level of detail isn't required day-to-day, but it demystifies what's happening "under the hood" and is common ground in more technical interviews.
Common Mistakes
- Assuming Query, Key, and Value are three separate pieces of input data — they're all derived from the same token representation, just via three different learned transformations
- Skipping the scaling step when explaining the formula — it's a small but specifically-motivated detail, not an arbitrary addition
Interview Relevance
"Walk me through the scaled dot-product attention formula" is a common question in more technical/research-oriented AI engineering interviews — being able to state the formula and explain why each step exists (not just recite it) is what distinguishes a strong answer.
Practice Question
Explain, in plain language, what would likely go wrong with attention if the scaling step (dividing by sqrt(key_dimension)) were removed.
