An embedding is a list of numbers (a vector) that represents the meaning of a piece of text — generated so that texts with similar meaning end up with similar vectors. This single idea is the foundation of semantic search, RAG, recommendation systems, and clustering.
The Pipeline: Text → Embedding → Vector → Similarity → Retrieval
Text: "How do I reset my password?"
↓ embedding model
Vector: [0.021, -0.184, 0.093, ..., 0.056] (hundreds to
thousands of numbers)
↓ compare against other vectors (e.g. a knowledge base)
Similarity: this vector is close to the vector for "password
recovery steps" — even though the wording is different
↓
Retrieval: return the "password recovery steps" document as
the most relevant matchThis is the mechanism that makes semantic search possible — matching by meaning, not just overlapping keywords. See Semantic Similarity and RAG, which is built almost entirely on this pipeline.
A Concrete Example of "Similar Meaning, Similar Vector"
"How do I reset my password?"
"I forgot my password, what do I do?"
"Password recovery steps"
→ These three phrases share almost no exact words in common,
but a good embedding model places their vectors close
together in vector space, because they mean roughly the
same thing.
"How do I reset my password?"
"What's the weather today?"
→ These share more surface-level similarity in sentence
structure than the examples above, but their vectors would
be far apart — meaning, not wording, drives the vector.What This Section Covers
| Note | Focus |
|---|---|
| Text Embeddings | Practical usage — calling an embedding API |
| Embedding Models | How embedding models differ from LLMs, and from each other |
| Vector Representations | The technical underpinning — what a vector actually encodes |
| Semantic Similarity, Cosine Similarity | Measuring how close two vectors are |
| Embedding Dimensions | What vector size means practically |
| Embedding Search | Using embeddings to retrieve relevant content |
| Best Practices | Practical guidance for real applications |
Common Mistakes
- Confusing embeddings (used for search/similarity) with the internal token embeddings inside a transformer's architecture (see Embeddings in Transformers) — related idea, different scope and purpose
- Assuming any embedding model works equally well for any task — models are often optimized differently (e.g. for search vs classification vs clustering)
Interview Relevance
Q: "Explain what an embedding is and why it's useful, without heavy math." — the "similar meaning → similar vector" framing, with a concrete example like the password-reset one above, is a strong, intuitive answer.
Practice Question
Explain why "cheap flights to Paris" and "affordable airfare to France" would be expected to have similar embedding vectors, despite sharing almost no exact words.
