Vector Representations

A vector representation encodes meaning as position in a high-dimensional space — each number in the vector is one coordinate, and the overall pattern of numbers captures something about the text's semantic content, learned entirely from training data rather than manually designed.

An Intuitive 2D Analogy (Real Embeddings Have Hundreds+ Dimensions)

Imagine plotting words on a simple 2D map:

           formal
             |
   "regards" •  • "sincerely"
             |
casual ------+------
             |  • "thanks!"
   "hey"   • |
             |
          informal

Words with similar formality end up near each other. Real embeddings work the same conceptual way, but across hundreds or thousands of dimensions simultaneously — capturing not just one axis like "formality," but many nuanced aspects of meaning at once, none of which are individually labeled or human-interpretable.

What Each Individual Number Means

Unlike the 2D example above, no single dimension in a real embedding vector has a clean, human-readable label like "formality" or "topic." The meaning is distributed across all dimensions together — this is a direct consequence of how embedding models are trained (learning to place similar texts close together, with no instruction about what any individual dimension should represent).

Distance = Similarity

vector("happy")  = [0.8, 0.2, -0.1, ...]
vector("joyful") = [0.75, 0.25, -0.15, ...]   ← close to "happy"
vector("angry")  = [-0.6, 0.7, 0.3, ...]      ← far from "happy"

The actual measurement of "how close" is typically cosine similarity — the specific mathematical technique for comparing two vectors' directions.

Practical Use Case

Understanding vectors as points in a meaning-space (not a lookup table or a set of labeled categories) explains why embedding-based search generalizes well to phrasings it's never seen before — it's comparing positions in a learned space, not matching against a fixed list of known patterns.

Common Mistakes

• Expecting to interpret individual dimensions of a real embedding vector as meaningful on their own — meaning is distributed across all dimensions jointly, not localized to any single one
• Treating embeddings as a form of exact keyword matching rather than a continuous, graded measure of similarity

Interview Relevance

"Can you point to what a specific dimension in an embedding vector 'means'?" — no, not in isolation; meaning is distributed jointly across all dimensions, a good check for whether a candidate actually understands embeddings vs just having memorized the term.

Practice Question

Explain, using the vector-space analogy, why "I love this product" and "This product is amazing" would be expected to have high similarity despite no shared exact vocabulary.

Related Notes

• Cosine Similarity
• Embedding Dimensions
• Embeddings in Transformers