Cosine Similarity

Cosine similarity is the standard way to measure how close two embedding vectors are — it measures the angle between them, not their raw distance, which is what makes it well-suited to comparing embeddings.

The Formula, Explained Simply

cosine_similarity(A, B) = (A · B) / (|A| × |B|)

Where:
  A · B  = dot product (multiply corresponding elements, sum them)
  |A|, |B| = the magnitude (length) of each vector

Result ranges from -1 to 1:
  1  → vectors point in the exact same direction (maximally similar)
  0  → vectors are unrelated (orthogonal)
  -1 → vectors point in opposite directions (maximally dissimilar)

Why "Angle" Instead of Raw Distance?

Cosine similarity ignores vector magnitude and focuses purely on direction — two vectors pointing the same way are considered maximally similar regardless of how "long" each one is. This matters because embedding vector magnitude isn't generally meaningful on its own the way direction is; a text repeated twice shouldn't necessarily be considered "more" of anything just because some representation of it might scale differently.

Worked Example (Simplified, 2D)

A = [1, 1]   (a simple 2D example, real embeddings have far more dimensions)
B = [2, 2]   (same direction as A, different magnitude)
C = [-1, -1] (opposite direction)

cosine_similarity(A, B) = 1.0   ← same direction = maximally similar,
                                   despite B being "longer"
cosine_similarity(A, C) = -1.0  ← opposite direction = maximally
                                   dissimilar

In Code

import numpy as np

def cosine_similarity(a, b):
    dot_product = np.dot(a, b)
    magnitude_a = np.linalg.norm(a)
    magnitude_b = np.linalg.norm(b)
    return dot_product / (magnitude_a * magnitude_b)

Practical Use Case

Every vector database and embedding-search system uses cosine similarity (or a closely related metric) as its core comparison operation — when you search a vector database for "most similar documents," this calculation (run efficiently at scale, see Vector Search) is what's happening under the hood.

Common Mistakes

• Confusing cosine similarity with Euclidean (straight-line) distance — they're related but distinct metrics, and most embedding-based systems specifically use cosine similarity
• Not normalizing vectors when a system expects normalized input, which can silently affect calculated similarity scores depending on the specific implementation

Interview Relevance

"Why is cosine similarity used for comparing embeddings instead of simple Euclidean distance?" — cosine similarity focuses on direction (semantic orientation) rather than magnitude, which better reflects meaning similarity independent of vector "length."

Practice Question

Given two embedding vectors with a cosine similarity of 0.92 and another pair with 0.15, explain what each result suggests about the underlying texts.

Related Notes

• Semantic Similarity
• Vector Representations
• Vector Search