Choosing a Distance Metric

Where we left off

Lesson 5 introduced <->, <#>, <=> and settled on <=> (cosine distance) as this course's default, because that's what Gemini's embeddings are trained for. This lesson shows why that choice matters, with a small worked example where the three operators actually disagree, then reconciles them.

Where cosine and inner product can disagree

Cosine distance cares only about a vector's direction; inner product (<#>) also cares about its length. That difference can flip a ranking: a vector pointing in almost the exact same direction as the query, but scaled down, can lose to a vector pointing somewhat further off but scaled up, under raw inner product, while cosine correctly prefers the one with the closer direction.

query = [1.0, 0.0]
close_direction_small = [0.1, 0.0]   # same direction as query, tiny magnitude
off_direction_large = [0.9, 0.436]   # further off-direction, large magnitude

Inserting exactly these two toy vectors and ranking them both ways shows the order actually flip: cosine correctly ranks close_direction_small first (it points in the exact same direction as the query), but raw inner product ranks off_direction_large first instead, purely because it's a longer vector.

Why this usually isn't a problem with real embeddings, but sometimes is

Embedding models are typically trained so that their outputs cluster around roughly one magnitude, and Gemini's is no exception, every real chunk from notes.txt comes back with almost the same vector length (roughly 0.58-0.59, not the dramatic toy-example difference above). That's why <#> often works fine in practice for a single embedding model's output. It stops being safe the moment you mix vectors from different sources, different models, differently truncated dimensions, or anything hand-constructed, exactly the toy example above.

Normalizing removes the ambiguity entirely

normalized = vector / np.linalg.norm(vector)

Once every vector has length exactly 1, cosine distance and (negative) inner product become mathematically identical rankings, only their absolute numbers differ. Some teams normalize at write time specifically so they can use <#> (typically the cheapest of the three operators to compute) with total confidence it agrees with cosine, without paying for the extra normalization work at every single query. This course keeps using <=> directly for simplicity, but knowing this trade exists matters once query volume gets large.