Migrations and Re-Embedding Drift

Where we left off

Lesson 8 warned about one kind of drift: editing text without re-embedding it. This lesson covers a subtler, easy-to-miss version of the same problem: changing how you embed, a new model, a new output_dimensionality, even a different task_type setting, and what happens to everything already stored.

The trap: same dimension, same column, silently incompatible vectors

old_model = GoogleGenerativeAIEmbeddings(
    model="models/gemini-embedding-001", output_dimensionality=768, task_type="RETRIEVAL_DOCUMENT"
)
new_model = GoogleGenerativeAIEmbeddings(
    model="models/gemini-embedding-001", output_dimensionality=768, task_type="SEMANTIC_SIMILARITY"
)

Both configurations produce a 768-number vector for the exact same text. Postgres has no way to notice anything changed, vector(768) accepts either one without complaint. But embedding the same sentence under both settings and comparing them directly reveals a cosine similarity of roughly 0.82 between them, not 1.0. They describe meaningfully different points in meaningfully different spaces. Store half your rows under the old configuration and half under the new one in the same column, and every search from then on is comparing vectors that were never meant to be compared, with no error, no warning, just steadily worse results.

The same trap applies to swapping models entirely, or dimensions

Anything that changes how an embedding is produced, a new model version, a different output_dimensionality truncation, a different task_type, a different provider, invalidates every vector stored under the old configuration. This includes situations that look harmless: bumping a model version string, a config change that seems unrelated to search quality.

The migration pattern: a versioned column, backfilled, then swapped

ALTER TABLE notes ADD COLUMN embedding_v2 vector(768);

1. Add the new embedding as a new column, don't overwrite the old one in place.
2. Backfill: re-embed every existing row's content with the new configuration, writing into embedding_v2, while embedding (the old column) keeps serving live searches, unaffected, undisturbed.
3. Verify: confirm embedding_v2 is populated for every row, and spot-check that searches against it return sensible results.
4. Cut over: point application queries at embedding_v2 (or rename columns), only once step 3 is confirmed.
5. Drop the old column, once nothing depends on it anymore.

with conn.cursor() as cur:
    cur.executemany(
        "UPDATE notes SET embedding_v2 = %s WHERE id = %s",
        [(Vector(vector), note_id) for vector, note_id in zip(new_vectors, ids)],
    )

This is the same shape as any zero-downtime schema migration, additive first, backfilled, verified, only then cut over, applied to the fact that an embedding is itself a kind of schema that can go stale.