Embeddings

An embedding is a list of numbers that represents an item in a learned vector space. Texts with similar meaning are often placed nearer to one another than unrelated texts, according to the model and similarity function being used. Applications use embeddings for semantic search, clustering, recommendations, duplicate detection, and retrieval-augmented generation.

An embedding is not a readable summary, a database key, or a guaranteed statement of meaning. It is a task-shaped numerical representation. The same sentence can receive different vectors from different models, and “near” only becomes useful after you define and test a retrieval task.

A small mental model

Imagine five support notes:

1. “I was charged twice this month.”
2. “My card shows a duplicate payment.”
3. “I cannot reset my password.”
4. “The login link has expired.”
5. “How do I export a report?”

A useful text-embedding model would likely place items 1 and 2 relatively close, and items 3 and 4 relatively close. Item 5 should be farther from both groups. The space may have hundreds or thousands of dimensions rather than the two dimensions in a diagram, but the retrieval idea is the same.

Sentence-BERT is an influential example of producing sentence representations that can be compared efficiently with similarity measures. Modern embedding systems vary widely, so treat the model identifier, preprocessing, and distance calculation as versioned parts of your index.

Similarity is a calculation

Common comparisons include cosine similarity, dot product, and Euclidean distance. They are not interchangeable in every setup. Follow the model’s documented recommendation and confirm how your vector database orders results.

Cosine similarity compares direction:

cosine(a, b) = dot(a, b) / (length(a) × length(b))

A higher cosine value usually means more similar direction. That does not establish factual agreement. “The device is safe outdoors” and “The device is not safe outdoors” share many terms and may appear close. Retrieval needs representative tests, metadata filters, and sometimes a reranker.

From documents to vector search

A typical pipeline is:

1. Split documents into meaningful chunks.
2. Preserve title, section, version, URL, permissions, and other metadata.
3. Generate an embedding for each chunk.
4. Store the vector with the chunk and metadata.
5. Embed a user query with the compatible model.
6. Find nearby vectors and apply filters or reranking.
7. Return passages, not vectors, to the user or generator.

Do not mix vectors from incompatible embedding models in one search space. When changing models, build a new index, evaluate it, and switch deliberately. Keep a content hash so unchanged chunks can be recognized.

Chunking changes what can be found

Embedding an entire manual can blur distinct topics. Embedding every sentence can separate a rule from its exception. Prefer chunks that preserve one coherent unit and its qualifiers. Store headings with the chunk when they disambiguate the text.

For a table, a useful chunk may include column names and one or more complete rows. For versioned policy, the effective date and version belong in metadata and often in the text representation. Similarity search cannot enforce access control or version validity unless your application applies those rules.

A retrieval diagnostic

Create at least twenty real questions and name the passages that should be returned. For each query, record:

• whether a required passage appears in the first k results;
• the rank of the first required passage;
• irrelevant passages in the top results;
• whether metadata filters removed a needed source;
• whether a lexical term, number, or product code was missed;
• whether two opposing passages were confused.

If semantic retrieval misses exact identifiers, combine it with keyword search. If the right passage appears at rank 18, test reranking. If it never appears, inspect ingestion and chunking before changing the answer prompt.

What embeddings do not solve

• They do not verify that a passage is current or authoritative.
• They do not guarantee that nearby text supports a claim.
• They do not apply user permissions by themselves.
• They do not make sensitive text anonymous; the original content and derived data still need protection.
• They do not remove the need for exact search when codes, names, dates, or quoted phrases matter.

The useful definition is therefore operational: an embedding is a learned vector representation that makes certain similarity comparisons efficient. Its quality is not an abstract property. It is measured by whether the right items are retrieved for the questions, corpus, filters, and failure costs in your application.