How Similarity Search Works in Vector Databases

A user asks: "what is the penalty for late payment?" The system has 200,000 document chunks stored in a vector database. Somewhere in there is section 4.3 of the terms of service: "Overdue balances incur a 1.5 percent monthly fee." The query and the document share one word. They use completely different phrasing to describe the same concept.

Similarity search finds that document. Not because it matched keywords. Because the query vector and the document vector are geometrically close in the high-dimensional embedding space where both were placed when the embedding model processed them.

Understanding how that geometric retrieval works is what this article covers: query vectorization, ANN candidate retrieval, distance calculation, ranking, and top-K selection. Those decisions determine whether the nearest vectors are found quickly and ordered correctly.

This article is part of the How Vector Databases Work Internally series. If you are new to the topic, start with what vector search is and then what a vector database is. This page stops when similarity search has produced its ranked top-K vector IDs. The vector query lifecycle article owns everything that surrounds that operation, including request parsing, filters, segment and shard merging, payload hydration, serialization, latency, and observability. The distance metrics are covered in depth in the cosine similarity vs Euclidean distance article.

What Similarity Search Is Doing Geometrically

Before covering the mechanics, the geometric intuition is worth establishing precisely.

Every embedding model maps text, images, or audio to a point in a high-dimensional vector space. The training process arranges those points so that items with similar meaning land close together and items with different meaning land far apart. When you search by similarity, you are asking: "find the stored points closest to this query point."

import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

# Three stored documents
stored = {
    "doc_a": "Late payments incur a monthly fee of 1.5 percent.",
    "doc_b": "Shipping takes 3 to 5 business days.",
    "doc_c": "You may return items within 30 days of purchase.",
}

# One query
query = "what is the penalty for late payment"

# Embed everything
stored_vecs = {k: model.encode(v, normalize_embeddings=True) for k, v in stored.items()}
query_vec   = model.encode(query, normalize_embeddings=True)

# Compute cosine similarity (dot product after L2 normalization)
scores = {k: float(np.dot(query_vec, v)) for k, v in stored_vecs.items()}
ranked = sorted(scores.items(), key=lambda x: x[1], reverse=True)

print("Similarity scores:")
for doc_id, score in ranked:
    print(f"  {doc_id}: {score:.4f}  '{stored[doc_id][:50]}'")

# Output:
# Similarity scores:
#   doc_a: 0.7841  'Late payments incur a monthly fee of 1.5 percent.'
#   doc_c: 0.3102  'You may return items within 30 days of purchase.'
#   doc_b: 0.1043  'Shipping takes 3 to 5 business days.'

"doc_a" scores 0.78 against the query even though "penalty" does not appear in the document. The embedding model learned during training that "late payment fee" and "penalty for late payment" occur in overlapping semantic contexts, so they land in similar regions of the embedding space. The math is just a dot product. The intelligence is in the learned geometry.

According to Couchbase's vector similarity guide, the index structure plays a crucial role in the similarity search process by guiding the search to relevant regions of the high-dimensional space, which helps narrow down the number of vector comparisons required.

The Two Phases of Similarity Search

Every production similarity search runs in two phases: retrieval and ranking.

Retrieval is the ANN phase. The index structure navigates to a candidate set of vectors that are approximately nearest to the query. This phase prioritizes speed. The goal is to return a superset of the true nearest neighbors without comparing against the entire collection.

Ranking is the scoring phase. Once candidates are retrieved, they are scored precisely by the distance metric, sorted, and reduced to the requested top-K. This phase prioritizes accurate ordering.

Phase 1: Retrieval (ANN search)
  Input:  query vector, k=10
  Output: candidate set of ~50 to 200 vector IDs (oversampled)
  Cost:   O(log n) comparisons using HNSW graph traversal
  Goal:   high recall - the true top-10 should be in this set

Phase 2: Ranking (exact scoring on candidates)
  Input:  candidate set of 50 to 200 vectors + query vector
  Output: sorted list of (id, score) pairs
  Cost:   O(candidates × dimensions) - fast, candidates << n
  Goal:   high precision - correct ordering of the candidate set

The separation between these phases is what makes similarity search at scale viable. Phase 1 discards 99.9 percent of the collection efficiently using the ANN index. Phase 2 applies exact scoring only to the small remaining candidate set.

Step 1: Query Vectorization

Before the search can begin, the query must be in the same vector space as the indexed documents. That means converting the raw text query to a float array using the same embedding model used during indexing.

import openai
import numpy as np

oai = openai.OpenAI(api_key="your-key")

def embed_query(text: str, model: str = "text-embedding-3-small") -> np.ndarray:
    """
    Convert a text query to a normalized embedding vector.
    Normalization to unit length converts cosine similarity to a dot product,
    which is faster and numerically more stable.
    """
    response = oai.embeddings.create(input=text, model=model)
    vec = np.array(response.data[0].embedding, dtype=np.float32)
    return vec / np.linalg.norm(vec)    # L2 normalize

query_vec = embed_query("what is the penalty for late payment")
print(f"Query vector shape: {query_vec.shape}")      # (1536,)
print(f"Query vector norm:  {np.linalg.norm(query_vec):.6f}")  # 1.000000

This step has two critical constraints. First, the embedding model must be identical to the model used during document indexing. Mixing models produces nonsensical similarity scores because different models learn different vector spaces with different geometries. Second, normalization matters: if documents were indexed as normalized vectors and the query is not normalized, cosine similarity scores will be incorrect.

Query vectorization happens before ANN traversal and must use the same model and preprocessing rules as the stored vectors. The query lifecycle guide explains how this step contributes to the full request path.

Step 2: ANN Index Traversal

With the query vector ready, the database sends it to the ANN index. The specific traversal depends on the index type. HNSW traversal is the most common in production systems.

HNSW Traversal Step by Step

The HNSW graph has multiple layers. Each layer is a graph where nodes (vectors) are connected to their nearest neighbors. Higher layers have fewer nodes and longer-range connections. Layer 0 is the base layer containing all vectors.

HNSW traversal for query vector q:

Layer 2 (coarse navigation):
  Enter at entry point node E
  Compute distance(E, q) = 0.42
  Check all neighbors of E: [A, F, G]
  distance(A, q) = 0.31 → move to A
  Check all neighbors of A: [B, E, H]
  No neighbor closer than A → descend to layer 1

Layer 1 (medium navigation):
  Enter at A
  Check neighbors of A: [B, C, E, D]
  distance(B, q) = 0.24 → move to B
  Check neighbors of B: [A, C, X, Y]
  distance(C, q) = 0.19 → move to C
  No neighbor closer than C → descend to layer 0

Layer 0 (precise local search):
  Enter at C
  Expand search to all neighbors and their neighbors
  using ef_search=64 candidate set
  Return top-K from explored candidates

The key parameter is ef_search (also written ef in some libraries), which controls how many candidates are tracked during the layer-0 search. A higher ef_search explores more of the local graph neighborhood, improving recall at the cost of more distance computations.

import faiss
import numpy as np

# Create HNSW index with ef_search tuning
d = 384   # dimension for all-MiniLM-L6-v2
index = faiss.IndexHNSWFlat(d, 16)     # M=16 connections per node
index.hnsw.efConstruction = 64         # quality during build
index.hnsw.efSearch = 64              # candidates during search (tune at query time)

# Add vectors
corpus = np.random.randn(100_000, d).astype(np.float32)
faiss.normalize_L2(corpus)
index.add(corpus)

query = np.random.randn(1, d).astype(np.float32)
faiss.normalize_L2(query)

# Search with default ef_search=64
distances, indices = index.search(query, k=10)

# Increase ef_search for higher recall (at higher search cost)
index.hnsw.efSearch = 128
distances_hq, indices_hq = index.search(query, k=10)

print(f"Result overlap: {len(set(indices[0]) & set(indices_hq[0]))} / 10")
# Result overlap: 9 / 10 - ef_search=128 found one extra true neighbor

The overlap between the two result sets shows how recall changes as ef_search increases. A larger value explores more candidates and usually improves recall, but requires more distance computations. Measure the trade-off against exact nearest-neighbor results on your own dataset.

According to Pinecone's similarity search guide, to reduce the computation complexity added by an exhaustive search, ANN search provides a massive performance boost on similarity search when dealing with large datasets by retrieving a close approximation of the nearest neighbor rather than the exact one.

Step 3: Distance Computation

During ANN traversal, distance computations happen constantly as the algorithm evaluates neighbors. Understanding the specific computation that runs at this step is important because it determines both correctness and performance.

Cosine Similarity (After Normalization)

For normalized vectors (L2 norm = 1), cosine similarity reduces to a dot product. This is the most common configuration for text embeddings.

import numpy as np

def cosine_similarity_normalized(a: np.ndarray, b: np.ndarray) -> float:
    """
    For L2-normalized vectors, cosine similarity = dot product.
    This is significantly faster than the full cosine formula because
    the norm computations (||a|| and ||b||) are both 1.0 and can be skipped.
    """
    return float(np.dot(a, b))

# Compare the two implementations on normalized vectors
a = np.random.randn(1536).astype(np.float32)
b = np.random.randn(1536).astype(np.float32)
a /= np.linalg.norm(a)
b /= np.linalg.norm(b)

full_cosine = np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
fast_dot    = np.dot(a, b)

print(f"Full cosine:  {full_cosine:.8f}")
print(f"Fast dot:     {fast_dot:.8f}")
print(f"Difference:   {abs(full_cosine - fast_dot):.2e}")
# Difference: 0.00e+00   - identical when vectors are normalized

Modern CPUs and GPUs implement dot products using SIMD (Single Instruction, Multiple Data) instructions that process multiple float values simultaneously. A single dot product on a 1536-dimensional float32 vector runs in a few microseconds on a modern CPU.

Euclidean Distance (L2)

For image embeddings and some audio models where vector magnitude carries signal, Euclidean distance is the correct metric.

def euclidean_distance(a: np.ndarray, b: np.ndarray) -> float:
    diff = a - b
    return float(np.sqrt(np.dot(diff, diff)))

# Euclidean distance is sensitive to magnitude
a_short = np.array([1.0, 0.0, 0.0])
a_long  = np.array([3.0, 0.0, 0.0])   # same direction, 3x longer
b       = np.array([0.9, 0.1, 0.0])

print(f"euclid(a_short, b) = {euclidean_distance(a_short, b):.4f}")  # 0.1414
print(f"euclid(a_long, b)  = {euclidean_distance(a_long, b):.4f}")   # 2.1024
print(f"cosine(a_short, b) = {cosine_similarity_normalized(a_short / np.linalg.norm(a_short), b / np.linalg.norm(b)):.4f}")  # 0.9950
print(f"cosine(a_long, b)  = {cosine_similarity_normalized(a_long / np.linalg.norm(a_long), b / np.linalg.norm(b)):.4f}")   # 0.9950

The output shows the critical difference: cosine similarity treats a_short and a_long as identical (they point in the same direction). Euclidean distance sees them as very different (they are at different positions in space). Choose the metric that matches the geometry of your embedding model.

The full mathematical comparison with geometric intuition is in the cosine similarity vs Euclidean distance article. The high-level takeaway: use cosine for text, use Euclidean for image and audio models where the embedding magnitude is meaningful.

Step 4: Rank Candidates and Select Top-K

ANN traversal produces a candidate set rather than the final answer. The search operation computes the configured distance or similarity for those candidates, orders them, and keeps the requested K nearest vectors.

def rank_top_k(query_vec, candidate_vectors, k=10):
    scored = [
        (vector_id, float(np.dot(query_vec, vector)))
        for vector_id, vector in candidate_vectors.items()
    ]
    return sorted(scored, key=lambda item: item[1], reverse=True)[:k]

For cosine similarity or dot product, higher scores rank first. For Euclidean distance, lower distances rank first. The result of similarity search is now complete: a ranked list of top-K vector IDs and their geometric scores.

What Happens After Similarity Search

The database still has work to do after the ranked candidates exist, but those operations belong to the query lifecycle rather than to the similarity calculation itself.

Similarity search
query vector → ANN candidates → exact distance scores → ranked top-K IDs
                                                        ↓
Query lifecycle
request parsing → filter strategy → segment and shard coordination → merge
                                      ↓
metadata evaluation → optional reranking → payload hydration → serialization

The vector query lifecycle guide explains segment and shard coordination, metadata filters, distributed merging, payload fetching, response serialization, latency attribution, and observability. Keeping those stages separate makes the boundary clear: similarity search finds and ranks nearby vectors; the lifecycle turns those candidates into a complete database response.

What the Similarity Score Tells You and What It Does Not

A similarity score of 0.82 from cosine search means the query vector and result vector are geometrically close in the embedding space. It does not necessarily mean the result is the best answer to the user's question.

The score is relative to a specific embedding model's learned geometry. A score of 0.82 from all-MiniLM-L6-v2 is not comparable to a score of 0.82 from text-embedding-3-large. The score is meaningful only within a single model's space and only in comparison to other scores from the same query.

Geometric proximity is not the same as factual correctness or answer quality. Similarity search can rank the closest vectors it was given, but it cannot guarantee that the highest-scoring item answers the user's question.

According to KX Systems' similarity search primer, similarity in a continuous vector space is not a binary match. The returned score measures geometric proximity under the configured metric; it should not be treated as a calibrated probability or confidence value.

Connecting Forward to the Technical Cluster Articles

Similarity search is the mathematical operation beneath many vector retrieval systems. The comparison of vector search vs semantic search explains how this operation differs from the broader goal of understanding and satisfying a user's search intent.

Each step in the similarity search pipeline described here is covered in depth in the dedicated cluster articles.

The distance metrics at step 3, cosine similarity and Euclidean distance, are covered mathematically with geometric intuition in the cosine similarity vs Euclidean distance article. That article covers when each is appropriate and how the choice affects result quality.

The ANN index traversal at step 2, specifically why approximate search is necessary and what accuracy it sacrifices, is the subject of the exact vs approximate nearest neighbor article. It covers how recall is measured and how to set your ANN parameters for a target recall threshold.

The HNSW graph traversal at step 2 is covered in full detail, with diagrams of the layered graph structure and the greedy navigation algorithm, in the HNSW algorithm article.

The alternative IVF cluster-based retrieval, which runs instead of HNSW for memory-constrained and very large-scale deployments, is covered in the IVF index article.

The vector indexing discipline as a whole, including how different index types are chosen and how index quality is measured, is covered in the vector indexing article.

The full request lifecycle from API call through all internal components to response serialization is covered in the vector query lifecycle article.

Summary

Similarity search in a vector database has a focused four-step boundary: convert the query into the indexed vector space, traverse an ANN index to collect candidates, calculate their distances or similarities, and return the ranked top-K vector IDs.

The main retrieval trade-off is between search cost and recall. Parameters such as HNSW's ef_search control how widely the index explores: larger candidate sets can recover more true neighbors but require more distance computations.

Accuracy is dominated by the quality of the embedding model and the chunking strategy used during indexing. If the query and the relevant document land in the same geometric neighborhood, similarity search will find it. If they land far apart because the model does not understand your domain or because the chunk is too large and its embedding is diffuse, no amount of ANN tuning will recover the result. The How Vector Databases Work Internally pillar covers the full architecture that surrounds this search pipeline.

Sources and Further Reading

1. Pinecone. What Is Similarity Search? pinecone.io/learn/what-is-similarity-search
2. Couchbase. What Is Vector Similarity Search? Benefits and Applications. couchbase.com/blog/vector-similarity-search
3. Instaclustr. What Is Vector Similarity Search? Pros, Cons, and 5 Tips. instaclustr.com/education/vector-database/what-is-vector-similarity-search-pros-cons-and-5-tips-for-success
4. Oracle. Similarity Search: Why AI Speaking in Vectors Is a Win for Users. oracle.com/database/ai-vector-search/similarity-search
5. Redis. What Is Vector Similarity? Metrics and Algorithms Explained. redis.io/blog/vector-similarity
6. KX Systems. How Vector Databases Search by Similarity: A Comprehensive Primer. medium.com/kx-systems/how-vector-databases-search-by-similarity
7. Weaviate. Vector Search Documentation. weaviate.io/developers/weaviate/search/similarity
8. Milvus. Similarity Metrics Documentation. milvus.io/docs/metric.md
9. FAISS. Getting Started Documentation. faiss.ai/index