Dense retrieval judges relevance by matching meaning. Each text becomes an embedding, a vector of numbers (for example 384) trained so that texts with similar meanings get nearby vectors. It is the idea of Word Embeddings, now used for whole passages.

Structure

  • retrieval becomes geometry: find the passage vectors that are closest to the question vector
  • a bi-encoder uses 2 transformer encoders, one for questions and one for passages, so and
  • relevance is the dot product
  • passages are encoded once, offline, so a query needs only 1 encoder pass plus a nearest-neighbour search

For Question A, "senior government official" and "politically exposed person" end up close together, so the PEP policy is found.

Cosine and dot product

  • cosine similarity measures the angle between 2 vectors, where 1 means the same direction and 0 means unrelated
  • the dot product also grows with the length of the vectors
  • after scaling every vector to length 1 (L2 normalisation), the dot product and the cosine are identical
  • for unit vectors the Euclidean distance gives the same ranking, since

Practical rules:

  • search with the same similarity that the model was trained with
  • never mix vectors from different embedding models
  • changing the embedding model means re-embedding the whole collection

Comparison with sparse retrieval

PointSparse retrievalDense retrieval
MatchesWordsMeaning
VectorWord counts, mostly zeroFull vector of about 384 numbers
Question A (paraphrase)Misses the PEP policyFinds it
Question B (exact code)Finds the exact entryConfuses KYC-E052 with KYC-E054, which read almost the same

Remembering tip

Sparse reads the words, dense reads the meaning. Each one fails where the other succeeds, so real systems use both.

Code

These are the 2 ways to make the passage vectors:

LSA embeddings (offline, no download): TF-IDF followed by truncated SVD, then scaled to length 1. LSA is a simple stand-in for a neural bi-encoder. It puts words that co-occur in nearby directions, which helps paraphrases, and it compresses away rare tokens, which hurts exact codes.

import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.decomposition import TruncatedSVD
from sklearn.preprocessing import normalize
 
tfidf = TfidfVectorizer(sublinear_tf=True, stop_words="english")
X = tfidf.fit_transform(DOCS)
svd = TruncatedSVD(n_components=16, random_state=0)       # compress to 16 dimensions
D = normalize(svd.fit_transform(X))                       # offline: one unit vector per passage
embed_query = lambda q: normalize(svd.transform(tfidf.transform([q])))[0]

Sentence-BERT (a neural bi-encoder, downloads about 90 MB, runs on CPU):

from sentence_transformers import SentenceTransformer
 
sbert = SentenceTransformer("all-MiniLM-L6-v2")
D = sbert.encode(DOCS, normalize_embeddings=True)                  # offline: one vector per passage
embed_query = lambda q: sbert.encode([q], normalize_embeddings=True)[0]

Either way the search is the same:

def dense_search(query, k=50):
    """Return the indices of the k passages whose vectors are closest to the query vector."""
    q = embed_query(query)                   # online: embed the question
    return list(np.argsort(-(D @ q))[:k])    # dot product of unit vectors equals cosine similarity

Notes:

  • normalize_embeddings=True makes D @ q a cosine similarity
  • the passage matrix D is built once, and only the question is embedded at query time
  • with LSA, the number of dimensions matters a lot for exact codes, since fewer dimensions throw away more rare tokens

A dense retriever is trained by contrast, see Contrastive Learning and InfoNCE Loss.