Retrieval decides what the LLM can see, so it is the right place to enforce who may see what, and which version of a document is current.

Metadata

Store metadata with every chunk:

  • source
  • version
  • effective date
  • department
  • classification

Pre-filtering and post-filtering

PointPre-filteringPost-filtering
WhenBefore the search, only chunks the user may see are searchedAfter the search, forbidden chunks are dropped from the top-
ResultCorrectFast, but can leave too few results
WeaknessCan slow ANN search when the filter is very selectiveThe top- may be mostly forbidden chunks

Rules

  • never rely on the prompt for access control, since an LLM told "do not reveal X" can be talked into revealing X
  • if X is never retrieved, it cannot leak
  • filter by effective date so that superseded policy versions are never retrieved

Access control belongs in the retriever, not in the prompt

Code

A pre-filter on a small collection: build a mask of allowed chunks and push the rest to the bottom of the ranking.

import numpy as np
from datetime import date
 
# one metadata dict per chunk
META = [
    {"dept": "compliance", "effective": date(2025, 1, 1), "superseded": False},
    {"dept": "hr",         "effective": date(2024, 6, 1), "superseded": True},
    # ...
]
 
def allowed(meta, user_depts, today):
    return (meta["dept"] in user_depts
            and meta["effective"] <= today
            and not meta["superseded"])
 
def filtered_dense_search(query, user_depts, k=5, today=date.today()):
    q = embed_query(query)
    mask = np.array([allowed(m, user_depts, today) for m in META])
    scores = np.where(mask, D @ q, -np.inf)       # forbidden chunks can never reach the top
    return list(np.argsort(-scores)[:k])

Notes:

  • -np.inf puts every forbidden chunk below all allowed ones
  • vector databases offer the same idea as a metadata filter argument on the search call

Once retrieval is built, it has to be measured, see Retrieval Metrics.