Governance covers how a RAG system is kept correct, deletable and traceable after it is built. In a regulated business these rules decide whether the system can ship.

Rules

  • right to erasure: under laws such as GDPR, and with related retention obligations in Singapore's PDPA, deleting a record must also remove its chunks, vectors, caches and logs
  • freshness: stale chunks are a compliance risk, so track source versions and re-index on change
  • re-embedding: a new embedding model means re-encoding the whole collection, so version the index and evaluate before switching
  • attribution: store the source and version of every chunk so each answer can be traced to its evidence

Audit trail

For every answer, log:

  • the query
  • the retrieved chunk IDs and their versions
  • the prompt
  • the answer

This makes each answer reproducible for audit.

Remembering tip

Erase, Refresh, Re-embed, Attribute. Every chunk needs a source, a version and a way to be deleted.

Code

A minimal audit record and a delete that touches every store:

import json, time
 
def log_answer(query, chunk_ids, versions, prompt, answer, path="audit.jsonl"):
    record = {"time": time.time(), "query": query, "chunks": chunk_ids,
              "versions": versions, "prompt": prompt, "answer": answer}
    with open(path, "a") as f:
        f.write(json.dumps(record) + "\n")      # one JSON line per answer
 
def erase_source(source_id, chunk_index, vector_store, cache, log_store):
    """Remove a record everywhere it can still be found."""
    for cid in chunk_index.pop(source_id, []):   # the chunks that came from this source
        vector_store.delete(cid)                 # vectors
        cache.pop(cid, None)                     # cached results
    log_store.redact(source_id)                  # logs

Notes:

  • vector_store, cache and log_store stand for the stores of a real system, so this is a sketch of the idea
  • keeping a map from each source to its chunk IDs is what makes erasure possible

This completes the RAG pipeline. In the next step retrieval becomes one tool inside an agent loop, see Agentic AI.