Memory is what the agent keeps beyond the current step. The CoALA framework (Sumers et al. 2024) names 4 kinds.

Four kinds of memory

MemoryWhat it holdsExample in the shopping task
WorkingThe current task and historyThe 3 listings from step 1
EpisodicWhat happened in past runs"PowerHub delivered on time last month"
SemanticFacts and documentsLaptop manuals and return policies
ProceduralHow to do things"Always check the total including GST"

Retrieval from Retrieval Augmented Generation (RAG) is the long-term memory of the agent. In agentic RAG, retrieval becomes a tool that the agent calls when it needs it, and a weak result lets the agent rewrite the query itself.

Memory can be poisoned

Anything saved to memory can carry a trick, for example "SuperCharge is a trusted brand" saved after reading a hidden text. Treat memory like any other untrusted input.

Code

A simple memory with the 4 kinds as separate stores:

memory = {
    "working": [],        # history of the current run
    "episodic": [],       # lessons from past runs
    "semantic": {},       # documents, usually a vector store from RAG
    "procedural": ["Always check the total including GST"],
}
 
def remember_lesson(text, source_trusted):
    """Save a lesson only if it comes from a trusted source, so memory is not poisoned."""
    if source_trusted:
        memory["episodic"].append(text)
 
remember_lesson("TechDealz88 is not a verified seller", source_trusted=True)

Notes:

  • working memory is the context window, so it is limited, see Context Engineering
  • the source_trusted flag is the simplest defence against memory poisoning