Overview
- this note is about how a navigating agent stores what it has seen, so it can plan
- three families, from least to most structured: unstructured memory, topological memory (a graph) and spatial memory (a grid map)
- each family comes with an example: Scene Memory Transformer, graph-based navigation, and Bayesian state tracking
Environment representation is how a navigating agent stores what it has seen, so it can plan where to go. There are three families, from least structured to most structured: unstructured memory, topological memory and spatial memory.
Unstructured memory
Keep everything, and maybe compress it.
- the agent stores its past observations as a list, without building a map
- this is the main regime today
- examples: Scene Memory Transformer for embodied agents in long-horizon tasks, and Habitat

In the Scene Memory Transformer:
- each input observation (image, pose and previous action ) is turned into an embedding
- the embeddings are stored in a scene memory
- a policy network with a Transformer encoder encodes the memory, and a decoder reads it for the current step
- a fully connected layer outputs the action
Topological memory
Build a graph of key nodes.
- each node is a place the agent has seen, and each edge says that two places are connected
- examples: Semi-parametric Topological Memory for Navigation, A Behavioral Approach to Visual Navigation with Graph Localization Networks, and Graph Attention Memory for Visual Navigation

With a graph, navigation has three steps:
- localization: find which nodes match the current observation and the goal observation
- planning: find a path through the graph from the current node to the goal node
- waypoint selection: pick the next node on the path as the short-term target
Spatial memory (cognitive map)
A spatially indexed matrix. The memory is a grid over the floor, and each cell stores information about that place.
- examples: Chasing Ghosts (instruction following as Bayesian state tracking) and Cognitive Mapping and Planning for Visual Navigation

In the Bayesian state tracking example:
- the agent keeps a confidence and belief map about the world from the previous time step (, )
- egomotion (how the agent moved, for example a 90 degree turn) is used to warp the old map so it lines up with the new position
- an encoder (ResNet-50) and decoder read the new image
- the warped old map and the new information are combined into an updated confidence and belief map (, )
Note
These representations are usually learned in simulation first, then moved to real robots, see Sim2Real Transfer.