Overview

  • this note is about how a navigating agent stores what it has seen, so it can plan
  • three families, from least to most structured: unstructured memory, topological memory (a graph) and spatial memory (a grid map)
  • each family comes with an example: Scene Memory Transformer, graph-based navigation, and Bayesian state tracking

Environment representation is how a navigating agent stores what it has seen, so it can plan where to go. There are three families, from least structured to most structured: unstructured memory, topological memory and spatial memory.

Unstructured memory

Keep everything, and maybe compress it.

  • the agent stores its past observations as a list, without building a map
  • this is the main regime today
  • examples: Scene Memory Transformer for embodied agents in long-horizon tasks, and Habitat

In the Scene Memory Transformer:

  1. each input observation (image, pose and previous action ) is turned into an embedding
  2. the embeddings are stored in a scene memory
  3. a policy network with a Transformer encoder encodes the memory, and a decoder reads it for the current step
  4. a fully connected layer outputs the action

Topological memory

Build a graph of key nodes.

  • each node is a place the agent has seen, and each edge says that two places are connected
  • examples: Semi-parametric Topological Memory for Navigation, A Behavioral Approach to Visual Navigation with Graph Localization Networks, and Graph Attention Memory for Visual Navigation

With a graph, navigation has three steps:

  1. localization: find which nodes match the current observation and the goal observation
  2. planning: find a path through the graph from the current node to the goal node
  3. waypoint selection: pick the next node on the path as the short-term target

Spatial memory (cognitive map)

A spatially indexed matrix. The memory is a grid over the floor, and each cell stores information about that place.

  • examples: Chasing Ghosts (instruction following as Bayesian state tracking) and Cognitive Mapping and Planning for Visual Navigation

In the Bayesian state tracking example:

  • the agent keeps a confidence and belief map about the world from the previous time step (, )
  • egomotion (how the agent moved, for example a 90 degree turn) is used to warp the old map so it lines up with the new position
  • an encoder (ResNet-50) and decoder read the new image
  • the warped old map and the new information are combined into an updated confidence and belief map (, )

Note

These representations are usually learned in simulation first, then moved to real robots, see Sim2Real Transfer.