Source

Adapted from my own handwritten and hand-typed notes from Year 1 to Year 3 of my EEE degree

Task environments are described along five pairs of properties.

Definitions

PropertyMeaning
Accessible vs inaccessiblethe sensors give the agent the complete state of the environment
Deterministic vs nondeterministicthe next state is completely set by the current state and the agent's action
Episodic vs sequentialeach episode is not affected by actions taken before
Static vs dynamicthe environment does not change while the agent is deciding
Discrete vs continuousthere is a limited number of distinct percepts and actions

Examples

Driverless taxiChessMinesweeperSlot machine
AccessibleNo, some traffic info is missingYes, whole board visibleNo, mines hiddenNo
DeterministicNo, cars may turn suddenlyYesNo, mines placed randomlyNo
EpisodicNo, depends on earlier drivingNo, depends on earlier movesNoYes, each pull independent
StaticNo, other cars keep movingYes without a clock, semi with a clockYesYes
DiscreteNo, speed and distance are real valuedYesYesYes

Reading the Table

Points:

  • the real world (driverless taxi) is the hardest case: inaccessible, nondeterministic, sequential, dynamic, continuous
  • chess with a clock is semi-dynamic, the board does not change but the time score does
  • a slot machine is a rare episodic example, each play is independent of the last

When the environment is accessible and deterministic, the agent can plan a full action sequence in advance, which is Problem Formulation for Search.