Source
Adapted from my own handwritten and hand-typed notes from Year 1 to Year 3 of my EEE degree
Task environments are described along five pairs of properties.
Definitions
| Property | Meaning |
|---|---|
| Accessible vs inaccessible | the sensors give the agent the complete state of the environment |
| Deterministic vs nondeterministic | the next state is completely set by the current state and the agent's action |
| Episodic vs sequential | each episode is not affected by actions taken before |
| Static vs dynamic | the environment does not change while the agent is deciding |
| Discrete vs continuous | there is a limited number of distinct percepts and actions |
Examples
| Driverless taxi | Chess | Minesweeper | Slot machine | |
|---|---|---|---|---|
| Accessible | No, some traffic info is missing | Yes, whole board visible | No, mines hidden | No |
| Deterministic | No, cars may turn suddenly | Yes | No, mines placed randomly | No |
| Episodic | No, depends on earlier driving | No, depends on earlier moves | No | Yes, each pull independent |
| Static | No, other cars keep moving | Yes without a clock, semi with a clock | Yes | Yes |
| Discrete | No, speed and distance are real valued | Yes | Yes | Yes |
Reading the Table
Points:
- the real world (driverless taxi) is the hardest case: inaccessible, nondeterministic, sequential, dynamic, continuous
- chess with a clock is semi-dynamic, the board does not change but the time score does
- a slot machine is a rare episodic example, each play is independent of the last
When the environment is accessible and deterministic, the agent can plan a full action sequence in advance, which is Problem Formulation for Search.