Source
Adapted from my own handwritten and hand-typed notes from Year 1 to Year 3 of my EEE degree
When actions have uncertain outcomes, a rational agent combines two things:
- a utility function: how good each outcome is
- outcome probabilities: how likely each outcome is, given the action
Together these give the expected utility.
Expected Utility
Where:
- is the action and the agent's evidence about the environment
- is the th possible outcome of doing
- is the utility of that outcome
Principle of maximum expected utility: choose the action with the highest .
Example: Robot
| Action | Outcomes | Expected utility |
|---|---|---|
| Turn right | hits wall (, ); finds target (, ) | |
| Turn left | falls in water (, ); finds target (, ) |
Choose turn right.
Agent Environment Interface
Agent and environment interact in discrete time steps :
- the agent observes state
- it produces action
- it receives reward and moves to the next state
When the utility depends on a whole sequence of such decisions, the problem is a sequential decision problem, modelled as a Markov decision process.