Notes

Rescorla Wagner Rule Model of classical conditioning in which learning is conceptualized in terms of associations between conditioned and unconditioned stimuli. Change in value $V\left( s_{t} \right)$ is proportional to the difference between actual and predicted reward.

$$V\left( s_{t} \right) \leftarrow V\left( s_{t} \right) + \eta\left\lbrack R - sum\left( V\left( s_{t} \right) \right) \right\rbrack$$

where: $s_{t}$ is the stimulus, $V\left( s_{t} \right)$ is the associative strength of conditioned stimulus $s_{t}$, R is the reward, $\eta$ is the learning rate, $sum\left( V\left( s_{t} \right) \right)$ is the sum of associative strengths of all conditioned stimuli (including $s_{t}$) that are presented on this trial (the n-th trial) and $|\left\lbrack R - sum\left( V\left( s_{t} \right) \right) \right\rbrack|$ is the surprise.

Two assumptions/hypotheses:

• Learning is driven by the error signal (formalized as notion of surprise).
• Summation of predictors is linear.