Notes

Hebbian Learning: Unsupervised Papers: "The Ups and Downs of Hebb Synapses" & "Unsupervised Learning of Digit Recognition Using Spike-Timing-Dependent Plasticity" & "Local Plasticity Rules Can Learn Deep Representations Using Self-Supervised Contrastive Predictions".

Error-driven learning appears to be much more necessary for deeper networks. A network was trained on the MNIST dataset using a basic Hebbian learning rule to cluster the data into separate digits and then learn a linear classifier on these digits.

Hebbian Learning: Three-Factor Rules Paper: "Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules".

Three-factor Hebbian learning rules integrate the pre- and postsynaptic firing with a third factor, M, which includes values such as the covariance-rule, TD learning, gated Hebbian learning, surprise-modulated STDP, etc. For a biological neuron, this M factor may be viewed in a variety of ways, as shown in the picture above. It may be seen as a representation of error, including backpropagated error. For example, in the apical dendrites (level 5 neurons) receive feedback signals from the next hierarchical layer, and the strong calcium channels in these apical dendrites allow for error signals to trigger calcium spikes that propagate down the cell. The calcium spike is therefore a possible representation of the error from the next layer, which would model backpropagation. Alternatively, neurons project to the next layer but some also project backwards to interneurons (an in turn, back to the apical dendritic layer), so the error signals reflect what is happening globally, in the next layer, and (through lateral inhibition, etc.) what is occurring in neighboring neurons. There is a motivation, as seen in the learning rules that are analogous to PCA, to inhibit neighboring neurons. In particular, this allows a neuron to potentially learn a useful unique representation instead of learning the same things as every other neuron. Another possibility includes extracellular calcium release from astrocytes, as this affects the external calcium concentration but also internal concentrations in neurons, thereby indirectly affecting the plasticity of said neuron.