Notes

Content of the Lecture

• Why (local) Neuronal Learning Rules are Important?
• The Perceptron Learning Rule
• ADALINE and the Delta Rule
• Recap of Hebbian Learning
• Oja's Rule
• The Covariance Rule
• Sanger's Rule
• The Calcium Rule
• The Bienenstock-Cooper-Monroe (BCM) Rule.
• The Triplet Rule (Pfister&Gerstner)
• Extensions of Hebbian Learning Rules (Neo-Hebbian)

Why (local) Neuronal Learning Rules are Important?

Learning rules, as the name imply, describe methods of learning from information. Various machine learning methods that we discuss elsewhere already describe ways in which the data available to us can be used to create an objective (or cost, or loss) function which gives us something concrete to optimize so we have a model that performs well on similar data. These methods describe global cost functions because these expressions are in terms of high-level representations in the model, often only the final output layer representations. In a very small toy model, such as a fully-connected neural network with no hidden layers, this may provide useful information for adapting individual neuronal connections. Expanding this model to add complexities such as additional neurons hidden layers leaves us with an architecture that we can understand and can enumerate, as well as the global objective which we continue to aim for. However, we now have little understanding of how individual connections should be modified in the training process to contribute to improving the global objective defined by some global cost function which makes claims describing how output representations should change to improve the model but no inherent claims describing how the changes can be implemented. To this end, the local learning rules we are about to discuss can alternatively be considered local optimization principles, as they are a small instance (typically involving only a few neurons) of our global optimization goal.

Error Minimization Rules The first category of learning rules we will discuss are those that focus on optimizing with respect to some error function.

Perceptron Learning Rule

Perceptron Learning Rule The perceptron learning rule was inspired by the model of neurons at the time, chiefly outlined in the McCulloch and Pitts Neuron. This model describes basic action potential propagation and involves multiple presynaptic neurons connected to the soma of a postsynaptic neuron. An action potential is triggered when sufficiently many presynaptic neurons (which may each contribute differently to the postsynaptic neuron based on their synaptic strengths) are activated such that the joint effects of their action potentials in the postsynaptic neuron exceeds the activation threshold, which triggers an action potential through the postsynaptic neuron. Analogously, the perceptron learning rule involves multiple input values, which are each connected with varying weights to an output node. The value emitted by the output node depends on whether the weighted contributions of the input values exceeds a specific threshold. Learning is the process of adjusting the weights of each input as well as the output threshold to achieve the desired goal. This can be denoted by a threshold linear transformation. Given an input vector of values, the output falls into two cases depending on whether the linear transformation is above or below threshold. Specifically, for an input vector x, corresponding weights w, and a threshold b, the output can be denoted as:

As a result, from a machine learning classification perspective, the perceptron learning rule describes a linear classifier as its decision is based on the result of a linear transformation. The standard algorithm (developed by Rosenblatt) for training according to this learning rule involves looping through every data sample, updating the weights w if and only if the current data sample x is misclassified, detailed below:

ADALINE & Delta Learning Rule

ADALINE (ADAptive LInear NEuron) Learning Rule This can be viewed as a slightly modified instance of the perceptron learning rule. In the perceptron rule, the threshold result of the weighted sum of inputs is used for updating the weights in each iteration. In ADALINE, the weighted sum of inputs itself is used to update the weights in training.

• They are both classifiers for binary classification.
• Both have a linear decision boundary.
• Both can learn iteratively, sample by sample (the Perceptron naturally, and ADALINE via stochastic gradient descent).
• Both use a threshold function.

DELTA Learning Rule The Delta Rule uses the difference between target activation (i.e., target output values) and obtained activation to drive learning. The weight updates from this equation aims to directly minimize a neuron's output error for a target value $t_i$ and output value $y_i$, which can be formulated using gradient descent minimizing the squared error between these values. This error can be formulated as:

$$E = \sum_{i}^{}{\frac{1}{2}\left( t_{i} - y_{i} \right)}^{2}$$

Finding the appropriate weight updates according to the gradient descent optimization method requires calculating the change in error with respect to each weight that we wish to update. This can be expressed as:

$$\frac{\partial E}{\partial w_{ji}} = \frac{\partial{\frac{1}{2}\left( t_{i} - y_{i} \right)}^{2}}{\partial w_{ji}}$$

Assuming a model structured similarly as in the perceptron and ADALINE learning rules with a single layer between inputs and the output value and using as the activation function $g(x) = \Theta(x)$ the heavyside function, this yields:

$$\mathrm{\Delta}w_{ji} = \alpha(t_{j} - y_{j})g'(h_{j})x_{i}$$

where we used $h_{j} = x_{i}w_{ji}$ as inputs, $y_{j} = g(h_{j})$ as outputs, $t_{j}$ as target value, learning rate $\alpha$ and the derivative g' of the activation function g. The weight update equation for the DELTA learning rule clearly shares some similarities with that of the perceptron learning rule. Both weight update equations contain an error term calculated by the difference between the target and output values $\left( t_{i} - y_{i} \right)$ multiplied with the input $x_{i}$. However, the DELTA learning rule adds some complexities as it incorporates a learning rate $\alpha$ to adapt learning as well as the derivative of the activation function applied to the sum of the inputs $g'(h_{j})$. While the perceptron learning rule, particularly in light of its well-defined algorithm, defines the problem in terms of shifting hyperplanes to adapt a decision boundary, the DELTA learning rule optimizes the sum of squared error for a model with an activation function applied to a linear output. As previously mentioned in the contrast between the perceptron and ADALINE rules, the perceptron rule will either reach a stable zero-error solution (in the case of linearly separable data) or continually oscillate (otherwise). In contrast, the DELTA rule due in part to its adaptable learning rate can continually converge to a minimum error solution.

DELTA Rule vs Perceptron Learning Rule We have seen that the DELTA rule and the Perceptron learning rule for training single-layer Perceptrons have a similar weight update equation. However, the two algorithms were obtained from very different theoretical starting points. The Perceptron learning rule was derived from a consideration of how we should shift around the decision hyper-planes for step function outputs, while the DELTA rule emerged from a gradient descent minimization of the Sum Squared Error for a linear output activation function. The Perceptron learning rule will converge to zero error and no weight changes in a fine number of steps if the problem is linearly separable, but otherwise the weights will keep oscillating. On the other hand, the DELTA rule will (for sufficiently small learning rates) always converge to a set of weights for which the error is a minimum, though the convergence to the precise target values will generally proceed at an ever decreasing rate proportional to the output discrepancies.

Biologically Plausible Rules The second category of learning rules we will discuss are those that draw inspiration from Neuroscience and Biology.

Hebbian Learning Rule

Hebbian Learning Rule and STDP "When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place on one or both cells such that A's efficiency as one of the cells firing B, is increased".

Spike-Timing Dependent Plasticity (STDP) is the Hebbian learning concept of weights between neuronal synapses changing over time based on the timing of their spikes. If a postsynaptic neuron fires at the same time or just after the presynaptic neuron fires, whether this is due to an action potential in the presynaptic neuron or other nearby neurons, this indicates that connections between these two neurons could be reinforced, which occurs by increasing the weights on these synaptic junctions to more efficiently propagate future action potentials. This is known as Long-Term Potentiation (LTP). However, if a postsynaptic neuron fires just before a presynaptic neuron, the junction in the pre- to postsynaptic direction is possibly unnecessary or counterproductive so the weight of these synapses decrease over time. This is known as Long-Term Depression (LTD). We can start by writing a simple Hebbian weight update as the product of the input and output with some scaling factor:

$$w = \alpha \times x \times y = \alpha \times x \times \lbrack w \bullet x\rbrack$$

The more closely aligned $w$ and x are, the larger $y = w^{T}x$ is, and by definition of the dot product $y = 0$ when w is orthogonal to x. This leads to the weight vector gradually pointing towards the input vector, or the cloud of input data in a dataset. We can mitigate this issue by applying a zero-mean transformation on our dataset to center the data around the origin, but this leads to a different problem of the weight vector tending to align with the direction of greatest variance.

Let's explore some other ways to express this Hebbian update rule:

$$\nabla w = \alpha \times x \times y = \alpha(w \bullet x)x = \alpha(x \bullet x^{T}) \bullet w$$

Side note: "When the input x and output y are correlated, their product is positive, which results in a positive update to the weight (increases). When they are uncorrelated, the product is close to 0, which results in a small or no-update to the weight".

where the last step involves a transformation of the inner product into an outer product.

We note that $x \bullet x^{T}$ is the correlation matrix of the vector x, which we denote as C. We now arrive at:

$$\nabla w = \alpha(x \bullet x^{T}) \bullet w = \alpha \times C \bullet w$$

which leads to the following update over time:

$$\frac{\partial w}{\partial t} = C \bullet w$$

The lecture slides go into some more detail describing that applying the classical solution for this expression, $w(t) = u \times e^{\lambda \times t}$ for some vector u, as $\lambda$ is positive the weight vectors will continue to increase and blow up.

Oja's Learning Rule

Oja's Rule Paper: "A Simplified Neuron Model as a Principal Component Analyzer".

A modification of the Hebbian rule above in which a weight decay term is added. As this weight decay term is proportional to $y^{2}$, a quadratic result, it eventually limits the magnitude of the weights w to unit length while maintaining the tendency of the weights to point in the direction of maximum variance.

$$\nabla w_{oja} = \nabla w - y^{2} \times w = \alpha \times x \times y - y^{2} \times w = \alpha \times y(x - y \times w)$$

Covariance Learning Rule

Covariance Rule Another modification of the Hebbian rule above uses an idea similar to mean-centering of the data, but instead of transforming the data, the weights w are updated using mean-centered inputs x and outputs y.

The last line depicts the difference between the mean of the product of x and y, $\left\langle y \times x \right\rangle$ and the product of the means $\left\langle y \right\rangle \times \left\langle x \right\rangle$. This rule solves a similar problem as Oja's rule, specifically the blowing up of weights over the training process. By subtracting the means when updating, weights updates can be negative as well as positive. The weights w increase when pre- and post-synaptic firing are positively correlated, and the change is proportional to the covariance of the firing rates.

Sanger's Learning Rule

Sanger's (PCA) Rule PCA Recap: It is a tool from statistics for data analysis. It can reveal structure in high-N-dimensional data that is not otherwise obvious. Like Hebbian learning, it discovers the direction of maximum variance in the data. But then in the (N - 1)-dimensional subspace perpendicular to that direction, it discovers the direction of maximum remaining variance, and so on for all N. The result is an ordered sequence of principal components. These are equivalently the eigenvectors of the correlation matrix C for zero-mean data, ordered by magnitude of eigenvalue in descending order. They are mutually orthogonal.

The idea is that we use a single Hebbian neuron that points in the direction of maximum variance, as described previously, and we view this as a principal component of the data. We subtract the contribution of this first principal component from the data, feeding the remaining data into a different neuron which subsequently identifies the direction of maximum variance in this data. This process can be repeated and resembles the addition of principal components in Principal Component Analysis (PCA).

We have seen that Hebbian learning, with appropriate provisions for preventing blow up, extracts the largest principal component. Let's take a look at two different neural network architectures capable of extracting more of them:

• Cascading Multiple Hebbian Neurons.
• Autoencoder Networks (we have already seen them before).

Algorithm:

1. Subtract the contribution of the first principal component.
2. Drive the difference into another Hebbian neuron.
3. This extracts the next principal component.
4. Subtract its contribution. Go to step 2. With N Hebbian neurons, we will get all N principal components.

Sejnowski's Infomax Network (ICA) Rule & Bienenstock-Cooper-Monroe (BCM) Rule

Sejnowski's Infomax Network (ICA) Rule Paper: "An Information-Maximisation Approach to Blind Separation and Blind Deconvolution".

The Infomax rule utilizes a nonlinear function and yields a method for implementing Independent Component Analysis (ICA).

Bienenstock-Cooper-Monroe Rule

Activity is measured by y along the horizontal axis, and with no activity no change in weights takes place. Activity below a certain threshold $\theta_{M}$ triggers the LTD regime, and activity above this threshold triggers the LTP regime. Measuring neuronal output across a certain window indicates that there is a biological basis for this. Above some threshold, the weight updates increased as the stimulation frequency was increased. This was tested experimentally in the hippocampus and primary visual cortex, stimulating inputs to a neuron and measuring its spiking frequency. In this experiment, below a stimulation frequency of about 10Hz the weight updates were negative, while above this stimulation frequency they were positive.

Triplet RUle (Pfister & Gerstner)

The Triplet Rule

Papers: "Triplets of Spikes in a Model of Spike Timing-Dependent Plasticity" & "A Triplet Spike-Timing-Dependent Plasticity Model Generalizes the BCM rule to Higher-Order Spatiotemporal Correlations".

The triplet rule extends the classical Hebbian STDP idea. Instead of only looking at single presynaptic and postsynaptic spikes, multiple recent spikes within a specified time window are considered. As shown above in the figure, LTD generally occurs when a presynaptic spike occurs just after a postsynaptic spike, even if (as is A3^-^) another presynaptic spike preceded the postsynaptic spike. Analogously, LTP tends to occur when a postsynaptic spike preceded the presynaptic spike, even if (as in A3^+^) another postsynaptic spike preceded the presynaptic spike. In this fashion, the consideration of the third spikes within the time window can lead to different outcomes. In both the triplet and BCM rules, above some threshold as the spiking frequency increases, the weight updates also increase (and are positive). The fundamental differences between these two rules are unclear.

Note that if we set A3+ = 0 and A3- = 0, the model becomes a classical pair-based STDP model (BCM).

Calcium Rule

The Calcium Rule Paper: "Calcium-Based Plasticity Model Explains Sensitivity of Synaptic Changes to Spike Pattern, Rate and Dendritic Location".

As shown in the previous picture, this rule specifies that the LTP regime applies based on the amount of time above the calcium threshold while the LTD regime applies below the threshold. The threshold is represented by the frequency of synaptic firing, indeed, low frequencies of synaptic firing (approx. 5Hz) produce LTD, while high frequencies of synaptic firing (approx. 50 to 100Hz) produce LTP. This rule results in the same basic STDP profile as before, and there are threshold parameters that can be changed to affect the dynamics.

Extensions of Hebbian Learning Rules (Neo-Hebbian)

Hebbian Learning: Unsupervised Papers: "The Ups and Downs of Hebb Synapses" & "Unsupervised Learning of Digit Recognition Using Spike-Timing-Dependent Plasticity" & "Local Plasticity Rules Can Learn Deep Representations Using Self-Supervised Contrastive Predictions".

Error-driven learning appears to be much more necessary for deeper networks. A network was trained on the MNIST dataset using a basic Hebbian learning rule to cluster the data into separate digits and then learn a linear classifier on these digits.

Hebbian Learning: Three-Factor Rules Paper: "Neuromodulated Spike-Timing-Dependent Plasticity, and Theory of Three-Factor Learning Rules".

Three-factor Hebbian learning rules integrate the pre- and postsynaptic firing with a third factor, M, which includes values such as the covariance-rule, TD learning, gated Hebbian learning, surprise-modulated STDP, etc. For a biological neuron, this M factor may be viewed in a variety of ways, as shown in the picture above. It may be seen as a representation of error, including backpropagated error. For example, in the apical dendrites (level 5 neurons) receive feedback signals from the next hierarchical layer, and the strong calcium channels in these apical dendrites allow for error signals to trigger calcium spikes that propagate down the cell. The calcium spike is therefore a possible representation of the error from the next layer, which would model backpropagation. Alternatively, neurons project to the next layer but some also project backwards to interneurons (an in turn, back to the apical dendritic layer), so the error signals reflect what is happening globally, in the next layer, and (through lateral inhibition, etc.) what is occurring in neighboring neurons. There is a motivation, as seen in the learning rules that are analogous to PCA, to inhibit neighboring neurons. In particular, this allows a neuron to potentially learn a useful unique representation instead of learning the same things as every other neuron. Another possibility includes extracellular calcium release from astrocytes, as this affects the external calcium concentration but also internal concentrations in neurons, thereby indirectly affecting the plasticity of said neuron.