Notes

Content of the Lecture

• What is Unsupervised Learning?
• Unsupervised Learning in the Brain
  • Unsupervised Learning in the cortex - Theory by K. Doya and D. Marr
  • Unsupervised Learning Experiments in Neuroscience
• Unsupervised Machine Learning
  • Non-Probabilistic UL Methods
    • Sparse Coding (Olshausen and Fields)
    • Simple Network Solutions: PCA, ICA (Infomax)
    • Autoencoders and Supervised Autoencoders
    • Contracting Autoencoders
    • Denoising and Sparse Autoencoders
    • The "Homomorphism Autoencoder"
    • Competitive Network Learning
  • Probabilistic (Generative) UL Methods
    • Boltzmann Machines
    • Contrastive Divergence (Hinton)
  • Self-Supervised Learning
    • PIXEL-RNN, PRED-NET

Unsupervised Learning - Introduction & Motivation

Motivation Geoffrey Hinton ("Learning Representations by Back-Propagating Errors") suggest networks should be able to become intelligent on their own, unsupervised, without backprop.

• Does not need labelled data. In practice we often assume a fixed number of labels and call them cluster centers.
• Is able to learn/reveal the intricate structure (features) and can be used for dimensionality reduction. This means we try to find projections in this lower dimension space.
• Is able to generate new data examples that are consistent with the statistics of the training data.
• Can predict future data examples (e.g., video frame prediction). If we know the data representation we can try to find an inverse function out of our lower dimensional space. Which means we can sometimes reconstruct high dimensional data.
• Can be used to detect familiar or out-of-distribution examples. If we freeze our clusters, and then switch to another data set, then we can calculate the new data distance to the original data cluster centers.
• Can facilitate future supervised learning.
• Can be used to cluster the data.
• Can be used to de-noise data. We lose some information when projecting to a lower dimensional space. We optimize out method to retain the most useful information, this process might also de-noise our data.
• Can be used to encode data in a particular way (e.g., for data compression).
• We assume that humans have learned the world in an almost unsupervised way.

The difference between a supervised regression task where the data points are separated by some function that represents a bound in between different classes and an unsupervised clustering where the algorithm highlights the intrinsic structure of the data.

Unsupervised Learning

• Non-Probabilistic Models:
  • Sparse Coding
  • Autoencoders
  • KNN, K-Means
• Probabilistic (Generative) Models:
  • Tractable Models
    • Fully Observed Belief Nets
    • Pixel RNNs
  • Non-Tractable Models
    • Boltzmann Machine
    • Variational Autoencoders
    • Helmholtz Machines
    • Many others
  • Generative Adversarial Networks (GAN)
  • Moment-to-Moment Networks (not covered)

Unsupervised Learning (UL) in the Brain

Unsupervised Learning in the Brain Papers: "Unsupervised Yearning", "Complementary Roles of Basal Ganglia and Cerebellum in Learning and Motor Control", "Development of the Brain Depends on the Visual Environment".

We are aware of the fact that the brain as well uses unsupervised learning. Several examples were collected for the lecture. In 1970 an experiment was conducted on cats. The kittens were housed from birth in a completely dark room, but from the age of 2 weeks they were put in a special apparatus for an average of about 5 hours a day. The kitten stood on a clear glass platform inside a tall cylinder of which the entire surface was covered in black and white stripes (in different experiments they used horizontal and vertical stripes). Those poor cats then were virtually blind for contours perpendicular to the orientation they had experienced. They recorded single neurons from primary visual cortex and found that almost all cats had their neurons trained to be most selective in direction of the stripes presented in the experiment. Interpretation: the neurons are the cluster centers and they move around during learning (growing up). When presented one stimulus only, then all cluster centers group in the same optimum.

In the picture we have a spike rate curve of a single neuron with respect to the neurons orientation. Experiments show that this distribution changes in adolescent subjects and becomes more rigid with increasing age.

Another group analyzed visual cortical activity of awake ferrets during development (2007). They provide a one-sentence summary: The relation between spontaneous activity and activity evoked by natural stimuli in the primary visual cortex reveals that the cortical circuit progressively adapts its internal model to the statistical structure of the environment. Paper: "Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment".

In the figures: Notation: Evoked and spontaneous (dark) neural activity (EA and SA). Multi-neural EA (aEA). In the top-left figure, the posterior distribution represented by EA is increasingly dominated by the prior distribution as brightness or contrast is decreased. In the right figure, ferrets either receiving no stimulus (middle) or viewing natural (top) or artificial stimuli (bottom) is used to construct neural activity distributions in young and adult animals. It reveals the level of statistical adaptation of the internal model to the stimulus ensemble. The internal model of young animals (left) is expected to show little adaptation to the natural environment and thus aEA for natural (and also for artificial) scenes should be different from SA. Adult animals (right) are expected to have adapted to natural scenes and thus to exhibit a high degree of similarity between SA and natural stimuli aEA, but not between SA and artificial stimuli aEA.

We now know that these distributions adapt, but from the presented experiments it is unclear what the conditions are to trigger an adaptation. Another experiment ("Stimuling Timing-Dependent Plasticity in Cortical processing of Orientation") shows that the relative timing of presynaptic and postsynaptic spikes plays a critical role in activity-induced synaptic orientation 9single unit recording in cat V1). Induction of a significant shift required that the interval between the pair fall within +- 40ms otherwise nothing changed. Another path to understand the learning in neural circuits leads to the recent advances in Deep Neural Networks (DNN). Several groups tried to map layers (as in DNN layers) to cortical regions. Several mapping strategies were found. We can show that dissimilarity matrices of regions in both systems look similar, especially in higher cortical regions vs deeper layers of neural networks. Interestingly, the animals we recorded from never knew any labels that were used to train the DNNs.

1st Experiment: The statistics of the neuronal activity has adapted to represent the input data statistics (spatial). 2nd Experiment: The statistics of the neuronal activity has adapted to represent the input temporal data statistics.

In the picture we have the confusion matrix of V4 neural units and units in artificial networks from a comparable depth.

Sparse Coding & Relation to Neuroscience

Sparse Coding The sparse code is found when each sample of a given data set is encoded by the strong activation of a relatively small set of neurons. For each item to be encoded, this is a different subset of all available neurons. Each image can be represented by a set of basis functions: $I(x,y) \approx \sum_{i}^{}a_{i}\phi_{i}(x,y) = \widehat{I}$.

We define an energy function which is essentially the true image I minus approximation $\overline{I}$ + regularizer S:

Presented in the lectures was a set of known regularizers $\mathbb{S}$:

For each image presentation E is minimized with respect to $a_{i}$. Thus, for a given image, the $a_{i}$ are determined from the equilibrium solution to the differential equation:

The $\phi_{i}$ then evolve by gradient descent on E averaged over many image presentations. The learning rule for updating $\phi$ is then:

Comments on Parameters and Operations:

• $a_{i}$ factors should be sparse, different for each sample.
• $\phi$ components, basis functions that are the same over all samples.
• $\tau$ scaling constant.
• $\lambda$ scaling constant: sparseness vs information perseverance.
• $\eta$ learning rate.
• $\left\langle x \right\rangle$ mean.

In the figure we have representative training images are shown at the left and the resulting basis functions that were learned from these examples are shown at the right. In a, images were composed of sparse pixels: each pixel was activated independently according to an exponential distribution. In b, images were composed similarly to a, except with gratings instead of pixels (i.e., sparse pixels in the Fourier domain). In c, images were composed of spare, non-orthogonal Gabor functions with the methods described by Field. In all cases, the basis functions were initialized to random initial conditions. The learned basis functions successfully recover the sparse components from which the images were composed.

Relation to Neuroscience Paper: "Spatial Structure of Neuronal Receptive Field in Awake Monkey Secondary Visual Cortex (V2)".

This paper shows that cells of sub-units in V1 have receptive fields that apply signal filtering that is very similar to sparse coding. In V2 they identified sub-units with spatial feature selectivity.

Non-Probabilistic UL - PCA, ICA (Infomax)

Infomax (ICA) Paper: "An Information-Maximisation Approach to Blind Separation and Blind Deconvolution".

Infomax is an optimization principle for artificial neural networks and other information processing systems. It prescribes that a function that maps a set of input values I to a set of output values O should be chosen or learned so as to maximize the average Shannon mutual information between I and O. One of the applications of Infomax has been to an independent component analysis (ICA) that finds independent signals by maximizing entropy. ICA via mutual information is one way to find independent components. The independence criterion is stronger than uncorrelatedness which is defined as:

$$\left\langle X_{1},X_{2} \right\rangle = \left\langle X_{2} \right\rangle\left\langle X_{1} \right\rangle = 0$$

Or

$$Cov\left( X_{1},X_{2} \right) = E\left\lbrack X_{1},X_{2} \right\rbrack - E\left\lbrack X_{1} \right\rbrack E\left\lbrack X_{2} \right\rbrack = 0$$

Remember: If two variables are uncorrelated, there is no linear relationship between them. However, this does not mean that they are independent. The other way works: if $X_{1}$ and$\ X_{2}$ are independent (with finite second moments), then they are uncorrelated.

For ICA we want statistical independence:

$$p\left( \alpha_{1:N} \right) = \prod_{i = 1}^{N}{p\left( \alpha_{i} \right)}$$

To measure the degree of dependence we look at the pairwise mutual information of two random variables X, Y. Mutual information is non-negative and symmetric:

We use the entropy:

Idea: X, Y independent if: $p(X,Y) = p(X)p(Y)$

For the discrete case we can then see best that last part of eq. is log(1) = 0:

In the picture below we have a Venn diagram of the Infomax objective: We want to maximize the entropy and minimize the mutual information. Entropy maximization forces the network to generalize.

• H(x) = Entropy of Distribution of the First Neuron's Output.
• H(x|y) = Conditional/Marginal Entropy.
• H(x,y) = Joint Entropy.
• I(x,y) = Mutual Information.

We want to ensure that the outputs $y_{i}$ are maximally independent. This is identical to requiring that the mutual information be small or alternatively that the joint entropy is large. Gradient ascent in this objective function is called INFOMAX (maximize the enclosed area representing both information quantities).

How do we actually implement Infomax? We can think of Infomax as a one layer linear neural network that produces: $\mathbf{y = Wx}$ such that outputs $y_{i}$ that are maximally independent:

but keep in mind we also maximize I(X;Y).

In the picture below we can see that the PCA features are very different from what we know that the brain uses for feature representation. However the ICA representation looks like the (Gabor-like) filters from the previous chapter.

• ICA is similar to Principal Component Analysis, except that we are looking for a transformation subject to the stronger requirement of independence, rather than uncorrelatedness.
• In general, no analytic solution (like eigenvalue decomposition for PCA) exists. Thus, ICA is typically implemented using neural network models and GD.
• For the ICA NN implementation, we need an architecture and an objective function to descend/climb.
• Results in N independent (or as independent as possible) components in an N-dimensional space; these don't need to be orthogonal.

Non-Probabilistic UL - Autoencoders & Supervised Autoencoders

Autoencoders

• $g\phi$: encoding function.
• $f\phi$: decoding function.
• $\phi$: parameters of encoding function (trainable).
• $\theta$: parameters of decoding function (trainable).
• MSE Loss:

$$L_{AE} = \frac{1}{2}\left\| \mathbf{x -}\mathbf{x}^{\mathbf{'}} \right\|^{2},\mathbf{x}^{\mathbf{'}} = f_{\theta}\left( g_{\phi}(x) \right)$$

The autoencoder is trained by gradient descent.

Semi-Supervised Autoencoder Paper: "Supervised Autoencoders: Improving Generalization Performance with Unsupervised Regularizers".

The supervised AE uses the latent space to define a second decoding pathway. This path is added as another term to the loss and one calculates the gradients from two different ends. In the shared part, these gradients then merge. This is called multi-task learning (having a shared pathway for different objectives).

$$L_{SAE} = \frac{1}{t}\sum_{i = 1}^{t}\left\lbrack L_{P}\left( \mathbf{x}_{\mathbf{i}},\mathbf{W}_{\mathbf{1:2}},\mathbf{y}_{\mathbf{i}} \right) + L_{v}\left( \mathbf{x}_{\mathbf{i}},\mathbf{W}_{\mathbf{1:4}},{\widehat{\mathbf{x}}}_{\mathbf{i}} \right) \right\rbrack$$

where:

• $L_{P}$ is the loss of label $y_{i}$
• $L_{v}$ is the loss of reconstruction ${\widehat{x}}_{i}$
• $\mathbf{W}_{\mathbf{1:2}}$ are the weights of encoder (that produce label y)
• $\mathbf{W}_{\mathbf{1:4}}$ are the weights of encoder + decoder.

Non-Probabilistic UL - Denoising & Sparse Autoencoders

De-Noising Autoencoder (DEA) Paper: "Extracting and Composing Robust Features with Denoising Autoencoders"

Since the autoencoder learns the identity function, we are facing the risk of "overfitting" when there are more network parameters than the number of data points. To avoid overfitting and improve the robustness, Denoising Autoencoder (Vincent et al. 2008) proposed a modification to the basic autoencoder. The input is partially corrupted by adding noises to or masking some values of the input vector in a stochastic manner. To "repair" the partially destroyed input, the denoising autoencoder has to discover and capture relationship between dimensions of input in order to infer missing pieces. Similar to dropout. Note: In the experiment of the original DAE paper, the noise is applied in that a fixed portion of input dimensions are selected at random and their values are forced to 0. This is very similar to Dropout but the denoising autoencoder was proposed in 2008, 4 years before the dropout paper (Hinton, et al. 2012).

Sparse Autoencoder The Sparse Autoencoder applies a sparsity constraint on the hidden unit activation to avoid overfitting and improve robustness. It forces the model to only have a small number of hidden units being activated at the same time.

Let's say there are $s_{l}$ neurons in the l-th hidden layer and the activation function for the j-th neuron in this layer is labelled as $a_{j}^{(l)}( \bullet ),\ \ j = 1,\ldots,s_{l}$. The fraction of activation of this neuron $\rho^{j}$ is expected to be a small number $\rho$, kowns as sparsity parameter; a common config is $\rho = 0.05$.

Keep in mind that we specify our desired target distribution that is $\rho$. Common activation functions include sigmoid, tanh, relu, leaky relu, etc. A neuron is activated when the value is close to 1 and inactive with a value close to 0.

Non-Probabilistic UL - Contracting Autoencoders

Contracting Autoencoder Similar to sparse autoencoder, Contractive Autoencoder (Rifai, et al., 2011) encourages the learned representation to stay in a contractive space for better robustness. It adds a term in the loss function to penalize the representation being too sensitive to the input, and thus improve the robustness to small perturbations around the training data points. The sensitivity is measured by the Frobenius norm of the Jacobian matrix of the encoder activations with respect to the input:

Where $h_{j}$ is one unit output in the compressed code $\mathbf{z} = f(x)$.

This penalty term is the sum of squares of all partial derivatives of the learned encoding with respect to input dimensions. The authors claimed that empirically this penalty was found to carve a representation that corresponds to a lower-dimensional non-linear manifold, while staying more invariant to majority directions orthogonal to the manifold.

Non-Probabilistic UL - "Homomorphism" Autoencoders

Homomorphism Autoencoder (HomoAE) Paper: "Homomorphism Autoencoder - Learning Group Structured Representations from Observed Transitions".

A Homomorphism Autoencoder (HomoAE) is a type of autoencoder that is trained to preserve the homomorphism (structure-preserving) properties of the input data in its encoded representation. This is achieved by adding a homomorphism constraint to the standard autoencoder loss function. The constraint ensures that the encoded representation preserves certain properties of the input, such as symmetry or commutativity. The decoder then tries to reconstruct the original input based on the encoded representation, which should also possess the same homomorphism properties. The result is a neural network that can learn to preserve structural information in the data and can be used for tasks such as dimensionality reduction, data generation, and anomaly detection.

From the paper: "How can we acquire world models that vertically represent the outside world both in terms of what is there and in terms of how our actions affect it? Can we acquire such models by interacting with the world, and can we state mathematical desiderata for their relationship with a hypothetical reality existing outside our heads? As machine learning is moving towards representations containing not just observation but also interventional knowledge, we study these problems using tools from representation learning and group theory. Under the assumption that our actuators act upon the world, we propose methods to learn internal representations of not just sensory information but also of actions that modify our sensory representations in a way that is consistent with the actions and transitions in the world. We use an autoencoder equipped with a group representation linearly acting on its latent space, trained on 2-step reconstruction such as to enforce a suitable homomorphism property on the group representation. Compared to existing work, our approach makes fewer assumption on the group representation and on which transformations the agent can sample from the group. We motivate our method theoretically, and demonstrate empirically that it can learn the correct representation of the groups and the topology of the environment. We also compare its performance in trajectory prediction with previous methods."

Non-Probabilistic UL - Competitive Network Learning

Competitive Learning Paper: "Feature Discovery by Competitive Learning"

Competitive learning is a form of unsupervised learning in artificial neural networks, in which nodes compete for the right to respond to a subset of the input data. A variant of Hebbian learning, competitive learning works by increasing the specialization of each node in the network. It is well suited to finding clusters within data. Imagine that we move our neuron around that space by adjusting the weights.

• Neurons/Nodes are all the same except for their weights.
• A competitive mechanism permits neurons to compete for the right to respond to a given subset of inputs, such that only one output neuron (or only one neuron per group), is active (i.e., "on") at a time.
• The neuron that wins the competition is called a "winner-take-all" neuron and is allowed to update its weights.
• During "learning" individual neurons of the network learn to specialize on ensembles of similar patterns and become "feature detectors" for different classes of input patterns.
• The competitive networks are able to recode sets of correlated inputs to a few output neurons.

The CL Algorithm The competitive learning algorithm (two clusters):

1. Let all inputs feed into two different nodes, so that every hidden node is connected to every input. Initialize the weights randomly between 0.0 and 1.0. Calculate the activity of each hidden node for the first input.
2. The hidden node with the highest output is the winner for the cluster to which the data point belongs.
3. The winner node updates each of its weights, thereby moving its weight vector towards the data point.
4. Repeat with the next data point.

In the figure we have an illustration of how the barrier would move if the blue neuron would move upwards in direction of the cluster center and the red one downwards (3 iterations are shown). However, after revisiting this example I think the blue one would occupy the lower cluster.

CL with Neural Networks We ask for the neuron with the closest weight vector:

We update our weights accordingly:

The figure above shows Competitive NN: the position of the two neurons after convergence (left). A new datapoint and the data points equidistance line to the cluster-center neurons (center). The network structure (right).

Self-Organizing Maps

When a training example is fed to the network, its Euclidean distance to all weight vectors is computed. The neuron whose weight vector is most similar to the input is called the Best Matching Unit (BMU). The weights of the BMU and neurons close to it in the SOM grid are adjusted towards the input vector. The magnitude of the change decreases with time and with the grid-distance from the BMU.

Summary Plots

Probabilistic (Generative) Unsupervised Learning

Probabilistic Generative Models "When one understands the causes, all vanished images can easily be found again in the brain through the impression of the cause. This is the true art of memory..."- Rene Descartes

We know that our data has some causes v, but it is hard to specify. We do not know the underlying distribution. We can see the real world as a generative model that produced our (observable) data u. Now, we want to mimic this process. We model the causes as prior p(v,G) and our generative model specifies the artificial data distribution p(u|v,G). We have a recognition model that maps the samples gathered in the "real" world somehow to our generative model (not clear from the slides how). Goal: Learn a good generative model that mimics the statistics of the data generation process. Approach: Given data, solve two problems:

1. Estimate the causes by computing the posterior.
2. Learn all parameters G of the model and the latent space statistics.

A very basic example is a mixture of Gaussians:

I assume these parameters are means, variances and mixture scaling factors. There are several ways how one could use a neural network for these challenges. Also known as "Maximum Likelihood Learning":

• Learn the recognition model
• Learn the generative model
• Learn/update the prior p(z).

In the figure we have the difference between a traditional embedding algorithm on the left where we have huge gaps in embedding space and a generative embedding method on the right that densely covers the space. This allows us to sample. Indeed, interpolation between the classes is possible with a generative model.

Probabilistic (Generative) UL - Boltzmann Machines

The (restricted) Boltzmann Machine (RBM) A restricted Boltzmann Machine (RBM) is a generative stochastic artificial neural network that can learn a probability distribution over its set of inputs. As their name implies, RBMs are a variant of Boltzmann machines, with the restriction that their neurons must form a bipartite graph. This means that in restricted Boltzmann machines there are only connections (dependencies) between hidden and visible units, and non between units of the same type (no hidden-hidden, nor visible-visible connections). Although learning is impracticable in general Boltzmann machines, it can be made quite efficient for RBMs. A deep Boltzmann machine (DBM) is a type of binary pairwise Markov random field (undirected probabilistic graphical model) with multiple layers of hidden random variables. Practical details:

• RBMs have two biases (visible and hidden).
• The hidden bias helps the RBM produce the activations on the forward pass, while
• The visible layer's biases help the RBM learn the reconstructions on the backward pass.

In the figure we have the difference between a general and a restricted Boltzmann machine. The weights (orange arrow) are probabilistic units with activation 0. Or 1. In Boltzmann machines, information flows forward and backwards.

In the (right) figure RBMs are similar to (reverse) autoencoders but use stochastic units with particular distribution instead of deterministic distribution. The task of training is to find out how these two sets of variables are connected to each other. (left) The difference between the hidden nodes which are probabilistic and the input nodes.

Probabilistic (Generative) UL - Contrastive Divergence

Training the Restricted Boltzmann Machine (Contrastive Divergence) Paper: "Training Products of Experts by Minimizing Contrastive Divergence", "Reducing the Dimensionality of Data with Neural Networks".

Is not used anymore because back-propagation works so well. Attempts were made to use RBMs as dimensionality reduction algorithm. Results were okay, compared to PCA, the clusters seem more dense and separated.

Training algorithm:

1. Take a sample $V_{0}$ and compute the hidden activation vector $h_{0}$. Call $V_{0}h_{0}^{T}$ (outer product) the positive gradient.
2. Sample from $h_{0}$, compute $V_{1}$, resample from $V_{1}$, compute $h_{1}$. Call $V_{1}h_{1}^{T}$ the negative gradient.
3. Update weights: $\nabla w = \eta\left( v_{0}h_{0}^{T} - v_{1}h_{1}^{T} \right)$.

Self-Supervised Learning - Pixel-RNN

Pixel-RNNs Paper: "Pixel Recurrent Neural Networks". The PixelRNN is a generative model for images. The network models conditional distribution of every individual pixel given previous pixels (to the left and to the top).

$$p\left( \mathbf{x} \right) = \prod_{i = 1}^{n^{2}}{p\left( x_{i} \middle| x_{1},\ldots,x_{i - 1} \right)}$$

In the figures we have the distribution over color space of a single pixel in the generative process of the PixelRNN.

Alternative GM approaches not covered in this lecture:

• Helmholtz Machine & the Wake-Sleep algorithm.
• Variational Autoencoders & the Re-Parametrization trick.
• General Adversarial Networks (GANs).
• Flow Nets (Estimating Image Flow).
• Wave Nets (Used for Speech Generation).
• Expectation Maximization (EM).
• Moment-to-Moment Networks.