Notes

Pixel-RNNs Paper: "Pixel Recurrent Neural Networks". The PixelRNN is a generative model for images. The network models conditional distribution of every individual pixel given previous pixels (to the left and to the top).

$$p\left( \mathbf{x} \right) = \prod_{i = 1}^{n^{2}}{p\left( x_{i} \middle| x_{1},\ldots,x_{i - 1} \right)}$$

In the figures we have the distribution over color space of a single pixel in the generative process of the PixelRNN.

Alternative GM approaches not covered in this lecture:

• Helmholtz Machine & the Wake-Sleep algorithm.
• Variational Autoencoders & the Re-Parametrization trick.
• General Adversarial Networks (GANs).
• Flow Nets (Estimating Image Flow).
• Wave Nets (Used for Speech Generation).
• Expectation Maximization (EM).
• Moment-to-Moment Networks.