Notes
A collection of fragments of understanding in the pursuit of deeper questions.
Contents
- Computational Vision
- Introduction to Data Science
- Translational Neuromodeling
- Deep Learning in Artificial & Biological Neuronal Networks
- Introduction to Neuroinformatics
- Intro to Neuroscience
- Systems Neuroscience
- Readings in Neuroinformatics
- Neurobiology of Consciousness
- Econometrics
- Principles of Finance
- Big Data & Databases
- Game Theory
- Intro to Blockchain
- Marketing Analytics
- Financial Accounting
- Fundamentals of Computer Science
- Foundamentals of Information Technology Law
- Statistics
- Computer Programming
- Macroeconomics
- EU Information Law & Data Economy
- Advanced Mathematics & Statistics
- Principles of Management
- Mathematics
- Microeconomics
Probabilistic (Generative) UL - Boltzmann Machines
The (restricted) Boltzmann Machine (RBM) A restricted Boltzmann Machine (RBM) is a generative stochastic artificial neural network that can learn a probability distribution over its set of inputs. As their name implies, RBMs are a variant of Boltzmann machines, with the restriction that their neurons must form a bipartite graph. This means that in restricted Boltzmann machines there are only connections (dependencies) between hidden and visible units, and non between units of the same type (no hidden-hidden, nor visible-visible connections). Although learning is impracticable in general Boltzmann machines, it can be made quite efficient for RBMs. A deep Boltzmann machine (DBM) is a type of binary pairwise Markov random field (undirected probabilistic graphical model) with multiple layers of hidden random variables. Practical details:
- RBMs have two biases (visible and hidden).
- The hidden bias helps the RBM produce the activations on the forward pass, while
- The visible layer's biases help the RBM learn the reconstructions on the backward pass.
In the figure we have the difference between a general and a restricted Boltzmann machine. The weights (orange arrow) are probabilistic units with activation 0. Or 1. In Boltzmann machines, information flows forward and backwards.
 |
 |
|---|
In the (right) figure RBMs are similar to (reverse) autoencoders but use stochastic units with particular distribution instead of deterministic distribution. The task of training is to find out how these two sets of variables are connected to each other. (left) The difference between the hidden nodes which are probabilistic and the input nodes.