Notes

The McCulloch & Pitts Neuron AKA The Perceptron

The Perceptron Learning Algorithm Can all logical operations be implemented by a McCulloch Pitts Neuron? OR yes, EQUALITY yes, AND yes, NOT yes, but it cannot implement XOR. Since there is no line which perfectly separates the two classes in the XOR problem. We can overcome this problem by combining multiple neurons in networks -> Neural Networks.

The Perceptron - Summary

• McCulloch-Pitts neurons implement a linear decision boundary (separating hyperplane)
• The weights and bias define the decision boundary
• They can implement many logical operations (AND, OR, NOT)
• They cannot implement XOR (not linearly separable)
• They can be trained on labeled datasets (supervised learning).

Learning to Recognize Handwritten Numbers (MNIST)

• Machine Learning Approach:
  • Linear Classifier: 88 - 92 accuracy.
  • K-Nearest Neighbor: 95 - 98 accuracy.
• Human: 99.8 accuracy.

A Neuronal Network for Classifying Handwritten Digits We can train a multi-layer NN through gradient descent (backpropagation) to minimize the error by changing the weights between neurons.