Notes

Bio-Plausible Deep Learning Through Control A novel, Bio-plausible Network Learning Algorithm: "Deep Feedback Control". It is based on a recurrent loop that stops when the output error is 0.

• u(t) is the error.
• Qi and QL are set to be the inverse of the feedforward weights.
• The error is multiplied by Qi and QL in the different layers to update the weights.
• The gradient is implicit.
• Hebbian type learning rule for Q that it updates to the transpose of the weight.

Advantages of the Deep Feedback Control (DFC) Algorithm:

• Supports continuous/asynchronous updates.
• Hidden layer activity does not need to be stored (no extra memory required).
• Highly parallelizable, but requires custom hardware.
• Very simple learning rule that is local in space and time solely based on the neuron's activity and effectively implements a delta rule.
• Absence of phases or back-propagation of errors (e.g., as in standard BP).
• The optimization approach "Gauss-Newton" which is fundamentally different from BP and standard gradient descent learning.