Notes

Artificial vs Neuromorphic Intelligence

• Artificial Neural Networks
  • Simulate abstract brain-inspired computing architectures on digital time-multiplexed computing substrates.
• Neuromorphic Architectures
  • Use the physics of electronic devices and circuits to emulate real neurons. They use physical (real) time and circuit dynamics to compute through their time evolution.
  • The physical hardware substrate is the algorithm.

Synapses vs Neurons

• In Biology
  • Pyramidal neurons have thousands of synapses, mainly distributed on the distal dentric arborization.
  • The excitatory drive from 10-40 inputs, discharging at an average rate of 100 spikes/s, should cause the postsynaptic neuron to discharge near 100 spikes/s.
• In Neurons
  • To first order approximation, a neuron has a gain of less than 1/10.
  • To produce one output spike, at least 10 input spikes should arrive in parallel to 10 different synapses.
  • An input spike train arriving onto a single synapse at 1kHz should produce one output spike train at 100Hz.

Dale's Principle In 1935, Sir Henry Dale hypothesized that a neuron extends its metabolic activity from the soma to all of its processes. In 1954, Sir John Eccles reformulated this principle to say that "neurons release the same set of transmitters at all of their synapses".

Dale's Law: In practice Dale's law requires neurons to be only excitatory or only inhibitory. A neuron cannot excite some targets and inhibit others. All projections of excitatory neurons can have only positive synaptic weights. All projections of inhibitory neurons can only have negative weights.

Dendrites The active electrical properties of dendrites shape neuronal input and output and are fundamental to brain function. They can implement analog signal processing, digital logic, and state dependent computations. These properties enable neocortical pyramidal neurons to classify linearly non-separable inputs - a computation conventionally thought to require multilayered networks. Recently it has even been hypothesized that there are multiple "Dendritic solutions to the credit assignment problem".

The Neocortex The neocortex is the newest part of the cerebral cortex to evolve. It is a distinguishing feature of mammals. In humans, it is 90% of the cerebral cortex and 76% of the entire brain.

The Canonical Micro-Circuit Across all areas of Neocortex

• Most cortical connections are local and excitatory.
• Most cortical excitatory synapses and almost all inhibitory ones originate from cortical neurons.
• Most of these synapses originate from neurons within the local cortical area.

Cortical Computation

• It is carried out almost entirely by local circuits.
• Recurrent circuits perform signal restoration.
• Control of gain is critical for cortical computation.

Model of Synapses

The First Models of Neurons

• Lapicque: The Integrate and Fire Model

• McCulloch & Pitts Neuron: "A logical calculus of the ideas immanent in nervous activity".

• Hodgkin & Huxley Model
  • Cell Lipid Bilayer (Membrane): Capacitance/Area
  • Leak Channels: Linear Conductance
  • Voltage-Gated Ion Channels: Conductance (V,t), voltage-dependent open-close probability.

Models of Neurons in Neural Networks The McCulloch&Pitts model quickly became extremely popular, and dominated the Artificial Neural Network scene for decades. Why? Isomorphism with calculus of logical propositions. In the hands of John Von Neumann, the McCulloch&Pitts model became the basis for the logical design of digital computers.