Notes

Lecturer: Benjamin Grewe

From temporal to rate coding, and single neurons to networks. How does the brain represent what we perceive? Perkel & Bullock (1968): The problem of neural coding is to elucidate "the representation and transformation of information in the nervous system".

Representation and Transformation of Information The simplest organism that uses spikes is the Paramecium (a "Swimming Neuron"), which represents some of the experience it has about the world. It can use such Action Potentials for movements.

The Coding Metaphor Considering three elements: Correspondence, Representation & Causality:

• The technical sense of a code is a correspondence between two domains, e.g., visual signals and spike trains. We call this relation a code to mean that spike trains specify the visual signals, as in a cipher: one can theoretically reconstruct the original message (visual signals) from the encoded message (spike trains) with some accuracy, a process called decoding.
• Not all cases of correlations in nature are considered instances of coding. Climate scientists, for example, rarely ask how rain encodes atmospheric pressure.
• Finally, we would not say that visual signals encode retinal spike trains, even though this would comply with the technical sense. The reason is the communication metaphor implicitly assumes a causal relation between the original message and the encoded message; here, spike trains result from visual signals by a causal process (transduction).

Encoding and Decoding of Information

• Encoding: How does a stimulus cause a pattern of responses?
  • Building an approximate mechanistic model of the world.
  • P(response|stimulus) = Encoding.
• Decoding: What do the responses tell us about the stimulus?
  • How can we reconstruct the stimulus?
  • P(stimulus|response) = Encoding
  • By recording neuronal responses in the cat visual cortex, we identify that the visual cortex encodes the orientation of a moving stimulus and has an orientation-specific organization. In rats, spatial information is coded via hippocampal place cells.

In general:

• Information is encoded by firing of single neurons and firing of populations of neurons.
• A neuron encodes information, fires to stimuli.
• Firing rate and spike timing encodes information.
• Spatial/temporal resolution of different measurement techniques tell us about the neural code.
• It is an issue to record from many neurons simultaneously.
• There is not much information in the slope of a spike.
• By recording neuronal responses from a stimuli, we can "see" how the brain encodes the stimuli.

Finding the Stimulus-Response Relation

Encoding Motor Output in Primates One of the first experiments investigating how is a motor command in arm reachment encoded by neurons's in the motor cortex. (Georgopoulus et. al., 1982).

Recording Neuronal Responses in Cat VI

Hubel & Wiesel wanted initially to find a neuron that was responsive to the black dot, they accidentally found the reaction to the edge of the paper onto which the black dot was depicted.

Orientation and Direction Selective Neurons in VI Neurons have a receptive field and they show direction selectivity to the stimuli.

Edge Filters in Primate Visual Cortex Edge filters constitute a way to represent in low-dimensional manner natural images, indeed with just a couple hundreds neurons you can reconstruct complex images through edges.

Paper: "Spatial Structure of Neuronal Receptive Field in Awake Monkey Secondary Visual Cortex (V2)".

Encoding Complex Stimuli in Primate V4 They proved the activity in V4 and figured out a way to design an experiment to understand what type of features maximally excite neurons in V4. Paper: "Neural Population Control Via Deep Image Synthesis".

Encoding Visual Stimuli in the Human Brain (Area MTL) They measured MTL neurons activity, they showed pics of people and measured that this patient had neurons responding to Jennifer Aniston's pictures. Hence, at MTL we have an high-representation of concepts, such as the Jennifer Aniston's character.

Encoding Spatial Information in Rata Hippocampus O'Keefe, M. B. Moser and E. Moser Nobel prize. It is possible to reconstruct the position of the mouse along the track based on the decoding of information encoded by spines. (Ziv and Schnitzer, 2013).

Which Features of the Spike Trains are the Signal? Rate Coding refers to information being carried by the firing rate. It is often argued, or assumed, that firing rate captures essentially all relevant information. (rate code means that I have a certain variable, which could be intensity or orientation, then I have a tuning curve and the more I tweak this variable the more I have a continuous reflection of my out-of-world variable and the spiking frequency of this neuron.) Temporal Coding may refer to several quite different ideas:

• Much of the information may be transmitted by a neuron during certain small intervals of time.
• Synchronous, or what one could call quasi-synchronous, firing of neurons within and across ensembles may carry important information.
• The precise timing, or pattern, of spikes may carry information.

Temporal vs Rate Code

• Rate Code
  • Rate = Average over time, single neuron, single run.
  • Definition of the mean firing rate via temporal average.
  • Neuronal gain function (curve). The output spike rate is given as a function of the total somatic input current I0.
  • Easy to understand, but no timing effects and misleading as more than one stimulus might be encoded.
  • It takes time to compute a temporal average and behavioral response time is shorter than integration time.
• Temporal Code
  • Phase Coding, when does the spike occur with respect to a background oscillatory cycle (phase delay of a spike).
  • Synchrony, when does the spike occur with respect to all the other spikes in the network? How synchronized is the activity?
  • Time to first spike, you have some stimulus and you measure the time until the neuron spikes. (short time - highly activated).

Phase Coding in Hippocampus Different cells responding to different stimuli encountered during the "trail". When the mouse is sleeping he replays the sequence faster but in the same order.

Hence, in the Hippocampus the information is mostly rate coded, but phase delay information (temporal coding) is also relevant. Indeed, the phase delay of spikes, with respect to the background oscillations, gives position cues that can be used to decode the position of the mouse.

Sound Localization by Measuring the Interaural Time Difference (ITD) The precise timing of spikes is directly used to hear and code the position of a prey (Barn Owl vs Mouse). The temporal delay between left and right ear is combined through delay lines. These neurons in the middle only activate when the stimuli arrive simultaneously.

How to Investigate the Stimulus Encoding of a Neuron? The same stimulus can be encoded very differently by different neurons. On the right we can see the factors that may cause such encoding differences.

In the cortex, most of the inputs for each neuron is not coming from the outside, but rather from neighboring neurons. In the cortex, approx. 4% of synaptic inputs are actually coming from the Thalamus and the Retina. Hence, the cortex is highly recurrent and the brain has a certain state that changes all the time, i.e., what we think. Depending on what we think, we might have different stimuli in the visual cortex.

What is the simplest possible relation between stimuli and encoded signals?

$$\mathbf{r}\left( \mathbf{t} \right)\mathbf{= f}\left\lbrack \mathbf{s}\left( \mathbf{t} \right) \right\rbrack\mathbf{\rightarrow}\left\lbrack \mathbf{f \bullet s}\left( \mathbf{t -}\tau \right) \right\rbrack$$

$$\mathbf{Linear\ Filter:\ r(t) =}\sum_{\mathbf{k = 0}}^{\mathbf{n}}{\mathbf{s}_{\mathbf{t - k}}\mathbf{\bullet}\mathbf{f}_{\mathbf{k}}}$$

The Neuron as a Temporal Filter

Linear Temporal Filter:

$$Discrete\ Form:\ r(t) = \sum_{\mathbf{k = 0}}^{\mathbf{n}}{\mathbf{s}_{\mathbf{t - k}}\mathbf{\bullet}\mathbf{f}_{\mathbf{k}}}$$

$$Continuous\ Form:\ r(t)\ = \ \int_{- \infty}^{t}{d\tau\ s(t - \tau) \bullet f(\tau)}$$

The Running Average Filter In this filter we take N time points and we average them.

Linear Temporal Filter:

$$Discrete\ Form:\ r(t) = \sum_{\mathbf{k = 0}}^{\mathbf{n}}{\mathbf{s}_{\mathbf{t - k}}\mathbf{\bullet}\mathbf{f}_{\mathbf{k}}}$$

$$Continuous\ Form:\ r(t)\ = \ \int_{- \infty}^{t}{d\tau\ s(t - \tau) \bullet f(\tau)}$$

The Leaky Average Filter

Linear Temporal Filter:

$$Discrete\ Form:\ r(t) = \sum_{\mathbf{k = 0}}^{\mathbf{n}}{\mathbf{s}_{\mathbf{t - k}}\mathbf{\bullet}\mathbf{f}_{\mathbf{k}}}$$

$$Continuous\ Form:\ r(t)\ = \ \int_{- \infty}^{t}{d\tau\ s(t - \tau) \bullet f(\tau)}$$

Basic Model of Linear Spatial Filtering (against the previous temporal filtering) This filter is local in space. The center is weighted positively, while the surround is weighted negatively (On/Off).

Combining Temporal and Spatial Filtering This is most likely what the brain is doing, i.e., integrate not only across space but also across time.

Combining Filtering with a Nonlinearity

Problems:

• Can spike rates be negative? (No)
• What happens if the stimulus becomes stronger and stronger?
• Can firing rates increase indefinitely? (No) How can we refine our fitting/modelling to be more biologically-plausible? The downside of this fitting is that I lose information relative to the dynamic range of encoding.

Linear Filter + Nonlinearity: $r(t)\ = \ g(\int_{\ }^{\ }{d\tau\ s(t - \tau) \bullet f(\tau))}$

Taking into Account Spatio-temporal Features

To Measure Population Activity in vivo it is possible to use Electrodes and Ca2+ Imaging. Now, we want to repeatedly sample the responses to a variety of stimuli so that we can characterize what feature combination triggers a spike or a behavior.

$$P\left( response \middle| stimulus \right) = P\left( response \middle| s_{1},s_{2},s_{3},\ldots,s_{n} \right)$$

After collecting data, if we don't have any labels for the stimuli, we use an unsupervised/clustering approach, otherwise a supervised approach to identify the characteristics that trigger a behavior.

Population coding refers to information available from ensembles that goes beyond simple summation of individual signals. It is often associated with the method of Georgopoulos et. al. (1996), but many scientists have also asked what an "ideal observer" could learn from a population of neurons.

• Different cells encode different ranges of the stimulus.
• Averaging over a population is often meaningless.
• Allows accurate reconstruction of the signal, also interpolated between peaks.
• Sparse coding: only few cells are activated.
• Retina as an example: different cells for different light wavelengths.
• A neuron encodes a stimulus, a neuronal population encodes behavior.

Finding the Single Neuron Response Vector & Projecting Stimuli in the direction of Neuronal Response (Encoding/Filtering)

Finding the I/O Function for a Single Neuron

The I/O function is:

$$P\left( spike \middle| stimulus \right) = P\left( spike \middle| s_{1} \right)$$

Where $s_{1}$ as identified by our linear filter.

The I/O function can be found from data using the Bayes' rule: $P\left( spike \middle| s_{1} \right) = \frac{P\left( s_{1} \middle| spike \right)P(spike)}{P\left( s_{1} \right)}$

Population Distance Metrics