Notes

Aversion Learning - Introduction

Some major circuits underlying innate and learned behaviour directed at: Aversion and Reward

Some Major Psychological Components of Processing Aversive Stimuli At psychological level stimuli can be either Innate/Inborn or Learned. An example of Innate stimuli is the communication of a danger through pheromones (chemosignals) which induces an unconditioned response, i.e., either freeze or flight. Built on genomically encoded response, a conditioned response can be learned.

• Pavlovian Stimulus - Stimulus Learning, in these experiments the animals learns to associate a conditioned stimulus (CS) (e.g., a bell ringing) to an unconditioned stimulus (US) (e.g., an electric shock). If this neutral stimulus (CS) can reliably predict the US, then such neutral stimulus will take on the emotional properties of the US. In this example, the response chosen by the mice is freezing.
• Operant Learning Response - US - US - Response, in this scenario the mice is in a conditioning where if a tone is presented the shock can occur in two situations:
  • In the first case, the shock occurs if the mice moves in another chamber of the cage and thus the mice learn to act Passive Avoidance = R - US Learning.
  • Otherwise, if the shock occurs in the same chamber the tone appeared, the mice will move to another chamber, thus learning Active Avoidance = R - US Learning. In this scenario, the mice shows a goal-directed behaviour, however they have no self-awareness.

Aversion Learning: What Is Learned Determines the Emotional-Behavioral State

Aversion Learning - Amygdala

Some Major Circuits in Aversion Processing: Focus on Amygdala People with lesioned amygdala face problems in feeling emotions. The amygdala is deeply involved in learning of aversive stimuli. The amygdala is made of several nuclei and the most important ones involved in learning aversive stimuli are: Basolateral Complex (BLA) and Central Nucleus (CEA). The amygdala is mainly composed of Glutamate and GABAergic inhibitory neurons.

Pavlovian, Classical or Stimulus-Stimulus Conditioning: Tone-Electroshock Conditioned Freezing

We know the importance of amygdala from these experiments: we have the animal in a chamber where a tone is played and a shock is exerted. If we present a neutral stimulus such that the end of the tone coincides with the shock, then we will quickly see not just an unconditioned response, but also a conditioned one. The conditioned stimulus takes on the emotional properties of the unconditioned one. At the first trial, the mice won't have any conditioned response, but already from the second trial the mice will learn (with a certain probability) and show a conditioned response. By increasing the salience of the unconditioned stimuli, the learning drastically increase as well. (Approx. from 50% after first trial to approx. 80% after first trial).

Amygdala Microcircuitry Cortex-like structure comprising glutamate long-range projections neurons (80%) and GABA interneurons (e.g., parvalbumin, somatostatin) (20%).

It is an example of Hebbian Associative Learning, when pavlovian conditioning is performed both the auditory system and the somatosensory system are involved, both projecting to thalamus, then cortices and then to through glutamate neurons in amygdala. As a result of some trials, the auditory neurons stimulated by the conditioned stimulus are capable of exciting the lateral amygdala neurons that were before only stimulated by the unconditioned one. However, the amygdala is not efficient in detecting and predicting unconditioned stimuli when a time delay occurs between the tone and the electroshock, in such cases the Hypothalamus plays a major role in predicting the unconditioned stimulus. So, the stimulus arrives to the central amygdala which in turn excites the lateral Hypothalamus (LH) that controls blood pressure, the Para-Ventricular-Nucleus (PVN) where Hormones are causing release of Cortisol are synthesized, which will then go back into the Amygdala to fixate emotional memory.

Optogenetic Investigation of Neural Circuits Underlying Emotional Behaviour AAV transduction-expression of light-sensitive proteins (opsins) in a specific regions/cell populations allows for the experimental regulation of the firing activity of those regions/cells. Optogenetic has been widely utilized to increase understanding of Pavlovian Aversion Learning.

Pavlovian Aversion Learning: Importance of Amygdala In Mouse, Optogenetic Inhibition (halorhodospin) of Basolateral amygdala (BLA) Glutamate neurons projecting to Central Nucleus (CeM) reduces Pavlovian (CS-US) aversion learning. Hence, they used an optic light to induce optogenetic inhibition when the unconditioned stimulus was presented. The inhibition of such pathway reduced learning, demonstrating that this pathway is important in the regulation of pavlovian aversive learning conditioning.

Aversion Learning - Habenula, Tegmentum, PAG, LHb, RMTg

Some Major Circuits in Aversion Processing: Focus on Habenula and Tegmentum In order to understand more of the behavioral responses, we have to go in the Periaqueductal Gray (PAG) or the Rostromedial Tegmental Nucleus (RMTg). There is a projection from the central amygdala to the PAG, so the GABAergic neurons are sending long-range projections.

Some Major Circuits in Aversion Processing: Focus on PAG, LHb, RMTg The Periaqueductal Gray contains GABAergic neurons targeted by GABA neurons from the central amygdala which are projecting to glutamate neurons in the PAG. When the neurons in central amygdala fires (i.e., when the lateral amygdala detects a CS or US) then they will inhibit GABA neurons in PAG, which will disinhibit Glutamate neurons in the PAG, these neurons project then to the Lateral Habenula (LHb) which excites VTA GABA interneurons and Rostromedial Tegmental Nucleus (RMTg) , which is in turn capable of inhibiting the dopamine neurons in the Ventral Tegmental Area (VTA). Hence, one of the key features of aversive stimuli processing is the inhibition of dopamine release, which is one of the main neurotransmitters underlying reward behavior.

Neural Circuitry of Pavlovian Aversion Behaviour Unconditioned response is Flight, Conditioned response is Freezing. If the neural circuitry underlying conditioned and unconditioned stimuli is the same, how is it possible that the reaction elicited is different?

Amygdala to PAG Pathway In Mouse, optogenetic excitation (ChR2) of dorsal Periaqueductal gray (dPAG) glutamate neurons causes: (1) Increased Running/Flight during dPAG neuronal firing. (2) Increased Freezing after dPAG neuronal firing.

It suggests 2 separate populations of neurons in dPAG: (1) Neurons for UR = Flight. (2) Neurons for CR = Freeze.

Importance of LHb/RMTg to Dopamine-Neuron Circuit to Aversion Processing Experiment in Macaques: a point is showed on a computer screen. If the point moves in a direction, then it is predictive of a tone and a reward. If the point moves in the other direction, then there is a different tone and no reward coming. Here the absence of reward is the aversive stimulus. What was seen is that:

• No-Reward CS results in phasic excitation of LHb neurons and phasic inhibition of VTA DA neurons.
• Reward CS results in phasic inhibition of LHb neurons and phasic excitation of VTA DA neurons. In the case of rats:
• Aversive CS or US results in phasic excitation of RMTg GABA neurons.
• Reward CS or US results in phasic inhibition of RMTg GABA neurons.

Some Major Circuits in Aversion Processing: Focus on Prefrontal Cortex Medial Prefrontal Cortex has long-range Glutamate connections to the Amygdala Basolateral complex (BLA) and the Nucleus Accumbens. It also receives dopaminergic neurons from VTA. PFC is fundamental in goal-directed behaviour.

Aversion Learning: Importance of Prefrontal Cortex to Amygdala Circuit In Mouse, Optogenetic excitation of infralimbic Prefrontal Cortex (vmPFC) glutamate neurons projecting to Basomedial amygdala (BMA) decreases freezing behaviour related to Pavlovian (CS-US) aversion learning - increased extinction learning (gradual decrease in response to a conditioned stimulus that occurs when the stimulus is presented without reinforcement) (Similar behavioural effect is observed with optogenetic excitation of BMA Glutamate neurons directly).

Overview of a Major Neurocircuit in Aversion Processing and Subsequent Behaviour

Reward Learning

Some Major Psychological Components of Processing Reward Stimuli The Innate and Learning responses underlying reward systems are the same of the aversive stimuli learning. However, experiments show that Pavlovian to Operant Transfer is possible, i.e., mice can learn to combine the two types of stimuli to reach the reward.

Pavlovian Reward Learning: Importance of VTA-NAc Dopamine Neurons

Pavlovian learning involves acquisition by a neutral stimulus of the emotional properties of an US based on their close temporal association. Dopamine neurons responding to UCS (Uncoditioned Reward Stimulus): High VTA Dopamine Neuron Burst firing. Once the association is learned, dopamine neurons respond shifts towards the conditioned stimulus (CS) and only slightly fire when reward is delivered. Thus, dopamine neurons respond to "Reward Prediction Error".