Notes

The Fascination of Hearing

Prologue: The Fascination of Hearing The vibration of an object (e.g., tuning fork) causes changes in air pressure. Hence, acoustic sound waves can be described as fluctuations in air pressure due to compression and rarefaction of air molecules. The auditory system transforms sound waves into distinct patterns of neural activity, which are then integrated with information from other senses (and cognition) to guide behavior and promote communication. The auditory system is not only essential for speech, but also for music, emotion recognition, sound localization, etc.

• Frequency range of hearing across species.
• Sound Localization:
  • Having two ears (instead of one) is important for sound localization.
  • Due to the perception of interaural time (ITD) and level (ILD) differences between the two ears we are able to localize sound sources in the environment.
  • Sound localization in the horizontal plane (azimuth) is based on ITD and ILD, while sound localization in the vertical plane is based on spectral filtering by ears, head and shoulders.
• Hearing enables:
  • To perceive and identify object from far away, vision doesn't.
  • To perceive sounds from all around the body with an angle of 360 degrees (vision only 180 degrees).
  • To maps the sound in space and to identify the spatial location of objects.
  • To recognize emotional states (e.g., speech prosody, sad, happy, fear, etc.)
  • Complex human faculties like music and language.

However, there is also a "dark side" of hearing such as hearing loss, tinnitus, amusia, etc.

The Basic Building Blocks of Acoustic Signals: Air Pressure and Waves

• Soundwave as variation in air pressure.
• Complex sound waveform: Each complex signal can be decomposed into sine and cosine waves.
• Each oscillation can be described in terms of amplitude and frequency. Frequency (Hz) = number of periods (cycles) per second.

Sound Processing Along the Hierarchy of the Auditory System

• Peripheral Auditory System: Outer, Middle and Inner Ear.
• Central Auditory System: Auditory Brainstem and Cortex.

The Ear

The Outer Ear The outer ear consists of the pinna, the ear canal and the tympanic membrane. The funnel-shaped pinna "collects" air pressure fluctuations from the environment and directs them through the ear canal to the tympanic membrane (eardrum), which constitutes the transition to the middle ear. The tympanic membrane vibrates in response to sounds (air pressure changes), and this vibration is transmitted to three ossicles situated in the middle ear (malleus, incus and stapes). The length and shape of the ear canal plays an important role in sound amplification.

The Middle Ear The middle ear is separated from the outer ear by the tympanic membrane (eardrum), and is responsible for the conversion of air pressure fluctuations into mechanical energy. The transmission of sounds from the outer to the middle ear takes place through the deflection of the tympanic membrane. The deflection of the tympanic membrane results in vibrations of the three ossicles (malleus, incus and stapes), which in turn convey energy to the fluid-filled cochlea in the inner ear.

The Inner Ear: Cochlea The cochlea is a fluid-filled tube which is divided into two major compartments by the basilar membrane, namely the scala vestibuli and tympani. The movement of the ossicles transfers the mechanical energy to the oval window of the cochlea, which sets the fluid in motion, from the oval to the round window (pressure compensation). The two liquid-filled tubes that run along the cochlea are separated by the basilar membrane. The basilar membrane moves up and down in response to incoming sound waves, which are converted into traveling waves on the basilar membrane. The Corti organ, which is situated on the basilar membrane, is responsible for the transduction of auditory signals into action potentials. The location-specific deflection of hair cells in the Corti organ activated calcium and potassium channels which lead to action potentials.

The Inner Ear: Tonotopic Organization of the Cochlea Basilar membrane vibration to sinusoids varies with frequency because its mechanical properties vary along its length. The basilar membrane is wider at the apex (most responsive to low frequencies) and stiffer at the base (most responsive to high frequencies). The mechanical properties of the basilar membrane are the initial source of tonotopic organization (frequency coding).

Recap

The pinna catches sound waves and deflects them into the external ear canal. Waves are amplified and directed to the eardrum, causing it to vibrate, which in turn vibrates ossicles. Ossicles amplify and convey vibrations to the oval window. Vibration of oval window sends waves through cochlear fluid causing the basilar and tectorial membranes to bend, which in turn cause cilia of outer hair cells, embedded in the tectorial membrane, to bend. This bending generates neural activity in hair cells.

Auditory Brain Pathways & Processing

Brainstem and Central Auditory Pathway

1. Cochlear Nucleus (start sound feature processing, frequency and sound onset/offset).
2. Superior Olives (Interaural Intensity Differences).
3. Inferior Colliculi (Somatosensory connections, multisensory).
4. Medial Geniculate (Integration and connecting to A1).
5. Auditory Cortex (A1).

Mapping of Elementary Auditory Functions in the Cortex

• Spectral and Temporal Auditory Processing:
  • Spectral processing involves the analysis of the frequency content of a sound, which is related to the perception of pitch.
  • Temporal processing involves the analysis of the timing and sequencing of sound events, which is related to the perception of rhythm, tempo and sound onset/offset.
  • The left auditory cortex favors the extraction of information from short temporal integration windows, whereas the right counterpart primarily relies on long integration windows.
• Intensity Coding
  • It refers to the way that the auditory system represents the loudness or amplitude of a sound. Neurons respond to different levels of sound intensity, with some neurons specifically sensitive to low-level sounds and others responsive to high-level sounds. The brain uses this information to construct a representation of the sound intensity and to determine the loudness of the sound.
• Timbre and Complexity Coding
• Auditory Object Recognition and Spatial Location ("What" and "Where" Streams)
  • "What" and "Where" streams in the auditory cortical system of primates (and humans). Similar to "what" and "where" streams in the visual system.
• Motor Perception
  • Moving vs Stationary auditory objects.

Speech Processing and Neural Oscillations

• Speech processing is mediated by neural tracking, which refers to the alignment of neural oscillations with the speech signal.
• High excitability phases of neurons align with information in speech (e.g., syllables, words, regularities, etc.).
• Neural oscillations track different linguistic units of the speech signal. Neural tracking of speech takes place at different time scales. Different methods, for example cross-correlation between the envelope of the speech signal and the EEG signal, phase coherence, etc.

Speech Segmentation Speech is a continuous acoustic signal without reliable gaps in between words or linguistic entities. The challenge is how to recognize word boundaries when no lexicon is available for word recognition. Once the word forms have been recognized, meaning assignment can be achieved through associative and contextual learning.

• Statistical Learning: Computation of transitional probabilities between adjacent syllables.
• Prosodic Bootstrapping: Detection of word boundaries based on rhythm, intonation or lexical stress cues.

The Vestibular System: Rotational Movements and Linear Accelerations Functions of the Vestibular System: register body motions, postural control (Vestibulo-spinal tract), maintain upright posture, measure gravity field, keep eyes still when head moves. Two organs in the inner ear (labyrinth):

• Semicircular tubes filled with fluid:
  • Oriented in three planes, one for each dimension in which we move the head (pitch, roll, jaw). Responsible for the coding of rotational movements.
• Otolith organs containing hair cells, and consisting of utricle and saccule:
  • Responsible for the coding of linear accelerations.
  • Infer body position in relation to gravity.
  • Detect changes in the direction and speed of movements.

Conclusions The auditory system enables to perceive air pressure fluctuations (frequency and amplitude) through complex processing steps in the outer, middle and inner ear. The cochlea is characterized by an excellent spectral (tonotopy) and temporal resolution. The neural codes are transmitted from the cochlea to the cortex via brainstem and central auditory system. Several specialized cortical modules contribute to different aspects of hearing (intensity, timbre and complexity coding, auditory object identification, speech and language processing, etc.). The vestibular system is fundamentally involved in balance and posture processing.