What happens in your brain when you read lips in a noisy room?

Updated

Visual cortex decodes lip movements, temporal cortex recognizes mouth and face shapes, premotor cortex subtly simulates the speech movements, Broca's area predicts the missing words, and auditory cortex merges this prediction with the noisy sound.

Visual Cortex, Temporal Cortex, Premotor Cortex, Broca's Area, Auditory Cortex lit up on the Brain Atlas 3D brain: what happens in your brain when you read lips in a noisy room
Figure. The brain regions that light up when you read lips in a noisy room, in order: Visual Cortex (lip tracking), then Temporal Cortex (face recognition), then Premotor Cortex (motor simulation), then Broca's Area (speech prediction), then Auditory Cortex (audio-visual merge).

Watch it light up on the 3D brain

Brain regions that light up

OrderRegionWhat it does here
1Visual CortexThe visual cortex processes the rapid movements of the speaker's lips.
2Temporal CortexTemporal cortex identifies the mouth and facial gestures as meaningful speech cues.
3Premotor CortexPremotor cortex may subtly simulate the lip movements, which some researchers think aids understanding.
4Broca's AreaBroca's area uses the simulated movements to predict the words being said.
5Auditory CortexAuditory cortex integrates the predicted words with the noisy sound to confirm what was said.

1. Visual Cortex: lip tracking

Step 1 of what happens in your brain when you read lips in a noisy room: Visual Cortex lit on the Brain Atlas 3D brain
Figure 1. Visual Cortex, lip tracking: The visual cortex processes the rapid movements of the speaker's lips.

Primary visual cortex decodes the rapid lip shapes and their timing from the incoming visual stream. This bottom-up sensory input becomes critical when the auditory signal is masked by background noise, making vision the primary channel for speech information.

2. Temporal Cortex: face recognition

Step 2 of what happens in your brain when you read lips in a noisy room: Visual Cortex, Temporal Cortex lit on the Brain Atlas 3D brain
Figure 2. Temporal Cortex, face recognition: Temporal cortex identifies the mouth and facial gestures as meaningful speech cues.

The temporal cortex, particularly regions specialized for face and biological motion, categorizes the dynamic lip patterns. It links visual forms to stored knowledge about how speech looks, enabling recognition of distinct phonemes.

3. Premotor Cortex: motor simulation

Step 3 of what happens in your brain when you read lips in a noisy room: Visual Cortex, Temporal Cortex, Premotor Cortex lit on the Brain Atlas 3D brain
Figure 3. Premotor Cortex, motor simulation: Premotor cortex may subtly simulate the lip movements, which some researchers think aids understanding.

Mirror-like activity in premotor cortex maps observed lip actions onto your own motor repertoire. This internal simulation helps decode the intended speech sounds by activating the same circuits used for producing them.

4. Broca's Area: speech prediction

Step 4 of what happens in your brain when you read lips in a noisy room: Visual Cortex, Temporal Cortex, Premotor Cortex, Broca's Area lit on the Brain Atlas 3D brain
Figure 4. Broca's Area, speech prediction: Broca's area uses the simulated movements to predict the words being said.

Broca's area, typically for speech production, reconstructs the likely linguistic message from the motor plan. It fills in missing syllables by anticipating the intended utterance, generating a top-down prediction of the speech content.

5. Auditory Cortex: audio-visual merge

Step 5 of what happens in your brain when you read lips in a noisy room: Visual Cortex, Temporal Cortex, Premotor Cortex, Broca's Area, Auditory Cortex lit on the Brain Atlas 3D brain
Figure 5. Auditory Cortex, audio-visual merge: Auditory cortex integrates the predicted words with the noisy sound to confirm what was said.

The auditory cortex receives top-down predictions from language areas and bottom-up noisy input from the ears. This cross-modal binding sharpens the perceived speech signal, allowing you to understand despite the degraded auditory information.

Good to know

  • The McGurk effect shows visual lip movements can override auditory speech perception, demonstrating strong audiovisual integration.
  • Mirror-like activity in premotor cortex is thought to occur during both performing and observing actions, potentially supporting motor simulation in lip reading.
  • In noisy environments, the brain relies more on visual speech cues to disambiguate sounds, a process known as audiovisual speech enhancement.

Lip reading vs. listening in quiet

In quiet listening, auditory cortex dominates and visual areas stay relatively inactive. In noisy lip reading, visual cortex and premotor cortex show strong activation to predict speech, while auditory cortex still integrates but relies on top-down signals.

Test yourself

  1. Which brain region first processes the visual information from lip movements?

  2. What role does Broca's area play in lip reading?

  3. Why is lip reading more important in a noisy room than in a quiet room?

0 of 3 answered · 0 correct

Simplified: many regions work together at once, and this shows the main players in the order they act.