Multimodal perception involves our senses working together to allow us to integrate information to provide an accurate representation of the world around us. The senses did not evolve in isolation and therefore, work together to give us a rich and unified experience. This integration is believed to occur in specific locations within the brain.
There are zones in the human brain where sensory information comes together and is integrated such as the Auditory, Visual and Motor Cortices. [Image: BruceBlaus]
The McGurk Effect.
So, hearing and visual cues work together when we are perceiving speech. When a person hears a different sound than what the visual information suggests it can cause a perceptual illusion, this is known as the ‘McGurk effect‘. Watch the clip below, to see the effect in action although it is believed that not everybody is susceptible to it.
When the senses conflict, depending on the stimulus being processed, one sense will override the other, in this case, our sight changes how we perceive what we hear being said. This is an effect that, just because you are aware of it, doesn’t mean it can ‘manually overridden’, our knowledge of the process does not reduce its impact upon our sesnses.