The associations between multisensory temporal processing and symptoms of schizophrenia
Introduction
Hallucinations are a positive symptom in schizophrenia (SCZ) that can present as false perceptions in any sensory modality, but commonly take the form of perceived auditory voices. They are often conceptualized as false attribution of internal voices to an external source. As such, hallucinations in SCZ are frequently linked to the audiovisual speech-perception network, including areas of superior temporal and inferior frontal (i.e., Broca's) cortex (Jardri et al., 2011). One cognitive operation of this network is the integration of information across the auditory and visual systems, forming coherent percepts that comprise our conscious experience (Stevenson et al., 2014a). Speech is a powerful example of audiovisual integration, though integration extends to all manner of sensory inputs: we seamlessly bind together audible speech signals with their associated visual cues, affording substantial behavioral and perceptual benefits, ranging from faster response times (Raab, 1962) to improved speech perception (Sumby and Pollack, 1954) in healthy participants but not as much in SCZ patients. For example, seeing a speaker's visual articulation enhances speech perception under noisy conditions in healthy participants but less so in SCZ patients (Ross et al., 2007). Similarly, SCZ patients are less susceptible to the McGurk effect (Pearl et al., 2009), where the mouth movements an individual sees can alter what they believe to “hear” a speaker to be saying (McGurk and MacDonald, 1976), despite preserved unisensory abilities (Ross et al., 2007).
Impaired sensory integration is a hallmark neurological “soft sign” of SCZ (Heinrichs and Buchanan, 1988) that is often noted at the time of an individual's first psychotic episode and is correlated with SCZ symptomatology (Williams et al., 2010). Most germane to this report is the possible link between alterations in sensory integration and positive symptoms in SCZ, most notably hallucinations (Postmes et al., 2014). Exploring an integration-hallucination link is motivated by the overlap in the neural substrates for audiovisual integration and hallucinations, specifically in regions of the audiovisual speech-perception network. For example, SCZ is associated with structural (Kim et al., 2003) and functional changes within the superior temporal cortex (Surguladze et al., 2001, Szycik et al., 2009). This same area of cortex is heavily implicated in multisensory temporal processing (Stevenson et al., 2010). Furthermore, individuals with SCZ exhibit alterations in temporal processing (Carroll et al., 2008, Davalos et al., 2002, Elvevag et al., 2003, Foucher et al., 2007, Freedman, 1974, Giersch et al., 2009, Lalanne et al., 2012, Tysk, 1983a, Tysk, 1983b, Volz et al., 2001), and impaired audiovisual temporal precision in SCZ has been linked to inaccurately attributing auditory components of speech to temporally disparate visual speech signals (Martin et al., 2013).
Given the relationship between temporal processing and sensory integration (Stevenson et al., 2012b), and links between sensory integration and hallucination in SCZ, we hypothesize that impaired temporal perception in SCZ may be associated with hallucinations in SCZ. To investigate this, we first measured auditory, visual, and multisensory temporal perception in SCZ patients and a group of matched controls, verifying the presence of temporal dysfunction in SCZ and assessing if temporal-perception deficits were uniquely multisensory. Second, and of paramount importance, we measured the severity of hallucinations in SCZ participants with the a priori prediction that changes in multisensory temporal processing would be predictive of hallucinations. This finding would point to shared mechanistic substrates for changes in audiovisual temporal integration and the presence and severity of hallucinations.
Section snippets
Overview
Participants completed four behavioral tasks: two unisensory timing tasks in which participants performed temporal order judgments (TOJ; “Which came first?”) with either auditory or visual stimuli, and two audiovisual timing tasks in which participants performed audiovisual simultaneity judgments (SJ; “Same time or different time?”), one with speech stimuli and one with simple flash-beep stimuli. Finally, participants completed standard metrics assessing SCZ symptomatology. Protocols were
Unisensory temporal perception
Auditory temporal perception was indexed via an auditory TOJ task. Responses were averaged for each SOA for each individual. A mixed linear model (MLM) was then used to measure the impact of SOA and diagnosis on response accuracy (Fig. 1A). In this 2-factor MLM, both factors of diagnosis (F(1,228.7) = 84.57, p < 0.001) and SOA (F(1,26.2) = 58.84, p < 0.001) significantly contributed, but the two did not interact (F(1,56.0) = 0.06, p < 0.81). To quantify this between-group difference, thresholds
Discussion
This study provides a novel view into the relationships between impaired temporal processing, multisensory integration, and hallucinations in SCZ. Three main findings are evident in the data. First, this study confirms that individuals with SCZ show decreased temporal acuity in both auditory and visual perception, as well as in audiovisual temporal perception. Second, SCZ participants exhibit impairments in multisensory temporal acuity that extend beyond these unisensory changes, suggesting a
Conclusions
Our results support the hypothesis that sensory disturbances, specifically those in the temporal processing realm, contribute to hallucinations in SCZ. SCZ is associated with auditory and visual temporal dysfunction, with additional multisensory temporal dysfunction beyond that predicted by these unisensory deficits. These audiovisual temporal perceptual disturbances are also significantly predictive of clinical measures of hallucination severity, supporting the hypothesis that hallucinations
Role of funding
Funding did not impact the outcome of this research in any way.
Contributions
This study was designed by authors RS, SP, CC, LM, and MW and conducted by RS, CC, and LM. Data were analyzed by RS and interpreted by RS, SP, MB, SF, and MW. The manuscript was drafted by RS, and edited and approved by all authors.
Financial disclosures
Dr. Stevenson reported no biomedical financial interests or potential conflicts of interest.
Dr. Park reported no biomedical financial interests or potential conflicts of interest.
Ms. Cochran reported no biomedical financial interests or potential conflicts of interest.
Ms. McIntosh reported no biomedical financial interests or potential conflicts of interest.
Mr. Noel reported no biomedical financial interests or potential conflicts of interest.
Dr. Barense reported no biomedical financial
Acknowledgements
R.S. was funded by a Banting Fellowship from the Canadian National Science and Engineering Research Counsel, the Autism Research Training Program funded by the Canadian Health Institutes of Health Research, and the National Institutes of Deafness and Communicative and Disorders (NIDCD 1F32 DC011993). S.P. was funded by grants from the National Alliance for Research in Schizophrenia and Affective Disorders, and the National Institutes for Mental Health, Gertrude Conaway Vanderbilt Endowment.
References (48)
- et al.
Behavioral, perceptual, and neural alterations in sensory and multisensory function in autism spectrum disorder
Prog. Neurobiol.
(2015) - et al.
Cortical folding abnormalities in schizophrenia patients with resistant auditory hallucinations
NeuroImage
(2008) - et al.
Temporal processing dysfunction in schizophrenia
Brain Cogn.
(2008) - et al.
Low time resolution in schizophrenia lengthened windows of simultaneity for visual, auditory and bimodal stimuli
Schizophr. Res.
(2007) - et al.
Morphology of the lateral superior temporal gyrus in neuroleptic naıve patients with schizophrenia: relationship to symptoms
Schizophr. Res.
(2003) - et al.
Superior temporal gyral volumes and laterality correlates of auditory hallucinations in schizophrenia
Biol. Psychiatry
(1999) - et al.
Temporal event structure and timing in schizophrenia: preserved binding in a longer “now”
Neuropsychologia
(2013) - et al.
Differences in audiovisual integration, as measured by McGurk phenomenon, among adult and adolescent patients with schizophrenia and age-matched healthy control groups
Compr. Psychiatry
(2009) - et al.
Schizophrenia as a self-disorder due to perceptual incoherence
Schizophr. Res.
(2014) - et al.
Impaired multisensory processing in schizophrenia: deficits in the visual enhancement of speech comprehension under noisy environmental conditions
Schizophr. Res.
(2007)
Neural processing of asynchronous audiovisual speech perception
NeuroImage
Discrete neural substrates underlie complementary audiovisual speech integration processes
Neuroimage
Audio-visual speech perception in schizophrenia: an fMRI study
Psychiatry Res.
Audiovisual integration of speech is disturbed in schizophrenia: an fMRI study
Schizophr. Res.
Auditory hallucinations in schizophrenia are associated with reduced functional connectivity of the temporo-parietal area
Biol. Psychiatry
Reduced multisensory integration in patients with schizophrenia on a target detection task
Neuropsychologia
Abnormal cortical folding patterns within Broca's area in schizophrenia: evidence from structural MRI
Schizophr. Res.
Auditory hallucinations and smaller superior temporal gyral volume in schizophrenia
Am. J. Psychiatry
Hallucinations in schizophrenia, sensory impairment, and brain disease: a unifying model
Behav. Brain Sci.
Dementia praecox oder Gruppe der Schizophrenien
Deficits in auditory and visual temporal perception in schizophrenia
Cogn. Neurodyn.
Duration judgements in patients with schizophrenia
Psychol. Med.
Stimulus intensity modulates multisensory temporal processing
Neuropsychologia
The subjective experience of perceptual and cognitive disturbances in schizophrenia. A review of autobiographical accounts
Arch. Gen. Psychiatry
Cited by (95)
Audiovisual illusion training improves multisensory temporal integration
2023, Consciousness and CognitionAbnormal dynamic functional network connectivity in first-episode, drug-naïve patients with major depressive disorder
2022, Journal of Affective DisordersTiming and schizophrenia: Time to get abstract?
2022, Psychiatry ResearchPerceptual inference, accuracy, and precision in temporal reproduction in schizophrenia
2022, Schizophrenia Research: Cognition