The effects of oxytocin and galantamine on objectively-defined vocal and facial expression: Data from the CIDAR study

Ambulatory audio and video recording provides a wealth of information which can be used for a broad range of applications, including digital phenotyping, telepsychiatry, and telepsychology. However, these technologies are in their infancy, and guidelines for their use and analysis have yet to be established. The current project used ambulatory assessment data from individuals with schizophrenia (N = 52) and controls (N = 55) over a week to assess factors influencing sufficiency and useability of video and audio data. Logistic multilevel models examined the effect of relevant variables on video provision and video quality. There was no difference by group in video provision or quality. Videos were less likely to be provided later in the study and later in the day. Video quality was lower later in the day, particularly for controls. Participants were more likely to provide videos if alone or at home than in other settings. Black participants were less likely to have analyzable video frames than White participants. These results suggest potential racial disparities in camera technologies and/or facial analysis algorithms. Implications of these findings and recommendations for future study development, such as instructions to provide to participants to optimize video quality, are discussed.

Blunted facial affect is a transdiagnostic component of Serious Mental Illness (SMI) and is associated with a host of negative outcomes. However, blunted facial affect is a poorly understood phenomenon, with no known cures or treatments. A critical step in better understanding its phenotypic expression involves clarifying which facial expressions are altered in specific ways and under what contexts. The current literature suggests that individuals with SMI show decreased positive facial expressions, but typical, or even increased negative facial expressions during laboratory tasks. While this literature has coalesced around general trends, significantly more nuance is available regarding what components facial expressions are atypical and how those components are associated with increased severity of clinical ratings. The present project leveraged computerized facial analysis to test whether clinician-rated blunted affect is driven by decreases in duration, intensity, or frequency of positive versus other facial expressions during a structured clinical interview. Stable outpatients meeting criteria for SMI (N = 59) were examined. Facial expression did not generally vary as a function of clinical diagnosis. Overall, clinically-rated blunted affect was not associated with positive expressions, but was associated with decreased surprise and increased anger, sadness, and fear expressions. Blunted affect is not a monolithic lack of expressivity, and increased precision in operationally defining it is critical for uncovering its causes and maintaining factors. Our discussion focuses on this effort, and on advancing digital phenotyping of blunted facial affect more generally.

Negative symptoms reflect one of the most debilitating aspects of one of the most debilitating diseases known to humankind. As yet, our treatments for negative symptoms are palliative at best and our understanding of their causes is relatively superficial. To address this, we are developing objective ambulatory tools for digitally phenotyping their severity which can be used outside the confines of the traditional clinical and research settings. The present study evaluated the feasibility, reliability and validity of ambulatory vocal acoustic and facial emotion expression analysis. Videos were provided by 25 patients with schizophrenia or schizoaffective disorder and 27 nonpsychiatric controls using inexpensive, non-invasive ambulatory recording methods. Controls provided 411 video recordings, and patients provided 377 video recordings; an average of 15.22 and 14.50 per participant per group respectively. The vast majority (over 80%) of these videos were usable for analysis. An empirically-supported, limited-feature vocal (7 features) and facial (3 features) set was examined. Within participants, these features varied considerably over time, but showed moderate to good test-retest reliability in many cases once contextual factors (e.g., activity involved in at the time of testing) were accounted for. Vocal and facial features showed statistically significant convergence with a “gold standard” negative symptom measure. Ambulatory vocal/facial features were more strongly associated with engagement in social or work activities in patients than negative symptom ratings. These data support the use of ambulatory vocal/facial analytic technologies for digital phenotyping of these negative symptoms.

Voice atypicalities have been a characteristic feature of schizophrenia since its first definitions. They are often associated with core negative symptoms such as flat affect and alogia, and with the social impairments seen in the disorder. This suggests that voice atypicalities may represent a marker of clinical features and social functioning in schizophrenia. We systematically reviewed and meta-analyzed the evidence for distinctive acoustic patterns in schizophrenia, as well as their relation to clinical features. We identified 46 articles, including 55 studies with a total of 1254 patients with schizophrenia and 699 healthy controls. Summary effect sizes (Hedges'g and Pearson's r) estimates were calculated using multilevel Bayesian modeling. We identified weak atypicalities in pitch variability (g = −0.55) related to flat affect, and stronger atypicalities in proportion of spoken time, speech rate, and pauses (g's between −0.75 and −1.89) related to alogia and flat affect. However, the effects were mostly modest (with the important exception of pause duration) compared to perceptual and clinical judgments, and characterized by large heterogeneity between studies. Moderator analyses revealed that tasks with a more demanding cognitive and social component showed larger effects both in contrasting patients and controls and in assessing symptomatology. In conclusion, studies of acoustic patterns are a promising but, yet unsystematic avenue for establishing markers of schizophrenia. We outline recommendations towards more cumulative, open, and theory-driven research.