Cognitive Assessment Interview (CAI): Validity as a co-primary measure of cognition across phases of schizophrenia

https://doi.org/10.1016/j.schres.2016.01.028Get rights and content

Abstract

Background

Progress has been made in developing interview-based measures for the assessment of cognitive functioning, such as the Cognitive Assessment Interview (CAI), as co-primary measures that compliment objective neurocognitive assessments and daily functioning. However, a few questions remain, including whether the relationships with objective cognitive measures and daily functioning are high enough to justify the CAI as an co-primary measure and whether patient-only assessments are valid.

Methods

Participants were first-episode schizophrenia patients (n = 60) and demographically-similar healthy controls (n = 35), chronic schizophrenia patients (n = 38) and demographically similar healthy controls (n = 19). Participants were assessed at baseline with an interview-based measure of cognitive functioning (CAI), a test of objective cognitive functioning, functional capacity, and role functioning at baseline, and in the first episode patients again 6 months later (n = 28).

Results

CAI ratings were correlated with objective cognitive functioning, functional capacity, and functional outcomes in first-episode schizophrenia patients at similar magnitudes as in chronic patients. Comparisons of first-episode and chronic patients with healthy controls indicated that the CAI sensitively detected deficits in schizophrenia. The relationship of CAI Patient-Only ratings with objective cognitive functioning, functional capacity, and daily functioning were comparable to CAI Rater scores that included informant information.

Conclusions

These results confirm in an independent sample the relationship of the CAI ratings with objectively measured cognition, functional capacity, and role functioning. Comparison of schizophrenia patients with healthy controls further validates the CAI as an co-primary measure of cognitive deficits. Also, CAI change scores were strongly related to objective cognitive change indicating sensitivity to change.

Introduction

A good deal of progress has been made in developing interview-based measures of cognition such as the Cognitive Assessment Interview (Ventura et al., 2010b) and the Schizophrenia Cognition Rating Scale (Keefe et al., 2006), as a method for assessing cognitive functioning. These measures have been shown to be reliable, valid, and sensitive to treatment effects (Keefe et al., 2015, Ventura et al., 2013). Early promising work on the CAI has shown modest relationships between interview-based measures and the following domains: objective cognition functioning (MATRICS Consensus Cognitive Battery; MCCB), functional capacity (UCSD Performance-based Skills Assessment: UPSA), and role functioning (Role Functioning Scale; RFS) (Green et al., 2011, Ventura et al., 2010a). In particular, recent validity work has shown stronger relationships with objective cognition and multiple domains of role functioning such as independent living, social interactions, family relationships, and school/work functioning (Ventura et al., 2013). Evidence is mounting which indicates that interview-based approaches to the assessment of cognition can serve a useful role as co-primary measures that capture elements of objective cognitive functioning that are related to role functioning (including: independent living, social relationships, school work functioning, and familial interactions).

As part of the MATRICS initiative, the Food and Drug Administration (FDA) suggested that the use of a single, objective cognitive performance-based measure should be adopted as part of a standardized approach to assessment for the approval of cognitive enhancing drugs. In addition, the FDA decided to require documentation of improvements on a functionally meaningful co-primary measure that has face validity for patients and clinicians (Buchanan et al., 2005). The UPSA was selected by the MATRICS committee as a recommended co-primary measure because of its high correlation with objective neurocognitive functioning. Given the importance of Patient Reported Outcome (PRO) measures in FDA clinical trials, further research on the relationship between the CAI and the MCCB could indicate that the CAI would prove to be useful as a co-primary measure. Research has shown that the CAI is moderately correlated with objective neurocognitive functioning (MCCB), functional capacity (UPSA), and role functioning (RFS). In addition, the CAI is close to patient reported functioning because patients and an informant are asked for their impression of the patient's cognitive functioning. The CAI is also a potential candidate for assessing cognitive change in clinical practice. However, more work is needed to understand how a CAI rater should go about obtaining informant information about a patient's cognitive functioning and whether patient ratings alone are sufficiently valid.

The CAI, is being proposed for use in situations where objective cognitive assessments are not practical, such as in medication management in clinical practice, when measurement of co-primary variables is needed such as in clinical trials, or when assessments that are more closely related to the patient's experience are desired. Several studies have already shown that the CAI is related to object cognitive performance, with correlations ranging from r = −.31 to r = −.41 and role functioning range from r = .32 to r = .49 (Ventura et al., 2010a, Ventura et al., 2013). Despite these moderate relationships between the CAI and objective measures of cognitive performance, the CAI is not being suggested as substitute for objective cognitive testing. Also, the CAI is not a measure of role functioning, but rather an assessment of how cognition influences functioning. The CAI has shown very high test–retest reliability (r = .83, n = 93) indicative of small practice effects making the CAI highly repeatable (Ventura et al., 2013). Even though practice effects are also small with the MCCB, the length of the battery administration and scoring process makes the MCCB less practical for routine aspects of clinical practice such as medication management.

As is the case with several interview-based instruments in psychiatry, the assumption is that the psychopathology being evaluated is not present, or is only observed at very low levels, in healthy individuals. Most schizophrenia patients assessed with the CAI are rated an average of 3.5 on a scale of “1” to “7,” where “7” indices severe impairment. There is an assumption that healthy individuals would score a “1” and are operating in their daily lives without cognitive impairments interfering with their functioning. Data to document that the ratings on the CAI for healthy subjects are indeed typically “1” would be very helpful for interpretation. Along those lines, the prevalence of cognitive difficulties that impact functioning in the general population has not been assessed with the CAI. The availability of normative data would aid in estimating the magnitude of cognitive change over time relative to the size of the initial deficit. Community normative data for the CAI could facilitate a valid interpretation of severity of cognitive difficulties across research settings and studies.

This aim of this study was to further explore the validity of CAI by examining: 1) cross-sectional relationships with objective cognitive performance tests (MCCB), functional capacity (UPSA), and role functioning (RFS) in first-episode schizophrenia patients, 2) the effect sizes for CAI ratings in schizophrenia patients relative to those of healthy controls, and 3) the sensitivity of the CAI to changes in objective measures of cognition (MCCB). In addition, we evaluated whether CAI ratings based on patient interview only, compared to the CAI ratings based on a combination of patient and informant information, correlated similarly with objective cognitive performance tests, functional capacity, and role functioning.

Section snippets

Subjects

The sample consisted of 60 first-episode schizophrenia patients and 35 demographically-similar controls participating in the pilot studies and the fourth phase of an NIMH-funded project focusing on the early course of schizophrenia (Nuechterlein et al., 2014, Subotnik et al., 2015), 38 chronic schizophrenia patients and 19 demographically-similar controls from the UCLA Center for Neurocognition and Emotion in Schizophrenia (Table 1). First-episode schizophrenia patients were enrolled in the

CAI discriminates between patients and controls

We used t-tests to address the question of whether patients perform significantly lower than controls on the CAI (Table 2). The CAI Rater score for first episode patients was significantly worse (higher scores indicate worse performance) compared to the matched controls [M = 3.56 vs. 1.36, F(1, 57) = 10.54, p < .01]. Chronic patients also had significantly higher mean CAI Rater scores compared to their matched controls [M = 3.19 vs. 1.35, F(1, 53) = 5.79, p < .01]. The CAI scores at baseline were

Discussion

This is the first study that used an interview-based assessment, the CAI, to assess cognitive functioning in first episode schizophrenia patients. We found that the cognitive deficits known to be present in the early course of schizophrenia can be detected with the CAI, just as they can in chronic patients. This is also the first study to compare CAI ratings in healthy control subjects to those of first-episode and chronic schizophrenia patients. The large effect size found in cognitive

Role of funding source

The funding source did not play a role in the study design, implementation, interpretation of results, preparation of manuscript drafts, or the publication of this paper.

Contributors

The CAI is an interview-based measure of cognition that was developed by Drs. Joseph Ventura, Robert Bilder, Steve Reise, and Richard Keefe. Drs. Keith Nuechterlein, Kenneth Subotnik, and Joseph Ventura obtained funding to conduct the primary study where the data were collected. Drs. Ventura and Nuechterlein trained and supervised research staff on the data collection process and the daily operations of the clinical staff. Dr. Ventura selected the variables of interest, participated in data

Conflict of interest

Joseph Ventura, Ph.D., has received funding from Janssen Scientific Affairs, LLC, Brain Plasticity, Inc., and Genentech, Inc. He has served as a consultant to Brain Plasticity, Inc., and Boehringer-Ingelheim, GmbH.

Kenneth L. Subotnik, Ph.D., has received research funding from Janssen Scientific Affairs, LLC, and Genentech, Inc., through grants to Drs. Nuechterlein and Ventura. He is a consultant to Otsuka America Pharmaceutical, Inc.

Keith H. Nuechterlein, Ph.D., has received funding from

Acknowledgement

Development of this instrument was supported by an unrestricted grant from Pfizer, Inc., and an NIMH R21 grant awarded to Joseph Ventura, Ph.D. This research was also supported by National Institute of Mental Health grants MH37705 (Principal Investigator K.H.N.) and P50 MH066286 (Principal Investigator K.H.N.) and supplemental support from an investigator-initiated grant from Janssen Scientific Affairs, LLC.

The authors would like to thank Robert Bilder, Ph.D., Steve Reise, Ph.D., and Richard

References (26)

  • S.H. Goodman et al.

    Assessing levels of adaptive functioning: the role functioning scale

    Community Ment. Health J.

    (1993)
  • R.S. Green et al.

    Selecting a rating scale for evaluating services to the chronically mentally ill

    Community Ment. Health J.

    (1987)
  • M.F. Green et al.

    Evaluation of functionally meaningful measures for clinical trials of cognition enhancement in schizophrenia

    Am. J. Psychiatr.

    (2011)
  • Cited by (13)

    • The Cognitive Assessment Interview (CAI): Association with neuropsychological scores and real-life functioning in a large sample of Italian subjects with schizophrenia

      2022, Schizophrenia Research
      Citation Excerpt :

      This finding supports the validity of the CAI in capturing the severity of cognitive impairment. In addition, the CAI global composite score was moderately/strongly associated with functional capacity and real-life functioning, in line with previous findings (Ventura et al., 2013; Bosgelmez et al., 2015; Sanchez-Torres et al., 2016; Ventura et al., 2016). Our findings add to this evidence also demonstrating the same pattern of correlations with measures of real-life functioning as those found for the MCCB scores.

    • Assessing Cognition and Social Cognition in Schizophrenia &amp; Related Disorders

      2019, A Clinical Introduction to Psychosis: Foundations for Clinical Psychologists and Neuropsychologists
    • The Cognitive Assessment Interview: A comparative study in first episode and chronic patients with psychosis

      2016, Schizophrenia Research
      Citation Excerpt :

      Among these studies, two report an association of the CAI with a scale of functionality, but only when considering the rater score (Durand et al., 2015; Gould et al., 2015). Our results are in line with those of Ventura et al. (Ventura et al., 2010, 2013, 2016), considering the non-FEP group. Specifically, they found moderate-to-strong correlations between the CAI and the composite MCCB score in three different samples of patients with schizophrenia.

    View all citing articles on Scopus
    View full text