Elsevier

Schizophrenia Research

Volume 159, Issue 1, October 2014, Pages 188-192
Schizophrenia Research

Limited practice effects and evaluation of expectation for change: MATRICS Consensus Cognitive Battery

https://doi.org/10.1016/j.schres.2014.08.004Get rights and content

Abstract

Understanding of the impact of antipsychotic medications on cognition requires differentiating between treatment effects and practice effects. This prospective study examines expectations for change on neuropsychological assessments and possible differential practice effects in community dwelling schizophrenia patients (n = 27) who are clinically stable and on a stable medication regimen when compared to demographically similar psychiatrically healthy controls (n = 29). All participants were administered the MATRICS Consensus Cognitive Battery (MCCB) twice over a period of four weeks. The use of Regression Based Norms for Change (RBNC) and Reliable Change Index (RCI) was completed to anchor estimates of meaningful change to a demographically similar control group. A repeated measures ANOVA was used to examine the effects of time and diagnosis on MCCB composite scores. A repeated measures MANOVA was used to examine the effects of time and diagnosis, and their interaction for MCCB subtests. Estimates of meaningful change are provided. A significant main effect was observed for time; no significant interactions were observed. There was no support for differential practice effects. In the absence of any behavioral, cognitive, or pharmaceutical interventions, these findings suggest limited change in performance over time in either group.

Introduction

Differentiating between treatment effects and practice effects is essential to understanding the impact of antipsychotic medications and cognitive rehabilitation techniques on cognition. One challenge facing schizophrenia researchers is estimating the extent of practice effects (Goldberg et al., 2010). Few clinical trials have included healthy control groups to quantify the impact that repeated exposure to the testing environment. Further, it is impractical/unethical to delay study treatments to establish practice effects on cognitive outcome measures in untreated patients. Despite the potential confounds associated with practice effects, improved cognitive performance has often been attributed to pharmacological treatments. This has led some authors to contend that improved test performance may not differ from practice effects observed in healthy controls.

One approach to addressing this issue is to assess practice effects with schizophrenia patients who are clinically stable and on a stable medication regimen compared to a demographically similar psychiatrically healthy control group tested at the same interval (Goldberg et al., 2010). By assessing change in this matter, one can better isolate practice effects from treatment effects in the context of clinical trials.

Cognitive impairments have been well documented in patients with schizophrenia (Blanchard and Neale, 1994, Bilder et al., 2000). These impairments are present across a wide range of cognitive domains, including executive function, visual attention, auditory attention, verbal memory, visual memory, motor skills, and visual perception (Heinrichs and Zakzanis, 1998, Hill et al., 2004), yet impairments can be heterogeneous in nature (Heinrichs et al., 2008). Level of severity typically ranges from moderate to severe (Heinrichs and Zakzanis, 1998), and these generalized cognitive impairments seem to play a critical role in functional outcome.

When accounting for improved performance over time, medication effects and/or practice effects are two of the primary explanations in schizophrenia spectrum populations. When investigating specific cognitive domains, practice effects have been observed across a broad range of cognitive domains in the context of clinical trials and research reports within schizophrenia spectrum populations (Hill et al., 2004, Goldberg et al., 2007, Goldberg et al., 2010). Interestingly, minimal practice effects have been observed on the composite score of the MCCB (Keefe et al., 2011).

One methodological challenge facing clinical trials evaluating either first generation antipsychotics (FGA) or second generation antipsychotics (SGA) has been the absence of a control group. By including a control group, one can quantify the impact that exposure to the testing environment has on improving performance during subsequent assessments. When nonpsychotic groups were compared to schizophrenia patients using SGA at similar testing intervals, the apparent cognitive benefit of SGA was consistent with practice effects observed in healthy populations (Goldberg et al., 2007). Thus, performance on cognitive measures may be artificially inflated at retest due to familiarity with the tests, increased comfort and confidence in test taking strategies, and other practice effects.

Differentiating between treatment effects and practice effects can be challenging. However, this is an area that is critical to understanding the cognitive impact of antipsychotic treatments and cognitive rehabilitation interventions. One way to address the issue of differentiating between treatment effects and practice effects is to establish expectations for meaningful change on neuropsychological measures. By establishing expectation for meaningful change through assessing stable schizophrenics on a stable medication regimen with no behavioral or therapeutic interventions, with a demographically similar control group, one would be able to better estimate practice effects. A unique way to complete this is to use conservative statistical techniques, such as RBNC and RCI, to anchor retest scores to demographically similar controls to establish expectations for change.

Prior studies assessing practice effects in schizophrenia are complicated by changes in medication status (clinical trials) or have failed to include controls to provide normative expectations for practice effects. Thus, this study was designed to prospectively examine practice effects in both clinically stable schizophrenia patients on stable medication regimens and demographically similar healthy controls. Further, to compliment that approach the present study also aims to provide estimates for meaningful change on the MCCB, based on RCI estimates anchored to the control group with RBNC statistics. This will provide future studies with a reference point for a range of expected variability on the MCCB. As with prior studies (Goldberg et al., 2007, Nuechterlein et al., 2008, Goldberg et al., 2010) practice effects were expected to be similar for both groups and no differential practice effects (i.e., one group experiencing greater practice effects) were predicted.

Section snippets

Participants

Participants were 27 community dwelling individuals who met criteria for schizophrenia spectrum disorders and 29 psychiatrically healthy individuals based on Structured Clinical Interview for DSM-IV (SCID). Patients were referred by primary care physicians, who provided medication status and dosage for each patient. Healthy participants were recruited from the community via local advertisements and a research registry. To limit effects of both acute illness and recent changes to medication

Repeated measures MANOVA

Repeated measures ANOVA for the MCCB composite score revealed a significant main effect for time [F (1, 54) = 13.40, p = .001], (effect size: d = 0.21; ŋp2 = 0.20) indicating that all groups improved over time. There was no main effect of diagnosis [F (1, 54) = 0.47, p = 0.50], (effect size: d = 0.18; ŋp2 = 0.01). The time by diagnosis interaction was non-significant [F (1, 54) = 0.08, p = 0.78] (effect size: d = 0.15; ŋp2 = 0.002), thus there was no evidence of differential practice effects over time. A repeated

Discussion

Differentiating between treatment effects and practice effects is essential to understanding the impact of antipsychotic medication on cognition. However, understanding the impact that practice effects have on cognition and cognitive improvement over time is a challenge for researchers in this area. One objective of this prospective study was to explore expectations for change due to practice effects by evaluating clinically stable schizophrenia patients on a stable medication regimen in the

Role of funding source

This study was supported by funds received from NIH/NIMH (MH072767 & MH083888). The funding source played no role in data analysis or interpretation.

Contributors

Peter Weiden, MD: Subject recruitment services.

Ellen Herbener, PhD: Clinical ratings and independent diagnostic evaluations.

Conflict of interest

There are no conflicts of interest to report.

Acknowledgments

This study was supported in part by NIMH grants MH072767 and MH083888.

References (19)

There are more references available in the full text version of this article.

Cited by (13)

  • Neurognitive function and symptom remission 2 years after ECT in major depressive disorders

    2019, Journal of Affective Disorders
    Citation Excerpt :

    However, the MCCB was assembled partly in order to permit repeated testing, presenting alternate versions of tests most vulnerable to practice effects (Nuechterlein et al., 2008). Moreover, the MCCB has shown limited practice effects across 4 weeks (Roseberry and Hill, 2014). With assessments over longer intervals, we do not expect this type of bias to occur in our data.

  • Significantly improved neurocognitive function in major depressive disorders 6 weeks after ECT

    2016, Journal of Affective Disorders
    Citation Excerpt :

    However, one of the explicit aims behind the MCCB test selection and standardization procedure was the development of a battery that allows repeated assessments at relatively short intervals, and alternate forms of the tests are used when appropriate (Nuechterlein et al., 2008). Moreover, other relevant studies have employed shorter follow-up periods than ours, with no reports of practice effects (Murrough et al., 2015; Roseberry and Hill, 2014). The major strengths of this study are the low drop-out rate and the use of a comprehensive, internationally validated neurocognitive test battery with strong psychometric properties in combination with subjective reports of memory complaints.

  • Cognitive course in first-episode psychosis and clinical correlates: A 4year longitudinal study using the MATRICS Consensus Cognitive Battery

    2015, Schizophrenia Research
    Citation Excerpt :

    The MCCB identified seven separable dysfunctional cognitive factors in the disorder, two of which (attention and working memory) are concurrent with the cognitive constructs of the NIMH Research Domain Criteria (RDoC) (Cuthbert and Insel, 2010). The MCCB demonstrates excellent test–retest reliability and minimal practice effects (Roseberry and Kristian Hill, 2014; Nuechterlein et al., 2008). Of the few FEP longitudinal studies that implemented the MCCB, Juuhl-Langseth et al. (2014) found that most neurocognitive deficits were relatively stable over two years apart from the course of processing speed.

View all citing articles on Scopus
View full text