The Brief Adherence Rating Scale (BARS) validated against electronic monitoring in assessing the antipsychotic medication adherence of outpatients with schizophrenia and schizoaffective disorder

Article Outline

• Abstract
• 1. Introduction
• 2. Materials and methods
  • 2.1. Participants
  • 2.2. Procedures and measures
    • 2.2.1. Antipsychotic medication
    • 2.2.2. Electronic monitoring and adherence
    • 2.2.3. Brief Adherence Rating Scale
    • 2.2.4. Symptom severity
    • 2.2.5. Raters
• 3. Data analysis
  • 3.1. Comparison of EM and BARS
  • 3.2. Crossinstrument agreement
  • 3.3. Reliability
  • 3.4. Concurrent validity
  • 3.5. Sensitivity and specificity
• 4. Results
  • 4.1. Participant characteristics
  • 4.2. Comparison of EM and BARS
  • 4.3. Crossinstrument agreement
  • 4.4. Internal and test–retest reliability
  • 4.5. Concurrent validity and correlation between adherence and PANSS
  • 4.6. Sensitivity and specificity
• 5. Discussion
• Role of funding source
• Contributors
• Conflict of interest
• Acknowledgments
• References
• Copyright

Abstract 

Among outpatients with schizophrenia, antipsychotic non-adherence is common, grossly under-detected by patients and their prescribers, and is associated with poor clinical outcomes. Using electronic monitoring (EM) as the reference standard we evaluated the reliability and validity as well as the sensitivity and specificity of a recently developed, brief, pencil-paper, clinician-administered adherence instrument [the Brief Adherence Rating Scale (BARS)] to assess the oral antipsychotic medication adherence of outpatients with schizophrenia and schizoaffective disorder. EM and BARS adherence and symptom severity ratings were gathered at baseline and prospectively at 6 monthly visits in 61 participants (n=35 with schizophrenia; n=26 with schizoaffective disorder). A significant positive relationship was found between mean BARS and EM adherence (β=0.98; r_s=0.59, p<0.0001). Cronbach's coefficient alpha revealed very high internal reliability for the BARS (α=0.92). A moderate-to-strong degree of test–retest reliability was also found for the BARS (β ranged from 0.53 to 0.92 and r_s ranged from 0.46 to 0.86). Regarding concurrent validity of the BARS, greater mean BARS adherence was significantly related to lower mean PANSS total scores (β=−0.40; r_s=−0.39, p=0.002) and to lower mean Positive symptom sub-scale scores (β=−0.08, p=.007; r_s=−0.28, p=.02). An initial 3-month monitoring period with the BARS also demonstrated good sensitivity (73%) and specificity (74%) in identifying non-adherent outpatients (defined as <70% mean EM adherence). Relative to EM, the BARS appears to provide valid, reliable, sensitive, and specific estimates of antipsychotic medication adherence of outpatients with schizophrenia and schizoaffective disorder. The BARS appears to be a promising candidate as a brief adherence assessment instrument for feasible use in community-based settings.

Keywords: Electronic monitoring, Antipsychotics, Medication adherence, Schizophrenia

Back to Article Outline

1. Introduction 

Assessing the antipsychotic medication adherence of outpatients with schizophrenia is a major challenge. Clinicians generally rely on patients' self-report of adherence (Osterberg and Blaschke, 2005). Clinical researchers have used various adherence assessment methods, including clinician rating, patient self-report, chart review, prescription renewal/pharmacy records, pill counts, and urine or blood analysis. These various methods have been extensively reviewed (Farmer, 1999, Osterberg and Blaschke, 2005, Velligan et al., 2006), and each method has its own critical shortcomings (Osterberg and Blaschke, 2005).

To date, the development of effective adherence assessment tools in schizophrenia has been hindered by a lack of objective, valid, and reliable adherence assessment methods capable of serving as an appropriate reference standard of patients' ongoing medication taking behavior in the community. The recent availability of electronic monitoring devices, however, now provides a reasonable “gold standard” by which other adherence assessment methods can be evaluated. Findings from recent clinical studies suggest that EM can effectively assess the antipsychotic adherence of outpatients with schizophrenia and schizoaffective disorder (Byerly et al., 2005a, Byerly et al., 2005b, Byerly et al., 2007, Diaz et al., 2001, Diaz et al., 2004, Nakonezny et al., 2008, Nakonezny and Byerly, 2006). However, there is a need for a simple, brief, reliable, valid, and less costly alternative to EM.

In the current study, using electronic monitoring as a “gold standard,” we evaluated the reliability and validity as well as the sensitivity and specificity of our recently developed, brief, pencil-paper, clinician-administered adherence instrument [the Brief Adherence Rating Scale (BARS)] to assess the oral antipsychotic medication adherence of outpatients with schizophrenia and schizoaffective disorder.

Back to Article Outline

2. Materials and methods 

2.1. Participants 

Sixty-one adult outpatients diagnosed with schizophrenia (n=35) or schizoaffective disorder (n=26), as established by the Structured Clinical Interview for DSM-IV, were recruited from three Dallas County public mental health outpatient clinics. The vast majority of participants were self-referred from flyers posted in the clinics (with a few participants being referred by their treating clinician). The flyers made no mention of medication adherence status or the investigational focus of the study. Participants were recruited and studied at their usual outpatient clinic sites. Participants had to be currently taking a single oral antipsychotic, and be at least 18 years of age. Participants who had received a depot antipsychotic within one treatment cycle prior to study initiation and those using a pillbox were excluded. There were no restrictions placed on the use of psychotropic or other medications other than those mentioned above for antipsychotics. The study protocol was approved by the Institutional Review Board of The University of Texas Southwestern Medical Center at Dallas, and written informed consent was obtained from all participants. Participants were paid $15 per hour of study participation.

2.2. Procedures and measures 

Participants took either a first-generation (n=25) or second-generation (n=36) oral antipsychotic during study participation. The particular antipsychotic medication the patient received was part of routine care at study entry. No participants switched antipsychotic medication type during the course of the trial. Twenty-five of the 61 outpatients took a first-generation, single oral antipsychotic (fluphenazine, N=1; haloperidol, N=7; loxapine, N=2; molindone, N=1; perphenazine, N=9; thiothixene, N=3; trifluoperazine, N=2) during study participation. Thirty-six of the 61 outpatients took a second-generation, single oral antipsychotic (aripiprazole, N=1; clozapine N=3; olanzapine, N=13, quetiapine, N=13, risperidone, N=4, ziprasidone, N=2) during study participation.

The reference medication adherence (or “gold standard”) was assessed with the Medication Event Monitoring System (MEMS^®), which is a medication vial cap that electronically recorded the date and time of bottle opening. The MEMS^® caps used in this study had no cueing mechanisms. The company that makes the MEMS® caps provided no support for the study.

The study period (March 2003–April 2004) included a screening, baseline, and up to 6 consecutive monthly EM adherence evaluations for each participant. The content of study visits was limited to adherence and clinical assessments. Participants were made aware of the purpose and function of the MEMS^® cap prior to study initiation, but they did not have access to EM-generated adherence results.

For a given patient, EM adherence was operationally defined as the proportion of oral antipsychotic medication vial caps openings in a given month relative to the prescribed doses for that month. Excessive bottle openings, however, did not count toward overall adherence. To improve the precision of EM in measuring adherence, we used a questionnaire at each visit to identify extra openings (e.g., opening of medication bottle by study staff) or the lack of bottle openings for valid reasons (e.g., pills taken away from the patient for a weekend). These openings (or lack thereof) were accounted for in the final adherence calculation.

Approximately 80% of the planned EM data was collected. Uncollected EM data resulted mostly from early discontinuation of participants, although data from 6 individual caps were unavailable because of 5 lost caps and 1 damaged cap.

The Brief Adherence Rating Scale (BARS) is a brief, pencil-paper, clinician-administered adherence assessment instrument recently developed by our group. It consists of 4 items: 3 questions and an overall visual analog rating scale to assess the proportion of doses taken by the patient in the past month (0%–100%). The visual analog scale rating serves as the final adherence determination. The three questions, adapted with permission from a lengthier adherence questionnaire used in the CATIE trial (McEvoy, 2003), inquired about patients' knowledge of their own medication regimen and episodes of missed medication taking. The three questions included: number of prescribed doses per day (question 1); number of days, over the past month, the patient did not take the prescribed doses (question 2); and number of days, over the past month, the patient took less than the prescribed doses (question 3).

The study period included a screening, baseline, and up to 6 consecutive monthly BARS adherence evaluations for each participant. The BARS adherence evaluations were assessed during the same time period as the EM adherence evaluations (so as to mitigate the threat of a time period effect). The BARS was administered by research assistants who were blinded to EM adherence results. However, BARS raters were aware of patient self-ratings of adherence, which consisted only of a visual analog scale (0%–100%) that was similar to item 4 of the BARS.

Symptom severity, as measured by the 30 item Positive and Negative Syndrome Scale (PANSS) total score (Kay et al., 1987), was assessed at baseline and then monthly over the 6-month study period. Higher PANSS scores reflect greater symptom severity.

The two research assistants who provided SCID and PANSS ratings possessed psychology undergraduate degrees and they had clinical experience in the evaluation of mental disorders. Rater training on the SCID included review and discussion of the SCID User's Guide and Administration Booklet, and completion and discussion of 10 case vignettes to make SCID diagnoses.

Training on the PANSS included the viewing and rating of three videotaped interviews of patients. These training sessions were followed by discussions with an experienced PANSS rater who compared the research assistant ratings to a “gold standard” (that was established by a consensus rating made by psychiatrists who were also experts in schizophrenia research).

Back to Article Outline

3. Data analysis 

3.1. Comparison of EM and BARS 

BARS adherence was compared to EM adherence (across the 6 monthly assessment periods) using a mixed linear model analysis of repeated measures. Restricted maximum likelihood estimation and Type 3 tests of fixed effects were used, with the Kenward–Roger correction (Kenward and Roger, 1997) applied to the Kronecker product covariance structure (unstructured covariance model@compound symmetry). The main effect of Instrument (BARS vs. EM) and the Instrument×Period interaction effect were examined. Simple instrument effects in each Period (month) were also assessed. The level of significance was set at p≤.05.

3.2. Crossinstrument agreement 

Simple linear robust regression (with MM estimation) and the Spearman rank-order correlation coefficient (r_s) were used to assess the linear relationship between BARS and EM adherence across all periods and at each separate period. In the regression analysis, EM adherence was the dependent variable and BARS adherence was the independent variable. The main purpose of using robust regression and Spearman's r_s was to provide resistant (stable) results in the presence of any potential outliers detected in the sample adherence data (with any slight departure from normality). The level of significance was set at p≤.05.

3.3. Reliability 

Internal consistency of BARS adherence over the six monthly monitoring periods was assessed using Cronbach's coefficient alpha (the mean inter-correlation of all possible split halves of the 6 adherence periods). Reliability of BARS was also assessed using a test–retest method in a simple linear regression and correlation analysis. Simple linear robust regression (with MM estimation) and the Spearman rank-order correlation coefficient (r_s) assessed the relationship between initial BARS adherence from various initial assessment periods (e.g., BARS adherence from month 1 alone, month 2 alone, mean of months 1 and 2, etc.) and subsequent BARS adherence. Robust regression and Spearman's r_s were used here for the same reasons mentioned earlier. Each initial BARS adherence period (initial assessment periods separately and average combinations of initial assessment periods) was an independent variable. Subsequent BARS adherence was the dependent variable and was based on BARS assessments for month 4 only, month 5 only, month 6 only as well as the average of the 3 monthly BARS assessments from months 4 thru 6. A separate linear regression analysis was conducted for each initial BARS adherence period on subsequent BARS adherence. The level of significance was set at p≤.05.

3.4. Concurrent validity 

Concurrent validity of BARS was assessed by the relationship between BARS adherence and PANSS total score using simple linear robust regression (with MM estimation). We also used the Spearman r_s to assess the linear association between BARS adherence and PANSS total score. Further, we used a simple linear robust regression and the Spearman r_s to assess the linear relationship between the 6-month average BARS adherence and the 6-month average Positive symptom sub-scale score and Negative symptom sub-scale score, respectively. The PANSS is a widely used criterion of symptom severity in clinical trials of schizophrenia. The criterion outcome variable, PANSS total score, was based on a PANSS assessment at each separate month over the 6-month study period as well as an average of the 6 monthly PANSS total score ratings. Also, for the current study, the Positive symptom sub-scale scores and Negative symptom sub-scale scores were based on an average of the 6 monthly assessments. BARS adherence was based on BARS assessments at each separate month over the 6-month study period as well as the average of all 6 monthly BARS assessments. The level of significance was set at p≤.05.

3.5. Sensitivity and specificity 

To further determine whether BARS accurately reflected participants' medication adherence, sensitivity and specificity were assessed between various BARS adherence cutoffs and the 6-month mean EM adherence of <70%. The <70% EM adherence cutoff was selected because it was both conservative in detecting non-adherence and consistent with definitions from prior published research (Lacro et al., 2002). Further, as additional (empirical) support for the <70% EM adherence cutoff, previous research suggests that schizophrenia patients who are <70% adherent to their antipsychotic medication are at greater risk of hospitalization than those who are ≥70% adherent (Weiden et al., 2004). The following BARS adherence cutoffs (which represented the mean adherence of an initial 3-month BARS assessment period and then separately the mean adherence of all 6 monthly BARS assessments) were used: <50%, <51%, <52%, …, <90%. This range of BARS adherence cutoffs represented a clinically meaningful (and encompassing) range to evaluate in relation to EM adherence. We treated EM adherence as the “gold standard.” Findings from recent clinical studies (including our own) (Byerly et al., 2007, Byerly et al., 2005a) suggest that EM can effectively identify patients who are non-adherent to their oral antipsychotic medication, and that such non-adherence is associated with poor clinical outcomes.

Back to Article Outline

4. Results 

4.1. Participant characteristics 

The study sample included 30 males (49.2%) and 31 females (50.8%), with an average age of 44.3 years, SD=9.1 (range 21–59 years). The average age at illness onset was 23.1 years (SD=10.1) and the average duration of illness was 20.9 years (SD=10.7). Participants included 40 (65.6%) African Americans, 19 (31.2%) Caucasians, and 2 (3.2%) Hispanics. Of these 61 participants, 24 (39.3%) had less than a high school education and 37 (60.7%) had at least a high school education. Also, of the 61 participants, 43 (70.5%) did not have the presence of a caregiver. The average baseline total PANSS score was 74.9 (SD=12.3). The 6-month average Positive symptom sub-scale score and Negative symptom sub-scale score was 19.6 (SD=3.3) and 15.6 (SD=3.8), respectively.

4.2. Comparison of EM and BARS 

The mixed model repeated measures analysis revealed no significant Instrument effect (EM vs. BARS; F=0.25, df=1, 92.1, p=.61) and no significant Instrument×Period interaction (F=1.61, df=5, 317, p=.16). Further, the test of simple Instrument effects (in each period) showed no significant instrument differences on adherence in any of the six prospective months (F's<1.16, p's>.28). Least-squares means and standard deviations for EM and BARS adherence at the 6 test periods are shown in Table 1 and Fig. 1.

Table 1. Instrument effect and Instrument×Time Period simple effects for mean level of EM and BARS adherence

	Adherence (%)
Month and instrument	M	SD	90% CI for M	n	F	p
Month 1 and instrument				61	0.85	0.36
EM	76.13	27.67	70.26 to 81.98
BARS	72.80	18.06	68.97 to 76.63
Month 2 and instrument				58	0.63	0.43
EMa	68.93	27.37	63.08 to 74.77
BARSb	71.78	17.77	67.92 to 75.64
Month 3 and instrument				54	0.17	0.67
EMa	66.14	27.36	60.30 to 71.99
BARSc	67.66	17.48	63.76 to 71.56
Month 4 and instrument				57	1.35	0.24
EMb	63.43	27.09	57.54 to 69.31
BARSb	67.65	17.79	63.78 to 71.51
Month 5 and instrument				51	0.13	0.71
EMd	64.56	26.21	58.55 to 70.57
BARSe	65.89	17.42	61.97 to 69.82
Month 6 and instrument				55	0.54	0.46
EMc	61.64	26.67	55.69 to 67.59
BARSc	64.32	17.52	60.41 to 68.23
Overall (months 1–6)				61	0.25	0.61
EM	66.81	23.08	61.76 to 71.85
BARS	68.35	15.05	65.04 to 71.67

Note. EM = electronic monitoring; BARS = Brief Adherence Rating Scale; Overall = 6-month average and represents the main effect of instrument. M = least-squares means; SD = standard deviation; 90% CI = 90% confidence interval for the expected mean value; n= sample size. F statistic was used to test for mean adherence differences between the two adherence instruments. p = p value associated with an upper one-tailed test.

• 
  • View Large Image
  • Download to PowerPoint

• Fig. 1. 

  Instrument×Period simple effects for mean level of EM and BARS adherence. The means reported in this figure represent the same least-squares means reported in Table 1. The mixed model repeated measures analysis revealed no significant Instrument effect across the 6 test periods (p=.61).

4.3. Crossinstrument agreement 

Simple linear robust regression and the Spearman rank-order correlation (r_s) revealed a significant positive relationship between BARS adherence and EM adherence across the average of the 6 monthly assessments (β=.98, r_s=.59, p<.0001). These regression and correlation results by individual month are reported in Table 2.

Table 2. The relationship between BARS and EM adherence by month

BARS and EM adherence by month	Robust regression analysis				Spearman correlation analysis
	β	χ2	p	n	rs	t	p	n
Month 1
Relation between BARS and EM adherence	0.71	45.60	<.0001	61	0.56	5.19	<.0001	61
Month 2
Relation between BARS and EM adherence	0.90	16.74	<.0001	58	0.42	3.46	0.0012	58
Month 3
Relation between BARS and EM adherence	0.97	25.51	<.0001	54	0.57	5.01	<.0001	54
Month 4
Relation between BARS and EM adherence	1.13	16.52	<.0001	57	0.49	4.16	0.0001	57
Month 5
Relation between BARS and EM adherence	0.50	6.13	0.0133	51	0.45	3.52	0.0008	51
Month 6
Relation between BARS and EM adherence	0.71	7.56	0.0060	55	0.46	3.77	0.0004	55
Overall (months 1–6)
Relation between BARS and EM adherence	0.98	33.16	<.0001	61	0.59	5.61	<.0001	61

Note. EM = electronic monitoring; BARS = Brief Adherence Rating Scale; β = unstandardized robust regression coefficient; χ² = observed test statistic for β; r_s = Spearman rank-order correlation coefficient; t = observed test statistic for r_s; p = two-tailed p value for the t statistic and one-tailed for the χ² statistic; n= sample size.

4.4. Internal and test–retest reliability 

Cronbach's coefficient alpha revealed very high internal reliability for the BARS across the 6 monthly assessment periods (α=.92). Simple linear robust regression and Spearman's r_s, which were used to estimate test–retest reliability, revealed a moderate-to-strong linear relationship between initial monthly BARS adherence and subsequent BARS adherence. For the various initial BARS assessment periods in relation to subsequent BARS adherence, robust regression coefficients ranged from 0.53 to 0.92 and Spearman's correlations (r_s) ranged from 0.46 to 0.86 (see Table 3).

Table 3. Estimates of test–retest reliability using simple linear robust regression and Spearman's r_s

	Regression and correlation analysis
Initial BARS adherence by monthb	BARS adherence for month 4a					BARS adherence for month 5a					BARS adherence for month 6a					BARS adherence for mean of months 4–6a
	rs	β	R2	χ2	p	rs	β	R2	χ2	p	rs	β	R2	χ2	p	rs	β	R2	χ2	p
BARS adherence for month 1	0.718	0.811	0.498	132.25	<0.0001	0.565	0.711	0.353	43.83	<0.0001	0.487	0.533	0.237	24.12	<0.0001	0.620	0.677	0.397	62.54	<0.0001
BARS adherence for month 2	0.778	0.876	0.482	91.14	<0.0001	0.636	0.838	0.375	42.78	<0.0001	0.460	0.561	0.187	14.80	<0.0001	0.684	0.706	0.384	39.59	<0.0001
BARS adherence for month 3	0.850	0.855	0.601	279.15	<0.0001	0.589	0.655	0.336	33.91	<0.0001	0.608	0.607	0.348	39.60	<0.0001	0.747	0.679	0.496	76.45	<0.0001
BARS adherence for mean of months 1 and 2	0.794	0.883	0.543	157.19	<0.0001	0.641	0.805	0.404	56.25	<0.0001	0.494	0.571	0.243	22.86	<0.0001	0.695	0.737	0.446	67.61	<0.0001
BARS adherence for mean of months 2 and 3	0.864	0.908	0.609	285.70	<0.0001	0.658	0.799	0.390	47.86	<0.0001	0.587	0.634	0.296	30.50	<0.0001	0.759	0.752	0.491	85.04	<0.0001
BARS adherence for mean of months 1, 2, and 3	0.855	0.926	0.610	279.59	<0.0001	0.627	0.794	0.393	50.02	<0.0001	0.547	0.619	0.288	29.92	<0.0001	0.737	0.773	0.494	90.39	<0.0001

Note. BARS = Brief Adherence Rating Scale; r_s = Spearman rank-order correlation coefficient; β = unstandardized robust regression coefficient (based on MM estimation); R² is from the robust regression model; χ² = observed test statistic for β; p = p value associated with r_s and β, respectively.

4.5. Concurrent validity and correlation between adherence and PANSS 

Both simple linear robust regression and the Spearman r_s revealed a significant inverse relationship between BARS adherence and Positive symptom sub-scale score (i.e., lower adherence was associated with more severe positive symptoms) (β=−0.08, p=.007; r_s=−0.28, p=.02). A significant inverse relationship was also found between BARS adherence and Negative symptom sub-scale score with the robust regression analysis (β=−0.09, p=.02), but this relationship did not quite reach statistical significance with the Spearman r_s (r_s=−0.23, p=.07). Further, simple linear robust regression and the Spearman r_s revealed a significant inverse relationship between BARS adherence and PANSS total score (β=−0.40, p<.0001; r_s=−0.39, p=.002). These regression and correlation results by individual month, along with raw means and standard deviations of BARS adherence and PANSS total scores, are reported in Table 4.

Table 4. The relationship between BARS adherence and PANSS total score by month

BARS adherence and PANSS total by month	Robust regression analysis				Spearman correlation analysis				Mean values for adherence and PANSS by month
									BARS adherence (%)			PANSS total score
	β	χ2	p	n	rs	t	p	n	M	SD	n	M	SD	n
Month 1									72.80	17.57	61	74.01	13.24	61
Relation between BARS adherence and PANSS total	−0.23	4.65	0.03	61	−0.27	−2.15	0.03	61
Month 2									72.58	15.85	58	74.13	13.54	58
Relation between BARS adherence and PANSS total	−0.32	7.83	0.005	58	−0.36	−2.89	0.005	58
Month 3									68.89	18.27	55	75.38	12.63	55
Relation between BARS adherence and PANSS total	−0.33	12.02	<.0001	55	−0.51	−4.34	<.0001	55
Month 4									68.15	17.72	58	75.81	12.80	58
Relation between BARS adherence and PANSS total	−0.44	18.97	<.0001	58	−0.54	−4.82	<.0001	58
Month 5									66.55	19.25	54	75.39	11.02	54
Relation between BARS adherence and PANSS total	−0.35	17.68	<.0001	54	−0.29	−2.20	0.03	54
Month 6									64.85	18.86	55	76.49	11.75	55
Relation between BARS adherence and PANSS total	−0.45	23.79	<.0001	55	−0.34	−2.65	0.01	55
Overall (months 1–6)									68.57	15.34	61	75.17	11.77	61
Relation between BARS adherence and PANSS total	−0.40	16.94	<.0001	61	−0.39	−3.25	0.002	61

Note. BARS = Brief Adherence Rating Scale; PANSS Total = Positive and Negative Syndrome Scale total score; β = unstandardized robust regression coefficient; χ² = observed test statistic for β; r_s = Spearman rank-order correlation coefficient; t = observed test statistic for r_s; p = two-tailed p value for the t statistic and one-tailed for the χ² statistic; n = sample size.

The means presented in this table are the arithmetic (raw) means. M = Mean; SD = standard deviation.

4.6. Sensitivity and specificity 

BARS adherence (with a cutoff of <71% mean adherence of the 6 monthly BARS assessments) maximized the combination of both sensitivity (73.1%) and specificity (71.4%), with an area under the curve of .72, in identifying EM adherence of <70% (defined as <70% mean adherence of the 6 monthly EM adherence assessments). Additionally, ROC results suggest that an initial, 3-month average, adherence assessment period with BARS can identify patients' oral antipsychotic medication non-adherence vis-à-vis EM adherence. An initial 3-month BARS assessment period with a cutoff of <74% mean adherence (mean of months 0, 1, and 2) maximized the combination of both sensitivity (73.1%) and specificity (74.3%), with an area under the curve of .73, in detecting non-adherence (defined as <70% mean adherence of the 6 monthly EM adherence assessments).

An analysis of the frequency distribution of the 6-month mean EM adherence cutoff of 70% (used in the current study) found that 57.4% (n=35) of the participants were deemed “adherent” (with ≥70% EM adherence) and 42.6% of the participants (n=26) were deemed “non-adherent” (with <70% EM adherence). Further, using the empirical cutoff of <71% mean BARS adherence of the 6 monthly BARS assessments, we found that 52.5% (n=32) of the participants were considered “adherent,” while 47.5% (n=29) were considered “non-adherent.” Similarly, based on the empirical cutoff of <74% mean BARS adherence of the initial 3-month BARS assessment period, we found that 54.1% (n=33) of the participants were considered “adherent,” whereas 45.9% (n=28) were considered “non-adherent.”

Back to Article Outline

5. Discussion 

In the current study, using electronic monitoring as a reference standard, we examined the reliability and validity as well as the sensitivity and specificity of the Brief Adherence Rating Scale (BARS) to assess the oral antipsychotic medication adherence of outpatients with schizophrenia and schizoaffective disorder. Estimates of oral antipsychotic medication adherence generated using the BARS were similar to those produced by EM across the 6-month study period, with the BARS slightly overestimating adherence as referenced to EM (albeit the difference was not statistically significant). The slight (non-significant) tendency of the BARS to overestimate adherence might have resulted from exposure of the BARS raters to patient self-ratings of adherence. In a prior study by our group, which was based on the same sample data as that used in the current study, patient self-estimates of adherence were 34% greater than that of EM (91% vs. 67%, respectively) (Byerly et al., 2007).

Cronbach's coefficient alpha revealed very high internal reliability for the BARS across the 6 monthly assessment periods (α=.92). The BARS also demonstrated good test–retest reliability. The BARS (with a cutoff of <71% for a 6-month monitoring period or <74% mean adherence for an initial 3-month monitoring period) further demonstrated good sensitivity (73%) and specificity (71%–74%) to detect non-adherent EM outpatients (defined as <70% mean adherence of the 6 monthly EM adherence assessments).

Concurrent validity of the BARS was established by the relationship between BARS adherence and the PANSS total score. We found that greater BARS adherence was significantly related to lower symptom severity in outpatients with schizophrenia or schizoaffective disorder. This finding is consistent with the few existing studies that found a significant relationship between worse antipsychotic medication adherence (based on chart review) and greater symptom severity in schizophrenia patients (Hudson et al., 2004, Kampman et al., 2002).

The present findings of the BARS may be contrasted with those previously reported using other pencil-paper instruments (scales) to assess medication adherence in persons with schizophrenia. A total of four such previous scales were tested against an objective reference standard (Dolder et al., 2004, Morisky et al., 1986, Thompson et al., 2000), with three of the four instruments demonstrating a statistically significant correlation with the reference adherence measure (Dolder et al., 2004, Thompson et al., 2000). These three scales, which demonstrated a statistically significant agreement with an objective reference standard, include: the Drug Attitude Inventory (DAI) 30-item (Thompson et al., 2000), the Medication Adherence Rating Scale (MARS) (Thompson et al., 2000), and the Brief Evaluation of Medication Influences (BEMIB) (Dolder et al., 2004). Unlike the BARS, the DAI-30 and MARS have not been validated in studies which exclusively used psychotic disorder patients (about 45% of the patients were diagnosed with mood and other psychiatric disorders) (Thompson et al., 2000). In addition, the DAI-30 and MARS were validated with blood levels of concurrent mood stabilizer medications (as the reference standard) instead of antipsychotic medication blood levels, and information about the sensitivity and specificity of the DAI-30 and MARS, respectively, in identifying antipsychotic non-adherence is not reported (Thompson et al., 2000). Potential drawbacks of the previous validation of the BEMIB for use in schizophrenia (Dolder et al., 2004) include: (a) the absence of validation in a general psychotic disorder population (the BEMIB has been validated in older, mostly Caucasian patients); (b) the use of prescription fill records as the objective reference standard (which is considered inferior to electronic monitoring, Farmer, 1999, Osterberg and Blaschke, 2005); and (c) a modest agreement with the objective reference standard (r=0.47) (Dolder et al., 2004).

In contrast to these other scales (DAI-30, MARS, BEMIB), the findings of the current study suggest that the BARS is perhaps the most desirable candidate for use in community treatment settings of persons with schizophrenia and schizoaffective disorder. The current study established the reliability, concurrent validity, sensitivity, and specificity of the BARS in outpatients with schizophrenia and schizoaffective disorder. The BARS (via the current study) is the only instrument that has been validated against electronically-monitored adherence as the reference standard. The BARS is simple, brief, and efficient in its administration. The BARS requires less than 60 min of staff training and the BARS takes less than 5 min to administer. Finally, the BARS rating is reported as a percent of adherence (0%–100%). In a recent review of adherence assessment in schizophrenia, Velligan et al. suggested that adherence should be reported as a percentage to allow comparison of adherence studies based on a common metric (Velligan et al., 2006).

The validity and reliability of the BARS is only as good as its reference method (electronic monitoring via MEMS^© caps in the current study). Previous research (including our own) suggests that electronic monitoring can effectively assess, with good reliability and concurrent validity, the antipsychotic medication adherence of outpatients with schizophrenia and schizoaffective disorder (Nakonezny et al., 2008) and the medication adherence of outpatients in non-psychiatric patient populations (e.g., HIV antiretroviral therapy patients, hypertensive patients) (Burnier et al., 2001, Liu et al., 2001). In fact, a recent review suggests that EM “provides the most accurate and valuable data on adherence in difficult clinical situations and in the setting of clinical trials and adherence research” (Osterberg and Blaschke, 2005).

The current study may be tempered by a few limitations. The study had a relatively brief study period and modest sample size. In addition, using a convenience sample might have limited the generalizability of the study results. In particular, the volunteers used in the current study may be more adherent than the general population of outpatients with schizophrenia or schizoaffective disorder and, thus, findings of the current study may have overestimated the adherence of what might be expected in the general population of outpatients with psychotic disorders. Further, because the BARS was performed by research assistants (albeit blind to electronic monitoring adherence results), it is possible that the BARS might perform differently when administered by the treating clinicians of patients in usual care settings. Finally, EM may overestimate adherence because the events captured by EM (date/time of bottle opening) does not ensure medication ingestion (Farmer, 1999, Osterberg and Blaschke, 2005). A period documenting a lack of medication bottle opening, however, most likely represents non-adherence (Osterberg and Blaschke, 2005).

To our knowledge, the current study is the first to establish the reliability, concurrent validity, sensitivity, and specificity of a brief, unobtrusive, pencil-paper, clinician-administered medication adherence instrument (the BARS) in assessing the oral antipsychotic medication adherence of outpatients with schizophrenia or schizoaffective disorder. Further, and of particular note, this is the first study to our knowledge to validate such an instrument against electronically-monitored adherence as the reference standard. The favorable psychometric properties of the BARS, including its ability to detect non-adherence to antipsychotic medication with good sensitivity and specificity, suggest that it is a promising candidate for future use as an adherence assessment instrument in community-based settings and populations. Additional studies, particularly evaluations of the BARS in other non-psychotic disorder patient populations and evaluations of the performance of the BARS when administered by patient's own clinicians, are warranted.

Back to Article Outline

Role of funding source 

Funding for this study was provided by NIMH K-Award Grant # 5 K23 MH064930, the Stanley Medical Research Institute, the Betty Jo Hay Distinguished Chair in Mental Health, and by a grant from Janssen Medical Affairs, LLC. All of the study sponsors had no further role in the study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the paper for publication.

Back to Article Outline

Contributors 

Dr. Byerly designed the study, wrote the protocol, and participated in manuscript drafting. Dr. Nakonezny developed the analysis plan, conducted the analyses, and participated in manuscript drafting. Dr. Rush assisted in the design of the study and protocol development and participated in manuscript drafting. All authors contributed to and have approved the final manuscript.

Back to Article Outline

Conflict of interest 

Dr. Byerly has received consultancies, speaker’s bureau honoraria, and grant funding from Janssen. All other authors declare that they have “no conflicts of interest.”

Back to Article Outline

Acknowledgments 

None.

Back to Article Outline

References 

1. Burnier M, Schneider MP, Chiolero A, Stubi CL, Brunner HR. Electronic compliance monitoring in resistant hypertension: the basis for rational therapeutic decisions. J. Hypertens. 2001;19:335–341
   • View In Article
   • MEDLINE
   • CrossRef
2. Byerly M, Fisher R, Whatley K, Holland R, Varghese F, Carmody T, et al. A comparison of electronic monitoring vs. clinician rating of antipsychotic adherence in outpatients with schizophrenia. Psychiatry Res. 2005;133:129–133
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (89 KB)
   • CrossRef
3. Byerly MJ, Fisher R, Carmody T, Rush AJ. A trial of compliance therapy in outpatients with schizophrenia or schizoaffective disorder. J. Clin. Psychiatry. 2005;66:997–1001
   • View In Article
   • MEDLINE
   • CrossRef
4. Byerly MJ, Thompson A, Carmody T, Bugno R, Erwin T, Kashner M, et al. Validity of electronically monitored medication adherence and conventional adherence measures in schizophrenia. Psychiatr. Serv. 2007;58:844–847
   • View In Article
   • MEDLINE
   • CrossRef
5. Diaz E, Levine HB, Sullivan MC, Sernyak MJ, Hawkins KA, Cramer JA, et al. Use of the Medication Event Monitoring System to estimate medication compliance in patients with schizophrenia. J. Psychiatry Neurosci. 2001;26:325–329
   • View In Article
   • MEDLINE
6. Diaz E, Neuse E, Sullivan MC, Pearsall HR, Woods SW. Adherence to conventional and atypical antipsychotics after hospital discharge. J. Clin. Psychiatry. 2004;65:354–360
   • View In Article
   • MEDLINE
   • CrossRef
7. Dolder CR, Lacro JP, Warren KA, Golshan S, Perkins DO, Jeste DV. Brief evaluation of medication influences and beliefs: development and testing of a brief scale for medication adherence. J. Clin. Psychopharmacol. 2004;24:404–409
   • View In Article
   • MEDLINE
   • CrossRef
8. Farmer KC. Methods for measuring and monitoring medication regimen adherence in clinical trials and clinical practice. Clin. Ther. 1999;21:1074–1090(discussion 1073)
   • View In Article
   • Abstract
   • Full-Text PDF (1281 KB)
   • CrossRef
9. Hudson TJ, Owen RR, Thrush CR, Han X, Pyne JM, Thapa P, et al. A pilot study of barriers to medication adherence in schizophrenia. J. Clin. Psychiatry. 2004;65:211–216
   • View In Article
   • MEDLINE
   • CrossRef
10. Kampman O, Laippala P, Vaananen J, Koivisto E, Kiviniemi P, Kilkku N, et al. Indicators of medication compliance in first-episode psychosis. Psychiatry Res. 2002;110:39–48
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (87 KB)
    • CrossRef
11. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr. Bull. 1987;13:261–276
    • View In Article
    • MEDLINE
12. Kenward MG, Roger JH. Small sample inference for fixed effects from restricted maximum likelihood. Biometrics. 1997;53:983–997
    • View In Article
    • MEDLINE
    • CrossRef
13. Lacro JP, Dunn LB, Dolder CR, Leckband SG, Jeste DV. Prevalence of and risk factors for medication nonadherence in patients with schizophrenia: a comprehensive review of recent literature. J. Clin. Psychiatry. 2002;63:892–909
    • View In Article
    • MEDLINE
    • CrossRef
14. Liu H, Golin CE, Miller LG, Hays RD, Beck CK, Sanandaji S, et al. A comparison study of multiple measures of adherence to HIV protease inhibitors. Ann. Intern. Med. 2001;134:968–977
    • View In Article
    • MEDLINE
15. McEvoy JP. CATIE Adherence Questionnaire. Adapted with Permission. 2003;
    • View In Article
16. Morisky DE, Green LW, Levine DM. Concurrent and predictive validity of a self-reported measure of medication adherence. Med. Care. 1986;24:67–74
    • View In Article
    • MEDLINE
    • CrossRef
17. Nakonezny PA, Byerly MJ. Electronically monitored adherence in outpatients with schizophrenia or schizoaffective disorder: a comparison of first- vs. second-generation antipsychotics. Schizophr. Res. 2006;82:107–114
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (142 KB)
    • CrossRef
18. Nakonezny P, Byerly M, Rush AJ. Electronic monitoring of antipsychotic medication adherence in outpatients with schizophrenia or schizoaffective disorder: An empirical evaluation of its reliability and predictive validity. Psychiatry Res. 2008;157:259–263
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (127 KB)
    • CrossRef
19. Osterberg L, Blaschke T. Adherence to medication. N. Engl. J. Med. 2005;353:487–497
    • View In Article
    • CrossRef
20. Thompson K, Kulkarni J, Sergejew AA. Reliability and validity of a new Medication Adherence Rating Scale (MARS) for the psychoses. Schizophr. Res. 2000;42:241–247
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (82 KB)
    • CrossRef
21. Velligan DI, Lam YW, Glahn DC, Barrett JA, Maples NJ, Ereshefsky L, et al. Defining and assessing adherence to oral antipsychotics: a review of the literature. Schizophr. Bull. 2006;32:724–742
    • View In Article
    • MEDLINE
    • CrossRef
22. Weiden PJ, Kozma C, Grogg A, Locklear J. Partial compliance and risk of rehospitalization among California Medicaid patients with schizophrenia. Psychiatr. Serv. 2004;55:886–891
    • View In Article
    • MEDLINE
    • CrossRef