CRS-R Properties

The psychometric properties of the CRS-R were investigated in a study published by Giacino, Kalmar and Whyte in 2004. The results of reliability, validity and diagnostic utility studies are summarized below.

Reliability
CRS-R Total Scores
Interrater reliability for the CRS-R total score was high (r squared=.84, p<.001) indicating that the scale yields reproducible findings across examiners. Test-retest reliability for the total score was also high (r squared=.94, p<.001) demonstrating adequate stability in patient performance over a brief assessment interval (i.e., 36 hours). Cross correlations representing the relationship between scores obtained by different raters on different days were in the moderate range (r squared=.79, p<.001). Wilcoxon analysis indicated that there was no systematic difference in the scores obtained by different raters on different days (p=.80) or by different raters on the same day (p=.10). Scores obtained by the same rater on different days, however, differed significantly with scores on day two tending to be higher than scores on day one.

CRS-R Subscale Scores
Kappa analyses were run on each subscale to determine the degree to which rater pairs agreed that patients’ responses were indicative of VS or MCS. For the interrater reliability analysis, kappa values were strong for the Auditory, Motor, Oromotor/Verbal and Communication subscales. Additionally, interrater agreement was 90% or higher for these subscales. Raters’ scores on the Visual subscale were moderately correlated and the 95% confidence interval around the kappa value was broad. Wilcoxon’s test indicated that there was a systematic difference between raters’ scores on the Visual subscale (p<.03). As the CRS-R is a standardized measure, we could not identify any rater-specific administration errors that could account for the difference in scores. Furthermore, analysis of the CRS-R record forms indicated that the disagreement in scores could not be attributed to differences in patients’ positioning or level of arousal at the time of examination. Among the 8 patients who were assigned different scores by each rater, all were in the same position during both examinations (i.e., lying in bed or sitting in a wheelchair). Additionally, patients’ arousal level (i.e., eyes open or eyes closed) immediately prior to the examination was noted to be different in only two cases. Because these scores were obtained on the same day, examiner error may have accounted for the broad confidence intervals associated with raters’ scores on the Visual subscale although patient fluctuation cannot be ruled out as a contributing factor. The 95% confidence interval was also broad for the Oromotor/Verbal subscale although there was no evidence of a systematic difference in scores between the two raters.

Test-retest reliability for the subscale scores was moderate to high with the exception of the Oromotor/Verbal subscale which fell in the low range. Confidence intervals were broad for the Auditory and Oromotor/Verbal subscales, however, percent agreement in these scores across the two examinations was 85% and 70%, respectively. Wilcoxon’s test showed a trend toward a systematic difference in ratings on the Oromotor/Verbal subscale (p<.06) with higher scores noted on day two. Systematic differences in scoring were not found on any of the other subscales in the test-retest analysis.

Diagnostic Agreement
Diagnoses established by two different raters were significantly related (k=.60, p=.03) indicating that examiners can distinguish VS, MCS and emergence from MCS using the CRS-R. Raters agreed on diagnosis in 16 of the 20 patients examined. Of the four cases in which raters disagreed on diagnosis, two involved a discrepancy between MCS and emergence from MCS and in two others, there was disagreement between VS and MCS. Diagnostic agreement in rater’s ratings over two examinations was strong (k=.82, p=.004) and moderately higher than the interrater agreement rate. In one case, the diagnosis changed from MCS on day one to VS on day two. A second patient was diagnosed with MCS on day one and as emergence from MCS on day two. Given that the examiner remained constant in this analysis, these data suggest that patient fluctuation contributes to the variability in CRS-R scores obtained over time.

Internal Consistency
The relationship between the CRS-R total score and the individual subscale scores was investigated using Cronbach’s alpha. This analysis resulted in an alpha value of .83 indicating that the CRS-R is a reasonably homogeneous measure of neurobehavioral function. Intercorrelations among the subscales consistently fellin the moderate range with the exception of the Visual and Oromotor/Verbal subscales which were poorly correlated. The Auditory and Visual subscales showed the strongest interrelationship.

Validity
Measures of Dispersion
The distribution of total scores on the CRS-R, CRS and DRS was examined to determine whether performance on each scale was evenly distributed across the range of possible scores. Among the three scales, CRS-R scores were most evenly distributed. The majority of total DRS scores were located at the severe end of the score range. The DRS also showed a long tail on the low (i.e. less severe) end of the scale. Of the three scales, the CRS-R was least skewed and the DRS most skewed.

Concurrent Validity
Total scores on the CRS-R were correlated with total scores on the CRS and DRS to establish concurrent validity. Spearman coefficients were significant between the CRS-R and the CRS (r squared=.97, p<.00001) and between the CRS-R and DRS (r squared= -90, p<.00001). The stronger association between the two versions of the CRS is expected given that some of the original CRS items were retained on the revised version.

Diagnostic Utility
To help discern the diagnostic utility of the CRS-R, each patient was assigned a diagnosis of VS or MCS following completion of the CRS-R and DRS. In 51 of the 80 patients assessed, both scales produced a diagnosis of MCS. An additional 19 patients received a diagnosis of VS on both measures. The overall rate of agreement in diagnosis was 87%. There were no cases in which the DRS found evidence of MCS while the CRS-R did not. Conversely, there were 10 cases in which the CRS-R profile supported a diagnosis of MCS while the DRS findings were indicative of VS. In all 10 of these cases, the CRS-R detected evidence of visual pursuit, a diagnostic feature of MCS that is not represented on the DRS.