Eye Position Information on CT Increases the Identification of Acute Ischemic Hypoattenuation

Methods

Two staff neuroradiologists and 2 neuroradiology fellows trained in the radiology department of our active stroke diagnosis and treatment center (the readers) reviewed 75 baseline CT hardcopy films, obtained within 3 hours of acute ischemia from subjects in the Interventional Management of Stroke Study (IMS I).⁷ Scans were submitted from 15 centers, obtained at 22 hospitals on 29 different CT scanners. Five additional baseline scans were deemed either of insufficient quality for study analysis or did not include the globes. The readers were blinded to the clinical data of the side of the suspected ischemia.

Films were reviewed by the 4 readers at time points 3 months apart, the first time with tape over the eyes on the images, the second time with the eyes visible. Readers were told that they were evaluating CT scans of subjects with acute stroke in IMS I and were asked to record the presence or absence of early hypoattenuation and the side if hypoattenuation was present. Readers were also asked if an HMCA was present and on which side.

Agreement statistics, κ or intraclass correlation, were calculated to determine agreement between readings. Interpretation of the κ statistics was the following: 0–0.4, poor; 0.4–0.7, good; and >0.7, excellent agreement. Specifically, the intraclass correlation was used to assess identification of hypoattenuation for each time point among the 4 readers. The κ statistic was used to assess agreement between each of the 2 readings by the same reader; to compare the readers with the original blinded hypoattenuation interpretation of the IMS neuroradiologist (T.A.T); and to compare the readers with baseline Alberta Stroke Program Early CT Score (ASPECTS) < 10 (indicating some hypoattenuation in the MCA distribution), as determined by a previous expert ASPECTS reading panel, who were unblinded to the clinical side of involvement.^8,9 A generalized estimating equations modeling approach was used to look at the overall effect of including eye information for agreement between readers and the study neuroradiologist.¹⁰

To establish the IMS neuroradiologist's interpretation of the presence of hypoattenuation as a valid comparator with the study readers, we also compared his baseline blinded interpretation with the unblinded interpretations of the ASPECTS panel. In addition, his reading was compared with 24-hour infarct side and infarct volumes (as measured digitally by using Cheshire software; G. Ramadas, personal communication, 2006), which might be considered indicators of the true-positive side and severity of infarct. The Wilcoxon rank sum test was used to compare volumes due to the non-normal distribution.

Results

Early hypoattenuation was identified on the side of subsequent CT infarct by at least 2 of the 4 study readers in 35/75 (47%), 3 of the 4 study readers (a simple majority) in 18/75 (24%), and all 4 in 9/75 (12%) for the eyes-available reading. The blinded IMS neuroradiologist identified early hypoattenuation in 41/75 (54.7%) scans. The unblinded ASPECTS reading panel identified ASPECTS < 10 (indicating some hypoattenuation in the MCA distribution) in 61/75 (81.3%).

Comparing readers to each other (inter-rater reliability), we found that individual κ agreements for the eyes-covered reading ranged from 0.1 to 0.38, with an associated intraclass correlation of 0.181. With eyes uncovered, the κ statistics improved, ranging from 0.18 to 0.50 with an intraclass correlation of 0.183.

The range of agreement between each reader's first and second interpretation and the study neuroradiologist's interpretation is shown in Table 1. The availability of eye information (second reading) was associated with significant improvement of agreement between the readers and the study neuroradiologist for early ischemic hypoattenuation (P = .02). Twenty-four-hour infarct volume in subjects with no hypoattenuation as determined by the study neuroradiologist was 21.4 mL, and for those with hypoattenuation, 61.3 mL (P = .004).

κ statistics for agreement for interpretations of the 4 study readers to the unblinded ASPECTS panel also increased from readings with eyes covered to readings with eyes uncovered (Table 2). The agreement was significant for 2 (P = .01) or 3 (P = .008) areas of involvement (ASPECTS < 9 and < 8, respectively). A similar trend with eyes uncovered for ASPECTS < 7 was observed.

The study neuroradiologist and the readings of the ASPECTS panel demonstrated increasing agreement for decreasing ASPECTS or for more areas of hypoattenuation. Table 3 includes κ statistics for agreement of the original blinded interpretation of the study neuroradiologist versus the ASPECTS of the unblinded panel. Agreement of the study neuroradiologist's blinded reading ranged from 0.25 for ASPECTS < 10 and increased for each point decrease in ASPECTS up to 0.51 for ASPECTS < 7.

All 4 study readers improved their sensitivity, specificity, positive predictive values, and accuracy of interpretations with the 3-month interval delay and the availability of eye-deviation information. When comparing readers' first and second readings for hypoattenuation, we noted increased mean values for sensitivity (46.8% first, 56.5% second), specificity (78.2%, 80.2%), positive predictive value (72.0%, 80.7%), negative predictive value (55.5%, 61.0%), and percentage agreement (61.0%, 67.5%). There was no differential effect of the presence of eye deviation for the change from first to second readings for experienced neuroradiologists compared with neuroradiology fellows.

HMCA was identified by at least 3 of 4 study readers in 31/75 (41.3%) subjects, but the availability of eye information (second reading) was not associated with significant improvement in agreement between the readers for HMCA (P = .42). The study neuroradiologist identified HMCA in 37.3% (28/75) of scans.

Discussion

Our results strongly suggest that availability of CT eye deviation information is useful in identifying hypoattenuation, even without knowledge of the patient's clinical data of suggested infarct side. Any sensitization to acute hypoattenuation may expedite the diagnosis of ischemic infarct, particularly in cases in which a patient's clinical data may be unclear, confusing, or absent. In some patients, an earlier imaging diagnosis may translate into earlier intervention.

Eye deviation on CT need not indicate cerebral hemisphere pathology. Definitive deviation is seen in approximately 10% and minimal deviation, in another 10% of all CT examinations (P. Minshew, unpublished data, 2004), even in patients without ischemia. Patients may be looking in several directions spontaneously. Involvement of the pontine gaze center by vertebrobasilar arterial occlusive disease may also cause eye deviation, but on the side opposite the brain involvement. Our population group included patients with 3 vertebrobasilar infarcts without eye deviation and without identified hypoattenuation by any reader, effectively serving as no-hemisphere-infarct internal controls.

In attempting to analyze the effect of eye information on interpretation, we found that no true gold standard of acute CT hypoattenuation exists. Comparing the readers' interpretations with 24-hour CT infarct development does not ensure that the hypoattenuation was present at baseline. Comparing the blinded readers' interpretations with the study neuroradiologist's blinded interpretation is potentially flawed, where one, the other, or both might be in error. Comparing the blinded readers' interpretations with the study neuroradiologist's blinded interpretation is perhaps more legitimate than comparing them with the unblinded ASPECTS reading panel's interpretation, in which clinical history might contribute to over-reading of hypoattenuation by the panel.

To assess the reliability of the study neuroradiologist's interpretation as a legitimate standard for comparison with the 4 study readers, we compared the study neuroradiologist's blinded readings with the unblinded scores of the 3-person ASPECTS reading panel and subsequent development of larger infarcts on 24-hour CT on the appropriate side. The legitimacy of using the study neuroradiologist's blinded interpretations is indeed supported by his level of correlation to the ASPECTS reading panel. Such comparison of the study neuroradiologist to the ASPECTS panel also allows assessment of the study neuroradiologist's potential bias caused by knowledge of the study group (ie, it evaluates any tendency to identify hypoattenuation where none may exist in a known major stroke population). In fact, the study neuroradiologist's blinded interpretations appear conservative, identifying hypoattenuation in 57.3% of subjects compared with 81.3% in the unblinded ASPECTS panel. Correlation of the presence of early hypoattenuation to larger 24-hour infarct volumes also supports the use of the study neuroradiologist's interpretation as a legitimate comparison with the readers' interpretations, insofar as his assignment of early hypoattenuation was associated with subsequent development of a significantly larger stroke volume at 24 hours on the appropriate side.

The availability of eye information improved readers' agreement with both the study neuroradiologist and the ASPECTS panel. Agreement with the ASPECTS panel was not uniform across all ASPECTS, however. Eye information did not improve agreement for hypoattenuation for ASPECTS < 10 or where at least a single area was involved. It did improve agreement where at least 2 or 3 areas were involved (ASPECTS < 8 or < 9), suggesting some inability or reluctance for these blinded readers to identify single or small areas of involvement, yet the ability and or willingness to do so when more than 1 area was involved. Lack of significance for ASPECTS < 7 suggests that observers were not aided by eye deviation when 4 or more ASPECTS zones of hypoattenuation were evident; larger hypoattenuation in effect negated the visual aid. The κ scores for ASPECTS < 7 were narrow in range (0.42–0.45), suggesting that some readers may have been aided more than others by greater hypoattenuation.

The κ agreement levels among the various reader groups were relatively low in these analyses. The nature of the scan population, in which scans were obtained with variable techniques at multiple centers on multiple scanners generally unfamiliar to the readers, may contribute greatly to levels of agreement observed. Comparing blinded and unblinded interpretations might be expected to lead to lower agreements for this subjective evaluation as well.

Identification of HMCA did not differ significantly with eyes uncovered on the second study. This may be an indication that readers were not merely improving their CT readings of findings with time, after an additional 3 months of experience. Of past studies that specifically analyzed hypoattenuation and HMCAs, prevalence values were 20%–70% and 28%, respectively.^11–13 That our readers' observations lie within the historic ranges for both signs suggests that knowledge of the stroke population does not appear to have created reader bias and increased rates of identification spuriously.

Previous data suggest that though interobserver agreement for acute hypoattenuaton is good, intraobserver agreement may be low.¹⁴ Attention to the presence of CED may be useful in sensitizing the reader to early hypoattenuation, having increased intraobserver and interobserver agreement in our analysis.

Conclusion

This exploratory analysis suggests that attention to eye deviation information on CT scans improves reader sensitivity to and identification of acute ischemic hypoattenuation. This positive improvement may provide clinical utility.