Observer Agreement in the Assessment of Endovascular Aneurysm Therapy and Aneurysm Recurrence

Abstract

BACKGROUND AND PURPOSE: Assessments of completeness of endovascular cerebral aneurysm therapy are commonly reported in the literature. We studied several aneurysm assessment scales with regard to observer variability, which directly affects validity of these scales.

MATERIALS AND METHODS: Initial aneurysm occlusion and occlusion at a follow-up angiogram at 3–6 months were assessed independently by 2 experienced observers. Assessments of each aneurysm were made using 3 different scales: 4-response (complete, dog ear, neck remnant, incomplete), 3-response (complete, near-complete, incomplete), and 2-response (complete or near-complete, incomplete). Assessments were also made of comparisons of initial treatment angiogram with follow-up angiogram using 2 different scales: 3-response (better, same, worse) and 2-point response (not worse, worse).

RESULTS: With assessments of both initial and follow-up angiograms, interobserver and intraobserver agreement was progressively worse with increasing response choices in the scales. Observer agreement on assessments of initial angiograms (κ values 0.48–0.67) was worse than that for follow-up angiograms (κ values 0.66–0.97). For the comparisons of the initial angiogram with the follow-up angiogram, there was worse observer agreement with the 3-response scale (κ values 0.64–0.71) than with the 2-response scale (κ values 0.78–0.89).

CONCLUSION: Interobserver and intraobserver variability are inherent to assessment scales of completeness of cerebral aneurysm therapy. Observer variability is substantially better in scales that offer fewer observer responses. However, scales with fewer observer responses may not identify aneurysm subgroups that have differing risks of recurrence and/or rehemorrhage.