Reliability of the Modified TICI Score among Endovascular Neurosurgeons

BACKGROUND AND PURPOSE: The modified TICI score is the benchmark for quantifying reperfusion after mechanical thrombectomy. There has been limited investigation into the reliability of this score. We aim to identify intra-rater and inter-rater reliability of the mTICI score among endovascular neurosurgeons.

MATERIALS AND METHODS: Four independent endovascular neurosurgeons (raters) reviewed angiograms of 67 patients at 2 time points. κ statistics assessed inter- and intrarater reliability and compared raters’-versus-proceduralists’ scores. Reliability was also assessed for occlusion location and by dichotomizing modified TICI scores (0–2a versus 2b–3).

RESULTS: Interrater reliability was moderate-to-substantial, weighted κ = 0.417–0.703, overall κ = 0.374 (P < .001). The dichotomized modified TICI score had moderate-to-substantial interrater agreement, κ statistics = 0.468–0.715, overall κ = 0.582 (P < .001). Intrarater reliability was moderate-to-almost perfect, weighted κ = 0.594–0.81. The dichotomized modified TICI score had substantial-to-almost perfect reliability, κ = 0.632–0.82. Proceduralists had fair-to-moderate agreement with raters, weighted κ = 0.348–0.574, and the dichotomized modified TICI score had fair-to-moderate agreement, κ = 0.365–0.544. When proceduralists and raters disagreed, proceduralists’ scores were higher in 79.6% of cases. M1 followed by ICA occlusions had the highest agreement.

CONCLUSIONS: The modified TICI score is a practical metric for assessing reperfusion after mechanical thrombectomy, though not without limitations. Agreement improved when scores were dichotomized around the clinically relevant threshold of successful revascularization. Interrater reliability improved with time, suggesting that formal training of interventionalists may improve reporting reliability. Agreement of the modified TICI scale is best with M1 and ICA occlusion and becomes less reliable with more distal or posterior circulation occlusions. These findings should be considered when developing research trials.