Abstract
BACKGROUND AND PURPOSE: The modified TICI Infarction grading system is a metric currently used to evaluate angiographic results of thrombectomy for large-vessel occlusion in ischemic stroke. Originally designed for evaluating MCA territories, it is currently used for other vessel occlusions, including the posterior circulation. We postulate that the modified TICI use for the posterior circulation is not accurate due to the different vascular territories supplied by vertebrobasilar vasculature, making grading more complex.
MATERIALS AND METHODS: We collected angiographic results from 30 patients who presented with acute posterior circulation occlusions between 2015 and 2018 and underwent thrombectomy in our institution. Eight observers were asked to evaluate the TICI scores before and after thrombectomy. The multirater statistics were computed using Fleiss κ analysis. Further data were collected regarding the potential brain territories at risk and the existence of atherosclerotic disease in the basilar artery.
RESULTS: The overall agreement κ reached 0.277 (SD, 0.013), which suggests a “fair” agreement among the raters. On average, 45% of observers achieved a high accuracy in predicting brain areas at risk of ischemia. As for the existence of basilar atherosclerotic disease, a high agreement (defined as at least 5 of 6 observers) was seen in 20 of the 30 patients.
CONCLUSIONS: Despite TICI being ubiquitous in stroke diagnostics, the high variability of posterior circulation TICI scores calls into question its use in these strokes. Other methods should be developed to assess recanalization in the posterior circulation.
ABBREVIATIONS:
- LVO
- = large-vessel occlusion
- mTICI
- = modified TICI
The TICI grading system is a metric developed by Higashida et al1 to evaluate the response of thrombolytic agents in acute ischemic strokes. Following the demonstration in controlled randomized studies that mechanical thrombectomy is superior to pharmacologic management in anterior ischemic strokes, the 5-point modified TICI (mTICI) scale became instrumental in analyzing pre- and postintervention cerebral perfusion and assessing procedural effectiveness. In the anterior circulatory system, the mTICI score may be a bona fide metric, which strongly correlates with the patient's prognosis and can be effectively used as a guide for clinical decision-making. However, does it make sense to talk about mTICI score in basilar artery occlusions?
Recent advances in endovascular therapies for stroke treatment allow rapid clot retrieval and restoration of perfusion to ischemic brain tissue.2 While patient recovery depends on myriad factors such as the time to reperfusion, the size of the infarct, and the location of the occluded vessel, the ability to achieve rapid and complete flow restoration to ischemic tissues is currently the primary goal of stroke treatment. The assessment of vessel patency not only guides the physician's operative plan, namely deciding on additional attempts at extracting clot versus ending the procedure, but is also a metric used posttreatment to gauge the patient's prognosis and the efficacy of the treatment.3,4 The mTICI scoring system is widely used to encode and compare angiographic results in acute stroke therapies.5 Despite much debate regarding its accuracy and precision and although modifications were suggested, the mTICI score is still the most widely used metric among interventional neuroradiologists.6,7
While the mTICI scoring system is mostly used to assess recanalization in the anterior circulation, recent technologic advances, such as low-profile stent retrievers and hypercompliant, atraumatic aspiration systems, have helped improve the effectiveness of mechanical thrombectomy in the posterior circulation.8⇓⇓-11 Although not yet demonstrated in a randomized trial, posterior vessel thrombectomy is promising, and several trials of the efficacy and safety of thrombectomy for acute basilar artery occlusions are ongoing.
For a lack of a better metric, the mTICI score is widely used for the assessment of the anterior and posterior circulation, but its popularity hides an ugly truth: The mTICI framework is an ill-adapted yardstick in the evaluation of the posterior circulation because the cerebrovascular anatomy is too convoluted, too diverse, and often incompletely rendered by conventional imaging to achieve either accuracy or precision. In the anterior system, some major interobserver variability is reported12 and the posterior side is even more complex. First, within posterior occlusions, there is ample variability, depending on the location of the occlusion, that translates into a variety of clinical presentations and, in turn, causes difficulty in scoring blood flow. For example, there are some critical differences between proximal and distal occlusions of the basilar artery. The prognosis of lateral medullary infarction due to PICA occlusion is more benign, mainly due to the absence of important motor dysfunction.13 Persistent top of the basilar occlusion after a failed thrombectomy attempt might lead to downward extension of thrombi, resulting in catastrophic bilateral midbrain or pontine infarction.14
Second, the multiple etiologies of the posterior circulation stroke15 add a layer of complexity in the scoring. Acute on chronic occlusion of a severely atherosclerotic basilar artery might be more difficult to treat, while it is less clinically devastating due to the development of collaterals.16 Cardioembolic basilar strokes typically lead to larger infarcts, due to larger clots and insufficiently developed collateral circulation.17 Third, often the severity of the disease does not follow the intuition gained from imaging, rendering the scoring moot. For example, the lack of the posterior cerebral artery on angiographic images might lead the operator to assign a low mTICI score because the posterior cerebral artery is a major artery of the posterior system and predicts a devastating stroke. However, the occlusions of the basilar perforators, which look more benign and would be scored higher on the mTICI scale, lead to far more serious conditions. We postulate that these major considerations are regarded differently by different operators, resulting in different mTICI grading.
The aim of this work was to examine whether the mTICI score is a bona fide metric of posterior vascularization states and whether it can reliably guide the clinical intent: attempt another pass for better recanalization or stop. Even though numerous outside parameters can influence the progression of the disease postintervention, we framed the decision to continue to treat or stop as a prediction of the likely course of postendovascular treatment. To test our hypotheses, we measured the interobserver reliability with the Fleiss κ of mTICI scoring of 8 trained raters in adjudicating outcomes of endovascular treatment for ischemic strokes of the posterior circulation.
MATERIALS AND METHODS
Patients
We collected angiographic results from 30 patients (men/women = 19:11, mean age = 68 years) who presented with acute posterior circulation occlusions between the 2015 and 2018 and were treated endovascularly in our institution (Department of Interventional Neuroradiology, Hospices Civils de Lyon, Lyon, France). The inclusion criteria were a clinical presentation with a basilar artery occlusion, at least 1 thrombectomy attempt, and proper pre- and postthrombectomy imaging.
Image Acquisition
DSA images from angiograms of the vertebrobasilar system were acquired during the thrombectomies using a biplane angiography system (Axiom Artis dBA; Siemens). Consecutive anterior-posterior and lateral angiographic images before and after thrombectomy from several consecutive arterial phases were obtained in JPEG format and converted to a movie file format. When they were available, we included images acquired from previous CT or MR imaging scans to assess the existence of patent collaterals, most notably posterior communicating arteries. The images were uploaded to an online survey.
Image Interpretation and mTICI Scoring
Eight observers were asked to assess pairs of angiographic images taken from 30 patients and to evaluate the mTICI scores before and after thrombectomy. They were instructed to use the modified 5-point mTICI scoring system defined as the following: grade 0, no perfusion; grade 1, contrast penetration with minimal perfusion; grade 2a, partial filling (less than half) of the entire vascular territory; grade 2b, perfusion of half or greater of the vascular territory; grade 3, complete and normal rate of perfusion. The mTICI guidelines were available to the graders before presentation of the cases.
Further questions concerned the potential brain territories at risk. The observers were asked to evaluate whether ≥1 of the following territories was at risk of ischemia: the occipital lobes, the thalamus, the brainstem, or the cerebellum. Six of the 8 observers were asked to evaluate the brain territories that were likely to be injured by the stroke because the other 2 observers were familiar with the postthrombectomy images and were excluded from this part of the study. Postthrombectomy imaging (brain CT or MR imaging) was analyzed by a single interpreter and was compared with the formal interpretation in the radiology report. Ischemic zones were defined as CT hypodensities or MR imaging hyperintensities on DWI or FLAIR imaging We also asked the observers whether they found evidence of basilar atherosclerotic disease (basilar artery stenosis or vessel irregularities) for each case.
Statistical Analysis
Interrater variability is a mathematic analysis tool that measures the degree of agreement among observers and can be used to evaluate the level of consensus and homogeneity of a ranking system. It is useful because it allows rating scale systems, such as mTICI, and quantifying their precision. We used the Fleiss κ to assess how closely related 8 trained neurointerventionalists grade the perfusion status of patients with stroke pre- and post-thrombectomy on the 5-step mTICI scale: 1) 0, 2) 1, 3) 2a, 4) 2b, and 5) 3. The Fleiss κ is a number computed from the observation data that describes the degree of agreement over that achieved by randomly assigning grades. Fleiss κ is computed from the following formula:
where 
measures the degree of agreement obtained above chance, and 
is a normalization coefficient that represents the maximum attainable degree of agreement. If the data are completely in agreement, meaning that all the reviewers graded all the cases exactly the same, then 
equals 1 and κ equals 1. In another scenario, if the judges select their answers randomly, 
equals 
and κ equals 0. Note that κ can become negative when raters are in disagreement more times than when the grades are assigned randomly.
The multirater statistics were computed using Fleiss κ analysis with the statistics software SPSS, Version 24 (IBM). The encoded macro takes the grades assigned by the trained observers as input and implements the methodology of Fleiss et al18 to compute the agreement among raters when the number of raters is >2. The output of the software package is the values of the κ coefficient, which can assume numeric values between −1 and 1. The Fleiss κ coefficients can be interpreted according to the guidelines proposed by Landis and Koch: 0 indicating poor, 0.01–0.20 slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, and 0.81–1.0 almost perfect agreement.
In addition, for every case, we computed the percentage of raters who agreed on predicting the potential brain territories at risk, and we defined a high degree of agreement when raters identified correctly at least 3 of the 4 zones of possible ischemia.
RESULTS
The interobserver agreement among the 8 observers is demonstrated in the Table. κ values are summarized for our observers' scores of angiographic outcomes according to the mTICI score. The overall agreement κ reached 0.277 (SD, 0.013), which suggests a “fair” agreement among the raters. Most interesting, 1 rater (rater 3) had a significantly lower agreement rate compared with the other 7 observers. We, therefore, retested the overall agreement excluding rater 3, which resulted in a rise of the overall κ to a value of 0.456 (SD, 0.018), indicating a “moderate” agreement among the observers.
Fleiss κ testa
Our analysis highlights diverging behaviors from the raters: In most cases, we observed drastic variability from clusters of raters, which directly questions the existence of a defined mTICI score. For example, case 6 (Fig 1) is an occlusion of the proximal basilar artery with a bilateral fetal configuration of the posterior communicating arteries, which divided the observers. Five of 8 observers (62.5%) assigned the score 2b, 1 (12.5%) gave it a 2a, and 2 (25%) assigned it a score of 1. On the contrary, we observed examples of more consistent analyses, in which the scores were equal or close. This result is demonstrated in case 26 (Fig 2), in which a patent posterior vasculature was achieved following thrombectomy and the analysis was more consensual: mTICI 2b (75%) and 3 (25%).
Case 6. After thrombectomy, interobserver variability.
Case 26, After thrombectomy, interobserver variability.
The Online Supplemental Data show the accuracy of assessment of territory damage. Almost all the cases had follow-up imaging (brain CT or MR imaging scans) (28/30). Twenty patients had follow-up CT scans, and 18, MR imaging scans (some had images available from >1 technique). One patient had no postthrombectomy follow-up available, and another patient had a massive subarachnoid hemorrhage, which did not allow proper evaluation of ischemia. We defined high accuracy as correctly identifying the postthrombectomy ischemic status of at least 3 of 4 potential zones (occipital lobes, thalamus, brainstem, and cerebellum). On average, 45% of observers achieved a high accuracy, meaning that they were able to correctly predict at least 75% of the brain areas at risk. In 6 of 28 thrombectomy cases, no observer reached high accuracy, and in 3 of 28 cases, all observers reached high accuracy.
Basilar artery atherosclerotic disease was graded by 6 of 8 observers. In 23 of 30 cases, at least 1 observer mentioned the existence of basilar atherosclerotic disease. A majority agreement of the existence of basilar atherosclerotic disease (at least 4 of 6 observers) was noted in 8 of the 30 cases. A high agreement (defined as at least 5 of 6 observers) for the existence or the lack of basilar atherosclerotic disease was seen in 20 of the 30 cases.
DISCUSSION
Recent advances in mechanical thrombectomy for the treatment of acute ischemic stroke with large-artery occlusion have made this treatment safe and extremely effective. Although the efficacy of mechanical thrombectomy for basilar artery occlusion has not yet been proved in a large, randomized, controlled study, it has become a preferred treatment strategy in many centers. The need for a reliable angiographic scale to assess the recanalization status of posterior strokes stems from the hypothesis—proved in anterior infarcts—that a higher grade of reperfusion leads to less ischemic damage and better functional outcomes. Therefore, such a scale might predict the outcome of patients undergoing posterior thrombectomy. Furthermore, a reliable angiographic scale is needed to quantify the efficacy and complication rates of different treatment methods as well as to compare the treatment options.
Our study shows that the interrater agreement among 8 experienced raters is only “fair,” which seriously questions the usefulness and existence of a reliable metric for posterior strokes.
Our study demonstrates the existence of major discrepancies in mTICI grading among experienced interventionalists, a conclusion that we somehow anticipated, considering that the mTICI score was designed to assess MCA strokes and may not be suited to the posterior system.
These results add to a prior study among 3 raters that failed to find substantial mTICI interrater agreement for posterior circulation strokes.19 In the anterior vascular territory, the interrater agreement of mTICI scores is low as well, a finding that may call for the emergence of a new metric altogether.7 There is much debate over the proper way to use the mTICI scale in analyses, ranging from clinical strategizing to the comparison of interventional techniques. There is also a growing consensus among clinicians that the mTICI needs to evolve to improve its clinical relevance and usefulness. Several groups proposed modified or refined versions of the mTICI scoring system: mTICI 2c that adds an additional level to the original scale and mTICI 67 (expanded mTICI 2b67).6,20⇓-22
Assessing patent arteries in the vertebrobasilar territory is even more difficult than in its anterior pendant for several reasons: First, the collateral circulation from the posterior communicating arteries makes the flow pattern more complex to visualize and analyze; second, the existence of vital perforating arteries that are not DSA-visible, such as the perforating arteries of the basilar artery, adds inherent uncertainty to the grading; and third, the significance of vascular territories is greater than in the anterior circulation. Notable examples are posterior strokes in which major arteries are irrigated, but small perforants supplying critical zones of the brain are occluded and can easily be missed on imaging because of either their size or larger arteries that are themselves irrigated through fluid redundancies of the circle of Willis. Specifically, strokes of the superior part of the basilar artery with occluded thalamic perforants but patent posterior cerebral arteries fed by the posterior communicating arteries are silent killers because they often lead to misdiagnoses.
The major differences of mTICI scoring among different operators is important because successful recanalization (mTICI 2b or 3) has been associated with good outcome and survival in basilar artery occlusions.23
Another interesting finding from our work is the inability of our experienced interventionists to predict follow-up MR imaging diffusion lesions or endangered territories. This has several possible explanations: the existence of collaterals, the existence of varied vascular architectures, and the different pathologies of brainstem strokes or the lack of data regarding time to recanalization and baseline perfusion. Given the difficulty in predicting ischemic lesions, as demonstrated by our data, we suggest including MR imaging diffusion imaging as a routine for posterior circulation postthrombectomy evaluations.
We noticed that in almost all cases, the observers overestimated the number of damaged territories. An interesting example is shown in case 15, in which most reviewers marked multiple territories at risk, while the control MR imaging and CT scan did not show any residual ischemia in the vertebrobasilar territory.
We also saw differences regarding the existence of basilar atherosclerotic disease. There was a high accordance of at least 5 of 6 observers regarding the existence of basilar atherosclerotic disease in only two-thirds of cases. Figures 3 and 4 show 2 cases that illustrate the variability of basilar atherosclerosis diagnostics. In case 1, there was an even split between reviewers on the existence of atherosclerosis in the basilar artery, and no clear diagnostic could be established. In stark contrast, all the reviewers agreed on the presence of atherosclerosis in the basilar artery in another case (case 16).
Case 1. Fifty percent of reviewers marked basilar atherosclerotic disease.
Case 16. All reviewers marked basilar atherosclerotic disease.
Basilar atherosclerosis might be determined when stenosis of 50%–99% is seen on initial or postthrombectomy angiography as defined be the Warfarin–Aspirin Symptomatic Intracranial Disease (WASID) study.24 Occlusion of the more proximal segment of the basilar artery is often associated with basilar artery atherosclerotic disease, while more distal occlusion usually results from the embolic source.25 Operators might look for vessel wall irregularities or changes in vessel diameter when defining atherosclerotic disease. The importance of correctly identifying basilar artery atherosclerosis might influence the thrombectomy method (the use of intracranial balloon angioplasty and stent placement for example) as well as the following choice of antiplatelet therapy.
Limitations
Our study has some limitations. We used electronic surveys to facilitate the access to our images of interventionalists from different centers. Not all images of the original DSA were available to the reviewers, and not all cases had data regarding collateral circulation. Data regarding the time from onset and time to recanalization were missing. This issue may have affected the reviewers' judgment regarding brainstem areas at risk. Last, our study did not test intrarater agreement, namely whether images with a defined “true score” are consistently graded similarly by the same grader, but this agreement would greatly complement the present study. Given that there is no “true value” for a particular case, the best surrogate that the authors can envision is a large consensus of experts—if it can be reached.
CONCLUSIONS
Despite being ubiquitous in stroke diagnostics, the mTICI score is an antiquated metric designed for the anterior cerebrovasculature that fails to describe the status of blood flow in the posterior circulation because it is both inaccurate and imprecise. Our data show that the interrater reliability among 8 experts is low for attributes such as occlusion severity, recanalization status, endangered territories, and the existence of basilar atherosclerotic disease. The high variability of posterior mTICI scores calls into question the very essence of the mTICI scale and its clinical relevance. A more suitable grading system should consider the territories involved (brainstem, cerebellum, thalamus, and occipital lobes), the occlusion site (proximal versus distal), and the lumen patency of the basilar artery (including clots, atherosclerosis, stent placement, and angioplasty attempts). To more accurately assess the success and infarct size of a procedure, we suggest routine MR imaging as an adjunctive technique.
Footnotes
Disclosures: Jean Raymond—UNRELATED: Employment: Center hospitalier de l'Université de Montréal. Philip M. White—UNRELATED: Consultancy: MicroVention, Terumo; Grants/Grants Pending: Stryker, Penumbra, Medtronic, MicroVention*; Payment for Development of Educational Presentations: European Society of Minimally Invasive Neurological Therapy. Christian A. Taschner—UNRELATED: Employment: Universitätsklinikum Freiburg; Expert Testimony: Expert Panel for medical device regulation, Commission of the European Union. Mikael Mazighi—UNRELATED: Consultancy: Acticor Biotech; Grants/Grants Pending: Agence Nationale pour la Recherche*; Payment for Development of Educational Presentations: Boerhinger Ingelheim. Francis Turjman—UNRELATED: Consultancy: Balt, Medtronic; Payment for Development of Educational Presentations: Medtronic, Balt. *Money paid to the institution.
References
- Received January 19, 2021.
- Accepted after revision March 9, 2021.
- © 2021 by American Journal of Neuroradiology
Jump to section
Related Articles
Cited By...
- No citing articles found.