Abstract
BACKGROUND: The Neck Imaging Reporting and Data System (NI-RADS) is a reporting template used in head and neck cancer posttreatment follow-up imaging.
PURPOSE: Our aim was to evaluate the pooled detection rates of the recurrence of head and neck squamous cell carcinoma based on each NI-RADS category and to compare the diagnostic accuracy between NI-RADS 2 and 3 cutoffs.
DATA SOURCES: The MEDLINE, Scopus, and EMBASE databases were searched.
STUDY SELECTION: This systematic review identified 7 studies with a total of 694 patients (1233 lesions) that were eligible for the meta-analysis.
DATA ANALYSIS: The meta-analysis of pooled recurrence detection rate estimates for each NI-RADS category and the diagnostic accuracy of recurrence with NI-RADS 3 or 2 as the cutoff was performed.
DATA SYNTHESIS: The estimated recurrence rates in each category for primary lesions were 74.4% for NI-RADS 3, 29.0% for NI-RADS 2, and 4.2% for NI-RADS 1. The estimated recurrence rates in each category for cervical lymph nodes were 73.3% for NI-RADS 3, 14.3% for NI-RADS 2, and 3.5% for NI-RADS 1. The area under the curve of the summary receiver operating characteristic for recurrence detection with NI-RADS 3 as the cutoff was 0.887 and 0.983, respectively, higher than 0.869 and 0.919 for the primary sites and cervical lymph nodes, respectively, with NI-RADS 2 as the cutoff.
LIMITATIONS: Given the heterogeneity of the data of the studies, the conclusions should be interpreted with caution.
CONCLUSIONS: This meta-analysis revealed estimated recurrence rates for each NI-RADS category for primary lesions and cervical lymph nodes and showed that NI-RADS 3 has a high diagnostic performance for detecting recurrence.
ABBREVIATIONS:
- AUC
- area under the curve
- CE-CT
- contrast-enhanced CT
- CE-MRI
- contrast-enhanced MRI
- DOR
- diagnostic odds ratio
- NI-RADS
- Neck Imaging Reporting and Data System
- sROC
- summary receiver operating characteristic
Follow-up imaging after head and neck cancer treatment is used for the assessment of the treatment response and the detection of recurrence. Recurrences may involve the primary site and/or cervical lymph nodes, and early detection of such recurrences may facilitate subsequent salvage therapy.1,2 Posttreatment follow-up imaging of head and neck cancer is often challenging, however, because of the anatomic complexity of the head and neck region, complex resection and reconstruction operations, and the posttreatment effects of radiation and chemotherapy that mimic recurrent disease. These factors affect radiologists’ interpretations, rendering them nonuniform and potentially suboptimal.3,4
The Neck Imaging Reporting and Data System (NI-RADS) is a head and neck cancer posttreatment follow-up imaging reporting template that was proposed by the American College of Radiology in 2016 to standardize imaging interpretation and communication between clinicians and radiologists.5 NI-RADS provides standardized terminology, report structure, and evaluation categories to convey the degree of suspicion of recurrence in the interpretation of imaging studies. The NI-RADS lexicon established for the evaluation of both posttreatment primary sites and cervical lymph nodes has 4 categories (category 1 [no evidence of recurrence], category 2 [low suspicion], category 3 [high suspicion], and category 4 [definitive recurrence]). NI-RADS 1–3 provide linked recommendations for clinical management along with an estimate of the degree of suspicion for recurrent head and neck cancer.
Most previously published studies of the diagnostic performance of NI-RADS have been limited by small sample sizes. Therefore, the purpose of this systematic article was to summarize the existing data and estimate the detection rate for recurrent head and neck squamous cell carcinoma for each NI-RADS category and to compare diagnostic test accuracy estimates using NI-RADS 3 versus 2 cutoffs for detecting recurrent lesions.
MATERIALS AND METHODS
Study Selection
This study was performed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 statement.6 We searched the Cochrane database and confirmed that there were no reviews/meta-analyses similar to the present research design. On November 10, 2022, MEDLINE via PubMed, Scopus, and EMBASE databases were screened using the following search terms, without any language or date limits: “Neck Imaging Reporting and Data System” or “NI-RADS.”
Inclusion criteria for this evaluation were as follows:
Data on the number of lesions in each NI-RADs category and the number of proven primary site or cervical lymph node recurrences
Data including either primary sites or lymph nodes in NI-RADs 1, 2, or 3
Data with contrast-enhanced CT (CE-CT), contrast-enhanced MRI (CE-MRI), PET/CT, or PET/MRI
Data with the pathology of squamous cell carcinoma only
Original studies that investigated human findings
In cases of duplicate publications, the highest quality or most recent publication was selected
Written in English
The exclusion criteria were as follows:
Studies published before 2016
Studies without an identified imaging period from treatment
The full text was unavailable
Studies with incomplete data
Review, case reports, and systematic review/meta-analysis articles
Books and conference proceedings only that lacked an associated peer-reviewed full-fledged publication
Data Extraction
Two board-certified radiologists with 13 and 9 years of experience, respectively, in head and neck radiology reviewed the full text of the eligible studies and extracted the following information from the included studies by consensus: first author’s name, study region, publication year, study period, study design, number of patients, age, sex, tumor subsite, pathology, treatment method, type of imaging technique, vendor and model of equipment used, imaging period from therapy, reference standard, and recurrent and nonrecurrent lesions for each NI-RADS category. Any disagreements were resolved by consensus agreement of the evaluators.
Quality and Risk Assessment
The Newcastle-Ottawa Scale was used to assess the quality of the included studies in accordance with the Cochrane Handbook for Systematic Reviews of Interventions for included nonrandomized studies.7,8 The scale rates the following 3 factors: selection (0–4 points), comparability (0–2 points), and exposure (0–3 points), with total scores ranging from 0 (lowest) to 9 (highest). Studies with scores of >6 were identified as “high-quality” choices.
Data Analyses
Proportional meta-analyses were performed using a random effects model or a common effects model to determine the estimated prevalence of recurrent disease for each NI-RADS category. The following data were available for this analysis: 2 articles for which all data from NI-RADS 1–3 of the primary lesions and cervical lymph nodes were available,9,10 1 article with only NI-RADS 3 data available for the primary lesions and cervical lymph nodes,11 1 article with only NI-RADS 2 and NI-RADS 3 data points available for the primary lesion,12 1 article with only primary lesion data from NI-RADS 1–3 due to an additional modification of the NI-RADS assessment for the cervical lymph node,13 1 article with only primary lesion data from NI-RADS 1–3 available,14 and 1 article with only cervical lymph node data from NI-RADS 1–3 available due to case overlap of primary lesion cases with another article.15 Forest plots were used to assess and summarize the data. Heterogeneity among the outcomes of the studies included in this article was evaluated using the I2 statistic.
Significant heterogeneity was indicated by a ratio >50% in I2 statistics. Publication bias was assessed using funnel plots. In 4 articles for which all case number data for recurrence and nonrecurrence from each of NI-RADS 1–3 at the primary lesions were available9,10,13,14 or in 3 articles for which all case number data for recurrence and nonrecurrence from each of NI-RADS 1–3 at the cervical lymph nodes were available,9,10,15 we divided the data for the diagnostic accuracy analysis into groups of NI-RADS 3 and NI-RADS 1/2 when NI-RADS 3 was used as the cutoff, and into groups of NI-RADS 2/3 and NI-RADS 1 when NI-RADS 2 was used as the cutoff. Data were pooled using random or fixed effects models to summarize the estimates of sensitivity, specificity, and diagnostic odds ratio (DOR). Bivariate models were used to construct summary receiver operator characteristic (sROC) curves and calculate the area under the curve (AUC). All statistical analyses were performed using R, Version 4.2.2 (http://www.r-project.org/).
RESULTS
Study Selection and Characteristics
Our initial search identified 257 records, and 185 remained after removing duplicates and/or conference proceedings and book chapters. In the next screening, 85 articles published before 2016, non-English language reports, review articles, case reports, and systematic reviews/meta-analyses were excluded. After applying the inclusion/exclusion criteria, we identified 7 articles with 694 patients (1233 lesions) for this review (Fig 1).9⇓⇓⇓⇓⇓-15
The PRISMA 2020 flow chart for the article-selection process. After applying the selection criteria, we identified 7 articles for the systematic review and the meta-analysis.
The data extracted from the 7 studies are outlined in the Online Supplemental Data. All were published between 2019 and 2022, with 2 studies from North America, 2 from South Asia, 2 from Africa, and 1 from Europe. The studies had a median Newcastle-Ottawa Scale score of 4 (range, 4−5). The 6 studies for which information regarding the sex of individual participants was available included 419 men and 191 women (male/female ratio = 2.2:1). The studies had a mean age range of 49–63.4 years and a median age range of 59–62 years. The primary tumor subsites in the studies included the nasopharynx, oropharynx, hypopharynx, larynx, oral cavity, sinonasal cavity, and salivary gland. The 6 studies for which treatment methods were available included radiation therapy, chemoradiotherapy, surgery, surgery plus radiation therapy, and surgery plus chemoradiotherapy. The imaging modalities in the studies were CE-CT, CE-MRI, PET/CT, or PET/CT with CE-CT. Imaging was performed >1.5−3 months after completion of therapy.
In the reference standard, histology and follow-up were used in all articles to determine recurrent lesions, whereas only follow-up was used in 3 articles to determine nonrecurrent lesions. In NI-RADS at the primary site, 6 articles were available for NI-RADS 3; five, for NI-RADS 2; and 4, for NI-RADS 1, for a total of 710 lesions evaluated. In NI-RADS for cervical lymph nodes, data for 4 articles were available for NI-RADS 3, three for NI-RADS 2, and 3 for NI-RADS 1, for a total of 523 lesions evaluated. For diagnostic performance evaluation using NI-RADS 3 or NI-RADS 2 as a cutoff, 4 articles were available for NI-RADS at the primary sites and 3 articles were available for NI-RADS at the lymph nodes. Seven studies reported 270 recurrent lesions and 963 nonrecurrent lesions at the primary sites and cervical lymph nodes.
Meta-analysis of Summarized Recurrence Rate Estimates for Each NI-RADS Category
A forest plot of the summarized estimates of the prevalence for each NI-RADS category of recurrent lesions at the primary sites or cervical lymph nodes is shown in Figs 2 and 3. At the primary site, the estimated recurrence rate for NI-RADS 3 was 74.4% (95% CI, 59.2%−85.4%; I2 = 74%); for NI-RADS 2, it was 29.0% (95% CI, 21.2%−38.2%; I2 = 47%); and for NI-RADS 1, it was 4.2% (95% CI, 2.6%−6.7%; I 2= 0%). For cervical lymph nodes, the estimated recurrence rates for NI-RADS 3, NI-RADS 2, and NI-RADS 1 were 73.3% (95% CI, 63.0%−81.5%; I2 = 38%), 14.3% (95% CI, 6.1%−30.0%; I2 = 0%), and 3.5% (95% CI, 2.1%−5.8%; I2 = 0%), respectively. Funnel plots of these results are shown in the Online Supplemental Data.
Forest plot of summary estimates of the prevalence of recurrence for each NI-RADS category at the primary site (A, NI-RADS 3. B, NI-RADS 2. C, NI-RADS 1).
Forest plot of summary estimates of the prevalence of recurrence for each NI-RADS category at the neck node (A, NI-RADS 3. B, NI-RADS 2. C, NI-RADS 1).
Meta-analysis of Diagnostic Test Accuracy of Recurrence with NI-RADS 3 or 2 as the Cutoff
Primary Lesion.
A forest plot of the summarized estimates of sensitivity, specificity, and DOR of recurrence detection with NI-RADS 3 or NI-RADS 2 as the cutoff in the primary lesion is shown in the Online Supplemental Data, and the sROC for diagnostic performance is shown in Fig 4. With NI-RADS 3 as the cutoff, the estimated sensitivity for recurrence was 60.6% (95% CI, 39.5%−78.4%), the estimated specificity was 94.6% (95% CI, 84.6%−98.3%), the estimated DOR was 26.6 (95% CI, 13.5−52.4), and the AUC in sROC was 0.887. With NI-RADS 2 as the cutoff, the estimated sensitivity for recurrence was 81.8% (95% CI, 54.5%−94.4%), the estimated specificity was 76.6% (95% CI, 29.5%−96.2%), the estimated DOR was 18.9 (95% CI, 9.4−37.9), and the AUC in sROC was 0.869.
sROC for diagnostic performance in the primary site. A, sROC with NI-RADS 3 as the cutoff. B, sROC with NI-RADS 2 as the cutoff.
Lymph Nodes.
A forest plot of the summarized estimates of sensitivity, specificity, and the DOR of recurrence detection with NI-RADS 3 or NI-RADS 2 as the cutoff for cervical lymph nodes is shown in the Online Supplemental Data, and the sROC for diagnostic performance is shown in Fig 5. With NI-RADS 3 as the cutoff, the estimated sensitivity for recurrence was 66.9% (95% CI, 28.9%–90.9%), the estimated specificity was 98.6% (95% CI, 96.7%–99.4%), the estimated DOR was 96.0 (95% CI, 31.1–296.2), and the AUC in sROC was 0.983. With NI-RADS 2 as the cutoff, the estimated sensitivity for recurrence was 75.7% (95% CI, 38.5%–94.0%), the estimated specificity was 89.3% (95% CI, 78.8%–95.0%), the estimated DOR was 21.3 (95% CI, 9.9–45.9), and the AUC in the sROC was 0.919.
sROC for diagnostic performance in the lymph node. A, sROC with NI-RADS 3 as the cutoff. B, sROC with NI-RADS 2 as the cutoff.
DISCUSSION
This systematic review and meta-analysis used 7 studies reporting 694 patients and 1233 lesions to calculate the summarized estimated detection rates of head and neck cancer recurrence for each NI-RADS category and to compare the diagnostic accuracy of NI-RADS 2 and 3 cutoffs to define the optimal cutoff value. The estimated recurrence rates in the categories of primary lesions and cervical lymph nodes were 73.3%−74.4%, 14.3%−29.0%, and 3.5%−4.2% for NI-RADS 3, NI-RADS 2, and NI-RADS 1, respectively. Furthermore, the summarized estimates of specificity, the DOR, and the AUC of the sROC for recurrence detection were higher when NI-RADS 3 was used as a cutoff in primary lesions or cervical lymph nodes than when NI-RADS 2 was used.
The current lexicon in NI-RADS is based on standardized report templates specific to CE-CT and PET/CE-CT, though NI-RADS can also be used for the interpretation of CE-MRI or PET/MR imaging.5,16 Therefore, in addition to CE-CT and PET/CT, there have been a number of articles regarding NI-RADS using CE-MRI and PET/MR imaging.10,12,14,15,17⇓⇓⇓-21 The inclusion of T2-weighted images, DWI, and ADC findings improves the diagnostic performance of NI-RADS.12 Other important MR imaging findings include quantitative values such as ADC values and dynamic contrast-enhanced MR imaging parameters, which are reported to be useful in differentiating recurrent head and neck cancer and posttreatment effects.22,23 Further reporting on the utility of these qualitative and quantitative MR imaging findings for NI-RADS incorporation or the establishment of a revised NI-RADS lexicon that includes these findings and assessment parameters may be warranted in the future but is beyond the scope of this work.
Each NI-RADS category has a different set of clinical recommendations that have been proposed as follows: NI-RADS 2: clinical evaluation of the concerning mucosal region, relatively close follow-up (∼ 3 months) or FDG-PET; and NI-RADS 3 recommends tissue correlation.5 The results of the current study showed that the estimated specificity, estimated DOR, and AUC for head and neck cancer recurrence for NI-RADS 3 as a cutoff were higher than those for NI-RADS 2 for both primary lesions and cervical lymph nodes, with an estimated recurrence rate in NI-RADS 3 as high as 74.5%−74.6%. These results support using NI-RADS 3 as a cutoff for recurrent lesion detection and proceeding to the linked clinical recommendation for NI-RADS 3 of biopsy, an invasive procedure. However, the estimated recurrence rates of 12.2%−29.8% for NI-RADS 2 in the results of this study are a relatively cautionary prevalence that cannot be considered safe. Therefore, close follow-up and direct testing as recommended by the current lexicon for NI-RADS 2 is validated by this analysis.
The utility of liquid biopsy for monitoring recurrence in head and neck cancer is promising24,25 and may have the potential to be incorporated into future NIRADS clinical recommendations.
Our study does have certain limitations, most notably the limited number of available studies. The Newcastle-Ottawa Scale score for all studies was low (4–5), suggesting a high risk of bias. Variations in the type of imaging technique, the location of head and neck cancers, time from treatment to imaging studies, and the reference method among the included studies could have influenced the heterogeneous outcomes. One study that included only NI-RADS 3 cases11 and another study that included only primary NI-RADS 2 and NI-RADS 3 cases12 may have disproportionately affected the results of the meta-analysis of proportions. Some funnel plots exhibited asymmetry indicating potential publication bias. Due to the limited number of included studies, however, a precise evaluation of publication bias was not feasible. Such heterogeneity and bias among studies must be considered a potential limitation when assessing the significance of this analysis.
The NI-RADS assessments and outcome differences may also have been influenced by the treatment method. In the NI-RADS, only local lesions could be classified into NI-RADS 2a and 2b.5 Of the articles included in this study, the number of recurrences and nonrecurrences in NI-RADS 2a and 2b could be extracted for only 2 articles.10,13 Due to the limited number of available articles, this study could not perform a meta-analysis of the NI-RADS 2a/2b. Further studies are warranted to comprehensively evaluate the diagnostic performance of the NI-RADS 2a/2b cutoff. Last, although random effects models were used in some of the tests to treat heterogeneity across studies, our conclusions should still be interpreted with caution because the underlying studies on this topic were not strongly based.26 Appropriately designed prospective large-scale trials are needed to validate the results of this study.
CONCLUSIONS
This meta-analysis demonstrated that NI-RADS 3 has a high diagnostic performance for detecting clinically significant recurrence and confirmed that a NI-RADS category 3 is the optimal cutoff value as a clinical recommendation linked to tissue sampling. Given that NI-RADS 2 lesions also have relatively high estimated recurrence rates, careful follow-up is mandatory in this group of patients. Considering the limitations, including procedural and methodologic heterogeneity among the eligible studies, further investigation and validation of this study are needed.
Acknowledgments
We gratefully acknowledge Mr Mark MacEachern (Taubman Health Sciences Library, University of Michigan) for his advice and Dr Roberto Rivera-de Choudens (Division of Neuroradiology, Department of Radiology, University of Michigan) for his English proofreading.
Footnotes
Disclosure forms provided by the authors are available with the full text and PDF of this article at www.ajnr.org.
References
- Received March 31, 2023.
- Accepted after revision August 12, 2023.
- © 2023 by American Journal of Neuroradiology
