Impact of SUSAN Denoising and ComBat Harmonization on Machine Learning Model Performance for Malignant Brain Neoplasms

Article Figures & Data

Download figure
Open in new tab
Download powerpoint
FIG 1.
Schematic depicting the study workflow.
Download figure
Open in new tab
Download powerpoint
FIG 2.
Violin plots for all 4 feature sets using the external data show the range of mAUC across different pipelines. Feature set 1: CE_ET and F_PTR; 2: CE_ET and T2_PTR; 3: CE_ET, A_ET and F_PTR; 4: CE_ET only. A indicates ADC; F, FLAIR.
Download figure
Open in new tab
Download powerpoint
FIG 3.
Maximum mAUC heatmaps for the internal and external data for the various models based on the ML algorithm. svmRBF indicates support vector machine-Gaussian kernel; XGB, extreme gradient boosting; RF, random forest; GBRM, generalized boosted regression mode; ENET, multinomial elastic net; MAX, maximum; svmPoly, support vector machine-polynomial kernel.
Download figure
Open in new tab
Download powerpoint
FIG 4.
Histogram plot showing mean differences between the internal and external data model performance for the different pipelines. The red line depicts the mean difference in model performance.

Table 1:

Patient demographic details, scanner, and class distributions in the internal and external data sets

Table 2:

Summary of top 3 performing models in the external data set for each feature set

Feature Set	Processing	Algorithm	Feature Selection	mAUC	LogLoss	Brier Score
CE_ET and F_PTR	SD/none	SVM-P	ICC	0.833	0.871	0.521
	SD/ComBat	GBRM	LinearComb	0.832	0.860	0.507
	SD/none	SVM-RBF	Corr	0.831	0.835	0.519
CE_ET and T2_PTR	None/ComBat	ENET	None	0.841	0.922	0.492
	None/ComBat	SVM-P	LASSO	0.840	0.896	0.505
	None/ComBat	SVM-P	linearComb	0.839	0.915	0.509
CE, ET, A, ET and F, PTR	SD/none	SVM-P	ICC	0.886	0.712	0.414
	SD/none	SVM-P	PCA	0.874	0.699	0.398
	None/none	SVM-P	ICC	0.873	0.764	0.433
CE_ET	SD/none	SVM-P	ICC	0.859	0.789	0.472
	SD/none	SVM-P	None	0.856	0.800	0.499
	SD/none	SVM-P	LASSO	0.856	0.786	0.494

Note:—ENET indicates multinomial elastic net; GBRM, generalized boosted regression mode; LASSO, least absolute shrinkage and selection operator; PCA, principal component analysis; SVM-P, support vector machine-polynomial kernel; SVM-RBF, support vector machine-Gaussian kernel; LinearComb, Linear combination filter; A, ADC; F, FLAIR; Corr, Correlation filter.

Table 3:

Top performing model for each feature set, along with the 3 other data-preprocessing results using the same modeling strategy (external data)

Feature Set	Processing	Algorithm	Feature Selection	mAUC	LogLoss	Brier Score	Best Model
CE_ET and F_PTR^{^a}	None/none	SVM-P	ICC	0.818	0.929	0.548	False
	SD/none^{^b}	SVM-P	ICC	0.833	0.871	0.521	True
	None/ComBat	SVM-P	ICC	0.808	1.029	0.588	False
	SD/ComBat	SVM-P	ICC	0.817	0.949	0.564	False
CE_ET and T2_PTR^{^a}	None/none	ENET	None	0.808	0.904	0.520	False
	SD/none	ENET	None	0.817	0.867	0.499	False
	None/ComBat^{^b}	ENET	None	0.841	0.922	0.492	True
	SD/ ComBat	ENET	None	0.835	0.891	0.487	False
CE_ET, A_ET and F_PTR^{^a}	None/none	SVM-P	ICC	0.873	0.764	0.433	False
	SD/none^{^b}	SVM-P	ICC	0.886	0.712	0.414	True
	None/ComBat	SVM-P	ICC	0.836	0.872	0.520	False
	SD/ ComBat	SVM-P	ICC	0.873	0.749	0.444	False
CE_ET^{^a}	None/none	SVM-P	ICC	0.819	0.881	0.499	False
	SD/None^{^b}	SVM-P	ICC	0.859	0.789	0.472	True
	None/ComBat	SVM-P	ICC	0.821	0.962	0.531	False
	SD/ ComBat	SVM-P	ICC	0.842	0.850	0.512	False

Note:—ENET indicates multinomial elastic net; SVM-P, support vector machine-polynomial kernel, LogLoss, ??????; A, ADC; F, FLAIR
↵^aRow indicates models using BIP only, but with otherwise the same modeling strategy.
↵^bRow indicates the top-performing model for each feature set.