Development, validation and recalibration of a prediction model for prediabetes: an EHR and NHANES-based study

Table 2 Model performance metrics and 95% confidence intervals for the LASSO and approximation models in the development cohort. Performance of the approximation model in 2017–2020 (pre-pandemic) NHANES is also shown

	LASSO^a	Approximation^a	External validation of approximation model in NHANES^b	Logistic recalibration of approximation model in NHANES^b	External validation of approximation model in weighted NHANES^b,c	Logistic recalibration of approximation model in weighted NHANES^b,c
N	22,635	22,635	2,348	2,348	2,348	2,348
Candidate variables	35	35
Number of variables selected	23	8
Number of coefficients	23	16	16	16	16	16
Model Performance
Calibration-in-the-large (calibration intercept)	0.000 (-0.031 to 0.032)	0.000 (-0.034 to 0.034)	0.102 (0.001 to 0.199)	-0.000 (-0.097 to 0.102)	0.033 (-0.114 to 0.181)	-0.000 (-0.155 to 0.150)
Calibration slope	1.099 (1.061 to 1.141)	0.992 (0.956 to 1.029)	1.097 (0.993 to 1.209)	1.000 (0.909 to 1.099)	1.103 (0.944 to 1.274)	1.000 (0.864 to 1.151)
Brier score	0.161 (0.158 to 0.163)	0.160 (0.157 to 0.163)	0.163 (0.154 to 0.172)	0.163 (0.155 to 0.171)	0.141 (0.128 to 0.154)	0.141 (0.129 to 0.154)
ICI^d	0.012 (0.009 to 0.016)	0.009 (0.005 to 0.013)	0.020 (0.009 to 0.036)	0.004	0.013 (0.009 to 0.037)	0.007
C-statistic	0.761 (0.755 to 0.768)	0.763 (0.756 to 0.770)	0.787 (0.768 to 0.805)	0.787 (0.767 to 0.807)	0.787 (0.765 to 0.808)	0.787 (0.766 to 0.808)

^aPerformance measures were corrected for optimism using 2,000 bootstrap resamples of the development EHR data, and the 95% confidence intervals for each optimism-corrected performance metric were derived from the 2.5th and 97.5th percentiles of these resamples
^b95% confidence intervals were derived from the 2.5th and 97.5th percentiles of 2,000 bootstrap resamples of the NHANES data
^cThe performance metrics for the weighted NHANES were weighted using normalized fasting subsample weights
^dThe integrated calibration index (ICI) is the average absolute difference between the predicted probabilities and observed probabilities derived from the locally weighted scatter plot smoother (LOWESS). For the weighted NHANES, the loess function was used with the normalized fasting subsample weights specified for the weights argument

ISSN: 1472-6947