Predicting Presence of Coronary Artery Disease
Predicting Presence of Coronary Artery Disease
We retrieved databases from 18 hospitals ( Table 1 and Web Appendix Table 2 ). The study population included 5677 patients (3283 men, 2394 women; mean age 58 and 60 years, respectively). Nearly all patients (5190, 91%) underwent CT based coronary angiography, which revealed obstructive coronary artery disease in 1634 (31%). Of these 1634 patients, 1083 (66%) underwent catheter based coronary angiography, which showed positive results in 886 (82%). Of the 3556 patients without obstructive disease on CT based coronary angiography, 526 (15%) underwent catheter based coronary angiography, which showed negative results in 498 (95%). Overall, 2062 (36%) patients underwent catheter based coronary angiography, with 1176 (57%) diagnosed with obstructive coronary artery disease. Missing values occurred in four (0.1%) patients for age, six (0.1%) for symptoms, 126 (2.2%) for hypertension, 189 (3.3%) for diabetes, 187 (3.3%) for dyslipidaemia, 155 (2.7%) for smoking, 354 (6.2%) for body mass index, and 810 (14%) for coronary calcium score.
Of the 3556 patients who did not have obstructive coronary artery disease revealed by CT based coronary angiography, 3030 (85%) did not undergo the catheter based procedure. Results for catheter based coronary angiography were imputed for these patients, and were mostly negative (range 97-98.4% across the multiple imputations), which accords with the high negative predictive value of the CT based procedure. Of the 1634 patients who had obstructive disease revealed by the CT based procedure, 551 (34%) did not undergo subsequent catheter based coronary angiography. For these patients, results for the catheter based procedure were imputed, and were mostly positive (range 65-77% across imputations), which accords with a reduced positive predictive value of the CT based procedure.
External Validation of the Duke Clinical Score. External validation of the Duke clinical score overestimated the probability of severe coronary artery disease, as observed in our dataset (Figure 1 and Web Appendix Table 3 ).
(Enlarge Image)
Figure 1.
Calibration plot of the Duke clinical score, in low prevalence datasets (n=4426). Distribution of predicted probabilities shown separately for patients with and without severe coronary artery disease. Triangles indicate observed proportions of severe disease, by tenths of predicted probability; 95% CI=confidence interval
Development of New Prediction Models.Table 2 summarises the results of the random effects analysis in logistic regression and the continuous net reclassification improvement ( Web Appendix Table 4 provides more detail). The prediction models are available as an online probability calculator (http://rcc.simpal.com/NpfpV5; Web Appendix Fig A2 ). In the clinical model, all predictors except body mass index were significantly associated with obstructive coronary artery disease. The clinical model improved the prediction, compared with the basic model (cross validated c statistic improved from 0.77 to 0.79; Table 2 ). Whereas an abnormal exercise electrocardiography had limited predictive value in the multivariable prediction model ( Web Appendix Table 5 ), the coronary calcium score was a major predictor, which increased the c statistic from 0.79 to 0.88 ( Table 2 ). Most predictor effects decreased after addition of the coronary calcium score; dyslipidaemia and smoking were no longer significant. We obtained similar results when using CT based coronary angiography as the outcome in patients who did not undergo catheter based coronary angiography.
Validation. Figure 2 and Web Appendix Table 6 show the cross validation results for the clinical model. The c statistic ranged from 0.78 to 0.81. The continuous net reclassification improvement was a measure of the relative change in the observed proportion when the predicted probability changes (Web Appendix), which was most favourable (102%) for the extended model compared with the clinical model ( Table 2 ).
(Enlarge Image)
Figure 2.
Validity of clinical model using the four largest low prevalence datasets and the smaller remaining low prevalence databases combined. Distribution of predicted probabilities shown separately for patients with and without obstructive coronary artery disease. Triangles indicate observed proportion of disease, by tenths of the predicted probability; 95% CI=95% confidence interval
Assessment of calibration-in-the-large showed a significant difference between the mean observed outcome and the predicted probability (clinical model), for Azienda Ospedaliero Universitaria Parma, Rotterdam, and the combined low prevalence hospitals (Figure 2). Logistic recalibration showed no significant differences between the overall hospital specific effects of the predictors compared with the overall effects of the predictors in the clinical model (Figure 2). When re-estimated in specific datasets, the predictor effects were not significantly different from the predictor effects in the clinical model, except for the effect of typical chest pain for Azienda Ospedaliero Universitaria Parma. The results indicated that predictor effects were similar across datasets ( Web Appendix Table 6 ).
Results
Data Collection and Study Population
We retrieved databases from 18 hospitals ( Table 1 and Web Appendix Table 2 ). The study population included 5677 patients (3283 men, 2394 women; mean age 58 and 60 years, respectively). Nearly all patients (5190, 91%) underwent CT based coronary angiography, which revealed obstructive coronary artery disease in 1634 (31%). Of these 1634 patients, 1083 (66%) underwent catheter based coronary angiography, which showed positive results in 886 (82%). Of the 3556 patients without obstructive disease on CT based coronary angiography, 526 (15%) underwent catheter based coronary angiography, which showed negative results in 498 (95%). Overall, 2062 (36%) patients underwent catheter based coronary angiography, with 1176 (57%) diagnosed with obstructive coronary artery disease. Missing values occurred in four (0.1%) patients for age, six (0.1%) for symptoms, 126 (2.2%) for hypertension, 189 (3.3%) for diabetes, 187 (3.3%) for dyslipidaemia, 155 (2.7%) for smoking, 354 (6.2%) for body mass index, and 810 (14%) for coronary calcium score.
Of the 3556 patients who did not have obstructive coronary artery disease revealed by CT based coronary angiography, 3030 (85%) did not undergo the catheter based procedure. Results for catheter based coronary angiography were imputed for these patients, and were mostly negative (range 97-98.4% across the multiple imputations), which accords with the high negative predictive value of the CT based procedure. Of the 1634 patients who had obstructive disease revealed by the CT based procedure, 551 (34%) did not undergo subsequent catheter based coronary angiography. For these patients, results for the catheter based procedure were imputed, and were mostly positive (range 65-77% across imputations), which accords with a reduced positive predictive value of the CT based procedure.
External Validation of the Duke Clinical Score. External validation of the Duke clinical score overestimated the probability of severe coronary artery disease, as observed in our dataset (Figure 1 and Web Appendix Table 3 ).
(Enlarge Image)
Figure 1.
Calibration plot of the Duke clinical score, in low prevalence datasets (n=4426). Distribution of predicted probabilities shown separately for patients with and without severe coronary artery disease. Triangles indicate observed proportions of severe disease, by tenths of predicted probability; 95% CI=confidence interval
Development of New Prediction Models.Table 2 summarises the results of the random effects analysis in logistic regression and the continuous net reclassification improvement ( Web Appendix Table 4 provides more detail). The prediction models are available as an online probability calculator (http://rcc.simpal.com/NpfpV5; Web Appendix Fig A2 ). In the clinical model, all predictors except body mass index were significantly associated with obstructive coronary artery disease. The clinical model improved the prediction, compared with the basic model (cross validated c statistic improved from 0.77 to 0.79; Table 2 ). Whereas an abnormal exercise electrocardiography had limited predictive value in the multivariable prediction model ( Web Appendix Table 5 ), the coronary calcium score was a major predictor, which increased the c statistic from 0.79 to 0.88 ( Table 2 ). Most predictor effects decreased after addition of the coronary calcium score; dyslipidaemia and smoking were no longer significant. We obtained similar results when using CT based coronary angiography as the outcome in patients who did not undergo catheter based coronary angiography.
Validation. Figure 2 and Web Appendix Table 6 show the cross validation results for the clinical model. The c statistic ranged from 0.78 to 0.81. The continuous net reclassification improvement was a measure of the relative change in the observed proportion when the predicted probability changes (Web Appendix), which was most favourable (102%) for the extended model compared with the clinical model ( Table 2 ).
(Enlarge Image)
Figure 2.
Validity of clinical model using the four largest low prevalence datasets and the smaller remaining low prevalence databases combined. Distribution of predicted probabilities shown separately for patients with and without obstructive coronary artery disease. Triangles indicate observed proportion of disease, by tenths of the predicted probability; 95% CI=95% confidence interval
Assessment of calibration-in-the-large showed a significant difference between the mean observed outcome and the predicted probability (clinical model), for Azienda Ospedaliero Universitaria Parma, Rotterdam, and the combined low prevalence hospitals (Figure 2). Logistic recalibration showed no significant differences between the overall hospital specific effects of the predictors compared with the overall effects of the predictors in the clinical model (Figure 2). When re-estimated in specific datasets, the predictor effects were not significantly different from the predictor effects in the clinical model, except for the effect of typical chest pain for Azienda Ospedaliero Universitaria Parma. The results indicated that predictor effects were similar across datasets ( Web Appendix Table 6 ).