User login
Estimating the Accuracy of Dobutamine Stress Echocardiography and Single-Photon Emission Computed Tomography among Patients Undergoing Noncardiac Surgery
Cardiac complications account for at least one-third of perioperative deaths, and lead to substantial morbidity and cost.1-4 Current guidelines recommend that patients undergo assessment of cardiac risk and functional status prior to noncardiac surgery.5 Preoperative cardiac stress testing is recommended for patients whose predicted cardiac risk exceeds 1%, whose functional status is limited, and for whom testing may change management.5
However, patients are not specifically selected according to risk of coronary artery disease (CAD) in current guidelines. The pretest probability of CAD may vary widely in this patient population, and the resultant accuracy of cardiac stress testing in making the diagnosis of CAD may vary as well.5 Meanwhile, CAD is a clear risk factor for perioperative cardiac events.6-8
Because the pretest probability of CAD is heterogeneous, the optimal modality of cardiac stress testing in this population is unclear. False-positive results would likely lead to inflated estimates of operative risk, expensive and high-risk downstream testing, and potentially cancellation of otherwise beneficial surgeries. Meanwhile, false-negative results would lead to overly optimistic estimates of surgical risk and potentially to surgical intervention at higher levels of risk than would be desirable. Current guidelines leave the selection of either dobutamine stress echocardiography (DSE) or pharmacological stress myocardial perfusion imaging to the clinician.5 To inform decisions regarding the selection of cardiac stress testing modality prior to noncardiac surgery, we conducted this study to estimate the diagnostic accuracy of DSE and single-photon emission computed tomography (SPECT) among this patient population.
METHODS
Surgical Cohort
The American College of Surgeons’ National Surgical Quality Improvement Program (NSQIP) samples patients undergoing surgery at participating hospitals and collects standardized clinical data on preoperative risk factors and postoperative complications.9 We acquired public use data from the 2009 NSQIP cohort, which included more than 336,000 surgical cases from 237 hospitals (principally in the United States). We excluded from our analysis patients undergoing cardiac surgery, patients with a prior diagnosis of CAD, and patients undergoing experimental surgeries. This left a sample of 300,462 for analysis.
Prediction of Dyslipidemia
The model we used to predict the presence of obstructive CAD required the presence or absence of dyslipidemia. A number of variables are common to both NSQIP and the National Health and Nutrition Examination Survey (NHANES), including age, weight, sex, tobacco use, diabetes, and prior stroke.10 Using those common variables, we developed a logistic regression to predict a diagnosis of dyslipidemia, applied that regression to the NSQIP cohort, and dichotomized. To assess the potential impact of misclassification, we performed separate sensitivity analyses in which either no patients or all patients had dyslipidemia.
Prediction of Obstructive CAD
To estimate the probability of obstructive CAD, we applied the risk prediction tool currently recommended by the European Society of Cardiology.11 The clinical version of this tool relies on age; sex; diagnoses of diabetes, hypertension, and dyslipidemia; active tobacco use; and chest pain characteristics to predict the probability of obstructive CAD on coronary angiography. We assumed that all patients in our cohort had nonspecific chest pain, the referent in the calculator.
Prediction of Perioperative Event Risk
To predict the probability of a perioperative cardiac event, we used the Myocardial Infarction or Cardiac Arrest (MICA) calculator, which was derived from an earlier cohort of NSQIP.12 All variables required for this prediction tool were included in the 2009 NSQIP cohort; our categorization of surgeries is included as an online appendix. MICA is one of three prediction tools included in the current American College of Cardiology/American Heart Association (ACC/AHA) guidelines.5
Prediction of Test Accuracy
We searched the MEDLINE database for estimates of the test characteristics of DSE and SPECT that adjusted for workup bias.13 (Also known as sequential-ordering bias, here we refer to the phenomenon whereby further workup is based on the results of diagnostic testing, resulting in underdiagnosis among patients with negative tests and falsely high estimates of sensitivity.14) Although other modalities of myocardial perfusion imaging exist, SPECT appears to be the most widely available, utilized, and studied modality of MPI.15 Our search strategy paired (“Sensitivity and Specificity” [MeSH Terms] AND “Coronary Disease/diagnostic imaging” [MAJR] AND “bias” [TIAB]) with (“Tomography, Emission-Computed, Single-Photon” [MAJR] OR “Echocardiography, Stress” [MAJR]). We reviewed the results for sensitivity and specificity estimates that corrected for workup bias. For each of SPECT and DSE, we drew the sensitivity and specificity from normal distributions based on literature estimates (see Table). We then calculated the expected accuracy of each modality for each patient in our dataset. All analyses were performed in Stata (version 14, College Station, Texas).
RESULTS
The median predicted probability of obstructive CAD was
Both accuracy and PPV were higher for DSE than for SPECT. The predicted accuracy of DSE was greater than that of SPECT in 73.5% of cases overall and in 60.5% of cases with a predicted operative cardiac risk greater than 1%. The mean PPV of DSE was 32.9% (median: 26.7%), while the equivalent PPVs for SPECT were 14.1% and 8.2%, respectively. Among cases with a predicted operative cardiac risk greater than 1%, the mean PPV of DSE was 57.5% (median: 60.2%), while the equivalent PPVs for SPECT were 29.8% and 26.7%, respectively.
DSE had a mean predicted accuracy of 93.0% (median: 96.2%), while SPECT had a mean accuracy of 92.6% (median: 95.6%). The predicted accuracies of DSE and SPECT are shown in the Figure, stratified by predicted perioperative risk across the 1% risk threshold currently used by ACC/AHA guidelines.
In our sensitivity analyses, dyslipidemia had little effect on the comparative accuracy. If no patients had dyslipidemia, DSE would have a higher accuracy than SPECT in 75.7% of cases. If all patients had dyslipidemia, DSE would have a higher predicted accuracy than SPECT in 72.8% of cases. For patients with an operative cardiac risk greater than 1%, the predicted accuracies were 65.0% and 59.4%, respectively.
DISCUSSION
In this study, we demonstrated that the expected accuracy of DSE in the diagnosis of obstructive CAD among patients undergoing noncardiac surgery is higher than that of SPECT. This finding was true in both unselected patients and those selected by a perioperative risk of greater than 1%. The use of SPECT, compared with DSE, would likely result in greater numbers of false-positive tests in this patient population and less accurate results overall.
Cardiac stress testing, as with any diagnostic test, is most useful at intermediate probabilities. Insofar as stress testing offers diagnostic value, our analysis suggests that, in the range of the predicted risk of CAD found in patients undergoing noncardiac surgery, DSE is a more efficient testing strategy. To the extent that making a diagnosis of CAD informs the decision to proceed to surgery, a more accurate test would be preferable. The lower cost of DSE, the lack of ionizing radiation, and the information provided by echocardiography regarding diagnoses other than CAD, if considered, would further amplify that preference.
However, it is important to note that both modalities have limited positive predictive value. In the median patient who meets the currently recommended 1% perioperative event risk threshold, SPECT would lead to 2.74 false positive results for every true positive result. DSE would produce approximately two false positive results for every three true positive results. If lower rates of false-positive testing are desired, different patient selection criteria are required.
A few key limitations of this work warrant discussion. First, our results likely overestimate the probability of obstructive CAD in this population. We assumed that all patients have nonspecific chest pain at the time of the preoperative evaluation, though many patients do not, in fact, have chest pain. Tools to estimate the pretest probability of CAD (such as the ESC tool that we used or the older Diamond-Forrester prediction) are intended to stratify higher-risk patients than are seen in a preoperative setting. If asymptomatic patients seen in preoperative risk assessment clinics have lower risk of CAD than what we have predicted, we will have understated the case for DSE. Moreover, cases sampled from hospitals participating in NSQIP are not representative of the national surgical population. This likely further inflates our estimates of CAD risk compared with the “true” surgical population. Finally, we cannot comment on current practice from these data. Current guidelines recommend preoperative cardiac stress testing for patients whose risk of a perioperative cardiac event exceeds 1%, whose functional status is poor or unknown, and only if said testing will change management.5 Using these data, we cannot determine the pretest probability of patients referred for stress testing before noncardiac surgery.
Still, this analysis suggests that, among patients undergoing noncardiac surgery, selecting patients according to the risk of perioperative events would result in a population at an overall comparatively low risk of CAD, and that in this population, DSE would be more accurate than SPECT for making the diagnosis of CAD. If a diagnosis of CAD would change the decision to proceed to surgery, DSE is likely to be a more efficient test than SPECT.
Acknowledgements
The authors would like to thank Wael Jaber, MD, for his thoughtful comments on a draft of this manuscript.
Disclosures
The authors have nothing to disclose.
Funding
The authors received no specific funding for this work.
1. Devereaux PJ, Xavier D, Pogue J, et al. Characteristics and short-term prognosis of perioperative myocardial infarction in patients undergoing noncardiac surgery: a cohort study. Ann Intern Med. 2011;154(8):523-528. doi: 10.7326/0003-4819-154-8-201104190-00003. PubMed
2. Udeh BL, Dalton JE, Hata JS, Udeh CI, Sessler DI. Economic trends from 2003 to 2010 for perioperative myocardial infarction: a retrospective, cohort study. Anesthesiology. 2014;121(1):36-45. doi: 10.1097/ALN.0000000000000233. PubMed
3. van Waes JAR, Nathoe HM, de Graaff JC, et al. Myocardial injury after noncardiac surgery and its association with short-term mortality. Circulation. 2013;127(23):2264-2271. doi: 10.1161/CIRCULATIONAHA.113.002128. PubMed
4. Wijeysundera DN, Beattie WS, Austin PC, Hux JE, Laupacis A. Non-invasive cardiac stress testing before elective major non-cardiac surgery: population based cohort study. BMJ. 2010;340(jan28 3):b5526-b5526. doi: 10.1136/bmj.b5526. PubMed
5. Fleisher LA, Fleischmann KE, Auerbach AD, et al. 2014 ACC/AHA Guideline on Perioperative Cardiovascular Evaluation and Management of Patients Undergoing Noncardiac Surgery. J Am Coll Cardiol. 2014;64(22):e77-e137. doi: 10.1016/j.jacc.2014.07.944. PubMed
6. Goldman L, Caldera DL, Nussbaum SR, et al. Multifactorial index of cardiac risk in noncardiac surgical procedures. N Engl J Med. 1977;297(16):845-850. doi:10.1056/NEJM197710202971601. PubMed
7. Devereaux PJ, Sessler DI. Cardiac Complications in patients undergoing major noncardiac surgery. N Engl J Med. 2015;373(23):2258-2269. doi:10.1056/NEJMra1502824. PubMed
8. Lee TH, Marcantonio ER, Mangione CM, et al. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999;100(10):1043-1049. doi: 10.1161/01.CIR.100.10.1043. PubMed
9. Cohen ME, Ko CY, Bilimoria KY, et al. Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg. 2013;217(2):336-46.e1. doi: 10.1016/j.jamcollsurg.2013.02.027. PubMed
10. National Health and Nutrition Examination Survey Data. https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?BeginYear=2011. Accessed April 20, 2018.
11. Genders TSS, Steyerberg EW, Hunink MGM, et al. Prediction model to estimate presence of coronary artery disease: retrospective pooled analysis of existing cohorts. BMJ. 2012;344:e3485. doi: 10.1136/bmj.e3485. PubMed
12. Gupta PK, Gupta H, Sundaram A, et al. Development and validation of a risk calculator for prediction of cardiac risk after surgery. Circulation. 2011;124(4):381-387. doi: 10.1161/CIRCULATIONAHA.110.015701. PubMed
13. Blackstone EH, Lauer MS. Caveat emptor: the treachery of work-up bias. J Thorac Cardiovasc Surg. 2004;128(3):341-344. doi: 10.1016/j.jtcvs.2004.03.039. PubMed
14. Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978;299(17):926-930. doi: 10.1056/NEJM197810262991705. PubMed
15. Jaarsma C, Leiner T, Bekkers SC, et al. Diagnostic performance of noninvasive myocardial perfusion imaging using single-photon emission computed tomography, cardiac magnetic resonance, and positron emission tomography imaging for the detection of obstructive coronary artery disease: a meta-analysis. J Am Coll Cardiol. 2012;59(19):1719-1728. doi: 10.1016/j.jacc.2011.12.040. PubMed
16. Geleijnse ML, Krenning BJ, Soliman OII, Nemes A, Galema TW, Cate ten FJ. Dobutamine stress echocardiography for the detection of coronary artery disease in women. Am J Cardiol. 2007;99(5):714-717. doi: 10.1016/j.amjcard.2006.09.124. PubMed
17. Miller TD, Hodge DO, Christian TF, Milavetz JJ, Bailey KR, Gibbons RJ. Effects of adjustment for referral bias on the sensitivity and specificity of single photon emission computed tomography for the diagnosis of coronary artery disease. Am J Med. 2002;112(4):290-297. doi: 10.1016/S0002-9343(01)01111-1. PubMed
Cardiac complications account for at least one-third of perioperative deaths, and lead to substantial morbidity and cost.1-4 Current guidelines recommend that patients undergo assessment of cardiac risk and functional status prior to noncardiac surgery.5 Preoperative cardiac stress testing is recommended for patients whose predicted cardiac risk exceeds 1%, whose functional status is limited, and for whom testing may change management.5
However, patients are not specifically selected according to risk of coronary artery disease (CAD) in current guidelines. The pretest probability of CAD may vary widely in this patient population, and the resultant accuracy of cardiac stress testing in making the diagnosis of CAD may vary as well.5 Meanwhile, CAD is a clear risk factor for perioperative cardiac events.6-8
Because the pretest probability of CAD is heterogeneous, the optimal modality of cardiac stress testing in this population is unclear. False-positive results would likely lead to inflated estimates of operative risk, expensive and high-risk downstream testing, and potentially cancellation of otherwise beneficial surgeries. Meanwhile, false-negative results would lead to overly optimistic estimates of surgical risk and potentially to surgical intervention at higher levels of risk than would be desirable. Current guidelines leave the selection of either dobutamine stress echocardiography (DSE) or pharmacological stress myocardial perfusion imaging to the clinician.5 To inform decisions regarding the selection of cardiac stress testing modality prior to noncardiac surgery, we conducted this study to estimate the diagnostic accuracy of DSE and single-photon emission computed tomography (SPECT) among this patient population.
METHODS
Surgical Cohort
The American College of Surgeons’ National Surgical Quality Improvement Program (NSQIP) samples patients undergoing surgery at participating hospitals and collects standardized clinical data on preoperative risk factors and postoperative complications.9 We acquired public use data from the 2009 NSQIP cohort, which included more than 336,000 surgical cases from 237 hospitals (principally in the United States). We excluded from our analysis patients undergoing cardiac surgery, patients with a prior diagnosis of CAD, and patients undergoing experimental surgeries. This left a sample of 300,462 for analysis.
Prediction of Dyslipidemia
The model we used to predict the presence of obstructive CAD required the presence or absence of dyslipidemia. A number of variables are common to both NSQIP and the National Health and Nutrition Examination Survey (NHANES), including age, weight, sex, tobacco use, diabetes, and prior stroke.10 Using those common variables, we developed a logistic regression to predict a diagnosis of dyslipidemia, applied that regression to the NSQIP cohort, and dichotomized. To assess the potential impact of misclassification, we performed separate sensitivity analyses in which either no patients or all patients had dyslipidemia.
Prediction of Obstructive CAD
To estimate the probability of obstructive CAD, we applied the risk prediction tool currently recommended by the European Society of Cardiology.11 The clinical version of this tool relies on age; sex; diagnoses of diabetes, hypertension, and dyslipidemia; active tobacco use; and chest pain characteristics to predict the probability of obstructive CAD on coronary angiography. We assumed that all patients in our cohort had nonspecific chest pain, the referent in the calculator.
Prediction of Perioperative Event Risk
To predict the probability of a perioperative cardiac event, we used the Myocardial Infarction or Cardiac Arrest (MICA) calculator, which was derived from an earlier cohort of NSQIP.12 All variables required for this prediction tool were included in the 2009 NSQIP cohort; our categorization of surgeries is included as an online appendix. MICA is one of three prediction tools included in the current American College of Cardiology/American Heart Association (ACC/AHA) guidelines.5
Prediction of Test Accuracy
We searched the MEDLINE database for estimates of the test characteristics of DSE and SPECT that adjusted for workup bias.13 (Also known as sequential-ordering bias, here we refer to the phenomenon whereby further workup is based on the results of diagnostic testing, resulting in underdiagnosis among patients with negative tests and falsely high estimates of sensitivity.14) Although other modalities of myocardial perfusion imaging exist, SPECT appears to be the most widely available, utilized, and studied modality of MPI.15 Our search strategy paired (“Sensitivity and Specificity” [MeSH Terms] AND “Coronary Disease/diagnostic imaging” [MAJR] AND “bias” [TIAB]) with (“Tomography, Emission-Computed, Single-Photon” [MAJR] OR “Echocardiography, Stress” [MAJR]). We reviewed the results for sensitivity and specificity estimates that corrected for workup bias. For each of SPECT and DSE, we drew the sensitivity and specificity from normal distributions based on literature estimates (see Table). We then calculated the expected accuracy of each modality for each patient in our dataset. All analyses were performed in Stata (version 14, College Station, Texas).
RESULTS
The median predicted probability of obstructive CAD was
Both accuracy and PPV were higher for DSE than for SPECT. The predicted accuracy of DSE was greater than that of SPECT in 73.5% of cases overall and in 60.5% of cases with a predicted operative cardiac risk greater than 1%. The mean PPV of DSE was 32.9% (median: 26.7%), while the equivalent PPVs for SPECT were 14.1% and 8.2%, respectively. Among cases with a predicted operative cardiac risk greater than 1%, the mean PPV of DSE was 57.5% (median: 60.2%), while the equivalent PPVs for SPECT were 29.8% and 26.7%, respectively.
DSE had a mean predicted accuracy of 93.0% (median: 96.2%), while SPECT had a mean accuracy of 92.6% (median: 95.6%). The predicted accuracies of DSE and SPECT are shown in the Figure, stratified by predicted perioperative risk across the 1% risk threshold currently used by ACC/AHA guidelines.
In our sensitivity analyses, dyslipidemia had little effect on the comparative accuracy. If no patients had dyslipidemia, DSE would have a higher accuracy than SPECT in 75.7% of cases. If all patients had dyslipidemia, DSE would have a higher predicted accuracy than SPECT in 72.8% of cases. For patients with an operative cardiac risk greater than 1%, the predicted accuracies were 65.0% and 59.4%, respectively.
DISCUSSION
In this study, we demonstrated that the expected accuracy of DSE in the diagnosis of obstructive CAD among patients undergoing noncardiac surgery is higher than that of SPECT. This finding was true in both unselected patients and those selected by a perioperative risk of greater than 1%. The use of SPECT, compared with DSE, would likely result in greater numbers of false-positive tests in this patient population and less accurate results overall.
Cardiac stress testing, as with any diagnostic test, is most useful at intermediate probabilities. Insofar as stress testing offers diagnostic value, our analysis suggests that, in the range of the predicted risk of CAD found in patients undergoing noncardiac surgery, DSE is a more efficient testing strategy. To the extent that making a diagnosis of CAD informs the decision to proceed to surgery, a more accurate test would be preferable. The lower cost of DSE, the lack of ionizing radiation, and the information provided by echocardiography regarding diagnoses other than CAD, if considered, would further amplify that preference.
However, it is important to note that both modalities have limited positive predictive value. In the median patient who meets the currently recommended 1% perioperative event risk threshold, SPECT would lead to 2.74 false positive results for every true positive result. DSE would produce approximately two false positive results for every three true positive results. If lower rates of false-positive testing are desired, different patient selection criteria are required.
A few key limitations of this work warrant discussion. First, our results likely overestimate the probability of obstructive CAD in this population. We assumed that all patients have nonspecific chest pain at the time of the preoperative evaluation, though many patients do not, in fact, have chest pain. Tools to estimate the pretest probability of CAD (such as the ESC tool that we used or the older Diamond-Forrester prediction) are intended to stratify higher-risk patients than are seen in a preoperative setting. If asymptomatic patients seen in preoperative risk assessment clinics have lower risk of CAD than what we have predicted, we will have understated the case for DSE. Moreover, cases sampled from hospitals participating in NSQIP are not representative of the national surgical population. This likely further inflates our estimates of CAD risk compared with the “true” surgical population. Finally, we cannot comment on current practice from these data. Current guidelines recommend preoperative cardiac stress testing for patients whose risk of a perioperative cardiac event exceeds 1%, whose functional status is poor or unknown, and only if said testing will change management.5 Using these data, we cannot determine the pretest probability of patients referred for stress testing before noncardiac surgery.
Still, this analysis suggests that, among patients undergoing noncardiac surgery, selecting patients according to the risk of perioperative events would result in a population at an overall comparatively low risk of CAD, and that in this population, DSE would be more accurate than SPECT for making the diagnosis of CAD. If a diagnosis of CAD would change the decision to proceed to surgery, DSE is likely to be a more efficient test than SPECT.
Acknowledgements
The authors would like to thank Wael Jaber, MD, for his thoughtful comments on a draft of this manuscript.
Disclosures
The authors have nothing to disclose.
Funding
The authors received no specific funding for this work.
Cardiac complications account for at least one-third of perioperative deaths, and lead to substantial morbidity and cost.1-4 Current guidelines recommend that patients undergo assessment of cardiac risk and functional status prior to noncardiac surgery.5 Preoperative cardiac stress testing is recommended for patients whose predicted cardiac risk exceeds 1%, whose functional status is limited, and for whom testing may change management.5
However, patients are not specifically selected according to risk of coronary artery disease (CAD) in current guidelines. The pretest probability of CAD may vary widely in this patient population, and the resultant accuracy of cardiac stress testing in making the diagnosis of CAD may vary as well.5 Meanwhile, CAD is a clear risk factor for perioperative cardiac events.6-8
Because the pretest probability of CAD is heterogeneous, the optimal modality of cardiac stress testing in this population is unclear. False-positive results would likely lead to inflated estimates of operative risk, expensive and high-risk downstream testing, and potentially cancellation of otherwise beneficial surgeries. Meanwhile, false-negative results would lead to overly optimistic estimates of surgical risk and potentially to surgical intervention at higher levels of risk than would be desirable. Current guidelines leave the selection of either dobutamine stress echocardiography (DSE) or pharmacological stress myocardial perfusion imaging to the clinician.5 To inform decisions regarding the selection of cardiac stress testing modality prior to noncardiac surgery, we conducted this study to estimate the diagnostic accuracy of DSE and single-photon emission computed tomography (SPECT) among this patient population.
METHODS
Surgical Cohort
The American College of Surgeons’ National Surgical Quality Improvement Program (NSQIP) samples patients undergoing surgery at participating hospitals and collects standardized clinical data on preoperative risk factors and postoperative complications.9 We acquired public use data from the 2009 NSQIP cohort, which included more than 336,000 surgical cases from 237 hospitals (principally in the United States). We excluded from our analysis patients undergoing cardiac surgery, patients with a prior diagnosis of CAD, and patients undergoing experimental surgeries. This left a sample of 300,462 for analysis.
Prediction of Dyslipidemia
The model we used to predict the presence of obstructive CAD required the presence or absence of dyslipidemia. A number of variables are common to both NSQIP and the National Health and Nutrition Examination Survey (NHANES), including age, weight, sex, tobacco use, diabetes, and prior stroke.10 Using those common variables, we developed a logistic regression to predict a diagnosis of dyslipidemia, applied that regression to the NSQIP cohort, and dichotomized. To assess the potential impact of misclassification, we performed separate sensitivity analyses in which either no patients or all patients had dyslipidemia.
Prediction of Obstructive CAD
To estimate the probability of obstructive CAD, we applied the risk prediction tool currently recommended by the European Society of Cardiology.11 The clinical version of this tool relies on age; sex; diagnoses of diabetes, hypertension, and dyslipidemia; active tobacco use; and chest pain characteristics to predict the probability of obstructive CAD on coronary angiography. We assumed that all patients in our cohort had nonspecific chest pain, the referent in the calculator.
Prediction of Perioperative Event Risk
To predict the probability of a perioperative cardiac event, we used the Myocardial Infarction or Cardiac Arrest (MICA) calculator, which was derived from an earlier cohort of NSQIP.12 All variables required for this prediction tool were included in the 2009 NSQIP cohort; our categorization of surgeries is included as an online appendix. MICA is one of three prediction tools included in the current American College of Cardiology/American Heart Association (ACC/AHA) guidelines.5
Prediction of Test Accuracy
We searched the MEDLINE database for estimates of the test characteristics of DSE and SPECT that adjusted for workup bias.13 (Also known as sequential-ordering bias, here we refer to the phenomenon whereby further workup is based on the results of diagnostic testing, resulting in underdiagnosis among patients with negative tests and falsely high estimates of sensitivity.14) Although other modalities of myocardial perfusion imaging exist, SPECT appears to be the most widely available, utilized, and studied modality of MPI.15 Our search strategy paired (“Sensitivity and Specificity” [MeSH Terms] AND “Coronary Disease/diagnostic imaging” [MAJR] AND “bias” [TIAB]) with (“Tomography, Emission-Computed, Single-Photon” [MAJR] OR “Echocardiography, Stress” [MAJR]). We reviewed the results for sensitivity and specificity estimates that corrected for workup bias. For each of SPECT and DSE, we drew the sensitivity and specificity from normal distributions based on literature estimates (see Table). We then calculated the expected accuracy of each modality for each patient in our dataset. All analyses were performed in Stata (version 14, College Station, Texas).
RESULTS
The median predicted probability of obstructive CAD was
Both accuracy and PPV were higher for DSE than for SPECT. The predicted accuracy of DSE was greater than that of SPECT in 73.5% of cases overall and in 60.5% of cases with a predicted operative cardiac risk greater than 1%. The mean PPV of DSE was 32.9% (median: 26.7%), while the equivalent PPVs for SPECT were 14.1% and 8.2%, respectively. Among cases with a predicted operative cardiac risk greater than 1%, the mean PPV of DSE was 57.5% (median: 60.2%), while the equivalent PPVs for SPECT were 29.8% and 26.7%, respectively.
DSE had a mean predicted accuracy of 93.0% (median: 96.2%), while SPECT had a mean accuracy of 92.6% (median: 95.6%). The predicted accuracies of DSE and SPECT are shown in the Figure, stratified by predicted perioperative risk across the 1% risk threshold currently used by ACC/AHA guidelines.
In our sensitivity analyses, dyslipidemia had little effect on the comparative accuracy. If no patients had dyslipidemia, DSE would have a higher accuracy than SPECT in 75.7% of cases. If all patients had dyslipidemia, DSE would have a higher predicted accuracy than SPECT in 72.8% of cases. For patients with an operative cardiac risk greater than 1%, the predicted accuracies were 65.0% and 59.4%, respectively.
DISCUSSION
In this study, we demonstrated that the expected accuracy of DSE in the diagnosis of obstructive CAD among patients undergoing noncardiac surgery is higher than that of SPECT. This finding was true in both unselected patients and those selected by a perioperative risk of greater than 1%. The use of SPECT, compared with DSE, would likely result in greater numbers of false-positive tests in this patient population and less accurate results overall.
Cardiac stress testing, as with any diagnostic test, is most useful at intermediate probabilities. Insofar as stress testing offers diagnostic value, our analysis suggests that, in the range of the predicted risk of CAD found in patients undergoing noncardiac surgery, DSE is a more efficient testing strategy. To the extent that making a diagnosis of CAD informs the decision to proceed to surgery, a more accurate test would be preferable. The lower cost of DSE, the lack of ionizing radiation, and the information provided by echocardiography regarding diagnoses other than CAD, if considered, would further amplify that preference.
However, it is important to note that both modalities have limited positive predictive value. In the median patient who meets the currently recommended 1% perioperative event risk threshold, SPECT would lead to 2.74 false positive results for every true positive result. DSE would produce approximately two false positive results for every three true positive results. If lower rates of false-positive testing are desired, different patient selection criteria are required.
A few key limitations of this work warrant discussion. First, our results likely overestimate the probability of obstructive CAD in this population. We assumed that all patients have nonspecific chest pain at the time of the preoperative evaluation, though many patients do not, in fact, have chest pain. Tools to estimate the pretest probability of CAD (such as the ESC tool that we used or the older Diamond-Forrester prediction) are intended to stratify higher-risk patients than are seen in a preoperative setting. If asymptomatic patients seen in preoperative risk assessment clinics have lower risk of CAD than what we have predicted, we will have understated the case for DSE. Moreover, cases sampled from hospitals participating in NSQIP are not representative of the national surgical population. This likely further inflates our estimates of CAD risk compared with the “true” surgical population. Finally, we cannot comment on current practice from these data. Current guidelines recommend preoperative cardiac stress testing for patients whose risk of a perioperative cardiac event exceeds 1%, whose functional status is poor or unknown, and only if said testing will change management.5 Using these data, we cannot determine the pretest probability of patients referred for stress testing before noncardiac surgery.
Still, this analysis suggests that, among patients undergoing noncardiac surgery, selecting patients according to the risk of perioperative events would result in a population at an overall comparatively low risk of CAD, and that in this population, DSE would be more accurate than SPECT for making the diagnosis of CAD. If a diagnosis of CAD would change the decision to proceed to surgery, DSE is likely to be a more efficient test than SPECT.
Acknowledgements
The authors would like to thank Wael Jaber, MD, for his thoughtful comments on a draft of this manuscript.
Disclosures
The authors have nothing to disclose.
Funding
The authors received no specific funding for this work.
1. Devereaux PJ, Xavier D, Pogue J, et al. Characteristics and short-term prognosis of perioperative myocardial infarction in patients undergoing noncardiac surgery: a cohort study. Ann Intern Med. 2011;154(8):523-528. doi: 10.7326/0003-4819-154-8-201104190-00003. PubMed
2. Udeh BL, Dalton JE, Hata JS, Udeh CI, Sessler DI. Economic trends from 2003 to 2010 for perioperative myocardial infarction: a retrospective, cohort study. Anesthesiology. 2014;121(1):36-45. doi: 10.1097/ALN.0000000000000233. PubMed
3. van Waes JAR, Nathoe HM, de Graaff JC, et al. Myocardial injury after noncardiac surgery and its association with short-term mortality. Circulation. 2013;127(23):2264-2271. doi: 10.1161/CIRCULATIONAHA.113.002128. PubMed
4. Wijeysundera DN, Beattie WS, Austin PC, Hux JE, Laupacis A. Non-invasive cardiac stress testing before elective major non-cardiac surgery: population based cohort study. BMJ. 2010;340(jan28 3):b5526-b5526. doi: 10.1136/bmj.b5526. PubMed
5. Fleisher LA, Fleischmann KE, Auerbach AD, et al. 2014 ACC/AHA Guideline on Perioperative Cardiovascular Evaluation and Management of Patients Undergoing Noncardiac Surgery. J Am Coll Cardiol. 2014;64(22):e77-e137. doi: 10.1016/j.jacc.2014.07.944. PubMed
6. Goldman L, Caldera DL, Nussbaum SR, et al. Multifactorial index of cardiac risk in noncardiac surgical procedures. N Engl J Med. 1977;297(16):845-850. doi:10.1056/NEJM197710202971601. PubMed
7. Devereaux PJ, Sessler DI. Cardiac Complications in patients undergoing major noncardiac surgery. N Engl J Med. 2015;373(23):2258-2269. doi:10.1056/NEJMra1502824. PubMed
8. Lee TH, Marcantonio ER, Mangione CM, et al. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999;100(10):1043-1049. doi: 10.1161/01.CIR.100.10.1043. PubMed
9. Cohen ME, Ko CY, Bilimoria KY, et al. Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg. 2013;217(2):336-46.e1. doi: 10.1016/j.jamcollsurg.2013.02.027. PubMed
10. National Health and Nutrition Examination Survey Data. https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?BeginYear=2011. Accessed April 20, 2018.
11. Genders TSS, Steyerberg EW, Hunink MGM, et al. Prediction model to estimate presence of coronary artery disease: retrospective pooled analysis of existing cohorts. BMJ. 2012;344:e3485. doi: 10.1136/bmj.e3485. PubMed
12. Gupta PK, Gupta H, Sundaram A, et al. Development and validation of a risk calculator for prediction of cardiac risk after surgery. Circulation. 2011;124(4):381-387. doi: 10.1161/CIRCULATIONAHA.110.015701. PubMed
13. Blackstone EH, Lauer MS. Caveat emptor: the treachery of work-up bias. J Thorac Cardiovasc Surg. 2004;128(3):341-344. doi: 10.1016/j.jtcvs.2004.03.039. PubMed
14. Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978;299(17):926-930. doi: 10.1056/NEJM197810262991705. PubMed
15. Jaarsma C, Leiner T, Bekkers SC, et al. Diagnostic performance of noninvasive myocardial perfusion imaging using single-photon emission computed tomography, cardiac magnetic resonance, and positron emission tomography imaging for the detection of obstructive coronary artery disease: a meta-analysis. J Am Coll Cardiol. 2012;59(19):1719-1728. doi: 10.1016/j.jacc.2011.12.040. PubMed
16. Geleijnse ML, Krenning BJ, Soliman OII, Nemes A, Galema TW, Cate ten FJ. Dobutamine stress echocardiography for the detection of coronary artery disease in women. Am J Cardiol. 2007;99(5):714-717. doi: 10.1016/j.amjcard.2006.09.124. PubMed
17. Miller TD, Hodge DO, Christian TF, Milavetz JJ, Bailey KR, Gibbons RJ. Effects of adjustment for referral bias on the sensitivity and specificity of single photon emission computed tomography for the diagnosis of coronary artery disease. Am J Med. 2002;112(4):290-297. doi: 10.1016/S0002-9343(01)01111-1. PubMed
1. Devereaux PJ, Xavier D, Pogue J, et al. Characteristics and short-term prognosis of perioperative myocardial infarction in patients undergoing noncardiac surgery: a cohort study. Ann Intern Med. 2011;154(8):523-528. doi: 10.7326/0003-4819-154-8-201104190-00003. PubMed
2. Udeh BL, Dalton JE, Hata JS, Udeh CI, Sessler DI. Economic trends from 2003 to 2010 for perioperative myocardial infarction: a retrospective, cohort study. Anesthesiology. 2014;121(1):36-45. doi: 10.1097/ALN.0000000000000233. PubMed
3. van Waes JAR, Nathoe HM, de Graaff JC, et al. Myocardial injury after noncardiac surgery and its association with short-term mortality. Circulation. 2013;127(23):2264-2271. doi: 10.1161/CIRCULATIONAHA.113.002128. PubMed
4. Wijeysundera DN, Beattie WS, Austin PC, Hux JE, Laupacis A. Non-invasive cardiac stress testing before elective major non-cardiac surgery: population based cohort study. BMJ. 2010;340(jan28 3):b5526-b5526. doi: 10.1136/bmj.b5526. PubMed
5. Fleisher LA, Fleischmann KE, Auerbach AD, et al. 2014 ACC/AHA Guideline on Perioperative Cardiovascular Evaluation and Management of Patients Undergoing Noncardiac Surgery. J Am Coll Cardiol. 2014;64(22):e77-e137. doi: 10.1016/j.jacc.2014.07.944. PubMed
6. Goldman L, Caldera DL, Nussbaum SR, et al. Multifactorial index of cardiac risk in noncardiac surgical procedures. N Engl J Med. 1977;297(16):845-850. doi:10.1056/NEJM197710202971601. PubMed
7. Devereaux PJ, Sessler DI. Cardiac Complications in patients undergoing major noncardiac surgery. N Engl J Med. 2015;373(23):2258-2269. doi:10.1056/NEJMra1502824. PubMed
8. Lee TH, Marcantonio ER, Mangione CM, et al. Derivation and prospective validation of a simple index for prediction of cardiac risk of major noncardiac surgery. Circulation. 1999;100(10):1043-1049. doi: 10.1161/01.CIR.100.10.1043. PubMed
9. Cohen ME, Ko CY, Bilimoria KY, et al. Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg. 2013;217(2):336-46.e1. doi: 10.1016/j.jamcollsurg.2013.02.027. PubMed
10. National Health and Nutrition Examination Survey Data. https://wwwn.cdc.gov/nchs/nhanes/continuousnhanes/default.aspx?BeginYear=2011. Accessed April 20, 2018.
11. Genders TSS, Steyerberg EW, Hunink MGM, et al. Prediction model to estimate presence of coronary artery disease: retrospective pooled analysis of existing cohorts. BMJ. 2012;344:e3485. doi: 10.1136/bmj.e3485. PubMed
12. Gupta PK, Gupta H, Sundaram A, et al. Development and validation of a risk calculator for prediction of cardiac risk after surgery. Circulation. 2011;124(4):381-387. doi: 10.1161/CIRCULATIONAHA.110.015701. PubMed
13. Blackstone EH, Lauer MS. Caveat emptor: the treachery of work-up bias. J Thorac Cardiovasc Surg. 2004;128(3):341-344. doi: 10.1016/j.jtcvs.2004.03.039. PubMed
14. Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978;299(17):926-930. doi: 10.1056/NEJM197810262991705. PubMed
15. Jaarsma C, Leiner T, Bekkers SC, et al. Diagnostic performance of noninvasive myocardial perfusion imaging using single-photon emission computed tomography, cardiac magnetic resonance, and positron emission tomography imaging for the detection of obstructive coronary artery disease: a meta-analysis. J Am Coll Cardiol. 2012;59(19):1719-1728. doi: 10.1016/j.jacc.2011.12.040. PubMed
16. Geleijnse ML, Krenning BJ, Soliman OII, Nemes A, Galema TW, Cate ten FJ. Dobutamine stress echocardiography for the detection of coronary artery disease in women. Am J Cardiol. 2007;99(5):714-717. doi: 10.1016/j.amjcard.2006.09.124. PubMed
17. Miller TD, Hodge DO, Christian TF, Milavetz JJ, Bailey KR, Gibbons RJ. Effects of adjustment for referral bias on the sensitivity and specificity of single photon emission computed tomography for the diagnosis of coronary artery disease. Am J Med. 2002;112(4):290-297. doi: 10.1016/S0002-9343(01)01111-1. PubMed
© 2018 Society of Hospital Medicine