SCIP Hasn't Improved Key SSI Outcomes

BOCA RATON, FLA. – Adherence to Surgical Care Improvement Program measures that are aimed at preventing surgical site infections has not had the desired effect, according to a large national Veterans Affairs study.

There is widespread agreement that reducing surgical site infections is a worthy quality improvement goal. These infections are associated with a twofold increase in mortality, a 60% increase in ICU admission, and a fivefold greater likelihood of hospital readmission after discharge.

But the VA study results showing that the SCIP hasn’t reduced surgical site infection rates call into question whether the program is worth continuing.

"SCIP adherence is not informative to third-party payers, administrators, or patients. The policy of continued SCIP measurement for public reporting and payment should be reevaluated," Dr. Mary T. Hawn declared in presenting the VA study findings at the annual meeting of the American Surgical Association.

SCIP is a multiyear partnership that was initiated in 2003 with the goal of reducing surgical morbidity and mortality at U.S. hospitals. Among the 10 national organizations that are represented on the SCIP steering committee are the American College of Surgeons, the American Hospital Association, the U.S. Department of Veterans Affairs, the Centers for Medicare and Medicaid Services, the Agency for Healthcare Research and Quality, the Centers for Disease Control and Prevention, and the Joint Commission.

Dr. Hawn presented a retrospective study of 60,853 procedures performed at 112 VA hospitals during 2005-2009. The outcome measure was the combined rate of superficial and deep surgical site infections (SSIs) occurring within 30 days of surgery.

The independent variables were adherence to each of five SCIP surgical site infection prevention measures that hospitals are required to collect and report. The five measures are timely administration of a prophylactic antibiotic, timely discontinuation of the antibiotic, appropriate antibiotic coverage, hair removal, and normothermia for colon procedures.

In addition, investigators tracked the impact of rates of adherence to all five measures in a given case, a combined metric they called composite SCIP.

This was the first study to use individual patient-level data for evaluating SCIP. Investigators were able to adjust for the presence of comorbid conditions that are known to affect the risk of SSIs, such as diabetes, dyspnea, and corticosteroid use, noted Dr. Hawn of the University of Alabama at Birmingham.

The overall SSI rate was 6.2%. It didn’t vary significantly over the 5-year study period, which began when the SCIP measures were first implemented in the VA system. Rates of adherence to the five SCIP measures quickly climbed to high levels during the first 6 months, probably because of the VA’s pay-for-performance incentives, she explained.

A first look at the data suggested that SCIP might be performing as intended. For example, there was an 81% adherence rate to the composite SCIP measure, and surgical cases meeting that standard had a 45% lower SSI rate than did those in which all five measures weren’t met. However, when patient-level SSI risk factors were introduced in a multivariate logistic regression analysis, there was no longer any association between SCIP adherence and SSI rates.

SCIP adherence rates ranged from a high of 86.4% for orthopedic surgery to a low of 60.4% for colorectal procedures. Adherence rates for gynecologic and vascular procedures were 85.9% and 81.6%, respectively.

In a separate analysis, Dr. Hawn and her colleagues looked at the relationship between a hospital’s adherence to SCIP measures and the institutional SSI rate. Once again, they found that there was none. The difference in hospital rates of SCIP adherence accounted for a mere 2% of the variation in hospital-wide SSI rates.

Dr. Hawn said that it is particularly troubling, in light of the VA data, to consider that public reporting of the SCIP adherence rates is being used to guide patients to what are supposed to be high-quality hospitals. "Are we really guiding patients to the right hospitals?" she asked.

Discussant Dr. David B. Hoyt noted that the collection of SCIP data constitutes a huge cost for American hospitals.

"It’s essential that quality measurement systems put in place actually correlate with improvement in quality. This study today is a critical example of how a well-intended process can in fact fail to reduce surgical infections. Overall, the SCIP program does not achieve its goal," said Dr. Hoyt, executive director of the American College of Surgeons.

"The data collection burden has increased in the last several years, and unless indicators that are ineffective are dropped, the expense of adding new indicators cannot be accommodated," he added.

Dr. Donald E. Fry commented that the SCIP measures are valid. The trouble is, they’re not inclusive.

"To paraphrase Paul Simon, ‘There must be 50 ways to get an SSI.’ And antibiotics are only a small portion of that," said Dr. Fry, executive vice president at Michael Pine and Associates, Chicago, an analytic health care consulting firm.

"I hope that this presentation will be a significant stimulus for us to go forward with not measuring silly process measures. This is not synchronized swimming. We need to be measuring outcomes. We need objective measures of what it is we’re trying to do, looking at how good hospitals do it well and bad ones don’t do it so well, and coming up with an entire strategy for SSIs," he said.

The SCIP study was funded by the VA. Dr. Hawn declared having no relevant financial interests. ☐

BOCA RATON, FLA. – Adherence to Surgical Care Improvement Program measures that are aimed at preventing surgical site infections has not had the desired effect, according to a large national Veterans Affairs study.

There is widespread agreement that reducing surgical site infections is a worthy quality improvement goal. These infections are associated with a twofold increase in mortality, a 60% increase in ICU admission, and a fivefold greater likelihood of hospital readmission after discharge.

But the VA study results showing that the SCIP hasn’t reduced surgical site infection rates call into question whether the program is worth continuing.

"SCIP adherence is not informative to third-party payers, administrators, or patients. The policy of continued SCIP measurement for public reporting and payment should be reevaluated," Dr. Mary T. Hawn declared in presenting the VA study findings at the annual meeting of the American Surgical Association.

SCIP is a multiyear partnership that was initiated in 2003 with the goal of reducing surgical morbidity and mortality at U.S. hospitals. Among the 10 national organizations that are represented on the SCIP steering committee are the American College of Surgeons, the American Hospital Association, the U.S. Department of Veterans Affairs, the Centers for Medicare and Medicaid Services, the Agency for Healthcare Research and Quality, the Centers for Disease Control and Prevention, and the Joint Commission.

Dr. Hawn presented a retrospective study of 60,853 procedures performed at 112 VA hospitals during 2005-2009. The outcome measure was the combined rate of superficial and deep surgical site infections (SSIs) occurring within 30 days of surgery.

The independent variables were adherence to each of five SCIP surgical site infection prevention measures that hospitals are required to collect and report. The five measures are timely administration of a prophylactic antibiotic, timely discontinuation of the antibiotic, appropriate antibiotic coverage, hair removal, and normothermia for colon procedures.

In addition, investigators tracked the impact of rates of adherence to all five measures in a given case, a combined metric they called composite SCIP.

This was the first study to use individual patient-level data for evaluating SCIP. Investigators were able to adjust for the presence of comorbid conditions that are known to affect the risk of SSIs, such as diabetes, dyspnea, and corticosteroid use, noted Dr. Hawn of the University of Alabama at Birmingham.

The overall SSI rate was 6.2%. It didn’t vary significantly over the 5-year study period, which began when the SCIP measures were first implemented in the VA system. Rates of adherence to the five SCIP measures quickly climbed to high levels during the first 6 months, probably because of the VA’s pay-for-performance incentives, she explained.

A first look at the data suggested that SCIP might be performing as intended. For example, there was an 81% adherence rate to the composite SCIP measure, and surgical cases meeting that standard had a 45% lower SSI rate than did those in which all five measures weren’t met. However, when patient-level SSI risk factors were introduced in a multivariate logistic regression analysis, there was no longer any association between SCIP adherence and SSI rates.

SCIP adherence rates ranged from a high of 86.4% for orthopedic surgery to a low of 60.4% for colorectal procedures. Adherence rates for gynecologic and vascular procedures were 85.9% and 81.6%, respectively.

In a separate analysis, Dr. Hawn and her colleagues looked at the relationship between a hospital’s adherence to SCIP measures and the institutional SSI rate. Once again, they found that there was none. The difference in hospital rates of SCIP adherence accounted for a mere 2% of the variation in hospital-wide SSI rates.

Dr. Hawn said that it is particularly troubling, in light of the VA data, to consider that public reporting of the SCIP adherence rates is being used to guide patients to what are supposed to be high-quality hospitals. "Are we really guiding patients to the right hospitals?" she asked.

Discussant Dr. David B. Hoyt noted that the collection of SCIP data constitutes a huge cost for American hospitals.

"It’s essential that quality measurement systems put in place actually correlate with improvement in quality. This study today is a critical example of how a well-intended process can in fact fail to reduce surgical infections. Overall, the SCIP program does not achieve its goal," said Dr. Hoyt, executive director of the American College of Surgeons.

"The data collection burden has increased in the last several years, and unless indicators that are ineffective are dropped, the expense of adding new indicators cannot be accommodated," he added.

Dr. Donald E. Fry commented that the SCIP measures are valid. The trouble is, they’re not inclusive.

"To paraphrase Paul Simon, ‘There must be 50 ways to get an SSI.’ And antibiotics are only a small portion of that," said Dr. Fry, executive vice president at Michael Pine and Associates, Chicago, an analytic health care consulting firm.

"I hope that this presentation will be a significant stimulus for us to go forward with not measuring silly process measures. This is not synchronized swimming. We need to be measuring outcomes. We need objective measures of what it is we’re trying to do, looking at how good hospitals do it well and bad ones don’t do it so well, and coming up with an entire strategy for SSIs," he said.

The SCIP study was funded by the VA. Dr. Hawn declared having no relevant financial interests. ☐

BOCA RATON, FLA. – Adherence to Surgical Care Improvement Program measures that are aimed at preventing surgical site infections has not had the desired effect, according to a large national Veterans Affairs study.

There is widespread agreement that reducing surgical site infections is a worthy quality improvement goal. These infections are associated with a twofold increase in mortality, a 60% increase in ICU admission, and a fivefold greater likelihood of hospital readmission after discharge.

But the VA study results showing that the SCIP hasn’t reduced surgical site infection rates call into question whether the program is worth continuing.

"SCIP adherence is not informative to third-party payers, administrators, or patients. The policy of continued SCIP measurement for public reporting and payment should be reevaluated," Dr. Mary T. Hawn declared in presenting the VA study findings at the annual meeting of the American Surgical Association.

SCIP is a multiyear partnership that was initiated in 2003 with the goal of reducing surgical morbidity and mortality at U.S. hospitals. Among the 10 national organizations that are represented on the SCIP steering committee are the American College of Surgeons, the American Hospital Association, the U.S. Department of Veterans Affairs, the Centers for Medicare and Medicaid Services, the Agency for Healthcare Research and Quality, the Centers for Disease Control and Prevention, and the Joint Commission.

Dr. Hawn presented a retrospective study of 60,853 procedures performed at 112 VA hospitals during 2005-2009. The outcome measure was the combined rate of superficial and deep surgical site infections (SSIs) occurring within 30 days of surgery.

The independent variables were adherence to each of five SCIP surgical site infection prevention measures that hospitals are required to collect and report. The five measures are timely administration of a prophylactic antibiotic, timely discontinuation of the antibiotic, appropriate antibiotic coverage, hair removal, and normothermia for colon procedures.

In addition, investigators tracked the impact of rates of adherence to all five measures in a given case, a combined metric they called composite SCIP.

This was the first study to use individual patient-level data for evaluating SCIP. Investigators were able to adjust for the presence of comorbid conditions that are known to affect the risk of SSIs, such as diabetes, dyspnea, and corticosteroid use, noted Dr. Hawn of the University of Alabama at Birmingham.

The overall SSI rate was 6.2%. It didn’t vary significantly over the 5-year study period, which began when the SCIP measures were first implemented in the VA system. Rates of adherence to the five SCIP measures quickly climbed to high levels during the first 6 months, probably because of the VA’s pay-for-performance incentives, she explained.

A first look at the data suggested that SCIP might be performing as intended. For example, there was an 81% adherence rate to the composite SCIP measure, and surgical cases meeting that standard had a 45% lower SSI rate than did those in which all five measures weren’t met. However, when patient-level SSI risk factors were introduced in a multivariate logistic regression analysis, there was no longer any association between SCIP adherence and SSI rates.

SCIP adherence rates ranged from a high of 86.4% for orthopedic surgery to a low of 60.4% for colorectal procedures. Adherence rates for gynecologic and vascular procedures were 85.9% and 81.6%, respectively.

In a separate analysis, Dr. Hawn and her colleagues looked at the relationship between a hospital’s adherence to SCIP measures and the institutional SSI rate. Once again, they found that there was none. The difference in hospital rates of SCIP adherence accounted for a mere 2% of the variation in hospital-wide SSI rates.

Dr. Hawn said that it is particularly troubling, in light of the VA data, to consider that public reporting of the SCIP adherence rates is being used to guide patients to what are supposed to be high-quality hospitals. "Are we really guiding patients to the right hospitals?" she asked.

Discussant Dr. David B. Hoyt noted that the collection of SCIP data constitutes a huge cost for American hospitals.

"It’s essential that quality measurement systems put in place actually correlate with improvement in quality. This study today is a critical example of how a well-intended process can in fact fail to reduce surgical infections. Overall, the SCIP program does not achieve its goal," said Dr. Hoyt, executive director of the American College of Surgeons.

"The data collection burden has increased in the last several years, and unless indicators that are ineffective are dropped, the expense of adding new indicators cannot be accommodated," he added.

Dr. Donald E. Fry commented that the SCIP measures are valid. The trouble is, they’re not inclusive.

"To paraphrase Paul Simon, ‘There must be 50 ways to get an SSI.’ And antibiotics are only a small portion of that," said Dr. Fry, executive vice president at Michael Pine and Associates, Chicago, an analytic health care consulting firm.

"I hope that this presentation will be a significant stimulus for us to go forward with not measuring silly process measures. This is not synchronized swimming. We need to be measuring outcomes. We need objective measures of what it is we’re trying to do, looking at how good hospitals do it well and bad ones don’t do it so well, and coming up with an entire strategy for SSIs," he said.

The SCIP study was funded by the VA. Dr. Hawn declared having no relevant financial interests. ☐