User login
Postoperative venous thromboembolism rates may not be an effective way of measuring hospital quality, according to Dr. Karl Y. Bilimoria and his colleagues.
The investigators calculated patient-level rates of venous thromboembolism as well as rates of imaging for VTE using data from the American Hospital Association and Medicare Compare from 2009 to 2010 from nearly 1 million patients discharged from 2,786 hospitals after a major surgery.
They sought to determine the association between hospital adherence to VTE reduction protocols (Surgical Care Improvement Project for VTE or SCIP-VTE-2) and risk-adjusted rates of VTE as measured by Patient Safety Indicator 12 (PSI-12) from the Agency for Healthcare Research and Quality. They also looked at how overall hospital quality scores correlated with VTE prophylaxis and risk-adjusted VTE scores.
They presented their findigns at the annual clinical congress of the American College of Surgeons and in JAMA [doi:10.1001/jama.2013.280048]).
Hospitals that adhered consistently to VTE reduction protocols paradoxically had higher PSI-12 scores, although not significantly so (P = .03). Hospitals with higher overall quality scores also adhered to VTE reduction protocols at a higher rate (93.3% in the lowest quartile vs. 95.5% in the highest) and had significantly higher risk-adjusted VTE event scores (P less than .001).
"Most important, hospital VTE rates were associated with the intensity of detecting VTE with imaging studies," the investigators said. Mean VTE diagnostic imaging rates ranged from 32/1,000 in the lowest quartile to 167/1,000 in the highest. Hospitals with the lowest imaging rates diagnosed 5.0 VTEs per 1,000 discharges, compared with hospitals with the highest imaging rates diagnosing 13.5 VTEs per 1,000 discharges.
In effect, PSI-12 scores the use of VTE imaging by hospitals instead of the quality of care provided, the investigators said. Further, surveillance bias impedes quality performance improvements; thus, decision making becomes more difficult for "patients seeking to identify a high-quality hospital."
In an accompanying editorial, Dr. Edwin H. Livingston, deputy editor of JAMA, noted that hypervigilance of VTEs might further worsen care in that "the very high compliance rate with VTE prophylaxis might result from many patients receiving treatments from which they are not likely to benefit. This is because current process measures were based on older guidelines that overestimated the benefits of VTE prophylaxis" (JAMA [doi:10.1001/jama.2013.280049]).
For that reason, Dr. Livingston recommended that public reporting of VTEs be "reconsidered or curtailed because few hospitals have sufficient numbers of patients to show statistically significant effects of prophylactic measures on VTE rates."
Measuring outcomes in general and safety events in particular is a complex proposition. This is particularly true when using patient safety indicators (PSIs) and hospital-acquired conditions as outcome metrics to compare performance across organizations.
In addition to the usual challenging nuances such as severity of illness adjustment, these indicators rely on accurate documentation and coding and as the Agency for Healthcare Research and Quality states: PSIs identify "potential in-hospital complications and adverse events following surgeries, procedures, and childbirth."
This is well meaning when an analytic team uses these metrics as part of a comprehensive quality and patient safety program to identify potential internal improvement opportunities. However, there are real limitations when using these metrics as outcomes that are tied to public reporting initiatives, payment incentives, and rankings.
The study in JAMA by Dr. Bilimoria and his colleagues highlights another limitation of some of these metrics – that of surveillance bias. Using PSI-12 (postoperative venous thromboembolism), risk-adjusted VTE rates were shown to correlate positively with intensity of imaging use (surveillance) and inversely with other measures of quality such as structure or process.
Thus, those with the highest VTE rates did everything right, but also looked for events more often.
This finding complicates the use of PSI-12 as an indicator to compare outcomes across health care systems. However, when used as an internal driver in the context of other local metrics of quality and safety, the original intent of PSI-12 as an indicator of potential hospital complications does not change.
This highlights the importance of health care systems in understanding the strengths and limitations of quality and safety metrics and in developing the analytic capabilities to turn data points into real opportunities to deliver better care, rather than going down the proverbial rabbit hole. Yet, regulatory agencies should also recognize that using imperfect metrics as a part of payment-reform initiatives needs to be done with extreme caution or there will be unintended consequences that do not lead to our collective goal of exceptional value in health care for our patients.
Dr. Robert Pendleton is chief medical quality officer for University of Utah Health Care, Salt Lake City.
Measuring outcomes in general and safety events in particular is a complex proposition. This is particularly true when using patient safety indicators (PSIs) and hospital-acquired conditions as outcome metrics to compare performance across organizations.
In addition to the usual challenging nuances such as severity of illness adjustment, these indicators rely on accurate documentation and coding and as the Agency for Healthcare Research and Quality states: PSIs identify "potential in-hospital complications and adverse events following surgeries, procedures, and childbirth."
This is well meaning when an analytic team uses these metrics as part of a comprehensive quality and patient safety program to identify potential internal improvement opportunities. However, there are real limitations when using these metrics as outcomes that are tied to public reporting initiatives, payment incentives, and rankings.
The study in JAMA by Dr. Bilimoria and his colleagues highlights another limitation of some of these metrics – that of surveillance bias. Using PSI-12 (postoperative venous thromboembolism), risk-adjusted VTE rates were shown to correlate positively with intensity of imaging use (surveillance) and inversely with other measures of quality such as structure or process.
Thus, those with the highest VTE rates did everything right, but also looked for events more often.
This finding complicates the use of PSI-12 as an indicator to compare outcomes across health care systems. However, when used as an internal driver in the context of other local metrics of quality and safety, the original intent of PSI-12 as an indicator of potential hospital complications does not change.
This highlights the importance of health care systems in understanding the strengths and limitations of quality and safety metrics and in developing the analytic capabilities to turn data points into real opportunities to deliver better care, rather than going down the proverbial rabbit hole. Yet, regulatory agencies should also recognize that using imperfect metrics as a part of payment-reform initiatives needs to be done with extreme caution or there will be unintended consequences that do not lead to our collective goal of exceptional value in health care for our patients.
Dr. Robert Pendleton is chief medical quality officer for University of Utah Health Care, Salt Lake City.
Measuring outcomes in general and safety events in particular is a complex proposition. This is particularly true when using patient safety indicators (PSIs) and hospital-acquired conditions as outcome metrics to compare performance across organizations.
In addition to the usual challenging nuances such as severity of illness adjustment, these indicators rely on accurate documentation and coding and as the Agency for Healthcare Research and Quality states: PSIs identify "potential in-hospital complications and adverse events following surgeries, procedures, and childbirth."
This is well meaning when an analytic team uses these metrics as part of a comprehensive quality and patient safety program to identify potential internal improvement opportunities. However, there are real limitations when using these metrics as outcomes that are tied to public reporting initiatives, payment incentives, and rankings.
The study in JAMA by Dr. Bilimoria and his colleagues highlights another limitation of some of these metrics – that of surveillance bias. Using PSI-12 (postoperative venous thromboembolism), risk-adjusted VTE rates were shown to correlate positively with intensity of imaging use (surveillance) and inversely with other measures of quality such as structure or process.
Thus, those with the highest VTE rates did everything right, but also looked for events more often.
This finding complicates the use of PSI-12 as an indicator to compare outcomes across health care systems. However, when used as an internal driver in the context of other local metrics of quality and safety, the original intent of PSI-12 as an indicator of potential hospital complications does not change.
This highlights the importance of health care systems in understanding the strengths and limitations of quality and safety metrics and in developing the analytic capabilities to turn data points into real opportunities to deliver better care, rather than going down the proverbial rabbit hole. Yet, regulatory agencies should also recognize that using imperfect metrics as a part of payment-reform initiatives needs to be done with extreme caution or there will be unintended consequences that do not lead to our collective goal of exceptional value in health care for our patients.
Dr. Robert Pendleton is chief medical quality officer for University of Utah Health Care, Salt Lake City.
Postoperative venous thromboembolism rates may not be an effective way of measuring hospital quality, according to Dr. Karl Y. Bilimoria and his colleagues.
The investigators calculated patient-level rates of venous thromboembolism as well as rates of imaging for VTE using data from the American Hospital Association and Medicare Compare from 2009 to 2010 from nearly 1 million patients discharged from 2,786 hospitals after a major surgery.
They sought to determine the association between hospital adherence to VTE reduction protocols (Surgical Care Improvement Project for VTE or SCIP-VTE-2) and risk-adjusted rates of VTE as measured by Patient Safety Indicator 12 (PSI-12) from the Agency for Healthcare Research and Quality. They also looked at how overall hospital quality scores correlated with VTE prophylaxis and risk-adjusted VTE scores.
They presented their findigns at the annual clinical congress of the American College of Surgeons and in JAMA [doi:10.1001/jama.2013.280048]).
Hospitals that adhered consistently to VTE reduction protocols paradoxically had higher PSI-12 scores, although not significantly so (P = .03). Hospitals with higher overall quality scores also adhered to VTE reduction protocols at a higher rate (93.3% in the lowest quartile vs. 95.5% in the highest) and had significantly higher risk-adjusted VTE event scores (P less than .001).
"Most important, hospital VTE rates were associated with the intensity of detecting VTE with imaging studies," the investigators said. Mean VTE diagnostic imaging rates ranged from 32/1,000 in the lowest quartile to 167/1,000 in the highest. Hospitals with the lowest imaging rates diagnosed 5.0 VTEs per 1,000 discharges, compared with hospitals with the highest imaging rates diagnosing 13.5 VTEs per 1,000 discharges.
In effect, PSI-12 scores the use of VTE imaging by hospitals instead of the quality of care provided, the investigators said. Further, surveillance bias impedes quality performance improvements; thus, decision making becomes more difficult for "patients seeking to identify a high-quality hospital."
In an accompanying editorial, Dr. Edwin H. Livingston, deputy editor of JAMA, noted that hypervigilance of VTEs might further worsen care in that "the very high compliance rate with VTE prophylaxis might result from many patients receiving treatments from which they are not likely to benefit. This is because current process measures were based on older guidelines that overestimated the benefits of VTE prophylaxis" (JAMA [doi:10.1001/jama.2013.280049]).
For that reason, Dr. Livingston recommended that public reporting of VTEs be "reconsidered or curtailed because few hospitals have sufficient numbers of patients to show statistically significant effects of prophylactic measures on VTE rates."
Postoperative venous thromboembolism rates may not be an effective way of measuring hospital quality, according to Dr. Karl Y. Bilimoria and his colleagues.
The investigators calculated patient-level rates of venous thromboembolism as well as rates of imaging for VTE using data from the American Hospital Association and Medicare Compare from 2009 to 2010 from nearly 1 million patients discharged from 2,786 hospitals after a major surgery.
They sought to determine the association between hospital adherence to VTE reduction protocols (Surgical Care Improvement Project for VTE or SCIP-VTE-2) and risk-adjusted rates of VTE as measured by Patient Safety Indicator 12 (PSI-12) from the Agency for Healthcare Research and Quality. They also looked at how overall hospital quality scores correlated with VTE prophylaxis and risk-adjusted VTE scores.
They presented their findigns at the annual clinical congress of the American College of Surgeons and in JAMA [doi:10.1001/jama.2013.280048]).
Hospitals that adhered consistently to VTE reduction protocols paradoxically had higher PSI-12 scores, although not significantly so (P = .03). Hospitals with higher overall quality scores also adhered to VTE reduction protocols at a higher rate (93.3% in the lowest quartile vs. 95.5% in the highest) and had significantly higher risk-adjusted VTE event scores (P less than .001).
"Most important, hospital VTE rates were associated with the intensity of detecting VTE with imaging studies," the investigators said. Mean VTE diagnostic imaging rates ranged from 32/1,000 in the lowest quartile to 167/1,000 in the highest. Hospitals with the lowest imaging rates diagnosed 5.0 VTEs per 1,000 discharges, compared with hospitals with the highest imaging rates diagnosing 13.5 VTEs per 1,000 discharges.
In effect, PSI-12 scores the use of VTE imaging by hospitals instead of the quality of care provided, the investigators said. Further, surveillance bias impedes quality performance improvements; thus, decision making becomes more difficult for "patients seeking to identify a high-quality hospital."
In an accompanying editorial, Dr. Edwin H. Livingston, deputy editor of JAMA, noted that hypervigilance of VTEs might further worsen care in that "the very high compliance rate with VTE prophylaxis might result from many patients receiving treatments from which they are not likely to benefit. This is because current process measures were based on older guidelines that overestimated the benefits of VTE prophylaxis" (JAMA [doi:10.1001/jama.2013.280049]).
For that reason, Dr. Livingston recommended that public reporting of VTEs be "reconsidered or curtailed because few hospitals have sufficient numbers of patients to show statistically significant effects of prophylactic measures on VTE rates."
Major finding: Hospitals that adhered consistently to VTE reduction protocols had higher rates of VTE, although not significantly so (P = .03).
Data source: Study of hospital risk-adjusted VTE prophylaxis adherence rates to postoperative VTE event rates in 2,786 hospitals.
Disclosures: The study was funded by the AHRQ and Northwestern University. Dr. Bilimoria has received honoraria from hospitals, professional societies, and continuing medical education companies for presentation on quality improvement.
