Can public reporting improve pediatric heart surgery?

 

Public reporting of cardiac surgery outcomes has been a disruptive force in cardiology, and especially daunting in pediatric cardiac surgery because of low case volumes and rare mortality. To ensure that public reporting achieves its original goals – providing transparency to the patient care process, holding providers accountable, informing decision making for health care consumers, reducing costs, encouraging more efficient use of health system resources, and improving patient care and outcomes – further study that includes use of appropriate risk adjustment is needed, according to commentaries in the April issue of the Journal of Thoracic and Cardiovascular Surgery.

The journal asked two groups to provide perspective on a study Adam D. DeVore, MD, of Duke University in Durham, N.C., and his coauthors published last year (J Am Coll Cardiol. 2016 Mar 1;67:963-72). The study analyzed Medicare claims data from 2006 to 2012 for 37,829 hospitalizations for heart attack, 100,189 for heart failure (HF), and 79,076 for pneumonia. Dr. DeVore and his colleagues found readmission rates for the three conditions did not significantly improve after public reporting protocols were implemented in 2009. However, the study did show a significant decrease in ED visits and observation stays for those with HF: from 2.3% to –0.8% for the former (P = .007); and from 15% to 4% for the latter (P = .04).

In their perspective, J. William Gaynor, MD, of the Children’s Hospital of Philadelphia and his coauthors cited four areas that require further study to validate the effectiveness of public reporting:

• The metrics must be accurate, reliably discern hospital quality, and account for high-risk cases without penalizing hospitals. “In pediatric cardiac surgery, this can be particularly challenging, owing to the very wide heterogeneity of disease and variability in case mix and volumes across centers,” Dr. Gaynor and his coauthors wrote. While methodology for case mix and patient characteristics have improved in recent years, further improvement is needed.

• Metrics must be clearly reported and easy for stakeholders to interpret. “This is critical if the data are to be used to steer patients toward higher-performing centers and/or to provide incentives for hospitals with lower performance to make improvements,” the researchers said.

• Regional reporting or a methodology that indicates where a hospital ranks within larger categories deserve further investigation as tools to help families choose a high-performing center, “ideally based on geography and on the particular type and complexity of disease,” Dr. Gaynor and his coauthors stated (J Thorac Cardiovasc Surg. 2017 Apr;153:904-7).

• Indirect standardization, a statistical methodology used to calculate risk-adjusted performance, could help consumers to interpret hospital performance more easily. This methodology might help classify a hospital with a low-complexity population as a high performer. “Developing better methods to convey this information to consumers is vital,” according to the researchers.

The perspective acknowledged several reports of an unintended consequence of public reporting: surgeons and centers avoiding higher-risk cases to skew their performance scores higher, thus restricting access to care. However, in a separate perspective on Dr. DeVore’s study, James S. Tweddell, MD, of Cincinnati Children’s Hospital Medical Center, and his coauthors, questioned the quality of the evidence on which Dr. Gaynor and his colleagues based their conclusion of risk aversion and limited access to care: a newspaper report from the United Kingdom.

Dr. Tweddell and his coauthors cited a New York state study (Ann Thorac Surg. 1999 Oct;68:1195-1200) that suggested “risk-averse behavior may not be all bad,” and “if not for public reporting, these less-qualified individuals would have been willing to take on high-risk cases” (J Thorac Cardiovasc Surg. 2017 Apr;153:908-11).

Dr. Tweddell and his coauthors noted, “The predominance of data suggest an overall beneficial impact of public reporting.” They cited a trial that showed a decrease in heart attack–related deaths after public reporting had been implemented (JAMA. 2009 Dec 2;302:2330-7); a 2012 Agency for Healthcare Research and Quality systemic review (Evidence Report No. 208) that showed that research on harm is limited, and most studies do not confirm potential harm; and a meta-analysis that found a 15% reduction in adverse events associated with public reporting (BMC Health Serv Res. 2016;16:296).

“Appropriate risk adjustment is critical to achieve effective and fair transparency, but there is little objective data of harm associated with public reporting,” Dr. Tweddell and his coauthors concluded. While examination of public reporting must continue, they said, “these efforts are likely to result in minor course changes and the effort to inform and educate our patients and their families must continue.”

Ds. Gaynor, Dr. Tweddell, and their coauthors reported having no financial disclosures.
 

 

Public reporting of cardiac surgery outcomes has been a disruptive force in cardiology, and especially daunting in pediatric cardiac surgery because of low case volumes and rare mortality. To ensure that public reporting achieves its original goals – providing transparency to the patient care process, holding providers accountable, informing decision making for health care consumers, reducing costs, encouraging more efficient use of health system resources, and improving patient care and outcomes – further study that includes use of appropriate risk adjustment is needed, according to commentaries in the April issue of the Journal of Thoracic and Cardiovascular Surgery.

The journal asked two groups to provide perspective on a study Adam D. DeVore, MD, of Duke University in Durham, N.C., and his coauthors published last year (J Am Coll Cardiol. 2016 Mar 1;67:963-72). The study analyzed Medicare claims data from 2006 to 2012 for 37,829 hospitalizations for heart attack, 100,189 for heart failure (HF), and 79,076 for pneumonia. Dr. DeVore and his colleagues found readmission rates for the three conditions did not significantly improve after public reporting protocols were implemented in 2009. However, the study did show a significant decrease in ED visits and observation stays for those with HF: from 2.3% to –0.8% for the former (P = .007); and from 15% to 4% for the latter (P = .04).

In their perspective, J. William Gaynor, MD, of the Children’s Hospital of Philadelphia and his coauthors cited four areas that require further study to validate the effectiveness of public reporting:

• The metrics must be accurate, reliably discern hospital quality, and account for high-risk cases without penalizing hospitals. “In pediatric cardiac surgery, this can be particularly challenging, owing to the very wide heterogeneity of disease and variability in case mix and volumes across centers,” Dr. Gaynor and his coauthors wrote. While methodology for case mix and patient characteristics have improved in recent years, further improvement is needed.

• Metrics must be clearly reported and easy for stakeholders to interpret. “This is critical if the data are to be used to steer patients toward higher-performing centers and/or to provide incentives for hospitals with lower performance to make improvements,” the researchers said.

• Regional reporting or a methodology that indicates where a hospital ranks within larger categories deserve further investigation as tools to help families choose a high-performing center, “ideally based on geography and on the particular type and complexity of disease,” Dr. Gaynor and his coauthors stated (J Thorac Cardiovasc Surg. 2017 Apr;153:904-7).

• Indirect standardization, a statistical methodology used to calculate risk-adjusted performance, could help consumers to interpret hospital performance more easily. This methodology might help classify a hospital with a low-complexity population as a high performer. “Developing better methods to convey this information to consumers is vital,” according to the researchers.

The perspective acknowledged several reports of an unintended consequence of public reporting: surgeons and centers avoiding higher-risk cases to skew their performance scores higher, thus restricting access to care. However, in a separate perspective on Dr. DeVore’s study, James S. Tweddell, MD, of Cincinnati Children’s Hospital Medical Center, and his coauthors, questioned the quality of the evidence on which Dr. Gaynor and his colleagues based their conclusion of risk aversion and limited access to care: a newspaper report from the United Kingdom.

Dr. Tweddell and his coauthors cited a New York state study (Ann Thorac Surg. 1999 Oct;68:1195-1200) that suggested “risk-averse behavior may not be all bad,” and “if not for public reporting, these less-qualified individuals would have been willing to take on high-risk cases” (J Thorac Cardiovasc Surg. 2017 Apr;153:908-11).

Dr. Tweddell and his coauthors noted, “The predominance of data suggest an overall beneficial impact of public reporting.” They cited a trial that showed a decrease in heart attack–related deaths after public reporting had been implemented (JAMA. 2009 Dec 2;302:2330-7); a 2012 Agency for Healthcare Research and Quality systemic review (Evidence Report No. 208) that showed that research on harm is limited, and most studies do not confirm potential harm; and a meta-analysis that found a 15% reduction in adverse events associated with public reporting (BMC Health Serv Res. 2016;16:296).

“Appropriate risk adjustment is critical to achieve effective and fair transparency, but there is little objective data of harm associated with public reporting,” Dr. Tweddell and his coauthors concluded. While examination of public reporting must continue, they said, “these efforts are likely to result in minor course changes and the effort to inform and educate our patients and their families must continue.”

Ds. Gaynor, Dr. Tweddell, and their coauthors reported having no financial disclosures.
 

 

Public reporting of cardiac surgery outcomes has been a disruptive force in cardiology, and especially daunting in pediatric cardiac surgery because of low case volumes and rare mortality. To ensure that public reporting achieves its original goals – providing transparency to the patient care process, holding providers accountable, informing decision making for health care consumers, reducing costs, encouraging more efficient use of health system resources, and improving patient care and outcomes – further study that includes use of appropriate risk adjustment is needed, according to commentaries in the April issue of the Journal of Thoracic and Cardiovascular Surgery.

The journal asked two groups to provide perspective on a study Adam D. DeVore, MD, of Duke University in Durham, N.C., and his coauthors published last year (J Am Coll Cardiol. 2016 Mar 1;67:963-72). The study analyzed Medicare claims data from 2006 to 2012 for 37,829 hospitalizations for heart attack, 100,189 for heart failure (HF), and 79,076 for pneumonia. Dr. DeVore and his colleagues found readmission rates for the three conditions did not significantly improve after public reporting protocols were implemented in 2009. However, the study did show a significant decrease in ED visits and observation stays for those with HF: from 2.3% to –0.8% for the former (P = .007); and from 15% to 4% for the latter (P = .04).

In their perspective, J. William Gaynor, MD, of the Children’s Hospital of Philadelphia and his coauthors cited four areas that require further study to validate the effectiveness of public reporting:

• The metrics must be accurate, reliably discern hospital quality, and account for high-risk cases without penalizing hospitals. “In pediatric cardiac surgery, this can be particularly challenging, owing to the very wide heterogeneity of disease and variability in case mix and volumes across centers,” Dr. Gaynor and his coauthors wrote. While methodology for case mix and patient characteristics have improved in recent years, further improvement is needed.

• Metrics must be clearly reported and easy for stakeholders to interpret. “This is critical if the data are to be used to steer patients toward higher-performing centers and/or to provide incentives for hospitals with lower performance to make improvements,” the researchers said.

• Regional reporting or a methodology that indicates where a hospital ranks within larger categories deserve further investigation as tools to help families choose a high-performing center, “ideally based on geography and on the particular type and complexity of disease,” Dr. Gaynor and his coauthors stated (J Thorac Cardiovasc Surg. 2017 Apr;153:904-7).

• Indirect standardization, a statistical methodology used to calculate risk-adjusted performance, could help consumers to interpret hospital performance more easily. This methodology might help classify a hospital with a low-complexity population as a high performer. “Developing better methods to convey this information to consumers is vital,” according to the researchers.

The perspective acknowledged several reports of an unintended consequence of public reporting: surgeons and centers avoiding higher-risk cases to skew their performance scores higher, thus restricting access to care. However, in a separate perspective on Dr. DeVore’s study, James S. Tweddell, MD, of Cincinnati Children’s Hospital Medical Center, and his coauthors, questioned the quality of the evidence on which Dr. Gaynor and his colleagues based their conclusion of risk aversion and limited access to care: a newspaper report from the United Kingdom.

Dr. Tweddell and his coauthors cited a New York state study (Ann Thorac Surg. 1999 Oct;68:1195-1200) that suggested “risk-averse behavior may not be all bad,” and “if not for public reporting, these less-qualified individuals would have been willing to take on high-risk cases” (J Thorac Cardiovasc Surg. 2017 Apr;153:908-11).

Dr. Tweddell and his coauthors noted, “The predominance of data suggest an overall beneficial impact of public reporting.” They cited a trial that showed a decrease in heart attack–related deaths after public reporting had been implemented (JAMA. 2009 Dec 2;302:2330-7); a 2012 Agency for Healthcare Research and Quality systemic review (Evidence Report No. 208) that showed that research on harm is limited, and most studies do not confirm potential harm; and a meta-analysis that found a 15% reduction in adverse events associated with public reporting (BMC Health Serv Res. 2016;16:296).

“Appropriate risk adjustment is critical to achieve effective and fair transparency, but there is little objective data of harm associated with public reporting,” Dr. Tweddell and his coauthors concluded. While examination of public reporting must continue, they said, “these efforts are likely to result in minor course changes and the effort to inform and educate our patients and their families must continue.”

Ds. Gaynor, Dr. Tweddell, and their coauthors reported having no financial disclosures.