P4P Programs May Not Improve Quality of Care

WASHINGTON — The few studies that have examined the effectiveness of incentivized pay-for-performance programs have found a mix of moderate to no improvement in quality measures, which, in some instances, have led to unintended consequences, according to Dr. Daniel B. Mark.

More than 100 reward or incentive programs have been started in the private U.S. health care sector under the control of employer groups or managed care organizations, said Dr. Mark, but congressionally authorized programs by the Centers for Medicare and Medicaid Services have received the most attention.

It is important to examine the evidence base that pay-for-performance programs actually improve quality because “people are making this association,” said Dr. Mark, director of the Outcomes Research and Assessment Group at the Duke (University) Clinical Research Institute, Durham, N.C.

During the last 20 years, incentivized performance programs have shown that “what you measure generally improves and what gets measured is generally what's easiest to measure. But the ease of measurement does not necessarily define the importance of the measurement.”

Little, if anything, is known about whether these initiatives are cost effective for the health care system at large, Dr. Mark noted at the annual meeting of the Heart Failure Society of America.

A systematic overview of 17 studies published during 1980–2005 on pay-for-performance programs found that 1 of 2 studies on system-level incentives had a positive result in which all performance measures improved. In nine studies of incentive programs aimed at the provider group level, seven had partially positive or fully positive results but had “quite small” effect sizes. Positive or partially positive results were seen in five of six programs at the physician level (Ann. Int. Med. 2006;145:265–72).

Nine of the studies were randomized and controlled, but eight of these had a sample size of fewer than 100 physicians or groups; the other study had fewer than 200 groups.

Programs in four studies appeared to have created unintended consequences, including “gaming the baseline level of illness,” avoiding sicker patients, and an improvement in documentation without any actual change in care. The studies did not include any information on the optimal duration of these programs or whether their effect persisted after the program was terminated. Only one study had a preliminary examination of cost-effectiveness.

Another study compared patients with acute non-ST-elevation myocardial infarction in 57 hospitals that participated in CMS Hospital Quality Incentive Demonstration and 113 control hospitals that did not participate to determine if a pay-for-performance strategy produced better quality of care. There was “very little evidence that there was any intervention effect,” Dr. Mark said. Measures that were not incentivized by CMS also did not appear to change (JAMA 2007;297:2373–80).

In the United Kingdom, family practice physicians participated in a pay-for-performance program in 2004 that focused on 146 quality indicators for 10 chronic diseases. The National Health Service substantially increased its deficit that year because the funds allocated for the project were used up by greater than predicted success in achieving the quality indicators. This led to an average increase in the physicians' pay of about $40,000 that year (N. Engl. J. Med. 2006;355:375–84).

Other investigators noted that in the 1998–2003 period prior to the NHS project, all of the quality indicators had already been improving, “so it's not clear how much the program's achievements can actually be attributed to the program itself,” he said (N. Engl. J. Med. 2007;357:181–90).

Another study showed that public reporting of quality measures alone could improve a set of quality indicators by the same magnitude as a pay-for performance program that included public reporting (N. Engl. J. Med. 2007;356:486–96).

Little, if anything, is known about whether these initiatives are cost effective for the health care system at large. DR. MARK

WASHINGTON — The few studies that have examined the effectiveness of incentivized pay-for-performance programs have found a mix of moderate to no improvement in quality measures, which, in some instances, have led to unintended consequences, according to Dr. Daniel B. Mark.

More than 100 reward or incentive programs have been started in the private U.S. health care sector under the control of employer groups or managed care organizations, said Dr. Mark, but congressionally authorized programs by the Centers for Medicare and Medicaid Services have received the most attention.

It is important to examine the evidence base that pay-for-performance programs actually improve quality because “people are making this association,” said Dr. Mark, director of the Outcomes Research and Assessment Group at the Duke (University) Clinical Research Institute, Durham, N.C.

During the last 20 years, incentivized performance programs have shown that “what you measure generally improves and what gets measured is generally what's easiest to measure. But the ease of measurement does not necessarily define the importance of the measurement.”

Little, if anything, is known about whether these initiatives are cost effective for the health care system at large, Dr. Mark noted at the annual meeting of the Heart Failure Society of America.

A systematic overview of 17 studies published during 1980–2005 on pay-for-performance programs found that 1 of 2 studies on system-level incentives had a positive result in which all performance measures improved. In nine studies of incentive programs aimed at the provider group level, seven had partially positive or fully positive results but had “quite small” effect sizes. Positive or partially positive results were seen in five of six programs at the physician level (Ann. Int. Med. 2006;145:265–72).

Nine of the studies were randomized and controlled, but eight of these had a sample size of fewer than 100 physicians or groups; the other study had fewer than 200 groups.

Programs in four studies appeared to have created unintended consequences, including “gaming the baseline level of illness,” avoiding sicker patients, and an improvement in documentation without any actual change in care. The studies did not include any information on the optimal duration of these programs or whether their effect persisted after the program was terminated. Only one study had a preliminary examination of cost-effectiveness.

Another study compared patients with acute non-ST-elevation myocardial infarction in 57 hospitals that participated in CMS Hospital Quality Incentive Demonstration and 113 control hospitals that did not participate to determine if a pay-for-performance strategy produced better quality of care. There was “very little evidence that there was any intervention effect,” Dr. Mark said. Measures that were not incentivized by CMS also did not appear to change (JAMA 2007;297:2373–80).

In the United Kingdom, family practice physicians participated in a pay-for-performance program in 2004 that focused on 146 quality indicators for 10 chronic diseases. The National Health Service substantially increased its deficit that year because the funds allocated for the project were used up by greater than predicted success in achieving the quality indicators. This led to an average increase in the physicians' pay of about $40,000 that year (N. Engl. J. Med. 2006;355:375–84).

Other investigators noted that in the 1998–2003 period prior to the NHS project, all of the quality indicators had already been improving, “so it's not clear how much the program's achievements can actually be attributed to the program itself,” he said (N. Engl. J. Med. 2007;357:181–90).

Another study showed that public reporting of quality measures alone could improve a set of quality indicators by the same magnitude as a pay-for performance program that included public reporting (N. Engl. J. Med. 2007;356:486–96).

Little, if anything, is known about whether these initiatives are cost effective for the health care system at large. DR. MARK

WASHINGTON — The few studies that have examined the effectiveness of incentivized pay-for-performance programs have found a mix of moderate to no improvement in quality measures, which, in some instances, have led to unintended consequences, according to Dr. Daniel B. Mark.

More than 100 reward or incentive programs have been started in the private U.S. health care sector under the control of employer groups or managed care organizations, said Dr. Mark, but congressionally authorized programs by the Centers for Medicare and Medicaid Services have received the most attention.

It is important to examine the evidence base that pay-for-performance programs actually improve quality because “people are making this association,” said Dr. Mark, director of the Outcomes Research and Assessment Group at the Duke (University) Clinical Research Institute, Durham, N.C.

During the last 20 years, incentivized performance programs have shown that “what you measure generally improves and what gets measured is generally what's easiest to measure. But the ease of measurement does not necessarily define the importance of the measurement.”

Little, if anything, is known about whether these initiatives are cost effective for the health care system at large, Dr. Mark noted at the annual meeting of the Heart Failure Society of America.

A systematic overview of 17 studies published during 1980–2005 on pay-for-performance programs found that 1 of 2 studies on system-level incentives had a positive result in which all performance measures improved. In nine studies of incentive programs aimed at the provider group level, seven had partially positive or fully positive results but had “quite small” effect sizes. Positive or partially positive results were seen in five of six programs at the physician level (Ann. Int. Med. 2006;145:265–72).

Nine of the studies were randomized and controlled, but eight of these had a sample size of fewer than 100 physicians or groups; the other study had fewer than 200 groups.

Programs in four studies appeared to have created unintended consequences, including “gaming the baseline level of illness,” avoiding sicker patients, and an improvement in documentation without any actual change in care. The studies did not include any information on the optimal duration of these programs or whether their effect persisted after the program was terminated. Only one study had a preliminary examination of cost-effectiveness.

Another study compared patients with acute non-ST-elevation myocardial infarction in 57 hospitals that participated in CMS Hospital Quality Incentive Demonstration and 113 control hospitals that did not participate to determine if a pay-for-performance strategy produced better quality of care. There was “very little evidence that there was any intervention effect,” Dr. Mark said. Measures that were not incentivized by CMS also did not appear to change (JAMA 2007;297:2373–80).

In the United Kingdom, family practice physicians participated in a pay-for-performance program in 2004 that focused on 146 quality indicators for 10 chronic diseases. The National Health Service substantially increased its deficit that year because the funds allocated for the project were used up by greater than predicted success in achieving the quality indicators. This led to an average increase in the physicians' pay of about $40,000 that year (N. Engl. J. Med. 2006;355:375–84).

Other investigators noted that in the 1998–2003 period prior to the NHS project, all of the quality indicators had already been improving, “so it's not clear how much the program's achievements can actually be attributed to the program itself,” he said (N. Engl. J. Med. 2007;357:181–90).

Another study showed that public reporting of quality measures alone could improve a set of quality indicators by the same magnitude as a pay-for performance program that included public reporting (N. Engl. J. Med. 2007;356:486–96).

Little, if anything, is known about whether these initiatives are cost effective for the health care system at large. DR. MARK