Pay for Performance Programs May Not Do Much for Quality

WASHINGTON – The few studies that have examined the effectiveness of incentivized pay-for-performance programs have found a mix of moderate to no improvement in quality measures, which, in some instances, have led to unintended consequences, Dr. Daniel B. Mark said at the annual meeting of the Heart Failure Society of America.

There are more than 100 reward or incentive programs that have started in the private U.S. health care sector under the control of employer groups or managed care organizations, but congressionally authorized programs by the Centers for Medicare and Medicaid Services have received the most attention, said Dr. Mark, director of the Outcomes Research and Assessment Group at the Duke (University) Clinical Research Institute, Durham, N.C.

During the last 20 years, incentivized performance programs have shown that “what you measure generally improves and what gets measured is generally what's easiest to measure. But the ease of measurement does not necessarily define the importance of the measurement,” he said.

A systematic overview of 17 studies published during 1980–2005 on pay-for-performance programs found that 1 of 2 studies on system-level incentives had a positive result in which all performance measures improved. In nine studies of incentive programs aimed at the provider group level, seven had partially positive or fully positive results but had “quite small” effect sizes. Positive or partially-positive results were seen in five of six programs at the physician level (Ann. Int. Med. 2006;145:265–72).

Nine of the studies were randomized and controlled, but eight of these had a sample size of fewer than 100 physicians or groups; the other study had fewer than 200 groups. “If these had been clinical trials, they would have all been considered extremely underpowered and preliminary,” Dr. Mark said.

Programs in four studies appeared to have created unintended consequences, including “gaming the baseline level of illness,” avoiding sicker patients, and an improvement in documentation in immunization studies without any actual change in the number of immunizations given or effect on care.

Another study compared patients with acute non-ST-elevation myocardial infarction in 57 hospitals that participated in CMs' Hospital Quality Incentive Demonstration and 113 control hospitals that did not participate in the program to determine if a pay-for-performance strategy produced better quality of care (JAMA 2007;297:2373–80). There was “very little evidence that there was any intervention effect,” according to Dr. Mark.

In the United Kingdom, family practice physicians participated in a pay-for-performance program in 2004 that focused on 146 quality indicators for 10 chronic diseases as well as measures related to the organization of care and the patient's experience. The National Health Service substantially increased its deficit that year because the greater than predicted success in achieving the quality indicators (83% achieved vs. an expected 75%) led to an average increase in the physicians' pay of about $40,000 (N. Engl. J. Med. 2006;355:375–84).

Other investigators noted that in the 1998–2003 period prior to the NHS project the quality indicators had already been improving, “so it's not clear how much the program's achievements can actually be attributed to the program itself,” he said (N. Engl. J. Med. 2007;357:181–90).

WASHINGTON – The few studies that have examined the effectiveness of incentivized pay-for-performance programs have found a mix of moderate to no improvement in quality measures, which, in some instances, have led to unintended consequences, Dr. Daniel B. Mark said at the annual meeting of the Heart Failure Society of America.

There are more than 100 reward or incentive programs that have started in the private U.S. health care sector under the control of employer groups or managed care organizations, but congressionally authorized programs by the Centers for Medicare and Medicaid Services have received the most attention, said Dr. Mark, director of the Outcomes Research and Assessment Group at the Duke (University) Clinical Research Institute, Durham, N.C.

During the last 20 years, incentivized performance programs have shown that “what you measure generally improves and what gets measured is generally what's easiest to measure. But the ease of measurement does not necessarily define the importance of the measurement,” he said.

A systematic overview of 17 studies published during 1980–2005 on pay-for-performance programs found that 1 of 2 studies on system-level incentives had a positive result in which all performance measures improved. In nine studies of incentive programs aimed at the provider group level, seven had partially positive or fully positive results but had “quite small” effect sizes. Positive or partially-positive results were seen in five of six programs at the physician level (Ann. Int. Med. 2006;145:265–72).

Nine of the studies were randomized and controlled, but eight of these had a sample size of fewer than 100 physicians or groups; the other study had fewer than 200 groups. “If these had been clinical trials, they would have all been considered extremely underpowered and preliminary,” Dr. Mark said.

Programs in four studies appeared to have created unintended consequences, including “gaming the baseline level of illness,” avoiding sicker patients, and an improvement in documentation in immunization studies without any actual change in the number of immunizations given or effect on care.

Another study compared patients with acute non-ST-elevation myocardial infarction in 57 hospitals that participated in CMs' Hospital Quality Incentive Demonstration and 113 control hospitals that did not participate in the program to determine if a pay-for-performance strategy produced better quality of care (JAMA 2007;297:2373–80). There was “very little evidence that there was any intervention effect,” according to Dr. Mark.

In the United Kingdom, family practice physicians participated in a pay-for-performance program in 2004 that focused on 146 quality indicators for 10 chronic diseases as well as measures related to the organization of care and the patient's experience. The National Health Service substantially increased its deficit that year because the greater than predicted success in achieving the quality indicators (83% achieved vs. an expected 75%) led to an average increase in the physicians' pay of about $40,000 (N. Engl. J. Med. 2006;355:375–84).

Other investigators noted that in the 1998–2003 period prior to the NHS project the quality indicators had already been improving, “so it's not clear how much the program's achievements can actually be attributed to the program itself,” he said (N. Engl. J. Med. 2007;357:181–90).

WASHINGTON – The few studies that have examined the effectiveness of incentivized pay-for-performance programs have found a mix of moderate to no improvement in quality measures, which, in some instances, have led to unintended consequences, Dr. Daniel B. Mark said at the annual meeting of the Heart Failure Society of America.

There are more than 100 reward or incentive programs that have started in the private U.S. health care sector under the control of employer groups or managed care organizations, but congressionally authorized programs by the Centers for Medicare and Medicaid Services have received the most attention, said Dr. Mark, director of the Outcomes Research and Assessment Group at the Duke (University) Clinical Research Institute, Durham, N.C.

During the last 20 years, incentivized performance programs have shown that “what you measure generally improves and what gets measured is generally what's easiest to measure. But the ease of measurement does not necessarily define the importance of the measurement,” he said.

A systematic overview of 17 studies published during 1980–2005 on pay-for-performance programs found that 1 of 2 studies on system-level incentives had a positive result in which all performance measures improved. In nine studies of incentive programs aimed at the provider group level, seven had partially positive or fully positive results but had “quite small” effect sizes. Positive or partially-positive results were seen in five of six programs at the physician level (Ann. Int. Med. 2006;145:265–72).

Nine of the studies were randomized and controlled, but eight of these had a sample size of fewer than 100 physicians or groups; the other study had fewer than 200 groups. “If these had been clinical trials, they would have all been considered extremely underpowered and preliminary,” Dr. Mark said.

Programs in four studies appeared to have created unintended consequences, including “gaming the baseline level of illness,” avoiding sicker patients, and an improvement in documentation in immunization studies without any actual change in the number of immunizations given or effect on care.

Another study compared patients with acute non-ST-elevation myocardial infarction in 57 hospitals that participated in CMs' Hospital Quality Incentive Demonstration and 113 control hospitals that did not participate in the program to determine if a pay-for-performance strategy produced better quality of care (JAMA 2007;297:2373–80). There was “very little evidence that there was any intervention effect,” according to Dr. Mark.

In the United Kingdom, family practice physicians participated in a pay-for-performance program in 2004 that focused on 146 quality indicators for 10 chronic diseases as well as measures related to the organization of care and the patient's experience. The National Health Service substantially increased its deficit that year because the greater than predicted success in achieving the quality indicators (83% achieved vs. an expected 75%) led to an average increase in the physicians' pay of about $40,000 (N. Engl. J. Med. 2006;355:375–84).

Other investigators noted that in the 1998–2003 period prior to the NHS project the quality indicators had already been improving, “so it's not clear how much the program's achievements can actually be attributed to the program itself,” he said (N. Engl. J. Med. 2007;357:181–90).