Health Policy

Health care professionals working in critical care settings have been overburdened because of the plethora of COVID-19 cases, which has led to symptoms of burnout in both physicians and nurses, findings from a new study show.

XiXinXing/ThinkStock

“Overburdening ICU professionals during an extended period of time leads to burnout,” said lead study author Niek Kok, MSc, of IQ healthcare, Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, the Netherlands. “All ICU professionals are at the risk of this, and in our study, the incidence of physicians experiencing burnout was significantly higher than that of nurses in June 2020.”

This burnout can be explained by conditions caused by the pandemic, he noted, such as the scarcity of staff and resources and having to work with colleagues who were not qualified to work in critical care but who were there out of necessity.

Mr. Kok presented the findings of the study at the Critical Care Congress sponsored by the Society of Critical Care Medicine.
 

Burnout highest among critical care physicians

The ICU can be a stressful environment for both patients and health care personnel, and burnout is not uncommon among ICU clinicians. However, COVID-19 has amplified the degree of burnout being experienced by clinicians working in this setting. Critical care physicians now top the list of physicians experiencing burnout, at 51%, up from 44% last year, according to the Medscape report ‘Death by 1000 Thousand Cuts’: Physician Burnout and Suicide Report 2021.

The Medscape Nurse Career Satisfaction Report 2020, while not restricted to those working in critical care, also reported higher rates of burnout, compared with the prepandemic period. The percentage of nurses reporting being “very burned out” prior to the pandemic was 4%. Six months into the pandemic, that percentage soared to 18%.

In this study, Mr. Kok and colleagues examined the prevalence and incidence of burnout symptoms and moral distress in health care professionals working in the ICU, both before and during the COVID-19 pandemic.

“When the COVID-19 pandemic surfaced in the Netherlands, the health care professionals in our hospitals were motivated to do everything they could to provide the best care possible,” said Mr. Kok. “Many of the ICU professionals immediately realized that they would have to work longer hours.”

However, the health care professionals that he spoke with did have mixed feelings. Some were afraid of being infected with the virus, while others said that “it was very interesting times for them and that gave them extra motivation to do the work.

“Some physicians [and] the WHO warned that COVID-19 is not going to weathered by a heroic sprint – it is an arduous marathon that is going to go hand in hand with burnout symptoms,” Mr. Kok added. “It will eat away at our qualified ICU staff.”
 

Before and after data on burnout

It was widely believed that the COVID-19 pandemic would increase burnout symptoms, as had been demonstrated in studies of previous pandemics. However, Mr. Kok emphasized that there are no before and after measurements that transcend cross-sectional designs.

“The claim [has been] that it increases burnout – but there are no assessments of how it progresses in ICU professionals through time,” he said. “So what we really need is a comparison [of] before and after the pandemic.”

It is quite difficult to obtain this type of information because disruptive events like the COVID-19 pandemic cannot be predicted, he said. Thus, it is challenging to get a baseline measurement. But Mr. Kok pointed out that the study has both “before and after” measurements.

“By coincidence really, we had baseline data to measure the impact of the COVID-19 pandemic and had information that was collected before the pandemic,” he said.

In January 2020, a study began looking at the effects of ethics meetings on moral distress in ICU professionals. Data had been collected on moral distress and burnout on ICU professionals in December 2019. The first COVID-19 cases appeared in the Netherlands in February 2020.

A follow-up study was then conducted in May and June 2020, several months into the pandemic.

The longitudinal open cohort study included all ICU personnel who were working in five units within a single university medical center, plus another adult ICU that was based in a separate teaching hospital.

A total of 352 health care professionals responded to a baseline survey in October through December 2019, and then 233 responded to a follow-up survey sent in May and June 2020. The authors measured burnout symptoms and moral distress with the Maslach Burnout Inventory and the Moral Distress Scale, respectively.
 

Findings

The overall prevalence of burnout symptoms was 23.0% prior to the pandemic, and that jumped to 36.1% at post-peak time. Higher rates of burnout were reported by nurses (38.0%) than physicians (28.6%).

However, the incidence rate of new burnout cases was higher among physicians, compared with nurses (26.7% vs 21.9%). Not surprisingly, a higher prevalence of burnout symptoms was observed in the post-peak period for all clinicians (odds ratio, 1.83; 95% confidence interval, 1.32-2.53), and was higher for nurses (odds ratio, 1.77; 95% confidence interval, 1.03-3.04), for those working overtime (OR, 2.11; 95% CI, 1.48-3.02), and for personnel who directly engaged in patient care (OR, 1.87; 95% CI, 1.35-2.60).

Physicians in general were much more likely to develop burnout symptoms related to the pandemic, compared with nurses (OR, 3.56; 95% CI, 1.06-12.21).

When looking at findings on moral distress, Kok pointed out that it often arises in situations when the health care professional knows the right thing to do but is prevented from doing so. “Morally distressful situations all rose from December to June,” said Mr. Kok. “Scarcity was the most distressing. The other was where colleagues were perceived to be less skilled, and this had to do with the recruitment of people from outside of the ICU to provide care.”

Moral distress from scarcity and unskilled colleagues were both significantly related to burnout, he noted.

In the final model, working in a COVID-19 unit, stress from scarcity of resources and people, stress from unskilled colleagues, and stress from unsafe conditions were all related to burnout. “The stress of physicians was significantly higher,” said Kok. “Even though nurses had higher baseline burnout, it became less pronounced in June 2020. This indicates that burnout was significantly higher in physicians.”

Thus, Mr. Kok and colleagues concluded that overburdening ICU professionals during an extended period of time leads to burnout, and all ICU workers are at risk.
 

Burnout rates higher in physicians

Weighing in on the study, Greg S. Martin, MD, FCCP, professor of medicine in the division of pulmonary, allergy, critical care and sleep medicine, Emory University, Atlanta, noted that the differences observed between physicians and nurses may have to do with the fact that “nurses have been smoldering all along and experiencing higher rates of burnout.

“They may have adapted better to the pandemic conditions, since they are more used to working overtime and short staffed, and spending far more time at the bedside,” he said. “Because of the volume of patients, physicians may be spending more hours doing patient care and are experiencing more burnout.”

Health care professionals working in critical care settings have been overburdened because of the plethora of COVID-19 cases, which has led to symptoms of burnout in both physicians and nurses, findings from a new study show.

XiXinXing/ThinkStock

“Overburdening ICU professionals during an extended period of time leads to burnout,” said lead study author Niek Kok, MSc, of IQ healthcare, Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, the Netherlands. “All ICU professionals are at the risk of this, and in our study, the incidence of physicians experiencing burnout was significantly higher than that of nurses in June 2020.”

This burnout can be explained by conditions caused by the pandemic, he noted, such as the scarcity of staff and resources and having to work with colleagues who were not qualified to work in critical care but who were there out of necessity.

Mr. Kok presented the findings of the study at the Critical Care Congress sponsored by the Society of Critical Care Medicine.
 

Burnout highest among critical care physicians

The ICU can be a stressful environment for both patients and health care personnel, and burnout is not uncommon among ICU clinicians. However, COVID-19 has amplified the degree of burnout being experienced by clinicians working in this setting. Critical care physicians now top the list of physicians experiencing burnout, at 51%, up from 44% last year, according to the Medscape report ‘Death by 1000 Thousand Cuts’: Physician Burnout and Suicide Report 2021.

The Medscape Nurse Career Satisfaction Report 2020, while not restricted to those working in critical care, also reported higher rates of burnout, compared with the prepandemic period. The percentage of nurses reporting being “very burned out” prior to the pandemic was 4%. Six months into the pandemic, that percentage soared to 18%.

In this study, Mr. Kok and colleagues examined the prevalence and incidence of burnout symptoms and moral distress in health care professionals working in the ICU, both before and during the COVID-19 pandemic.

“When the COVID-19 pandemic surfaced in the Netherlands, the health care professionals in our hospitals were motivated to do everything they could to provide the best care possible,” said Mr. Kok. “Many of the ICU professionals immediately realized that they would have to work longer hours.”

However, the health care professionals that he spoke with did have mixed feelings. Some were afraid of being infected with the virus, while others said that “it was very interesting times for them and that gave them extra motivation to do the work.

“Some physicians [and] the WHO warned that COVID-19 is not going to weathered by a heroic sprint – it is an arduous marathon that is going to go hand in hand with burnout symptoms,” Mr. Kok added. “It will eat away at our qualified ICU staff.”
 

Before and after data on burnout

It was widely believed that the COVID-19 pandemic would increase burnout symptoms, as had been demonstrated in studies of previous pandemics. However, Mr. Kok emphasized that there are no before and after measurements that transcend cross-sectional designs.

“The claim [has been] that it increases burnout – but there are no assessments of how it progresses in ICU professionals through time,” he said. “So what we really need is a comparison [of] before and after the pandemic.”

It is quite difficult to obtain this type of information because disruptive events like the COVID-19 pandemic cannot be predicted, he said. Thus, it is challenging to get a baseline measurement. But Mr. Kok pointed out that the study has both “before and after” measurements.

“By coincidence really, we had baseline data to measure the impact of the COVID-19 pandemic and had information that was collected before the pandemic,” he said.

In January 2020, a study began looking at the effects of ethics meetings on moral distress in ICU professionals. Data had been collected on moral distress and burnout on ICU professionals in December 2019. The first COVID-19 cases appeared in the Netherlands in February 2020.

A follow-up study was then conducted in May and June 2020, several months into the pandemic.

The longitudinal open cohort study included all ICU personnel who were working in five units within a single university medical center, plus another adult ICU that was based in a separate teaching hospital.

A total of 352 health care professionals responded to a baseline survey in October through December 2019, and then 233 responded to a follow-up survey sent in May and June 2020. The authors measured burnout symptoms and moral distress with the Maslach Burnout Inventory and the Moral Distress Scale, respectively.
 

Findings

The overall prevalence of burnout symptoms was 23.0% prior to the pandemic, and that jumped to 36.1% at post-peak time. Higher rates of burnout were reported by nurses (38.0%) than physicians (28.6%).

However, the incidence rate of new burnout cases was higher among physicians, compared with nurses (26.7% vs 21.9%). Not surprisingly, a higher prevalence of burnout symptoms was observed in the post-peak period for all clinicians (odds ratio, 1.83; 95% confidence interval, 1.32-2.53), and was higher for nurses (odds ratio, 1.77; 95% confidence interval, 1.03-3.04), for those working overtime (OR, 2.11; 95% CI, 1.48-3.02), and for personnel who directly engaged in patient care (OR, 1.87; 95% CI, 1.35-2.60).

Physicians in general were much more likely to develop burnout symptoms related to the pandemic, compared with nurses (OR, 3.56; 95% CI, 1.06-12.21).

When looking at findings on moral distress, Kok pointed out that it often arises in situations when the health care professional knows the right thing to do but is prevented from doing so. “Morally distressful situations all rose from December to June,” said Mr. Kok. “Scarcity was the most distressing. The other was where colleagues were perceived to be less skilled, and this had to do with the recruitment of people from outside of the ICU to provide care.”

Moral distress from scarcity and unskilled colleagues were both significantly related to burnout, he noted.

In the final model, working in a COVID-19 unit, stress from scarcity of resources and people, stress from unskilled colleagues, and stress from unsafe conditions were all related to burnout. “The stress of physicians was significantly higher,” said Kok. “Even though nurses had higher baseline burnout, it became less pronounced in June 2020. This indicates that burnout was significantly higher in physicians.”

Thus, Mr. Kok and colleagues concluded that overburdening ICU professionals during an extended period of time leads to burnout, and all ICU workers are at risk.
 

Burnout rates higher in physicians

Weighing in on the study, Greg S. Martin, MD, FCCP, professor of medicine in the division of pulmonary, allergy, critical care and sleep medicine, Emory University, Atlanta, noted that the differences observed between physicians and nurses may have to do with the fact that “nurses have been smoldering all along and experiencing higher rates of burnout.

“They may have adapted better to the pandemic conditions, since they are more used to working overtime and short staffed, and spending far more time at the bedside,” he said. “Because of the volume of patients, physicians may be spending more hours doing patient care and are experiencing more burnout.”

Health care professionals working in critical care settings have been overburdened because of the plethora of COVID-19 cases, which has led to symptoms of burnout in both physicians and nurses, findings from a new study show.

XiXinXing/ThinkStock

“Overburdening ICU professionals during an extended period of time leads to burnout,” said lead study author Niek Kok, MSc, of IQ healthcare, Radboud University Medical Center, Radboud Institute for Health Sciences, Nijmegen, the Netherlands. “All ICU professionals are at the risk of this, and in our study, the incidence of physicians experiencing burnout was significantly higher than that of nurses in June 2020.”

This burnout can be explained by conditions caused by the pandemic, he noted, such as the scarcity of staff and resources and having to work with colleagues who were not qualified to work in critical care but who were there out of necessity.

Mr. Kok presented the findings of the study at the Critical Care Congress sponsored by the Society of Critical Care Medicine.
 

Burnout highest among critical care physicians

The ICU can be a stressful environment for both patients and health care personnel, and burnout is not uncommon among ICU clinicians. However, COVID-19 has amplified the degree of burnout being experienced by clinicians working in this setting. Critical care physicians now top the list of physicians experiencing burnout, at 51%, up from 44% last year, according to the Medscape report ‘Death by 1000 Thousand Cuts’: Physician Burnout and Suicide Report 2021.

The Medscape Nurse Career Satisfaction Report 2020, while not restricted to those working in critical care, also reported higher rates of burnout, compared with the prepandemic period. The percentage of nurses reporting being “very burned out” prior to the pandemic was 4%. Six months into the pandemic, that percentage soared to 18%.

In this study, Mr. Kok and colleagues examined the prevalence and incidence of burnout symptoms and moral distress in health care professionals working in the ICU, both before and during the COVID-19 pandemic.

“When the COVID-19 pandemic surfaced in the Netherlands, the health care professionals in our hospitals were motivated to do everything they could to provide the best care possible,” said Mr. Kok. “Many of the ICU professionals immediately realized that they would have to work longer hours.”

However, the health care professionals that he spoke with did have mixed feelings. Some were afraid of being infected with the virus, while others said that “it was very interesting times for them and that gave them extra motivation to do the work.

“Some physicians [and] the WHO warned that COVID-19 is not going to weathered by a heroic sprint – it is an arduous marathon that is going to go hand in hand with burnout symptoms,” Mr. Kok added. “It will eat away at our qualified ICU staff.”
 

Before and after data on burnout

It was widely believed that the COVID-19 pandemic would increase burnout symptoms, as had been demonstrated in studies of previous pandemics. However, Mr. Kok emphasized that there are no before and after measurements that transcend cross-sectional designs.

“The claim [has been] that it increases burnout – but there are no assessments of how it progresses in ICU professionals through time,” he said. “So what we really need is a comparison [of] before and after the pandemic.”

It is quite difficult to obtain this type of information because disruptive events like the COVID-19 pandemic cannot be predicted, he said. Thus, it is challenging to get a baseline measurement. But Mr. Kok pointed out that the study has both “before and after” measurements.

“By coincidence really, we had baseline data to measure the impact of the COVID-19 pandemic and had information that was collected before the pandemic,” he said.

In January 2020, a study began looking at the effects of ethics meetings on moral distress in ICU professionals. Data had been collected on moral distress and burnout on ICU professionals in December 2019. The first COVID-19 cases appeared in the Netherlands in February 2020.

A follow-up study was then conducted in May and June 2020, several months into the pandemic.

The longitudinal open cohort study included all ICU personnel who were working in five units within a single university medical center, plus another adult ICU that was based in a separate teaching hospital.

A total of 352 health care professionals responded to a baseline survey in October through December 2019, and then 233 responded to a follow-up survey sent in May and June 2020. The authors measured burnout symptoms and moral distress with the Maslach Burnout Inventory and the Moral Distress Scale, respectively.
 

Findings

The overall prevalence of burnout symptoms was 23.0% prior to the pandemic, and that jumped to 36.1% at post-peak time. Higher rates of burnout were reported by nurses (38.0%) than physicians (28.6%).

However, the incidence rate of new burnout cases was higher among physicians, compared with nurses (26.7% vs 21.9%). Not surprisingly, a higher prevalence of burnout symptoms was observed in the post-peak period for all clinicians (odds ratio, 1.83; 95% confidence interval, 1.32-2.53), and was higher for nurses (odds ratio, 1.77; 95% confidence interval, 1.03-3.04), for those working overtime (OR, 2.11; 95% CI, 1.48-3.02), and for personnel who directly engaged in patient care (OR, 1.87; 95% CI, 1.35-2.60).

Physicians in general were much more likely to develop burnout symptoms related to the pandemic, compared with nurses (OR, 3.56; 95% CI, 1.06-12.21).

When looking at findings on moral distress, Kok pointed out that it often arises in situations when the health care professional knows the right thing to do but is prevented from doing so. “Morally distressful situations all rose from December to June,” said Mr. Kok. “Scarcity was the most distressing. The other was where colleagues were perceived to be less skilled, and this had to do with the recruitment of people from outside of the ICU to provide care.”

Moral distress from scarcity and unskilled colleagues were both significantly related to burnout, he noted.

In the final model, working in a COVID-19 unit, stress from scarcity of resources and people, stress from unskilled colleagues, and stress from unsafe conditions were all related to burnout. “The stress of physicians was significantly higher,” said Kok. “Even though nurses had higher baseline burnout, it became less pronounced in June 2020. This indicates that burnout was significantly higher in physicians.”

Thus, Mr. Kok and colleagues concluded that overburdening ICU professionals during an extended period of time leads to burnout, and all ICU workers are at risk.
 

Burnout rates higher in physicians

Weighing in on the study, Greg S. Martin, MD, FCCP, professor of medicine in the division of pulmonary, allergy, critical care and sleep medicine, Emory University, Atlanta, noted that the differences observed between physicians and nurses may have to do with the fact that “nurses have been smoldering all along and experiencing higher rates of burnout.

“They may have adapted better to the pandemic conditions, since they are more used to working overtime and short staffed, and spending far more time at the bedside,” he said. “Because of the volume of patients, physicians may be spending more hours doing patient care and are experiencing more burnout.”

States that implemented mask mandates in 2020 saw a decline in the growth of COVID-19 hospitalizations between March and October 2020, according to a new study published Feb. 5 in the CDC’s Morbidity and Mortality Weekly Report.

Hospitalization growth rates declined by 5.5 percentage points for adults between ages 18-64 about 3 weeks after the mandates were implemented, compared with climbing growth rates in the 4 weeks before mandates.

CDC Director Rochelle Walensky said she was pleased to see the results, but that it’s “too early” to tell whether President Joe Biden’s recent mask orders have had an effect on cases and hospitalizations in 2021.

“We’re going to be watching the mask data very carefully,” she said during a news briefing with the White House COVID-19 Response Team on Feb. 5. “I think it’s probably still a bit too early to tell, but I’m encouraged with the decrease in case rates right now.”

In another study published Feb. 5 in the Morbidity and Mortality Weekly Report, trained observers tracked mask use at six universities with mask mandates between September and November 2020. Overall, observers reported that about 92% of people wore masks correctly indoors, which varied based on the type of mask.

About 97% of people used N95 masks correctly, compared with 92% who used cloth masks, and 79% who used bandanas, scarves, or neck gaiters. Cloth masks were most common, and bandanas and scarves were least common.

The Biden administration is considering whether to send masks directly to American households to encourage people to wear them, according to NBC News. The White House COVID-19 Response Team is debating the logistics of mailing out masks, including how many to send and what the mask material would be, the news outlet reported.

Wisconsin Gov. Tony Evers reissued a new statewide mask mandate on Feb. 4, just an hour after the Republican-controlled legislature voted to repeal his previous mandate, according to The Associated Press. Gov. Evers said his priority is to keep people safe and that wearing a mask is the easiest way to do so.

“If the legislature keeps playing politics and we don’t keep wearing masks, we’re going to see more preventable deaths,” he said. “It’s going to take even longer to get our state and our economy back on track.”

A version of this article first appeared on WebMD.com.

States that implemented mask mandates in 2020 saw a decline in the growth of COVID-19 hospitalizations between March and October 2020, according to a new study published Feb. 5 in the CDC’s Morbidity and Mortality Weekly Report.

Hospitalization growth rates declined by 5.5 percentage points for adults between ages 18-64 about 3 weeks after the mandates were implemented, compared with climbing growth rates in the 4 weeks before mandates.

CDC Director Rochelle Walensky said she was pleased to see the results, but that it’s “too early” to tell whether President Joe Biden’s recent mask orders have had an effect on cases and hospitalizations in 2021.

“We’re going to be watching the mask data very carefully,” she said during a news briefing with the White House COVID-19 Response Team on Feb. 5. “I think it’s probably still a bit too early to tell, but I’m encouraged with the decrease in case rates right now.”

In another study published Feb. 5 in the Morbidity and Mortality Weekly Report, trained observers tracked mask use at six universities with mask mandates between September and November 2020. Overall, observers reported that about 92% of people wore masks correctly indoors, which varied based on the type of mask.

About 97% of people used N95 masks correctly, compared with 92% who used cloth masks, and 79% who used bandanas, scarves, or neck gaiters. Cloth masks were most common, and bandanas and scarves were least common.

The Biden administration is considering whether to send masks directly to American households to encourage people to wear them, according to NBC News. The White House COVID-19 Response Team is debating the logistics of mailing out masks, including how many to send and what the mask material would be, the news outlet reported.

Wisconsin Gov. Tony Evers reissued a new statewide mask mandate on Feb. 4, just an hour after the Republican-controlled legislature voted to repeal his previous mandate, according to The Associated Press. Gov. Evers said his priority is to keep people safe and that wearing a mask is the easiest way to do so.

“If the legislature keeps playing politics and we don’t keep wearing masks, we’re going to see more preventable deaths,” he said. “It’s going to take even longer to get our state and our economy back on track.”

A version of this article first appeared on WebMD.com.

States that implemented mask mandates in 2020 saw a decline in the growth of COVID-19 hospitalizations between March and October 2020, according to a new study published Feb. 5 in the CDC’s Morbidity and Mortality Weekly Report.

Hospitalization growth rates declined by 5.5 percentage points for adults between ages 18-64 about 3 weeks after the mandates were implemented, compared with climbing growth rates in the 4 weeks before mandates.

CDC Director Rochelle Walensky said she was pleased to see the results, but that it’s “too early” to tell whether President Joe Biden’s recent mask orders have had an effect on cases and hospitalizations in 2021.

“We’re going to be watching the mask data very carefully,” she said during a news briefing with the White House COVID-19 Response Team on Feb. 5. “I think it’s probably still a bit too early to tell, but I’m encouraged with the decrease in case rates right now.”

In another study published Feb. 5 in the Morbidity and Mortality Weekly Report, trained observers tracked mask use at six universities with mask mandates between September and November 2020. Overall, observers reported that about 92% of people wore masks correctly indoors, which varied based on the type of mask.

About 97% of people used N95 masks correctly, compared with 92% who used cloth masks, and 79% who used bandanas, scarves, or neck gaiters. Cloth masks were most common, and bandanas and scarves were least common.

The Biden administration is considering whether to send masks directly to American households to encourage people to wear them, according to NBC News. The White House COVID-19 Response Team is debating the logistics of mailing out masks, including how many to send and what the mask material would be, the news outlet reported.

Wisconsin Gov. Tony Evers reissued a new statewide mask mandate on Feb. 4, just an hour after the Republican-controlled legislature voted to repeal his previous mandate, according to The Associated Press. Gov. Evers said his priority is to keep people safe and that wearing a mask is the easiest way to do so.

“If the legislature keeps playing politics and we don’t keep wearing masks, we’re going to see more preventable deaths,” he said. “It’s going to take even longer to get our state and our economy back on track.”

A version of this article first appeared on WebMD.com.

The Food and Drug Administration has revised its emergency use authorization for COVID-19 convalescent plasma on the basis of the latest available data.

The revision states that only high-titer COVID-19 convalescent plasma can be used and only in hospitalized patients who are early in the disease course and those with impaired humoral immunity who cannot produce an adequate antibody response.

The revisions stem from new clinical trial data analyzed or reported since the original EUA was issued in August 2020. The original EUA did not have these restrictions.

“This and other changes to the EUA represent important updates to the use of convalescent plasma for the treatment of COVID-19 patients,” Peter Marks, MD, PhD, director, FDA Center for Biologics Evaluation and Research, said in a statement announcing the revisions.

“COVID-19 convalescent plasma used according to the revised EUA may have efficacy, and its known and potential benefits outweigh its known and potential risks,” the FDA said.

The agency said it revoked use of low-titer COVID-19 convalescent plasma on the basis of new data from clinical trials, including randomized, controlled trials, that have failed to demonstrate that low-titer convalescent plasma may be effective in the treatment of hospitalized patients with COVID-19.

The FDA’s updated fact sheet for health care providers on the use of COVID-19 convalescent plasma also notes that transfusion of COVID-19 convalescent plasma late in the disease course, following respiratory failure requiring intubation and mechanical ventilation, hasn’t been found to have clinical benefit.

The revised EUA also includes several additional tests that can be used to manufacture COVID-19 convalescent plasma.

“With this update, nine tests are now included in the EUA for testing plasma donations for anti-SARS-CoV-2 antibodies as a manufacturing step to determine suitability before release,” the FDA said.

A version of this article first appeared on Medscape.com.

The Food and Drug Administration has revised its emergency use authorization for COVID-19 convalescent plasma on the basis of the latest available data.

The revision states that only high-titer COVID-19 convalescent plasma can be used and only in hospitalized patients who are early in the disease course and those with impaired humoral immunity who cannot produce an adequate antibody response.

The revisions stem from new clinical trial data analyzed or reported since the original EUA was issued in August 2020. The original EUA did not have these restrictions.

“This and other changes to the EUA represent important updates to the use of convalescent plasma for the treatment of COVID-19 patients,” Peter Marks, MD, PhD, director, FDA Center for Biologics Evaluation and Research, said in a statement announcing the revisions.

“COVID-19 convalescent plasma used according to the revised EUA may have efficacy, and its known and potential benefits outweigh its known and potential risks,” the FDA said.

The agency said it revoked use of low-titer COVID-19 convalescent plasma on the basis of new data from clinical trials, including randomized, controlled trials, that have failed to demonstrate that low-titer convalescent plasma may be effective in the treatment of hospitalized patients with COVID-19.

The FDA’s updated fact sheet for health care providers on the use of COVID-19 convalescent plasma also notes that transfusion of COVID-19 convalescent plasma late in the disease course, following respiratory failure requiring intubation and mechanical ventilation, hasn’t been found to have clinical benefit.

The revised EUA also includes several additional tests that can be used to manufacture COVID-19 convalescent plasma.

“With this update, nine tests are now included in the EUA for testing plasma donations for anti-SARS-CoV-2 antibodies as a manufacturing step to determine suitability before release,” the FDA said.

A version of this article first appeared on Medscape.com.

The Food and Drug Administration has revised its emergency use authorization for COVID-19 convalescent plasma on the basis of the latest available data.

The revision states that only high-titer COVID-19 convalescent plasma can be used and only in hospitalized patients who are early in the disease course and those with impaired humoral immunity who cannot produce an adequate antibody response.

The revisions stem from new clinical trial data analyzed or reported since the original EUA was issued in August 2020. The original EUA did not have these restrictions.

“This and other changes to the EUA represent important updates to the use of convalescent plasma for the treatment of COVID-19 patients,” Peter Marks, MD, PhD, director, FDA Center for Biologics Evaluation and Research, said in a statement announcing the revisions.

“COVID-19 convalescent plasma used according to the revised EUA may have efficacy, and its known and potential benefits outweigh its known and potential risks,” the FDA said.

The agency said it revoked use of low-titer COVID-19 convalescent plasma on the basis of new data from clinical trials, including randomized, controlled trials, that have failed to demonstrate that low-titer convalescent plasma may be effective in the treatment of hospitalized patients with COVID-19.

The FDA’s updated fact sheet for health care providers on the use of COVID-19 convalescent plasma also notes that transfusion of COVID-19 convalescent plasma late in the disease course, following respiratory failure requiring intubation and mechanical ventilation, hasn’t been found to have clinical benefit.

The revised EUA also includes several additional tests that can be used to manufacture COVID-19 convalescent plasma.

“With this update, nine tests are now included in the EUA for testing plasma donations for anti-SARS-CoV-2 antibodies as a manufacturing step to determine suitability before release,” the FDA said.

A version of this article first appeared on Medscape.com.

The world of telemedicine—especially teledermatology—had been a sleepy underutilized afterthought for most physicians until we were faced with a global pandemic the likes of which none of us had seen in our lifetimes. And just like that, teledermatology went from an afterthought to part of the “new normal.” Although those of us already practicing telemedicine knew of potential pitfalls and concerns, this great social experiment of throwing everyone into unexplored territory led to a great deal of frustration with technology and workflows that were not optimized for dermatology visits. The process is still changing, and the technical aspects of conducting teledermatology visits will no doubt improve, but what about the bigger question of reimbursement? Without adequate payments and financial models, the long-term future of telemedicine is uncertain, so an understanding of the current and likely future landscape of telemedicine reimbursement is critical.

Waivers During the Public Health Emergency

The declaration of a public health emergency (PHE)allowed for significant flexibility by the Centers for Medicare & Medicaid Services (CMS) during the coronavirus disease 2019 (COVID-19) pandemic. Importantly, the CMS was permitted to act quickly to allow telehealth to flourish during the worst of the pandemic and throughout the declared PHE, which has been extended several times already. Currently, the PHE is set to expire on April 20, 2021, but may be extended again if the pandemic is ongoing. The most important of these waivers was probably the removal of both the originating site and geographic requirements for telehealth services.¹ Prior to the COVID-19 PHE, a patient would have to travel to a doctor’s office, hospital, or skilled nursing facility to receive telehealth care (originating site requirement), and even then this was only allowed in defined rural areas of the country (geographic requirement). Both of these requirements were waived, allowing for any patient to receive telehealth services within their own homes. Concurrently, the requirement that patients must have an established relationship with the provider (ie, telehealth could not be used to provide care to new patients) also was waived.¹

In the spirit of expanding access to care and providing reasonable reimbursement for medical services, other changes were made for which the CMS should be commended. In acknowledging that many Medicare/Medicaid beneficiaries may not have access to devices that permit real-time, 2-way audio/video communication, which previously were necessary to qualify for a telehealth encounter, the CMS decided to cover telephone visits and provide reimbursement at the level of an established visit.¹ They also changed the billing structure to remove the place of service (POS) designation for telehealth (POS 02) and replace it with the normal physician’s office POS designation (usually POS 11), bringing back a telehealth modifier (modifier -95) in the process. The benefit of this change is solely to increase reimbursement for these services, as telehealth POS services generally are covered at lower facility rates, whereas POS 11 codes are reimbursed at the full level of a nonfacility physician’s office rate.

Finally, other waivers such as the Office of Civil Rights’ decision to waive HIPAA (Health Insurance Portability and Accountability Act) violations for telehealth platforms during the PHE allowed offices to take on telemedicine quickly without having to implement a new infrastructure.² Numerous codes were added to the list of covered services for telehealth, but these generally are not relevant for dermatologists. The CMS also allowed physicians’ offices to waive the patient responsibility/co-pay during the COVID-19 PHE, which previously was not allowed due to concerns about the anti-kickback statute.¹ These co-pay waivers were intended to remove another barrier to care for patients who were hesitant to participate in virtual visits. For the most part, the waiver of state licensing requirements is a bit less useful. As part of the CMS waiver, providers technically are allowed to see out-of-state Medicare/Medicaid beneficiaries, but state licensing laws are still in effect; thus, in the absence of a blanket state-level waiver (which some states enacted, modeled after the Uniform Emergency Volunteer Health Practitioner Act of 2006³), providers still cannot see most out-of-state patients from a legal and malpractice coverage standpoint.

An important flexibility during the COVID-19 PHE is one that often is underrecognized. The CMS has been clear about the ability to provide direct supervision for advanced practice providers (APPs) and residents via telehealth during the PHE, which allows for incident-to billing for APPs at remote sites given that the supervising physician is immediately available via an interactive, 2-way, live audio/video telecommunications method. It also allows for direct supervision of APPs and residents using such technology. For dermatology, which does not have a primary care waiver, an attending must still directly supervise each patient and see the patient via a live audio/video modality but does not have to be on-site to do so. This is a very interesting concept that, if extended, could truly impact practice management for the long-term.

Response From Commercial Insurance Carriers

Tracking along with the CMS waivers and flexibilities during the PHE, most commercial carriers quickly adopted similar policies to cover telehealth services. It should be noted that for most commercial insurance carriers, the coverage was already broader than Medicare/Medicaid coverage for telehealth prior to the PHE, so in many ways it is an extension of that concept and acceptance of telemedicine as a whole. What is sometimes confusing, though, is that various policies and requirements around billing exist; for example, while most carriers emulated the POS requirements that the CMS adopted, some carriers still stuck with the telemedicine POS but paid full in-office visit rates for those codes. Some carriers adopted higher reimbursement rates for telephone visits, similar to the CMS, while others instructed providers to just bill for the established office visit codes and allowed for telephone-only visits to qualify for these billing codes. Some carriers also waived co-pays for telehealth visits for their members (whether related to COVID-19 or not). It is beyond the scope of this article to delve into the specifics, which may vary not only by carrier but by region and plan. However, it is important to stay on top of one’s insurance carriers to find out what their latest directives are for billing for telehealth.

Postpandemic Teledermatology

What about the future of teledermatology? Although many dermatologists have adopted telehealth services out of necessity during the COVID-19 PHE, the jury is still out on the long-term forecast for telemedicine in dermatology. Concerns about liability/malpractice and technology issues abound, and for many, the headaches of teledermatology—such as trying to focus on a blurry photograph of a nevus that the patient is concerned about—make it unappealing. Some of these issues will be addressed by better technology, but the reimbursement structure must continue for teledermatology to remain in widespread use.

Currently, the biggest question facing telehealth is whether the waivers for originating site and geographic requirements will be able to continue. The CMS itself does not have the statutory authority to make these changes permanent and was only allowed to act due to a waiver under section 1135 of the Social Security Act during a PHE. It would take an act of Congress to change the law to allow for this specific expansion of telehealth services. A number of federal bills, including S 2741 (Creating Opportunities Now for Necessary and Effective Care Technologies [CONNECT] for Health Act of 2019) and S 4796 (Fair Care Act of 2020) from the Senate, contain such provisions, but none have been passed at the time of writing. There does seem to be broad support of the concept of expanding telemedicine access, such as noted by New York State Governor Andrew Cuomo in his 2021 State of the State address,⁴ but it remains to be seen when action will come.

Some regulations, such as the HIPAA waiver and the ability to waive co-pays, are not slated to continue after the pandemic. The ability to supervise residents via telehealth (real-time audio/video) has been made permanent, but only in rural areas. Direct supervision of APPs via telehealth will continue through the end of the calendar year of the PHE or the end of 2021, whichever comes later, but it remains to be seen whether remote supervision will continue. The CMS has stated in its comments that it is looking at this issue closely and may establish certain guardrails to ensure quality of care is maintained.¹ Telephone/audio-only visits also may come under further scrutiny, but research has supported the concept that patients who are more likely to gain access through audio-only modalities are older, Medicare/Medicaid (vs commercial), and Black (vs White) patients,⁵ so it would indeed introduce an unfair barrier to access if such coverage was rolled back.

Final Thoughts

Overall, we have made much progress in teledermatology. Once utilized by a small fraction of dermatologists, the vast majority of us turned to teledermatology to sustain our practices during the COVID-19 pandemic. Moving forward, there are 2 critical factors to consider: continued technological innovation and permanent coverage for telehealth reimbursement at in-office visit levels. With these challenges resolved, we can move forward and consider novel models that may be able to deliver dermatologic care to a broader patient population, thereby solving the critical issue of access to care for so many patients in need in our country.

The world of telemedicine—especially teledermatology—had been a sleepy underutilized afterthought for most physicians until we were faced with a global pandemic the likes of which none of us had seen in our lifetimes. And just like that, teledermatology went from an afterthought to part of the “new normal.” Although those of us already practicing telemedicine knew of potential pitfalls and concerns, this great social experiment of throwing everyone into unexplored territory led to a great deal of frustration with technology and workflows that were not optimized for dermatology visits. The process is still changing, and the technical aspects of conducting teledermatology visits will no doubt improve, but what about the bigger question of reimbursement? Without adequate payments and financial models, the long-term future of telemedicine is uncertain, so an understanding of the current and likely future landscape of telemedicine reimbursement is critical.

Waivers During the Public Health Emergency

The declaration of a public health emergency (PHE)allowed for significant flexibility by the Centers for Medicare & Medicaid Services (CMS) during the coronavirus disease 2019 (COVID-19) pandemic. Importantly, the CMS was permitted to act quickly to allow telehealth to flourish during the worst of the pandemic and throughout the declared PHE, which has been extended several times already. Currently, the PHE is set to expire on April 20, 2021, but may be extended again if the pandemic is ongoing. The most important of these waivers was probably the removal of both the originating site and geographic requirements for telehealth services.¹ Prior to the COVID-19 PHE, a patient would have to travel to a doctor’s office, hospital, or skilled nursing facility to receive telehealth care (originating site requirement), and even then this was only allowed in defined rural areas of the country (geographic requirement). Both of these requirements were waived, allowing for any patient to receive telehealth services within their own homes. Concurrently, the requirement that patients must have an established relationship with the provider (ie, telehealth could not be used to provide care to new patients) also was waived.¹

In the spirit of expanding access to care and providing reasonable reimbursement for medical services, other changes were made for which the CMS should be commended. In acknowledging that many Medicare/Medicaid beneficiaries may not have access to devices that permit real-time, 2-way audio/video communication, which previously were necessary to qualify for a telehealth encounter, the CMS decided to cover telephone visits and provide reimbursement at the level of an established visit.¹ They also changed the billing structure to remove the place of service (POS) designation for telehealth (POS 02) and replace it with the normal physician’s office POS designation (usually POS 11), bringing back a telehealth modifier (modifier -95) in the process. The benefit of this change is solely to increase reimbursement for these services, as telehealth POS services generally are covered at lower facility rates, whereas POS 11 codes are reimbursed at the full level of a nonfacility physician’s office rate.

Finally, other waivers such as the Office of Civil Rights’ decision to waive HIPAA (Health Insurance Portability and Accountability Act) violations for telehealth platforms during the PHE allowed offices to take on telemedicine quickly without having to implement a new infrastructure.² Numerous codes were added to the list of covered services for telehealth, but these generally are not relevant for dermatologists. The CMS also allowed physicians’ offices to waive the patient responsibility/co-pay during the COVID-19 PHE, which previously was not allowed due to concerns about the anti-kickback statute.¹ These co-pay waivers were intended to remove another barrier to care for patients who were hesitant to participate in virtual visits. For the most part, the waiver of state licensing requirements is a bit less useful. As part of the CMS waiver, providers technically are allowed to see out-of-state Medicare/Medicaid beneficiaries, but state licensing laws are still in effect; thus, in the absence of a blanket state-level waiver (which some states enacted, modeled after the Uniform Emergency Volunteer Health Practitioner Act of 2006³), providers still cannot see most out-of-state patients from a legal and malpractice coverage standpoint.

An important flexibility during the COVID-19 PHE is one that often is underrecognized. The CMS has been clear about the ability to provide direct supervision for advanced practice providers (APPs) and residents via telehealth during the PHE, which allows for incident-to billing for APPs at remote sites given that the supervising physician is immediately available via an interactive, 2-way, live audio/video telecommunications method. It also allows for direct supervision of APPs and residents using such technology. For dermatology, which does not have a primary care waiver, an attending must still directly supervise each patient and see the patient via a live audio/video modality but does not have to be on-site to do so. This is a very interesting concept that, if extended, could truly impact practice management for the long-term.

Response From Commercial Insurance Carriers

Tracking along with the CMS waivers and flexibilities during the PHE, most commercial carriers quickly adopted similar policies to cover telehealth services. It should be noted that for most commercial insurance carriers, the coverage was already broader than Medicare/Medicaid coverage for telehealth prior to the PHE, so in many ways it is an extension of that concept and acceptance of telemedicine as a whole. What is sometimes confusing, though, is that various policies and requirements around billing exist; for example, while most carriers emulated the POS requirements that the CMS adopted, some carriers still stuck with the telemedicine POS but paid full in-office visit rates for those codes. Some carriers adopted higher reimbursement rates for telephone visits, similar to the CMS, while others instructed providers to just bill for the established office visit codes and allowed for telephone-only visits to qualify for these billing codes. Some carriers also waived co-pays for telehealth visits for their members (whether related to COVID-19 or not). It is beyond the scope of this article to delve into the specifics, which may vary not only by carrier but by region and plan. However, it is important to stay on top of one’s insurance carriers to find out what their latest directives are for billing for telehealth.

Postpandemic Teledermatology

What about the future of teledermatology? Although many dermatologists have adopted telehealth services out of necessity during the COVID-19 PHE, the jury is still out on the long-term forecast for telemedicine in dermatology. Concerns about liability/malpractice and technology issues abound, and for many, the headaches of teledermatology—such as trying to focus on a blurry photograph of a nevus that the patient is concerned about—make it unappealing. Some of these issues will be addressed by better technology, but the reimbursement structure must continue for teledermatology to remain in widespread use.

Currently, the biggest question facing telehealth is whether the waivers for originating site and geographic requirements will be able to continue. The CMS itself does not have the statutory authority to make these changes permanent and was only allowed to act due to a waiver under section 1135 of the Social Security Act during a PHE. It would take an act of Congress to change the law to allow for this specific expansion of telehealth services. A number of federal bills, including S 2741 (Creating Opportunities Now for Necessary and Effective Care Technologies [CONNECT] for Health Act of 2019) and S 4796 (Fair Care Act of 2020) from the Senate, contain such provisions, but none have been passed at the time of writing. There does seem to be broad support of the concept of expanding telemedicine access, such as noted by New York State Governor Andrew Cuomo in his 2021 State of the State address,⁴ but it remains to be seen when action will come.

Some regulations, such as the HIPAA waiver and the ability to waive co-pays, are not slated to continue after the pandemic. The ability to supervise residents via telehealth (real-time audio/video) has been made permanent, but only in rural areas. Direct supervision of APPs via telehealth will continue through the end of the calendar year of the PHE or the end of 2021, whichever comes later, but it remains to be seen whether remote supervision will continue. The CMS has stated in its comments that it is looking at this issue closely and may establish certain guardrails to ensure quality of care is maintained.¹ Telephone/audio-only visits also may come under further scrutiny, but research has supported the concept that patients who are more likely to gain access through audio-only modalities are older, Medicare/Medicaid (vs commercial), and Black (vs White) patients,⁵ so it would indeed introduce an unfair barrier to access if such coverage was rolled back.

Final Thoughts

Overall, we have made much progress in teledermatology. Once utilized by a small fraction of dermatologists, the vast majority of us turned to teledermatology to sustain our practices during the COVID-19 pandemic. Moving forward, there are 2 critical factors to consider: continued technological innovation and permanent coverage for telehealth reimbursement at in-office visit levels. With these challenges resolved, we can move forward and consider novel models that may be able to deliver dermatologic care to a broader patient population, thereby solving the critical issue of access to care for so many patients in need in our country.

The world of telemedicine—especially teledermatology—had been a sleepy underutilized afterthought for most physicians until we were faced with a global pandemic the likes of which none of us had seen in our lifetimes. And just like that, teledermatology went from an afterthought to part of the “new normal.” Although those of us already practicing telemedicine knew of potential pitfalls and concerns, this great social experiment of throwing everyone into unexplored territory led to a great deal of frustration with technology and workflows that were not optimized for dermatology visits. The process is still changing, and the technical aspects of conducting teledermatology visits will no doubt improve, but what about the bigger question of reimbursement? Without adequate payments and financial models, the long-term future of telemedicine is uncertain, so an understanding of the current and likely future landscape of telemedicine reimbursement is critical.

Waivers During the Public Health Emergency

The declaration of a public health emergency (PHE)allowed for significant flexibility by the Centers for Medicare & Medicaid Services (CMS) during the coronavirus disease 2019 (COVID-19) pandemic. Importantly, the CMS was permitted to act quickly to allow telehealth to flourish during the worst of the pandemic and throughout the declared PHE, which has been extended several times already. Currently, the PHE is set to expire on April 20, 2021, but may be extended again if the pandemic is ongoing. The most important of these waivers was probably the removal of both the originating site and geographic requirements for telehealth services.¹ Prior to the COVID-19 PHE, a patient would have to travel to a doctor’s office, hospital, or skilled nursing facility to receive telehealth care (originating site requirement), and even then this was only allowed in defined rural areas of the country (geographic requirement). Both of these requirements were waived, allowing for any patient to receive telehealth services within their own homes. Concurrently, the requirement that patients must have an established relationship with the provider (ie, telehealth could not be used to provide care to new patients) also was waived.¹

In the spirit of expanding access to care and providing reasonable reimbursement for medical services, other changes were made for which the CMS should be commended. In acknowledging that many Medicare/Medicaid beneficiaries may not have access to devices that permit real-time, 2-way audio/video communication, which previously were necessary to qualify for a telehealth encounter, the CMS decided to cover telephone visits and provide reimbursement at the level of an established visit.¹ They also changed the billing structure to remove the place of service (POS) designation for telehealth (POS 02) and replace it with the normal physician’s office POS designation (usually POS 11), bringing back a telehealth modifier (modifier -95) in the process. The benefit of this change is solely to increase reimbursement for these services, as telehealth POS services generally are covered at lower facility rates, whereas POS 11 codes are reimbursed at the full level of a nonfacility physician’s office rate.

Finally, other waivers such as the Office of Civil Rights’ decision to waive HIPAA (Health Insurance Portability and Accountability Act) violations for telehealth platforms during the PHE allowed offices to take on telemedicine quickly without having to implement a new infrastructure.² Numerous codes were added to the list of covered services for telehealth, but these generally are not relevant for dermatologists. The CMS also allowed physicians’ offices to waive the patient responsibility/co-pay during the COVID-19 PHE, which previously was not allowed due to concerns about the anti-kickback statute.¹ These co-pay waivers were intended to remove another barrier to care for patients who were hesitant to participate in virtual visits. For the most part, the waiver of state licensing requirements is a bit less useful. As part of the CMS waiver, providers technically are allowed to see out-of-state Medicare/Medicaid beneficiaries, but state licensing laws are still in effect; thus, in the absence of a blanket state-level waiver (which some states enacted, modeled after the Uniform Emergency Volunteer Health Practitioner Act of 2006³), providers still cannot see most out-of-state patients from a legal and malpractice coverage standpoint.

An important flexibility during the COVID-19 PHE is one that often is underrecognized. The CMS has been clear about the ability to provide direct supervision for advanced practice providers (APPs) and residents via telehealth during the PHE, which allows for incident-to billing for APPs at remote sites given that the supervising physician is immediately available via an interactive, 2-way, live audio/video telecommunications method. It also allows for direct supervision of APPs and residents using such technology. For dermatology, which does not have a primary care waiver, an attending must still directly supervise each patient and see the patient via a live audio/video modality but does not have to be on-site to do so. This is a very interesting concept that, if extended, could truly impact practice management for the long-term.

Response From Commercial Insurance Carriers

Tracking along with the CMS waivers and flexibilities during the PHE, most commercial carriers quickly adopted similar policies to cover telehealth services. It should be noted that for most commercial insurance carriers, the coverage was already broader than Medicare/Medicaid coverage for telehealth prior to the PHE, so in many ways it is an extension of that concept and acceptance of telemedicine as a whole. What is sometimes confusing, though, is that various policies and requirements around billing exist; for example, while most carriers emulated the POS requirements that the CMS adopted, some carriers still stuck with the telemedicine POS but paid full in-office visit rates for those codes. Some carriers adopted higher reimbursement rates for telephone visits, similar to the CMS, while others instructed providers to just bill for the established office visit codes and allowed for telephone-only visits to qualify for these billing codes. Some carriers also waived co-pays for telehealth visits for their members (whether related to COVID-19 or not). It is beyond the scope of this article to delve into the specifics, which may vary not only by carrier but by region and plan. However, it is important to stay on top of one’s insurance carriers to find out what their latest directives are for billing for telehealth.

Postpandemic Teledermatology

What about the future of teledermatology? Although many dermatologists have adopted telehealth services out of necessity during the COVID-19 PHE, the jury is still out on the long-term forecast for telemedicine in dermatology. Concerns about liability/malpractice and technology issues abound, and for many, the headaches of teledermatology—such as trying to focus on a blurry photograph of a nevus that the patient is concerned about—make it unappealing. Some of these issues will be addressed by better technology, but the reimbursement structure must continue for teledermatology to remain in widespread use.

Currently, the biggest question facing telehealth is whether the waivers for originating site and geographic requirements will be able to continue. The CMS itself does not have the statutory authority to make these changes permanent and was only allowed to act due to a waiver under section 1135 of the Social Security Act during a PHE. It would take an act of Congress to change the law to allow for this specific expansion of telehealth services. A number of federal bills, including S 2741 (Creating Opportunities Now for Necessary and Effective Care Technologies [CONNECT] for Health Act of 2019) and S 4796 (Fair Care Act of 2020) from the Senate, contain such provisions, but none have been passed at the time of writing. There does seem to be broad support of the concept of expanding telemedicine access, such as noted by New York State Governor Andrew Cuomo in his 2021 State of the State address,⁴ but it remains to be seen when action will come.

Some regulations, such as the HIPAA waiver and the ability to waive co-pays, are not slated to continue after the pandemic. The ability to supervise residents via telehealth (real-time audio/video) has been made permanent, but only in rural areas. Direct supervision of APPs via telehealth will continue through the end of the calendar year of the PHE or the end of 2021, whichever comes later, but it remains to be seen whether remote supervision will continue. The CMS has stated in its comments that it is looking at this issue closely and may establish certain guardrails to ensure quality of care is maintained.¹ Telephone/audio-only visits also may come under further scrutiny, but research has supported the concept that patients who are more likely to gain access through audio-only modalities are older, Medicare/Medicaid (vs commercial), and Black (vs White) patients,⁵ so it would indeed introduce an unfair barrier to access if such coverage was rolled back.

Final Thoughts

Overall, we have made much progress in teledermatology. Once utilized by a small fraction of dermatologists, the vast majority of us turned to teledermatology to sustain our practices during the COVID-19 pandemic. Moving forward, there are 2 critical factors to consider: continued technological innovation and permanent coverage for telehealth reimbursement at in-office visit levels. With these challenges resolved, we can move forward and consider novel models that may be able to deliver dermatologic care to a broader patient population, thereby solving the critical issue of access to care for so many patients in need in our country.

The Biden administration has halted a Trump administration initiative that would have allowed more physicians to prescribe buprenorphine for opioid use disorder (OUD).

Under the Trump administration’s plan, many doctors would be exempt from taking a day’s training before they could prescribe buprenorphine for OUD.

On Jan. 25, 2021, citing anonymous sources, the Washington Post reported that this action by the Biden administration was likely. At the time, there were concerns about whether the Department of Health & Human Services had the legal authority to make this policy change, the Post reported. The Substance Abuse and Mental Health Services Administration subsequently announced the derailment of the buprenorphine proposal on its website.

The plan was unveiled Jan. 14 in the final days of the Trump administration. In SAMHSA’s view, the proposal was made “prematurely.” The SAMHSA statement did not detail the reasons for abandoning the Jan. 14 proposal. It had been scheduled to take effect upon publication in the Federal Register.

Instead of finalizing it in this way, the HHS said it would work with other federal agencies to “increase access to buprenorphine, reduce overdose rates and save lives.”

The HHS decision to scupper the proposal disappointed many physician groups. In a letter dated Jan. 27, several physician groups called on the Biden administration to proceed with the Trump proposal.

Under current federal law, physicians who wish to prescribe buprenorphine outside of opioid treatment programs must take an 8-hour course and receive a waiver from the Drug Enforcement Administration, the letter noted. It was signed by the American College of Emergency Physicians, the American Medical Association, and other organizations.
 

Treatment barrier

After taking the training course, it can take 60-90 days for physicians to receive the waiver. The license application can then be submitted. Physician groups argue that this so-called X-waiver requirement creates a barrier to providing medication-assisted treatment.

“Due to the stigma, some clinicians are not willing to pursue this DEA license or even engage in treatment of patients with [OUD],” the letter said.

The Trump administration’s proposal would have limited most physicians to treating no more than 30 patients with buprenorphine for OUD at any one time. This cap would not have applied to hospital-based physicians, such as those practicing emergency medicine, the HHS noted in a statement. The policy would have applied to only physicians who already have registered with the DEA.

Patrice A. Harris, MD, the immediate past president of the AMA and chair of the organization’s Opioid Task Force, was among the many physicians who supported the Trump administration proposal.

“It is estimated that more than 2 million Americans need treatment for opioid use disorder, but only a small percentage actually receive treatment,” Dr. Harris said in statement. Dr. Harris also noted that overdose deaths have reportedly accelerated during the COVID-19 pandemic.

Centers for Disease Control and Prevention data show there were more than 83,000 drug overdose deaths in the United States in the 12 months ending in June 2020. That is the highest number of overdose deaths ever recorded in a 12-month period and is an increase of more than 21%, compared with the previous year.
 

A ‘disappointment’

On Jan. 28, Dr. Harris said the decision to drop the plan was a disappointment.

“We encourage the current administration to quickly develop a path forward that removes the burdensome waiver requirement, thus allowing more physicians to prescribe this lifesaving medication,” she said in a statement sent to this news organization.

In a Jan. 26 statement, the American Society of Addiction Medicine urged Congress to eliminate the X-waiver and called for more education and training in the treatment of patients who struggle with opioids.

In the 116th session of Congress, which ended on Jan. 3, there was bipartisan support for proposed legislation to ease requirements for buprenorphine prescribing. A House bill had more than 90 Democratic and 21 Republican sponsors. A companion Senate bill had three Democratic and three Republican Sponsors, including Sen. Maggie Hassan (D-N.H.). On Jan. 25, Dr. Hassan tweeted that she would be seeking an explanation from the Biden administration if it halted the plan to ease the waiver restriction.

“Medication-assisted treatment can save lives, and the buprenorphine waiver requirement should be eliminated so that physicians can more easily prescribe it to those who need it,” she said.

Many clinicians and policy experts turned to Twitter to urge an easing of buprenorphine prescribing, using the hashtag “Xthexwaiver.”

Among them was the official who put forward the Jan. 14 proposal, Brett Giroir, MD. He served as assistant secretary for health during the Trump administration.

 

Objections

In its Jan. 25 article, the Washington Post referred to an article in Alcoholism and Drug Abuse Weekly in which a top federal official in the Trump administration objected to Dr. Giroir’s plan.

Elinore F. McCance-Katz, MD, PhD, who served as the assistant secretary of HHS for SAMHSA, had earlier proposed raising the cap for addiction experts. Alcoholism and Drug Abuse Weekly quotes Dr. McCance-Katz as saying the Trump buprenorphine proposal was “unfair to the incoming administration.”

“The Biden administration has so much work to do to get their programs and policies into place, and to do something like this at the 11th hour that could get doctors into trouble – it’s heinous,” she said in the article.

Dr. McCance-Katz had resigned before the Trump administration proposal was unveiled. On Jan. 7, she issued a public notice announcing she would resign, citing concerns about the previous day’s attack on the U.S. Capitol.

“It had been my plan to stay until the change in administration occurred, but my plans abruptly changed last evening when, on my way back from visiting an excellent residential treatment program in New York, I saw the violent takeover of the Capitol building,” she said.

On Twitter, Roland Flores, MD, an anesthesiologist and pain specialist, urged his colleagues to consider the need for more education among clinicians who treat OUD. He jousted a bit with those favoring a swift drive to “XtheXwaiver” and questioned their arguments about the burden of the current rules.

“I think ‘all this red tape’ is a little bit of an exaggeration – it’s an 8-hour online course, and an application,” Dr. Flores tweeted in one exchange. “But #XtheXwaiver is fine – it’s probably rooted in stigma. It’s unlikely to make much difference tho. The waiver wasn’t the thing keeping docs from prescribing.”

A version of this article first appeared on Medscape.com.

The Biden administration has halted a Trump administration initiative that would have allowed more physicians to prescribe buprenorphine for opioid use disorder (OUD).

Under the Trump administration’s plan, many doctors would be exempt from taking a day’s training before they could prescribe buprenorphine for OUD.

On Jan. 25, 2021, citing anonymous sources, the Washington Post reported that this action by the Biden administration was likely. At the time, there were concerns about whether the Department of Health & Human Services had the legal authority to make this policy change, the Post reported. The Substance Abuse and Mental Health Services Administration subsequently announced the derailment of the buprenorphine proposal on its website.

The plan was unveiled Jan. 14 in the final days of the Trump administration. In SAMHSA’s view, the proposal was made “prematurely.” The SAMHSA statement did not detail the reasons for abandoning the Jan. 14 proposal. It had been scheduled to take effect upon publication in the Federal Register.

Instead of finalizing it in this way, the HHS said it would work with other federal agencies to “increase access to buprenorphine, reduce overdose rates and save lives.”

The HHS decision to scupper the proposal disappointed many physician groups. In a letter dated Jan. 27, several physician groups called on the Biden administration to proceed with the Trump proposal.

Under current federal law, physicians who wish to prescribe buprenorphine outside of opioid treatment programs must take an 8-hour course and receive a waiver from the Drug Enforcement Administration, the letter noted. It was signed by the American College of Emergency Physicians, the American Medical Association, and other organizations.
 

Treatment barrier

After taking the training course, it can take 60-90 days for physicians to receive the waiver. The license application can then be submitted. Physician groups argue that this so-called X-waiver requirement creates a barrier to providing medication-assisted treatment.

“Due to the stigma, some clinicians are not willing to pursue this DEA license or even engage in treatment of patients with [OUD],” the letter said.

The Trump administration’s proposal would have limited most physicians to treating no more than 30 patients with buprenorphine for OUD at any one time. This cap would not have applied to hospital-based physicians, such as those practicing emergency medicine, the HHS noted in a statement. The policy would have applied to only physicians who already have registered with the DEA.

Patrice A. Harris, MD, the immediate past president of the AMA and chair of the organization’s Opioid Task Force, was among the many physicians who supported the Trump administration proposal.

“It is estimated that more than 2 million Americans need treatment for opioid use disorder, but only a small percentage actually receive treatment,” Dr. Harris said in statement. Dr. Harris also noted that overdose deaths have reportedly accelerated during the COVID-19 pandemic.

Centers for Disease Control and Prevention data show there were more than 83,000 drug overdose deaths in the United States in the 12 months ending in June 2020. That is the highest number of overdose deaths ever recorded in a 12-month period and is an increase of more than 21%, compared with the previous year.
 

A ‘disappointment’

On Jan. 28, Dr. Harris said the decision to drop the plan was a disappointment.

“We encourage the current administration to quickly develop a path forward that removes the burdensome waiver requirement, thus allowing more physicians to prescribe this lifesaving medication,” she said in a statement sent to this news organization.

In a Jan. 26 statement, the American Society of Addiction Medicine urged Congress to eliminate the X-waiver and called for more education and training in the treatment of patients who struggle with opioids.

In the 116th session of Congress, which ended on Jan. 3, there was bipartisan support for proposed legislation to ease requirements for buprenorphine prescribing. A House bill had more than 90 Democratic and 21 Republican sponsors. A companion Senate bill had three Democratic and three Republican Sponsors, including Sen. Maggie Hassan (D-N.H.). On Jan. 25, Dr. Hassan tweeted that she would be seeking an explanation from the Biden administration if it halted the plan to ease the waiver restriction.

“Medication-assisted treatment can save lives, and the buprenorphine waiver requirement should be eliminated so that physicians can more easily prescribe it to those who need it,” she said.

Many clinicians and policy experts turned to Twitter to urge an easing of buprenorphine prescribing, using the hashtag “Xthexwaiver.”

Among them was the official who put forward the Jan. 14 proposal, Brett Giroir, MD. He served as assistant secretary for health during the Trump administration.

 

Objections

In its Jan. 25 article, the Washington Post referred to an article in Alcoholism and Drug Abuse Weekly in which a top federal official in the Trump administration objected to Dr. Giroir’s plan.

Elinore F. McCance-Katz, MD, PhD, who served as the assistant secretary of HHS for SAMHSA, had earlier proposed raising the cap for addiction experts. Alcoholism and Drug Abuse Weekly quotes Dr. McCance-Katz as saying the Trump buprenorphine proposal was “unfair to the incoming administration.”

“The Biden administration has so much work to do to get their programs and policies into place, and to do something like this at the 11th hour that could get doctors into trouble – it’s heinous,” she said in the article.

Dr. McCance-Katz had resigned before the Trump administration proposal was unveiled. On Jan. 7, she issued a public notice announcing she would resign, citing concerns about the previous day’s attack on the U.S. Capitol.

“It had been my plan to stay until the change in administration occurred, but my plans abruptly changed last evening when, on my way back from visiting an excellent residential treatment program in New York, I saw the violent takeover of the Capitol building,” she said.

On Twitter, Roland Flores, MD, an anesthesiologist and pain specialist, urged his colleagues to consider the need for more education among clinicians who treat OUD. He jousted a bit with those favoring a swift drive to “XtheXwaiver” and questioned their arguments about the burden of the current rules.

“I think ‘all this red tape’ is a little bit of an exaggeration – it’s an 8-hour online course, and an application,” Dr. Flores tweeted in one exchange. “But #XtheXwaiver is fine – it’s probably rooted in stigma. It’s unlikely to make much difference tho. The waiver wasn’t the thing keeping docs from prescribing.”

A version of this article first appeared on Medscape.com.

The Biden administration has halted a Trump administration initiative that would have allowed more physicians to prescribe buprenorphine for opioid use disorder (OUD).

Under the Trump administration’s plan, many doctors would be exempt from taking a day’s training before they could prescribe buprenorphine for OUD.

On Jan. 25, 2021, citing anonymous sources, the Washington Post reported that this action by the Biden administration was likely. At the time, there were concerns about whether the Department of Health & Human Services had the legal authority to make this policy change, the Post reported. The Substance Abuse and Mental Health Services Administration subsequently announced the derailment of the buprenorphine proposal on its website.

The plan was unveiled Jan. 14 in the final days of the Trump administration. In SAMHSA’s view, the proposal was made “prematurely.” The SAMHSA statement did not detail the reasons for abandoning the Jan. 14 proposal. It had been scheduled to take effect upon publication in the Federal Register.

Instead of finalizing it in this way, the HHS said it would work with other federal agencies to “increase access to buprenorphine, reduce overdose rates and save lives.”

The HHS decision to scupper the proposal disappointed many physician groups. In a letter dated Jan. 27, several physician groups called on the Biden administration to proceed with the Trump proposal.

Under current federal law, physicians who wish to prescribe buprenorphine outside of opioid treatment programs must take an 8-hour course and receive a waiver from the Drug Enforcement Administration, the letter noted. It was signed by the American College of Emergency Physicians, the American Medical Association, and other organizations.
 

Treatment barrier

After taking the training course, it can take 60-90 days for physicians to receive the waiver. The license application can then be submitted. Physician groups argue that this so-called X-waiver requirement creates a barrier to providing medication-assisted treatment.

“Due to the stigma, some clinicians are not willing to pursue this DEA license or even engage in treatment of patients with [OUD],” the letter said.

The Trump administration’s proposal would have limited most physicians to treating no more than 30 patients with buprenorphine for OUD at any one time. This cap would not have applied to hospital-based physicians, such as those practicing emergency medicine, the HHS noted in a statement. The policy would have applied to only physicians who already have registered with the DEA.

Patrice A. Harris, MD, the immediate past president of the AMA and chair of the organization’s Opioid Task Force, was among the many physicians who supported the Trump administration proposal.

“It is estimated that more than 2 million Americans need treatment for opioid use disorder, but only a small percentage actually receive treatment,” Dr. Harris said in statement. Dr. Harris also noted that overdose deaths have reportedly accelerated during the COVID-19 pandemic.

Centers for Disease Control and Prevention data show there were more than 83,000 drug overdose deaths in the United States in the 12 months ending in June 2020. That is the highest number of overdose deaths ever recorded in a 12-month period and is an increase of more than 21%, compared with the previous year.
 

A ‘disappointment’

On Jan. 28, Dr. Harris said the decision to drop the plan was a disappointment.

“We encourage the current administration to quickly develop a path forward that removes the burdensome waiver requirement, thus allowing more physicians to prescribe this lifesaving medication,” she said in a statement sent to this news organization.

In a Jan. 26 statement, the American Society of Addiction Medicine urged Congress to eliminate the X-waiver and called for more education and training in the treatment of patients who struggle with opioids.

In the 116th session of Congress, which ended on Jan. 3, there was bipartisan support for proposed legislation to ease requirements for buprenorphine prescribing. A House bill had more than 90 Democratic and 21 Republican sponsors. A companion Senate bill had three Democratic and three Republican Sponsors, including Sen. Maggie Hassan (D-N.H.). On Jan. 25, Dr. Hassan tweeted that she would be seeking an explanation from the Biden administration if it halted the plan to ease the waiver restriction.

“Medication-assisted treatment can save lives, and the buprenorphine waiver requirement should be eliminated so that physicians can more easily prescribe it to those who need it,” she said.

Many clinicians and policy experts turned to Twitter to urge an easing of buprenorphine prescribing, using the hashtag “Xthexwaiver.”

Among them was the official who put forward the Jan. 14 proposal, Brett Giroir, MD. He served as assistant secretary for health during the Trump administration.

 

Objections

In its Jan. 25 article, the Washington Post referred to an article in Alcoholism and Drug Abuse Weekly in which a top federal official in the Trump administration objected to Dr. Giroir’s plan.

Elinore F. McCance-Katz, MD, PhD, who served as the assistant secretary of HHS for SAMHSA, had earlier proposed raising the cap for addiction experts. Alcoholism and Drug Abuse Weekly quotes Dr. McCance-Katz as saying the Trump buprenorphine proposal was “unfair to the incoming administration.”

“The Biden administration has so much work to do to get their programs and policies into place, and to do something like this at the 11th hour that could get doctors into trouble – it’s heinous,” she said in the article.

Dr. McCance-Katz had resigned before the Trump administration proposal was unveiled. On Jan. 7, she issued a public notice announcing she would resign, citing concerns about the previous day’s attack on the U.S. Capitol.

“It had been my plan to stay until the change in administration occurred, but my plans abruptly changed last evening when, on my way back from visiting an excellent residential treatment program in New York, I saw the violent takeover of the Capitol building,” she said.

On Twitter, Roland Flores, MD, an anesthesiologist and pain specialist, urged his colleagues to consider the need for more education among clinicians who treat OUD. He jousted a bit with those favoring a swift drive to “XtheXwaiver” and questioned their arguments about the burden of the current rules.

“I think ‘all this red tape’ is a little bit of an exaggeration – it’s an 8-hour online course, and an application,” Dr. Flores tweeted in one exchange. “But #XtheXwaiver is fine – it’s probably rooted in stigma. It’s unlikely to make much difference tho. The waiver wasn’t the thing keeping docs from prescribing.”

A version of this article first appeared on Medscape.com.

The National Institutes of Health has launched a database to track COVID-19–related neurologic symptoms, complications, and outcomes as well as the effects of the virus on preexisting neurologic conditions.

“We know COVID-19 can disrupt multiple body systems, but the effects of the virus and the body’s response to COVID-19 infection on the brain, spinal cord, nerves, and muscle can be particularly devastating and contribute to persistence of disability even after the virus is cleared,” said Barbara Karp, MD, program director at the National Institute of Neurological Disorders and Stroke.

“There is an urgent need to understand COVID-19–related neurological problems, which not uncommonly include headaches, fatigue, cognitive difficulties, stroke, pain, and sleep disorders as well as some very rare complications of serious infections,” said Dr. Karp.

The COVID-19 NeuroDatabank/BioBank (NeuroCOVID) is funded by the NINDS. It was created and will be maintained by researchers at NYU Langone Health in New York.

The project is led by Andrea Troxel, ScD, professor of population health, and Eva Petkova, PhD, professor of population health and child and adolescent psychiatry, both at New York University.

“We’ve built a pretty comprehensive database that will accept deidentified patient information about new neurological issues that coincide with their COVID disease or worsening of preexisting neurological problems,” said Dr. Troxel. “In addition, we have a bio repository that will accept almost any kind of biological sample, such as blood, plasma, cerebrospinal fluid, and tissue,” she said.

“Neuroimages are very difficult to store because the files are so enormous, but we’ve had some questions about that, and we’re looking into whether we can accommodate neuroimages,” Dr. Troxel noted.

Dr. Troxel said a “blast of information and invitations” has gone out in an effort to acquire data and biospecimens. “We’ve been really pleased with the amount of interest already, interest not only from large academic medical centers, as you might expect, but also from some smaller stand-alone clinics and even some individuals who have either experienced some of these neurological problems of COVID or know those who have and are really eager to try to provide information,” she added.

Researchers interested in using data and biosamples from the database may submit requests to the NeuroCOVID Steering Committee. More information is available online on the NeuroCOVID website.

A version of this article first appeared on Medscape.com.

The National Institutes of Health has launched a database to track COVID-19–related neurologic symptoms, complications, and outcomes as well as the effects of the virus on preexisting neurologic conditions.

“We know COVID-19 can disrupt multiple body systems, but the effects of the virus and the body’s response to COVID-19 infection on the brain, spinal cord, nerves, and muscle can be particularly devastating and contribute to persistence of disability even after the virus is cleared,” said Barbara Karp, MD, program director at the National Institute of Neurological Disorders and Stroke.

“There is an urgent need to understand COVID-19–related neurological problems, which not uncommonly include headaches, fatigue, cognitive difficulties, stroke, pain, and sleep disorders as well as some very rare complications of serious infections,” said Dr. Karp.

The COVID-19 NeuroDatabank/BioBank (NeuroCOVID) is funded by the NINDS. It was created and will be maintained by researchers at NYU Langone Health in New York.

The project is led by Andrea Troxel, ScD, professor of population health, and Eva Petkova, PhD, professor of population health and child and adolescent psychiatry, both at New York University.

“We’ve built a pretty comprehensive database that will accept deidentified patient information about new neurological issues that coincide with their COVID disease or worsening of preexisting neurological problems,” said Dr. Troxel. “In addition, we have a bio repository that will accept almost any kind of biological sample, such as blood, plasma, cerebrospinal fluid, and tissue,” she said.

“Neuroimages are very difficult to store because the files are so enormous, but we’ve had some questions about that, and we’re looking into whether we can accommodate neuroimages,” Dr. Troxel noted.

Dr. Troxel said a “blast of information and invitations” has gone out in an effort to acquire data and biospecimens. “We’ve been really pleased with the amount of interest already, interest not only from large academic medical centers, as you might expect, but also from some smaller stand-alone clinics and even some individuals who have either experienced some of these neurological problems of COVID or know those who have and are really eager to try to provide information,” she added.

Researchers interested in using data and biosamples from the database may submit requests to the NeuroCOVID Steering Committee. More information is available online on the NeuroCOVID website.

A version of this article first appeared on Medscape.com.

The National Institutes of Health has launched a database to track COVID-19–related neurologic symptoms, complications, and outcomes as well as the effects of the virus on preexisting neurologic conditions.

“We know COVID-19 can disrupt multiple body systems, but the effects of the virus and the body’s response to COVID-19 infection on the brain, spinal cord, nerves, and muscle can be particularly devastating and contribute to persistence of disability even after the virus is cleared,” said Barbara Karp, MD, program director at the National Institute of Neurological Disorders and Stroke.

“There is an urgent need to understand COVID-19–related neurological problems, which not uncommonly include headaches, fatigue, cognitive difficulties, stroke, pain, and sleep disorders as well as some very rare complications of serious infections,” said Dr. Karp.

The COVID-19 NeuroDatabank/BioBank (NeuroCOVID) is funded by the NINDS. It was created and will be maintained by researchers at NYU Langone Health in New York.

The project is led by Andrea Troxel, ScD, professor of population health, and Eva Petkova, PhD, professor of population health and child and adolescent psychiatry, both at New York University.

“We’ve built a pretty comprehensive database that will accept deidentified patient information about new neurological issues that coincide with their COVID disease or worsening of preexisting neurological problems,” said Dr. Troxel. “In addition, we have a bio repository that will accept almost any kind of biological sample, such as blood, plasma, cerebrospinal fluid, and tissue,” she said.

“Neuroimages are very difficult to store because the files are so enormous, but we’ve had some questions about that, and we’re looking into whether we can accommodate neuroimages,” Dr. Troxel noted.

Dr. Troxel said a “blast of information and invitations” has gone out in an effort to acquire data and biospecimens. “We’ve been really pleased with the amount of interest already, interest not only from large academic medical centers, as you might expect, but also from some smaller stand-alone clinics and even some individuals who have either experienced some of these neurological problems of COVID or know those who have and are really eager to try to provide information,” she added.

Researchers interested in using data and biosamples from the database may submit requests to the NeuroCOVID Steering Committee. More information is available online on the NeuroCOVID website.

A version of this article first appeared on Medscape.com.

All patients receiving active cancer treatment should receive a COVID-19 vaccine and should be prioritized for vaccination, according to preliminary recommendations from the National Comprehensive Cancer Network (NCCN).

Vaccination timing considerations vary based on factors such as cancer and treatment type, and reasons for delaying vaccination in the general public also apply to cancer patients (recent COVID-19 exposure, for example).

In general, however, patients with cancer should be assigned to Centers for Disease Control and Prevention priority group 1 b/c and immunized when vaccination is available to them, the guidelines state. Exceptions to this recommendation include:

• Patients undergoing hematopoietic stem cell transplant or receiving engineered cellular therapy such as chimeric antigen receptor T-cell therapy. Vaccination should be delayed for at least 3 months in these patients to maximize vaccine efficacy. Caregivers of these patients, however, should be immunized when possible.
• Patients with hematologic malignancies who are receiving intensive cytotoxic chemotherapy, such as cytarabine- or anthracycline-based regimens for acute myeloid leukemia. Vaccination in these patients should be delayed until absolute neutrophil count recovery.
• Patients undergoing major surgery. Vaccination should occur at least a few days before or after surgery.
• Patients who have experienced a severe or immediate adverse reaction to any of the ingredients in the mRNA COVID-19 vaccines.

Conversely, vaccination should occur when available in patients with hematologic malignancies and marrow failure who are expected to have limited or no recovery, patients with hematologic malignancies who are on long-term maintenance therapy, and patients with solid tumors who are receiving cytotoxic chemotherapy, targeted therapy, checkpoint inhibitors and other immunotherapy, or radiotherapy.

Caregivers, household contacts, and other close contacts who are 16 years of age and older should be vaccinated whenever they are eligible.
 

Unique concerns in patients with cancer

The NCCN recommendations were developed to address the unique issues and concerns with respect to patients with cancer, who have an increased risk of severe illness from SARS-CoV-2 infection. But the guidelines come with a caveat: “[t]here are limited safety and efficacy data in these patients,” the NCCN emphasized in a press statement.

“Right now, there is urgent need and limited data,” Steven Pergam, MD, co-leader of the NCCN COVID-19 Vaccination Committee, said in the statement.

“Our number one goal is helping to get the vaccine to as many people as we can,” Dr. Pergam said. “That means following existing national and regional directions for prioritizing people who are more likely to face death or severe illness from COVID-19.”

Dr. Pergam, associate professor at Fred Hutchinson Cancer Research Center in Seattle, further explained that “people receiving active cancer treatment are at greater risk for worse outcomes from COVID-19, particularly if they are older and have additional comorbidities, like immunosuppression.”

NCCN’s recommendations couldn’t have come at a better time for patients with cancer, according to Nora Disis, MD, a professor at the University of Washington in Seattle.

“The NCCN’s recommendations to prioritize COVID vaccinations for cancer patients on active treatment is an important step forward in protecting our patients from the infection,” Dr. Disis said in an interview.

“Cancer patients may be at higher risk for the complications seen with infection. In addition, cancer is a disease of older people, and a good number of our patients have the comorbidities that would predict a poorer outcome if they should become sick,” Dr. Disis added. “With the correct treatment, many patients with cancer will be long-term survivors. It is important that they be protected from infection with COVID to realize their best outcome.”
 

Additional vaccine considerations

The NCCN recommendations also address several other issues of importance for cancer patients, including:

• Deprioritizing other vaccines. COVID-19 vaccines should take precedence over other vaccines because data on dual vaccination are lacking. The NCCN recommends waiting 14 days after COVID-19 vaccination to deliver other vaccines.
• Vaccinating clinical trial participants. Trial leads should be consulted to prevent protocol violations or exclusions.
• Decision-making in the setting of limited vaccine availability. The NCCN noted that decisions on allocation must be made in accordance with state and local vaccine guidance but suggests prioritizing appropriate patients on active treatment, those planning to start treatment, and those who have just completed treatment. Additional risk factors for these patients, as well as other factors associated with risk for adverse COVID-19 outcomes, should also be considered. These include advanced age, comorbidities, and adverse social and demographic factors such as poverty and limited health care access.
• The need for ongoing prevention measures. Vaccines have been shown to decrease the incidence of COVID-19 and related complications, but it remains unclear whether vaccines prevent infection and subsequent transmission. This means everyone should continue following prevention recommendations, such as wearing masks and avoiding crowds.

The NCCN stressed that these recommendations are “intended to be a living document that is constantly evolving – it will be updated rapidly whenever new data comes out, as well as any potential new vaccines that may get approved in the future.” The NCCN also noted that the advisory committee will meet regularly to refine the recommendations as needed.

Dr. Pergam disclosed relationships with Chimerix Inc., Merck & Co., Global Life Technologies Inc., and Sanofi-Aventis. Dr. Disis disclosed grants from Pfizer, Bavarian Nordisk, Janssen, and Precigen. She is the founder of EpiThany and editor-in-chief of JAMA Oncology.

[email protected]

All patients receiving active cancer treatment should receive a COVID-19 vaccine and should be prioritized for vaccination, according to preliminary recommendations from the National Comprehensive Cancer Network (NCCN).

Vaccination timing considerations vary based on factors such as cancer and treatment type, and reasons for delaying vaccination in the general public also apply to cancer patients (recent COVID-19 exposure, for example).

In general, however, patients with cancer should be assigned to Centers for Disease Control and Prevention priority group 1 b/c and immunized when vaccination is available to them, the guidelines state. Exceptions to this recommendation include:

• Patients undergoing hematopoietic stem cell transplant or receiving engineered cellular therapy such as chimeric antigen receptor T-cell therapy. Vaccination should be delayed for at least 3 months in these patients to maximize vaccine efficacy. Caregivers of these patients, however, should be immunized when possible.
• Patients with hematologic malignancies who are receiving intensive cytotoxic chemotherapy, such as cytarabine- or anthracycline-based regimens for acute myeloid leukemia. Vaccination in these patients should be delayed until absolute neutrophil count recovery.
• Patients undergoing major surgery. Vaccination should occur at least a few days before or after surgery.
• Patients who have experienced a severe or immediate adverse reaction to any of the ingredients in the mRNA COVID-19 vaccines.

Conversely, vaccination should occur when available in patients with hematologic malignancies and marrow failure who are expected to have limited or no recovery, patients with hematologic malignancies who are on long-term maintenance therapy, and patients with solid tumors who are receiving cytotoxic chemotherapy, targeted therapy, checkpoint inhibitors and other immunotherapy, or radiotherapy.

Caregivers, household contacts, and other close contacts who are 16 years of age and older should be vaccinated whenever they are eligible.
 

Unique concerns in patients with cancer

The NCCN recommendations were developed to address the unique issues and concerns with respect to patients with cancer, who have an increased risk of severe illness from SARS-CoV-2 infection. But the guidelines come with a caveat: “[t]here are limited safety and efficacy data in these patients,” the NCCN emphasized in a press statement.

“Right now, there is urgent need and limited data,” Steven Pergam, MD, co-leader of the NCCN COVID-19 Vaccination Committee, said in the statement.

“Our number one goal is helping to get the vaccine to as many people as we can,” Dr. Pergam said. “That means following existing national and regional directions for prioritizing people who are more likely to face death or severe illness from COVID-19.”

Dr. Pergam, associate professor at Fred Hutchinson Cancer Research Center in Seattle, further explained that “people receiving active cancer treatment are at greater risk for worse outcomes from COVID-19, particularly if they are older and have additional comorbidities, like immunosuppression.”

NCCN’s recommendations couldn’t have come at a better time for patients with cancer, according to Nora Disis, MD, a professor at the University of Washington in Seattle.

“The NCCN’s recommendations to prioritize COVID vaccinations for cancer patients on active treatment is an important step forward in protecting our patients from the infection,” Dr. Disis said in an interview.

“Cancer patients may be at higher risk for the complications seen with infection. In addition, cancer is a disease of older people, and a good number of our patients have the comorbidities that would predict a poorer outcome if they should become sick,” Dr. Disis added. “With the correct treatment, many patients with cancer will be long-term survivors. It is important that they be protected from infection with COVID to realize their best outcome.”
 

Additional vaccine considerations

The NCCN recommendations also address several other issues of importance for cancer patients, including:

• Deprioritizing other vaccines. COVID-19 vaccines should take precedence over other vaccines because data on dual vaccination are lacking. The NCCN recommends waiting 14 days after COVID-19 vaccination to deliver other vaccines.
• Vaccinating clinical trial participants. Trial leads should be consulted to prevent protocol violations or exclusions.
• Decision-making in the setting of limited vaccine availability. The NCCN noted that decisions on allocation must be made in accordance with state and local vaccine guidance but suggests prioritizing appropriate patients on active treatment, those planning to start treatment, and those who have just completed treatment. Additional risk factors for these patients, as well as other factors associated with risk for adverse COVID-19 outcomes, should also be considered. These include advanced age, comorbidities, and adverse social and demographic factors such as poverty and limited health care access.
• The need for ongoing prevention measures. Vaccines have been shown to decrease the incidence of COVID-19 and related complications, but it remains unclear whether vaccines prevent infection and subsequent transmission. This means everyone should continue following prevention recommendations, such as wearing masks and avoiding crowds.

The NCCN stressed that these recommendations are “intended to be a living document that is constantly evolving – it will be updated rapidly whenever new data comes out, as well as any potential new vaccines that may get approved in the future.” The NCCN also noted that the advisory committee will meet regularly to refine the recommendations as needed.

Dr. Pergam disclosed relationships with Chimerix Inc., Merck & Co., Global Life Technologies Inc., and Sanofi-Aventis. Dr. Disis disclosed grants from Pfizer, Bavarian Nordisk, Janssen, and Precigen. She is the founder of EpiThany and editor-in-chief of JAMA Oncology.

[email protected]

All patients receiving active cancer treatment should receive a COVID-19 vaccine and should be prioritized for vaccination, according to preliminary recommendations from the National Comprehensive Cancer Network (NCCN).

Vaccination timing considerations vary based on factors such as cancer and treatment type, and reasons for delaying vaccination in the general public also apply to cancer patients (recent COVID-19 exposure, for example).

In general, however, patients with cancer should be assigned to Centers for Disease Control and Prevention priority group 1 b/c and immunized when vaccination is available to them, the guidelines state. Exceptions to this recommendation include:

• Patients undergoing hematopoietic stem cell transplant or receiving engineered cellular therapy such as chimeric antigen receptor T-cell therapy. Vaccination should be delayed for at least 3 months in these patients to maximize vaccine efficacy. Caregivers of these patients, however, should be immunized when possible.
• Patients with hematologic malignancies who are receiving intensive cytotoxic chemotherapy, such as cytarabine- or anthracycline-based regimens for acute myeloid leukemia. Vaccination in these patients should be delayed until absolute neutrophil count recovery.
• Patients undergoing major surgery. Vaccination should occur at least a few days before or after surgery.
• Patients who have experienced a severe or immediate adverse reaction to any of the ingredients in the mRNA COVID-19 vaccines.

Conversely, vaccination should occur when available in patients with hematologic malignancies and marrow failure who are expected to have limited or no recovery, patients with hematologic malignancies who are on long-term maintenance therapy, and patients with solid tumors who are receiving cytotoxic chemotherapy, targeted therapy, checkpoint inhibitors and other immunotherapy, or radiotherapy.

Caregivers, household contacts, and other close contacts who are 16 years of age and older should be vaccinated whenever they are eligible.
 

Unique concerns in patients with cancer

The NCCN recommendations were developed to address the unique issues and concerns with respect to patients with cancer, who have an increased risk of severe illness from SARS-CoV-2 infection. But the guidelines come with a caveat: “[t]here are limited safety and efficacy data in these patients,” the NCCN emphasized in a press statement.

“Right now, there is urgent need and limited data,” Steven Pergam, MD, co-leader of the NCCN COVID-19 Vaccination Committee, said in the statement.

“Our number one goal is helping to get the vaccine to as many people as we can,” Dr. Pergam said. “That means following existing national and regional directions for prioritizing people who are more likely to face death or severe illness from COVID-19.”

Dr. Pergam, associate professor at Fred Hutchinson Cancer Research Center in Seattle, further explained that “people receiving active cancer treatment are at greater risk for worse outcomes from COVID-19, particularly if they are older and have additional comorbidities, like immunosuppression.”

NCCN’s recommendations couldn’t have come at a better time for patients with cancer, according to Nora Disis, MD, a professor at the University of Washington in Seattle.

“The NCCN’s recommendations to prioritize COVID vaccinations for cancer patients on active treatment is an important step forward in protecting our patients from the infection,” Dr. Disis said in an interview.

“Cancer patients may be at higher risk for the complications seen with infection. In addition, cancer is a disease of older people, and a good number of our patients have the comorbidities that would predict a poorer outcome if they should become sick,” Dr. Disis added. “With the correct treatment, many patients with cancer will be long-term survivors. It is important that they be protected from infection with COVID to realize their best outcome.”
 

Additional vaccine considerations

The NCCN recommendations also address several other issues of importance for cancer patients, including:

• Deprioritizing other vaccines. COVID-19 vaccines should take precedence over other vaccines because data on dual vaccination are lacking. The NCCN recommends waiting 14 days after COVID-19 vaccination to deliver other vaccines.
• Vaccinating clinical trial participants. Trial leads should be consulted to prevent protocol violations or exclusions.
• Decision-making in the setting of limited vaccine availability. The NCCN noted that decisions on allocation must be made in accordance with state and local vaccine guidance but suggests prioritizing appropriate patients on active treatment, those planning to start treatment, and those who have just completed treatment. Additional risk factors for these patients, as well as other factors associated with risk for adverse COVID-19 outcomes, should also be considered. These include advanced age, comorbidities, and adverse social and demographic factors such as poverty and limited health care access.
• The need for ongoing prevention measures. Vaccines have been shown to decrease the incidence of COVID-19 and related complications, but it remains unclear whether vaccines prevent infection and subsequent transmission. This means everyone should continue following prevention recommendations, such as wearing masks and avoiding crowds.

The NCCN stressed that these recommendations are “intended to be a living document that is constantly evolving – it will be updated rapidly whenever new data comes out, as well as any potential new vaccines that may get approved in the future.” The NCCN also noted that the advisory committee will meet regularly to refine the recommendations as needed.

Dr. Pergam disclosed relationships with Chimerix Inc., Merck & Co., Global Life Technologies Inc., and Sanofi-Aventis. Dr. Disis disclosed grants from Pfizer, Bavarian Nordisk, Janssen, and Precigen. She is the founder of EpiThany and editor-in-chief of JAMA Oncology.

[email protected]

The Biden administration is exploring whether factories can be retrofitted to produce more of the Pfizer/BioNTech and Moderna COVID-19 mRNA vaccines to speed up vaccination of the vast majority of Americans.

The announcement comes as the nation is on track to see 479,000-514,000 deaths by the end of February, said Rochelle Walensky, MD, the director of the Centers for Disease Control and Prevention.

Dr. Walensky, speaking to reporters Wednesday in the first briefing from the White House COVID-19 Response Team, said that 1.6 million COVID-19 shots had been administered each day over the past week and that 3.4 million Americans have been fully vaccinated with two doses.

More than 500 million doses will be needed to vaccinate every American older than 16 years, Andy Slavitt, the senior advisor to the COVID-19 response team, told reporters. Pfizer and Moderna are due to deliver an additional 200 million doses near the end of March, and President Biden is seeking to purchase another 200 million doses from the companies, said Mr. Slavitt.

But it may not be enough. Whether companies can retrofit factories to produce vaccines is “something that’s under active exploration,” Mr. Slavitt said.

“This is a national emergency,” said Jeff Zients, the White House COVID-19 response coordinator. “Everything is on the table across the whole supply chain,” he said. He noted that the administration was also buying low-dead-space syringes to help extract an additional sixth dose from every Pfizer vial.

Mr. Slavitt said the team had identified 12 areas in which Mr. Biden was authorized to use the Defense Production Act to spur the manufacture of items such as masks and COVID-19 diagnostics.
 

More sequencing needed

As new variants emerge, vaccine makers and the CDC are racing to stay a step ahead. “RNA viruses mutate all the time – that’s what they do, that’s their business,” said Anthony Fauci, MD, director of the National Institute of Allergy and Infectious Diseases and Mr. Biden’s chief medical adviser, in the briefing.

Three concerning variants have emerged: the B117, which is circulating widely in the United Kingdom; the B1.351 in South Africa; and the P.1 in Brazil. As of Jan. 26, no cases involving the B1.351 variant have been detected in the United States; one person with the P.1 variant was identified in Minnesota. The CDC has identified 308 cases of the U.K. variant in 26 states, said Dr. Walensky.

The United States is dismally behind in surveillance and sequencing of variants, said Zients. “We are 43rd in the world at genomic sequencing,” which he said was “totally unacceptable.”

Dr. Walensky said the CDC is working on improving data collection and sequencing, but she said more money is needed to “do the amount of sequencing and surveillance that we need in order to be able to detect these when they first start to emerge.”

Both she and Mr. Zients called on Congress to pass Mr. Biden’s proposed American Rescue package, which includes more money for sequencing.

Dr. Fauci said the National Institutes of Health was collaborating with the CDC to determine whether other newly emerging variants pose any threat – such as increased transmissibility or lethality or some other functional characteristic. Scientists will also monitor “in real-time” whether current vaccines continue to make neutralizing antibodies against these mutants.

“With the U.K. variant, what we’re seeing is a very slight, if at all, impact on vaccine-induced antibodies and very little impact on anything else,” he said. With the South African variant, there is “a multifold diminution in the in vitro neutralization by vaccine-induced antibodies,” but “it still is well within the cushion of protection” for the current vaccines.

But, he added, “we have to be concerned looking forward of what the further evolution of this might be.” The anti-COVID monoclonal antibodies – bamlanivimab and the combination of casirivimab and imdevimab – are “more seriously inhibited by this South African strain,” which is spurring development of new monoclonals.

Dr. Fauci also noted that the Johnson & Johnson/Janssen vaccine that is in development – for which phase 3 data may be released within days – was tested in South Africa and Brazil in addition to the United States. The comparative data could help researchers and clinicians make better-informed decisions about what vaccine to use if the South African variant “seeds itself in the U.S.”

A version of this article first appeared on Medscape.com.

The Biden administration is exploring whether factories can be retrofitted to produce more of the Pfizer/BioNTech and Moderna COVID-19 mRNA vaccines to speed up vaccination of the vast majority of Americans.

The announcement comes as the nation is on track to see 479,000-514,000 deaths by the end of February, said Rochelle Walensky, MD, the director of the Centers for Disease Control and Prevention.

Dr. Walensky, speaking to reporters Wednesday in the first briefing from the White House COVID-19 Response Team, said that 1.6 million COVID-19 shots had been administered each day over the past week and that 3.4 million Americans have been fully vaccinated with two doses.

More than 500 million doses will be needed to vaccinate every American older than 16 years, Andy Slavitt, the senior advisor to the COVID-19 response team, told reporters. Pfizer and Moderna are due to deliver an additional 200 million doses near the end of March, and President Biden is seeking to purchase another 200 million doses from the companies, said Mr. Slavitt.

But it may not be enough. Whether companies can retrofit factories to produce vaccines is “something that’s under active exploration,” Mr. Slavitt said.

“This is a national emergency,” said Jeff Zients, the White House COVID-19 response coordinator. “Everything is on the table across the whole supply chain,” he said. He noted that the administration was also buying low-dead-space syringes to help extract an additional sixth dose from every Pfizer vial.

Mr. Slavitt said the team had identified 12 areas in which Mr. Biden was authorized to use the Defense Production Act to spur the manufacture of items such as masks and COVID-19 diagnostics.
 

More sequencing needed

As new variants emerge, vaccine makers and the CDC are racing to stay a step ahead. “RNA viruses mutate all the time – that’s what they do, that’s their business,” said Anthony Fauci, MD, director of the National Institute of Allergy and Infectious Diseases and Mr. Biden’s chief medical adviser, in the briefing.

Three concerning variants have emerged: the B117, which is circulating widely in the United Kingdom; the B1.351 in South Africa; and the P.1 in Brazil. As of Jan. 26, no cases involving the B1.351 variant have been detected in the United States; one person with the P.1 variant was identified in Minnesota. The CDC has identified 308 cases of the U.K. variant in 26 states, said Dr. Walensky.

The United States is dismally behind in surveillance and sequencing of variants, said Zients. “We are 43rd in the world at genomic sequencing,” which he said was “totally unacceptable.”

Dr. Walensky said the CDC is working on improving data collection and sequencing, but she said more money is needed to “do the amount of sequencing and surveillance that we need in order to be able to detect these when they first start to emerge.”

Both she and Mr. Zients called on Congress to pass Mr. Biden’s proposed American Rescue package, which includes more money for sequencing.

Dr. Fauci said the National Institutes of Health was collaborating with the CDC to determine whether other newly emerging variants pose any threat – such as increased transmissibility or lethality or some other functional characteristic. Scientists will also monitor “in real-time” whether current vaccines continue to make neutralizing antibodies against these mutants.

“With the U.K. variant, what we’re seeing is a very slight, if at all, impact on vaccine-induced antibodies and very little impact on anything else,” he said. With the South African variant, there is “a multifold diminution in the in vitro neutralization by vaccine-induced antibodies,” but “it still is well within the cushion of protection” for the current vaccines.

But, he added, “we have to be concerned looking forward of what the further evolution of this might be.” The anti-COVID monoclonal antibodies – bamlanivimab and the combination of casirivimab and imdevimab – are “more seriously inhibited by this South African strain,” which is spurring development of new monoclonals.

Dr. Fauci also noted that the Johnson & Johnson/Janssen vaccine that is in development – for which phase 3 data may be released within days – was tested in South Africa and Brazil in addition to the United States. The comparative data could help researchers and clinicians make better-informed decisions about what vaccine to use if the South African variant “seeds itself in the U.S.”

A version of this article first appeared on Medscape.com.

The Biden administration is exploring whether factories can be retrofitted to produce more of the Pfizer/BioNTech and Moderna COVID-19 mRNA vaccines to speed up vaccination of the vast majority of Americans.

The announcement comes as the nation is on track to see 479,000-514,000 deaths by the end of February, said Rochelle Walensky, MD, the director of the Centers for Disease Control and Prevention.

Dr. Walensky, speaking to reporters Wednesday in the first briefing from the White House COVID-19 Response Team, said that 1.6 million COVID-19 shots had been administered each day over the past week and that 3.4 million Americans have been fully vaccinated with two doses.

More than 500 million doses will be needed to vaccinate every American older than 16 years, Andy Slavitt, the senior advisor to the COVID-19 response team, told reporters. Pfizer and Moderna are due to deliver an additional 200 million doses near the end of March, and President Biden is seeking to purchase another 200 million doses from the companies, said Mr. Slavitt.

But it may not be enough. Whether companies can retrofit factories to produce vaccines is “something that’s under active exploration,” Mr. Slavitt said.

“This is a national emergency,” said Jeff Zients, the White House COVID-19 response coordinator. “Everything is on the table across the whole supply chain,” he said. He noted that the administration was also buying low-dead-space syringes to help extract an additional sixth dose from every Pfizer vial.

Mr. Slavitt said the team had identified 12 areas in which Mr. Biden was authorized to use the Defense Production Act to spur the manufacture of items such as masks and COVID-19 diagnostics.
 

More sequencing needed

As new variants emerge, vaccine makers and the CDC are racing to stay a step ahead. “RNA viruses mutate all the time – that’s what they do, that’s their business,” said Anthony Fauci, MD, director of the National Institute of Allergy and Infectious Diseases and Mr. Biden’s chief medical adviser, in the briefing.

Three concerning variants have emerged: the B117, which is circulating widely in the United Kingdom; the B1.351 in South Africa; and the P.1 in Brazil. As of Jan. 26, no cases involving the B1.351 variant have been detected in the United States; one person with the P.1 variant was identified in Minnesota. The CDC has identified 308 cases of the U.K. variant in 26 states, said Dr. Walensky.

The United States is dismally behind in surveillance and sequencing of variants, said Zients. “We are 43rd in the world at genomic sequencing,” which he said was “totally unacceptable.”

Dr. Walensky said the CDC is working on improving data collection and sequencing, but she said more money is needed to “do the amount of sequencing and surveillance that we need in order to be able to detect these when they first start to emerge.”

Both she and Mr. Zients called on Congress to pass Mr. Biden’s proposed American Rescue package, which includes more money for sequencing.

Dr. Fauci said the National Institutes of Health was collaborating with the CDC to determine whether other newly emerging variants pose any threat – such as increased transmissibility or lethality or some other functional characteristic. Scientists will also monitor “in real-time” whether current vaccines continue to make neutralizing antibodies against these mutants.

“With the U.K. variant, what we’re seeing is a very slight, if at all, impact on vaccine-induced antibodies and very little impact on anything else,” he said. With the South African variant, there is “a multifold diminution in the in vitro neutralization by vaccine-induced antibodies,” but “it still is well within the cushion of protection” for the current vaccines.

But, he added, “we have to be concerned looking forward of what the further evolution of this might be.” The anti-COVID monoclonal antibodies – bamlanivimab and the combination of casirivimab and imdevimab – are “more seriously inhibited by this South African strain,” which is spurring development of new monoclonals.

Dr. Fauci also noted that the Johnson & Johnson/Janssen vaccine that is in development – for which phase 3 data may be released within days – was tested in South Africa and Brazil in addition to the United States. The comparative data could help researchers and clinicians make better-informed decisions about what vaccine to use if the South African variant “seeds itself in the U.S.”

A version of this article first appeared on Medscape.com.

From the Milwaukee County Behavioral Health Division, Milwaukee, WI.

Abstract

Objective: Implementation of the Quadruple Aim of health care must begin with a clearly articulated set of concepts, or core domains (CDs), that comprise each aim. These CDs can then be operationalized with existing or new measures. If aligned to the organization’s mission and strategic goals, these CDs have the potential to focus quality improvement activities and reduce measurement burden. This article represents the efforts of a publicly funded behavioral health system to operationalize the Quadruple Aim through the development of CDs.

Methods: Various stakeholders across the organization were consulted on their perceptions of the Quadruple Aim and the CDs they believed should support it. Then, a review of existing literature on core metrics for health care and population health was completed, summarized, and integrated with the stakeholder feedback.

Results: These efforts led to the development and adoption of 15 CDs, with an accompanying literature review and set of recommendations of new and existing measures for each domain.

Conclusions: It is possible to create a comprehensive yet economical set of CDs and attendant measures that can be implemented in a staged, scalable, enterprise manner. It is hoped that the process articulated here, and the accompanying literature review, may be of some benefit to other public or government-run health systems in their own quality improvement journey to operationalize the Quadruple Aim by developing a set of CDs.

Keywords: quality measures; quality improvement; adult behavioral health.

First articulated in 2008, the Triple Aim proposes that health care systems should simultaneously seek to improve the patient’s experience of care, improve the health of populations, and reduce the per capita costs of care for populations.¹ More recently, some have argued that health care provider burnout can deleteriously impact the attainment of the Triple Aim and have therefore advocated for an expanded focus to include a fourth Aim, the work life quality of the staff.² Milwaukee County Behavioral Health Division (BHD), a publicly funded, county-based behavioral health care system in Milwaukee, Wisconsin, recently adopted the Quadruple Aim as the framework by which it will organize its quality activities.

Although originally developed for medical organizations, BHD believes that the Quadruple Aim has strong applicability to county-level behavioral health services. Many county-based behavioral health divisions provide a variety of programs to large segments of the county based on financial eligibility and/or clinical need, and thus often have responsibilities to populations or subpopulations, rather than programs. County health divisions, such as Milwaukee County’s Department of Health and Human Services, are often asked to improve outcomes and client experience of care with neutral growth budgets and less reliance on taxes to fund programs, while simultaneously attracting and retaining competent staff.

Crucial to the effective implementation of the Quadruple Aim, however, is a clear set of population- level measures that help organizations assess their progress.³ Unfortunately, as some authors have noted, evaluation of the Quadruple Aim remains a challenge because the “concepts of (population) health, quality of care and costs are not unanimously defined and measures for these concepts are under construction.”⁴ Several authors have provided some guidance to assist in the development of a set of measures that effectively capture the elements of the Quadruple Aim.^5,6 However, the recent rapid proliferation of quality measures in health care^7,8 has been both burdensome and costly for providers.^9,10 Any measures adopted should not only be as meaningful as possible with regards to assessing progress towards the basic aims of health care, but should also be parsimonious, to limit measurement burden for providers (and patients) and focus attention on important issues.^11,12

To select the most effective, parsimonious set of measures possible, one must first select a set of key foci from among the many possible areas of focus that the core measure is intended to represent. The core domains (CDs), if appropriately consistent with the strategic goals of the organization, provide a mechanism to orient the efforts of the organization at every level and help every staff member of the organization understand how his or her work impacts the progress towards these goals.¹¹ The CDs, therefore, represent the opportunity to affect a greater integration of efforts across the organization toward these shared aims, creating uniformity of purpose at every level. Further, increasing organizational attention on the CDs can also help to reduce measurement burden by streamlining and focusing the data capture processes on the most valuable elements of quality and health, and discarding other extraneous measures (albeit not at the expense of other reporting requirements).¹¹ The remainder of this article describes the CDs selected by BHD to assess its progress toward implementation of the Quadruple Aim and are organized by the Aim which they best represent.

Methods

To effectively implement the Quadruple Aim at BHD, it was necessary to clearly define the subpopulation of focus for our efforts.⁶ In this case, the subpopulation of interest was defined as all adult clients (18 years and older) who received at least 1 service encounter within a specified time frame from a program that BHD either operated or contracted with to provide care. Services provided by the BHD network include everything from psychiatric inpatient services to mental health and addiction treatment and care management. A limited array of social services, including housing and employment services, is also available to eligible consumers. BHD is the county-run behavioral health provider for individuals who are uninsured or underinsured in Milwaukee County, a demographically diverse, primarily urban county of approximately 950,000 people located in Wisconsin. Approximately 15,000 adults receive services at BHD each year.

This work began by obtaining executive sponsorship for the project, in this case from the Chief Operations Officer and Executive Medical Director of BHD. With their backing, an initial review of the literature produced a preliminary set of possible domains, for which we created working definitions. We then made a list of key stakeholders throughout BHD to whom we needed to present the idea of the Quadruple Aim, and the CDs under each Aim, to secure their support. These stakeholders, which included individuals involved in quality activities, program managers, and executive leadership, were strategically selected based on their relative influence within the organization. A set of brief presentations and handouts explaining the project were then developed and shared at different focus groups with these stakeholders over the course of 6 months. These focus groups served to not only educate the organization about the Quadruple Aim and the CDs but afforded participants an opportunity to provide feedback as well.

During the focus groups, we asked participants which domains they believed were most important (were “core”) when operationalizing the Quadruple Aim. The focus groups provided feedback on the domain definitions, feedback that was used to develop uniform, mutually agreed upon definitions for the CDs that were generalizable to all departments at BHD, regardless of the focus of their services within the continuum of care or the continuum of age. This was a crucial step, as it will eventually enable BHD to aggregate data across departments, even if there are minor discrepancies in the specific items they use to assess the CDs. Comments from the focus groups ultimately resulted in a truncated list of domains and definitions, which, coupled with the literature review, resulted in our final set of CDs.

During our review of the literature, we also looked for items that we felt could best represent each CD in the briefest, most meaningful way. (These items were not meant to supersede existing data, but to provide examples that could be implemented with existing data or recommendations that could be utilized in the absence of existing data.) During this process, we made every effort to make use of existing data-reporting requirements. For example, if we had a state mandate to collect data on housing status, we attempted to leverage this required data point to represent the CD related to housing. In other cases, we attempted to utilize claims or other administrative data to operationalize the CD, such as in the cost-of-care metric articulated in the section the Third Aim. For CDs for which no data existed or were insufficient, we emphasized the use of single- versus multi-item scales. For example, if we found a single-item global assessment of quality of life that had good psychometric properties relative to its longer parent scale, we selected the single item. This approach to item selection allowed us to create the most efficient, parsimonious set of measures possible, which we believed would enable us to comprehensively assess all the CDs with the least amount of burden to staff and clients. These items were presented at stakeholder focus groups, during which we asked for comments on the existing measures in their program or department and gave them the opportunity to comment on the new recommended measures.

A working definition is provided for each CD, followed by a brief review of the research base supporting its inclusion in the final list. The item(s) selected by BHD to represent each CD and the source of the item(s) are then supplied. These items were based either on measures currently collected because of existing reporting mandates or, in the case where extant measures were not available, on new items that demonstrated acceptable psychometric properties in the research literature. The CDs and items are organized by the Aim they best represent. A full list of the CDs by Quadruple Aim and items by CD is provided in the Appendix of the online version of this article. This article concludes with a brief summary of this effort and a discussion of how staff will utilize these items at different levels throughout the BHD system.

The First Aim: Population Health

Health Outcomes

Deaths. This can be defined as the cause of death, as determined by the medical examiner’s office (where appropriate) or as the age at time of death. This CD can also be reported as proportion of deaths considered premature (eg, before age 75) or calculated as total years of potential life lost.

Brief review and suggested item(s). Rates and causes of premature mortality are critical foci for the County Health Rankings & Roadmaps,¹³ the Institute for Healthcare Improvement’s “Guide to Measuring the Triple Aim,”⁶ the Centers for Disease Control and Prevention’s “Community Health Assessment for Population Health Improvement,”¹⁴ and the Institute of Medicine’s (IOM) “Vital Signs: Core Metrics for Health and Health Care Progress.”¹¹ There is ample evidence that individuals with serious mental illness are at increased risk of early mortality relative to the general population,^15-18 and this risk applies to those with substance use disorders as well.^15,19-20 BHD tracks all deaths that occur while patients are receiving BHD-funded, community-based services.

Self-Reported Health and Well-Being. This CD asks patients to rate their current physical and mental health status, as well as their overall quality of life.

Brief review and suggested item(s): Self-rated physical health. Premature mortality among individuals with behavioral health issues appears to be due, in large part, to their increased vulnerability to the development of medical comorbidities.^16,21 A single self-rating question has demonstrated considerable sensitivity to premature mortality,^22,23 with predictive properties up to a decade prior to death.^24,25 Further, self-rated health has been associated with subsequent functional decline,^26,27 acute service utilization,^28,29 and overall health care costs.²⁸

Brief review and suggested item(s): Self-rated mental health. Mental health disorders are associated with significant disability worldwide,³⁰ and comorbid mental health issues can exacerbate the course of other medical problems. For example, depression is associated with increased rates of mortality among individuals with diabetes and³¹ cardiovascular disease,³² as well as with rates of overall mortality,³³ and psychiatric comorbidity is associated with longer lengths of stay and higher costs among patients hospitalized for medical problems.³⁴ Research has found that a single-item measure of self-rated mental health is associated with the presence of psychiatric diagnoses, psychiatric symptoms, and subsequent depression and serious mental illness up to 1 year post-assessment.^35,36 There is even evidence that self-rated mental health may be more strongly associated with self-ratings of overall health than self-ratings of physical health.³⁷

Brief review and suggested item(s): Self-rated quality of life. Quality of life is a critical component of the recovery journey and overall health.³⁸ For example, the County Health Rankings & Roadmaps lists “quality of life” as 1 of its key “health outcomes” in its County Health Rankings.¹³ As some authors have noted, quality of life is often inferred from other “objective” recovery domains, such as employment, health status, or housing status. However, there is evidence that these objective domains are functionally distinct from the inherently subjective construct of quality of life.³⁹ This has led other authors to conclude that these domains should be assessed separately when evaluating outcomes.⁴⁰ Single-item quality of life assessments have been used in research with individuals with cancer,⁴¹ adults with disabilities,⁴² patients with cystic fibrosis,⁴³ and children with epilepsy.⁴⁴ For this effort, BHD selected the first global quality of life item from the World Health Organization’s WHOQOL-BREF quality of life assessment,⁴⁵ an item used in other quality of life research.⁴⁶

Health Factors

Substance Use. This CD is a composite of 4 different types of substance use, any recent heavy alcohol use (defined as 5 or more drinks in one sitting), any recent drug use, any recent prescription drug abuse, and any recent tobacco use.

Brief review and suggested item(s). As noted, substance use disorders confer an increased risk for early mortality^15,19 and are significantly implicated in disease disability burden worldwide.³⁰ Substance use has also been associated with both the onset^47,48 and exacerbation of mental health diagnoses.^49-51 Further, substance use appears to heighten the risk of violence in the general population⁵² and especially among those with a co-occurring mental illness.^53,54 The County Health Rankings & Roadmaps list alcohol and drug use as key behaviors to address to improve the overall health of a given county,¹³ and the Centers for Medicare & Medicaid Services (CMS) has endorsed initiation and engagement in addiction treatment as one of the measures in its Adult Core Set.⁵⁵

Tobacco use continues to be one of the most significant risk factors for early mortality worldwide, and evidence indicates that it is associated with a lower life expectancy of nearly 10 years.⁵⁶ Unfortunately, rates of tobacco use are even higher among those with severe mental illness relative to the general population, and their rates of smoking cessation are lower.^57,58 Tobacco use is a significant risk factor for the high rates of early mortality in individuals with severe mental illness.¹⁸ Further, a recent meta-analysis noted that, relative to those who continued to smoke, those who ceased smoking had reduced rates of psychological distress and increased quality of life rankings.⁵⁹ Reducing tobacco use is one of the key components of the County Health Rankings & Roadmaps, and medication assistance with smoking and tobacco use cessation is also listed in the CMS Adult Core Set.^13,55

An accumulating body of evidence suggests that single-item measures can adequately detect alcohol^60-62 and drug use disorders.^60-64 McNeely and colleagues recently developed and tested a brief 4-item screen, the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool.^65,66 Preliminary evidence suggests that the TAPS tool can effectively identify the presence of problematic and disordered use of tobacco, alcohol, prescription medications, and other drugs.^65-67 BHD will use the 4 items from the TAPS tool to represent its substance use CD.

Education/Employment Status. This CD assesses the proportion of BHD members who have completed high school, who are in some type of educational or training program, or who are engaged in some type of employment activity (defined as full-time, part-time, supported, sheltered workshop, or as a full-time homemaker).

Brief review and suggested item(s). Research indicates that unemployment is a risk factor for mortality, even after controlling for other risk factors (eg, age, sex, socioeconomic status [SES], health).⁶⁸ Unemployment is associated with poorer physical and mental health in the general population and among those with disabilities.^69-71 Promisingly, evidence suggests that gaining employment or re-employment is associated with better health,⁷² even for individuals with substance use disorders⁷³ or moderate⁷⁴ to severe mental health disorders.^75-78 Some authors have even proposed that, above and beyond the associated health benefits, employment may also help to realize a modest cost savings due to reduced service utilization and disability.^79,80 Employment is a core tenet in the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) model of recovery,⁸¹ and is also listed as an important recovery goal for individuals with behavioral health issues.⁸² BHD collects data on employment status on all the patients it serves as part of its state-mandated reporting requirements and will use this item in the CD data set.⁸³

Living Situation. This is measured as the proportion of people who live in permanent, supportive, stable housing; it may also be measured as the percentage of the population living with severe housing problems or who are homeless.

Brief review and suggested item(s). Housing problems can be conceptualized as 3 inter-related components: conditions within the home, neighborhood conditions, and housing affordability, each of which can contribute uniquely to poorer physical and mental health of individuals and families⁸⁴ and to educational outcomes for children.^85,86 Further, individuals who are homeless have a standardized mortality ratio 2 to 5 times that of the general population,^87-89 even after controlling for low income status,⁹⁰ and some evidence suggests these rates are even higher among unsheltered versus sheltered homeless individuals.⁹¹ Interventions to improve the condition of housing have demonstrated positive impacts on both physical and mental health,⁹² and a recent study found that individuals receiving housing assistance in the form of public housing or multifamily housing from the Department of Housing and Urban Development had better self-rated physical and mental health relative to individuals on the wait list for housing assistance.⁹³ Moreover, the provision of housing has been shown to promote reductions in substance use and health service utilization among homeless individuals with substance use disorders.⁹⁴ Rog and colleagues reviewed the literature on permanent supportive housing for individuals with substance use or mental health disorders who were homeless or disabled, and found that provision of housing led to reduced rates of homelessness, emergency department (ED) and inpatient utilization and increased consumer satisfaction.⁹⁵

Importantly, evidence suggests that housing is viewed as facilitative of recovery. For example, in a recent qualitative study of homeless individuals with mental illness, housing was seen as a critical first step in recovery, providing a sense of security, increasing feelings of personal independence and autonomy, improving perceptions of health and well-being, and affording a stable environment to rebuild relationships with important others.⁹⁶ BHD collects data on housing status on all the patients it serves as part of its state-mandated reporting requirements and will utilize this item in the CD data set.⁸³

Social Relationships. This is defined as recent interactions with family, supportive networks (formal and informal), and other recovery services.

Brief review and suggested item(s). Research has long established that social relationships have a significant impact on health, including rates of mortality as well as physical and mental health morbidity.^97-99 Social connectedness is another of the pillars supporting an individual’s recovery in SAMHSA’s formulation. Several reviews of the recovery literature^38,82 support its importance to the recovery process and inclusion in any assessment of holistic recovery. Social support has been shown to promote recovery among individuals with severe mental illness^100-102 and substance use disorders,¹⁰³ and may mitigate the progression of chronic, life-threatening physical illnesses.⁹⁷ For the purposes of BHD’s CD data set, the social support question from the “100 Million Healthier Lives Common Questionnaire for Adults” will be used to assess individuals’ perceived adequacy of social support.¹⁰⁴

Legal Involvement. Defined as involvement with the civil or criminal justice system, including arrests, imprisonment, or detainment.

Brief review and suggested item(s). Involvement in the criminal justice system is both disruptive for the individual in recovery and expensive to the larger health care system.¹⁰⁵ Individuals with substance use¹⁰⁶ and severe mental health disorders¹⁰⁷ are over-represented in the prison system, and evidence suggests that general physical and mental health declines while individuals are in prison.^108,109 Perhaps even more concerning, numerous studies have demonstrated an increase in mortality rates for individuals recently released from prison relative to the general population, particularly during the period immediately following release.^108-110 This relationship may even persist long term.¹¹¹ Further, research indicates that individuals recently released from prison have increased emergency care and hospital utilization.^112,113

Incarceration can have significant impacts on the health of the broader community as well. For example, research has found an association between parental incarceration to rates of infant mortality,¹¹⁴ increased behavioral and developmental problems of children of incarcerated parents,^115,116 lower rates of child support payments,¹¹⁷ and poorer cardiovascular health of female partners of incarcerated individuals.¹¹⁸ Formerly incarcerated individuals experience slower wage growth as well.¹¹⁹ However, evidence also indicates that engagement in mental health¹²⁰ and substance abuse¹²¹ treatment can reduce the likelihood of subsequent recidivism. As part of its state-mandated reporting, BHD is required to provide information on the criminal justice system involvement of its clients in the previous 6 months, including whether they have been jailed or imprisoned,⁸³ and this will function as its measure of legal involvement in its CD data set.

Socioeconomic Status. Socioeconomic status is the social standing or class of an individual or group. It is often measured as a combination of education, income, and occupation. It can also be defined subjectively, such as one’s evaluation of status relative to similar others or based on an individual’s interpretation of her or his financial needs.

Brief review and suggested item(s). A large body of evidence supports the existence of a robust relationship between lower SES and poor health, including mortality and chronic medical diseases,^122-124 as well as mental illness.^125-127 Although previous research has examined this relationship using objective indicators of SES (eg, income, education level, occupation), there has recently been an increased interest in exploring the relationship of subjective SES with health indices. Subjective SES is generally assessed by asking individuals to rate themselves relative to others in the society in which they live, in terms of wealth, occupation, educational level, or other indicators of social status. Evidence suggests that subjective SES is associated with objective measures of SES,^128-130 and relates to measures of physical and mental health as well, even after controlling for objective SES.^130-135 BHD will be using a modified version of the Subject SES Scale,^131,135 which is deployed in the “100 Million Healthier Lives Common Questionnaire for Adults.”¹⁰⁴

Acute Service Use. This is defined as an admission to a medical or psychiatric emergency room or to a medical or psychiatric hospital or to a detoxification facility.

Brief review and suggested item(s). The CMS Adult Core Set includes “plan all cause readmissions” as a key quality metric.⁵⁵ Hospital readmissions are also endorsed by the National Committee on Quality Assurance as one of its Health Effectiveness Data and Information Set (HEDIS) measures and by the National Quality Forum. Readmissions, despite their widespread endorsement, are a somewhat controversial measure. Although readmissions are costly to the health care system,¹³⁶ the relationship between readmissions and quality is inconsistent. For example, Krumholz and colleagues¹³⁷ found differential rates of readmission for the same patient discharged from 2 different hospitals, which were categorized based on previous readmission rates, suggesting that hospitals do have different levels of performance even when treating the same patient. However, other data indicate that 30-day, all-cause, risk-standardized readmission rates are not associated with hospital 30-day, all-cause, risk-standardized mortality rates.¹³⁸

Chin and colleague found that readmissions to the hospital that occurred more than 7 days post-discharge were likely due to community- and household-related factors, rather than hospital-related quality factors.¹³⁹ Transitional care interventions that have successfully reduced 30-day readmission rates are most often multicomponent and focus not just on hospital-based interventions (eg, discharge planning, education) but on follow-up care in the community by formal supports (eg, in-home visits, telephone calls, outpatient clinic appointments, case management) and informal supports (eg, family and friends).^140-143 Further, qualitative evidence suggests that some individuals perceive psychiatric hospitalizations to be the result of insufficient resources or unsuccessful attempts to maintain their stability in the community.¹⁴⁴ Thus, unplanned or avoidable hospital readmissions may represent a failure of the continuum of care not only from the perspective of the health care system, but from the patient perspective as well.

Frequent or nonurgent use of EDs is conceptually similar to excessive or avoidable inpatient utilization in several ways. For example, overuse of EDs is costly, with some estimates suggesting that it is responsible for up to $38 billion in wasteful spending each year.¹⁴⁵ Individuals with frequent ED visits have a greater disease burden¹⁴⁶ and an increased risk of mortality compared to nonfrequent users.¹⁴⁷ Research suggests that individuals who visit the ED for non-urgent issues do so because of perceived difficulties associated with accessing primary care, and the convenience of EDs relative to primary care.^148-150 Moreover, similar to the hospital readmission literature discussed earlier, successful strategies to reduce high rates of ED utilization generally focus on continuum of care interventions, such as provision of case management services.^151-155

This evidence implies that frequent ED utilization and hospital readmissions may not be a fundamental issue of quality (or lack thereof) in hospitals or EDs but rather a lack of, or ineffectual, transitional and continuum of care strategies and services. To underscore this point, some authors have argued that a system that is excessively crisis-oriented hinders recovery because it is reactive rather than proactive, predicated on the notion that one’s condition must deteriorate to receive care.¹⁵⁶

Although some organizations may have access to claims data or may function as self-contained health systems (eg, the Veterans Health Administration [VHA] ), others may not have access to such data. In the absence of claims data, patient self-report of service utilization has been used as a proxy for actual agency records.¹⁵⁷ Although concordance between medical and/or agency records and patient self-report has been variable,¹⁵⁷ evidence generally suggests that rates of agreement are higher the shorter the recall time interval.^158,159 BHD does not have access to comprehensive claims data and has therefore chosen to use 5 dichotomously scored (yes/no) questions—related to medical inpatient, medical ED, psychiatric inpatient, psychiatric ED, and detoxification use in the last 30 days—to represent the CD of acute service utilization.

The Second Aim: Quality of Care

Safety

Safety is defined as avoiding injuries to patients from the care that is intended to help them.

Brief review and suggested item(s). As noted in “Crossing the Quality Chasm,” the IOM’s seminal document, “the health care environment should be safe for all patients, in all of its processes, all the time.”¹⁶⁰ The landmark Harvard Medical Practice Study in 1991 found that adverse events occurred in nearly 4% of all hospital admissions and, among these, over a quarter were due to negligence.¹⁶¹ Other estimates of adverse events range as high as 17%.¹⁶² Indeed, a recent article by Makary and Daniel estimated that medical errors may be the third leading cause of death in the United States.¹⁶³ Unfortunately, research on safety in the mental health field has lagged behind that of physical health,¹⁶⁴ with evidence indicating that research in nonhospital settings in mental health care may be particularly scarce.¹⁶⁵ In a study of adverse events that occurred in psychiatric inpatient units in the VHA system between 2015 and 2016, Mills and colleagues found that of the 87 root cause analysis reports, suicide attempts were the most frequent, and, among safety events, falls were the most frequently reported, followed by medication events.¹⁶⁶ Another report on data collected from psychiatric inpatient units in the VHA revealed that nearly one-fifth of patients experienced a safety event, over half of which were deemed preventable.¹⁶⁷ These numbers likely represent an underestimation of the true volume of safety events, as another study by the same research group found that less than 40% of safety events described in patient medical records were documented in the incident reporting system.¹⁶⁸ BHD will utilize the total number of complaints and incident reports submitted within a given time frame as its “safety” metric in the CD data set.

Wait Time for Service

The CD is defined as the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Brief review and suggested item(s). “Timeliness” was listed among the 6 aims for improvement in “Crossing the Quality Chasm” in 2001, and it remains no less relevant today.¹⁶⁰ For example, evidence indicates that access to primary care is inversely related to avoidable hospitalizations.¹⁶⁹ One study found that, of patients hospitalized for cardiovascular problems, those who had difficulty accessing routine care post discharge had higher 30-day readmission rates.¹⁷⁰ Among VHA patients, longer wait times are associated with more avoidable hospitalizations and higher rates of mortality.¹⁷¹ Longer wait times appear to decrease the likelihood of attending a first appointment for individuals with substance use^172,173 and mental health disorders.¹⁷⁴ Importantly, longer wait times are associated with lower ratings of the patient experience of care, including perceptions of the quality of and satisfaction with care,¹⁷⁵ and may be associated with worse outcomes for individuals in early intervention for psychosis treatment.¹⁷⁶ For the purposes of the CD data set, BHD will monitor the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Patient satisfaction is defined as the degree of patients’ satisfaction with the care they have received.

Brief review and suggested item(s). Research has consistently demonstrated the relationship of the patient’s experience of care to a variety of safety and clinical effectiveness measures in medical health care,¹⁷⁷ and the therapeutic alliance is one of the most consistent predictors of outcomes in behavioral health, regardless of therapeutic modality.¹⁷⁸ Patient satisfaction is a commonly assessed aspect of the patient experience of care. Patient satisfaction scores have been correlated with patient adherence to recommended treatment regimens, care quality, and health outcomes.¹⁷⁹ For example, Aiken et al found that patient satisfaction with hospital care was associated with higher ratings of the quality and safety of nursing care in these hospitals.¹⁸⁰ Increased satisfaction with inpatient care has been associated with lower 30-day readmission rates for patients with acute myocardial infarction, heart failure, and pneumonia,¹⁸¹ and patients with schizophrenia who reported higher treatment satisfaction also reported better quality of life.^182,183 Many satisfaction survey options exist to evaluate this CD, including the Consumer Assessment of Healthcare Providers and Systems and the Client Satisfaction Questionnaire; BHD will utilize an outpatient behavioral health survey from a third-party vendor.

The Third Aim: Cost of Care

Cost of Care

This can be defined as the average cost to provide care per patient per month.

Brief review and suggested item(s). Per capita cost, or rather, the total cost of providing care to a circumscribed population divided by the total population, has been espoused as an important metric for the Triple Aim and the County Health Rankings.^6,13 Indeed, between 1960 and 2016, per capita expenditures for health care have grown 70-fold, and the percent of the national gross domestic product accounted for by health expenditures has more than tripled (5.0% to 17.9%).¹⁸⁴ One of the more common metrics deployed for assessing health care cost is the per capita per month cost, or rather, the per member per month cost of the predefined population for a given health care system.^6,185,186 In fact, some authors have proposed that cost of care can be used not only to track efficient resource allocation, but can also be a proxy for a healthier population as well (ie, as health improves, individuals use fewer and less-expensive services, thus costing the system less).¹⁸⁷ To assess this metric, BHD will calculate the total amount billed for patient care provided within BHD’s health network each month (irrespective of funding source) and then divide this sum by the number of members served each month. Although this measure does not account for care received at other health care facilities outside BHD’s provider network, nor does it include all the overhead costs associated with the care provided by BHD itself, it is consistent with the claims-based approach used or recommended by other authors.^6,188

The Fourth Aim: Staff Well-being

Staff Quality of Work Life

This can be defined as the quality of the work life of health care clinicians and staff.

Brief review and suggested item(s). Some authors have suggested that the Triple Aim framework is incomplete and have proffered compelling arguments that provider well-being and the quality of work life constitutes a fourth aim.² Provider burnout is prevalent in both medical^2,189 and behavioral health care.^190,191 Burnout among health care professionals has been associated with higher rates of perceived medical errors,¹⁹² lower patient satisfaction scores,^189,193 lower rates of provider empathy,¹⁹⁴ more negative attitudes towards patients,¹⁹⁵ and poorer staff mental and physical health.¹⁹¹

Burnout is also associated with higher rates of absenteeism, turnover intentions, and turnover.^{190,191,196,197} However, burnout is not the only predictor of staff turnover; for example, turnover rates are a useful proxy for staff quality of work life for several reasons.¹⁹⁸ First, turnover is associated with substantial direct and indirect costs, including lost productivity, increased errors, and lost revenue and recruitment costs, with some turnover cost estimates as high as $17 billion for physicians and $14 billion for nurses annually.^199-201 Second, research indicates that staff turnover can have a deleterious impact on implementation of evidence-based interventions.^202-205 Finally, consistent with the philosophy of utilizing existing data sources for the CD measures, turnover can be relatively easily extracted from administrative data for operated or contracted programs, and its collection does not place any additional burden on staff. As a large behavioral health system that is both a provider and payer of care, BHD will therefore examine the turnover rates of its internal administrative and clinical staff as well as the turnover of staff in its contracted provider network as its measures for the Staff Quality of Work Life CD.

Clinical Implications

These metrics can be deployed at any level of the organization. Clinicians may use 1 or more of the measures to track the recovery of individual clients, or in aggregate for their entire caseload. Similarly, managers can use these measures to assess the overall effectiveness of the programs for which they are responsible. Executive leaders can evaluate the impact of several programs or the system of care on the health of a subpopulation of clients with a specific condition, or for all their enrolled members. Further, not all measures need be utilized for every dashboard or evaluative effort. The benefit of a comprehensive set of measures lies in their flexibility—1 or more of the measures may be selected depending on the project being implemented or the interests of the stakeholder.

It is important to note that many of the CDs (and their accompanying measures) are aligned to/consistent with social determinants of health.^206,207 Evidence suggests that social determinants make substantial contributions to the overall health of individuals and populations and may even account for a greater proportion of variance in health outcomes than health care itself.²⁰⁸ The measures articulated here, therefore, can be used to assess whether and how effectively care provision has addressed these social determinants, as well as the relative impact their resolution may have on other health outcomes (eg, mortality, self-rated health).

These measures can also be used to stratify clients by clinical severity or degree of socioeconomic deprivation. The ability to adjust for risk has many applications in health care, particularly when organizations are attempting to implement value-based purchasing models, such as pay-for-performance contracts or other alternative payment models (population health-based payment models).²⁰⁹ Indeed, once fully implemented, the CDs and measures will enable BHD to more effectively build and execute different conceptual models of “value” (see references 210 and 211 for examples). We will be able to assess the progress our clients have made in care, the cost associated with that degree of improvement, the experience of those clients receiving that care, and the clinical and social variables that may influence the relative degree of improvement (or lack thereof). Thus, the CDs provide a conceptual and data-driven foundation for the Quadruple Aim and any quality initiatives that either catalyze or augment its implementation.

This article provides an overview of the CDs selected by BHD to help organize, focus, advance, and track its quality efforts within the framework of the Quadruple Aim. Although items aligned to each of these CDs are offered, the CDs themselves have been broadly conceptualized such that they can flexibly admit a variety of possible items and/or assessments to operationalize each CD and thus have potential applicability to other behavioral health systems, particularly public systems that have state-mandated and other data reporting requirements.

Bearing in mind the burden that growing data collection requirements can have on the provision of quality care and staff work satisfaction and burnout,^10,212 the CDs (and the items selected to represent each) are designed with “strategic parsimony” in mind. Although the CDs are inclusive in that they cover care quality, cost of care, staff quality of life, and general population health, only CDs and items undergirded by a solid evidence base and high value with regards to BHD’s mission and values, as determined by key stakeholders, were selected. Moreover, BHD attempted to make use of existing data collection and reporting mandates when selecting the final pool of items to reduce the measurement burden on staff and clients. Thus, the final set of CDs and items are designed to be comprehensive yet economical.

The CDs are deeply interrelated. Although each CD may be individually viewed as a valuable metric, improvements in any 1 CD will impact the others (eg, increasing care quality should impact population health, increasing staff quality of life should impact the quality of care). Moreover, this idea of interrelatedness acknowledges the need to view health systems and the populations they serve holistically, in that improvement is not simply the degree of change in any given metric (whether individually or collectively), but rather something more entirely. The concepts of value, quality, and health are complex, multidimensional, and dynamic, and the CDs that comprise these concepts should not be considered independently from one another. The CDs (and items) offered in this article are scalable in that they can be used at different levels of an organization depending on the question or stakeholder, and can be used individually or in combination with one another. Moreover, they are adaptable to a variety of risk-adjusted program, population health, and value-based evaluation models. It is hoped that the process articulated here, and the accompanying literature review, may benefit other public or government-run health systems in their own quality journey to operationalize the Quadruple Aim by developing a set of CDs.

Corresponding author: Walter Matthew Drymalski, PhD; [email protected].

Financial disclosures: None.

From the Milwaukee County Behavioral Health Division, Milwaukee, WI.

Abstract

Objective: Implementation of the Quadruple Aim of health care must begin with a clearly articulated set of concepts, or core domains (CDs), that comprise each aim. These CDs can then be operationalized with existing or new measures. If aligned to the organization’s mission and strategic goals, these CDs have the potential to focus quality improvement activities and reduce measurement burden. This article represents the efforts of a publicly funded behavioral health system to operationalize the Quadruple Aim through the development of CDs.

Methods: Various stakeholders across the organization were consulted on their perceptions of the Quadruple Aim and the CDs they believed should support it. Then, a review of existing literature on core metrics for health care and population health was completed, summarized, and integrated with the stakeholder feedback.

Results: These efforts led to the development and adoption of 15 CDs, with an accompanying literature review and set of recommendations of new and existing measures for each domain.

Conclusions: It is possible to create a comprehensive yet economical set of CDs and attendant measures that can be implemented in a staged, scalable, enterprise manner. It is hoped that the process articulated here, and the accompanying literature review, may be of some benefit to other public or government-run health systems in their own quality improvement journey to operationalize the Quadruple Aim by developing a set of CDs.

Keywords: quality measures; quality improvement; adult behavioral health.

First articulated in 2008, the Triple Aim proposes that health care systems should simultaneously seek to improve the patient’s experience of care, improve the health of populations, and reduce the per capita costs of care for populations.¹ More recently, some have argued that health care provider burnout can deleteriously impact the attainment of the Triple Aim and have therefore advocated for an expanded focus to include a fourth Aim, the work life quality of the staff.² Milwaukee County Behavioral Health Division (BHD), a publicly funded, county-based behavioral health care system in Milwaukee, Wisconsin, recently adopted the Quadruple Aim as the framework by which it will organize its quality activities.

Although originally developed for medical organizations, BHD believes that the Quadruple Aim has strong applicability to county-level behavioral health services. Many county-based behavioral health divisions provide a variety of programs to large segments of the county based on financial eligibility and/or clinical need, and thus often have responsibilities to populations or subpopulations, rather than programs. County health divisions, such as Milwaukee County’s Department of Health and Human Services, are often asked to improve outcomes and client experience of care with neutral growth budgets and less reliance on taxes to fund programs, while simultaneously attracting and retaining competent staff.

Crucial to the effective implementation of the Quadruple Aim, however, is a clear set of population- level measures that help organizations assess their progress.³ Unfortunately, as some authors have noted, evaluation of the Quadruple Aim remains a challenge because the “concepts of (population) health, quality of care and costs are not unanimously defined and measures for these concepts are under construction.”⁴ Several authors have provided some guidance to assist in the development of a set of measures that effectively capture the elements of the Quadruple Aim.^5,6 However, the recent rapid proliferation of quality measures in health care^7,8 has been both burdensome and costly for providers.^9,10 Any measures adopted should not only be as meaningful as possible with regards to assessing progress towards the basic aims of health care, but should also be parsimonious, to limit measurement burden for providers (and patients) and focus attention on important issues.^11,12

To select the most effective, parsimonious set of measures possible, one must first select a set of key foci from among the many possible areas of focus that the core measure is intended to represent. The core domains (CDs), if appropriately consistent with the strategic goals of the organization, provide a mechanism to orient the efforts of the organization at every level and help every staff member of the organization understand how his or her work impacts the progress towards these goals.¹¹ The CDs, therefore, represent the opportunity to affect a greater integration of efforts across the organization toward these shared aims, creating uniformity of purpose at every level. Further, increasing organizational attention on the CDs can also help to reduce measurement burden by streamlining and focusing the data capture processes on the most valuable elements of quality and health, and discarding other extraneous measures (albeit not at the expense of other reporting requirements).¹¹ The remainder of this article describes the CDs selected by BHD to assess its progress toward implementation of the Quadruple Aim and are organized by the Aim which they best represent.

Methods

To effectively implement the Quadruple Aim at BHD, it was necessary to clearly define the subpopulation of focus for our efforts.⁶ In this case, the subpopulation of interest was defined as all adult clients (18 years and older) who received at least 1 service encounter within a specified time frame from a program that BHD either operated or contracted with to provide care. Services provided by the BHD network include everything from psychiatric inpatient services to mental health and addiction treatment and care management. A limited array of social services, including housing and employment services, is also available to eligible consumers. BHD is the county-run behavioral health provider for individuals who are uninsured or underinsured in Milwaukee County, a demographically diverse, primarily urban county of approximately 950,000 people located in Wisconsin. Approximately 15,000 adults receive services at BHD each year.

This work began by obtaining executive sponsorship for the project, in this case from the Chief Operations Officer and Executive Medical Director of BHD. With their backing, an initial review of the literature produced a preliminary set of possible domains, for which we created working definitions. We then made a list of key stakeholders throughout BHD to whom we needed to present the idea of the Quadruple Aim, and the CDs under each Aim, to secure their support. These stakeholders, which included individuals involved in quality activities, program managers, and executive leadership, were strategically selected based on their relative influence within the organization. A set of brief presentations and handouts explaining the project were then developed and shared at different focus groups with these stakeholders over the course of 6 months. These focus groups served to not only educate the organization about the Quadruple Aim and the CDs but afforded participants an opportunity to provide feedback as well.

During the focus groups, we asked participants which domains they believed were most important (were “core”) when operationalizing the Quadruple Aim. The focus groups provided feedback on the domain definitions, feedback that was used to develop uniform, mutually agreed upon definitions for the CDs that were generalizable to all departments at BHD, regardless of the focus of their services within the continuum of care or the continuum of age. This was a crucial step, as it will eventually enable BHD to aggregate data across departments, even if there are minor discrepancies in the specific items they use to assess the CDs. Comments from the focus groups ultimately resulted in a truncated list of domains and definitions, which, coupled with the literature review, resulted in our final set of CDs.

During our review of the literature, we also looked for items that we felt could best represent each CD in the briefest, most meaningful way. (These items were not meant to supersede existing data, but to provide examples that could be implemented with existing data or recommendations that could be utilized in the absence of existing data.) During this process, we made every effort to make use of existing data-reporting requirements. For example, if we had a state mandate to collect data on housing status, we attempted to leverage this required data point to represent the CD related to housing. In other cases, we attempted to utilize claims or other administrative data to operationalize the CD, such as in the cost-of-care metric articulated in the section the Third Aim. For CDs for which no data existed or were insufficient, we emphasized the use of single- versus multi-item scales. For example, if we found a single-item global assessment of quality of life that had good psychometric properties relative to its longer parent scale, we selected the single item. This approach to item selection allowed us to create the most efficient, parsimonious set of measures possible, which we believed would enable us to comprehensively assess all the CDs with the least amount of burden to staff and clients. These items were presented at stakeholder focus groups, during which we asked for comments on the existing measures in their program or department and gave them the opportunity to comment on the new recommended measures.

A working definition is provided for each CD, followed by a brief review of the research base supporting its inclusion in the final list. The item(s) selected by BHD to represent each CD and the source of the item(s) are then supplied. These items were based either on measures currently collected because of existing reporting mandates or, in the case where extant measures were not available, on new items that demonstrated acceptable psychometric properties in the research literature. The CDs and items are organized by the Aim they best represent. A full list of the CDs by Quadruple Aim and items by CD is provided in the Appendix of the online version of this article. This article concludes with a brief summary of this effort and a discussion of how staff will utilize these items at different levels throughout the BHD system.

The First Aim: Population Health

Health Outcomes

Deaths. This can be defined as the cause of death, as determined by the medical examiner’s office (where appropriate) or as the age at time of death. This CD can also be reported as proportion of deaths considered premature (eg, before age 75) or calculated as total years of potential life lost.

Brief review and suggested item(s). Rates and causes of premature mortality are critical foci for the County Health Rankings & Roadmaps,¹³ the Institute for Healthcare Improvement’s “Guide to Measuring the Triple Aim,”⁶ the Centers for Disease Control and Prevention’s “Community Health Assessment for Population Health Improvement,”¹⁴ and the Institute of Medicine’s (IOM) “Vital Signs: Core Metrics for Health and Health Care Progress.”¹¹ There is ample evidence that individuals with serious mental illness are at increased risk of early mortality relative to the general population,^15-18 and this risk applies to those with substance use disorders as well.^15,19-20 BHD tracks all deaths that occur while patients are receiving BHD-funded, community-based services.

Self-Reported Health and Well-Being. This CD asks patients to rate their current physical and mental health status, as well as their overall quality of life.

Brief review and suggested item(s): Self-rated physical health. Premature mortality among individuals with behavioral health issues appears to be due, in large part, to their increased vulnerability to the development of medical comorbidities.^16,21 A single self-rating question has demonstrated considerable sensitivity to premature mortality,^22,23 with predictive properties up to a decade prior to death.^24,25 Further, self-rated health has been associated with subsequent functional decline,^26,27 acute service utilization,^28,29 and overall health care costs.²⁸

Brief review and suggested item(s): Self-rated mental health. Mental health disorders are associated with significant disability worldwide,³⁰ and comorbid mental health issues can exacerbate the course of other medical problems. For example, depression is associated with increased rates of mortality among individuals with diabetes and³¹ cardiovascular disease,³² as well as with rates of overall mortality,³³ and psychiatric comorbidity is associated with longer lengths of stay and higher costs among patients hospitalized for medical problems.³⁴ Research has found that a single-item measure of self-rated mental health is associated with the presence of psychiatric diagnoses, psychiatric symptoms, and subsequent depression and serious mental illness up to 1 year post-assessment.^35,36 There is even evidence that self-rated mental health may be more strongly associated with self-ratings of overall health than self-ratings of physical health.³⁷

Brief review and suggested item(s): Self-rated quality of life. Quality of life is a critical component of the recovery journey and overall health.³⁸ For example, the County Health Rankings & Roadmaps lists “quality of life” as 1 of its key “health outcomes” in its County Health Rankings.¹³ As some authors have noted, quality of life is often inferred from other “objective” recovery domains, such as employment, health status, or housing status. However, there is evidence that these objective domains are functionally distinct from the inherently subjective construct of quality of life.³⁹ This has led other authors to conclude that these domains should be assessed separately when evaluating outcomes.⁴⁰ Single-item quality of life assessments have been used in research with individuals with cancer,⁴¹ adults with disabilities,⁴² patients with cystic fibrosis,⁴³ and children with epilepsy.⁴⁴ For this effort, BHD selected the first global quality of life item from the World Health Organization’s WHOQOL-BREF quality of life assessment,⁴⁵ an item used in other quality of life research.⁴⁶

Health Factors

Substance Use. This CD is a composite of 4 different types of substance use, any recent heavy alcohol use (defined as 5 or more drinks in one sitting), any recent drug use, any recent prescription drug abuse, and any recent tobacco use.

Brief review and suggested item(s). As noted, substance use disorders confer an increased risk for early mortality^15,19 and are significantly implicated in disease disability burden worldwide.³⁰ Substance use has also been associated with both the onset^47,48 and exacerbation of mental health diagnoses.^49-51 Further, substance use appears to heighten the risk of violence in the general population⁵² and especially among those with a co-occurring mental illness.^53,54 The County Health Rankings & Roadmaps list alcohol and drug use as key behaviors to address to improve the overall health of a given county,¹³ and the Centers for Medicare & Medicaid Services (CMS) has endorsed initiation and engagement in addiction treatment as one of the measures in its Adult Core Set.⁵⁵

Tobacco use continues to be one of the most significant risk factors for early mortality worldwide, and evidence indicates that it is associated with a lower life expectancy of nearly 10 years.⁵⁶ Unfortunately, rates of tobacco use are even higher among those with severe mental illness relative to the general population, and their rates of smoking cessation are lower.^57,58 Tobacco use is a significant risk factor for the high rates of early mortality in individuals with severe mental illness.¹⁸ Further, a recent meta-analysis noted that, relative to those who continued to smoke, those who ceased smoking had reduced rates of psychological distress and increased quality of life rankings.⁵⁹ Reducing tobacco use is one of the key components of the County Health Rankings & Roadmaps, and medication assistance with smoking and tobacco use cessation is also listed in the CMS Adult Core Set.^13,55

An accumulating body of evidence suggests that single-item measures can adequately detect alcohol^60-62 and drug use disorders.^60-64 McNeely and colleagues recently developed and tested a brief 4-item screen, the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool.^65,66 Preliminary evidence suggests that the TAPS tool can effectively identify the presence of problematic and disordered use of tobacco, alcohol, prescription medications, and other drugs.^65-67 BHD will use the 4 items from the TAPS tool to represent its substance use CD.

Education/Employment Status. This CD assesses the proportion of BHD members who have completed high school, who are in some type of educational or training program, or who are engaged in some type of employment activity (defined as full-time, part-time, supported, sheltered workshop, or as a full-time homemaker).

Brief review and suggested item(s). Research indicates that unemployment is a risk factor for mortality, even after controlling for other risk factors (eg, age, sex, socioeconomic status [SES], health).⁶⁸ Unemployment is associated with poorer physical and mental health in the general population and among those with disabilities.^69-71 Promisingly, evidence suggests that gaining employment or re-employment is associated with better health,⁷² even for individuals with substance use disorders⁷³ or moderate⁷⁴ to severe mental health disorders.^75-78 Some authors have even proposed that, above and beyond the associated health benefits, employment may also help to realize a modest cost savings due to reduced service utilization and disability.^79,80 Employment is a core tenet in the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) model of recovery,⁸¹ and is also listed as an important recovery goal for individuals with behavioral health issues.⁸² BHD collects data on employment status on all the patients it serves as part of its state-mandated reporting requirements and will use this item in the CD data set.⁸³

Living Situation. This is measured as the proportion of people who live in permanent, supportive, stable housing; it may also be measured as the percentage of the population living with severe housing problems or who are homeless.

Brief review and suggested item(s). Housing problems can be conceptualized as 3 inter-related components: conditions within the home, neighborhood conditions, and housing affordability, each of which can contribute uniquely to poorer physical and mental health of individuals and families⁸⁴ and to educational outcomes for children.^85,86 Further, individuals who are homeless have a standardized mortality ratio 2 to 5 times that of the general population,^87-89 even after controlling for low income status,⁹⁰ and some evidence suggests these rates are even higher among unsheltered versus sheltered homeless individuals.⁹¹ Interventions to improve the condition of housing have demonstrated positive impacts on both physical and mental health,⁹² and a recent study found that individuals receiving housing assistance in the form of public housing or multifamily housing from the Department of Housing and Urban Development had better self-rated physical and mental health relative to individuals on the wait list for housing assistance.⁹³ Moreover, the provision of housing has been shown to promote reductions in substance use and health service utilization among homeless individuals with substance use disorders.⁹⁴ Rog and colleagues reviewed the literature on permanent supportive housing for individuals with substance use or mental health disorders who were homeless or disabled, and found that provision of housing led to reduced rates of homelessness, emergency department (ED) and inpatient utilization and increased consumer satisfaction.⁹⁵

Importantly, evidence suggests that housing is viewed as facilitative of recovery. For example, in a recent qualitative study of homeless individuals with mental illness, housing was seen as a critical first step in recovery, providing a sense of security, increasing feelings of personal independence and autonomy, improving perceptions of health and well-being, and affording a stable environment to rebuild relationships with important others.⁹⁶ BHD collects data on housing status on all the patients it serves as part of its state-mandated reporting requirements and will utilize this item in the CD data set.⁸³

Social Relationships. This is defined as recent interactions with family, supportive networks (formal and informal), and other recovery services.

Brief review and suggested item(s). Research has long established that social relationships have a significant impact on health, including rates of mortality as well as physical and mental health morbidity.^97-99 Social connectedness is another of the pillars supporting an individual’s recovery in SAMHSA’s formulation. Several reviews of the recovery literature^38,82 support its importance to the recovery process and inclusion in any assessment of holistic recovery. Social support has been shown to promote recovery among individuals with severe mental illness^100-102 and substance use disorders,¹⁰³ and may mitigate the progression of chronic, life-threatening physical illnesses.⁹⁷ For the purposes of BHD’s CD data set, the social support question from the “100 Million Healthier Lives Common Questionnaire for Adults” will be used to assess individuals’ perceived adequacy of social support.¹⁰⁴

Legal Involvement. Defined as involvement with the civil or criminal justice system, including arrests, imprisonment, or detainment.

Brief review and suggested item(s). Involvement in the criminal justice system is both disruptive for the individual in recovery and expensive to the larger health care system.¹⁰⁵ Individuals with substance use¹⁰⁶ and severe mental health disorders¹⁰⁷ are over-represented in the prison system, and evidence suggests that general physical and mental health declines while individuals are in prison.^108,109 Perhaps even more concerning, numerous studies have demonstrated an increase in mortality rates for individuals recently released from prison relative to the general population, particularly during the period immediately following release.^108-110 This relationship may even persist long term.¹¹¹ Further, research indicates that individuals recently released from prison have increased emergency care and hospital utilization.^112,113

Incarceration can have significant impacts on the health of the broader community as well. For example, research has found an association between parental incarceration to rates of infant mortality,¹¹⁴ increased behavioral and developmental problems of children of incarcerated parents,^115,116 lower rates of child support payments,¹¹⁷ and poorer cardiovascular health of female partners of incarcerated individuals.¹¹⁸ Formerly incarcerated individuals experience slower wage growth as well.¹¹⁹ However, evidence also indicates that engagement in mental health¹²⁰ and substance abuse¹²¹ treatment can reduce the likelihood of subsequent recidivism. As part of its state-mandated reporting, BHD is required to provide information on the criminal justice system involvement of its clients in the previous 6 months, including whether they have been jailed or imprisoned,⁸³ and this will function as its measure of legal involvement in its CD data set.

Socioeconomic Status. Socioeconomic status is the social standing or class of an individual or group. It is often measured as a combination of education, income, and occupation. It can also be defined subjectively, such as one’s evaluation of status relative to similar others or based on an individual’s interpretation of her or his financial needs.

Brief review and suggested item(s). A large body of evidence supports the existence of a robust relationship between lower SES and poor health, including mortality and chronic medical diseases,^122-124 as well as mental illness.^125-127 Although previous research has examined this relationship using objective indicators of SES (eg, income, education level, occupation), there has recently been an increased interest in exploring the relationship of subjective SES with health indices. Subjective SES is generally assessed by asking individuals to rate themselves relative to others in the society in which they live, in terms of wealth, occupation, educational level, or other indicators of social status. Evidence suggests that subjective SES is associated with objective measures of SES,^128-130 and relates to measures of physical and mental health as well, even after controlling for objective SES.^130-135 BHD will be using a modified version of the Subject SES Scale,^131,135 which is deployed in the “100 Million Healthier Lives Common Questionnaire for Adults.”¹⁰⁴

Acute Service Use. This is defined as an admission to a medical or psychiatric emergency room or to a medical or psychiatric hospital or to a detoxification facility.

Brief review and suggested item(s). The CMS Adult Core Set includes “plan all cause readmissions” as a key quality metric.⁵⁵ Hospital readmissions are also endorsed by the National Committee on Quality Assurance as one of its Health Effectiveness Data and Information Set (HEDIS) measures and by the National Quality Forum. Readmissions, despite their widespread endorsement, are a somewhat controversial measure. Although readmissions are costly to the health care system,¹³⁶ the relationship between readmissions and quality is inconsistent. For example, Krumholz and colleagues¹³⁷ found differential rates of readmission for the same patient discharged from 2 different hospitals, which were categorized based on previous readmission rates, suggesting that hospitals do have different levels of performance even when treating the same patient. However, other data indicate that 30-day, all-cause, risk-standardized readmission rates are not associated with hospital 30-day, all-cause, risk-standardized mortality rates.¹³⁸

Chin and colleague found that readmissions to the hospital that occurred more than 7 days post-discharge were likely due to community- and household-related factors, rather than hospital-related quality factors.¹³⁹ Transitional care interventions that have successfully reduced 30-day readmission rates are most often multicomponent and focus not just on hospital-based interventions (eg, discharge planning, education) but on follow-up care in the community by formal supports (eg, in-home visits, telephone calls, outpatient clinic appointments, case management) and informal supports (eg, family and friends).^140-143 Further, qualitative evidence suggests that some individuals perceive psychiatric hospitalizations to be the result of insufficient resources or unsuccessful attempts to maintain their stability in the community.¹⁴⁴ Thus, unplanned or avoidable hospital readmissions may represent a failure of the continuum of care not only from the perspective of the health care system, but from the patient perspective as well.

Frequent or nonurgent use of EDs is conceptually similar to excessive or avoidable inpatient utilization in several ways. For example, overuse of EDs is costly, with some estimates suggesting that it is responsible for up to $38 billion in wasteful spending each year.¹⁴⁵ Individuals with frequent ED visits have a greater disease burden¹⁴⁶ and an increased risk of mortality compared to nonfrequent users.¹⁴⁷ Research suggests that individuals who visit the ED for non-urgent issues do so because of perceived difficulties associated with accessing primary care, and the convenience of EDs relative to primary care.^148-150 Moreover, similar to the hospital readmission literature discussed earlier, successful strategies to reduce high rates of ED utilization generally focus on continuum of care interventions, such as provision of case management services.^151-155

This evidence implies that frequent ED utilization and hospital readmissions may not be a fundamental issue of quality (or lack thereof) in hospitals or EDs but rather a lack of, or ineffectual, transitional and continuum of care strategies and services. To underscore this point, some authors have argued that a system that is excessively crisis-oriented hinders recovery because it is reactive rather than proactive, predicated on the notion that one’s condition must deteriorate to receive care.¹⁵⁶

Although some organizations may have access to claims data or may function as self-contained health systems (eg, the Veterans Health Administration [VHA] ), others may not have access to such data. In the absence of claims data, patient self-report of service utilization has been used as a proxy for actual agency records.¹⁵⁷ Although concordance between medical and/or agency records and patient self-report has been variable,¹⁵⁷ evidence generally suggests that rates of agreement are higher the shorter the recall time interval.^158,159 BHD does not have access to comprehensive claims data and has therefore chosen to use 5 dichotomously scored (yes/no) questions—related to medical inpatient, medical ED, psychiatric inpatient, psychiatric ED, and detoxification use in the last 30 days—to represent the CD of acute service utilization.

The Second Aim: Quality of Care

Safety

Safety is defined as avoiding injuries to patients from the care that is intended to help them.

Brief review and suggested item(s). As noted in “Crossing the Quality Chasm,” the IOM’s seminal document, “the health care environment should be safe for all patients, in all of its processes, all the time.”¹⁶⁰ The landmark Harvard Medical Practice Study in 1991 found that adverse events occurred in nearly 4% of all hospital admissions and, among these, over a quarter were due to negligence.¹⁶¹ Other estimates of adverse events range as high as 17%.¹⁶² Indeed, a recent article by Makary and Daniel estimated that medical errors may be the third leading cause of death in the United States.¹⁶³ Unfortunately, research on safety in the mental health field has lagged behind that of physical health,¹⁶⁴ with evidence indicating that research in nonhospital settings in mental health care may be particularly scarce.¹⁶⁵ In a study of adverse events that occurred in psychiatric inpatient units in the VHA system between 2015 and 2016, Mills and colleagues found that of the 87 root cause analysis reports, suicide attempts were the most frequent, and, among safety events, falls were the most frequently reported, followed by medication events.¹⁶⁶ Another report on data collected from psychiatric inpatient units in the VHA revealed that nearly one-fifth of patients experienced a safety event, over half of which were deemed preventable.¹⁶⁷ These numbers likely represent an underestimation of the true volume of safety events, as another study by the same research group found that less than 40% of safety events described in patient medical records were documented in the incident reporting system.¹⁶⁸ BHD will utilize the total number of complaints and incident reports submitted within a given time frame as its “safety” metric in the CD data set.

Wait Time for Service

The CD is defined as the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Brief review and suggested item(s). “Timeliness” was listed among the 6 aims for improvement in “Crossing the Quality Chasm” in 2001, and it remains no less relevant today.¹⁶⁰ For example, evidence indicates that access to primary care is inversely related to avoidable hospitalizations.¹⁶⁹ One study found that, of patients hospitalized for cardiovascular problems, those who had difficulty accessing routine care post discharge had higher 30-day readmission rates.¹⁷⁰ Among VHA patients, longer wait times are associated with more avoidable hospitalizations and higher rates of mortality.¹⁷¹ Longer wait times appear to decrease the likelihood of attending a first appointment for individuals with substance use^172,173 and mental health disorders.¹⁷⁴ Importantly, longer wait times are associated with lower ratings of the patient experience of care, including perceptions of the quality of and satisfaction with care,¹⁷⁵ and may be associated with worse outcomes for individuals in early intervention for psychosis treatment.¹⁷⁶ For the purposes of the CD data set, BHD will monitor the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Patient satisfaction is defined as the degree of patients’ satisfaction with the care they have received.

Brief review and suggested item(s). Research has consistently demonstrated the relationship of the patient’s experience of care to a variety of safety and clinical effectiveness measures in medical health care,¹⁷⁷ and the therapeutic alliance is one of the most consistent predictors of outcomes in behavioral health, regardless of therapeutic modality.¹⁷⁸ Patient satisfaction is a commonly assessed aspect of the patient experience of care. Patient satisfaction scores have been correlated with patient adherence to recommended treatment regimens, care quality, and health outcomes.¹⁷⁹ For example, Aiken et al found that patient satisfaction with hospital care was associated with higher ratings of the quality and safety of nursing care in these hospitals.¹⁸⁰ Increased satisfaction with inpatient care has been associated with lower 30-day readmission rates for patients with acute myocardial infarction, heart failure, and pneumonia,¹⁸¹ and patients with schizophrenia who reported higher treatment satisfaction also reported better quality of life.^182,183 Many satisfaction survey options exist to evaluate this CD, including the Consumer Assessment of Healthcare Providers and Systems and the Client Satisfaction Questionnaire; BHD will utilize an outpatient behavioral health survey from a third-party vendor.

The Third Aim: Cost of Care

Cost of Care

This can be defined as the average cost to provide care per patient per month.

Brief review and suggested item(s). Per capita cost, or rather, the total cost of providing care to a circumscribed population divided by the total population, has been espoused as an important metric for the Triple Aim and the County Health Rankings.^6,13 Indeed, between 1960 and 2016, per capita expenditures for health care have grown 70-fold, and the percent of the national gross domestic product accounted for by health expenditures has more than tripled (5.0% to 17.9%).¹⁸⁴ One of the more common metrics deployed for assessing health care cost is the per capita per month cost, or rather, the per member per month cost of the predefined population for a given health care system.^6,185,186 In fact, some authors have proposed that cost of care can be used not only to track efficient resource allocation, but can also be a proxy for a healthier population as well (ie, as health improves, individuals use fewer and less-expensive services, thus costing the system less).¹⁸⁷ To assess this metric, BHD will calculate the total amount billed for patient care provided within BHD’s health network each month (irrespective of funding source) and then divide this sum by the number of members served each month. Although this measure does not account for care received at other health care facilities outside BHD’s provider network, nor does it include all the overhead costs associated with the care provided by BHD itself, it is consistent with the claims-based approach used or recommended by other authors.^6,188

The Fourth Aim: Staff Well-being

Staff Quality of Work Life

This can be defined as the quality of the work life of health care clinicians and staff.

Brief review and suggested item(s). Some authors have suggested that the Triple Aim framework is incomplete and have proffered compelling arguments that provider well-being and the quality of work life constitutes a fourth aim.² Provider burnout is prevalent in both medical^2,189 and behavioral health care.^190,191 Burnout among health care professionals has been associated with higher rates of perceived medical errors,¹⁹² lower patient satisfaction scores,^189,193 lower rates of provider empathy,¹⁹⁴ more negative attitudes towards patients,¹⁹⁵ and poorer staff mental and physical health.¹⁹¹

Burnout is also associated with higher rates of absenteeism, turnover intentions, and turnover.^{190,191,196,197} However, burnout is not the only predictor of staff turnover; for example, turnover rates are a useful proxy for staff quality of work life for several reasons.¹⁹⁸ First, turnover is associated with substantial direct and indirect costs, including lost productivity, increased errors, and lost revenue and recruitment costs, with some turnover cost estimates as high as $17 billion for physicians and $14 billion for nurses annually.^199-201 Second, research indicates that staff turnover can have a deleterious impact on implementation of evidence-based interventions.^202-205 Finally, consistent with the philosophy of utilizing existing data sources for the CD measures, turnover can be relatively easily extracted from administrative data for operated or contracted programs, and its collection does not place any additional burden on staff. As a large behavioral health system that is both a provider and payer of care, BHD will therefore examine the turnover rates of its internal administrative and clinical staff as well as the turnover of staff in its contracted provider network as its measures for the Staff Quality of Work Life CD.

Clinical Implications

These metrics can be deployed at any level of the organization. Clinicians may use 1 or more of the measures to track the recovery of individual clients, or in aggregate for their entire caseload. Similarly, managers can use these measures to assess the overall effectiveness of the programs for which they are responsible. Executive leaders can evaluate the impact of several programs or the system of care on the health of a subpopulation of clients with a specific condition, or for all their enrolled members. Further, not all measures need be utilized for every dashboard or evaluative effort. The benefit of a comprehensive set of measures lies in their flexibility—1 or more of the measures may be selected depending on the project being implemented or the interests of the stakeholder.

It is important to note that many of the CDs (and their accompanying measures) are aligned to/consistent with social determinants of health.^206,207 Evidence suggests that social determinants make substantial contributions to the overall health of individuals and populations and may even account for a greater proportion of variance in health outcomes than health care itself.²⁰⁸ The measures articulated here, therefore, can be used to assess whether and how effectively care provision has addressed these social determinants, as well as the relative impact their resolution may have on other health outcomes (eg, mortality, self-rated health).

These measures can also be used to stratify clients by clinical severity or degree of socioeconomic deprivation. The ability to adjust for risk has many applications in health care, particularly when organizations are attempting to implement value-based purchasing models, such as pay-for-performance contracts or other alternative payment models (population health-based payment models).²⁰⁹ Indeed, once fully implemented, the CDs and measures will enable BHD to more effectively build and execute different conceptual models of “value” (see references 210 and 211 for examples). We will be able to assess the progress our clients have made in care, the cost associated with that degree of improvement, the experience of those clients receiving that care, and the clinical and social variables that may influence the relative degree of improvement (or lack thereof). Thus, the CDs provide a conceptual and data-driven foundation for the Quadruple Aim and any quality initiatives that either catalyze or augment its implementation.

This article provides an overview of the CDs selected by BHD to help organize, focus, advance, and track its quality efforts within the framework of the Quadruple Aim. Although items aligned to each of these CDs are offered, the CDs themselves have been broadly conceptualized such that they can flexibly admit a variety of possible items and/or assessments to operationalize each CD and thus have potential applicability to other behavioral health systems, particularly public systems that have state-mandated and other data reporting requirements.

Bearing in mind the burden that growing data collection requirements can have on the provision of quality care and staff work satisfaction and burnout,^10,212 the CDs (and the items selected to represent each) are designed with “strategic parsimony” in mind. Although the CDs are inclusive in that they cover care quality, cost of care, staff quality of life, and general population health, only CDs and items undergirded by a solid evidence base and high value with regards to BHD’s mission and values, as determined by key stakeholders, were selected. Moreover, BHD attempted to make use of existing data collection and reporting mandates when selecting the final pool of items to reduce the measurement burden on staff and clients. Thus, the final set of CDs and items are designed to be comprehensive yet economical.

The CDs are deeply interrelated. Although each CD may be individually viewed as a valuable metric, improvements in any 1 CD will impact the others (eg, increasing care quality should impact population health, increasing staff quality of life should impact the quality of care). Moreover, this idea of interrelatedness acknowledges the need to view health systems and the populations they serve holistically, in that improvement is not simply the degree of change in any given metric (whether individually or collectively), but rather something more entirely. The concepts of value, quality, and health are complex, multidimensional, and dynamic, and the CDs that comprise these concepts should not be considered independently from one another. The CDs (and items) offered in this article are scalable in that they can be used at different levels of an organization depending on the question or stakeholder, and can be used individually or in combination with one another. Moreover, they are adaptable to a variety of risk-adjusted program, population health, and value-based evaluation models. It is hoped that the process articulated here, and the accompanying literature review, may benefit other public or government-run health systems in their own quality journey to operationalize the Quadruple Aim by developing a set of CDs.

Corresponding author: Walter Matthew Drymalski, PhD; [email protected].

Financial disclosures: None.

From the Milwaukee County Behavioral Health Division, Milwaukee, WI.

Abstract

Objective: Implementation of the Quadruple Aim of health care must begin with a clearly articulated set of concepts, or core domains (CDs), that comprise each aim. These CDs can then be operationalized with existing or new measures. If aligned to the organization’s mission and strategic goals, these CDs have the potential to focus quality improvement activities and reduce measurement burden. This article represents the efforts of a publicly funded behavioral health system to operationalize the Quadruple Aim through the development of CDs.

Methods: Various stakeholders across the organization were consulted on their perceptions of the Quadruple Aim and the CDs they believed should support it. Then, a review of existing literature on core metrics for health care and population health was completed, summarized, and integrated with the stakeholder feedback.

Results: These efforts led to the development and adoption of 15 CDs, with an accompanying literature review and set of recommendations of new and existing measures for each domain.

Conclusions: It is possible to create a comprehensive yet economical set of CDs and attendant measures that can be implemented in a staged, scalable, enterprise manner. It is hoped that the process articulated here, and the accompanying literature review, may be of some benefit to other public or government-run health systems in their own quality improvement journey to operationalize the Quadruple Aim by developing a set of CDs.

Keywords: quality measures; quality improvement; adult behavioral health.

First articulated in 2008, the Triple Aim proposes that health care systems should simultaneously seek to improve the patient’s experience of care, improve the health of populations, and reduce the per capita costs of care for populations.¹ More recently, some have argued that health care provider burnout can deleteriously impact the attainment of the Triple Aim and have therefore advocated for an expanded focus to include a fourth Aim, the work life quality of the staff.² Milwaukee County Behavioral Health Division (BHD), a publicly funded, county-based behavioral health care system in Milwaukee, Wisconsin, recently adopted the Quadruple Aim as the framework by which it will organize its quality activities.

Although originally developed for medical organizations, BHD believes that the Quadruple Aim has strong applicability to county-level behavioral health services. Many county-based behavioral health divisions provide a variety of programs to large segments of the county based on financial eligibility and/or clinical need, and thus often have responsibilities to populations or subpopulations, rather than programs. County health divisions, such as Milwaukee County’s Department of Health and Human Services, are often asked to improve outcomes and client experience of care with neutral growth budgets and less reliance on taxes to fund programs, while simultaneously attracting and retaining competent staff.

Crucial to the effective implementation of the Quadruple Aim, however, is a clear set of population- level measures that help organizations assess their progress.³ Unfortunately, as some authors have noted, evaluation of the Quadruple Aim remains a challenge because the “concepts of (population) health, quality of care and costs are not unanimously defined and measures for these concepts are under construction.”⁴ Several authors have provided some guidance to assist in the development of a set of measures that effectively capture the elements of the Quadruple Aim.^5,6 However, the recent rapid proliferation of quality measures in health care^7,8 has been both burdensome and costly for providers.^9,10 Any measures adopted should not only be as meaningful as possible with regards to assessing progress towards the basic aims of health care, but should also be parsimonious, to limit measurement burden for providers (and patients) and focus attention on important issues.^11,12

To select the most effective, parsimonious set of measures possible, one must first select a set of key foci from among the many possible areas of focus that the core measure is intended to represent. The core domains (CDs), if appropriately consistent with the strategic goals of the organization, provide a mechanism to orient the efforts of the organization at every level and help every staff member of the organization understand how his or her work impacts the progress towards these goals.¹¹ The CDs, therefore, represent the opportunity to affect a greater integration of efforts across the organization toward these shared aims, creating uniformity of purpose at every level. Further, increasing organizational attention on the CDs can also help to reduce measurement burden by streamlining and focusing the data capture processes on the most valuable elements of quality and health, and discarding other extraneous measures (albeit not at the expense of other reporting requirements).¹¹ The remainder of this article describes the CDs selected by BHD to assess its progress toward implementation of the Quadruple Aim and are organized by the Aim which they best represent.

Methods

To effectively implement the Quadruple Aim at BHD, it was necessary to clearly define the subpopulation of focus for our efforts.⁶ In this case, the subpopulation of interest was defined as all adult clients (18 years and older) who received at least 1 service encounter within a specified time frame from a program that BHD either operated or contracted with to provide care. Services provided by the BHD network include everything from psychiatric inpatient services to mental health and addiction treatment and care management. A limited array of social services, including housing and employment services, is also available to eligible consumers. BHD is the county-run behavioral health provider for individuals who are uninsured or underinsured in Milwaukee County, a demographically diverse, primarily urban county of approximately 950,000 people located in Wisconsin. Approximately 15,000 adults receive services at BHD each year.

This work began by obtaining executive sponsorship for the project, in this case from the Chief Operations Officer and Executive Medical Director of BHD. With their backing, an initial review of the literature produced a preliminary set of possible domains, for which we created working definitions. We then made a list of key stakeholders throughout BHD to whom we needed to present the idea of the Quadruple Aim, and the CDs under each Aim, to secure their support. These stakeholders, which included individuals involved in quality activities, program managers, and executive leadership, were strategically selected based on their relative influence within the organization. A set of brief presentations and handouts explaining the project were then developed and shared at different focus groups with these stakeholders over the course of 6 months. These focus groups served to not only educate the organization about the Quadruple Aim and the CDs but afforded participants an opportunity to provide feedback as well.

During the focus groups, we asked participants which domains they believed were most important (were “core”) when operationalizing the Quadruple Aim. The focus groups provided feedback on the domain definitions, feedback that was used to develop uniform, mutually agreed upon definitions for the CDs that were generalizable to all departments at BHD, regardless of the focus of their services within the continuum of care or the continuum of age. This was a crucial step, as it will eventually enable BHD to aggregate data across departments, even if there are minor discrepancies in the specific items they use to assess the CDs. Comments from the focus groups ultimately resulted in a truncated list of domains and definitions, which, coupled with the literature review, resulted in our final set of CDs.

During our review of the literature, we also looked for items that we felt could best represent each CD in the briefest, most meaningful way. (These items were not meant to supersede existing data, but to provide examples that could be implemented with existing data or recommendations that could be utilized in the absence of existing data.) During this process, we made every effort to make use of existing data-reporting requirements. For example, if we had a state mandate to collect data on housing status, we attempted to leverage this required data point to represent the CD related to housing. In other cases, we attempted to utilize claims or other administrative data to operationalize the CD, such as in the cost-of-care metric articulated in the section the Third Aim. For CDs for which no data existed or were insufficient, we emphasized the use of single- versus multi-item scales. For example, if we found a single-item global assessment of quality of life that had good psychometric properties relative to its longer parent scale, we selected the single item. This approach to item selection allowed us to create the most efficient, parsimonious set of measures possible, which we believed would enable us to comprehensively assess all the CDs with the least amount of burden to staff and clients. These items were presented at stakeholder focus groups, during which we asked for comments on the existing measures in their program or department and gave them the opportunity to comment on the new recommended measures.

A working definition is provided for each CD, followed by a brief review of the research base supporting its inclusion in the final list. The item(s) selected by BHD to represent each CD and the source of the item(s) are then supplied. These items were based either on measures currently collected because of existing reporting mandates or, in the case where extant measures were not available, on new items that demonstrated acceptable psychometric properties in the research literature. The CDs and items are organized by the Aim they best represent. A full list of the CDs by Quadruple Aim and items by CD is provided in the Appendix of the online version of this article. This article concludes with a brief summary of this effort and a discussion of how staff will utilize these items at different levels throughout the BHD system.

The First Aim: Population Health

Health Outcomes

Deaths. This can be defined as the cause of death, as determined by the medical examiner’s office (where appropriate) or as the age at time of death. This CD can also be reported as proportion of deaths considered premature (eg, before age 75) or calculated as total years of potential life lost.

Brief review and suggested item(s). Rates and causes of premature mortality are critical foci for the County Health Rankings & Roadmaps,¹³ the Institute for Healthcare Improvement’s “Guide to Measuring the Triple Aim,”⁶ the Centers for Disease Control and Prevention’s “Community Health Assessment for Population Health Improvement,”¹⁴ and the Institute of Medicine’s (IOM) “Vital Signs: Core Metrics for Health and Health Care Progress.”¹¹ There is ample evidence that individuals with serious mental illness are at increased risk of early mortality relative to the general population,^15-18 and this risk applies to those with substance use disorders as well.^15,19-20 BHD tracks all deaths that occur while patients are receiving BHD-funded, community-based services.

Self-Reported Health and Well-Being. This CD asks patients to rate their current physical and mental health status, as well as their overall quality of life.

Brief review and suggested item(s): Self-rated physical health. Premature mortality among individuals with behavioral health issues appears to be due, in large part, to their increased vulnerability to the development of medical comorbidities.^16,21 A single self-rating question has demonstrated considerable sensitivity to premature mortality,^22,23 with predictive properties up to a decade prior to death.^24,25 Further, self-rated health has been associated with subsequent functional decline,^26,27 acute service utilization,^28,29 and overall health care costs.²⁸

Brief review and suggested item(s): Self-rated mental health. Mental health disorders are associated with significant disability worldwide,³⁰ and comorbid mental health issues can exacerbate the course of other medical problems. For example, depression is associated with increased rates of mortality among individuals with diabetes and³¹ cardiovascular disease,³² as well as with rates of overall mortality,³³ and psychiatric comorbidity is associated with longer lengths of stay and higher costs among patients hospitalized for medical problems.³⁴ Research has found that a single-item measure of self-rated mental health is associated with the presence of psychiatric diagnoses, psychiatric symptoms, and subsequent depression and serious mental illness up to 1 year post-assessment.^35,36 There is even evidence that self-rated mental health may be more strongly associated with self-ratings of overall health than self-ratings of physical health.³⁷

Brief review and suggested item(s): Self-rated quality of life. Quality of life is a critical component of the recovery journey and overall health.³⁸ For example, the County Health Rankings & Roadmaps lists “quality of life” as 1 of its key “health outcomes” in its County Health Rankings.¹³ As some authors have noted, quality of life is often inferred from other “objective” recovery domains, such as employment, health status, or housing status. However, there is evidence that these objective domains are functionally distinct from the inherently subjective construct of quality of life.³⁹ This has led other authors to conclude that these domains should be assessed separately when evaluating outcomes.⁴⁰ Single-item quality of life assessments have been used in research with individuals with cancer,⁴¹ adults with disabilities,⁴² patients with cystic fibrosis,⁴³ and children with epilepsy.⁴⁴ For this effort, BHD selected the first global quality of life item from the World Health Organization’s WHOQOL-BREF quality of life assessment,⁴⁵ an item used in other quality of life research.⁴⁶

Health Factors

Substance Use. This CD is a composite of 4 different types of substance use, any recent heavy alcohol use (defined as 5 or more drinks in one sitting), any recent drug use, any recent prescription drug abuse, and any recent tobacco use.

Brief review and suggested item(s). As noted, substance use disorders confer an increased risk for early mortality^15,19 and are significantly implicated in disease disability burden worldwide.³⁰ Substance use has also been associated with both the onset^47,48 and exacerbation of mental health diagnoses.^49-51 Further, substance use appears to heighten the risk of violence in the general population⁵² and especially among those with a co-occurring mental illness.^53,54 The County Health Rankings & Roadmaps list alcohol and drug use as key behaviors to address to improve the overall health of a given county,¹³ and the Centers for Medicare & Medicaid Services (CMS) has endorsed initiation and engagement in addiction treatment as one of the measures in its Adult Core Set.⁵⁵

Tobacco use continues to be one of the most significant risk factors for early mortality worldwide, and evidence indicates that it is associated with a lower life expectancy of nearly 10 years.⁵⁶ Unfortunately, rates of tobacco use are even higher among those with severe mental illness relative to the general population, and their rates of smoking cessation are lower.^57,58 Tobacco use is a significant risk factor for the high rates of early mortality in individuals with severe mental illness.¹⁸ Further, a recent meta-analysis noted that, relative to those who continued to smoke, those who ceased smoking had reduced rates of psychological distress and increased quality of life rankings.⁵⁹ Reducing tobacco use is one of the key components of the County Health Rankings & Roadmaps, and medication assistance with smoking and tobacco use cessation is also listed in the CMS Adult Core Set.^13,55

An accumulating body of evidence suggests that single-item measures can adequately detect alcohol^60-62 and drug use disorders.^60-64 McNeely and colleagues recently developed and tested a brief 4-item screen, the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool.^65,66 Preliminary evidence suggests that the TAPS tool can effectively identify the presence of problematic and disordered use of tobacco, alcohol, prescription medications, and other drugs.^65-67 BHD will use the 4 items from the TAPS tool to represent its substance use CD.

Education/Employment Status. This CD assesses the proportion of BHD members who have completed high school, who are in some type of educational or training program, or who are engaged in some type of employment activity (defined as full-time, part-time, supported, sheltered workshop, or as a full-time homemaker).

Brief review and suggested item(s). Research indicates that unemployment is a risk factor for mortality, even after controlling for other risk factors (eg, age, sex, socioeconomic status [SES], health).⁶⁸ Unemployment is associated with poorer physical and mental health in the general population and among those with disabilities.^69-71 Promisingly, evidence suggests that gaining employment or re-employment is associated with better health,⁷² even for individuals with substance use disorders⁷³ or moderate⁷⁴ to severe mental health disorders.^75-78 Some authors have even proposed that, above and beyond the associated health benefits, employment may also help to realize a modest cost savings due to reduced service utilization and disability.^79,80 Employment is a core tenet in the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) model of recovery,⁸¹ and is also listed as an important recovery goal for individuals with behavioral health issues.⁸² BHD collects data on employment status on all the patients it serves as part of its state-mandated reporting requirements and will use this item in the CD data set.⁸³

Living Situation. This is measured as the proportion of people who live in permanent, supportive, stable housing; it may also be measured as the percentage of the population living with severe housing problems or who are homeless.

Brief review and suggested item(s). Housing problems can be conceptualized as 3 inter-related components: conditions within the home, neighborhood conditions, and housing affordability, each of which can contribute uniquely to poorer physical and mental health of individuals and families⁸⁴ and to educational outcomes for children.^85,86 Further, individuals who are homeless have a standardized mortality ratio 2 to 5 times that of the general population,^87-89 even after controlling for low income status,⁹⁰ and some evidence suggests these rates are even higher among unsheltered versus sheltered homeless individuals.⁹¹ Interventions to improve the condition of housing have demonstrated positive impacts on both physical and mental health,⁹² and a recent study found that individuals receiving housing assistance in the form of public housing or multifamily housing from the Department of Housing and Urban Development had better self-rated physical and mental health relative to individuals on the wait list for housing assistance.⁹³ Moreover, the provision of housing has been shown to promote reductions in substance use and health service utilization among homeless individuals with substance use disorders.⁹⁴ Rog and colleagues reviewed the literature on permanent supportive housing for individuals with substance use or mental health disorders who were homeless or disabled, and found that provision of housing led to reduced rates of homelessness, emergency department (ED) and inpatient utilization and increased consumer satisfaction.⁹⁵

Importantly, evidence suggests that housing is viewed as facilitative of recovery. For example, in a recent qualitative study of homeless individuals with mental illness, housing was seen as a critical first step in recovery, providing a sense of security, increasing feelings of personal independence and autonomy, improving perceptions of health and well-being, and affording a stable environment to rebuild relationships with important others.⁹⁶ BHD collects data on housing status on all the patients it serves as part of its state-mandated reporting requirements and will utilize this item in the CD data set.⁸³

Social Relationships. This is defined as recent interactions with family, supportive networks (formal and informal), and other recovery services.

Brief review and suggested item(s). Research has long established that social relationships have a significant impact on health, including rates of mortality as well as physical and mental health morbidity.^97-99 Social connectedness is another of the pillars supporting an individual’s recovery in SAMHSA’s formulation. Several reviews of the recovery literature^38,82 support its importance to the recovery process and inclusion in any assessment of holistic recovery. Social support has been shown to promote recovery among individuals with severe mental illness^100-102 and substance use disorders,¹⁰³ and may mitigate the progression of chronic, life-threatening physical illnesses.⁹⁷ For the purposes of BHD’s CD data set, the social support question from the “100 Million Healthier Lives Common Questionnaire for Adults” will be used to assess individuals’ perceived adequacy of social support.¹⁰⁴

Legal Involvement. Defined as involvement with the civil or criminal justice system, including arrests, imprisonment, or detainment.

Brief review and suggested item(s). Involvement in the criminal justice system is both disruptive for the individual in recovery and expensive to the larger health care system.¹⁰⁵ Individuals with substance use¹⁰⁶ and severe mental health disorders¹⁰⁷ are over-represented in the prison system, and evidence suggests that general physical and mental health declines while individuals are in prison.^108,109 Perhaps even more concerning, numerous studies have demonstrated an increase in mortality rates for individuals recently released from prison relative to the general population, particularly during the period immediately following release.^108-110 This relationship may even persist long term.¹¹¹ Further, research indicates that individuals recently released from prison have increased emergency care and hospital utilization.^112,113

Incarceration can have significant impacts on the health of the broader community as well. For example, research has found an association between parental incarceration to rates of infant mortality,¹¹⁴ increased behavioral and developmental problems of children of incarcerated parents,^115,116 lower rates of child support payments,¹¹⁷ and poorer cardiovascular health of female partners of incarcerated individuals.¹¹⁸ Formerly incarcerated individuals experience slower wage growth as well.¹¹⁹ However, evidence also indicates that engagement in mental health¹²⁰ and substance abuse¹²¹ treatment can reduce the likelihood of subsequent recidivism. As part of its state-mandated reporting, BHD is required to provide information on the criminal justice system involvement of its clients in the previous 6 months, including whether they have been jailed or imprisoned,⁸³ and this will function as its measure of legal involvement in its CD data set.

Socioeconomic Status. Socioeconomic status is the social standing or class of an individual or group. It is often measured as a combination of education, income, and occupation. It can also be defined subjectively, such as one’s evaluation of status relative to similar others or based on an individual’s interpretation of her or his financial needs.

Brief review and suggested item(s). A large body of evidence supports the existence of a robust relationship between lower SES and poor health, including mortality and chronic medical diseases,^122-124 as well as mental illness.^125-127 Although previous research has examined this relationship using objective indicators of SES (eg, income, education level, occupation), there has recently been an increased interest in exploring the relationship of subjective SES with health indices. Subjective SES is generally assessed by asking individuals to rate themselves relative to others in the society in which they live, in terms of wealth, occupation, educational level, or other indicators of social status. Evidence suggests that subjective SES is associated with objective measures of SES,^128-130 and relates to measures of physical and mental health as well, even after controlling for objective SES.^130-135 BHD will be using a modified version of the Subject SES Scale,^131,135 which is deployed in the “100 Million Healthier Lives Common Questionnaire for Adults.”¹⁰⁴

Acute Service Use. This is defined as an admission to a medical or psychiatric emergency room or to a medical or psychiatric hospital or to a detoxification facility.

Brief review and suggested item(s). The CMS Adult Core Set includes “plan all cause readmissions” as a key quality metric.⁵⁵ Hospital readmissions are also endorsed by the National Committee on Quality Assurance as one of its Health Effectiveness Data and Information Set (HEDIS) measures and by the National Quality Forum. Readmissions, despite their widespread endorsement, are a somewhat controversial measure. Although readmissions are costly to the health care system,¹³⁶ the relationship between readmissions and quality is inconsistent. For example, Krumholz and colleagues¹³⁷ found differential rates of readmission for the same patient discharged from 2 different hospitals, which were categorized based on previous readmission rates, suggesting that hospitals do have different levels of performance even when treating the same patient. However, other data indicate that 30-day, all-cause, risk-standardized readmission rates are not associated with hospital 30-day, all-cause, risk-standardized mortality rates.¹³⁸

Chin and colleague found that readmissions to the hospital that occurred more than 7 days post-discharge were likely due to community- and household-related factors, rather than hospital-related quality factors.¹³⁹ Transitional care interventions that have successfully reduced 30-day readmission rates are most often multicomponent and focus not just on hospital-based interventions (eg, discharge planning, education) but on follow-up care in the community by formal supports (eg, in-home visits, telephone calls, outpatient clinic appointments, case management) and informal supports (eg, family and friends).^140-143 Further, qualitative evidence suggests that some individuals perceive psychiatric hospitalizations to be the result of insufficient resources or unsuccessful attempts to maintain their stability in the community.¹⁴⁴ Thus, unplanned or avoidable hospital readmissions may represent a failure of the continuum of care not only from the perspective of the health care system, but from the patient perspective as well.

Frequent or nonurgent use of EDs is conceptually similar to excessive or avoidable inpatient utilization in several ways. For example, overuse of EDs is costly, with some estimates suggesting that it is responsible for up to $38 billion in wasteful spending each year.¹⁴⁵ Individuals with frequent ED visits have a greater disease burden¹⁴⁶ and an increased risk of mortality compared to nonfrequent users.¹⁴⁷ Research suggests that individuals who visit the ED for non-urgent issues do so because of perceived difficulties associated with accessing primary care, and the convenience of EDs relative to primary care.^148-150 Moreover, similar to the hospital readmission literature discussed earlier, successful strategies to reduce high rates of ED utilization generally focus on continuum of care interventions, such as provision of case management services.^151-155

This evidence implies that frequent ED utilization and hospital readmissions may not be a fundamental issue of quality (or lack thereof) in hospitals or EDs but rather a lack of, or ineffectual, transitional and continuum of care strategies and services. To underscore this point, some authors have argued that a system that is excessively crisis-oriented hinders recovery because it is reactive rather than proactive, predicated on the notion that one’s condition must deteriorate to receive care.¹⁵⁶

Although some organizations may have access to claims data or may function as self-contained health systems (eg, the Veterans Health Administration [VHA] ), others may not have access to such data. In the absence of claims data, patient self-report of service utilization has been used as a proxy for actual agency records.¹⁵⁷ Although concordance between medical and/or agency records and patient self-report has been variable,¹⁵⁷ evidence generally suggests that rates of agreement are higher the shorter the recall time interval.^158,159 BHD does not have access to comprehensive claims data and has therefore chosen to use 5 dichotomously scored (yes/no) questions—related to medical inpatient, medical ED, psychiatric inpatient, psychiatric ED, and detoxification use in the last 30 days—to represent the CD of acute service utilization.

The Second Aim: Quality of Care

Safety

Safety is defined as avoiding injuries to patients from the care that is intended to help them.

Brief review and suggested item(s). As noted in “Crossing the Quality Chasm,” the IOM’s seminal document, “the health care environment should be safe for all patients, in all of its processes, all the time.”¹⁶⁰ The landmark Harvard Medical Practice Study in 1991 found that adverse events occurred in nearly 4% of all hospital admissions and, among these, over a quarter were due to negligence.¹⁶¹ Other estimates of adverse events range as high as 17%.¹⁶² Indeed, a recent article by Makary and Daniel estimated that medical errors may be the third leading cause of death in the United States.¹⁶³ Unfortunately, research on safety in the mental health field has lagged behind that of physical health,¹⁶⁴ with evidence indicating that research in nonhospital settings in mental health care may be particularly scarce.¹⁶⁵ In a study of adverse events that occurred in psychiatric inpatient units in the VHA system between 2015 and 2016, Mills and colleagues found that of the 87 root cause analysis reports, suicide attempts were the most frequent, and, among safety events, falls were the most frequently reported, followed by medication events.¹⁶⁶ Another report on data collected from psychiatric inpatient units in the VHA revealed that nearly one-fifth of patients experienced a safety event, over half of which were deemed preventable.¹⁶⁷ These numbers likely represent an underestimation of the true volume of safety events, as another study by the same research group found that less than 40% of safety events described in patient medical records were documented in the incident reporting system.¹⁶⁸ BHD will utilize the total number of complaints and incident reports submitted within a given time frame as its “safety” metric in the CD data set.

Wait Time for Service

The CD is defined as the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Brief review and suggested item(s). “Timeliness” was listed among the 6 aims for improvement in “Crossing the Quality Chasm” in 2001, and it remains no less relevant today.¹⁶⁰ For example, evidence indicates that access to primary care is inversely related to avoidable hospitalizations.¹⁶⁹ One study found that, of patients hospitalized for cardiovascular problems, those who had difficulty accessing routine care post discharge had higher 30-day readmission rates.¹⁷⁰ Among VHA patients, longer wait times are associated with more avoidable hospitalizations and higher rates of mortality.¹⁷¹ Longer wait times appear to decrease the likelihood of attending a first appointment for individuals with substance use^172,173 and mental health disorders.¹⁷⁴ Importantly, longer wait times are associated with lower ratings of the patient experience of care, including perceptions of the quality of and satisfaction with care,¹⁷⁵ and may be associated with worse outcomes for individuals in early intervention for psychosis treatment.¹⁷⁶ For the purposes of the CD data set, BHD will monitor the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Patient satisfaction is defined as the degree of patients’ satisfaction with the care they have received.

Brief review and suggested item(s). Research has consistently demonstrated the relationship of the patient’s experience of care to a variety of safety and clinical effectiveness measures in medical health care,¹⁷⁷ and the therapeutic alliance is one of the most consistent predictors of outcomes in behavioral health, regardless of therapeutic modality.¹⁷⁸ Patient satisfaction is a commonly assessed aspect of the patient experience of care. Patient satisfaction scores have been correlated with patient adherence to recommended treatment regimens, care quality, and health outcomes.¹⁷⁹ For example, Aiken et al found that patient satisfaction with hospital care was associated with higher ratings of the quality and safety of nursing care in these hospitals.¹⁸⁰ Increased satisfaction with inpatient care has been associated with lower 30-day readmission rates for patients with acute myocardial infarction, heart failure, and pneumonia,¹⁸¹ and patients with schizophrenia who reported higher treatment satisfaction also reported better quality of life.^182,183 Many satisfaction survey options exist to evaluate this CD, including the Consumer Assessment of Healthcare Providers and Systems and the Client Satisfaction Questionnaire; BHD will utilize an outpatient behavioral health survey from a third-party vendor.

The Third Aim: Cost of Care

Cost of Care

This can be defined as the average cost to provide care per patient per month.

Brief review and suggested item(s). Per capita cost, or rather, the total cost of providing care to a circumscribed population divided by the total population, has been espoused as an important metric for the Triple Aim and the County Health Rankings.^6,13 Indeed, between 1960 and 2016, per capita expenditures for health care have grown 70-fold, and the percent of the national gross domestic product accounted for by health expenditures has more than tripled (5.0% to 17.9%).¹⁸⁴ One of the more common metrics deployed for assessing health care cost is the per capita per month cost, or rather, the per member per month cost of the predefined population for a given health care system.^6,185,186 In fact, some authors have proposed that cost of care can be used not only to track efficient resource allocation, but can also be a proxy for a healthier population as well (ie, as health improves, individuals use fewer and less-expensive services, thus costing the system less).¹⁸⁷ To assess this metric, BHD will calculate the total amount billed for patient care provided within BHD’s health network each month (irrespective of funding source) and then divide this sum by the number of members served each month. Although this measure does not account for care received at other health care facilities outside BHD’s provider network, nor does it include all the overhead costs associated with the care provided by BHD itself, it is consistent with the claims-based approach used or recommended by other authors.^6,188

The Fourth Aim: Staff Well-being

Staff Quality of Work Life

This can be defined as the quality of the work life of health care clinicians and staff.

Brief review and suggested item(s). Some authors have suggested that the Triple Aim framework is incomplete and have proffered compelling arguments that provider well-being and the quality of work life constitutes a fourth aim.² Provider burnout is prevalent in both medical^2,189 and behavioral health care.^190,191 Burnout among health care professionals has been associated with higher rates of perceived medical errors,¹⁹² lower patient satisfaction scores,^189,193 lower rates of provider empathy,¹⁹⁴ more negative attitudes towards patients,¹⁹⁵ and poorer staff mental and physical health.¹⁹¹

Burnout is also associated with higher rates of absenteeism, turnover intentions, and turnover.^{190,191,196,197} However, burnout is not the only predictor of staff turnover; for example, turnover rates are a useful proxy for staff quality of work life for several reasons.¹⁹⁸ First, turnover is associated with substantial direct and indirect costs, including lost productivity, increased errors, and lost revenue and recruitment costs, with some turnover cost estimates as high as $17 billion for physicians and $14 billion for nurses annually.^199-201 Second, research indicates that staff turnover can have a deleterious impact on implementation of evidence-based interventions.^202-205 Finally, consistent with the philosophy of utilizing existing data sources for the CD measures, turnover can be relatively easily extracted from administrative data for operated or contracted programs, and its collection does not place any additional burden on staff. As a large behavioral health system that is both a provider and payer of care, BHD will therefore examine the turnover rates of its internal administrative and clinical staff as well as the turnover of staff in its contracted provider network as its measures for the Staff Quality of Work Life CD.

Clinical Implications

These metrics can be deployed at any level of the organization. Clinicians may use 1 or more of the measures to track the recovery of individual clients, or in aggregate for their entire caseload. Similarly, managers can use these measures to assess the overall effectiveness of the programs for which they are responsible. Executive leaders can evaluate the impact of several programs or the system of care on the health of a subpopulation of clients with a specific condition, or for all their enrolled members. Further, not all measures need be utilized for every dashboard or evaluative effort. The benefit of a comprehensive set of measures lies in their flexibility—1 or more of the measures may be selected depending on the project being implemented or the interests of the stakeholder.

It is important to note that many of the CDs (and their accompanying measures) are aligned to/consistent with social determinants of health.^206,207 Evidence suggests that social determinants make substantial contributions to the overall health of individuals and populations and may even account for a greater proportion of variance in health outcomes than health care itself.²⁰⁸ The measures articulated here, therefore, can be used to assess whether and how effectively care provision has addressed these social determinants, as well as the relative impact their resolution may have on other health outcomes (eg, mortality, self-rated health).

These measures can also be used to stratify clients by clinical severity or degree of socioeconomic deprivation. The ability to adjust for risk has many applications in health care, particularly when organizations are attempting to implement value-based purchasing models, such as pay-for-performance contracts or other alternative payment models (population health-based payment models).²⁰⁹ Indeed, once fully implemented, the CDs and measures will enable BHD to more effectively build and execute different conceptual models of “value” (see references 210 and 211 for examples). We will be able to assess the progress our clients have made in care, the cost associated with that degree of improvement, the experience of those clients receiving that care, and the clinical and social variables that may influence the relative degree of improvement (or lack thereof). Thus, the CDs provide a conceptual and data-driven foundation for the Quadruple Aim and any quality initiatives that either catalyze or augment its implementation.

This article provides an overview of the CDs selected by BHD to help organize, focus, advance, and track its quality efforts within the framework of the Quadruple Aim. Although items aligned to each of these CDs are offered, the CDs themselves have been broadly conceptualized such that they can flexibly admit a variety of possible items and/or assessments to operationalize each CD and thus have potential applicability to other behavioral health systems, particularly public systems that have state-mandated and other data reporting requirements.

Bearing in mind the burden that growing data collection requirements can have on the provision of quality care and staff work satisfaction and burnout,^10,212 the CDs (and the items selected to represent each) are designed with “strategic parsimony” in mind. Although the CDs are inclusive in that they cover care quality, cost of care, staff quality of life, and general population health, only CDs and items undergirded by a solid evidence base and high value with regards to BHD’s mission and values, as determined by key stakeholders, were selected. Moreover, BHD attempted to make use of existing data collection and reporting mandates when selecting the final pool of items to reduce the measurement burden on staff and clients. Thus, the final set of CDs and items are designed to be comprehensive yet economical.

The CDs are deeply interrelated. Although each CD may be individually viewed as a valuable metric, improvements in any 1 CD will impact the others (eg, increasing care quality should impact population health, increasing staff quality of life should impact the quality of care). Moreover, this idea of interrelatedness acknowledges the need to view health systems and the populations they serve holistically, in that improvement is not simply the degree of change in any given metric (whether individually or collectively), but rather something more entirely. The concepts of value, quality, and health are complex, multidimensional, and dynamic, and the CDs that comprise these concepts should not be considered independently from one another. The CDs (and items) offered in this article are scalable in that they can be used at different levels of an organization depending on the question or stakeholder, and can be used individually or in combination with one another. Moreover, they are adaptable to a variety of risk-adjusted program, population health, and value-based evaluation models. It is hoped that the process articulated here, and the accompanying literature review, may benefit other public or government-run health systems in their own quality journey to operationalize the Quadruple Aim by developing a set of CDs.

Corresponding author: Walter Matthew Drymalski, PhD; [email protected].

Financial disclosures: None.

From Banner Health Corporation, Phoenix, AZ.

Background: Health care providers are routinely incentivized with pay-for-performance (P4P) metrics to increase the quality of care. In an inpatient setting, P4P models typically measure quality by attributing each patient’s outcome to a single provider even though many providers routinely care for the patient. This study investigates a new attribution approach aiming to distribute each outcome across all providers who provided care.

Methods: The methodology relies on a multi-membership model and is demonstrated in the Banner Health system using 3 clinical outcome measures (length of stay, 30-day readmissions, and mortality) and responses to 3 survey questions that measure a patient’s perception of their care. The new approach is compared to the “standard” method, which attributes each patient to only 1 provider.

Results: When ranking by clinical outcomes, both methods were concordant 72.1% to 82.1% of the time for top-half/bottom-half rankings, with a median percentile difference between 7 and 15. When ranking by survey scores, there was more agreement, with concordance between 84.1% and 86.6% and a median percentile difference between 11 and 13. Last, Pearson correlation coefficients of the paired percentiles ranged from 0.56 to 0.78.

Conclusion: The new approach provides a fairer solution when measuring provider performance.

Keywords: patient attribution; PAMM; PAPR; random effect model; pay for performance.

Providers practicing in hospitals are routinely evaluated based on their performance and, in many cases, are financially incentivized for a better-than-average performance within a pay-for-performance (P4P) model. The use of P4P models is based on the belief that they will “improve, motivate, and enhance providers to pursue aggressively and ultimately achieve the quality performance targets thus decreasing the number of medical errors with less malpractice events.”¹ Although P4P models continue to be a movement in health care, they have been challenging to implement.

One concern involves the general quality of implementation, such as defining metrics and targets, setting payout amounts, managing technology and market conditions, and gauging the level of transparency to the provider.² Another challenge, and the focus of this project, are concerns around measuring performance to avoid perceptions of unfairness. This concern can be minimized if the attribution is handled in a fairer way, by spreading it across all providers who affected the outcome, both in a positive or negative direction.³

To implement these models, the performance of providers needs to be measured and tracked periodically. This requires linking, or attributing, a patient’s outcome to a provider, which is almost always the attending or discharging provider (ie, a single provider).³ In this single-provider attribution approach, one provider will receive all the credit (good or bad) for their respective patients’ outcomes, even though the provider may have seen the patient only a fraction of the time during the hospitalization. Attributing outcomes—for example, length of stay (LOS), readmission rate, mortality rate, net promoter score (NPS)—using this approach reduces the validity of metrics designed to measure provider performance, especially in a rotating provider environment where many providers interact with and care for a patient. For example, the quality of providers’ interpersonal skills and competence were among the strongest determinants of patient satisfaction,⁴ but it is not credible that this is solely based on the last provider during a hospitalization.

Proportionally distributing the attribution of an outcome has been used successfully in other contexts. Typically, a statistical modeling approach using a multi-membership framework is used because it can handle the sometimes-complicated relationships within the hierarchy. It also allows for auxiliary variables to be introduced, which can help explain and control for exogenous effects.^5-7 For example, in the education setting, standardized testing is administered to students at defined years of schooling: at grades 4, 8, and 10, for instance. The progress of students, measured as the academic gains between test years, are proportionally attributed to all the teachers who the student has had between the test years. These partial attributions are combined to evaluate an overall teacher performance.^8,9

Although the multi-membership framework has been used in other industries, it has yet to be applied in measuring provider performance. The purpose of this project is to investigate the impact of using a multi-provider approach compared to the standard single-provider approach. The findings may lead to modifications in the way a provider’s performance is measured and, thus, how providers are compensated. A similar study investigated the impact of proportionally distributing patients’ outcomes across all rotating providers using a weighting method based on billing practices to measure the partial impact of each provider.³

This study is different in 2 fundamental ways. First, attribution is weighted based on the number of clinically documented interactions (via clinical notes) between a patient and all rotating providers during the hospitalization. Second, performance is measured via multi-membership models, which can estimate the effect (both positive and negative) that a provider has on an outcome, even when caring for a patient a fraction of the time during the hospitalization.

Methods

Setting

Banner Health is a non-profit, multi-hospital health care system across 6 states in the western United States that is uniquely positioned to study provider quality attribution models. It not only has a large number of providers and serves a broad patient population, but Banner Health also uses an instance of Cerner (Kansas City, MO), an enterprise-level electronic health record (EHR) system that connects all its facilities and allows for advanced analytics across its system.

For this study, we included only general medicine and surgery patients admitted and discharged from the inpatient setting between January 1, 2018, and December 31, 2018, who were between 18 and 89 years old at admission, and who had a LOS between 1 and 14 days. Visit- and patient-level data were collected from Cerner, while outcome data, and corresponding expected outcome data, were obtained from Premier, Inc. (Charlotte, NC) using their CareScience methodologies.¹⁰ To measure patient experience, response data were extracted from post-discharge surveys administered by InMoment (Salt Lake City, UT).

Provider Attribution Models

Provider Attribution by Physician of Record (PAPR). In the standard approach, denoted here as the PAPR model, 1 provider—typically the attending or discharging provider, which may be the same person—is attributed to the entire hospitalization. This provider is responsible for the patient’s care, and all patient outcomes are aggregated and attributed to the provider to gauge his or her performance. The PAPR model is the most popular form of attribution across many health care systems and is routinely used for P4P incentives.

In this study, the discharging provider was used when attributing hospitalizations using the PAPR model. Providers responsible for fewer than 12 discharges in the calendar year were excluded. Because of the directness of this type of attribution, the performance of 1 provider does not account for the performance of the other rotating providers during hospitalizations.

Provider Attribution by Multiple Membership (PAMM). In contrast, we introduce another attribution approach here that is designed to assign partial attribution to each provider who cares for the patient during the hospitalization. To aggregate the partial attributions, and possibly control for any exogenous or risk-based factors, a multiple-membership, or multi-member (MM), model is used. The MM model can measure the effect of a provider on an outcome even when the patient-to-provider relationship is complex, such as in a rotating provider environment.⁸

The purpose of this study is to compare attribution models and to determine whether there are meaningful differences between them. Therefore, for comparison purposes, the same discharging providers using the PAPR approach are eligible for the PAMM approach, so that both attribution models are using the same set of providers. All other providers are excluded because their performance would not be comparable to the PAPR approach.

While there are many ways to document provider-to-patient interactions, 2 methods are available in almost all health care systems. The first method is to link a provider’s billing charges to each patient-day combination. This approach limits the attribution to 1 provider per patient per day because multiple rotating providers cannot charge for the same patient-day combination.³ However, many providers interact with a patient on the same day, so using this approach excludes non-billed provider-to-patient interactions.

The second method, which was used in this study, relies on documented clinical notes within the EHR to determine how attribution is shared. In this approach, attribution is weighted based on the authorship of 3 types of eligible clinical notes: admitting history/physical notes (during admission), progress notes (during subsequent days), and discharge summary notes (during final discharge). This will (likely) result in many providers being linked to a patient on each day, which better reflects the clinical setting (Figure). Recently, clinical notes were used to attribute care of patients in an inpatient setting, and it was found that this approach provides a reliable way of tracking interactions and assigning ownership.¹¹

The provider-level attribution weights are based on the share of authorships of eligible note types. Specifically, for each provider j, let a_ij be the total count of eligible note types for hospitalization i authored by provider j, and let a_i be the overall total count of eligible note types for hospitalization i. Then the attribution weight is

(Eq. 1)

for hospitalization i and provider j. Note that ∑_jw_ij = 1: in other words, the total attribution, summed across all providers, is constrained to be 1 for each hospitalization.

Patient Outcomes

Outcomes were chosen based on their routine use in health care systems as standards when evaluating provider performance. This study included 6 outcomes: inpatient LOS, inpatient mortality, 30-day inpatient readmission, and patient responses from 3 survey questions. These outcomes can be collected without any manual chart reviews, and therefore are viewed as objective outcomes of provider performance.

Each outcome was aggregated for each provider using both attribution methods independently. For the PAPR method, observed-to-expected (OE) indices for LOS, mortality, and readmissions were calculated along with average patient survey scores. For the PAMM method, provider-level random effects from the fitted models were used. In both cases, the calculated measures were used for ranking purposes when determining top (or bottom) providers for each outcome.

Individual Provider Metrics for the PAPR Method

Inpatient LOS Index. Hospital inpatient LOS was measured as the number of days between admission date and discharge date. For each hospital visit, an expected LOS was determined using Premier’s CareScience Analytics (CSA) risk-adjustment methodology.¹⁰ The CSA methodology for LOS incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let o_ij and e_ij be the observed and expected LOS, respectively, for hospitalization i = 1,…,n_j attributed to provider j. Then the inpatient LOS index for provider j is L_j = ∑_io_ij⁄∑_ie_ij.

Inpatient Mortality Index. Inpatient mortality was defined as the death of the patient during hospitalization. For each hospitalization, an expected mortality probability was determined using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for mortality incorporates a patient’s demographics and comorbidities.

Just as before, let n_j be the number of hospitalizations attributed to provider j. Let m_ij = 1 if the patient died during hospitalization i = 1, … , n_j attributed to provider j; m_ij = 0 otherwise. Let p_ij^(m) be the corresponding expected mortality probability. Then the inpatient mortality index for provider j is M_j = ∑_im_ij⁄∑_ip_ij^(m).

30-Day Inpatient Readmission Index. A 30-day inpatient readmission was defined as the event when a patient is discharged and readmits back into the inpatient setting within 30 days. The inclusion criteria defined by the Centers for Medicare and Medicaid Services (CMS) all-cause hospital-wide readmission measure was used and, consequently, planned readmissions were excluded.¹² Readmissions could occur at any Banner hospital, including the same hospital. For each hospital visit, an expected readmission probability was derived using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for readmissions incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let r_ij = 1 if the patient had a readmission following hospitalization i = 1, … , n_j attributed to provider j; r_ij = 0 otherwise. Let p_ij^(r) be the corresponding expected readmission probability. Then the 30-day inpatient readmission index for provider j is R_j = ∑_ir_ij ⁄∑_ip_ij^(r).

Patient Survey Scores. The satisfaction of the patient’s experience during hospitalization was measured via post-discharge surveys administered by InMoment. Two survey questions were selected because they related directly to a provider’s interaction with the patient: “My interactions with doctors were excellent” (Doctor) and “I received the best possible care” (Care). A third question, “I would recommend this hospital to my family and friends,” was selected as a proxy measure of the overall experience and, in the aggregate, is referred to as the net promoter score (NPS).^13,14 The responses were measured on an 11-point Likert scale, ranging from “Strongly Disagree” (0) to “Strongly Agree” (10); “N/A” or missing responses were excluded.

The Likert responses were coded to 3 discrete values as follows: if the value was between 0 and 6, then -1 (ie, detractor); between 7 and 8 (ie, neutral), then 0; otherwise 1 (ie, promoter). Averaging these coded responses results in a patient survey score for each question. Specifically, let n_j be the number of hospitalizations attributed to provider j in which the patient responded to the survey question. Let s_ij ∈{−1, 0, 1} be the coded response linked to hospitalization i = 1, … , n_j attributed to provider j. Then the patient experience score for provider j is S_j = ∑_is_ij⁄n_j.

Handling Ties in Provider Performance Measures. Because ties can occur in the PAPR approach for all measures, a tie-breaking strategy is needed. For LOS indices, ties are less likely because their numerator is strictly greater than 0, and expected LOS values are typically distinct enough. Indeed, no ties were found in this study for LOS indices. However, mortality and readmission indices can routinely result in ties when the best possible index is achieved, such as 0 deaths or readmissions among attributed hospitalizations. To help differentiate between those indices in the PAPR approach, the total estimated risk (denominator) was utilized as a secondary scoring criterion.

Mortality and readmission metrics were addressed by sorting first by the outcome (mortality index), and second by the denominator (total estimated risk). For example, if provider A has the same mortality rate as provider B, then provider A would be ranked higher if the denominator was larger, indicating a higher risk for mortality.

Similarly, it was very common for providers to have the same overall average rating for a survey question. Therefore, the denominator (number of respondents) was used to break ties. However, the denominator sorting was bidirectional. For example, if the tied score was positive (more promoters than detractors) for providers A and B, then provider A would be ranked higher if the denominator was larger. Conversely, if the tied score between providers A and B was neutral or negative (more detractors than promoters), then provider A would be ranked lower if the denominator was larger.

Individual Provider Metrics for the PAMM Method

For the PAMM method, model-based metrics were derived using a MM model.⁸ Specifically, let J be the number of rotating providers in a health care system. Let Y_i be an outcome of interest from hospitalization i, X_1i, …, X_pi be fixed effects or covariates, and ß₁, …, ß_p be the coefficients for the respective covariates. Then the generalized MM statistical model is

(Eq. 2)

where g(μ_i) is a link function between the mean of the outcome, μ_i, and its linear predictor, ß₀, is the marginal intercept, w_ij represents the attribution weight of provider j on hospitalization i (described in Equation 1), and γ_j represents the random effect of provider j on the outcome with γ_j~N(0,σ_γ²).

For the mortality and readmission binary outcomes, logistic regression was performed using a logit link function, with the corresponding expected probability as the only fixed covariate. The expected probabilities were first converted into odds and then log-transformed before entering the model. For LOS, Poisson regression was performed using a log link function with the log-transformed expected LOS as the only fixed covariate. For coded patient experience responses, an ordered logistic regression was performed using a cumulative logit link function (no fixed effects were added).

MM Model-based Metrics. Each fitted MM model produces a predicted random effect for each provider. The provider-specific random effects can be interpreted as the unobserved influence of each provider on the outcome after controlling for any fixed effect included in the model. Therefore, the provider-specific random effects were used to evaluate the relative provider performance, which is analogous to the individual provider-level metrics used in the PAPR method.

Measuring provider performance using a MM model is more flexible and robust to outliers compared to the standard approach using OE indices or simple averages. First, although not investigated here, the effect of patient-, visit-, provider-, and/or temporal-level covariates can be controlled when evaluating provider performance. For example, a patient’s socioeconomic status, a provider’s workload, and seasonal factors can be added to the MM model. These external factors are not accounted for in OE indices.

Another advantage of using predicted random effects is the concept of “shrinkage.” The process of estimating random effects inherently accounts for small sample sizes (when providers do not treat a large enough sample of patients) and/or when there is a large ratio of patient variance to provider variance (for instance, when patient outcome variability is much higher compared to provider performance variability). In both cases, the estimation of the random effect is pulled ever closer to 0, signaling that the provider performance is closer to the population average. See Henderson¹⁵ and Mood¹⁶ for further details.

In contrast, OE indices can result in unreliable estimates when a provider has not cared for many patients. This is especially prevalent when the outcome is binary with a low probability of occurring, such as mortality. Indeed, provider-level mortality OE indices are routinely 0 when the patient counts are low, which skews performance rankings unfairly. Finally, OE indices also ignore the magnitude of the variance of an outcome between providers and patients, which can be large.

Comparison Methodology

In this study, we seek to compare the 2 methods of attribution, PAPR and PAMM, to determine whether there are meaningful differences between them when measuring provider performance. Using retrospective data described in the next section, each attribution method was used independently to derive provider-level metrics. To assess relative performance, percentiles were assigned to each provider based on their metric values so that, in the end, there were 2 percentile ranks for each provider for each metric.

Using these paired percentiles, we derived the following measures of concordance, similar to Herzke, Michtalik³: (1) the percent concordance measure—defined as the number of providers who landed in the top half (greater than the median) or bottom half under both attribution models—divided by the total number of providers; (2) the median of the absolute difference in percentiles under both attribution models; and (3) the Pearson correlation coefficient of the paired provider ranks. The first measure is a global measure of concordance between the 2 approaches and would be expected to be 50% by chance. The second measure gauges how an individual provider’s rank is affected by the change in attribution methodologies. The third measure is a statistical measure of linear correlation of the paired percentiles and was not included in the Herzke, Michtalik³ study.

All statistical analyses were performed on SAS (version 9.4; Cary, NC) and the MM models were fitted using PROC GLIMMIX with the EFFECT statement. The Banner Health Institutional Review Board approved this study.

Results

Descriptive Statistics

A total of 58,730 hospitalizations were included, of which care was provided by 963 unique providers across 25 acute care and critical access hospitals. Table 1 contains patient characteristics, and Table 2 depicts overall unadjusted outcomes. Providers responsible for less than 12 discharges in the calendar year were excluded from both approaches. Also, some hospitalizations were excluded when expected values were not available.

Multi-Membership Model Results

Table 3 displays the results after independently fitting MM models to each of the 3 clinical outcomes. Along with a marginal intercept, the only covariate in each model was the corresponding expected value after a transformation. This was added to use the same information that is typically used in OE indices, therefore allowing for a proper comparison between the 2 attribution methods. The provider-level variance represents the between-provider variation and measures the amount of influence providers have on the corresponding outcome after controlling for any covariates in the model. A provider-level variance of 0 would indicate that providers do not have any influence on the outcome. While the mortality and readmission model results can be compared to each other, the LOS model cannot given its different scale and transformation altogether.

The results in Table 3 suggest that each expected value covariate is highly correlated with its corresponding outcome, which is the anticipated conclusion given that they are constructed in this fashion. The estimated provider-level variances indicate that, after including an expected value in the model, providers have less of an influence on a patient’s LOS and likelihood of being readmitted. On the other hand, the results suggest that providers have much more influence on the likelihood of a patient dying in the hospital, even after controlling for an expected mortality covariate.

Table 4 shows the results after independently fitting MM-ordered logistic models to each of the 3 survey questions. The similar provider-level variances suggest that providers have the same influence on the patient’s perception of the quality of their interactions with the doctor (Doctor), the quality of the care they received (Care), and their likelihood to recommend a friend or family member to the hospital (NPS).

Comparison Results Between Both Attribution Methods

Table 5 compares the 2 attribution methods when ranking providers based on their performance on each outcome measure. The comparison metrics gauge how well the 2 methods agree overall (percent concordance), agree at the provider level (absolute percentile difference and interquartile range [IQR]), and how the paired percentiles linearly correlate to each other (Pearson correlation coefficient).

LOS, by a small margin, had the lowest concordance of clinical outcomes (72.1%), followed by mortality (75.9%) and readmissions (82.1%). Generally, the survey scores had higher percent concordance than the clinical outcome measures, with Doctor at 84.1%, Care at 85.9%, and NPS having the highest percent concordance at 86.6%. Given that by chance the percent concordance is expected to be 50%, there was notable discordance, especially with the clinical outcome measures. Using LOS performance as an example, one attribution methodology would rank a provider in the top half or bottom half, while the other attribution methodology would rank the same provider exactly the opposite way about 28% of the time.

The median absolute percentile difference between the 2 methods was more modest (between 7 and 15). Still, there were some providers whose performance ranking was heavily impacted by the attribution methodology that was used. This was especially true when evaluating performance for certain clinical measures, where the attribution method that was used could change the provider performance percentile by up to 90 levels.

The paired percentiles were positively correlated when ranking performance using any of the 6 measures. This suggests that both methodologies assess performance generally in the same direction, irrespective of the methodology and measure. We did not investigate more complex correlation measures and left this for future research.

It should be noted that ties occurred much more frequently with the PAPR method than when using PAMM and therefore required tie-breaking rules to be designed. Given the nature of OE indices, PAPR methodology is especially sensitive to ties whenever the measure includes counting the number of events (for example, mortality and readmissions) and whenever there are many providers with very few attributed patients. On the other hand, using the PAMM method is much more robust against ties given that the summation of all the weighted attributed outcomes will rarely result in ties, even with a nominal set of providers.

In this study, the PAMM methodology was introduced and was used to assess relative provider performance on 3 clinical outcome measures and 3 patient survey scores. The new approach aims to distribute each outcome among all providers who provided care for a patient in an inpatient setting. Clinical notes were used to account for patient-to-provider interactions, and fitted MM statistical models were used to compute the effects that each provider had on each outcome. The provider effect was introduced as a random effect, and the set of predicted random effects was used to rank the performance of each provider.

The PAMM approach was compared to the more traditional methodology, PAPR, where each patient is attributed to only 1 provider: the discharging physician in this study. Using this approach, OE indices of clinical outcomes and averages of survey scores were used to rank the performance of each provider. This approach resulted in many ties, which were broken based on the number of hospitalizations, although other tie-breaking methods may be used in practice.

Both methodologies showed modest concordance with each other for the clinical outcomes, but higher concordance for the patient survey scores. This was also true when using the Pearson correlation coefficient to assess agreement. The 1 outcome measure that showed the least concordance and least linear correlation between methods was LOS, which would suggest that LOS performance is more sensitive to the attribution methodology that is used. However, it was the least concordant by a small margin.

Furthermore, although the medians of the absolute percentile differences were small, there were some providers who had large deviations, suggesting that some providers would move from being shown as high-performers to low-performers and vice versa based on the chosen attribution method. We investigated examples of this and determined that the root cause was the difference in effective sample sizes for a provider. For the PAPR method, the effective sample size is simply the number of hospitalizations attributed to the provider. For the PAMM method, the effective sample size is the sum of all non-zero weights across all hospitalizations where the provider cared for a patient. By and large, the PAMM methodology provides more information of the provider effect on an outcome than the PAPR approach because every provider-patient interaction is considered. For example, providers who do not routinely discharge patients, but often care for patients, will have rankings that differ dramatically between the 2 methods.

The PAMM methodology has many statistical advantages that were not fully utilized in this comparative study. For example, we did not include any covariates in the MM models except for the expected value of the outcome, when it was available. Still, it is known that other covariates can impact an outcome as well, such as the patient’s age, socioeconomic indicators, existing chronic conditions, and severity of hospitalization, which can be added to the MM models as fixed effects. In this way, the PAMM approach can control for these other covariates, which are typically outside of the control of providers but typically ignored using OE indices. Therefore, using the PAMM approach would provide a fairer comparison of provider performance.

Using the PAMM method, most providers had a large sample size to assess their performance once all the weighted interactions were included. Still, there were a few who did not care for many patients for a variety of reasons. In these scenarios, MM models “borrow” strength from other providers to produce a more robust predicted provider effect by using a weighted average between the overall population trend and the specific provider outcomes (see Rao and Molina¹⁷). As a result, PAMM is a more suitable approach when the sample sizes of patients attributed to providers can be small.

One of the most interesting findings of this study was the relative size of the provider-level variance to the size of the fixed effect in each model (Table 3). Except for mortality, these variances suggest that there is a small difference in performance from one provider to another. However, these should be interpreted as the variance when only 1 provider is involved in the care of a patient. When multiple providers are involved, using basic statistical theory, the overall provider-level variance will be σ_γ² ∑w_ij² (see Equation 2). For example, the estimated variance among providers for LOS was 0.03 (on a log scale), but, using the scenario in the Figure, the overall provider-level variance for this hospitalization will be 0.03 (0.375²+ 0.125²+ 0.5²) = 0.012. Hence, the combined effect of providers on LOS is less than would be expected. Indeed, as more providers are involved with a patient’s care, the more their combined influence on an outcome is diluted.

In this study, the PAMM approach placed an equal weight on all provider-patient interactions via clinical note authorship, but that may not be optimal in some settings. For example, it may make more sense to set a higher weight on the provider who admitted or discharged the patient while placing less (or 0) weight on all other interactions. In the extreme, if the full weight were placed on 1 provider interaction (eg, during discharge, then the MM model would be reduced to a one-way random effects model. The flexibility of weighting interactions is a feature of the PAMM approach, but any weighting framework must be transparent to the providers before implementation.

Conclusion

This study demonstrates that the PAMM approach is a feasible option within a large health care organization. For P4P programs to be successful, providers must be able to trust that their performance will be fairly assessed and that all provider-patient interactions are captured to provide a full comparison amongst their peers. The PAMM methodology is one solution to spread the positive (and negative) outcomes across all providers who cared for a patient and therefore, if implemented, would add trust and fairness when measuring and assessing provider performance.

Acknowledgments: The authors thank Barrie Bradley for his support in the initial stages of this research and Dr. Syed Ismail Jafri for his help and support on the standard approaches of assessing and measuring provider performances.

Corresponding author: Rachel Ginn, MS, Banner Health Corporation, 2901 N. Central Ave., Phoenix, AZ 85012; [email protected].

Financial disclosures: None.

From Banner Health Corporation, Phoenix, AZ.

Background: Health care providers are routinely incentivized with pay-for-performance (P4P) metrics to increase the quality of care. In an inpatient setting, P4P models typically measure quality by attributing each patient’s outcome to a single provider even though many providers routinely care for the patient. This study investigates a new attribution approach aiming to distribute each outcome across all providers who provided care.

Methods: The methodology relies on a multi-membership model and is demonstrated in the Banner Health system using 3 clinical outcome measures (length of stay, 30-day readmissions, and mortality) and responses to 3 survey questions that measure a patient’s perception of their care. The new approach is compared to the “standard” method, which attributes each patient to only 1 provider.

Results: When ranking by clinical outcomes, both methods were concordant 72.1% to 82.1% of the time for top-half/bottom-half rankings, with a median percentile difference between 7 and 15. When ranking by survey scores, there was more agreement, with concordance between 84.1% and 86.6% and a median percentile difference between 11 and 13. Last, Pearson correlation coefficients of the paired percentiles ranged from 0.56 to 0.78.

Conclusion: The new approach provides a fairer solution when measuring provider performance.

Keywords: patient attribution; PAMM; PAPR; random effect model; pay for performance.

Providers practicing in hospitals are routinely evaluated based on their performance and, in many cases, are financially incentivized for a better-than-average performance within a pay-for-performance (P4P) model. The use of P4P models is based on the belief that they will “improve, motivate, and enhance providers to pursue aggressively and ultimately achieve the quality performance targets thus decreasing the number of medical errors with less malpractice events.”¹ Although P4P models continue to be a movement in health care, they have been challenging to implement.

One concern involves the general quality of implementation, such as defining metrics and targets, setting payout amounts, managing technology and market conditions, and gauging the level of transparency to the provider.² Another challenge, and the focus of this project, are concerns around measuring performance to avoid perceptions of unfairness. This concern can be minimized if the attribution is handled in a fairer way, by spreading it across all providers who affected the outcome, both in a positive or negative direction.³

To implement these models, the performance of providers needs to be measured and tracked periodically. This requires linking, or attributing, a patient’s outcome to a provider, which is almost always the attending or discharging provider (ie, a single provider).³ In this single-provider attribution approach, one provider will receive all the credit (good or bad) for their respective patients’ outcomes, even though the provider may have seen the patient only a fraction of the time during the hospitalization. Attributing outcomes—for example, length of stay (LOS), readmission rate, mortality rate, net promoter score (NPS)—using this approach reduces the validity of metrics designed to measure provider performance, especially in a rotating provider environment where many providers interact with and care for a patient. For example, the quality of providers’ interpersonal skills and competence were among the strongest determinants of patient satisfaction,⁴ but it is not credible that this is solely based on the last provider during a hospitalization.

Proportionally distributing the attribution of an outcome has been used successfully in other contexts. Typically, a statistical modeling approach using a multi-membership framework is used because it can handle the sometimes-complicated relationships within the hierarchy. It also allows for auxiliary variables to be introduced, which can help explain and control for exogenous effects.^5-7 For example, in the education setting, standardized testing is administered to students at defined years of schooling: at grades 4, 8, and 10, for instance. The progress of students, measured as the academic gains between test years, are proportionally attributed to all the teachers who the student has had between the test years. These partial attributions are combined to evaluate an overall teacher performance.^8,9

Although the multi-membership framework has been used in other industries, it has yet to be applied in measuring provider performance. The purpose of this project is to investigate the impact of using a multi-provider approach compared to the standard single-provider approach. The findings may lead to modifications in the way a provider’s performance is measured and, thus, how providers are compensated. A similar study investigated the impact of proportionally distributing patients’ outcomes across all rotating providers using a weighting method based on billing practices to measure the partial impact of each provider.³

This study is different in 2 fundamental ways. First, attribution is weighted based on the number of clinically documented interactions (via clinical notes) between a patient and all rotating providers during the hospitalization. Second, performance is measured via multi-membership models, which can estimate the effect (both positive and negative) that a provider has on an outcome, even when caring for a patient a fraction of the time during the hospitalization.

Methods

Setting

Banner Health is a non-profit, multi-hospital health care system across 6 states in the western United States that is uniquely positioned to study provider quality attribution models. It not only has a large number of providers and serves a broad patient population, but Banner Health also uses an instance of Cerner (Kansas City, MO), an enterprise-level electronic health record (EHR) system that connects all its facilities and allows for advanced analytics across its system.

For this study, we included only general medicine and surgery patients admitted and discharged from the inpatient setting between January 1, 2018, and December 31, 2018, who were between 18 and 89 years old at admission, and who had a LOS between 1 and 14 days. Visit- and patient-level data were collected from Cerner, while outcome data, and corresponding expected outcome data, were obtained from Premier, Inc. (Charlotte, NC) using their CareScience methodologies.¹⁰ To measure patient experience, response data were extracted from post-discharge surveys administered by InMoment (Salt Lake City, UT).

Provider Attribution Models

Provider Attribution by Physician of Record (PAPR). In the standard approach, denoted here as the PAPR model, 1 provider—typically the attending or discharging provider, which may be the same person—is attributed to the entire hospitalization. This provider is responsible for the patient’s care, and all patient outcomes are aggregated and attributed to the provider to gauge his or her performance. The PAPR model is the most popular form of attribution across many health care systems and is routinely used for P4P incentives.

In this study, the discharging provider was used when attributing hospitalizations using the PAPR model. Providers responsible for fewer than 12 discharges in the calendar year were excluded. Because of the directness of this type of attribution, the performance of 1 provider does not account for the performance of the other rotating providers during hospitalizations.

Provider Attribution by Multiple Membership (PAMM). In contrast, we introduce another attribution approach here that is designed to assign partial attribution to each provider who cares for the patient during the hospitalization. To aggregate the partial attributions, and possibly control for any exogenous or risk-based factors, a multiple-membership, or multi-member (MM), model is used. The MM model can measure the effect of a provider on an outcome even when the patient-to-provider relationship is complex, such as in a rotating provider environment.⁸

The purpose of this study is to compare attribution models and to determine whether there are meaningful differences between them. Therefore, for comparison purposes, the same discharging providers using the PAPR approach are eligible for the PAMM approach, so that both attribution models are using the same set of providers. All other providers are excluded because their performance would not be comparable to the PAPR approach.

While there are many ways to document provider-to-patient interactions, 2 methods are available in almost all health care systems. The first method is to link a provider’s billing charges to each patient-day combination. This approach limits the attribution to 1 provider per patient per day because multiple rotating providers cannot charge for the same patient-day combination.³ However, many providers interact with a patient on the same day, so using this approach excludes non-billed provider-to-patient interactions.

The second method, which was used in this study, relies on documented clinical notes within the EHR to determine how attribution is shared. In this approach, attribution is weighted based on the authorship of 3 types of eligible clinical notes: admitting history/physical notes (during admission), progress notes (during subsequent days), and discharge summary notes (during final discharge). This will (likely) result in many providers being linked to a patient on each day, which better reflects the clinical setting (Figure). Recently, clinical notes were used to attribute care of patients in an inpatient setting, and it was found that this approach provides a reliable way of tracking interactions and assigning ownership.¹¹

The provider-level attribution weights are based on the share of authorships of eligible note types. Specifically, for each provider j, let a_ij be the total count of eligible note types for hospitalization i authored by provider j, and let a_i be the overall total count of eligible note types for hospitalization i. Then the attribution weight is

(Eq. 1)

for hospitalization i and provider j. Note that ∑_jw_ij = 1: in other words, the total attribution, summed across all providers, is constrained to be 1 for each hospitalization.

Patient Outcomes

Outcomes were chosen based on their routine use in health care systems as standards when evaluating provider performance. This study included 6 outcomes: inpatient LOS, inpatient mortality, 30-day inpatient readmission, and patient responses from 3 survey questions. These outcomes can be collected without any manual chart reviews, and therefore are viewed as objective outcomes of provider performance.

Each outcome was aggregated for each provider using both attribution methods independently. For the PAPR method, observed-to-expected (OE) indices for LOS, mortality, and readmissions were calculated along with average patient survey scores. For the PAMM method, provider-level random effects from the fitted models were used. In both cases, the calculated measures were used for ranking purposes when determining top (or bottom) providers for each outcome.

Individual Provider Metrics for the PAPR Method

Inpatient LOS Index. Hospital inpatient LOS was measured as the number of days between admission date and discharge date. For each hospital visit, an expected LOS was determined using Premier’s CareScience Analytics (CSA) risk-adjustment methodology.¹⁰ The CSA methodology for LOS incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let o_ij and e_ij be the observed and expected LOS, respectively, for hospitalization i = 1,…,n_j attributed to provider j. Then the inpatient LOS index for provider j is L_j = ∑_io_ij⁄∑_ie_ij.

Inpatient Mortality Index. Inpatient mortality was defined as the death of the patient during hospitalization. For each hospitalization, an expected mortality probability was determined using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for mortality incorporates a patient’s demographics and comorbidities.

Just as before, let n_j be the number of hospitalizations attributed to provider j. Let m_ij = 1 if the patient died during hospitalization i = 1, … , n_j attributed to provider j; m_ij = 0 otherwise. Let p_ij^(m) be the corresponding expected mortality probability. Then the inpatient mortality index for provider j is M_j = ∑_im_ij⁄∑_ip_ij^(m).

30-Day Inpatient Readmission Index. A 30-day inpatient readmission was defined as the event when a patient is discharged and readmits back into the inpatient setting within 30 days. The inclusion criteria defined by the Centers for Medicare and Medicaid Services (CMS) all-cause hospital-wide readmission measure was used and, consequently, planned readmissions were excluded.¹² Readmissions could occur at any Banner hospital, including the same hospital. For each hospital visit, an expected readmission probability was derived using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for readmissions incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let r_ij = 1 if the patient had a readmission following hospitalization i = 1, … , n_j attributed to provider j; r_ij = 0 otherwise. Let p_ij^(r) be the corresponding expected readmission probability. Then the 30-day inpatient readmission index for provider j is R_j = ∑_ir_ij ⁄∑_ip_ij^(r).

Patient Survey Scores. The satisfaction of the patient’s experience during hospitalization was measured via post-discharge surveys administered by InMoment. Two survey questions were selected because they related directly to a provider’s interaction with the patient: “My interactions with doctors were excellent” (Doctor) and “I received the best possible care” (Care). A third question, “I would recommend this hospital to my family and friends,” was selected as a proxy measure of the overall experience and, in the aggregate, is referred to as the net promoter score (NPS).^13,14 The responses were measured on an 11-point Likert scale, ranging from “Strongly Disagree” (0) to “Strongly Agree” (10); “N/A” or missing responses were excluded.

The Likert responses were coded to 3 discrete values as follows: if the value was between 0 and 6, then -1 (ie, detractor); between 7 and 8 (ie, neutral), then 0; otherwise 1 (ie, promoter). Averaging these coded responses results in a patient survey score for each question. Specifically, let n_j be the number of hospitalizations attributed to provider j in which the patient responded to the survey question. Let s_ij ∈{−1, 0, 1} be the coded response linked to hospitalization i = 1, … , n_j attributed to provider j. Then the patient experience score for provider j is S_j = ∑_is_ij⁄n_j.

Handling Ties in Provider Performance Measures. Because ties can occur in the PAPR approach for all measures, a tie-breaking strategy is needed. For LOS indices, ties are less likely because their numerator is strictly greater than 0, and expected LOS values are typically distinct enough. Indeed, no ties were found in this study for LOS indices. However, mortality and readmission indices can routinely result in ties when the best possible index is achieved, such as 0 deaths or readmissions among attributed hospitalizations. To help differentiate between those indices in the PAPR approach, the total estimated risk (denominator) was utilized as a secondary scoring criterion.

Mortality and readmission metrics were addressed by sorting first by the outcome (mortality index), and second by the denominator (total estimated risk). For example, if provider A has the same mortality rate as provider B, then provider A would be ranked higher if the denominator was larger, indicating a higher risk for mortality.

Similarly, it was very common for providers to have the same overall average rating for a survey question. Therefore, the denominator (number of respondents) was used to break ties. However, the denominator sorting was bidirectional. For example, if the tied score was positive (more promoters than detractors) for providers A and B, then provider A would be ranked higher if the denominator was larger. Conversely, if the tied score between providers A and B was neutral or negative (more detractors than promoters), then provider A would be ranked lower if the denominator was larger.

Individual Provider Metrics for the PAMM Method

For the PAMM method, model-based metrics were derived using a MM model.⁸ Specifically, let J be the number of rotating providers in a health care system. Let Y_i be an outcome of interest from hospitalization i, X_1i, …, X_pi be fixed effects or covariates, and ß₁, …, ß_p be the coefficients for the respective covariates. Then the generalized MM statistical model is

(Eq. 2)

where g(μ_i) is a link function between the mean of the outcome, μ_i, and its linear predictor, ß₀, is the marginal intercept, w_ij represents the attribution weight of provider j on hospitalization i (described in Equation 1), and γ_j represents the random effect of provider j on the outcome with γ_j~N(0,σ_γ²).

For the mortality and readmission binary outcomes, logistic regression was performed using a logit link function, with the corresponding expected probability as the only fixed covariate. The expected probabilities were first converted into odds and then log-transformed before entering the model. For LOS, Poisson regression was performed using a log link function with the log-transformed expected LOS as the only fixed covariate. For coded patient experience responses, an ordered logistic regression was performed using a cumulative logit link function (no fixed effects were added).

MM Model-based Metrics. Each fitted MM model produces a predicted random effect for each provider. The provider-specific random effects can be interpreted as the unobserved influence of each provider on the outcome after controlling for any fixed effect included in the model. Therefore, the provider-specific random effects were used to evaluate the relative provider performance, which is analogous to the individual provider-level metrics used in the PAPR method.

Measuring provider performance using a MM model is more flexible and robust to outliers compared to the standard approach using OE indices or simple averages. First, although not investigated here, the effect of patient-, visit-, provider-, and/or temporal-level covariates can be controlled when evaluating provider performance. For example, a patient’s socioeconomic status, a provider’s workload, and seasonal factors can be added to the MM model. These external factors are not accounted for in OE indices.

Another advantage of using predicted random effects is the concept of “shrinkage.” The process of estimating random effects inherently accounts for small sample sizes (when providers do not treat a large enough sample of patients) and/or when there is a large ratio of patient variance to provider variance (for instance, when patient outcome variability is much higher compared to provider performance variability). In both cases, the estimation of the random effect is pulled ever closer to 0, signaling that the provider performance is closer to the population average. See Henderson¹⁵ and Mood¹⁶ for further details.

In contrast, OE indices can result in unreliable estimates when a provider has not cared for many patients. This is especially prevalent when the outcome is binary with a low probability of occurring, such as mortality. Indeed, provider-level mortality OE indices are routinely 0 when the patient counts are low, which skews performance rankings unfairly. Finally, OE indices also ignore the magnitude of the variance of an outcome between providers and patients, which can be large.

Comparison Methodology

In this study, we seek to compare the 2 methods of attribution, PAPR and PAMM, to determine whether there are meaningful differences between them when measuring provider performance. Using retrospective data described in the next section, each attribution method was used independently to derive provider-level metrics. To assess relative performance, percentiles were assigned to each provider based on their metric values so that, in the end, there were 2 percentile ranks for each provider for each metric.

Using these paired percentiles, we derived the following measures of concordance, similar to Herzke, Michtalik³: (1) the percent concordance measure—defined as the number of providers who landed in the top half (greater than the median) or bottom half under both attribution models—divided by the total number of providers; (2) the median of the absolute difference in percentiles under both attribution models; and (3) the Pearson correlation coefficient of the paired provider ranks. The first measure is a global measure of concordance between the 2 approaches and would be expected to be 50% by chance. The second measure gauges how an individual provider’s rank is affected by the change in attribution methodologies. The third measure is a statistical measure of linear correlation of the paired percentiles and was not included in the Herzke, Michtalik³ study.

All statistical analyses were performed on SAS (version 9.4; Cary, NC) and the MM models were fitted using PROC GLIMMIX with the EFFECT statement. The Banner Health Institutional Review Board approved this study.

Results

Descriptive Statistics

A total of 58,730 hospitalizations were included, of which care was provided by 963 unique providers across 25 acute care and critical access hospitals. Table 1 contains patient characteristics, and Table 2 depicts overall unadjusted outcomes. Providers responsible for less than 12 discharges in the calendar year were excluded from both approaches. Also, some hospitalizations were excluded when expected values were not available.

Multi-Membership Model Results

Table 3 displays the results after independently fitting MM models to each of the 3 clinical outcomes. Along with a marginal intercept, the only covariate in each model was the corresponding expected value after a transformation. This was added to use the same information that is typically used in OE indices, therefore allowing for a proper comparison between the 2 attribution methods. The provider-level variance represents the between-provider variation and measures the amount of influence providers have on the corresponding outcome after controlling for any covariates in the model. A provider-level variance of 0 would indicate that providers do not have any influence on the outcome. While the mortality and readmission model results can be compared to each other, the LOS model cannot given its different scale and transformation altogether.

The results in Table 3 suggest that each expected value covariate is highly correlated with its corresponding outcome, which is the anticipated conclusion given that they are constructed in this fashion. The estimated provider-level variances indicate that, after including an expected value in the model, providers have less of an influence on a patient’s LOS and likelihood of being readmitted. On the other hand, the results suggest that providers have much more influence on the likelihood of a patient dying in the hospital, even after controlling for an expected mortality covariate.

Table 4 shows the results after independently fitting MM-ordered logistic models to each of the 3 survey questions. The similar provider-level variances suggest that providers have the same influence on the patient’s perception of the quality of their interactions with the doctor (Doctor), the quality of the care they received (Care), and their likelihood to recommend a friend or family member to the hospital (NPS).

Comparison Results Between Both Attribution Methods

Table 5 compares the 2 attribution methods when ranking providers based on their performance on each outcome measure. The comparison metrics gauge how well the 2 methods agree overall (percent concordance), agree at the provider level (absolute percentile difference and interquartile range [IQR]), and how the paired percentiles linearly correlate to each other (Pearson correlation coefficient).

LOS, by a small margin, had the lowest concordance of clinical outcomes (72.1%), followed by mortality (75.9%) and readmissions (82.1%). Generally, the survey scores had higher percent concordance than the clinical outcome measures, with Doctor at 84.1%, Care at 85.9%, and NPS having the highest percent concordance at 86.6%. Given that by chance the percent concordance is expected to be 50%, there was notable discordance, especially with the clinical outcome measures. Using LOS performance as an example, one attribution methodology would rank a provider in the top half or bottom half, while the other attribution methodology would rank the same provider exactly the opposite way about 28% of the time.

The median absolute percentile difference between the 2 methods was more modest (between 7 and 15). Still, there were some providers whose performance ranking was heavily impacted by the attribution methodology that was used. This was especially true when evaluating performance for certain clinical measures, where the attribution method that was used could change the provider performance percentile by up to 90 levels.

The paired percentiles were positively correlated when ranking performance using any of the 6 measures. This suggests that both methodologies assess performance generally in the same direction, irrespective of the methodology and measure. We did not investigate more complex correlation measures and left this for future research.

It should be noted that ties occurred much more frequently with the PAPR method than when using PAMM and therefore required tie-breaking rules to be designed. Given the nature of OE indices, PAPR methodology is especially sensitive to ties whenever the measure includes counting the number of events (for example, mortality and readmissions) and whenever there are many providers with very few attributed patients. On the other hand, using the PAMM method is much more robust against ties given that the summation of all the weighted attributed outcomes will rarely result in ties, even with a nominal set of providers.

In this study, the PAMM methodology was introduced and was used to assess relative provider performance on 3 clinical outcome measures and 3 patient survey scores. The new approach aims to distribute each outcome among all providers who provided care for a patient in an inpatient setting. Clinical notes were used to account for patient-to-provider interactions, and fitted MM statistical models were used to compute the effects that each provider had on each outcome. The provider effect was introduced as a random effect, and the set of predicted random effects was used to rank the performance of each provider.

The PAMM approach was compared to the more traditional methodology, PAPR, where each patient is attributed to only 1 provider: the discharging physician in this study. Using this approach, OE indices of clinical outcomes and averages of survey scores were used to rank the performance of each provider. This approach resulted in many ties, which were broken based on the number of hospitalizations, although other tie-breaking methods may be used in practice.

Both methodologies showed modest concordance with each other for the clinical outcomes, but higher concordance for the patient survey scores. This was also true when using the Pearson correlation coefficient to assess agreement. The 1 outcome measure that showed the least concordance and least linear correlation between methods was LOS, which would suggest that LOS performance is more sensitive to the attribution methodology that is used. However, it was the least concordant by a small margin.

Furthermore, although the medians of the absolute percentile differences were small, there were some providers who had large deviations, suggesting that some providers would move from being shown as high-performers to low-performers and vice versa based on the chosen attribution method. We investigated examples of this and determined that the root cause was the difference in effective sample sizes for a provider. For the PAPR method, the effective sample size is simply the number of hospitalizations attributed to the provider. For the PAMM method, the effective sample size is the sum of all non-zero weights across all hospitalizations where the provider cared for a patient. By and large, the PAMM methodology provides more information of the provider effect on an outcome than the PAPR approach because every provider-patient interaction is considered. For example, providers who do not routinely discharge patients, but often care for patients, will have rankings that differ dramatically between the 2 methods.

The PAMM methodology has many statistical advantages that were not fully utilized in this comparative study. For example, we did not include any covariates in the MM models except for the expected value of the outcome, when it was available. Still, it is known that other covariates can impact an outcome as well, such as the patient’s age, socioeconomic indicators, existing chronic conditions, and severity of hospitalization, which can be added to the MM models as fixed effects. In this way, the PAMM approach can control for these other covariates, which are typically outside of the control of providers but typically ignored using OE indices. Therefore, using the PAMM approach would provide a fairer comparison of provider performance.

Using the PAMM method, most providers had a large sample size to assess their performance once all the weighted interactions were included. Still, there were a few who did not care for many patients for a variety of reasons. In these scenarios, MM models “borrow” strength from other providers to produce a more robust predicted provider effect by using a weighted average between the overall population trend and the specific provider outcomes (see Rao and Molina¹⁷). As a result, PAMM is a more suitable approach when the sample sizes of patients attributed to providers can be small.

One of the most interesting findings of this study was the relative size of the provider-level variance to the size of the fixed effect in each model (Table 3). Except for mortality, these variances suggest that there is a small difference in performance from one provider to another. However, these should be interpreted as the variance when only 1 provider is involved in the care of a patient. When multiple providers are involved, using basic statistical theory, the overall provider-level variance will be σ_γ² ∑w_ij² (see Equation 2). For example, the estimated variance among providers for LOS was 0.03 (on a log scale), but, using the scenario in the Figure, the overall provider-level variance for this hospitalization will be 0.03 (0.375²+ 0.125²+ 0.5²) = 0.012. Hence, the combined effect of providers on LOS is less than would be expected. Indeed, as more providers are involved with a patient’s care, the more their combined influence on an outcome is diluted.

In this study, the PAMM approach placed an equal weight on all provider-patient interactions via clinical note authorship, but that may not be optimal in some settings. For example, it may make more sense to set a higher weight on the provider who admitted or discharged the patient while placing less (or 0) weight on all other interactions. In the extreme, if the full weight were placed on 1 provider interaction (eg, during discharge, then the MM model would be reduced to a one-way random effects model. The flexibility of weighting interactions is a feature of the PAMM approach, but any weighting framework must be transparent to the providers before implementation.

Conclusion

This study demonstrates that the PAMM approach is a feasible option within a large health care organization. For P4P programs to be successful, providers must be able to trust that their performance will be fairly assessed and that all provider-patient interactions are captured to provide a full comparison amongst their peers. The PAMM methodology is one solution to spread the positive (and negative) outcomes across all providers who cared for a patient and therefore, if implemented, would add trust and fairness when measuring and assessing provider performance.

Acknowledgments: The authors thank Barrie Bradley for his support in the initial stages of this research and Dr. Syed Ismail Jafri for his help and support on the standard approaches of assessing and measuring provider performances.

Corresponding author: Rachel Ginn, MS, Banner Health Corporation, 2901 N. Central Ave., Phoenix, AZ 85012; [email protected].

Financial disclosures: None.

From Banner Health Corporation, Phoenix, AZ.

Background: Health care providers are routinely incentivized with pay-for-performance (P4P) metrics to increase the quality of care. In an inpatient setting, P4P models typically measure quality by attributing each patient’s outcome to a single provider even though many providers routinely care for the patient. This study investigates a new attribution approach aiming to distribute each outcome across all providers who provided care.

Methods: The methodology relies on a multi-membership model and is demonstrated in the Banner Health system using 3 clinical outcome measures (length of stay, 30-day readmissions, and mortality) and responses to 3 survey questions that measure a patient’s perception of their care. The new approach is compared to the “standard” method, which attributes each patient to only 1 provider.

Results: When ranking by clinical outcomes, both methods were concordant 72.1% to 82.1% of the time for top-half/bottom-half rankings, with a median percentile difference between 7 and 15. When ranking by survey scores, there was more agreement, with concordance between 84.1% and 86.6% and a median percentile difference between 11 and 13. Last, Pearson correlation coefficients of the paired percentiles ranged from 0.56 to 0.78.

Conclusion: The new approach provides a fairer solution when measuring provider performance.

Keywords: patient attribution; PAMM; PAPR; random effect model; pay for performance.

Providers practicing in hospitals are routinely evaluated based on their performance and, in many cases, are financially incentivized for a better-than-average performance within a pay-for-performance (P4P) model. The use of P4P models is based on the belief that they will “improve, motivate, and enhance providers to pursue aggressively and ultimately achieve the quality performance targets thus decreasing the number of medical errors with less malpractice events.”¹ Although P4P models continue to be a movement in health care, they have been challenging to implement.

One concern involves the general quality of implementation, such as defining metrics and targets, setting payout amounts, managing technology and market conditions, and gauging the level of transparency to the provider.² Another challenge, and the focus of this project, are concerns around measuring performance to avoid perceptions of unfairness. This concern can be minimized if the attribution is handled in a fairer way, by spreading it across all providers who affected the outcome, both in a positive or negative direction.³

To implement these models, the performance of providers needs to be measured and tracked periodically. This requires linking, or attributing, a patient’s outcome to a provider, which is almost always the attending or discharging provider (ie, a single provider).³ In this single-provider attribution approach, one provider will receive all the credit (good or bad) for their respective patients’ outcomes, even though the provider may have seen the patient only a fraction of the time during the hospitalization. Attributing outcomes—for example, length of stay (LOS), readmission rate, mortality rate, net promoter score (NPS)—using this approach reduces the validity of metrics designed to measure provider performance, especially in a rotating provider environment where many providers interact with and care for a patient. For example, the quality of providers’ interpersonal skills and competence were among the strongest determinants of patient satisfaction,⁴ but it is not credible that this is solely based on the last provider during a hospitalization.

Proportionally distributing the attribution of an outcome has been used successfully in other contexts. Typically, a statistical modeling approach using a multi-membership framework is used because it can handle the sometimes-complicated relationships within the hierarchy. It also allows for auxiliary variables to be introduced, which can help explain and control for exogenous effects.^5-7 For example, in the education setting, standardized testing is administered to students at defined years of schooling: at grades 4, 8, and 10, for instance. The progress of students, measured as the academic gains between test years, are proportionally attributed to all the teachers who the student has had between the test years. These partial attributions are combined to evaluate an overall teacher performance.^8,9

Although the multi-membership framework has been used in other industries, it has yet to be applied in measuring provider performance. The purpose of this project is to investigate the impact of using a multi-provider approach compared to the standard single-provider approach. The findings may lead to modifications in the way a provider’s performance is measured and, thus, how providers are compensated. A similar study investigated the impact of proportionally distributing patients’ outcomes across all rotating providers using a weighting method based on billing practices to measure the partial impact of each provider.³

This study is different in 2 fundamental ways. First, attribution is weighted based on the number of clinically documented interactions (via clinical notes) between a patient and all rotating providers during the hospitalization. Second, performance is measured via multi-membership models, which can estimate the effect (both positive and negative) that a provider has on an outcome, even when caring for a patient a fraction of the time during the hospitalization.

Methods

Setting

Banner Health is a non-profit, multi-hospital health care system across 6 states in the western United States that is uniquely positioned to study provider quality attribution models. It not only has a large number of providers and serves a broad patient population, but Banner Health also uses an instance of Cerner (Kansas City, MO), an enterprise-level electronic health record (EHR) system that connects all its facilities and allows for advanced analytics across its system.

For this study, we included only general medicine and surgery patients admitted and discharged from the inpatient setting between January 1, 2018, and December 31, 2018, who were between 18 and 89 years old at admission, and who had a LOS between 1 and 14 days. Visit- and patient-level data were collected from Cerner, while outcome data, and corresponding expected outcome data, were obtained from Premier, Inc. (Charlotte, NC) using their CareScience methodologies.¹⁰ To measure patient experience, response data were extracted from post-discharge surveys administered by InMoment (Salt Lake City, UT).

Provider Attribution Models

Provider Attribution by Physician of Record (PAPR). In the standard approach, denoted here as the PAPR model, 1 provider—typically the attending or discharging provider, which may be the same person—is attributed to the entire hospitalization. This provider is responsible for the patient’s care, and all patient outcomes are aggregated and attributed to the provider to gauge his or her performance. The PAPR model is the most popular form of attribution across many health care systems and is routinely used for P4P incentives.

In this study, the discharging provider was used when attributing hospitalizations using the PAPR model. Providers responsible for fewer than 12 discharges in the calendar year were excluded. Because of the directness of this type of attribution, the performance of 1 provider does not account for the performance of the other rotating providers during hospitalizations.

Provider Attribution by Multiple Membership (PAMM). In contrast, we introduce another attribution approach here that is designed to assign partial attribution to each provider who cares for the patient during the hospitalization. To aggregate the partial attributions, and possibly control for any exogenous or risk-based factors, a multiple-membership, or multi-member (MM), model is used. The MM model can measure the effect of a provider on an outcome even when the patient-to-provider relationship is complex, such as in a rotating provider environment.⁸

The purpose of this study is to compare attribution models and to determine whether there are meaningful differences between them. Therefore, for comparison purposes, the same discharging providers using the PAPR approach are eligible for the PAMM approach, so that both attribution models are using the same set of providers. All other providers are excluded because their performance would not be comparable to the PAPR approach.

While there are many ways to document provider-to-patient interactions, 2 methods are available in almost all health care systems. The first method is to link a provider’s billing charges to each patient-day combination. This approach limits the attribution to 1 provider per patient per day because multiple rotating providers cannot charge for the same patient-day combination.³ However, many providers interact with a patient on the same day, so using this approach excludes non-billed provider-to-patient interactions.

The second method, which was used in this study, relies on documented clinical notes within the EHR to determine how attribution is shared. In this approach, attribution is weighted based on the authorship of 3 types of eligible clinical notes: admitting history/physical notes (during admission), progress notes (during subsequent days), and discharge summary notes (during final discharge). This will (likely) result in many providers being linked to a patient on each day, which better reflects the clinical setting (Figure). Recently, clinical notes were used to attribute care of patients in an inpatient setting, and it was found that this approach provides a reliable way of tracking interactions and assigning ownership.¹¹

The provider-level attribution weights are based on the share of authorships of eligible note types. Specifically, for each provider j, let a_ij be the total count of eligible note types for hospitalization i authored by provider j, and let a_i be the overall total count of eligible note types for hospitalization i. Then the attribution weight is

(Eq. 1)

for hospitalization i and provider j. Note that ∑_jw_ij = 1: in other words, the total attribution, summed across all providers, is constrained to be 1 for each hospitalization.

Patient Outcomes

Outcomes were chosen based on their routine use in health care systems as standards when evaluating provider performance. This study included 6 outcomes: inpatient LOS, inpatient mortality, 30-day inpatient readmission, and patient responses from 3 survey questions. These outcomes can be collected without any manual chart reviews, and therefore are viewed as objective outcomes of provider performance.

Each outcome was aggregated for each provider using both attribution methods independently. For the PAPR method, observed-to-expected (OE) indices for LOS, mortality, and readmissions were calculated along with average patient survey scores. For the PAMM method, provider-level random effects from the fitted models were used. In both cases, the calculated measures were used for ranking purposes when determining top (or bottom) providers for each outcome.

Individual Provider Metrics for the PAPR Method

Inpatient LOS Index. Hospital inpatient LOS was measured as the number of days between admission date and discharge date. For each hospital visit, an expected LOS was determined using Premier’s CareScience Analytics (CSA) risk-adjustment methodology.¹⁰ The CSA methodology for LOS incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let o_ij and e_ij be the observed and expected LOS, respectively, for hospitalization i = 1,…,n_j attributed to provider j. Then the inpatient LOS index for provider j is L_j = ∑_io_ij⁄∑_ie_ij.

Inpatient Mortality Index. Inpatient mortality was defined as the death of the patient during hospitalization. For each hospitalization, an expected mortality probability was determined using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for mortality incorporates a patient’s demographics and comorbidities.

Just as before, let n_j be the number of hospitalizations attributed to provider j. Let m_ij = 1 if the patient died during hospitalization i = 1, … , n_j attributed to provider j; m_ij = 0 otherwise. Let p_ij^(m) be the corresponding expected mortality probability. Then the inpatient mortality index for provider j is M_j = ∑_im_ij⁄∑_ip_ij^(m).

30-Day Inpatient Readmission Index. A 30-day inpatient readmission was defined as the event when a patient is discharged and readmits back into the inpatient setting within 30 days. The inclusion criteria defined by the Centers for Medicare and Medicaid Services (CMS) all-cause hospital-wide readmission measure was used and, consequently, planned readmissions were excluded.¹² Readmissions could occur at any Banner hospital, including the same hospital. For each hospital visit, an expected readmission probability was derived using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for readmissions incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let r_ij = 1 if the patient had a readmission following hospitalization i = 1, … , n_j attributed to provider j; r_ij = 0 otherwise. Let p_ij^(r) be the corresponding expected readmission probability. Then the 30-day inpatient readmission index for provider j is R_j = ∑_ir_ij ⁄∑_ip_ij^(r).

Patient Survey Scores. The satisfaction of the patient’s experience during hospitalization was measured via post-discharge surveys administered by InMoment. Two survey questions were selected because they related directly to a provider’s interaction with the patient: “My interactions with doctors were excellent” (Doctor) and “I received the best possible care” (Care). A third question, “I would recommend this hospital to my family and friends,” was selected as a proxy measure of the overall experience and, in the aggregate, is referred to as the net promoter score (NPS).^13,14 The responses were measured on an 11-point Likert scale, ranging from “Strongly Disagree” (0) to “Strongly Agree” (10); “N/A” or missing responses were excluded.

The Likert responses were coded to 3 discrete values as follows: if the value was between 0 and 6, then -1 (ie, detractor); between 7 and 8 (ie, neutral), then 0; otherwise 1 (ie, promoter). Averaging these coded responses results in a patient survey score for each question. Specifically, let n_j be the number of hospitalizations attributed to provider j in which the patient responded to the survey question. Let s_ij ∈{−1, 0, 1} be the coded response linked to hospitalization i = 1, … , n_j attributed to provider j. Then the patient experience score for provider j is S_j = ∑_is_ij⁄n_j.

Handling Ties in Provider Performance Measures. Because ties can occur in the PAPR approach for all measures, a tie-breaking strategy is needed. For LOS indices, ties are less likely because their numerator is strictly greater than 0, and expected LOS values are typically distinct enough. Indeed, no ties were found in this study for LOS indices. However, mortality and readmission indices can routinely result in ties when the best possible index is achieved, such as 0 deaths or readmissions among attributed hospitalizations. To help differentiate between those indices in the PAPR approach, the total estimated risk (denominator) was utilized as a secondary scoring criterion.

Mortality and readmission metrics were addressed by sorting first by the outcome (mortality index), and second by the denominator (total estimated risk). For example, if provider A has the same mortality rate as provider B, then provider A would be ranked higher if the denominator was larger, indicating a higher risk for mortality.

Similarly, it was very common for providers to have the same overall average rating for a survey question. Therefore, the denominator (number of respondents) was used to break ties. However, the denominator sorting was bidirectional. For example, if the tied score was positive (more promoters than detractors) for providers A and B, then provider A would be ranked higher if the denominator was larger. Conversely, if the tied score between providers A and B was neutral or negative (more detractors than promoters), then provider A would be ranked lower if the denominator was larger.

Individual Provider Metrics for the PAMM Method

For the PAMM method, model-based metrics were derived using a MM model.⁸ Specifically, let J be the number of rotating providers in a health care system. Let Y_i be an outcome of interest from hospitalization i, X_1i, …, X_pi be fixed effects or covariates, and ß₁, …, ß_p be the coefficients for the respective covariates. Then the generalized MM statistical model is

(Eq. 2)

where g(μ_i) is a link function between the mean of the outcome, μ_i, and its linear predictor, ß₀, is the marginal intercept, w_ij represents the attribution weight of provider j on hospitalization i (described in Equation 1), and γ_j represents the random effect of provider j on the outcome with γ_j~N(0,σ_γ²).

For the mortality and readmission binary outcomes, logistic regression was performed using a logit link function, with the corresponding expected probability as the only fixed covariate. The expected probabilities were first converted into odds and then log-transformed before entering the model. For LOS, Poisson regression was performed using a log link function with the log-transformed expected LOS as the only fixed covariate. For coded patient experience responses, an ordered logistic regression was performed using a cumulative logit link function (no fixed effects were added).

MM Model-based Metrics. Each fitted MM model produces a predicted random effect for each provider. The provider-specific random effects can be interpreted as the unobserved influence of each provider on the outcome after controlling for any fixed effect included in the model. Therefore, the provider-specific random effects were used to evaluate the relative provider performance, which is analogous to the individual provider-level metrics used in the PAPR method.

Measuring provider performance using a MM model is more flexible and robust to outliers compared to the standard approach using OE indices or simple averages. First, although not investigated here, the effect of patient-, visit-, provider-, and/or temporal-level covariates can be controlled when evaluating provider performance. For example, a patient’s socioeconomic status, a provider’s workload, and seasonal factors can be added to the MM model. These external factors are not accounted for in OE indices.

Another advantage of using predicted random effects is the concept of “shrinkage.” The process of estimating random effects inherently accounts for small sample sizes (when providers do not treat a large enough sample of patients) and/or when there is a large ratio of patient variance to provider variance (for instance, when patient outcome variability is much higher compared to provider performance variability). In both cases, the estimation of the random effect is pulled ever closer to 0, signaling that the provider performance is closer to the population average. See Henderson¹⁵ and Mood¹⁶ for further details.

In contrast, OE indices can result in unreliable estimates when a provider has not cared for many patients. This is especially prevalent when the outcome is binary with a low probability of occurring, such as mortality. Indeed, provider-level mortality OE indices are routinely 0 when the patient counts are low, which skews performance rankings unfairly. Finally, OE indices also ignore the magnitude of the variance of an outcome between providers and patients, which can be large.

Comparison Methodology

In this study, we seek to compare the 2 methods of attribution, PAPR and PAMM, to determine whether there are meaningful differences between them when measuring provider performance. Using retrospective data described in the next section, each attribution method was used independently to derive provider-level metrics. To assess relative performance, percentiles were assigned to each provider based on their metric values so that, in the end, there were 2 percentile ranks for each provider for each metric.

Using these paired percentiles, we derived the following measures of concordance, similar to Herzke, Michtalik³: (1) the percent concordance measure—defined as the number of providers who landed in the top half (greater than the median) or bottom half under both attribution models—divided by the total number of providers; (2) the median of the absolute difference in percentiles under both attribution models; and (3) the Pearson correlation coefficient of the paired provider ranks. The first measure is a global measure of concordance between the 2 approaches and would be expected to be 50% by chance. The second measure gauges how an individual provider’s rank is affected by the change in attribution methodologies. The third measure is a statistical measure of linear correlation of the paired percentiles and was not included in the Herzke, Michtalik³ study.

All statistical analyses were performed on SAS (version 9.4; Cary, NC) and the MM models were fitted using PROC GLIMMIX with the EFFECT statement. The Banner Health Institutional Review Board approved this study.

Results

Descriptive Statistics

A total of 58,730 hospitalizations were included, of which care was provided by 963 unique providers across 25 acute care and critical access hospitals. Table 1 contains patient characteristics, and Table 2 depicts overall unadjusted outcomes. Providers responsible for less than 12 discharges in the calendar year were excluded from both approaches. Also, some hospitalizations were excluded when expected values were not available.

Multi-Membership Model Results

Table 3 displays the results after independently fitting MM models to each of the 3 clinical outcomes. Along with a marginal intercept, the only covariate in each model was the corresponding expected value after a transformation. This was added to use the same information that is typically used in OE indices, therefore allowing for a proper comparison between the 2 attribution methods. The provider-level variance represents the between-provider variation and measures the amount of influence providers have on the corresponding outcome after controlling for any covariates in the model. A provider-level variance of 0 would indicate that providers do not have any influence on the outcome. While the mortality and readmission model results can be compared to each other, the LOS model cannot given its different scale and transformation altogether.

The results in Table 3 suggest that each expected value covariate is highly correlated with its corresponding outcome, which is the anticipated conclusion given that they are constructed in this fashion. The estimated provider-level variances indicate that, after including an expected value in the model, providers have less of an influence on a patient’s LOS and likelihood of being readmitted. On the other hand, the results suggest that providers have much more influence on the likelihood of a patient dying in the hospital, even after controlling for an expected mortality covariate.

Table 4 shows the results after independently fitting MM-ordered logistic models to each of the 3 survey questions. The similar provider-level variances suggest that providers have the same influence on the patient’s perception of the quality of their interactions with the doctor (Doctor), the quality of the care they received (Care), and their likelihood to recommend a friend or family member to the hospital (NPS).

Comparison Results Between Both Attribution Methods

Table 5 compares the 2 attribution methods when ranking providers based on their performance on each outcome measure. The comparison metrics gauge how well the 2 methods agree overall (percent concordance), agree at the provider level (absolute percentile difference and interquartile range [IQR]), and how the paired percentiles linearly correlate to each other (Pearson correlation coefficient).

LOS, by a small margin, had the lowest concordance of clinical outcomes (72.1%), followed by mortality (75.9%) and readmissions (82.1%). Generally, the survey scores had higher percent concordance than the clinical outcome measures, with Doctor at 84.1%, Care at 85.9%, and NPS having the highest percent concordance at 86.6%. Given that by chance the percent concordance is expected to be 50%, there was notable discordance, especially with the clinical outcome measures. Using LOS performance as an example, one attribution methodology would rank a provider in the top half or bottom half, while the other attribution methodology would rank the same provider exactly the opposite way about 28% of the time.

The median absolute percentile difference between the 2 methods was more modest (between 7 and 15). Still, there were some providers whose performance ranking was heavily impacted by the attribution methodology that was used. This was especially true when evaluating performance for certain clinical measures, where the attribution method that was used could change the provider performance percentile by up to 90 levels.

The paired percentiles were positively correlated when ranking performance using any of the 6 measures. This suggests that both methodologies assess performance generally in the same direction, irrespective of the methodology and measure. We did not investigate more complex correlation measures and left this for future research.

It should be noted that ties occurred much more frequently with the PAPR method than when using PAMM and therefore required tie-breaking rules to be designed. Given the nature of OE indices, PAPR methodology is especially sensitive to ties whenever the measure includes counting the number of events (for example, mortality and readmissions) and whenever there are many providers with very few attributed patients. On the other hand, using the PAMM method is much more robust against ties given that the summation of all the weighted attributed outcomes will rarely result in ties, even with a nominal set of providers.

In this study, the PAMM methodology was introduced and was used to assess relative provider performance on 3 clinical outcome measures and 3 patient survey scores. The new approach aims to distribute each outcome among all providers who provided care for a patient in an inpatient setting. Clinical notes were used to account for patient-to-provider interactions, and fitted MM statistical models were used to compute the effects that each provider had on each outcome. The provider effect was introduced as a random effect, and the set of predicted random effects was used to rank the performance of each provider.

The PAMM approach was compared to the more traditional methodology, PAPR, where each patient is attributed to only 1 provider: the discharging physician in this study. Using this approach, OE indices of clinical outcomes and averages of survey scores were used to rank the performance of each provider. This approach resulted in many ties, which were broken based on the number of hospitalizations, although other tie-breaking methods may be used in practice.

Both methodologies showed modest concordance with each other for the clinical outcomes, but higher concordance for the patient survey scores. This was also true when using the Pearson correlation coefficient to assess agreement. The 1 outcome measure that showed the least concordance and least linear correlation between methods was LOS, which would suggest that LOS performance is more sensitive to the attribution methodology that is used. However, it was the least concordant by a small margin.

Furthermore, although the medians of the absolute percentile differences were small, there were some providers who had large deviations, suggesting that some providers would move from being shown as high-performers to low-performers and vice versa based on the chosen attribution method. We investigated examples of this and determined that the root cause was the difference in effective sample sizes for a provider. For the PAPR method, the effective sample size is simply the number of hospitalizations attributed to the provider. For the PAMM method, the effective sample size is the sum of all non-zero weights across all hospitalizations where the provider cared for a patient. By and large, the PAMM methodology provides more information of the provider effect on an outcome than the PAPR approach because every provider-patient interaction is considered. For example, providers who do not routinely discharge patients, but often care for patients, will have rankings that differ dramatically between the 2 methods.

The PAMM methodology has many statistical advantages that were not fully utilized in this comparative study. For example, we did not include any covariates in the MM models except for the expected value of the outcome, when it was available. Still, it is known that other covariates can impact an outcome as well, such as the patient’s age, socioeconomic indicators, existing chronic conditions, and severity of hospitalization, which can be added to the MM models as fixed effects. In this way, the PAMM approach can control for these other covariates, which are typically outside of the control of providers but typically ignored using OE indices. Therefore, using the PAMM approach would provide a fairer comparison of provider performance.

Using the PAMM method, most providers had a large sample size to assess their performance once all the weighted interactions were included. Still, there were a few who did not care for many patients for a variety of reasons. In these scenarios, MM models “borrow” strength from other providers to produce a more robust predicted provider effect by using a weighted average between the overall population trend and the specific provider outcomes (see Rao and Molina¹⁷). As a result, PAMM is a more suitable approach when the sample sizes of patients attributed to providers can be small.

One of the most interesting findings of this study was the relative size of the provider-level variance to the size of the fixed effect in each model (Table 3). Except for mortality, these variances suggest that there is a small difference in performance from one provider to another. However, these should be interpreted as the variance when only 1 provider is involved in the care of a patient. When multiple providers are involved, using basic statistical theory, the overall provider-level variance will be σ_γ² ∑w_ij² (see Equation 2). For example, the estimated variance among providers for LOS was 0.03 (on a log scale), but, using the scenario in the Figure, the overall provider-level variance for this hospitalization will be 0.03 (0.375²+ 0.125²+ 0.5²) = 0.012. Hence, the combined effect of providers on LOS is less than would be expected. Indeed, as more providers are involved with a patient’s care, the more their combined influence on an outcome is diluted.

In this study, the PAMM approach placed an equal weight on all provider-patient interactions via clinical note authorship, but that may not be optimal in some settings. For example, it may make more sense to set a higher weight on the provider who admitted or discharged the patient while placing less (or 0) weight on all other interactions. In the extreme, if the full weight were placed on 1 provider interaction (eg, during discharge, then the MM model would be reduced to a one-way random effects model. The flexibility of weighting interactions is a feature of the PAMM approach, but any weighting framework must be transparent to the providers before implementation.

Conclusion

This study demonstrates that the PAMM approach is a feasible option within a large health care organization. For P4P programs to be successful, providers must be able to trust that their performance will be fairly assessed and that all provider-patient interactions are captured to provide a full comparison amongst their peers. The PAMM methodology is one solution to spread the positive (and negative) outcomes across all providers who cared for a patient and therefore, if implemented, would add trust and fairness when measuring and assessing provider performance.

Acknowledgments: The authors thank Barrie Bradley for his support in the initial stages of this research and Dr. Syed Ismail Jafri for his help and support on the standard approaches of assessing and measuring provider performances.

Corresponding author: Rachel Ginn, MS, Banner Health Corporation, 2901 N. Central Ave., Phoenix, AZ 85012; [email protected].

Financial disclosures: None.