Just as Brazil surpassed 200,000 deaths from COVID-19 on Jan. 7, news from Bahia added another layer of concern: A platform case report in a preprint detailed the first case of reinfection in that state, apparently caused by a new strain, one having the E484K mutation.

That variant, now called Brazil P.1, has migrated to the United States. The Minnesota Department of Health announced on Jan. 25 the nation’s first known COVID-19 case associated with it.

The mutation is located in the protein gene of the virus’ spike, which forms the crown structure of coronaviruses and is responsible for the virus’ binding to human cells. The E484K mutation is now the focus because it’s associated with mutations that escape the immune system’s neutralizing antibodies.

“This mutation is at the center of worldwide concern, and it is the first time that it has appeared in a reinfection,” the study’s first author, Bruno Solano de Freitas Souza, MD, a researcher at the Salvador regional unit of Instituto D’Or of Teaching and Research, based at Hospital São Rafael, Salvador, Brazil, explained in an interview.

“We will wait for the sample from Bahia to confirm the case from the perspective of the Ministry of Health’s surveillance network,” said Fernando Motta, PhD, deputy head of the Laboratory for Respiratory Virus and Measles at the Oswaldo Cruz Institute in Rio de Janeiro, which acts as a national reference center for respiratory viruses with the Brazilian Ministry of Health (MS) and as a reference for the World Health Organization.
 

A case of reinfection

The case patient that led to the alarm was a 45-year-old woman who is a health care executive. She had no comorbidities. The team had been following health care professionals and patients who had tested positive on reverse transcription–polymerase chain reaction (RT-PCR) testing more than once to understand whether they represented cases of prolonged viral persistence or new infections.

The woman had symptoms of viral infection on two occasions (May 26 and Oct. 26). On both occasions, results of RT-PCR testing for SARS-CoV-2 on nasopharyngeal samples were positive. In the first episode, the patient had diarrhea, myalgia, asthenia, and odynophagia for about 7 days. She returned to activities 21 days later. In the second episode, she had more severe symptoms that lasted longer, but she still did not require hospitalization.

“It was the first confirmed case of reinfection in Bahia, and in the second episode, we observed a mutation that could have an impact on the ability of antibodies to neutralize the virus,” Dr. Souza said. “The research continues with the investigation of cases in which the patient has a positive SARS-CoV-2 RT-PCR more than once in an interval greater than 45 days, to have a higher level of evidence.”

He stressed that “it is very important to reinforce measures to control the pandemic, social distance, use of masks, and speed up vaccination to be able to control the circulation of the virus, while monitoring the evolution of it.”
 

On alert for more cases

A person who twice tests positive for SARS-CoV-2 on real-time RT-PCR is suspected of having been reinfected, provided 90 or more days have elapsed between the two episodes, regardless of the condition observed. To confirm the suspected case, the samples must be sent to reference laboratories according to a plan established by the Ministry of Health in Brazil.

A health professional living in the Brazilian city of Natal represented the first confirmed case of reinfection by the new coronavirus in Brazil. That case was announced on Dec. 10, 2020.

“We communicated this case of reinfection to the MS in early December 2020. And the second sample already had the E484K mutation on the spike, as in the case of Bahia,” said Dr. Motta.

The first step in differentiating reinfection from persistence is to observe differences in the genotyping of the virus. For the technique to be successful, Dr. Souza said, researchers need a large amount of viral genetic material, which usually cannot be obtained.

“That is why there are many more suspected than confirmed cases,” Dr. Souza explained. He admitted that, although there are few cases, “it is increasingly clear that reinfection is a reality.”
 

Markers of mutations

What worried the researchers most was not only the possibility of reinfection but also the fact that preliminary analyses showed a specific mutation.

“The E484K mutation is present in a group of variants identified in South Africa that have been associated with increased infectivity and has been observed in a strain recently described in Brazil,” Dr. Souza said.

Mutations are expected, appear spontaneously, and in most cases have no effects on transmission or clinical outcome – they are simply used as markers and are useful for contact tracing or studying transmission routes. But some mutations can last because they provide an advantage for the pathogen, even if only momentary. In the case of SARS-CoV-2, mutations in the protein spike gene (S) are relevant because they may give clues to that advantage – as well as to changes in infectivity, transmission potential, antibodies, and response to vaccines.

A variant of the virus that has eight changes that affect the protein S gene – and several others in different genes – is behind the increase in the number of cases in London and southeastern England. Researchers from the University of São Paulo identified one of the factors that made this new variant – classified as B.1.1.7 – more infectious.

With bioinformatics tools, they found that the protein S gene in the new viral strain has a stronger molecular interaction with the ACE2 receptor, which is on the surface of human cells and to which the virus binds, making infection possible. The variant has already spread to the rest of the world, and the first two cases have been confirmed in Brazil by the Adolf Lutz Institute.

The alert for a new variant in Africa – similar to B.1.1.7 in the United Kingdom in that it carries nine changes in protein S at position 501 – was made by the Brazilian virologist Tulio de Oliveira, PhD.

“We found that this strain seems to be spreading much faster,” Dr. Oliveira, who is with the University of KwaZulu Natal, told the journal Science. His work first alerted British scientists to the importance of the position N501Y.

“The new variants just described in the United Kingdom and South Africa are slightly more transmissible and have already been identified in cases imported into Brazil,” Dr. Motta said. “Unfortunately, we believe it is only a matter of time before it becomes indigenous.”
 

The viral family grows

Viruses such as SARS-CoV-2 are classified into strains on the basis of small differences in their genetic material. Since Dec. 26, 2020, in addition to the British and South African variants, it appears the Carioca lineage also is a player.

In a preprint article, researchers analyzed the evolution of the epidemic in Rio de Janeiro from April 2020 until just before the new increase in incidence in December. They compared the complete sequences of the viral genome of 180 patients from different municipalities. The study, which is being jointly conducted by members of the Federal University of Rio de Janeiro and the National Laboratory for Scientific Computing, identified a new variant of SARS-CoV-2 that has five unique mutations (from one of the predominant strains). Concern arose because, in addition to those five genetic changes, many of the samples had a sixth – the well-known E484K mutation.

“The three lines – the U.K., South Africa, and Brazil – were almost synchronous publications, but there is no clear evidence that they have any kind of common ancestry,” Carolina M. Voloch, PhD, the article’s first author and a biologist and researcher at the Molecular Virology Laboratory and associate professor in the department of genetics at the Federal University of Rio de Janeiro, said in an interview.

Dr. Voloch’s research focuses on the use of bioinformatics tools to study the molecular, phylogenetic, and genomic evolution of viruses.

“The emergence of new strains is common for viruses,” she said. “It can be happening anywhere in the world at any time.”

She stressed that identifying when mutations emerge will help to define the new Brazilian lineage. Researchers are working to determine whether the neutralizing antibodies of patients who have been infected with other strains respond to this Rio de Janeiro strain.

“We hope to soon be sharing these results,” Dr. Voloch said.

The article’s authors estimated that the new strain likely appeared in early July. They say more analysis is needed to predict whether the changes have a major effect on viral infectivity, the host’s immune response, or the severity of the disease. Asked about the lineage that caused the reinfection in Bahia, Dr. Voloch said she hadn’t yet contacted the authors to conduct a joint analysis but added that the data disclosed in the preprint would not represent the same variant.

“There are only two of the five mutations that characterize the Rio de Janeiro lineage. However, it has the E484K mutation that is present in more than 94% of the samples of the new variant of Rio,” she said.

She added that there’s a possibility of reinfection by the lineage that’s circulating in Rio de Janeiro and in other states, as well as countries such as the United States, the United Kingdom, and Japan.

“The Carioca virus is being exported to the rest of the world,” Dr. Voloch said.
 

Virus’ diversity still unknown

Researchers now know that SARS-CoV-2 probably circulated silently in Brazil as early as February 2020 and reached all the nation’s regions before air travel was restricted. Since the first half of 2020, there have been two predominant strains.

“More than a dozen strains have been identified in Brazil, but more important than counting strains to identify the speed with which they arise – which is directly associated with the rate of infection, which is very high in the country,” said Dr. Motta.

The so-called variant of Rio de Janeiro, he said, has also been detected in other states in four regions of Brazil. The key to documenting variants is to get a more representative sample with genomes from other parts of the country.

As of Jan. 10, a total of 347,000 complete genome sequences had been shared globally through open databases since SARS-CoV-2 was first identified, but the contribution of countries is uneven. Although the cost and complexity of genetic sequencing has dropped significantly over time, effective sequencing programs still require substantial investments in personnel, equipment, reagents, and bioinformatics infrastructure.

According to Dr. Voloch, it will only be possible to combat the new coronavirus by knowing its diversity and understanding how it evolves. The Fiocruz Genomic Network has made an infographic available so researchers can track the strains circulating in Brazil. It›s the result of collaboration between researchers from Fiocruz and the GISAID Initiative, an international partnership that promotes rapid data sharing.

As of Jan. 5, researchers in Brazil had studied 1,897 genomes – not nearly enough.

“In Brazil, there is little testing and even less sequencing,” lamented Dr. Souza.

“In the U.K., 1 in 600 cases is sequenced. In Brazil it is less than 1 in 10 million cases,” Dr. Voloch added.

So far, no decisive factors for public health, such as greater virulence or greater transmissibility, have been identified in any of the strains established in Brazil. The million-dollar question is whether the emergence of new strains could have an impact on the effectiveness of vaccines being administered today.

“In one way or another, the vaccine is our best bet ever, even if in the future we identify escapist mutants and have to modify it,” Dr. Motta said. “It is what we do annually with influenza.”

Dr. Voloch, Dr. Motta, and Dr. Souza disclosed no relevant financial relationships.

A version of this article first appeared on the Portuguese edition of Medscape.com.

Just as Brazil surpassed 200,000 deaths from COVID-19 on Jan. 7, news from Bahia added another layer of concern: A platform case report in a preprint detailed the first case of reinfection in that state, apparently caused by a new strain, one having the E484K mutation.

That variant, now called Brazil P.1, has migrated to the United States. The Minnesota Department of Health announced on Jan. 25 the nation’s first known COVID-19 case associated with it.

The mutation is located in the protein gene of the virus’ spike, which forms the crown structure of coronaviruses and is responsible for the virus’ binding to human cells. The E484K mutation is now the focus because it’s associated with mutations that escape the immune system’s neutralizing antibodies.

“This mutation is at the center of worldwide concern, and it is the first time that it has appeared in a reinfection,” the study’s first author, Bruno Solano de Freitas Souza, MD, a researcher at the Salvador regional unit of Instituto D’Or of Teaching and Research, based at Hospital São Rafael, Salvador, Brazil, explained in an interview.

“We will wait for the sample from Bahia to confirm the case from the perspective of the Ministry of Health’s surveillance network,” said Fernando Motta, PhD, deputy head of the Laboratory for Respiratory Virus and Measles at the Oswaldo Cruz Institute in Rio de Janeiro, which acts as a national reference center for respiratory viruses with the Brazilian Ministry of Health (MS) and as a reference for the World Health Organization.
 

A case of reinfection

The case patient that led to the alarm was a 45-year-old woman who is a health care executive. She had no comorbidities. The team had been following health care professionals and patients who had tested positive on reverse transcription–polymerase chain reaction (RT-PCR) testing more than once to understand whether they represented cases of prolonged viral persistence or new infections.

The woman had symptoms of viral infection on two occasions (May 26 and Oct. 26). On both occasions, results of RT-PCR testing for SARS-CoV-2 on nasopharyngeal samples were positive. In the first episode, the patient had diarrhea, myalgia, asthenia, and odynophagia for about 7 days. She returned to activities 21 days later. In the second episode, she had more severe symptoms that lasted longer, but she still did not require hospitalization.

“It was the first confirmed case of reinfection in Bahia, and in the second episode, we observed a mutation that could have an impact on the ability of antibodies to neutralize the virus,” Dr. Souza said. “The research continues with the investigation of cases in which the patient has a positive SARS-CoV-2 RT-PCR more than once in an interval greater than 45 days, to have a higher level of evidence.”

He stressed that “it is very important to reinforce measures to control the pandemic, social distance, use of masks, and speed up vaccination to be able to control the circulation of the virus, while monitoring the evolution of it.”
 

On alert for more cases

A person who twice tests positive for SARS-CoV-2 on real-time RT-PCR is suspected of having been reinfected, provided 90 or more days have elapsed between the two episodes, regardless of the condition observed. To confirm the suspected case, the samples must be sent to reference laboratories according to a plan established by the Ministry of Health in Brazil.

A health professional living in the Brazilian city of Natal represented the first confirmed case of reinfection by the new coronavirus in Brazil. That case was announced on Dec. 10, 2020.

“We communicated this case of reinfection to the MS in early December 2020. And the second sample already had the E484K mutation on the spike, as in the case of Bahia,” said Dr. Motta.

The first step in differentiating reinfection from persistence is to observe differences in the genotyping of the virus. For the technique to be successful, Dr. Souza said, researchers need a large amount of viral genetic material, which usually cannot be obtained.

“That is why there are many more suspected than confirmed cases,” Dr. Souza explained. He admitted that, although there are few cases, “it is increasingly clear that reinfection is a reality.”
 

Markers of mutations

What worried the researchers most was not only the possibility of reinfection but also the fact that preliminary analyses showed a specific mutation.

“The E484K mutation is present in a group of variants identified in South Africa that have been associated with increased infectivity and has been observed in a strain recently described in Brazil,” Dr. Souza said.

Mutations are expected, appear spontaneously, and in most cases have no effects on transmission or clinical outcome – they are simply used as markers and are useful for contact tracing or studying transmission routes. But some mutations can last because they provide an advantage for the pathogen, even if only momentary. In the case of SARS-CoV-2, mutations in the protein spike gene (S) are relevant because they may give clues to that advantage – as well as to changes in infectivity, transmission potential, antibodies, and response to vaccines.

A variant of the virus that has eight changes that affect the protein S gene – and several others in different genes – is behind the increase in the number of cases in London and southeastern England. Researchers from the University of São Paulo identified one of the factors that made this new variant – classified as B.1.1.7 – more infectious.

With bioinformatics tools, they found that the protein S gene in the new viral strain has a stronger molecular interaction with the ACE2 receptor, which is on the surface of human cells and to which the virus binds, making infection possible. The variant has already spread to the rest of the world, and the first two cases have been confirmed in Brazil by the Adolf Lutz Institute.

The alert for a new variant in Africa – similar to B.1.1.7 in the United Kingdom in that it carries nine changes in protein S at position 501 – was made by the Brazilian virologist Tulio de Oliveira, PhD.

“We found that this strain seems to be spreading much faster,” Dr. Oliveira, who is with the University of KwaZulu Natal, told the journal Science. His work first alerted British scientists to the importance of the position N501Y.

“The new variants just described in the United Kingdom and South Africa are slightly more transmissible and have already been identified in cases imported into Brazil,” Dr. Motta said. “Unfortunately, we believe it is only a matter of time before it becomes indigenous.”
 

The viral family grows

Viruses such as SARS-CoV-2 are classified into strains on the basis of small differences in their genetic material. Since Dec. 26, 2020, in addition to the British and South African variants, it appears the Carioca lineage also is a player.

In a preprint article, researchers analyzed the evolution of the epidemic in Rio de Janeiro from April 2020 until just before the new increase in incidence in December. They compared the complete sequences of the viral genome of 180 patients from different municipalities. The study, which is being jointly conducted by members of the Federal University of Rio de Janeiro and the National Laboratory for Scientific Computing, identified a new variant of SARS-CoV-2 that has five unique mutations (from one of the predominant strains). Concern arose because, in addition to those five genetic changes, many of the samples had a sixth – the well-known E484K mutation.

“The three lines – the U.K., South Africa, and Brazil – were almost synchronous publications, but there is no clear evidence that they have any kind of common ancestry,” Carolina M. Voloch, PhD, the article’s first author and a biologist and researcher at the Molecular Virology Laboratory and associate professor in the department of genetics at the Federal University of Rio de Janeiro, said in an interview.

Dr. Voloch’s research focuses on the use of bioinformatics tools to study the molecular, phylogenetic, and genomic evolution of viruses.

“The emergence of new strains is common for viruses,” she said. “It can be happening anywhere in the world at any time.”

She stressed that identifying when mutations emerge will help to define the new Brazilian lineage. Researchers are working to determine whether the neutralizing antibodies of patients who have been infected with other strains respond to this Rio de Janeiro strain.

“We hope to soon be sharing these results,” Dr. Voloch said.

The article’s authors estimated that the new strain likely appeared in early July. They say more analysis is needed to predict whether the changes have a major effect on viral infectivity, the host’s immune response, or the severity of the disease. Asked about the lineage that caused the reinfection in Bahia, Dr. Voloch said she hadn’t yet contacted the authors to conduct a joint analysis but added that the data disclosed in the preprint would not represent the same variant.

“There are only two of the five mutations that characterize the Rio de Janeiro lineage. However, it has the E484K mutation that is present in more than 94% of the samples of the new variant of Rio,” she said.

She added that there’s a possibility of reinfection by the lineage that’s circulating in Rio de Janeiro and in other states, as well as countries such as the United States, the United Kingdom, and Japan.

“The Carioca virus is being exported to the rest of the world,” Dr. Voloch said.
 

Virus’ diversity still unknown

Researchers now know that SARS-CoV-2 probably circulated silently in Brazil as early as February 2020 and reached all the nation’s regions before air travel was restricted. Since the first half of 2020, there have been two predominant strains.

“More than a dozen strains have been identified in Brazil, but more important than counting strains to identify the speed with which they arise – which is directly associated with the rate of infection, which is very high in the country,” said Dr. Motta.

The so-called variant of Rio de Janeiro, he said, has also been detected in other states in four regions of Brazil. The key to documenting variants is to get a more representative sample with genomes from other parts of the country.

As of Jan. 10, a total of 347,000 complete genome sequences had been shared globally through open databases since SARS-CoV-2 was first identified, but the contribution of countries is uneven. Although the cost and complexity of genetic sequencing has dropped significantly over time, effective sequencing programs still require substantial investments in personnel, equipment, reagents, and bioinformatics infrastructure.

According to Dr. Voloch, it will only be possible to combat the new coronavirus by knowing its diversity and understanding how it evolves. The Fiocruz Genomic Network has made an infographic available so researchers can track the strains circulating in Brazil. It›s the result of collaboration between researchers from Fiocruz and the GISAID Initiative, an international partnership that promotes rapid data sharing.

As of Jan. 5, researchers in Brazil had studied 1,897 genomes – not nearly enough.

“In Brazil, there is little testing and even less sequencing,” lamented Dr. Souza.

“In the U.K., 1 in 600 cases is sequenced. In Brazil it is less than 1 in 10 million cases,” Dr. Voloch added.

So far, no decisive factors for public health, such as greater virulence or greater transmissibility, have been identified in any of the strains established in Brazil. The million-dollar question is whether the emergence of new strains could have an impact on the effectiveness of vaccines being administered today.

“In one way or another, the vaccine is our best bet ever, even if in the future we identify escapist mutants and have to modify it,” Dr. Motta said. “It is what we do annually with influenza.”

Dr. Voloch, Dr. Motta, and Dr. Souza disclosed no relevant financial relationships.

A version of this article first appeared on the Portuguese edition of Medscape.com.

Just as Brazil surpassed 200,000 deaths from COVID-19 on Jan. 7, news from Bahia added another layer of concern: A platform case report in a preprint detailed the first case of reinfection in that state, apparently caused by a new strain, one having the E484K mutation.

That variant, now called Brazil P.1, has migrated to the United States. The Minnesota Department of Health announced on Jan. 25 the nation’s first known COVID-19 case associated with it.

The mutation is located in the protein gene of the virus’ spike, which forms the crown structure of coronaviruses and is responsible for the virus’ binding to human cells. The E484K mutation is now the focus because it’s associated with mutations that escape the immune system’s neutralizing antibodies.

“This mutation is at the center of worldwide concern, and it is the first time that it has appeared in a reinfection,” the study’s first author, Bruno Solano de Freitas Souza, MD, a researcher at the Salvador regional unit of Instituto D’Or of Teaching and Research, based at Hospital São Rafael, Salvador, Brazil, explained in an interview.

“We will wait for the sample from Bahia to confirm the case from the perspective of the Ministry of Health’s surveillance network,” said Fernando Motta, PhD, deputy head of the Laboratory for Respiratory Virus and Measles at the Oswaldo Cruz Institute in Rio de Janeiro, which acts as a national reference center for respiratory viruses with the Brazilian Ministry of Health (MS) and as a reference for the World Health Organization.
 

A case of reinfection

The case patient that led to the alarm was a 45-year-old woman who is a health care executive. She had no comorbidities. The team had been following health care professionals and patients who had tested positive on reverse transcription–polymerase chain reaction (RT-PCR) testing more than once to understand whether they represented cases of prolonged viral persistence or new infections.

The woman had symptoms of viral infection on two occasions (May 26 and Oct. 26). On both occasions, results of RT-PCR testing for SARS-CoV-2 on nasopharyngeal samples were positive. In the first episode, the patient had diarrhea, myalgia, asthenia, and odynophagia for about 7 days. She returned to activities 21 days later. In the second episode, she had more severe symptoms that lasted longer, but she still did not require hospitalization.

“It was the first confirmed case of reinfection in Bahia, and in the second episode, we observed a mutation that could have an impact on the ability of antibodies to neutralize the virus,” Dr. Souza said. “The research continues with the investigation of cases in which the patient has a positive SARS-CoV-2 RT-PCR more than once in an interval greater than 45 days, to have a higher level of evidence.”

He stressed that “it is very important to reinforce measures to control the pandemic, social distance, use of masks, and speed up vaccination to be able to control the circulation of the virus, while monitoring the evolution of it.”
 

On alert for more cases

A person who twice tests positive for SARS-CoV-2 on real-time RT-PCR is suspected of having been reinfected, provided 90 or more days have elapsed between the two episodes, regardless of the condition observed. To confirm the suspected case, the samples must be sent to reference laboratories according to a plan established by the Ministry of Health in Brazil.

A health professional living in the Brazilian city of Natal represented the first confirmed case of reinfection by the new coronavirus in Brazil. That case was announced on Dec. 10, 2020.

“We communicated this case of reinfection to the MS in early December 2020. And the second sample already had the E484K mutation on the spike, as in the case of Bahia,” said Dr. Motta.

The first step in differentiating reinfection from persistence is to observe differences in the genotyping of the virus. For the technique to be successful, Dr. Souza said, researchers need a large amount of viral genetic material, which usually cannot be obtained.

“That is why there are many more suspected than confirmed cases,” Dr. Souza explained. He admitted that, although there are few cases, “it is increasingly clear that reinfection is a reality.”
 

Markers of mutations

What worried the researchers most was not only the possibility of reinfection but also the fact that preliminary analyses showed a specific mutation.

“The E484K mutation is present in a group of variants identified in South Africa that have been associated with increased infectivity and has been observed in a strain recently described in Brazil,” Dr. Souza said.

Mutations are expected, appear spontaneously, and in most cases have no effects on transmission or clinical outcome – they are simply used as markers and are useful for contact tracing or studying transmission routes. But some mutations can last because they provide an advantage for the pathogen, even if only momentary. In the case of SARS-CoV-2, mutations in the protein spike gene (S) are relevant because they may give clues to that advantage – as well as to changes in infectivity, transmission potential, antibodies, and response to vaccines.

A variant of the virus that has eight changes that affect the protein S gene – and several others in different genes – is behind the increase in the number of cases in London and southeastern England. Researchers from the University of São Paulo identified one of the factors that made this new variant – classified as B.1.1.7 – more infectious.

With bioinformatics tools, they found that the protein S gene in the new viral strain has a stronger molecular interaction with the ACE2 receptor, which is on the surface of human cells and to which the virus binds, making infection possible. The variant has already spread to the rest of the world, and the first two cases have been confirmed in Brazil by the Adolf Lutz Institute.

The alert for a new variant in Africa – similar to B.1.1.7 in the United Kingdom in that it carries nine changes in protein S at position 501 – was made by the Brazilian virologist Tulio de Oliveira, PhD.

“We found that this strain seems to be spreading much faster,” Dr. Oliveira, who is with the University of KwaZulu Natal, told the journal Science. His work first alerted British scientists to the importance of the position N501Y.

“The new variants just described in the United Kingdom and South Africa are slightly more transmissible and have already been identified in cases imported into Brazil,” Dr. Motta said. “Unfortunately, we believe it is only a matter of time before it becomes indigenous.”
 

The viral family grows

Viruses such as SARS-CoV-2 are classified into strains on the basis of small differences in their genetic material. Since Dec. 26, 2020, in addition to the British and South African variants, it appears the Carioca lineage also is a player.

In a preprint article, researchers analyzed the evolution of the epidemic in Rio de Janeiro from April 2020 until just before the new increase in incidence in December. They compared the complete sequences of the viral genome of 180 patients from different municipalities. The study, which is being jointly conducted by members of the Federal University of Rio de Janeiro and the National Laboratory for Scientific Computing, identified a new variant of SARS-CoV-2 that has five unique mutations (from one of the predominant strains). Concern arose because, in addition to those five genetic changes, many of the samples had a sixth – the well-known E484K mutation.

“The three lines – the U.K., South Africa, and Brazil – were almost synchronous publications, but there is no clear evidence that they have any kind of common ancestry,” Carolina M. Voloch, PhD, the article’s first author and a biologist and researcher at the Molecular Virology Laboratory and associate professor in the department of genetics at the Federal University of Rio de Janeiro, said in an interview.

Dr. Voloch’s research focuses on the use of bioinformatics tools to study the molecular, phylogenetic, and genomic evolution of viruses.

“The emergence of new strains is common for viruses,” she said. “It can be happening anywhere in the world at any time.”

She stressed that identifying when mutations emerge will help to define the new Brazilian lineage. Researchers are working to determine whether the neutralizing antibodies of patients who have been infected with other strains respond to this Rio de Janeiro strain.

“We hope to soon be sharing these results,” Dr. Voloch said.

The article’s authors estimated that the new strain likely appeared in early July. They say more analysis is needed to predict whether the changes have a major effect on viral infectivity, the host’s immune response, or the severity of the disease. Asked about the lineage that caused the reinfection in Bahia, Dr. Voloch said she hadn’t yet contacted the authors to conduct a joint analysis but added that the data disclosed in the preprint would not represent the same variant.

“There are only two of the five mutations that characterize the Rio de Janeiro lineage. However, it has the E484K mutation that is present in more than 94% of the samples of the new variant of Rio,” she said.

She added that there’s a possibility of reinfection by the lineage that’s circulating in Rio de Janeiro and in other states, as well as countries such as the United States, the United Kingdom, and Japan.

“The Carioca virus is being exported to the rest of the world,” Dr. Voloch said.
 

Virus’ diversity still unknown

Researchers now know that SARS-CoV-2 probably circulated silently in Brazil as early as February 2020 and reached all the nation’s regions before air travel was restricted. Since the first half of 2020, there have been two predominant strains.

“More than a dozen strains have been identified in Brazil, but more important than counting strains to identify the speed with which they arise – which is directly associated with the rate of infection, which is very high in the country,” said Dr. Motta.

The so-called variant of Rio de Janeiro, he said, has also been detected in other states in four regions of Brazil. The key to documenting variants is to get a more representative sample with genomes from other parts of the country.

As of Jan. 10, a total of 347,000 complete genome sequences had been shared globally through open databases since SARS-CoV-2 was first identified, but the contribution of countries is uneven. Although the cost and complexity of genetic sequencing has dropped significantly over time, effective sequencing programs still require substantial investments in personnel, equipment, reagents, and bioinformatics infrastructure.

According to Dr. Voloch, it will only be possible to combat the new coronavirus by knowing its diversity and understanding how it evolves. The Fiocruz Genomic Network has made an infographic available so researchers can track the strains circulating in Brazil. It›s the result of collaboration between researchers from Fiocruz and the GISAID Initiative, an international partnership that promotes rapid data sharing.

As of Jan. 5, researchers in Brazil had studied 1,897 genomes – not nearly enough.

“In Brazil, there is little testing and even less sequencing,” lamented Dr. Souza.

“In the U.K., 1 in 600 cases is sequenced. In Brazil it is less than 1 in 10 million cases,” Dr. Voloch added.

So far, no decisive factors for public health, such as greater virulence or greater transmissibility, have been identified in any of the strains established in Brazil. The million-dollar question is whether the emergence of new strains could have an impact on the effectiveness of vaccines being administered today.

“In one way or another, the vaccine is our best bet ever, even if in the future we identify escapist mutants and have to modify it,” Dr. Motta said. “It is what we do annually with influenza.”

Dr. Voloch, Dr. Motta, and Dr. Souza disclosed no relevant financial relationships.

A version of this article first appeared on the Portuguese edition of Medscape.com.

To the Editor:

Pembrolizumab, a humanized monoclonal anti–programmed cell death protein 1 (PD-1) antibody, acts by blocking negative immune regulators such as PD-1.¹ Since its approval by the US Food and Drug Administration in 2014, the use of PD-1 inhibitors such as pembrolizumab has dramatically increased, and they are now the standard of care for cancers such as melanoma, lung cancer, and renal cell carcinoma.^2,3 With increased use comes a better understanding of the cutaneous adverse effects that may occur. To date, almost 50% of patients treated with PD-1 inhibitors will develop an adverse cutaneous reaction.⁴ Thus far, cases of patients developing vitiligo, bullous pemphigoid, psoriasis, granulomatous skin reactions, severe cutaneous reactions (ie, toxic epidermal necrolysis), lupus erythematosus, and lichenoid reactions have been described.^3,5,6 There are fewer than 30 documented cases of lichenoid reactions due to anti–PD-1 treatment described in the literature, increasing the importance of case reports to demonstrate a full range of cutaneous findings.³ We present a case of a reaction to pembrolizumab with an eruption of lichenoid papules predominantly involving the hands and feet as well as nail changes.

A 60-year-old man with ocular melanoma metastatic to the right lung, transverse colon, and right axillary lymph nodes presented with a chief concern of growing skin lesions present for 6 weeks on the hands and feet. The lesions were tender to the touch and occasionally drained a clear fluid. He also reported nail fragility. Of note, the patient was being treated for metastatic melanoma with pembrolizumab infusions every 3 weeks, which started 6 weeks prior to the onset of the eruption. 

Physical examination demonstrated lichenoid papules on the dorsal and ventral aspects of the hands and feet (Figure 1), as well as longitudinal ridging on numerous fingernails and mild koilonychia. A punch biopsy revealed lichenoid interface dermatitis with irregular epidermal hyperplasia (Figure 2). A diagnosis of hypertrophic lichen planus–like drug eruption in response to pembrolizumab was made and clobetasol cream was prescribed.

At 1-month follow-up, the patient reported notable improvement with clobetasol, and he was transitioned to tacrolimus ointment 0.1%. He continued to improve until a month later when he reported new lesions arising a week after a pembrolizumab infusion. He continued to use clobetasol cream for flares and tacrolimus ointment for maintenance.

Almost 3 months after the initial visit, the patient presented with inflammation around his right third fingernail of 1 week’s duration, with more notable fragility than his other nails. No trauma was described, and the nail abnormality was attributed to pembrolizumab. Clobetasol cream and biotin 3 mg daily resulted in improvement, and no other nails were affected in a similar way.

Programmed cell death protein 1 blockers are associated with a variety of adverse events including hypothyroidism, gastrointestinal abnormalities, fatigue, and skin disorders.⁷ In one study (N=83), cutaneous adverse drug events were found to occur in 42% (35/83) of patients following pembrolizumab therapy, with the most common cutaneous lesions being maculopapular eruptions (29% [24/83]), pruritus (12% [10/83]), and hypopigmentation (8% [7/83]).⁵

A total of 29 cases of lichenoid dermatitis following anti–PD-1 therapy have been described in the literature.³ Cases range from an eruption of photodistributed hyperkeratotic papules and plaques to hypertrophic vesiculobullous lesions.^3,6 Suggested pathophysiology involves blocking the interaction of programmed death ligand 1 on keratinocytes with PD-1 on T cells.³ Management typically includes topical or systemic steroids. Cyclosporine and acitretin also have been successful in a small number of patients. Most patients continue anti–PD-1 treatment with systemic therapy.³

Our patient represents a similar lichenoid eruption; however, the distribution on the dorsal and ventral aspects of the hands and feet as well as nail dystrophy make the presentation unique. Anticancer drugs that increase the T-cell immune response by altering the complex signaling among T cells, antigen-presenting cells, and tumor cells have been associated with cutaneous eruptions. Although the exact mechanism is still not fully understood, clinical suspicion of a pembrolizumab reaction should remain high on the differential in the setting of hyperkeratotic papules in association with anti–PD-1 therapy.

To the Editor:

Pembrolizumab, a humanized monoclonal anti–programmed cell death protein 1 (PD-1) antibody, acts by blocking negative immune regulators such as PD-1.¹ Since its approval by the US Food and Drug Administration in 2014, the use of PD-1 inhibitors such as pembrolizumab has dramatically increased, and they are now the standard of care for cancers such as melanoma, lung cancer, and renal cell carcinoma.^2,3 With increased use comes a better understanding of the cutaneous adverse effects that may occur. To date, almost 50% of patients treated with PD-1 inhibitors will develop an adverse cutaneous reaction.⁴ Thus far, cases of patients developing vitiligo, bullous pemphigoid, psoriasis, granulomatous skin reactions, severe cutaneous reactions (ie, toxic epidermal necrolysis), lupus erythematosus, and lichenoid reactions have been described.^3,5,6 There are fewer than 30 documented cases of lichenoid reactions due to anti–PD-1 treatment described in the literature, increasing the importance of case reports to demonstrate a full range of cutaneous findings.³ We present a case of a reaction to pembrolizumab with an eruption of lichenoid papules predominantly involving the hands and feet as well as nail changes.

A 60-year-old man with ocular melanoma metastatic to the right lung, transverse colon, and right axillary lymph nodes presented with a chief concern of growing skin lesions present for 6 weeks on the hands and feet. The lesions were tender to the touch and occasionally drained a clear fluid. He also reported nail fragility. Of note, the patient was being treated for metastatic melanoma with pembrolizumab infusions every 3 weeks, which started 6 weeks prior to the onset of the eruption. 

Physical examination demonstrated lichenoid papules on the dorsal and ventral aspects of the hands and feet (Figure 1), as well as longitudinal ridging on numerous fingernails and mild koilonychia. A punch biopsy revealed lichenoid interface dermatitis with irregular epidermal hyperplasia (Figure 2). A diagnosis of hypertrophic lichen planus–like drug eruption in response to pembrolizumab was made and clobetasol cream was prescribed.

At 1-month follow-up, the patient reported notable improvement with clobetasol, and he was transitioned to tacrolimus ointment 0.1%. He continued to improve until a month later when he reported new lesions arising a week after a pembrolizumab infusion. He continued to use clobetasol cream for flares and tacrolimus ointment for maintenance.

Almost 3 months after the initial visit, the patient presented with inflammation around his right third fingernail of 1 week’s duration, with more notable fragility than his other nails. No trauma was described, and the nail abnormality was attributed to pembrolizumab. Clobetasol cream and biotin 3 mg daily resulted in improvement, and no other nails were affected in a similar way.

Programmed cell death protein 1 blockers are associated with a variety of adverse events including hypothyroidism, gastrointestinal abnormalities, fatigue, and skin disorders.⁷ In one study (N=83), cutaneous adverse drug events were found to occur in 42% (35/83) of patients following pembrolizumab therapy, with the most common cutaneous lesions being maculopapular eruptions (29% [24/83]), pruritus (12% [10/83]), and hypopigmentation (8% [7/83]).⁵

A total of 29 cases of lichenoid dermatitis following anti–PD-1 therapy have been described in the literature.³ Cases range from an eruption of photodistributed hyperkeratotic papules and plaques to hypertrophic vesiculobullous lesions.^3,6 Suggested pathophysiology involves blocking the interaction of programmed death ligand 1 on keratinocytes with PD-1 on T cells.³ Management typically includes topical or systemic steroids. Cyclosporine and acitretin also have been successful in a small number of patients. Most patients continue anti–PD-1 treatment with systemic therapy.³

Our patient represents a similar lichenoid eruption; however, the distribution on the dorsal and ventral aspects of the hands and feet as well as nail dystrophy make the presentation unique. Anticancer drugs that increase the T-cell immune response by altering the complex signaling among T cells, antigen-presenting cells, and tumor cells have been associated with cutaneous eruptions. Although the exact mechanism is still not fully understood, clinical suspicion of a pembrolizumab reaction should remain high on the differential in the setting of hyperkeratotic papules in association with anti–PD-1 therapy.

To the Editor:

Pembrolizumab, a humanized monoclonal anti–programmed cell death protein 1 (PD-1) antibody, acts by blocking negative immune regulators such as PD-1.¹ Since its approval by the US Food and Drug Administration in 2014, the use of PD-1 inhibitors such as pembrolizumab has dramatically increased, and they are now the standard of care for cancers such as melanoma, lung cancer, and renal cell carcinoma.^2,3 With increased use comes a better understanding of the cutaneous adverse effects that may occur. To date, almost 50% of patients treated with PD-1 inhibitors will develop an adverse cutaneous reaction.⁴ Thus far, cases of patients developing vitiligo, bullous pemphigoid, psoriasis, granulomatous skin reactions, severe cutaneous reactions (ie, toxic epidermal necrolysis), lupus erythematosus, and lichenoid reactions have been described.^3,5,6 There are fewer than 30 documented cases of lichenoid reactions due to anti–PD-1 treatment described in the literature, increasing the importance of case reports to demonstrate a full range of cutaneous findings.³ We present a case of a reaction to pembrolizumab with an eruption of lichenoid papules predominantly involving the hands and feet as well as nail changes.

A 60-year-old man with ocular melanoma metastatic to the right lung, transverse colon, and right axillary lymph nodes presented with a chief concern of growing skin lesions present for 6 weeks on the hands and feet. The lesions were tender to the touch and occasionally drained a clear fluid. He also reported nail fragility. Of note, the patient was being treated for metastatic melanoma with pembrolizumab infusions every 3 weeks, which started 6 weeks prior to the onset of the eruption. 

Physical examination demonstrated lichenoid papules on the dorsal and ventral aspects of the hands and feet (Figure 1), as well as longitudinal ridging on numerous fingernails and mild koilonychia. A punch biopsy revealed lichenoid interface dermatitis with irregular epidermal hyperplasia (Figure 2). A diagnosis of hypertrophic lichen planus–like drug eruption in response to pembrolizumab was made and clobetasol cream was prescribed.

At 1-month follow-up, the patient reported notable improvement with clobetasol, and he was transitioned to tacrolimus ointment 0.1%. He continued to improve until a month later when he reported new lesions arising a week after a pembrolizumab infusion. He continued to use clobetasol cream for flares and tacrolimus ointment for maintenance.

Almost 3 months after the initial visit, the patient presented with inflammation around his right third fingernail of 1 week’s duration, with more notable fragility than his other nails. No trauma was described, and the nail abnormality was attributed to pembrolizumab. Clobetasol cream and biotin 3 mg daily resulted in improvement, and no other nails were affected in a similar way.

Programmed cell death protein 1 blockers are associated with a variety of adverse events including hypothyroidism, gastrointestinal abnormalities, fatigue, and skin disorders.⁷ In one study (N=83), cutaneous adverse drug events were found to occur in 42% (35/83) of patients following pembrolizumab therapy, with the most common cutaneous lesions being maculopapular eruptions (29% [24/83]), pruritus (12% [10/83]), and hypopigmentation (8% [7/83]).⁵

A total of 29 cases of lichenoid dermatitis following anti–PD-1 therapy have been described in the literature.³ Cases range from an eruption of photodistributed hyperkeratotic papules and plaques to hypertrophic vesiculobullous lesions.^3,6 Suggested pathophysiology involves blocking the interaction of programmed death ligand 1 on keratinocytes with PD-1 on T cells.³ Management typically includes topical or systemic steroids. Cyclosporine and acitretin also have been successful in a small number of patients. Most patients continue anti–PD-1 treatment with systemic therapy.³

Our patient represents a similar lichenoid eruption; however, the distribution on the dorsal and ventral aspects of the hands and feet as well as nail dystrophy make the presentation unique. Anticancer drugs that increase the T-cell immune response by altering the complex signaling among T cells, antigen-presenting cells, and tumor cells have been associated with cutaneous eruptions. Although the exact mechanism is still not fully understood, clinical suspicion of a pembrolizumab reaction should remain high on the differential in the setting of hyperkeratotic papules in association with anti–PD-1 therapy.

Seven days into his presidency, Joe Biden announced that he is taking new steps to speed vaccines to Americans.

The president said he would increase the supply of vaccines to states from 8.6 million doses to 10 million doses per week, a 16% increase, for at least the next 3 weeks.

He said he was working to give states more advanced notice of their allotments so they could better plan their campaigns. He also said doses would be doled out based on population.

“We will both increase the supply and give our state and local partners more certainty about when doses will arrive,” he said Tuesday.

Finally, Mr. Biden announced that the United States would “soon be able to confirm” the purchase of 200 million more doses of the Pfizer and Moderna vaccines – 100 million of each – to effectively double the nation’s supply by “early summer.” That would increase the nation’s supply enough to fully vaccinate 300 million Americans by fall.

Mr. Biden said he was also working to shift the focus to getting more doses to economically disadvantaged communities and rural areas, which have fallen further behind as the vaccine rollout has faltered.

Even with these steps, Mr. Biden stressed that it would take months for vaccines to curb infections and deaths. He said, for the time being, masks, not vaccines, are the best way to save lives.

“The brutal truth is its going to take months before we get the majority of Americans vaccinated. Months,” he said, adding that wearing masks until at least April could save to save 50,000 lives.

“Let me be clear,” Mr. Biden said, “Things are going to get worse before they get better.

“We didn’t get into this mess overnight. It’s going to take months for us to turn things around. But let me be equally clear we’re going to get through this. We will defeat this pandemic,” he said.

A version of this article first appeared on WebMD.com.

Seven days into his presidency, Joe Biden announced that he is taking new steps to speed vaccines to Americans.

The president said he would increase the supply of vaccines to states from 8.6 million doses to 10 million doses per week, a 16% increase, for at least the next 3 weeks.

He said he was working to give states more advanced notice of their allotments so they could better plan their campaigns. He also said doses would be doled out based on population.

“We will both increase the supply and give our state and local partners more certainty about when doses will arrive,” he said Tuesday.

Finally, Mr. Biden announced that the United States would “soon be able to confirm” the purchase of 200 million more doses of the Pfizer and Moderna vaccines – 100 million of each – to effectively double the nation’s supply by “early summer.” That would increase the nation’s supply enough to fully vaccinate 300 million Americans by fall.

Mr. Biden said he was also working to shift the focus to getting more doses to economically disadvantaged communities and rural areas, which have fallen further behind as the vaccine rollout has faltered.

Even with these steps, Mr. Biden stressed that it would take months for vaccines to curb infections and deaths. He said, for the time being, masks, not vaccines, are the best way to save lives.

“The brutal truth is its going to take months before we get the majority of Americans vaccinated. Months,” he said, adding that wearing masks until at least April could save to save 50,000 lives.

“Let me be clear,” Mr. Biden said, “Things are going to get worse before they get better.

“We didn’t get into this mess overnight. It’s going to take months for us to turn things around. But let me be equally clear we’re going to get through this. We will defeat this pandemic,” he said.

A version of this article first appeared on WebMD.com.

Seven days into his presidency, Joe Biden announced that he is taking new steps to speed vaccines to Americans.

The president said he would increase the supply of vaccines to states from 8.6 million doses to 10 million doses per week, a 16% increase, for at least the next 3 weeks.

He said he was working to give states more advanced notice of their allotments so they could better plan their campaigns. He also said doses would be doled out based on population.

“We will both increase the supply and give our state and local partners more certainty about when doses will arrive,” he said Tuesday.

Finally, Mr. Biden announced that the United States would “soon be able to confirm” the purchase of 200 million more doses of the Pfizer and Moderna vaccines – 100 million of each – to effectively double the nation’s supply by “early summer.” That would increase the nation’s supply enough to fully vaccinate 300 million Americans by fall.

Mr. Biden said he was also working to shift the focus to getting more doses to economically disadvantaged communities and rural areas, which have fallen further behind as the vaccine rollout has faltered.

Even with these steps, Mr. Biden stressed that it would take months for vaccines to curb infections and deaths. He said, for the time being, masks, not vaccines, are the best way to save lives.

“The brutal truth is its going to take months before we get the majority of Americans vaccinated. Months,” he said, adding that wearing masks until at least April could save to save 50,000 lives.

“Let me be clear,” Mr. Biden said, “Things are going to get worse before they get better.

“We didn’t get into this mess overnight. It’s going to take months for us to turn things around. But let me be equally clear we’re going to get through this. We will defeat this pandemic,” he said.

A version of this article first appeared on WebMD.com.

Although inflammatory bowel disease (IBD) affects primarily White patients, about one-quarter of cases are found in non-White racial and ethnic groups. Various factors have combined to lead to disparities in treatment and outcomes for non-Whites with IBD.

Ethnic and racial disparities, along with socioeconomic factors, were the subject of a presentation by Ruby Greywoode, MD, at the annual congress of the Crohn’s & Colitis Foundation and the American Gastroenterological Association.

“Historical and present-day realities of racial inequity and factors that contribute to socioeconomic status [include] educational and housing policies, employment practices, and generational wealth. Addressing health disparities requires acknowledging these systemic factors,” said Dr. Greywoode, who is with Montefiore Medical Center in New York.

An important concept in discussing health disparity is social determinants of health, which refers to nonbiological factors that affect health and health outcomes. These are “the conditions in which people live, work, learn, and play that affect their health and their quality of life,” said Dr. Greywoode. 

Dr. Greywoode shared examples of social determinants that affect economic stability and financial worry. One study found that one in six IBD patients reported not taking their medications because of cost considerations. A survey of about 900 adults showed that 1 in 4 delayed medical care – half of those because of cost; patients who delayed care were 2.5 times more likely to report an IBD flare in the previous year.

Another important issue is food insecurity. Other presenters at the session emphasized the importance of high-quality nutrition in IBD, and Dr. Greywoode presented one survey showing that only 9% of patients who had both food security and social support reported cost-related medication nonadherence. Among those that had either food insecurity or poor social support, 12% reported cost-related medication nonadherence, but the proportion jumped to 57% among patients who had both food insecurity and lack of social support.

Session comoderator Kelly Issokson noted that socioeconomic factors often interfere with adoption of healthy diets. Whole foods and plant-based foods are expensive, and the financial pressures of the COVID-19 epidemic have made that worse. “Millions of people are slipping into poverty and food insecurity. This is one of the things she highlighted as factors in medication nonadherence,” said Ms. Issokson, who is the clinical nutritional coordinator at the digestive disease clinic at Cedars-Sinai Medical Center in Los Angeles.

Dr. Greywoode also described studies that looked at race, socioeconomic status, and IBD outcomes. A review from 2013 showed disparities among Whites, African Americans, and Hispanics with respect to undergoing ulcerative colitis–related colectomy and Crohn’s disease–related bowel resection. Ulcerative colitis patients on Medicaid had 230% greater in-patient mortality, compared with patients with private insurance, even after adjustment for multiple confounders.

But inequities are not static. “Since this publication, we have numerous other studies drawing conclusions that sometimes agree with and sometimes conflict with it. My belief is that health disparities in IBD will continue to be an active area of research. We know that it takes vigilance to identify, track, and address any disparities when they do arise,” said Dr. Greywoode.

Dr. Greywoode also noted that phenotypic differences based on race and ethnicity influence disparities. She showed results from a meta-analysis that found a difference in the frequency of perianal Crohn’s disease by race and ethnicity; the highest frequency occurred in Black patients (31%), followed by Asians (22%), Whites (14%), and Hispanics (13%). Another study showed that African American patients with Crohn’s disease were more likely to develop a new abscess (adjusted odds ratio, 2.27; 95% confidence interval, 1.31-3.93) or anal fissure (aOR, 1.76; 95% CI, 1.01-3.07), and were also more likely to be initiated on an anti–tumor necrosis factor drug (aOR, 1.85; 95% CI, 1.09-3.14).

Those differences underscore the need to recognize that IBD is not just a disease for White patients. “As we move forward in IBD research, we recognize that individuals of European ancestry are not the only ones who have IBD. There is a growing diverse racial and ethnic population with IBD,” said Dr. Greywoode.

She noted that, in the United States, it is estimated that about one in four adult patients are non-Hispanic African American, Hispanic, Asians, or other ethnicities. Nevertheless, Whites are overrepresented among participants in IBD clinical trials. Some trials are composed of as much as 95% White patients, and sometimes race isn’t even listed. “It’s unclear if [race/ethnicity data are] not collected or not deemed important, but we know that what is not collected is not measured, and what is not measured can’t be evaluated, either to praise or constructively criticize,” said Dr. Greywoode.

Fortunately, there are efforts in place to improve representation in clinical trials. There has been a mandate for almost 3 decades that federally funded research must include racial and ethnic minorities who have been traditionally underrepresented. The Food and Drug Administration has also provided guidance to industry to improve diversity in clinical trial participation, and industry groups have developed strategies, including improved representation among investigators and related early-career development programs. At the community and independent health care practice levels, clinical trial networks encourage patient participation with regulatory and data management support to bolster practices with insufficient resources.

Underrepresentation in clinical trials resonated with comoderator Tina Aswani Omprakash, who is a patient advocate and a master’s in public health student at the Icahn School of Medicine at Mount Sinai, New York. She called for greater awareness among physicians that IBD can occur among people of all backgrounds. “[Providers] would look at me and [say]: ‘There’s no way that, as a South Asian woman, you have that kind of disease.’ There’s that lack of believability,” said Ms. Aswani Omprakash.

Greater recognition of the diversity of patients, as well as the phenotypic differences found among ethnicities, could also inform clinical trial participation and, ultimately, more personalized medicine. “We have to look at these things, observe how they’re affecting populations differently, so that we can have proper medication solutions,” said Ms. Aswani Omprakash.

Dr. Greywoode and Ms. Issokson have no relevant financial disclosures. Ms. Aswani Omprakash has consulted for Genentech, AbbVie, Janssen, and Arena.

The AGA applauds researchers who are working to raise our awareness of health disparities in digestive diseases. The AGA is committed to addressing this important societal issue head on. Learn more about AGA’s commitment through the AGA Equity Project.

Updated Feb. 17, 2021.
 

Although inflammatory bowel disease (IBD) affects primarily White patients, about one-quarter of cases are found in non-White racial and ethnic groups. Various factors have combined to lead to disparities in treatment and outcomes for non-Whites with IBD.

Ethnic and racial disparities, along with socioeconomic factors, were the subject of a presentation by Ruby Greywoode, MD, at the annual congress of the Crohn’s & Colitis Foundation and the American Gastroenterological Association.

“Historical and present-day realities of racial inequity and factors that contribute to socioeconomic status [include] educational and housing policies, employment practices, and generational wealth. Addressing health disparities requires acknowledging these systemic factors,” said Dr. Greywoode, who is with Montefiore Medical Center in New York.

An important concept in discussing health disparity is social determinants of health, which refers to nonbiological factors that affect health and health outcomes. These are “the conditions in which people live, work, learn, and play that affect their health and their quality of life,” said Dr. Greywoode. 

Dr. Greywoode shared examples of social determinants that affect economic stability and financial worry. One study found that one in six IBD patients reported not taking their medications because of cost considerations. A survey of about 900 adults showed that 1 in 4 delayed medical care – half of those because of cost; patients who delayed care were 2.5 times more likely to report an IBD flare in the previous year.

Another important issue is food insecurity. Other presenters at the session emphasized the importance of high-quality nutrition in IBD, and Dr. Greywoode presented one survey showing that only 9% of patients who had both food security and social support reported cost-related medication nonadherence. Among those that had either food insecurity or poor social support, 12% reported cost-related medication nonadherence, but the proportion jumped to 57% among patients who had both food insecurity and lack of social support.

Session comoderator Kelly Issokson noted that socioeconomic factors often interfere with adoption of healthy diets. Whole foods and plant-based foods are expensive, and the financial pressures of the COVID-19 epidemic have made that worse. “Millions of people are slipping into poverty and food insecurity. This is one of the things she highlighted as factors in medication nonadherence,” said Ms. Issokson, who is the clinical nutritional coordinator at the digestive disease clinic at Cedars-Sinai Medical Center in Los Angeles.

Dr. Greywoode also described studies that looked at race, socioeconomic status, and IBD outcomes. A review from 2013 showed disparities among Whites, African Americans, and Hispanics with respect to undergoing ulcerative colitis–related colectomy and Crohn’s disease–related bowel resection. Ulcerative colitis patients on Medicaid had 230% greater in-patient mortality, compared with patients with private insurance, even after adjustment for multiple confounders.

But inequities are not static. “Since this publication, we have numerous other studies drawing conclusions that sometimes agree with and sometimes conflict with it. My belief is that health disparities in IBD will continue to be an active area of research. We know that it takes vigilance to identify, track, and address any disparities when they do arise,” said Dr. Greywoode.

Dr. Greywoode also noted that phenotypic differences based on race and ethnicity influence disparities. She showed results from a meta-analysis that found a difference in the frequency of perianal Crohn’s disease by race and ethnicity; the highest frequency occurred in Black patients (31%), followed by Asians (22%), Whites (14%), and Hispanics (13%). Another study showed that African American patients with Crohn’s disease were more likely to develop a new abscess (adjusted odds ratio, 2.27; 95% confidence interval, 1.31-3.93) or anal fissure (aOR, 1.76; 95% CI, 1.01-3.07), and were also more likely to be initiated on an anti–tumor necrosis factor drug (aOR, 1.85; 95% CI, 1.09-3.14).

Those differences underscore the need to recognize that IBD is not just a disease for White patients. “As we move forward in IBD research, we recognize that individuals of European ancestry are not the only ones who have IBD. There is a growing diverse racial and ethnic population with IBD,” said Dr. Greywoode.

She noted that, in the United States, it is estimated that about one in four adult patients are non-Hispanic African American, Hispanic, Asians, or other ethnicities. Nevertheless, Whites are overrepresented among participants in IBD clinical trials. Some trials are composed of as much as 95% White patients, and sometimes race isn’t even listed. “It’s unclear if [race/ethnicity data are] not collected or not deemed important, but we know that what is not collected is not measured, and what is not measured can’t be evaluated, either to praise or constructively criticize,” said Dr. Greywoode.

Fortunately, there are efforts in place to improve representation in clinical trials. There has been a mandate for almost 3 decades that federally funded research must include racial and ethnic minorities who have been traditionally underrepresented. The Food and Drug Administration has also provided guidance to industry to improve diversity in clinical trial participation, and industry groups have developed strategies, including improved representation among investigators and related early-career development programs. At the community and independent health care practice levels, clinical trial networks encourage patient participation with regulatory and data management support to bolster practices with insufficient resources.

Underrepresentation in clinical trials resonated with comoderator Tina Aswani Omprakash, who is a patient advocate and a master’s in public health student at the Icahn School of Medicine at Mount Sinai, New York. She called for greater awareness among physicians that IBD can occur among people of all backgrounds. “[Providers] would look at me and [say]: ‘There’s no way that, as a South Asian woman, you have that kind of disease.’ There’s that lack of believability,” said Ms. Aswani Omprakash.

Greater recognition of the diversity of patients, as well as the phenotypic differences found among ethnicities, could also inform clinical trial participation and, ultimately, more personalized medicine. “We have to look at these things, observe how they’re affecting populations differently, so that we can have proper medication solutions,” said Ms. Aswani Omprakash.

Dr. Greywoode and Ms. Issokson have no relevant financial disclosures. Ms. Aswani Omprakash has consulted for Genentech, AbbVie, Janssen, and Arena.

The AGA applauds researchers who are working to raise our awareness of health disparities in digestive diseases. The AGA is committed to addressing this important societal issue head on. Learn more about AGA’s commitment through the AGA Equity Project.

Updated Feb. 17, 2021.
 

Although inflammatory bowel disease (IBD) affects primarily White patients, about one-quarter of cases are found in non-White racial and ethnic groups. Various factors have combined to lead to disparities in treatment and outcomes for non-Whites with IBD.

Ethnic and racial disparities, along with socioeconomic factors, were the subject of a presentation by Ruby Greywoode, MD, at the annual congress of the Crohn’s & Colitis Foundation and the American Gastroenterological Association.

“Historical and present-day realities of racial inequity and factors that contribute to socioeconomic status [include] educational and housing policies, employment practices, and generational wealth. Addressing health disparities requires acknowledging these systemic factors,” said Dr. Greywoode, who is with Montefiore Medical Center in New York.

An important concept in discussing health disparity is social determinants of health, which refers to nonbiological factors that affect health and health outcomes. These are “the conditions in which people live, work, learn, and play that affect their health and their quality of life,” said Dr. Greywoode. 

Dr. Greywoode shared examples of social determinants that affect economic stability and financial worry. One study found that one in six IBD patients reported not taking their medications because of cost considerations. A survey of about 900 adults showed that 1 in 4 delayed medical care – half of those because of cost; patients who delayed care were 2.5 times more likely to report an IBD flare in the previous year.

Another important issue is food insecurity. Other presenters at the session emphasized the importance of high-quality nutrition in IBD, and Dr. Greywoode presented one survey showing that only 9% of patients who had both food security and social support reported cost-related medication nonadherence. Among those that had either food insecurity or poor social support, 12% reported cost-related medication nonadherence, but the proportion jumped to 57% among patients who had both food insecurity and lack of social support.

Session comoderator Kelly Issokson noted that socioeconomic factors often interfere with adoption of healthy diets. Whole foods and plant-based foods are expensive, and the financial pressures of the COVID-19 epidemic have made that worse. “Millions of people are slipping into poverty and food insecurity. This is one of the things she highlighted as factors in medication nonadherence,” said Ms. Issokson, who is the clinical nutritional coordinator at the digestive disease clinic at Cedars-Sinai Medical Center in Los Angeles.

Dr. Greywoode also described studies that looked at race, socioeconomic status, and IBD outcomes. A review from 2013 showed disparities among Whites, African Americans, and Hispanics with respect to undergoing ulcerative colitis–related colectomy and Crohn’s disease–related bowel resection. Ulcerative colitis patients on Medicaid had 230% greater in-patient mortality, compared with patients with private insurance, even after adjustment for multiple confounders.

But inequities are not static. “Since this publication, we have numerous other studies drawing conclusions that sometimes agree with and sometimes conflict with it. My belief is that health disparities in IBD will continue to be an active area of research. We know that it takes vigilance to identify, track, and address any disparities when they do arise,” said Dr. Greywoode.

Dr. Greywoode also noted that phenotypic differences based on race and ethnicity influence disparities. She showed results from a meta-analysis that found a difference in the frequency of perianal Crohn’s disease by race and ethnicity; the highest frequency occurred in Black patients (31%), followed by Asians (22%), Whites (14%), and Hispanics (13%). Another study showed that African American patients with Crohn’s disease were more likely to develop a new abscess (adjusted odds ratio, 2.27; 95% confidence interval, 1.31-3.93) or anal fissure (aOR, 1.76; 95% CI, 1.01-3.07), and were also more likely to be initiated on an anti–tumor necrosis factor drug (aOR, 1.85; 95% CI, 1.09-3.14).

Those differences underscore the need to recognize that IBD is not just a disease for White patients. “As we move forward in IBD research, we recognize that individuals of European ancestry are not the only ones who have IBD. There is a growing diverse racial and ethnic population with IBD,” said Dr. Greywoode.

She noted that, in the United States, it is estimated that about one in four adult patients are non-Hispanic African American, Hispanic, Asians, or other ethnicities. Nevertheless, Whites are overrepresented among participants in IBD clinical trials. Some trials are composed of as much as 95% White patients, and sometimes race isn’t even listed. “It’s unclear if [race/ethnicity data are] not collected or not deemed important, but we know that what is not collected is not measured, and what is not measured can’t be evaluated, either to praise or constructively criticize,” said Dr. Greywoode.

Fortunately, there are efforts in place to improve representation in clinical trials. There has been a mandate for almost 3 decades that federally funded research must include racial and ethnic minorities who have been traditionally underrepresented. The Food and Drug Administration has also provided guidance to industry to improve diversity in clinical trial participation, and industry groups have developed strategies, including improved representation among investigators and related early-career development programs. At the community and independent health care practice levels, clinical trial networks encourage patient participation with regulatory and data management support to bolster practices with insufficient resources.

Underrepresentation in clinical trials resonated with comoderator Tina Aswani Omprakash, who is a patient advocate and a master’s in public health student at the Icahn School of Medicine at Mount Sinai, New York. She called for greater awareness among physicians that IBD can occur among people of all backgrounds. “[Providers] would look at me and [say]: ‘There’s no way that, as a South Asian woman, you have that kind of disease.’ There’s that lack of believability,” said Ms. Aswani Omprakash.

Greater recognition of the diversity of patients, as well as the phenotypic differences found among ethnicities, could also inform clinical trial participation and, ultimately, more personalized medicine. “We have to look at these things, observe how they’re affecting populations differently, so that we can have proper medication solutions,” said Ms. Aswani Omprakash.

Dr. Greywoode and Ms. Issokson have no relevant financial disclosures. Ms. Aswani Omprakash has consulted for Genentech, AbbVie, Janssen, and Arena.

The AGA applauds researchers who are working to raise our awareness of health disparities in digestive diseases. The AGA is committed to addressing this important societal issue head on. Learn more about AGA’s commitment through the AGA Equity Project.

Updated Feb. 17, 2021.
 

Since the first baby was born after a uterus transplantation in Sweden in 2014, uterus transplantation has been rapidly transitioning toward clinical reality.¹ Several teams in the United States and multiple teams worldwide have performed the procedure, with the total number of worldwide surgeries performed nearing 100.

Uterus transplantation is the first and only true treatment for women with absolute uterine factor infertility – estimated to affect 1 in 500 women – and is filling an unmet need for this population of women. Women who have sought participation in uterus transplantation research have had complex and meaningful reasons and motivations for doing so.² Combined with an accumulation of successful pregnancies, this makes continued research and technical improvement a worthy endeavor.

Most of the births thus far have occurred through the living-donor model; the initial Swedish trial involved nine women, seven of whom completed the procedure with viable transplants from living donors, and gave birth to eight healthy children. (Two required hysterectomy prior to attempted embryo transfer.³)

The Cleveland Clinic opted to build its first – and still ongoing – trial focusing on deceased-donor uterus transplants on the premise that such an approach obviates any risk to the donor and presents the fewest ethical challenges at the current time. Of eight uterus transplants performed thus far at the Cleveland Clinic, there have been three live births and two graft failures. As of early 2021, there was one ongoing pregnancy and two patients in preparation for embryo transfer.

Thus far, neither the living- nor deceased-donor model of uterus transplantation has been demonstrated to be superior. However, as data accrues from deceased donor studies, we will be able to more directly compare outcomes.

In the meantime, alongside a rapid ascent of clinical landmarks – the first live birth in the United States from living-donor uterus transplantation in 2017 at Baylor University Medical Center in Houston,⁴ for instance, and the first live birth in the United States from deceased-donor uterus transplantation in 2019 at the Cleveland Clinic ^– there have been significant improvements in surgical retrieval of the uterus and in the optimization of graft performance.⁵

Most notably, the utero-ovarian vein has been used successfully in living donors to achieve venous drainage of the graft. This has lessened the risks of deep pelvic dissection in the living donor and made the transition to laparoscopic and robotic approaches in the living donor much easier.
 

Donor procurement, venous drainage

Adequate circulatory inflow and outflow for the transplanted uterus are essential both for the prevention of ischemia and thrombosis, which have been major causes of graft failure, and for meeting the increased demands of blood flow during pregnancy. Of the two, the outflow is the more challenging component.

Courtesy Cleveland Clinic

Venous drainage traditionally has been accomplished through the use of the uterine veins, which drain into the internal iliac veins; often the vascular graft will include a portion of the internal iliac vessel which can be connected via anastomoses to the external iliac vein classically in deceased donors. Typically, the gynecologic surgeon on the team performs the vaginal anastomosis and suspension of the uterus, while the transplant surgeons perform the venous and arterial anastomoses.

In the living-donor model, procurement and dissection of these often unpredictable and tortuous complexes in the deep pelvis – particularly the branching uterine veins that lie in close proximity to the ureter, bladder, other blood vessels, and rectum – can be risky. The anatomic variants in the uterine vein are numerous, and even in one patient, a comprehensive dissection on one side cannot be expected to be mirrored on the contralateral side.

Courtesy Cleveland Clinic

In addition to the risk of injury to the donor, the anastomosis may be unsuccessful as the veins are thinly walled and challenging to suture. As such, multiple modifications have been developed, often adapted to the donor’s anatomy and the caliber and accessibility of vessels. Preoperative vascular imaging with CT and/or MRI may help to identify suitable candidates and also may facilitate presurgical planning of which vessels may be selected for use.

Recently, surgeons performing living-donor transplantations have successfully used the more accessible and less risky ovarian and/or utero-ovarian veins for venous anastomosis. In 2019, for instance, a team in Pune, India, reported laparoscopically dissecting the donor ovarian veins and a portion of the internal iliac artery, and completing anastomosis with bilateral donor internal iliac arteries to recipient internal iliac arteries, and bilateral donor ovarian veins to recipient external iliac veins.⁶ It is significant that these smaller-caliber vessels were found to able to support the uterus through pregnancy.

Courtesy Cleveland Clinic

We must be cautious, however, to avoid removing donors’ ovaries. Oophorectomy for women in their 40s can result in significant long-term medical sequelae. Surgeons at Baylor have achieved at least one live birth after harvesting the donor’s utero-ovarian veins while conserving the ovaries – a significant advancement for the living-donor model.⁴

There is tremendous interest in developing minimally invasive approaches to further reduce living-donor risk. The Swedish team has completed a series of eight robotic hysterectomies in living-donor uterus transplantations as part of a second trial. Addressing the reality of a learning curve, their study was designed around a step-wise approach, mastering initial steps first – e.g., dissections of the uterovaginal fossa, arteries, and ureters – and ultimately converting to laparotomy.⁷ In the United States, Baylor University has now completed at least five completely robotic living-donor hysterectomies with complete vaginal extraction.

Published data on robotic surgery suggests that surgical access and perioperative visualization of the vessels may be improved. And as minimally invasive approaches are adopted and improved, the length of donor surgery – 10-13 hours of operating room time in the original Swedish series – should diminish, as should the morbidity associated with laparotomy.

Courtesy Cleveland Clinic

Surgical acquisition of a uterine graft from a deceased donor diminishes concerns for injury to nearby structures. Therefore, although it is a technically similar procedure, a deceased-donor model allows more flexibility with the length, caliber, and number of vessels that can be used for anastomosis. The internal iliac vessels and even portions of the external iliac vessels and ovarian vessels can be used to allow maximum flexibility.⁸

Surgical technique for uterus recipients

For the recipient surgery, entry is achieved via a midline, vertical laparotomy. The external iliac vessels are exposed, and the sites of vascular anastomoses are identified. The peritoneal reflection of the bladder is identified and dissected away to expose the anterior vagina, and the vagina is opened to a diameter that matches the donor, typically using a monopolar electrosurgical cutting instrument.

Courtesy Cleveland Clinic

The vault of the donor vagina will be attached to the recipient’s existing vagina or vaginal pouch. It is important to identify recipient vaginal mucosa and incorporate it into the vaginal anastomosis to reduce the risk of vaginal stricture. We recommend that the vaginal mucosa be tagged with PDS II sutures or grasped with allis clamps to prevent retraction.

Surgical teams have taken multiple approaches to vaginal anastomosis. The Cleveland Clinic has used both a running suture as well as a horizontal mattress stitch for closure. For the latter, a 30-inch double-armed 2.0 Vicryl allows for complete suturing of the recipient vagina – with eight stitches placed circumferentially – before the uterus is placed. Both ends of the suture are passed intra-abdominal to intravaginal in the recipient.⁹

Once the donor uterus is suspended, attention focuses on vascular anastomosis, with bilateral end-to-side anastomosis between the donor anterior division of the internal iliac arteries and the external iliac vessels of the recipient, and with venous drainage commonly achieved through the uterine veins draining into the internal or external iliac vein of the recipient. As mentioned, recent cases involving living donors have also demonstrated success with the use of ovarian and/or utero-ovarian veins. Care should be taken to avoid having tension or twisting across the anastomosis.

Courtesy Cleveland Clinic

After adequate graft perfusion is confirmed, with the uterus turning from a dusky color to a pink and well-perfused organ, the vaginal anastomosis is completed, with the arms of the double-armed suture passed through the donor vagina, from intravaginal to intra-abdominal. Tension should be evenly spread along the recipient and donor vagina in order to reduce the formation of granulation tissue and the severity of future vaginal stricturing.

For uterine fixation, polypropylene sutures are placed between the graft uterosacral ligaments and recipient uterine rudiments, and between the graft round ligaments and the recipient pelvic side wall at the level of the deep inguinal ring.

Current uterus transplantation protocols require removal of the uterus after one or two live births are achieved, so that recipients will not be exposed to long-term immunosuppression.
 

Complications and controversies

Postoperative vaginal strictures can make embryo transfer difficult and are a common complication in both living- and deceased-donor models. The Cleveland Clinic team has applied techniques from vaginal reconstructive surgery to try to reduce the occurrence of postoperative strictures – mainly increasing attention paid to anastomosis tissue–site preparation and closure of the anastomosis using a tension-free interrupted suture technique, as described above.⁹ The jury is out on whether such changes are sufficient, and a more complete understanding of the causes of vaginal stricture is needed.

Other perioperative complications include infection and graft thrombosis, both of which typically result in urgent graft hysterectomy. During pregnancy, one of our patients experienced abnormal placentation, though this was not thought to be related to uterus transplantation.⁵

The U.S. Uterus Transplant Consortium (USUTC) is a group of active programs that are sharing ideas and outcomes and advocating for continued research in this rapidly developing field. Uterine transplants require collaboration with transplant surgery, transplant medicine, infectious disease, gynecologic surgery, high-risk obstetrics, and other specialties. While significant progress has been made in a short period of time, uterine transplantation is still in its early stages, and transplants should be done in institutions that have the capacity for mentorship, bioethical oversight, and long-term follow-up of donors, recipients, and offspring.

The USUTC has recently proposed guidelines for nomenclature related to operative technique, vascular anatomy, and uterine transplantation outcomes.¹⁰ It proposes standardizing the names for the four veins originating from the uterus (to eliminate current inconsistency), which will be important as optimal strategies for vascular anastomoses are discussed and determined.

In addition, the consortium is creating a registry for the rigorous collection of data on procedures and outcomes (from menstruation and pregnancy through delivery, graft removal, and long-term follow-up). A registry has also been proposed by the International Society for Uterine Transplantation.

A major question remains in our field: Is the living-donor or deceased-donor uterus transplant the best approach? Knowledge of the quality of the uterus is greater preoperatively within a living-donor model, but no matter how minimally invasive the technique, the donor still assumes some risk of prolonged surgery and extensive pelvic dissection for a transplant that is not lifesaving.

On the other hand, deceased-donor transplants require additional layers of organization and coordination, and the availability of suitable deceased-donor uteri will likely not be sufficient to meet the current demand. Many of us in the field believe that the future of uterine transplantation will involve some combination of living- and deceased-donor transplants – similar to other solid organ transplant programs.

Dr. Flyckt and Dr. Richards reported that they have no relevant financial disclosures.

Correction, 2/2/21: An earlier version of this article misstated Dr. Richards' name in the photo caption.
 

References

1. Lancet. 2015;14:385:607-16.

2. AJOB Empir Bioeth. 2019;10(1):23-5.

3. Transplantation. 2020;104(7):1312-5.

4. Am J Transplant. 2018;18(5):1270-4.

5. Am J Obstet Gynecol. 2020;223(2):143-51.

6. J Minimally Invasive Gynecol. 2019;26:628-35.

7. Acta Obstet Gynecol Scand. 2020;99(9):1222-9.

8. Fertil Steril. 2018;110(1):183.

9. Fertil Steril. 2020 Jul 16. doi: 10.1016/j.fertnstert.2020.05.017

10 Am J Transplant. 2020;20(12):3319-25.

Since the first baby was born after a uterus transplantation in Sweden in 2014, uterus transplantation has been rapidly transitioning toward clinical reality.¹ Several teams in the United States and multiple teams worldwide have performed the procedure, with the total number of worldwide surgeries performed nearing 100.

Uterus transplantation is the first and only true treatment for women with absolute uterine factor infertility – estimated to affect 1 in 500 women – and is filling an unmet need for this population of women. Women who have sought participation in uterus transplantation research have had complex and meaningful reasons and motivations for doing so.² Combined with an accumulation of successful pregnancies, this makes continued research and technical improvement a worthy endeavor.

Most of the births thus far have occurred through the living-donor model; the initial Swedish trial involved nine women, seven of whom completed the procedure with viable transplants from living donors, and gave birth to eight healthy children. (Two required hysterectomy prior to attempted embryo transfer.³)

The Cleveland Clinic opted to build its first – and still ongoing – trial focusing on deceased-donor uterus transplants on the premise that such an approach obviates any risk to the donor and presents the fewest ethical challenges at the current time. Of eight uterus transplants performed thus far at the Cleveland Clinic, there have been three live births and two graft failures. As of early 2021, there was one ongoing pregnancy and two patients in preparation for embryo transfer.

Thus far, neither the living- nor deceased-donor model of uterus transplantation has been demonstrated to be superior. However, as data accrues from deceased donor studies, we will be able to more directly compare outcomes.

In the meantime, alongside a rapid ascent of clinical landmarks – the first live birth in the United States from living-donor uterus transplantation in 2017 at Baylor University Medical Center in Houston,⁴ for instance, and the first live birth in the United States from deceased-donor uterus transplantation in 2019 at the Cleveland Clinic ^– there have been significant improvements in surgical retrieval of the uterus and in the optimization of graft performance.⁵

Most notably, the utero-ovarian vein has been used successfully in living donors to achieve venous drainage of the graft. This has lessened the risks of deep pelvic dissection in the living donor and made the transition to laparoscopic and robotic approaches in the living donor much easier.
 

Donor procurement, venous drainage

Adequate circulatory inflow and outflow for the transplanted uterus are essential both for the prevention of ischemia and thrombosis, which have been major causes of graft failure, and for meeting the increased demands of blood flow during pregnancy. Of the two, the outflow is the more challenging component.

Courtesy Cleveland Clinic

Venous drainage traditionally has been accomplished through the use of the uterine veins, which drain into the internal iliac veins; often the vascular graft will include a portion of the internal iliac vessel which can be connected via anastomoses to the external iliac vein classically in deceased donors. Typically, the gynecologic surgeon on the team performs the vaginal anastomosis and suspension of the uterus, while the transplant surgeons perform the venous and arterial anastomoses.

In the living-donor model, procurement and dissection of these often unpredictable and tortuous complexes in the deep pelvis – particularly the branching uterine veins that lie in close proximity to the ureter, bladder, other blood vessels, and rectum – can be risky. The anatomic variants in the uterine vein are numerous, and even in one patient, a comprehensive dissection on one side cannot be expected to be mirrored on the contralateral side.

Courtesy Cleveland Clinic

In addition to the risk of injury to the donor, the anastomosis may be unsuccessful as the veins are thinly walled and challenging to suture. As such, multiple modifications have been developed, often adapted to the donor’s anatomy and the caliber and accessibility of vessels. Preoperative vascular imaging with CT and/or MRI may help to identify suitable candidates and also may facilitate presurgical planning of which vessels may be selected for use.

Recently, surgeons performing living-donor transplantations have successfully used the more accessible and less risky ovarian and/or utero-ovarian veins for venous anastomosis. In 2019, for instance, a team in Pune, India, reported laparoscopically dissecting the donor ovarian veins and a portion of the internal iliac artery, and completing anastomosis with bilateral donor internal iliac arteries to recipient internal iliac arteries, and bilateral donor ovarian veins to recipient external iliac veins.⁶ It is significant that these smaller-caliber vessels were found to able to support the uterus through pregnancy.

Courtesy Cleveland Clinic

We must be cautious, however, to avoid removing donors’ ovaries. Oophorectomy for women in their 40s can result in significant long-term medical sequelae. Surgeons at Baylor have achieved at least one live birth after harvesting the donor’s utero-ovarian veins while conserving the ovaries – a significant advancement for the living-donor model.⁴

There is tremendous interest in developing minimally invasive approaches to further reduce living-donor risk. The Swedish team has completed a series of eight robotic hysterectomies in living-donor uterus transplantations as part of a second trial. Addressing the reality of a learning curve, their study was designed around a step-wise approach, mastering initial steps first – e.g., dissections of the uterovaginal fossa, arteries, and ureters – and ultimately converting to laparotomy.⁷ In the United States, Baylor University has now completed at least five completely robotic living-donor hysterectomies with complete vaginal extraction.

Published data on robotic surgery suggests that surgical access and perioperative visualization of the vessels may be improved. And as minimally invasive approaches are adopted and improved, the length of donor surgery – 10-13 hours of operating room time in the original Swedish series – should diminish, as should the morbidity associated with laparotomy.

Courtesy Cleveland Clinic

Surgical acquisition of a uterine graft from a deceased donor diminishes concerns for injury to nearby structures. Therefore, although it is a technically similar procedure, a deceased-donor model allows more flexibility with the length, caliber, and number of vessels that can be used for anastomosis. The internal iliac vessels and even portions of the external iliac vessels and ovarian vessels can be used to allow maximum flexibility.⁸

Surgical technique for uterus recipients

For the recipient surgery, entry is achieved via a midline, vertical laparotomy. The external iliac vessels are exposed, and the sites of vascular anastomoses are identified. The peritoneal reflection of the bladder is identified and dissected away to expose the anterior vagina, and the vagina is opened to a diameter that matches the donor, typically using a monopolar electrosurgical cutting instrument.

Courtesy Cleveland Clinic

The vault of the donor vagina will be attached to the recipient’s existing vagina or vaginal pouch. It is important to identify recipient vaginal mucosa and incorporate it into the vaginal anastomosis to reduce the risk of vaginal stricture. We recommend that the vaginal mucosa be tagged with PDS II sutures or grasped with allis clamps to prevent retraction.

Surgical teams have taken multiple approaches to vaginal anastomosis. The Cleveland Clinic has used both a running suture as well as a horizontal mattress stitch for closure. For the latter, a 30-inch double-armed 2.0 Vicryl allows for complete suturing of the recipient vagina – with eight stitches placed circumferentially – before the uterus is placed. Both ends of the suture are passed intra-abdominal to intravaginal in the recipient.⁹

Once the donor uterus is suspended, attention focuses on vascular anastomosis, with bilateral end-to-side anastomosis between the donor anterior division of the internal iliac arteries and the external iliac vessels of the recipient, and with venous drainage commonly achieved through the uterine veins draining into the internal or external iliac vein of the recipient. As mentioned, recent cases involving living donors have also demonstrated success with the use of ovarian and/or utero-ovarian veins. Care should be taken to avoid having tension or twisting across the anastomosis.

Courtesy Cleveland Clinic

After adequate graft perfusion is confirmed, with the uterus turning from a dusky color to a pink and well-perfused organ, the vaginal anastomosis is completed, with the arms of the double-armed suture passed through the donor vagina, from intravaginal to intra-abdominal. Tension should be evenly spread along the recipient and donor vagina in order to reduce the formation of granulation tissue and the severity of future vaginal stricturing.

For uterine fixation, polypropylene sutures are placed between the graft uterosacral ligaments and recipient uterine rudiments, and between the graft round ligaments and the recipient pelvic side wall at the level of the deep inguinal ring.

Current uterus transplantation protocols require removal of the uterus after one or two live births are achieved, so that recipients will not be exposed to long-term immunosuppression.
 

Complications and controversies

Postoperative vaginal strictures can make embryo transfer difficult and are a common complication in both living- and deceased-donor models. The Cleveland Clinic team has applied techniques from vaginal reconstructive surgery to try to reduce the occurrence of postoperative strictures – mainly increasing attention paid to anastomosis tissue–site preparation and closure of the anastomosis using a tension-free interrupted suture technique, as described above.⁹ The jury is out on whether such changes are sufficient, and a more complete understanding of the causes of vaginal stricture is needed.

Other perioperative complications include infection and graft thrombosis, both of which typically result in urgent graft hysterectomy. During pregnancy, one of our patients experienced abnormal placentation, though this was not thought to be related to uterus transplantation.⁵

The U.S. Uterus Transplant Consortium (USUTC) is a group of active programs that are sharing ideas and outcomes and advocating for continued research in this rapidly developing field. Uterine transplants require collaboration with transplant surgery, transplant medicine, infectious disease, gynecologic surgery, high-risk obstetrics, and other specialties. While significant progress has been made in a short period of time, uterine transplantation is still in its early stages, and transplants should be done in institutions that have the capacity for mentorship, bioethical oversight, and long-term follow-up of donors, recipients, and offspring.

The USUTC has recently proposed guidelines for nomenclature related to operative technique, vascular anatomy, and uterine transplantation outcomes.¹⁰ It proposes standardizing the names for the four veins originating from the uterus (to eliminate current inconsistency), which will be important as optimal strategies for vascular anastomoses are discussed and determined.

In addition, the consortium is creating a registry for the rigorous collection of data on procedures and outcomes (from menstruation and pregnancy through delivery, graft removal, and long-term follow-up). A registry has also been proposed by the International Society for Uterine Transplantation.

A major question remains in our field: Is the living-donor or deceased-donor uterus transplant the best approach? Knowledge of the quality of the uterus is greater preoperatively within a living-donor model, but no matter how minimally invasive the technique, the donor still assumes some risk of prolonged surgery and extensive pelvic dissection for a transplant that is not lifesaving.

On the other hand, deceased-donor transplants require additional layers of organization and coordination, and the availability of suitable deceased-donor uteri will likely not be sufficient to meet the current demand. Many of us in the field believe that the future of uterine transplantation will involve some combination of living- and deceased-donor transplants – similar to other solid organ transplant programs.

Dr. Flyckt and Dr. Richards reported that they have no relevant financial disclosures.

Correction, 2/2/21: An earlier version of this article misstated Dr. Richards' name in the photo caption.
 

References

1. Lancet. 2015;14:385:607-16.

2. AJOB Empir Bioeth. 2019;10(1):23-5.

3. Transplantation. 2020;104(7):1312-5.

4. Am J Transplant. 2018;18(5):1270-4.

5. Am J Obstet Gynecol. 2020;223(2):143-51.

6. J Minimally Invasive Gynecol. 2019;26:628-35.

7. Acta Obstet Gynecol Scand. 2020;99(9):1222-9.

8. Fertil Steril. 2018;110(1):183.

9. Fertil Steril. 2020 Jul 16. doi: 10.1016/j.fertnstert.2020.05.017

10 Am J Transplant. 2020;20(12):3319-25.

Since the first baby was born after a uterus transplantation in Sweden in 2014, uterus transplantation has been rapidly transitioning toward clinical reality.¹ Several teams in the United States and multiple teams worldwide have performed the procedure, with the total number of worldwide surgeries performed nearing 100.

Uterus transplantation is the first and only true treatment for women with absolute uterine factor infertility – estimated to affect 1 in 500 women – and is filling an unmet need for this population of women. Women who have sought participation in uterus transplantation research have had complex and meaningful reasons and motivations for doing so.² Combined with an accumulation of successful pregnancies, this makes continued research and technical improvement a worthy endeavor.

Most of the births thus far have occurred through the living-donor model; the initial Swedish trial involved nine women, seven of whom completed the procedure with viable transplants from living donors, and gave birth to eight healthy children. (Two required hysterectomy prior to attempted embryo transfer.³)

The Cleveland Clinic opted to build its first – and still ongoing – trial focusing on deceased-donor uterus transplants on the premise that such an approach obviates any risk to the donor and presents the fewest ethical challenges at the current time. Of eight uterus transplants performed thus far at the Cleveland Clinic, there have been three live births and two graft failures. As of early 2021, there was one ongoing pregnancy and two patients in preparation for embryo transfer.

Thus far, neither the living- nor deceased-donor model of uterus transplantation has been demonstrated to be superior. However, as data accrues from deceased donor studies, we will be able to more directly compare outcomes.

In the meantime, alongside a rapid ascent of clinical landmarks – the first live birth in the United States from living-donor uterus transplantation in 2017 at Baylor University Medical Center in Houston,⁴ for instance, and the first live birth in the United States from deceased-donor uterus transplantation in 2019 at the Cleveland Clinic ^– there have been significant improvements in surgical retrieval of the uterus and in the optimization of graft performance.⁵

Most notably, the utero-ovarian vein has been used successfully in living donors to achieve venous drainage of the graft. This has lessened the risks of deep pelvic dissection in the living donor and made the transition to laparoscopic and robotic approaches in the living donor much easier.
 

Donor procurement, venous drainage

Adequate circulatory inflow and outflow for the transplanted uterus are essential both for the prevention of ischemia and thrombosis, which have been major causes of graft failure, and for meeting the increased demands of blood flow during pregnancy. Of the two, the outflow is the more challenging component.

Courtesy Cleveland Clinic

Venous drainage traditionally has been accomplished through the use of the uterine veins, which drain into the internal iliac veins; often the vascular graft will include a portion of the internal iliac vessel which can be connected via anastomoses to the external iliac vein classically in deceased donors. Typically, the gynecologic surgeon on the team performs the vaginal anastomosis and suspension of the uterus, while the transplant surgeons perform the venous and arterial anastomoses.

In the living-donor model, procurement and dissection of these often unpredictable and tortuous complexes in the deep pelvis – particularly the branching uterine veins that lie in close proximity to the ureter, bladder, other blood vessels, and rectum – can be risky. The anatomic variants in the uterine vein are numerous, and even in one patient, a comprehensive dissection on one side cannot be expected to be mirrored on the contralateral side.

Courtesy Cleveland Clinic

In addition to the risk of injury to the donor, the anastomosis may be unsuccessful as the veins are thinly walled and challenging to suture. As such, multiple modifications have been developed, often adapted to the donor’s anatomy and the caliber and accessibility of vessels. Preoperative vascular imaging with CT and/or MRI may help to identify suitable candidates and also may facilitate presurgical planning of which vessels may be selected for use.

Recently, surgeons performing living-donor transplantations have successfully used the more accessible and less risky ovarian and/or utero-ovarian veins for venous anastomosis. In 2019, for instance, a team in Pune, India, reported laparoscopically dissecting the donor ovarian veins and a portion of the internal iliac artery, and completing anastomosis with bilateral donor internal iliac arteries to recipient internal iliac arteries, and bilateral donor ovarian veins to recipient external iliac veins.⁶ It is significant that these smaller-caliber vessels were found to able to support the uterus through pregnancy.

Courtesy Cleveland Clinic

We must be cautious, however, to avoid removing donors’ ovaries. Oophorectomy for women in their 40s can result in significant long-term medical sequelae. Surgeons at Baylor have achieved at least one live birth after harvesting the donor’s utero-ovarian veins while conserving the ovaries – a significant advancement for the living-donor model.⁴

There is tremendous interest in developing minimally invasive approaches to further reduce living-donor risk. The Swedish team has completed a series of eight robotic hysterectomies in living-donor uterus transplantations as part of a second trial. Addressing the reality of a learning curve, their study was designed around a step-wise approach, mastering initial steps first – e.g., dissections of the uterovaginal fossa, arteries, and ureters – and ultimately converting to laparotomy.⁷ In the United States, Baylor University has now completed at least five completely robotic living-donor hysterectomies with complete vaginal extraction.

Published data on robotic surgery suggests that surgical access and perioperative visualization of the vessels may be improved. And as minimally invasive approaches are adopted and improved, the length of donor surgery – 10-13 hours of operating room time in the original Swedish series – should diminish, as should the morbidity associated with laparotomy.

Courtesy Cleveland Clinic

Surgical acquisition of a uterine graft from a deceased donor diminishes concerns for injury to nearby structures. Therefore, although it is a technically similar procedure, a deceased-donor model allows more flexibility with the length, caliber, and number of vessels that can be used for anastomosis. The internal iliac vessels and even portions of the external iliac vessels and ovarian vessels can be used to allow maximum flexibility.⁸

Surgical technique for uterus recipients

For the recipient surgery, entry is achieved via a midline, vertical laparotomy. The external iliac vessels are exposed, and the sites of vascular anastomoses are identified. The peritoneal reflection of the bladder is identified and dissected away to expose the anterior vagina, and the vagina is opened to a diameter that matches the donor, typically using a monopolar electrosurgical cutting instrument.

Courtesy Cleveland Clinic

The vault of the donor vagina will be attached to the recipient’s existing vagina or vaginal pouch. It is important to identify recipient vaginal mucosa and incorporate it into the vaginal anastomosis to reduce the risk of vaginal stricture. We recommend that the vaginal mucosa be tagged with PDS II sutures or grasped with allis clamps to prevent retraction.

Surgical teams have taken multiple approaches to vaginal anastomosis. The Cleveland Clinic has used both a running suture as well as a horizontal mattress stitch for closure. For the latter, a 30-inch double-armed 2.0 Vicryl allows for complete suturing of the recipient vagina – with eight stitches placed circumferentially – before the uterus is placed. Both ends of the suture are passed intra-abdominal to intravaginal in the recipient.⁹

Once the donor uterus is suspended, attention focuses on vascular anastomosis, with bilateral end-to-side anastomosis between the donor anterior division of the internal iliac arteries and the external iliac vessels of the recipient, and with venous drainage commonly achieved through the uterine veins draining into the internal or external iliac vein of the recipient. As mentioned, recent cases involving living donors have also demonstrated success with the use of ovarian and/or utero-ovarian veins. Care should be taken to avoid having tension or twisting across the anastomosis.

Courtesy Cleveland Clinic

After adequate graft perfusion is confirmed, with the uterus turning from a dusky color to a pink and well-perfused organ, the vaginal anastomosis is completed, with the arms of the double-armed suture passed through the donor vagina, from intravaginal to intra-abdominal. Tension should be evenly spread along the recipient and donor vagina in order to reduce the formation of granulation tissue and the severity of future vaginal stricturing.

For uterine fixation, polypropylene sutures are placed between the graft uterosacral ligaments and recipient uterine rudiments, and between the graft round ligaments and the recipient pelvic side wall at the level of the deep inguinal ring.

Current uterus transplantation protocols require removal of the uterus after one or two live births are achieved, so that recipients will not be exposed to long-term immunosuppression.
 

Complications and controversies

Postoperative vaginal strictures can make embryo transfer difficult and are a common complication in both living- and deceased-donor models. The Cleveland Clinic team has applied techniques from vaginal reconstructive surgery to try to reduce the occurrence of postoperative strictures – mainly increasing attention paid to anastomosis tissue–site preparation and closure of the anastomosis using a tension-free interrupted suture technique, as described above.⁹ The jury is out on whether such changes are sufficient, and a more complete understanding of the causes of vaginal stricture is needed.

Other perioperative complications include infection and graft thrombosis, both of which typically result in urgent graft hysterectomy. During pregnancy, one of our patients experienced abnormal placentation, though this was not thought to be related to uterus transplantation.⁵

The U.S. Uterus Transplant Consortium (USUTC) is a group of active programs that are sharing ideas and outcomes and advocating for continued research in this rapidly developing field. Uterine transplants require collaboration with transplant surgery, transplant medicine, infectious disease, gynecologic surgery, high-risk obstetrics, and other specialties. While significant progress has been made in a short period of time, uterine transplantation is still in its early stages, and transplants should be done in institutions that have the capacity for mentorship, bioethical oversight, and long-term follow-up of donors, recipients, and offspring.

The USUTC has recently proposed guidelines for nomenclature related to operative technique, vascular anatomy, and uterine transplantation outcomes.¹⁰ It proposes standardizing the names for the four veins originating from the uterus (to eliminate current inconsistency), which will be important as optimal strategies for vascular anastomoses are discussed and determined.

In addition, the consortium is creating a registry for the rigorous collection of data on procedures and outcomes (from menstruation and pregnancy through delivery, graft removal, and long-term follow-up). A registry has also been proposed by the International Society for Uterine Transplantation.

A major question remains in our field: Is the living-donor or deceased-donor uterus transplant the best approach? Knowledge of the quality of the uterus is greater preoperatively within a living-donor model, but no matter how minimally invasive the technique, the donor still assumes some risk of prolonged surgery and extensive pelvic dissection for a transplant that is not lifesaving.

On the other hand, deceased-donor transplants require additional layers of organization and coordination, and the availability of suitable deceased-donor uteri will likely not be sufficient to meet the current demand. Many of us in the field believe that the future of uterine transplantation will involve some combination of living- and deceased-donor transplants – similar to other solid organ transplant programs.

Dr. Flyckt and Dr. Richards reported that they have no relevant financial disclosures.

Correction, 2/2/21: An earlier version of this article misstated Dr. Richards' name in the photo caption.
 

References

1. Lancet. 2015;14:385:607-16.

2. AJOB Empir Bioeth. 2019;10(1):23-5.

3. Transplantation. 2020;104(7):1312-5.

4. Am J Transplant. 2018;18(5):1270-4.

5. Am J Obstet Gynecol. 2020;223(2):143-51.

6. J Minimally Invasive Gynecol. 2019;26:628-35.

7. Acta Obstet Gynecol Scand. 2020;99(9):1222-9.

8. Fertil Steril. 2018;110(1):183.

9. Fertil Steril. 2020 Jul 16. doi: 10.1016/j.fertnstert.2020.05.017

10 Am J Transplant. 2020;20(12):3319-25.

Until the advent of uterus transplantation, there was no restorative procedure available to a woman presenting with an absent uterus or nonfunctioning uterus; that is, absolute uterine factor infertility (AUFI). It is estimated that 1 in 500 women of childbearing age are affected by AUFI.^1,2 An absent uterus may be secondary to uterine agenesis or Mayer-Rokitansky-Küster-Hauser syndrome (MRKH), which occurs in 1 in 4,500 women.^3,4 (Because women with MRKH have a normal karyotype, their children can be normal, without urogenital malformations.⁵)

Given the fact that roughly 240,000 hysterectomies are performed in the United States each year for women aged under 44 years, hysterectomy is the most common cause of acquired AUFI.⁶AUFI may also be secondary to a uterus that will not support a viable pregnancy; that is, a nonfunctional uterus. In this case, medical or surgical treatment is impossible to enable normal physiological uterine function to produce a successful pregnancy. Causal factors include Müllerian anomalies, severe intrauterine adhesions/Asherman syndrome, uterine fibroids not amendable to surgical therapy, and radiation injury not responsive to medical therapy.

Prior to uterus transplantation, parenthood could only be achieved via adoption, foster parenting, or gestational carrier. While utilizing a gestational carrier is legal in most U.S. states, most countries of western Europe as well as Brazil and Japan, to name a few, do not allow the use of gestational carriers. For some women, moreover, the desire is not only to have a baby, but to carry a child as well.

For this edition of the Master Class in Gynecologic Surgery, I have enlisted the assistance of Rebecca Flyckt, MD, division chief of reproductive endocrinology and infertility at University Hospitals Cleveland Medical Center and associate professor at Case Western Reserve University, Cleveland, and Elliott G. Richards, MD, director of reproductive endocrinology and infertility research at the Cleveland Clinic, to discuss the current and future state of uterus transplantation.

Dr. Flyckt and Dr. Richards have both contributed to the uterus transplantation team at the Cleveland Clinic and are founding members of the U.S. Uterus Transplant Consortium. They are well published in the field of minimally invasive gynecology and reproductive endocrinology and infertility. It is truly a pleasure to welcome them both to this edition of the Master Class in Gynecologic Surgery.

References

1. Fertil Steril. 2014 May;101(5):1228-36.

2. Acta Biomater. 2014 Dec;10(12):5034-42.

3. Hum Reprod Update. Mar-Apr 2001;7(2):161-74.

4. Obstet Gynecol Surv. 2000 Oct;55(10):644-9.

5. Fertil Steril. 1997 Feb;67(2):387-9

6. Am J Public Health. 2003 Feb;93(2):307-12.
 

Dr. Miller is professor of obstetrics & gynecology in the department of clinical sciences, Rosalind Franklin University, North Chicago, and director of minimally invasive gynecologic surgery at Advocate Lutheran General Hospital, Park Ridge, Ill. Dr. Miller reported that he has no disclosures relevant to this Master Class. Email him at [email protected].

Until the advent of uterus transplantation, there was no restorative procedure available to a woman presenting with an absent uterus or nonfunctioning uterus; that is, absolute uterine factor infertility (AUFI). It is estimated that 1 in 500 women of childbearing age are affected by AUFI.^1,2 An absent uterus may be secondary to uterine agenesis or Mayer-Rokitansky-Küster-Hauser syndrome (MRKH), which occurs in 1 in 4,500 women.^3,4 (Because women with MRKH have a normal karyotype, their children can be normal, without urogenital malformations.⁵)

Given the fact that roughly 240,000 hysterectomies are performed in the United States each year for women aged under 44 years, hysterectomy is the most common cause of acquired AUFI.⁶AUFI may also be secondary to a uterus that will not support a viable pregnancy; that is, a nonfunctional uterus. In this case, medical or surgical treatment is impossible to enable normal physiological uterine function to produce a successful pregnancy. Causal factors include Müllerian anomalies, severe intrauterine adhesions/Asherman syndrome, uterine fibroids not amendable to surgical therapy, and radiation injury not responsive to medical therapy.

Prior to uterus transplantation, parenthood could only be achieved via adoption, foster parenting, or gestational carrier. While utilizing a gestational carrier is legal in most U.S. states, most countries of western Europe as well as Brazil and Japan, to name a few, do not allow the use of gestational carriers. For some women, moreover, the desire is not only to have a baby, but to carry a child as well.

For this edition of the Master Class in Gynecologic Surgery, I have enlisted the assistance of Rebecca Flyckt, MD, division chief of reproductive endocrinology and infertility at University Hospitals Cleveland Medical Center and associate professor at Case Western Reserve University, Cleveland, and Elliott G. Richards, MD, director of reproductive endocrinology and infertility research at the Cleveland Clinic, to discuss the current and future state of uterus transplantation.

Dr. Flyckt and Dr. Richards have both contributed to the uterus transplantation team at the Cleveland Clinic and are founding members of the U.S. Uterus Transplant Consortium. They are well published in the field of minimally invasive gynecology and reproductive endocrinology and infertility. It is truly a pleasure to welcome them both to this edition of the Master Class in Gynecologic Surgery.

References

1. Fertil Steril. 2014 May;101(5):1228-36.

2. Acta Biomater. 2014 Dec;10(12):5034-42.

3. Hum Reprod Update. Mar-Apr 2001;7(2):161-74.

4. Obstet Gynecol Surv. 2000 Oct;55(10):644-9.

5. Fertil Steril. 1997 Feb;67(2):387-9

6. Am J Public Health. 2003 Feb;93(2):307-12.
 

Dr. Miller is professor of obstetrics & gynecology in the department of clinical sciences, Rosalind Franklin University, North Chicago, and director of minimally invasive gynecologic surgery at Advocate Lutheran General Hospital, Park Ridge, Ill. Dr. Miller reported that he has no disclosures relevant to this Master Class. Email him at [email protected].

Until the advent of uterus transplantation, there was no restorative procedure available to a woman presenting with an absent uterus or nonfunctioning uterus; that is, absolute uterine factor infertility (AUFI). It is estimated that 1 in 500 women of childbearing age are affected by AUFI.^1,2 An absent uterus may be secondary to uterine agenesis or Mayer-Rokitansky-Küster-Hauser syndrome (MRKH), which occurs in 1 in 4,500 women.^3,4 (Because women with MRKH have a normal karyotype, their children can be normal, without urogenital malformations.⁵)

Given the fact that roughly 240,000 hysterectomies are performed in the United States each year for women aged under 44 years, hysterectomy is the most common cause of acquired AUFI.⁶AUFI may also be secondary to a uterus that will not support a viable pregnancy; that is, a nonfunctional uterus. In this case, medical or surgical treatment is impossible to enable normal physiological uterine function to produce a successful pregnancy. Causal factors include Müllerian anomalies, severe intrauterine adhesions/Asherman syndrome, uterine fibroids not amendable to surgical therapy, and radiation injury not responsive to medical therapy.

Prior to uterus transplantation, parenthood could only be achieved via adoption, foster parenting, or gestational carrier. While utilizing a gestational carrier is legal in most U.S. states, most countries of western Europe as well as Brazil and Japan, to name a few, do not allow the use of gestational carriers. For some women, moreover, the desire is not only to have a baby, but to carry a child as well.

For this edition of the Master Class in Gynecologic Surgery, I have enlisted the assistance of Rebecca Flyckt, MD, division chief of reproductive endocrinology and infertility at University Hospitals Cleveland Medical Center and associate professor at Case Western Reserve University, Cleveland, and Elliott G. Richards, MD, director of reproductive endocrinology and infertility research at the Cleveland Clinic, to discuss the current and future state of uterus transplantation.

Dr. Flyckt and Dr. Richards have both contributed to the uterus transplantation team at the Cleveland Clinic and are founding members of the U.S. Uterus Transplant Consortium. They are well published in the field of minimally invasive gynecology and reproductive endocrinology and infertility. It is truly a pleasure to welcome them both to this edition of the Master Class in Gynecologic Surgery.

References

1. Fertil Steril. 2014 May;101(5):1228-36.

2. Acta Biomater. 2014 Dec;10(12):5034-42.

3. Hum Reprod Update. Mar-Apr 2001;7(2):161-74.

4. Obstet Gynecol Surv. 2000 Oct;55(10):644-9.

5. Fertil Steril. 1997 Feb;67(2):387-9

6. Am J Public Health. 2003 Feb;93(2):307-12.
 

Dr. Miller is professor of obstetrics & gynecology in the department of clinical sciences, Rosalind Franklin University, North Chicago, and director of minimally invasive gynecologic surgery at Advocate Lutheran General Hospital, Park Ridge, Ill. Dr. Miller reported that he has no disclosures relevant to this Master Class. Email him at [email protected].

From the Milwaukee County Behavioral Health Division, Milwaukee, WI.

Abstract

Objective: Implementation of the Quadruple Aim of health care must begin with a clearly articulated set of concepts, or core domains (CDs), that comprise each aim. These CDs can then be operationalized with existing or new measures. If aligned to the organization’s mission and strategic goals, these CDs have the potential to focus quality improvement activities and reduce measurement burden. This article represents the efforts of a publicly funded behavioral health system to operationalize the Quadruple Aim through the development of CDs.

Methods: Various stakeholders across the organization were consulted on their perceptions of the Quadruple Aim and the CDs they believed should support it. Then, a review of existing literature on core metrics for health care and population health was completed, summarized, and integrated with the stakeholder feedback.

Results: These efforts led to the development and adoption of 15 CDs, with an accompanying literature review and set of recommendations of new and existing measures for each domain.

Conclusions: It is possible to create a comprehensive yet economical set of CDs and attendant measures that can be implemented in a staged, scalable, enterprise manner. It is hoped that the process articulated here, and the accompanying literature review, may be of some benefit to other public or government-run health systems in their own quality improvement journey to operationalize the Quadruple Aim by developing a set of CDs.

Keywords: quality measures; quality improvement; adult behavioral health.

First articulated in 2008, the Triple Aim proposes that health care systems should simultaneously seek to improve the patient’s experience of care, improve the health of populations, and reduce the per capita costs of care for populations.¹ More recently, some have argued that health care provider burnout can deleteriously impact the attainment of the Triple Aim and have therefore advocated for an expanded focus to include a fourth Aim, the work life quality of the staff.² Milwaukee County Behavioral Health Division (BHD), a publicly funded, county-based behavioral health care system in Milwaukee, Wisconsin, recently adopted the Quadruple Aim as the framework by which it will organize its quality activities.

Although originally developed for medical organizations, BHD believes that the Quadruple Aim has strong applicability to county-level behavioral health services. Many county-based behavioral health divisions provide a variety of programs to large segments of the county based on financial eligibility and/or clinical need, and thus often have responsibilities to populations or subpopulations, rather than programs. County health divisions, such as Milwaukee County’s Department of Health and Human Services, are often asked to improve outcomes and client experience of care with neutral growth budgets and less reliance on taxes to fund programs, while simultaneously attracting and retaining competent staff.

Crucial to the effective implementation of the Quadruple Aim, however, is a clear set of population- level measures that help organizations assess their progress.³ Unfortunately, as some authors have noted, evaluation of the Quadruple Aim remains a challenge because the “concepts of (population) health, quality of care and costs are not unanimously defined and measures for these concepts are under construction.”⁴ Several authors have provided some guidance to assist in the development of a set of measures that effectively capture the elements of the Quadruple Aim.^5,6 However, the recent rapid proliferation of quality measures in health care^7,8 has been both burdensome and costly for providers.^9,10 Any measures adopted should not only be as meaningful as possible with regards to assessing progress towards the basic aims of health care, but should also be parsimonious, to limit measurement burden for providers (and patients) and focus attention on important issues.^11,12

To select the most effective, parsimonious set of measures possible, one must first select a set of key foci from among the many possible areas of focus that the core measure is intended to represent. The core domains (CDs), if appropriately consistent with the strategic goals of the organization, provide a mechanism to orient the efforts of the organization at every level and help every staff member of the organization understand how his or her work impacts the progress towards these goals.¹¹ The CDs, therefore, represent the opportunity to affect a greater integration of efforts across the organization toward these shared aims, creating uniformity of purpose at every level. Further, increasing organizational attention on the CDs can also help to reduce measurement burden by streamlining and focusing the data capture processes on the most valuable elements of quality and health, and discarding other extraneous measures (albeit not at the expense of other reporting requirements).¹¹ The remainder of this article describes the CDs selected by BHD to assess its progress toward implementation of the Quadruple Aim and are organized by the Aim which they best represent.

Methods

To effectively implement the Quadruple Aim at BHD, it was necessary to clearly define the subpopulation of focus for our efforts.⁶ In this case, the subpopulation of interest was defined as all adult clients (18 years and older) who received at least 1 service encounter within a specified time frame from a program that BHD either operated or contracted with to provide care. Services provided by the BHD network include everything from psychiatric inpatient services to mental health and addiction treatment and care management. A limited array of social services, including housing and employment services, is also available to eligible consumers. BHD is the county-run behavioral health provider for individuals who are uninsured or underinsured in Milwaukee County, a demographically diverse, primarily urban county of approximately 950,000 people located in Wisconsin. Approximately 15,000 adults receive services at BHD each year.

This work began by obtaining executive sponsorship for the project, in this case from the Chief Operations Officer and Executive Medical Director of BHD. With their backing, an initial review of the literature produced a preliminary set of possible domains, for which we created working definitions. We then made a list of key stakeholders throughout BHD to whom we needed to present the idea of the Quadruple Aim, and the CDs under each Aim, to secure their support. These stakeholders, which included individuals involved in quality activities, program managers, and executive leadership, were strategically selected based on their relative influence within the organization. A set of brief presentations and handouts explaining the project were then developed and shared at different focus groups with these stakeholders over the course of 6 months. These focus groups served to not only educate the organization about the Quadruple Aim and the CDs but afforded participants an opportunity to provide feedback as well.

During the focus groups, we asked participants which domains they believed were most important (were “core”) when operationalizing the Quadruple Aim. The focus groups provided feedback on the domain definitions, feedback that was used to develop uniform, mutually agreed upon definitions for the CDs that were generalizable to all departments at BHD, regardless of the focus of their services within the continuum of care or the continuum of age. This was a crucial step, as it will eventually enable BHD to aggregate data across departments, even if there are minor discrepancies in the specific items they use to assess the CDs. Comments from the focus groups ultimately resulted in a truncated list of domains and definitions, which, coupled with the literature review, resulted in our final set of CDs.

During our review of the literature, we also looked for items that we felt could best represent each CD in the briefest, most meaningful way. (These items were not meant to supersede existing data, but to provide examples that could be implemented with existing data or recommendations that could be utilized in the absence of existing data.) During this process, we made every effort to make use of existing data-reporting requirements. For example, if we had a state mandate to collect data on housing status, we attempted to leverage this required data point to represent the CD related to housing. In other cases, we attempted to utilize claims or other administrative data to operationalize the CD, such as in the cost-of-care metric articulated in the section the Third Aim. For CDs for which no data existed or were insufficient, we emphasized the use of single- versus multi-item scales. For example, if we found a single-item global assessment of quality of life that had good psychometric properties relative to its longer parent scale, we selected the single item. This approach to item selection allowed us to create the most efficient, parsimonious set of measures possible, which we believed would enable us to comprehensively assess all the CDs with the least amount of burden to staff and clients. These items were presented at stakeholder focus groups, during which we asked for comments on the existing measures in their program or department and gave them the opportunity to comment on the new recommended measures.

A working definition is provided for each CD, followed by a brief review of the research base supporting its inclusion in the final list. The item(s) selected by BHD to represent each CD and the source of the item(s) are then supplied. These items were based either on measures currently collected because of existing reporting mandates or, in the case where extant measures were not available, on new items that demonstrated acceptable psychometric properties in the research literature. The CDs and items are organized by the Aim they best represent. A full list of the CDs by Quadruple Aim and items by CD is provided in the Appendix of the online version of this article. This article concludes with a brief summary of this effort and a discussion of how staff will utilize these items at different levels throughout the BHD system.

The First Aim: Population Health

Health Outcomes

Deaths. This can be defined as the cause of death, as determined by the medical examiner’s office (where appropriate) or as the age at time of death. This CD can also be reported as proportion of deaths considered premature (eg, before age 75) or calculated as total years of potential life lost.

Brief review and suggested item(s). Rates and causes of premature mortality are critical foci for the County Health Rankings & Roadmaps,¹³ the Institute for Healthcare Improvement’s “Guide to Measuring the Triple Aim,”⁶ the Centers for Disease Control and Prevention’s “Community Health Assessment for Population Health Improvement,”¹⁴ and the Institute of Medicine’s (IOM) “Vital Signs: Core Metrics for Health and Health Care Progress.”¹¹ There is ample evidence that individuals with serious mental illness are at increased risk of early mortality relative to the general population,^15-18 and this risk applies to those with substance use disorders as well.^15,19-20 BHD tracks all deaths that occur while patients are receiving BHD-funded, community-based services.

Self-Reported Health and Well-Being. This CD asks patients to rate their current physical and mental health status, as well as their overall quality of life.

Brief review and suggested item(s): Self-rated physical health. Premature mortality among individuals with behavioral health issues appears to be due, in large part, to their increased vulnerability to the development of medical comorbidities.^16,21 A single self-rating question has demonstrated considerable sensitivity to premature mortality,^22,23 with predictive properties up to a decade prior to death.^24,25 Further, self-rated health has been associated with subsequent functional decline,^26,27 acute service utilization,^28,29 and overall health care costs.²⁸

Brief review and suggested item(s): Self-rated mental health. Mental health disorders are associated with significant disability worldwide,³⁰ and comorbid mental health issues can exacerbate the course of other medical problems. For example, depression is associated with increased rates of mortality among individuals with diabetes and³¹ cardiovascular disease,³² as well as with rates of overall mortality,³³ and psychiatric comorbidity is associated with longer lengths of stay and higher costs among patients hospitalized for medical problems.³⁴ Research has found that a single-item measure of self-rated mental health is associated with the presence of psychiatric diagnoses, psychiatric symptoms, and subsequent depression and serious mental illness up to 1 year post-assessment.^35,36 There is even evidence that self-rated mental health may be more strongly associated with self-ratings of overall health than self-ratings of physical health.³⁷

Brief review and suggested item(s): Self-rated quality of life. Quality of life is a critical component of the recovery journey and overall health.³⁸ For example, the County Health Rankings & Roadmaps lists “quality of life” as 1 of its key “health outcomes” in its County Health Rankings.¹³ As some authors have noted, quality of life is often inferred from other “objective” recovery domains, such as employment, health status, or housing status. However, there is evidence that these objective domains are functionally distinct from the inherently subjective construct of quality of life.³⁹ This has led other authors to conclude that these domains should be assessed separately when evaluating outcomes.⁴⁰ Single-item quality of life assessments have been used in research with individuals with cancer,⁴¹ adults with disabilities,⁴² patients with cystic fibrosis,⁴³ and children with epilepsy.⁴⁴ For this effort, BHD selected the first global quality of life item from the World Health Organization’s WHOQOL-BREF quality of life assessment,⁴⁵ an item used in other quality of life research.⁴⁶

Health Factors

Substance Use. This CD is a composite of 4 different types of substance use, any recent heavy alcohol use (defined as 5 or more drinks in one sitting), any recent drug use, any recent prescription drug abuse, and any recent tobacco use.

Brief review and suggested item(s). As noted, substance use disorders confer an increased risk for early mortality^15,19 and are significantly implicated in disease disability burden worldwide.³⁰ Substance use has also been associated with both the onset^47,48 and exacerbation of mental health diagnoses.^49-51 Further, substance use appears to heighten the risk of violence in the general population⁵² and especially among those with a co-occurring mental illness.^53,54 The County Health Rankings & Roadmaps list alcohol and drug use as key behaviors to address to improve the overall health of a given county,¹³ and the Centers for Medicare & Medicaid Services (CMS) has endorsed initiation and engagement in addiction treatment as one of the measures in its Adult Core Set.⁵⁵

Tobacco use continues to be one of the most significant risk factors for early mortality worldwide, and evidence indicates that it is associated with a lower life expectancy of nearly 10 years.⁵⁶ Unfortunately, rates of tobacco use are even higher among those with severe mental illness relative to the general population, and their rates of smoking cessation are lower.^57,58 Tobacco use is a significant risk factor for the high rates of early mortality in individuals with severe mental illness.¹⁸ Further, a recent meta-analysis noted that, relative to those who continued to smoke, those who ceased smoking had reduced rates of psychological distress and increased quality of life rankings.⁵⁹ Reducing tobacco use is one of the key components of the County Health Rankings & Roadmaps, and medication assistance with smoking and tobacco use cessation is also listed in the CMS Adult Core Set.^13,55

An accumulating body of evidence suggests that single-item measures can adequately detect alcohol^60-62 and drug use disorders.^60-64 McNeely and colleagues recently developed and tested a brief 4-item screen, the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool.^65,66 Preliminary evidence suggests that the TAPS tool can effectively identify the presence of problematic and disordered use of tobacco, alcohol, prescription medications, and other drugs.^65-67 BHD will use the 4 items from the TAPS tool to represent its substance use CD.

Education/Employment Status. This CD assesses the proportion of BHD members who have completed high school, who are in some type of educational or training program, or who are engaged in some type of employment activity (defined as full-time, part-time, supported, sheltered workshop, or as a full-time homemaker).

Brief review and suggested item(s). Research indicates that unemployment is a risk factor for mortality, even after controlling for other risk factors (eg, age, sex, socioeconomic status [SES], health).⁶⁸ Unemployment is associated with poorer physical and mental health in the general population and among those with disabilities.^69-71 Promisingly, evidence suggests that gaining employment or re-employment is associated with better health,⁷² even for individuals with substance use disorders⁷³ or moderate⁷⁴ to severe mental health disorders.^75-78 Some authors have even proposed that, above and beyond the associated health benefits, employment may also help to realize a modest cost savings due to reduced service utilization and disability.^79,80 Employment is a core tenet in the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) model of recovery,⁸¹ and is also listed as an important recovery goal for individuals with behavioral health issues.⁸² BHD collects data on employment status on all the patients it serves as part of its state-mandated reporting requirements and will use this item in the CD data set.⁸³

Living Situation. This is measured as the proportion of people who live in permanent, supportive, stable housing; it may also be measured as the percentage of the population living with severe housing problems or who are homeless.

Brief review and suggested item(s). Housing problems can be conceptualized as 3 inter-related components: conditions within the home, neighborhood conditions, and housing affordability, each of which can contribute uniquely to poorer physical and mental health of individuals and families⁸⁴ and to educational outcomes for children.^85,86 Further, individuals who are homeless have a standardized mortality ratio 2 to 5 times that of the general population,^87-89 even after controlling for low income status,⁹⁰ and some evidence suggests these rates are even higher among unsheltered versus sheltered homeless individuals.⁹¹ Interventions to improve the condition of housing have demonstrated positive impacts on both physical and mental health,⁹² and a recent study found that individuals receiving housing assistance in the form of public housing or multifamily housing from the Department of Housing and Urban Development had better self-rated physical and mental health relative to individuals on the wait list for housing assistance.⁹³ Moreover, the provision of housing has been shown to promote reductions in substance use and health service utilization among homeless individuals with substance use disorders.⁹⁴ Rog and colleagues reviewed the literature on permanent supportive housing for individuals with substance use or mental health disorders who were homeless or disabled, and found that provision of housing led to reduced rates of homelessness, emergency department (ED) and inpatient utilization and increased consumer satisfaction.⁹⁵

Importantly, evidence suggests that housing is viewed as facilitative of recovery. For example, in a recent qualitative study of homeless individuals with mental illness, housing was seen as a critical first step in recovery, providing a sense of security, increasing feelings of personal independence and autonomy, improving perceptions of health and well-being, and affording a stable environment to rebuild relationships with important others.⁹⁶ BHD collects data on housing status on all the patients it serves as part of its state-mandated reporting requirements and will utilize this item in the CD data set.⁸³

Social Relationships. This is defined as recent interactions with family, supportive networks (formal and informal), and other recovery services.

Brief review and suggested item(s). Research has long established that social relationships have a significant impact on health, including rates of mortality as well as physical and mental health morbidity.^97-99 Social connectedness is another of the pillars supporting an individual’s recovery in SAMHSA’s formulation. Several reviews of the recovery literature^38,82 support its importance to the recovery process and inclusion in any assessment of holistic recovery. Social support has been shown to promote recovery among individuals with severe mental illness^100-102 and substance use disorders,¹⁰³ and may mitigate the progression of chronic, life-threatening physical illnesses.⁹⁷ For the purposes of BHD’s CD data set, the social support question from the “100 Million Healthier Lives Common Questionnaire for Adults” will be used to assess individuals’ perceived adequacy of social support.¹⁰⁴

Legal Involvement. Defined as involvement with the civil or criminal justice system, including arrests, imprisonment, or detainment.

Brief review and suggested item(s). Involvement in the criminal justice system is both disruptive for the individual in recovery and expensive to the larger health care system.¹⁰⁵ Individuals with substance use¹⁰⁶ and severe mental health disorders¹⁰⁷ are over-represented in the prison system, and evidence suggests that general physical and mental health declines while individuals are in prison.^108,109 Perhaps even more concerning, numerous studies have demonstrated an increase in mortality rates for individuals recently released from prison relative to the general population, particularly during the period immediately following release.^108-110 This relationship may even persist long term.¹¹¹ Further, research indicates that individuals recently released from prison have increased emergency care and hospital utilization.^112,113

Incarceration can have significant impacts on the health of the broader community as well. For example, research has found an association between parental incarceration to rates of infant mortality,¹¹⁴ increased behavioral and developmental problems of children of incarcerated parents,^115,116 lower rates of child support payments,¹¹⁷ and poorer cardiovascular health of female partners of incarcerated individuals.¹¹⁸ Formerly incarcerated individuals experience slower wage growth as well.¹¹⁹ However, evidence also indicates that engagement in mental health¹²⁰ and substance abuse¹²¹ treatment can reduce the likelihood of subsequent recidivism. As part of its state-mandated reporting, BHD is required to provide information on the criminal justice system involvement of its clients in the previous 6 months, including whether they have been jailed or imprisoned,⁸³ and this will function as its measure of legal involvement in its CD data set.

Socioeconomic Status. Socioeconomic status is the social standing or class of an individual or group. It is often measured as a combination of education, income, and occupation. It can also be defined subjectively, such as one’s evaluation of status relative to similar others or based on an individual’s interpretation of her or his financial needs.

Brief review and suggested item(s). A large body of evidence supports the existence of a robust relationship between lower SES and poor health, including mortality and chronic medical diseases,^122-124 as well as mental illness.^125-127 Although previous research has examined this relationship using objective indicators of SES (eg, income, education level, occupation), there has recently been an increased interest in exploring the relationship of subjective SES with health indices. Subjective SES is generally assessed by asking individuals to rate themselves relative to others in the society in which they live, in terms of wealth, occupation, educational level, or other indicators of social status. Evidence suggests that subjective SES is associated with objective measures of SES,^128-130 and relates to measures of physical and mental health as well, even after controlling for objective SES.^130-135 BHD will be using a modified version of the Subject SES Scale,^131,135 which is deployed in the “100 Million Healthier Lives Common Questionnaire for Adults.”¹⁰⁴

Acute Service Use. This is defined as an admission to a medical or psychiatric emergency room or to a medical or psychiatric hospital or to a detoxification facility.

Brief review and suggested item(s). The CMS Adult Core Set includes “plan all cause readmissions” as a key quality metric.⁵⁵ Hospital readmissions are also endorsed by the National Committee on Quality Assurance as one of its Health Effectiveness Data and Information Set (HEDIS) measures and by the National Quality Forum. Readmissions, despite their widespread endorsement, are a somewhat controversial measure. Although readmissions are costly to the health care system,¹³⁶ the relationship between readmissions and quality is inconsistent. For example, Krumholz and colleagues¹³⁷ found differential rates of readmission for the same patient discharged from 2 different hospitals, which were categorized based on previous readmission rates, suggesting that hospitals do have different levels of performance even when treating the same patient. However, other data indicate that 30-day, all-cause, risk-standardized readmission rates are not associated with hospital 30-day, all-cause, risk-standardized mortality rates.¹³⁸

Chin and colleague found that readmissions to the hospital that occurred more than 7 days post-discharge were likely due to community- and household-related factors, rather than hospital-related quality factors.¹³⁹ Transitional care interventions that have successfully reduced 30-day readmission rates are most often multicomponent and focus not just on hospital-based interventions (eg, discharge planning, education) but on follow-up care in the community by formal supports (eg, in-home visits, telephone calls, outpatient clinic appointments, case management) and informal supports (eg, family and friends).^140-143 Further, qualitative evidence suggests that some individuals perceive psychiatric hospitalizations to be the result of insufficient resources or unsuccessful attempts to maintain their stability in the community.¹⁴⁴ Thus, unplanned or avoidable hospital readmissions may represent a failure of the continuum of care not only from the perspective of the health care system, but from the patient perspective as well.

Frequent or nonurgent use of EDs is conceptually similar to excessive or avoidable inpatient utilization in several ways. For example, overuse of EDs is costly, with some estimates suggesting that it is responsible for up to $38 billion in wasteful spending each year.¹⁴⁵ Individuals with frequent ED visits have a greater disease burden¹⁴⁶ and an increased risk of mortality compared to nonfrequent users.¹⁴⁷ Research suggests that individuals who visit the ED for non-urgent issues do so because of perceived difficulties associated with accessing primary care, and the convenience of EDs relative to primary care.^148-150 Moreover, similar to the hospital readmission literature discussed earlier, successful strategies to reduce high rates of ED utilization generally focus on continuum of care interventions, such as provision of case management services.^151-155

This evidence implies that frequent ED utilization and hospital readmissions may not be a fundamental issue of quality (or lack thereof) in hospitals or EDs but rather a lack of, or ineffectual, transitional and continuum of care strategies and services. To underscore this point, some authors have argued that a system that is excessively crisis-oriented hinders recovery because it is reactive rather than proactive, predicated on the notion that one’s condition must deteriorate to receive care.¹⁵⁶

Although some organizations may have access to claims data or may function as self-contained health systems (eg, the Veterans Health Administration [VHA] ), others may not have access to such data. In the absence of claims data, patient self-report of service utilization has been used as a proxy for actual agency records.¹⁵⁷ Although concordance between medical and/or agency records and patient self-report has been variable,¹⁵⁷ evidence generally suggests that rates of agreement are higher the shorter the recall time interval.^158,159 BHD does not have access to comprehensive claims data and has therefore chosen to use 5 dichotomously scored (yes/no) questions—related to medical inpatient, medical ED, psychiatric inpatient, psychiatric ED, and detoxification use in the last 30 days—to represent the CD of acute service utilization.

The Second Aim: Quality of Care

Safety

Safety is defined as avoiding injuries to patients from the care that is intended to help them.

Brief review and suggested item(s). As noted in “Crossing the Quality Chasm,” the IOM’s seminal document, “the health care environment should be safe for all patients, in all of its processes, all the time.”¹⁶⁰ The landmark Harvard Medical Practice Study in 1991 found that adverse events occurred in nearly 4% of all hospital admissions and, among these, over a quarter were due to negligence.¹⁶¹ Other estimates of adverse events range as high as 17%.¹⁶² Indeed, a recent article by Makary and Daniel estimated that medical errors may be the third leading cause of death in the United States.¹⁶³ Unfortunately, research on safety in the mental health field has lagged behind that of physical health,¹⁶⁴ with evidence indicating that research in nonhospital settings in mental health care may be particularly scarce.¹⁶⁵ In a study of adverse events that occurred in psychiatric inpatient units in the VHA system between 2015 and 2016, Mills and colleagues found that of the 87 root cause analysis reports, suicide attempts were the most frequent, and, among safety events, falls were the most frequently reported, followed by medication events.¹⁶⁶ Another report on data collected from psychiatric inpatient units in the VHA revealed that nearly one-fifth of patients experienced a safety event, over half of which were deemed preventable.¹⁶⁷ These numbers likely represent an underestimation of the true volume of safety events, as another study by the same research group found that less than 40% of safety events described in patient medical records were documented in the incident reporting system.¹⁶⁸ BHD will utilize the total number of complaints and incident reports submitted within a given time frame as its “safety” metric in the CD data set.

Wait Time for Service

The CD is defined as the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Brief review and suggested item(s). “Timeliness” was listed among the 6 aims for improvement in “Crossing the Quality Chasm” in 2001, and it remains no less relevant today.¹⁶⁰ For example, evidence indicates that access to primary care is inversely related to avoidable hospitalizations.¹⁶⁹ One study found that, of patients hospitalized for cardiovascular problems, those who had difficulty accessing routine care post discharge had higher 30-day readmission rates.¹⁷⁰ Among VHA patients, longer wait times are associated with more avoidable hospitalizations and higher rates of mortality.¹⁷¹ Longer wait times appear to decrease the likelihood of attending a first appointment for individuals with substance use^172,173 and mental health disorders.¹⁷⁴ Importantly, longer wait times are associated with lower ratings of the patient experience of care, including perceptions of the quality of and satisfaction with care,¹⁷⁵ and may be associated with worse outcomes for individuals in early intervention for psychosis treatment.¹⁷⁶ For the purposes of the CD data set, BHD will monitor the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Patient satisfaction is defined as the degree of patients’ satisfaction with the care they have received.

Brief review and suggested item(s). Research has consistently demonstrated the relationship of the patient’s experience of care to a variety of safety and clinical effectiveness measures in medical health care,¹⁷⁷ and the therapeutic alliance is one of the most consistent predictors of outcomes in behavioral health, regardless of therapeutic modality.¹⁷⁸ Patient satisfaction is a commonly assessed aspect of the patient experience of care. Patient satisfaction scores have been correlated with patient adherence to recommended treatment regimens, care quality, and health outcomes.¹⁷⁹ For example, Aiken et al found that patient satisfaction with hospital care was associated with higher ratings of the quality and safety of nursing care in these hospitals.¹⁸⁰ Increased satisfaction with inpatient care has been associated with lower 30-day readmission rates for patients with acute myocardial infarction, heart failure, and pneumonia,¹⁸¹ and patients with schizophrenia who reported higher treatment satisfaction also reported better quality of life.^182,183 Many satisfaction survey options exist to evaluate this CD, including the Consumer Assessment of Healthcare Providers and Systems and the Client Satisfaction Questionnaire; BHD will utilize an outpatient behavioral health survey from a third-party vendor.

The Third Aim: Cost of Care

Cost of Care

This can be defined as the average cost to provide care per patient per month.

Brief review and suggested item(s). Per capita cost, or rather, the total cost of providing care to a circumscribed population divided by the total population, has been espoused as an important metric for the Triple Aim and the County Health Rankings.^6,13 Indeed, between 1960 and 2016, per capita expenditures for health care have grown 70-fold, and the percent of the national gross domestic product accounted for by health expenditures has more than tripled (5.0% to 17.9%).¹⁸⁴ One of the more common metrics deployed for assessing health care cost is the per capita per month cost, or rather, the per member per month cost of the predefined population for a given health care system.^6,185,186 In fact, some authors have proposed that cost of care can be used not only to track efficient resource allocation, but can also be a proxy for a healthier population as well (ie, as health improves, individuals use fewer and less-expensive services, thus costing the system less).¹⁸⁷ To assess this metric, BHD will calculate the total amount billed for patient care provided within BHD’s health network each month (irrespective of funding source) and then divide this sum by the number of members served each month. Although this measure does not account for care received at other health care facilities outside BHD’s provider network, nor does it include all the overhead costs associated with the care provided by BHD itself, it is consistent with the claims-based approach used or recommended by other authors.^6,188

The Fourth Aim: Staff Well-being

Staff Quality of Work Life

This can be defined as the quality of the work life of health care clinicians and staff.

Brief review and suggested item(s). Some authors have suggested that the Triple Aim framework is incomplete and have proffered compelling arguments that provider well-being and the quality of work life constitutes a fourth aim.² Provider burnout is prevalent in both medical^2,189 and behavioral health care.^190,191 Burnout among health care professionals has been associated with higher rates of perceived medical errors,¹⁹² lower patient satisfaction scores,^189,193 lower rates of provider empathy,¹⁹⁴ more negative attitudes towards patients,¹⁹⁵ and poorer staff mental and physical health.¹⁹¹

Burnout is also associated with higher rates of absenteeism, turnover intentions, and turnover.^{190,191,196,197} However, burnout is not the only predictor of staff turnover; for example, turnover rates are a useful proxy for staff quality of work life for several reasons.¹⁹⁸ First, turnover is associated with substantial direct and indirect costs, including lost productivity, increased errors, and lost revenue and recruitment costs, with some turnover cost estimates as high as $17 billion for physicians and $14 billion for nurses annually.^199-201 Second, research indicates that staff turnover can have a deleterious impact on implementation of evidence-based interventions.^202-205 Finally, consistent with the philosophy of utilizing existing data sources for the CD measures, turnover can be relatively easily extracted from administrative data for operated or contracted programs, and its collection does not place any additional burden on staff. As a large behavioral health system that is both a provider and payer of care, BHD will therefore examine the turnover rates of its internal administrative and clinical staff as well as the turnover of staff in its contracted provider network as its measures for the Staff Quality of Work Life CD.

Clinical Implications

These metrics can be deployed at any level of the organization. Clinicians may use 1 or more of the measures to track the recovery of individual clients, or in aggregate for their entire caseload. Similarly, managers can use these measures to assess the overall effectiveness of the programs for which they are responsible. Executive leaders can evaluate the impact of several programs or the system of care on the health of a subpopulation of clients with a specific condition, or for all their enrolled members. Further, not all measures need be utilized for every dashboard or evaluative effort. The benefit of a comprehensive set of measures lies in their flexibility—1 or more of the measures may be selected depending on the project being implemented or the interests of the stakeholder.

It is important to note that many of the CDs (and their accompanying measures) are aligned to/consistent with social determinants of health.^206,207 Evidence suggests that social determinants make substantial contributions to the overall health of individuals and populations and may even account for a greater proportion of variance in health outcomes than health care itself.²⁰⁸ The measures articulated here, therefore, can be used to assess whether and how effectively care provision has addressed these social determinants, as well as the relative impact their resolution may have on other health outcomes (eg, mortality, self-rated health).

These measures can also be used to stratify clients by clinical severity or degree of socioeconomic deprivation. The ability to adjust for risk has many applications in health care, particularly when organizations are attempting to implement value-based purchasing models, such as pay-for-performance contracts or other alternative payment models (population health-based payment models).²⁰⁹ Indeed, once fully implemented, the CDs and measures will enable BHD to more effectively build and execute different conceptual models of “value” (see references 210 and 211 for examples). We will be able to assess the progress our clients have made in care, the cost associated with that degree of improvement, the experience of those clients receiving that care, and the clinical and social variables that may influence the relative degree of improvement (or lack thereof). Thus, the CDs provide a conceptual and data-driven foundation for the Quadruple Aim and any quality initiatives that either catalyze or augment its implementation.

This article provides an overview of the CDs selected by BHD to help organize, focus, advance, and track its quality efforts within the framework of the Quadruple Aim. Although items aligned to each of these CDs are offered, the CDs themselves have been broadly conceptualized such that they can flexibly admit a variety of possible items and/or assessments to operationalize each CD and thus have potential applicability to other behavioral health systems, particularly public systems that have state-mandated and other data reporting requirements.

Bearing in mind the burden that growing data collection requirements can have on the provision of quality care and staff work satisfaction and burnout,^10,212 the CDs (and the items selected to represent each) are designed with “strategic parsimony” in mind. Although the CDs are inclusive in that they cover care quality, cost of care, staff quality of life, and general population health, only CDs and items undergirded by a solid evidence base and high value with regards to BHD’s mission and values, as determined by key stakeholders, were selected. Moreover, BHD attempted to make use of existing data collection and reporting mandates when selecting the final pool of items to reduce the measurement burden on staff and clients. Thus, the final set of CDs and items are designed to be comprehensive yet economical.

The CDs are deeply interrelated. Although each CD may be individually viewed as a valuable metric, improvements in any 1 CD will impact the others (eg, increasing care quality should impact population health, increasing staff quality of life should impact the quality of care). Moreover, this idea of interrelatedness acknowledges the need to view health systems and the populations they serve holistically, in that improvement is not simply the degree of change in any given metric (whether individually or collectively), but rather something more entirely. The concepts of value, quality, and health are complex, multidimensional, and dynamic, and the CDs that comprise these concepts should not be considered independently from one another. The CDs (and items) offered in this article are scalable in that they can be used at different levels of an organization depending on the question or stakeholder, and can be used individually or in combination with one another. Moreover, they are adaptable to a variety of risk-adjusted program, population health, and value-based evaluation models. It is hoped that the process articulated here, and the accompanying literature review, may benefit other public or government-run health systems in their own quality journey to operationalize the Quadruple Aim by developing a set of CDs.

Corresponding author: Walter Matthew Drymalski, PhD; [email protected].

Financial disclosures: None.

From the Milwaukee County Behavioral Health Division, Milwaukee, WI.

Abstract

Objective: Implementation of the Quadruple Aim of health care must begin with a clearly articulated set of concepts, or core domains (CDs), that comprise each aim. These CDs can then be operationalized with existing or new measures. If aligned to the organization’s mission and strategic goals, these CDs have the potential to focus quality improvement activities and reduce measurement burden. This article represents the efforts of a publicly funded behavioral health system to operationalize the Quadruple Aim through the development of CDs.

Methods: Various stakeholders across the organization were consulted on their perceptions of the Quadruple Aim and the CDs they believed should support it. Then, a review of existing literature on core metrics for health care and population health was completed, summarized, and integrated with the stakeholder feedback.

Results: These efforts led to the development and adoption of 15 CDs, with an accompanying literature review and set of recommendations of new and existing measures for each domain.

Conclusions: It is possible to create a comprehensive yet economical set of CDs and attendant measures that can be implemented in a staged, scalable, enterprise manner. It is hoped that the process articulated here, and the accompanying literature review, may be of some benefit to other public or government-run health systems in their own quality improvement journey to operationalize the Quadruple Aim by developing a set of CDs.

Keywords: quality measures; quality improvement; adult behavioral health.

First articulated in 2008, the Triple Aim proposes that health care systems should simultaneously seek to improve the patient’s experience of care, improve the health of populations, and reduce the per capita costs of care for populations.¹ More recently, some have argued that health care provider burnout can deleteriously impact the attainment of the Triple Aim and have therefore advocated for an expanded focus to include a fourth Aim, the work life quality of the staff.² Milwaukee County Behavioral Health Division (BHD), a publicly funded, county-based behavioral health care system in Milwaukee, Wisconsin, recently adopted the Quadruple Aim as the framework by which it will organize its quality activities.

Although originally developed for medical organizations, BHD believes that the Quadruple Aim has strong applicability to county-level behavioral health services. Many county-based behavioral health divisions provide a variety of programs to large segments of the county based on financial eligibility and/or clinical need, and thus often have responsibilities to populations or subpopulations, rather than programs. County health divisions, such as Milwaukee County’s Department of Health and Human Services, are often asked to improve outcomes and client experience of care with neutral growth budgets and less reliance on taxes to fund programs, while simultaneously attracting and retaining competent staff.

Crucial to the effective implementation of the Quadruple Aim, however, is a clear set of population- level measures that help organizations assess their progress.³ Unfortunately, as some authors have noted, evaluation of the Quadruple Aim remains a challenge because the “concepts of (population) health, quality of care and costs are not unanimously defined and measures for these concepts are under construction.”⁴ Several authors have provided some guidance to assist in the development of a set of measures that effectively capture the elements of the Quadruple Aim.^5,6 However, the recent rapid proliferation of quality measures in health care^7,8 has been both burdensome and costly for providers.^9,10 Any measures adopted should not only be as meaningful as possible with regards to assessing progress towards the basic aims of health care, but should also be parsimonious, to limit measurement burden for providers (and patients) and focus attention on important issues.^11,12

To select the most effective, parsimonious set of measures possible, one must first select a set of key foci from among the many possible areas of focus that the core measure is intended to represent. The core domains (CDs), if appropriately consistent with the strategic goals of the organization, provide a mechanism to orient the efforts of the organization at every level and help every staff member of the organization understand how his or her work impacts the progress towards these goals.¹¹ The CDs, therefore, represent the opportunity to affect a greater integration of efforts across the organization toward these shared aims, creating uniformity of purpose at every level. Further, increasing organizational attention on the CDs can also help to reduce measurement burden by streamlining and focusing the data capture processes on the most valuable elements of quality and health, and discarding other extraneous measures (albeit not at the expense of other reporting requirements).¹¹ The remainder of this article describes the CDs selected by BHD to assess its progress toward implementation of the Quadruple Aim and are organized by the Aim which they best represent.

Methods

To effectively implement the Quadruple Aim at BHD, it was necessary to clearly define the subpopulation of focus for our efforts.⁶ In this case, the subpopulation of interest was defined as all adult clients (18 years and older) who received at least 1 service encounter within a specified time frame from a program that BHD either operated or contracted with to provide care. Services provided by the BHD network include everything from psychiatric inpatient services to mental health and addiction treatment and care management. A limited array of social services, including housing and employment services, is also available to eligible consumers. BHD is the county-run behavioral health provider for individuals who are uninsured or underinsured in Milwaukee County, a demographically diverse, primarily urban county of approximately 950,000 people located in Wisconsin. Approximately 15,000 adults receive services at BHD each year.

This work began by obtaining executive sponsorship for the project, in this case from the Chief Operations Officer and Executive Medical Director of BHD. With their backing, an initial review of the literature produced a preliminary set of possible domains, for which we created working definitions. We then made a list of key stakeholders throughout BHD to whom we needed to present the idea of the Quadruple Aim, and the CDs under each Aim, to secure their support. These stakeholders, which included individuals involved in quality activities, program managers, and executive leadership, were strategically selected based on their relative influence within the organization. A set of brief presentations and handouts explaining the project were then developed and shared at different focus groups with these stakeholders over the course of 6 months. These focus groups served to not only educate the organization about the Quadruple Aim and the CDs but afforded participants an opportunity to provide feedback as well.

During the focus groups, we asked participants which domains they believed were most important (were “core”) when operationalizing the Quadruple Aim. The focus groups provided feedback on the domain definitions, feedback that was used to develop uniform, mutually agreed upon definitions for the CDs that were generalizable to all departments at BHD, regardless of the focus of their services within the continuum of care or the continuum of age. This was a crucial step, as it will eventually enable BHD to aggregate data across departments, even if there are minor discrepancies in the specific items they use to assess the CDs. Comments from the focus groups ultimately resulted in a truncated list of domains and definitions, which, coupled with the literature review, resulted in our final set of CDs.

During our review of the literature, we also looked for items that we felt could best represent each CD in the briefest, most meaningful way. (These items were not meant to supersede existing data, but to provide examples that could be implemented with existing data or recommendations that could be utilized in the absence of existing data.) During this process, we made every effort to make use of existing data-reporting requirements. For example, if we had a state mandate to collect data on housing status, we attempted to leverage this required data point to represent the CD related to housing. In other cases, we attempted to utilize claims or other administrative data to operationalize the CD, such as in the cost-of-care metric articulated in the section the Third Aim. For CDs for which no data existed or were insufficient, we emphasized the use of single- versus multi-item scales. For example, if we found a single-item global assessment of quality of life that had good psychometric properties relative to its longer parent scale, we selected the single item. This approach to item selection allowed us to create the most efficient, parsimonious set of measures possible, which we believed would enable us to comprehensively assess all the CDs with the least amount of burden to staff and clients. These items were presented at stakeholder focus groups, during which we asked for comments on the existing measures in their program or department and gave them the opportunity to comment on the new recommended measures.

A working definition is provided for each CD, followed by a brief review of the research base supporting its inclusion in the final list. The item(s) selected by BHD to represent each CD and the source of the item(s) are then supplied. These items were based either on measures currently collected because of existing reporting mandates or, in the case where extant measures were not available, on new items that demonstrated acceptable psychometric properties in the research literature. The CDs and items are organized by the Aim they best represent. A full list of the CDs by Quadruple Aim and items by CD is provided in the Appendix of the online version of this article. This article concludes with a brief summary of this effort and a discussion of how staff will utilize these items at different levels throughout the BHD system.

The First Aim: Population Health

Health Outcomes

Deaths. This can be defined as the cause of death, as determined by the medical examiner’s office (where appropriate) or as the age at time of death. This CD can also be reported as proportion of deaths considered premature (eg, before age 75) or calculated as total years of potential life lost.

Brief review and suggested item(s). Rates and causes of premature mortality are critical foci for the County Health Rankings & Roadmaps,¹³ the Institute for Healthcare Improvement’s “Guide to Measuring the Triple Aim,”⁶ the Centers for Disease Control and Prevention’s “Community Health Assessment for Population Health Improvement,”¹⁴ and the Institute of Medicine’s (IOM) “Vital Signs: Core Metrics for Health and Health Care Progress.”¹¹ There is ample evidence that individuals with serious mental illness are at increased risk of early mortality relative to the general population,^15-18 and this risk applies to those with substance use disorders as well.^15,19-20 BHD tracks all deaths that occur while patients are receiving BHD-funded, community-based services.

Self-Reported Health and Well-Being. This CD asks patients to rate their current physical and mental health status, as well as their overall quality of life.

Brief review and suggested item(s): Self-rated physical health. Premature mortality among individuals with behavioral health issues appears to be due, in large part, to their increased vulnerability to the development of medical comorbidities.^16,21 A single self-rating question has demonstrated considerable sensitivity to premature mortality,^22,23 with predictive properties up to a decade prior to death.^24,25 Further, self-rated health has been associated with subsequent functional decline,^26,27 acute service utilization,^28,29 and overall health care costs.²⁸

Brief review and suggested item(s): Self-rated mental health. Mental health disorders are associated with significant disability worldwide,³⁰ and comorbid mental health issues can exacerbate the course of other medical problems. For example, depression is associated with increased rates of mortality among individuals with diabetes and³¹ cardiovascular disease,³² as well as with rates of overall mortality,³³ and psychiatric comorbidity is associated with longer lengths of stay and higher costs among patients hospitalized for medical problems.³⁴ Research has found that a single-item measure of self-rated mental health is associated with the presence of psychiatric diagnoses, psychiatric symptoms, and subsequent depression and serious mental illness up to 1 year post-assessment.^35,36 There is even evidence that self-rated mental health may be more strongly associated with self-ratings of overall health than self-ratings of physical health.³⁷

Brief review and suggested item(s): Self-rated quality of life. Quality of life is a critical component of the recovery journey and overall health.³⁸ For example, the County Health Rankings & Roadmaps lists “quality of life” as 1 of its key “health outcomes” in its County Health Rankings.¹³ As some authors have noted, quality of life is often inferred from other “objective” recovery domains, such as employment, health status, or housing status. However, there is evidence that these objective domains are functionally distinct from the inherently subjective construct of quality of life.³⁹ This has led other authors to conclude that these domains should be assessed separately when evaluating outcomes.⁴⁰ Single-item quality of life assessments have been used in research with individuals with cancer,⁴¹ adults with disabilities,⁴² patients with cystic fibrosis,⁴³ and children with epilepsy.⁴⁴ For this effort, BHD selected the first global quality of life item from the World Health Organization’s WHOQOL-BREF quality of life assessment,⁴⁵ an item used in other quality of life research.⁴⁶

Health Factors

Substance Use. This CD is a composite of 4 different types of substance use, any recent heavy alcohol use (defined as 5 or more drinks in one sitting), any recent drug use, any recent prescription drug abuse, and any recent tobacco use.

Brief review and suggested item(s). As noted, substance use disorders confer an increased risk for early mortality^15,19 and are significantly implicated in disease disability burden worldwide.³⁰ Substance use has also been associated with both the onset^47,48 and exacerbation of mental health diagnoses.^49-51 Further, substance use appears to heighten the risk of violence in the general population⁵² and especially among those with a co-occurring mental illness.^53,54 The County Health Rankings & Roadmaps list alcohol and drug use as key behaviors to address to improve the overall health of a given county,¹³ and the Centers for Medicare & Medicaid Services (CMS) has endorsed initiation and engagement in addiction treatment as one of the measures in its Adult Core Set.⁵⁵

Tobacco use continues to be one of the most significant risk factors for early mortality worldwide, and evidence indicates that it is associated with a lower life expectancy of nearly 10 years.⁵⁶ Unfortunately, rates of tobacco use are even higher among those with severe mental illness relative to the general population, and their rates of smoking cessation are lower.^57,58 Tobacco use is a significant risk factor for the high rates of early mortality in individuals with severe mental illness.¹⁸ Further, a recent meta-analysis noted that, relative to those who continued to smoke, those who ceased smoking had reduced rates of psychological distress and increased quality of life rankings.⁵⁹ Reducing tobacco use is one of the key components of the County Health Rankings & Roadmaps, and medication assistance with smoking and tobacco use cessation is also listed in the CMS Adult Core Set.^13,55

An accumulating body of evidence suggests that single-item measures can adequately detect alcohol^60-62 and drug use disorders.^60-64 McNeely and colleagues recently developed and tested a brief 4-item screen, the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool.^65,66 Preliminary evidence suggests that the TAPS tool can effectively identify the presence of problematic and disordered use of tobacco, alcohol, prescription medications, and other drugs.^65-67 BHD will use the 4 items from the TAPS tool to represent its substance use CD.

Education/Employment Status. This CD assesses the proportion of BHD members who have completed high school, who are in some type of educational or training program, or who are engaged in some type of employment activity (defined as full-time, part-time, supported, sheltered workshop, or as a full-time homemaker).

Brief review and suggested item(s). Research indicates that unemployment is a risk factor for mortality, even after controlling for other risk factors (eg, age, sex, socioeconomic status [SES], health).⁶⁸ Unemployment is associated with poorer physical and mental health in the general population and among those with disabilities.^69-71 Promisingly, evidence suggests that gaining employment or re-employment is associated with better health,⁷² even for individuals with substance use disorders⁷³ or moderate⁷⁴ to severe mental health disorders.^75-78 Some authors have even proposed that, above and beyond the associated health benefits, employment may also help to realize a modest cost savings due to reduced service utilization and disability.^79,80 Employment is a core tenet in the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) model of recovery,⁸¹ and is also listed as an important recovery goal for individuals with behavioral health issues.⁸² BHD collects data on employment status on all the patients it serves as part of its state-mandated reporting requirements and will use this item in the CD data set.⁸³

Living Situation. This is measured as the proportion of people who live in permanent, supportive, stable housing; it may also be measured as the percentage of the population living with severe housing problems or who are homeless.

Brief review and suggested item(s). Housing problems can be conceptualized as 3 inter-related components: conditions within the home, neighborhood conditions, and housing affordability, each of which can contribute uniquely to poorer physical and mental health of individuals and families⁸⁴ and to educational outcomes for children.^85,86 Further, individuals who are homeless have a standardized mortality ratio 2 to 5 times that of the general population,^87-89 even after controlling for low income status,⁹⁰ and some evidence suggests these rates are even higher among unsheltered versus sheltered homeless individuals.⁹¹ Interventions to improve the condition of housing have demonstrated positive impacts on both physical and mental health,⁹² and a recent study found that individuals receiving housing assistance in the form of public housing or multifamily housing from the Department of Housing and Urban Development had better self-rated physical and mental health relative to individuals on the wait list for housing assistance.⁹³ Moreover, the provision of housing has been shown to promote reductions in substance use and health service utilization among homeless individuals with substance use disorders.⁹⁴ Rog and colleagues reviewed the literature on permanent supportive housing for individuals with substance use or mental health disorders who were homeless or disabled, and found that provision of housing led to reduced rates of homelessness, emergency department (ED) and inpatient utilization and increased consumer satisfaction.⁹⁵

Importantly, evidence suggests that housing is viewed as facilitative of recovery. For example, in a recent qualitative study of homeless individuals with mental illness, housing was seen as a critical first step in recovery, providing a sense of security, increasing feelings of personal independence and autonomy, improving perceptions of health and well-being, and affording a stable environment to rebuild relationships with important others.⁹⁶ BHD collects data on housing status on all the patients it serves as part of its state-mandated reporting requirements and will utilize this item in the CD data set.⁸³

Social Relationships. This is defined as recent interactions with family, supportive networks (formal and informal), and other recovery services.

Brief review and suggested item(s). Research has long established that social relationships have a significant impact on health, including rates of mortality as well as physical and mental health morbidity.^97-99 Social connectedness is another of the pillars supporting an individual’s recovery in SAMHSA’s formulation. Several reviews of the recovery literature^38,82 support its importance to the recovery process and inclusion in any assessment of holistic recovery. Social support has been shown to promote recovery among individuals with severe mental illness^100-102 and substance use disorders,¹⁰³ and may mitigate the progression of chronic, life-threatening physical illnesses.⁹⁷ For the purposes of BHD’s CD data set, the social support question from the “100 Million Healthier Lives Common Questionnaire for Adults” will be used to assess individuals’ perceived adequacy of social support.¹⁰⁴

Legal Involvement. Defined as involvement with the civil or criminal justice system, including arrests, imprisonment, or detainment.

Brief review and suggested item(s). Involvement in the criminal justice system is both disruptive for the individual in recovery and expensive to the larger health care system.¹⁰⁵ Individuals with substance use¹⁰⁶ and severe mental health disorders¹⁰⁷ are over-represented in the prison system, and evidence suggests that general physical and mental health declines while individuals are in prison.^108,109 Perhaps even more concerning, numerous studies have demonstrated an increase in mortality rates for individuals recently released from prison relative to the general population, particularly during the period immediately following release.^108-110 This relationship may even persist long term.¹¹¹ Further, research indicates that individuals recently released from prison have increased emergency care and hospital utilization.^112,113

Incarceration can have significant impacts on the health of the broader community as well. For example, research has found an association between parental incarceration to rates of infant mortality,¹¹⁴ increased behavioral and developmental problems of children of incarcerated parents,^115,116 lower rates of child support payments,¹¹⁷ and poorer cardiovascular health of female partners of incarcerated individuals.¹¹⁸ Formerly incarcerated individuals experience slower wage growth as well.¹¹⁹ However, evidence also indicates that engagement in mental health¹²⁰ and substance abuse¹²¹ treatment can reduce the likelihood of subsequent recidivism. As part of its state-mandated reporting, BHD is required to provide information on the criminal justice system involvement of its clients in the previous 6 months, including whether they have been jailed or imprisoned,⁸³ and this will function as its measure of legal involvement in its CD data set.

Socioeconomic Status. Socioeconomic status is the social standing or class of an individual or group. It is often measured as a combination of education, income, and occupation. It can also be defined subjectively, such as one’s evaluation of status relative to similar others or based on an individual’s interpretation of her or his financial needs.

Brief review and suggested item(s). A large body of evidence supports the existence of a robust relationship between lower SES and poor health, including mortality and chronic medical diseases,^122-124 as well as mental illness.^125-127 Although previous research has examined this relationship using objective indicators of SES (eg, income, education level, occupation), there has recently been an increased interest in exploring the relationship of subjective SES with health indices. Subjective SES is generally assessed by asking individuals to rate themselves relative to others in the society in which they live, in terms of wealth, occupation, educational level, or other indicators of social status. Evidence suggests that subjective SES is associated with objective measures of SES,^128-130 and relates to measures of physical and mental health as well, even after controlling for objective SES.^130-135 BHD will be using a modified version of the Subject SES Scale,^131,135 which is deployed in the “100 Million Healthier Lives Common Questionnaire for Adults.”¹⁰⁴

Acute Service Use. This is defined as an admission to a medical or psychiatric emergency room or to a medical or psychiatric hospital or to a detoxification facility.

Brief review and suggested item(s). The CMS Adult Core Set includes “plan all cause readmissions” as a key quality metric.⁵⁵ Hospital readmissions are also endorsed by the National Committee on Quality Assurance as one of its Health Effectiveness Data and Information Set (HEDIS) measures and by the National Quality Forum. Readmissions, despite their widespread endorsement, are a somewhat controversial measure. Although readmissions are costly to the health care system,¹³⁶ the relationship between readmissions and quality is inconsistent. For example, Krumholz and colleagues¹³⁷ found differential rates of readmission for the same patient discharged from 2 different hospitals, which were categorized based on previous readmission rates, suggesting that hospitals do have different levels of performance even when treating the same patient. However, other data indicate that 30-day, all-cause, risk-standardized readmission rates are not associated with hospital 30-day, all-cause, risk-standardized mortality rates.¹³⁸

Chin and colleague found that readmissions to the hospital that occurred more than 7 days post-discharge were likely due to community- and household-related factors, rather than hospital-related quality factors.¹³⁹ Transitional care interventions that have successfully reduced 30-day readmission rates are most often multicomponent and focus not just on hospital-based interventions (eg, discharge planning, education) but on follow-up care in the community by formal supports (eg, in-home visits, telephone calls, outpatient clinic appointments, case management) and informal supports (eg, family and friends).^140-143 Further, qualitative evidence suggests that some individuals perceive psychiatric hospitalizations to be the result of insufficient resources or unsuccessful attempts to maintain their stability in the community.¹⁴⁴ Thus, unplanned or avoidable hospital readmissions may represent a failure of the continuum of care not only from the perspective of the health care system, but from the patient perspective as well.

Frequent or nonurgent use of EDs is conceptually similar to excessive or avoidable inpatient utilization in several ways. For example, overuse of EDs is costly, with some estimates suggesting that it is responsible for up to $38 billion in wasteful spending each year.¹⁴⁵ Individuals with frequent ED visits have a greater disease burden¹⁴⁶ and an increased risk of mortality compared to nonfrequent users.¹⁴⁷ Research suggests that individuals who visit the ED for non-urgent issues do so because of perceived difficulties associated with accessing primary care, and the convenience of EDs relative to primary care.^148-150 Moreover, similar to the hospital readmission literature discussed earlier, successful strategies to reduce high rates of ED utilization generally focus on continuum of care interventions, such as provision of case management services.^151-155

This evidence implies that frequent ED utilization and hospital readmissions may not be a fundamental issue of quality (or lack thereof) in hospitals or EDs but rather a lack of, or ineffectual, transitional and continuum of care strategies and services. To underscore this point, some authors have argued that a system that is excessively crisis-oriented hinders recovery because it is reactive rather than proactive, predicated on the notion that one’s condition must deteriorate to receive care.¹⁵⁶

Although some organizations may have access to claims data or may function as self-contained health systems (eg, the Veterans Health Administration [VHA] ), others may not have access to such data. In the absence of claims data, patient self-report of service utilization has been used as a proxy for actual agency records.¹⁵⁷ Although concordance between medical and/or agency records and patient self-report has been variable,¹⁵⁷ evidence generally suggests that rates of agreement are higher the shorter the recall time interval.^158,159 BHD does not have access to comprehensive claims data and has therefore chosen to use 5 dichotomously scored (yes/no) questions—related to medical inpatient, medical ED, psychiatric inpatient, psychiatric ED, and detoxification use in the last 30 days—to represent the CD of acute service utilization.

The Second Aim: Quality of Care

Safety

Safety is defined as avoiding injuries to patients from the care that is intended to help them.

Brief review and suggested item(s). As noted in “Crossing the Quality Chasm,” the IOM’s seminal document, “the health care environment should be safe for all patients, in all of its processes, all the time.”¹⁶⁰ The landmark Harvard Medical Practice Study in 1991 found that adverse events occurred in nearly 4% of all hospital admissions and, among these, over a quarter were due to negligence.¹⁶¹ Other estimates of adverse events range as high as 17%.¹⁶² Indeed, a recent article by Makary and Daniel estimated that medical errors may be the third leading cause of death in the United States.¹⁶³ Unfortunately, research on safety in the mental health field has lagged behind that of physical health,¹⁶⁴ with evidence indicating that research in nonhospital settings in mental health care may be particularly scarce.¹⁶⁵ In a study of adverse events that occurred in psychiatric inpatient units in the VHA system between 2015 and 2016, Mills and colleagues found that of the 87 root cause analysis reports, suicide attempts were the most frequent, and, among safety events, falls were the most frequently reported, followed by medication events.¹⁶⁶ Another report on data collected from psychiatric inpatient units in the VHA revealed that nearly one-fifth of patients experienced a safety event, over half of which were deemed preventable.¹⁶⁷ These numbers likely represent an underestimation of the true volume of safety events, as another study by the same research group found that less than 40% of safety events described in patient medical records were documented in the incident reporting system.¹⁶⁸ BHD will utilize the total number of complaints and incident reports submitted within a given time frame as its “safety” metric in the CD data set.

Wait Time for Service

The CD is defined as the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Brief review and suggested item(s). “Timeliness” was listed among the 6 aims for improvement in “Crossing the Quality Chasm” in 2001, and it remains no less relevant today.¹⁶⁰ For example, evidence indicates that access to primary care is inversely related to avoidable hospitalizations.¹⁶⁹ One study found that, of patients hospitalized for cardiovascular problems, those who had difficulty accessing routine care post discharge had higher 30-day readmission rates.¹⁷⁰ Among VHA patients, longer wait times are associated with more avoidable hospitalizations and higher rates of mortality.¹⁷¹ Longer wait times appear to decrease the likelihood of attending a first appointment for individuals with substance use^172,173 and mental health disorders.¹⁷⁴ Importantly, longer wait times are associated with lower ratings of the patient experience of care, including perceptions of the quality of and satisfaction with care,¹⁷⁵ and may be associated with worse outcomes for individuals in early intervention for psychosis treatment.¹⁷⁶ For the purposes of the CD data set, BHD will monitor the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Patient satisfaction is defined as the degree of patients’ satisfaction with the care they have received.

Brief review and suggested item(s). Research has consistently demonstrated the relationship of the patient’s experience of care to a variety of safety and clinical effectiveness measures in medical health care,¹⁷⁷ and the therapeutic alliance is one of the most consistent predictors of outcomes in behavioral health, regardless of therapeutic modality.¹⁷⁸ Patient satisfaction is a commonly assessed aspect of the patient experience of care. Patient satisfaction scores have been correlated with patient adherence to recommended treatment regimens, care quality, and health outcomes.¹⁷⁹ For example, Aiken et al found that patient satisfaction with hospital care was associated with higher ratings of the quality and safety of nursing care in these hospitals.¹⁸⁰ Increased satisfaction with inpatient care has been associated with lower 30-day readmission rates for patients with acute myocardial infarction, heart failure, and pneumonia,¹⁸¹ and patients with schizophrenia who reported higher treatment satisfaction also reported better quality of life.^182,183 Many satisfaction survey options exist to evaluate this CD, including the Consumer Assessment of Healthcare Providers and Systems and the Client Satisfaction Questionnaire; BHD will utilize an outpatient behavioral health survey from a third-party vendor.

The Third Aim: Cost of Care

Cost of Care

This can be defined as the average cost to provide care per patient per month.

Brief review and suggested item(s). Per capita cost, or rather, the total cost of providing care to a circumscribed population divided by the total population, has been espoused as an important metric for the Triple Aim and the County Health Rankings.^6,13 Indeed, between 1960 and 2016, per capita expenditures for health care have grown 70-fold, and the percent of the national gross domestic product accounted for by health expenditures has more than tripled (5.0% to 17.9%).¹⁸⁴ One of the more common metrics deployed for assessing health care cost is the per capita per month cost, or rather, the per member per month cost of the predefined population for a given health care system.^6,185,186 In fact, some authors have proposed that cost of care can be used not only to track efficient resource allocation, but can also be a proxy for a healthier population as well (ie, as health improves, individuals use fewer and less-expensive services, thus costing the system less).¹⁸⁷ To assess this metric, BHD will calculate the total amount billed for patient care provided within BHD’s health network each month (irrespective of funding source) and then divide this sum by the number of members served each month. Although this measure does not account for care received at other health care facilities outside BHD’s provider network, nor does it include all the overhead costs associated with the care provided by BHD itself, it is consistent with the claims-based approach used or recommended by other authors.^6,188

The Fourth Aim: Staff Well-being

Staff Quality of Work Life

This can be defined as the quality of the work life of health care clinicians and staff.

Brief review and suggested item(s). Some authors have suggested that the Triple Aim framework is incomplete and have proffered compelling arguments that provider well-being and the quality of work life constitutes a fourth aim.² Provider burnout is prevalent in both medical^2,189 and behavioral health care.^190,191 Burnout among health care professionals has been associated with higher rates of perceived medical errors,¹⁹² lower patient satisfaction scores,^189,193 lower rates of provider empathy,¹⁹⁴ more negative attitudes towards patients,¹⁹⁵ and poorer staff mental and physical health.¹⁹¹

Burnout is also associated with higher rates of absenteeism, turnover intentions, and turnover.^{190,191,196,197} However, burnout is not the only predictor of staff turnover; for example, turnover rates are a useful proxy for staff quality of work life for several reasons.¹⁹⁸ First, turnover is associated with substantial direct and indirect costs, including lost productivity, increased errors, and lost revenue and recruitment costs, with some turnover cost estimates as high as $17 billion for physicians and $14 billion for nurses annually.^199-201 Second, research indicates that staff turnover can have a deleterious impact on implementation of evidence-based interventions.^202-205 Finally, consistent with the philosophy of utilizing existing data sources for the CD measures, turnover can be relatively easily extracted from administrative data for operated or contracted programs, and its collection does not place any additional burden on staff. As a large behavioral health system that is both a provider and payer of care, BHD will therefore examine the turnover rates of its internal administrative and clinical staff as well as the turnover of staff in its contracted provider network as its measures for the Staff Quality of Work Life CD.

Clinical Implications

These metrics can be deployed at any level of the organization. Clinicians may use 1 or more of the measures to track the recovery of individual clients, or in aggregate for their entire caseload. Similarly, managers can use these measures to assess the overall effectiveness of the programs for which they are responsible. Executive leaders can evaluate the impact of several programs or the system of care on the health of a subpopulation of clients with a specific condition, or for all their enrolled members. Further, not all measures need be utilized for every dashboard or evaluative effort. The benefit of a comprehensive set of measures lies in their flexibility—1 or more of the measures may be selected depending on the project being implemented or the interests of the stakeholder.

It is important to note that many of the CDs (and their accompanying measures) are aligned to/consistent with social determinants of health.^206,207 Evidence suggests that social determinants make substantial contributions to the overall health of individuals and populations and may even account for a greater proportion of variance in health outcomes than health care itself.²⁰⁸ The measures articulated here, therefore, can be used to assess whether and how effectively care provision has addressed these social determinants, as well as the relative impact their resolution may have on other health outcomes (eg, mortality, self-rated health).

These measures can also be used to stratify clients by clinical severity or degree of socioeconomic deprivation. The ability to adjust for risk has many applications in health care, particularly when organizations are attempting to implement value-based purchasing models, such as pay-for-performance contracts or other alternative payment models (population health-based payment models).²⁰⁹ Indeed, once fully implemented, the CDs and measures will enable BHD to more effectively build and execute different conceptual models of “value” (see references 210 and 211 for examples). We will be able to assess the progress our clients have made in care, the cost associated with that degree of improvement, the experience of those clients receiving that care, and the clinical and social variables that may influence the relative degree of improvement (or lack thereof). Thus, the CDs provide a conceptual and data-driven foundation for the Quadruple Aim and any quality initiatives that either catalyze or augment its implementation.

This article provides an overview of the CDs selected by BHD to help organize, focus, advance, and track its quality efforts within the framework of the Quadruple Aim. Although items aligned to each of these CDs are offered, the CDs themselves have been broadly conceptualized such that they can flexibly admit a variety of possible items and/or assessments to operationalize each CD and thus have potential applicability to other behavioral health systems, particularly public systems that have state-mandated and other data reporting requirements.

Bearing in mind the burden that growing data collection requirements can have on the provision of quality care and staff work satisfaction and burnout,^10,212 the CDs (and the items selected to represent each) are designed with “strategic parsimony” in mind. Although the CDs are inclusive in that they cover care quality, cost of care, staff quality of life, and general population health, only CDs and items undergirded by a solid evidence base and high value with regards to BHD’s mission and values, as determined by key stakeholders, were selected. Moreover, BHD attempted to make use of existing data collection and reporting mandates when selecting the final pool of items to reduce the measurement burden on staff and clients. Thus, the final set of CDs and items are designed to be comprehensive yet economical.

The CDs are deeply interrelated. Although each CD may be individually viewed as a valuable metric, improvements in any 1 CD will impact the others (eg, increasing care quality should impact population health, increasing staff quality of life should impact the quality of care). Moreover, this idea of interrelatedness acknowledges the need to view health systems and the populations they serve holistically, in that improvement is not simply the degree of change in any given metric (whether individually or collectively), but rather something more entirely. The concepts of value, quality, and health are complex, multidimensional, and dynamic, and the CDs that comprise these concepts should not be considered independently from one another. The CDs (and items) offered in this article are scalable in that they can be used at different levels of an organization depending on the question or stakeholder, and can be used individually or in combination with one another. Moreover, they are adaptable to a variety of risk-adjusted program, population health, and value-based evaluation models. It is hoped that the process articulated here, and the accompanying literature review, may benefit other public or government-run health systems in their own quality journey to operationalize the Quadruple Aim by developing a set of CDs.

Corresponding author: Walter Matthew Drymalski, PhD; [email protected].

Financial disclosures: None.

From the Milwaukee County Behavioral Health Division, Milwaukee, WI.

Abstract

Objective: Implementation of the Quadruple Aim of health care must begin with a clearly articulated set of concepts, or core domains (CDs), that comprise each aim. These CDs can then be operationalized with existing or new measures. If aligned to the organization’s mission and strategic goals, these CDs have the potential to focus quality improvement activities and reduce measurement burden. This article represents the efforts of a publicly funded behavioral health system to operationalize the Quadruple Aim through the development of CDs.

Methods: Various stakeholders across the organization were consulted on their perceptions of the Quadruple Aim and the CDs they believed should support it. Then, a review of existing literature on core metrics for health care and population health was completed, summarized, and integrated with the stakeholder feedback.

Results: These efforts led to the development and adoption of 15 CDs, with an accompanying literature review and set of recommendations of new and existing measures for each domain.

Conclusions: It is possible to create a comprehensive yet economical set of CDs and attendant measures that can be implemented in a staged, scalable, enterprise manner. It is hoped that the process articulated here, and the accompanying literature review, may be of some benefit to other public or government-run health systems in their own quality improvement journey to operationalize the Quadruple Aim by developing a set of CDs.

Keywords: quality measures; quality improvement; adult behavioral health.

First articulated in 2008, the Triple Aim proposes that health care systems should simultaneously seek to improve the patient’s experience of care, improve the health of populations, and reduce the per capita costs of care for populations.¹ More recently, some have argued that health care provider burnout can deleteriously impact the attainment of the Triple Aim and have therefore advocated for an expanded focus to include a fourth Aim, the work life quality of the staff.² Milwaukee County Behavioral Health Division (BHD), a publicly funded, county-based behavioral health care system in Milwaukee, Wisconsin, recently adopted the Quadruple Aim as the framework by which it will organize its quality activities.

Although originally developed for medical organizations, BHD believes that the Quadruple Aim has strong applicability to county-level behavioral health services. Many county-based behavioral health divisions provide a variety of programs to large segments of the county based on financial eligibility and/or clinical need, and thus often have responsibilities to populations or subpopulations, rather than programs. County health divisions, such as Milwaukee County’s Department of Health and Human Services, are often asked to improve outcomes and client experience of care with neutral growth budgets and less reliance on taxes to fund programs, while simultaneously attracting and retaining competent staff.

Crucial to the effective implementation of the Quadruple Aim, however, is a clear set of population- level measures that help organizations assess their progress.³ Unfortunately, as some authors have noted, evaluation of the Quadruple Aim remains a challenge because the “concepts of (population) health, quality of care and costs are not unanimously defined and measures for these concepts are under construction.”⁴ Several authors have provided some guidance to assist in the development of a set of measures that effectively capture the elements of the Quadruple Aim.^5,6 However, the recent rapid proliferation of quality measures in health care^7,8 has been both burdensome and costly for providers.^9,10 Any measures adopted should not only be as meaningful as possible with regards to assessing progress towards the basic aims of health care, but should also be parsimonious, to limit measurement burden for providers (and patients) and focus attention on important issues.^11,12

To select the most effective, parsimonious set of measures possible, one must first select a set of key foci from among the many possible areas of focus that the core measure is intended to represent. The core domains (CDs), if appropriately consistent with the strategic goals of the organization, provide a mechanism to orient the efforts of the organization at every level and help every staff member of the organization understand how his or her work impacts the progress towards these goals.¹¹ The CDs, therefore, represent the opportunity to affect a greater integration of efforts across the organization toward these shared aims, creating uniformity of purpose at every level. Further, increasing organizational attention on the CDs can also help to reduce measurement burden by streamlining and focusing the data capture processes on the most valuable elements of quality and health, and discarding other extraneous measures (albeit not at the expense of other reporting requirements).¹¹ The remainder of this article describes the CDs selected by BHD to assess its progress toward implementation of the Quadruple Aim and are organized by the Aim which they best represent.

Methods

To effectively implement the Quadruple Aim at BHD, it was necessary to clearly define the subpopulation of focus for our efforts.⁶ In this case, the subpopulation of interest was defined as all adult clients (18 years and older) who received at least 1 service encounter within a specified time frame from a program that BHD either operated or contracted with to provide care. Services provided by the BHD network include everything from psychiatric inpatient services to mental health and addiction treatment and care management. A limited array of social services, including housing and employment services, is also available to eligible consumers. BHD is the county-run behavioral health provider for individuals who are uninsured or underinsured in Milwaukee County, a demographically diverse, primarily urban county of approximately 950,000 people located in Wisconsin. Approximately 15,000 adults receive services at BHD each year.

This work began by obtaining executive sponsorship for the project, in this case from the Chief Operations Officer and Executive Medical Director of BHD. With their backing, an initial review of the literature produced a preliminary set of possible domains, for which we created working definitions. We then made a list of key stakeholders throughout BHD to whom we needed to present the idea of the Quadruple Aim, and the CDs under each Aim, to secure their support. These stakeholders, which included individuals involved in quality activities, program managers, and executive leadership, were strategically selected based on their relative influence within the organization. A set of brief presentations and handouts explaining the project were then developed and shared at different focus groups with these stakeholders over the course of 6 months. These focus groups served to not only educate the organization about the Quadruple Aim and the CDs but afforded participants an opportunity to provide feedback as well.

During the focus groups, we asked participants which domains they believed were most important (were “core”) when operationalizing the Quadruple Aim. The focus groups provided feedback on the domain definitions, feedback that was used to develop uniform, mutually agreed upon definitions for the CDs that were generalizable to all departments at BHD, regardless of the focus of their services within the continuum of care or the continuum of age. This was a crucial step, as it will eventually enable BHD to aggregate data across departments, even if there are minor discrepancies in the specific items they use to assess the CDs. Comments from the focus groups ultimately resulted in a truncated list of domains and definitions, which, coupled with the literature review, resulted in our final set of CDs.

During our review of the literature, we also looked for items that we felt could best represent each CD in the briefest, most meaningful way. (These items were not meant to supersede existing data, but to provide examples that could be implemented with existing data or recommendations that could be utilized in the absence of existing data.) During this process, we made every effort to make use of existing data-reporting requirements. For example, if we had a state mandate to collect data on housing status, we attempted to leverage this required data point to represent the CD related to housing. In other cases, we attempted to utilize claims or other administrative data to operationalize the CD, such as in the cost-of-care metric articulated in the section the Third Aim. For CDs for which no data existed or were insufficient, we emphasized the use of single- versus multi-item scales. For example, if we found a single-item global assessment of quality of life that had good psychometric properties relative to its longer parent scale, we selected the single item. This approach to item selection allowed us to create the most efficient, parsimonious set of measures possible, which we believed would enable us to comprehensively assess all the CDs with the least amount of burden to staff and clients. These items were presented at stakeholder focus groups, during which we asked for comments on the existing measures in their program or department and gave them the opportunity to comment on the new recommended measures.

A working definition is provided for each CD, followed by a brief review of the research base supporting its inclusion in the final list. The item(s) selected by BHD to represent each CD and the source of the item(s) are then supplied. These items were based either on measures currently collected because of existing reporting mandates or, in the case where extant measures were not available, on new items that demonstrated acceptable psychometric properties in the research literature. The CDs and items are organized by the Aim they best represent. A full list of the CDs by Quadruple Aim and items by CD is provided in the Appendix of the online version of this article. This article concludes with a brief summary of this effort and a discussion of how staff will utilize these items at different levels throughout the BHD system.

The First Aim: Population Health

Health Outcomes

Deaths. This can be defined as the cause of death, as determined by the medical examiner’s office (where appropriate) or as the age at time of death. This CD can also be reported as proportion of deaths considered premature (eg, before age 75) or calculated as total years of potential life lost.

Brief review and suggested item(s). Rates and causes of premature mortality are critical foci for the County Health Rankings & Roadmaps,¹³ the Institute for Healthcare Improvement’s “Guide to Measuring the Triple Aim,”⁶ the Centers for Disease Control and Prevention’s “Community Health Assessment for Population Health Improvement,”¹⁴ and the Institute of Medicine’s (IOM) “Vital Signs: Core Metrics for Health and Health Care Progress.”¹¹ There is ample evidence that individuals with serious mental illness are at increased risk of early mortality relative to the general population,^15-18 and this risk applies to those with substance use disorders as well.^15,19-20 BHD tracks all deaths that occur while patients are receiving BHD-funded, community-based services.

Self-Reported Health and Well-Being. This CD asks patients to rate their current physical and mental health status, as well as their overall quality of life.

Brief review and suggested item(s): Self-rated physical health. Premature mortality among individuals with behavioral health issues appears to be due, in large part, to their increased vulnerability to the development of medical comorbidities.^16,21 A single self-rating question has demonstrated considerable sensitivity to premature mortality,^22,23 with predictive properties up to a decade prior to death.^24,25 Further, self-rated health has been associated with subsequent functional decline,^26,27 acute service utilization,^28,29 and overall health care costs.²⁸

Brief review and suggested item(s): Self-rated mental health. Mental health disorders are associated with significant disability worldwide,³⁰ and comorbid mental health issues can exacerbate the course of other medical problems. For example, depression is associated with increased rates of mortality among individuals with diabetes and³¹ cardiovascular disease,³² as well as with rates of overall mortality,³³ and psychiatric comorbidity is associated with longer lengths of stay and higher costs among patients hospitalized for medical problems.³⁴ Research has found that a single-item measure of self-rated mental health is associated with the presence of psychiatric diagnoses, psychiatric symptoms, and subsequent depression and serious mental illness up to 1 year post-assessment.^35,36 There is even evidence that self-rated mental health may be more strongly associated with self-ratings of overall health than self-ratings of physical health.³⁷

Brief review and suggested item(s): Self-rated quality of life. Quality of life is a critical component of the recovery journey and overall health.³⁸ For example, the County Health Rankings & Roadmaps lists “quality of life” as 1 of its key “health outcomes” in its County Health Rankings.¹³ As some authors have noted, quality of life is often inferred from other “objective” recovery domains, such as employment, health status, or housing status. However, there is evidence that these objective domains are functionally distinct from the inherently subjective construct of quality of life.³⁹ This has led other authors to conclude that these domains should be assessed separately when evaluating outcomes.⁴⁰ Single-item quality of life assessments have been used in research with individuals with cancer,⁴¹ adults with disabilities,⁴² patients with cystic fibrosis,⁴³ and children with epilepsy.⁴⁴ For this effort, BHD selected the first global quality of life item from the World Health Organization’s WHOQOL-BREF quality of life assessment,⁴⁵ an item used in other quality of life research.⁴⁶

Health Factors

Substance Use. This CD is a composite of 4 different types of substance use, any recent heavy alcohol use (defined as 5 or more drinks in one sitting), any recent drug use, any recent prescription drug abuse, and any recent tobacco use.

Brief review and suggested item(s). As noted, substance use disorders confer an increased risk for early mortality^15,19 and are significantly implicated in disease disability burden worldwide.³⁰ Substance use has also been associated with both the onset^47,48 and exacerbation of mental health diagnoses.^49-51 Further, substance use appears to heighten the risk of violence in the general population⁵² and especially among those with a co-occurring mental illness.^53,54 The County Health Rankings & Roadmaps list alcohol and drug use as key behaviors to address to improve the overall health of a given county,¹³ and the Centers for Medicare & Medicaid Services (CMS) has endorsed initiation and engagement in addiction treatment as one of the measures in its Adult Core Set.⁵⁵

Tobacco use continues to be one of the most significant risk factors for early mortality worldwide, and evidence indicates that it is associated with a lower life expectancy of nearly 10 years.⁵⁶ Unfortunately, rates of tobacco use are even higher among those with severe mental illness relative to the general population, and their rates of smoking cessation are lower.^57,58 Tobacco use is a significant risk factor for the high rates of early mortality in individuals with severe mental illness.¹⁸ Further, a recent meta-analysis noted that, relative to those who continued to smoke, those who ceased smoking had reduced rates of psychological distress and increased quality of life rankings.⁵⁹ Reducing tobacco use is one of the key components of the County Health Rankings & Roadmaps, and medication assistance with smoking and tobacco use cessation is also listed in the CMS Adult Core Set.^13,55

An accumulating body of evidence suggests that single-item measures can adequately detect alcohol^60-62 and drug use disorders.^60-64 McNeely and colleagues recently developed and tested a brief 4-item screen, the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool.^65,66 Preliminary evidence suggests that the TAPS tool can effectively identify the presence of problematic and disordered use of tobacco, alcohol, prescription medications, and other drugs.^65-67 BHD will use the 4 items from the TAPS tool to represent its substance use CD.

Education/Employment Status. This CD assesses the proportion of BHD members who have completed high school, who are in some type of educational or training program, or who are engaged in some type of employment activity (defined as full-time, part-time, supported, sheltered workshop, or as a full-time homemaker).

Brief review and suggested item(s). Research indicates that unemployment is a risk factor for mortality, even after controlling for other risk factors (eg, age, sex, socioeconomic status [SES], health).⁶⁸ Unemployment is associated with poorer physical and mental health in the general population and among those with disabilities.^69-71 Promisingly, evidence suggests that gaining employment or re-employment is associated with better health,⁷² even for individuals with substance use disorders⁷³ or moderate⁷⁴ to severe mental health disorders.^75-78 Some authors have even proposed that, above and beyond the associated health benefits, employment may also help to realize a modest cost savings due to reduced service utilization and disability.^79,80 Employment is a core tenet in the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) model of recovery,⁸¹ and is also listed as an important recovery goal for individuals with behavioral health issues.⁸² BHD collects data on employment status on all the patients it serves as part of its state-mandated reporting requirements and will use this item in the CD data set.⁸³

Living Situation. This is measured as the proportion of people who live in permanent, supportive, stable housing; it may also be measured as the percentage of the population living with severe housing problems or who are homeless.

Brief review and suggested item(s). Housing problems can be conceptualized as 3 inter-related components: conditions within the home, neighborhood conditions, and housing affordability, each of which can contribute uniquely to poorer physical and mental health of individuals and families⁸⁴ and to educational outcomes for children.^85,86 Further, individuals who are homeless have a standardized mortality ratio 2 to 5 times that of the general population,^87-89 even after controlling for low income status,⁹⁰ and some evidence suggests these rates are even higher among unsheltered versus sheltered homeless individuals.⁹¹ Interventions to improve the condition of housing have demonstrated positive impacts on both physical and mental health,⁹² and a recent study found that individuals receiving housing assistance in the form of public housing or multifamily housing from the Department of Housing and Urban Development had better self-rated physical and mental health relative to individuals on the wait list for housing assistance.⁹³ Moreover, the provision of housing has been shown to promote reductions in substance use and health service utilization among homeless individuals with substance use disorders.⁹⁴ Rog and colleagues reviewed the literature on permanent supportive housing for individuals with substance use or mental health disorders who were homeless or disabled, and found that provision of housing led to reduced rates of homelessness, emergency department (ED) and inpatient utilization and increased consumer satisfaction.⁹⁵

Importantly, evidence suggests that housing is viewed as facilitative of recovery. For example, in a recent qualitative study of homeless individuals with mental illness, housing was seen as a critical first step in recovery, providing a sense of security, increasing feelings of personal independence and autonomy, improving perceptions of health and well-being, and affording a stable environment to rebuild relationships with important others.⁹⁶ BHD collects data on housing status on all the patients it serves as part of its state-mandated reporting requirements and will utilize this item in the CD data set.⁸³

Social Relationships. This is defined as recent interactions with family, supportive networks (formal and informal), and other recovery services.

Brief review and suggested item(s). Research has long established that social relationships have a significant impact on health, including rates of mortality as well as physical and mental health morbidity.^97-99 Social connectedness is another of the pillars supporting an individual’s recovery in SAMHSA’s formulation. Several reviews of the recovery literature^38,82 support its importance to the recovery process and inclusion in any assessment of holistic recovery. Social support has been shown to promote recovery among individuals with severe mental illness^100-102 and substance use disorders,¹⁰³ and may mitigate the progression of chronic, life-threatening physical illnesses.⁹⁷ For the purposes of BHD’s CD data set, the social support question from the “100 Million Healthier Lives Common Questionnaire for Adults” will be used to assess individuals’ perceived adequacy of social support.¹⁰⁴

Legal Involvement. Defined as involvement with the civil or criminal justice system, including arrests, imprisonment, or detainment.

Brief review and suggested item(s). Involvement in the criminal justice system is both disruptive for the individual in recovery and expensive to the larger health care system.¹⁰⁵ Individuals with substance use¹⁰⁶ and severe mental health disorders¹⁰⁷ are over-represented in the prison system, and evidence suggests that general physical and mental health declines while individuals are in prison.^108,109 Perhaps even more concerning, numerous studies have demonstrated an increase in mortality rates for individuals recently released from prison relative to the general population, particularly during the period immediately following release.^108-110 This relationship may even persist long term.¹¹¹ Further, research indicates that individuals recently released from prison have increased emergency care and hospital utilization.^112,113

Incarceration can have significant impacts on the health of the broader community as well. For example, research has found an association between parental incarceration to rates of infant mortality,¹¹⁴ increased behavioral and developmental problems of children of incarcerated parents,^115,116 lower rates of child support payments,¹¹⁷ and poorer cardiovascular health of female partners of incarcerated individuals.¹¹⁸ Formerly incarcerated individuals experience slower wage growth as well.¹¹⁹ However, evidence also indicates that engagement in mental health¹²⁰ and substance abuse¹²¹ treatment can reduce the likelihood of subsequent recidivism. As part of its state-mandated reporting, BHD is required to provide information on the criminal justice system involvement of its clients in the previous 6 months, including whether they have been jailed or imprisoned,⁸³ and this will function as its measure of legal involvement in its CD data set.

Socioeconomic Status. Socioeconomic status is the social standing or class of an individual or group. It is often measured as a combination of education, income, and occupation. It can also be defined subjectively, such as one’s evaluation of status relative to similar others or based on an individual’s interpretation of her or his financial needs.

Brief review and suggested item(s). A large body of evidence supports the existence of a robust relationship between lower SES and poor health, including mortality and chronic medical diseases,^122-124 as well as mental illness.^125-127 Although previous research has examined this relationship using objective indicators of SES (eg, income, education level, occupation), there has recently been an increased interest in exploring the relationship of subjective SES with health indices. Subjective SES is generally assessed by asking individuals to rate themselves relative to others in the society in which they live, in terms of wealth, occupation, educational level, or other indicators of social status. Evidence suggests that subjective SES is associated with objective measures of SES,^128-130 and relates to measures of physical and mental health as well, even after controlling for objective SES.^130-135 BHD will be using a modified version of the Subject SES Scale,^131,135 which is deployed in the “100 Million Healthier Lives Common Questionnaire for Adults.”¹⁰⁴

Acute Service Use. This is defined as an admission to a medical or psychiatric emergency room or to a medical or psychiatric hospital or to a detoxification facility.

Brief review and suggested item(s). The CMS Adult Core Set includes “plan all cause readmissions” as a key quality metric.⁵⁵ Hospital readmissions are also endorsed by the National Committee on Quality Assurance as one of its Health Effectiveness Data and Information Set (HEDIS) measures and by the National Quality Forum. Readmissions, despite their widespread endorsement, are a somewhat controversial measure. Although readmissions are costly to the health care system,¹³⁶ the relationship between readmissions and quality is inconsistent. For example, Krumholz and colleagues¹³⁷ found differential rates of readmission for the same patient discharged from 2 different hospitals, which were categorized based on previous readmission rates, suggesting that hospitals do have different levels of performance even when treating the same patient. However, other data indicate that 30-day, all-cause, risk-standardized readmission rates are not associated with hospital 30-day, all-cause, risk-standardized mortality rates.¹³⁸

Chin and colleague found that readmissions to the hospital that occurred more than 7 days post-discharge were likely due to community- and household-related factors, rather than hospital-related quality factors.¹³⁹ Transitional care interventions that have successfully reduced 30-day readmission rates are most often multicomponent and focus not just on hospital-based interventions (eg, discharge planning, education) but on follow-up care in the community by formal supports (eg, in-home visits, telephone calls, outpatient clinic appointments, case management) and informal supports (eg, family and friends).^140-143 Further, qualitative evidence suggests that some individuals perceive psychiatric hospitalizations to be the result of insufficient resources or unsuccessful attempts to maintain their stability in the community.¹⁴⁴ Thus, unplanned or avoidable hospital readmissions may represent a failure of the continuum of care not only from the perspective of the health care system, but from the patient perspective as well.

Frequent or nonurgent use of EDs is conceptually similar to excessive or avoidable inpatient utilization in several ways. For example, overuse of EDs is costly, with some estimates suggesting that it is responsible for up to $38 billion in wasteful spending each year.¹⁴⁵ Individuals with frequent ED visits have a greater disease burden¹⁴⁶ and an increased risk of mortality compared to nonfrequent users.¹⁴⁷ Research suggests that individuals who visit the ED for non-urgent issues do so because of perceived difficulties associated with accessing primary care, and the convenience of EDs relative to primary care.^148-150 Moreover, similar to the hospital readmission literature discussed earlier, successful strategies to reduce high rates of ED utilization generally focus on continuum of care interventions, such as provision of case management services.^151-155

This evidence implies that frequent ED utilization and hospital readmissions may not be a fundamental issue of quality (or lack thereof) in hospitals or EDs but rather a lack of, or ineffectual, transitional and continuum of care strategies and services. To underscore this point, some authors have argued that a system that is excessively crisis-oriented hinders recovery because it is reactive rather than proactive, predicated on the notion that one’s condition must deteriorate to receive care.¹⁵⁶

Although some organizations may have access to claims data or may function as self-contained health systems (eg, the Veterans Health Administration [VHA] ), others may not have access to such data. In the absence of claims data, patient self-report of service utilization has been used as a proxy for actual agency records.¹⁵⁷ Although concordance between medical and/or agency records and patient self-report has been variable,¹⁵⁷ evidence generally suggests that rates of agreement are higher the shorter the recall time interval.^158,159 BHD does not have access to comprehensive claims data and has therefore chosen to use 5 dichotomously scored (yes/no) questions—related to medical inpatient, medical ED, psychiatric inpatient, psychiatric ED, and detoxification use in the last 30 days—to represent the CD of acute service utilization.

The Second Aim: Quality of Care

Safety

Safety is defined as avoiding injuries to patients from the care that is intended to help them.

Brief review and suggested item(s). As noted in “Crossing the Quality Chasm,” the IOM’s seminal document, “the health care environment should be safe for all patients, in all of its processes, all the time.”¹⁶⁰ The landmark Harvard Medical Practice Study in 1991 found that adverse events occurred in nearly 4% of all hospital admissions and, among these, over a quarter were due to negligence.¹⁶¹ Other estimates of adverse events range as high as 17%.¹⁶² Indeed, a recent article by Makary and Daniel estimated that medical errors may be the third leading cause of death in the United States.¹⁶³ Unfortunately, research on safety in the mental health field has lagged behind that of physical health,¹⁶⁴ with evidence indicating that research in nonhospital settings in mental health care may be particularly scarce.¹⁶⁵ In a study of adverse events that occurred in psychiatric inpatient units in the VHA system between 2015 and 2016, Mills and colleagues found that of the 87 root cause analysis reports, suicide attempts were the most frequent, and, among safety events, falls were the most frequently reported, followed by medication events.¹⁶⁶ Another report on data collected from psychiatric inpatient units in the VHA revealed that nearly one-fifth of patients experienced a safety event, over half of which were deemed preventable.¹⁶⁷ These numbers likely represent an underestimation of the true volume of safety events, as another study by the same research group found that less than 40% of safety events described in patient medical records were documented in the incident reporting system.¹⁶⁸ BHD will utilize the total number of complaints and incident reports submitted within a given time frame as its “safety” metric in the CD data set.

Wait Time for Service

The CD is defined as the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Brief review and suggested item(s). “Timeliness” was listed among the 6 aims for improvement in “Crossing the Quality Chasm” in 2001, and it remains no less relevant today.¹⁶⁰ For example, evidence indicates that access to primary care is inversely related to avoidable hospitalizations.¹⁶⁹ One study found that, of patients hospitalized for cardiovascular problems, those who had difficulty accessing routine care post discharge had higher 30-day readmission rates.¹⁷⁰ Among VHA patients, longer wait times are associated with more avoidable hospitalizations and higher rates of mortality.¹⁷¹ Longer wait times appear to decrease the likelihood of attending a first appointment for individuals with substance use^172,173 and mental health disorders.¹⁷⁴ Importantly, longer wait times are associated with lower ratings of the patient experience of care, including perceptions of the quality of and satisfaction with care,¹⁷⁵ and may be associated with worse outcomes for individuals in early intervention for psychosis treatment.¹⁷⁶ For the purposes of the CD data set, BHD will monitor the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.

Patient satisfaction is defined as the degree of patients’ satisfaction with the care they have received.

Brief review and suggested item(s). Research has consistently demonstrated the relationship of the patient’s experience of care to a variety of safety and clinical effectiveness measures in medical health care,¹⁷⁷ and the therapeutic alliance is one of the most consistent predictors of outcomes in behavioral health, regardless of therapeutic modality.¹⁷⁸ Patient satisfaction is a commonly assessed aspect of the patient experience of care. Patient satisfaction scores have been correlated with patient adherence to recommended treatment regimens, care quality, and health outcomes.¹⁷⁹ For example, Aiken et al found that patient satisfaction with hospital care was associated with higher ratings of the quality and safety of nursing care in these hospitals.¹⁸⁰ Increased satisfaction with inpatient care has been associated with lower 30-day readmission rates for patients with acute myocardial infarction, heart failure, and pneumonia,¹⁸¹ and patients with schizophrenia who reported higher treatment satisfaction also reported better quality of life.^182,183 Many satisfaction survey options exist to evaluate this CD, including the Consumer Assessment of Healthcare Providers and Systems and the Client Satisfaction Questionnaire; BHD will utilize an outpatient behavioral health survey from a third-party vendor.

The Third Aim: Cost of Care

Cost of Care

This can be defined as the average cost to provide care per patient per month.

Brief review and suggested item(s). Per capita cost, or rather, the total cost of providing care to a circumscribed population divided by the total population, has been espoused as an important metric for the Triple Aim and the County Health Rankings.^6,13 Indeed, between 1960 and 2016, per capita expenditures for health care have grown 70-fold, and the percent of the national gross domestic product accounted for by health expenditures has more than tripled (5.0% to 17.9%).¹⁸⁴ One of the more common metrics deployed for assessing health care cost is the per capita per month cost, or rather, the per member per month cost of the predefined population for a given health care system.^6,185,186 In fact, some authors have proposed that cost of care can be used not only to track efficient resource allocation, but can also be a proxy for a healthier population as well (ie, as health improves, individuals use fewer and less-expensive services, thus costing the system less).¹⁸⁷ To assess this metric, BHD will calculate the total amount billed for patient care provided within BHD’s health network each month (irrespective of funding source) and then divide this sum by the number of members served each month. Although this measure does not account for care received at other health care facilities outside BHD’s provider network, nor does it include all the overhead costs associated with the care provided by BHD itself, it is consistent with the claims-based approach used or recommended by other authors.^6,188

The Fourth Aim: Staff Well-being

Staff Quality of Work Life

This can be defined as the quality of the work life of health care clinicians and staff.

Brief review and suggested item(s). Some authors have suggested that the Triple Aim framework is incomplete and have proffered compelling arguments that provider well-being and the quality of work life constitutes a fourth aim.² Provider burnout is prevalent in both medical^2,189 and behavioral health care.^190,191 Burnout among health care professionals has been associated with higher rates of perceived medical errors,¹⁹² lower patient satisfaction scores,^189,193 lower rates of provider empathy,¹⁹⁴ more negative attitudes towards patients,¹⁹⁵ and poorer staff mental and physical health.¹⁹¹

Burnout is also associated with higher rates of absenteeism, turnover intentions, and turnover.^{190,191,196,197} However, burnout is not the only predictor of staff turnover; for example, turnover rates are a useful proxy for staff quality of work life for several reasons.¹⁹⁸ First, turnover is associated with substantial direct and indirect costs, including lost productivity, increased errors, and lost revenue and recruitment costs, with some turnover cost estimates as high as $17 billion for physicians and $14 billion for nurses annually.^199-201 Second, research indicates that staff turnover can have a deleterious impact on implementation of evidence-based interventions.^202-205 Finally, consistent with the philosophy of utilizing existing data sources for the CD measures, turnover can be relatively easily extracted from administrative data for operated or contracted programs, and its collection does not place any additional burden on staff. As a large behavioral health system that is both a provider and payer of care, BHD will therefore examine the turnover rates of its internal administrative and clinical staff as well as the turnover of staff in its contracted provider network as its measures for the Staff Quality of Work Life CD.

Clinical Implications

These metrics can be deployed at any level of the organization. Clinicians may use 1 or more of the measures to track the recovery of individual clients, or in aggregate for their entire caseload. Similarly, managers can use these measures to assess the overall effectiveness of the programs for which they are responsible. Executive leaders can evaluate the impact of several programs or the system of care on the health of a subpopulation of clients with a specific condition, or for all their enrolled members. Further, not all measures need be utilized for every dashboard or evaluative effort. The benefit of a comprehensive set of measures lies in their flexibility—1 or more of the measures may be selected depending on the project being implemented or the interests of the stakeholder.

It is important to note that many of the CDs (and their accompanying measures) are aligned to/consistent with social determinants of health.^206,207 Evidence suggests that social determinants make substantial contributions to the overall health of individuals and populations and may even account for a greater proportion of variance in health outcomes than health care itself.²⁰⁸ The measures articulated here, therefore, can be used to assess whether and how effectively care provision has addressed these social determinants, as well as the relative impact their resolution may have on other health outcomes (eg, mortality, self-rated health).

These measures can also be used to stratify clients by clinical severity or degree of socioeconomic deprivation. The ability to adjust for risk has many applications in health care, particularly when organizations are attempting to implement value-based purchasing models, such as pay-for-performance contracts or other alternative payment models (population health-based payment models).²⁰⁹ Indeed, once fully implemented, the CDs and measures will enable BHD to more effectively build and execute different conceptual models of “value” (see references 210 and 211 for examples). We will be able to assess the progress our clients have made in care, the cost associated with that degree of improvement, the experience of those clients receiving that care, and the clinical and social variables that may influence the relative degree of improvement (or lack thereof). Thus, the CDs provide a conceptual and data-driven foundation for the Quadruple Aim and any quality initiatives that either catalyze or augment its implementation.

This article provides an overview of the CDs selected by BHD to help organize, focus, advance, and track its quality efforts within the framework of the Quadruple Aim. Although items aligned to each of these CDs are offered, the CDs themselves have been broadly conceptualized such that they can flexibly admit a variety of possible items and/or assessments to operationalize each CD and thus have potential applicability to other behavioral health systems, particularly public systems that have state-mandated and other data reporting requirements.

Bearing in mind the burden that growing data collection requirements can have on the provision of quality care and staff work satisfaction and burnout,^10,212 the CDs (and the items selected to represent each) are designed with “strategic parsimony” in mind. Although the CDs are inclusive in that they cover care quality, cost of care, staff quality of life, and general population health, only CDs and items undergirded by a solid evidence base and high value with regards to BHD’s mission and values, as determined by key stakeholders, were selected. Moreover, BHD attempted to make use of existing data collection and reporting mandates when selecting the final pool of items to reduce the measurement burden on staff and clients. Thus, the final set of CDs and items are designed to be comprehensive yet economical.

The CDs are deeply interrelated. Although each CD may be individually viewed as a valuable metric, improvements in any 1 CD will impact the others (eg, increasing care quality should impact population health, increasing staff quality of life should impact the quality of care). Moreover, this idea of interrelatedness acknowledges the need to view health systems and the populations they serve holistically, in that improvement is not simply the degree of change in any given metric (whether individually or collectively), but rather something more entirely. The concepts of value, quality, and health are complex, multidimensional, and dynamic, and the CDs that comprise these concepts should not be considered independently from one another. The CDs (and items) offered in this article are scalable in that they can be used at different levels of an organization depending on the question or stakeholder, and can be used individually or in combination with one another. Moreover, they are adaptable to a variety of risk-adjusted program, population health, and value-based evaluation models. It is hoped that the process articulated here, and the accompanying literature review, may benefit other public or government-run health systems in their own quality journey to operationalize the Quadruple Aim by developing a set of CDs.

Corresponding author: Walter Matthew Drymalski, PhD; [email protected].

Financial disclosures: None.

From Dignity Health Methodist Hospital of Sacramento Family Medicine Residency Program, Sacramento, CA (Dr. Oldach); Nationwide Children’s Hospital, Columbus, OH (Dr. Irwin); OhioHealth Research Institute, Columbus, OH (Dr. Pershing); Department of Clinical Transformation, OhioHealth, Columbus, OH (Dr. Zigmont and Dr. Gascon); and Department of Geriatrics, OhioHealth, Columbus, OH (Dr. Skully).

Abstract

Objective: An interdisciplinary committee was formed to identify factors contributing to surgical delays in urgent hip fracture repair at an urban, level 1 trauma center, with the goal of reducing preoperative time to less than 24 hours. Surgical optimization was identified as a primary, modifiable factor, as surgeons were reluctant to clear patients for surgery without cardiac consultation. Preoperative transthoracic echocardiogram (TTE) was recommended as a safe alternative to cardiac consultation in most patients.

Methods: A retrospective review was conducted for patients who underwent urgent hip fracture repair between January 2010 and April 2014 (n = 316). Time to medical optimization, time to surgery, hospital length of stay, and anesthesia induction were compared for 3 patient groups of interest: those who received (1) neither TTE nor cardiology consultation (ie, direct to surgery); (2) a preoperative TTE; or (3) preoperative cardiac consultation.

Results: There were significant between-group differences in medical optimization time (P = 0.001) and mean time to surgery (P < 0.001) when comparing the 3 groups of interest. Patients in the preoperative cardiac consult group had the longest times, followed by the TTE and direct-to-surgery groups. There were no differences in the type of induction agent used across treatment groups when stratifying by ejection fraction.

Conclusion: Preoperative TTE allows for decreased preoperative time compared to a cardiology consultation. It provides an easily implemented inter-departmental, intra-institutional intervention to decrease preoperative time in patients presenting with hip fractures.

Keywords: surgical delay; preoperative risk stratification; process improvement.

Hip fractures are common, expensive, and associated with poor outcomes.^1,2 Ample literature suggests that morbidity, mortality, and cost of care may be reduced by minimizing surgical delays.^3-5 While individual reports indicate mixed evidence, in a 2010 meta-analysis, surgery within 72 hours was associated with significant reductions in pneumonia and pressure sores, as well as a 19% reduction in all-cause mortality through 1 year.⁶ Additional reviews suggest evidence of improved patient outcomes (pain, length of stay, non-union, and/or mortality) when surgery occurs early, within 12 to 72 hours after injury.^4,6,7 Regardless of the definition of “early surgery” used, surgical delay remains a challenge, often due to organizational factors, including admission day of the week and hospital staffing, and patient characteristics, such as comorbidities, echocardiographic findings, age, and insurance status.^7-9

Among factors that contribute to surgical delays, the need for preoperative cardiovascular risk stratification is significantly modifiable.¹⁰ The American College of Cardiology (ACC)/American Heart Association (AHA) Task Force risk stratification framework for preoperative cardiac testing assists clinicians in determining surgical urgency, active cardiac conditions, cardiovascular risk factors, and functional capacity of each patient, and is well established for low- or intermediate-risk patients.¹¹ Specifically, metabolic equivalents (METs) measurements are used to identify medically stable patients with good or excellent functional capacity versus poor or unknown functional status. Patients with ≥ 4 METs may proceed to surgery without further testing; patients with < 4 METs may either proceed with planned surgery or undergo additional testing. Patients with a perceived increased risk profile who require urgent or semi-urgent hip fracture repair may be confounded by disagreement about required preoperative cardiac testing.

At OhioHealth Grant Medical Center (GMC), an urban, level 1 trauma center, the consideration of further preoperative noninvasive testing frequently contributed to surgical delays. In 2009, hip fracture patients arriving to the emergency department (ED) waited an average of 51 hours before being transferred to the operating room (OR) for surgery. Presuming prompt surgery is both desirable and feasible, the Grant Hip Fracture Management Committee (GHFMC) was developed in order to expedite surgeries in hip fracture patients. The GHFMC recommended a preoperative hip fracture protocol, and the outcomes from protocol implementation are described in this article.

Methods

This study was approved by the OhioHealth Institutional Review Board, with a waiver of the informed consent requirement. Medical records from patients treated at GMC during the time period between January 2010 and April 2014 (ie, following implementation of GHFMC recommendations) were retrospectively reviewed to identify the extent to which the use of preoperative transthoracic echocardiography (TTE) reduced average time to surgery and total length of stay, compared to cardiac consultation. This chart review included 316 participants and was used to identify primary induction agent utilized, time to medical optimization, time to surgery, and total length of hospital stay.

Intervention

The GHFMC conducted a 9-month quality improvement project to decrease ED-to-OR time to less than 24 hours for hip fracture patients. The multidisciplinary committee consisted of physicians from orthopedic surgery, anesthesia, hospital medicine, and geriatrics, along with key administrators and nurse outcomes managers. While there is lack of complete clarity surrounding optimal surgical timing, the committee decided that surgery within 24 hours would be beneficial for the majority of patients and therefore was considered a prudent goal.

Based on identified barriers that contributed to surgical delays, several process improvement strategies were implemented, including admitting patients to the hospitalist service, engaging the orthopedic trauma team, and implementing pre- and postoperative protocols and order sets (eg, ED and pain management order sets). Specific emphasis was placed on establishing guidelines for determining medical optimization. In the absence of established guidelines, medical optimization was determined at the discretion of the attending physician. The necessity of preoperative cardiac assessment was based, in part, on physician concerns about determining safe anesthesia protocols and hemodynamically managing patients who may have occult heart disease, specifically those patients with low functional capacity (< 4 METs) and/or inability to accurately communicate their medical history.

Many hip fractures result from a fall, and it may be unclear whether the fall causing a fracture was purely mechanical or indicative of a distinct acute or chronic illness. As a result, many patients received cardiac consultations, with or without pharmacologic stress testing, adding another 24 to 36 hours to preoperative time. As invasive preoperative cardiac procedures generally result in surgical delays without improving outcomes,¹¹ the committee recommended that clinicians reserve preoperative cardiac consultation for patients with active cardiac conditions.

In lieu of cardiac consultation, the committee suggested preoperative TTE. While use of TTE has not been shown to improve preoperative risk stratification in routine noncardiac surgeries, it has been shown to provide clinically useful information in patients at high risk for cardiac complications.¹¹ There was consensus for incorporating preoperative TTE for several reasons: (1) the patients with hip fractures were not “routine,” and often did not have a reliable medical history; (2) a large percentage of patients had cardiac risk factors; (3) patients with undiagnosed aortic stenosis, severe left ventricular dysfunction, or severe pulmonary hypertension would likely have altered intraoperative fluid management; and (4) in supplanting cardiac consultations, TTE would likely expedite patients’ ED-to-OR times. Therefore, the GHFMC created a recommendation of ordering urgent TTE for patients who were unable to exercise at ≥ 4 METs but needed urgent hip fracture surgery.

In order to evaluate the success of the new protocol, the ED-to-OR times were calculated for a cohort of patients who underwent surgery for hip fracture following algorithm implementation.

Participants

A chart review was conducted for patients admitted to GMC between January 2010 and April 2014 for operative treatment of a hip fracture. Exclusion criteria included lack of radiologist-diagnosed hip fracture, periprosthetic hip fracture, or multiple traumas. Electronic patient charts were reviewed by investigators (KI and BO) using a standardized, electronic abstraction form for 3 groups of patients who (1) proceeded directly to planned surgery without TTE or cardiac consultation (direct-to-surgery group); (2) received preoperative TTE but not a cardiac consultation (TTE-only group); or (3) received preoperative cardiac consultation (cardiac consult group).

Measures

Demographics, comorbid conditions, MET score, anesthesia protocol, and in-hospital morbidity and mortality were extracted from medical charts. Medical optimization time was determined by the latest time stamp of 1 of the following: time that the final consulting specialist stated that the patient was stable for surgery; time that the hospitalist described the patient as being ready for surgery; time that the TTE report was certified by the reading cardiologist; or time that the hospitalist described the outcome of completed preoperative risk stratification. Time elapsed prior to medical optimization, surgery, and discharge were calculated using differences between the patient’s arrival date and time at the ED, first recorded time of medical optimization, surgical start time (from the surgical report), and discharge time, respectively.

To assess whether the TTE protocol may have affected anesthesia selection, the induction agent (etomidate or propofol) was abstracted from anesthesia reports and stratified by the ejection fraction of each patient: very low (≤ 35%), low (36%–50%), or normal (> 50%). Patients without an echocardiogram report were assumed to have a normal ejection fraction for this analysis.

Analysis

Descriptive statistics were produced using mean and standard deviation (SD) for continuous variables and frequency and percentage for categorical variables. To determine whether statistically significant differences existed between the 3 groups, the Kruskal-Wallis test was used to compare skewed continuous variables, and Pearson’s chi-square test was used to compare categorical variables. Due to differences in baseline patient characteristics across the 3 treatment groups, inverse probability weights were used to adjust for group differences (using a multinomial logit treatment model) while comparing differences in outcome variables. This modeling strategy does not rely on any assumptions for the distribution of the outcome variable. Covariates were considered for inclusion in the treatment or outcome model if they were significantly associated (P < 0.05) with the group variable. Additionally, anesthetic agent (etomidate or propofol) was compared across the treatment groups after stratifying by ejection fraction to identify whether any differences existed in anesthesia regimen. Patients who were prescribed more than 1 anesthetic agent (n = 2) or an agent that was not of interest were removed from the analysis (n = 13). Stata (version 14) was used for analysis. All other missing data with respect to the tested variables were omitted in the analysis for that variable. Any disagreements about abstraction were resolved through consensus between the investigators.

Results

A total of 316 cases met inclusion criteria, including 108 direct-to-surgery patients, 143 preoperative TTE patients, and 65 cardiac consult patients. Patient demographics and preoperative characteristics are shown in Table 1. The average age for all patients was 76.5 years of age (SD, 12.89; IQR, 34-97); however, direct-to-surgery patients were significantly (P < 0.001) younger (71.2 years; SD, 14.2; interquartile range [IQR], 34-95 years) than TTE-only patients (79.0 years; SD, 11.5; IQR, 35-97 years) and cardiac consult patients (79.57 years; SD, 10.63; IQR, 49-97 years). The majority of patients were female (69.9%) and experienced a fall prior to admission (94%). Almost three-fourths of patients had 1 or more cardiac risk factors (73.7%), including history of congestive heart failure (CHF; 19%), coronary artery disease (CAD; 26.3%), chronic obstructive pulmonary disease (COPD; 19.3%), or aortic stenosis (AS; 3.5%). Due to between-group differences in these comorbid conditions, confounding factors were adjusted for in subsequent analyses.

As shown in Table 2, before adjustment for confounding factors, there were significant between-group differences in medical optimization time for patients in all 3 groups. After adjustment for treatment differences using age and number of comorbid diseases, and medical optimization time differences using age and COPD, fewer between-group differences were statistically significant. Patients who received a cardiac consult had an 18.44-hour longer medical optimization time compared to patients who went directly to surgery (29.136 vs 10.696 hours; P = 0.001). Optimization remained approximately 5 hours longer for the TTE-only group than for the direct-to-surgery group; however, this difference was not significant (P = 0.075).

When comparing differences in ED-to-OR time for the 3 groups after adjusting the probability of treatment for age and the number of comorbid conditions, and adjusting the probability of ED-to-OR time for age, COPD, and CHF, significant differences remained in ED-to-OR times across all groups. Specifically, patients in the direct-to-surgery group experienced the shortest time (mean, 20.64 hours), compared to patients in the TTE-only group (mean, 26.32; P = 0.04) or patients in the cardiac consult group (mean, 36.08; P < 0.001). TTE-only patients had a longer time of 5.68 hours, compared to the direct-to-surgery group, and patients in the preoperative cardiac consult group were on average 15.44 hours longer than the direct-to-surgery group.

When comparing differences in the length of stay for the 3 groups before statistical adjustments, differences were observed; however, after removing the confounding factors related to treatment (age and CAD) and the outcome (age and the number of comorbid conditions), there were no statistically significant differences in the length of stay for the 3 groups. Average length of stay was 131 hours for direct-to-surgery patients, 142 hours for TTE-only patients, and 141 hours for cardiac consult patients.

The use of different anesthetic agents was compared for patients in the 3 groups. The majority of patients in the study (87.7%) were given propofol, and there were no differences after stratifying by ejection fraction (Table 3).

Discussion

The GHFMC was created to reduce surgical delays for hip fracture. Medical optimization was considered a primary, modifiable factor given that surgeons were reluctant to proceed without a cardiac consult. To address this gap, the committee recommended a preoperative TTE for patients with low or unknown functional status. This threshold provides a quick and easy method for stratifying patients who previously required risk stratification by a cardiologist, which often resulted in surgery delays.

In their recommendations for implementation of hip fracture quality improvement projects, the Geriatric Fracture Center emphasizes the importance of multidisciplinary physician leadership along with standardization of approach across patients.¹² This recommendation is supported by increasing evidence that orthogeriatric collaborations are associated with decreased mortality and length of stay.¹³ The GHFMC and subsequent interventions reflect this approach, allowing for collaboration to identify cross-disciplinary procedural barriers to care. In our institution, addressing identified procedural barriers to care was associated with a reduction in the average time to surgery from 51 hours to 25.3 hours.

Multiple approaches have been attempted to decrease presurgical time in hip fracture patients in various settings. Prehospital interventions, such as providing ambulances with checklists and ability to bypass the ED, have not been shown to decrease time to surgery for hip fracture patients, though similar strategies have been successful in other conditions, such as stroke.^14,15 In-hospital procedures, such as implementation of a hip fracture protocol and reduction of preoperative interventions, have more consistently been found to decrease time to surgery and in-hospital mortality.^16,17 However, reduced delays have not been found universally. Luttrell and Nana found that preoperative TTE resulted in approximately 30.8-hour delays from the ED to OR, compared to patients who did not receive a preoperative TTE.¹⁸ However, in that study hospitalists used TTE at their own discretion, and there may have been confounding factors contributing to delays. When used as part of a protocol targeting patients with poor or unknown functional capacity, we believe that preoperative TTE results in modest surgical delays yet provides clinically useful information about each patient.

ACC/AHA preoperative guidelines were updated after we implemented our intervention and now recommend that patients with poor or unknown functional capacity in whom stress testing will not influence care proceed to surgery “according to guideline-directed medical care.”¹¹ While routine use of preoperative evaluation of left ventricular function is not recommended, assessing left ventricular function may be reasonable for patients with heart failure with a change in clinical status. Guidelines also recommend that patients with clinically suspected valvular stenosis undergo preoperative echocardiography.¹¹

Limitations

This study has several limitations. First, due to resource limitations, a substantial period of time elapsed between implementation of the new protocol and the analysis of the data set. That is, the hip fracture protocol assessed in this paper occurred from January 2010 through April 2014, and final analysis of the data set occurred in April 2020. This limitation precludes our ability to formally assess any pre- or post-protocol changes in patient outcomes. Second, randomization was not used to create groups that were balanced in differing health characteristics (ie, patients with noncardiac-related surgeries, patients in different age groups); however, the use of inverse probability treatment regression analysis was a way to statistically address these between-group differences. Moreover, this study is limited by the factors that were measured; unmeasured factors cannot be accounted for. Third, health care providers working at the hospital during this time were aware of the goal to decrease presurgical time, possibly creating exaggerated effects compared to a blinded trial. Finally, although this intervention is likely translatable to other centers, these results represent the experiences of a single level 1 trauma center and may not be replicable elsewhere.

Conclusion

Preoperative TTE in lieu of cardiac consultation has several advantages. First, it requires interdepartmental collaboration for implementation, but can be implemented through a single hospital or hospital system. Unlike prehospital interventions, preoperative urgent TTE for patients with low functional capacity does not require the support of emergency medical technicians, ambulance services, or other hospitals in the region. Second, while costs are associated with TTE, they are offset by a reduction in expensive consultations with specialists, surgical delays, and longer lengths of stay. Third, despite likely increased ED-to-OR times compared to no intervention, urgent TTE decreases time to surgery compared with cardiology consultation. Prior to the GHFMC, the ED-to-OR time at our institution was 51 hours. In contrast, the mean time following the GHFMC-led protocol was less than half that, at 25.3 hours (SD, 19.1 hours). In fact, nearly two-thirds (65.2%) of the patients evaluated in this study underwent surgery within 24 hours of admission. This improvement in presurgical time was attributed, in part, to the implementation of preoperative TTE over cardiology consultations.

Acknowledgments: The authors thank Jenny Williams, RN, who was instrumental in obtaining the data set for analysis, and Shauna Ayres, MPH, from the OhioHealth Research Institute, who provided writing and technical assistance.

Corresponding author: Robert Skully, MD, OhioHealth Family Medicine Grant, 290 East Town St., Columbus, OH 43215; [email protected].

Funding: This work was supported by the OhioHealth Summer Research Externship Program.

Financial disclosures: None.

From Dignity Health Methodist Hospital of Sacramento Family Medicine Residency Program, Sacramento, CA (Dr. Oldach); Nationwide Children’s Hospital, Columbus, OH (Dr. Irwin); OhioHealth Research Institute, Columbus, OH (Dr. Pershing); Department of Clinical Transformation, OhioHealth, Columbus, OH (Dr. Zigmont and Dr. Gascon); and Department of Geriatrics, OhioHealth, Columbus, OH (Dr. Skully).

Abstract

Objective: An interdisciplinary committee was formed to identify factors contributing to surgical delays in urgent hip fracture repair at an urban, level 1 trauma center, with the goal of reducing preoperative time to less than 24 hours. Surgical optimization was identified as a primary, modifiable factor, as surgeons were reluctant to clear patients for surgery without cardiac consultation. Preoperative transthoracic echocardiogram (TTE) was recommended as a safe alternative to cardiac consultation in most patients.

Methods: A retrospective review was conducted for patients who underwent urgent hip fracture repair between January 2010 and April 2014 (n = 316). Time to medical optimization, time to surgery, hospital length of stay, and anesthesia induction were compared for 3 patient groups of interest: those who received (1) neither TTE nor cardiology consultation (ie, direct to surgery); (2) a preoperative TTE; or (3) preoperative cardiac consultation.

Results: There were significant between-group differences in medical optimization time (P = 0.001) and mean time to surgery (P < 0.001) when comparing the 3 groups of interest. Patients in the preoperative cardiac consult group had the longest times, followed by the TTE and direct-to-surgery groups. There were no differences in the type of induction agent used across treatment groups when stratifying by ejection fraction.

Conclusion: Preoperative TTE allows for decreased preoperative time compared to a cardiology consultation. It provides an easily implemented inter-departmental, intra-institutional intervention to decrease preoperative time in patients presenting with hip fractures.

Keywords: surgical delay; preoperative risk stratification; process improvement.

Hip fractures are common, expensive, and associated with poor outcomes.^1,2 Ample literature suggests that morbidity, mortality, and cost of care may be reduced by minimizing surgical delays.^3-5 While individual reports indicate mixed evidence, in a 2010 meta-analysis, surgery within 72 hours was associated with significant reductions in pneumonia and pressure sores, as well as a 19% reduction in all-cause mortality through 1 year.⁶ Additional reviews suggest evidence of improved patient outcomes (pain, length of stay, non-union, and/or mortality) when surgery occurs early, within 12 to 72 hours after injury.^4,6,7 Regardless of the definition of “early surgery” used, surgical delay remains a challenge, often due to organizational factors, including admission day of the week and hospital staffing, and patient characteristics, such as comorbidities, echocardiographic findings, age, and insurance status.^7-9

Among factors that contribute to surgical delays, the need for preoperative cardiovascular risk stratification is significantly modifiable.¹⁰ The American College of Cardiology (ACC)/American Heart Association (AHA) Task Force risk stratification framework for preoperative cardiac testing assists clinicians in determining surgical urgency, active cardiac conditions, cardiovascular risk factors, and functional capacity of each patient, and is well established for low- or intermediate-risk patients.¹¹ Specifically, metabolic equivalents (METs) measurements are used to identify medically stable patients with good or excellent functional capacity versus poor or unknown functional status. Patients with ≥ 4 METs may proceed to surgery without further testing; patients with < 4 METs may either proceed with planned surgery or undergo additional testing. Patients with a perceived increased risk profile who require urgent or semi-urgent hip fracture repair may be confounded by disagreement about required preoperative cardiac testing.

At OhioHealth Grant Medical Center (GMC), an urban, level 1 trauma center, the consideration of further preoperative noninvasive testing frequently contributed to surgical delays. In 2009, hip fracture patients arriving to the emergency department (ED) waited an average of 51 hours before being transferred to the operating room (OR) for surgery. Presuming prompt surgery is both desirable and feasible, the Grant Hip Fracture Management Committee (GHFMC) was developed in order to expedite surgeries in hip fracture patients. The GHFMC recommended a preoperative hip fracture protocol, and the outcomes from protocol implementation are described in this article.

Methods

This study was approved by the OhioHealth Institutional Review Board, with a waiver of the informed consent requirement. Medical records from patients treated at GMC during the time period between January 2010 and April 2014 (ie, following implementation of GHFMC recommendations) were retrospectively reviewed to identify the extent to which the use of preoperative transthoracic echocardiography (TTE) reduced average time to surgery and total length of stay, compared to cardiac consultation. This chart review included 316 participants and was used to identify primary induction agent utilized, time to medical optimization, time to surgery, and total length of hospital stay.

Intervention

The GHFMC conducted a 9-month quality improvement project to decrease ED-to-OR time to less than 24 hours for hip fracture patients. The multidisciplinary committee consisted of physicians from orthopedic surgery, anesthesia, hospital medicine, and geriatrics, along with key administrators and nurse outcomes managers. While there is lack of complete clarity surrounding optimal surgical timing, the committee decided that surgery within 24 hours would be beneficial for the majority of patients and therefore was considered a prudent goal.

Based on identified barriers that contributed to surgical delays, several process improvement strategies were implemented, including admitting patients to the hospitalist service, engaging the orthopedic trauma team, and implementing pre- and postoperative protocols and order sets (eg, ED and pain management order sets). Specific emphasis was placed on establishing guidelines for determining medical optimization. In the absence of established guidelines, medical optimization was determined at the discretion of the attending physician. The necessity of preoperative cardiac assessment was based, in part, on physician concerns about determining safe anesthesia protocols and hemodynamically managing patients who may have occult heart disease, specifically those patients with low functional capacity (< 4 METs) and/or inability to accurately communicate their medical history.

Many hip fractures result from a fall, and it may be unclear whether the fall causing a fracture was purely mechanical or indicative of a distinct acute or chronic illness. As a result, many patients received cardiac consultations, with or without pharmacologic stress testing, adding another 24 to 36 hours to preoperative time. As invasive preoperative cardiac procedures generally result in surgical delays without improving outcomes,¹¹ the committee recommended that clinicians reserve preoperative cardiac consultation for patients with active cardiac conditions.

In lieu of cardiac consultation, the committee suggested preoperative TTE. While use of TTE has not been shown to improve preoperative risk stratification in routine noncardiac surgeries, it has been shown to provide clinically useful information in patients at high risk for cardiac complications.¹¹ There was consensus for incorporating preoperative TTE for several reasons: (1) the patients with hip fractures were not “routine,” and often did not have a reliable medical history; (2) a large percentage of patients had cardiac risk factors; (3) patients with undiagnosed aortic stenosis, severe left ventricular dysfunction, or severe pulmonary hypertension would likely have altered intraoperative fluid management; and (4) in supplanting cardiac consultations, TTE would likely expedite patients’ ED-to-OR times. Therefore, the GHFMC created a recommendation of ordering urgent TTE for patients who were unable to exercise at ≥ 4 METs but needed urgent hip fracture surgery.

In order to evaluate the success of the new protocol, the ED-to-OR times were calculated for a cohort of patients who underwent surgery for hip fracture following algorithm implementation.

Participants

A chart review was conducted for patients admitted to GMC between January 2010 and April 2014 for operative treatment of a hip fracture. Exclusion criteria included lack of radiologist-diagnosed hip fracture, periprosthetic hip fracture, or multiple traumas. Electronic patient charts were reviewed by investigators (KI and BO) using a standardized, electronic abstraction form for 3 groups of patients who (1) proceeded directly to planned surgery without TTE or cardiac consultation (direct-to-surgery group); (2) received preoperative TTE but not a cardiac consultation (TTE-only group); or (3) received preoperative cardiac consultation (cardiac consult group).

Measures

Demographics, comorbid conditions, MET score, anesthesia protocol, and in-hospital morbidity and mortality were extracted from medical charts. Medical optimization time was determined by the latest time stamp of 1 of the following: time that the final consulting specialist stated that the patient was stable for surgery; time that the hospitalist described the patient as being ready for surgery; time that the TTE report was certified by the reading cardiologist; or time that the hospitalist described the outcome of completed preoperative risk stratification. Time elapsed prior to medical optimization, surgery, and discharge were calculated using differences between the patient’s arrival date and time at the ED, first recorded time of medical optimization, surgical start time (from the surgical report), and discharge time, respectively.

To assess whether the TTE protocol may have affected anesthesia selection, the induction agent (etomidate or propofol) was abstracted from anesthesia reports and stratified by the ejection fraction of each patient: very low (≤ 35%), low (36%–50%), or normal (> 50%). Patients without an echocardiogram report were assumed to have a normal ejection fraction for this analysis.

Analysis

Descriptive statistics were produced using mean and standard deviation (SD) for continuous variables and frequency and percentage for categorical variables. To determine whether statistically significant differences existed between the 3 groups, the Kruskal-Wallis test was used to compare skewed continuous variables, and Pearson’s chi-square test was used to compare categorical variables. Due to differences in baseline patient characteristics across the 3 treatment groups, inverse probability weights were used to adjust for group differences (using a multinomial logit treatment model) while comparing differences in outcome variables. This modeling strategy does not rely on any assumptions for the distribution of the outcome variable. Covariates were considered for inclusion in the treatment or outcome model if they were significantly associated (P < 0.05) with the group variable. Additionally, anesthetic agent (etomidate or propofol) was compared across the treatment groups after stratifying by ejection fraction to identify whether any differences existed in anesthesia regimen. Patients who were prescribed more than 1 anesthetic agent (n = 2) or an agent that was not of interest were removed from the analysis (n = 13). Stata (version 14) was used for analysis. All other missing data with respect to the tested variables were omitted in the analysis for that variable. Any disagreements about abstraction were resolved through consensus between the investigators.

Results

A total of 316 cases met inclusion criteria, including 108 direct-to-surgery patients, 143 preoperative TTE patients, and 65 cardiac consult patients. Patient demographics and preoperative characteristics are shown in Table 1. The average age for all patients was 76.5 years of age (SD, 12.89; IQR, 34-97); however, direct-to-surgery patients were significantly (P < 0.001) younger (71.2 years; SD, 14.2; interquartile range [IQR], 34-95 years) than TTE-only patients (79.0 years; SD, 11.5; IQR, 35-97 years) and cardiac consult patients (79.57 years; SD, 10.63; IQR, 49-97 years). The majority of patients were female (69.9%) and experienced a fall prior to admission (94%). Almost three-fourths of patients had 1 or more cardiac risk factors (73.7%), including history of congestive heart failure (CHF; 19%), coronary artery disease (CAD; 26.3%), chronic obstructive pulmonary disease (COPD; 19.3%), or aortic stenosis (AS; 3.5%). Due to between-group differences in these comorbid conditions, confounding factors were adjusted for in subsequent analyses.

As shown in Table 2, before adjustment for confounding factors, there were significant between-group differences in medical optimization time for patients in all 3 groups. After adjustment for treatment differences using age and number of comorbid diseases, and medical optimization time differences using age and COPD, fewer between-group differences were statistically significant. Patients who received a cardiac consult had an 18.44-hour longer medical optimization time compared to patients who went directly to surgery (29.136 vs 10.696 hours; P = 0.001). Optimization remained approximately 5 hours longer for the TTE-only group than for the direct-to-surgery group; however, this difference was not significant (P = 0.075).

When comparing differences in ED-to-OR time for the 3 groups after adjusting the probability of treatment for age and the number of comorbid conditions, and adjusting the probability of ED-to-OR time for age, COPD, and CHF, significant differences remained in ED-to-OR times across all groups. Specifically, patients in the direct-to-surgery group experienced the shortest time (mean, 20.64 hours), compared to patients in the TTE-only group (mean, 26.32; P = 0.04) or patients in the cardiac consult group (mean, 36.08; P < 0.001). TTE-only patients had a longer time of 5.68 hours, compared to the direct-to-surgery group, and patients in the preoperative cardiac consult group were on average 15.44 hours longer than the direct-to-surgery group.

When comparing differences in the length of stay for the 3 groups before statistical adjustments, differences were observed; however, after removing the confounding factors related to treatment (age and CAD) and the outcome (age and the number of comorbid conditions), there were no statistically significant differences in the length of stay for the 3 groups. Average length of stay was 131 hours for direct-to-surgery patients, 142 hours for TTE-only patients, and 141 hours for cardiac consult patients.

The use of different anesthetic agents was compared for patients in the 3 groups. The majority of patients in the study (87.7%) were given propofol, and there were no differences after stratifying by ejection fraction (Table 3).

Discussion

The GHFMC was created to reduce surgical delays for hip fracture. Medical optimization was considered a primary, modifiable factor given that surgeons were reluctant to proceed without a cardiac consult. To address this gap, the committee recommended a preoperative TTE for patients with low or unknown functional status. This threshold provides a quick and easy method for stratifying patients who previously required risk stratification by a cardiologist, which often resulted in surgery delays.

In their recommendations for implementation of hip fracture quality improvement projects, the Geriatric Fracture Center emphasizes the importance of multidisciplinary physician leadership along with standardization of approach across patients.¹² This recommendation is supported by increasing evidence that orthogeriatric collaborations are associated with decreased mortality and length of stay.¹³ The GHFMC and subsequent interventions reflect this approach, allowing for collaboration to identify cross-disciplinary procedural barriers to care. In our institution, addressing identified procedural barriers to care was associated with a reduction in the average time to surgery from 51 hours to 25.3 hours.

Multiple approaches have been attempted to decrease presurgical time in hip fracture patients in various settings. Prehospital interventions, such as providing ambulances with checklists and ability to bypass the ED, have not been shown to decrease time to surgery for hip fracture patients, though similar strategies have been successful in other conditions, such as stroke.^14,15 In-hospital procedures, such as implementation of a hip fracture protocol and reduction of preoperative interventions, have more consistently been found to decrease time to surgery and in-hospital mortality.^16,17 However, reduced delays have not been found universally. Luttrell and Nana found that preoperative TTE resulted in approximately 30.8-hour delays from the ED to OR, compared to patients who did not receive a preoperative TTE.¹⁸ However, in that study hospitalists used TTE at their own discretion, and there may have been confounding factors contributing to delays. When used as part of a protocol targeting patients with poor or unknown functional capacity, we believe that preoperative TTE results in modest surgical delays yet provides clinically useful information about each patient.

ACC/AHA preoperative guidelines were updated after we implemented our intervention and now recommend that patients with poor or unknown functional capacity in whom stress testing will not influence care proceed to surgery “according to guideline-directed medical care.”¹¹ While routine use of preoperative evaluation of left ventricular function is not recommended, assessing left ventricular function may be reasonable for patients with heart failure with a change in clinical status. Guidelines also recommend that patients with clinically suspected valvular stenosis undergo preoperative echocardiography.¹¹

Limitations

This study has several limitations. First, due to resource limitations, a substantial period of time elapsed between implementation of the new protocol and the analysis of the data set. That is, the hip fracture protocol assessed in this paper occurred from January 2010 through April 2014, and final analysis of the data set occurred in April 2020. This limitation precludes our ability to formally assess any pre- or post-protocol changes in patient outcomes. Second, randomization was not used to create groups that were balanced in differing health characteristics (ie, patients with noncardiac-related surgeries, patients in different age groups); however, the use of inverse probability treatment regression analysis was a way to statistically address these between-group differences. Moreover, this study is limited by the factors that were measured; unmeasured factors cannot be accounted for. Third, health care providers working at the hospital during this time were aware of the goal to decrease presurgical time, possibly creating exaggerated effects compared to a blinded trial. Finally, although this intervention is likely translatable to other centers, these results represent the experiences of a single level 1 trauma center and may not be replicable elsewhere.

Conclusion

Preoperative TTE in lieu of cardiac consultation has several advantages. First, it requires interdepartmental collaboration for implementation, but can be implemented through a single hospital or hospital system. Unlike prehospital interventions, preoperative urgent TTE for patients with low functional capacity does not require the support of emergency medical technicians, ambulance services, or other hospitals in the region. Second, while costs are associated with TTE, they are offset by a reduction in expensive consultations with specialists, surgical delays, and longer lengths of stay. Third, despite likely increased ED-to-OR times compared to no intervention, urgent TTE decreases time to surgery compared with cardiology consultation. Prior to the GHFMC, the ED-to-OR time at our institution was 51 hours. In contrast, the mean time following the GHFMC-led protocol was less than half that, at 25.3 hours (SD, 19.1 hours). In fact, nearly two-thirds (65.2%) of the patients evaluated in this study underwent surgery within 24 hours of admission. This improvement in presurgical time was attributed, in part, to the implementation of preoperative TTE over cardiology consultations.

Acknowledgments: The authors thank Jenny Williams, RN, who was instrumental in obtaining the data set for analysis, and Shauna Ayres, MPH, from the OhioHealth Research Institute, who provided writing and technical assistance.

Corresponding author: Robert Skully, MD, OhioHealth Family Medicine Grant, 290 East Town St., Columbus, OH 43215; [email protected].

Funding: This work was supported by the OhioHealth Summer Research Externship Program.

Financial disclosures: None.

From Dignity Health Methodist Hospital of Sacramento Family Medicine Residency Program, Sacramento, CA (Dr. Oldach); Nationwide Children’s Hospital, Columbus, OH (Dr. Irwin); OhioHealth Research Institute, Columbus, OH (Dr. Pershing); Department of Clinical Transformation, OhioHealth, Columbus, OH (Dr. Zigmont and Dr. Gascon); and Department of Geriatrics, OhioHealth, Columbus, OH (Dr. Skully).

Abstract

Objective: An interdisciplinary committee was formed to identify factors contributing to surgical delays in urgent hip fracture repair at an urban, level 1 trauma center, with the goal of reducing preoperative time to less than 24 hours. Surgical optimization was identified as a primary, modifiable factor, as surgeons were reluctant to clear patients for surgery without cardiac consultation. Preoperative transthoracic echocardiogram (TTE) was recommended as a safe alternative to cardiac consultation in most patients.

Methods: A retrospective review was conducted for patients who underwent urgent hip fracture repair between January 2010 and April 2014 (n = 316). Time to medical optimization, time to surgery, hospital length of stay, and anesthesia induction were compared for 3 patient groups of interest: those who received (1) neither TTE nor cardiology consultation (ie, direct to surgery); (2) a preoperative TTE; or (3) preoperative cardiac consultation.

Results: There were significant between-group differences in medical optimization time (P = 0.001) and mean time to surgery (P < 0.001) when comparing the 3 groups of interest. Patients in the preoperative cardiac consult group had the longest times, followed by the TTE and direct-to-surgery groups. There were no differences in the type of induction agent used across treatment groups when stratifying by ejection fraction.

Conclusion: Preoperative TTE allows for decreased preoperative time compared to a cardiology consultation. It provides an easily implemented inter-departmental, intra-institutional intervention to decrease preoperative time in patients presenting with hip fractures.

Keywords: surgical delay; preoperative risk stratification; process improvement.

Hip fractures are common, expensive, and associated with poor outcomes.^1,2 Ample literature suggests that morbidity, mortality, and cost of care may be reduced by minimizing surgical delays.^3-5 While individual reports indicate mixed evidence, in a 2010 meta-analysis, surgery within 72 hours was associated with significant reductions in pneumonia and pressure sores, as well as a 19% reduction in all-cause mortality through 1 year.⁶ Additional reviews suggest evidence of improved patient outcomes (pain, length of stay, non-union, and/or mortality) when surgery occurs early, within 12 to 72 hours after injury.^4,6,7 Regardless of the definition of “early surgery” used, surgical delay remains a challenge, often due to organizational factors, including admission day of the week and hospital staffing, and patient characteristics, such as comorbidities, echocardiographic findings, age, and insurance status.^7-9

Among factors that contribute to surgical delays, the need for preoperative cardiovascular risk stratification is significantly modifiable.¹⁰ The American College of Cardiology (ACC)/American Heart Association (AHA) Task Force risk stratification framework for preoperative cardiac testing assists clinicians in determining surgical urgency, active cardiac conditions, cardiovascular risk factors, and functional capacity of each patient, and is well established for low- or intermediate-risk patients.¹¹ Specifically, metabolic equivalents (METs) measurements are used to identify medically stable patients with good or excellent functional capacity versus poor or unknown functional status. Patients with ≥ 4 METs may proceed to surgery without further testing; patients with < 4 METs may either proceed with planned surgery or undergo additional testing. Patients with a perceived increased risk profile who require urgent or semi-urgent hip fracture repair may be confounded by disagreement about required preoperative cardiac testing.

At OhioHealth Grant Medical Center (GMC), an urban, level 1 trauma center, the consideration of further preoperative noninvasive testing frequently contributed to surgical delays. In 2009, hip fracture patients arriving to the emergency department (ED) waited an average of 51 hours before being transferred to the operating room (OR) for surgery. Presuming prompt surgery is both desirable and feasible, the Grant Hip Fracture Management Committee (GHFMC) was developed in order to expedite surgeries in hip fracture patients. The GHFMC recommended a preoperative hip fracture protocol, and the outcomes from protocol implementation are described in this article.

Methods

This study was approved by the OhioHealth Institutional Review Board, with a waiver of the informed consent requirement. Medical records from patients treated at GMC during the time period between January 2010 and April 2014 (ie, following implementation of GHFMC recommendations) were retrospectively reviewed to identify the extent to which the use of preoperative transthoracic echocardiography (TTE) reduced average time to surgery and total length of stay, compared to cardiac consultation. This chart review included 316 participants and was used to identify primary induction agent utilized, time to medical optimization, time to surgery, and total length of hospital stay.

Intervention

The GHFMC conducted a 9-month quality improvement project to decrease ED-to-OR time to less than 24 hours for hip fracture patients. The multidisciplinary committee consisted of physicians from orthopedic surgery, anesthesia, hospital medicine, and geriatrics, along with key administrators and nurse outcomes managers. While there is lack of complete clarity surrounding optimal surgical timing, the committee decided that surgery within 24 hours would be beneficial for the majority of patients and therefore was considered a prudent goal.

Based on identified barriers that contributed to surgical delays, several process improvement strategies were implemented, including admitting patients to the hospitalist service, engaging the orthopedic trauma team, and implementing pre- and postoperative protocols and order sets (eg, ED and pain management order sets). Specific emphasis was placed on establishing guidelines for determining medical optimization. In the absence of established guidelines, medical optimization was determined at the discretion of the attending physician. The necessity of preoperative cardiac assessment was based, in part, on physician concerns about determining safe anesthesia protocols and hemodynamically managing patients who may have occult heart disease, specifically those patients with low functional capacity (< 4 METs) and/or inability to accurately communicate their medical history.

Many hip fractures result from a fall, and it may be unclear whether the fall causing a fracture was purely mechanical or indicative of a distinct acute or chronic illness. As a result, many patients received cardiac consultations, with or without pharmacologic stress testing, adding another 24 to 36 hours to preoperative time. As invasive preoperative cardiac procedures generally result in surgical delays without improving outcomes,¹¹ the committee recommended that clinicians reserve preoperative cardiac consultation for patients with active cardiac conditions.

In lieu of cardiac consultation, the committee suggested preoperative TTE. While use of TTE has not been shown to improve preoperative risk stratification in routine noncardiac surgeries, it has been shown to provide clinically useful information in patients at high risk for cardiac complications.¹¹ There was consensus for incorporating preoperative TTE for several reasons: (1) the patients with hip fractures were not “routine,” and often did not have a reliable medical history; (2) a large percentage of patients had cardiac risk factors; (3) patients with undiagnosed aortic stenosis, severe left ventricular dysfunction, or severe pulmonary hypertension would likely have altered intraoperative fluid management; and (4) in supplanting cardiac consultations, TTE would likely expedite patients’ ED-to-OR times. Therefore, the GHFMC created a recommendation of ordering urgent TTE for patients who were unable to exercise at ≥ 4 METs but needed urgent hip fracture surgery.

In order to evaluate the success of the new protocol, the ED-to-OR times were calculated for a cohort of patients who underwent surgery for hip fracture following algorithm implementation.

Participants

A chart review was conducted for patients admitted to GMC between January 2010 and April 2014 for operative treatment of a hip fracture. Exclusion criteria included lack of radiologist-diagnosed hip fracture, periprosthetic hip fracture, or multiple traumas. Electronic patient charts were reviewed by investigators (KI and BO) using a standardized, electronic abstraction form for 3 groups of patients who (1) proceeded directly to planned surgery without TTE or cardiac consultation (direct-to-surgery group); (2) received preoperative TTE but not a cardiac consultation (TTE-only group); or (3) received preoperative cardiac consultation (cardiac consult group).

Measures

Demographics, comorbid conditions, MET score, anesthesia protocol, and in-hospital morbidity and mortality were extracted from medical charts. Medical optimization time was determined by the latest time stamp of 1 of the following: time that the final consulting specialist stated that the patient was stable for surgery; time that the hospitalist described the patient as being ready for surgery; time that the TTE report was certified by the reading cardiologist; or time that the hospitalist described the outcome of completed preoperative risk stratification. Time elapsed prior to medical optimization, surgery, and discharge were calculated using differences between the patient’s arrival date and time at the ED, first recorded time of medical optimization, surgical start time (from the surgical report), and discharge time, respectively.

To assess whether the TTE protocol may have affected anesthesia selection, the induction agent (etomidate or propofol) was abstracted from anesthesia reports and stratified by the ejection fraction of each patient: very low (≤ 35%), low (36%–50%), or normal (> 50%). Patients without an echocardiogram report were assumed to have a normal ejection fraction for this analysis.

Analysis

Descriptive statistics were produced using mean and standard deviation (SD) for continuous variables and frequency and percentage for categorical variables. To determine whether statistically significant differences existed between the 3 groups, the Kruskal-Wallis test was used to compare skewed continuous variables, and Pearson’s chi-square test was used to compare categorical variables. Due to differences in baseline patient characteristics across the 3 treatment groups, inverse probability weights were used to adjust for group differences (using a multinomial logit treatment model) while comparing differences in outcome variables. This modeling strategy does not rely on any assumptions for the distribution of the outcome variable. Covariates were considered for inclusion in the treatment or outcome model if they were significantly associated (P < 0.05) with the group variable. Additionally, anesthetic agent (etomidate or propofol) was compared across the treatment groups after stratifying by ejection fraction to identify whether any differences existed in anesthesia regimen. Patients who were prescribed more than 1 anesthetic agent (n = 2) or an agent that was not of interest were removed from the analysis (n = 13). Stata (version 14) was used for analysis. All other missing data with respect to the tested variables were omitted in the analysis for that variable. Any disagreements about abstraction were resolved through consensus between the investigators.

Results

A total of 316 cases met inclusion criteria, including 108 direct-to-surgery patients, 143 preoperative TTE patients, and 65 cardiac consult patients. Patient demographics and preoperative characteristics are shown in Table 1. The average age for all patients was 76.5 years of age (SD, 12.89; IQR, 34-97); however, direct-to-surgery patients were significantly (P < 0.001) younger (71.2 years; SD, 14.2; interquartile range [IQR], 34-95 years) than TTE-only patients (79.0 years; SD, 11.5; IQR, 35-97 years) and cardiac consult patients (79.57 years; SD, 10.63; IQR, 49-97 years). The majority of patients were female (69.9%) and experienced a fall prior to admission (94%). Almost three-fourths of patients had 1 or more cardiac risk factors (73.7%), including history of congestive heart failure (CHF; 19%), coronary artery disease (CAD; 26.3%), chronic obstructive pulmonary disease (COPD; 19.3%), or aortic stenosis (AS; 3.5%). Due to between-group differences in these comorbid conditions, confounding factors were adjusted for in subsequent analyses.

As shown in Table 2, before adjustment for confounding factors, there were significant between-group differences in medical optimization time for patients in all 3 groups. After adjustment for treatment differences using age and number of comorbid diseases, and medical optimization time differences using age and COPD, fewer between-group differences were statistically significant. Patients who received a cardiac consult had an 18.44-hour longer medical optimization time compared to patients who went directly to surgery (29.136 vs 10.696 hours; P = 0.001). Optimization remained approximately 5 hours longer for the TTE-only group than for the direct-to-surgery group; however, this difference was not significant (P = 0.075).

When comparing differences in ED-to-OR time for the 3 groups after adjusting the probability of treatment for age and the number of comorbid conditions, and adjusting the probability of ED-to-OR time for age, COPD, and CHF, significant differences remained in ED-to-OR times across all groups. Specifically, patients in the direct-to-surgery group experienced the shortest time (mean, 20.64 hours), compared to patients in the TTE-only group (mean, 26.32; P = 0.04) or patients in the cardiac consult group (mean, 36.08; P < 0.001). TTE-only patients had a longer time of 5.68 hours, compared to the direct-to-surgery group, and patients in the preoperative cardiac consult group were on average 15.44 hours longer than the direct-to-surgery group.

When comparing differences in the length of stay for the 3 groups before statistical adjustments, differences were observed; however, after removing the confounding factors related to treatment (age and CAD) and the outcome (age and the number of comorbid conditions), there were no statistically significant differences in the length of stay for the 3 groups. Average length of stay was 131 hours for direct-to-surgery patients, 142 hours for TTE-only patients, and 141 hours for cardiac consult patients.

The use of different anesthetic agents was compared for patients in the 3 groups. The majority of patients in the study (87.7%) were given propofol, and there were no differences after stratifying by ejection fraction (Table 3).

Discussion

The GHFMC was created to reduce surgical delays for hip fracture. Medical optimization was considered a primary, modifiable factor given that surgeons were reluctant to proceed without a cardiac consult. To address this gap, the committee recommended a preoperative TTE for patients with low or unknown functional status. This threshold provides a quick and easy method for stratifying patients who previously required risk stratification by a cardiologist, which often resulted in surgery delays.

In their recommendations for implementation of hip fracture quality improvement projects, the Geriatric Fracture Center emphasizes the importance of multidisciplinary physician leadership along with standardization of approach across patients.¹² This recommendation is supported by increasing evidence that orthogeriatric collaborations are associated with decreased mortality and length of stay.¹³ The GHFMC and subsequent interventions reflect this approach, allowing for collaboration to identify cross-disciplinary procedural barriers to care. In our institution, addressing identified procedural barriers to care was associated with a reduction in the average time to surgery from 51 hours to 25.3 hours.

Multiple approaches have been attempted to decrease presurgical time in hip fracture patients in various settings. Prehospital interventions, such as providing ambulances with checklists and ability to bypass the ED, have not been shown to decrease time to surgery for hip fracture patients, though similar strategies have been successful in other conditions, such as stroke.^14,15 In-hospital procedures, such as implementation of a hip fracture protocol and reduction of preoperative interventions, have more consistently been found to decrease time to surgery and in-hospital mortality.^16,17 However, reduced delays have not been found universally. Luttrell and Nana found that preoperative TTE resulted in approximately 30.8-hour delays from the ED to OR, compared to patients who did not receive a preoperative TTE.¹⁸ However, in that study hospitalists used TTE at their own discretion, and there may have been confounding factors contributing to delays. When used as part of a protocol targeting patients with poor or unknown functional capacity, we believe that preoperative TTE results in modest surgical delays yet provides clinically useful information about each patient.

ACC/AHA preoperative guidelines were updated after we implemented our intervention and now recommend that patients with poor or unknown functional capacity in whom stress testing will not influence care proceed to surgery “according to guideline-directed medical care.”¹¹ While routine use of preoperative evaluation of left ventricular function is not recommended, assessing left ventricular function may be reasonable for patients with heart failure with a change in clinical status. Guidelines also recommend that patients with clinically suspected valvular stenosis undergo preoperative echocardiography.¹¹

Limitations

This study has several limitations. First, due to resource limitations, a substantial period of time elapsed between implementation of the new protocol and the analysis of the data set. That is, the hip fracture protocol assessed in this paper occurred from January 2010 through April 2014, and final analysis of the data set occurred in April 2020. This limitation precludes our ability to formally assess any pre- or post-protocol changes in patient outcomes. Second, randomization was not used to create groups that were balanced in differing health characteristics (ie, patients with noncardiac-related surgeries, patients in different age groups); however, the use of inverse probability treatment regression analysis was a way to statistically address these between-group differences. Moreover, this study is limited by the factors that were measured; unmeasured factors cannot be accounted for. Third, health care providers working at the hospital during this time were aware of the goal to decrease presurgical time, possibly creating exaggerated effects compared to a blinded trial. Finally, although this intervention is likely translatable to other centers, these results represent the experiences of a single level 1 trauma center and may not be replicable elsewhere.

Conclusion

Preoperative TTE in lieu of cardiac consultation has several advantages. First, it requires interdepartmental collaboration for implementation, but can be implemented through a single hospital or hospital system. Unlike prehospital interventions, preoperative urgent TTE for patients with low functional capacity does not require the support of emergency medical technicians, ambulance services, or other hospitals in the region. Second, while costs are associated with TTE, they are offset by a reduction in expensive consultations with specialists, surgical delays, and longer lengths of stay. Third, despite likely increased ED-to-OR times compared to no intervention, urgent TTE decreases time to surgery compared with cardiology consultation. Prior to the GHFMC, the ED-to-OR time at our institution was 51 hours. In contrast, the mean time following the GHFMC-led protocol was less than half that, at 25.3 hours (SD, 19.1 hours). In fact, nearly two-thirds (65.2%) of the patients evaluated in this study underwent surgery within 24 hours of admission. This improvement in presurgical time was attributed, in part, to the implementation of preoperative TTE over cardiology consultations.

Acknowledgments: The authors thank Jenny Williams, RN, who was instrumental in obtaining the data set for analysis, and Shauna Ayres, MPH, from the OhioHealth Research Institute, who provided writing and technical assistance.

Corresponding author: Robert Skully, MD, OhioHealth Family Medicine Grant, 290 East Town St., Columbus, OH 43215; [email protected].

Funding: This work was supported by the OhioHealth Summer Research Externship Program.

Financial disclosures: None.

From Banner Health Corporation, Phoenix, AZ.

Background: Health care providers are routinely incentivized with pay-for-performance (P4P) metrics to increase the quality of care. In an inpatient setting, P4P models typically measure quality by attributing each patient’s outcome to a single provider even though many providers routinely care for the patient. This study investigates a new attribution approach aiming to distribute each outcome across all providers who provided care.

Methods: The methodology relies on a multi-membership model and is demonstrated in the Banner Health system using 3 clinical outcome measures (length of stay, 30-day readmissions, and mortality) and responses to 3 survey questions that measure a patient’s perception of their care. The new approach is compared to the “standard” method, which attributes each patient to only 1 provider.

Results: When ranking by clinical outcomes, both methods were concordant 72.1% to 82.1% of the time for top-half/bottom-half rankings, with a median percentile difference between 7 and 15. When ranking by survey scores, there was more agreement, with concordance between 84.1% and 86.6% and a median percentile difference between 11 and 13. Last, Pearson correlation coefficients of the paired percentiles ranged from 0.56 to 0.78.

Conclusion: The new approach provides a fairer solution when measuring provider performance.

Keywords: patient attribution; PAMM; PAPR; random effect model; pay for performance.

Providers practicing in hospitals are routinely evaluated based on their performance and, in many cases, are financially incentivized for a better-than-average performance within a pay-for-performance (P4P) model. The use of P4P models is based on the belief that they will “improve, motivate, and enhance providers to pursue aggressively and ultimately achieve the quality performance targets thus decreasing the number of medical errors with less malpractice events.”¹ Although P4P models continue to be a movement in health care, they have been challenging to implement.

One concern involves the general quality of implementation, such as defining metrics and targets, setting payout amounts, managing technology and market conditions, and gauging the level of transparency to the provider.² Another challenge, and the focus of this project, are concerns around measuring performance to avoid perceptions of unfairness. This concern can be minimized if the attribution is handled in a fairer way, by spreading it across all providers who affected the outcome, both in a positive or negative direction.³

To implement these models, the performance of providers needs to be measured and tracked periodically. This requires linking, or attributing, a patient’s outcome to a provider, which is almost always the attending or discharging provider (ie, a single provider).³ In this single-provider attribution approach, one provider will receive all the credit (good or bad) for their respective patients’ outcomes, even though the provider may have seen the patient only a fraction of the time during the hospitalization. Attributing outcomes—for example, length of stay (LOS), readmission rate, mortality rate, net promoter score (NPS)—using this approach reduces the validity of metrics designed to measure provider performance, especially in a rotating provider environment where many providers interact with and care for a patient. For example, the quality of providers’ interpersonal skills and competence were among the strongest determinants of patient satisfaction,⁴ but it is not credible that this is solely based on the last provider during a hospitalization.

Proportionally distributing the attribution of an outcome has been used successfully in other contexts. Typically, a statistical modeling approach using a multi-membership framework is used because it can handle the sometimes-complicated relationships within the hierarchy. It also allows for auxiliary variables to be introduced, which can help explain and control for exogenous effects.^5-7 For example, in the education setting, standardized testing is administered to students at defined years of schooling: at grades 4, 8, and 10, for instance. The progress of students, measured as the academic gains between test years, are proportionally attributed to all the teachers who the student has had between the test years. These partial attributions are combined to evaluate an overall teacher performance.^8,9

Although the multi-membership framework has been used in other industries, it has yet to be applied in measuring provider performance. The purpose of this project is to investigate the impact of using a multi-provider approach compared to the standard single-provider approach. The findings may lead to modifications in the way a provider’s performance is measured and, thus, how providers are compensated. A similar study investigated the impact of proportionally distributing patients’ outcomes across all rotating providers using a weighting method based on billing practices to measure the partial impact of each provider.³

This study is different in 2 fundamental ways. First, attribution is weighted based on the number of clinically documented interactions (via clinical notes) between a patient and all rotating providers during the hospitalization. Second, performance is measured via multi-membership models, which can estimate the effect (both positive and negative) that a provider has on an outcome, even when caring for a patient a fraction of the time during the hospitalization.

Methods

Setting

Banner Health is a non-profit, multi-hospital health care system across 6 states in the western United States that is uniquely positioned to study provider quality attribution models. It not only has a large number of providers and serves a broad patient population, but Banner Health also uses an instance of Cerner (Kansas City, MO), an enterprise-level electronic health record (EHR) system that connects all its facilities and allows for advanced analytics across its system.

For this study, we included only general medicine and surgery patients admitted and discharged from the inpatient setting between January 1, 2018, and December 31, 2018, who were between 18 and 89 years old at admission, and who had a LOS between 1 and 14 days. Visit- and patient-level data were collected from Cerner, while outcome data, and corresponding expected outcome data, were obtained from Premier, Inc. (Charlotte, NC) using their CareScience methodologies.¹⁰ To measure patient experience, response data were extracted from post-discharge surveys administered by InMoment (Salt Lake City, UT).

Provider Attribution Models

Provider Attribution by Physician of Record (PAPR). In the standard approach, denoted here as the PAPR model, 1 provider—typically the attending or discharging provider, which may be the same person—is attributed to the entire hospitalization. This provider is responsible for the patient’s care, and all patient outcomes are aggregated and attributed to the provider to gauge his or her performance. The PAPR model is the most popular form of attribution across many health care systems and is routinely used for P4P incentives.

In this study, the discharging provider was used when attributing hospitalizations using the PAPR model. Providers responsible for fewer than 12 discharges in the calendar year were excluded. Because of the directness of this type of attribution, the performance of 1 provider does not account for the performance of the other rotating providers during hospitalizations.

Provider Attribution by Multiple Membership (PAMM). In contrast, we introduce another attribution approach here that is designed to assign partial attribution to each provider who cares for the patient during the hospitalization. To aggregate the partial attributions, and possibly control for any exogenous or risk-based factors, a multiple-membership, or multi-member (MM), model is used. The MM model can measure the effect of a provider on an outcome even when the patient-to-provider relationship is complex, such as in a rotating provider environment.⁸

The purpose of this study is to compare attribution models and to determine whether there are meaningful differences between them. Therefore, for comparison purposes, the same discharging providers using the PAPR approach are eligible for the PAMM approach, so that both attribution models are using the same set of providers. All other providers are excluded because their performance would not be comparable to the PAPR approach.

While there are many ways to document provider-to-patient interactions, 2 methods are available in almost all health care systems. The first method is to link a provider’s billing charges to each patient-day combination. This approach limits the attribution to 1 provider per patient per day because multiple rotating providers cannot charge for the same patient-day combination.³ However, many providers interact with a patient on the same day, so using this approach excludes non-billed provider-to-patient interactions.

The second method, which was used in this study, relies on documented clinical notes within the EHR to determine how attribution is shared. In this approach, attribution is weighted based on the authorship of 3 types of eligible clinical notes: admitting history/physical notes (during admission), progress notes (during subsequent days), and discharge summary notes (during final discharge). This will (likely) result in many providers being linked to a patient on each day, which better reflects the clinical setting (Figure). Recently, clinical notes were used to attribute care of patients in an inpatient setting, and it was found that this approach provides a reliable way of tracking interactions and assigning ownership.¹¹

The provider-level attribution weights are based on the share of authorships of eligible note types. Specifically, for each provider j, let a_ij be the total count of eligible note types for hospitalization i authored by provider j, and let a_i be the overall total count of eligible note types for hospitalization i. Then the attribution weight is

(Eq. 1)

for hospitalization i and provider j. Note that ∑_jw_ij = 1: in other words, the total attribution, summed across all providers, is constrained to be 1 for each hospitalization.

Patient Outcomes

Outcomes were chosen based on their routine use in health care systems as standards when evaluating provider performance. This study included 6 outcomes: inpatient LOS, inpatient mortality, 30-day inpatient readmission, and patient responses from 3 survey questions. These outcomes can be collected without any manual chart reviews, and therefore are viewed as objective outcomes of provider performance.

Each outcome was aggregated for each provider using both attribution methods independently. For the PAPR method, observed-to-expected (OE) indices for LOS, mortality, and readmissions were calculated along with average patient survey scores. For the PAMM method, provider-level random effects from the fitted models were used. In both cases, the calculated measures were used for ranking purposes when determining top (or bottom) providers for each outcome.

Individual Provider Metrics for the PAPR Method

Inpatient LOS Index. Hospital inpatient LOS was measured as the number of days between admission date and discharge date. For each hospital visit, an expected LOS was determined using Premier’s CareScience Analytics (CSA) risk-adjustment methodology.¹⁰ The CSA methodology for LOS incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let o_ij and e_ij be the observed and expected LOS, respectively, for hospitalization i = 1,…,n_j attributed to provider j. Then the inpatient LOS index for provider j is L_j = ∑_io_ij⁄∑_ie_ij.

Inpatient Mortality Index. Inpatient mortality was defined as the death of the patient during hospitalization. For each hospitalization, an expected mortality probability was determined using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for mortality incorporates a patient’s demographics and comorbidities.

Just as before, let n_j be the number of hospitalizations attributed to provider j. Let m_ij = 1 if the patient died during hospitalization i = 1, … , n_j attributed to provider j; m_ij = 0 otherwise. Let p_ij^(m) be the corresponding expected mortality probability. Then the inpatient mortality index for provider j is M_j = ∑_im_ij⁄∑_ip_ij^(m).

30-Day Inpatient Readmission Index. A 30-day inpatient readmission was defined as the event when a patient is discharged and readmits back into the inpatient setting within 30 days. The inclusion criteria defined by the Centers for Medicare and Medicaid Services (CMS) all-cause hospital-wide readmission measure was used and, consequently, planned readmissions were excluded.¹² Readmissions could occur at any Banner hospital, including the same hospital. For each hospital visit, an expected readmission probability was derived using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for readmissions incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let r_ij = 1 if the patient had a readmission following hospitalization i = 1, … , n_j attributed to provider j; r_ij = 0 otherwise. Let p_ij^(r) be the corresponding expected readmission probability. Then the 30-day inpatient readmission index for provider j is R_j = ∑_ir_ij ⁄∑_ip_ij^(r).

Patient Survey Scores. The satisfaction of the patient’s experience during hospitalization was measured via post-discharge surveys administered by InMoment. Two survey questions were selected because they related directly to a provider’s interaction with the patient: “My interactions with doctors were excellent” (Doctor) and “I received the best possible care” (Care). A third question, “I would recommend this hospital to my family and friends,” was selected as a proxy measure of the overall experience and, in the aggregate, is referred to as the net promoter score (NPS).^13,14 The responses were measured on an 11-point Likert scale, ranging from “Strongly Disagree” (0) to “Strongly Agree” (10); “N/A” or missing responses were excluded.

The Likert responses were coded to 3 discrete values as follows: if the value was between 0 and 6, then -1 (ie, detractor); between 7 and 8 (ie, neutral), then 0; otherwise 1 (ie, promoter). Averaging these coded responses results in a patient survey score for each question. Specifically, let n_j be the number of hospitalizations attributed to provider j in which the patient responded to the survey question. Let s_ij ∈{−1, 0, 1} be the coded response linked to hospitalization i = 1, … , n_j attributed to provider j. Then the patient experience score for provider j is S_j = ∑_is_ij⁄n_j.

Handling Ties in Provider Performance Measures. Because ties can occur in the PAPR approach for all measures, a tie-breaking strategy is needed. For LOS indices, ties are less likely because their numerator is strictly greater than 0, and expected LOS values are typically distinct enough. Indeed, no ties were found in this study for LOS indices. However, mortality and readmission indices can routinely result in ties when the best possible index is achieved, such as 0 deaths or readmissions among attributed hospitalizations. To help differentiate between those indices in the PAPR approach, the total estimated risk (denominator) was utilized as a secondary scoring criterion.

Mortality and readmission metrics were addressed by sorting first by the outcome (mortality index), and second by the denominator (total estimated risk). For example, if provider A has the same mortality rate as provider B, then provider A would be ranked higher if the denominator was larger, indicating a higher risk for mortality.

Similarly, it was very common for providers to have the same overall average rating for a survey question. Therefore, the denominator (number of respondents) was used to break ties. However, the denominator sorting was bidirectional. For example, if the tied score was positive (more promoters than detractors) for providers A and B, then provider A would be ranked higher if the denominator was larger. Conversely, if the tied score between providers A and B was neutral or negative (more detractors than promoters), then provider A would be ranked lower if the denominator was larger.

Individual Provider Metrics for the PAMM Method

For the PAMM method, model-based metrics were derived using a MM model.⁸ Specifically, let J be the number of rotating providers in a health care system. Let Y_i be an outcome of interest from hospitalization i, X_1i, …, X_pi be fixed effects or covariates, and ß₁, …, ß_p be the coefficients for the respective covariates. Then the generalized MM statistical model is

(Eq. 2)

where g(μ_i) is a link function between the mean of the outcome, μ_i, and its linear predictor, ß₀, is the marginal intercept, w_ij represents the attribution weight of provider j on hospitalization i (described in Equation 1), and γ_j represents the random effect of provider j on the outcome with γ_j~N(0,σ_γ²).

For the mortality and readmission binary outcomes, logistic regression was performed using a logit link function, with the corresponding expected probability as the only fixed covariate. The expected probabilities were first converted into odds and then log-transformed before entering the model. For LOS, Poisson regression was performed using a log link function with the log-transformed expected LOS as the only fixed covariate. For coded patient experience responses, an ordered logistic regression was performed using a cumulative logit link function (no fixed effects were added).

MM Model-based Metrics. Each fitted MM model produces a predicted random effect for each provider. The provider-specific random effects can be interpreted as the unobserved influence of each provider on the outcome after controlling for any fixed effect included in the model. Therefore, the provider-specific random effects were used to evaluate the relative provider performance, which is analogous to the individual provider-level metrics used in the PAPR method.

Measuring provider performance using a MM model is more flexible and robust to outliers compared to the standard approach using OE indices or simple averages. First, although not investigated here, the effect of patient-, visit-, provider-, and/or temporal-level covariates can be controlled when evaluating provider performance. For example, a patient’s socioeconomic status, a provider’s workload, and seasonal factors can be added to the MM model. These external factors are not accounted for in OE indices.

Another advantage of using predicted random effects is the concept of “shrinkage.” The process of estimating random effects inherently accounts for small sample sizes (when providers do not treat a large enough sample of patients) and/or when there is a large ratio of patient variance to provider variance (for instance, when patient outcome variability is much higher compared to provider performance variability). In both cases, the estimation of the random effect is pulled ever closer to 0, signaling that the provider performance is closer to the population average. See Henderson¹⁵ and Mood¹⁶ for further details.

In contrast, OE indices can result in unreliable estimates when a provider has not cared for many patients. This is especially prevalent when the outcome is binary with a low probability of occurring, such as mortality. Indeed, provider-level mortality OE indices are routinely 0 when the patient counts are low, which skews performance rankings unfairly. Finally, OE indices also ignore the magnitude of the variance of an outcome between providers and patients, which can be large.

Comparison Methodology

In this study, we seek to compare the 2 methods of attribution, PAPR and PAMM, to determine whether there are meaningful differences between them when measuring provider performance. Using retrospective data described in the next section, each attribution method was used independently to derive provider-level metrics. To assess relative performance, percentiles were assigned to each provider based on their metric values so that, in the end, there were 2 percentile ranks for each provider for each metric.

Using these paired percentiles, we derived the following measures of concordance, similar to Herzke, Michtalik³: (1) the percent concordance measure—defined as the number of providers who landed in the top half (greater than the median) or bottom half under both attribution models—divided by the total number of providers; (2) the median of the absolute difference in percentiles under both attribution models; and (3) the Pearson correlation coefficient of the paired provider ranks. The first measure is a global measure of concordance between the 2 approaches and would be expected to be 50% by chance. The second measure gauges how an individual provider’s rank is affected by the change in attribution methodologies. The third measure is a statistical measure of linear correlation of the paired percentiles and was not included in the Herzke, Michtalik³ study.

All statistical analyses were performed on SAS (version 9.4; Cary, NC) and the MM models were fitted using PROC GLIMMIX with the EFFECT statement. The Banner Health Institutional Review Board approved this study.

Results

Descriptive Statistics

A total of 58,730 hospitalizations were included, of which care was provided by 963 unique providers across 25 acute care and critical access hospitals. Table 1 contains patient characteristics, and Table 2 depicts overall unadjusted outcomes. Providers responsible for less than 12 discharges in the calendar year were excluded from both approaches. Also, some hospitalizations were excluded when expected values were not available.

Multi-Membership Model Results

Table 3 displays the results after independently fitting MM models to each of the 3 clinical outcomes. Along with a marginal intercept, the only covariate in each model was the corresponding expected value after a transformation. This was added to use the same information that is typically used in OE indices, therefore allowing for a proper comparison between the 2 attribution methods. The provider-level variance represents the between-provider variation and measures the amount of influence providers have on the corresponding outcome after controlling for any covariates in the model. A provider-level variance of 0 would indicate that providers do not have any influence on the outcome. While the mortality and readmission model results can be compared to each other, the LOS model cannot given its different scale and transformation altogether.

The results in Table 3 suggest that each expected value covariate is highly correlated with its corresponding outcome, which is the anticipated conclusion given that they are constructed in this fashion. The estimated provider-level variances indicate that, after including an expected value in the model, providers have less of an influence on a patient’s LOS and likelihood of being readmitted. On the other hand, the results suggest that providers have much more influence on the likelihood of a patient dying in the hospital, even after controlling for an expected mortality covariate.

Table 4 shows the results after independently fitting MM-ordered logistic models to each of the 3 survey questions. The similar provider-level variances suggest that providers have the same influence on the patient’s perception of the quality of their interactions with the doctor (Doctor), the quality of the care they received (Care), and their likelihood to recommend a friend or family member to the hospital (NPS).

Comparison Results Between Both Attribution Methods

Table 5 compares the 2 attribution methods when ranking providers based on their performance on each outcome measure. The comparison metrics gauge how well the 2 methods agree overall (percent concordance), agree at the provider level (absolute percentile difference and interquartile range [IQR]), and how the paired percentiles linearly correlate to each other (Pearson correlation coefficient).

LOS, by a small margin, had the lowest concordance of clinical outcomes (72.1%), followed by mortality (75.9%) and readmissions (82.1%). Generally, the survey scores had higher percent concordance than the clinical outcome measures, with Doctor at 84.1%, Care at 85.9%, and NPS having the highest percent concordance at 86.6%. Given that by chance the percent concordance is expected to be 50%, there was notable discordance, especially with the clinical outcome measures. Using LOS performance as an example, one attribution methodology would rank a provider in the top half or bottom half, while the other attribution methodology would rank the same provider exactly the opposite way about 28% of the time.

The median absolute percentile difference between the 2 methods was more modest (between 7 and 15). Still, there were some providers whose performance ranking was heavily impacted by the attribution methodology that was used. This was especially true when evaluating performance for certain clinical measures, where the attribution method that was used could change the provider performance percentile by up to 90 levels.

The paired percentiles were positively correlated when ranking performance using any of the 6 measures. This suggests that both methodologies assess performance generally in the same direction, irrespective of the methodology and measure. We did not investigate more complex correlation measures and left this for future research.

It should be noted that ties occurred much more frequently with the PAPR method than when using PAMM and therefore required tie-breaking rules to be designed. Given the nature of OE indices, PAPR methodology is especially sensitive to ties whenever the measure includes counting the number of events (for example, mortality and readmissions) and whenever there are many providers with very few attributed patients. On the other hand, using the PAMM method is much more robust against ties given that the summation of all the weighted attributed outcomes will rarely result in ties, even with a nominal set of providers.

In this study, the PAMM methodology was introduced and was used to assess relative provider performance on 3 clinical outcome measures and 3 patient survey scores. The new approach aims to distribute each outcome among all providers who provided care for a patient in an inpatient setting. Clinical notes were used to account for patient-to-provider interactions, and fitted MM statistical models were used to compute the effects that each provider had on each outcome. The provider effect was introduced as a random effect, and the set of predicted random effects was used to rank the performance of each provider.

The PAMM approach was compared to the more traditional methodology, PAPR, where each patient is attributed to only 1 provider: the discharging physician in this study. Using this approach, OE indices of clinical outcomes and averages of survey scores were used to rank the performance of each provider. This approach resulted in many ties, which were broken based on the number of hospitalizations, although other tie-breaking methods may be used in practice.

Both methodologies showed modest concordance with each other for the clinical outcomes, but higher concordance for the patient survey scores. This was also true when using the Pearson correlation coefficient to assess agreement. The 1 outcome measure that showed the least concordance and least linear correlation between methods was LOS, which would suggest that LOS performance is more sensitive to the attribution methodology that is used. However, it was the least concordant by a small margin.

Furthermore, although the medians of the absolute percentile differences were small, there were some providers who had large deviations, suggesting that some providers would move from being shown as high-performers to low-performers and vice versa based on the chosen attribution method. We investigated examples of this and determined that the root cause was the difference in effective sample sizes for a provider. For the PAPR method, the effective sample size is simply the number of hospitalizations attributed to the provider. For the PAMM method, the effective sample size is the sum of all non-zero weights across all hospitalizations where the provider cared for a patient. By and large, the PAMM methodology provides more information of the provider effect on an outcome than the PAPR approach because every provider-patient interaction is considered. For example, providers who do not routinely discharge patients, but often care for patients, will have rankings that differ dramatically between the 2 methods.

The PAMM methodology has many statistical advantages that were not fully utilized in this comparative study. For example, we did not include any covariates in the MM models except for the expected value of the outcome, when it was available. Still, it is known that other covariates can impact an outcome as well, such as the patient’s age, socioeconomic indicators, existing chronic conditions, and severity of hospitalization, which can be added to the MM models as fixed effects. In this way, the PAMM approach can control for these other covariates, which are typically outside of the control of providers but typically ignored using OE indices. Therefore, using the PAMM approach would provide a fairer comparison of provider performance.

Using the PAMM method, most providers had a large sample size to assess their performance once all the weighted interactions were included. Still, there were a few who did not care for many patients for a variety of reasons. In these scenarios, MM models “borrow” strength from other providers to produce a more robust predicted provider effect by using a weighted average between the overall population trend and the specific provider outcomes (see Rao and Molina¹⁷). As a result, PAMM is a more suitable approach when the sample sizes of patients attributed to providers can be small.

One of the most interesting findings of this study was the relative size of the provider-level variance to the size of the fixed effect in each model (Table 3). Except for mortality, these variances suggest that there is a small difference in performance from one provider to another. However, these should be interpreted as the variance when only 1 provider is involved in the care of a patient. When multiple providers are involved, using basic statistical theory, the overall provider-level variance will be σ_γ² ∑w_ij² (see Equation 2). For example, the estimated variance among providers for LOS was 0.03 (on a log scale), but, using the scenario in the Figure, the overall provider-level variance for this hospitalization will be 0.03 (0.375²+ 0.125²+ 0.5²) = 0.012. Hence, the combined effect of providers on LOS is less than would be expected. Indeed, as more providers are involved with a patient’s care, the more their combined influence on an outcome is diluted.

In this study, the PAMM approach placed an equal weight on all provider-patient interactions via clinical note authorship, but that may not be optimal in some settings. For example, it may make more sense to set a higher weight on the provider who admitted or discharged the patient while placing less (or 0) weight on all other interactions. In the extreme, if the full weight were placed on 1 provider interaction (eg, during discharge, then the MM model would be reduced to a one-way random effects model. The flexibility of weighting interactions is a feature of the PAMM approach, but any weighting framework must be transparent to the providers before implementation.

Conclusion

This study demonstrates that the PAMM approach is a feasible option within a large health care organization. For P4P programs to be successful, providers must be able to trust that their performance will be fairly assessed and that all provider-patient interactions are captured to provide a full comparison amongst their peers. The PAMM methodology is one solution to spread the positive (and negative) outcomes across all providers who cared for a patient and therefore, if implemented, would add trust and fairness when measuring and assessing provider performance.

Acknowledgments: The authors thank Barrie Bradley for his support in the initial stages of this research and Dr. Syed Ismail Jafri for his help and support on the standard approaches of assessing and measuring provider performances.

Corresponding author: Rachel Ginn, MS, Banner Health Corporation, 2901 N. Central Ave., Phoenix, AZ 85012; [email protected].

Financial disclosures: None.

From Banner Health Corporation, Phoenix, AZ.

Background: Health care providers are routinely incentivized with pay-for-performance (P4P) metrics to increase the quality of care. In an inpatient setting, P4P models typically measure quality by attributing each patient’s outcome to a single provider even though many providers routinely care for the patient. This study investigates a new attribution approach aiming to distribute each outcome across all providers who provided care.

Methods: The methodology relies on a multi-membership model and is demonstrated in the Banner Health system using 3 clinical outcome measures (length of stay, 30-day readmissions, and mortality) and responses to 3 survey questions that measure a patient’s perception of their care. The new approach is compared to the “standard” method, which attributes each patient to only 1 provider.

Results: When ranking by clinical outcomes, both methods were concordant 72.1% to 82.1% of the time for top-half/bottom-half rankings, with a median percentile difference between 7 and 15. When ranking by survey scores, there was more agreement, with concordance between 84.1% and 86.6% and a median percentile difference between 11 and 13. Last, Pearson correlation coefficients of the paired percentiles ranged from 0.56 to 0.78.

Conclusion: The new approach provides a fairer solution when measuring provider performance.

Keywords: patient attribution; PAMM; PAPR; random effect model; pay for performance.

Providers practicing in hospitals are routinely evaluated based on their performance and, in many cases, are financially incentivized for a better-than-average performance within a pay-for-performance (P4P) model. The use of P4P models is based on the belief that they will “improve, motivate, and enhance providers to pursue aggressively and ultimately achieve the quality performance targets thus decreasing the number of medical errors with less malpractice events.”¹ Although P4P models continue to be a movement in health care, they have been challenging to implement.

One concern involves the general quality of implementation, such as defining metrics and targets, setting payout amounts, managing technology and market conditions, and gauging the level of transparency to the provider.² Another challenge, and the focus of this project, are concerns around measuring performance to avoid perceptions of unfairness. This concern can be minimized if the attribution is handled in a fairer way, by spreading it across all providers who affected the outcome, both in a positive or negative direction.³

To implement these models, the performance of providers needs to be measured and tracked periodically. This requires linking, or attributing, a patient’s outcome to a provider, which is almost always the attending or discharging provider (ie, a single provider).³ In this single-provider attribution approach, one provider will receive all the credit (good or bad) for their respective patients’ outcomes, even though the provider may have seen the patient only a fraction of the time during the hospitalization. Attributing outcomes—for example, length of stay (LOS), readmission rate, mortality rate, net promoter score (NPS)—using this approach reduces the validity of metrics designed to measure provider performance, especially in a rotating provider environment where many providers interact with and care for a patient. For example, the quality of providers’ interpersonal skills and competence were among the strongest determinants of patient satisfaction,⁴ but it is not credible that this is solely based on the last provider during a hospitalization.

Proportionally distributing the attribution of an outcome has been used successfully in other contexts. Typically, a statistical modeling approach using a multi-membership framework is used because it can handle the sometimes-complicated relationships within the hierarchy. It also allows for auxiliary variables to be introduced, which can help explain and control for exogenous effects.^5-7 For example, in the education setting, standardized testing is administered to students at defined years of schooling: at grades 4, 8, and 10, for instance. The progress of students, measured as the academic gains between test years, are proportionally attributed to all the teachers who the student has had between the test years. These partial attributions are combined to evaluate an overall teacher performance.^8,9

Although the multi-membership framework has been used in other industries, it has yet to be applied in measuring provider performance. The purpose of this project is to investigate the impact of using a multi-provider approach compared to the standard single-provider approach. The findings may lead to modifications in the way a provider’s performance is measured and, thus, how providers are compensated. A similar study investigated the impact of proportionally distributing patients’ outcomes across all rotating providers using a weighting method based on billing practices to measure the partial impact of each provider.³

This study is different in 2 fundamental ways. First, attribution is weighted based on the number of clinically documented interactions (via clinical notes) between a patient and all rotating providers during the hospitalization. Second, performance is measured via multi-membership models, which can estimate the effect (both positive and negative) that a provider has on an outcome, even when caring for a patient a fraction of the time during the hospitalization.

Methods

Setting

Banner Health is a non-profit, multi-hospital health care system across 6 states in the western United States that is uniquely positioned to study provider quality attribution models. It not only has a large number of providers and serves a broad patient population, but Banner Health also uses an instance of Cerner (Kansas City, MO), an enterprise-level electronic health record (EHR) system that connects all its facilities and allows for advanced analytics across its system.

For this study, we included only general medicine and surgery patients admitted and discharged from the inpatient setting between January 1, 2018, and December 31, 2018, who were between 18 and 89 years old at admission, and who had a LOS between 1 and 14 days. Visit- and patient-level data were collected from Cerner, while outcome data, and corresponding expected outcome data, were obtained from Premier, Inc. (Charlotte, NC) using their CareScience methodologies.¹⁰ To measure patient experience, response data were extracted from post-discharge surveys administered by InMoment (Salt Lake City, UT).

Provider Attribution Models

Provider Attribution by Physician of Record (PAPR). In the standard approach, denoted here as the PAPR model, 1 provider—typically the attending or discharging provider, which may be the same person—is attributed to the entire hospitalization. This provider is responsible for the patient’s care, and all patient outcomes are aggregated and attributed to the provider to gauge his or her performance. The PAPR model is the most popular form of attribution across many health care systems and is routinely used for P4P incentives.

In this study, the discharging provider was used when attributing hospitalizations using the PAPR model. Providers responsible for fewer than 12 discharges in the calendar year were excluded. Because of the directness of this type of attribution, the performance of 1 provider does not account for the performance of the other rotating providers during hospitalizations.

Provider Attribution by Multiple Membership (PAMM). In contrast, we introduce another attribution approach here that is designed to assign partial attribution to each provider who cares for the patient during the hospitalization. To aggregate the partial attributions, and possibly control for any exogenous or risk-based factors, a multiple-membership, or multi-member (MM), model is used. The MM model can measure the effect of a provider on an outcome even when the patient-to-provider relationship is complex, such as in a rotating provider environment.⁸

The purpose of this study is to compare attribution models and to determine whether there are meaningful differences between them. Therefore, for comparison purposes, the same discharging providers using the PAPR approach are eligible for the PAMM approach, so that both attribution models are using the same set of providers. All other providers are excluded because their performance would not be comparable to the PAPR approach.

While there are many ways to document provider-to-patient interactions, 2 methods are available in almost all health care systems. The first method is to link a provider’s billing charges to each patient-day combination. This approach limits the attribution to 1 provider per patient per day because multiple rotating providers cannot charge for the same patient-day combination.³ However, many providers interact with a patient on the same day, so using this approach excludes non-billed provider-to-patient interactions.

The second method, which was used in this study, relies on documented clinical notes within the EHR to determine how attribution is shared. In this approach, attribution is weighted based on the authorship of 3 types of eligible clinical notes: admitting history/physical notes (during admission), progress notes (during subsequent days), and discharge summary notes (during final discharge). This will (likely) result in many providers being linked to a patient on each day, which better reflects the clinical setting (Figure). Recently, clinical notes were used to attribute care of patients in an inpatient setting, and it was found that this approach provides a reliable way of tracking interactions and assigning ownership.¹¹

The provider-level attribution weights are based on the share of authorships of eligible note types. Specifically, for each provider j, let a_ij be the total count of eligible note types for hospitalization i authored by provider j, and let a_i be the overall total count of eligible note types for hospitalization i. Then the attribution weight is

(Eq. 1)

for hospitalization i and provider j. Note that ∑_jw_ij = 1: in other words, the total attribution, summed across all providers, is constrained to be 1 for each hospitalization.

Patient Outcomes

Outcomes were chosen based on their routine use in health care systems as standards when evaluating provider performance. This study included 6 outcomes: inpatient LOS, inpatient mortality, 30-day inpatient readmission, and patient responses from 3 survey questions. These outcomes can be collected without any manual chart reviews, and therefore are viewed as objective outcomes of provider performance.

Each outcome was aggregated for each provider using both attribution methods independently. For the PAPR method, observed-to-expected (OE) indices for LOS, mortality, and readmissions were calculated along with average patient survey scores. For the PAMM method, provider-level random effects from the fitted models were used. In both cases, the calculated measures were used for ranking purposes when determining top (or bottom) providers for each outcome.

Individual Provider Metrics for the PAPR Method

Inpatient LOS Index. Hospital inpatient LOS was measured as the number of days between admission date and discharge date. For each hospital visit, an expected LOS was determined using Premier’s CareScience Analytics (CSA) risk-adjustment methodology.¹⁰ The CSA methodology for LOS incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let o_ij and e_ij be the observed and expected LOS, respectively, for hospitalization i = 1,…,n_j attributed to provider j. Then the inpatient LOS index for provider j is L_j = ∑_io_ij⁄∑_ie_ij.

Inpatient Mortality Index. Inpatient mortality was defined as the death of the patient during hospitalization. For each hospitalization, an expected mortality probability was determined using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for mortality incorporates a patient’s demographics and comorbidities.

Just as before, let n_j be the number of hospitalizations attributed to provider j. Let m_ij = 1 if the patient died during hospitalization i = 1, … , n_j attributed to provider j; m_ij = 0 otherwise. Let p_ij^(m) be the corresponding expected mortality probability. Then the inpatient mortality index for provider j is M_j = ∑_im_ij⁄∑_ip_ij^(m).

30-Day Inpatient Readmission Index. A 30-day inpatient readmission was defined as the event when a patient is discharged and readmits back into the inpatient setting within 30 days. The inclusion criteria defined by the Centers for Medicare and Medicaid Services (CMS) all-cause hospital-wide readmission measure was used and, consequently, planned readmissions were excluded.¹² Readmissions could occur at any Banner hospital, including the same hospital. For each hospital visit, an expected readmission probability was derived using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for readmissions incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let r_ij = 1 if the patient had a readmission following hospitalization i = 1, … , n_j attributed to provider j; r_ij = 0 otherwise. Let p_ij^(r) be the corresponding expected readmission probability. Then the 30-day inpatient readmission index for provider j is R_j = ∑_ir_ij ⁄∑_ip_ij^(r).

Patient Survey Scores. The satisfaction of the patient’s experience during hospitalization was measured via post-discharge surveys administered by InMoment. Two survey questions were selected because they related directly to a provider’s interaction with the patient: “My interactions with doctors were excellent” (Doctor) and “I received the best possible care” (Care). A third question, “I would recommend this hospital to my family and friends,” was selected as a proxy measure of the overall experience and, in the aggregate, is referred to as the net promoter score (NPS).^13,14 The responses were measured on an 11-point Likert scale, ranging from “Strongly Disagree” (0) to “Strongly Agree” (10); “N/A” or missing responses were excluded.

The Likert responses were coded to 3 discrete values as follows: if the value was between 0 and 6, then -1 (ie, detractor); between 7 and 8 (ie, neutral), then 0; otherwise 1 (ie, promoter). Averaging these coded responses results in a patient survey score for each question. Specifically, let n_j be the number of hospitalizations attributed to provider j in which the patient responded to the survey question. Let s_ij ∈{−1, 0, 1} be the coded response linked to hospitalization i = 1, … , n_j attributed to provider j. Then the patient experience score for provider j is S_j = ∑_is_ij⁄n_j.

Handling Ties in Provider Performance Measures. Because ties can occur in the PAPR approach for all measures, a tie-breaking strategy is needed. For LOS indices, ties are less likely because their numerator is strictly greater than 0, and expected LOS values are typically distinct enough. Indeed, no ties were found in this study for LOS indices. However, mortality and readmission indices can routinely result in ties when the best possible index is achieved, such as 0 deaths or readmissions among attributed hospitalizations. To help differentiate between those indices in the PAPR approach, the total estimated risk (denominator) was utilized as a secondary scoring criterion.

Mortality and readmission metrics were addressed by sorting first by the outcome (mortality index), and second by the denominator (total estimated risk). For example, if provider A has the same mortality rate as provider B, then provider A would be ranked higher if the denominator was larger, indicating a higher risk for mortality.

Similarly, it was very common for providers to have the same overall average rating for a survey question. Therefore, the denominator (number of respondents) was used to break ties. However, the denominator sorting was bidirectional. For example, if the tied score was positive (more promoters than detractors) for providers A and B, then provider A would be ranked higher if the denominator was larger. Conversely, if the tied score between providers A and B was neutral or negative (more detractors than promoters), then provider A would be ranked lower if the denominator was larger.

Individual Provider Metrics for the PAMM Method

For the PAMM method, model-based metrics were derived using a MM model.⁸ Specifically, let J be the number of rotating providers in a health care system. Let Y_i be an outcome of interest from hospitalization i, X_1i, …, X_pi be fixed effects or covariates, and ß₁, …, ß_p be the coefficients for the respective covariates. Then the generalized MM statistical model is

(Eq. 2)

where g(μ_i) is a link function between the mean of the outcome, μ_i, and its linear predictor, ß₀, is the marginal intercept, w_ij represents the attribution weight of provider j on hospitalization i (described in Equation 1), and γ_j represents the random effect of provider j on the outcome with γ_j~N(0,σ_γ²).

For the mortality and readmission binary outcomes, logistic regression was performed using a logit link function, with the corresponding expected probability as the only fixed covariate. The expected probabilities were first converted into odds and then log-transformed before entering the model. For LOS, Poisson regression was performed using a log link function with the log-transformed expected LOS as the only fixed covariate. For coded patient experience responses, an ordered logistic regression was performed using a cumulative logit link function (no fixed effects were added).

MM Model-based Metrics. Each fitted MM model produces a predicted random effect for each provider. The provider-specific random effects can be interpreted as the unobserved influence of each provider on the outcome after controlling for any fixed effect included in the model. Therefore, the provider-specific random effects were used to evaluate the relative provider performance, which is analogous to the individual provider-level metrics used in the PAPR method.

Measuring provider performance using a MM model is more flexible and robust to outliers compared to the standard approach using OE indices or simple averages. First, although not investigated here, the effect of patient-, visit-, provider-, and/or temporal-level covariates can be controlled when evaluating provider performance. For example, a patient’s socioeconomic status, a provider’s workload, and seasonal factors can be added to the MM model. These external factors are not accounted for in OE indices.

Another advantage of using predicted random effects is the concept of “shrinkage.” The process of estimating random effects inherently accounts for small sample sizes (when providers do not treat a large enough sample of patients) and/or when there is a large ratio of patient variance to provider variance (for instance, when patient outcome variability is much higher compared to provider performance variability). In both cases, the estimation of the random effect is pulled ever closer to 0, signaling that the provider performance is closer to the population average. See Henderson¹⁵ and Mood¹⁶ for further details.

In contrast, OE indices can result in unreliable estimates when a provider has not cared for many patients. This is especially prevalent when the outcome is binary with a low probability of occurring, such as mortality. Indeed, provider-level mortality OE indices are routinely 0 when the patient counts are low, which skews performance rankings unfairly. Finally, OE indices also ignore the magnitude of the variance of an outcome between providers and patients, which can be large.

Comparison Methodology

In this study, we seek to compare the 2 methods of attribution, PAPR and PAMM, to determine whether there are meaningful differences between them when measuring provider performance. Using retrospective data described in the next section, each attribution method was used independently to derive provider-level metrics. To assess relative performance, percentiles were assigned to each provider based on their metric values so that, in the end, there were 2 percentile ranks for each provider for each metric.

Using these paired percentiles, we derived the following measures of concordance, similar to Herzke, Michtalik³: (1) the percent concordance measure—defined as the number of providers who landed in the top half (greater than the median) or bottom half under both attribution models—divided by the total number of providers; (2) the median of the absolute difference in percentiles under both attribution models; and (3) the Pearson correlation coefficient of the paired provider ranks. The first measure is a global measure of concordance between the 2 approaches and would be expected to be 50% by chance. The second measure gauges how an individual provider’s rank is affected by the change in attribution methodologies. The third measure is a statistical measure of linear correlation of the paired percentiles and was not included in the Herzke, Michtalik³ study.

All statistical analyses were performed on SAS (version 9.4; Cary, NC) and the MM models were fitted using PROC GLIMMIX with the EFFECT statement. The Banner Health Institutional Review Board approved this study.

Results

Descriptive Statistics

A total of 58,730 hospitalizations were included, of which care was provided by 963 unique providers across 25 acute care and critical access hospitals. Table 1 contains patient characteristics, and Table 2 depicts overall unadjusted outcomes. Providers responsible for less than 12 discharges in the calendar year were excluded from both approaches. Also, some hospitalizations were excluded when expected values were not available.

Multi-Membership Model Results

Table 3 displays the results after independently fitting MM models to each of the 3 clinical outcomes. Along with a marginal intercept, the only covariate in each model was the corresponding expected value after a transformation. This was added to use the same information that is typically used in OE indices, therefore allowing for a proper comparison between the 2 attribution methods. The provider-level variance represents the between-provider variation and measures the amount of influence providers have on the corresponding outcome after controlling for any covariates in the model. A provider-level variance of 0 would indicate that providers do not have any influence on the outcome. While the mortality and readmission model results can be compared to each other, the LOS model cannot given its different scale and transformation altogether.

The results in Table 3 suggest that each expected value covariate is highly correlated with its corresponding outcome, which is the anticipated conclusion given that they are constructed in this fashion. The estimated provider-level variances indicate that, after including an expected value in the model, providers have less of an influence on a patient’s LOS and likelihood of being readmitted. On the other hand, the results suggest that providers have much more influence on the likelihood of a patient dying in the hospital, even after controlling for an expected mortality covariate.

Table 4 shows the results after independently fitting MM-ordered logistic models to each of the 3 survey questions. The similar provider-level variances suggest that providers have the same influence on the patient’s perception of the quality of their interactions with the doctor (Doctor), the quality of the care they received (Care), and their likelihood to recommend a friend or family member to the hospital (NPS).

Comparison Results Between Both Attribution Methods

Table 5 compares the 2 attribution methods when ranking providers based on their performance on each outcome measure. The comparison metrics gauge how well the 2 methods agree overall (percent concordance), agree at the provider level (absolute percentile difference and interquartile range [IQR]), and how the paired percentiles linearly correlate to each other (Pearson correlation coefficient).

LOS, by a small margin, had the lowest concordance of clinical outcomes (72.1%), followed by mortality (75.9%) and readmissions (82.1%). Generally, the survey scores had higher percent concordance than the clinical outcome measures, with Doctor at 84.1%, Care at 85.9%, and NPS having the highest percent concordance at 86.6%. Given that by chance the percent concordance is expected to be 50%, there was notable discordance, especially with the clinical outcome measures. Using LOS performance as an example, one attribution methodology would rank a provider in the top half or bottom half, while the other attribution methodology would rank the same provider exactly the opposite way about 28% of the time.

The median absolute percentile difference between the 2 methods was more modest (between 7 and 15). Still, there were some providers whose performance ranking was heavily impacted by the attribution methodology that was used. This was especially true when evaluating performance for certain clinical measures, where the attribution method that was used could change the provider performance percentile by up to 90 levels.

The paired percentiles were positively correlated when ranking performance using any of the 6 measures. This suggests that both methodologies assess performance generally in the same direction, irrespective of the methodology and measure. We did not investigate more complex correlation measures and left this for future research.

It should be noted that ties occurred much more frequently with the PAPR method than when using PAMM and therefore required tie-breaking rules to be designed. Given the nature of OE indices, PAPR methodology is especially sensitive to ties whenever the measure includes counting the number of events (for example, mortality and readmissions) and whenever there are many providers with very few attributed patients. On the other hand, using the PAMM method is much more robust against ties given that the summation of all the weighted attributed outcomes will rarely result in ties, even with a nominal set of providers.

In this study, the PAMM methodology was introduced and was used to assess relative provider performance on 3 clinical outcome measures and 3 patient survey scores. The new approach aims to distribute each outcome among all providers who provided care for a patient in an inpatient setting. Clinical notes were used to account for patient-to-provider interactions, and fitted MM statistical models were used to compute the effects that each provider had on each outcome. The provider effect was introduced as a random effect, and the set of predicted random effects was used to rank the performance of each provider.

The PAMM approach was compared to the more traditional methodology, PAPR, where each patient is attributed to only 1 provider: the discharging physician in this study. Using this approach, OE indices of clinical outcomes and averages of survey scores were used to rank the performance of each provider. This approach resulted in many ties, which were broken based on the number of hospitalizations, although other tie-breaking methods may be used in practice.

Both methodologies showed modest concordance with each other for the clinical outcomes, but higher concordance for the patient survey scores. This was also true when using the Pearson correlation coefficient to assess agreement. The 1 outcome measure that showed the least concordance and least linear correlation between methods was LOS, which would suggest that LOS performance is more sensitive to the attribution methodology that is used. However, it was the least concordant by a small margin.

Furthermore, although the medians of the absolute percentile differences were small, there were some providers who had large deviations, suggesting that some providers would move from being shown as high-performers to low-performers and vice versa based on the chosen attribution method. We investigated examples of this and determined that the root cause was the difference in effective sample sizes for a provider. For the PAPR method, the effective sample size is simply the number of hospitalizations attributed to the provider. For the PAMM method, the effective sample size is the sum of all non-zero weights across all hospitalizations where the provider cared for a patient. By and large, the PAMM methodology provides more information of the provider effect on an outcome than the PAPR approach because every provider-patient interaction is considered. For example, providers who do not routinely discharge patients, but often care for patients, will have rankings that differ dramatically between the 2 methods.

The PAMM methodology has many statistical advantages that were not fully utilized in this comparative study. For example, we did not include any covariates in the MM models except for the expected value of the outcome, when it was available. Still, it is known that other covariates can impact an outcome as well, such as the patient’s age, socioeconomic indicators, existing chronic conditions, and severity of hospitalization, which can be added to the MM models as fixed effects. In this way, the PAMM approach can control for these other covariates, which are typically outside of the control of providers but typically ignored using OE indices. Therefore, using the PAMM approach would provide a fairer comparison of provider performance.

Using the PAMM method, most providers had a large sample size to assess their performance once all the weighted interactions were included. Still, there were a few who did not care for many patients for a variety of reasons. In these scenarios, MM models “borrow” strength from other providers to produce a more robust predicted provider effect by using a weighted average between the overall population trend and the specific provider outcomes (see Rao and Molina¹⁷). As a result, PAMM is a more suitable approach when the sample sizes of patients attributed to providers can be small.

One of the most interesting findings of this study was the relative size of the provider-level variance to the size of the fixed effect in each model (Table 3). Except for mortality, these variances suggest that there is a small difference in performance from one provider to another. However, these should be interpreted as the variance when only 1 provider is involved in the care of a patient. When multiple providers are involved, using basic statistical theory, the overall provider-level variance will be σ_γ² ∑w_ij² (see Equation 2). For example, the estimated variance among providers for LOS was 0.03 (on a log scale), but, using the scenario in the Figure, the overall provider-level variance for this hospitalization will be 0.03 (0.375²+ 0.125²+ 0.5²) = 0.012. Hence, the combined effect of providers on LOS is less than would be expected. Indeed, as more providers are involved with a patient’s care, the more their combined influence on an outcome is diluted.

In this study, the PAMM approach placed an equal weight on all provider-patient interactions via clinical note authorship, but that may not be optimal in some settings. For example, it may make more sense to set a higher weight on the provider who admitted or discharged the patient while placing less (or 0) weight on all other interactions. In the extreme, if the full weight were placed on 1 provider interaction (eg, during discharge, then the MM model would be reduced to a one-way random effects model. The flexibility of weighting interactions is a feature of the PAMM approach, but any weighting framework must be transparent to the providers before implementation.

Conclusion

This study demonstrates that the PAMM approach is a feasible option within a large health care organization. For P4P programs to be successful, providers must be able to trust that their performance will be fairly assessed and that all provider-patient interactions are captured to provide a full comparison amongst their peers. The PAMM methodology is one solution to spread the positive (and negative) outcomes across all providers who cared for a patient and therefore, if implemented, would add trust and fairness when measuring and assessing provider performance.

Acknowledgments: The authors thank Barrie Bradley for his support in the initial stages of this research and Dr. Syed Ismail Jafri for his help and support on the standard approaches of assessing and measuring provider performances.

Corresponding author: Rachel Ginn, MS, Banner Health Corporation, 2901 N. Central Ave., Phoenix, AZ 85012; [email protected].

Financial disclosures: None.

From Banner Health Corporation, Phoenix, AZ.

Background: Health care providers are routinely incentivized with pay-for-performance (P4P) metrics to increase the quality of care. In an inpatient setting, P4P models typically measure quality by attributing each patient’s outcome to a single provider even though many providers routinely care for the patient. This study investigates a new attribution approach aiming to distribute each outcome across all providers who provided care.

Methods: The methodology relies on a multi-membership model and is demonstrated in the Banner Health system using 3 clinical outcome measures (length of stay, 30-day readmissions, and mortality) and responses to 3 survey questions that measure a patient’s perception of their care. The new approach is compared to the “standard” method, which attributes each patient to only 1 provider.

Results: When ranking by clinical outcomes, both methods were concordant 72.1% to 82.1% of the time for top-half/bottom-half rankings, with a median percentile difference between 7 and 15. When ranking by survey scores, there was more agreement, with concordance between 84.1% and 86.6% and a median percentile difference between 11 and 13. Last, Pearson correlation coefficients of the paired percentiles ranged from 0.56 to 0.78.

Conclusion: The new approach provides a fairer solution when measuring provider performance.

Keywords: patient attribution; PAMM; PAPR; random effect model; pay for performance.

Providers practicing in hospitals are routinely evaluated based on their performance and, in many cases, are financially incentivized for a better-than-average performance within a pay-for-performance (P4P) model. The use of P4P models is based on the belief that they will “improve, motivate, and enhance providers to pursue aggressively and ultimately achieve the quality performance targets thus decreasing the number of medical errors with less malpractice events.”¹ Although P4P models continue to be a movement in health care, they have been challenging to implement.

One concern involves the general quality of implementation, such as defining metrics and targets, setting payout amounts, managing technology and market conditions, and gauging the level of transparency to the provider.² Another challenge, and the focus of this project, are concerns around measuring performance to avoid perceptions of unfairness. This concern can be minimized if the attribution is handled in a fairer way, by spreading it across all providers who affected the outcome, both in a positive or negative direction.³

To implement these models, the performance of providers needs to be measured and tracked periodically. This requires linking, or attributing, a patient’s outcome to a provider, which is almost always the attending or discharging provider (ie, a single provider).³ In this single-provider attribution approach, one provider will receive all the credit (good or bad) for their respective patients’ outcomes, even though the provider may have seen the patient only a fraction of the time during the hospitalization. Attributing outcomes—for example, length of stay (LOS), readmission rate, mortality rate, net promoter score (NPS)—using this approach reduces the validity of metrics designed to measure provider performance, especially in a rotating provider environment where many providers interact with and care for a patient. For example, the quality of providers’ interpersonal skills and competence were among the strongest determinants of patient satisfaction,⁴ but it is not credible that this is solely based on the last provider during a hospitalization.

Proportionally distributing the attribution of an outcome has been used successfully in other contexts. Typically, a statistical modeling approach using a multi-membership framework is used because it can handle the sometimes-complicated relationships within the hierarchy. It also allows for auxiliary variables to be introduced, which can help explain and control for exogenous effects.^5-7 For example, in the education setting, standardized testing is administered to students at defined years of schooling: at grades 4, 8, and 10, for instance. The progress of students, measured as the academic gains between test years, are proportionally attributed to all the teachers who the student has had between the test years. These partial attributions are combined to evaluate an overall teacher performance.^8,9

Although the multi-membership framework has been used in other industries, it has yet to be applied in measuring provider performance. The purpose of this project is to investigate the impact of using a multi-provider approach compared to the standard single-provider approach. The findings may lead to modifications in the way a provider’s performance is measured and, thus, how providers are compensated. A similar study investigated the impact of proportionally distributing patients’ outcomes across all rotating providers using a weighting method based on billing practices to measure the partial impact of each provider.³

This study is different in 2 fundamental ways. First, attribution is weighted based on the number of clinically documented interactions (via clinical notes) between a patient and all rotating providers during the hospitalization. Second, performance is measured via multi-membership models, which can estimate the effect (both positive and negative) that a provider has on an outcome, even when caring for a patient a fraction of the time during the hospitalization.

Methods

Setting

Banner Health is a non-profit, multi-hospital health care system across 6 states in the western United States that is uniquely positioned to study provider quality attribution models. It not only has a large number of providers and serves a broad patient population, but Banner Health also uses an instance of Cerner (Kansas City, MO), an enterprise-level electronic health record (EHR) system that connects all its facilities and allows for advanced analytics across its system.

For this study, we included only general medicine and surgery patients admitted and discharged from the inpatient setting between January 1, 2018, and December 31, 2018, who were between 18 and 89 years old at admission, and who had a LOS between 1 and 14 days. Visit- and patient-level data were collected from Cerner, while outcome data, and corresponding expected outcome data, were obtained from Premier, Inc. (Charlotte, NC) using their CareScience methodologies.¹⁰ To measure patient experience, response data were extracted from post-discharge surveys administered by InMoment (Salt Lake City, UT).

Provider Attribution Models

Provider Attribution by Physician of Record (PAPR). In the standard approach, denoted here as the PAPR model, 1 provider—typically the attending or discharging provider, which may be the same person—is attributed to the entire hospitalization. This provider is responsible for the patient’s care, and all patient outcomes are aggregated and attributed to the provider to gauge his or her performance. The PAPR model is the most popular form of attribution across many health care systems and is routinely used for P4P incentives.

In this study, the discharging provider was used when attributing hospitalizations using the PAPR model. Providers responsible for fewer than 12 discharges in the calendar year were excluded. Because of the directness of this type of attribution, the performance of 1 provider does not account for the performance of the other rotating providers during hospitalizations.

Provider Attribution by Multiple Membership (PAMM). In contrast, we introduce another attribution approach here that is designed to assign partial attribution to each provider who cares for the patient during the hospitalization. To aggregate the partial attributions, and possibly control for any exogenous or risk-based factors, a multiple-membership, or multi-member (MM), model is used. The MM model can measure the effect of a provider on an outcome even when the patient-to-provider relationship is complex, such as in a rotating provider environment.⁸

The purpose of this study is to compare attribution models and to determine whether there are meaningful differences between them. Therefore, for comparison purposes, the same discharging providers using the PAPR approach are eligible for the PAMM approach, so that both attribution models are using the same set of providers. All other providers are excluded because their performance would not be comparable to the PAPR approach.

While there are many ways to document provider-to-patient interactions, 2 methods are available in almost all health care systems. The first method is to link a provider’s billing charges to each patient-day combination. This approach limits the attribution to 1 provider per patient per day because multiple rotating providers cannot charge for the same patient-day combination.³ However, many providers interact with a patient on the same day, so using this approach excludes non-billed provider-to-patient interactions.

The second method, which was used in this study, relies on documented clinical notes within the EHR to determine how attribution is shared. In this approach, attribution is weighted based on the authorship of 3 types of eligible clinical notes: admitting history/physical notes (during admission), progress notes (during subsequent days), and discharge summary notes (during final discharge). This will (likely) result in many providers being linked to a patient on each day, which better reflects the clinical setting (Figure). Recently, clinical notes were used to attribute care of patients in an inpatient setting, and it was found that this approach provides a reliable way of tracking interactions and assigning ownership.¹¹

The provider-level attribution weights are based on the share of authorships of eligible note types. Specifically, for each provider j, let a_ij be the total count of eligible note types for hospitalization i authored by provider j, and let a_i be the overall total count of eligible note types for hospitalization i. Then the attribution weight is

(Eq. 1)

for hospitalization i and provider j. Note that ∑_jw_ij = 1: in other words, the total attribution, summed across all providers, is constrained to be 1 for each hospitalization.

Patient Outcomes

Outcomes were chosen based on their routine use in health care systems as standards when evaluating provider performance. This study included 6 outcomes: inpatient LOS, inpatient mortality, 30-day inpatient readmission, and patient responses from 3 survey questions. These outcomes can be collected without any manual chart reviews, and therefore are viewed as objective outcomes of provider performance.

Each outcome was aggregated for each provider using both attribution methods independently. For the PAPR method, observed-to-expected (OE) indices for LOS, mortality, and readmissions were calculated along with average patient survey scores. For the PAMM method, provider-level random effects from the fitted models were used. In both cases, the calculated measures were used for ranking purposes when determining top (or bottom) providers for each outcome.

Individual Provider Metrics for the PAPR Method

Inpatient LOS Index. Hospital inpatient LOS was measured as the number of days between admission date and discharge date. For each hospital visit, an expected LOS was determined using Premier’s CareScience Analytics (CSA) risk-adjustment methodology.¹⁰ The CSA methodology for LOS incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let o_ij and e_ij be the observed and expected LOS, respectively, for hospitalization i = 1,…,n_j attributed to provider j. Then the inpatient LOS index for provider j is L_j = ∑_io_ij⁄∑_ie_ij.

Inpatient Mortality Index. Inpatient mortality was defined as the death of the patient during hospitalization. For each hospitalization, an expected mortality probability was determined using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for mortality incorporates a patient’s demographics and comorbidities.

Just as before, let n_j be the number of hospitalizations attributed to provider j. Let m_ij = 1 if the patient died during hospitalization i = 1, … , n_j attributed to provider j; m_ij = 0 otherwise. Let p_ij^(m) be the corresponding expected mortality probability. Then the inpatient mortality index for provider j is M_j = ∑_im_ij⁄∑_ip_ij^(m).

30-Day Inpatient Readmission Index. A 30-day inpatient readmission was defined as the event when a patient is discharged and readmits back into the inpatient setting within 30 days. The inclusion criteria defined by the Centers for Medicare and Medicaid Services (CMS) all-cause hospital-wide readmission measure was used and, consequently, planned readmissions were excluded.¹² Readmissions could occur at any Banner hospital, including the same hospital. For each hospital visit, an expected readmission probability was derived using Premier’s CSA risk-adjustment methodology.¹⁰ The CSA methodology for readmissions incorporates a patient’s clinical history, demographics, and visit-related administrative information.

Let n_j be the number of hospitalizations attributed to provider j. Let r_ij = 1 if the patient had a readmission following hospitalization i = 1, … , n_j attributed to provider j; r_ij = 0 otherwise. Let p_ij^(r) be the corresponding expected readmission probability. Then the 30-day inpatient readmission index for provider j is R_j = ∑_ir_ij ⁄∑_ip_ij^(r).

Patient Survey Scores. The satisfaction of the patient’s experience during hospitalization was measured via post-discharge surveys administered by InMoment. Two survey questions were selected because they related directly to a provider’s interaction with the patient: “My interactions with doctors were excellent” (Doctor) and “I received the best possible care” (Care). A third question, “I would recommend this hospital to my family and friends,” was selected as a proxy measure of the overall experience and, in the aggregate, is referred to as the net promoter score (NPS).^13,14 The responses were measured on an 11-point Likert scale, ranging from “Strongly Disagree” (0) to “Strongly Agree” (10); “N/A” or missing responses were excluded.

The Likert responses were coded to 3 discrete values as follows: if the value was between 0 and 6, then -1 (ie, detractor); between 7 and 8 (ie, neutral), then 0; otherwise 1 (ie, promoter). Averaging these coded responses results in a patient survey score for each question. Specifically, let n_j be the number of hospitalizations attributed to provider j in which the patient responded to the survey question. Let s_ij ∈{−1, 0, 1} be the coded response linked to hospitalization i = 1, … , n_j attributed to provider j. Then the patient experience score for provider j is S_j = ∑_is_ij⁄n_j.

Handling Ties in Provider Performance Measures. Because ties can occur in the PAPR approach for all measures, a tie-breaking strategy is needed. For LOS indices, ties are less likely because their numerator is strictly greater than 0, and expected LOS values are typically distinct enough. Indeed, no ties were found in this study for LOS indices. However, mortality and readmission indices can routinely result in ties when the best possible index is achieved, such as 0 deaths or readmissions among attributed hospitalizations. To help differentiate between those indices in the PAPR approach, the total estimated risk (denominator) was utilized as a secondary scoring criterion.

Mortality and readmission metrics were addressed by sorting first by the outcome (mortality index), and second by the denominator (total estimated risk). For example, if provider A has the same mortality rate as provider B, then provider A would be ranked higher if the denominator was larger, indicating a higher risk for mortality.

Similarly, it was very common for providers to have the same overall average rating for a survey question. Therefore, the denominator (number of respondents) was used to break ties. However, the denominator sorting was bidirectional. For example, if the tied score was positive (more promoters than detractors) for providers A and B, then provider A would be ranked higher if the denominator was larger. Conversely, if the tied score between providers A and B was neutral or negative (more detractors than promoters), then provider A would be ranked lower if the denominator was larger.

Individual Provider Metrics for the PAMM Method

For the PAMM method, model-based metrics were derived using a MM model.⁸ Specifically, let J be the number of rotating providers in a health care system. Let Y_i be an outcome of interest from hospitalization i, X_1i, …, X_pi be fixed effects or covariates, and ß₁, …, ß_p be the coefficients for the respective covariates. Then the generalized MM statistical model is

(Eq. 2)

where g(μ_i) is a link function between the mean of the outcome, μ_i, and its linear predictor, ß₀, is the marginal intercept, w_ij represents the attribution weight of provider j on hospitalization i (described in Equation 1), and γ_j represents the random effect of provider j on the outcome with γ_j~N(0,σ_γ²).

For the mortality and readmission binary outcomes, logistic regression was performed using a logit link function, with the corresponding expected probability as the only fixed covariate. The expected probabilities were first converted into odds and then log-transformed before entering the model. For LOS, Poisson regression was performed using a log link function with the log-transformed expected LOS as the only fixed covariate. For coded patient experience responses, an ordered logistic regression was performed using a cumulative logit link function (no fixed effects were added).

MM Model-based Metrics. Each fitted MM model produces a predicted random effect for each provider. The provider-specific random effects can be interpreted as the unobserved influence of each provider on the outcome after controlling for any fixed effect included in the model. Therefore, the provider-specific random effects were used to evaluate the relative provider performance, which is analogous to the individual provider-level metrics used in the PAPR method.

Measuring provider performance using a MM model is more flexible and robust to outliers compared to the standard approach using OE indices or simple averages. First, although not investigated here, the effect of patient-, visit-, provider-, and/or temporal-level covariates can be controlled when evaluating provider performance. For example, a patient’s socioeconomic status, a provider’s workload, and seasonal factors can be added to the MM model. These external factors are not accounted for in OE indices.

Another advantage of using predicted random effects is the concept of “shrinkage.” The process of estimating random effects inherently accounts for small sample sizes (when providers do not treat a large enough sample of patients) and/or when there is a large ratio of patient variance to provider variance (for instance, when patient outcome variability is much higher compared to provider performance variability). In both cases, the estimation of the random effect is pulled ever closer to 0, signaling that the provider performance is closer to the population average. See Henderson¹⁵ and Mood¹⁶ for further details.

In contrast, OE indices can result in unreliable estimates when a provider has not cared for many patients. This is especially prevalent when the outcome is binary with a low probability of occurring, such as mortality. Indeed, provider-level mortality OE indices are routinely 0 when the patient counts are low, which skews performance rankings unfairly. Finally, OE indices also ignore the magnitude of the variance of an outcome between providers and patients, which can be large.

Comparison Methodology

In this study, we seek to compare the 2 methods of attribution, PAPR and PAMM, to determine whether there are meaningful differences between them when measuring provider performance. Using retrospective data described in the next section, each attribution method was used independently to derive provider-level metrics. To assess relative performance, percentiles were assigned to each provider based on their metric values so that, in the end, there were 2 percentile ranks for each provider for each metric.

Using these paired percentiles, we derived the following measures of concordance, similar to Herzke, Michtalik³: (1) the percent concordance measure—defined as the number of providers who landed in the top half (greater than the median) or bottom half under both attribution models—divided by the total number of providers; (2) the median of the absolute difference in percentiles under both attribution models; and (3) the Pearson correlation coefficient of the paired provider ranks. The first measure is a global measure of concordance between the 2 approaches and would be expected to be 50% by chance. The second measure gauges how an individual provider’s rank is affected by the change in attribution methodologies. The third measure is a statistical measure of linear correlation of the paired percentiles and was not included in the Herzke, Michtalik³ study.

All statistical analyses were performed on SAS (version 9.4; Cary, NC) and the MM models were fitted using PROC GLIMMIX with the EFFECT statement. The Banner Health Institutional Review Board approved this study.

Results

Descriptive Statistics

A total of 58,730 hospitalizations were included, of which care was provided by 963 unique providers across 25 acute care and critical access hospitals. Table 1 contains patient characteristics, and Table 2 depicts overall unadjusted outcomes. Providers responsible for less than 12 discharges in the calendar year were excluded from both approaches. Also, some hospitalizations were excluded when expected values were not available.

Multi-Membership Model Results

Table 3 displays the results after independently fitting MM models to each of the 3 clinical outcomes. Along with a marginal intercept, the only covariate in each model was the corresponding expected value after a transformation. This was added to use the same information that is typically used in OE indices, therefore allowing for a proper comparison between the 2 attribution methods. The provider-level variance represents the between-provider variation and measures the amount of influence providers have on the corresponding outcome after controlling for any covariates in the model. A provider-level variance of 0 would indicate that providers do not have any influence on the outcome. While the mortality and readmission model results can be compared to each other, the LOS model cannot given its different scale and transformation altogether.

The results in Table 3 suggest that each expected value covariate is highly correlated with its corresponding outcome, which is the anticipated conclusion given that they are constructed in this fashion. The estimated provider-level variances indicate that, after including an expected value in the model, providers have less of an influence on a patient’s LOS and likelihood of being readmitted. On the other hand, the results suggest that providers have much more influence on the likelihood of a patient dying in the hospital, even after controlling for an expected mortality covariate.

Table 4 shows the results after independently fitting MM-ordered logistic models to each of the 3 survey questions. The similar provider-level variances suggest that providers have the same influence on the patient’s perception of the quality of their interactions with the doctor (Doctor), the quality of the care they received (Care), and their likelihood to recommend a friend or family member to the hospital (NPS).

Comparison Results Between Both Attribution Methods

Table 5 compares the 2 attribution methods when ranking providers based on their performance on each outcome measure. The comparison metrics gauge how well the 2 methods agree overall (percent concordance), agree at the provider level (absolute percentile difference and interquartile range [IQR]), and how the paired percentiles linearly correlate to each other (Pearson correlation coefficient).

LOS, by a small margin, had the lowest concordance of clinical outcomes (72.1%), followed by mortality (75.9%) and readmissions (82.1%). Generally, the survey scores had higher percent concordance than the clinical outcome measures, with Doctor at 84.1%, Care at 85.9%, and NPS having the highest percent concordance at 86.6%. Given that by chance the percent concordance is expected to be 50%, there was notable discordance, especially with the clinical outcome measures. Using LOS performance as an example, one attribution methodology would rank a provider in the top half or bottom half, while the other attribution methodology would rank the same provider exactly the opposite way about 28% of the time.

The median absolute percentile difference between the 2 methods was more modest (between 7 and 15). Still, there were some providers whose performance ranking was heavily impacted by the attribution methodology that was used. This was especially true when evaluating performance for certain clinical measures, where the attribution method that was used could change the provider performance percentile by up to 90 levels.

The paired percentiles were positively correlated when ranking performance using any of the 6 measures. This suggests that both methodologies assess performance generally in the same direction, irrespective of the methodology and measure. We did not investigate more complex correlation measures and left this for future research.

It should be noted that ties occurred much more frequently with the PAPR method than when using PAMM and therefore required tie-breaking rules to be designed. Given the nature of OE indices, PAPR methodology is especially sensitive to ties whenever the measure includes counting the number of events (for example, mortality and readmissions) and whenever there are many providers with very few attributed patients. On the other hand, using the PAMM method is much more robust against ties given that the summation of all the weighted attributed outcomes will rarely result in ties, even with a nominal set of providers.

In this study, the PAMM methodology was introduced and was used to assess relative provider performance on 3 clinical outcome measures and 3 patient survey scores. The new approach aims to distribute each outcome among all providers who provided care for a patient in an inpatient setting. Clinical notes were used to account for patient-to-provider interactions, and fitted MM statistical models were used to compute the effects that each provider had on each outcome. The provider effect was introduced as a random effect, and the set of predicted random effects was used to rank the performance of each provider.

The PAMM approach was compared to the more traditional methodology, PAPR, where each patient is attributed to only 1 provider: the discharging physician in this study. Using this approach, OE indices of clinical outcomes and averages of survey scores were used to rank the performance of each provider. This approach resulted in many ties, which were broken based on the number of hospitalizations, although other tie-breaking methods may be used in practice.

Both methodologies showed modest concordance with each other for the clinical outcomes, but higher concordance for the patient survey scores. This was also true when using the Pearson correlation coefficient to assess agreement. The 1 outcome measure that showed the least concordance and least linear correlation between methods was LOS, which would suggest that LOS performance is more sensitive to the attribution methodology that is used. However, it was the least concordant by a small margin.

Furthermore, although the medians of the absolute percentile differences were small, there were some providers who had large deviations, suggesting that some providers would move from being shown as high-performers to low-performers and vice versa based on the chosen attribution method. We investigated examples of this and determined that the root cause was the difference in effective sample sizes for a provider. For the PAPR method, the effective sample size is simply the number of hospitalizations attributed to the provider. For the PAMM method, the effective sample size is the sum of all non-zero weights across all hospitalizations where the provider cared for a patient. By and large, the PAMM methodology provides more information of the provider effect on an outcome than the PAPR approach because every provider-patient interaction is considered. For example, providers who do not routinely discharge patients, but often care for patients, will have rankings that differ dramatically between the 2 methods.

The PAMM methodology has many statistical advantages that were not fully utilized in this comparative study. For example, we did not include any covariates in the MM models except for the expected value of the outcome, when it was available. Still, it is known that other covariates can impact an outcome as well, such as the patient’s age, socioeconomic indicators, existing chronic conditions, and severity of hospitalization, which can be added to the MM models as fixed effects. In this way, the PAMM approach can control for these other covariates, which are typically outside of the control of providers but typically ignored using OE indices. Therefore, using the PAMM approach would provide a fairer comparison of provider performance.

Using the PAMM method, most providers had a large sample size to assess their performance once all the weighted interactions were included. Still, there were a few who did not care for many patients for a variety of reasons. In these scenarios, MM models “borrow” strength from other providers to produce a more robust predicted provider effect by using a weighted average between the overall population trend and the specific provider outcomes (see Rao and Molina¹⁷). As a result, PAMM is a more suitable approach when the sample sizes of patients attributed to providers can be small.

One of the most interesting findings of this study was the relative size of the provider-level variance to the size of the fixed effect in each model (Table 3). Except for mortality, these variances suggest that there is a small difference in performance from one provider to another. However, these should be interpreted as the variance when only 1 provider is involved in the care of a patient. When multiple providers are involved, using basic statistical theory, the overall provider-level variance will be σ_γ² ∑w_ij² (see Equation 2). For example, the estimated variance among providers for LOS was 0.03 (on a log scale), but, using the scenario in the Figure, the overall provider-level variance for this hospitalization will be 0.03 (0.375²+ 0.125²+ 0.5²) = 0.012. Hence, the combined effect of providers on LOS is less than would be expected. Indeed, as more providers are involved with a patient’s care, the more their combined influence on an outcome is diluted.

In this study, the PAMM approach placed an equal weight on all provider-patient interactions via clinical note authorship, but that may not be optimal in some settings. For example, it may make more sense to set a higher weight on the provider who admitted or discharged the patient while placing less (or 0) weight on all other interactions. In the extreme, if the full weight were placed on 1 provider interaction (eg, during discharge, then the MM model would be reduced to a one-way random effects model. The flexibility of weighting interactions is a feature of the PAMM approach, but any weighting framework must be transparent to the providers before implementation.

Conclusion

This study demonstrates that the PAMM approach is a feasible option within a large health care organization. For P4P programs to be successful, providers must be able to trust that their performance will be fairly assessed and that all provider-patient interactions are captured to provide a full comparison amongst their peers. The PAMM methodology is one solution to spread the positive (and negative) outcomes across all providers who cared for a patient and therefore, if implemented, would add trust and fairness when measuring and assessing provider performance.

Acknowledgments: The authors thank Barrie Bradley for his support in the initial stages of this research and Dr. Syed Ismail Jafri for his help and support on the standard approaches of assessing and measuring provider performances.

Corresponding author: Rachel Ginn, MS, Banner Health Corporation, 2901 N. Central Ave., Phoenix, AZ 85012; [email protected].

Financial disclosures: None.