Médecins Sans Frontières (MSF/Doctors Without Borders) announced early closure of its phase 2/3 trial of a 6-month multidrug regimen for multidrug-resistant tuberculosis (MDR-TB) because an independent data safety and monitoring board (DSMB) determined that the drug combination in the study regimen was superior to current therapy, according to a press release.

The trial, called TB PRACTECAL, compared the current local standard of care with a 6-month regimen of bedaquiline, pretomanid, linezolid, and moxifloxacin. The interim analysis included 242 patients and the randomized, controlled trial was conducted in sites in Belarus, South Africa, and Uzbekistan.

The preliminary data will be shared with the World Health Organization soon and will also be submitted to a peer-reviewed journal. If it withstands further reviews, as is anticipated, the trial would support the first solely oral regimen for MDR-TB.

In 2019, an estimated 465,000 people developed MDR-TB and 182,000 died. The global burden of TB at that time was about 10 million new cases, many with coexisting HIV.

Current treatment for MDR-TB lasts 9-20 months and is complicated by the need for painful shots and toxic antibiotics. Side effects can include psychiatric problems from quinolones, isoniazid, ethambutol, or cycloserine; deafness from aminoglycosides; and bone marrow suppression from linezolid, among other toxicities.

It’s hoped that the shorter regimen will reduce toxicity and improve patient compliance. Poor adherence to treatment is a major driver of further drug resistance. Current regimens require up to 20 pills per day as well as daily injections.

In a prepared statement from MSF, David Moore, MD, MSc, London School of Hygiene and Tropical Medicine, a member of the TB-PRACTECAL trial’s steering committee, concluded: “The findings could transform the way we treat patients with drug-resistant forms of TB worldwide, who have been neglected for too long.”

This good news is particularly welcome as, in the time of COVID-19, “an estimated 1.4 million fewer people received care for tuberculosis in 2020 than in 2019,” according to the WHO. The drop, an overall 21% reduction in patients beginning treatment, ranged as high as 42% in Indonesia.

Although awaiting complete data, Madhukar Pai, MD, PhD, associate director of the McGill International TB Centre, McGill University, Montreal, shares Dr. Moore’s enthusiasm. In an interview, Dr. Pai compared MDR-TB with extensively drug-resistant TB (XDR-TB).

“I’m excited about the possibility that these trial results might help shorten MDR-TB treatment to 6 months,” said Dr. Pai. “That will be a huge relief to all patients battling drug-resistant disease. The 6-month BPaL regimen (bedaquiline, pretomanid, and linezolid) regimen works well in XDR-TB. So, I would expect the TB PRACTECAL regimen with one added drug (moxifloxacin) to work well in MDR-TB, which is less severe than XDR-TB. Between these two regimens, if we can bring down MDR and XDR treatment to 6 months, all oral, that would be a huge advance.”

The expense of bedaquiline has been a long-standing concern in the global health community. Janssen, a subsidiary of Johnson & Johnson, has reduced the price to $340 per 6-month treatment course for more than 135 eligible low- and middle-income countries.

Previously, the tiered pricing structure was different for low-, middle-, and high-income countries (U.S. $900, $3,000, and $30,000, respectively). “The global TB community has asked Janssen to drop the price of bedaquiline to a level no higher than $32 per month – double the price at which researchers estimated bedaquiline could be sold for a profit,” according to the Treatment Action Group A major source of contention over pricing has been that there has been considerable public investment in the drug›s development.

Dr. Pai concluded: “Bedaquiline is likely the most important drug in both 6-month regimens. We need to work harder to make bedaquiline, an excellent drug, more affordable and accessible.”

While the full data is not yet publicly available, TB PRACTECAL was a randomized, controlled, multicenter study. The fact that enrollment was discontinued early by the DSMB suggests the efficacy data was compelling and that this completely oral regimen will become the standard of care.

Dr. Stone is an infectious disease specialist and author of Resilience: One Family’s Story of Hope and Triumph Over Evil and of Conducting Clinical Research, the essential guide to the topic. A version of this article first appeared on Medscape.com.

Médecins Sans Frontières (MSF/Doctors Without Borders) announced early closure of its phase 2/3 trial of a 6-month multidrug regimen for multidrug-resistant tuberculosis (MDR-TB) because an independent data safety and monitoring board (DSMB) determined that the drug combination in the study regimen was superior to current therapy, according to a press release.

The trial, called TB PRACTECAL, compared the current local standard of care with a 6-month regimen of bedaquiline, pretomanid, linezolid, and moxifloxacin. The interim analysis included 242 patients and the randomized, controlled trial was conducted in sites in Belarus, South Africa, and Uzbekistan.

The preliminary data will be shared with the World Health Organization soon and will also be submitted to a peer-reviewed journal. If it withstands further reviews, as is anticipated, the trial would support the first solely oral regimen for MDR-TB.

In 2019, an estimated 465,000 people developed MDR-TB and 182,000 died. The global burden of TB at that time was about 10 million new cases, many with coexisting HIV.

Current treatment for MDR-TB lasts 9-20 months and is complicated by the need for painful shots and toxic antibiotics. Side effects can include psychiatric problems from quinolones, isoniazid, ethambutol, or cycloserine; deafness from aminoglycosides; and bone marrow suppression from linezolid, among other toxicities.

It’s hoped that the shorter regimen will reduce toxicity and improve patient compliance. Poor adherence to treatment is a major driver of further drug resistance. Current regimens require up to 20 pills per day as well as daily injections.

In a prepared statement from MSF, David Moore, MD, MSc, London School of Hygiene and Tropical Medicine, a member of the TB-PRACTECAL trial’s steering committee, concluded: “The findings could transform the way we treat patients with drug-resistant forms of TB worldwide, who have been neglected for too long.”

This good news is particularly welcome as, in the time of COVID-19, “an estimated 1.4 million fewer people received care for tuberculosis in 2020 than in 2019,” according to the WHO. The drop, an overall 21% reduction in patients beginning treatment, ranged as high as 42% in Indonesia.

Although awaiting complete data, Madhukar Pai, MD, PhD, associate director of the McGill International TB Centre, McGill University, Montreal, shares Dr. Moore’s enthusiasm. In an interview, Dr. Pai compared MDR-TB with extensively drug-resistant TB (XDR-TB).

“I’m excited about the possibility that these trial results might help shorten MDR-TB treatment to 6 months,” said Dr. Pai. “That will be a huge relief to all patients battling drug-resistant disease. The 6-month BPaL regimen (bedaquiline, pretomanid, and linezolid) regimen works well in XDR-TB. So, I would expect the TB PRACTECAL regimen with one added drug (moxifloxacin) to work well in MDR-TB, which is less severe than XDR-TB. Between these two regimens, if we can bring down MDR and XDR treatment to 6 months, all oral, that would be a huge advance.”

The expense of bedaquiline has been a long-standing concern in the global health community. Janssen, a subsidiary of Johnson & Johnson, has reduced the price to $340 per 6-month treatment course for more than 135 eligible low- and middle-income countries.

Previously, the tiered pricing structure was different for low-, middle-, and high-income countries (U.S. $900, $3,000, and $30,000, respectively). “The global TB community has asked Janssen to drop the price of bedaquiline to a level no higher than $32 per month – double the price at which researchers estimated bedaquiline could be sold for a profit,” according to the Treatment Action Group A major source of contention over pricing has been that there has been considerable public investment in the drug›s development.

Dr. Pai concluded: “Bedaquiline is likely the most important drug in both 6-month regimens. We need to work harder to make bedaquiline, an excellent drug, more affordable and accessible.”

While the full data is not yet publicly available, TB PRACTECAL was a randomized, controlled, multicenter study. The fact that enrollment was discontinued early by the DSMB suggests the efficacy data was compelling and that this completely oral regimen will become the standard of care.

Dr. Stone is an infectious disease specialist and author of Resilience: One Family’s Story of Hope and Triumph Over Evil and of Conducting Clinical Research, the essential guide to the topic. A version of this article first appeared on Medscape.com.

Médecins Sans Frontières (MSF/Doctors Without Borders) announced early closure of its phase 2/3 trial of a 6-month multidrug regimen for multidrug-resistant tuberculosis (MDR-TB) because an independent data safety and monitoring board (DSMB) determined that the drug combination in the study regimen was superior to current therapy, according to a press release.

The trial, called TB PRACTECAL, compared the current local standard of care with a 6-month regimen of bedaquiline, pretomanid, linezolid, and moxifloxacin. The interim analysis included 242 patients and the randomized, controlled trial was conducted in sites in Belarus, South Africa, and Uzbekistan.

The preliminary data will be shared with the World Health Organization soon and will also be submitted to a peer-reviewed journal. If it withstands further reviews, as is anticipated, the trial would support the first solely oral regimen for MDR-TB.

In 2019, an estimated 465,000 people developed MDR-TB and 182,000 died. The global burden of TB at that time was about 10 million new cases, many with coexisting HIV.

Current treatment for MDR-TB lasts 9-20 months and is complicated by the need for painful shots and toxic antibiotics. Side effects can include psychiatric problems from quinolones, isoniazid, ethambutol, or cycloserine; deafness from aminoglycosides; and bone marrow suppression from linezolid, among other toxicities.

It’s hoped that the shorter regimen will reduce toxicity and improve patient compliance. Poor adherence to treatment is a major driver of further drug resistance. Current regimens require up to 20 pills per day as well as daily injections.

In a prepared statement from MSF, David Moore, MD, MSc, London School of Hygiene and Tropical Medicine, a member of the TB-PRACTECAL trial’s steering committee, concluded: “The findings could transform the way we treat patients with drug-resistant forms of TB worldwide, who have been neglected for too long.”

This good news is particularly welcome as, in the time of COVID-19, “an estimated 1.4 million fewer people received care for tuberculosis in 2020 than in 2019,” according to the WHO. The drop, an overall 21% reduction in patients beginning treatment, ranged as high as 42% in Indonesia.

Although awaiting complete data, Madhukar Pai, MD, PhD, associate director of the McGill International TB Centre, McGill University, Montreal, shares Dr. Moore’s enthusiasm. In an interview, Dr. Pai compared MDR-TB with extensively drug-resistant TB (XDR-TB).

“I’m excited about the possibility that these trial results might help shorten MDR-TB treatment to 6 months,” said Dr. Pai. “That will be a huge relief to all patients battling drug-resistant disease. The 6-month BPaL regimen (bedaquiline, pretomanid, and linezolid) regimen works well in XDR-TB. So, I would expect the TB PRACTECAL regimen with one added drug (moxifloxacin) to work well in MDR-TB, which is less severe than XDR-TB. Between these two regimens, if we can bring down MDR and XDR treatment to 6 months, all oral, that would be a huge advance.”

The expense of bedaquiline has been a long-standing concern in the global health community. Janssen, a subsidiary of Johnson & Johnson, has reduced the price to $340 per 6-month treatment course for more than 135 eligible low- and middle-income countries.

Previously, the tiered pricing structure was different for low-, middle-, and high-income countries (U.S. $900, $3,000, and $30,000, respectively). “The global TB community has asked Janssen to drop the price of bedaquiline to a level no higher than $32 per month – double the price at which researchers estimated bedaquiline could be sold for a profit,” according to the Treatment Action Group A major source of contention over pricing has been that there has been considerable public investment in the drug›s development.

Dr. Pai concluded: “Bedaquiline is likely the most important drug in both 6-month regimens. We need to work harder to make bedaquiline, an excellent drug, more affordable and accessible.”

While the full data is not yet publicly available, TB PRACTECAL was a randomized, controlled, multicenter study. The fact that enrollment was discontinued early by the DSMB suggests the efficacy data was compelling and that this completely oral regimen will become the standard of care.

Dr. Stone is an infectious disease specialist and author of Resilience: One Family’s Story of Hope and Triumph Over Evil and of Conducting Clinical Research, the essential guide to the topic. A version of this article first appeared on Medscape.com.

Higher levels of vitamin D than traditionally considered sufficient may help prevent COVID-19 infection – particularly in Black patients, shows a new single-center, retrospective study looking at the role of vitamin D in prevention of infection.

The study, published recently in JAMA Network Open, noted that expert opinion varies as to what “sufficient” levels of vitamin D are, some define this as 30 ng/mL, while others cite 40 ng/mL or greater.

In their discussion, the authors also noted that their results showed the “risk of positive COVID-19 test results decreased significantly with increased vitamin D level of 30 ng/mL or greater when measured as a continuous variable.”

“These new results tell us that having vitamin D levels above those normally considered sufficient is associated with decreased risk of testing positive for COVID-19, at least in Black individuals,” lead author, David Meltzer, MD, chief of hospital medicine at the University of Chicago, said in a press release from his institution.

“These findings suggest that randomized clinical trials to determine whether increasing vitamin D levels to greater than 30-40 ng/mL affect COVID-19 risk are warranted, especially in Black individuals,” he and his coauthors said.
 

Vitamin D at time of testing most strongly associated with COVID risk

An earlier study by the same researchers found that vitamin D deficiency (less than 20 ng/mL) may raise the risk of testing positive for COVID-19 in people from various ethnicities, as reported by this news organization.

Data for this latest study were drawn from electronic health records for 4,638 individuals at the University of Chicago Medicine and were used to examine whether the likelihood of a positive COVID-19 test was associated with a person’s most recent vitamin D level (within the previous year), and whether there was any effect of ethnicity on this outcome.

Mean age was 52.8 years, 69% were women, 49% were Black, 43% White, and 8% were another race/ethnicity. A total of 27% of the individuals were deficient in vitamin D (less than 20 ng/mL), 27% had insufficient levels (20-30 ng/mL), 22% had sufficient levels (30-40 ng/mL), and the remaining 24% had levels of 40 ng/mL or greater.

In total, 333 (7%) of people tested positive for COVID-19, including 102 (5%) Whites and 211 (9%) Blacks. And 36% of Black individuals who tested positive for COVID-19 were classified as vitamin D deficient, compared with 16% of Whites.

A positive test result for COVID-19 was not significantly associated with vitamin D levels in white individuals but was in Black individuals.

In Black people, compared with levels of at least 40 ng/mL, vitamin D levels of 30-40 ng/mL were associated with an incidence rate ratio of 2.64 for COVID-19 positivity (P = .01). For levels of 20-30 ng/mL, the IRR was 1.69 (P = 0.21); and for less than 20 ng/mL the IRR was 2.55 (P = .009).

The researchers also found that the risk of positive test results with lower vitamin D levels increased when those levels were lower just prior to the positive COVID-19 test, lending “support [to] the idea that vitamin D level at the time of testing is most strongly associated with COVID-19 risk,” they wrote.
 

Try upping vitamin D levels to 40 ng/mL or greater to prevent COVID?

In their discussion, the authors noted that significant association of vitamin D levels with COVID-19 risk in Blacks but not in Whites, “could reflect their higher COVID-19 risk, to which socioeconomic factors and structural inequities clearly contribute.

“Biological susceptibility to vitamin D deficiency may also be less frequent in White than Black individuals, since lighter skin increases vitamin D production in response to sunlight, and vitamin D binding proteins may vary by race and affect vitamin D bioavailability.”

Given less than 10% of U.S. adults have a vitamin D level greater than 40 ng/mL, the study findings increase the urgency to consider whether increased sun exposure or supplementation could reduce COVID-19 risk, according to the authors.

“When increased sun exposure is impractical, achieving vitamin D levels of 40 ng/mL or greater typically requires greater supplementation than currently recommended for most individuals of 600-800 IU/d vitamin D3,” they added.

However, Dr. Meltzer also acknowledged that “this is an observational study. We can see that there’s an association between vitamin D levels and likelihood of a COVID-19 diagnosis, but we don’t know exactly why that is, or whether these results are due to the vitamin D directly or other related biological factors.”

All in all, the authors suggested that randomized clinical trials are needed to understand if vitamin D can reduce COVID-19 risk, and as such they should include doses of supplements likely to increase vitamin D to at least 40 ng/mL, and perhaps even higher, although they pointed out that the latter must be achieved safely.

“Studies should also consider the role of vitamin D testing, loading doses, dose adjustments for individuals who are obese or overweight, risks for hypercalcemia, and strategies to monitor for and mitigate hypercalcemia, and that non-White populations, such as Black individuals, may have greater needs for supplementation,” they outlined.

They are now recruiting participants for two separate clinical trials testing the efficacy of vitamin D supplements for preventing COVID-19.

The authors disclosed no relevant financial relationships.

A version of this article first appeared on Medscape.com.

Higher levels of vitamin D than traditionally considered sufficient may help prevent COVID-19 infection – particularly in Black patients, shows a new single-center, retrospective study looking at the role of vitamin D in prevention of infection.

The study, published recently in JAMA Network Open, noted that expert opinion varies as to what “sufficient” levels of vitamin D are, some define this as 30 ng/mL, while others cite 40 ng/mL or greater.

In their discussion, the authors also noted that their results showed the “risk of positive COVID-19 test results decreased significantly with increased vitamin D level of 30 ng/mL or greater when measured as a continuous variable.”

“These new results tell us that having vitamin D levels above those normally considered sufficient is associated with decreased risk of testing positive for COVID-19, at least in Black individuals,” lead author, David Meltzer, MD, chief of hospital medicine at the University of Chicago, said in a press release from his institution.

“These findings suggest that randomized clinical trials to determine whether increasing vitamin D levels to greater than 30-40 ng/mL affect COVID-19 risk are warranted, especially in Black individuals,” he and his coauthors said.
 

Vitamin D at time of testing most strongly associated with COVID risk

An earlier study by the same researchers found that vitamin D deficiency (less than 20 ng/mL) may raise the risk of testing positive for COVID-19 in people from various ethnicities, as reported by this news organization.

Data for this latest study were drawn from electronic health records for 4,638 individuals at the University of Chicago Medicine and were used to examine whether the likelihood of a positive COVID-19 test was associated with a person’s most recent vitamin D level (within the previous year), and whether there was any effect of ethnicity on this outcome.

Mean age was 52.8 years, 69% were women, 49% were Black, 43% White, and 8% were another race/ethnicity. A total of 27% of the individuals were deficient in vitamin D (less than 20 ng/mL), 27% had insufficient levels (20-30 ng/mL), 22% had sufficient levels (30-40 ng/mL), and the remaining 24% had levels of 40 ng/mL or greater.

In total, 333 (7%) of people tested positive for COVID-19, including 102 (5%) Whites and 211 (9%) Blacks. And 36% of Black individuals who tested positive for COVID-19 were classified as vitamin D deficient, compared with 16% of Whites.

A positive test result for COVID-19 was not significantly associated with vitamin D levels in white individuals but was in Black individuals.

In Black people, compared with levels of at least 40 ng/mL, vitamin D levels of 30-40 ng/mL were associated with an incidence rate ratio of 2.64 for COVID-19 positivity (P = .01). For levels of 20-30 ng/mL, the IRR was 1.69 (P = 0.21); and for less than 20 ng/mL the IRR was 2.55 (P = .009).

The researchers also found that the risk of positive test results with lower vitamin D levels increased when those levels were lower just prior to the positive COVID-19 test, lending “support [to] the idea that vitamin D level at the time of testing is most strongly associated with COVID-19 risk,” they wrote.
 

Try upping vitamin D levels to 40 ng/mL or greater to prevent COVID?

In their discussion, the authors noted that significant association of vitamin D levels with COVID-19 risk in Blacks but not in Whites, “could reflect their higher COVID-19 risk, to which socioeconomic factors and structural inequities clearly contribute.

“Biological susceptibility to vitamin D deficiency may also be less frequent in White than Black individuals, since lighter skin increases vitamin D production in response to sunlight, and vitamin D binding proteins may vary by race and affect vitamin D bioavailability.”

Given less than 10% of U.S. adults have a vitamin D level greater than 40 ng/mL, the study findings increase the urgency to consider whether increased sun exposure or supplementation could reduce COVID-19 risk, according to the authors.

“When increased sun exposure is impractical, achieving vitamin D levels of 40 ng/mL or greater typically requires greater supplementation than currently recommended for most individuals of 600-800 IU/d vitamin D3,” they added.

However, Dr. Meltzer also acknowledged that “this is an observational study. We can see that there’s an association between vitamin D levels and likelihood of a COVID-19 diagnosis, but we don’t know exactly why that is, or whether these results are due to the vitamin D directly or other related biological factors.”

All in all, the authors suggested that randomized clinical trials are needed to understand if vitamin D can reduce COVID-19 risk, and as such they should include doses of supplements likely to increase vitamin D to at least 40 ng/mL, and perhaps even higher, although they pointed out that the latter must be achieved safely.

“Studies should also consider the role of vitamin D testing, loading doses, dose adjustments for individuals who are obese or overweight, risks for hypercalcemia, and strategies to monitor for and mitigate hypercalcemia, and that non-White populations, such as Black individuals, may have greater needs for supplementation,” they outlined.

They are now recruiting participants for two separate clinical trials testing the efficacy of vitamin D supplements for preventing COVID-19.

The authors disclosed no relevant financial relationships.

A version of this article first appeared on Medscape.com.

Higher levels of vitamin D than traditionally considered sufficient may help prevent COVID-19 infection – particularly in Black patients, shows a new single-center, retrospective study looking at the role of vitamin D in prevention of infection.

The study, published recently in JAMA Network Open, noted that expert opinion varies as to what “sufficient” levels of vitamin D are, some define this as 30 ng/mL, while others cite 40 ng/mL or greater.

In their discussion, the authors also noted that their results showed the “risk of positive COVID-19 test results decreased significantly with increased vitamin D level of 30 ng/mL or greater when measured as a continuous variable.”

“These new results tell us that having vitamin D levels above those normally considered sufficient is associated with decreased risk of testing positive for COVID-19, at least in Black individuals,” lead author, David Meltzer, MD, chief of hospital medicine at the University of Chicago, said in a press release from his institution.

“These findings suggest that randomized clinical trials to determine whether increasing vitamin D levels to greater than 30-40 ng/mL affect COVID-19 risk are warranted, especially in Black individuals,” he and his coauthors said.
 

Vitamin D at time of testing most strongly associated with COVID risk

An earlier study by the same researchers found that vitamin D deficiency (less than 20 ng/mL) may raise the risk of testing positive for COVID-19 in people from various ethnicities, as reported by this news organization.

Data for this latest study were drawn from electronic health records for 4,638 individuals at the University of Chicago Medicine and were used to examine whether the likelihood of a positive COVID-19 test was associated with a person’s most recent vitamin D level (within the previous year), and whether there was any effect of ethnicity on this outcome.

Mean age was 52.8 years, 69% were women, 49% were Black, 43% White, and 8% were another race/ethnicity. A total of 27% of the individuals were deficient in vitamin D (less than 20 ng/mL), 27% had insufficient levels (20-30 ng/mL), 22% had sufficient levels (30-40 ng/mL), and the remaining 24% had levels of 40 ng/mL or greater.

In total, 333 (7%) of people tested positive for COVID-19, including 102 (5%) Whites and 211 (9%) Blacks. And 36% of Black individuals who tested positive for COVID-19 were classified as vitamin D deficient, compared with 16% of Whites.

A positive test result for COVID-19 was not significantly associated with vitamin D levels in white individuals but was in Black individuals.

In Black people, compared with levels of at least 40 ng/mL, vitamin D levels of 30-40 ng/mL were associated with an incidence rate ratio of 2.64 for COVID-19 positivity (P = .01). For levels of 20-30 ng/mL, the IRR was 1.69 (P = 0.21); and for less than 20 ng/mL the IRR was 2.55 (P = .009).

The researchers also found that the risk of positive test results with lower vitamin D levels increased when those levels were lower just prior to the positive COVID-19 test, lending “support [to] the idea that vitamin D level at the time of testing is most strongly associated with COVID-19 risk,” they wrote.
 

Try upping vitamin D levels to 40 ng/mL or greater to prevent COVID?

In their discussion, the authors noted that significant association of vitamin D levels with COVID-19 risk in Blacks but not in Whites, “could reflect their higher COVID-19 risk, to which socioeconomic factors and structural inequities clearly contribute.

“Biological susceptibility to vitamin D deficiency may also be less frequent in White than Black individuals, since lighter skin increases vitamin D production in response to sunlight, and vitamin D binding proteins may vary by race and affect vitamin D bioavailability.”

Given less than 10% of U.S. adults have a vitamin D level greater than 40 ng/mL, the study findings increase the urgency to consider whether increased sun exposure or supplementation could reduce COVID-19 risk, according to the authors.

“When increased sun exposure is impractical, achieving vitamin D levels of 40 ng/mL or greater typically requires greater supplementation than currently recommended for most individuals of 600-800 IU/d vitamin D3,” they added.

However, Dr. Meltzer also acknowledged that “this is an observational study. We can see that there’s an association between vitamin D levels and likelihood of a COVID-19 diagnosis, but we don’t know exactly why that is, or whether these results are due to the vitamin D directly or other related biological factors.”

All in all, the authors suggested that randomized clinical trials are needed to understand if vitamin D can reduce COVID-19 risk, and as such they should include doses of supplements likely to increase vitamin D to at least 40 ng/mL, and perhaps even higher, although they pointed out that the latter must be achieved safely.

“Studies should also consider the role of vitamin D testing, loading doses, dose adjustments for individuals who are obese or overweight, risks for hypercalcemia, and strategies to monitor for and mitigate hypercalcemia, and that non-White populations, such as Black individuals, may have greater needs for supplementation,” they outlined.

They are now recruiting participants for two separate clinical trials testing the efficacy of vitamin D supplements for preventing COVID-19.

The authors disclosed no relevant financial relationships.

A version of this article first appeared on Medscape.com.

Among individuals who have received a hematopoietic stem cell transplant (HSCT), often used in the treatment of blood cancers, rates of survival are poor for those who develop COVID-19.

The probability of survival 30 days after being diagnosed with COVID-19 is only 68% for persons who have received an allogeneic HSCT and 67% for autologous HSCT recipients, according to new data from the Center for International Blood and Marrow Transplant Research.

These findings underscore the need for “stringent surveillance and aggressive treatment measures” in this population, Akshay Sharma, MBBS, of St. Jude Children’s Research Hospital, Memphis, and colleagues wrote.

The findings were published online March 1, 2021, in The Lancet Haematology.

The study is “of importance for physicians caring for HSCT recipients worldwide,” Mathieu Leclerc, MD, and Sébastien Maury, MD, Hôpital Henri Mondor, Créteil, France, commented in an accompanying editorial.
 

Study details

For their study, Dr. Sharma and colleagues analyzed outcomes for all HSCT recipients who developed COVID-19 and whose cases were reported to the CIBMTR. Of 318 such patients, 184 had undergone allogeneic HSCT, and 134 had undergone autologous HSCT.

Overall, about half of these patients (49%) had mild COVID-19.

Severe COVID-19 that required mechanical ventilation developed in 15% and 13% of the allogeneic and autologous HSCT recipients, respectively.

About one-fifth of patients died: 22% and 19% of allogeneic and autologous HSCT recipients, respectively.

Factors associated with greater mortality risk included age of 50 years or older (hazard ratio, 2.53), male sex (HR, 3.53), and development of COVID-19 within 12 months of undergoing HSCT (HR, 2.67).

Among autologous HSCT recipients, lymphoma was associated with higher mortality risk in comparison with a plasma cell disorder or myeloma (HR, 2.41), the authors noted.

“Two important messages can be drawn from the results reported by Sharma and colleagues,” Dr. Leclerc and Dr. Maury wrote in their editorial. “The first is the confirmation that the prognosis of COVID-19 is particularly poor in HSCT recipients, and that its prevention, in the absence of any specific curative treatment with sufficient efficacy, should be at the forefront of concerns.”

The second relates to the risk factors for death among HSCT recipients who develop COVID-19. In addition to previously known risk factors, such as age and gender, the investigators identified transplant-specific factors potentially associated with prognosis – namely, the nearly threefold increase in death among allogeneic HSCT recipients who develop COVID-19 within 12 months of transplant, they explained.

However, the findings are limited by a substantial amount of missing data, short follow-up, and the possibility of selection bias, they noted.

“Further large and well-designed studies with longer follow-up are needed to confirm and refine the results,” the editorialists wrote.

“[A] better understanding of the distinctive features of COVID-19 infection in HSCT recipients will be a necessary and essential step toward improvement of the remarkably poor prognosis observed in this setting,” they added.

The study was funded by the American Society of Hematology; the Leukemia and Lymphoma Society; the National Cancer Institute; the National Heart, Lung and Blood Institute; the National Institute of Allergy and Infectious Diseases; the National Institutes of Health; the Health Resources and Services Administration; and the Office of Naval Research. Dr. Sharma receives support for the conduct of industry-sponsored trials from Vertex Pharmaceuticals, CRISPR Therapeutics, and Novartis and consulting fees from Spotlight Therapeutics. Dr. Leclerc and Dr. Maury disclosed no relevant financial relationships.

A version of this article first appeared on Medscape.com.

Among individuals who have received a hematopoietic stem cell transplant (HSCT), often used in the treatment of blood cancers, rates of survival are poor for those who develop COVID-19.

The probability of survival 30 days after being diagnosed with COVID-19 is only 68% for persons who have received an allogeneic HSCT and 67% for autologous HSCT recipients, according to new data from the Center for International Blood and Marrow Transplant Research.

These findings underscore the need for “stringent surveillance and aggressive treatment measures” in this population, Akshay Sharma, MBBS, of St. Jude Children’s Research Hospital, Memphis, and colleagues wrote.

The findings were published online March 1, 2021, in The Lancet Haematology.

The study is “of importance for physicians caring for HSCT recipients worldwide,” Mathieu Leclerc, MD, and Sébastien Maury, MD, Hôpital Henri Mondor, Créteil, France, commented in an accompanying editorial.
 

Study details

For their study, Dr. Sharma and colleagues analyzed outcomes for all HSCT recipients who developed COVID-19 and whose cases were reported to the CIBMTR. Of 318 such patients, 184 had undergone allogeneic HSCT, and 134 had undergone autologous HSCT.

Overall, about half of these patients (49%) had mild COVID-19.

Severe COVID-19 that required mechanical ventilation developed in 15% and 13% of the allogeneic and autologous HSCT recipients, respectively.

About one-fifth of patients died: 22% and 19% of allogeneic and autologous HSCT recipients, respectively.

Factors associated with greater mortality risk included age of 50 years or older (hazard ratio, 2.53), male sex (HR, 3.53), and development of COVID-19 within 12 months of undergoing HSCT (HR, 2.67).

Among autologous HSCT recipients, lymphoma was associated with higher mortality risk in comparison with a plasma cell disorder or myeloma (HR, 2.41), the authors noted.

“Two important messages can be drawn from the results reported by Sharma and colleagues,” Dr. Leclerc and Dr. Maury wrote in their editorial. “The first is the confirmation that the prognosis of COVID-19 is particularly poor in HSCT recipients, and that its prevention, in the absence of any specific curative treatment with sufficient efficacy, should be at the forefront of concerns.”

The second relates to the risk factors for death among HSCT recipients who develop COVID-19. In addition to previously known risk factors, such as age and gender, the investigators identified transplant-specific factors potentially associated with prognosis – namely, the nearly threefold increase in death among allogeneic HSCT recipients who develop COVID-19 within 12 months of transplant, they explained.

However, the findings are limited by a substantial amount of missing data, short follow-up, and the possibility of selection bias, they noted.

“Further large and well-designed studies with longer follow-up are needed to confirm and refine the results,” the editorialists wrote.

“[A] better understanding of the distinctive features of COVID-19 infection in HSCT recipients will be a necessary and essential step toward improvement of the remarkably poor prognosis observed in this setting,” they added.

The study was funded by the American Society of Hematology; the Leukemia and Lymphoma Society; the National Cancer Institute; the National Heart, Lung and Blood Institute; the National Institute of Allergy and Infectious Diseases; the National Institutes of Health; the Health Resources and Services Administration; and the Office of Naval Research. Dr. Sharma receives support for the conduct of industry-sponsored trials from Vertex Pharmaceuticals, CRISPR Therapeutics, and Novartis and consulting fees from Spotlight Therapeutics. Dr. Leclerc and Dr. Maury disclosed no relevant financial relationships.

A version of this article first appeared on Medscape.com.

Among individuals who have received a hematopoietic stem cell transplant (HSCT), often used in the treatment of blood cancers, rates of survival are poor for those who develop COVID-19.

The probability of survival 30 days after being diagnosed with COVID-19 is only 68% for persons who have received an allogeneic HSCT and 67% for autologous HSCT recipients, according to new data from the Center for International Blood and Marrow Transplant Research.

These findings underscore the need for “stringent surveillance and aggressive treatment measures” in this population, Akshay Sharma, MBBS, of St. Jude Children’s Research Hospital, Memphis, and colleagues wrote.

The findings were published online March 1, 2021, in The Lancet Haematology.

The study is “of importance for physicians caring for HSCT recipients worldwide,” Mathieu Leclerc, MD, and Sébastien Maury, MD, Hôpital Henri Mondor, Créteil, France, commented in an accompanying editorial.
 

Study details

For their study, Dr. Sharma and colleagues analyzed outcomes for all HSCT recipients who developed COVID-19 and whose cases were reported to the CIBMTR. Of 318 such patients, 184 had undergone allogeneic HSCT, and 134 had undergone autologous HSCT.

Overall, about half of these patients (49%) had mild COVID-19.

Severe COVID-19 that required mechanical ventilation developed in 15% and 13% of the allogeneic and autologous HSCT recipients, respectively.

About one-fifth of patients died: 22% and 19% of allogeneic and autologous HSCT recipients, respectively.

Factors associated with greater mortality risk included age of 50 years or older (hazard ratio, 2.53), male sex (HR, 3.53), and development of COVID-19 within 12 months of undergoing HSCT (HR, 2.67).

Among autologous HSCT recipients, lymphoma was associated with higher mortality risk in comparison with a plasma cell disorder or myeloma (HR, 2.41), the authors noted.

“Two important messages can be drawn from the results reported by Sharma and colleagues,” Dr. Leclerc and Dr. Maury wrote in their editorial. “The first is the confirmation that the prognosis of COVID-19 is particularly poor in HSCT recipients, and that its prevention, in the absence of any specific curative treatment with sufficient efficacy, should be at the forefront of concerns.”

The second relates to the risk factors for death among HSCT recipients who develop COVID-19. In addition to previously known risk factors, such as age and gender, the investigators identified transplant-specific factors potentially associated with prognosis – namely, the nearly threefold increase in death among allogeneic HSCT recipients who develop COVID-19 within 12 months of transplant, they explained.

However, the findings are limited by a substantial amount of missing data, short follow-up, and the possibility of selection bias, they noted.

“Further large and well-designed studies with longer follow-up are needed to confirm and refine the results,” the editorialists wrote.

“[A] better understanding of the distinctive features of COVID-19 infection in HSCT recipients will be a necessary and essential step toward improvement of the remarkably poor prognosis observed in this setting,” they added.

The study was funded by the American Society of Hematology; the Leukemia and Lymphoma Society; the National Cancer Institute; the National Heart, Lung and Blood Institute; the National Institute of Allergy and Infectious Diseases; the National Institutes of Health; the Health Resources and Services Administration; and the Office of Naval Research. Dr. Sharma receives support for the conduct of industry-sponsored trials from Vertex Pharmaceuticals, CRISPR Therapeutics, and Novartis and consulting fees from Spotlight Therapeutics. Dr. Leclerc and Dr. Maury disclosed no relevant financial relationships.

A version of this article first appeared on Medscape.com.

Study Overview

Objective. To evaluate the efficacy and safety of lenvatinib in combination with everolimus or pembrolizumab compared with sunitinib alone for the treatment of newly diagnosed advanced clear cell renal cell carcinoma (ccRCC).

Design. Global, multicenter, randomized, open-label, phase 3 trial.

Intervention. Patients were randomized in a 1:1:1 ratio to receive treatment with 1 of 3 regimens: lenvatinib 20 mg daily plus pembrolizumab 200 mg on day 1 of each 21-day cycle; lenvatinib 18 mg daily plus everolimus 5 mg once daily for each 21-day cycle; or sunitinib 50 mg daily for 4 weeks followed by 2 weeks off. Patients were stratified according to geographic region and Memorial Sloan Kettering Cancer Center (MSKCC) prognostic risk group.

Setting and participants. A total of 1417 patients were screened, and 1069 patients underwent randomization between October 2016 and July 2019: 355 patients were randomized to the lenvatinib plus pembrolizumab group, 357 were randomized to the lenvatinib plus everolimus group, and 357 were randomized to the sunitinib alone group. The patients must have had a diagnosis of previously untreated advanced renal cell carcinoma with a clear-cell component. All the patients need to have a Karnofsky performance status of at least 70, adequate renal function, and controlled blood pressure with or without antihypertensive medications.

Main outcome measures. The primary endpoint assessed the progression-free survival (PFS) as evaluated by independent review committee using RECIST, version 1.1. Imaging was performed at the time of screening and every 8 weeks thereafter. Secondary endpoints were safety, overall survival (OS), and objective response rate as well as investigator-assessed PFS. Also, they assessed the duration of response. During the treatment period, the safety and adverse events were assessed up to 30 days from the last dose of the trial drug.

Main results. The baseline characteristics were well balanced between the treatment groups. More than 70% of enrolled participants were male. Approximately 60% of participants were MSKCC intermediate risk, 27% were favorable risk, and 9% were poor risk. Patients with a PD-L1 combined positive score of 1% or more represented 30% of the population. The remainder had a PD-L1 combined positive score of <1% (30%) or such data were not available (38%). Liver metastases were present in 17% of patients at baseline in each group, and 70% of patients had a prior nephrectomy. The data cutoff occurred in August 2020 for PFS and the median follow-up for OS was 26.6 months. Around 40% of the participants in the lenvatinib plus pembrolizumab group, 18.8% in the sunitinib group, and 31% in the lenvatinib plus everolimus group were still receiving trial treatment at data cutoff. The leading cause for discontinuing therapy was disease progression. Approximately 50% of patients in the lenvatinib/everolimus group and sunitinib group received subsequent checkpoint inhibitor therapy after progression.

The median PFS in the lenvatinib plus pembrolizumab group was significantly longer than in the sunitinib group, 23.9 months vs 9.2 months (hazard ratio [HR], 0.39; 95% CI, 0.32-0.49; P < 0.001). The median PFS was also significantly longer in the lenvatinib plus everolimus group compared with sunitinib, 14.7 vs 9.2 months (HR 0.65; 95% CI 0.53-0.80; P < 0.001). The PFS benefit favored the lenvatinib combination groups over sunitinib in all subgroups, including the MSKCC prognostic risk groups. The median OS was not reached with any treatment, with 79% of patients in the lenvatinib plus pembrolizumab group, 66% of patients in the lenvatinib plus everolimus group, and 70% in the sunitinib group still alive at 24 months. Survival was significantly longer in the lenvatinib plus pembrolizumab group compared with sunitinib (HR, 0.66; 95% CI, 0.49-0.88; P = 0.005). The OS favored lenvatinib/pembrolizumab over sunitinib in the PD-L1 positive or negative groups. The median duration of response in the lenvatinib plus pembrolizumab group was 25.8 months compared to 16.6 months and 14.6 months in the lenvatinib plus everolimus and sunitinib groups, respectively. Complete response rates were higher in the lenvatinib plus pembrolizumab group (16%) compared with lenvatinib/everolimus (9.8%) or sunitinib (4.2%). The median time to response was around 1.9 months in all 3 groups.

The most frequent adverse events seen in all groups were diarrhea, hypertension, fatigue, and nausea. Hypothyroidism was seen more frequently in the lenvatinib plus pembrolizumab group (47%). Grade 3 adverse events were seen in approximately 80% of patients in all groups. The most common grade 3 or higher adverse event was hypertension in all 3 groups. The median time for discontinuing treatment due to side effects was 8.97 months in the lenvatinib plus pembrolizumab arm, 5.49 months in the lenvatinib plus everolimus group, and 4.57 months in the sunitinib group. In the lenvatinib plus pembrolizumab group, 15 patients had grade 5 adverse events; 11 participants had fatal events not related to disease progression. In the lenvatinib plus everolimus group, there were 22 patients with grade 5 events, with 10 fatal events not related to disease progression. In the sunitinib group, 11 patients had grade 5 events, and only 2 fatal events were not linked to disease progression.

Conclusion. The combination of lenvatinib plus pembrolizumab significantly prolongs PFS and OS compared with sunitinib in patients with previously untreated and advanced ccRCC. The median OS has not yet been reached.

Commentary

The results of the current phase 3 CLEAR trial highlight the efficacy and safety of lenvatinib plus pembrolizumab as a first-line treatment in advanced ccRCC. This trial adds to the rapidly growing body of literature supporting the notion that the combination of anti-PD-1 based therapy with either CTLA-4 antibodies or VEGF receptor tyrosine kinase inhibitors (TKI) improves outcomes in previously untreated patients with advanced ccRCC. Previously presented data from Keynote-426 (pembrolizumab plus axitinib), Checkmate-214 (nivolumab plus ipilimumab), and Javelin Renal 101 (Avelumab plus axitinib) have also shown improved outcomes with combination therapy in the frontline setting.^1-4 While the landscape of therapeutic options in the frontline setting continues to grow, there remains lack of clarity as to how to tailor our therapeutic decisions for specific patient populations. The exception would be nivolumab and ipilimumab, which are currently indicated for IMDC intermediate- or poor-risk patients.

The combination of VEGFR TKI therapy and PD-1 antibodies provides rapid disease control, with a median time to response in the current study of 1.9 months, and, generally speaking, a low risk of progression in the first 6 months of therapy. While cross-trial comparisons are always problematic, the PFS reported in this study and others with VEGFR TKI and PD-1 antibody combinations is quite impressive and surpasses that noted in Checkmate 214.³ While the median OS survival has not yet been reached, the long duration of PFS and complete response rate of 16% in this study certainly make this an attractive frontline option for newly diagnosed patients with advanced ccRCC. Longer follow-up is needed to confirm the survival benefit noted.

Applications for Clinical Practice

The current data support the use VEGFR TKI and anti-PD1 therapy in the frontline setting. How to choose between such combination regimens or combination immunotherapy remains unclear, and further biomarker-based assessments are needed to help guide therapeutic decisions for our patients.

Study Overview

Objective. To evaluate the efficacy and safety of lenvatinib in combination with everolimus or pembrolizumab compared with sunitinib alone for the treatment of newly diagnosed advanced clear cell renal cell carcinoma (ccRCC).

Design. Global, multicenter, randomized, open-label, phase 3 trial.

Intervention. Patients were randomized in a 1:1:1 ratio to receive treatment with 1 of 3 regimens: lenvatinib 20 mg daily plus pembrolizumab 200 mg on day 1 of each 21-day cycle; lenvatinib 18 mg daily plus everolimus 5 mg once daily for each 21-day cycle; or sunitinib 50 mg daily for 4 weeks followed by 2 weeks off. Patients were stratified according to geographic region and Memorial Sloan Kettering Cancer Center (MSKCC) prognostic risk group.

Setting and participants. A total of 1417 patients were screened, and 1069 patients underwent randomization between October 2016 and July 2019: 355 patients were randomized to the lenvatinib plus pembrolizumab group, 357 were randomized to the lenvatinib plus everolimus group, and 357 were randomized to the sunitinib alone group. The patients must have had a diagnosis of previously untreated advanced renal cell carcinoma with a clear-cell component. All the patients need to have a Karnofsky performance status of at least 70, adequate renal function, and controlled blood pressure with or without antihypertensive medications.

Main outcome measures. The primary endpoint assessed the progression-free survival (PFS) as evaluated by independent review committee using RECIST, version 1.1. Imaging was performed at the time of screening and every 8 weeks thereafter. Secondary endpoints were safety, overall survival (OS), and objective response rate as well as investigator-assessed PFS. Also, they assessed the duration of response. During the treatment period, the safety and adverse events were assessed up to 30 days from the last dose of the trial drug.

Main results. The baseline characteristics were well balanced between the treatment groups. More than 70% of enrolled participants were male. Approximately 60% of participants were MSKCC intermediate risk, 27% were favorable risk, and 9% were poor risk. Patients with a PD-L1 combined positive score of 1% or more represented 30% of the population. The remainder had a PD-L1 combined positive score of <1% (30%) or such data were not available (38%). Liver metastases were present in 17% of patients at baseline in each group, and 70% of patients had a prior nephrectomy. The data cutoff occurred in August 2020 for PFS and the median follow-up for OS was 26.6 months. Around 40% of the participants in the lenvatinib plus pembrolizumab group, 18.8% in the sunitinib group, and 31% in the lenvatinib plus everolimus group were still receiving trial treatment at data cutoff. The leading cause for discontinuing therapy was disease progression. Approximately 50% of patients in the lenvatinib/everolimus group and sunitinib group received subsequent checkpoint inhibitor therapy after progression.

The median PFS in the lenvatinib plus pembrolizumab group was significantly longer than in the sunitinib group, 23.9 months vs 9.2 months (hazard ratio [HR], 0.39; 95% CI, 0.32-0.49; P < 0.001). The median PFS was also significantly longer in the lenvatinib plus everolimus group compared with sunitinib, 14.7 vs 9.2 months (HR 0.65; 95% CI 0.53-0.80; P < 0.001). The PFS benefit favored the lenvatinib combination groups over sunitinib in all subgroups, including the MSKCC prognostic risk groups. The median OS was not reached with any treatment, with 79% of patients in the lenvatinib plus pembrolizumab group, 66% of patients in the lenvatinib plus everolimus group, and 70% in the sunitinib group still alive at 24 months. Survival was significantly longer in the lenvatinib plus pembrolizumab group compared with sunitinib (HR, 0.66; 95% CI, 0.49-0.88; P = 0.005). The OS favored lenvatinib/pembrolizumab over sunitinib in the PD-L1 positive or negative groups. The median duration of response in the lenvatinib plus pembrolizumab group was 25.8 months compared to 16.6 months and 14.6 months in the lenvatinib plus everolimus and sunitinib groups, respectively. Complete response rates were higher in the lenvatinib plus pembrolizumab group (16%) compared with lenvatinib/everolimus (9.8%) or sunitinib (4.2%). The median time to response was around 1.9 months in all 3 groups.

The most frequent adverse events seen in all groups were diarrhea, hypertension, fatigue, and nausea. Hypothyroidism was seen more frequently in the lenvatinib plus pembrolizumab group (47%). Grade 3 adverse events were seen in approximately 80% of patients in all groups. The most common grade 3 or higher adverse event was hypertension in all 3 groups. The median time for discontinuing treatment due to side effects was 8.97 months in the lenvatinib plus pembrolizumab arm, 5.49 months in the lenvatinib plus everolimus group, and 4.57 months in the sunitinib group. In the lenvatinib plus pembrolizumab group, 15 patients had grade 5 adverse events; 11 participants had fatal events not related to disease progression. In the lenvatinib plus everolimus group, there were 22 patients with grade 5 events, with 10 fatal events not related to disease progression. In the sunitinib group, 11 patients had grade 5 events, and only 2 fatal events were not linked to disease progression.

Conclusion. The combination of lenvatinib plus pembrolizumab significantly prolongs PFS and OS compared with sunitinib in patients with previously untreated and advanced ccRCC. The median OS has not yet been reached.

Commentary

The results of the current phase 3 CLEAR trial highlight the efficacy and safety of lenvatinib plus pembrolizumab as a first-line treatment in advanced ccRCC. This trial adds to the rapidly growing body of literature supporting the notion that the combination of anti-PD-1 based therapy with either CTLA-4 antibodies or VEGF receptor tyrosine kinase inhibitors (TKI) improves outcomes in previously untreated patients with advanced ccRCC. Previously presented data from Keynote-426 (pembrolizumab plus axitinib), Checkmate-214 (nivolumab plus ipilimumab), and Javelin Renal 101 (Avelumab plus axitinib) have also shown improved outcomes with combination therapy in the frontline setting.^1-4 While the landscape of therapeutic options in the frontline setting continues to grow, there remains lack of clarity as to how to tailor our therapeutic decisions for specific patient populations. The exception would be nivolumab and ipilimumab, which are currently indicated for IMDC intermediate- or poor-risk patients.

The combination of VEGFR TKI therapy and PD-1 antibodies provides rapid disease control, with a median time to response in the current study of 1.9 months, and, generally speaking, a low risk of progression in the first 6 months of therapy. While cross-trial comparisons are always problematic, the PFS reported in this study and others with VEGFR TKI and PD-1 antibody combinations is quite impressive and surpasses that noted in Checkmate 214.³ While the median OS survival has not yet been reached, the long duration of PFS and complete response rate of 16% in this study certainly make this an attractive frontline option for newly diagnosed patients with advanced ccRCC. Longer follow-up is needed to confirm the survival benefit noted.

Applications for Clinical Practice

The current data support the use VEGFR TKI and anti-PD1 therapy in the frontline setting. How to choose between such combination regimens or combination immunotherapy remains unclear, and further biomarker-based assessments are needed to help guide therapeutic decisions for our patients.

Study Overview

Objective. To evaluate the efficacy and safety of lenvatinib in combination with everolimus or pembrolizumab compared with sunitinib alone for the treatment of newly diagnosed advanced clear cell renal cell carcinoma (ccRCC).

Design. Global, multicenter, randomized, open-label, phase 3 trial.

Intervention. Patients were randomized in a 1:1:1 ratio to receive treatment with 1 of 3 regimens: lenvatinib 20 mg daily plus pembrolizumab 200 mg on day 1 of each 21-day cycle; lenvatinib 18 mg daily plus everolimus 5 mg once daily for each 21-day cycle; or sunitinib 50 mg daily for 4 weeks followed by 2 weeks off. Patients were stratified according to geographic region and Memorial Sloan Kettering Cancer Center (MSKCC) prognostic risk group.

Setting and participants. A total of 1417 patients were screened, and 1069 patients underwent randomization between October 2016 and July 2019: 355 patients were randomized to the lenvatinib plus pembrolizumab group, 357 were randomized to the lenvatinib plus everolimus group, and 357 were randomized to the sunitinib alone group. The patients must have had a diagnosis of previously untreated advanced renal cell carcinoma with a clear-cell component. All the patients need to have a Karnofsky performance status of at least 70, adequate renal function, and controlled blood pressure with or without antihypertensive medications.

Main outcome measures. The primary endpoint assessed the progression-free survival (PFS) as evaluated by independent review committee using RECIST, version 1.1. Imaging was performed at the time of screening and every 8 weeks thereafter. Secondary endpoints were safety, overall survival (OS), and objective response rate as well as investigator-assessed PFS. Also, they assessed the duration of response. During the treatment period, the safety and adverse events were assessed up to 30 days from the last dose of the trial drug.

Main results. The baseline characteristics were well balanced between the treatment groups. More than 70% of enrolled participants were male. Approximately 60% of participants were MSKCC intermediate risk, 27% were favorable risk, and 9% were poor risk. Patients with a PD-L1 combined positive score of 1% or more represented 30% of the population. The remainder had a PD-L1 combined positive score of <1% (30%) or such data were not available (38%). Liver metastases were present in 17% of patients at baseline in each group, and 70% of patients had a prior nephrectomy. The data cutoff occurred in August 2020 for PFS and the median follow-up for OS was 26.6 months. Around 40% of the participants in the lenvatinib plus pembrolizumab group, 18.8% in the sunitinib group, and 31% in the lenvatinib plus everolimus group were still receiving trial treatment at data cutoff. The leading cause for discontinuing therapy was disease progression. Approximately 50% of patients in the lenvatinib/everolimus group and sunitinib group received subsequent checkpoint inhibitor therapy after progression.

The median PFS in the lenvatinib plus pembrolizumab group was significantly longer than in the sunitinib group, 23.9 months vs 9.2 months (hazard ratio [HR], 0.39; 95% CI, 0.32-0.49; P < 0.001). The median PFS was also significantly longer in the lenvatinib plus everolimus group compared with sunitinib, 14.7 vs 9.2 months (HR 0.65; 95% CI 0.53-0.80; P < 0.001). The PFS benefit favored the lenvatinib combination groups over sunitinib in all subgroups, including the MSKCC prognostic risk groups. The median OS was not reached with any treatment, with 79% of patients in the lenvatinib plus pembrolizumab group, 66% of patients in the lenvatinib plus everolimus group, and 70% in the sunitinib group still alive at 24 months. Survival was significantly longer in the lenvatinib plus pembrolizumab group compared with sunitinib (HR, 0.66; 95% CI, 0.49-0.88; P = 0.005). The OS favored lenvatinib/pembrolizumab over sunitinib in the PD-L1 positive or negative groups. The median duration of response in the lenvatinib plus pembrolizumab group was 25.8 months compared to 16.6 months and 14.6 months in the lenvatinib plus everolimus and sunitinib groups, respectively. Complete response rates were higher in the lenvatinib plus pembrolizumab group (16%) compared with lenvatinib/everolimus (9.8%) or sunitinib (4.2%). The median time to response was around 1.9 months in all 3 groups.

The most frequent adverse events seen in all groups were diarrhea, hypertension, fatigue, and nausea. Hypothyroidism was seen more frequently in the lenvatinib plus pembrolizumab group (47%). Grade 3 adverse events were seen in approximately 80% of patients in all groups. The most common grade 3 or higher adverse event was hypertension in all 3 groups. The median time for discontinuing treatment due to side effects was 8.97 months in the lenvatinib plus pembrolizumab arm, 5.49 months in the lenvatinib plus everolimus group, and 4.57 months in the sunitinib group. In the lenvatinib plus pembrolizumab group, 15 patients had grade 5 adverse events; 11 participants had fatal events not related to disease progression. In the lenvatinib plus everolimus group, there were 22 patients with grade 5 events, with 10 fatal events not related to disease progression. In the sunitinib group, 11 patients had grade 5 events, and only 2 fatal events were not linked to disease progression.

Conclusion. The combination of lenvatinib plus pembrolizumab significantly prolongs PFS and OS compared with sunitinib in patients with previously untreated and advanced ccRCC. The median OS has not yet been reached.

Commentary

The results of the current phase 3 CLEAR trial highlight the efficacy and safety of lenvatinib plus pembrolizumab as a first-line treatment in advanced ccRCC. This trial adds to the rapidly growing body of literature supporting the notion that the combination of anti-PD-1 based therapy with either CTLA-4 antibodies or VEGF receptor tyrosine kinase inhibitors (TKI) improves outcomes in previously untreated patients with advanced ccRCC. Previously presented data from Keynote-426 (pembrolizumab plus axitinib), Checkmate-214 (nivolumab plus ipilimumab), and Javelin Renal 101 (Avelumab plus axitinib) have also shown improved outcomes with combination therapy in the frontline setting.^1-4 While the landscape of therapeutic options in the frontline setting continues to grow, there remains lack of clarity as to how to tailor our therapeutic decisions for specific patient populations. The exception would be nivolumab and ipilimumab, which are currently indicated for IMDC intermediate- or poor-risk patients.

The combination of VEGFR TKI therapy and PD-1 antibodies provides rapid disease control, with a median time to response in the current study of 1.9 months, and, generally speaking, a low risk of progression in the first 6 months of therapy. While cross-trial comparisons are always problematic, the PFS reported in this study and others with VEGFR TKI and PD-1 antibody combinations is quite impressive and surpasses that noted in Checkmate 214.³ While the median OS survival has not yet been reached, the long duration of PFS and complete response rate of 16% in this study certainly make this an attractive frontline option for newly diagnosed patients with advanced ccRCC. Longer follow-up is needed to confirm the survival benefit noted.

Applications for Clinical Practice

The current data support the use VEGFR TKI and anti-PD1 therapy in the frontline setting. How to choose between such combination regimens or combination immunotherapy remains unclear, and further biomarker-based assessments are needed to help guide therapeutic decisions for our patients.

From The Ohio State University College of Medicine Department of Family and Community Medicine, Columbus, OH (Candy Magaña, Jná Báez, Christine Junk, Drs. Ahmad, Conroy, and Olayiwola); The Ohio State University College of Medicine Center for Primary Care Innovation and Transformation (Candy Magaña, Jná Báez, and Dr. Olayiwola); and The Ohio State University Wexner Medical Center (Christine Harsh, Erica Esposito).

Much has been discussed about the growing crisis of professional dissatisfaction among physicians, with increasing efforts being made to incorporate physician wellness into health system strategies that move from the Triple to the Quadruple Aim.¹ For many years, our health care system has been focused on improving the health of populations, optimizing the patient experience, and reducing the cost of care (Triple Aim). The inclusion of the fourth aim, improving the experience of the teams that deliver care, has become paramount in achieving the other aims.

An area often overlooked in this focus on wellness, however, is the importance of the earliest days of employment to shape and predict long-term career contentment. This is a missed opportunity, as data suggest that organizations with standardized onboarding programs boast a 62% increased productivity rate and a 50% greater retention rate among new hires.^2,3 Moreover, a study by the International Institute for Management Development found that businesses lose an estimated $37 billion annually because employees do not fully understand their jobs.⁴ The report ties losses to “actions taken by employees who have misunderstood or misinterpreted company policies, business processes, job function, or a combination of the three.” Additionally, onboarding programs that focus strictly on technical or functional orientation tasks miss important opportunities for culture integration during the onboarding process.⁵ It is therefore imperative to look to effective models of employee onboarding to develop systems that position physicians and practices for success.

Challenges With Traditional Physician Onboarding

In recent years, the Department of Family and Community Medicine at The Ohio State University College of Medicine has experienced rapid organizational change. Like many primary care systems nationwide responding to disruption in health care and changing demands on the clinical workforce, the department has hired new leadership, revised strategic priorities, and witnessed an influx of faculty and staff. It has also planned an expansion of ambulatory services that will more than double the clinical workforce over the next 3 years. While an exciting time, there has been a growing need to align strategy, culture, and human capital during these changes.

As we entered this phase of transformation, we recognized that our highly individualized, ad hoc orientation system presented shortcomings. During the act of revamping our physician recruitment process, stakeholder workgroup members specifically noted that improvement efforts were needed regarding new physician orientation, as no consistent structures were previously in place. New physician orientation had been a major gap for years, resulting in dissatisfaction in the first few months of physician practice, early physician turnover, and staff frustration. For physicians, we continued to learn about their frustration and unanswered questions regarding expectations, norms, structures, and processes.

Many new hires were left with a kind of “trial by fire” entry into their roles. On the first day of clinic, a new physician would most likely need to simultaneously see patients, learn the nuances of the electronic health record (EHR), figure out where the break room was located, and quickly learn population health issues for the patients they were serving. Opportunities to meet key clinic site leadership would be at random, and new physicians might not have the opportunity to meet leadership or staff until months into their tenure; this did not allow for a sense of belonging or understanding of the many resources available to them. We learned that the quality of these ad hoc orientations also varied based on the experience and priorities of each practice’s clinic and administrative leaders, who themselves felt ill-equipped to provide a consistent, robust, and confidence-building experience. In addition, practice site management was rarely given advance time to prepare for the arrival of new physicians, which resulted in physicians perceiving practices to be unwelcoming and disorganized. Their first days were often spent with patients in clinic with no structured orientation and without understanding workflows or having systems practice knowledge.

Institutionally, the interview process satisfied some transfer of knowledge, but we were unclear of what was being consistently shared and understood in the multiple ambulatory locations where our physicians enter practice. More importantly, we knew we were missing a critical opportunity to use orientation to imbue other values of diversity and inclusion, health equity, and operational excellence into the workforce. Based on anecdotal insights from employees and our own review of successful onboarding approaches from other industries, we also knew a more structured welcoming process would predict greater long-term career satisfaction for physicians and create a foundation for providing optimal care for patients when clinical encounters began.

Reengineering Physician Onboarding

In 2019, our department developed a multipronged approach to physician onboarding, which is already paying dividends in easing acculturation and fostering team cohesion. The department tapped its Center for Primary Care Innovation and Transformation (PCIT) to direct this effort, based on its expertise in practice transformation, clinical transformation and adaptations, and workflow efficiency through process and quality improvement. The PCIT team provides support to the department and the entire health system focused on technology and innovation, health equity, and health care efficiency.⁶ They applied many of the tools used in the Clinical Transformation in Technology approach to lead this initiative.⁷

The PCIT team began identifying key stakeholders (department, clinical and ambulatory leadership, clinicians and clinical staff, community partners, human resources, and resident physicians), and then engaging those individuals in dialogue surrounding orientation needs. During scheduled in-person and virtual work sessions, stakeholders were asked to provide input on pain points for new physicians and clinic leadership and were then empowered to create an onboarding program. Applying health care quality improvement techniques, we leveraged workflow mapping, current and future state planning, and goal setting, led by the skilled process improvement and clinical transformation specialists. We coordinated a multidisciplinary process improvement team that included clinic administrators, medical directors, human resources, administrative staff, ambulatory and resident leadership, clinical leadership, and recruitment liaisons. This diverse group of leadership and staff was brought together to address these critical identified gaps and weaknesses in new physician onboarding.

Through a series of learning sessions, the workgroup provided input that was used to form an itemized physician onboarding schedule, which was then leveraged to develop Plan-Do-Study-Act (PDSA) cycles, collecting feedback in real time. Some issues that seem small can cause major distress for new physicians. For example, in our inaugural orientation implementation, a physician provided feedback that they wanted to obtain information on setting up their work email on their personal devices and was having considerable trouble figuring out how to do so. This particular topic was not initially included in the first iteration of the Department’s orientation program. We rapidly sought out different ways to embed that into the onboarding experience. The first PDSA involved integrating the university information technology team (IT) into the process but was not successful because it required extra work for the new physician and reliance on the IT schedule. The next attempt was to have IT train a department staff member, but again, this still required that the physician find time to connect with that staff member. Finally, we decided to obtain a useful tip sheet that clearly outlined the process and could be included in orientation materials. This gave the new physicians control over how and when they would work on this issue. Based on these learnings, this was incorporated as a standing agenda item and resource for incoming physicians.

Essential Elements of Effective Onboarding

The new physician onboarding program consists of 5 key elements: (1) 2-week acclimation period; (2) peer learning and connection; (3) training before beginning patient care; (4) standardization, transparency, and accountability in all processes; (5) ongoing feedback for continued program improvement with individual support (Figure).

The program begins with a 2-week period of intentional investment in individual success, during which time no patients are scheduled. In week 1, we work with new hires to set expectations for performance, understand departmental norms, and introduce culture. Physicians meet formally and informally with department and institutional leadership, as well as attend team meetings and trainings that include a range of administrative and compliance requirements, such as quality standards and expectations, compliance, billing and coding specific to family medicine, EHR management, and institutionally mandated orientations. We are also adding implicit bias and antiracism training during this period, which are essential to creating a culture of unity and belonging.

During week 2, we focus on clinic-level orientation, assigning new hires an orientation buddy and a department sponsor, such as a physician lead or medical director. Physicians spend time with leadership at their clinic as they nurture relationships important for mentorship, sponsorship, and peer support. They also meet care team members, including front desk associates, medical assistants, behavioral health clinicians, nutritionists, social workers, pharmacists, and other key colleagues and care team members. This introduces the physician to the clinical environment and physical space as well as acclimates the physician to workflows and feedback loops for regular interaction.

When physicians ultimately begin patient care, they begin with an expected productivity rate of 50%, followed by an expected productivity rate of 75%, and then an expected productivity rate of 100%. This steady increase occurs over 3 to 4 weeks depending on the physician’s comfort level. They are also provided monthly reports on work relative value unit performance so that they can track and adapt practice patterns as necessary.More details on the program can be found in Appendix 1.

Takeaways From the Implementation of the New Program

Give time for new physicians to focus on acclimating to the role and environment.

The initial 2-week period of transition—without direct patient care—ensures that physicians feel comfortable in their new ecosystem. This also supports personal transitions, as many new hires are managing relocation and acclimating themselves and their families to new settings. Even residents from our training program who returned as attending physicians found this flexibility and slow reentry essential. This also gives the clinic time to orient to an additional provider, nurture them into the team culture, and develop relationships with the care team.

Cultivate spaces for shared learning, problem-solving, and peer connection.

Orientation is delivered primarily through group learning sessions with cohorts of new physicians, thus developing spaces for networking, fostering psychological safety, encouraging personal and professional rapport, emphasizing interactive learning, and reinforcing scheduling blocks at the departmental level. New hires also participate in peer shadowing to develop clinical competencies and are assigned a workplace buddy to foster a sense of belonging and create opportunities for additional knowledge sharing and cross-training.

Strengthen physician knowledge base, confidence, and comfort in the workplace before beginning direct patient care.

Without fluency in the workflows, culture, and operations of a practice, the urgency to have physicians begin clinical care can result in frustration for the physician, patients, and clinical and administrative staff. Therefore, we complete essential training prior to seeing any patients. This includes clinical workflows, referral processes, use of alternate modalities of care (eg, telehealth, eConsults), billing protocols, population health training, patient resources, office resources, and other essential daily processes and tools. This creates efficiency in administrative management, increased productivity, and better understanding of resources available for patients’ medical, social, and behavioral needs when patient care begins.

Embrace standardization, transparency, and accountability in as many processes as possible.

Standardized knowledge-sharing and checklists are mandated at every step of the orientation process, requiring sign off from the physician lead, practice manager, and new physicians upon completion. This offers all parties the opportunity to play a role in the delivery of and accountability for skills transfer and empowers new hires to press pause if they feel unsure about any domain in the training. It is also essential in guaranteeing that all physicians—regardless of which ambulatory location they practice in—receive consistent information and expectations. A sample checklist can be found in Appendix 2.

Commit to collecting and acting on feedback for continued program improvement and individual support.

As physicians complete the program, it is necessary to create structures to measure and enhance its impact, as well as evaluate how physicians are faring following the program. Each physician completes surveys at the end of the orientation program, attends a 90-day post-program check-in with the department chair, and receives follow-up trainings on advanced topics as they become more deeply embedded in the organization.

Lessons Learned

Feedback from surveys and 90-day check-ins with leadership and physicians reflect a high degree of clarity on job roles and duties, a sense of team camaraderie, easier system navigation, and a strong sense of support. We do recognize that sustaining change takes time and our study is limited by data demonstrating the impact of these efforts. We look forward to sharing more robust data from surveys and qualitative interviews with physicians, clinical leadership, and staff in the future. Our team will conduct interviews at 90-day and 180-day checkpoints with new physicians who have gone through this program, followed by a check-in after 1 year. Additionally, new physicians as well as key stakeholders, such as physician leads, practice managers, and members of the recruitment team, have started to participate in short surveys. These are designed to better understand their experiences, what worked well, what can be improved, and the overall satisfaction of the physician and other members of the extended care team.

What follows are some comments made by the initial group of physicians that went through this program and participated in follow-up interviews:

“I really feel like part of a bigger team.”

“I knew exactly what do to when I walked into the exam room on clinic Day 1.”

“It was great to make deep connections during the early process of joining.”

“Having a buddy to direct questions and ideas to is amazing and empowering.”

“Even though the orientation was long, I felt that I learned so much that I would not have otherwise.”

“Thank you for not letting me crash and burn!”

“Great culture! I love understanding our values of health equity, diversity, and inclusion.”

In the months since our endeavor began, we have learned just how essential it is to fully and effectively integrate new hires into the organization for their own satisfaction and success—and ours. Indeed, we cannot expect to achieve the Quadruple Aim without investing in the kind of transparent and intentional orientation process that defines expectations, aligns cultural values, mitigates costly and stressful operational misunderstandings, and communicates to physicians that, not only do they belong, but their sense of belonging is our priority. While we have yet to understand the impact of this program on the fourth aim of the Quadruple Aim, we are hopeful that the benefits will be far-reaching.

It is our ultimate hope that programs like this: (1) give physicians the confidence needed to create impactful patient-centered experiences; (2) enable physicians to become more cost-effective and efficient in care delivery; (3) allow physicians to understand the populations they are serving and access tools available to mitigate health disparities and other barriers; and (4) improve the collective experience of every member of the care team, practice leadership, and clinician-patient partnership.

Corresponding author: J. Nwando Olayiwola, MD, MPH, FAAFP, The Ohio State University College of Medicine, Department of Family and Community Medicine, 2231 N High St, Ste 250, Columbus, OH 43210; [email protected].

Financial disclosures: None.

Keywords: physician onboarding; Quadruple Aim; leadership; clinician satisfaction; care team satisfaction.

From The Ohio State University College of Medicine Department of Family and Community Medicine, Columbus, OH (Candy Magaña, Jná Báez, Christine Junk, Drs. Ahmad, Conroy, and Olayiwola); The Ohio State University College of Medicine Center for Primary Care Innovation and Transformation (Candy Magaña, Jná Báez, and Dr. Olayiwola); and The Ohio State University Wexner Medical Center (Christine Harsh, Erica Esposito).

Much has been discussed about the growing crisis of professional dissatisfaction among physicians, with increasing efforts being made to incorporate physician wellness into health system strategies that move from the Triple to the Quadruple Aim.¹ For many years, our health care system has been focused on improving the health of populations, optimizing the patient experience, and reducing the cost of care (Triple Aim). The inclusion of the fourth aim, improving the experience of the teams that deliver care, has become paramount in achieving the other aims.

An area often overlooked in this focus on wellness, however, is the importance of the earliest days of employment to shape and predict long-term career contentment. This is a missed opportunity, as data suggest that organizations with standardized onboarding programs boast a 62% increased productivity rate and a 50% greater retention rate among new hires.^2,3 Moreover, a study by the International Institute for Management Development found that businesses lose an estimated $37 billion annually because employees do not fully understand their jobs.⁴ The report ties losses to “actions taken by employees who have misunderstood or misinterpreted company policies, business processes, job function, or a combination of the three.” Additionally, onboarding programs that focus strictly on technical or functional orientation tasks miss important opportunities for culture integration during the onboarding process.⁵ It is therefore imperative to look to effective models of employee onboarding to develop systems that position physicians and practices for success.

Challenges With Traditional Physician Onboarding

In recent years, the Department of Family and Community Medicine at The Ohio State University College of Medicine has experienced rapid organizational change. Like many primary care systems nationwide responding to disruption in health care and changing demands on the clinical workforce, the department has hired new leadership, revised strategic priorities, and witnessed an influx of faculty and staff. It has also planned an expansion of ambulatory services that will more than double the clinical workforce over the next 3 years. While an exciting time, there has been a growing need to align strategy, culture, and human capital during these changes.

As we entered this phase of transformation, we recognized that our highly individualized, ad hoc orientation system presented shortcomings. During the act of revamping our physician recruitment process, stakeholder workgroup members specifically noted that improvement efforts were needed regarding new physician orientation, as no consistent structures were previously in place. New physician orientation had been a major gap for years, resulting in dissatisfaction in the first few months of physician practice, early physician turnover, and staff frustration. For physicians, we continued to learn about their frustration and unanswered questions regarding expectations, norms, structures, and processes.

Many new hires were left with a kind of “trial by fire” entry into their roles. On the first day of clinic, a new physician would most likely need to simultaneously see patients, learn the nuances of the electronic health record (EHR), figure out where the break room was located, and quickly learn population health issues for the patients they were serving. Opportunities to meet key clinic site leadership would be at random, and new physicians might not have the opportunity to meet leadership or staff until months into their tenure; this did not allow for a sense of belonging or understanding of the many resources available to them. We learned that the quality of these ad hoc orientations also varied based on the experience and priorities of each practice’s clinic and administrative leaders, who themselves felt ill-equipped to provide a consistent, robust, and confidence-building experience. In addition, practice site management was rarely given advance time to prepare for the arrival of new physicians, which resulted in physicians perceiving practices to be unwelcoming and disorganized. Their first days were often spent with patients in clinic with no structured orientation and without understanding workflows or having systems practice knowledge.

Institutionally, the interview process satisfied some transfer of knowledge, but we were unclear of what was being consistently shared and understood in the multiple ambulatory locations where our physicians enter practice. More importantly, we knew we were missing a critical opportunity to use orientation to imbue other values of diversity and inclusion, health equity, and operational excellence into the workforce. Based on anecdotal insights from employees and our own review of successful onboarding approaches from other industries, we also knew a more structured welcoming process would predict greater long-term career satisfaction for physicians and create a foundation for providing optimal care for patients when clinical encounters began.

Reengineering Physician Onboarding

In 2019, our department developed a multipronged approach to physician onboarding, which is already paying dividends in easing acculturation and fostering team cohesion. The department tapped its Center for Primary Care Innovation and Transformation (PCIT) to direct this effort, based on its expertise in practice transformation, clinical transformation and adaptations, and workflow efficiency through process and quality improvement. The PCIT team provides support to the department and the entire health system focused on technology and innovation, health equity, and health care efficiency.⁶ They applied many of the tools used in the Clinical Transformation in Technology approach to lead this initiative.⁷

The PCIT team began identifying key stakeholders (department, clinical and ambulatory leadership, clinicians and clinical staff, community partners, human resources, and resident physicians), and then engaging those individuals in dialogue surrounding orientation needs. During scheduled in-person and virtual work sessions, stakeholders were asked to provide input on pain points for new physicians and clinic leadership and were then empowered to create an onboarding program. Applying health care quality improvement techniques, we leveraged workflow mapping, current and future state planning, and goal setting, led by the skilled process improvement and clinical transformation specialists. We coordinated a multidisciplinary process improvement team that included clinic administrators, medical directors, human resources, administrative staff, ambulatory and resident leadership, clinical leadership, and recruitment liaisons. This diverse group of leadership and staff was brought together to address these critical identified gaps and weaknesses in new physician onboarding.

Through a series of learning sessions, the workgroup provided input that was used to form an itemized physician onboarding schedule, which was then leveraged to develop Plan-Do-Study-Act (PDSA) cycles, collecting feedback in real time. Some issues that seem small can cause major distress for new physicians. For example, in our inaugural orientation implementation, a physician provided feedback that they wanted to obtain information on setting up their work email on their personal devices and was having considerable trouble figuring out how to do so. This particular topic was not initially included in the first iteration of the Department’s orientation program. We rapidly sought out different ways to embed that into the onboarding experience. The first PDSA involved integrating the university information technology team (IT) into the process but was not successful because it required extra work for the new physician and reliance on the IT schedule. The next attempt was to have IT train a department staff member, but again, this still required that the physician find time to connect with that staff member. Finally, we decided to obtain a useful tip sheet that clearly outlined the process and could be included in orientation materials. This gave the new physicians control over how and when they would work on this issue. Based on these learnings, this was incorporated as a standing agenda item and resource for incoming physicians.

Essential Elements of Effective Onboarding

The new physician onboarding program consists of 5 key elements: (1) 2-week acclimation period; (2) peer learning and connection; (3) training before beginning patient care; (4) standardization, transparency, and accountability in all processes; (5) ongoing feedback for continued program improvement with individual support (Figure).

The program begins with a 2-week period of intentional investment in individual success, during which time no patients are scheduled. In week 1, we work with new hires to set expectations for performance, understand departmental norms, and introduce culture. Physicians meet formally and informally with department and institutional leadership, as well as attend team meetings and trainings that include a range of administrative and compliance requirements, such as quality standards and expectations, compliance, billing and coding specific to family medicine, EHR management, and institutionally mandated orientations. We are also adding implicit bias and antiracism training during this period, which are essential to creating a culture of unity and belonging.

During week 2, we focus on clinic-level orientation, assigning new hires an orientation buddy and a department sponsor, such as a physician lead or medical director. Physicians spend time with leadership at their clinic as they nurture relationships important for mentorship, sponsorship, and peer support. They also meet care team members, including front desk associates, medical assistants, behavioral health clinicians, nutritionists, social workers, pharmacists, and other key colleagues and care team members. This introduces the physician to the clinical environment and physical space as well as acclimates the physician to workflows and feedback loops for regular interaction.

When physicians ultimately begin patient care, they begin with an expected productivity rate of 50%, followed by an expected productivity rate of 75%, and then an expected productivity rate of 100%. This steady increase occurs over 3 to 4 weeks depending on the physician’s comfort level. They are also provided monthly reports on work relative value unit performance so that they can track and adapt practice patterns as necessary.More details on the program can be found in Appendix 1.

Takeaways From the Implementation of the New Program

Give time for new physicians to focus on acclimating to the role and environment.

The initial 2-week period of transition—without direct patient care—ensures that physicians feel comfortable in their new ecosystem. This also supports personal transitions, as many new hires are managing relocation and acclimating themselves and their families to new settings. Even residents from our training program who returned as attending physicians found this flexibility and slow reentry essential. This also gives the clinic time to orient to an additional provider, nurture them into the team culture, and develop relationships with the care team.

Cultivate spaces for shared learning, problem-solving, and peer connection.

Orientation is delivered primarily through group learning sessions with cohorts of new physicians, thus developing spaces for networking, fostering psychological safety, encouraging personal and professional rapport, emphasizing interactive learning, and reinforcing scheduling blocks at the departmental level. New hires also participate in peer shadowing to develop clinical competencies and are assigned a workplace buddy to foster a sense of belonging and create opportunities for additional knowledge sharing and cross-training.

Strengthen physician knowledge base, confidence, and comfort in the workplace before beginning direct patient care.

Without fluency in the workflows, culture, and operations of a practice, the urgency to have physicians begin clinical care can result in frustration for the physician, patients, and clinical and administrative staff. Therefore, we complete essential training prior to seeing any patients. This includes clinical workflows, referral processes, use of alternate modalities of care (eg, telehealth, eConsults), billing protocols, population health training, patient resources, office resources, and other essential daily processes and tools. This creates efficiency in administrative management, increased productivity, and better understanding of resources available for patients’ medical, social, and behavioral needs when patient care begins.

Embrace standardization, transparency, and accountability in as many processes as possible.

Standardized knowledge-sharing and checklists are mandated at every step of the orientation process, requiring sign off from the physician lead, practice manager, and new physicians upon completion. This offers all parties the opportunity to play a role in the delivery of and accountability for skills transfer and empowers new hires to press pause if they feel unsure about any domain in the training. It is also essential in guaranteeing that all physicians—regardless of which ambulatory location they practice in—receive consistent information and expectations. A sample checklist can be found in Appendix 2.

Commit to collecting and acting on feedback for continued program improvement and individual support.

As physicians complete the program, it is necessary to create structures to measure and enhance its impact, as well as evaluate how physicians are faring following the program. Each physician completes surveys at the end of the orientation program, attends a 90-day post-program check-in with the department chair, and receives follow-up trainings on advanced topics as they become more deeply embedded in the organization.

Lessons Learned

Feedback from surveys and 90-day check-ins with leadership and physicians reflect a high degree of clarity on job roles and duties, a sense of team camaraderie, easier system navigation, and a strong sense of support. We do recognize that sustaining change takes time and our study is limited by data demonstrating the impact of these efforts. We look forward to sharing more robust data from surveys and qualitative interviews with physicians, clinical leadership, and staff in the future. Our team will conduct interviews at 90-day and 180-day checkpoints with new physicians who have gone through this program, followed by a check-in after 1 year. Additionally, new physicians as well as key stakeholders, such as physician leads, practice managers, and members of the recruitment team, have started to participate in short surveys. These are designed to better understand their experiences, what worked well, what can be improved, and the overall satisfaction of the physician and other members of the extended care team.

What follows are some comments made by the initial group of physicians that went through this program and participated in follow-up interviews:

“I really feel like part of a bigger team.”

“I knew exactly what do to when I walked into the exam room on clinic Day 1.”

“It was great to make deep connections during the early process of joining.”

“Having a buddy to direct questions and ideas to is amazing and empowering.”

“Even though the orientation was long, I felt that I learned so much that I would not have otherwise.”

“Thank you for not letting me crash and burn!”

“Great culture! I love understanding our values of health equity, diversity, and inclusion.”

In the months since our endeavor began, we have learned just how essential it is to fully and effectively integrate new hires into the organization for their own satisfaction and success—and ours. Indeed, we cannot expect to achieve the Quadruple Aim without investing in the kind of transparent and intentional orientation process that defines expectations, aligns cultural values, mitigates costly and stressful operational misunderstandings, and communicates to physicians that, not only do they belong, but their sense of belonging is our priority. While we have yet to understand the impact of this program on the fourth aim of the Quadruple Aim, we are hopeful that the benefits will be far-reaching.

It is our ultimate hope that programs like this: (1) give physicians the confidence needed to create impactful patient-centered experiences; (2) enable physicians to become more cost-effective and efficient in care delivery; (3) allow physicians to understand the populations they are serving and access tools available to mitigate health disparities and other barriers; and (4) improve the collective experience of every member of the care team, practice leadership, and clinician-patient partnership.

Corresponding author: J. Nwando Olayiwola, MD, MPH, FAAFP, The Ohio State University College of Medicine, Department of Family and Community Medicine, 2231 N High St, Ste 250, Columbus, OH 43210; [email protected].

Financial disclosures: None.

Keywords: physician onboarding; Quadruple Aim; leadership; clinician satisfaction; care team satisfaction.

From The Ohio State University College of Medicine Department of Family and Community Medicine, Columbus, OH (Candy Magaña, Jná Báez, Christine Junk, Drs. Ahmad, Conroy, and Olayiwola); The Ohio State University College of Medicine Center for Primary Care Innovation and Transformation (Candy Magaña, Jná Báez, and Dr. Olayiwola); and The Ohio State University Wexner Medical Center (Christine Harsh, Erica Esposito).

Much has been discussed about the growing crisis of professional dissatisfaction among physicians, with increasing efforts being made to incorporate physician wellness into health system strategies that move from the Triple to the Quadruple Aim.¹ For many years, our health care system has been focused on improving the health of populations, optimizing the patient experience, and reducing the cost of care (Triple Aim). The inclusion of the fourth aim, improving the experience of the teams that deliver care, has become paramount in achieving the other aims.

An area often overlooked in this focus on wellness, however, is the importance of the earliest days of employment to shape and predict long-term career contentment. This is a missed opportunity, as data suggest that organizations with standardized onboarding programs boast a 62% increased productivity rate and a 50% greater retention rate among new hires.^2,3 Moreover, a study by the International Institute for Management Development found that businesses lose an estimated $37 billion annually because employees do not fully understand their jobs.⁴ The report ties losses to “actions taken by employees who have misunderstood or misinterpreted company policies, business processes, job function, or a combination of the three.” Additionally, onboarding programs that focus strictly on technical or functional orientation tasks miss important opportunities for culture integration during the onboarding process.⁵ It is therefore imperative to look to effective models of employee onboarding to develop systems that position physicians and practices for success.

Challenges With Traditional Physician Onboarding

In recent years, the Department of Family and Community Medicine at The Ohio State University College of Medicine has experienced rapid organizational change. Like many primary care systems nationwide responding to disruption in health care and changing demands on the clinical workforce, the department has hired new leadership, revised strategic priorities, and witnessed an influx of faculty and staff. It has also planned an expansion of ambulatory services that will more than double the clinical workforce over the next 3 years. While an exciting time, there has been a growing need to align strategy, culture, and human capital during these changes.

As we entered this phase of transformation, we recognized that our highly individualized, ad hoc orientation system presented shortcomings. During the act of revamping our physician recruitment process, stakeholder workgroup members specifically noted that improvement efforts were needed regarding new physician orientation, as no consistent structures were previously in place. New physician orientation had been a major gap for years, resulting in dissatisfaction in the first few months of physician practice, early physician turnover, and staff frustration. For physicians, we continued to learn about their frustration and unanswered questions regarding expectations, norms, structures, and processes.

Many new hires were left with a kind of “trial by fire” entry into their roles. On the first day of clinic, a new physician would most likely need to simultaneously see patients, learn the nuances of the electronic health record (EHR), figure out where the break room was located, and quickly learn population health issues for the patients they were serving. Opportunities to meet key clinic site leadership would be at random, and new physicians might not have the opportunity to meet leadership or staff until months into their tenure; this did not allow for a sense of belonging or understanding of the many resources available to them. We learned that the quality of these ad hoc orientations also varied based on the experience and priorities of each practice’s clinic and administrative leaders, who themselves felt ill-equipped to provide a consistent, robust, and confidence-building experience. In addition, practice site management was rarely given advance time to prepare for the arrival of new physicians, which resulted in physicians perceiving practices to be unwelcoming and disorganized. Their first days were often spent with patients in clinic with no structured orientation and without understanding workflows or having systems practice knowledge.

Institutionally, the interview process satisfied some transfer of knowledge, but we were unclear of what was being consistently shared and understood in the multiple ambulatory locations where our physicians enter practice. More importantly, we knew we were missing a critical opportunity to use orientation to imbue other values of diversity and inclusion, health equity, and operational excellence into the workforce. Based on anecdotal insights from employees and our own review of successful onboarding approaches from other industries, we also knew a more structured welcoming process would predict greater long-term career satisfaction for physicians and create a foundation for providing optimal care for patients when clinical encounters began.

Reengineering Physician Onboarding

In 2019, our department developed a multipronged approach to physician onboarding, which is already paying dividends in easing acculturation and fostering team cohesion. The department tapped its Center for Primary Care Innovation and Transformation (PCIT) to direct this effort, based on its expertise in practice transformation, clinical transformation and adaptations, and workflow efficiency through process and quality improvement. The PCIT team provides support to the department and the entire health system focused on technology and innovation, health equity, and health care efficiency.⁶ They applied many of the tools used in the Clinical Transformation in Technology approach to lead this initiative.⁷

The PCIT team began identifying key stakeholders (department, clinical and ambulatory leadership, clinicians and clinical staff, community partners, human resources, and resident physicians), and then engaging those individuals in dialogue surrounding orientation needs. During scheduled in-person and virtual work sessions, stakeholders were asked to provide input on pain points for new physicians and clinic leadership and were then empowered to create an onboarding program. Applying health care quality improvement techniques, we leveraged workflow mapping, current and future state planning, and goal setting, led by the skilled process improvement and clinical transformation specialists. We coordinated a multidisciplinary process improvement team that included clinic administrators, medical directors, human resources, administrative staff, ambulatory and resident leadership, clinical leadership, and recruitment liaisons. This diverse group of leadership and staff was brought together to address these critical identified gaps and weaknesses in new physician onboarding.

Through a series of learning sessions, the workgroup provided input that was used to form an itemized physician onboarding schedule, which was then leveraged to develop Plan-Do-Study-Act (PDSA) cycles, collecting feedback in real time. Some issues that seem small can cause major distress for new physicians. For example, in our inaugural orientation implementation, a physician provided feedback that they wanted to obtain information on setting up their work email on their personal devices and was having considerable trouble figuring out how to do so. This particular topic was not initially included in the first iteration of the Department’s orientation program. We rapidly sought out different ways to embed that into the onboarding experience. The first PDSA involved integrating the university information technology team (IT) into the process but was not successful because it required extra work for the new physician and reliance on the IT schedule. The next attempt was to have IT train a department staff member, but again, this still required that the physician find time to connect with that staff member. Finally, we decided to obtain a useful tip sheet that clearly outlined the process and could be included in orientation materials. This gave the new physicians control over how and when they would work on this issue. Based on these learnings, this was incorporated as a standing agenda item and resource for incoming physicians.

Essential Elements of Effective Onboarding

The new physician onboarding program consists of 5 key elements: (1) 2-week acclimation period; (2) peer learning and connection; (3) training before beginning patient care; (4) standardization, transparency, and accountability in all processes; (5) ongoing feedback for continued program improvement with individual support (Figure).

The program begins with a 2-week period of intentional investment in individual success, during which time no patients are scheduled. In week 1, we work with new hires to set expectations for performance, understand departmental norms, and introduce culture. Physicians meet formally and informally with department and institutional leadership, as well as attend team meetings and trainings that include a range of administrative and compliance requirements, such as quality standards and expectations, compliance, billing and coding specific to family medicine, EHR management, and institutionally mandated orientations. We are also adding implicit bias and antiracism training during this period, which are essential to creating a culture of unity and belonging.

During week 2, we focus on clinic-level orientation, assigning new hires an orientation buddy and a department sponsor, such as a physician lead or medical director. Physicians spend time with leadership at their clinic as they nurture relationships important for mentorship, sponsorship, and peer support. They also meet care team members, including front desk associates, medical assistants, behavioral health clinicians, nutritionists, social workers, pharmacists, and other key colleagues and care team members. This introduces the physician to the clinical environment and physical space as well as acclimates the physician to workflows and feedback loops for regular interaction.

When physicians ultimately begin patient care, they begin with an expected productivity rate of 50%, followed by an expected productivity rate of 75%, and then an expected productivity rate of 100%. This steady increase occurs over 3 to 4 weeks depending on the physician’s comfort level. They are also provided monthly reports on work relative value unit performance so that they can track and adapt practice patterns as necessary.More details on the program can be found in Appendix 1.

Takeaways From the Implementation of the New Program

Give time for new physicians to focus on acclimating to the role and environment.

The initial 2-week period of transition—without direct patient care—ensures that physicians feel comfortable in their new ecosystem. This also supports personal transitions, as many new hires are managing relocation and acclimating themselves and their families to new settings. Even residents from our training program who returned as attending physicians found this flexibility and slow reentry essential. This also gives the clinic time to orient to an additional provider, nurture them into the team culture, and develop relationships with the care team.

Cultivate spaces for shared learning, problem-solving, and peer connection.

Orientation is delivered primarily through group learning sessions with cohorts of new physicians, thus developing spaces for networking, fostering psychological safety, encouraging personal and professional rapport, emphasizing interactive learning, and reinforcing scheduling blocks at the departmental level. New hires also participate in peer shadowing to develop clinical competencies and are assigned a workplace buddy to foster a sense of belonging and create opportunities for additional knowledge sharing and cross-training.

Strengthen physician knowledge base, confidence, and comfort in the workplace before beginning direct patient care.

Without fluency in the workflows, culture, and operations of a practice, the urgency to have physicians begin clinical care can result in frustration for the physician, patients, and clinical and administrative staff. Therefore, we complete essential training prior to seeing any patients. This includes clinical workflows, referral processes, use of alternate modalities of care (eg, telehealth, eConsults), billing protocols, population health training, patient resources, office resources, and other essential daily processes and tools. This creates efficiency in administrative management, increased productivity, and better understanding of resources available for patients’ medical, social, and behavioral needs when patient care begins.

Embrace standardization, transparency, and accountability in as many processes as possible.

Standardized knowledge-sharing and checklists are mandated at every step of the orientation process, requiring sign off from the physician lead, practice manager, and new physicians upon completion. This offers all parties the opportunity to play a role in the delivery of and accountability for skills transfer and empowers new hires to press pause if they feel unsure about any domain in the training. It is also essential in guaranteeing that all physicians—regardless of which ambulatory location they practice in—receive consistent information and expectations. A sample checklist can be found in Appendix 2.

Commit to collecting and acting on feedback for continued program improvement and individual support.

As physicians complete the program, it is necessary to create structures to measure and enhance its impact, as well as evaluate how physicians are faring following the program. Each physician completes surveys at the end of the orientation program, attends a 90-day post-program check-in with the department chair, and receives follow-up trainings on advanced topics as they become more deeply embedded in the organization.

Lessons Learned

Feedback from surveys and 90-day check-ins with leadership and physicians reflect a high degree of clarity on job roles and duties, a sense of team camaraderie, easier system navigation, and a strong sense of support. We do recognize that sustaining change takes time and our study is limited by data demonstrating the impact of these efforts. We look forward to sharing more robust data from surveys and qualitative interviews with physicians, clinical leadership, and staff in the future. Our team will conduct interviews at 90-day and 180-day checkpoints with new physicians who have gone through this program, followed by a check-in after 1 year. Additionally, new physicians as well as key stakeholders, such as physician leads, practice managers, and members of the recruitment team, have started to participate in short surveys. These are designed to better understand their experiences, what worked well, what can be improved, and the overall satisfaction of the physician and other members of the extended care team.

What follows are some comments made by the initial group of physicians that went through this program and participated in follow-up interviews:

“I really feel like part of a bigger team.”

“I knew exactly what do to when I walked into the exam room on clinic Day 1.”

“It was great to make deep connections during the early process of joining.”

“Having a buddy to direct questions and ideas to is amazing and empowering.”

“Even though the orientation was long, I felt that I learned so much that I would not have otherwise.”

“Thank you for not letting me crash and burn!”

“Great culture! I love understanding our values of health equity, diversity, and inclusion.”

In the months since our endeavor began, we have learned just how essential it is to fully and effectively integrate new hires into the organization for their own satisfaction and success—and ours. Indeed, we cannot expect to achieve the Quadruple Aim without investing in the kind of transparent and intentional orientation process that defines expectations, aligns cultural values, mitigates costly and stressful operational misunderstandings, and communicates to physicians that, not only do they belong, but their sense of belonging is our priority. While we have yet to understand the impact of this program on the fourth aim of the Quadruple Aim, we are hopeful that the benefits will be far-reaching.

It is our ultimate hope that programs like this: (1) give physicians the confidence needed to create impactful patient-centered experiences; (2) enable physicians to become more cost-effective and efficient in care delivery; (3) allow physicians to understand the populations they are serving and access tools available to mitigate health disparities and other barriers; and (4) improve the collective experience of every member of the care team, practice leadership, and clinician-patient partnership.

Corresponding author: J. Nwando Olayiwola, MD, MPH, FAAFP, The Ohio State University College of Medicine, Department of Family and Community Medicine, 2231 N High St, Ste 250, Columbus, OH 43210; [email protected].

Financial disclosures: None.

Keywords: physician onboarding; Quadruple Aim; leadership; clinician satisfaction; care team satisfaction.

From Adelante Healthcare, Mesa, AZ (Dr. Chin), University Hospitals of Cleveland, Cleveland, OH (Drs. Delozier, Bascug, Levine, Bejanishvili, and Wynbrandt and Janet C. Peachey, Rachel M. Cerminara, and Sharon M. Darkovich), and Houston Methodist Hospitals, Houston, TX (Dr. Bhakta).

Abstract

Background: Resident physicians play an active role in the reporting of errors that occur in patient care. Previous studies indicate that residents significantly underreport errors in patient care.

Methods: Fifty-four of 80 eligible residents enrolled at University Hospitals–Regional Hospitals (UH-RH) during the 2018-2019 academic year completed a survey assessing their knowledge and experience in completing Patient Advocacy and Shared Stories (PASS) reports, which serve as incident reports in the UH health system in reporting errors in patient care. A series of interventions aimed at educating residents about the PASS report system were then conducted. The 54 residents who completed the first survey received it again 4 months later.

Results: Residents demonstrated greater understanding of when filing PASS reports was appropriate after the intervention, as significantly more residents reported having been involved in a situation where they should have filed a PASS report but did not (P = 0.036).

Conclusion: In this study, residents often did not report errors in patient care because they simply did not know the process for doing so. In addition, many residents often felt that the reporting of patient errors could be used as a form of retaliation.

Keywords: resident physicians; quality improvement; high-value care; medical errors; patient safety.

Resident physicians play a critical role in patient care. Residents undergo extensive supervised training in order to one day be able to practice medicine in an unsupervised setting, with the goal of providing the highest quality of care possible. One study reported that primary care provided by residents in a training program is of similar or higher quality than that provided by attending physicians.¹

Besides providing high-quality care, it is important that residents play an active role in the reporting of errors that occur regarding patient care as well as in identifying events that may compromise patient safety and quality.² In fact, increased reporting of patient errors has been shown to decrease liability-related costs for hospitals.³ Unfortunately, physicians, and residents in particular, have historically been poor reporters of errors in patient care.⁴ This is especially true when comparing physicians to other health professionals, such as nurses, in error reporting.⁵

Several studies have examined the involvement of residents in reporting errors in patient care. One recent study showed that a graduate medical education financial incentive program significantly increased the number of patient safety events reported by residents and fellows.⁶ This study, along with several others, supports the concept of using incentives to help improve the reporting of errors in patient care for physicians in training.^7-10 Another study used Quality Improvement Knowledge Assessment Tool (QIKAT) scores to assess quality improvement (QI) knowledge. The study demonstrated that self-assessment scores of QI skills using QIKAT scores improved following a targeted intervention.¹¹ Because further information on the involvement and attitudes of residents in reporting errors in patient care is needed, University Hospitals of Cleveland (UH) designed and implemented a QI study during the 2018-2019 academic year. This prospective study used anonymous surveys to objectively examine the involvement and attitudes of residents in reporting errors in patient care.

Methods

The UH health system uses Patient Advocacy and Shared Stories (PASS) reports as incident reports to not only disclose errors in patient care but also to identify any events that may compromise patient safety and quality. Based on preliminary review, nurses, ancillary staff, and administrators file the majority of PASS reports.

The study group consisted of residents at University Hospitals–Regional Hospitals (UH-RH), which is comprised of 2 hospitals: University Hospitals–Richmond Medical Center (UH-RMC) and University Hospitals –Bedford Medical Center (UH-BMC). UH-RMC and UH-BMC are 2 medium-sized university-affiliated community hospitals located in the Cleveland metropolitan area in Northeast Ohio. Both serve as clinical training sites for Case Western Reserve University School of Medicine and Lake Erie College of Osteopathic Medicine, the latter of which helped fund this study. The study was submitted to the Institutional Review Board (IRB) of University Hospitals of Cleveland and granted “not human subjects research” status as a QI study.

Surveys

UH-RH offers residency programs in dermatology, emergency medicine, family medicine, internal medicine, orthopedic surgery, and physical medicine and rehabilitation, along with a 1-year transitional/preliminary year. A total of 80 residents enrolled at UH-RH during the 2018-2019 academic year. All 80 residents at UH-RH received an email in December 2018 asking them to complete an anonymous survey regarding the PASS report system. The survey was administered using the REDCap software system and consisted of 15 multiple-choice questions. As an incentive for completing the survey, residents were offered a $10 Amazon gift card. The gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey. At the end of the week, 54 of 80 residents completed the first survey.

Following the first survey, efforts were undertaken by the study authors, in conjunction with the quality improvement department at UH-RH, to educate residents about the PASS report system. These interventions included giving a lecture on the PASS report system during resident didactic sessions, sending an email to all residents about the PASS report system, and providing residents an opportunity to complete an optional online training course regarding the PASS report system. As an incentive for completing the online training course, residents were offered a $10 Amazon gift card. As before, the gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine.

A second survey was administered in April 2019, 4 months after the first survey. To determine whether the intervention made an impact on the involvement and attitudes of residents in the reporting errors in patient care, only residents who completed the first survey were sent the second survey. The second survey consisted of the same questions as the first survey and was also administered using the REDCap software system. As an incentive for completing the survey, residents were offered another $10 Amazon gift card, again were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey.

Analysis

Chi-square analyses were utilized to examine differences between preintervention and postintervention responses across categories. All analyses were conducted using R statistical software, version 3.6.1 (R Foundation for Statistical Computing).

Results

A total of 54 of 80 eligible residents responded to the first survey (Table). Twenty-nine of 54 eligible residents responded to the second survey. Postintervention, significantly more residents indicated being involved in a situation where they should have filed a PASS report but did not (58.6% vs 53.7%; P = 0.036). Improvement was seen in PASS knowledge postintervention, where fewer residents reported not knowing how to file a PASS report (31.5% vs 55.2%; P = 0.059). No other improvements were significant, nor were there significant differences in responses between any other categories pre- and postintervention.

Discussion

Errors in patient care are a common occurrence in the hospital setting. Reporting errors when they happen is important for hospitals to gain data and better care for patients, but studies show that patient errors are usually underreported. This is concerning, as data on errors and other aspects of patient care are needed to inform quality improvement programs.

This study measured residents’ attitudes and knowledge regarding the filing of a PASS report. It also aimed to increase both the frequency of and knowledge about filing a PASS report with interventions. The results from each survey indicated a statistically significant increase in knowledge of when to file a PASS report. In the first survey, 53.7% of residents responded they they were involved in an instance where they should have filed a PASS report but did not. In the second survey, 58.5% of residents reported being involved in an instance where they should have filed a PASS report but did not. This difference was statistically significant (P = 0.036), sugesting that the intervention was successful at increasing residents’ knowledge regarding PASS reports and the appropriate times to file a PASS report.

The survey results also showed a trend toward increasing aggregate knowledge level of how to file PASS reports on the first survey and second surveys (from 31.5% vs 55.2%. This demonstrates an increase in knowledge of how to file a PASS report among residents at our hospital after the intervention. It should be noted that the intervention that was performed in this study was simple, easy to perform, and can be completed at any hospital system that uses a similar system for reporting patient errors.

Another important trend indicating the effectiveness of the intervention was a 15% increase in knowledge of what the PASS report acronym stands for, along with a 13.1% aggregate increase in the number of residents who filed a PASS report. This indicated that residents may have wanted to file a PASS report previously but simply did not know how to until the intervention. In addition, there was also a decrease in the aggregate percentages of residents who had never filed a PASS report and an increase in how many PASS reports were filed.

While PASS reports are a great way for hospitals to gain data and insight into problems at their sites, there was also a negative view of PASS reports. For example, a large percentage of residents indicated that filing a PASS report would not make any difference and that PASS reports are often used as a form of retaliation, either against themselves as the submitter or the person(s) mentioned in the PASS report. More specifically, more than 50% of residents felt that PASS reports were sometimes or often used as a form of retaliation against others. While many residents correctly identified in the survey that PASS reports are not equivalent to a “write-up,” it is concerning that they still feel there is a strong potential for retaliation when filing a PASS report. This finding is unfortunate but matches the results of a multicenter study that found that 44.6% of residents felt uncomfortable reporting patient errors, possibly secondary to fear of retaliation, along with issues with the reporting system.¹²

It is interesting to note that a minority of residents indicated that they feel that PASS reports are filed as often as they should be (25.9% on first survey and 24.1% on second survey). This is concerning, as the data gathered through PASS reports is used to improve patient care. However, the percentage reported in our study, although low, is higher than that reported in a similar study involving patients with Medicare insurance, which showed that only 14% of patient safety events were reported.¹³ These results demonstrate that further interventions are necessary in order to ensure that a PASS report is filed each time a patient safety event occurs.

Another finding of note is that the majority of residents also feel that the process of filing a PASS report is too time consuming. The majority of residents who have completed a PASS report stated that it took them between 10 and 20 minutes to complete a PASS report, but those same individuals also feel that it should take < 10 minutes to complete a PASS report. This is an important issue for hospital systems to address. Reducing the time it takes to file a PASS report may facilitate an increase in the amount of PASS reports filed.

We administered our surveys using email outreach to residents asking them to complete an anonymous online survey regarding the PASS report system using the REDCap software system. Researchers have various ways of administering surveys, ranging from paper surveys, emails, and even mobile apps. One study showed that online surveys tend to have higher response rates compared to non-online surveys, such as paper surveys and telephone surveys, which is likely due to the ease of use of online surveys.¹⁴ At the same time, unsolicited email surveys have been shown to have a negative influence on response rates. Mobile apps are a new way of administering surveys. However, research has not found any significant difference in the time required to complete the survey using mobile apps compared to other forms of administering surveys. In addition, surveys using mobile apps did not have increased response rates compared to other forms of administering surveys.¹⁵

To increase the response rate of our surveys, we offered gift cards to the study population for completing the survey. Studies have shown that surveys that offer incentives tend to have higher response rates than surveys that do not.¹⁶ Also, in addition to serving as a method for gathering data from our study population, we used our surveys as an intervention to increase awareness of PASS reporting, as reported in other studies. For example, another study used the HABITS questionnaire to not only gather information about children’s diet, but also to promote behavioral change towards healthy eating habits.¹⁷

This study had several limitations. First, the study was conducted using an anonymous online survey, which means we could not clarify questions that residents found confusing or needed further explanation. For example, 17 residents indicated in the first survey that they knew how to PASS report, but 19 residents indicated in the same survey that they have filed a PASS report in the past.

A second limitation of the study was that fewer residents completed the second survey (29 of 54 eligible residents) compared to the first survey (54 of 80 eligible residents). This may have impacted the results of the analysis, as certain findings were not statistically significant, despite trends in the data.

A third limitation of the study is that not all of the residents that completed the first and second surveys completed the entire intervention. For example, some residents did not attend the didactic lecture discussing PASS reports, and as such may not have received the appropriate training prior to completing the second survey.

The findings from this study can be used by the residency programs at UH-RH and by residency programs across the country to improve the involvement and attitudes of residents in reporting errors in patient care. Hospital staff need to be encouraged and educated on how to better report patient errors and the importance of reporting these errors. It would benefit hospital systems to provide continued and targeted training to familiarize physicians with the process of reporting patient errors, and take steps to reduce the time it takes to report patient errors. By increasing the reporting of errors, hospitals will be able to improve patient care through initiatives aimed at preventing errors.

Conclusion

Residents play an important role in providing high-quality care for patients. Part of providing high-quality care is the reporting of errors in patient care when they occur. Physicians, and in particular, residents, have historically underreported errors in patient care. Part of this underreporting results from residents not knowing or understanding the process of filing a report and feeling that the reports could be used as a form of retaliation. For hospital systems to continue to improve patient care, it is important for residents to not only know how to report errors in patient care but to feel comfortable doing so.

Corresponding author: Andrew J. Chin, DO, MS, MPH, Department of Internal Medicine, Adelante Healthcare, 1705 W Main St, Mesa, AZ 85201; [email protected].

Financial disclosures: None.

Funding: This study was funded by a research grant provided by Lake Eric College of Osteopathic Medicine to Andrew J. Chin and Anish Bhakta.

From Adelante Healthcare, Mesa, AZ (Dr. Chin), University Hospitals of Cleveland, Cleveland, OH (Drs. Delozier, Bascug, Levine, Bejanishvili, and Wynbrandt and Janet C. Peachey, Rachel M. Cerminara, and Sharon M. Darkovich), and Houston Methodist Hospitals, Houston, TX (Dr. Bhakta).

Abstract

Background: Resident physicians play an active role in the reporting of errors that occur in patient care. Previous studies indicate that residents significantly underreport errors in patient care.

Methods: Fifty-four of 80 eligible residents enrolled at University Hospitals–Regional Hospitals (UH-RH) during the 2018-2019 academic year completed a survey assessing their knowledge and experience in completing Patient Advocacy and Shared Stories (PASS) reports, which serve as incident reports in the UH health system in reporting errors in patient care. A series of interventions aimed at educating residents about the PASS report system were then conducted. The 54 residents who completed the first survey received it again 4 months later.

Results: Residents demonstrated greater understanding of when filing PASS reports was appropriate after the intervention, as significantly more residents reported having been involved in a situation where they should have filed a PASS report but did not (P = 0.036).

Conclusion: In this study, residents often did not report errors in patient care because they simply did not know the process for doing so. In addition, many residents often felt that the reporting of patient errors could be used as a form of retaliation.

Keywords: resident physicians; quality improvement; high-value care; medical errors; patient safety.

Resident physicians play a critical role in patient care. Residents undergo extensive supervised training in order to one day be able to practice medicine in an unsupervised setting, with the goal of providing the highest quality of care possible. One study reported that primary care provided by residents in a training program is of similar or higher quality than that provided by attending physicians.¹

Besides providing high-quality care, it is important that residents play an active role in the reporting of errors that occur regarding patient care as well as in identifying events that may compromise patient safety and quality.² In fact, increased reporting of patient errors has been shown to decrease liability-related costs for hospitals.³ Unfortunately, physicians, and residents in particular, have historically been poor reporters of errors in patient care.⁴ This is especially true when comparing physicians to other health professionals, such as nurses, in error reporting.⁵

Several studies have examined the involvement of residents in reporting errors in patient care. One recent study showed that a graduate medical education financial incentive program significantly increased the number of patient safety events reported by residents and fellows.⁶ This study, along with several others, supports the concept of using incentives to help improve the reporting of errors in patient care for physicians in training.^7-10 Another study used Quality Improvement Knowledge Assessment Tool (QIKAT) scores to assess quality improvement (QI) knowledge. The study demonstrated that self-assessment scores of QI skills using QIKAT scores improved following a targeted intervention.¹¹ Because further information on the involvement and attitudes of residents in reporting errors in patient care is needed, University Hospitals of Cleveland (UH) designed and implemented a QI study during the 2018-2019 academic year. This prospective study used anonymous surveys to objectively examine the involvement and attitudes of residents in reporting errors in patient care.

Methods

The UH health system uses Patient Advocacy and Shared Stories (PASS) reports as incident reports to not only disclose errors in patient care but also to identify any events that may compromise patient safety and quality. Based on preliminary review, nurses, ancillary staff, and administrators file the majority of PASS reports.

The study group consisted of residents at University Hospitals–Regional Hospitals (UH-RH), which is comprised of 2 hospitals: University Hospitals–Richmond Medical Center (UH-RMC) and University Hospitals –Bedford Medical Center (UH-BMC). UH-RMC and UH-BMC are 2 medium-sized university-affiliated community hospitals located in the Cleveland metropolitan area in Northeast Ohio. Both serve as clinical training sites for Case Western Reserve University School of Medicine and Lake Erie College of Osteopathic Medicine, the latter of which helped fund this study. The study was submitted to the Institutional Review Board (IRB) of University Hospitals of Cleveland and granted “not human subjects research” status as a QI study.

Surveys

UH-RH offers residency programs in dermatology, emergency medicine, family medicine, internal medicine, orthopedic surgery, and physical medicine and rehabilitation, along with a 1-year transitional/preliminary year. A total of 80 residents enrolled at UH-RH during the 2018-2019 academic year. All 80 residents at UH-RH received an email in December 2018 asking them to complete an anonymous survey regarding the PASS report system. The survey was administered using the REDCap software system and consisted of 15 multiple-choice questions. As an incentive for completing the survey, residents were offered a $10 Amazon gift card. The gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey. At the end of the week, 54 of 80 residents completed the first survey.

Following the first survey, efforts were undertaken by the study authors, in conjunction with the quality improvement department at UH-RH, to educate residents about the PASS report system. These interventions included giving a lecture on the PASS report system during resident didactic sessions, sending an email to all residents about the PASS report system, and providing residents an opportunity to complete an optional online training course regarding the PASS report system. As an incentive for completing the online training course, residents were offered a $10 Amazon gift card. As before, the gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine.

A second survey was administered in April 2019, 4 months after the first survey. To determine whether the intervention made an impact on the involvement and attitudes of residents in the reporting errors in patient care, only residents who completed the first survey were sent the second survey. The second survey consisted of the same questions as the first survey and was also administered using the REDCap software system. As an incentive for completing the survey, residents were offered another $10 Amazon gift card, again were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey.

Analysis

Chi-square analyses were utilized to examine differences between preintervention and postintervention responses across categories. All analyses were conducted using R statistical software, version 3.6.1 (R Foundation for Statistical Computing).

Results

A total of 54 of 80 eligible residents responded to the first survey (Table). Twenty-nine of 54 eligible residents responded to the second survey. Postintervention, significantly more residents indicated being involved in a situation where they should have filed a PASS report but did not (58.6% vs 53.7%; P = 0.036). Improvement was seen in PASS knowledge postintervention, where fewer residents reported not knowing how to file a PASS report (31.5% vs 55.2%; P = 0.059). No other improvements were significant, nor were there significant differences in responses between any other categories pre- and postintervention.

Discussion

Errors in patient care are a common occurrence in the hospital setting. Reporting errors when they happen is important for hospitals to gain data and better care for patients, but studies show that patient errors are usually underreported. This is concerning, as data on errors and other aspects of patient care are needed to inform quality improvement programs.

This study measured residents’ attitudes and knowledge regarding the filing of a PASS report. It also aimed to increase both the frequency of and knowledge about filing a PASS report with interventions. The results from each survey indicated a statistically significant increase in knowledge of when to file a PASS report. In the first survey, 53.7% of residents responded they they were involved in an instance where they should have filed a PASS report but did not. In the second survey, 58.5% of residents reported being involved in an instance where they should have filed a PASS report but did not. This difference was statistically significant (P = 0.036), sugesting that the intervention was successful at increasing residents’ knowledge regarding PASS reports and the appropriate times to file a PASS report.

The survey results also showed a trend toward increasing aggregate knowledge level of how to file PASS reports on the first survey and second surveys (from 31.5% vs 55.2%. This demonstrates an increase in knowledge of how to file a PASS report among residents at our hospital after the intervention. It should be noted that the intervention that was performed in this study was simple, easy to perform, and can be completed at any hospital system that uses a similar system for reporting patient errors.

Another important trend indicating the effectiveness of the intervention was a 15% increase in knowledge of what the PASS report acronym stands for, along with a 13.1% aggregate increase in the number of residents who filed a PASS report. This indicated that residents may have wanted to file a PASS report previously but simply did not know how to until the intervention. In addition, there was also a decrease in the aggregate percentages of residents who had never filed a PASS report and an increase in how many PASS reports were filed.

While PASS reports are a great way for hospitals to gain data and insight into problems at their sites, there was also a negative view of PASS reports. For example, a large percentage of residents indicated that filing a PASS report would not make any difference and that PASS reports are often used as a form of retaliation, either against themselves as the submitter or the person(s) mentioned in the PASS report. More specifically, more than 50% of residents felt that PASS reports were sometimes or often used as a form of retaliation against others. While many residents correctly identified in the survey that PASS reports are not equivalent to a “write-up,” it is concerning that they still feel there is a strong potential for retaliation when filing a PASS report. This finding is unfortunate but matches the results of a multicenter study that found that 44.6% of residents felt uncomfortable reporting patient errors, possibly secondary to fear of retaliation, along with issues with the reporting system.¹²

It is interesting to note that a minority of residents indicated that they feel that PASS reports are filed as often as they should be (25.9% on first survey and 24.1% on second survey). This is concerning, as the data gathered through PASS reports is used to improve patient care. However, the percentage reported in our study, although low, is higher than that reported in a similar study involving patients with Medicare insurance, which showed that only 14% of patient safety events were reported.¹³ These results demonstrate that further interventions are necessary in order to ensure that a PASS report is filed each time a patient safety event occurs.

Another finding of note is that the majority of residents also feel that the process of filing a PASS report is too time consuming. The majority of residents who have completed a PASS report stated that it took them between 10 and 20 minutes to complete a PASS report, but those same individuals also feel that it should take < 10 minutes to complete a PASS report. This is an important issue for hospital systems to address. Reducing the time it takes to file a PASS report may facilitate an increase in the amount of PASS reports filed.

We administered our surveys using email outreach to residents asking them to complete an anonymous online survey regarding the PASS report system using the REDCap software system. Researchers have various ways of administering surveys, ranging from paper surveys, emails, and even mobile apps. One study showed that online surveys tend to have higher response rates compared to non-online surveys, such as paper surveys and telephone surveys, which is likely due to the ease of use of online surveys.¹⁴ At the same time, unsolicited email surveys have been shown to have a negative influence on response rates. Mobile apps are a new way of administering surveys. However, research has not found any significant difference in the time required to complete the survey using mobile apps compared to other forms of administering surveys. In addition, surveys using mobile apps did not have increased response rates compared to other forms of administering surveys.¹⁵

To increase the response rate of our surveys, we offered gift cards to the study population for completing the survey. Studies have shown that surveys that offer incentives tend to have higher response rates than surveys that do not.¹⁶ Also, in addition to serving as a method for gathering data from our study population, we used our surveys as an intervention to increase awareness of PASS reporting, as reported in other studies. For example, another study used the HABITS questionnaire to not only gather information about children’s diet, but also to promote behavioral change towards healthy eating habits.¹⁷

This study had several limitations. First, the study was conducted using an anonymous online survey, which means we could not clarify questions that residents found confusing or needed further explanation. For example, 17 residents indicated in the first survey that they knew how to PASS report, but 19 residents indicated in the same survey that they have filed a PASS report in the past.

A second limitation of the study was that fewer residents completed the second survey (29 of 54 eligible residents) compared to the first survey (54 of 80 eligible residents). This may have impacted the results of the analysis, as certain findings were not statistically significant, despite trends in the data.

A third limitation of the study is that not all of the residents that completed the first and second surveys completed the entire intervention. For example, some residents did not attend the didactic lecture discussing PASS reports, and as such may not have received the appropriate training prior to completing the second survey.

The findings from this study can be used by the residency programs at UH-RH and by residency programs across the country to improve the involvement and attitudes of residents in reporting errors in patient care. Hospital staff need to be encouraged and educated on how to better report patient errors and the importance of reporting these errors. It would benefit hospital systems to provide continued and targeted training to familiarize physicians with the process of reporting patient errors, and take steps to reduce the time it takes to report patient errors. By increasing the reporting of errors, hospitals will be able to improve patient care through initiatives aimed at preventing errors.

Conclusion

Residents play an important role in providing high-quality care for patients. Part of providing high-quality care is the reporting of errors in patient care when they occur. Physicians, and in particular, residents, have historically underreported errors in patient care. Part of this underreporting results from residents not knowing or understanding the process of filing a report and feeling that the reports could be used as a form of retaliation. For hospital systems to continue to improve patient care, it is important for residents to not only know how to report errors in patient care but to feel comfortable doing so.

Corresponding author: Andrew J. Chin, DO, MS, MPH, Department of Internal Medicine, Adelante Healthcare, 1705 W Main St, Mesa, AZ 85201; [email protected].

Financial disclosures: None.

Funding: This study was funded by a research grant provided by Lake Eric College of Osteopathic Medicine to Andrew J. Chin and Anish Bhakta.

From Adelante Healthcare, Mesa, AZ (Dr. Chin), University Hospitals of Cleveland, Cleveland, OH (Drs. Delozier, Bascug, Levine, Bejanishvili, and Wynbrandt and Janet C. Peachey, Rachel M. Cerminara, and Sharon M. Darkovich), and Houston Methodist Hospitals, Houston, TX (Dr. Bhakta).

Abstract

Background: Resident physicians play an active role in the reporting of errors that occur in patient care. Previous studies indicate that residents significantly underreport errors in patient care.

Methods: Fifty-four of 80 eligible residents enrolled at University Hospitals–Regional Hospitals (UH-RH) during the 2018-2019 academic year completed a survey assessing their knowledge and experience in completing Patient Advocacy and Shared Stories (PASS) reports, which serve as incident reports in the UH health system in reporting errors in patient care. A series of interventions aimed at educating residents about the PASS report system were then conducted. The 54 residents who completed the first survey received it again 4 months later.

Results: Residents demonstrated greater understanding of when filing PASS reports was appropriate after the intervention, as significantly more residents reported having been involved in a situation where they should have filed a PASS report but did not (P = 0.036).

Conclusion: In this study, residents often did not report errors in patient care because they simply did not know the process for doing so. In addition, many residents often felt that the reporting of patient errors could be used as a form of retaliation.

Keywords: resident physicians; quality improvement; high-value care; medical errors; patient safety.

Resident physicians play a critical role in patient care. Residents undergo extensive supervised training in order to one day be able to practice medicine in an unsupervised setting, with the goal of providing the highest quality of care possible. One study reported that primary care provided by residents in a training program is of similar or higher quality than that provided by attending physicians.¹

Besides providing high-quality care, it is important that residents play an active role in the reporting of errors that occur regarding patient care as well as in identifying events that may compromise patient safety and quality.² In fact, increased reporting of patient errors has been shown to decrease liability-related costs for hospitals.³ Unfortunately, physicians, and residents in particular, have historically been poor reporters of errors in patient care.⁴ This is especially true when comparing physicians to other health professionals, such as nurses, in error reporting.⁵

Several studies have examined the involvement of residents in reporting errors in patient care. One recent study showed that a graduate medical education financial incentive program significantly increased the number of patient safety events reported by residents and fellows.⁶ This study, along with several others, supports the concept of using incentives to help improve the reporting of errors in patient care for physicians in training.^7-10 Another study used Quality Improvement Knowledge Assessment Tool (QIKAT) scores to assess quality improvement (QI) knowledge. The study demonstrated that self-assessment scores of QI skills using QIKAT scores improved following a targeted intervention.¹¹ Because further information on the involvement and attitudes of residents in reporting errors in patient care is needed, University Hospitals of Cleveland (UH) designed and implemented a QI study during the 2018-2019 academic year. This prospective study used anonymous surveys to objectively examine the involvement and attitudes of residents in reporting errors in patient care.

Methods

The UH health system uses Patient Advocacy and Shared Stories (PASS) reports as incident reports to not only disclose errors in patient care but also to identify any events that may compromise patient safety and quality. Based on preliminary review, nurses, ancillary staff, and administrators file the majority of PASS reports.

The study group consisted of residents at University Hospitals–Regional Hospitals (UH-RH), which is comprised of 2 hospitals: University Hospitals–Richmond Medical Center (UH-RMC) and University Hospitals –Bedford Medical Center (UH-BMC). UH-RMC and UH-BMC are 2 medium-sized university-affiliated community hospitals located in the Cleveland metropolitan area in Northeast Ohio. Both serve as clinical training sites for Case Western Reserve University School of Medicine and Lake Erie College of Osteopathic Medicine, the latter of which helped fund this study. The study was submitted to the Institutional Review Board (IRB) of University Hospitals of Cleveland and granted “not human subjects research” status as a QI study.

Surveys

UH-RH offers residency programs in dermatology, emergency medicine, family medicine, internal medicine, orthopedic surgery, and physical medicine and rehabilitation, along with a 1-year transitional/preliminary year. A total of 80 residents enrolled at UH-RH during the 2018-2019 academic year. All 80 residents at UH-RH received an email in December 2018 asking them to complete an anonymous survey regarding the PASS report system. The survey was administered using the REDCap software system and consisted of 15 multiple-choice questions. As an incentive for completing the survey, residents were offered a $10 Amazon gift card. The gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey. At the end of the week, 54 of 80 residents completed the first survey.

Following the first survey, efforts were undertaken by the study authors, in conjunction with the quality improvement department at UH-RH, to educate residents about the PASS report system. These interventions included giving a lecture on the PASS report system during resident didactic sessions, sending an email to all residents about the PASS report system, and providing residents an opportunity to complete an optional online training course regarding the PASS report system. As an incentive for completing the online training course, residents were offered a $10 Amazon gift card. As before, the gift cards were funded through a research grant from Lake Erie College of Osteopathic Medicine.

A second survey was administered in April 2019, 4 months after the first survey. To determine whether the intervention made an impact on the involvement and attitudes of residents in the reporting errors in patient care, only residents who completed the first survey were sent the second survey. The second survey consisted of the same questions as the first survey and was also administered using the REDCap software system. As an incentive for completing the survey, residents were offered another $10 Amazon gift card, again were funded through a research grant from Lake Erie College of Osteopathic Medicine. Residents were given 1 week to complete the survey.

Analysis

Chi-square analyses were utilized to examine differences between preintervention and postintervention responses across categories. All analyses were conducted using R statistical software, version 3.6.1 (R Foundation for Statistical Computing).

Results

A total of 54 of 80 eligible residents responded to the first survey (Table). Twenty-nine of 54 eligible residents responded to the second survey. Postintervention, significantly more residents indicated being involved in a situation where they should have filed a PASS report but did not (58.6% vs 53.7%; P = 0.036). Improvement was seen in PASS knowledge postintervention, where fewer residents reported not knowing how to file a PASS report (31.5% vs 55.2%; P = 0.059). No other improvements were significant, nor were there significant differences in responses between any other categories pre- and postintervention.

Discussion

Errors in patient care are a common occurrence in the hospital setting. Reporting errors when they happen is important for hospitals to gain data and better care for patients, but studies show that patient errors are usually underreported. This is concerning, as data on errors and other aspects of patient care are needed to inform quality improvement programs.

This study measured residents’ attitudes and knowledge regarding the filing of a PASS report. It also aimed to increase both the frequency of and knowledge about filing a PASS report with interventions. The results from each survey indicated a statistically significant increase in knowledge of when to file a PASS report. In the first survey, 53.7% of residents responded they they were involved in an instance where they should have filed a PASS report but did not. In the second survey, 58.5% of residents reported being involved in an instance where they should have filed a PASS report but did not. This difference was statistically significant (P = 0.036), sugesting that the intervention was successful at increasing residents’ knowledge regarding PASS reports and the appropriate times to file a PASS report.

The survey results also showed a trend toward increasing aggregate knowledge level of how to file PASS reports on the first survey and second surveys (from 31.5% vs 55.2%. This demonstrates an increase in knowledge of how to file a PASS report among residents at our hospital after the intervention. It should be noted that the intervention that was performed in this study was simple, easy to perform, and can be completed at any hospital system that uses a similar system for reporting patient errors.

Another important trend indicating the effectiveness of the intervention was a 15% increase in knowledge of what the PASS report acronym stands for, along with a 13.1% aggregate increase in the number of residents who filed a PASS report. This indicated that residents may have wanted to file a PASS report previously but simply did not know how to until the intervention. In addition, there was also a decrease in the aggregate percentages of residents who had never filed a PASS report and an increase in how many PASS reports were filed.

While PASS reports are a great way for hospitals to gain data and insight into problems at their sites, there was also a negative view of PASS reports. For example, a large percentage of residents indicated that filing a PASS report would not make any difference and that PASS reports are often used as a form of retaliation, either against themselves as the submitter or the person(s) mentioned in the PASS report. More specifically, more than 50% of residents felt that PASS reports were sometimes or often used as a form of retaliation against others. While many residents correctly identified in the survey that PASS reports are not equivalent to a “write-up,” it is concerning that they still feel there is a strong potential for retaliation when filing a PASS report. This finding is unfortunate but matches the results of a multicenter study that found that 44.6% of residents felt uncomfortable reporting patient errors, possibly secondary to fear of retaliation, along with issues with the reporting system.¹²

It is interesting to note that a minority of residents indicated that they feel that PASS reports are filed as often as they should be (25.9% on first survey and 24.1% on second survey). This is concerning, as the data gathered through PASS reports is used to improve patient care. However, the percentage reported in our study, although low, is higher than that reported in a similar study involving patients with Medicare insurance, which showed that only 14% of patient safety events were reported.¹³ These results demonstrate that further interventions are necessary in order to ensure that a PASS report is filed each time a patient safety event occurs.

Another finding of note is that the majority of residents also feel that the process of filing a PASS report is too time consuming. The majority of residents who have completed a PASS report stated that it took them between 10 and 20 minutes to complete a PASS report, but those same individuals also feel that it should take < 10 minutes to complete a PASS report. This is an important issue for hospital systems to address. Reducing the time it takes to file a PASS report may facilitate an increase in the amount of PASS reports filed.

We administered our surveys using email outreach to residents asking them to complete an anonymous online survey regarding the PASS report system using the REDCap software system. Researchers have various ways of administering surveys, ranging from paper surveys, emails, and even mobile apps. One study showed that online surveys tend to have higher response rates compared to non-online surveys, such as paper surveys and telephone surveys, which is likely due to the ease of use of online surveys.¹⁴ At the same time, unsolicited email surveys have been shown to have a negative influence on response rates. Mobile apps are a new way of administering surveys. However, research has not found any significant difference in the time required to complete the survey using mobile apps compared to other forms of administering surveys. In addition, surveys using mobile apps did not have increased response rates compared to other forms of administering surveys.¹⁵

To increase the response rate of our surveys, we offered gift cards to the study population for completing the survey. Studies have shown that surveys that offer incentives tend to have higher response rates than surveys that do not.¹⁶ Also, in addition to serving as a method for gathering data from our study population, we used our surveys as an intervention to increase awareness of PASS reporting, as reported in other studies. For example, another study used the HABITS questionnaire to not only gather information about children’s diet, but also to promote behavioral change towards healthy eating habits.¹⁷

This study had several limitations. First, the study was conducted using an anonymous online survey, which means we could not clarify questions that residents found confusing or needed further explanation. For example, 17 residents indicated in the first survey that they knew how to PASS report, but 19 residents indicated in the same survey that they have filed a PASS report in the past.

A second limitation of the study was that fewer residents completed the second survey (29 of 54 eligible residents) compared to the first survey (54 of 80 eligible residents). This may have impacted the results of the analysis, as certain findings were not statistically significant, despite trends in the data.

A third limitation of the study is that not all of the residents that completed the first and second surveys completed the entire intervention. For example, some residents did not attend the didactic lecture discussing PASS reports, and as such may not have received the appropriate training prior to completing the second survey.

The findings from this study can be used by the residency programs at UH-RH and by residency programs across the country to improve the involvement and attitudes of residents in reporting errors in patient care. Hospital staff need to be encouraged and educated on how to better report patient errors and the importance of reporting these errors. It would benefit hospital systems to provide continued and targeted training to familiarize physicians with the process of reporting patient errors, and take steps to reduce the time it takes to report patient errors. By increasing the reporting of errors, hospitals will be able to improve patient care through initiatives aimed at preventing errors.

Conclusion

Residents play an important role in providing high-quality care for patients. Part of providing high-quality care is the reporting of errors in patient care when they occur. Physicians, and in particular, residents, have historically underreported errors in patient care. Part of this underreporting results from residents not knowing or understanding the process of filing a report and feeling that the reports could be used as a form of retaliation. For hospital systems to continue to improve patient care, it is important for residents to not only know how to report errors in patient care but to feel comfortable doing so.

Corresponding author: Andrew J. Chin, DO, MS, MPH, Department of Internal Medicine, Adelante Healthcare, 1705 W Main St, Mesa, AZ 85201; [email protected].

Financial disclosures: None.

Funding: This study was funded by a research grant provided by Lake Eric College of Osteopathic Medicine to Andrew J. Chin and Anish Bhakta.

From Mount Sinai Medical Center, Miami Beach, FL.

Abstract

Objective: To develop and implement a process for administering COVID-19 monoclonal antibody infusions for outpatients with mild or moderate COVID-19 at high risk for hospitalization, using multidisciplinary collaboration, US Food and Drug Administration (FDA) guidance, and infection prevention standards.

Methods: When monoclonal antibody therapy became available for mild or moderate COVID-19 outpatients via Emergency Use Authorization (EUA), our institution sought to provide this therapy option to our patients. We describe the process for planning, implementing, and maintaining a successful program for administering novel therapies based on FDA guidance and infection prevention standards. Key components of our implementation process were multidisciplinary planning involving decision makers and stakeholders; setting realistic goals in the process; team communication; and measuring and reporting quality improvement on a regular basis.

Results: A total of 790 COVID-19 monoclonal antibody infusions were administered from November 20, 2020 to March 5, 2021. Steps to minimize the likelihood of adverse drug reactions were implemented and a low incidence (< 1%) has occurred. There has been no concern from staff regarding infection during the process. Rarely, patients have raised cost-related concerns, typically due to incomplete communication regarding billing prior to the infusion. Patients, families, nursing staff, physicians, pharmacy, and hospital administration have expressed satisfaction with the program.

Conclusion: This process can provide a template for other hospitals or health care delivery facilities to provide novel therapies to patients with mild or moderate COVID-19 in a safe and effective manner.

Keywords: COVID-19; monoclonal antibody; infusion; emergency use authorization.

SARS-CoV-2 and the disease it causes, COVID-19, have transformed from scientific vernacular to common household terms. It began with a cluster of pneumonia cases of unknown etiology in December 2019 in Wuhan, China, with physicians there reporting a novel coronavirus strain (2019-nCoV), now referred to as SARS-CoV-2. Rapid spread of this virus resulted in the World Health Organization (WHO) declaring an international public health emergency. Since this time, the virus has evolved into a worldwide pandemic. COVID-19 has dramatically impacted our society, resulting in more than 2.63 million global deaths as of this writing, of which more than 527,000 deaths have occurred in the United States.¹ This novel virus has resulted in a flurry of literature, research, therapies, and collaboration across multiple disciplines in an effort to prevent, treat, and mitigate cases and complications of this disease.

On November 9, 2020, and November 21, 2020, the US Food and Drug Administration (FDA) issued Emergency Use Authorizations (EUA) for 2 novel COVID-19 monoclonal therapies, bamlanivimab^2-3 and casirivimab/imdevimab,^3-4 respectively. The EUAs granted permission for these therapies to be administered for the treatment of mild to moderate COVID-19 in adult and pediatric patients (≥ 12 years and weighing at least 40 kg) with positive results of direct SARS-CoV-2 viral testing and who are at high risk for progressing to severe COVID-19 and/or hospitalization. The therapies work by targeting the SARS-CoV-2 spike protein and subsequent attachment to human angiotensin-converting enzyme 2 receptors. Clinical trial data leading to the EUA demonstrated a reduction in viral load, safe outcome, and most importantly, fewer hospitalization and emergency room visits, as compared to the placebo group.^5-7 The use of monoclonal antibodies is not new and gained recognition during the Ebola crisis, when the monoclonal antibody to the Ebola virus showed a significant survival benefit.⁸ Providing monoclonal antibody therapy soon after symptom onset aligns with a shift from the onset of the pandemic to the current focus on the administration of pharmaceutical therapy early in the disease course. This shift prevents progression to severe COVID-19, with the goal of reducing patient mortality, hospitalizations, and strain on health care systems.

The availability of novel neutralizing monoclonal antibodies for COVID-19 led to discussions of how to incorporate these therapies as new options for patients. Our institution networked with colleagues from multiple disciplines to discuss processes and policies for the safe administration of the monoclonal antibody infusion therapies. Federal health leaders urge more use of monoclonal antibodies, but many hospitals have been unable to successfully implement infusions due to staff and logistical challenges.⁹ This article presents a viable process that hospitals can use to provide these novel therapies to outpatients with mild to moderate COVID-19.

The Mount Sinai Medical Center, Florida Experience

Mount Sinai Medical Center in Miami Beach, Florida, is the largest private, independent, not-for-profit teaching hospital in South Florida, comprising 672 licensed beds and supporting 150,000 emergency department (ED) visits annually. Per the EUA criteria for use, COVID-19 monoclonal antibody therapies are not authorized for patients who are hospitalized or who require oxygen therapy due to COVID-19. Therefore, options for outpatient administration needed to be evaluated. Directly following the first EUA press release, a task force of key stakeholders was assembled to brainstorm and develop a process to offer this therapy to the community. A multidisciplinary task force with representation from the ED, nursing, primary care, hospital medicine, pharmacy, risk management, billing, information technology, infection prevention, and senior level leadership participated (Table).

The task force reviewed institutional outpatient locations to determine whether offering this service would be feasible (eg, ED, ambulatory care facilities, cancer center). The ED was selected because it would offer the largest array of appointment times to meet the community needs with around-the-clock availability. While Mount Sinai Medical Center offers care in 3 emergency center locations in Aventura, Hialeah, and Miami Beach, it was determined to initiate the infusions at the main campus center in Miami Beach only. The main campus affords an onsite pharmacy with suitable staffing to prepare the anticipated volume of infusions in a timely manner, as both therapies have short stabilities following preparation. Thus, it was decided that patients from freestanding emergency centers in Aventura and Hialeah would be moved to the Miami Beach ED location to receive therapy. Operating at a single site also allowed for more rapid implementation, monitoring, and ability to make modifications more easily. Discussions for the possible expansion of COVID-19 monoclonal antibody infusions at satellite locations are underway.

On November 20, 2020, 11 days after the formation of the multidisciplinary task force, the first COVID-19 monoclonal infusion was successfully administered. Figure 1 depicts the timeline from assessment to program implementation. Critical to implementation was the involvement of decision makers from all necessary departments early in the planning process to ensure that standard operating procedures were followed and that the patients, community, and organization had a positive experience. This allowed for simultaneous planning of electronic health record (Epic; EHR) builds, departmental workflows, and staff education, as described in the following section. Figure 2 shows the patient safety activities included in the implementation process.

On the day of bamlanivimab EUA release, email communication was shared among hospital leadership with details of the press release. Departments were quickly involved to initiate a task force to assess if and how this therapy could be offered at Mount Sinai Medical Center. The following sections explain the role of each stakeholder and their essential role to operationalize these novel EUA treatment options. The task force was organized and led by our chief medical officer and chief nursing officer.

Information Technology

Medication Ordering and Documentation EHR and Smart Pumps. Early in the pandemic, the antimicrobial stewardship (ASP) clinical coordinator became the designated point person for pharmacy assessment of novel COVID-19 therapies. As such, this pharmacist began reviewing the bamlanivimab and, later, the casirivimab/imdevimab EUA Fact Sheet for Health Care Providers. All necessary elements for the complete and safe ordering and dispensing of the medication were developed and reviewed by pharmacy administration and ED nursing leadership for input, prior to submitting to the information technology team for implementation. Building the COVID-19 monoclonal medication records into the EHR allowed for detailed direction (ie, administration and preparation instructions) to be consistently applied. The medication records were also built into hospital smart pumps so that nurses could access prepopulated, accurate volumes and infusion rates to minimize errors.

Order Set Development. The pharmacy medication build was added to a comprehensive order set (Figure 3), which was then developed to guide prescribers and standardize the process around ordering of COVID-19 monoclonal therapies. While these therapies are new, oncology monoclonal therapies are regularly administered to outpatients at Mount Sinai Cancer Center. The cancer center was therefore consulted on their process surrounding best practices in administration of monoclonal antibody therapies. This included protocols for medications used in pretreatment and management of hypersensitivity reactions and potential adverse drug reactions of both COVID-19 monoclonal therapies. These medication orders were selected by default in the order set to ensure that all patients received premedications aimed at minimizing the risk of hypersensitivity reaction, and had as-needed medication orders, in the event a hypersensitivity reaction occurred. Reducing hypersensitivity reaction risk is important as well to increase the likelihood that the patient would receive full therapy, as management of this adverse drug reactions involves possible cessation of therapy depending on the level of severity. The pharmacy department also ensured these medications were stocked in ED automated dispensing cabinets to promote quick access. In addition to the aforementioned nursing orders, we added EUA criteria for use and hyperlinks to the Fact Sheets for Patients and Caregivers and Health Care Providers for each monoclonal therapy, and restricted ordering to ED physicians, nurse practitioners, and physician assistants.

The order set underwent multidisciplinary review by pharmacy administration, the chair of emergency medicine, physicians, and ED nursing leadership prior to presentation and approval by the Pharmacy and Therapeutics Committee. Lastly, at time of implementation, the order set was added to the ED preference list, preventing inpatient access. Additionally, as a patient safety action, free- standing orders of COVID-19 monoclonal therapies were disabled, so providers could only order therapies via the approved, comprehensive order set.

Preliminary Assessment Tool. A provider assessment tool was developed to document patient-specific EUA criteria for use during initial assessment (Figure 4). This tool serves as a checklist and is visible to the full multidisciplinary team in the patient’s EHR. It is used as a resource at the time of pharmacist verification and ED physician assessment to ensure criteria for use are met.

Patient Referral. Patients with symptoms or concerns of COVID-19 exposure can make physician appointments via telemedicine or in person at Mount Sinai Medical Center’s primary care and specialty offices. At the time of patient encounter, physicians suspecting a COVID-19 diagnosis will refer patients for outpatient COVID-19 polymerase chain reaction (PCR) laboratory testing, which has an approximate 24-hour turnaround to results. Physicians also assess whether the patient meets EUA criteria for use, pending results of testing. In the event a patient meets EUA criteria for use, the physician provides patient counseling and requests verbal consent. Following this, the physician enters a note in the EHR describing the patient’s condition, criteria for use evaluation, and the patient’s verbal agreement to therapy. This preliminary screening is beneficial to begin planning with both the patient and ED to minimize delays. Patients are notified of the results of their test once available. If the COVID-19 PCR test returns positive, the physician will call the ED at the main campus and schedule the patient for COVID-19 monoclonal therapy. As the desired timeframe for administering COVID-19 monoclonal therapies is within less than 10 days of symptom onset, timely scheduling of appointments is crucial. Infusion appointments are typically provided the same or next day. The patients are informed that they must bring documentation of their positive COVID-19 PCR test to their ED visit. Lastly, because patients are pretreated with medication that may potentially impair driving, they are instructed that they cannot drive themselves home; ride shares also are not allowed in order to limit the spread of infection.

Emergency Department

Patient Arrival and Screening. A COVID-19 patient can be evaluated in the ED 1 of 2 ways. The first option is via outpatient office referral, as described previously. Upon arrival to the ED, a second screening is performed to ensure the patient still meets EUA criteria for use and the positive COVID-19 PCR test result is confirmed. If the patient no longer meets criteria, the patient is triaged accordingly, including evaluation for higher-level care (eg, supplemental oxygen, hospital admission). The second optoion is via new patient walk-ins without outpatient physician referral (Figure 4). In these cases, an initial screening is performed, documenting EUA criteria for use in the preliminary assessment (Figure 5). Physicians will consider an outside COVID-19 test as valid, so long as documentation is readily available confirming a positive PCR result. Otherwise, an in-house COVID-19 PCR test will be performed, which has a 2-hour turnaround time.

Infusion Schedule. The ED offers a total of 16 COVID-19 monoclonal infusions slots daily. These are broken up into 4 infusion time blocks (eg, 8 am, 12 pm, 4 pm, 8 pm), with each infusion time block consisting of 4 available patient appointments. A list of scheduled infusions for the day is emailed to the pharmacy department every morning, and patients are instructed to arrive 1 hour prior to their appointment time. This allows time for patient registration, assessment, and pharmacy notification in advance of order entry. For logistical purposes, and as a patient safety initiative to reduce the likelihood of medication errors, each of the available COVID-19 monoclonal antibodies is offered on a designated day. Bamlanivimab is offered on Tuesday, Thursday, Saturday, and Sunday, while casirivimab/imdevimab is offered Monday, Wednesday, and Friday. This provides flexibility to adjust should supply deviate based on Department of Health allocation or should new therapy options within this class of medication become available.

Patient Education. Prior to administration of the monoclonal therapy, physician and nursing staff obtain a formal, written patient consent for therapy and provide patients with the option of participating in the institutional review board (IRB) approved study. Details of this are discussed in the risk management and IRB sections of the article. Nursing staff also provides the medication-specific Fact Sheet for Patients and Caregivers in either Spanish or English, which is also included as a hyperlink on the COVID-19 Monoclonal Antibody Order Set for ease of access. Interpreter services are available for patients who speak other languages. An ED decentralized pharmacist is also available onsite Monday through Friday from 12 pm to 8:30 pm to supplement education and serve as a resource for any questions.

Infusion Ordering. Once the patient is ready to begin therapy, the he/she is brought to a dedicated overflow area of the ED. There are few, if any, patients in this location, and it is adjacent to the main emergency center for easy access by the patients, nurses, pharmacists, and physicians. The physician then enters orders in the EHR using the COVID-19 Monoclonal Antibody Order Set (Figure 3). Three discrete questions were built into the medication order: (1) Was patient consent obtained? (2) Was the Fact Sheet for Patient/Caregiver provided to the patient? (3) Is the patient COVID-19 PCR-positive? These questions were built as hard stops so that the medication orders cannot be placed without a response. This serves as another double-check to ensure processes are followed and helps facilitate timely verification by the pharmacist.

Medication Administration. One nurse is dedicated to administering the monoclonal therapies scheduled at 8 am and 12 pm and another at 4 pm and 8 pm. Each appointment block is 4 hours in duration to allow adequate time for patient registration, infusion, and postinfusion observation. The nurse administers the premedications and COVID-19 monoclonal therapy, and observes the patient for the required 1-hour postadministration observation period. Nursing orders detailing monitoring parameters for mild, moderate, and severe reactions, along with associated medication orders to administer in the event they occur, are detailed in the nursing orders of the COVID-19 Monoclonal Antibody Order Set (Figure 3). Prior to administration, the nurse scans each medication and the patient’s wrist identification band, and documents the time of administration within the EHR medication administration report.

Pharmacy Department

Medication Receipt Process. Inventory is currently allocated biweekly from the state department of health and will soon be transitioning to a direct order system. The pharmacy technician in charge of deliveries notifies the pharmacy Antimicrobial Stewardship Program (ASP) clinical coordinator upon receipt of the monoclonal therapies. Bamlanivimab is supplied as 1 vial per dose, whereas casirivimab/imdevimab is supplied as 4 vials or 8 vials per dose, depending how it is shipped. To reduce the likelihood of medication errors, the ASP clinical coordinator assembles each of the casirivimab/imdevimab vials into kits, where 1 kit equals 1 dose. Labels are then affixed to each kit indicating the medication name, number of vials which equal a full dose, and pharmacist signature. The kits are stored in a dedicated refrigerator, and inventory logs are affixed to the outside of the refrigerator and updated daily. This inventory is also communicated daily to ED physician, nursing, and pharmacy leadership, as well as the director of patient safety, who reports weekly usage to the state Department of Health and Human Services. These weekly reports are used to determine allocation amounts.

Medication Verification and Delivery. The Mount Sinai Medical Center pharmacist staffing model consists of centralized order entry and specialized, decentralized positions. All orders are verified by the ED pharmacist when scheduled (not a 24/7 service) and by the designated pharmacist for all other times. At the time of medication verification, the pharmacist documents patient-specific EUA criteria for use and confirms that consent was obtained and the Fact Sheet for Patients/Caregivers was provided. A pharmacist intervention was developed to assist with this documentation. Pharmacists input smart text “.COVIDmonoclonal” and a drop-down menu of EUA criteria for use appears. The pharmacist reviews the patient care notes and medication order question responses to ascertain this information, contacting the ED prescriber if further clarification is required. This verification serves as another check to ensure processes put in place are followed. Lastly, intravenous preparation and delivery are electronically recorded in the EHR, and the medications require nursing signature at the time of delivery to ensure a formal chain of custody.

Risk Management

At Mount Sinai Medical Center, all EUA and investigational therapies require patient consent. Consistent with this requirement, a COVID-19 monoclonal specific consent was developed by risk management. This is provided to every patient receiving a COVID-19 monoclonal infusion, in addition to the FDA EUA Fact Sheet for Patients and Caregivers, and documented as part of their EHR. The questions providers must answer are built into the order set to ensure this process is followed and these patient safety checks are incorporated into the workflow.

Billing and Finance Department

In alignment with Mount Sinai Medical Center’s mission to provide high-quality health care to its diverse community through teaching, research, charity care, and financial responsibility, it was determined that this therapy would be provided to all patients regardless of insurance type, including those who are uninsured. The billing and finance department was consulted prior to this service being offered, to provide patients with accurate and pertinent information. The billing and finance department provided guidance on how to document patient encounters at time of registration to facilitate appropriate billing. At this time, the medication is free of charge, but nonmedication-related ED fees apply. This is explained to patients so there is a clear understanding prior to booking their appointment.

Infection Prevention

As patients receiving COVID-19 monoclonal therapies can transmit the virus to others, measures to ensure protection for other patients and staff are vital. To minimize exposure, specific nursing and physician staff from the ED are assigned to the treatment of these patients, and patients receive infusions and postobservation monitoring in a designated wing of the ED. Additionally, all staff who interact with these patients are required to don full personal protective equipment. This includes not only physicians and nurses but all specialties such as physician assistants, nurse practitioners, pharmacists, and laboratory technicians. Moreover, patients are not permitted to go home in a ride share and are counseled on Centers for Disease Control and Prevention quarantining following infusion.

Measurement of Process and Outcomes and Reporting

IRB approval was sought and obtained early during initiation of this service, allowing study consent to be offered to patients at the time general consent was obtained, which maximized patient recruitment and streamlined workflow. The study is a prospective observational research study to determine the impact of administration of COVID-19 monoclonal antibody therapy on length of symptoms, chronic illness, and rate of hospitalization. Most patients were eager to participate and offer their assistance to the scientific community during this pandemic.

Staff Education

In order to successfully implement this multidisciplinary EUA treatment option, comprehensive staff education was paramount after the workflow was developed. Prior to the first day of infusions, nurses and pharmacists were provided education during multiple huddle announcements. The pharmacy team also provided screen captures via email to the pharmacists so they could become familiar with the order set, intervention documentation, and location of the preliminary assessment of EUA criteria for use at the time of order verification. The emergency medicine department chair and chief medical officer also provided education via several virtual meetings and email to referring physicians (specialists and primary care) and residents in the emergency centers involved in COVID-19 monoclonal therapy-related patient care.

Factors Contributing to Success

We believe the reasons for continued success of this process are multifactorial and include the following key elements. Multidisciplinary planning, which included decision makers and all stakeholders, began at the time the idea was conceived. This allowed quick implementation of this service by efficiently navigating barriers to engaging impacted staff early on. Throughout this process, the authors set realistic step-wise goals. While navigating through the many details to implementation described, we also kept in mind the big picture, which was to provide this potentially lifesaving therapy to as many qualifying members of our community as possible. This included being flexible with the process and adapting when needed to achieve this ultimate goal. A focus on safety remained a priority to minimize possible errors and enhance patient and staff satisfaction. The optimization of the EHR streamlined workflow, provided point-of-care resources, and enhanced patient safety. Additionally, the target date set for implementation allowed staff and department leads adequate time to plan for and anticipate the changes. Serving only 1 patient on the first day allowed time for staff to experience this new process hands-on and provided opportunity for focused education. This team communication was essential to implementing this project, including staff training of processes and procedures prior to go-live. Early incorporation of IRB approval allowed the experience to be assessed and considered for contribution to the scientific literature to tackle this novel virus that has impacted our communities locally, nationally, and abroad. Moreover, continued measurement and reporting on a regular basis leads to performance improvement. The process outlined here can be adapted to incorporate other new therapies in the future, such as the recent February 9, 2021, EUA of the COVID-19 monoclonal antibody combination bamlanivimab and etesevimab.¹⁰

Conclusion

We administered 790 COVID-19 monoclonal antibody infusions between November 20, 2020 and March 5, 2021. Steps to minimize the likelihood of hypersensitivity reactions were implemented, and a low incidence (< 1%) has been observed. There has been no incidence of infection, concern from staff about infection prevention, or risk of infection during the processes. There have been very infrequent cost-related concerns raised by patients, typically due to incomplete communication regarding billing prior to the infusion. To address these issues, staff education has been provided to enhance patient instruction on this topic. The program has provided patient and family satisfaction, as well nursing, physician, pharmacist, clinical staff, and hospital administration pride and gratification. Setting up a new program to provide a 4-hour patient encounter to infuse therapy to high-risk patients with COVID-19 requires commitment and effort. This article describes the experience, ideas, and formula others may consider using to set up such a program. Through networking and formal phone calls and meetings about monoclonal antibody therapy, we have heard about other institutions who have not been able to institute this program due to various barriers to implementation. We hope our experience serves as a resource for others to provide this therapy to their patients and expand access in an effort to mitigate COVID-19 consequences and cases affecting our communities.

Corresponding author: Kathleen Jodoin, PharmD, BCPS, Mount Sinai Medical Center, 4300 Alton Rd, Miami Beach, FL 33140; [email protected].

Financial disclosures: None.

From Mount Sinai Medical Center, Miami Beach, FL.

Abstract

Objective: To develop and implement a process for administering COVID-19 monoclonal antibody infusions for outpatients with mild or moderate COVID-19 at high risk for hospitalization, using multidisciplinary collaboration, US Food and Drug Administration (FDA) guidance, and infection prevention standards.

Methods: When monoclonal antibody therapy became available for mild or moderate COVID-19 outpatients via Emergency Use Authorization (EUA), our institution sought to provide this therapy option to our patients. We describe the process for planning, implementing, and maintaining a successful program for administering novel therapies based on FDA guidance and infection prevention standards. Key components of our implementation process were multidisciplinary planning involving decision makers and stakeholders; setting realistic goals in the process; team communication; and measuring and reporting quality improvement on a regular basis.

Results: A total of 790 COVID-19 monoclonal antibody infusions were administered from November 20, 2020 to March 5, 2021. Steps to minimize the likelihood of adverse drug reactions were implemented and a low incidence (< 1%) has occurred. There has been no concern from staff regarding infection during the process. Rarely, patients have raised cost-related concerns, typically due to incomplete communication regarding billing prior to the infusion. Patients, families, nursing staff, physicians, pharmacy, and hospital administration have expressed satisfaction with the program.

Conclusion: This process can provide a template for other hospitals or health care delivery facilities to provide novel therapies to patients with mild or moderate COVID-19 in a safe and effective manner.

Keywords: COVID-19; monoclonal antibody; infusion; emergency use authorization.

SARS-CoV-2 and the disease it causes, COVID-19, have transformed from scientific vernacular to common household terms. It began with a cluster of pneumonia cases of unknown etiology in December 2019 in Wuhan, China, with physicians there reporting a novel coronavirus strain (2019-nCoV), now referred to as SARS-CoV-2. Rapid spread of this virus resulted in the World Health Organization (WHO) declaring an international public health emergency. Since this time, the virus has evolved into a worldwide pandemic. COVID-19 has dramatically impacted our society, resulting in more than 2.63 million global deaths as of this writing, of which more than 527,000 deaths have occurred in the United States.¹ This novel virus has resulted in a flurry of literature, research, therapies, and collaboration across multiple disciplines in an effort to prevent, treat, and mitigate cases and complications of this disease.

On November 9, 2020, and November 21, 2020, the US Food and Drug Administration (FDA) issued Emergency Use Authorizations (EUA) for 2 novel COVID-19 monoclonal therapies, bamlanivimab^2-3 and casirivimab/imdevimab,^3-4 respectively. The EUAs granted permission for these therapies to be administered for the treatment of mild to moderate COVID-19 in adult and pediatric patients (≥ 12 years and weighing at least 40 kg) with positive results of direct SARS-CoV-2 viral testing and who are at high risk for progressing to severe COVID-19 and/or hospitalization. The therapies work by targeting the SARS-CoV-2 spike protein and subsequent attachment to human angiotensin-converting enzyme 2 receptors. Clinical trial data leading to the EUA demonstrated a reduction in viral load, safe outcome, and most importantly, fewer hospitalization and emergency room visits, as compared to the placebo group.^5-7 The use of monoclonal antibodies is not new and gained recognition during the Ebola crisis, when the monoclonal antibody to the Ebola virus showed a significant survival benefit.⁸ Providing monoclonal antibody therapy soon after symptom onset aligns with a shift from the onset of the pandemic to the current focus on the administration of pharmaceutical therapy early in the disease course. This shift prevents progression to severe COVID-19, with the goal of reducing patient mortality, hospitalizations, and strain on health care systems.

The availability of novel neutralizing monoclonal antibodies for COVID-19 led to discussions of how to incorporate these therapies as new options for patients. Our institution networked with colleagues from multiple disciplines to discuss processes and policies for the safe administration of the monoclonal antibody infusion therapies. Federal health leaders urge more use of monoclonal antibodies, but many hospitals have been unable to successfully implement infusions due to staff and logistical challenges.⁹ This article presents a viable process that hospitals can use to provide these novel therapies to outpatients with mild to moderate COVID-19.

The Mount Sinai Medical Center, Florida Experience

Mount Sinai Medical Center in Miami Beach, Florida, is the largest private, independent, not-for-profit teaching hospital in South Florida, comprising 672 licensed beds and supporting 150,000 emergency department (ED) visits annually. Per the EUA criteria for use, COVID-19 monoclonal antibody therapies are not authorized for patients who are hospitalized or who require oxygen therapy due to COVID-19. Therefore, options for outpatient administration needed to be evaluated. Directly following the first EUA press release, a task force of key stakeholders was assembled to brainstorm and develop a process to offer this therapy to the community. A multidisciplinary task force with representation from the ED, nursing, primary care, hospital medicine, pharmacy, risk management, billing, information technology, infection prevention, and senior level leadership participated (Table).

The task force reviewed institutional outpatient locations to determine whether offering this service would be feasible (eg, ED, ambulatory care facilities, cancer center). The ED was selected because it would offer the largest array of appointment times to meet the community needs with around-the-clock availability. While Mount Sinai Medical Center offers care in 3 emergency center locations in Aventura, Hialeah, and Miami Beach, it was determined to initiate the infusions at the main campus center in Miami Beach only. The main campus affords an onsite pharmacy with suitable staffing to prepare the anticipated volume of infusions in a timely manner, as both therapies have short stabilities following preparation. Thus, it was decided that patients from freestanding emergency centers in Aventura and Hialeah would be moved to the Miami Beach ED location to receive therapy. Operating at a single site also allowed for more rapid implementation, monitoring, and ability to make modifications more easily. Discussions for the possible expansion of COVID-19 monoclonal antibody infusions at satellite locations are underway.

On November 20, 2020, 11 days after the formation of the multidisciplinary task force, the first COVID-19 monoclonal infusion was successfully administered. Figure 1 depicts the timeline from assessment to program implementation. Critical to implementation was the involvement of decision makers from all necessary departments early in the planning process to ensure that standard operating procedures were followed and that the patients, community, and organization had a positive experience. This allowed for simultaneous planning of electronic health record (Epic; EHR) builds, departmental workflows, and staff education, as described in the following section. Figure 2 shows the patient safety activities included in the implementation process.

On the day of bamlanivimab EUA release, email communication was shared among hospital leadership with details of the press release. Departments were quickly involved to initiate a task force to assess if and how this therapy could be offered at Mount Sinai Medical Center. The following sections explain the role of each stakeholder and their essential role to operationalize these novel EUA treatment options. The task force was organized and led by our chief medical officer and chief nursing officer.

Information Technology

Medication Ordering and Documentation EHR and Smart Pumps. Early in the pandemic, the antimicrobial stewardship (ASP) clinical coordinator became the designated point person for pharmacy assessment of novel COVID-19 therapies. As such, this pharmacist began reviewing the bamlanivimab and, later, the casirivimab/imdevimab EUA Fact Sheet for Health Care Providers. All necessary elements for the complete and safe ordering and dispensing of the medication were developed and reviewed by pharmacy administration and ED nursing leadership for input, prior to submitting to the information technology team for implementation. Building the COVID-19 monoclonal medication records into the EHR allowed for detailed direction (ie, administration and preparation instructions) to be consistently applied. The medication records were also built into hospital smart pumps so that nurses could access prepopulated, accurate volumes and infusion rates to minimize errors.

Order Set Development. The pharmacy medication build was added to a comprehensive order set (Figure 3), which was then developed to guide prescribers and standardize the process around ordering of COVID-19 monoclonal therapies. While these therapies are new, oncology monoclonal therapies are regularly administered to outpatients at Mount Sinai Cancer Center. The cancer center was therefore consulted on their process surrounding best practices in administration of monoclonal antibody therapies. This included protocols for medications used in pretreatment and management of hypersensitivity reactions and potential adverse drug reactions of both COVID-19 monoclonal therapies. These medication orders were selected by default in the order set to ensure that all patients received premedications aimed at minimizing the risk of hypersensitivity reaction, and had as-needed medication orders, in the event a hypersensitivity reaction occurred. Reducing hypersensitivity reaction risk is important as well to increase the likelihood that the patient would receive full therapy, as management of this adverse drug reactions involves possible cessation of therapy depending on the level of severity. The pharmacy department also ensured these medications were stocked in ED automated dispensing cabinets to promote quick access. In addition to the aforementioned nursing orders, we added EUA criteria for use and hyperlinks to the Fact Sheets for Patients and Caregivers and Health Care Providers for each monoclonal therapy, and restricted ordering to ED physicians, nurse practitioners, and physician assistants.

The order set underwent multidisciplinary review by pharmacy administration, the chair of emergency medicine, physicians, and ED nursing leadership prior to presentation and approval by the Pharmacy and Therapeutics Committee. Lastly, at time of implementation, the order set was added to the ED preference list, preventing inpatient access. Additionally, as a patient safety action, free- standing orders of COVID-19 monoclonal therapies were disabled, so providers could only order therapies via the approved, comprehensive order set.

Preliminary Assessment Tool. A provider assessment tool was developed to document patient-specific EUA criteria for use during initial assessment (Figure 4). This tool serves as a checklist and is visible to the full multidisciplinary team in the patient’s EHR. It is used as a resource at the time of pharmacist verification and ED physician assessment to ensure criteria for use are met.

Patient Referral. Patients with symptoms or concerns of COVID-19 exposure can make physician appointments via telemedicine or in person at Mount Sinai Medical Center’s primary care and specialty offices. At the time of patient encounter, physicians suspecting a COVID-19 diagnosis will refer patients for outpatient COVID-19 polymerase chain reaction (PCR) laboratory testing, which has an approximate 24-hour turnaround to results. Physicians also assess whether the patient meets EUA criteria for use, pending results of testing. In the event a patient meets EUA criteria for use, the physician provides patient counseling and requests verbal consent. Following this, the physician enters a note in the EHR describing the patient’s condition, criteria for use evaluation, and the patient’s verbal agreement to therapy. This preliminary screening is beneficial to begin planning with both the patient and ED to minimize delays. Patients are notified of the results of their test once available. If the COVID-19 PCR test returns positive, the physician will call the ED at the main campus and schedule the patient for COVID-19 monoclonal therapy. As the desired timeframe for administering COVID-19 monoclonal therapies is within less than 10 days of symptom onset, timely scheduling of appointments is crucial. Infusion appointments are typically provided the same or next day. The patients are informed that they must bring documentation of their positive COVID-19 PCR test to their ED visit. Lastly, because patients are pretreated with medication that may potentially impair driving, they are instructed that they cannot drive themselves home; ride shares also are not allowed in order to limit the spread of infection.

Emergency Department

Patient Arrival and Screening. A COVID-19 patient can be evaluated in the ED 1 of 2 ways. The first option is via outpatient office referral, as described previously. Upon arrival to the ED, a second screening is performed to ensure the patient still meets EUA criteria for use and the positive COVID-19 PCR test result is confirmed. If the patient no longer meets criteria, the patient is triaged accordingly, including evaluation for higher-level care (eg, supplemental oxygen, hospital admission). The second optoion is via new patient walk-ins without outpatient physician referral (Figure 4). In these cases, an initial screening is performed, documenting EUA criteria for use in the preliminary assessment (Figure 5). Physicians will consider an outside COVID-19 test as valid, so long as documentation is readily available confirming a positive PCR result. Otherwise, an in-house COVID-19 PCR test will be performed, which has a 2-hour turnaround time.

Infusion Schedule. The ED offers a total of 16 COVID-19 monoclonal infusions slots daily. These are broken up into 4 infusion time blocks (eg, 8 am, 12 pm, 4 pm, 8 pm), with each infusion time block consisting of 4 available patient appointments. A list of scheduled infusions for the day is emailed to the pharmacy department every morning, and patients are instructed to arrive 1 hour prior to their appointment time. This allows time for patient registration, assessment, and pharmacy notification in advance of order entry. For logistical purposes, and as a patient safety initiative to reduce the likelihood of medication errors, each of the available COVID-19 monoclonal antibodies is offered on a designated day. Bamlanivimab is offered on Tuesday, Thursday, Saturday, and Sunday, while casirivimab/imdevimab is offered Monday, Wednesday, and Friday. This provides flexibility to adjust should supply deviate based on Department of Health allocation or should new therapy options within this class of medication become available.

Patient Education. Prior to administration of the monoclonal therapy, physician and nursing staff obtain a formal, written patient consent for therapy and provide patients with the option of participating in the institutional review board (IRB) approved study. Details of this are discussed in the risk management and IRB sections of the article. Nursing staff also provides the medication-specific Fact Sheet for Patients and Caregivers in either Spanish or English, which is also included as a hyperlink on the COVID-19 Monoclonal Antibody Order Set for ease of access. Interpreter services are available for patients who speak other languages. An ED decentralized pharmacist is also available onsite Monday through Friday from 12 pm to 8:30 pm to supplement education and serve as a resource for any questions.

Infusion Ordering. Once the patient is ready to begin therapy, the he/she is brought to a dedicated overflow area of the ED. There are few, if any, patients in this location, and it is adjacent to the main emergency center for easy access by the patients, nurses, pharmacists, and physicians. The physician then enters orders in the EHR using the COVID-19 Monoclonal Antibody Order Set (Figure 3). Three discrete questions were built into the medication order: (1) Was patient consent obtained? (2) Was the Fact Sheet for Patient/Caregiver provided to the patient? (3) Is the patient COVID-19 PCR-positive? These questions were built as hard stops so that the medication orders cannot be placed without a response. This serves as another double-check to ensure processes are followed and helps facilitate timely verification by the pharmacist.

Medication Administration. One nurse is dedicated to administering the monoclonal therapies scheduled at 8 am and 12 pm and another at 4 pm and 8 pm. Each appointment block is 4 hours in duration to allow adequate time for patient registration, infusion, and postinfusion observation. The nurse administers the premedications and COVID-19 monoclonal therapy, and observes the patient for the required 1-hour postadministration observation period. Nursing orders detailing monitoring parameters for mild, moderate, and severe reactions, along with associated medication orders to administer in the event they occur, are detailed in the nursing orders of the COVID-19 Monoclonal Antibody Order Set (Figure 3). Prior to administration, the nurse scans each medication and the patient’s wrist identification band, and documents the time of administration within the EHR medication administration report.

Pharmacy Department

Medication Receipt Process. Inventory is currently allocated biweekly from the state department of health and will soon be transitioning to a direct order system. The pharmacy technician in charge of deliveries notifies the pharmacy Antimicrobial Stewardship Program (ASP) clinical coordinator upon receipt of the monoclonal therapies. Bamlanivimab is supplied as 1 vial per dose, whereas casirivimab/imdevimab is supplied as 4 vials or 8 vials per dose, depending how it is shipped. To reduce the likelihood of medication errors, the ASP clinical coordinator assembles each of the casirivimab/imdevimab vials into kits, where 1 kit equals 1 dose. Labels are then affixed to each kit indicating the medication name, number of vials which equal a full dose, and pharmacist signature. The kits are stored in a dedicated refrigerator, and inventory logs are affixed to the outside of the refrigerator and updated daily. This inventory is also communicated daily to ED physician, nursing, and pharmacy leadership, as well as the director of patient safety, who reports weekly usage to the state Department of Health and Human Services. These weekly reports are used to determine allocation amounts.

Medication Verification and Delivery. The Mount Sinai Medical Center pharmacist staffing model consists of centralized order entry and specialized, decentralized positions. All orders are verified by the ED pharmacist when scheduled (not a 24/7 service) and by the designated pharmacist for all other times. At the time of medication verification, the pharmacist documents patient-specific EUA criteria for use and confirms that consent was obtained and the Fact Sheet for Patients/Caregivers was provided. A pharmacist intervention was developed to assist with this documentation. Pharmacists input smart text “.COVIDmonoclonal” and a drop-down menu of EUA criteria for use appears. The pharmacist reviews the patient care notes and medication order question responses to ascertain this information, contacting the ED prescriber if further clarification is required. This verification serves as another check to ensure processes put in place are followed. Lastly, intravenous preparation and delivery are electronically recorded in the EHR, and the medications require nursing signature at the time of delivery to ensure a formal chain of custody.

Risk Management

At Mount Sinai Medical Center, all EUA and investigational therapies require patient consent. Consistent with this requirement, a COVID-19 monoclonal specific consent was developed by risk management. This is provided to every patient receiving a COVID-19 monoclonal infusion, in addition to the FDA EUA Fact Sheet for Patients and Caregivers, and documented as part of their EHR. The questions providers must answer are built into the order set to ensure this process is followed and these patient safety checks are incorporated into the workflow.

Billing and Finance Department

In alignment with Mount Sinai Medical Center’s mission to provide high-quality health care to its diverse community through teaching, research, charity care, and financial responsibility, it was determined that this therapy would be provided to all patients regardless of insurance type, including those who are uninsured. The billing and finance department was consulted prior to this service being offered, to provide patients with accurate and pertinent information. The billing and finance department provided guidance on how to document patient encounters at time of registration to facilitate appropriate billing. At this time, the medication is free of charge, but nonmedication-related ED fees apply. This is explained to patients so there is a clear understanding prior to booking their appointment.

Infection Prevention

As patients receiving COVID-19 monoclonal therapies can transmit the virus to others, measures to ensure protection for other patients and staff are vital. To minimize exposure, specific nursing and physician staff from the ED are assigned to the treatment of these patients, and patients receive infusions and postobservation monitoring in a designated wing of the ED. Additionally, all staff who interact with these patients are required to don full personal protective equipment. This includes not only physicians and nurses but all specialties such as physician assistants, nurse practitioners, pharmacists, and laboratory technicians. Moreover, patients are not permitted to go home in a ride share and are counseled on Centers for Disease Control and Prevention quarantining following infusion.

Measurement of Process and Outcomes and Reporting

IRB approval was sought and obtained early during initiation of this service, allowing study consent to be offered to patients at the time general consent was obtained, which maximized patient recruitment and streamlined workflow. The study is a prospective observational research study to determine the impact of administration of COVID-19 monoclonal antibody therapy on length of symptoms, chronic illness, and rate of hospitalization. Most patients were eager to participate and offer their assistance to the scientific community during this pandemic.

Staff Education

In order to successfully implement this multidisciplinary EUA treatment option, comprehensive staff education was paramount after the workflow was developed. Prior to the first day of infusions, nurses and pharmacists were provided education during multiple huddle announcements. The pharmacy team also provided screen captures via email to the pharmacists so they could become familiar with the order set, intervention documentation, and location of the preliminary assessment of EUA criteria for use at the time of order verification. The emergency medicine department chair and chief medical officer also provided education via several virtual meetings and email to referring physicians (specialists and primary care) and residents in the emergency centers involved in COVID-19 monoclonal therapy-related patient care.

Factors Contributing to Success

We believe the reasons for continued success of this process are multifactorial and include the following key elements. Multidisciplinary planning, which included decision makers and all stakeholders, began at the time the idea was conceived. This allowed quick implementation of this service by efficiently navigating barriers to engaging impacted staff early on. Throughout this process, the authors set realistic step-wise goals. While navigating through the many details to implementation described, we also kept in mind the big picture, which was to provide this potentially lifesaving therapy to as many qualifying members of our community as possible. This included being flexible with the process and adapting when needed to achieve this ultimate goal. A focus on safety remained a priority to minimize possible errors and enhance patient and staff satisfaction. The optimization of the EHR streamlined workflow, provided point-of-care resources, and enhanced patient safety. Additionally, the target date set for implementation allowed staff and department leads adequate time to plan for and anticipate the changes. Serving only 1 patient on the first day allowed time for staff to experience this new process hands-on and provided opportunity for focused education. This team communication was essential to implementing this project, including staff training of processes and procedures prior to go-live. Early incorporation of IRB approval allowed the experience to be assessed and considered for contribution to the scientific literature to tackle this novel virus that has impacted our communities locally, nationally, and abroad. Moreover, continued measurement and reporting on a regular basis leads to performance improvement. The process outlined here can be adapted to incorporate other new therapies in the future, such as the recent February 9, 2021, EUA of the COVID-19 monoclonal antibody combination bamlanivimab and etesevimab.¹⁰

Conclusion

We administered 790 COVID-19 monoclonal antibody infusions between November 20, 2020 and March 5, 2021. Steps to minimize the likelihood of hypersensitivity reactions were implemented, and a low incidence (< 1%) has been observed. There has been no incidence of infection, concern from staff about infection prevention, or risk of infection during the processes. There have been very infrequent cost-related concerns raised by patients, typically due to incomplete communication regarding billing prior to the infusion. To address these issues, staff education has been provided to enhance patient instruction on this topic. The program has provided patient and family satisfaction, as well nursing, physician, pharmacist, clinical staff, and hospital administration pride and gratification. Setting up a new program to provide a 4-hour patient encounter to infuse therapy to high-risk patients with COVID-19 requires commitment and effort. This article describes the experience, ideas, and formula others may consider using to set up such a program. Through networking and formal phone calls and meetings about monoclonal antibody therapy, we have heard about other institutions who have not been able to institute this program due to various barriers to implementation. We hope our experience serves as a resource for others to provide this therapy to their patients and expand access in an effort to mitigate COVID-19 consequences and cases affecting our communities.

Corresponding author: Kathleen Jodoin, PharmD, BCPS, Mount Sinai Medical Center, 4300 Alton Rd, Miami Beach, FL 33140; [email protected].

Financial disclosures: None.

From Mount Sinai Medical Center, Miami Beach, FL.

Abstract

Objective: To develop and implement a process for administering COVID-19 monoclonal antibody infusions for outpatients with mild or moderate COVID-19 at high risk for hospitalization, using multidisciplinary collaboration, US Food and Drug Administration (FDA) guidance, and infection prevention standards.

Methods: When monoclonal antibody therapy became available for mild or moderate COVID-19 outpatients via Emergency Use Authorization (EUA), our institution sought to provide this therapy option to our patients. We describe the process for planning, implementing, and maintaining a successful program for administering novel therapies based on FDA guidance and infection prevention standards. Key components of our implementation process were multidisciplinary planning involving decision makers and stakeholders; setting realistic goals in the process; team communication; and measuring and reporting quality improvement on a regular basis.

Results: A total of 790 COVID-19 monoclonal antibody infusions were administered from November 20, 2020 to March 5, 2021. Steps to minimize the likelihood of adverse drug reactions were implemented and a low incidence (< 1%) has occurred. There has been no concern from staff regarding infection during the process. Rarely, patients have raised cost-related concerns, typically due to incomplete communication regarding billing prior to the infusion. Patients, families, nursing staff, physicians, pharmacy, and hospital administration have expressed satisfaction with the program.

Conclusion: This process can provide a template for other hospitals or health care delivery facilities to provide novel therapies to patients with mild or moderate COVID-19 in a safe and effective manner.

Keywords: COVID-19; monoclonal antibody; infusion; emergency use authorization.

SARS-CoV-2 and the disease it causes, COVID-19, have transformed from scientific vernacular to common household terms. It began with a cluster of pneumonia cases of unknown etiology in December 2019 in Wuhan, China, with physicians there reporting a novel coronavirus strain (2019-nCoV), now referred to as SARS-CoV-2. Rapid spread of this virus resulted in the World Health Organization (WHO) declaring an international public health emergency. Since this time, the virus has evolved into a worldwide pandemic. COVID-19 has dramatically impacted our society, resulting in more than 2.63 million global deaths as of this writing, of which more than 527,000 deaths have occurred in the United States.¹ This novel virus has resulted in a flurry of literature, research, therapies, and collaboration across multiple disciplines in an effort to prevent, treat, and mitigate cases and complications of this disease.

On November 9, 2020, and November 21, 2020, the US Food and Drug Administration (FDA) issued Emergency Use Authorizations (EUA) for 2 novel COVID-19 monoclonal therapies, bamlanivimab^2-3 and casirivimab/imdevimab,^3-4 respectively. The EUAs granted permission for these therapies to be administered for the treatment of mild to moderate COVID-19 in adult and pediatric patients (≥ 12 years and weighing at least 40 kg) with positive results of direct SARS-CoV-2 viral testing and who are at high risk for progressing to severe COVID-19 and/or hospitalization. The therapies work by targeting the SARS-CoV-2 spike protein and subsequent attachment to human angiotensin-converting enzyme 2 receptors. Clinical trial data leading to the EUA demonstrated a reduction in viral load, safe outcome, and most importantly, fewer hospitalization and emergency room visits, as compared to the placebo group.^5-7 The use of monoclonal antibodies is not new and gained recognition during the Ebola crisis, when the monoclonal antibody to the Ebola virus showed a significant survival benefit.⁸ Providing monoclonal antibody therapy soon after symptom onset aligns with a shift from the onset of the pandemic to the current focus on the administration of pharmaceutical therapy early in the disease course. This shift prevents progression to severe COVID-19, with the goal of reducing patient mortality, hospitalizations, and strain on health care systems.

The availability of novel neutralizing monoclonal antibodies for COVID-19 led to discussions of how to incorporate these therapies as new options for patients. Our institution networked with colleagues from multiple disciplines to discuss processes and policies for the safe administration of the monoclonal antibody infusion therapies. Federal health leaders urge more use of monoclonal antibodies, but many hospitals have been unable to successfully implement infusions due to staff and logistical challenges.⁹ This article presents a viable process that hospitals can use to provide these novel therapies to outpatients with mild to moderate COVID-19.

The Mount Sinai Medical Center, Florida Experience

Mount Sinai Medical Center in Miami Beach, Florida, is the largest private, independent, not-for-profit teaching hospital in South Florida, comprising 672 licensed beds and supporting 150,000 emergency department (ED) visits annually. Per the EUA criteria for use, COVID-19 monoclonal antibody therapies are not authorized for patients who are hospitalized or who require oxygen therapy due to COVID-19. Therefore, options for outpatient administration needed to be evaluated. Directly following the first EUA press release, a task force of key stakeholders was assembled to brainstorm and develop a process to offer this therapy to the community. A multidisciplinary task force with representation from the ED, nursing, primary care, hospital medicine, pharmacy, risk management, billing, information technology, infection prevention, and senior level leadership participated (Table).

The task force reviewed institutional outpatient locations to determine whether offering this service would be feasible (eg, ED, ambulatory care facilities, cancer center). The ED was selected because it would offer the largest array of appointment times to meet the community needs with around-the-clock availability. While Mount Sinai Medical Center offers care in 3 emergency center locations in Aventura, Hialeah, and Miami Beach, it was determined to initiate the infusions at the main campus center in Miami Beach only. The main campus affords an onsite pharmacy with suitable staffing to prepare the anticipated volume of infusions in a timely manner, as both therapies have short stabilities following preparation. Thus, it was decided that patients from freestanding emergency centers in Aventura and Hialeah would be moved to the Miami Beach ED location to receive therapy. Operating at a single site also allowed for more rapid implementation, monitoring, and ability to make modifications more easily. Discussions for the possible expansion of COVID-19 monoclonal antibody infusions at satellite locations are underway.

On November 20, 2020, 11 days after the formation of the multidisciplinary task force, the first COVID-19 monoclonal infusion was successfully administered. Figure 1 depicts the timeline from assessment to program implementation. Critical to implementation was the involvement of decision makers from all necessary departments early in the planning process to ensure that standard operating procedures were followed and that the patients, community, and organization had a positive experience. This allowed for simultaneous planning of electronic health record (Epic; EHR) builds, departmental workflows, and staff education, as described in the following section. Figure 2 shows the patient safety activities included in the implementation process.

On the day of bamlanivimab EUA release, email communication was shared among hospital leadership with details of the press release. Departments were quickly involved to initiate a task force to assess if and how this therapy could be offered at Mount Sinai Medical Center. The following sections explain the role of each stakeholder and their essential role to operationalize these novel EUA treatment options. The task force was organized and led by our chief medical officer and chief nursing officer.

Information Technology

Medication Ordering and Documentation EHR and Smart Pumps. Early in the pandemic, the antimicrobial stewardship (ASP) clinical coordinator became the designated point person for pharmacy assessment of novel COVID-19 therapies. As such, this pharmacist began reviewing the bamlanivimab and, later, the casirivimab/imdevimab EUA Fact Sheet for Health Care Providers. All necessary elements for the complete and safe ordering and dispensing of the medication were developed and reviewed by pharmacy administration and ED nursing leadership for input, prior to submitting to the information technology team for implementation. Building the COVID-19 monoclonal medication records into the EHR allowed for detailed direction (ie, administration and preparation instructions) to be consistently applied. The medication records were also built into hospital smart pumps so that nurses could access prepopulated, accurate volumes and infusion rates to minimize errors.

Order Set Development. The pharmacy medication build was added to a comprehensive order set (Figure 3), which was then developed to guide prescribers and standardize the process around ordering of COVID-19 monoclonal therapies. While these therapies are new, oncology monoclonal therapies are regularly administered to outpatients at Mount Sinai Cancer Center. The cancer center was therefore consulted on their process surrounding best practices in administration of monoclonal antibody therapies. This included protocols for medications used in pretreatment and management of hypersensitivity reactions and potential adverse drug reactions of both COVID-19 monoclonal therapies. These medication orders were selected by default in the order set to ensure that all patients received premedications aimed at minimizing the risk of hypersensitivity reaction, and had as-needed medication orders, in the event a hypersensitivity reaction occurred. Reducing hypersensitivity reaction risk is important as well to increase the likelihood that the patient would receive full therapy, as management of this adverse drug reactions involves possible cessation of therapy depending on the level of severity. The pharmacy department also ensured these medications were stocked in ED automated dispensing cabinets to promote quick access. In addition to the aforementioned nursing orders, we added EUA criteria for use and hyperlinks to the Fact Sheets for Patients and Caregivers and Health Care Providers for each monoclonal therapy, and restricted ordering to ED physicians, nurse practitioners, and physician assistants.

The order set underwent multidisciplinary review by pharmacy administration, the chair of emergency medicine, physicians, and ED nursing leadership prior to presentation and approval by the Pharmacy and Therapeutics Committee. Lastly, at time of implementation, the order set was added to the ED preference list, preventing inpatient access. Additionally, as a patient safety action, free- standing orders of COVID-19 monoclonal therapies were disabled, so providers could only order therapies via the approved, comprehensive order set.

Preliminary Assessment Tool. A provider assessment tool was developed to document patient-specific EUA criteria for use during initial assessment (Figure 4). This tool serves as a checklist and is visible to the full multidisciplinary team in the patient’s EHR. It is used as a resource at the time of pharmacist verification and ED physician assessment to ensure criteria for use are met.

Patient Referral. Patients with symptoms or concerns of COVID-19 exposure can make physician appointments via telemedicine or in person at Mount Sinai Medical Center’s primary care and specialty offices. At the time of patient encounter, physicians suspecting a COVID-19 diagnosis will refer patients for outpatient COVID-19 polymerase chain reaction (PCR) laboratory testing, which has an approximate 24-hour turnaround to results. Physicians also assess whether the patient meets EUA criteria for use, pending results of testing. In the event a patient meets EUA criteria for use, the physician provides patient counseling and requests verbal consent. Following this, the physician enters a note in the EHR describing the patient’s condition, criteria for use evaluation, and the patient’s verbal agreement to therapy. This preliminary screening is beneficial to begin planning with both the patient and ED to minimize delays. Patients are notified of the results of their test once available. If the COVID-19 PCR test returns positive, the physician will call the ED at the main campus and schedule the patient for COVID-19 monoclonal therapy. As the desired timeframe for administering COVID-19 monoclonal therapies is within less than 10 days of symptom onset, timely scheduling of appointments is crucial. Infusion appointments are typically provided the same or next day. The patients are informed that they must bring documentation of their positive COVID-19 PCR test to their ED visit. Lastly, because patients are pretreated with medication that may potentially impair driving, they are instructed that they cannot drive themselves home; ride shares also are not allowed in order to limit the spread of infection.

Emergency Department

Patient Arrival and Screening. A COVID-19 patient can be evaluated in the ED 1 of 2 ways. The first option is via outpatient office referral, as described previously. Upon arrival to the ED, a second screening is performed to ensure the patient still meets EUA criteria for use and the positive COVID-19 PCR test result is confirmed. If the patient no longer meets criteria, the patient is triaged accordingly, including evaluation for higher-level care (eg, supplemental oxygen, hospital admission). The second optoion is via new patient walk-ins without outpatient physician referral (Figure 4). In these cases, an initial screening is performed, documenting EUA criteria for use in the preliminary assessment (Figure 5). Physicians will consider an outside COVID-19 test as valid, so long as documentation is readily available confirming a positive PCR result. Otherwise, an in-house COVID-19 PCR test will be performed, which has a 2-hour turnaround time.

Infusion Schedule. The ED offers a total of 16 COVID-19 monoclonal infusions slots daily. These are broken up into 4 infusion time blocks (eg, 8 am, 12 pm, 4 pm, 8 pm), with each infusion time block consisting of 4 available patient appointments. A list of scheduled infusions for the day is emailed to the pharmacy department every morning, and patients are instructed to arrive 1 hour prior to their appointment time. This allows time for patient registration, assessment, and pharmacy notification in advance of order entry. For logistical purposes, and as a patient safety initiative to reduce the likelihood of medication errors, each of the available COVID-19 monoclonal antibodies is offered on a designated day. Bamlanivimab is offered on Tuesday, Thursday, Saturday, and Sunday, while casirivimab/imdevimab is offered Monday, Wednesday, and Friday. This provides flexibility to adjust should supply deviate based on Department of Health allocation or should new therapy options within this class of medication become available.

Patient Education. Prior to administration of the monoclonal therapy, physician and nursing staff obtain a formal, written patient consent for therapy and provide patients with the option of participating in the institutional review board (IRB) approved study. Details of this are discussed in the risk management and IRB sections of the article. Nursing staff also provides the medication-specific Fact Sheet for Patients and Caregivers in either Spanish or English, which is also included as a hyperlink on the COVID-19 Monoclonal Antibody Order Set for ease of access. Interpreter services are available for patients who speak other languages. An ED decentralized pharmacist is also available onsite Monday through Friday from 12 pm to 8:30 pm to supplement education and serve as a resource for any questions.

Infusion Ordering. Once the patient is ready to begin therapy, the he/she is brought to a dedicated overflow area of the ED. There are few, if any, patients in this location, and it is adjacent to the main emergency center for easy access by the patients, nurses, pharmacists, and physicians. The physician then enters orders in the EHR using the COVID-19 Monoclonal Antibody Order Set (Figure 3). Three discrete questions were built into the medication order: (1) Was patient consent obtained? (2) Was the Fact Sheet for Patient/Caregiver provided to the patient? (3) Is the patient COVID-19 PCR-positive? These questions were built as hard stops so that the medication orders cannot be placed without a response. This serves as another double-check to ensure processes are followed and helps facilitate timely verification by the pharmacist.

Medication Administration. One nurse is dedicated to administering the monoclonal therapies scheduled at 8 am and 12 pm and another at 4 pm and 8 pm. Each appointment block is 4 hours in duration to allow adequate time for patient registration, infusion, and postinfusion observation. The nurse administers the premedications and COVID-19 monoclonal therapy, and observes the patient for the required 1-hour postadministration observation period. Nursing orders detailing monitoring parameters for mild, moderate, and severe reactions, along with associated medication orders to administer in the event they occur, are detailed in the nursing orders of the COVID-19 Monoclonal Antibody Order Set (Figure 3). Prior to administration, the nurse scans each medication and the patient’s wrist identification band, and documents the time of administration within the EHR medication administration report.

Pharmacy Department

Medication Receipt Process. Inventory is currently allocated biweekly from the state department of health and will soon be transitioning to a direct order system. The pharmacy technician in charge of deliveries notifies the pharmacy Antimicrobial Stewardship Program (ASP) clinical coordinator upon receipt of the monoclonal therapies. Bamlanivimab is supplied as 1 vial per dose, whereas casirivimab/imdevimab is supplied as 4 vials or 8 vials per dose, depending how it is shipped. To reduce the likelihood of medication errors, the ASP clinical coordinator assembles each of the casirivimab/imdevimab vials into kits, where 1 kit equals 1 dose. Labels are then affixed to each kit indicating the medication name, number of vials which equal a full dose, and pharmacist signature. The kits are stored in a dedicated refrigerator, and inventory logs are affixed to the outside of the refrigerator and updated daily. This inventory is also communicated daily to ED physician, nursing, and pharmacy leadership, as well as the director of patient safety, who reports weekly usage to the state Department of Health and Human Services. These weekly reports are used to determine allocation amounts.

Medication Verification and Delivery. The Mount Sinai Medical Center pharmacist staffing model consists of centralized order entry and specialized, decentralized positions. All orders are verified by the ED pharmacist when scheduled (not a 24/7 service) and by the designated pharmacist for all other times. At the time of medication verification, the pharmacist documents patient-specific EUA criteria for use and confirms that consent was obtained and the Fact Sheet for Patients/Caregivers was provided. A pharmacist intervention was developed to assist with this documentation. Pharmacists input smart text “.COVIDmonoclonal” and a drop-down menu of EUA criteria for use appears. The pharmacist reviews the patient care notes and medication order question responses to ascertain this information, contacting the ED prescriber if further clarification is required. This verification serves as another check to ensure processes put in place are followed. Lastly, intravenous preparation and delivery are electronically recorded in the EHR, and the medications require nursing signature at the time of delivery to ensure a formal chain of custody.

Risk Management

At Mount Sinai Medical Center, all EUA and investigational therapies require patient consent. Consistent with this requirement, a COVID-19 monoclonal specific consent was developed by risk management. This is provided to every patient receiving a COVID-19 monoclonal infusion, in addition to the FDA EUA Fact Sheet for Patients and Caregivers, and documented as part of their EHR. The questions providers must answer are built into the order set to ensure this process is followed and these patient safety checks are incorporated into the workflow.

Billing and Finance Department

In alignment with Mount Sinai Medical Center’s mission to provide high-quality health care to its diverse community through teaching, research, charity care, and financial responsibility, it was determined that this therapy would be provided to all patients regardless of insurance type, including those who are uninsured. The billing and finance department was consulted prior to this service being offered, to provide patients with accurate and pertinent information. The billing and finance department provided guidance on how to document patient encounters at time of registration to facilitate appropriate billing. At this time, the medication is free of charge, but nonmedication-related ED fees apply. This is explained to patients so there is a clear understanding prior to booking their appointment.

Infection Prevention

As patients receiving COVID-19 monoclonal therapies can transmit the virus to others, measures to ensure protection for other patients and staff are vital. To minimize exposure, specific nursing and physician staff from the ED are assigned to the treatment of these patients, and patients receive infusions and postobservation monitoring in a designated wing of the ED. Additionally, all staff who interact with these patients are required to don full personal protective equipment. This includes not only physicians and nurses but all specialties such as physician assistants, nurse practitioners, pharmacists, and laboratory technicians. Moreover, patients are not permitted to go home in a ride share and are counseled on Centers for Disease Control and Prevention quarantining following infusion.

Measurement of Process and Outcomes and Reporting

IRB approval was sought and obtained early during initiation of this service, allowing study consent to be offered to patients at the time general consent was obtained, which maximized patient recruitment and streamlined workflow. The study is a prospective observational research study to determine the impact of administration of COVID-19 monoclonal antibody therapy on length of symptoms, chronic illness, and rate of hospitalization. Most patients were eager to participate and offer their assistance to the scientific community during this pandemic.

Staff Education

In order to successfully implement this multidisciplinary EUA treatment option, comprehensive staff education was paramount after the workflow was developed. Prior to the first day of infusions, nurses and pharmacists were provided education during multiple huddle announcements. The pharmacy team also provided screen captures via email to the pharmacists so they could become familiar with the order set, intervention documentation, and location of the preliminary assessment of EUA criteria for use at the time of order verification. The emergency medicine department chair and chief medical officer also provided education via several virtual meetings and email to referring physicians (specialists and primary care) and residents in the emergency centers involved in COVID-19 monoclonal therapy-related patient care.

Factors Contributing to Success

We believe the reasons for continued success of this process are multifactorial and include the following key elements. Multidisciplinary planning, which included decision makers and all stakeholders, began at the time the idea was conceived. This allowed quick implementation of this service by efficiently navigating barriers to engaging impacted staff early on. Throughout this process, the authors set realistic step-wise goals. While navigating through the many details to implementation described, we also kept in mind the big picture, which was to provide this potentially lifesaving therapy to as many qualifying members of our community as possible. This included being flexible with the process and adapting when needed to achieve this ultimate goal. A focus on safety remained a priority to minimize possible errors and enhance patient and staff satisfaction. The optimization of the EHR streamlined workflow, provided point-of-care resources, and enhanced patient safety. Additionally, the target date set for implementation allowed staff and department leads adequate time to plan for and anticipate the changes. Serving only 1 patient on the first day allowed time for staff to experience this new process hands-on and provided opportunity for focused education. This team communication was essential to implementing this project, including staff training of processes and procedures prior to go-live. Early incorporation of IRB approval allowed the experience to be assessed and considered for contribution to the scientific literature to tackle this novel virus that has impacted our communities locally, nationally, and abroad. Moreover, continued measurement and reporting on a regular basis leads to performance improvement. The process outlined here can be adapted to incorporate other new therapies in the future, such as the recent February 9, 2021, EUA of the COVID-19 monoclonal antibody combination bamlanivimab and etesevimab.¹⁰

Conclusion

We administered 790 COVID-19 monoclonal antibody infusions between November 20, 2020 and March 5, 2021. Steps to minimize the likelihood of hypersensitivity reactions were implemented, and a low incidence (< 1%) has been observed. There has been no incidence of infection, concern from staff about infection prevention, or risk of infection during the processes. There have been very infrequent cost-related concerns raised by patients, typically due to incomplete communication regarding billing prior to the infusion. To address these issues, staff education has been provided to enhance patient instruction on this topic. The program has provided patient and family satisfaction, as well nursing, physician, pharmacist, clinical staff, and hospital administration pride and gratification. Setting up a new program to provide a 4-hour patient encounter to infuse therapy to high-risk patients with COVID-19 requires commitment and effort. This article describes the experience, ideas, and formula others may consider using to set up such a program. Through networking and formal phone calls and meetings about monoclonal antibody therapy, we have heard about other institutions who have not been able to institute this program due to various barriers to implementation. We hope our experience serves as a resource for others to provide this therapy to their patients and expand access in an effort to mitigate COVID-19 consequences and cases affecting our communities.

Corresponding author: Kathleen Jodoin, PharmD, BCPS, Mount Sinai Medical Center, 4300 Alton Rd, Miami Beach, FL 33140; [email protected].

Financial disclosures: None.

From Baylor College of Medicine, Houston, TX (Drs. Spezia-Lindner, Montealegre, Muldrew, and Suarez) and Harris Health System, Houston, TX (Shanna L. Harris, Maria Daheri, and Drs. Muldrew and Suarez).

Abstract

Objective: To characterize and analyze the prevalence, indications for, and outcomes of fecal immunochemical testing (FIT) in acute patient care within a safety net health care system’s emergency departments (EDs) and inpatient settings.

Design: Retrospective cohort study derived from administrative data.

Setting: A large, urban, safety net health care delivery system in Texas. The data gathered were from the health care system’s 2 primary hospitals and their associated EDs. This health care system utilizes FIT exclusively for fecal occult blood testing.

Participants: Adults ≥18 years who underwent FIT in the ED or inpatient setting between August 2016 and March 2017. Chart review abstractions were performed on a sample (n = 382) from the larger subset.

Measurements: Primary data points included total FITs performed in acute patient care during the study period, basic demographic data, FIT indications, FIT result, receipt of invasive diagnostic follow-up, and result of invasive diagnostic follow-up. Multivariable log-binomial regression was used to calculate risk ratios (RRs) to assess the association between FIT result and receipt of diagnostic follow-up. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result.

Results: During the 8-month study period, 2718 FITs were performed in the ED and inpatient setting, comprising 5.7% of system-wide FITs. Of the 382 patients included in the chart review who underwent acute care FIT, a majority had their test performed in the ED (304, 79.6%), 133 of which were positive (34.8%). The most common indication for FIT was evidence of overt gastrointestinal (GI) bleed (207, 54.2%), followed by anemia (84, 22.0%). While a positive FIT result was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72; P < 0.001), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00; P = 0.003). Of patients who underwent FIT and received diagnostic follow-up (n = 110), 48.2% were FIT negative. These patients were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86). Of the 382 patients in the study, 4 (1.0%) were subsequently diagnosed with colorectal cancer (CRC). Of those 4 patients, 1 (25%) was FIT positive.

Conclusion: FIT is being utilized in acute patient care outside of its established indication for CRC screening in asymptomatic, average-risk adults. Our study demonstrates that FIT is not useful in acute patient care.

Keywords: FOBT; FIT; fecal immunochemical testing; inpatient.

Despite the exclusive validation of FOBTs for use in CRC screening, studies have demonstrated that they are commonly used for a multitude of additional indications in emergency department (ED) and inpatient settings, most aimed at detecting or confirming GI blood loss. This may lead to inappropriate patient management, including the receipt of unnecessary follow-up procedures, which can incur significant costs to the patient and the health system.^13-19 These costs may be particularly burdensome in safety net health systems (ie, those that offer access to care regardless of the patient’s ability to pay), which serve a large proportion of socioeconomically disadvantaged individuals in the United States.^20,21 To our knowledge, no published study to date has specifically investigated the role of FIT in acute patient management.

This study characterizes the use of FIT in acute patient care within a large, urban, safety net health care system. Through a retrospective review of administrative data and patient charts, we evaluated FIT use prevalence, indications, and patient outcomes in the ED and inpatient settings.

Methods

Setting

This study was conducted in a large, urban, county-based integrated delivery system in Houston, Texas, that provides health care services to one of the largest uninsured and underinsured populations in the country.²² The health system includes 2 main hospitals and more than 20 ambulatory care clinics. Within its ambulatory care clinics, the health system implements a population-based screening strategy using stool-based testing. All adults aged 50 years or older who are due for FIT are identified through the health-maintenance module of the electronic medical record (EMR) and offered a take-home FIT. The health system utilizes FIT exclusively (OC-Light S FIT, Polymedco, Cortlandt Manor, NY); no guaiac-based assays are available.

Design and Data Collection

We began by using administrative records to determine the proportion of FITs conducted health system-wide that were ordered and completed in the acute care setting over the study period (August 2016-March 2017). Specifically, we used aggregate quality metric reports, which quantify the number of FITs conducted at each health system clinic and hospital each month, to calculate the proportion of FITs done in the ED and inpatient hospital setting.

We then conducted a retrospective cohort study of 382 adult patients who received FIT in the EDs and inpatient wards in both of the health system’s hospitals over the study period. All data were collected by retrospective chart review in Epic (Madison, WI) EMRs. Sampling was performed by selecting the medical record numbers corresponding to the first 50 completed FITs chronologically each month over the 8-month period, with a total of 400 charts reviewed.

Data collected included basic patient demographics, location of FIT ordering (ED vs inpatient), primary service ordering FIT, FIT indication, FIT result, and receipt and results of invasive diagnostic follow-up. Demographics collected included age, biological sex, race (self-selected), and insurance coverage.

FIT indication was determined based on resident or attending physician notes. The history of present illness, physical exam, and assessment and plan section of notes were reviewed by the lead author for a specific statement of indication for FIT or for evidence of clinical presentation for which FIT could reasonably be ordered. Indications were iteratively reviewed and collapsed into 6 different categories: anemia, iron deficiency with or without anemia, overt GIB, suspected GIB/miscellaneous, non-bloody diarrhea, and no indication identified. Overt GIB was defined as reported or witnessed hematemesis, coffee-ground emesis, hematochezia, bright red blood per rectum, or melena irrespective of time frame (current or remote) or chronicity (acute, subacute, or chronic). In cases where signs of overt bleed were not witnessed by medical professionals, determination of conditions such as melena or coffee-ground emesis were made based on health care providers’ assessment of patient history as documented in his or her notes. Suspected GIB/miscellaneous was defined with the following parameters: any new drop in hemoglobin, abdominal pain, anorectal pain, non-bloody vomiting, hemoptysis, isolated rising blood urea nitrogen, or patient noticing blood on self, clothing, or in the commode without an identified source. Patients who were anemic and found to have iron deficiency on recent lab studies (within 6 months) were reflexively categorized into iron deficiency with or without anemia as opposed to the “anemia” category, which was comprised of any anemia without recent iron studies or non-iron deficient anemia. FIT result was determined by test result entry in Epic, with results either reading positive or negative.

Diagnostic follow-up, for our purposes, was defined as receipt of an invasive procedure or surgery, including esophagogastroduodenoscopy (EGD), colonoscopy, flexible sigmoidoscopy, diagnostic and/or therapeutic abdominal surgical intervention, or any combination of these. Results of diagnostic follow-up were coded as normal or abnormal. A normal result was determined if all procedures performed were listed as normal or as “no pathological findings” on the operative or endoscopic report. Any reported pathologic findings on the operative/endoscopic report were coded as abnormal.

Statistical Analysis

Proportions were used to describe demographic characteristics of patients who received a FIT in acute hospital settings. Bivariable tables and Chi-square tests were used to compare indications and outcomes for FIT-positive and FIT-negative patients. The association between receipt of an invasive diagnostic follow-up (outcome) and the results of an inpatient FIT (predictor) was assessed using multivariable log-binomial regression to calculate risk ratios (RRs) and corresponding 95% confidence intervals. Log-binomial regression was used over logistic regression given that adjusted odds ratios generated by logistic regression often overestimate the association between the risk factor and the outcome when the outcome is common,²³ as in the case of diagnostic follow-up. The model was adjusted for variables selected a priori, specifically, age, gender, and FIT indication. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result (negative vs positive).

Results

During the 8-month study period, there were 2718 FITs ordered and completed in the acute care setting, compared to 44,662 FITs ordered and completed in the outpatient setting (5.7% performed during acute care).

Among the 400 charts reviewed, 7 were excluded from the analysis because they were duplicates from the same patient, and 11 were excluded due to insufficient information in the patient’s medical record, resulting in 382 patients included in the analysis. Patient demographic characteristics are described in Table 1. Patients were predominantly Hispanic/Latino or Black/African American (51.0% and 32.5%, respectively), a majority had insurance through the county health system (50.5%), and most were male (58.1%). The average age of those receiving FIT was 52 years (standard deviation, 14.8 years), with 40.8% being under the age of 50. For a majority of patients, FIT was ordered in the ED by emergency medicine providers (79.8%). The remaining FITs were ordered by providers in 12 different inpatient departments. Of the FITs ordered, 35.1% were positive.

Indications for ordering FIT are listed in Table 2. The largest proportion of FITs were ordered for overt signs of GIB (54.2%), followed by anemia (22.0%), suspected GIB/miscellaneous reasons (12.3%), iron deficiency with or without anemia (7.6%), and non-bloody diarrhea (2.1%). In 1.8% of cases, no indication for FIT was found in the EMR. No FITs were ordered for the indication of CRC detection. Of these indication categories, overt GIB yielded the highest percentage of FIT positive results (44.0%), and non-bloody diarrhea yielded the lowest (0%).

A total of 110 patients (28.7%) underwent FIT and received invasive diagnostic follow-up. Of these 110 patients, 57 (51.8%) underwent EGD (2 of whom had further surgical intervention), 21 (19.1%) underwent colonoscopy (1 of whom had further surgical intervention), 25 (22.7%) underwent dual EGD and colonoscopy, 1 (0.9%) underwent flexible sigmoidoscopy, and 6 (5.5%) directly underwent abdominal surgical intervention. There was a significantly higher rate of diagnostic follow-up for FIT-positive vs FIT-negative patients (42.9% vs 21.3%; P < 0.001). However, of the 110 patients who underwent subsequent diagnostic follow-up, 48.2% were FIT negative. FIT-negative patients who received diagnostic follow-up were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86).

Of the 382 patients in the study, 4 were diagnosed with CRC through diagnostic follow-up (1.0%). Of those 4 patients, 1 was FIT positive.

The results of the multivariable analyses to evaluate predictors of diagnostic colonoscopy are described in Table 3. Variables in the final model were FITresult, age, and FIT indication. After adjusting for other variables in the model, receipt of diagnostic follow-up was significantly associated with having a positive FIT (adjusted RR, 1.72; P < 0.001) and an overt GIB as an indication (adjusted RR, 2.00; P < 0.01).

Discussion

During the time frame of our study, 5.7% of all FITs ordered within our health system were ordered in the acute patient care setting at our hospitals. The most common indication was overt GIB, which was the indication for 54.2% of patients. Of note, none of the FITs ordered in the acute patient care setting were ordered for CRC screening. These findings support the evidence in the literature that stool-based screening tests, including FIT, are commonly used in US health care systems for diagnostic purposes and risk stratification in acute patient care to detect GIBs.^13-18

Our data suggest that FIT was not a clinically useful test in determining a patient’s need for diagnostic follow-up. While having a positive FIT was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00). This salient finding is evidence that a thorough clinical history and physical exam may more strongly predict whether a patient will undergo endoscopy or other follow-up than a FIT result. These findings support other studies in the literature that have called into question the utility of FOBTs in these acute settings.^13-19 Under such circumstances, FOBTs have been shown to rarely influence patient management and thus represent an unnecessary expense.^13–17 Additionally, in some cases, FOBT use in these settings may negatively affect patient outcomes. Such adverse effects include delaying treatment until results are returned or obfuscating indicated management with the results (eg, a patient with indications for colonoscopy not being referred due to a negative FOBT).^13,14,17

We found that, for patients who subsequently went on to have diagnostic follow-up (most commonly endoscopy), there was no difference in the likelihood of FIT-positive and FIT-negative patients to have an abnormality discovered (91.2% vs 90.6%; P = 0.86). This analysis demonstrates no post-hoc support for FIT positivity as a predictor of presence of pathology in patients who were discriminately selected for diagnostic follow-up on clinical grounds by gastroenterologists and surgeons. It does, however, further support that clinical judgment about the need for diagnostic follow-up—irrespective of FIT result—has a very high yield for discovery of pathology in the acute setting.

There are multiple reasons why FOBTs, and specifically FIT, contribute little in management decisions for patients with suspected GI blood loss. Use of FIT raises concern for both false-negatives and false-positives when used outside of its indication. Regarding false- negatives, FIT is an unreliable test for detection of blood loss from the upper GI tract. As FITs utilize antibodies to detect the presence of globin, a byproduct of red blood cell breakdown, it is expected that FIT would fail to detect many cases of upper GI bleeding, as globin is broken down in the upper GI tract.²⁴ This fact is part of what has made FIT a more effective CRC screening test than its guaiac-based counterparts—it has greater specificity for lower GI tract blood loss compared to tests relying on detection of heme.⁸ While guaiac-based assays like Hemoccult have also been shown to be poor tests in acute patient care, they may more frequently, though still unreliably, detect blood of upper GI origin. We believe that part of the ongoing use of FIT in patients with a suspected upper GIB may be from lack of understanding among providers on the mechanistic difference between gFOBTs and FITs, even though gFOBTs also yield highly unreliable results.

FIT does not have the same risk of false-positive results that guaiac-based tests have, which can yield positive results with extra-intestinal blood ingestion, aspirin, or alcohol use; insignificant GI bleeding; and consumption of peroxidase-containing foods.^13,17,25 However, from a clinical standpoint, there are several scenarios of insignificant bleeding that would yield a positive FIT result, such as hemorrhoids, which are common in the US population.^26,27 Additionally, in the ED, where most FITs were performed in our study, it is possible that samples for FITs are being obtained via digital rectal exam (DRE) given patients’ acuity of medical conditions and time constraints. However, FIT has been validated when using a formed stool sample. Obtaining FIT via DRE may lead to microtrauma to the rectum, which could hypothetically yield a positive FIT.

Strengths of this study include its use of in-depth chart data on a large number of FIT-positive patients, which allowed us to discern indications, outcomes, and other clinical data that may have influenced clinical decision-making. Additionally, whereas other studies that address FOBT use in acute patient care have focused on guaiac-based assays, our findings regarding the lack of utility of FIT are novel and have particular relevance as FITs continue to grow in popularity. Nonetheless, there are certain limitations future research should seek to address. In this study, the diagnostic follow-up result was coded by presence or absence of pathologic findings but did not qualify findings by severity or attempt to determine whether the pathology noted on diagnostic follow-up was the definitive source of the suspected GI bleed. These variables could help determine whether there was a difference in severity of bleeding between FIT-positive and FIT-negative patients and could potentially be studied with a prospective research design. Our own study was not designed to address the question of whether FIT result informs patient management decisions. To answer this directly, interviews would have to be conducted with those making the follow-up decision (ie, endoscopists and surgeons). Additionally, this study was not adequately powered to make determinations on the efficacy of FIT in the acute care setting for detection of CRC. As mentioned, only 1 of the 4 patients (25%) who went on to be diagnosed with CRC on follow-up was initially FIT-positive. This would require further investigation.

Conclusion

FIT is being utilized for diagnostic purposes in the acute care of symptomatic patients, which is a misuse of an established screening test for CRC. While our study was not designed to answer whether and how often a FIT result informs subsequent patient management, our results indicate that FIT is an ineffective diagnostic and risk-stratification tool when used in the acute care setting. Our findings add to existing evidence that indicates FOBTs should not be used in acute patient care.

Taken as a whole, the results of our study add to a growing body of evidence demonstrating no role for FOBTs, and specifically FIT, in acute patient care. In light of this evidence, some health care systems have already demonstrated success with system-wide disinvestment from the test in acute patient care settings, with one group publishing about their disinvestment process.²⁸ After completion of our study, our preliminary data were presented to leadership from the internal medicine, emergency medicine, and laboratory divisions within our health care delivery system to galvanize complete disinvestment of FIT from acute care at our hospitals, a policy that was put into effect in July 2019.

Corresponding author: Nathaniel J. Spezia-Lindner, MD, Baylor College of Medicine, 7200 Cambridge St, BCM 903, Ste A10.197, Houston, TX 77030; [email protected].

Financial disclosures: None.

Funding: Cancer Prevention and Research Institute of Texas, CPRIT (PP170094, PDs: ML Jibaja-Weiss and JR Montealegre).

From Baylor College of Medicine, Houston, TX (Drs. Spezia-Lindner, Montealegre, Muldrew, and Suarez) and Harris Health System, Houston, TX (Shanna L. Harris, Maria Daheri, and Drs. Muldrew and Suarez).

Abstract

Objective: To characterize and analyze the prevalence, indications for, and outcomes of fecal immunochemical testing (FIT) in acute patient care within a safety net health care system’s emergency departments (EDs) and inpatient settings.

Design: Retrospective cohort study derived from administrative data.

Setting: A large, urban, safety net health care delivery system in Texas. The data gathered were from the health care system’s 2 primary hospitals and their associated EDs. This health care system utilizes FIT exclusively for fecal occult blood testing.

Participants: Adults ≥18 years who underwent FIT in the ED or inpatient setting between August 2016 and March 2017. Chart review abstractions were performed on a sample (n = 382) from the larger subset.

Measurements: Primary data points included total FITs performed in acute patient care during the study period, basic demographic data, FIT indications, FIT result, receipt of invasive diagnostic follow-up, and result of invasive diagnostic follow-up. Multivariable log-binomial regression was used to calculate risk ratios (RRs) to assess the association between FIT result and receipt of diagnostic follow-up. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result.

Results: During the 8-month study period, 2718 FITs were performed in the ED and inpatient setting, comprising 5.7% of system-wide FITs. Of the 382 patients included in the chart review who underwent acute care FIT, a majority had their test performed in the ED (304, 79.6%), 133 of which were positive (34.8%). The most common indication for FIT was evidence of overt gastrointestinal (GI) bleed (207, 54.2%), followed by anemia (84, 22.0%). While a positive FIT result was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72; P < 0.001), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00; P = 0.003). Of patients who underwent FIT and received diagnostic follow-up (n = 110), 48.2% were FIT negative. These patients were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86). Of the 382 patients in the study, 4 (1.0%) were subsequently diagnosed with colorectal cancer (CRC). Of those 4 patients, 1 (25%) was FIT positive.

Conclusion: FIT is being utilized in acute patient care outside of its established indication for CRC screening in asymptomatic, average-risk adults. Our study demonstrates that FIT is not useful in acute patient care.

Keywords: FOBT; FIT; fecal immunochemical testing; inpatient.

Despite the exclusive validation of FOBTs for use in CRC screening, studies have demonstrated that they are commonly used for a multitude of additional indications in emergency department (ED) and inpatient settings, most aimed at detecting or confirming GI blood loss. This may lead to inappropriate patient management, including the receipt of unnecessary follow-up procedures, which can incur significant costs to the patient and the health system.^13-19 These costs may be particularly burdensome in safety net health systems (ie, those that offer access to care regardless of the patient’s ability to pay), which serve a large proportion of socioeconomically disadvantaged individuals in the United States.^20,21 To our knowledge, no published study to date has specifically investigated the role of FIT in acute patient management.

This study characterizes the use of FIT in acute patient care within a large, urban, safety net health care system. Through a retrospective review of administrative data and patient charts, we evaluated FIT use prevalence, indications, and patient outcomes in the ED and inpatient settings.

Methods

Setting

This study was conducted in a large, urban, county-based integrated delivery system in Houston, Texas, that provides health care services to one of the largest uninsured and underinsured populations in the country.²² The health system includes 2 main hospitals and more than 20 ambulatory care clinics. Within its ambulatory care clinics, the health system implements a population-based screening strategy using stool-based testing. All adults aged 50 years or older who are due for FIT are identified through the health-maintenance module of the electronic medical record (EMR) and offered a take-home FIT. The health system utilizes FIT exclusively (OC-Light S FIT, Polymedco, Cortlandt Manor, NY); no guaiac-based assays are available.

Design and Data Collection

We began by using administrative records to determine the proportion of FITs conducted health system-wide that were ordered and completed in the acute care setting over the study period (August 2016-March 2017). Specifically, we used aggregate quality metric reports, which quantify the number of FITs conducted at each health system clinic and hospital each month, to calculate the proportion of FITs done in the ED and inpatient hospital setting.

We then conducted a retrospective cohort study of 382 adult patients who received FIT in the EDs and inpatient wards in both of the health system’s hospitals over the study period. All data were collected by retrospective chart review in Epic (Madison, WI) EMRs. Sampling was performed by selecting the medical record numbers corresponding to the first 50 completed FITs chronologically each month over the 8-month period, with a total of 400 charts reviewed.

Data collected included basic patient demographics, location of FIT ordering (ED vs inpatient), primary service ordering FIT, FIT indication, FIT result, and receipt and results of invasive diagnostic follow-up. Demographics collected included age, biological sex, race (self-selected), and insurance coverage.

FIT indication was determined based on resident or attending physician notes. The history of present illness, physical exam, and assessment and plan section of notes were reviewed by the lead author for a specific statement of indication for FIT or for evidence of clinical presentation for which FIT could reasonably be ordered. Indications were iteratively reviewed and collapsed into 6 different categories: anemia, iron deficiency with or without anemia, overt GIB, suspected GIB/miscellaneous, non-bloody diarrhea, and no indication identified. Overt GIB was defined as reported or witnessed hematemesis, coffee-ground emesis, hematochezia, bright red blood per rectum, or melena irrespective of time frame (current or remote) or chronicity (acute, subacute, or chronic). In cases where signs of overt bleed were not witnessed by medical professionals, determination of conditions such as melena or coffee-ground emesis were made based on health care providers’ assessment of patient history as documented in his or her notes. Suspected GIB/miscellaneous was defined with the following parameters: any new drop in hemoglobin, abdominal pain, anorectal pain, non-bloody vomiting, hemoptysis, isolated rising blood urea nitrogen, or patient noticing blood on self, clothing, or in the commode without an identified source. Patients who were anemic and found to have iron deficiency on recent lab studies (within 6 months) were reflexively categorized into iron deficiency with or without anemia as opposed to the “anemia” category, which was comprised of any anemia without recent iron studies or non-iron deficient anemia. FIT result was determined by test result entry in Epic, with results either reading positive or negative.

Diagnostic follow-up, for our purposes, was defined as receipt of an invasive procedure or surgery, including esophagogastroduodenoscopy (EGD), colonoscopy, flexible sigmoidoscopy, diagnostic and/or therapeutic abdominal surgical intervention, or any combination of these. Results of diagnostic follow-up were coded as normal or abnormal. A normal result was determined if all procedures performed were listed as normal or as “no pathological findings” on the operative or endoscopic report. Any reported pathologic findings on the operative/endoscopic report were coded as abnormal.

Statistical Analysis

Proportions were used to describe demographic characteristics of patients who received a FIT in acute hospital settings. Bivariable tables and Chi-square tests were used to compare indications and outcomes for FIT-positive and FIT-negative patients. The association between receipt of an invasive diagnostic follow-up (outcome) and the results of an inpatient FIT (predictor) was assessed using multivariable log-binomial regression to calculate risk ratios (RRs) and corresponding 95% confidence intervals. Log-binomial regression was used over logistic regression given that adjusted odds ratios generated by logistic regression often overestimate the association between the risk factor and the outcome when the outcome is common,²³ as in the case of diagnostic follow-up. The model was adjusted for variables selected a priori, specifically, age, gender, and FIT indication. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result (negative vs positive).

Results

During the 8-month study period, there were 2718 FITs ordered and completed in the acute care setting, compared to 44,662 FITs ordered and completed in the outpatient setting (5.7% performed during acute care).

Among the 400 charts reviewed, 7 were excluded from the analysis because they were duplicates from the same patient, and 11 were excluded due to insufficient information in the patient’s medical record, resulting in 382 patients included in the analysis. Patient demographic characteristics are described in Table 1. Patients were predominantly Hispanic/Latino or Black/African American (51.0% and 32.5%, respectively), a majority had insurance through the county health system (50.5%), and most were male (58.1%). The average age of those receiving FIT was 52 years (standard deviation, 14.8 years), with 40.8% being under the age of 50. For a majority of patients, FIT was ordered in the ED by emergency medicine providers (79.8%). The remaining FITs were ordered by providers in 12 different inpatient departments. Of the FITs ordered, 35.1% were positive.

Indications for ordering FIT are listed in Table 2. The largest proportion of FITs were ordered for overt signs of GIB (54.2%), followed by anemia (22.0%), suspected GIB/miscellaneous reasons (12.3%), iron deficiency with or without anemia (7.6%), and non-bloody diarrhea (2.1%). In 1.8% of cases, no indication for FIT was found in the EMR. No FITs were ordered for the indication of CRC detection. Of these indication categories, overt GIB yielded the highest percentage of FIT positive results (44.0%), and non-bloody diarrhea yielded the lowest (0%).

A total of 110 patients (28.7%) underwent FIT and received invasive diagnostic follow-up. Of these 110 patients, 57 (51.8%) underwent EGD (2 of whom had further surgical intervention), 21 (19.1%) underwent colonoscopy (1 of whom had further surgical intervention), 25 (22.7%) underwent dual EGD and colonoscopy, 1 (0.9%) underwent flexible sigmoidoscopy, and 6 (5.5%) directly underwent abdominal surgical intervention. There was a significantly higher rate of diagnostic follow-up for FIT-positive vs FIT-negative patients (42.9% vs 21.3%; P < 0.001). However, of the 110 patients who underwent subsequent diagnostic follow-up, 48.2% were FIT negative. FIT-negative patients who received diagnostic follow-up were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86).

Of the 382 patients in the study, 4 were diagnosed with CRC through diagnostic follow-up (1.0%). Of those 4 patients, 1 was FIT positive.

The results of the multivariable analyses to evaluate predictors of diagnostic colonoscopy are described in Table 3. Variables in the final model were FITresult, age, and FIT indication. After adjusting for other variables in the model, receipt of diagnostic follow-up was significantly associated with having a positive FIT (adjusted RR, 1.72; P < 0.001) and an overt GIB as an indication (adjusted RR, 2.00; P < 0.01).

Discussion

During the time frame of our study, 5.7% of all FITs ordered within our health system were ordered in the acute patient care setting at our hospitals. The most common indication was overt GIB, which was the indication for 54.2% of patients. Of note, none of the FITs ordered in the acute patient care setting were ordered for CRC screening. These findings support the evidence in the literature that stool-based screening tests, including FIT, are commonly used in US health care systems for diagnostic purposes and risk stratification in acute patient care to detect GIBs.^13-18

Our data suggest that FIT was not a clinically useful test in determining a patient’s need for diagnostic follow-up. While having a positive FIT was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00). This salient finding is evidence that a thorough clinical history and physical exam may more strongly predict whether a patient will undergo endoscopy or other follow-up than a FIT result. These findings support other studies in the literature that have called into question the utility of FOBTs in these acute settings.^13-19 Under such circumstances, FOBTs have been shown to rarely influence patient management and thus represent an unnecessary expense.^13–17 Additionally, in some cases, FOBT use in these settings may negatively affect patient outcomes. Such adverse effects include delaying treatment until results are returned or obfuscating indicated management with the results (eg, a patient with indications for colonoscopy not being referred due to a negative FOBT).^13,14,17

We found that, for patients who subsequently went on to have diagnostic follow-up (most commonly endoscopy), there was no difference in the likelihood of FIT-positive and FIT-negative patients to have an abnormality discovered (91.2% vs 90.6%; P = 0.86). This analysis demonstrates no post-hoc support for FIT positivity as a predictor of presence of pathology in patients who were discriminately selected for diagnostic follow-up on clinical grounds by gastroenterologists and surgeons. It does, however, further support that clinical judgment about the need for diagnostic follow-up—irrespective of FIT result—has a very high yield for discovery of pathology in the acute setting.

There are multiple reasons why FOBTs, and specifically FIT, contribute little in management decisions for patients with suspected GI blood loss. Use of FIT raises concern for both false-negatives and false-positives when used outside of its indication. Regarding false- negatives, FIT is an unreliable test for detection of blood loss from the upper GI tract. As FITs utilize antibodies to detect the presence of globin, a byproduct of red blood cell breakdown, it is expected that FIT would fail to detect many cases of upper GI bleeding, as globin is broken down in the upper GI tract.²⁴ This fact is part of what has made FIT a more effective CRC screening test than its guaiac-based counterparts—it has greater specificity for lower GI tract blood loss compared to tests relying on detection of heme.⁸ While guaiac-based assays like Hemoccult have also been shown to be poor tests in acute patient care, they may more frequently, though still unreliably, detect blood of upper GI origin. We believe that part of the ongoing use of FIT in patients with a suspected upper GIB may be from lack of understanding among providers on the mechanistic difference between gFOBTs and FITs, even though gFOBTs also yield highly unreliable results.

FIT does not have the same risk of false-positive results that guaiac-based tests have, which can yield positive results with extra-intestinal blood ingestion, aspirin, or alcohol use; insignificant GI bleeding; and consumption of peroxidase-containing foods.^13,17,25 However, from a clinical standpoint, there are several scenarios of insignificant bleeding that would yield a positive FIT result, such as hemorrhoids, which are common in the US population.^26,27 Additionally, in the ED, where most FITs were performed in our study, it is possible that samples for FITs are being obtained via digital rectal exam (DRE) given patients’ acuity of medical conditions and time constraints. However, FIT has been validated when using a formed stool sample. Obtaining FIT via DRE may lead to microtrauma to the rectum, which could hypothetically yield a positive FIT.

Strengths of this study include its use of in-depth chart data on a large number of FIT-positive patients, which allowed us to discern indications, outcomes, and other clinical data that may have influenced clinical decision-making. Additionally, whereas other studies that address FOBT use in acute patient care have focused on guaiac-based assays, our findings regarding the lack of utility of FIT are novel and have particular relevance as FITs continue to grow in popularity. Nonetheless, there are certain limitations future research should seek to address. In this study, the diagnostic follow-up result was coded by presence or absence of pathologic findings but did not qualify findings by severity or attempt to determine whether the pathology noted on diagnostic follow-up was the definitive source of the suspected GI bleed. These variables could help determine whether there was a difference in severity of bleeding between FIT-positive and FIT-negative patients and could potentially be studied with a prospective research design. Our own study was not designed to address the question of whether FIT result informs patient management decisions. To answer this directly, interviews would have to be conducted with those making the follow-up decision (ie, endoscopists and surgeons). Additionally, this study was not adequately powered to make determinations on the efficacy of FIT in the acute care setting for detection of CRC. As mentioned, only 1 of the 4 patients (25%) who went on to be diagnosed with CRC on follow-up was initially FIT-positive. This would require further investigation.

Conclusion

FIT is being utilized for diagnostic purposes in the acute care of symptomatic patients, which is a misuse of an established screening test for CRC. While our study was not designed to answer whether and how often a FIT result informs subsequent patient management, our results indicate that FIT is an ineffective diagnostic and risk-stratification tool when used in the acute care setting. Our findings add to existing evidence that indicates FOBTs should not be used in acute patient care.

Taken as a whole, the results of our study add to a growing body of evidence demonstrating no role for FOBTs, and specifically FIT, in acute patient care. In light of this evidence, some health care systems have already demonstrated success with system-wide disinvestment from the test in acute patient care settings, with one group publishing about their disinvestment process.²⁸ After completion of our study, our preliminary data were presented to leadership from the internal medicine, emergency medicine, and laboratory divisions within our health care delivery system to galvanize complete disinvestment of FIT from acute care at our hospitals, a policy that was put into effect in July 2019.

Corresponding author: Nathaniel J. Spezia-Lindner, MD, Baylor College of Medicine, 7200 Cambridge St, BCM 903, Ste A10.197, Houston, TX 77030; [email protected].

Financial disclosures: None.

Funding: Cancer Prevention and Research Institute of Texas, CPRIT (PP170094, PDs: ML Jibaja-Weiss and JR Montealegre).

From Baylor College of Medicine, Houston, TX (Drs. Spezia-Lindner, Montealegre, Muldrew, and Suarez) and Harris Health System, Houston, TX (Shanna L. Harris, Maria Daheri, and Drs. Muldrew and Suarez).

Abstract

Objective: To characterize and analyze the prevalence, indications for, and outcomes of fecal immunochemical testing (FIT) in acute patient care within a safety net health care system’s emergency departments (EDs) and inpatient settings.

Design: Retrospective cohort study derived from administrative data.

Setting: A large, urban, safety net health care delivery system in Texas. The data gathered were from the health care system’s 2 primary hospitals and their associated EDs. This health care system utilizes FIT exclusively for fecal occult blood testing.

Participants: Adults ≥18 years who underwent FIT in the ED or inpatient setting between August 2016 and March 2017. Chart review abstractions were performed on a sample (n = 382) from the larger subset.

Measurements: Primary data points included total FITs performed in acute patient care during the study period, basic demographic data, FIT indications, FIT result, receipt of invasive diagnostic follow-up, and result of invasive diagnostic follow-up. Multivariable log-binomial regression was used to calculate risk ratios (RRs) to assess the association between FIT result and receipt of diagnostic follow-up. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result.

Results: During the 8-month study period, 2718 FITs were performed in the ED and inpatient setting, comprising 5.7% of system-wide FITs. Of the 382 patients included in the chart review who underwent acute care FIT, a majority had their test performed in the ED (304, 79.6%), 133 of which were positive (34.8%). The most common indication for FIT was evidence of overt gastrointestinal (GI) bleed (207, 54.2%), followed by anemia (84, 22.0%). While a positive FIT result was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72; P < 0.001), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00; P = 0.003). Of patients who underwent FIT and received diagnostic follow-up (n = 110), 48.2% were FIT negative. These patients were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86). Of the 382 patients in the study, 4 (1.0%) were subsequently diagnosed with colorectal cancer (CRC). Of those 4 patients, 1 (25%) was FIT positive.

Conclusion: FIT is being utilized in acute patient care outside of its established indication for CRC screening in asymptomatic, average-risk adults. Our study demonstrates that FIT is not useful in acute patient care.

Keywords: FOBT; FIT; fecal immunochemical testing; inpatient.

Despite the exclusive validation of FOBTs for use in CRC screening, studies have demonstrated that they are commonly used for a multitude of additional indications in emergency department (ED) and inpatient settings, most aimed at detecting or confirming GI blood loss. This may lead to inappropriate patient management, including the receipt of unnecessary follow-up procedures, which can incur significant costs to the patient and the health system.^13-19 These costs may be particularly burdensome in safety net health systems (ie, those that offer access to care regardless of the patient’s ability to pay), which serve a large proportion of socioeconomically disadvantaged individuals in the United States.^20,21 To our knowledge, no published study to date has specifically investigated the role of FIT in acute patient management.

This study characterizes the use of FIT in acute patient care within a large, urban, safety net health care system. Through a retrospective review of administrative data and patient charts, we evaluated FIT use prevalence, indications, and patient outcomes in the ED and inpatient settings.

Methods

Setting

This study was conducted in a large, urban, county-based integrated delivery system in Houston, Texas, that provides health care services to one of the largest uninsured and underinsured populations in the country.²² The health system includes 2 main hospitals and more than 20 ambulatory care clinics. Within its ambulatory care clinics, the health system implements a population-based screening strategy using stool-based testing. All adults aged 50 years or older who are due for FIT are identified through the health-maintenance module of the electronic medical record (EMR) and offered a take-home FIT. The health system utilizes FIT exclusively (OC-Light S FIT, Polymedco, Cortlandt Manor, NY); no guaiac-based assays are available.

Design and Data Collection

We began by using administrative records to determine the proportion of FITs conducted health system-wide that were ordered and completed in the acute care setting over the study period (August 2016-March 2017). Specifically, we used aggregate quality metric reports, which quantify the number of FITs conducted at each health system clinic and hospital each month, to calculate the proportion of FITs done in the ED and inpatient hospital setting.

We then conducted a retrospective cohort study of 382 adult patients who received FIT in the EDs and inpatient wards in both of the health system’s hospitals over the study period. All data were collected by retrospective chart review in Epic (Madison, WI) EMRs. Sampling was performed by selecting the medical record numbers corresponding to the first 50 completed FITs chronologically each month over the 8-month period, with a total of 400 charts reviewed.

Data collected included basic patient demographics, location of FIT ordering (ED vs inpatient), primary service ordering FIT, FIT indication, FIT result, and receipt and results of invasive diagnostic follow-up. Demographics collected included age, biological sex, race (self-selected), and insurance coverage.

FIT indication was determined based on resident or attending physician notes. The history of present illness, physical exam, and assessment and plan section of notes were reviewed by the lead author for a specific statement of indication for FIT or for evidence of clinical presentation for which FIT could reasonably be ordered. Indications were iteratively reviewed and collapsed into 6 different categories: anemia, iron deficiency with or without anemia, overt GIB, suspected GIB/miscellaneous, non-bloody diarrhea, and no indication identified. Overt GIB was defined as reported or witnessed hematemesis, coffee-ground emesis, hematochezia, bright red blood per rectum, or melena irrespective of time frame (current or remote) or chronicity (acute, subacute, or chronic). In cases where signs of overt bleed were not witnessed by medical professionals, determination of conditions such as melena or coffee-ground emesis were made based on health care providers’ assessment of patient history as documented in his or her notes. Suspected GIB/miscellaneous was defined with the following parameters: any new drop in hemoglobin, abdominal pain, anorectal pain, non-bloody vomiting, hemoptysis, isolated rising blood urea nitrogen, or patient noticing blood on self, clothing, or in the commode without an identified source. Patients who were anemic and found to have iron deficiency on recent lab studies (within 6 months) were reflexively categorized into iron deficiency with or without anemia as opposed to the “anemia” category, which was comprised of any anemia without recent iron studies or non-iron deficient anemia. FIT result was determined by test result entry in Epic, with results either reading positive or negative.

Diagnostic follow-up, for our purposes, was defined as receipt of an invasive procedure or surgery, including esophagogastroduodenoscopy (EGD), colonoscopy, flexible sigmoidoscopy, diagnostic and/or therapeutic abdominal surgical intervention, or any combination of these. Results of diagnostic follow-up were coded as normal or abnormal. A normal result was determined if all procedures performed were listed as normal or as “no pathological findings” on the operative or endoscopic report. Any reported pathologic findings on the operative/endoscopic report were coded as abnormal.

Statistical Analysis

Proportions were used to describe demographic characteristics of patients who received a FIT in acute hospital settings. Bivariable tables and Chi-square tests were used to compare indications and outcomes for FIT-positive and FIT-negative patients. The association between receipt of an invasive diagnostic follow-up (outcome) and the results of an inpatient FIT (predictor) was assessed using multivariable log-binomial regression to calculate risk ratios (RRs) and corresponding 95% confidence intervals. Log-binomial regression was used over logistic regression given that adjusted odds ratios generated by logistic regression often overestimate the association between the risk factor and the outcome when the outcome is common,²³ as in the case of diagnostic follow-up. The model was adjusted for variables selected a priori, specifically, age, gender, and FIT indication. Chi-square analysis was used to compare the proportion of abnormal findings on diagnostic follow-up by FIT result (negative vs positive).

Results

During the 8-month study period, there were 2718 FITs ordered and completed in the acute care setting, compared to 44,662 FITs ordered and completed in the outpatient setting (5.7% performed during acute care).

Among the 400 charts reviewed, 7 were excluded from the analysis because they were duplicates from the same patient, and 11 were excluded due to insufficient information in the patient’s medical record, resulting in 382 patients included in the analysis. Patient demographic characteristics are described in Table 1. Patients were predominantly Hispanic/Latino or Black/African American (51.0% and 32.5%, respectively), a majority had insurance through the county health system (50.5%), and most were male (58.1%). The average age of those receiving FIT was 52 years (standard deviation, 14.8 years), with 40.8% being under the age of 50. For a majority of patients, FIT was ordered in the ED by emergency medicine providers (79.8%). The remaining FITs were ordered by providers in 12 different inpatient departments. Of the FITs ordered, 35.1% were positive.

Indications for ordering FIT are listed in Table 2. The largest proportion of FITs were ordered for overt signs of GIB (54.2%), followed by anemia (22.0%), suspected GIB/miscellaneous reasons (12.3%), iron deficiency with or without anemia (7.6%), and non-bloody diarrhea (2.1%). In 1.8% of cases, no indication for FIT was found in the EMR. No FITs were ordered for the indication of CRC detection. Of these indication categories, overt GIB yielded the highest percentage of FIT positive results (44.0%), and non-bloody diarrhea yielded the lowest (0%).

A total of 110 patients (28.7%) underwent FIT and received invasive diagnostic follow-up. Of these 110 patients, 57 (51.8%) underwent EGD (2 of whom had further surgical intervention), 21 (19.1%) underwent colonoscopy (1 of whom had further surgical intervention), 25 (22.7%) underwent dual EGD and colonoscopy, 1 (0.9%) underwent flexible sigmoidoscopy, and 6 (5.5%) directly underwent abdominal surgical intervention. There was a significantly higher rate of diagnostic follow-up for FIT-positive vs FIT-negative patients (42.9% vs 21.3%; P < 0.001). However, of the 110 patients who underwent subsequent diagnostic follow-up, 48.2% were FIT negative. FIT-negative patients who received diagnostic follow-up were just as likely to have an abnormal finding as FIT-positive patients (90.6% vs 91.2%; P = 0.86).

Of the 382 patients in the study, 4 were diagnosed with CRC through diagnostic follow-up (1.0%). Of those 4 patients, 1 was FIT positive.

The results of the multivariable analyses to evaluate predictors of diagnostic colonoscopy are described in Table 3. Variables in the final model were FITresult, age, and FIT indication. After adjusting for other variables in the model, receipt of diagnostic follow-up was significantly associated with having a positive FIT (adjusted RR, 1.72; P < 0.001) and an overt GIB as an indication (adjusted RR, 2.00; P < 0.01).

Discussion

During the time frame of our study, 5.7% of all FITs ordered within our health system were ordered in the acute patient care setting at our hospitals. The most common indication was overt GIB, which was the indication for 54.2% of patients. Of note, none of the FITs ordered in the acute patient care setting were ordered for CRC screening. These findings support the evidence in the literature that stool-based screening tests, including FIT, are commonly used in US health care systems for diagnostic purposes and risk stratification in acute patient care to detect GIBs.^13-18

Our data suggest that FIT was not a clinically useful test in determining a patient’s need for diagnostic follow-up. While having a positive FIT was significantly associated with obtaining a diagnostic exam in multivariate analysis (RR, 1.72), having signs of overt GI bleeding was a stronger predictor of diagnostic follow-up (RR, 2.00). This salient finding is evidence that a thorough clinical history and physical exam may more strongly predict whether a patient will undergo endoscopy or other follow-up than a FIT result. These findings support other studies in the literature that have called into question the utility of FOBTs in these acute settings.^13-19 Under such circumstances, FOBTs have been shown to rarely influence patient management and thus represent an unnecessary expense.^13–17 Additionally, in some cases, FOBT use in these settings may negatively affect patient outcomes. Such adverse effects include delaying treatment until results are returned or obfuscating indicated management with the results (eg, a patient with indications for colonoscopy not being referred due to a negative FOBT).^13,14,17

We found that, for patients who subsequently went on to have diagnostic follow-up (most commonly endoscopy), there was no difference in the likelihood of FIT-positive and FIT-negative patients to have an abnormality discovered (91.2% vs 90.6%; P = 0.86). This analysis demonstrates no post-hoc support for FIT positivity as a predictor of presence of pathology in patients who were discriminately selected for diagnostic follow-up on clinical grounds by gastroenterologists and surgeons. It does, however, further support that clinical judgment about the need for diagnostic follow-up—irrespective of FIT result—has a very high yield for discovery of pathology in the acute setting.

There are multiple reasons why FOBTs, and specifically FIT, contribute little in management decisions for patients with suspected GI blood loss. Use of FIT raises concern for both false-negatives and false-positives when used outside of its indication. Regarding false- negatives, FIT is an unreliable test for detection of blood loss from the upper GI tract. As FITs utilize antibodies to detect the presence of globin, a byproduct of red blood cell breakdown, it is expected that FIT would fail to detect many cases of upper GI bleeding, as globin is broken down in the upper GI tract.²⁴ This fact is part of what has made FIT a more effective CRC screening test than its guaiac-based counterparts—it has greater specificity for lower GI tract blood loss compared to tests relying on detection of heme.⁸ While guaiac-based assays like Hemoccult have also been shown to be poor tests in acute patient care, they may more frequently, though still unreliably, detect blood of upper GI origin. We believe that part of the ongoing use of FIT in patients with a suspected upper GIB may be from lack of understanding among providers on the mechanistic difference between gFOBTs and FITs, even though gFOBTs also yield highly unreliable results.

FIT does not have the same risk of false-positive results that guaiac-based tests have, which can yield positive results with extra-intestinal blood ingestion, aspirin, or alcohol use; insignificant GI bleeding; and consumption of peroxidase-containing foods.^13,17,25 However, from a clinical standpoint, there are several scenarios of insignificant bleeding that would yield a positive FIT result, such as hemorrhoids, which are common in the US population.^26,27 Additionally, in the ED, where most FITs were performed in our study, it is possible that samples for FITs are being obtained via digital rectal exam (DRE) given patients’ acuity of medical conditions and time constraints. However, FIT has been validated when using a formed stool sample. Obtaining FIT via DRE may lead to microtrauma to the rectum, which could hypothetically yield a positive FIT.

Strengths of this study include its use of in-depth chart data on a large number of FIT-positive patients, which allowed us to discern indications, outcomes, and other clinical data that may have influenced clinical decision-making. Additionally, whereas other studies that address FOBT use in acute patient care have focused on guaiac-based assays, our findings regarding the lack of utility of FIT are novel and have particular relevance as FITs continue to grow in popularity. Nonetheless, there are certain limitations future research should seek to address. In this study, the diagnostic follow-up result was coded by presence or absence of pathologic findings but did not qualify findings by severity or attempt to determine whether the pathology noted on diagnostic follow-up was the definitive source of the suspected GI bleed. These variables could help determine whether there was a difference in severity of bleeding between FIT-positive and FIT-negative patients and could potentially be studied with a prospective research design. Our own study was not designed to address the question of whether FIT result informs patient management decisions. To answer this directly, interviews would have to be conducted with those making the follow-up decision (ie, endoscopists and surgeons). Additionally, this study was not adequately powered to make determinations on the efficacy of FIT in the acute care setting for detection of CRC. As mentioned, only 1 of the 4 patients (25%) who went on to be diagnosed with CRC on follow-up was initially FIT-positive. This would require further investigation.

Conclusion

FIT is being utilized for diagnostic purposes in the acute care of symptomatic patients, which is a misuse of an established screening test for CRC. While our study was not designed to answer whether and how often a FIT result informs subsequent patient management, our results indicate that FIT is an ineffective diagnostic and risk-stratification tool when used in the acute care setting. Our findings add to existing evidence that indicates FOBTs should not be used in acute patient care.

Taken as a whole, the results of our study add to a growing body of evidence demonstrating no role for FOBTs, and specifically FIT, in acute patient care. In light of this evidence, some health care systems have already demonstrated success with system-wide disinvestment from the test in acute patient care settings, with one group publishing about their disinvestment process.²⁸ After completion of our study, our preliminary data were presented to leadership from the internal medicine, emergency medicine, and laboratory divisions within our health care delivery system to galvanize complete disinvestment of FIT from acute care at our hospitals, a policy that was put into effect in July 2019.

Corresponding author: Nathaniel J. Spezia-Lindner, MD, Baylor College of Medicine, 7200 Cambridge St, BCM 903, Ste A10.197, Houston, TX 77030; [email protected].

Financial disclosures: None.

Funding: Cancer Prevention and Research Institute of Texas, CPRIT (PP170094, PDs: ML Jibaja-Weiss and JR Montealegre).

From The Johns Hopkins Hospital, Baltimore, MD (Dr. Muganlinskaya and Dr. Skojec, retired); The George Washington University, Washington, DC (Dr. Posey); and Johns Hopkins University, Baltimore, MD (Dr. Resar).

Abstract

Objective: Assessing the risk characteristics of patients with acute myocardial infarction (MI) can help providers make appropriate referral decisions. This quality improvement project sought to improve timely, appropriate referrals among patients with type I MI by adding a risk assessment, the AMI READMITS score, to the existing referral protocol.

Methods: Patients’ chart data were analyzed to assess changes in referrals and timely follow-up appointments from pre-intervention to intervention. A survey assessed providers’ satisfaction with the new referral protocol.

Results: Among 57 patients (n = 29 preintervention; n = 28 intervention), documented referrals increased significantly from 66% to 89% (χ2 = 4.571, df = 1, P = 0.033); and timely appointments increased by 10%, which was not significant (χ2 = 3.550, df = 2, P = 0.169). Most providers agreed that the new protocol was easy to use, useful in making referral decisions, and improved the referral process. All agreed the risk score should be incorporated into electronic clinical notes. Provider opinions related to implementing the risk score in clinical practice were mixed. Qualitative feedback suggests this was due to limited validation of the AMI READMITS score in reducing readmissions.

Conclusions: Our risk-based referral protocol helped to increase appropriate referrals among patients with type I MI. Provider adoption may be enhanced by incorporating the protocol into electronic clinical notes. Research to further validate the accuracy of the AMI READMITS score in predicting readmissions may support adoption of the protocol in clinical practice.

Keywords: quality improvement; type I myocardial infarction; referral process; readmission risk; risk assessment; chart review.

Early follow-up after discharge is an important strategy to reduce the risk of unplanned hospital readmissions among patients with various conditions.^1-3 While patient confounding factors, such as chronic health problems, environment, socioeconomic status, and literacy, make it difficult to avoid all unplanned readmissions, early follow-up may help providers identify and appropriately manage some health-related issues, and as such is a pivotal element of a readmission prevention strategy.⁴ There is evidence that patients with non-ST elevation myocardial infarction (NSTEMI) who have an outpatient appointment with a physician within 7 days after discharge have a lower risk of 30-day readmission.⁵

Our hospital’s postmyocardial infarction clinic was created to prevent unplanned readmissions within 30 days after discharge among patients with type I myocardial infarction (MI). Since inception, the number of referrals has been much lower than expected. In 2018, the total number of patients discharged from the hospital with type I MI and any troponin I level above 0.40 ng/mL was 313. Most of these patients were discharged from the hospital’s cardiac units; however, only 91 referrals were made. To increase referrals, the cardiology nurse practitioners (NPs) developed a post-MI referral protocol (Figure 1). However, this protocol was not consistently used and referrals to the clinic remained low.

Evidence-based risk assessment tools have the potential to increase effective patient management. For example, cardiology providers at the hospital utilize various scores, such as CHA₂DS₂-VASc⁶ and the Society of Thoracic Surgery risk score,⁷ to plan patient management. Among the scores used to predict unplanned readmissions for MI patients, the most promising is the AMI READMITS score.⁸ Unlike other nonspecific prediction models, the AMI READMITS score was developed based on variables extracted from the electronic health records (EHRs) of patients who were hospitalized for MI and readmitted within 30 days after discharge. Recognizing the potential to increase referrals by integrating an MI-specific risk assessment, this quality improvement study modified the existing referral protocol to include the patients’ AMI READMITS score and recommendations for follow-up.

Currently, there are no clear recommendations on how soon after discharge patients with MI should undergo follow-up. As research data vary, we selected 7 days follow-up for patients from high risk groups based on the “See you in 7” initiative for patients with heart failure (HF) and MI,^9,10 as well as evidence that patients with NSTEMI have a lower risk of 30-day readmission if they have follow-up within 7 days after discharge⁵; and we selected 14 days follow-up for patients from low-risk groups based on evidence that postdischarge follow-up within 14 days reduces risk of 30-day readmission in patients with acute myocardial infarction (AMI) and/or acutely decompensated HF.¹¹

Methods

This project was designed to answer the following question: For adult patients with type I MI, does implementation of a readmission risk assessment referral protocol increase the percentage of referrals and appointments scheduled within a recommended time? Anticipated outcomes included: (1) increased referrals to a cardiologist or the post-MI clinic; (2) increased scheduled follow-up appointments within 7 to 14 days; (3) provider satisfaction with the usability and usefulness of the new protocol; and (4) consistent provider adoption of the new risk assessment referral protocol.

To evaluate the degree to which these outcomes were achieved, we reviewed patient charts for 2 months prior and 2 months during implementation of the new referral protocol. As shown in Figure 2, the new protocol added the following process steps to the existing protocol: calculation of the AMI READMITS score, recommendations for follow-up based on patients’ risk score, and guidance to refer patients to the post-MI clinic if patients did not have an appointment with a cardiologist within 7 to 14 days after discharge. Patients’ risk assessment scores were obtained from forms completed by clinicians during the intervention. Clinician’s perceptions related to the usability and usefulness of the new protocol and feedback related to its long-term adoption were assessed using a descriptive survey.

Population

The project population included all adult patients (≥ 18 years old) with type I MI who were admitted or transferred to the hospital, had a percutaneous coronary intervention (PCI), or were managed without PCI and discharged from the hospital’s cardiac care unit (CCU) and progressive cardiac care unit (PCCU). The criteria for type I MI included the “detection of a rise and/or fall of cardiac troponin with at least 1 value above the 99th percentile and with at least 1 of the following: symptoms of acute myocardial ischemia; new ischemic electrocardiographic (ECG) changes; development of new pathological Q waves; imaging evidence of new loss of viable myocardium or new regional wall motion abnormality in a pattern consistent with an ischemic etiology; identification of a coronary thrombus by angiography including intracoronary imaging or by autopsy.”¹² The study excluded patients with type I MI who were referred for coronary bypass surgery.

Intervention

The revised risk assessment protocol was implemented within the CCU and PCCU. The lead investigator met with each provider to discuss the role of the post-MI clinic, current referral rates, the purpose of the project, and the new referral process to be completed during the project for each patient discharged with type I MI. Cardiology NPs, fellows, and residents were asked to use the risk-assessment form to calculate patients’ risk for readmission, and refer patients to the post-MI clinic if an appointment with a cardiologist was not available within 7 to 14 days after discharge. Every week during the intervention phase, the investigator sent reminder emails to ensure form completion. Providers were asked to calculate and write the score, the discharge and referral dates, where referrals were made (a cardiologist or the post-MI clinic), date of appointment, and reason for not scheduling an appointment or not referring on the risk assessment form, and to drop the completed forms in specific labeled boxes located at the CCU and PCCU work stations. The investigator collected the completed forms weekly. When the number of discharged patients did not match the number of completed forms, the investigator followed up with discharging providers to understand why.

Data and Data Collection

Data to determine whether the use of the new protocol increased discharge referrals among patients with type I MI within the recommended timeframes were collected by electronic chart review. Data included discharging unit, patients’ age, gender, admission and discharge date, diagnosis, referral to a cardiologist and the post-MI clinic, and appointment date. Clinical data needed to calculate the AMI READMITS score was also collected: PCI within 24 hours, serum creatinine, systolic blood pressure (SBP), brain natriuretic peptide (BNP), and diabetes status.

Data to assess provider satisfaction with the usability and usefulness of the new protocol were gathered through an online survey. The survey included 1 question related to the providers’ role, 1 question asking whether they used the risk assessment for each patient, and 5 Likert-items assessing the ease of usage. An additional open-ended question asked providers to share feedback related to integrating the AMI READMITS risk assessment score to the post-MI referral protocol long term.

To evaluate how consistently providers utilized the new referral protocol when discharging patients with type I MI, the number of completed forms was compared with the number of those patients who were discharged.

Statistical Analysis

Descriptive statistics were used to summarize patient demographics and to calculate the frequency of referrals before and during the intervention. Chi-square statistics were calculated to determine whether the change in percentage of referrals and timely referrals was significant. Descriptive statistics were used to determine the level of provider satisfaction related to each survey item. A content analysis method was used to synthesize themes from the open-ended question asking clinicians to share their feedback related to the new protocol.

Results

Fifty-seven patients met the study inclusion criteria: 29 patients during the preintervention phase and 28 patients during the intervention phase. There were 35 male (61.4%) and 22 female (38.6%) patients. Twenty-five patients (43.9%) were from age groups 41 through 60 years and 61 through 80 years, respectively, representing the majority of included patients. Seven patients (12.3%) were from the 81 years and older age group. There were no patients in the age group 18 through 40 years. Based on the AMI READMITS score calculation, 57.9% (n = 33) patients were from a low-risk group (includes extremely low and low risk for readmission) and 42.1% (n = 24) were from a high-risk group (includes moderate, high, and extremely high risk for readmission).

Provider adoption of the new protocol during the intervention was high. Referral forms were completed for 82% (n = 23) of the 28 patients during the intervention. Analysis findings showed a statistically significant increase in documented referrals after implementing the new referral protocol. During the preintervention phase, 66% (n = 19) of patients with type I MI were referred to see a cardiologist or an NP at a post-MI clinic and there was no documented referral for 34% (n = 10) of patients. During the intervention phase, 89% (n = 25) of patients were referred and there was no documented referral for 11% (n = 3) of patients. Chi-square results indicated that the increase in referrals was significant (χ2 = 4.571, df = 1, P = 0.033).

Data analysis examined whether patient referrals fell within the recommended timeframe of 7 days for the high-risk group (included moderate-to-extremely high risk) and 14 days for the low-risk group (included low-to-extremely low risk). During the preintervention phase, 31% (n = 9) of patient referrals were scheduled as recommended; 28% (n = 8) of patient referrals were scheduled but delayed; and there was no referral date documented for 41% (n = 12) of patients. During the intervention phase, referrals scheduled as recommended increased to 53% (n = 15); 25% (n = 7) of referrals were scheduled but delayed; and there was no referral date documented for 21.4% (n = 6) of patients. The change in appointments scheduled as recommended was not significant (χ2 = 3.550, df = 2, P = 0.169).

Surveys were emailed to 25 cardiology fellows and 3 cardiology NPs who participated in this study. Eighteen of the 28 clinicians (15 cardiology fellows and 3 cardiology NPs) responded for a response rate of 64%. One of several residents who rotated through the CCU and PCCU during the intervention also completed the survey, for a total of 19 participants. When asked if the protocol was easy to use, 79% agreed or strongly agreed. Eighteen of the 19 participants (95%) agreed or strongly agreed that the protocol was useful in making referral decisions. Sixty-eight percent agreed or strongly agreed that the AMI READMITS risk assessment score improves referral process. All participants agreed or strongly agreed that there should be an option to incorporate the AMI READMITS risk assessment score into electronic clinical notes. When asked whether the AMI READMITS risk score should be implemented in clinical practice, responses were mixed (Figure 3). A common theme among the 4 participants who responded with comments was the need for additional data to validate the usefulness of the AMI READMITS to reduce readmissions. In addition, 1 participant commented that “manual calculation [of the risk score] is not ideal.”

Discussion

This project demonstrated that implementing an evidence-based referral protocol integrating the AMI-READMITS score can increase timely postdischarge referrals among patients with type I MI. The percentage of appropriately scheduled appointments increased during the intervention phase; however, a relatively high number of appointments were scheduled outside of the recommended timeframe, similar to preintervention. Thus, while the new protocol increased referrals and provider documentation of these referrals, it appears that challenges in scheduling timely referral appointments remained. This project did not examine the reasons for delayed appointments.

The survey findings indicated that providers were generally satisfied with the usability and usefulness of the new risk assessment protocol. A large majority agreed or strongly agreed that it was easy to use and useful in making referral decisions, and most agreed or strongly agreed that it improves the referral process. Mixed opinions regarding implementing the AMI READMITS score in clinical practice, combined with qualitative findings, suggest that a lack of external validation of the AMI READMITS presents a barrier to its long-term adoption. All providers who participated in the survey agreed or strongly agreed that the risk assessment should be incorporated into electronic clinical notes. We have begun the process of working with the EHR vendor to automate the AMI risk-assessment within the referral work-flow, which will provide an opportunity for a follow-up quality improvement study.

This quality improvement project has several limitations. First, it implemented a small change in 2 inpatient units at 1 hospital using a simple pre- posttest design. Therefore, the findings are not generalizable to other settings. Prior to the intervention, some referrals may have been made without documentation. While the authors were able to trace undocumented referrals for patients who were referred to the post-MI clinic or to a cardiologist affiliated with the hospital, some patients may have been referred to cardiologists who were not affiliated with the hospital. Another limitation was that the self-created provider survey used was not tested in other clinical settings; thus, it cannot be determined whether the sensitivity and specificity of the survey questions are high. In addition, the clinical providers who participated in the study knew the study team, which may have influenced their behavior during the study period. Furthermore, the identified improvement in clinicians’ referral practices may not be sustainable due to the complexity and effort required to manually calculate the risk score. This limitation could be eliminated by integrating the risk score calculation into the EHR.

Conclusion

Early follow-up after discharge plays an important role in supporting patients’ self-management of some risk factors (ie, diet, weight, and smoking) and identifying gaps in postdischarge care which may lead to readmission. This project provides evidence that integrating the AMI READMITS risk assessment score into the referral process can help to guide discharge decision-making and increase timely, appropriate referrals for patients with MI. Integration of a specific risk assessment, such as the AMI READMITS, within the post-MI referral protocol may help clinicians make more efficient, educated referral decisions. Future studies should explore more specifically how and why the new protocol impacts clinicians’ decision-making and behavior related to post-MI referrals. In addition, future studies should investigate challenges associated with scheduling postdischarge appointments. It will be important to investigate how integration of the new protocol within the EHR may increase efficiency, consistency, and provider satisfaction with the new referral process. Additional research investigating the effects of the AMI READMITS score on readmissions reduction will be important to promote long-term adoption of the improved referral protocol in clinical practice.

Acknowledgments: The authors thank Shelly Conaway, ANP-BC, MSN, Angela Street, ANP-BC, MSN, Andrew Geis, ACNP-BC, MSN, Richard P. Jones II, MD, Eunice Young, MD, Joy Rothwell, MSN, RN-BC, Allison Olazo, MBA, MSN, RN-BC, Elizabeth Heck, RN-BC, and Matthew Trojanowski, MHA, MS, RRT, CSSBB for their support of this study.

Corresponding author: Nailya Muganlinskaya, DNP, MPH, ACNP-BC, MSN, The Johns Hopkins Hospital, 1800 Orleans St, Baltimore, MD 21287; [email protected].

Financial disclosures: None.

From The Johns Hopkins Hospital, Baltimore, MD (Dr. Muganlinskaya and Dr. Skojec, retired); The George Washington University, Washington, DC (Dr. Posey); and Johns Hopkins University, Baltimore, MD (Dr. Resar).

Abstract

Objective: Assessing the risk characteristics of patients with acute myocardial infarction (MI) can help providers make appropriate referral decisions. This quality improvement project sought to improve timely, appropriate referrals among patients with type I MI by adding a risk assessment, the AMI READMITS score, to the existing referral protocol.

Methods: Patients’ chart data were analyzed to assess changes in referrals and timely follow-up appointments from pre-intervention to intervention. A survey assessed providers’ satisfaction with the new referral protocol.

Results: Among 57 patients (n = 29 preintervention; n = 28 intervention), documented referrals increased significantly from 66% to 89% (χ2 = 4.571, df = 1, P = 0.033); and timely appointments increased by 10%, which was not significant (χ2 = 3.550, df = 2, P = 0.169). Most providers agreed that the new protocol was easy to use, useful in making referral decisions, and improved the referral process. All agreed the risk score should be incorporated into electronic clinical notes. Provider opinions related to implementing the risk score in clinical practice were mixed. Qualitative feedback suggests this was due to limited validation of the AMI READMITS score in reducing readmissions.

Conclusions: Our risk-based referral protocol helped to increase appropriate referrals among patients with type I MI. Provider adoption may be enhanced by incorporating the protocol into electronic clinical notes. Research to further validate the accuracy of the AMI READMITS score in predicting readmissions may support adoption of the protocol in clinical practice.

Keywords: quality improvement; type I myocardial infarction; referral process; readmission risk; risk assessment; chart review.

Early follow-up after discharge is an important strategy to reduce the risk of unplanned hospital readmissions among patients with various conditions.^1-3 While patient confounding factors, such as chronic health problems, environment, socioeconomic status, and literacy, make it difficult to avoid all unplanned readmissions, early follow-up may help providers identify and appropriately manage some health-related issues, and as such is a pivotal element of a readmission prevention strategy.⁴ There is evidence that patients with non-ST elevation myocardial infarction (NSTEMI) who have an outpatient appointment with a physician within 7 days after discharge have a lower risk of 30-day readmission.⁵

Our hospital’s postmyocardial infarction clinic was created to prevent unplanned readmissions within 30 days after discharge among patients with type I myocardial infarction (MI). Since inception, the number of referrals has been much lower than expected. In 2018, the total number of patients discharged from the hospital with type I MI and any troponin I level above 0.40 ng/mL was 313. Most of these patients were discharged from the hospital’s cardiac units; however, only 91 referrals were made. To increase referrals, the cardiology nurse practitioners (NPs) developed a post-MI referral protocol (Figure 1). However, this protocol was not consistently used and referrals to the clinic remained low.

Evidence-based risk assessment tools have the potential to increase effective patient management. For example, cardiology providers at the hospital utilize various scores, such as CHA₂DS₂-VASc⁶ and the Society of Thoracic Surgery risk score,⁷ to plan patient management. Among the scores used to predict unplanned readmissions for MI patients, the most promising is the AMI READMITS score.⁸ Unlike other nonspecific prediction models, the AMI READMITS score was developed based on variables extracted from the electronic health records (EHRs) of patients who were hospitalized for MI and readmitted within 30 days after discharge. Recognizing the potential to increase referrals by integrating an MI-specific risk assessment, this quality improvement study modified the existing referral protocol to include the patients’ AMI READMITS score and recommendations for follow-up.

Currently, there are no clear recommendations on how soon after discharge patients with MI should undergo follow-up. As research data vary, we selected 7 days follow-up for patients from high risk groups based on the “See you in 7” initiative for patients with heart failure (HF) and MI,^9,10 as well as evidence that patients with NSTEMI have a lower risk of 30-day readmission if they have follow-up within 7 days after discharge⁵; and we selected 14 days follow-up for patients from low-risk groups based on evidence that postdischarge follow-up within 14 days reduces risk of 30-day readmission in patients with acute myocardial infarction (AMI) and/or acutely decompensated HF.¹¹

Methods

This project was designed to answer the following question: For adult patients with type I MI, does implementation of a readmission risk assessment referral protocol increase the percentage of referrals and appointments scheduled within a recommended time? Anticipated outcomes included: (1) increased referrals to a cardiologist or the post-MI clinic; (2) increased scheduled follow-up appointments within 7 to 14 days; (3) provider satisfaction with the usability and usefulness of the new protocol; and (4) consistent provider adoption of the new risk assessment referral protocol.

To evaluate the degree to which these outcomes were achieved, we reviewed patient charts for 2 months prior and 2 months during implementation of the new referral protocol. As shown in Figure 2, the new protocol added the following process steps to the existing protocol: calculation of the AMI READMITS score, recommendations for follow-up based on patients’ risk score, and guidance to refer patients to the post-MI clinic if patients did not have an appointment with a cardiologist within 7 to 14 days after discharge. Patients’ risk assessment scores were obtained from forms completed by clinicians during the intervention. Clinician’s perceptions related to the usability and usefulness of the new protocol and feedback related to its long-term adoption were assessed using a descriptive survey.

Population

The project population included all adult patients (≥ 18 years old) with type I MI who were admitted or transferred to the hospital, had a percutaneous coronary intervention (PCI), or were managed without PCI and discharged from the hospital’s cardiac care unit (CCU) and progressive cardiac care unit (PCCU). The criteria for type I MI included the “detection of a rise and/or fall of cardiac troponin with at least 1 value above the 99th percentile and with at least 1 of the following: symptoms of acute myocardial ischemia; new ischemic electrocardiographic (ECG) changes; development of new pathological Q waves; imaging evidence of new loss of viable myocardium or new regional wall motion abnormality in a pattern consistent with an ischemic etiology; identification of a coronary thrombus by angiography including intracoronary imaging or by autopsy.”¹² The study excluded patients with type I MI who were referred for coronary bypass surgery.

Intervention

The revised risk assessment protocol was implemented within the CCU and PCCU. The lead investigator met with each provider to discuss the role of the post-MI clinic, current referral rates, the purpose of the project, and the new referral process to be completed during the project for each patient discharged with type I MI. Cardiology NPs, fellows, and residents were asked to use the risk-assessment form to calculate patients’ risk for readmission, and refer patients to the post-MI clinic if an appointment with a cardiologist was not available within 7 to 14 days after discharge. Every week during the intervention phase, the investigator sent reminder emails to ensure form completion. Providers were asked to calculate and write the score, the discharge and referral dates, where referrals were made (a cardiologist or the post-MI clinic), date of appointment, and reason for not scheduling an appointment or not referring on the risk assessment form, and to drop the completed forms in specific labeled boxes located at the CCU and PCCU work stations. The investigator collected the completed forms weekly. When the number of discharged patients did not match the number of completed forms, the investigator followed up with discharging providers to understand why.

Data and Data Collection

Data to determine whether the use of the new protocol increased discharge referrals among patients with type I MI within the recommended timeframes were collected by electronic chart review. Data included discharging unit, patients’ age, gender, admission and discharge date, diagnosis, referral to a cardiologist and the post-MI clinic, and appointment date. Clinical data needed to calculate the AMI READMITS score was also collected: PCI within 24 hours, serum creatinine, systolic blood pressure (SBP), brain natriuretic peptide (BNP), and diabetes status.

Data to assess provider satisfaction with the usability and usefulness of the new protocol were gathered through an online survey. The survey included 1 question related to the providers’ role, 1 question asking whether they used the risk assessment for each patient, and 5 Likert-items assessing the ease of usage. An additional open-ended question asked providers to share feedback related to integrating the AMI READMITS risk assessment score to the post-MI referral protocol long term.

To evaluate how consistently providers utilized the new referral protocol when discharging patients with type I MI, the number of completed forms was compared with the number of those patients who were discharged.

Statistical Analysis

Descriptive statistics were used to summarize patient demographics and to calculate the frequency of referrals before and during the intervention. Chi-square statistics were calculated to determine whether the change in percentage of referrals and timely referrals was significant. Descriptive statistics were used to determine the level of provider satisfaction related to each survey item. A content analysis method was used to synthesize themes from the open-ended question asking clinicians to share their feedback related to the new protocol.

Results

Fifty-seven patients met the study inclusion criteria: 29 patients during the preintervention phase and 28 patients during the intervention phase. There were 35 male (61.4%) and 22 female (38.6%) patients. Twenty-five patients (43.9%) were from age groups 41 through 60 years and 61 through 80 years, respectively, representing the majority of included patients. Seven patients (12.3%) were from the 81 years and older age group. There were no patients in the age group 18 through 40 years. Based on the AMI READMITS score calculation, 57.9% (n = 33) patients were from a low-risk group (includes extremely low and low risk for readmission) and 42.1% (n = 24) were from a high-risk group (includes moderate, high, and extremely high risk for readmission).

Provider adoption of the new protocol during the intervention was high. Referral forms were completed for 82% (n = 23) of the 28 patients during the intervention. Analysis findings showed a statistically significant increase in documented referrals after implementing the new referral protocol. During the preintervention phase, 66% (n = 19) of patients with type I MI were referred to see a cardiologist or an NP at a post-MI clinic and there was no documented referral for 34% (n = 10) of patients. During the intervention phase, 89% (n = 25) of patients were referred and there was no documented referral for 11% (n = 3) of patients. Chi-square results indicated that the increase in referrals was significant (χ2 = 4.571, df = 1, P = 0.033).

Data analysis examined whether patient referrals fell within the recommended timeframe of 7 days for the high-risk group (included moderate-to-extremely high risk) and 14 days for the low-risk group (included low-to-extremely low risk). During the preintervention phase, 31% (n = 9) of patient referrals were scheduled as recommended; 28% (n = 8) of patient referrals were scheduled but delayed; and there was no referral date documented for 41% (n = 12) of patients. During the intervention phase, referrals scheduled as recommended increased to 53% (n = 15); 25% (n = 7) of referrals were scheduled but delayed; and there was no referral date documented for 21.4% (n = 6) of patients. The change in appointments scheduled as recommended was not significant (χ2 = 3.550, df = 2, P = 0.169).

Surveys were emailed to 25 cardiology fellows and 3 cardiology NPs who participated in this study. Eighteen of the 28 clinicians (15 cardiology fellows and 3 cardiology NPs) responded for a response rate of 64%. One of several residents who rotated through the CCU and PCCU during the intervention also completed the survey, for a total of 19 participants. When asked if the protocol was easy to use, 79% agreed or strongly agreed. Eighteen of the 19 participants (95%) agreed or strongly agreed that the protocol was useful in making referral decisions. Sixty-eight percent agreed or strongly agreed that the AMI READMITS risk assessment score improves referral process. All participants agreed or strongly agreed that there should be an option to incorporate the AMI READMITS risk assessment score into electronic clinical notes. When asked whether the AMI READMITS risk score should be implemented in clinical practice, responses were mixed (Figure 3). A common theme among the 4 participants who responded with comments was the need for additional data to validate the usefulness of the AMI READMITS to reduce readmissions. In addition, 1 participant commented that “manual calculation [of the risk score] is not ideal.”

Discussion

This project demonstrated that implementing an evidence-based referral protocol integrating the AMI-READMITS score can increase timely postdischarge referrals among patients with type I MI. The percentage of appropriately scheduled appointments increased during the intervention phase; however, a relatively high number of appointments were scheduled outside of the recommended timeframe, similar to preintervention. Thus, while the new protocol increased referrals and provider documentation of these referrals, it appears that challenges in scheduling timely referral appointments remained. This project did not examine the reasons for delayed appointments.

The survey findings indicated that providers were generally satisfied with the usability and usefulness of the new risk assessment protocol. A large majority agreed or strongly agreed that it was easy to use and useful in making referral decisions, and most agreed or strongly agreed that it improves the referral process. Mixed opinions regarding implementing the AMI READMITS score in clinical practice, combined with qualitative findings, suggest that a lack of external validation of the AMI READMITS presents a barrier to its long-term adoption. All providers who participated in the survey agreed or strongly agreed that the risk assessment should be incorporated into electronic clinical notes. We have begun the process of working with the EHR vendor to automate the AMI risk-assessment within the referral work-flow, which will provide an opportunity for a follow-up quality improvement study.

This quality improvement project has several limitations. First, it implemented a small change in 2 inpatient units at 1 hospital using a simple pre- posttest design. Therefore, the findings are not generalizable to other settings. Prior to the intervention, some referrals may have been made without documentation. While the authors were able to trace undocumented referrals for patients who were referred to the post-MI clinic or to a cardiologist affiliated with the hospital, some patients may have been referred to cardiologists who were not affiliated with the hospital. Another limitation was that the self-created provider survey used was not tested in other clinical settings; thus, it cannot be determined whether the sensitivity and specificity of the survey questions are high. In addition, the clinical providers who participated in the study knew the study team, which may have influenced their behavior during the study period. Furthermore, the identified improvement in clinicians’ referral practices may not be sustainable due to the complexity and effort required to manually calculate the risk score. This limitation could be eliminated by integrating the risk score calculation into the EHR.

Conclusion

Early follow-up after discharge plays an important role in supporting patients’ self-management of some risk factors (ie, diet, weight, and smoking) and identifying gaps in postdischarge care which may lead to readmission. This project provides evidence that integrating the AMI READMITS risk assessment score into the referral process can help to guide discharge decision-making and increase timely, appropriate referrals for patients with MI. Integration of a specific risk assessment, such as the AMI READMITS, within the post-MI referral protocol may help clinicians make more efficient, educated referral decisions. Future studies should explore more specifically how and why the new protocol impacts clinicians’ decision-making and behavior related to post-MI referrals. In addition, future studies should investigate challenges associated with scheduling postdischarge appointments. It will be important to investigate how integration of the new protocol within the EHR may increase efficiency, consistency, and provider satisfaction with the new referral process. Additional research investigating the effects of the AMI READMITS score on readmissions reduction will be important to promote long-term adoption of the improved referral protocol in clinical practice.

Acknowledgments: The authors thank Shelly Conaway, ANP-BC, MSN, Angela Street, ANP-BC, MSN, Andrew Geis, ACNP-BC, MSN, Richard P. Jones II, MD, Eunice Young, MD, Joy Rothwell, MSN, RN-BC, Allison Olazo, MBA, MSN, RN-BC, Elizabeth Heck, RN-BC, and Matthew Trojanowski, MHA, MS, RRT, CSSBB for their support of this study.

Corresponding author: Nailya Muganlinskaya, DNP, MPH, ACNP-BC, MSN, The Johns Hopkins Hospital, 1800 Orleans St, Baltimore, MD 21287; [email protected].

Financial disclosures: None.

From The Johns Hopkins Hospital, Baltimore, MD (Dr. Muganlinskaya and Dr. Skojec, retired); The George Washington University, Washington, DC (Dr. Posey); and Johns Hopkins University, Baltimore, MD (Dr. Resar).

Abstract

Objective: Assessing the risk characteristics of patients with acute myocardial infarction (MI) can help providers make appropriate referral decisions. This quality improvement project sought to improve timely, appropriate referrals among patients with type I MI by adding a risk assessment, the AMI READMITS score, to the existing referral protocol.

Methods: Patients’ chart data were analyzed to assess changes in referrals and timely follow-up appointments from pre-intervention to intervention. A survey assessed providers’ satisfaction with the new referral protocol.

Results: Among 57 patients (n = 29 preintervention; n = 28 intervention), documented referrals increased significantly from 66% to 89% (χ2 = 4.571, df = 1, P = 0.033); and timely appointments increased by 10%, which was not significant (χ2 = 3.550, df = 2, P = 0.169). Most providers agreed that the new protocol was easy to use, useful in making referral decisions, and improved the referral process. All agreed the risk score should be incorporated into electronic clinical notes. Provider opinions related to implementing the risk score in clinical practice were mixed. Qualitative feedback suggests this was due to limited validation of the AMI READMITS score in reducing readmissions.

Conclusions: Our risk-based referral protocol helped to increase appropriate referrals among patients with type I MI. Provider adoption may be enhanced by incorporating the protocol into electronic clinical notes. Research to further validate the accuracy of the AMI READMITS score in predicting readmissions may support adoption of the protocol in clinical practice.

Keywords: quality improvement; type I myocardial infarction; referral process; readmission risk; risk assessment; chart review.

Early follow-up after discharge is an important strategy to reduce the risk of unplanned hospital readmissions among patients with various conditions.^1-3 While patient confounding factors, such as chronic health problems, environment, socioeconomic status, and literacy, make it difficult to avoid all unplanned readmissions, early follow-up may help providers identify and appropriately manage some health-related issues, and as such is a pivotal element of a readmission prevention strategy.⁴ There is evidence that patients with non-ST elevation myocardial infarction (NSTEMI) who have an outpatient appointment with a physician within 7 days after discharge have a lower risk of 30-day readmission.⁵

Our hospital’s postmyocardial infarction clinic was created to prevent unplanned readmissions within 30 days after discharge among patients with type I myocardial infarction (MI). Since inception, the number of referrals has been much lower than expected. In 2018, the total number of patients discharged from the hospital with type I MI and any troponin I level above 0.40 ng/mL was 313. Most of these patients were discharged from the hospital’s cardiac units; however, only 91 referrals were made. To increase referrals, the cardiology nurse practitioners (NPs) developed a post-MI referral protocol (Figure 1). However, this protocol was not consistently used and referrals to the clinic remained low.

Evidence-based risk assessment tools have the potential to increase effective patient management. For example, cardiology providers at the hospital utilize various scores, such as CHA₂DS₂-VASc⁶ and the Society of Thoracic Surgery risk score,⁷ to plan patient management. Among the scores used to predict unplanned readmissions for MI patients, the most promising is the AMI READMITS score.⁸ Unlike other nonspecific prediction models, the AMI READMITS score was developed based on variables extracted from the electronic health records (EHRs) of patients who were hospitalized for MI and readmitted within 30 days after discharge. Recognizing the potential to increase referrals by integrating an MI-specific risk assessment, this quality improvement study modified the existing referral protocol to include the patients’ AMI READMITS score and recommendations for follow-up.

Currently, there are no clear recommendations on how soon after discharge patients with MI should undergo follow-up. As research data vary, we selected 7 days follow-up for patients from high risk groups based on the “See you in 7” initiative for patients with heart failure (HF) and MI,^9,10 as well as evidence that patients with NSTEMI have a lower risk of 30-day readmission if they have follow-up within 7 days after discharge⁵; and we selected 14 days follow-up for patients from low-risk groups based on evidence that postdischarge follow-up within 14 days reduces risk of 30-day readmission in patients with acute myocardial infarction (AMI) and/or acutely decompensated HF.¹¹

Methods

This project was designed to answer the following question: For adult patients with type I MI, does implementation of a readmission risk assessment referral protocol increase the percentage of referrals and appointments scheduled within a recommended time? Anticipated outcomes included: (1) increased referrals to a cardiologist or the post-MI clinic; (2) increased scheduled follow-up appointments within 7 to 14 days; (3) provider satisfaction with the usability and usefulness of the new protocol; and (4) consistent provider adoption of the new risk assessment referral protocol.

To evaluate the degree to which these outcomes were achieved, we reviewed patient charts for 2 months prior and 2 months during implementation of the new referral protocol. As shown in Figure 2, the new protocol added the following process steps to the existing protocol: calculation of the AMI READMITS score, recommendations for follow-up based on patients’ risk score, and guidance to refer patients to the post-MI clinic if patients did not have an appointment with a cardiologist within 7 to 14 days after discharge. Patients’ risk assessment scores were obtained from forms completed by clinicians during the intervention. Clinician’s perceptions related to the usability and usefulness of the new protocol and feedback related to its long-term adoption were assessed using a descriptive survey.

Population

The project population included all adult patients (≥ 18 years old) with type I MI who were admitted or transferred to the hospital, had a percutaneous coronary intervention (PCI), or were managed without PCI and discharged from the hospital’s cardiac care unit (CCU) and progressive cardiac care unit (PCCU). The criteria for type I MI included the “detection of a rise and/or fall of cardiac troponin with at least 1 value above the 99th percentile and with at least 1 of the following: symptoms of acute myocardial ischemia; new ischemic electrocardiographic (ECG) changes; development of new pathological Q waves; imaging evidence of new loss of viable myocardium or new regional wall motion abnormality in a pattern consistent with an ischemic etiology; identification of a coronary thrombus by angiography including intracoronary imaging or by autopsy.”¹² The study excluded patients with type I MI who were referred for coronary bypass surgery.

Intervention

The revised risk assessment protocol was implemented within the CCU and PCCU. The lead investigator met with each provider to discuss the role of the post-MI clinic, current referral rates, the purpose of the project, and the new referral process to be completed during the project for each patient discharged with type I MI. Cardiology NPs, fellows, and residents were asked to use the risk-assessment form to calculate patients’ risk for readmission, and refer patients to the post-MI clinic if an appointment with a cardiologist was not available within 7 to 14 days after discharge. Every week during the intervention phase, the investigator sent reminder emails to ensure form completion. Providers were asked to calculate and write the score, the discharge and referral dates, where referrals were made (a cardiologist or the post-MI clinic), date of appointment, and reason for not scheduling an appointment or not referring on the risk assessment form, and to drop the completed forms in specific labeled boxes located at the CCU and PCCU work stations. The investigator collected the completed forms weekly. When the number of discharged patients did not match the number of completed forms, the investigator followed up with discharging providers to understand why.

Data and Data Collection

Data to determine whether the use of the new protocol increased discharge referrals among patients with type I MI within the recommended timeframes were collected by electronic chart review. Data included discharging unit, patients’ age, gender, admission and discharge date, diagnosis, referral to a cardiologist and the post-MI clinic, and appointment date. Clinical data needed to calculate the AMI READMITS score was also collected: PCI within 24 hours, serum creatinine, systolic blood pressure (SBP), brain natriuretic peptide (BNP), and diabetes status.

Data to assess provider satisfaction with the usability and usefulness of the new protocol were gathered through an online survey. The survey included 1 question related to the providers’ role, 1 question asking whether they used the risk assessment for each patient, and 5 Likert-items assessing the ease of usage. An additional open-ended question asked providers to share feedback related to integrating the AMI READMITS risk assessment score to the post-MI referral protocol long term.

To evaluate how consistently providers utilized the new referral protocol when discharging patients with type I MI, the number of completed forms was compared with the number of those patients who were discharged.

Statistical Analysis

Descriptive statistics were used to summarize patient demographics and to calculate the frequency of referrals before and during the intervention. Chi-square statistics were calculated to determine whether the change in percentage of referrals and timely referrals was significant. Descriptive statistics were used to determine the level of provider satisfaction related to each survey item. A content analysis method was used to synthesize themes from the open-ended question asking clinicians to share their feedback related to the new protocol.

Results

Fifty-seven patients met the study inclusion criteria: 29 patients during the preintervention phase and 28 patients during the intervention phase. There were 35 male (61.4%) and 22 female (38.6%) patients. Twenty-five patients (43.9%) were from age groups 41 through 60 years and 61 through 80 years, respectively, representing the majority of included patients. Seven patients (12.3%) were from the 81 years and older age group. There were no patients in the age group 18 through 40 years. Based on the AMI READMITS score calculation, 57.9% (n = 33) patients were from a low-risk group (includes extremely low and low risk for readmission) and 42.1% (n = 24) were from a high-risk group (includes moderate, high, and extremely high risk for readmission).

Provider adoption of the new protocol during the intervention was high. Referral forms were completed for 82% (n = 23) of the 28 patients during the intervention. Analysis findings showed a statistically significant increase in documented referrals after implementing the new referral protocol. During the preintervention phase, 66% (n = 19) of patients with type I MI were referred to see a cardiologist or an NP at a post-MI clinic and there was no documented referral for 34% (n = 10) of patients. During the intervention phase, 89% (n = 25) of patients were referred and there was no documented referral for 11% (n = 3) of patients. Chi-square results indicated that the increase in referrals was significant (χ2 = 4.571, df = 1, P = 0.033).

Data analysis examined whether patient referrals fell within the recommended timeframe of 7 days for the high-risk group (included moderate-to-extremely high risk) and 14 days for the low-risk group (included low-to-extremely low risk). During the preintervention phase, 31% (n = 9) of patient referrals were scheduled as recommended; 28% (n = 8) of patient referrals were scheduled but delayed; and there was no referral date documented for 41% (n = 12) of patients. During the intervention phase, referrals scheduled as recommended increased to 53% (n = 15); 25% (n = 7) of referrals were scheduled but delayed; and there was no referral date documented for 21.4% (n = 6) of patients. The change in appointments scheduled as recommended was not significant (χ2 = 3.550, df = 2, P = 0.169).

Surveys were emailed to 25 cardiology fellows and 3 cardiology NPs who participated in this study. Eighteen of the 28 clinicians (15 cardiology fellows and 3 cardiology NPs) responded for a response rate of 64%. One of several residents who rotated through the CCU and PCCU during the intervention also completed the survey, for a total of 19 participants. When asked if the protocol was easy to use, 79% agreed or strongly agreed. Eighteen of the 19 participants (95%) agreed or strongly agreed that the protocol was useful in making referral decisions. Sixty-eight percent agreed or strongly agreed that the AMI READMITS risk assessment score improves referral process. All participants agreed or strongly agreed that there should be an option to incorporate the AMI READMITS risk assessment score into electronic clinical notes. When asked whether the AMI READMITS risk score should be implemented in clinical practice, responses were mixed (Figure 3). A common theme among the 4 participants who responded with comments was the need for additional data to validate the usefulness of the AMI READMITS to reduce readmissions. In addition, 1 participant commented that “manual calculation [of the risk score] is not ideal.”

Discussion

This project demonstrated that implementing an evidence-based referral protocol integrating the AMI-READMITS score can increase timely postdischarge referrals among patients with type I MI. The percentage of appropriately scheduled appointments increased during the intervention phase; however, a relatively high number of appointments were scheduled outside of the recommended timeframe, similar to preintervention. Thus, while the new protocol increased referrals and provider documentation of these referrals, it appears that challenges in scheduling timely referral appointments remained. This project did not examine the reasons for delayed appointments.

The survey findings indicated that providers were generally satisfied with the usability and usefulness of the new risk assessment protocol. A large majority agreed or strongly agreed that it was easy to use and useful in making referral decisions, and most agreed or strongly agreed that it improves the referral process. Mixed opinions regarding implementing the AMI READMITS score in clinical practice, combined with qualitative findings, suggest that a lack of external validation of the AMI READMITS presents a barrier to its long-term adoption. All providers who participated in the survey agreed or strongly agreed that the risk assessment should be incorporated into electronic clinical notes. We have begun the process of working with the EHR vendor to automate the AMI risk-assessment within the referral work-flow, which will provide an opportunity for a follow-up quality improvement study.

This quality improvement project has several limitations. First, it implemented a small change in 2 inpatient units at 1 hospital using a simple pre- posttest design. Therefore, the findings are not generalizable to other settings. Prior to the intervention, some referrals may have been made without documentation. While the authors were able to trace undocumented referrals for patients who were referred to the post-MI clinic or to a cardiologist affiliated with the hospital, some patients may have been referred to cardiologists who were not affiliated with the hospital. Another limitation was that the self-created provider survey used was not tested in other clinical settings; thus, it cannot be determined whether the sensitivity and specificity of the survey questions are high. In addition, the clinical providers who participated in the study knew the study team, which may have influenced their behavior during the study period. Furthermore, the identified improvement in clinicians’ referral practices may not be sustainable due to the complexity and effort required to manually calculate the risk score. This limitation could be eliminated by integrating the risk score calculation into the EHR.

Conclusion

Early follow-up after discharge plays an important role in supporting patients’ self-management of some risk factors (ie, diet, weight, and smoking) and identifying gaps in postdischarge care which may lead to readmission. This project provides evidence that integrating the AMI READMITS risk assessment score into the referral process can help to guide discharge decision-making and increase timely, appropriate referrals for patients with MI. Integration of a specific risk assessment, such as the AMI READMITS, within the post-MI referral protocol may help clinicians make more efficient, educated referral decisions. Future studies should explore more specifically how and why the new protocol impacts clinicians’ decision-making and behavior related to post-MI referrals. In addition, future studies should investigate challenges associated with scheduling postdischarge appointments. It will be important to investigate how integration of the new protocol within the EHR may increase efficiency, consistency, and provider satisfaction with the new referral process. Additional research investigating the effects of the AMI READMITS score on readmissions reduction will be important to promote long-term adoption of the improved referral protocol in clinical practice.

Acknowledgments: The authors thank Shelly Conaway, ANP-BC, MSN, Angela Street, ANP-BC, MSN, Andrew Geis, ACNP-BC, MSN, Richard P. Jones II, MD, Eunice Young, MD, Joy Rothwell, MSN, RN-BC, Allison Olazo, MBA, MSN, RN-BC, Elizabeth Heck, RN-BC, and Matthew Trojanowski, MHA, MS, RRT, CSSBB for their support of this study.

Corresponding author: Nailya Muganlinskaya, DNP, MPH, ACNP-BC, MSN, The Johns Hopkins Hospital, 1800 Orleans St, Baltimore, MD 21287; [email protected].

Financial disclosures: None.

The U.S. Senate voted mostly along party lines Wednesday to confirm Vice Adm. Vivek H. Murthy, MD, MBA, to serve as the 21st Surgeon General of the United States.

Seven Republicans – Bill Cassidy (La.), Susan Collins (Maine), Roger Marshall (Kan.), Susan Murkowski (Alaska), Rob Portman (Ohio), Mitt Romney (Utah), and Dan Sullivan (Alaska) – joined all the Democrats and independents in the 57-43 vote approving Dr. Murthy’s nomination.

Dr. Murthy, 43, previously served as the 19th Surgeon General, from December 2014 to April 2017, when he was asked to step down by President Donald J. Trump.

Surgeons General serve 4-year terms.

During his first tenure, Dr. Murthy issued the first-ever Surgeon General’s report on the crisis of addiction and issued a call to action to doctors to help battle the opioid crisis.

When Dr. Murthy was nominated by President-elect Joseph R. Biden Jr. in December, he was acting as cochair of the incoming administration’s COVID-19 transition advisory board.

Early in 2020, before the COVID-19 pandemic hit, Dr. Murthy published a timely book: “Together: The Healing Power of Human Connection in a Sometimes Lonely World”.

He earned his bachelor’s degree from Harvard and his MD and MBA degrees from Yale. He completed his internal medicine residency at Brigham and Women’s Hospital in Boston, where he also served as a hospitalist, and later joined Harvard Medical School as a faculty member in internal medicine.

He is married to Alice Chen, MD. The couple have two children.
 

A version of this article first appeared on WebMD.com.

The U.S. Senate voted mostly along party lines Wednesday to confirm Vice Adm. Vivek H. Murthy, MD, MBA, to serve as the 21st Surgeon General of the United States.

Seven Republicans – Bill Cassidy (La.), Susan Collins (Maine), Roger Marshall (Kan.), Susan Murkowski (Alaska), Rob Portman (Ohio), Mitt Romney (Utah), and Dan Sullivan (Alaska) – joined all the Democrats and independents in the 57-43 vote approving Dr. Murthy’s nomination.

Dr. Murthy, 43, previously served as the 19th Surgeon General, from December 2014 to April 2017, when he was asked to step down by President Donald J. Trump.

Surgeons General serve 4-year terms.

During his first tenure, Dr. Murthy issued the first-ever Surgeon General’s report on the crisis of addiction and issued a call to action to doctors to help battle the opioid crisis.

When Dr. Murthy was nominated by President-elect Joseph R. Biden Jr. in December, he was acting as cochair of the incoming administration’s COVID-19 transition advisory board.

Early in 2020, before the COVID-19 pandemic hit, Dr. Murthy published a timely book: “Together: The Healing Power of Human Connection in a Sometimes Lonely World”.

He earned his bachelor’s degree from Harvard and his MD and MBA degrees from Yale. He completed his internal medicine residency at Brigham and Women’s Hospital in Boston, where he also served as a hospitalist, and later joined Harvard Medical School as a faculty member in internal medicine.

He is married to Alice Chen, MD. The couple have two children.
 

A version of this article first appeared on WebMD.com.

The U.S. Senate voted mostly along party lines Wednesday to confirm Vice Adm. Vivek H. Murthy, MD, MBA, to serve as the 21st Surgeon General of the United States.

Seven Republicans – Bill Cassidy (La.), Susan Collins (Maine), Roger Marshall (Kan.), Susan Murkowski (Alaska), Rob Portman (Ohio), Mitt Romney (Utah), and Dan Sullivan (Alaska) – joined all the Democrats and independents in the 57-43 vote approving Dr. Murthy’s nomination.

Dr. Murthy, 43, previously served as the 19th Surgeon General, from December 2014 to April 2017, when he was asked to step down by President Donald J. Trump.

Surgeons General serve 4-year terms.

During his first tenure, Dr. Murthy issued the first-ever Surgeon General’s report on the crisis of addiction and issued a call to action to doctors to help battle the opioid crisis.

When Dr. Murthy was nominated by President-elect Joseph R. Biden Jr. in December, he was acting as cochair of the incoming administration’s COVID-19 transition advisory board.

Early in 2020, before the COVID-19 pandemic hit, Dr. Murthy published a timely book: “Together: The Healing Power of Human Connection in a Sometimes Lonely World”.

He earned his bachelor’s degree from Harvard and his MD and MBA degrees from Yale. He completed his internal medicine residency at Brigham and Women’s Hospital in Boston, where he also served as a hospitalist, and later joined Harvard Medical School as a faculty member in internal medicine.

He is married to Alice Chen, MD. The couple have two children.
 

A version of this article first appeared on WebMD.com.