User login
Impact of an Educational and Laboratory Stewardship Intervention on Inpatient COVID-19 Therapeutics at a Veterans Affairs Medical Center
Throughout the COVID-19 pandemic, health care professionals (HCPs), including emergency medicine physicians and hospitalists, have been continuously challenged to maintain an up-to-date clinical practice on COVID-19 therapeutics as new evidence emerged.1,2 In the early part of the pandemic, these included not only appropriate and time-sensitive prescriptions of COVID-19 therapeutics, such as remdesivir and dexamethasone, but also judicious use of empiric antibiotics given the low prevalence for bacterial coinfection in early disease.3-6 Alongside this, curbing the excessive laboratory testing of these patients during the pandemic was important not only to minimize costs but also to reduce potential iatrogenic harm and extended length of stay (LOS).7
At the beginning of the pandemic in March 2020 at the US Department of Veterans Affairs (VA) North Texas Health Care System (VANTHCS) Dallas VA Medical Center (DVAMC), we attempted to provide therapeutic guidance for physicians primarily through direct infectious disease (ID) consultation (in-person or electronic).8 This was secondarily supported by a pharmacist and ID physician–curated “living guidance” document on COVID-19 care accessible to all physicians through the DVAMC electronic health record (EHR) and intranet.
As the alpha variant (lineage B.1.1.7) of COVID-19 began spreading throughout North Texas in the winter of 2020, we implemented a targeted educational intervention toward the hospitalist group taking care of patients with COVID-19 with the primary goal of improving the accuracy of COVID-19 therapeutics while minimizing the consultative burden on ID clinical and pharmacy staff. This initiative consisted of (1) proactive guideline dissemination through email and text messages; (2) virtual didactics; and (3) physician reminders during the consultation process. Our ultimate aims were to improve hospitalist-led appropriate prescriptions of remdesivir and dexamethasone, reducing empiric antibiotic days of therapy in patients with COVID-19 at low risk of bacterial coinfection, and reducing laboratory orders that were not indicated for the management of these patients. Following this intervention and the resolution of the second wave, we retrospectively assessed the temporal trends of COVID-19 practices by hospitalists and associated patterns of ID consultation in the DVAMC from October 1, 2020, to March 31, 2021.
METHODS
The educational intervention was carried out at the DVAMC, a 1A high complex facility with more than 200 inpatient beds and part of the VANTHCS. During the study period, patients admitted with COVID-19 were located either on a closed floor (managed by the hospitalist team) or in a closed intensive care unit (ICU) (managed by the pulmonary/critical care team) contingent on the level of care or oxygen supplementation required. ID and other subspecialties provided consultation services as requested by hospitalists or ICU teams either electronically or in person. During the study period, 66 hospitalists were involved in the care of the patients: 59 (89.5%) permanent staff, 4 (6.0%) fee-basis physicians, and 3 (4.5%) moonlighting fellows.
Educational Initiative
We delivered educational sessions to the hospitalists, using collaboration software with video meeting capability every 1 to 2 months beginning in December 2020. An additional session focused on reducing empiric antibiotic prescriptions was also delivered to the emergency medicine department, based on feedback from the hospitalist group. The content for the educational sessions came from informal surveys of both ID trainees assigned to the consultation service and hospitalists, covering the following topics: understanding the stages of COVID-19 illness (virologic replication vs inflammatory) and rationales for therapy; assessing disease severity; indications and use of remdesivir; indications and use of dexamethasone; assessing for bacterial coinfections; when an ID consultation is required; management algorithm for COVID-19; and locating guidelines on the intranet. About 15 to 20 physicians participated in each session. In addition, slides of these didactics and updated institutional COVID-19 guidelines were disseminated to the hospitalist group via email and text messaging. We also linked the intranet institution guidelines in our communication, including a revised user-friendly flowchart (eAppendix).
Laboratory Stewardship Initiative
Laboratory stewardship initiatives were implemented by modifying suggested orders on the admission of patients with COVID-19 and directly educating hospitalist and emergency medicine physicians on evidence-based laboratory orders. At the beginning of the pandemic, a broad admission order set was established at DVAMC, based on the then limited knowledge of the course of infection with COVID-19. This order set allowed the admitting physicians to efficiently order laboratory tests for patients, especially during the demanding increase in patient volume experienced by DVAMC.
As new evidence emerged during the pandemic, many of the laboratory orders were reviewed for clinical utility during care for the patient with COVID-19 per the latest guidance. In December 2020, the admission orders for patients with COVID-19 were revised to reflect better laboratory stewardship to reduce cost and harm. The ID section revised the laboratory orders and disseminated the new order set to admitting physicians. Specifically, the admission order set removed the following laboratory tests available for selection: routine blood cultures, interleukin 6 (IL-6) level, and Legionella sputum culture. These laboratory orders were removed based on the lack of supporting evidence in persons admitted with COVID-19.9 In addition to modification of the admission order set, educational sessions were held with hospitalists to disseminate knowledge of the new changes and address any concerns.
Observations of Care
This study was approved by the VANTHCS Institutional Review Board (protocol code 20-047). Records were retrospectively reviewed for patients admitted to DVAMC for COVID-19 under hospitalist care (patients admitted directly to the ICU were excluded) from October 1, 2020, to March 31, 2021. Age, sex, race and ethnicity, and comorbidities were collected from the EHR. In addition clinical measures such as maximum oxygen requirement during admission (none, nasal cannula of 2-4 L/min, high flow/bilevel positive airway pressure [BiPAP] or mechanical ventilation), proven presence of coinfection (defined as the isolation of a probable pathogen in pure culture and/or clinically determined by ID specialist evaluation), and the average LOS also were collected. For laboratory stewardship data, a retrospective chart review was conducted to determine the total number of blood cultures obtained within 24 hours of admission per month during the study period. Both IL-6 levels and Legionella sputum culture data were collected as the total number of laboratory orders per month, as it was assumed that most of these orders were obtained for patients admitted with COVID-19.
Individual patient-level data were extracted to calculate monthly percentages of ID consultations for COVID-19 by the hospitalist team, adherence to institutional guidelines for dexamethasone and remdesivir prescriptions, and empiric antibiotic prescriptions for patients with COVID-19, including use of a priori adjudication criteria to determine justified vs unjustified empiric use. These criteria included asymmetric chest X-ray infiltrates concerning for bacterial pneumonia; peripheral white blood cell count > 11 K/μL; critical respiratory failure in the emergency department (ED) and being transferred to the ICU; and ID consultation recommended. Because the total number of antibiotics was not being analyzed but rather just the use of antibiotics for the justified and unjustified groups, antibiotic days were reported as the length of therapy (LOT).10 A subset analysis was performed on antibiotic prescriptions by the hospitalist group focusing on those with mild-to-moderate oxygen requirements (no high flow, noninvasive or invasive ventilatory methods) and excluding infections with a proven microbiologic entity.
Differences in demographic and clinical characteristics of patients with COVID-19 admitted from October 1, 2020, to March 31, 2021, were assessed using ANOVA, χ2, and Kruskal-Wallis test. χ2 was used to compare the difference in total laboratory orders for routine blood cultures, IL-6 levels, and Legionella sputum cultures between pre-intervention (October to December 2020) and postintervention (January to March 2021). These pre- and postintervention periods were determined based on the timing of revised admission orders in the EHR and initiation of focused educational sessions starting in late December 2020 and early January 2021. Linear regressions were used to examine the possible 6-month trend of the percentage of patients receiving ID consultation for appropriate dexamethasone prescriptions, appropriate remdesivir prescriptions, appropriate antibiotic coadministration, and mean number of antibiotic days per patient. Linear and logistic regression were also used to assess the trend in LOS over the 6 months while adjusting for age, race and ethnicity, sex, and coinfections. All analyses were performed using SAS 9.4. Statistical significance was defined as P < .05.
RESULTS
From October 1, 2020, to March 31, 2021, there were 565 admissions for COVID-19, which peaked in January 2021 with 163. Analysis of the patient characteristics showed no statistically significant difference for age, sex, oxygen requirements during admission, or proven presence of coinfection between the months of interest (Table 1). 
The number of blood cultures obtained in the first 24 hours of admission significantly decreased from 58.1% of admissions in October 2020 to 34.8% of admissions in March 2021 (P < .01) (Table 2). 
We observed trends that coincided with the educational efforts. The rate of dexamethasone and remdesivir prescriptions for eligible patients that followed guidelines without ID consultation grew from 0% to 22.2% (P < .01) and 0% to 16.7% (P = .01), respectively. The remaining correct prescriptions for dexamethasone or remdesivir were instituted only after ID consultation. These improvements were seen in tandem with decreased reliance on ID consultation for admitted patients with COVID-19 overall (86.5% in October 2020 to 56.5% in March 2021; P < .01).
After applying a priori justified antibiotic use criteria, we found that the overall degree of empiric unjustified antibiotic use remained high for patients admitted with COVID-19 (36.5%-60.3%) and was largely driven by prescriptions from the ED. However, further analysis revealed a statistically significant decrease in empiric antibiotic LOT per patient during the study period from 3.0 days in October 2020 to 0.9 days in March 2021 (P < .01). In addition, there was a statistically significant change in the mean (SD) LOS, which decreased from 16.3 (17.8) days in October 2020 to 9.7 (13.0) days in March 2021 (P = .02).
DISCUSSION
As the COVID-19 pandemic has evolved, the ability to enact up-to-date guidance is crucial to streamlining patient care, improving time to COVID-19–specific therapies, and minimizing the burden on subspecialty consultation services. At DVAMC, we initiated a targeted and deliberate educational effort directed toward hospitalist and ED groups combined with a laboratory stewardship effort over 6 months to improve the implementation of COVID-19 therapeutics, reduce empiric antibiotic use without reliance on ID consultation services, and reduce the number of unnecessary laboratory orders for admitted patients with COVID-19. During this time, we observed modest but statistically significant improvements in the accuracy of dexamethasone and remdesivir prescribing. In addition, we observed statistically significant improvement in the average LOT per patient regarding antibiotic use and overall decreased LOS. These improvements were seen in parallel with decreasing requests for ID consultation, suggesting that they were attributable in part to increasing self-confidence and efficacy in COVID-19 practices by the hospitalist group. Modification of the COVID-19 admission order set for our facility resulted in substantial decreases in orders for blood cultures, IL-6 levels, and sputum cultures for Legionella.
ID consultation, either in person or remotely, has been instrumental in assisting physicians in COVID-19 management and has been shown to reduce morbidity, mortality, and patient LOS in other infections.11,12 However, in scenarios where ID consultation is not available or in limited supply, accessibility, familiarity, and confidence of primary practitioners to use therapeutic guidance material are integral. Frequent and accessible guidance for the management of COVID-19 has been provided by the National Institutes of Health and the Infectious Diseases Society of America.13,14 Other mechanisms of assisting physicians in both test ordering and therapeutics include clinical decision support tools built into the EHR and the use of a smartphone digital application.15 Guidance needs to be adapted to the context of the facility, including available resources and specific restrictions and/or prohibitions on therapeutics (eg, mandatory ID consultation or approval). In our facility, while COVID-19 therapeutic living guidance documents were maintained and accessible through the intranet, proactive dissemination and redirection were important steps in enabling the use of these documents.
Limitations
We acknowledge several limitations to this study. Most important, the correlations we observed do not represent causation. Our analysis was not designed to ascertain the direct impact of any single or combined educational and laboratory stewardship intervention from this study, and we acknowledge that the improvements in part could be related to increased experience and confidence with COVID-19 management that occurred over time independent of our programs. Furthermore, we acknowledge that several areas of COVID-19 management did not improve over time (such as overall empiric antibiotic use from the ED) or had very modest improvements (hospitalist-initiated remdesivir use). These results underscore the complex dynamics and contextual barriers to rapidly implementing guideline-based care at VANTHCS. Potential factors include insufficient reach to all physicians, variable learner motivation, and therapeutic momentum of antibiotic use carried forward from the ED.16,17 These factors should be considered as grounds for further study. Another limitation was the inability to track viewership and engagement of our COVID-19 guidance document. Without the use metrics, it is difficult to know the individual impact of the document regarding the changing trends in COVID-19 management we observed during the study period.
Conclusions
We report improvements in COVID-19 therapeutic prescriptions and the use of antibiotics and laboratory testing over 6 months at the DVAMC. This was correlated with a deliberate COVID-19 educational initiative that included antibiotic and laboratory stewardship interventions with simultaneous decreased reliance on ID consultation. These efforts lend support to the proof of the principle of combined educational and laboratory stewardship interventions to improve the care of COVID-19 patients, especially where ID support may not be available or is accessed remotely.
1. Dagens A, Sigfrid L, Cai E, et al. Scope, quality, and inclusivity of clinical guidelines produced early in the covid-19 pandemic: rapid review. BMJ. 2020;369:m1936. Published 2020 May 26. doi:10.1136/bmj.m1936
2. Dhivagaran T, Abbas U, Butt F, Arunasalam L, Chang O. Critical appraisal of clinical practice guidelines for the management of COVID-19: protocol for a systematic review. Syst Rev. 2021;10(1):317. Published 2021 Dec 22. doi:10.1186/s13643-021-01871-7
3. Garcia-Vidal C, Sanjuan G, Moreno-García E, et al. Incidence of co-infections and superinfections in hospitalized patients with COVID-19: a retrospective cohort study. Clin Microbiol Infect. 2021;27(1):83-88. doi:10.1016/j.cmi.2020.07.041
4. Karaba SM, Jones G, Helsel T, et al. Prevalence of co-infection at the time of hospital admission in covid-19 patients, a multicenter study. Open Forum Infect Dis. 2020;8(1):ofaa578. Published 2020 Dec 21. doi:10.1093/ofid/ofaa578
5. RECOVERY Collaborative Group, Horby P, Lim WS, et al. Dexamethasone in hospitalized patients with Covid-19. N Engl J Med. 2021;384(8):693-704. doi:10.1056/NEJMoa2021436
6. Beigel JH, Tomashek KM, Dodd LE, et al. Remdesivir for the treatment of covid-19 - final report. N Engl J Med. 2020;383(19):1813-1826. doi:10.1056/NEJMoa2007764
7. Durant TJS, Peaper DR, Ferguson D, Schulz WL. Impact of COVID-19 pandemic on laboratory utilization. J Appl Lab Med. 2020;5(6):1194-1205. doi:10.1093/jalm/jfaa121
8. Yagnik KJ, Saad HA, King HL, Bedimo RJ, Lehmann CU, Medford RJ. Characteristics and outcomes of infectious diseases electronic COVID-19 consultations at a multisite academic health system. Cureus. 2021;13(11):e19203. Published 2021 Nov 2. doi:10.7759/cureus.19203
9. Rawson TM, Moore LSP, Zhu N, et al. Bacterial and fungal coinfection in individuals with coronavirus: a rapid review to support COVID-19 antimicrobial prescribing. Clin Infect Dis. 2020;71(9):2459-2468. doi:10.1093/cid/ciaa530
10. Yarrington ME, Moehring RW. Basic, advanced, and novel metrics to guide antibiotic use assessments. Curr Treat Options Infect Dis. 2019;11(2):145-160. doi:10.1007/s40506-019-00188-3
11. Bai AD, Showler A, Burry L, et al. Impact of infectious disease consultation on quality of care, mortality, and length of stay in Staphylococcus aureus bacteremia: results from a large multicenter cohort study. Clin Infect Dis. 2015;60(10):1451-1461. doi:10.1093/cid/civ120
12. Mejia-Chew C, O’Halloran JA, Olsen MA, et al. Effect of infectious disease consultation on mortality and treatment of patients with candida bloodstream infections: a retrospective, cohort study. Lancet Infect Dis. 2019;19(12):1336-1344. doi:10.1016/S1473-3099(19)30405-0
13. Coronavirus Disease 2019 (COVID-19) Treatment Guidelines. National Institutes of Health (US); April 21, 2021. Accessed February 14, 2023. https://files.covid19treatmentguidelines.nih.gov/guidelines/covid19treatmentguidelines.pdf
14. Bhimraj A, Morgan RL, Shumaker AH, et al. Infectious Diseases Society of America guidelines on the treatment and management of patients with COVID-19. Clin Infect Dis. 2020;ciaa478. doi:10.1093/cid/ciaa478
15. Suraj V, Del Vecchio Fitz C, Kleiman LB, et al. SMART COVID Navigator, a clinical decision support tool for COVID-19 treatment: design and development study. J Med Internet Res. 2022;24(2):e29279. Published 2022 Feb 18. doi:10.2196/29279
16. Pendharkar SR, Minty E, Shukalek CB, et al. Description of a multi-faceted COVID-19 pandemic physician workforce plan at a multi-site academic health system. J Gen Intern Med. 2021;36(5):1310-1318. doi:10.1007/s11606-020-06543-1
17. Pulia MS, Wolf I, Schulz LT, Pop-Vicas A, Schwei RJ, Lindenauer PK. COVID-19: an emerging threat to antibiotic stewardship in the emergency department. West J Emerg Med. 2020;21(5):1283-1286. Published 2020 Aug 7. doi:10.5811/westjem.2020.7.48848
Throughout the COVID-19 pandemic, health care professionals (HCPs), including emergency medicine physicians and hospitalists, have been continuously challenged to maintain an up-to-date clinical practice on COVID-19 therapeutics as new evidence emerged.1,2 In the early part of the pandemic, these included not only appropriate and time-sensitive prescriptions of COVID-19 therapeutics, such as remdesivir and dexamethasone, but also judicious use of empiric antibiotics given the low prevalence for bacterial coinfection in early disease.3-6 Alongside this, curbing the excessive laboratory testing of these patients during the pandemic was important not only to minimize costs but also to reduce potential iatrogenic harm and extended length of stay (LOS).7
At the beginning of the pandemic in March 2020 at the US Department of Veterans Affairs (VA) North Texas Health Care System (VANTHCS) Dallas VA Medical Center (DVAMC), we attempted to provide therapeutic guidance for physicians primarily through direct infectious disease (ID) consultation (in-person or electronic).8 This was secondarily supported by a pharmacist and ID physician–curated “living guidance” document on COVID-19 care accessible to all physicians through the DVAMC electronic health record (EHR) and intranet.
As the alpha variant (lineage B.1.1.7) of COVID-19 began spreading throughout North Texas in the winter of 2020, we implemented a targeted educational intervention toward the hospitalist group taking care of patients with COVID-19 with the primary goal of improving the accuracy of COVID-19 therapeutics while minimizing the consultative burden on ID clinical and pharmacy staff. This initiative consisted of (1) proactive guideline dissemination through email and text messages; (2) virtual didactics; and (3) physician reminders during the consultation process. Our ultimate aims were to improve hospitalist-led appropriate prescriptions of remdesivir and dexamethasone, reducing empiric antibiotic days of therapy in patients with COVID-19 at low risk of bacterial coinfection, and reducing laboratory orders that were not indicated for the management of these patients. Following this intervention and the resolution of the second wave, we retrospectively assessed the temporal trends of COVID-19 practices by hospitalists and associated patterns of ID consultation in the DVAMC from October 1, 2020, to March 31, 2021.
METHODS
The educational intervention was carried out at the DVAMC, a 1A high complex facility with more than 200 inpatient beds and part of the VANTHCS. During the study period, patients admitted with COVID-19 were located either on a closed floor (managed by the hospitalist team) or in a closed intensive care unit (ICU) (managed by the pulmonary/critical care team) contingent on the level of care or oxygen supplementation required. ID and other subspecialties provided consultation services as requested by hospitalists or ICU teams either electronically or in person. During the study period, 66 hospitalists were involved in the care of the patients: 59 (89.5%) permanent staff, 4 (6.0%) fee-basis physicians, and 3 (4.5%) moonlighting fellows.
Educational Initiative
We delivered educational sessions to the hospitalists, using collaboration software with video meeting capability every 1 to 2 months beginning in December 2020. An additional session focused on reducing empiric antibiotic prescriptions was also delivered to the emergency medicine department, based on feedback from the hospitalist group. The content for the educational sessions came from informal surveys of both ID trainees assigned to the consultation service and hospitalists, covering the following topics: understanding the stages of COVID-19 illness (virologic replication vs inflammatory) and rationales for therapy; assessing disease severity; indications and use of remdesivir; indications and use of dexamethasone; assessing for bacterial coinfections; when an ID consultation is required; management algorithm for COVID-19; and locating guidelines on the intranet. About 15 to 20 physicians participated in each session. In addition, slides of these didactics and updated institutional COVID-19 guidelines were disseminated to the hospitalist group via email and text messaging. We also linked the intranet institution guidelines in our communication, including a revised user-friendly flowchart (eAppendix).
Laboratory Stewardship Initiative
Laboratory stewardship initiatives were implemented by modifying suggested orders on the admission of patients with COVID-19 and directly educating hospitalist and emergency medicine physicians on evidence-based laboratory orders. At the beginning of the pandemic, a broad admission order set was established at DVAMC, based on the then limited knowledge of the course of infection with COVID-19. This order set allowed the admitting physicians to efficiently order laboratory tests for patients, especially during the demanding increase in patient volume experienced by DVAMC.
As new evidence emerged during the pandemic, many of the laboratory orders were reviewed for clinical utility during care for the patient with COVID-19 per the latest guidance. In December 2020, the admission orders for patients with COVID-19 were revised to reflect better laboratory stewardship to reduce cost and harm. The ID section revised the laboratory orders and disseminated the new order set to admitting physicians. Specifically, the admission order set removed the following laboratory tests available for selection: routine blood cultures, interleukin 6 (IL-6) level, and Legionella sputum culture. These laboratory orders were removed based on the lack of supporting evidence in persons admitted with COVID-19.9 In addition to modification of the admission order set, educational sessions were held with hospitalists to disseminate knowledge of the new changes and address any concerns.
Observations of Care
This study was approved by the VANTHCS Institutional Review Board (protocol code 20-047). Records were retrospectively reviewed for patients admitted to DVAMC for COVID-19 under hospitalist care (patients admitted directly to the ICU were excluded) from October 1, 2020, to March 31, 2021. Age, sex, race and ethnicity, and comorbidities were collected from the EHR. In addition clinical measures such as maximum oxygen requirement during admission (none, nasal cannula of 2-4 L/min, high flow/bilevel positive airway pressure [BiPAP] or mechanical ventilation), proven presence of coinfection (defined as the isolation of a probable pathogen in pure culture and/or clinically determined by ID specialist evaluation), and the average LOS also were collected. For laboratory stewardship data, a retrospective chart review was conducted to determine the total number of blood cultures obtained within 24 hours of admission per month during the study period. Both IL-6 levels and Legionella sputum culture data were collected as the total number of laboratory orders per month, as it was assumed that most of these orders were obtained for patients admitted with COVID-19.
Individual patient-level data were extracted to calculate monthly percentages of ID consultations for COVID-19 by the hospitalist team, adherence to institutional guidelines for dexamethasone and remdesivir prescriptions, and empiric antibiotic prescriptions for patients with COVID-19, including use of a priori adjudication criteria to determine justified vs unjustified empiric use. These criteria included asymmetric chest X-ray infiltrates concerning for bacterial pneumonia; peripheral white blood cell count > 11 K/μL; critical respiratory failure in the emergency department (ED) and being transferred to the ICU; and ID consultation recommended. Because the total number of antibiotics was not being analyzed but rather just the use of antibiotics for the justified and unjustified groups, antibiotic days were reported as the length of therapy (LOT).10 A subset analysis was performed on antibiotic prescriptions by the hospitalist group focusing on those with mild-to-moderate oxygen requirements (no high flow, noninvasive or invasive ventilatory methods) and excluding infections with a proven microbiologic entity.
Differences in demographic and clinical characteristics of patients with COVID-19 admitted from October 1, 2020, to March 31, 2021, were assessed using ANOVA, χ2, and Kruskal-Wallis test. χ2 was used to compare the difference in total laboratory orders for routine blood cultures, IL-6 levels, and Legionella sputum cultures between pre-intervention (October to December 2020) and postintervention (January to March 2021). These pre- and postintervention periods were determined based on the timing of revised admission orders in the EHR and initiation of focused educational sessions starting in late December 2020 and early January 2021. Linear regressions were used to examine the possible 6-month trend of the percentage of patients receiving ID consultation for appropriate dexamethasone prescriptions, appropriate remdesivir prescriptions, appropriate antibiotic coadministration, and mean number of antibiotic days per patient. Linear and logistic regression were also used to assess the trend in LOS over the 6 months while adjusting for age, race and ethnicity, sex, and coinfections. All analyses were performed using SAS 9.4. Statistical significance was defined as P < .05.
RESULTS
From October 1, 2020, to March 31, 2021, there were 565 admissions for COVID-19, which peaked in January 2021 with 163. Analysis of the patient characteristics showed no statistically significant difference for age, sex, oxygen requirements during admission, or proven presence of coinfection between the months of interest (Table 1).
The number of blood cultures obtained in the first 24 hours of admission significantly decreased from 58.1% of admissions in October 2020 to 34.8% of admissions in March 2021 (P < .01) (Table 2).
We observed trends that coincided with the educational efforts. The rate of dexamethasone and remdesivir prescriptions for eligible patients that followed guidelines without ID consultation grew from 0% to 22.2% (P < .01) and 0% to 16.7% (P = .01), respectively. The remaining correct prescriptions for dexamethasone or remdesivir were instituted only after ID consultation. These improvements were seen in tandem with decreased reliance on ID consultation for admitted patients with COVID-19 overall (86.5% in October 2020 to 56.5% in March 2021; P < .01).
After applying a priori justified antibiotic use criteria, we found that the overall degree of empiric unjustified antibiotic use remained high for patients admitted with COVID-19 (36.5%-60.3%) and was largely driven by prescriptions from the ED. However, further analysis revealed a statistically significant decrease in empiric antibiotic LOT per patient during the study period from 3.0 days in October 2020 to 0.9 days in March 2021 (P < .01). In addition, there was a statistically significant change in the mean (SD) LOS, which decreased from 16.3 (17.8) days in October 2020 to 9.7 (13.0) days in March 2021 (P = .02).
DISCUSSION
As the COVID-19 pandemic has evolved, the ability to enact up-to-date guidance is crucial to streamlining patient care, improving time to COVID-19–specific therapies, and minimizing the burden on subspecialty consultation services. At DVAMC, we initiated a targeted and deliberate educational effort directed toward hospitalist and ED groups combined with a laboratory stewardship effort over 6 months to improve the implementation of COVID-19 therapeutics, reduce empiric antibiotic use without reliance on ID consultation services, and reduce the number of unnecessary laboratory orders for admitted patients with COVID-19. During this time, we observed modest but statistically significant improvements in the accuracy of dexamethasone and remdesivir prescribing. In addition, we observed statistically significant improvement in the average LOT per patient regarding antibiotic use and overall decreased LOS. These improvements were seen in parallel with decreasing requests for ID consultation, suggesting that they were attributable in part to increasing self-confidence and efficacy in COVID-19 practices by the hospitalist group. Modification of the COVID-19 admission order set for our facility resulted in substantial decreases in orders for blood cultures, IL-6 levels, and sputum cultures for Legionella.
ID consultation, either in person or remotely, has been instrumental in assisting physicians in COVID-19 management and has been shown to reduce morbidity, mortality, and patient LOS in other infections.11,12 However, in scenarios where ID consultation is not available or in limited supply, accessibility, familiarity, and confidence of primary practitioners to use therapeutic guidance material are integral. Frequent and accessible guidance for the management of COVID-19 has been provided by the National Institutes of Health and the Infectious Diseases Society of America.13,14 Other mechanisms of assisting physicians in both test ordering and therapeutics include clinical decision support tools built into the EHR and the use of a smartphone digital application.15 Guidance needs to be adapted to the context of the facility, including available resources and specific restrictions and/or prohibitions on therapeutics (eg, mandatory ID consultation or approval). In our facility, while COVID-19 therapeutic living guidance documents were maintained and accessible through the intranet, proactive dissemination and redirection were important steps in enabling the use of these documents.
Limitations
We acknowledge several limitations to this study. Most important, the correlations we observed do not represent causation. Our analysis was not designed to ascertain the direct impact of any single or combined educational and laboratory stewardship intervention from this study, and we acknowledge that the improvements in part could be related to increased experience and confidence with COVID-19 management that occurred over time independent of our programs. Furthermore, we acknowledge that several areas of COVID-19 management did not improve over time (such as overall empiric antibiotic use from the ED) or had very modest improvements (hospitalist-initiated remdesivir use). These results underscore the complex dynamics and contextual barriers to rapidly implementing guideline-based care at VANTHCS. Potential factors include insufficient reach to all physicians, variable learner motivation, and therapeutic momentum of antibiotic use carried forward from the ED.16,17 These factors should be considered as grounds for further study. Another limitation was the inability to track viewership and engagement of our COVID-19 guidance document. Without the use metrics, it is difficult to know the individual impact of the document regarding the changing trends in COVID-19 management we observed during the study period.
Conclusions
We report improvements in COVID-19 therapeutic prescriptions and the use of antibiotics and laboratory testing over 6 months at the DVAMC. This was correlated with a deliberate COVID-19 educational initiative that included antibiotic and laboratory stewardship interventions with simultaneous decreased reliance on ID consultation. These efforts lend support to the proof of the principle of combined educational and laboratory stewardship interventions to improve the care of COVID-19 patients, especially where ID support may not be available or is accessed remotely.
Throughout the COVID-19 pandemic, health care professionals (HCPs), including emergency medicine physicians and hospitalists, have been continuously challenged to maintain an up-to-date clinical practice on COVID-19 therapeutics as new evidence emerged.1,2 In the early part of the pandemic, these included not only appropriate and time-sensitive prescriptions of COVID-19 therapeutics, such as remdesivir and dexamethasone, but also judicious use of empiric antibiotics given the low prevalence for bacterial coinfection in early disease.3-6 Alongside this, curbing the excessive laboratory testing of these patients during the pandemic was important not only to minimize costs but also to reduce potential iatrogenic harm and extended length of stay (LOS).7
At the beginning of the pandemic in March 2020 at the US Department of Veterans Affairs (VA) North Texas Health Care System (VANTHCS) Dallas VA Medical Center (DVAMC), we attempted to provide therapeutic guidance for physicians primarily through direct infectious disease (ID) consultation (in-person or electronic).8 This was secondarily supported by a pharmacist and ID physician–curated “living guidance” document on COVID-19 care accessible to all physicians through the DVAMC electronic health record (EHR) and intranet.
As the alpha variant (lineage B.1.1.7) of COVID-19 began spreading throughout North Texas in the winter of 2020, we implemented a targeted educational intervention toward the hospitalist group taking care of patients with COVID-19 with the primary goal of improving the accuracy of COVID-19 therapeutics while minimizing the consultative burden on ID clinical and pharmacy staff. This initiative consisted of (1) proactive guideline dissemination through email and text messages; (2) virtual didactics; and (3) physician reminders during the consultation process. Our ultimate aims were to improve hospitalist-led appropriate prescriptions of remdesivir and dexamethasone, reducing empiric antibiotic days of therapy in patients with COVID-19 at low risk of bacterial coinfection, and reducing laboratory orders that were not indicated for the management of these patients. Following this intervention and the resolution of the second wave, we retrospectively assessed the temporal trends of COVID-19 practices by hospitalists and associated patterns of ID consultation in the DVAMC from October 1, 2020, to March 31, 2021.
METHODS
The educational intervention was carried out at the DVAMC, a 1A high complex facility with more than 200 inpatient beds and part of the VANTHCS. During the study period, patients admitted with COVID-19 were located either on a closed floor (managed by the hospitalist team) or in a closed intensive care unit (ICU) (managed by the pulmonary/critical care team) contingent on the level of care or oxygen supplementation required. ID and other subspecialties provided consultation services as requested by hospitalists or ICU teams either electronically or in person. During the study period, 66 hospitalists were involved in the care of the patients: 59 (89.5%) permanent staff, 4 (6.0%) fee-basis physicians, and 3 (4.5%) moonlighting fellows.
Educational Initiative
We delivered educational sessions to the hospitalists, using collaboration software with video meeting capability every 1 to 2 months beginning in December 2020. An additional session focused on reducing empiric antibiotic prescriptions was also delivered to the emergency medicine department, based on feedback from the hospitalist group. The content for the educational sessions came from informal surveys of both ID trainees assigned to the consultation service and hospitalists, covering the following topics: understanding the stages of COVID-19 illness (virologic replication vs inflammatory) and rationales for therapy; assessing disease severity; indications and use of remdesivir; indications and use of dexamethasone; assessing for bacterial coinfections; when an ID consultation is required; management algorithm for COVID-19; and locating guidelines on the intranet. About 15 to 20 physicians participated in each session. In addition, slides of these didactics and updated institutional COVID-19 guidelines were disseminated to the hospitalist group via email and text messaging. We also linked the intranet institution guidelines in our communication, including a revised user-friendly flowchart (eAppendix).
Laboratory Stewardship Initiative
Laboratory stewardship initiatives were implemented by modifying suggested orders on the admission of patients with COVID-19 and directly educating hospitalist and emergency medicine physicians on evidence-based laboratory orders. At the beginning of the pandemic, a broad admission order set was established at DVAMC, based on the then limited knowledge of the course of infection with COVID-19. This order set allowed the admitting physicians to efficiently order laboratory tests for patients, especially during the demanding increase in patient volume experienced by DVAMC.
As new evidence emerged during the pandemic, many of the laboratory orders were reviewed for clinical utility during care for the patient with COVID-19 per the latest guidance. In December 2020, the admission orders for patients with COVID-19 were revised to reflect better laboratory stewardship to reduce cost and harm. The ID section revised the laboratory orders and disseminated the new order set to admitting physicians. Specifically, the admission order set removed the following laboratory tests available for selection: routine blood cultures, interleukin 6 (IL-6) level, and Legionella sputum culture. These laboratory orders were removed based on the lack of supporting evidence in persons admitted with COVID-19.9 In addition to modification of the admission order set, educational sessions were held with hospitalists to disseminate knowledge of the new changes and address any concerns.
Observations of Care
This study was approved by the VANTHCS Institutional Review Board (protocol code 20-047). Records were retrospectively reviewed for patients admitted to DVAMC for COVID-19 under hospitalist care (patients admitted directly to the ICU were excluded) from October 1, 2020, to March 31, 2021. Age, sex, race and ethnicity, and comorbidities were collected from the EHR. In addition clinical measures such as maximum oxygen requirement during admission (none, nasal cannula of 2-4 L/min, high flow/bilevel positive airway pressure [BiPAP] or mechanical ventilation), proven presence of coinfection (defined as the isolation of a probable pathogen in pure culture and/or clinically determined by ID specialist evaluation), and the average LOS also were collected. For laboratory stewardship data, a retrospective chart review was conducted to determine the total number of blood cultures obtained within 24 hours of admission per month during the study period. Both IL-6 levels and Legionella sputum culture data were collected as the total number of laboratory orders per month, as it was assumed that most of these orders were obtained for patients admitted with COVID-19.
Individual patient-level data were extracted to calculate monthly percentages of ID consultations for COVID-19 by the hospitalist team, adherence to institutional guidelines for dexamethasone and remdesivir prescriptions, and empiric antibiotic prescriptions for patients with COVID-19, including use of a priori adjudication criteria to determine justified vs unjustified empiric use. These criteria included asymmetric chest X-ray infiltrates concerning for bacterial pneumonia; peripheral white blood cell count > 11 K/μL; critical respiratory failure in the emergency department (ED) and being transferred to the ICU; and ID consultation recommended. Because the total number of antibiotics was not being analyzed but rather just the use of antibiotics for the justified and unjustified groups, antibiotic days were reported as the length of therapy (LOT).10 A subset analysis was performed on antibiotic prescriptions by the hospitalist group focusing on those with mild-to-moderate oxygen requirements (no high flow, noninvasive or invasive ventilatory methods) and excluding infections with a proven microbiologic entity.
Differences in demographic and clinical characteristics of patients with COVID-19 admitted from October 1, 2020, to March 31, 2021, were assessed using ANOVA, χ2, and Kruskal-Wallis test. χ2 was used to compare the difference in total laboratory orders for routine blood cultures, IL-6 levels, and Legionella sputum cultures between pre-intervention (October to December 2020) and postintervention (January to March 2021). These pre- and postintervention periods were determined based on the timing of revised admission orders in the EHR and initiation of focused educational sessions starting in late December 2020 and early January 2021. Linear regressions were used to examine the possible 6-month trend of the percentage of patients receiving ID consultation for appropriate dexamethasone prescriptions, appropriate remdesivir prescriptions, appropriate antibiotic coadministration, and mean number of antibiotic days per patient. Linear and logistic regression were also used to assess the trend in LOS over the 6 months while adjusting for age, race and ethnicity, sex, and coinfections. All analyses were performed using SAS 9.4. Statistical significance was defined as P < .05.
RESULTS
From October 1, 2020, to March 31, 2021, there were 565 admissions for COVID-19, which peaked in January 2021 with 163. Analysis of the patient characteristics showed no statistically significant difference for age, sex, oxygen requirements during admission, or proven presence of coinfection between the months of interest (Table 1).
The number of blood cultures obtained in the first 24 hours of admission significantly decreased from 58.1% of admissions in October 2020 to 34.8% of admissions in March 2021 (P < .01) (Table 2).
We observed trends that coincided with the educational efforts. The rate of dexamethasone and remdesivir prescriptions for eligible patients that followed guidelines without ID consultation grew from 0% to 22.2% (P < .01) and 0% to 16.7% (P = .01), respectively. The remaining correct prescriptions for dexamethasone or remdesivir were instituted only after ID consultation. These improvements were seen in tandem with decreased reliance on ID consultation for admitted patients with COVID-19 overall (86.5% in October 2020 to 56.5% in March 2021; P < .01).
After applying a priori justified antibiotic use criteria, we found that the overall degree of empiric unjustified antibiotic use remained high for patients admitted with COVID-19 (36.5%-60.3%) and was largely driven by prescriptions from the ED. However, further analysis revealed a statistically significant decrease in empiric antibiotic LOT per patient during the study period from 3.0 days in October 2020 to 0.9 days in March 2021 (P < .01). In addition, there was a statistically significant change in the mean (SD) LOS, which decreased from 16.3 (17.8) days in October 2020 to 9.7 (13.0) days in March 2021 (P = .02).
DISCUSSION
As the COVID-19 pandemic has evolved, the ability to enact up-to-date guidance is crucial to streamlining patient care, improving time to COVID-19–specific therapies, and minimizing the burden on subspecialty consultation services. At DVAMC, we initiated a targeted and deliberate educational effort directed toward hospitalist and ED groups combined with a laboratory stewardship effort over 6 months to improve the implementation of COVID-19 therapeutics, reduce empiric antibiotic use without reliance on ID consultation services, and reduce the number of unnecessary laboratory orders for admitted patients with COVID-19. During this time, we observed modest but statistically significant improvements in the accuracy of dexamethasone and remdesivir prescribing. In addition, we observed statistically significant improvement in the average LOT per patient regarding antibiotic use and overall decreased LOS. These improvements were seen in parallel with decreasing requests for ID consultation, suggesting that they were attributable in part to increasing self-confidence and efficacy in COVID-19 practices by the hospitalist group. Modification of the COVID-19 admission order set for our facility resulted in substantial decreases in orders for blood cultures, IL-6 levels, and sputum cultures for Legionella.
ID consultation, either in person or remotely, has been instrumental in assisting physicians in COVID-19 management and has been shown to reduce morbidity, mortality, and patient LOS in other infections.11,12 However, in scenarios where ID consultation is not available or in limited supply, accessibility, familiarity, and confidence of primary practitioners to use therapeutic guidance material are integral. Frequent and accessible guidance for the management of COVID-19 has been provided by the National Institutes of Health and the Infectious Diseases Society of America.13,14 Other mechanisms of assisting physicians in both test ordering and therapeutics include clinical decision support tools built into the EHR and the use of a smartphone digital application.15 Guidance needs to be adapted to the context of the facility, including available resources and specific restrictions and/or prohibitions on therapeutics (eg, mandatory ID consultation or approval). In our facility, while COVID-19 therapeutic living guidance documents were maintained and accessible through the intranet, proactive dissemination and redirection were important steps in enabling the use of these documents.
Limitations
We acknowledge several limitations to this study. Most important, the correlations we observed do not represent causation. Our analysis was not designed to ascertain the direct impact of any single or combined educational and laboratory stewardship intervention from this study, and we acknowledge that the improvements in part could be related to increased experience and confidence with COVID-19 management that occurred over time independent of our programs. Furthermore, we acknowledge that several areas of COVID-19 management did not improve over time (such as overall empiric antibiotic use from the ED) or had very modest improvements (hospitalist-initiated remdesivir use). These results underscore the complex dynamics and contextual barriers to rapidly implementing guideline-based care at VANTHCS. Potential factors include insufficient reach to all physicians, variable learner motivation, and therapeutic momentum of antibiotic use carried forward from the ED.16,17 These factors should be considered as grounds for further study. Another limitation was the inability to track viewership and engagement of our COVID-19 guidance document. Without the use metrics, it is difficult to know the individual impact of the document regarding the changing trends in COVID-19 management we observed during the study period.
Conclusions
We report improvements in COVID-19 therapeutic prescriptions and the use of antibiotics and laboratory testing over 6 months at the DVAMC. This was correlated with a deliberate COVID-19 educational initiative that included antibiotic and laboratory stewardship interventions with simultaneous decreased reliance on ID consultation. These efforts lend support to the proof of the principle of combined educational and laboratory stewardship interventions to improve the care of COVID-19 patients, especially where ID support may not be available or is accessed remotely.
1. Dagens A, Sigfrid L, Cai E, et al. Scope, quality, and inclusivity of clinical guidelines produced early in the covid-19 pandemic: rapid review. BMJ. 2020;369:m1936. Published 2020 May 26. doi:10.1136/bmj.m1936
2. Dhivagaran T, Abbas U, Butt F, Arunasalam L, Chang O. Critical appraisal of clinical practice guidelines for the management of COVID-19: protocol for a systematic review. Syst Rev. 2021;10(1):317. Published 2021 Dec 22. doi:10.1186/s13643-021-01871-7
3. Garcia-Vidal C, Sanjuan G, Moreno-García E, et al. Incidence of co-infections and superinfections in hospitalized patients with COVID-19: a retrospective cohort study. Clin Microbiol Infect. 2021;27(1):83-88. doi:10.1016/j.cmi.2020.07.041
4. Karaba SM, Jones G, Helsel T, et al. Prevalence of co-infection at the time of hospital admission in covid-19 patients, a multicenter study. Open Forum Infect Dis. 2020;8(1):ofaa578. Published 2020 Dec 21. doi:10.1093/ofid/ofaa578
5. RECOVERY Collaborative Group, Horby P, Lim WS, et al. Dexamethasone in hospitalized patients with Covid-19. N Engl J Med. 2021;384(8):693-704. doi:10.1056/NEJMoa2021436
6. Beigel JH, Tomashek KM, Dodd LE, et al. Remdesivir for the treatment of covid-19 - final report. N Engl J Med. 2020;383(19):1813-1826. doi:10.1056/NEJMoa2007764
7. Durant TJS, Peaper DR, Ferguson D, Schulz WL. Impact of COVID-19 pandemic on laboratory utilization. J Appl Lab Med. 2020;5(6):1194-1205. doi:10.1093/jalm/jfaa121
8. Yagnik KJ, Saad HA, King HL, Bedimo RJ, Lehmann CU, Medford RJ. Characteristics and outcomes of infectious diseases electronic COVID-19 consultations at a multisite academic health system. Cureus. 2021;13(11):e19203. Published 2021 Nov 2. doi:10.7759/cureus.19203
9. Rawson TM, Moore LSP, Zhu N, et al. Bacterial and fungal coinfection in individuals with coronavirus: a rapid review to support COVID-19 antimicrobial prescribing. Clin Infect Dis. 2020;71(9):2459-2468. doi:10.1093/cid/ciaa530
10. Yarrington ME, Moehring RW. Basic, advanced, and novel metrics to guide antibiotic use assessments. Curr Treat Options Infect Dis. 2019;11(2):145-160. doi:10.1007/s40506-019-00188-3
11. Bai AD, Showler A, Burry L, et al. Impact of infectious disease consultation on quality of care, mortality, and length of stay in Staphylococcus aureus bacteremia: results from a large multicenter cohort study. Clin Infect Dis. 2015;60(10):1451-1461. doi:10.1093/cid/civ120
12. Mejia-Chew C, O’Halloran JA, Olsen MA, et al. Effect of infectious disease consultation on mortality and treatment of patients with candida bloodstream infections: a retrospective, cohort study. Lancet Infect Dis. 2019;19(12):1336-1344. doi:10.1016/S1473-3099(19)30405-0
13. Coronavirus Disease 2019 (COVID-19) Treatment Guidelines. National Institutes of Health (US); April 21, 2021. Accessed February 14, 2023. https://files.covid19treatmentguidelines.nih.gov/guidelines/covid19treatmentguidelines.pdf
14. Bhimraj A, Morgan RL, Shumaker AH, et al. Infectious Diseases Society of America guidelines on the treatment and management of patients with COVID-19. Clin Infect Dis. 2020;ciaa478. doi:10.1093/cid/ciaa478
15. Suraj V, Del Vecchio Fitz C, Kleiman LB, et al. SMART COVID Navigator, a clinical decision support tool for COVID-19 treatment: design and development study. J Med Internet Res. 2022;24(2):e29279. Published 2022 Feb 18. doi:10.2196/29279
16. Pendharkar SR, Minty E, Shukalek CB, et al. Description of a multi-faceted COVID-19 pandemic physician workforce plan at a multi-site academic health system. J Gen Intern Med. 2021;36(5):1310-1318. doi:10.1007/s11606-020-06543-1
17. Pulia MS, Wolf I, Schulz LT, Pop-Vicas A, Schwei RJ, Lindenauer PK. COVID-19: an emerging threat to antibiotic stewardship in the emergency department. West J Emerg Med. 2020;21(5):1283-1286. Published 2020 Aug 7. doi:10.5811/westjem.2020.7.48848
1. Dagens A, Sigfrid L, Cai E, et al. Scope, quality, and inclusivity of clinical guidelines produced early in the covid-19 pandemic: rapid review. BMJ. 2020;369:m1936. Published 2020 May 26. doi:10.1136/bmj.m1936
2. Dhivagaran T, Abbas U, Butt F, Arunasalam L, Chang O. Critical appraisal of clinical practice guidelines for the management of COVID-19: protocol for a systematic review. Syst Rev. 2021;10(1):317. Published 2021 Dec 22. doi:10.1186/s13643-021-01871-7
3. Garcia-Vidal C, Sanjuan G, Moreno-García E, et al. Incidence of co-infections and superinfections in hospitalized patients with COVID-19: a retrospective cohort study. Clin Microbiol Infect. 2021;27(1):83-88. doi:10.1016/j.cmi.2020.07.041
4. Karaba SM, Jones G, Helsel T, et al. Prevalence of co-infection at the time of hospital admission in covid-19 patients, a multicenter study. Open Forum Infect Dis. 2020;8(1):ofaa578. Published 2020 Dec 21. doi:10.1093/ofid/ofaa578
5. RECOVERY Collaborative Group, Horby P, Lim WS, et al. Dexamethasone in hospitalized patients with Covid-19. N Engl J Med. 2021;384(8):693-704. doi:10.1056/NEJMoa2021436
6. Beigel JH, Tomashek KM, Dodd LE, et al. Remdesivir for the treatment of covid-19 - final report. N Engl J Med. 2020;383(19):1813-1826. doi:10.1056/NEJMoa2007764
7. Durant TJS, Peaper DR, Ferguson D, Schulz WL. Impact of COVID-19 pandemic on laboratory utilization. J Appl Lab Med. 2020;5(6):1194-1205. doi:10.1093/jalm/jfaa121
8. Yagnik KJ, Saad HA, King HL, Bedimo RJ, Lehmann CU, Medford RJ. Characteristics and outcomes of infectious diseases electronic COVID-19 consultations at a multisite academic health system. Cureus. 2021;13(11):e19203. Published 2021 Nov 2. doi:10.7759/cureus.19203
9. Rawson TM, Moore LSP, Zhu N, et al. Bacterial and fungal coinfection in individuals with coronavirus: a rapid review to support COVID-19 antimicrobial prescribing. Clin Infect Dis. 2020;71(9):2459-2468. doi:10.1093/cid/ciaa530
10. Yarrington ME, Moehring RW. Basic, advanced, and novel metrics to guide antibiotic use assessments. Curr Treat Options Infect Dis. 2019;11(2):145-160. doi:10.1007/s40506-019-00188-3
11. Bai AD, Showler A, Burry L, et al. Impact of infectious disease consultation on quality of care, mortality, and length of stay in Staphylococcus aureus bacteremia: results from a large multicenter cohort study. Clin Infect Dis. 2015;60(10):1451-1461. doi:10.1093/cid/civ120
12. Mejia-Chew C, O’Halloran JA, Olsen MA, et al. Effect of infectious disease consultation on mortality and treatment of patients with candida bloodstream infections: a retrospective, cohort study. Lancet Infect Dis. 2019;19(12):1336-1344. doi:10.1016/S1473-3099(19)30405-0
13. Coronavirus Disease 2019 (COVID-19) Treatment Guidelines. National Institutes of Health (US); April 21, 2021. Accessed February 14, 2023. https://files.covid19treatmentguidelines.nih.gov/guidelines/covid19treatmentguidelines.pdf
14. Bhimraj A, Morgan RL, Shumaker AH, et al. Infectious Diseases Society of America guidelines on the treatment and management of patients with COVID-19. Clin Infect Dis. 2020;ciaa478. doi:10.1093/cid/ciaa478
15. Suraj V, Del Vecchio Fitz C, Kleiman LB, et al. SMART COVID Navigator, a clinical decision support tool for COVID-19 treatment: design and development study. J Med Internet Res. 2022;24(2):e29279. Published 2022 Feb 18. doi:10.2196/29279
16. Pendharkar SR, Minty E, Shukalek CB, et al. Description of a multi-faceted COVID-19 pandemic physician workforce plan at a multi-site academic health system. J Gen Intern Med. 2021;36(5):1310-1318. doi:10.1007/s11606-020-06543-1
17. Pulia MS, Wolf I, Schulz LT, Pop-Vicas A, Schwei RJ, Lindenauer PK. COVID-19: an emerging threat to antibiotic stewardship in the emergency department. West J Emerg Med. 2020;21(5):1283-1286. Published 2020 Aug 7. doi:10.5811/westjem.2020.7.48848