Angela M Statile, MD, MEd

Febrile infants aged 0 to 60 days often undergo diagnostic testing to evaluate for invasive bacterial infections (IBI; ie, bacteremia and meningitis) and are subsequently hospitalized pending culture results. Only 1% to 2% of infants 0 to 60 days old have an IBI,^1-3 and most hospitalized infants are discharged once physicians feel confident that pathogens are unlikely to be isolated from blood and cerebrospinal fluid (CSF) cultures. Practice regarding duration of hospitalization while awaiting blood and CSF culture results is not standardized in this population. Longer hospitalizations can lead to increased costs and familial stress, including difficulty with breastfeeding and anxiety in newly postpartum mothers.^4,5

In 2010, an institutional evidence-based guideline for the management of febrile infants aged 0 to 60 days recommended discharge after 36 hours of observation if all cultures were negative.⁶ However, recent studies demonstrate that 85% to 93% of pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 Assuming a 2% prevalence of IBI, if 15% of pathogens were identified after 24 hours of incubation, only one out of 333 infants would have an IBI identified after 24 hours of hospital observation.⁷

Furthermore, a review of our institution’s electronic health records (EHR) over the past 5 years revealed that an observation period of 24 hours would have resulted in the discharge of three infants with an IBI. Two infants had bacteremia; both were discharged from the emergency department (ED) without antibiotics, returned to care after cultures were reported positive at 27 hours, and had no adverse outcomes. The third infant had meningitis, but also had an abnormal CSF Gram stain, which led to a longer hospitalization.

In 2019, our institution appraised the emerging literature and institutional data supporting the low absolute risk of missed IBI, and also leveraged local consensus among key stakeholders to update its evidence-based guideline for the evaluation and management of febrile infants aged 60 days and younger. The updated guideline recommends that clinicians consider discharging well-appearing neonates and infants if blood and CSF cultures remain negative at 24 hours.¹⁰ The objective of this study was to decrease the average hospital culture observation time (COT; culture incubation to hospital discharge) from 38 to 30 hours over a 12-month period in febrile infants aged 0 to 60 days.

Context

Improvement efforts were conducted at Cincinnati Children’s Hospital Medical Center (CCHMC), a large, urban, academic hospital that admitted more than 8,000 noncritically ill patients to the hospital medicine (HM) service from July 1, 2018, through June 30, 2019. Hospital medicine teams, located at both the main and satellite campuses, are staffed by attending physicians, fellows, residents, medical students, and nurse practitioners. The two campuses, which are about 20 miles apart, share clinician providers but have distinct nursing pools.

Microbiology services for all CCHMC patients are provided at the main campus. Blood and CSF cultures at the satellite campus are transported to the main campus for incubation and monitoring via an urgent courier service. The microbiology laboratory at CCHMC uses a continuous monitoring system for blood cultures (BACT/ALERT Virtuo, BioMérieux). The system automatically alerts laboratory technicians of positive cultures; these results are reported to clinical providers within 30 minutes of detection. Laboratory technicians manually evaluate CSF cultures once daily for 5 days.

Improvement Team

Our improvement team included three HM attending physicians; two HM fellows; a pediatric chief resident; two nurses, who represented nursing pools at the main and satellite campuses; and a clinical pharmacist, who is a co-leader of the antimicrobial stewardship program at CCHMC. Supporting members for the improvement team included the CCHMC laboratory director; the microbiology laboratory director; an infectious disease physician, who is a co-leader of the antimicrobial stewardship program; and nursing directors of the HM units at both campuses.

Evidence-Based Guideline

Our improvement initiative was based on recommendations from the updated CCHMC Evidence-Based Care Guideline for Management of Infants 0 to 60 days with Fever of Unknown Source.¹⁰ This guideline, published in May 2019, was developed by a multidisciplinary working group composed of key stakeholders from HM, community pediatrics, emergency medicine, the pediatric residency program, infectious disease, and laboratory medicine. Several improvement team members were participants on the committee that published the evidence-based guideline. The committee first performed a systematic literature review and critical appraisal of the literature. Care recommendations were formulated via a consensus process directed by best evidence, patient and family preferences, and clinical expertise; the recommendations were subsequently reviewed and approved by clinical experts who were not involved in the development process.

Based on evidence review and multistakeholder consensus, the updated guideline recommends clinicians consider discharging neonates and infants aged 60 days and younger if there is no culture growth after an observation period of 24 hours (as documented in the EHR) and patients are otherwise medically ready for discharge (ie, well appearing with adequate oral intake).^10,11 In addition, prior to discharge, there must be a documented working phone number on file for the patient’s parents/guardians, an established outpatient follow-up plan within 24 hours, and communication with the primary pediatrician who is in agreement with discharge at 24 hours.

Study Population

Infants 0 to 60 days old who had a documented or reported fever without an apparent source based on history and physical exam upon presentation to the ED, and who were subsequently admitted to the HM service at CCHMC between October 30, 2018, and July 10, 2020, were eligible for inclusion. We excluded infants who were admitted to other clinical services (eg, intensive care unit); had organisms identified on blood, urine, or CSF culture within 24 hours of incubation; had positive herpes simplex virus testing; had skin/soft tissue infections or another clearly documented source of bacterial infection; or had an alternative indication for hospitalization (eg, need for intravenous fluid or deep suctioning) after cultures had incubated for 24 hours. Infants who had a positive blood, urine, or CSF culture result after 24 hours of incubation were included in the study population. Organisms were classified as pathogen or contaminant based on treatment decisions made by the care team.

Improvement Activities

Key drivers critical to success of the improvement efforts were: (1) clearly defined standard of care for duration of observation in febrile infants 0 to 60 days old; (2) improved understanding of microbiology lab procedures; (3) effective communication of discharge criteria between providers and nurses; and (4) transparency of data with feedback (Figure 1).

Education and Structured Dissemination of Evidence-Based Guideline

The CCHMC febrile infant guideline¹⁰ was disseminated to HM physicians, residents, and nurses via the following means: (1) in-person announcements at staff meetings and educational conferences, (2) published highlights from the guideline in weekly newsletters, and (3) email announcements. Additionally, members of the study team educated HM attending physicians, nursing staff from the medical units at both campuses, and resident physicians about recent studies demonstrating safety of shorter length of stay (LOS) in febrile infants aged 0 to 60 days. The study team also provided residents, physicians, and nurses with data on the number of positive blood and CSF cultures and outcomes of patients at CCHMC within the past 5 years. In addition, team members led a journal club for residents discussing an article⁷ describing time-to-positivity of blood and CSF cultures in febrile infants. For ongoing engagement, the evidence-based guideline and a detailed explanation of microbiology procedures were published in the resident handbook, an internal resource that includes vital clinical pearls and practice guidelines across specialties. (Each resident receives an updated hard copy each year, and there is also an online link to the resource in the EHR.) Information about the guideline and COT was also included in the monthly chief resident’s orientation script, which is relayed to all residents on the first day of their HM rotation.

Clear Communication of Microbiology Procedures

Team members created a detailed process map describing the processing protocols for blood and CSF cultures collected at both CCHMC campuses. This information was shared with HM attending physicians and nurses via in-person announcements at staff meetings, flyers in team workrooms, and email communications. Residents received information on microbiology protocols via in-person announcements at educational conferences and dissemination in the weekly residency newsletter.Important information communicated included:

1. Definition of culture start time. We conveyed that there may be a delay of up to 4 hours between culture collection at the satellite campus and culture incubation at the main campus laboratory. As a result, the time of blood or CSF sample arrival to the main campus laboratory was a more accurate reflection of the culture incubation start time than the culture collection time.

2. Explanation of CSF culture processing. We discussed the process by which these cultures are plated upon arrival at the microbiology laboratory and read once per day in the morning. Therefore, a culture incubated at midnight would be evaluated once at 9 hours and not again until 33 hours.

Modification of Febrile Infant Order Set

Enhancements to the febrile infant order set improved communication and cultivated a shared mental model regarding discharge goals among all members of the care team. The EHR order set for febrile infants was updated as follows: (1) mandatory free-text fields that established the culture start time for blood and CSF cultures were added, (2) culture start time was clearly defined (ie, the time culture arrives at the main campus laboratory), and (3) a change was made in the default discharge criteria¹¹ to “culture observation for 24 hours,” with the ability to modify COT (Appendix Figure 1). We embedded hyperlinks to the guideline and microbiology process map within the updated order set, which allowed providers to easily access this information and refresh their knowledge of the recommendations (Appendix Figure 1).

Identification of Failures and Follow-up With Near-Time Feedback

All cases of febrile infants were tracked weekly. For infants hospitalized longer than 24 hours, the study team contacted the discharging clinicians to discuss reasons for prolonged hospitalization, with an emphasis on identifying system-level barriers to earlier discharge.

Study of the Interventions

The institutional microbiology database was queried weekly to identify all infants 0 to 60 days old who had a blood culture obtained and were hospitalized on the HM service. Study team members conducted targeted EHR review to determine whether patients met exclusion criteria and to identify reasons for prolonged COT. Baseline data were collected retrospectively for a 3-month period prior to initiation of improvement activities. During the study period, queries were conducted weekly and reviewed by study team members to evaluate the impact of improvement activities and to inform new interventions.

Measures

Our primary outcome measure was COT, defined as the hours between final culture incubation and hospital discharge. The operational definition for “final culture incubation” was the documented time of arrival of the last collected culture to the microbiology laboratory. Our goal COT was 30 hours to account for a subset of patients whose blood and/or CSF culture were obtained overnight (ie, after 9 pm), since subsequent discharge times would likely and practically be delayed beyond 24 hours. Our secondary outcome measure was LOS, defined as the time between ED arrival and hospital discharge. Process measures included the proportion of patients for whom the febrile infant EHR order set was used and the proportion of patients for whom medical discharge criteria (ie, blood and CSF culture observed for ”xx” hours) and culture incubation start times were entered using the order set. Balancing measures included identification of IBI after hospital discharge, 48-hour ED revisits, and 7-day hospital readmissions.

Analysis

Measures were evaluated using statistical process control charts and run charts, and Western Electric rules were employed to determine special cause variation.¹² Annotated X-bar S control charts tracked the impact of improvement activities on average COT and LOS for all infants. Given that a relatively small number of patients (ie, two to four) met inclusion criteria each week, average COT was calculated per five patients.

This study was considered exempt from review by the CCHMC Institutional Review Board.

Of the 184 infants in this study, 46 were included as part of baseline data collection, and 138 were included during the intervention period. The median age was 26.6 days (range, 3-59 days); 52% of patients were female; two-thirds were non-Hispanic White; 22% were Black, and 5% were Hispanic (Appendix Table).

Average COT decreased from 38 hours to 32 hours with improvement activities (Figure 2) and was sustained for a total of 17 months. There were small decreases in COT after initial education was provided to attendings, nurses, and residents.

After the launch of the updated order set, median usage of the EHR order set increased from 50% to 80%. Medical discharge criteria were entered for 80 (96%) of the 83 patients for whom the updated order set was applied; culture incubation start times were entered for 78 (94%) of these patients.

No infants in our cohort were found to have IBI after hospital discharge. There were no ED revisits within 48 hours of discharge, and there were no hospital readmissions within 7 days of index discharge. Furthermore, none of the patients included in the study had growth of a pathogenic organism after 24 hours.

Of the 138 infants hospitalized during the intervention period, 77 (56%) had a COT greater than 30 hours. Among these 77 patients, 49 (64%) had their final culture incubated between 9 pm and 4 am; Furthermore, 11 (14%) had missing, abnormal, pretreated, or uninterpretable CSF studies, 7 (9%) had ongoing fevers, and 4 (5%) remained hospitalized due to family preference or inability to obtain timely outpatient follow-up.

Our study aimed to decrease the average COT from 38 hours to 30 hours among hospitalized infants aged 60 days and younger over a period of 12 months. An intervention featuring implementation of an evidence-based guideline through education, laboratory procedure transparency, creation of a standardized EHR order set, and near-time feedback was associated with a shorter average COT of 32 hours, sustained over a 17-month period. No infants with bacteremia or meningitis were inappropriately discharged during this study.

Interpretation

Prior to our improvement efforts, most febrile infants at CCHMC were observed for at least 36 hours based on a prior institutional guideline,⁶ despite recent evidence suggesting that most pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 The goal of this improvement initiative was to bridge the gap between emerging evidence and clinical practice by developing and disseminating an updated evidence-based guideline to safely decrease the hospital observation time in febrile infants aged 60 days and younger.

Similar to previous studies aimed at improving diagnosis and management among febrile infants,^13-16 generation and structured dissemination of an institutional evidence-based guideline was crucial to safely shortening COT in our population. These prior studies established a goal COT of 36 to 42 hours for hospitalized febrile infants.^13,15,16 Our study incorporated emerging evidence and local experience into an updated evidence-based practice guideline to further reduce COT to 32 hours for hospitalized infants. Key factors contributing to our success included multidisciplinary engagement, specifically partnering with nurses and resident physicians in designing and implementing our initiatives. Furthermore, improved transparency of culture monitoring practices allowed clinicians to better understand the recommended observation periods. Finally, we employed a standardized EHR order set as a no-cost, one-time, high-reliability intervention to establish 24 hours of culture monitoring as the default and to enhance transparency around start time for culture incubation.

Average COT remained stable at 32 hours for 17 months after initiation of the intervention. During the intervention period, 64% patients with hospital stays longer than 30 hours had cultures obtained between 9 pm to 4 am. These patients often remained hospitalized for longer than 30 hours to allow for a daytime hospital discharge. Additionally, CSF cultures were only monitored manually once per day between 8 am and 10 am. As a result, CSF cultures obtained in the evening (eg, 9 pm) would be evaluated once at roughly 12 hours of incubation, and then the following morning at 36 hours of incubation. In cases where CSF studies (eg, cell count, protein, Gram stain) were abnormal, uninterpretable, or could not be obtained, clinicians monitored CSF cultures closer to 36 hours from incubation. While evidence-based guidelines and local data support safe early discharge of febrile infants, clinicians presented with incomplete or uninterpretable data were appropriately more likely to observe infants for longer periods to confirm negative cultures.

Limitations

The study has several limitations. First, this single-center study was conducted at a quaternary care medical center with a robust quality improvement infrastructure. Our interventions took advantage of the existing processes in place that ensure timely discharge of medically ready patients.¹¹ Furthermore, microbiology laboratory practices are unique to our institution. These factors limit the generalizability of this work. Second, due to small numbers of eligible infants, analyses were conducted per five patients. Infrequent hospitalizations limited our ability to learn quickly from PDSA cycles. Finally, we did not measure cost savings attributable to shorter hospital stays. However, in addition to financial savings from charges and decreased nonmedical costs such as lost earnings and childcare,¹⁷ shorter hospitalizations have many additional benefits, such as promoting bonding and breastfeeding and decreasing exposure to nosocomial infections. Shorter hospitalizations, with clearly communicated discharge times, also serve to optimize patient throughput.

Implementation of a clinical practice guideline resulted in reduction of average COT from 38 to 32 hours in febrile infants aged 60 days and younger, with no cases of missed IBI. Engagement of multidisciplinary stakeholders in the generation and structured dissemination of the evidence-based guideline, improved transparency of the microbiological blood and CSF culture process, and standardization of EHR order sets were crucial to the success of this work. Cultures incubated overnight and daily CSF culture-monitoring practices primarily contributed to an average LOS of more than 30 hours.

Future work will include collaboration with emergency physicians to improve evaluation efficiency and decrease LOS in the ED for febrile infants. Additionally, creation of an automated data dashboard of COT and LOS will provide clinicians with real-time feedback on hospitalization practices.

The authors thank Dr Jeffrey Simmons, MD, MSc, as well as the members of the 2019 Fever of Uncertain Source Evidence-Based Guideline Committee. We also thank the James M Anderson Center for Health System Excellence and the Rapid Cycle Improvement Collaborative for their support with guideline development as well as design and execution of our improvement efforts.

Febrile infants aged 0 to 60 days often undergo diagnostic testing to evaluate for invasive bacterial infections (IBI; ie, bacteremia and meningitis) and are subsequently hospitalized pending culture results. Only 1% to 2% of infants 0 to 60 days old have an IBI,^1-3 and most hospitalized infants are discharged once physicians feel confident that pathogens are unlikely to be isolated from blood and cerebrospinal fluid (CSF) cultures. Practice regarding duration of hospitalization while awaiting blood and CSF culture results is not standardized in this population. Longer hospitalizations can lead to increased costs and familial stress, including difficulty with breastfeeding and anxiety in newly postpartum mothers.^4,5

In 2010, an institutional evidence-based guideline for the management of febrile infants aged 0 to 60 days recommended discharge after 36 hours of observation if all cultures were negative.⁶ However, recent studies demonstrate that 85% to 93% of pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 Assuming a 2% prevalence of IBI, if 15% of pathogens were identified after 24 hours of incubation, only one out of 333 infants would have an IBI identified after 24 hours of hospital observation.⁷

Furthermore, a review of our institution’s electronic health records (EHR) over the past 5 years revealed that an observation period of 24 hours would have resulted in the discharge of three infants with an IBI. Two infants had bacteremia; both were discharged from the emergency department (ED) without antibiotics, returned to care after cultures were reported positive at 27 hours, and had no adverse outcomes. The third infant had meningitis, but also had an abnormal CSF Gram stain, which led to a longer hospitalization.

In 2019, our institution appraised the emerging literature and institutional data supporting the low absolute risk of missed IBI, and also leveraged local consensus among key stakeholders to update its evidence-based guideline for the evaluation and management of febrile infants aged 60 days and younger. The updated guideline recommends that clinicians consider discharging well-appearing neonates and infants if blood and CSF cultures remain negative at 24 hours.¹⁰ The objective of this study was to decrease the average hospital culture observation time (COT; culture incubation to hospital discharge) from 38 to 30 hours over a 12-month period in febrile infants aged 0 to 60 days.

Context

Improvement efforts were conducted at Cincinnati Children’s Hospital Medical Center (CCHMC), a large, urban, academic hospital that admitted more than 8,000 noncritically ill patients to the hospital medicine (HM) service from July 1, 2018, through June 30, 2019. Hospital medicine teams, located at both the main and satellite campuses, are staffed by attending physicians, fellows, residents, medical students, and nurse practitioners. The two campuses, which are about 20 miles apart, share clinician providers but have distinct nursing pools.

Microbiology services for all CCHMC patients are provided at the main campus. Blood and CSF cultures at the satellite campus are transported to the main campus for incubation and monitoring via an urgent courier service. The microbiology laboratory at CCHMC uses a continuous monitoring system for blood cultures (BACT/ALERT Virtuo, BioMérieux). The system automatically alerts laboratory technicians of positive cultures; these results are reported to clinical providers within 30 minutes of detection. Laboratory technicians manually evaluate CSF cultures once daily for 5 days.

Improvement Team

Our improvement team included three HM attending physicians; two HM fellows; a pediatric chief resident; two nurses, who represented nursing pools at the main and satellite campuses; and a clinical pharmacist, who is a co-leader of the antimicrobial stewardship program at CCHMC. Supporting members for the improvement team included the CCHMC laboratory director; the microbiology laboratory director; an infectious disease physician, who is a co-leader of the antimicrobial stewardship program; and nursing directors of the HM units at both campuses.

Evidence-Based Guideline

Our improvement initiative was based on recommendations from the updated CCHMC Evidence-Based Care Guideline for Management of Infants 0 to 60 days with Fever of Unknown Source.¹⁰ This guideline, published in May 2019, was developed by a multidisciplinary working group composed of key stakeholders from HM, community pediatrics, emergency medicine, the pediatric residency program, infectious disease, and laboratory medicine. Several improvement team members were participants on the committee that published the evidence-based guideline. The committee first performed a systematic literature review and critical appraisal of the literature. Care recommendations were formulated via a consensus process directed by best evidence, patient and family preferences, and clinical expertise; the recommendations were subsequently reviewed and approved by clinical experts who were not involved in the development process.

Based on evidence review and multistakeholder consensus, the updated guideline recommends clinicians consider discharging neonates and infants aged 60 days and younger if there is no culture growth after an observation period of 24 hours (as documented in the EHR) and patients are otherwise medically ready for discharge (ie, well appearing with adequate oral intake).^10,11 In addition, prior to discharge, there must be a documented working phone number on file for the patient’s parents/guardians, an established outpatient follow-up plan within 24 hours, and communication with the primary pediatrician who is in agreement with discharge at 24 hours.

Study Population

Infants 0 to 60 days old who had a documented or reported fever without an apparent source based on history and physical exam upon presentation to the ED, and who were subsequently admitted to the HM service at CCHMC between October 30, 2018, and July 10, 2020, were eligible for inclusion. We excluded infants who were admitted to other clinical services (eg, intensive care unit); had organisms identified on blood, urine, or CSF culture within 24 hours of incubation; had positive herpes simplex virus testing; had skin/soft tissue infections or another clearly documented source of bacterial infection; or had an alternative indication for hospitalization (eg, need for intravenous fluid or deep suctioning) after cultures had incubated for 24 hours. Infants who had a positive blood, urine, or CSF culture result after 24 hours of incubation were included in the study population. Organisms were classified as pathogen or contaminant based on treatment decisions made by the care team.

Improvement Activities

Key drivers critical to success of the improvement efforts were: (1) clearly defined standard of care for duration of observation in febrile infants 0 to 60 days old; (2) improved understanding of microbiology lab procedures; (3) effective communication of discharge criteria between providers and nurses; and (4) transparency of data with feedback (Figure 1).

Education and Structured Dissemination of Evidence-Based Guideline

The CCHMC febrile infant guideline¹⁰ was disseminated to HM physicians, residents, and nurses via the following means: (1) in-person announcements at staff meetings and educational conferences, (2) published highlights from the guideline in weekly newsletters, and (3) email announcements. Additionally, members of the study team educated HM attending physicians, nursing staff from the medical units at both campuses, and resident physicians about recent studies demonstrating safety of shorter length of stay (LOS) in febrile infants aged 0 to 60 days. The study team also provided residents, physicians, and nurses with data on the number of positive blood and CSF cultures and outcomes of patients at CCHMC within the past 5 years. In addition, team members led a journal club for residents discussing an article⁷ describing time-to-positivity of blood and CSF cultures in febrile infants. For ongoing engagement, the evidence-based guideline and a detailed explanation of microbiology procedures were published in the resident handbook, an internal resource that includes vital clinical pearls and practice guidelines across specialties. (Each resident receives an updated hard copy each year, and there is also an online link to the resource in the EHR.) Information about the guideline and COT was also included in the monthly chief resident’s orientation script, which is relayed to all residents on the first day of their HM rotation.

Clear Communication of Microbiology Procedures

Team members created a detailed process map describing the processing protocols for blood and CSF cultures collected at both CCHMC campuses. This information was shared with HM attending physicians and nurses via in-person announcements at staff meetings, flyers in team workrooms, and email communications. Residents received information on microbiology protocols via in-person announcements at educational conferences and dissemination in the weekly residency newsletter.Important information communicated included:

1. Definition of culture start time. We conveyed that there may be a delay of up to 4 hours between culture collection at the satellite campus and culture incubation at the main campus laboratory. As a result, the time of blood or CSF sample arrival to the main campus laboratory was a more accurate reflection of the culture incubation start time than the culture collection time.

2. Explanation of CSF culture processing. We discussed the process by which these cultures are plated upon arrival at the microbiology laboratory and read once per day in the morning. Therefore, a culture incubated at midnight would be evaluated once at 9 hours and not again until 33 hours.

Modification of Febrile Infant Order Set

Enhancements to the febrile infant order set improved communication and cultivated a shared mental model regarding discharge goals among all members of the care team. The EHR order set for febrile infants was updated as follows: (1) mandatory free-text fields that established the culture start time for blood and CSF cultures were added, (2) culture start time was clearly defined (ie, the time culture arrives at the main campus laboratory), and (3) a change was made in the default discharge criteria¹¹ to “culture observation for 24 hours,” with the ability to modify COT (Appendix Figure 1). We embedded hyperlinks to the guideline and microbiology process map within the updated order set, which allowed providers to easily access this information and refresh their knowledge of the recommendations (Appendix Figure 1).

Identification of Failures and Follow-up With Near-Time Feedback

All cases of febrile infants were tracked weekly. For infants hospitalized longer than 24 hours, the study team contacted the discharging clinicians to discuss reasons for prolonged hospitalization, with an emphasis on identifying system-level barriers to earlier discharge.

Study of the Interventions

The institutional microbiology database was queried weekly to identify all infants 0 to 60 days old who had a blood culture obtained and were hospitalized on the HM service. Study team members conducted targeted EHR review to determine whether patients met exclusion criteria and to identify reasons for prolonged COT. Baseline data were collected retrospectively for a 3-month period prior to initiation of improvement activities. During the study period, queries were conducted weekly and reviewed by study team members to evaluate the impact of improvement activities and to inform new interventions.

Measures

Our primary outcome measure was COT, defined as the hours between final culture incubation and hospital discharge. The operational definition for “final culture incubation” was the documented time of arrival of the last collected culture to the microbiology laboratory. Our goal COT was 30 hours to account for a subset of patients whose blood and/or CSF culture were obtained overnight (ie, after 9 pm), since subsequent discharge times would likely and practically be delayed beyond 24 hours. Our secondary outcome measure was LOS, defined as the time between ED arrival and hospital discharge. Process measures included the proportion of patients for whom the febrile infant EHR order set was used and the proportion of patients for whom medical discharge criteria (ie, blood and CSF culture observed for ”xx” hours) and culture incubation start times were entered using the order set. Balancing measures included identification of IBI after hospital discharge, 48-hour ED revisits, and 7-day hospital readmissions.

Analysis

Measures were evaluated using statistical process control charts and run charts, and Western Electric rules were employed to determine special cause variation.¹² Annotated X-bar S control charts tracked the impact of improvement activities on average COT and LOS for all infants. Given that a relatively small number of patients (ie, two to four) met inclusion criteria each week, average COT was calculated per five patients.

This study was considered exempt from review by the CCHMC Institutional Review Board.

Of the 184 infants in this study, 46 were included as part of baseline data collection, and 138 were included during the intervention period. The median age was 26.6 days (range, 3-59 days); 52% of patients were female; two-thirds were non-Hispanic White; 22% were Black, and 5% were Hispanic (Appendix Table).

Average COT decreased from 38 hours to 32 hours with improvement activities (Figure 2) and was sustained for a total of 17 months. There were small decreases in COT after initial education was provided to attendings, nurses, and residents.

After the launch of the updated order set, median usage of the EHR order set increased from 50% to 80%. Medical discharge criteria were entered for 80 (96%) of the 83 patients for whom the updated order set was applied; culture incubation start times were entered for 78 (94%) of these patients.

No infants in our cohort were found to have IBI after hospital discharge. There were no ED revisits within 48 hours of discharge, and there were no hospital readmissions within 7 days of index discharge. Furthermore, none of the patients included in the study had growth of a pathogenic organism after 24 hours.

Of the 138 infants hospitalized during the intervention period, 77 (56%) had a COT greater than 30 hours. Among these 77 patients, 49 (64%) had their final culture incubated between 9 pm and 4 am; Furthermore, 11 (14%) had missing, abnormal, pretreated, or uninterpretable CSF studies, 7 (9%) had ongoing fevers, and 4 (5%) remained hospitalized due to family preference or inability to obtain timely outpatient follow-up.

Our study aimed to decrease the average COT from 38 hours to 30 hours among hospitalized infants aged 60 days and younger over a period of 12 months. An intervention featuring implementation of an evidence-based guideline through education, laboratory procedure transparency, creation of a standardized EHR order set, and near-time feedback was associated with a shorter average COT of 32 hours, sustained over a 17-month period. No infants with bacteremia or meningitis were inappropriately discharged during this study.

Interpretation

Prior to our improvement efforts, most febrile infants at CCHMC were observed for at least 36 hours based on a prior institutional guideline,⁶ despite recent evidence suggesting that most pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 The goal of this improvement initiative was to bridge the gap between emerging evidence and clinical practice by developing and disseminating an updated evidence-based guideline to safely decrease the hospital observation time in febrile infants aged 60 days and younger.

Similar to previous studies aimed at improving diagnosis and management among febrile infants,^13-16 generation and structured dissemination of an institutional evidence-based guideline was crucial to safely shortening COT in our population. These prior studies established a goal COT of 36 to 42 hours for hospitalized febrile infants.^13,15,16 Our study incorporated emerging evidence and local experience into an updated evidence-based practice guideline to further reduce COT to 32 hours for hospitalized infants. Key factors contributing to our success included multidisciplinary engagement, specifically partnering with nurses and resident physicians in designing and implementing our initiatives. Furthermore, improved transparency of culture monitoring practices allowed clinicians to better understand the recommended observation periods. Finally, we employed a standardized EHR order set as a no-cost, one-time, high-reliability intervention to establish 24 hours of culture monitoring as the default and to enhance transparency around start time for culture incubation.

Average COT remained stable at 32 hours for 17 months after initiation of the intervention. During the intervention period, 64% patients with hospital stays longer than 30 hours had cultures obtained between 9 pm to 4 am. These patients often remained hospitalized for longer than 30 hours to allow for a daytime hospital discharge. Additionally, CSF cultures were only monitored manually once per day between 8 am and 10 am. As a result, CSF cultures obtained in the evening (eg, 9 pm) would be evaluated once at roughly 12 hours of incubation, and then the following morning at 36 hours of incubation. In cases where CSF studies (eg, cell count, protein, Gram stain) were abnormal, uninterpretable, or could not be obtained, clinicians monitored CSF cultures closer to 36 hours from incubation. While evidence-based guidelines and local data support safe early discharge of febrile infants, clinicians presented with incomplete or uninterpretable data were appropriately more likely to observe infants for longer periods to confirm negative cultures.

Limitations

The study has several limitations. First, this single-center study was conducted at a quaternary care medical center with a robust quality improvement infrastructure. Our interventions took advantage of the existing processes in place that ensure timely discharge of medically ready patients.¹¹ Furthermore, microbiology laboratory practices are unique to our institution. These factors limit the generalizability of this work. Second, due to small numbers of eligible infants, analyses were conducted per five patients. Infrequent hospitalizations limited our ability to learn quickly from PDSA cycles. Finally, we did not measure cost savings attributable to shorter hospital stays. However, in addition to financial savings from charges and decreased nonmedical costs such as lost earnings and childcare,¹⁷ shorter hospitalizations have many additional benefits, such as promoting bonding and breastfeeding and decreasing exposure to nosocomial infections. Shorter hospitalizations, with clearly communicated discharge times, also serve to optimize patient throughput.

Implementation of a clinical practice guideline resulted in reduction of average COT from 38 to 32 hours in febrile infants aged 60 days and younger, with no cases of missed IBI. Engagement of multidisciplinary stakeholders in the generation and structured dissemination of the evidence-based guideline, improved transparency of the microbiological blood and CSF culture process, and standardization of EHR order sets were crucial to the success of this work. Cultures incubated overnight and daily CSF culture-monitoring practices primarily contributed to an average LOS of more than 30 hours.

Future work will include collaboration with emergency physicians to improve evaluation efficiency and decrease LOS in the ED for febrile infants. Additionally, creation of an automated data dashboard of COT and LOS will provide clinicians with real-time feedback on hospitalization practices.

The authors thank Dr Jeffrey Simmons, MD, MSc, as well as the members of the 2019 Fever of Uncertain Source Evidence-Based Guideline Committee. We also thank the James M Anderson Center for Health System Excellence and the Rapid Cycle Improvement Collaborative for their support with guideline development as well as design and execution of our improvement efforts.

Febrile infants aged 0 to 60 days often undergo diagnostic testing to evaluate for invasive bacterial infections (IBI; ie, bacteremia and meningitis) and are subsequently hospitalized pending culture results. Only 1% to 2% of infants 0 to 60 days old have an IBI,^1-3 and most hospitalized infants are discharged once physicians feel confident that pathogens are unlikely to be isolated from blood and cerebrospinal fluid (CSF) cultures. Practice regarding duration of hospitalization while awaiting blood and CSF culture results is not standardized in this population. Longer hospitalizations can lead to increased costs and familial stress, including difficulty with breastfeeding and anxiety in newly postpartum mothers.^4,5

In 2010, an institutional evidence-based guideline for the management of febrile infants aged 0 to 60 days recommended discharge after 36 hours of observation if all cultures were negative.⁶ However, recent studies demonstrate that 85% to 93% of pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 Assuming a 2% prevalence of IBI, if 15% of pathogens were identified after 24 hours of incubation, only one out of 333 infants would have an IBI identified after 24 hours of hospital observation.⁷

Furthermore, a review of our institution’s electronic health records (EHR) over the past 5 years revealed that an observation period of 24 hours would have resulted in the discharge of three infants with an IBI. Two infants had bacteremia; both were discharged from the emergency department (ED) without antibiotics, returned to care after cultures were reported positive at 27 hours, and had no adverse outcomes. The third infant had meningitis, but also had an abnormal CSF Gram stain, which led to a longer hospitalization.

In 2019, our institution appraised the emerging literature and institutional data supporting the low absolute risk of missed IBI, and also leveraged local consensus among key stakeholders to update its evidence-based guideline for the evaluation and management of febrile infants aged 60 days and younger. The updated guideline recommends that clinicians consider discharging well-appearing neonates and infants if blood and CSF cultures remain negative at 24 hours.¹⁰ The objective of this study was to decrease the average hospital culture observation time (COT; culture incubation to hospital discharge) from 38 to 30 hours over a 12-month period in febrile infants aged 0 to 60 days.

Context

Improvement efforts were conducted at Cincinnati Children’s Hospital Medical Center (CCHMC), a large, urban, academic hospital that admitted more than 8,000 noncritically ill patients to the hospital medicine (HM) service from July 1, 2018, through June 30, 2019. Hospital medicine teams, located at both the main and satellite campuses, are staffed by attending physicians, fellows, residents, medical students, and nurse practitioners. The two campuses, which are about 20 miles apart, share clinician providers but have distinct nursing pools.

Microbiology services for all CCHMC patients are provided at the main campus. Blood and CSF cultures at the satellite campus are transported to the main campus for incubation and monitoring via an urgent courier service. The microbiology laboratory at CCHMC uses a continuous monitoring system for blood cultures (BACT/ALERT Virtuo, BioMérieux). The system automatically alerts laboratory technicians of positive cultures; these results are reported to clinical providers within 30 minutes of detection. Laboratory technicians manually evaluate CSF cultures once daily for 5 days.

Improvement Team

Our improvement team included three HM attending physicians; two HM fellows; a pediatric chief resident; two nurses, who represented nursing pools at the main and satellite campuses; and a clinical pharmacist, who is a co-leader of the antimicrobial stewardship program at CCHMC. Supporting members for the improvement team included the CCHMC laboratory director; the microbiology laboratory director; an infectious disease physician, who is a co-leader of the antimicrobial stewardship program; and nursing directors of the HM units at both campuses.

Evidence-Based Guideline

Our improvement initiative was based on recommendations from the updated CCHMC Evidence-Based Care Guideline for Management of Infants 0 to 60 days with Fever of Unknown Source.¹⁰ This guideline, published in May 2019, was developed by a multidisciplinary working group composed of key stakeholders from HM, community pediatrics, emergency medicine, the pediatric residency program, infectious disease, and laboratory medicine. Several improvement team members were participants on the committee that published the evidence-based guideline. The committee first performed a systematic literature review and critical appraisal of the literature. Care recommendations were formulated via a consensus process directed by best evidence, patient and family preferences, and clinical expertise; the recommendations were subsequently reviewed and approved by clinical experts who were not involved in the development process.

Based on evidence review and multistakeholder consensus, the updated guideline recommends clinicians consider discharging neonates and infants aged 60 days and younger if there is no culture growth after an observation period of 24 hours (as documented in the EHR) and patients are otherwise medically ready for discharge (ie, well appearing with adequate oral intake).^10,11 In addition, prior to discharge, there must be a documented working phone number on file for the patient’s parents/guardians, an established outpatient follow-up plan within 24 hours, and communication with the primary pediatrician who is in agreement with discharge at 24 hours.

Study Population

Infants 0 to 60 days old who had a documented or reported fever without an apparent source based on history and physical exam upon presentation to the ED, and who were subsequently admitted to the HM service at CCHMC between October 30, 2018, and July 10, 2020, were eligible for inclusion. We excluded infants who were admitted to other clinical services (eg, intensive care unit); had organisms identified on blood, urine, or CSF culture within 24 hours of incubation; had positive herpes simplex virus testing; had skin/soft tissue infections or another clearly documented source of bacterial infection; or had an alternative indication for hospitalization (eg, need for intravenous fluid or deep suctioning) after cultures had incubated for 24 hours. Infants who had a positive blood, urine, or CSF culture result after 24 hours of incubation were included in the study population. Organisms were classified as pathogen or contaminant based on treatment decisions made by the care team.

Improvement Activities

Key drivers critical to success of the improvement efforts were: (1) clearly defined standard of care for duration of observation in febrile infants 0 to 60 days old; (2) improved understanding of microbiology lab procedures; (3) effective communication of discharge criteria between providers and nurses; and (4) transparency of data with feedback (Figure 1).

Education and Structured Dissemination of Evidence-Based Guideline

The CCHMC febrile infant guideline¹⁰ was disseminated to HM physicians, residents, and nurses via the following means: (1) in-person announcements at staff meetings and educational conferences, (2) published highlights from the guideline in weekly newsletters, and (3) email announcements. Additionally, members of the study team educated HM attending physicians, nursing staff from the medical units at both campuses, and resident physicians about recent studies demonstrating safety of shorter length of stay (LOS) in febrile infants aged 0 to 60 days. The study team also provided residents, physicians, and nurses with data on the number of positive blood and CSF cultures and outcomes of patients at CCHMC within the past 5 years. In addition, team members led a journal club for residents discussing an article⁷ describing time-to-positivity of blood and CSF cultures in febrile infants. For ongoing engagement, the evidence-based guideline and a detailed explanation of microbiology procedures were published in the resident handbook, an internal resource that includes vital clinical pearls and practice guidelines across specialties. (Each resident receives an updated hard copy each year, and there is also an online link to the resource in the EHR.) Information about the guideline and COT was also included in the monthly chief resident’s orientation script, which is relayed to all residents on the first day of their HM rotation.

Clear Communication of Microbiology Procedures

Team members created a detailed process map describing the processing protocols for blood and CSF cultures collected at both CCHMC campuses. This information was shared with HM attending physicians and nurses via in-person announcements at staff meetings, flyers in team workrooms, and email communications. Residents received information on microbiology protocols via in-person announcements at educational conferences and dissemination in the weekly residency newsletter.Important information communicated included:

1. Definition of culture start time. We conveyed that there may be a delay of up to 4 hours between culture collection at the satellite campus and culture incubation at the main campus laboratory. As a result, the time of blood or CSF sample arrival to the main campus laboratory was a more accurate reflection of the culture incubation start time than the culture collection time.

2. Explanation of CSF culture processing. We discussed the process by which these cultures are plated upon arrival at the microbiology laboratory and read once per day in the morning. Therefore, a culture incubated at midnight would be evaluated once at 9 hours and not again until 33 hours.

Modification of Febrile Infant Order Set

Enhancements to the febrile infant order set improved communication and cultivated a shared mental model regarding discharge goals among all members of the care team. The EHR order set for febrile infants was updated as follows: (1) mandatory free-text fields that established the culture start time for blood and CSF cultures were added, (2) culture start time was clearly defined (ie, the time culture arrives at the main campus laboratory), and (3) a change was made in the default discharge criteria¹¹ to “culture observation for 24 hours,” with the ability to modify COT (Appendix Figure 1). We embedded hyperlinks to the guideline and microbiology process map within the updated order set, which allowed providers to easily access this information and refresh their knowledge of the recommendations (Appendix Figure 1).

Identification of Failures and Follow-up With Near-Time Feedback

All cases of febrile infants were tracked weekly. For infants hospitalized longer than 24 hours, the study team contacted the discharging clinicians to discuss reasons for prolonged hospitalization, with an emphasis on identifying system-level barriers to earlier discharge.

Study of the Interventions

The institutional microbiology database was queried weekly to identify all infants 0 to 60 days old who had a blood culture obtained and were hospitalized on the HM service. Study team members conducted targeted EHR review to determine whether patients met exclusion criteria and to identify reasons for prolonged COT. Baseline data were collected retrospectively for a 3-month period prior to initiation of improvement activities. During the study period, queries were conducted weekly and reviewed by study team members to evaluate the impact of improvement activities and to inform new interventions.

Measures

Our primary outcome measure was COT, defined as the hours between final culture incubation and hospital discharge. The operational definition for “final culture incubation” was the documented time of arrival of the last collected culture to the microbiology laboratory. Our goal COT was 30 hours to account for a subset of patients whose blood and/or CSF culture were obtained overnight (ie, after 9 pm), since subsequent discharge times would likely and practically be delayed beyond 24 hours. Our secondary outcome measure was LOS, defined as the time between ED arrival and hospital discharge. Process measures included the proportion of patients for whom the febrile infant EHR order set was used and the proportion of patients for whom medical discharge criteria (ie, blood and CSF culture observed for ”xx” hours) and culture incubation start times were entered using the order set. Balancing measures included identification of IBI after hospital discharge, 48-hour ED revisits, and 7-day hospital readmissions.

Analysis

Measures were evaluated using statistical process control charts and run charts, and Western Electric rules were employed to determine special cause variation.¹² Annotated X-bar S control charts tracked the impact of improvement activities on average COT and LOS for all infants. Given that a relatively small number of patients (ie, two to four) met inclusion criteria each week, average COT was calculated per five patients.

This study was considered exempt from review by the CCHMC Institutional Review Board.

Of the 184 infants in this study, 46 were included as part of baseline data collection, and 138 were included during the intervention period. The median age was 26.6 days (range, 3-59 days); 52% of patients were female; two-thirds were non-Hispanic White; 22% were Black, and 5% were Hispanic (Appendix Table).

Average COT decreased from 38 hours to 32 hours with improvement activities (Figure 2) and was sustained for a total of 17 months. There were small decreases in COT after initial education was provided to attendings, nurses, and residents.

After the launch of the updated order set, median usage of the EHR order set increased from 50% to 80%. Medical discharge criteria were entered for 80 (96%) of the 83 patients for whom the updated order set was applied; culture incubation start times were entered for 78 (94%) of these patients.

No infants in our cohort were found to have IBI after hospital discharge. There were no ED revisits within 48 hours of discharge, and there were no hospital readmissions within 7 days of index discharge. Furthermore, none of the patients included in the study had growth of a pathogenic organism after 24 hours.

Of the 138 infants hospitalized during the intervention period, 77 (56%) had a COT greater than 30 hours. Among these 77 patients, 49 (64%) had their final culture incubated between 9 pm and 4 am; Furthermore, 11 (14%) had missing, abnormal, pretreated, or uninterpretable CSF studies, 7 (9%) had ongoing fevers, and 4 (5%) remained hospitalized due to family preference or inability to obtain timely outpatient follow-up.

Our study aimed to decrease the average COT from 38 hours to 30 hours among hospitalized infants aged 60 days and younger over a period of 12 months. An intervention featuring implementation of an evidence-based guideline through education, laboratory procedure transparency, creation of a standardized EHR order set, and near-time feedback was associated with a shorter average COT of 32 hours, sustained over a 17-month period. No infants with bacteremia or meningitis were inappropriately discharged during this study.

Interpretation

Prior to our improvement efforts, most febrile infants at CCHMC were observed for at least 36 hours based on a prior institutional guideline,⁶ despite recent evidence suggesting that most pathogens in blood and CSF cultures grow within 24 hours of incubation.^7-9 The goal of this improvement initiative was to bridge the gap between emerging evidence and clinical practice by developing and disseminating an updated evidence-based guideline to safely decrease the hospital observation time in febrile infants aged 60 days and younger.

Similar to previous studies aimed at improving diagnosis and management among febrile infants,^13-16 generation and structured dissemination of an institutional evidence-based guideline was crucial to safely shortening COT in our population. These prior studies established a goal COT of 36 to 42 hours for hospitalized febrile infants.^13,15,16 Our study incorporated emerging evidence and local experience into an updated evidence-based practice guideline to further reduce COT to 32 hours for hospitalized infants. Key factors contributing to our success included multidisciplinary engagement, specifically partnering with nurses and resident physicians in designing and implementing our initiatives. Furthermore, improved transparency of culture monitoring practices allowed clinicians to better understand the recommended observation periods. Finally, we employed a standardized EHR order set as a no-cost, one-time, high-reliability intervention to establish 24 hours of culture monitoring as the default and to enhance transparency around start time for culture incubation.

Average COT remained stable at 32 hours for 17 months after initiation of the intervention. During the intervention period, 64% patients with hospital stays longer than 30 hours had cultures obtained between 9 pm to 4 am. These patients often remained hospitalized for longer than 30 hours to allow for a daytime hospital discharge. Additionally, CSF cultures were only monitored manually once per day between 8 am and 10 am. As a result, CSF cultures obtained in the evening (eg, 9 pm) would be evaluated once at roughly 12 hours of incubation, and then the following morning at 36 hours of incubation. In cases where CSF studies (eg, cell count, protein, Gram stain) were abnormal, uninterpretable, or could not be obtained, clinicians monitored CSF cultures closer to 36 hours from incubation. While evidence-based guidelines and local data support safe early discharge of febrile infants, clinicians presented with incomplete or uninterpretable data were appropriately more likely to observe infants for longer periods to confirm negative cultures.

Limitations

The study has several limitations. First, this single-center study was conducted at a quaternary care medical center with a robust quality improvement infrastructure. Our interventions took advantage of the existing processes in place that ensure timely discharge of medically ready patients.¹¹ Furthermore, microbiology laboratory practices are unique to our institution. These factors limit the generalizability of this work. Second, due to small numbers of eligible infants, analyses were conducted per five patients. Infrequent hospitalizations limited our ability to learn quickly from PDSA cycles. Finally, we did not measure cost savings attributable to shorter hospital stays. However, in addition to financial savings from charges and decreased nonmedical costs such as lost earnings and childcare,¹⁷ shorter hospitalizations have many additional benefits, such as promoting bonding and breastfeeding and decreasing exposure to nosocomial infections. Shorter hospitalizations, with clearly communicated discharge times, also serve to optimize patient throughput.

Implementation of a clinical practice guideline resulted in reduction of average COT from 38 to 32 hours in febrile infants aged 60 days and younger, with no cases of missed IBI. Engagement of multidisciplinary stakeholders in the generation and structured dissemination of the evidence-based guideline, improved transparency of the microbiological blood and CSF culture process, and standardization of EHR order sets were crucial to the success of this work. Cultures incubated overnight and daily CSF culture-monitoring practices primarily contributed to an average LOS of more than 30 hours.

Future work will include collaboration with emergency physicians to improve evaluation efficiency and decrease LOS in the ED for febrile infants. Additionally, creation of an automated data dashboard of COT and LOS will provide clinicians with real-time feedback on hospitalization practices.

The authors thank Dr Jeffrey Simmons, MD, MSc, as well as the members of the 2019 Fever of Uncertain Source Evidence-Based Guideline Committee. We also thank the James M Anderson Center for Health System Excellence and the Rapid Cycle Improvement Collaborative for their support with guideline development as well as design and execution of our improvement efforts.

Readmission rates are frequently used as a hospital quality metric, with use including payment incentive at the hospital level,¹ specific condition quality measurement,² balancing measures for quality improvement projects,^3-5 transition success,^6,7 and use in public hospital rankings.⁸ Currently, four methods are commonly used to evaluate pediatric readmissions, each with strengths and limitations, including the following (Appendix Table 1):

1. All-cause readmissions: A measure of any readmission within a given time period regardless of the reason for readmission.⁹

2. Unplanned readmission/time flag: A measure intended to identify unplanned readmissions. This measure relies on time designations within the electronic health record. The time between hospital registration and admission is calculated, and if the readmission is registered more than 24 hours prior to admission, the readmission is considered planned.¹⁰ Hereafter, this measure will be referred to as the time flag measure.

3. Pediatric all-condition readmission (PACR): A measure intended to identify unplanned readmission through the exclusion of certain procedures and diagnoses.¹¹

4. Potentially preventable readmission (PPR): A method to identify preventable readmissions based on a proprietary algorithm developed by 3M Health Information Systems.^12,13

While all four of these measures are used to assess quality, there is little known about these measures’ ability to exclude planned readmissions and identify only preventable pediatric readmission, which conceptually is most relevant to the quality of care. However, many of these measures were not intended to capture preventability, but instead capture the related issue of whether the readmission was planned. Therefore, we sought to evaluate the four readmission measures as they relate to both preventability and unplanned status as determined through medical record review with multidisciplinary care provider input.

As part of a hospital-wide readmission reduction quality improvement collaborative at a free-standing tertiary care children’s hospital, clinicians from hospital medicine, cardiology, neonatology, and neurology teams reviewed 30-day readmissions using a standardized abstraction tool. All readmission events (observation or inpatient encounter) after any discharge (observation or inpatient encounter) from eligible units were reviewed; therefore, each hospitalization was a potential index hospitalization. We classified the preventability of each readmission with use of a previously described Likert scale with high interrater reliability.¹⁴ For these analyses, readmissions were considered preventable if the reviewing team rated them as either “more likely preventable” or “preventable in most circumstances.” Each readmission was also evaluated as planned or unplanned. Methods for readmission review and classification are in the Appendix.

We included all readmissions between July 2014 and June 2016. We compared the medical record review classifications with the assessments from each of the four measures of pediatric readmission. We calculated sensitivity and specificity for both outcomes (planned/unplanned and preventable/not preventable) for all four measures. For standardization of discussion, we categorized description of measure performance as “very poor” as less than 50%, “poor” between 50%-75%, “fair” as 75%-85%, “good” as 85%-90%, “very good” as 90%-95% and excellent as greater than 95%. We also calculated positive and negative predictive value (PPV and NPV) over plausible ranges of prevalence using the sensitivity and specificity of each comparison (Appendix).

Of note, certain exclusions are outlined by the PACR and PPR algorithms. The PACR evaluates only readmission events that occur in children younger than 18 years. The PPR algorithm does not assign preventability if either the index or readmission event is classified as an observation stay or if it is part of a larger chain of readmissions.

Among 30-day readmissions considered, 1,643 were eligible for medical record review; 1,125 reviews were completed by the clinical teams (68.5%). The median time to readmission was 7 days (interquartile range [IQR], 4-18). Most children were non-Hispanic White (71%) or Black (20%). The median age at hospitalization was 2.3 years (IQR 0.4-12.1). Most children had Medicaid (56%) or private (41%) insurance. Most of the reviews were performed in cardiology (43%) and hospital medicine (37%) with patients in neurology (13%) and neonatology (7%) constituting the remaining reviews. Uncontrolled advancement of chronic disease was the most common readmission category on medical record review (25.1%), followed by unrelated readmission (20.7%), scheduled readmission (20.4%), and progression of acute disease (16.6%) (Appendix Table 2).

Assessment of Preventable and Unplanned Readmissions

On multidisciplinary medical record review, most readmissions were classified as not preventable (84.5%). Specifically, 64% were not preventable and unplanned; 20% were deemed not preventable and planned. Only 15% were classified as unplanned and preventable and 1% as planned and preventable (Appendix Figure: Population A/B).

Matching Chart Review to the Four Algorithms

All 1,125 readmissions were assessed by the all-cause and time flag readmission measures (Appendix Figure: Population A/B). After applying algorithm exclusions (details in Appendix), only 804 of the 1,125 (71.5%) reviewed readmissions matched for PACR readmission comparison (Appendix Figure: Population C); 487 of the 1,125 (43.3%) of the reviewed readmissions matched for PPR comparison (Appendix Figure: Population D).

All-Cause

Because all-cause determines only if a readmission occurs, the measure is by definition 100% sensitive and 0% specific in both assessment of preventability and unplanned readmission (Table: Section A).

Time Flag

The time flag measure identified 80% (866/1,112) of the readmissions as unplanned. This measure had very good sensitivity but very poor specificity in identifying preventable readmissions, which corresponded to very poor PPV and good to excellent NPV. In terms of identifying unplanned readmissions, the time flag measure had excellent sensitivity and very good specificity, which corresponded to very good to excellent PPV and good to very good NPV (Table: Section B).

PACR

The PACR algorithm identified 75% (599/796) of readmissions as unplanned. The PACR has good sensitivity but very poor specificity in identifying preventable readmissions, which corresponded to very poor PPV and fair to very good NPV. In terms of identifying unplanned readmissions, the PACR had fair sensitivity but poor specificity, which corresponded to fair PPV and poor NPV (Table: Section C).

PPR

The PPR algorithm identified 53% (257/487) of admissions as potentially preventable. The PPR algorithm had poor sensitivity and specificity in identifying preventable readmissions, which corresponded to very poor PPV and fair to very good NPV. In terms of identifying unplanned readmissions, the PPR algorithm had poor sensitivity and fair specificity in identifying unplanned readmissions, which corresponded to fair to good PPV and very poor to poor NPV (Table: Section D).

Evaluation of Excluded Readmission Events

Because both the PACR and PPR had large numbers of algorithm exclusions, we describe the preventability and unplanned assessment of the excluded readmission events. Both algorithms excluded preventable events. Of the 321 readmissions excluded by the PACR algorithm, 13.4% were classified as preventable by chart review. Likewise, 14.9% of 638 readmissions excluded by PPR were classified as preventable by chart review.

The ability to accurately capture preventable pediatric readmission is a goal for hospital quality experts and health policymakers alike. Of the four commonly used readmission measures to assess readmission, only PPR is designed to focus on preventability. Unfortunately, none of these four measures is adequately sensitive or specific to identify preventable readmissions; all measures had very poor PPV for preventability. Of the four measures, the time flag measure had the best sensitivity, specificity, PPV, and NPV for identifying unplanned readmissions.

The overall percentage of unplanned readmissions identified by both the time flag and by PACR measures match the overall percentage of unplanned readmissions identified in chart review: The time flag measure identified 80% of admissions as unplanned versus 79% identified by chart review (Appendix Figure: Population A/B); PACR classified 75% as unplanned versus 81% identified by chart review for PACR-eligible readmissions (Appendix Figure: Population C). In contrast, the PPR algorithm classified many more readmissions as potentially preventable (53%) than were identified by chart review at only 16% (Appendix Figure: Population D). The PACR and PPR algorithms also exclude a significant number of readmissions that are unplanned and a smaller, but not trivial, number of readmissions that are preventable; these exclusions limit their accuracy.

The ability to apply these four measures in real time during a hospitalization varies by metric. Two of the measures, the all-cause and time flag, can be applied during a readmission event, which is appealing for quality improvement initiatives. These measures allow for notification of providers that a current hospitalization is a readmission event, which allows providers the opportunity to learn from these events as they occur (Appendix Table 1). While “unplanned” is not the same as “potentially preventable,” almost all potentially preventable readmissions are unplanned; therefore, accurately identifying unplanned readmissions is more beneficial than all-cause. Additionally, a low all-cause readmission rate can be indicative of poor access to scheduled procedures. Nevertheless, all-cause readmission is sometimes used to measure quality.^1,8 While the time flag measure may be more useful for quality improvement initiatives and hospital providers, it relies on hospital registration time, which is not widely available in administrative data sources and, therefore, has limited usefulness to policymakers.

Both PACR and PPR require administrative claims analysis, which is appealing from a policy standpoint. However, the reliance on claims data means the inclusion/exclusion of events can occur only retrospectively, which limits the usefulness of these measures in learning and intervening in real time. When the two measures are compared, PACR offers better sensitivity and PPR offers better specificity with regard to identifying unplanned readmission. The PPR software overcalls preventable readmissions, identifying more readmissions as preventable than there actually are. Nevertheless, Medicaid in several states uses PPR for payment incentive.^1,15-17 Given the poor performance of PPR in assessing both preventable and unplanned pediatric readmission, the use of this measure as a quality metric should be limited.

This study should be considered in the context of several limitations. Because the assessment of preventability was determined as part of a learning quality improvement collaborative and not as a planned research endeavor, not all readmission reviews were completed nor were other existent tools¹⁸ that allow for preventability assessment via more structured medical record review used. Second, we reviewed cases only from certain clinical services, which would limit generalizability of these findings to all pediatric admissions. However, given the low sensitivity and specificity of some of the metrics, we would not anticipate that the addition of other types of admissions would improve the sensitivity and specificity enough to ensure reliability. Third, while we relied on an established method to determine preventability, prior work has demonstrated that additional information gathered from families may change preventability.¹⁹ Finally, due to the exclusions required by the PPR and PACR algorithms, not all readmission events were reviewed. However, these exclusions reflect the actual specifications of use for both measures.

The PPR software has poor fidelity in identifying preventable and unplanned pediatric readmission; this finding has broad policy implications given how widely it is used by state Medicaid offices to assess financial penalties. Among the four pediatric readmission measures used, the time flag metric best identifies unplanned readmissions.

The authors have no conflicts of interest or financial relationships relevant to this article to disclose.

Dr Auger’s research is supported by a grant from the Agency for Healthcare Research and Quality (1K08HS204735-01A1). The project described was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health, under Award Number 5UL1TR001425-04. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Readmission rates are frequently used as a hospital quality metric, with use including payment incentive at the hospital level,¹ specific condition quality measurement,² balancing measures for quality improvement projects,^3-5 transition success,^6,7 and use in public hospital rankings.⁸ Currently, four methods are commonly used to evaluate pediatric readmissions, each with strengths and limitations, including the following (Appendix Table 1):

1. All-cause readmissions: A measure of any readmission within a given time period regardless of the reason for readmission.⁹

2. Unplanned readmission/time flag: A measure intended to identify unplanned readmissions. This measure relies on time designations within the electronic health record. The time between hospital registration and admission is calculated, and if the readmission is registered more than 24 hours prior to admission, the readmission is considered planned.¹⁰ Hereafter, this measure will be referred to as the time flag measure.

3. Pediatric all-condition readmission (PACR): A measure intended to identify unplanned readmission through the exclusion of certain procedures and diagnoses.¹¹

4. Potentially preventable readmission (PPR): A method to identify preventable readmissions based on a proprietary algorithm developed by 3M Health Information Systems.^12,13

While all four of these measures are used to assess quality, there is little known about these measures’ ability to exclude planned readmissions and identify only preventable pediatric readmission, which conceptually is most relevant to the quality of care. However, many of these measures were not intended to capture preventability, but instead capture the related issue of whether the readmission was planned. Therefore, we sought to evaluate the four readmission measures as they relate to both preventability and unplanned status as determined through medical record review with multidisciplinary care provider input.

As part of a hospital-wide readmission reduction quality improvement collaborative at a free-standing tertiary care children’s hospital, clinicians from hospital medicine, cardiology, neonatology, and neurology teams reviewed 30-day readmissions using a standardized abstraction tool. All readmission events (observation or inpatient encounter) after any discharge (observation or inpatient encounter) from eligible units were reviewed; therefore, each hospitalization was a potential index hospitalization. We classified the preventability of each readmission with use of a previously described Likert scale with high interrater reliability.¹⁴ For these analyses, readmissions were considered preventable if the reviewing team rated them as either “more likely preventable” or “preventable in most circumstances.” Each readmission was also evaluated as planned or unplanned. Methods for readmission review and classification are in the Appendix.

We included all readmissions between July 2014 and June 2016. We compared the medical record review classifications with the assessments from each of the four measures of pediatric readmission. We calculated sensitivity and specificity for both outcomes (planned/unplanned and preventable/not preventable) for all four measures. For standardization of discussion, we categorized description of measure performance as “very poor” as less than 50%, “poor” between 50%-75%, “fair” as 75%-85%, “good” as 85%-90%, “very good” as 90%-95% and excellent as greater than 95%. We also calculated positive and negative predictive value (PPV and NPV) over plausible ranges of prevalence using the sensitivity and specificity of each comparison (Appendix).

Of note, certain exclusions are outlined by the PACR and PPR algorithms. The PACR evaluates only readmission events that occur in children younger than 18 years. The PPR algorithm does not assign preventability if either the index or readmission event is classified as an observation stay or if it is part of a larger chain of readmissions.

Among 30-day readmissions considered, 1,643 were eligible for medical record review; 1,125 reviews were completed by the clinical teams (68.5%). The median time to readmission was 7 days (interquartile range [IQR], 4-18). Most children were non-Hispanic White (71%) or Black (20%). The median age at hospitalization was 2.3 years (IQR 0.4-12.1). Most children had Medicaid (56%) or private (41%) insurance. Most of the reviews were performed in cardiology (43%) and hospital medicine (37%) with patients in neurology (13%) and neonatology (7%) constituting the remaining reviews. Uncontrolled advancement of chronic disease was the most common readmission category on medical record review (25.1%), followed by unrelated readmission (20.7%), scheduled readmission (20.4%), and progression of acute disease (16.6%) (Appendix Table 2).

Assessment of Preventable and Unplanned Readmissions

On multidisciplinary medical record review, most readmissions were classified as not preventable (84.5%). Specifically, 64% were not preventable and unplanned; 20% were deemed not preventable and planned. Only 15% were classified as unplanned and preventable and 1% as planned and preventable (Appendix Figure: Population A/B).

Matching Chart Review to the Four Algorithms

All 1,125 readmissions were assessed by the all-cause and time flag readmission measures (Appendix Figure: Population A/B). After applying algorithm exclusions (details in Appendix), only 804 of the 1,125 (71.5%) reviewed readmissions matched for PACR readmission comparison (Appendix Figure: Population C); 487 of the 1,125 (43.3%) of the reviewed readmissions matched for PPR comparison (Appendix Figure: Population D).

All-Cause

Because all-cause determines only if a readmission occurs, the measure is by definition 100% sensitive and 0% specific in both assessment of preventability and unplanned readmission (Table: Section A).

Time Flag

The time flag measure identified 80% (866/1,112) of the readmissions as unplanned. This measure had very good sensitivity but very poor specificity in identifying preventable readmissions, which corresponded to very poor PPV and good to excellent NPV. In terms of identifying unplanned readmissions, the time flag measure had excellent sensitivity and very good specificity, which corresponded to very good to excellent PPV and good to very good NPV (Table: Section B).

PACR

The PACR algorithm identified 75% (599/796) of readmissions as unplanned. The PACR has good sensitivity but very poor specificity in identifying preventable readmissions, which corresponded to very poor PPV and fair to very good NPV. In terms of identifying unplanned readmissions, the PACR had fair sensitivity but poor specificity, which corresponded to fair PPV and poor NPV (Table: Section C).

PPR

The PPR algorithm identified 53% (257/487) of admissions as potentially preventable. The PPR algorithm had poor sensitivity and specificity in identifying preventable readmissions, which corresponded to very poor PPV and fair to very good NPV. In terms of identifying unplanned readmissions, the PPR algorithm had poor sensitivity and fair specificity in identifying unplanned readmissions, which corresponded to fair to good PPV and very poor to poor NPV (Table: Section D).

Evaluation of Excluded Readmission Events

Because both the PACR and PPR had large numbers of algorithm exclusions, we describe the preventability and unplanned assessment of the excluded readmission events. Both algorithms excluded preventable events. Of the 321 readmissions excluded by the PACR algorithm, 13.4% were classified as preventable by chart review. Likewise, 14.9% of 638 readmissions excluded by PPR were classified as preventable by chart review.

The ability to accurately capture preventable pediatric readmission is a goal for hospital quality experts and health policymakers alike. Of the four commonly used readmission measures to assess readmission, only PPR is designed to focus on preventability. Unfortunately, none of these four measures is adequately sensitive or specific to identify preventable readmissions; all measures had very poor PPV for preventability. Of the four measures, the time flag measure had the best sensitivity, specificity, PPV, and NPV for identifying unplanned readmissions.

The overall percentage of unplanned readmissions identified by both the time flag and by PACR measures match the overall percentage of unplanned readmissions identified in chart review: The time flag measure identified 80% of admissions as unplanned versus 79% identified by chart review (Appendix Figure: Population A/B); PACR classified 75% as unplanned versus 81% identified by chart review for PACR-eligible readmissions (Appendix Figure: Population C). In contrast, the PPR algorithm classified many more readmissions as potentially preventable (53%) than were identified by chart review at only 16% (Appendix Figure: Population D). The PACR and PPR algorithms also exclude a significant number of readmissions that are unplanned and a smaller, but not trivial, number of readmissions that are preventable; these exclusions limit their accuracy.

The ability to apply these four measures in real time during a hospitalization varies by metric. Two of the measures, the all-cause and time flag, can be applied during a readmission event, which is appealing for quality improvement initiatives. These measures allow for notification of providers that a current hospitalization is a readmission event, which allows providers the opportunity to learn from these events as they occur (Appendix Table 1). While “unplanned” is not the same as “potentially preventable,” almost all potentially preventable readmissions are unplanned; therefore, accurately identifying unplanned readmissions is more beneficial than all-cause. Additionally, a low all-cause readmission rate can be indicative of poor access to scheduled procedures. Nevertheless, all-cause readmission is sometimes used to measure quality.^1,8 While the time flag measure may be more useful for quality improvement initiatives and hospital providers, it relies on hospital registration time, which is not widely available in administrative data sources and, therefore, has limited usefulness to policymakers.

Both PACR and PPR require administrative claims analysis, which is appealing from a policy standpoint. However, the reliance on claims data means the inclusion/exclusion of events can occur only retrospectively, which limits the usefulness of these measures in learning and intervening in real time. When the two measures are compared, PACR offers better sensitivity and PPR offers better specificity with regard to identifying unplanned readmission. The PPR software overcalls preventable readmissions, identifying more readmissions as preventable than there actually are. Nevertheless, Medicaid in several states uses PPR for payment incentive.^1,15-17 Given the poor performance of PPR in assessing both preventable and unplanned pediatric readmission, the use of this measure as a quality metric should be limited.

This study should be considered in the context of several limitations. Because the assessment of preventability was determined as part of a learning quality improvement collaborative and not as a planned research endeavor, not all readmission reviews were completed nor were other existent tools¹⁸ that allow for preventability assessment via more structured medical record review used. Second, we reviewed cases only from certain clinical services, which would limit generalizability of these findings to all pediatric admissions. However, given the low sensitivity and specificity of some of the metrics, we would not anticipate that the addition of other types of admissions would improve the sensitivity and specificity enough to ensure reliability. Third, while we relied on an established method to determine preventability, prior work has demonstrated that additional information gathered from families may change preventability.¹⁹ Finally, due to the exclusions required by the PPR and PACR algorithms, not all readmission events were reviewed. However, these exclusions reflect the actual specifications of use for both measures.

The PPR software has poor fidelity in identifying preventable and unplanned pediatric readmission; this finding has broad policy implications given how widely it is used by state Medicaid offices to assess financial penalties. Among the four pediatric readmission measures used, the time flag metric best identifies unplanned readmissions.

The authors have no conflicts of interest or financial relationships relevant to this article to disclose.

Dr Auger’s research is supported by a grant from the Agency for Healthcare Research and Quality (1K08HS204735-01A1). The project described was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health, under Award Number 5UL1TR001425-04. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Readmission rates are frequently used as a hospital quality metric, with use including payment incentive at the hospital level,¹ specific condition quality measurement,² balancing measures for quality improvement projects,^3-5 transition success,^6,7 and use in public hospital rankings.⁸ Currently, four methods are commonly used to evaluate pediatric readmissions, each with strengths and limitations, including the following (Appendix Table 1):

1. All-cause readmissions: A measure of any readmission within a given time period regardless of the reason for readmission.⁹

2. Unplanned readmission/time flag: A measure intended to identify unplanned readmissions. This measure relies on time designations within the electronic health record. The time between hospital registration and admission is calculated, and if the readmission is registered more than 24 hours prior to admission, the readmission is considered planned.¹⁰ Hereafter, this measure will be referred to as the time flag measure.

3. Pediatric all-condition readmission (PACR): A measure intended to identify unplanned readmission through the exclusion of certain procedures and diagnoses.¹¹

4. Potentially preventable readmission (PPR): A method to identify preventable readmissions based on a proprietary algorithm developed by 3M Health Information Systems.^12,13

While all four of these measures are used to assess quality, there is little known about these measures’ ability to exclude planned readmissions and identify only preventable pediatric readmission, which conceptually is most relevant to the quality of care. However, many of these measures were not intended to capture preventability, but instead capture the related issue of whether the readmission was planned. Therefore, we sought to evaluate the four readmission measures as they relate to both preventability and unplanned status as determined through medical record review with multidisciplinary care provider input.

As part of a hospital-wide readmission reduction quality improvement collaborative at a free-standing tertiary care children’s hospital, clinicians from hospital medicine, cardiology, neonatology, and neurology teams reviewed 30-day readmissions using a standardized abstraction tool. All readmission events (observation or inpatient encounter) after any discharge (observation or inpatient encounter) from eligible units were reviewed; therefore, each hospitalization was a potential index hospitalization. We classified the preventability of each readmission with use of a previously described Likert scale with high interrater reliability.¹⁴ For these analyses, readmissions were considered preventable if the reviewing team rated them as either “more likely preventable” or “preventable in most circumstances.” Each readmission was also evaluated as planned or unplanned. Methods for readmission review and classification are in the Appendix.

We included all readmissions between July 2014 and June 2016. We compared the medical record review classifications with the assessments from each of the four measures of pediatric readmission. We calculated sensitivity and specificity for both outcomes (planned/unplanned and preventable/not preventable) for all four measures. For standardization of discussion, we categorized description of measure performance as “very poor” as less than 50%, “poor” between 50%-75%, “fair” as 75%-85%, “good” as 85%-90%, “very good” as 90%-95% and excellent as greater than 95%. We also calculated positive and negative predictive value (PPV and NPV) over plausible ranges of prevalence using the sensitivity and specificity of each comparison (Appendix).

Of note, certain exclusions are outlined by the PACR and PPR algorithms. The PACR evaluates only readmission events that occur in children younger than 18 years. The PPR algorithm does not assign preventability if either the index or readmission event is classified as an observation stay or if it is part of a larger chain of readmissions.

Among 30-day readmissions considered, 1,643 were eligible for medical record review; 1,125 reviews were completed by the clinical teams (68.5%). The median time to readmission was 7 days (interquartile range [IQR], 4-18). Most children were non-Hispanic White (71%) or Black (20%). The median age at hospitalization was 2.3 years (IQR 0.4-12.1). Most children had Medicaid (56%) or private (41%) insurance. Most of the reviews were performed in cardiology (43%) and hospital medicine (37%) with patients in neurology (13%) and neonatology (7%) constituting the remaining reviews. Uncontrolled advancement of chronic disease was the most common readmission category on medical record review (25.1%), followed by unrelated readmission (20.7%), scheduled readmission (20.4%), and progression of acute disease (16.6%) (Appendix Table 2).

Assessment of Preventable and Unplanned Readmissions

On multidisciplinary medical record review, most readmissions were classified as not preventable (84.5%). Specifically, 64% were not preventable and unplanned; 20% were deemed not preventable and planned. Only 15% were classified as unplanned and preventable and 1% as planned and preventable (Appendix Figure: Population A/B).

Matching Chart Review to the Four Algorithms

All 1,125 readmissions were assessed by the all-cause and time flag readmission measures (Appendix Figure: Population A/B). After applying algorithm exclusions (details in Appendix), only 804 of the 1,125 (71.5%) reviewed readmissions matched for PACR readmission comparison (Appendix Figure: Population C); 487 of the 1,125 (43.3%) of the reviewed readmissions matched for PPR comparison (Appendix Figure: Population D).

All-Cause

Because all-cause determines only if a readmission occurs, the measure is by definition 100% sensitive and 0% specific in both assessment of preventability and unplanned readmission (Table: Section A).

Time Flag

The time flag measure identified 80% (866/1,112) of the readmissions as unplanned. This measure had very good sensitivity but very poor specificity in identifying preventable readmissions, which corresponded to very poor PPV and good to excellent NPV. In terms of identifying unplanned readmissions, the time flag measure had excellent sensitivity and very good specificity, which corresponded to very good to excellent PPV and good to very good NPV (Table: Section B).

PACR

The PACR algorithm identified 75% (599/796) of readmissions as unplanned. The PACR has good sensitivity but very poor specificity in identifying preventable readmissions, which corresponded to very poor PPV and fair to very good NPV. In terms of identifying unplanned readmissions, the PACR had fair sensitivity but poor specificity, which corresponded to fair PPV and poor NPV (Table: Section C).

PPR

The PPR algorithm identified 53% (257/487) of admissions as potentially preventable. The PPR algorithm had poor sensitivity and specificity in identifying preventable readmissions, which corresponded to very poor PPV and fair to very good NPV. In terms of identifying unplanned readmissions, the PPR algorithm had poor sensitivity and fair specificity in identifying unplanned readmissions, which corresponded to fair to good PPV and very poor to poor NPV (Table: Section D).

Evaluation of Excluded Readmission Events

Because both the PACR and PPR had large numbers of algorithm exclusions, we describe the preventability and unplanned assessment of the excluded readmission events. Both algorithms excluded preventable events. Of the 321 readmissions excluded by the PACR algorithm, 13.4% were classified as preventable by chart review. Likewise, 14.9% of 638 readmissions excluded by PPR were classified as preventable by chart review.

The ability to accurately capture preventable pediatric readmission is a goal for hospital quality experts and health policymakers alike. Of the four commonly used readmission measures to assess readmission, only PPR is designed to focus on preventability. Unfortunately, none of these four measures is adequately sensitive or specific to identify preventable readmissions; all measures had very poor PPV for preventability. Of the four measures, the time flag measure had the best sensitivity, specificity, PPV, and NPV for identifying unplanned readmissions.

The overall percentage of unplanned readmissions identified by both the time flag and by PACR measures match the overall percentage of unplanned readmissions identified in chart review: The time flag measure identified 80% of admissions as unplanned versus 79% identified by chart review (Appendix Figure: Population A/B); PACR classified 75% as unplanned versus 81% identified by chart review for PACR-eligible readmissions (Appendix Figure: Population C). In contrast, the PPR algorithm classified many more readmissions as potentially preventable (53%) than were identified by chart review at only 16% (Appendix Figure: Population D). The PACR and PPR algorithms also exclude a significant number of readmissions that are unplanned and a smaller, but not trivial, number of readmissions that are preventable; these exclusions limit their accuracy.

The ability to apply these four measures in real time during a hospitalization varies by metric. Two of the measures, the all-cause and time flag, can be applied during a readmission event, which is appealing for quality improvement initiatives. These measures allow for notification of providers that a current hospitalization is a readmission event, which allows providers the opportunity to learn from these events as they occur (Appendix Table 1). While “unplanned” is not the same as “potentially preventable,” almost all potentially preventable readmissions are unplanned; therefore, accurately identifying unplanned readmissions is more beneficial than all-cause. Additionally, a low all-cause readmission rate can be indicative of poor access to scheduled procedures. Nevertheless, all-cause readmission is sometimes used to measure quality.^1,8 While the time flag measure may be more useful for quality improvement initiatives and hospital providers, it relies on hospital registration time, which is not widely available in administrative data sources and, therefore, has limited usefulness to policymakers.

Both PACR and PPR require administrative claims analysis, which is appealing from a policy standpoint. However, the reliance on claims data means the inclusion/exclusion of events can occur only retrospectively, which limits the usefulness of these measures in learning and intervening in real time. When the two measures are compared, PACR offers better sensitivity and PPR offers better specificity with regard to identifying unplanned readmission. The PPR software overcalls preventable readmissions, identifying more readmissions as preventable than there actually are. Nevertheless, Medicaid in several states uses PPR for payment incentive.^1,15-17 Given the poor performance of PPR in assessing both preventable and unplanned pediatric readmission, the use of this measure as a quality metric should be limited.

This study should be considered in the context of several limitations. Because the assessment of preventability was determined as part of a learning quality improvement collaborative and not as a planned research endeavor, not all readmission reviews were completed nor were other existent tools¹⁸ that allow for preventability assessment via more structured medical record review used. Second, we reviewed cases only from certain clinical services, which would limit generalizability of these findings to all pediatric admissions. However, given the low sensitivity and specificity of some of the metrics, we would not anticipate that the addition of other types of admissions would improve the sensitivity and specificity enough to ensure reliability. Third, while we relied on an established method to determine preventability, prior work has demonstrated that additional information gathered from families may change preventability.¹⁹ Finally, due to the exclusions required by the PPR and PACR algorithms, not all readmission events were reviewed. However, these exclusions reflect the actual specifications of use for both measures.

The PPR software has poor fidelity in identifying preventable and unplanned pediatric readmission; this finding has broad policy implications given how widely it is used by state Medicaid offices to assess financial penalties. Among the four pediatric readmission measures used, the time flag metric best identifies unplanned readmissions.

The authors have no conflicts of interest or financial relationships relevant to this article to disclose.

Dr Auger’s research is supported by a grant from the Agency for Healthcare Research and Quality (1K08HS204735-01A1). The project described was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health, under Award Number 5UL1TR001425-04. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Establish a Command Team

We benefit from having an existing divisional leadership structure comprising the director, medical directors of our clinical service lines, directors of education and community integration, and associate directors of clinical operations, research, and quality. This established team provides us broad representation of team member expertise and ideas. We maintain our weekly leadership team meeting through video chat and have added daily 30-minute virtual huddles to provide updates from our respective areas and discuss logistical challenges and planning. We use ad hoc phone meetings with relevant team members to address issues of immediate concern.

In the absence of a formal leadership team structure, establish a command team comprising representative leaders of your varied groups (eg, clinical operations, quality improvement, education, research, and business).

Collaborate With Institutional Response

Align divisional command team actions with the institutional response. Our clinical operations leader serves as our primary representative on the institutional emergency preparedness team. This participation allows bidirectional communication, both for institutional updates to be shared with division members and division-specific initiatives to be shared with institutional leadership to facilitate learning across the system.

In conjunction with hospital leadership, our division created a special isolation unit (SIU) to isolate patients positive for COVID-19 and persons under investigation. The institutional emergency preparedness team highlighted the need for such a unit, and our divisional leadership team developed the physician staffing model and medical care delivery system. We collaborated with key stakeholders, including nurses, respiratory therapists, other patient care services members, and subspecialists. The SIU leadership, which includes representatives from hospital medicine, nursing, respiratory therapy, and hospital operations, holds regular phone huddles to provide support and enlist resources based on identified gaps, which allows the frontline SIU physicians to focus on patient care. The calls initially occurred twice daily, but we transitioned to a once-daily schedule after routines were established and resources were procured.

Communicate With Everyone

Frequent communication with the clinical staff is paramount given the rapidly evolving operational changes and medical management recommendations. The divisional leadership team provides frequent email updates to the attending physicians on clinical shifts to communicate clinical updates, send reminders to conserve personal protective equipment (PPE), and share links to COVID-19 resources.

We use our weekly divisional meetings, now held virtually, to provide updates and to allow staff to ask questions and provide input. These meetings routinely include our nonclinical staff, such as administrative assistants and research coordinators, to ensure all team members’ voices are heard and skill sets are utilized. Our divisional infrastructure promotes dialogue and transparency, which is key to our division’s culture. Applying a learning health network approach has allowed us to generate new ideas, accelerate improvement, and encourage everyone to be a part of our community focused on improving outcomes.⁶ We continue to leverage this approach in our pandemic response.

One idea generated from this approach prompted us to create a centralized communication forum, using Microsoft Teams, to serve as a repository for the most up-to-date information related to COVID-19, the SIU, and general information, including links to divisional and institutional resources.

Maintain Nonclinical Operations

Nonclinical staff are working remotely. The business director and research director hold daily calls with the administrative staff and research coordinators, respectively, to discuss workload and to reallocate responsibilities as needed. This approach allows the business, administrative, and research support teams to function efficiently and redistribute work as the nonclinical priorities shift to meet divisional needs.

Establish a Backup Pool

We anticipate needing a larger pool of backup providers in the event of ill or quarantined staff or in case of increased patient volumes. The latter may be less likely for pediatric patients based on early studies^3-5 but could occur if our free-standing children’s hospital expands to include the care of adult patients. We asked physicians to volunteer for backup shifts to augment our existing “jeopardy” backup system with a greater request to those with a lower clinical full-time equivalence. Each day, two backup shift positions are filled by volunteers, with additional positions added on days when medicine-­pediatrics providers are scheduled for shifts in case they are needed at the university (adult) hospital.

Minimize Staffing to Reserve Pool

We monitor census closely on all service lines, including our consult service lines and secondary inpatient site, with plans to dissolve unnecessary consult services and combine medical teams, when feasible, to reduce the risk of staff exposure and maintain reserves. For example, after elective procedures were canceled, we reduced physician staffing of our surgical comanagement service to the minimal necessary coverage. We assign nonpatient-facing clinical duties to physicians who are called off their shift, in quarantine, or mildly ill to help off-load the clinical burden. Such duties include accepting direct admission phone calls, triaging patient care calls, entering orders remotely, and assisting with care coordination needs.

Anticipate Adult Care Needs

Our pediatric institution admits select groups of adult patients with congenital or complex healthcare needs who require specialized care. Hospitalists board certified in both pediatrics and internal medicine provide consultative services to many of these patients. Anticipating that these physicians may be needed in adult facilities, we plan to dissolve this consult service and utilize our reserve pool of providers to cover their pediatric shifts if needed. Additionally, if our hospital expands coverage for adult patients, these medicine-pediatrics providers will be instrumental in coordinating that expanded effort and will serve as leaders for teams of physicians and advanced practice providers with limited or no adult medicine training.

Special Isolation Unit

Logistic planning for our SIU evolved over the first few patients with rapid-cycle feedback and learning with each admission. This feedback was facilitated with our twice-daily huddle calls, which involved all key stakeholders, including nursing and respiratory therapy representatives. For division physician staffing, higher-risk team members are excluded from working on this unit. Because the SIU was developed to care for all patients positive for COVID-19 and persons under investigation, subspecialty patients not typically cared for by Hospital Medicine at our institution are being admitted to this unit. Therefore, subspecialty divisions assign attending physicians to provide consultative services to the SIU. These consultants use the unit’s telemedicine capabilities, when feasible, to limit staff exposure and conserve PPE. Our hospital medicine leaders in the SIU proactively worked with subspecialty divisions that are anticipated to have more admissions given their at-risk patient populations, such as pulmonary medicine, cardiology, and oncology. They specifically developed staffing plans for these patients if the SIU census becomes unsustainable under Hospital Medicine alone.

Healthcare workers are experiencing numerous stressors at work and home during this tumultuous time. Our workforce is at risk of developing emotional distress and mental health concerns. A cross-sectional survey of more than 1,200 healthcare workers in China who cared for COVID-19 patients found that many experienced symptoms of psychological distress (71%), as well as depression (51%), anxiety (44%), and insomnia (34%).⁷ Hospital medicine groups should consider methods to support their staff to mitigate stressors and promote self-care.

Anticipate Childcare Issues

When we were faced with impending school and daycare closures, we surveyed our division to assess childcare needs (Table) and share resources. We created a system of emergency childcare coverage options by connecting parents with similarly aged children and who lived in geographic proximity. This approach to childcare contingency planning was shared with and adopted by other divisions within the institution.

Build Support Measures

To support each other during this particularly stressful time, we divided division members into groups or “support pods,” each facilitated by a leadership team member. Group text messages and weekly phone or video chats have promoted connectivity and peer support.

Promote Self-care

The divisional leadership team provides food and drink for staff on clinical shifts. We also collated self-care resources to share via a central repository. These resources include ideas for meditation, home education for children, parenting, exercise, faith communities, entertainment, methods to support our local community through volunteerism and donations, and mental health resources, as well as online links to these resources.

Adult health systems will be disproportionately affected as this pandemic evolves. Pediatric hospitalists have the unique opportunity to support the response efforts by maintaining teams that are flexible and adaptable to evolving community needs. To do this, team leaders need to promote transparency, share learnings, and leverage the diverse skills of team members to ensure we are ready to meet the challenges of the moment.

Establish a Command Team

We benefit from having an existing divisional leadership structure comprising the director, medical directors of our clinical service lines, directors of education and community integration, and associate directors of clinical operations, research, and quality. This established team provides us broad representation of team member expertise and ideas. We maintain our weekly leadership team meeting through video chat and have added daily 30-minute virtual huddles to provide updates from our respective areas and discuss logistical challenges and planning. We use ad hoc phone meetings with relevant team members to address issues of immediate concern.

In the absence of a formal leadership team structure, establish a command team comprising representative leaders of your varied groups (eg, clinical operations, quality improvement, education, research, and business).

Collaborate With Institutional Response

Align divisional command team actions with the institutional response. Our clinical operations leader serves as our primary representative on the institutional emergency preparedness team. This participation allows bidirectional communication, both for institutional updates to be shared with division members and division-specific initiatives to be shared with institutional leadership to facilitate learning across the system.

In conjunction with hospital leadership, our division created a special isolation unit (SIU) to isolate patients positive for COVID-19 and persons under investigation. The institutional emergency preparedness team highlighted the need for such a unit, and our divisional leadership team developed the physician staffing model and medical care delivery system. We collaborated with key stakeholders, including nurses, respiratory therapists, other patient care services members, and subspecialists. The SIU leadership, which includes representatives from hospital medicine, nursing, respiratory therapy, and hospital operations, holds regular phone huddles to provide support and enlist resources based on identified gaps, which allows the frontline SIU physicians to focus on patient care. The calls initially occurred twice daily, but we transitioned to a once-daily schedule after routines were established and resources were procured.

Communicate With Everyone

Frequent communication with the clinical staff is paramount given the rapidly evolving operational changes and medical management recommendations. The divisional leadership team provides frequent email updates to the attending physicians on clinical shifts to communicate clinical updates, send reminders to conserve personal protective equipment (PPE), and share links to COVID-19 resources.

We use our weekly divisional meetings, now held virtually, to provide updates and to allow staff to ask questions and provide input. These meetings routinely include our nonclinical staff, such as administrative assistants and research coordinators, to ensure all team members’ voices are heard and skill sets are utilized. Our divisional infrastructure promotes dialogue and transparency, which is key to our division’s culture. Applying a learning health network approach has allowed us to generate new ideas, accelerate improvement, and encourage everyone to be a part of our community focused on improving outcomes.⁶ We continue to leverage this approach in our pandemic response.

One idea generated from this approach prompted us to create a centralized communication forum, using Microsoft Teams, to serve as a repository for the most up-to-date information related to COVID-19, the SIU, and general information, including links to divisional and institutional resources.

Maintain Nonclinical Operations

Nonclinical staff are working remotely. The business director and research director hold daily calls with the administrative staff and research coordinators, respectively, to discuss workload and to reallocate responsibilities as needed. This approach allows the business, administrative, and research support teams to function efficiently and redistribute work as the nonclinical priorities shift to meet divisional needs.

Establish a Backup Pool

We anticipate needing a larger pool of backup providers in the event of ill or quarantined staff or in case of increased patient volumes. The latter may be less likely for pediatric patients based on early studies^3-5 but could occur if our free-standing children’s hospital expands to include the care of adult patients. We asked physicians to volunteer for backup shifts to augment our existing “jeopardy” backup system with a greater request to those with a lower clinical full-time equivalence. Each day, two backup shift positions are filled by volunteers, with additional positions added on days when medicine-­pediatrics providers are scheduled for shifts in case they are needed at the university (adult) hospital.

Minimize Staffing to Reserve Pool

We monitor census closely on all service lines, including our consult service lines and secondary inpatient site, with plans to dissolve unnecessary consult services and combine medical teams, when feasible, to reduce the risk of staff exposure and maintain reserves. For example, after elective procedures were canceled, we reduced physician staffing of our surgical comanagement service to the minimal necessary coverage. We assign nonpatient-facing clinical duties to physicians who are called off their shift, in quarantine, or mildly ill to help off-load the clinical burden. Such duties include accepting direct admission phone calls, triaging patient care calls, entering orders remotely, and assisting with care coordination needs.

Anticipate Adult Care Needs

Our pediatric institution admits select groups of adult patients with congenital or complex healthcare needs who require specialized care. Hospitalists board certified in both pediatrics and internal medicine provide consultative services to many of these patients. Anticipating that these physicians may be needed in adult facilities, we plan to dissolve this consult service and utilize our reserve pool of providers to cover their pediatric shifts if needed. Additionally, if our hospital expands coverage for adult patients, these medicine-pediatrics providers will be instrumental in coordinating that expanded effort and will serve as leaders for teams of physicians and advanced practice providers with limited or no adult medicine training.

Special Isolation Unit

Logistic planning for our SIU evolved over the first few patients with rapid-cycle feedback and learning with each admission. This feedback was facilitated with our twice-daily huddle calls, which involved all key stakeholders, including nursing and respiratory therapy representatives. For division physician staffing, higher-risk team members are excluded from working on this unit. Because the SIU was developed to care for all patients positive for COVID-19 and persons under investigation, subspecialty patients not typically cared for by Hospital Medicine at our institution are being admitted to this unit. Therefore, subspecialty divisions assign attending physicians to provide consultative services to the SIU. These consultants use the unit’s telemedicine capabilities, when feasible, to limit staff exposure and conserve PPE. Our hospital medicine leaders in the SIU proactively worked with subspecialty divisions that are anticipated to have more admissions given their at-risk patient populations, such as pulmonary medicine, cardiology, and oncology. They specifically developed staffing plans for these patients if the SIU census becomes unsustainable under Hospital Medicine alone.

Healthcare workers are experiencing numerous stressors at work and home during this tumultuous time. Our workforce is at risk of developing emotional distress and mental health concerns. A cross-sectional survey of more than 1,200 healthcare workers in China who cared for COVID-19 patients found that many experienced symptoms of psychological distress (71%), as well as depression (51%), anxiety (44%), and insomnia (34%).⁷ Hospital medicine groups should consider methods to support their staff to mitigate stressors and promote self-care.

Anticipate Childcare Issues

When we were faced with impending school and daycare closures, we surveyed our division to assess childcare needs (Table) and share resources. We created a system of emergency childcare coverage options by connecting parents with similarly aged children and who lived in geographic proximity. This approach to childcare contingency planning was shared with and adopted by other divisions within the institution.

Build Support Measures

To support each other during this particularly stressful time, we divided division members into groups or “support pods,” each facilitated by a leadership team member. Group text messages and weekly phone or video chats have promoted connectivity and peer support.

Promote Self-care

The divisional leadership team provides food and drink for staff on clinical shifts. We also collated self-care resources to share via a central repository. These resources include ideas for meditation, home education for children, parenting, exercise, faith communities, entertainment, methods to support our local community through volunteerism and donations, and mental health resources, as well as online links to these resources.

Adult health systems will be disproportionately affected as this pandemic evolves. Pediatric hospitalists have the unique opportunity to support the response efforts by maintaining teams that are flexible and adaptable to evolving community needs. To do this, team leaders need to promote transparency, share learnings, and leverage the diverse skills of team members to ensure we are ready to meet the challenges of the moment.

Establish a Command Team

We benefit from having an existing divisional leadership structure comprising the director, medical directors of our clinical service lines, directors of education and community integration, and associate directors of clinical operations, research, and quality. This established team provides us broad representation of team member expertise and ideas. We maintain our weekly leadership team meeting through video chat and have added daily 30-minute virtual huddles to provide updates from our respective areas and discuss logistical challenges and planning. We use ad hoc phone meetings with relevant team members to address issues of immediate concern.

In the absence of a formal leadership team structure, establish a command team comprising representative leaders of your varied groups (eg, clinical operations, quality improvement, education, research, and business).

Collaborate With Institutional Response

Align divisional command team actions with the institutional response. Our clinical operations leader serves as our primary representative on the institutional emergency preparedness team. This participation allows bidirectional communication, both for institutional updates to be shared with division members and division-specific initiatives to be shared with institutional leadership to facilitate learning across the system.

In conjunction with hospital leadership, our division created a special isolation unit (SIU) to isolate patients positive for COVID-19 and persons under investigation. The institutional emergency preparedness team highlighted the need for such a unit, and our divisional leadership team developed the physician staffing model and medical care delivery system. We collaborated with key stakeholders, including nurses, respiratory therapists, other patient care services members, and subspecialists. The SIU leadership, which includes representatives from hospital medicine, nursing, respiratory therapy, and hospital operations, holds regular phone huddles to provide support and enlist resources based on identified gaps, which allows the frontline SIU physicians to focus on patient care. The calls initially occurred twice daily, but we transitioned to a once-daily schedule after routines were established and resources were procured.

Communicate With Everyone

Frequent communication with the clinical staff is paramount given the rapidly evolving operational changes and medical management recommendations. The divisional leadership team provides frequent email updates to the attending physicians on clinical shifts to communicate clinical updates, send reminders to conserve personal protective equipment (PPE), and share links to COVID-19 resources.

We use our weekly divisional meetings, now held virtually, to provide updates and to allow staff to ask questions and provide input. These meetings routinely include our nonclinical staff, such as administrative assistants and research coordinators, to ensure all team members’ voices are heard and skill sets are utilized. Our divisional infrastructure promotes dialogue and transparency, which is key to our division’s culture. Applying a learning health network approach has allowed us to generate new ideas, accelerate improvement, and encourage everyone to be a part of our community focused on improving outcomes.⁶ We continue to leverage this approach in our pandemic response.

One idea generated from this approach prompted us to create a centralized communication forum, using Microsoft Teams, to serve as a repository for the most up-to-date information related to COVID-19, the SIU, and general information, including links to divisional and institutional resources.

Maintain Nonclinical Operations

Nonclinical staff are working remotely. The business director and research director hold daily calls with the administrative staff and research coordinators, respectively, to discuss workload and to reallocate responsibilities as needed. This approach allows the business, administrative, and research support teams to function efficiently and redistribute work as the nonclinical priorities shift to meet divisional needs.

Establish a Backup Pool

We anticipate needing a larger pool of backup providers in the event of ill or quarantined staff or in case of increased patient volumes. The latter may be less likely for pediatric patients based on early studies^3-5 but could occur if our free-standing children’s hospital expands to include the care of adult patients. We asked physicians to volunteer for backup shifts to augment our existing “jeopardy” backup system with a greater request to those with a lower clinical full-time equivalence. Each day, two backup shift positions are filled by volunteers, with additional positions added on days when medicine-­pediatrics providers are scheduled for shifts in case they are needed at the university (adult) hospital.

Minimize Staffing to Reserve Pool

We monitor census closely on all service lines, including our consult service lines and secondary inpatient site, with plans to dissolve unnecessary consult services and combine medical teams, when feasible, to reduce the risk of staff exposure and maintain reserves. For example, after elective procedures were canceled, we reduced physician staffing of our surgical comanagement service to the minimal necessary coverage. We assign nonpatient-facing clinical duties to physicians who are called off their shift, in quarantine, or mildly ill to help off-load the clinical burden. Such duties include accepting direct admission phone calls, triaging patient care calls, entering orders remotely, and assisting with care coordination needs.

Anticipate Adult Care Needs

Our pediatric institution admits select groups of adult patients with congenital or complex healthcare needs who require specialized care. Hospitalists board certified in both pediatrics and internal medicine provide consultative services to many of these patients. Anticipating that these physicians may be needed in adult facilities, we plan to dissolve this consult service and utilize our reserve pool of providers to cover their pediatric shifts if needed. Additionally, if our hospital expands coverage for adult patients, these medicine-pediatrics providers will be instrumental in coordinating that expanded effort and will serve as leaders for teams of physicians and advanced practice providers with limited or no adult medicine training.

Special Isolation Unit

Logistic planning for our SIU evolved over the first few patients with rapid-cycle feedback and learning with each admission. This feedback was facilitated with our twice-daily huddle calls, which involved all key stakeholders, including nursing and respiratory therapy representatives. For division physician staffing, higher-risk team members are excluded from working on this unit. Because the SIU was developed to care for all patients positive for COVID-19 and persons under investigation, subspecialty patients not typically cared for by Hospital Medicine at our institution are being admitted to this unit. Therefore, subspecialty divisions assign attending physicians to provide consultative services to the SIU. These consultants use the unit’s telemedicine capabilities, when feasible, to limit staff exposure and conserve PPE. Our hospital medicine leaders in the SIU proactively worked with subspecialty divisions that are anticipated to have more admissions given their at-risk patient populations, such as pulmonary medicine, cardiology, and oncology. They specifically developed staffing plans for these patients if the SIU census becomes unsustainable under Hospital Medicine alone.

Healthcare workers are experiencing numerous stressors at work and home during this tumultuous time. Our workforce is at risk of developing emotional distress and mental health concerns. A cross-sectional survey of more than 1,200 healthcare workers in China who cared for COVID-19 patients found that many experienced symptoms of psychological distress (71%), as well as depression (51%), anxiety (44%), and insomnia (34%).⁷ Hospital medicine groups should consider methods to support their staff to mitigate stressors and promote self-care.

Anticipate Childcare Issues

When we were faced with impending school and daycare closures, we surveyed our division to assess childcare needs (Table) and share resources. We created a system of emergency childcare coverage options by connecting parents with similarly aged children and who lived in geographic proximity. This approach to childcare contingency planning was shared with and adopted by other divisions within the institution.

Build Support Measures

To support each other during this particularly stressful time, we divided division members into groups or “support pods,” each facilitated by a leadership team member. Group text messages and weekly phone or video chats have promoted connectivity and peer support.

Promote Self-care

The divisional leadership team provides food and drink for staff on clinical shifts. We also collated self-care resources to share via a central repository. These resources include ideas for meditation, home education for children, parenting, exercise, faith communities, entertainment, methods to support our local community through volunteerism and donations, and mental health resources, as well as online links to these resources.

Adult health systems will be disproportionately affected as this pandemic evolves. Pediatric hospitalists have the unique opportunity to support the response efforts by maintaining teams that are flexible and adaptable to evolving community needs. To do this, team leaders need to promote transparency, share learnings, and leverage the diverse skills of team members to ensure we are ready to meet the challenges of the moment.

Readmission rates are used as metrics for care quality and reimbursement, with penalties applied to hospitals with higher than expected rates¹ and up to 30% of pediatric readmissions deemed potentially preventable.² There is a paucity of information on how to prevent pediatric readmissions,³ yet pediatric hospitals are tasked with implementing interventions for readmission reduction.

The Hospital to Home Outcomes (H2O) trial was a 2-arm, randomized controlled trial in which patients discharged from hospital medicine and neuroscience services at a single institution were randomized to receive a single home visit from a registered nurse (RN) within 96 hours of discharge.⁴ RNs completed a structured nurse visit designed specifically for the trial. Lists of “red flags” or warning signs associated with common diagnoses were provided to assist RNs in standardizing education about when to seek additional care. The hypothesis was that the postdischarge visits would result in lower reutilization rates (unplanned readmissions, emergency department [ED] visits, and urgent care visits).⁵

Unexpectedly, children randomized to receive the postdischarge nurse visit had higher rates of 30-day unplanned healthcare reutilization, with children randomly assigned to the intervention demonstrating higher odds of 30-day healthcare use (OR 1.33; 95% CI 1.003-1.76).⁴ We sought to understand perspectives on these unanticipated findings by obtaining input from relevant stakeholders. There were 2 goals for the qualitative analysis: first, to understand possible explanations of the increased reutilization finding; second, to elicit suggestions for improving the nurse visit intervention.

We selected an in-depth qualitative approach, using interviews and focus groups to explore underlying explanations for the increase in 30-day unplanned healthcare reutilization among those randomized to receive the postdischarge nurse visit during the H2O trial.⁴ Input was sought from 4 stakeholder groups—parents, primary care physicians (PCPs), hospital medicine physicians, and home care RNs—in an effort to triangulate data sources and elicit rich and diverse opinions. Approval was obtained from the Institutional Review Board prior to conducting the study.

Recruitment
Parents

Because we conducted interviews approximately 1 year after the trial’s conclusion, we purposefully selected families who were enrolled in the latter portion of the H2O trial in order to enhance recall. Beginning with the last families in the study, we sequentially contacted families in reverse order. We contacted 10 families in each of 4 categories (intervention/reutilization, intervention/no reutilization, control/reutilization, control/no reutilization). A total of 3 attempts were made by telephone to contact each family. Participants received a grocery store gift card for participating in the study.

Primary Care Physicians

We conducted focus groups with a purposive sample of physicians recruited from 2 community practices and 1 hospital-­owned practice.

Hospital Medicine Physicians

We conducted focus groups with a purposive sample of physicians from our Division of Hospital Medicine. There was a varying level of knowledge of the original trial; however, none of the participants were collaborators in the trial.

Home Care RNs

We conducted focus groups with a subset of RNs who were involved with trial visits. All RNs were members of the pediatric home care division associated with the hospital with specific training in caring for patients at home.

Data Collection

The study team designed question guides for each stakeholder group (Appendix 1). While questions were tailored for specific stakeholders, all guides included the following topics: benefits and challenges of nurse visits, suggestions for improving the intervention in future trials, and reactions to the trial results (once presented to participants). Only the results of the intention-to-treat (ITT) analysis were shared with stakeholders because ITT is considered the gold standard for trial analysis and allows easy understanding of the results.

A single investigator (A.L.) conducted parental interviews by telephone. Focus groups for PCPs, hospital medicine physicians, and RN groups were held at practice locations in private conference rooms and were conducted by trained moderators (S.N.S., A.L., and H.T.C.). Moderators probed responses to the open-ended questions to delve deeply into issues. The question guides were modified in an iterative fashion to include new concepts raised during interviews or focus groups. All interviews and focus groups were recorded and transcribed verbatim with all identifiable information redacted.

Data Analysis

During multiple cycles of inductive thematic analysis,⁶ we examined, discussed, interpreted, and organized responses to the open-ended questions,^6,7 analyzing each stakeholder group separately. First, transcripts were shared with and reviewed by the entire multidisciplinary team (12 members) which included hospital medicine physicians, PCPs, home care nursing leaders, a nurse scientist, a parent representative, research coordinators, and a qualitative research methodologist. Second, team members convened to discuss overall concepts and ideas and created the preliminary coding frameworks. Third, a smaller subgroup (research coordinator [A.L]., hospital medicine physician [S.R.], parent representative [M.M.], and qualitative research methodologist [S.N.S.]), refined the unique coding framework for each stakeholder group and then independently applied codes to participant comments. This subgroup met regularly to reach consensus about the assigned codes and to further refine the codebooks. The codes were organized into major and minor themes based on recurring patterns in the data and the salience or emphasis given by participants. The subgroup’s work was reviewed and discussed on an ongoing basis by the entire multidisciplinary team. Triangulation of the data was achieved in multiple ways. The preliminary results were shared in several forums, and feedback was solicited and incorporated. Two of 4 members of the subgroup analytic team were not part of the trial planning or data collection, providing a potentially broader perspective. All coding decisions were maintained in an electronic database, and an audit trail was created to document codebook revisions.

A total of 33 parents participated in the interviews (intervention/readmit [8], intervention/no readmit [8], control/readmit [8], and control/no readmit [9]). Although we selected families from all 4 categories, we were not able to explore qualitative differences between these groups because of the relatively low numbers of participants. Parent data was very limited as interviews were brief and “control” parents had not received the intervention. Three focus groups were held with PCPs (7 participants in total), 2 focus groups were held with hospital medicine physicians (12 participants), and 2 focus groups were held with RNs (10 participants).

Goal 1: Explanation of Reutilization Rates

During interviews and focus groups, the results of the H2O trial were discussed, and stakeholders were asked to comment on potential explanations of the findings. 4 major themes and 5 minor themes emerged from analysis of the transcripts (summarized in Table 1).

Theme 1: Appropriateness of Patient Reutilization

Hospital medicine physicians and home care RNs questioned whether the reutilization events were clinically indicated. RNs wondered whether children who reutilized the ED were also readmitted to the hospital; many perceived that if the child was ill enough to be readmitted, then the ED revisit was warranted (Table 2). Parents commented on parental decision-making and changes in clinical status of the child leading to reutilization (Table 2).

Theme 2: Impact of Red Flags/Warning Sign Instructions on Family’s Reutilization Decisions

Theme 3: Hospital-Affiliated RNs “Directing Traffic” Back to Hospital

Both physician groups were concerned that, because the study was conducted by hospital-employed nurses, families might have been more likely to reaccess care at the hospital. Thus, the connection with the hospital was strengthened in the H2O model, potentially at the expense of the connection with PCPs. Physicians hypothesized that families might “still feel part of the medical system,” so families would return to the hospital if there was a problem. PCPs emphasized that there may have been straightforward situations that could have been handled appropriately in the outpatient office (Table 2).

Theme 4: Home Visit RNs Had a Low Threshold for Escalating Care

Parents and PCPs hypothesized that RNs are more conservative and, therefore, would have had a low threshold to refer back to the hospital if there were concerns in the home. One parent commented: “I guess, nurses are just by trade accustomed to erring on the side of caution and medical intervention instead of letting time take its course. … They’re more apt to say it’s better off to go to the hospital and have everything be fine” (Table 2).

Minor Themes

Participants also explained reutilization in ways that coalesced into 5 minor themes: (1) families receiving a visit might perceive that their child was sicker; (2) patients in the control group did not reutilize enough; (3) receiving more education on a child’s illness drives reutilization; (4) provider access issues; and (5) variability of RN experience may determine whether escalated care. Supportive quotations found in Appendix 2.

We directly asked parents if they would want a nurse home visit in the future after discussing the results of the study. Almost all of the parents in the intervention group and most of the parents in the control group were in favor of receiving a visit, even knowing that patients who had received a visit were more likely to reutilize care.

Goal 2: Suggestions for Improving Intervention Design

Three major themes and 3 minor themes were related to improving the design of the intervention (Table 1).

Theme 1: Need for Improved Postdischarge Communication

All stakeholder groups highlighted postdischarge communication as an area that could be improved. Parents were frustrated with regard to attempts to connect with inpatient physicians after discharge. PCPs suggested developing pathways for the RN to connect with the primary care office as opposed to the hospital. Hospital medicine physicians discussed a lack of consensus regarding patient ownership following discharge and were uncertain about what types of postdischarge symptoms PCPs would be comfortable managing. RNs described specific situations when they had difficulty contacting a physician to escalate care (Table 3).

Theme 2: Individualizing Home Visits—One Size Does Not Fit All

All stakeholder groups also encouraged “individualization” of home visits according to patient and family characteristics, diagnosis, and both timing and severity of illness. PCPs recommended visits only for certain diagnoses. Hospital medicine physicians voiced similar sentiments as the PCPs and added that worrisome family dynamics during a hospitalization, such as a lack of engagement with the medical team, might also warrant a visit. RNs suggested visits for those families with more concerns, for example, those with young children or children recovering from an acute respiratory illness (Table 3).

Theme 3: Providing Context for and Framing of Red Flags

Physicians and nurses suggested providing more context to “red flag” instructions and education. RNs emphasized that some families seemed to benefit from the opportunity to discuss their postdischarge concerns with a medical professional. Others appreciated concrete written instructions that spelled out how to respond in certain situations (Table 3).

Minor Themes

Three minor themes were revealed regarding intervention design improvement (Table 1): (1) streamlining the discharge process; (2) improving the definition of the scope and goal of intervention; and (3) extending inpatient team expertise post discharge. Supportive quotations can be found in Appendix 3.

When stakeholders were asked about why postdischarge RN visits led to increased postdischarge urgent healthcare visits, they questioned the appropriateness of the reutilization events, wondered about the lack of context for the warning signs that nurses provided families as part of the intervention, worried that families were encouraged to return to the hospital because of the ties of the trial to the hospital, and suggested that RNs had a low threshold to refer patients back to the hospital. When asked about how to design an improved nurse visit to better support families, stakeholders emphasized improving communication, individualizing the visit, and providing context around the red-flag discussion, enabling more nuanced instructions about how to respond to specific events.

A synthesis of themes suggests that potential drivers for increased utilization rates may lie in the design and goals of the initial project. The intervention was designed to support families and enhance education after discharge, with components derived from pretrial focus groups with families after a hospital discharge.⁸ The intervention was not designed to divert patients from the ED nor did it enhance access to the PCP. A second trial of the intervention adapted to a phone call also failed to decrease reutilization rates.⁹ Both physician stakeholder groups perceived that the intervention directed traffic back to the hospital because of the intervention design. Coupled with the perception that the red flags may have changed a family’s threshold for seeking care and/or that an RN may be more apt to refer back to care, this failure to push utilization to the primary care office may explain the unexpected trial results. Despite the stakeholders’ perception of enhanced connection back to the hospital as a result of the nurse visit, in analysis of visit referral patterns, a referral was made directly back to the ED in only 4 of the 651 trial visits (Tubbs-Cooley H, Riddle SR, Gold JM, et al.; under review. Pediatric clinical and social concerns identified by home visit nurses in the immediate postdischarge period 2020).

Both H2O trials demonstrated improved recall of red flags by parents who received the intervention, which may be important given the stakeholders’ perspectives that the red flags may not have been contextualized well enough. Yet neither trial demonstrated any differences in postdischarge coping or time to return to normal routine. In interviews with parents, despite the clearly stated results of increased reutilization, intervention parents endorsed a desire for a home visit in the future, raising the possibility that our outcome measures did not capture parents’ priorities adequately.

When asked to recommend design improvements of the intervention, 2 major themes (improvement in communication and individualization of visits) were discussed by all stakeholder groups, providing actionable information to modify or create new interventions. Focus groups with clinicians suggested that communication challenges may have influenced reutilization likelihood during the postdischarge period. RNs expressed uncertainty about who to call with problems or questions at the time of a home visit. This was compounded by difficulty reaching physicians. Both hospital medicine physicians and PCPs identified system challenges including questions of patient ownership, variable PCP practice communication preferences, and difficulty in identifying a partnered staff member (on either end of the inpatient-outpatient continuum) who was familiar with a specific patient. While the communication issues raised may reflect difficulties in our local healthcare system, there is broad evidence of postdischarge communication challenges. In adults, postdischarge communication failures between home health staff and physicians are associated with an increased risk of readmission.¹⁰ The real or perceived lack of communication between inpatient and outpatient providers can add to parental confusion post discharge.¹¹ Although there have been efforts to improve the reliability of communication across this gulf,^12,13 it is not clear whether changes to discharge communication could help to avoid pediatric reutilization events.¹⁴

The theme of individualization of the home nurse visit is consistent with evidence regarding the impact of focusing the intervention on patients with specific diagnoses or demographics. In adults, reduced reutilization associated with postdischarge home nurse visits has been described in specific populations such as patients with heart failure and chronic obstructive pulmonary disease.¹⁵ Impact of home nurse visits on patients within diagnosis-specific populations with certain demographics (such as advanced age) has also been described.¹⁶ In the pediatric population, readmission rates vary widely by diagnosis.¹⁷ A systematic review of interventions to reduce pediatric readmissions found increased impact of discharge interventions in specific populations (asthma, oncology, and NICU).³

Next steps may lie in interventions in targeted populations that function as part of a care continuum bridging the patient from the inpatient to the outpatient setting. A home nurse visit as part of this discharge structure may prove to have more impact on reducing reutilization. One population which accounts for a large proportion of readmissions and where there has been recent focus on discharge transition of care has been children with medical complexity.¹⁸ This group was largely excluded from the H2O trial. Postdischarge home nurse visits in this population have been found to be feasible and address many questions and problems, but the effect on readmission is less clear.¹⁹ Family priorities and preferences related to preparation for discharge, including family engagement, respect for discharge readiness, and goal of returning to normal routines, may be areas on which to focus with future interventions in this population.²⁰ In summary, although widespread postdischarge interventions (home nurse visit⁴ and nurse telephone call⁹) have not been found to be effective, targeting interventions to specific populations by diagnosis or demographic factors may prove to be more effective in reducing pediatric reutilization.

There were several strengths to this study. This qualitative approach allowed us to elucidate potential explanations for the H2O trial results from multiple perspectives. The multidisciplinary composition of our analytic team and the use of an iterative process sparked diverse contributions in a dynamic, ongoing discussion and interpretation of our data.

This study should be considered in the context of several limitations. For families and RNs, there was a time lag between participation in the trial and participation in the qualitative study call or focus group which could lead to difficulty recalling details. Only families who received the intervention could give opinions on their experience of the nurse visit, while families in the control group were asked to hypothesize. Focus groups with hospital medicine physicians and PCPs were purposive samples, and complete demographic information of participants was not collected.

Key stakeholders reflecting on a postdischarge RN visit trial suggested multiple potential explanations for the unexpected increase in reutilization in children randomized to the intervention. Certain participants questioned whether all reutilization events were appropriate or necessary. Others expressed concerns that the H2O intervention lacked context and directed children back to the hospital instead of the PCP. Parents, PCPs, hospital medicine physicians, and RNs all suggested that future transition-focused interventions should enhance postdischarge communication, strengthen connection to the PCP, and be more effectively tailored to the needs of the individual patient and family.

Acknowledgments

Collaborators: H2O Trial Study Group: Joanne Bachus, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Monica L Borell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lenisa V Chang, MA, PhD; Patricia Crawford, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sarah A Ferris, BA, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Judy A Heilman BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Jane C Khoury, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen Lawley, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lynne O’Donnell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Hadley S Sauers-Ford, MPH, Department of Pediatrics, UC Davis Health, Sacramento, California; Anita N Shah, DO, MPH, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lauren G Solan, MD, Med, University of Rochester, Rochester, New York; Heidi J Sucharew, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen P Sullivan, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Christine M White, MD, MAT, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio.

Readmission rates are used as metrics for care quality and reimbursement, with penalties applied to hospitals with higher than expected rates¹ and up to 30% of pediatric readmissions deemed potentially preventable.² There is a paucity of information on how to prevent pediatric readmissions,³ yet pediatric hospitals are tasked with implementing interventions for readmission reduction.

The Hospital to Home Outcomes (H2O) trial was a 2-arm, randomized controlled trial in which patients discharged from hospital medicine and neuroscience services at a single institution were randomized to receive a single home visit from a registered nurse (RN) within 96 hours of discharge.⁴ RNs completed a structured nurse visit designed specifically for the trial. Lists of “red flags” or warning signs associated with common diagnoses were provided to assist RNs in standardizing education about when to seek additional care. The hypothesis was that the postdischarge visits would result in lower reutilization rates (unplanned readmissions, emergency department [ED] visits, and urgent care visits).⁵

Unexpectedly, children randomized to receive the postdischarge nurse visit had higher rates of 30-day unplanned healthcare reutilization, with children randomly assigned to the intervention demonstrating higher odds of 30-day healthcare use (OR 1.33; 95% CI 1.003-1.76).⁴ We sought to understand perspectives on these unanticipated findings by obtaining input from relevant stakeholders. There were 2 goals for the qualitative analysis: first, to understand possible explanations of the increased reutilization finding; second, to elicit suggestions for improving the nurse visit intervention.

We selected an in-depth qualitative approach, using interviews and focus groups to explore underlying explanations for the increase in 30-day unplanned healthcare reutilization among those randomized to receive the postdischarge nurse visit during the H2O trial.⁴ Input was sought from 4 stakeholder groups—parents, primary care physicians (PCPs), hospital medicine physicians, and home care RNs—in an effort to triangulate data sources and elicit rich and diverse opinions. Approval was obtained from the Institutional Review Board prior to conducting the study.

Recruitment
Parents

Because we conducted interviews approximately 1 year after the trial’s conclusion, we purposefully selected families who were enrolled in the latter portion of the H2O trial in order to enhance recall. Beginning with the last families in the study, we sequentially contacted families in reverse order. We contacted 10 families in each of 4 categories (intervention/reutilization, intervention/no reutilization, control/reutilization, control/no reutilization). A total of 3 attempts were made by telephone to contact each family. Participants received a grocery store gift card for participating in the study.

Primary Care Physicians

We conducted focus groups with a purposive sample of physicians recruited from 2 community practices and 1 hospital-­owned practice.

Hospital Medicine Physicians

We conducted focus groups with a purposive sample of physicians from our Division of Hospital Medicine. There was a varying level of knowledge of the original trial; however, none of the participants were collaborators in the trial.

Home Care RNs

We conducted focus groups with a subset of RNs who were involved with trial visits. All RNs were members of the pediatric home care division associated with the hospital with specific training in caring for patients at home.

Data Collection

The study team designed question guides for each stakeholder group (Appendix 1). While questions were tailored for specific stakeholders, all guides included the following topics: benefits and challenges of nurse visits, suggestions for improving the intervention in future trials, and reactions to the trial results (once presented to participants). Only the results of the intention-to-treat (ITT) analysis were shared with stakeholders because ITT is considered the gold standard for trial analysis and allows easy understanding of the results.

A single investigator (A.L.) conducted parental interviews by telephone. Focus groups for PCPs, hospital medicine physicians, and RN groups were held at practice locations in private conference rooms and were conducted by trained moderators (S.N.S., A.L., and H.T.C.). Moderators probed responses to the open-ended questions to delve deeply into issues. The question guides were modified in an iterative fashion to include new concepts raised during interviews or focus groups. All interviews and focus groups were recorded and transcribed verbatim with all identifiable information redacted.

Data Analysis

During multiple cycles of inductive thematic analysis,⁶ we examined, discussed, interpreted, and organized responses to the open-ended questions,^6,7 analyzing each stakeholder group separately. First, transcripts were shared with and reviewed by the entire multidisciplinary team (12 members) which included hospital medicine physicians, PCPs, home care nursing leaders, a nurse scientist, a parent representative, research coordinators, and a qualitative research methodologist. Second, team members convened to discuss overall concepts and ideas and created the preliminary coding frameworks. Third, a smaller subgroup (research coordinator [A.L]., hospital medicine physician [S.R.], parent representative [M.M.], and qualitative research methodologist [S.N.S.]), refined the unique coding framework for each stakeholder group and then independently applied codes to participant comments. This subgroup met regularly to reach consensus about the assigned codes and to further refine the codebooks. The codes were organized into major and minor themes based on recurring patterns in the data and the salience or emphasis given by participants. The subgroup’s work was reviewed and discussed on an ongoing basis by the entire multidisciplinary team. Triangulation of the data was achieved in multiple ways. The preliminary results were shared in several forums, and feedback was solicited and incorporated. Two of 4 members of the subgroup analytic team were not part of the trial planning or data collection, providing a potentially broader perspective. All coding decisions were maintained in an electronic database, and an audit trail was created to document codebook revisions.

A total of 33 parents participated in the interviews (intervention/readmit [8], intervention/no readmit [8], control/readmit [8], and control/no readmit [9]). Although we selected families from all 4 categories, we were not able to explore qualitative differences between these groups because of the relatively low numbers of participants. Parent data was very limited as interviews were brief and “control” parents had not received the intervention. Three focus groups were held with PCPs (7 participants in total), 2 focus groups were held with hospital medicine physicians (12 participants), and 2 focus groups were held with RNs (10 participants).

Goal 1: Explanation of Reutilization Rates

During interviews and focus groups, the results of the H2O trial were discussed, and stakeholders were asked to comment on potential explanations of the findings. 4 major themes and 5 minor themes emerged from analysis of the transcripts (summarized in Table 1).

Theme 1: Appropriateness of Patient Reutilization

Hospital medicine physicians and home care RNs questioned whether the reutilization events were clinically indicated. RNs wondered whether children who reutilized the ED were also readmitted to the hospital; many perceived that if the child was ill enough to be readmitted, then the ED revisit was warranted (Table 2). Parents commented on parental decision-making and changes in clinical status of the child leading to reutilization (Table 2).

Theme 2: Impact of Red Flags/Warning Sign Instructions on Family’s Reutilization Decisions

Theme 3: Hospital-Affiliated RNs “Directing Traffic” Back to Hospital

Both physician groups were concerned that, because the study was conducted by hospital-employed nurses, families might have been more likely to reaccess care at the hospital. Thus, the connection with the hospital was strengthened in the H2O model, potentially at the expense of the connection with PCPs. Physicians hypothesized that families might “still feel part of the medical system,” so families would return to the hospital if there was a problem. PCPs emphasized that there may have been straightforward situations that could have been handled appropriately in the outpatient office (Table 2).

Theme 4: Home Visit RNs Had a Low Threshold for Escalating Care

Parents and PCPs hypothesized that RNs are more conservative and, therefore, would have had a low threshold to refer back to the hospital if there were concerns in the home. One parent commented: “I guess, nurses are just by trade accustomed to erring on the side of caution and medical intervention instead of letting time take its course. … They’re more apt to say it’s better off to go to the hospital and have everything be fine” (Table 2).

Minor Themes

Participants also explained reutilization in ways that coalesced into 5 minor themes: (1) families receiving a visit might perceive that their child was sicker; (2) patients in the control group did not reutilize enough; (3) receiving more education on a child’s illness drives reutilization; (4) provider access issues; and (5) variability of RN experience may determine whether escalated care. Supportive quotations found in Appendix 2.

We directly asked parents if they would want a nurse home visit in the future after discussing the results of the study. Almost all of the parents in the intervention group and most of the parents in the control group were in favor of receiving a visit, even knowing that patients who had received a visit were more likely to reutilize care.

Goal 2: Suggestions for Improving Intervention Design

Three major themes and 3 minor themes were related to improving the design of the intervention (Table 1).

Theme 1: Need for Improved Postdischarge Communication

All stakeholder groups highlighted postdischarge communication as an area that could be improved. Parents were frustrated with regard to attempts to connect with inpatient physicians after discharge. PCPs suggested developing pathways for the RN to connect with the primary care office as opposed to the hospital. Hospital medicine physicians discussed a lack of consensus regarding patient ownership following discharge and were uncertain about what types of postdischarge symptoms PCPs would be comfortable managing. RNs described specific situations when they had difficulty contacting a physician to escalate care (Table 3).

Theme 2: Individualizing Home Visits—One Size Does Not Fit All

All stakeholder groups also encouraged “individualization” of home visits according to patient and family characteristics, diagnosis, and both timing and severity of illness. PCPs recommended visits only for certain diagnoses. Hospital medicine physicians voiced similar sentiments as the PCPs and added that worrisome family dynamics during a hospitalization, such as a lack of engagement with the medical team, might also warrant a visit. RNs suggested visits for those families with more concerns, for example, those with young children or children recovering from an acute respiratory illness (Table 3).

Theme 3: Providing Context for and Framing of Red Flags

Physicians and nurses suggested providing more context to “red flag” instructions and education. RNs emphasized that some families seemed to benefit from the opportunity to discuss their postdischarge concerns with a medical professional. Others appreciated concrete written instructions that spelled out how to respond in certain situations (Table 3).

Minor Themes

Three minor themes were revealed regarding intervention design improvement (Table 1): (1) streamlining the discharge process; (2) improving the definition of the scope and goal of intervention; and (3) extending inpatient team expertise post discharge. Supportive quotations can be found in Appendix 3.

When stakeholders were asked about why postdischarge RN visits led to increased postdischarge urgent healthcare visits, they questioned the appropriateness of the reutilization events, wondered about the lack of context for the warning signs that nurses provided families as part of the intervention, worried that families were encouraged to return to the hospital because of the ties of the trial to the hospital, and suggested that RNs had a low threshold to refer patients back to the hospital. When asked about how to design an improved nurse visit to better support families, stakeholders emphasized improving communication, individualizing the visit, and providing context around the red-flag discussion, enabling more nuanced instructions about how to respond to specific events.

A synthesis of themes suggests that potential drivers for increased utilization rates may lie in the design and goals of the initial project. The intervention was designed to support families and enhance education after discharge, with components derived from pretrial focus groups with families after a hospital discharge.⁸ The intervention was not designed to divert patients from the ED nor did it enhance access to the PCP. A second trial of the intervention adapted to a phone call also failed to decrease reutilization rates.⁹ Both physician stakeholder groups perceived that the intervention directed traffic back to the hospital because of the intervention design. Coupled with the perception that the red flags may have changed a family’s threshold for seeking care and/or that an RN may be more apt to refer back to care, this failure to push utilization to the primary care office may explain the unexpected trial results. Despite the stakeholders’ perception of enhanced connection back to the hospital as a result of the nurse visit, in analysis of visit referral patterns, a referral was made directly back to the ED in only 4 of the 651 trial visits (Tubbs-Cooley H, Riddle SR, Gold JM, et al.; under review. Pediatric clinical and social concerns identified by home visit nurses in the immediate postdischarge period 2020).

Both H2O trials demonstrated improved recall of red flags by parents who received the intervention, which may be important given the stakeholders’ perspectives that the red flags may not have been contextualized well enough. Yet neither trial demonstrated any differences in postdischarge coping or time to return to normal routine. In interviews with parents, despite the clearly stated results of increased reutilization, intervention parents endorsed a desire for a home visit in the future, raising the possibility that our outcome measures did not capture parents’ priorities adequately.

When asked to recommend design improvements of the intervention, 2 major themes (improvement in communication and individualization of visits) were discussed by all stakeholder groups, providing actionable information to modify or create new interventions. Focus groups with clinicians suggested that communication challenges may have influenced reutilization likelihood during the postdischarge period. RNs expressed uncertainty about who to call with problems or questions at the time of a home visit. This was compounded by difficulty reaching physicians. Both hospital medicine physicians and PCPs identified system challenges including questions of patient ownership, variable PCP practice communication preferences, and difficulty in identifying a partnered staff member (on either end of the inpatient-outpatient continuum) who was familiar with a specific patient. While the communication issues raised may reflect difficulties in our local healthcare system, there is broad evidence of postdischarge communication challenges. In adults, postdischarge communication failures between home health staff and physicians are associated with an increased risk of readmission.¹⁰ The real or perceived lack of communication between inpatient and outpatient providers can add to parental confusion post discharge.¹¹ Although there have been efforts to improve the reliability of communication across this gulf,^12,13 it is not clear whether changes to discharge communication could help to avoid pediatric reutilization events.¹⁴

The theme of individualization of the home nurse visit is consistent with evidence regarding the impact of focusing the intervention on patients with specific diagnoses or demographics. In adults, reduced reutilization associated with postdischarge home nurse visits has been described in specific populations such as patients with heart failure and chronic obstructive pulmonary disease.¹⁵ Impact of home nurse visits on patients within diagnosis-specific populations with certain demographics (such as advanced age) has also been described.¹⁶ In the pediatric population, readmission rates vary widely by diagnosis.¹⁷ A systematic review of interventions to reduce pediatric readmissions found increased impact of discharge interventions in specific populations (asthma, oncology, and NICU).³

Next steps may lie in interventions in targeted populations that function as part of a care continuum bridging the patient from the inpatient to the outpatient setting. A home nurse visit as part of this discharge structure may prove to have more impact on reducing reutilization. One population which accounts for a large proportion of readmissions and where there has been recent focus on discharge transition of care has been children with medical complexity.¹⁸ This group was largely excluded from the H2O trial. Postdischarge home nurse visits in this population have been found to be feasible and address many questions and problems, but the effect on readmission is less clear.¹⁹ Family priorities and preferences related to preparation for discharge, including family engagement, respect for discharge readiness, and goal of returning to normal routines, may be areas on which to focus with future interventions in this population.²⁰ In summary, although widespread postdischarge interventions (home nurse visit⁴ and nurse telephone call⁹) have not been found to be effective, targeting interventions to specific populations by diagnosis or demographic factors may prove to be more effective in reducing pediatric reutilization.

There were several strengths to this study. This qualitative approach allowed us to elucidate potential explanations for the H2O trial results from multiple perspectives. The multidisciplinary composition of our analytic team and the use of an iterative process sparked diverse contributions in a dynamic, ongoing discussion and interpretation of our data.

This study should be considered in the context of several limitations. For families and RNs, there was a time lag between participation in the trial and participation in the qualitative study call or focus group which could lead to difficulty recalling details. Only families who received the intervention could give opinions on their experience of the nurse visit, while families in the control group were asked to hypothesize. Focus groups with hospital medicine physicians and PCPs were purposive samples, and complete demographic information of participants was not collected.

Key stakeholders reflecting on a postdischarge RN visit trial suggested multiple potential explanations for the unexpected increase in reutilization in children randomized to the intervention. Certain participants questioned whether all reutilization events were appropriate or necessary. Others expressed concerns that the H2O intervention lacked context and directed children back to the hospital instead of the PCP. Parents, PCPs, hospital medicine physicians, and RNs all suggested that future transition-focused interventions should enhance postdischarge communication, strengthen connection to the PCP, and be more effectively tailored to the needs of the individual patient and family.

Acknowledgments

Collaborators: H2O Trial Study Group: Joanne Bachus, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Monica L Borell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lenisa V Chang, MA, PhD; Patricia Crawford, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sarah A Ferris, BA, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Judy A Heilman BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Jane C Khoury, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen Lawley, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lynne O’Donnell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Hadley S Sauers-Ford, MPH, Department of Pediatrics, UC Davis Health, Sacramento, California; Anita N Shah, DO, MPH, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lauren G Solan, MD, Med, University of Rochester, Rochester, New York; Heidi J Sucharew, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen P Sullivan, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Christine M White, MD, MAT, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio.

Readmission rates are used as metrics for care quality and reimbursement, with penalties applied to hospitals with higher than expected rates¹ and up to 30% of pediatric readmissions deemed potentially preventable.² There is a paucity of information on how to prevent pediatric readmissions,³ yet pediatric hospitals are tasked with implementing interventions for readmission reduction.

The Hospital to Home Outcomes (H2O) trial was a 2-arm, randomized controlled trial in which patients discharged from hospital medicine and neuroscience services at a single institution were randomized to receive a single home visit from a registered nurse (RN) within 96 hours of discharge.⁴ RNs completed a structured nurse visit designed specifically for the trial. Lists of “red flags” or warning signs associated with common diagnoses were provided to assist RNs in standardizing education about when to seek additional care. The hypothesis was that the postdischarge visits would result in lower reutilization rates (unplanned readmissions, emergency department [ED] visits, and urgent care visits).⁵

Unexpectedly, children randomized to receive the postdischarge nurse visit had higher rates of 30-day unplanned healthcare reutilization, with children randomly assigned to the intervention demonstrating higher odds of 30-day healthcare use (OR 1.33; 95% CI 1.003-1.76).⁴ We sought to understand perspectives on these unanticipated findings by obtaining input from relevant stakeholders. There were 2 goals for the qualitative analysis: first, to understand possible explanations of the increased reutilization finding; second, to elicit suggestions for improving the nurse visit intervention.

We selected an in-depth qualitative approach, using interviews and focus groups to explore underlying explanations for the increase in 30-day unplanned healthcare reutilization among those randomized to receive the postdischarge nurse visit during the H2O trial.⁴ Input was sought from 4 stakeholder groups—parents, primary care physicians (PCPs), hospital medicine physicians, and home care RNs—in an effort to triangulate data sources and elicit rich and diverse opinions. Approval was obtained from the Institutional Review Board prior to conducting the study.

Recruitment
Parents

Because we conducted interviews approximately 1 year after the trial’s conclusion, we purposefully selected families who were enrolled in the latter portion of the H2O trial in order to enhance recall. Beginning with the last families in the study, we sequentially contacted families in reverse order. We contacted 10 families in each of 4 categories (intervention/reutilization, intervention/no reutilization, control/reutilization, control/no reutilization). A total of 3 attempts were made by telephone to contact each family. Participants received a grocery store gift card for participating in the study.

Primary Care Physicians

We conducted focus groups with a purposive sample of physicians recruited from 2 community practices and 1 hospital-­owned practice.

Hospital Medicine Physicians

We conducted focus groups with a purposive sample of physicians from our Division of Hospital Medicine. There was a varying level of knowledge of the original trial; however, none of the participants were collaborators in the trial.

Home Care RNs

We conducted focus groups with a subset of RNs who were involved with trial visits. All RNs were members of the pediatric home care division associated with the hospital with specific training in caring for patients at home.

Data Collection

The study team designed question guides for each stakeholder group (Appendix 1). While questions were tailored for specific stakeholders, all guides included the following topics: benefits and challenges of nurse visits, suggestions for improving the intervention in future trials, and reactions to the trial results (once presented to participants). Only the results of the intention-to-treat (ITT) analysis were shared with stakeholders because ITT is considered the gold standard for trial analysis and allows easy understanding of the results.

A single investigator (A.L.) conducted parental interviews by telephone. Focus groups for PCPs, hospital medicine physicians, and RN groups were held at practice locations in private conference rooms and were conducted by trained moderators (S.N.S., A.L., and H.T.C.). Moderators probed responses to the open-ended questions to delve deeply into issues. The question guides were modified in an iterative fashion to include new concepts raised during interviews or focus groups. All interviews and focus groups were recorded and transcribed verbatim with all identifiable information redacted.

Data Analysis

During multiple cycles of inductive thematic analysis,⁶ we examined, discussed, interpreted, and organized responses to the open-ended questions,^6,7 analyzing each stakeholder group separately. First, transcripts were shared with and reviewed by the entire multidisciplinary team (12 members) which included hospital medicine physicians, PCPs, home care nursing leaders, a nurse scientist, a parent representative, research coordinators, and a qualitative research methodologist. Second, team members convened to discuss overall concepts and ideas and created the preliminary coding frameworks. Third, a smaller subgroup (research coordinator [A.L]., hospital medicine physician [S.R.], parent representative [M.M.], and qualitative research methodologist [S.N.S.]), refined the unique coding framework for each stakeholder group and then independently applied codes to participant comments. This subgroup met regularly to reach consensus about the assigned codes and to further refine the codebooks. The codes were organized into major and minor themes based on recurring patterns in the data and the salience or emphasis given by participants. The subgroup’s work was reviewed and discussed on an ongoing basis by the entire multidisciplinary team. Triangulation of the data was achieved in multiple ways. The preliminary results were shared in several forums, and feedback was solicited and incorporated. Two of 4 members of the subgroup analytic team were not part of the trial planning or data collection, providing a potentially broader perspective. All coding decisions were maintained in an electronic database, and an audit trail was created to document codebook revisions.

A total of 33 parents participated in the interviews (intervention/readmit [8], intervention/no readmit [8], control/readmit [8], and control/no readmit [9]). Although we selected families from all 4 categories, we were not able to explore qualitative differences between these groups because of the relatively low numbers of participants. Parent data was very limited as interviews were brief and “control” parents had not received the intervention. Three focus groups were held with PCPs (7 participants in total), 2 focus groups were held with hospital medicine physicians (12 participants), and 2 focus groups were held with RNs (10 participants).

Goal 1: Explanation of Reutilization Rates

During interviews and focus groups, the results of the H2O trial were discussed, and stakeholders were asked to comment on potential explanations of the findings. 4 major themes and 5 minor themes emerged from analysis of the transcripts (summarized in Table 1).

Theme 1: Appropriateness of Patient Reutilization

Hospital medicine physicians and home care RNs questioned whether the reutilization events were clinically indicated. RNs wondered whether children who reutilized the ED were also readmitted to the hospital; many perceived that if the child was ill enough to be readmitted, then the ED revisit was warranted (Table 2). Parents commented on parental decision-making and changes in clinical status of the child leading to reutilization (Table 2).

Theme 2: Impact of Red Flags/Warning Sign Instructions on Family’s Reutilization Decisions

Theme 3: Hospital-Affiliated RNs “Directing Traffic” Back to Hospital

Both physician groups were concerned that, because the study was conducted by hospital-employed nurses, families might have been more likely to reaccess care at the hospital. Thus, the connection with the hospital was strengthened in the H2O model, potentially at the expense of the connection with PCPs. Physicians hypothesized that families might “still feel part of the medical system,” so families would return to the hospital if there was a problem. PCPs emphasized that there may have been straightforward situations that could have been handled appropriately in the outpatient office (Table 2).

Theme 4: Home Visit RNs Had a Low Threshold for Escalating Care

Parents and PCPs hypothesized that RNs are more conservative and, therefore, would have had a low threshold to refer back to the hospital if there were concerns in the home. One parent commented: “I guess, nurses are just by trade accustomed to erring on the side of caution and medical intervention instead of letting time take its course. … They’re more apt to say it’s better off to go to the hospital and have everything be fine” (Table 2).

Minor Themes

Participants also explained reutilization in ways that coalesced into 5 minor themes: (1) families receiving a visit might perceive that their child was sicker; (2) patients in the control group did not reutilize enough; (3) receiving more education on a child’s illness drives reutilization; (4) provider access issues; and (5) variability of RN experience may determine whether escalated care. Supportive quotations found in Appendix 2.

We directly asked parents if they would want a nurse home visit in the future after discussing the results of the study. Almost all of the parents in the intervention group and most of the parents in the control group were in favor of receiving a visit, even knowing that patients who had received a visit were more likely to reutilize care.

Goal 2: Suggestions for Improving Intervention Design

Three major themes and 3 minor themes were related to improving the design of the intervention (Table 1).

Theme 1: Need for Improved Postdischarge Communication

All stakeholder groups highlighted postdischarge communication as an area that could be improved. Parents were frustrated with regard to attempts to connect with inpatient physicians after discharge. PCPs suggested developing pathways for the RN to connect with the primary care office as opposed to the hospital. Hospital medicine physicians discussed a lack of consensus regarding patient ownership following discharge and were uncertain about what types of postdischarge symptoms PCPs would be comfortable managing. RNs described specific situations when they had difficulty contacting a physician to escalate care (Table 3).

Theme 2: Individualizing Home Visits—One Size Does Not Fit All

All stakeholder groups also encouraged “individualization” of home visits according to patient and family characteristics, diagnosis, and both timing and severity of illness. PCPs recommended visits only for certain diagnoses. Hospital medicine physicians voiced similar sentiments as the PCPs and added that worrisome family dynamics during a hospitalization, such as a lack of engagement with the medical team, might also warrant a visit. RNs suggested visits for those families with more concerns, for example, those with young children or children recovering from an acute respiratory illness (Table 3).

Theme 3: Providing Context for and Framing of Red Flags

Physicians and nurses suggested providing more context to “red flag” instructions and education. RNs emphasized that some families seemed to benefit from the opportunity to discuss their postdischarge concerns with a medical professional. Others appreciated concrete written instructions that spelled out how to respond in certain situations (Table 3).

Minor Themes

Three minor themes were revealed regarding intervention design improvement (Table 1): (1) streamlining the discharge process; (2) improving the definition of the scope and goal of intervention; and (3) extending inpatient team expertise post discharge. Supportive quotations can be found in Appendix 3.

When stakeholders were asked about why postdischarge RN visits led to increased postdischarge urgent healthcare visits, they questioned the appropriateness of the reutilization events, wondered about the lack of context for the warning signs that nurses provided families as part of the intervention, worried that families were encouraged to return to the hospital because of the ties of the trial to the hospital, and suggested that RNs had a low threshold to refer patients back to the hospital. When asked about how to design an improved nurse visit to better support families, stakeholders emphasized improving communication, individualizing the visit, and providing context around the red-flag discussion, enabling more nuanced instructions about how to respond to specific events.

A synthesis of themes suggests that potential drivers for increased utilization rates may lie in the design and goals of the initial project. The intervention was designed to support families and enhance education after discharge, with components derived from pretrial focus groups with families after a hospital discharge.⁸ The intervention was not designed to divert patients from the ED nor did it enhance access to the PCP. A second trial of the intervention adapted to a phone call also failed to decrease reutilization rates.⁹ Both physician stakeholder groups perceived that the intervention directed traffic back to the hospital because of the intervention design. Coupled with the perception that the red flags may have changed a family’s threshold for seeking care and/or that an RN may be more apt to refer back to care, this failure to push utilization to the primary care office may explain the unexpected trial results. Despite the stakeholders’ perception of enhanced connection back to the hospital as a result of the nurse visit, in analysis of visit referral patterns, a referral was made directly back to the ED in only 4 of the 651 trial visits (Tubbs-Cooley H, Riddle SR, Gold JM, et al.; under review. Pediatric clinical and social concerns identified by home visit nurses in the immediate postdischarge period 2020).

Both H2O trials demonstrated improved recall of red flags by parents who received the intervention, which may be important given the stakeholders’ perspectives that the red flags may not have been contextualized well enough. Yet neither trial demonstrated any differences in postdischarge coping or time to return to normal routine. In interviews with parents, despite the clearly stated results of increased reutilization, intervention parents endorsed a desire for a home visit in the future, raising the possibility that our outcome measures did not capture parents’ priorities adequately.

When asked to recommend design improvements of the intervention, 2 major themes (improvement in communication and individualization of visits) were discussed by all stakeholder groups, providing actionable information to modify or create new interventions. Focus groups with clinicians suggested that communication challenges may have influenced reutilization likelihood during the postdischarge period. RNs expressed uncertainty about who to call with problems or questions at the time of a home visit. This was compounded by difficulty reaching physicians. Both hospital medicine physicians and PCPs identified system challenges including questions of patient ownership, variable PCP practice communication preferences, and difficulty in identifying a partnered staff member (on either end of the inpatient-outpatient continuum) who was familiar with a specific patient. While the communication issues raised may reflect difficulties in our local healthcare system, there is broad evidence of postdischarge communication challenges. In adults, postdischarge communication failures between home health staff and physicians are associated with an increased risk of readmission.¹⁰ The real or perceived lack of communication between inpatient and outpatient providers can add to parental confusion post discharge.¹¹ Although there have been efforts to improve the reliability of communication across this gulf,^12,13 it is not clear whether changes to discharge communication could help to avoid pediatric reutilization events.¹⁴

The theme of individualization of the home nurse visit is consistent with evidence regarding the impact of focusing the intervention on patients with specific diagnoses or demographics. In adults, reduced reutilization associated with postdischarge home nurse visits has been described in specific populations such as patients with heart failure and chronic obstructive pulmonary disease.¹⁵ Impact of home nurse visits on patients within diagnosis-specific populations with certain demographics (such as advanced age) has also been described.¹⁶ In the pediatric population, readmission rates vary widely by diagnosis.¹⁷ A systematic review of interventions to reduce pediatric readmissions found increased impact of discharge interventions in specific populations (asthma, oncology, and NICU).³

Next steps may lie in interventions in targeted populations that function as part of a care continuum bridging the patient from the inpatient to the outpatient setting. A home nurse visit as part of this discharge structure may prove to have more impact on reducing reutilization. One population which accounts for a large proportion of readmissions and where there has been recent focus on discharge transition of care has been children with medical complexity.¹⁸ This group was largely excluded from the H2O trial. Postdischarge home nurse visits in this population have been found to be feasible and address many questions and problems, but the effect on readmission is less clear.¹⁹ Family priorities and preferences related to preparation for discharge, including family engagement, respect for discharge readiness, and goal of returning to normal routines, may be areas on which to focus with future interventions in this population.²⁰ In summary, although widespread postdischarge interventions (home nurse visit⁴ and nurse telephone call⁹) have not been found to be effective, targeting interventions to specific populations by diagnosis or demographic factors may prove to be more effective in reducing pediatric reutilization.

There were several strengths to this study. This qualitative approach allowed us to elucidate potential explanations for the H2O trial results from multiple perspectives. The multidisciplinary composition of our analytic team and the use of an iterative process sparked diverse contributions in a dynamic, ongoing discussion and interpretation of our data.

This study should be considered in the context of several limitations. For families and RNs, there was a time lag between participation in the trial and participation in the qualitative study call or focus group which could lead to difficulty recalling details. Only families who received the intervention could give opinions on their experience of the nurse visit, while families in the control group were asked to hypothesize. Focus groups with hospital medicine physicians and PCPs were purposive samples, and complete demographic information of participants was not collected.

Key stakeholders reflecting on a postdischarge RN visit trial suggested multiple potential explanations for the unexpected increase in reutilization in children randomized to the intervention. Certain participants questioned whether all reutilization events were appropriate or necessary. Others expressed concerns that the H2O intervention lacked context and directed children back to the hospital instead of the PCP. Parents, PCPs, hospital medicine physicians, and RNs all suggested that future transition-focused interventions should enhance postdischarge communication, strengthen connection to the PCP, and be more effectively tailored to the needs of the individual patient and family.

Acknowledgments

Collaborators: H2O Trial Study Group: Joanne Bachus, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Monica L Borell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lenisa V Chang, MA, PhD; Patricia Crawford, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sarah A Ferris, BA, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Judy A Heilman BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Jane C Khoury, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen Lawley, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lynne O’Donnell, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Hadley S Sauers-Ford, MPH, Department of Pediatrics, UC Davis Health, Sacramento, California; Anita N Shah, DO, MPH, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Lauren G Solan, MD, Med, University of Rochester, Rochester, New York; Heidi J Sucharew, PhD, Division of Biostatistics and Epidemiology, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Karen P Sullivan, BSN, RN, Department of Patient Services, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Christine M White, MD, MAT, Division of Hospital Medicine, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio.

Viral bronchiolitis is the most common indication for infant hospitalization in the United States.¹ The treatment mainstay remains supportive care, including supplemental oxygen when indicated.¹ High flow nasal cannula (HFNC) therapy delivers humidified, heated air blended with oxygen, allowing much higher flow rates than standard nasal cannula therapy and is being used more frequently in inpatient settings.

Infants and toddlers with bronchiolitis develop increased work of breathing to preserve oxygenation and ventilation in the setting of altered airway resistance and lung compliance.^2,3 In addition to oxygen supplementation, HFNC is used to reduce work of breathing through several mechanisms:^2-6 (1) Nasopharyngeal dead space washout clears oxygen-depleted gas at the end of expiration, facilitating alveolar ventilation (ie, carbon dioxide retention improves); (2) High flow rates match increased inspiratory flow demands of acutely ill patients, reducing nasopharyngeal inspiratory resistance and optimizing dead space washout, thus decreasing work of breathing; (3) Adequate flow rates generate distending pressure, which prevents pharyngeal collapse, supports lung recruitment, and reduces respiratory effort (demonstrated in younger infants); and (4) HFNC systems heat and humidify the breathing gas, reducing the metabolic work required to condition cool, dry gas and improving conductance and pulmonary compliance.^2-5

HFNC therapy is used more commonly in acute care units despite limited literature on its effectiveness outside the intensive care unit (ICU).^7,8 We asked the question, “Does use of HFNC therapy for infants with bronchiolitis hospitalized in acute care units result in improved outcomes when compared with standard nasal cannula oxygen therapy, including length of stay (LOS), oxygen therapy duration, and preventing escalations of care such as ICU transfer, positive pressure ventilation, and intubation?” Also, do published studies provide guidance for the initiation and management of HFNC? We focused our search on studies published in the last five years that included patients with bronchiolitis treated with HFNC outside the ICU; here, we review those studies most relevant to pediatric hospitalists.

No guideline exists for initiating flow or fraction of inspired oxygen (FiO₂). HFNC may be initiated for hypoxia, increased work of breathing, or both in patients with bronchiolitis. To achieve optimal dead space washout, inspiratory flow, and distending pressure, initial flow rates should be 1.5 to 2 L/kg/min, particularly for infants and young children.^2-5 Weiler et al.³ evaluated the breathing effort of ICU patients at 0.5, 1, 1.5, and 2 L/kg/min and found optimal flow rates for improved work of breathing were 1.5-2 L/kg/min. The smallest patients, ≤8 kg, saw the greatest benefit, a finding likely explained by larger anatomic dead space in infants/small children compared with older children.³ For older/larger children (>20 kg), an initial flow closer to 1 L/kg/min is often appropriate.⁵ When used for hypoxia, initiating flow without supplemental FiO₂ may improve oxygenation by flushing nasopharyngeal dead space. FiO₂ should be titrated to achieve the goal set by the treatment team, often ≥90%. Improvement in heart rate and peripheral oxygen saturation (SpO₂) can be observed within 60 minutes of initiating HFNC in patients responsive to therapy.⁶

HFNC therapy is safe when used correctly.^6,9,10Potential adverse effects include pneumothorax, pressure injury, mucosal injury/bleeding, and delayed escalation to invasive ventilation. While difficult to quantify, recent studies report low rates or no serious HFNC complications. For example, only 2 of 1,127 patients supported with HFNC developed a pneumothorax and neither required evacuation.^2,9-12

Inclusion criteria and HFNC protocols vary among published studies. Most HFNC protocols reviewed may not have optimally supported all of the patients in their HFNC groups, often by limiting flow to <2 L/kg/min.^6-9,11,12 These variables may explain the disparate results, with some studies demonstrating apparent benefits and others no difference.^7,9,10,12

Two studies of infants with bronchiolitis showed HFNC therapy may prevent ICU transfer, but this benefit may be limited to rescue when standard oxygen therapy fails, rather than as a superior initial support modality.^7,9 Kepreotes et al.⁹ reported a single-center, randomized controlled trial comparing HFNC with standard oxygen therapy with 101 patients in each treatment arm. The primary outcome, median time to wean off oxygen, was not significantly different between the two groups: 24 hours (95% CI: 18-28) in the HFNC group versus 20 hours in the standard therapy group (95% CI: 17-34). The HFNC group had fewer treatment failures (abnormal heart rate, respiratory rate, SpO₂ <90%, or severe respiratory distress score while on maximum therapy) than the standard therapy group, and 20 (63%) of the 33 patients who failed standard therapy were rescued with HFNC, avoiding transfer to the ICU. Fourteen patients from the HFNC group and 12 from the standard oxygen group required transfer to the ICU for support escalation. Although this study did not show a significant difference in oxygen weaning time between groups, it appears to support HFNC use as a rescue modality to reduce or prevent ICU transfer.⁹ Franklin et al.¹⁰ conducted a multicenter, randomized, controlled trial to compare standard nasal cannula oxygen therapy with HFNC (2 L/kg/min) in 1,472 patients. Patients receiving HFNC had lower care escalation rates due to treatment failure, defined as the presence of at least three of four clinical criteria and the clinician determining escalation was indicated. Oxygen therapy duration, ICU admission rates, and LOS were not significantly different between groups. Similar to the previous study, a large portion of the standard therapy patients who failed treatment (102 of 167) crossed over to the HFNC arm in an attempt to avoid ICU transfer. Twelve patients required intubation: 8 (1%) receiving HFNC and 4 (0.5%) receiving the standard therapy.¹⁰

Two additional studies, both with study design limitations, did not demonstrate differences in ICU transfer rates and had variable differences in outcomes. Riese et al.⁷ retrospectively assessed HFNC use outside the ICU at one institution and included 936 patients admitted before and 1,001 patients admitted after HFNC guideline implementation on the wards. Flow rates were based on age and not weight. They found no difference in LOS, ICU transfer rate, ICU LOS, intubation rates, or 30-day readmission rates, though HFNC use increased over time. The HFNC guideline is a potentially significant limitation as it may not have provided optimal flow rates to all subjects given it was based on age rather than weight. Milani et al.¹² performed a single-center observational study of 36 infants aged <12 months, treated for bronchiolitis on the ward, who were informally assigned to HFNC or standard therapy based upon HFNC device availability. HFNC flow rate was determined by the equation: L/min = 8 mL/kg × respiratory rate × 0.3. Using mean weight and respiratory rate for patients in this group, it appears patients in the HFNC group were treated with flow rates less than the 1.5-2 L/kg/min recommended to be effective.^2,3,12 Despite this, clinical improvement was faster in the HFNC group, including respiratory rate and effort, ability to feed, days on oxygen supplementation, and hospital LOS. ICU admission was not different between the two groups.¹² The Table compares the four studies discussed above.

Many children’s hospitals have extended the use of HFNC outside the ICU for children with bronchiolitis despite the paucity of evidence demonstrating its benefit over standard flow oxygen. Given variation in protocols, study designs, outcomes, and number of patients studied, it is difficult to assess its efficacy outside the ICU. However, based on the studies reviewed herein, HFNC therapy does not appear to decrease LOS, time on oxygen, or escalations of care, such as ICU transfers, positive pressure ventilation, or intubation, when used as a primary therapy.^7,9,11,12 Future research will ideally use optimal flow rates to determine the effectiveness of HFNC on acute care units. Although not addressed in the above studies, additional benefits to be considered in future studies include: (1) increased critical care capacity by allowing patients to be supported on the floor and (2) the ability for patients to remain closer to home when HFNC is used in the community hospital setting.

In each of the large, randomized studies reviewed, most (66%-75%) patients treated with standard low flow oxygen were supported successfully and did not require escalation to HFNC.^9,10 Hospitalists should continue to use standard low flow oxygen as first-line respiratory support for patients with bronchiolitis.¹ No evidence supports the use of HFNC therapy early in a child’s inpatient course; rather, it should be used when standard oxygen therapy fails. Future research should focus on better elucidating which patients will benefit most from HFNC to prevent overuse.

Viral bronchiolitis is the most common indication for infant hospitalization in the United States.¹ The treatment mainstay remains supportive care, including supplemental oxygen when indicated.¹ High flow nasal cannula (HFNC) therapy delivers humidified, heated air blended with oxygen, allowing much higher flow rates than standard nasal cannula therapy and is being used more frequently in inpatient settings.

Infants and toddlers with bronchiolitis develop increased work of breathing to preserve oxygenation and ventilation in the setting of altered airway resistance and lung compliance.^2,3 In addition to oxygen supplementation, HFNC is used to reduce work of breathing through several mechanisms:^2-6 (1) Nasopharyngeal dead space washout clears oxygen-depleted gas at the end of expiration, facilitating alveolar ventilation (ie, carbon dioxide retention improves); (2) High flow rates match increased inspiratory flow demands of acutely ill patients, reducing nasopharyngeal inspiratory resistance and optimizing dead space washout, thus decreasing work of breathing; (3) Adequate flow rates generate distending pressure, which prevents pharyngeal collapse, supports lung recruitment, and reduces respiratory effort (demonstrated in younger infants); and (4) HFNC systems heat and humidify the breathing gas, reducing the metabolic work required to condition cool, dry gas and improving conductance and pulmonary compliance.^2-5

HFNC therapy is used more commonly in acute care units despite limited literature on its effectiveness outside the intensive care unit (ICU).^7,8 We asked the question, “Does use of HFNC therapy for infants with bronchiolitis hospitalized in acute care units result in improved outcomes when compared with standard nasal cannula oxygen therapy, including length of stay (LOS), oxygen therapy duration, and preventing escalations of care such as ICU transfer, positive pressure ventilation, and intubation?” Also, do published studies provide guidance for the initiation and management of HFNC? We focused our search on studies published in the last five years that included patients with bronchiolitis treated with HFNC outside the ICU; here, we review those studies most relevant to pediatric hospitalists.

No guideline exists for initiating flow or fraction of inspired oxygen (FiO₂). HFNC may be initiated for hypoxia, increased work of breathing, or both in patients with bronchiolitis. To achieve optimal dead space washout, inspiratory flow, and distending pressure, initial flow rates should be 1.5 to 2 L/kg/min, particularly for infants and young children.^2-5 Weiler et al.³ evaluated the breathing effort of ICU patients at 0.5, 1, 1.5, and 2 L/kg/min and found optimal flow rates for improved work of breathing were 1.5-2 L/kg/min. The smallest patients, ≤8 kg, saw the greatest benefit, a finding likely explained by larger anatomic dead space in infants/small children compared with older children.³ For older/larger children (>20 kg), an initial flow closer to 1 L/kg/min is often appropriate.⁵ When used for hypoxia, initiating flow without supplemental FiO₂ may improve oxygenation by flushing nasopharyngeal dead space. FiO₂ should be titrated to achieve the goal set by the treatment team, often ≥90%. Improvement in heart rate and peripheral oxygen saturation (SpO₂) can be observed within 60 minutes of initiating HFNC in patients responsive to therapy.⁶

HFNC therapy is safe when used correctly.^6,9,10Potential adverse effects include pneumothorax, pressure injury, mucosal injury/bleeding, and delayed escalation to invasive ventilation. While difficult to quantify, recent studies report low rates or no serious HFNC complications. For example, only 2 of 1,127 patients supported with HFNC developed a pneumothorax and neither required evacuation.^2,9-12

Inclusion criteria and HFNC protocols vary among published studies. Most HFNC protocols reviewed may not have optimally supported all of the patients in their HFNC groups, often by limiting flow to <2 L/kg/min.^6-9,11,12 These variables may explain the disparate results, with some studies demonstrating apparent benefits and others no difference.^7,9,10,12

Two studies of infants with bronchiolitis showed HFNC therapy may prevent ICU transfer, but this benefit may be limited to rescue when standard oxygen therapy fails, rather than as a superior initial support modality.^7,9 Kepreotes et al.⁹ reported a single-center, randomized controlled trial comparing HFNC with standard oxygen therapy with 101 patients in each treatment arm. The primary outcome, median time to wean off oxygen, was not significantly different between the two groups: 24 hours (95% CI: 18-28) in the HFNC group versus 20 hours in the standard therapy group (95% CI: 17-34). The HFNC group had fewer treatment failures (abnormal heart rate, respiratory rate, SpO₂ <90%, or severe respiratory distress score while on maximum therapy) than the standard therapy group, and 20 (63%) of the 33 patients who failed standard therapy were rescued with HFNC, avoiding transfer to the ICU. Fourteen patients from the HFNC group and 12 from the standard oxygen group required transfer to the ICU for support escalation. Although this study did not show a significant difference in oxygen weaning time between groups, it appears to support HFNC use as a rescue modality to reduce or prevent ICU transfer.⁹ Franklin et al.¹⁰ conducted a multicenter, randomized, controlled trial to compare standard nasal cannula oxygen therapy with HFNC (2 L/kg/min) in 1,472 patients. Patients receiving HFNC had lower care escalation rates due to treatment failure, defined as the presence of at least three of four clinical criteria and the clinician determining escalation was indicated. Oxygen therapy duration, ICU admission rates, and LOS were not significantly different between groups. Similar to the previous study, a large portion of the standard therapy patients who failed treatment (102 of 167) crossed over to the HFNC arm in an attempt to avoid ICU transfer. Twelve patients required intubation: 8 (1%) receiving HFNC and 4 (0.5%) receiving the standard therapy.¹⁰

Two additional studies, both with study design limitations, did not demonstrate differences in ICU transfer rates and had variable differences in outcomes. Riese et al.⁷ retrospectively assessed HFNC use outside the ICU at one institution and included 936 patients admitted before and 1,001 patients admitted after HFNC guideline implementation on the wards. Flow rates were based on age and not weight. They found no difference in LOS, ICU transfer rate, ICU LOS, intubation rates, or 30-day readmission rates, though HFNC use increased over time. The HFNC guideline is a potentially significant limitation as it may not have provided optimal flow rates to all subjects given it was based on age rather than weight. Milani et al.¹² performed a single-center observational study of 36 infants aged <12 months, treated for bronchiolitis on the ward, who were informally assigned to HFNC or standard therapy based upon HFNC device availability. HFNC flow rate was determined by the equation: L/min = 8 mL/kg × respiratory rate × 0.3. Using mean weight and respiratory rate for patients in this group, it appears patients in the HFNC group were treated with flow rates less than the 1.5-2 L/kg/min recommended to be effective.^2,3,12 Despite this, clinical improvement was faster in the HFNC group, including respiratory rate and effort, ability to feed, days on oxygen supplementation, and hospital LOS. ICU admission was not different between the two groups.¹² The Table compares the four studies discussed above.

Many children’s hospitals have extended the use of HFNC outside the ICU for children with bronchiolitis despite the paucity of evidence demonstrating its benefit over standard flow oxygen. Given variation in protocols, study designs, outcomes, and number of patients studied, it is difficult to assess its efficacy outside the ICU. However, based on the studies reviewed herein, HFNC therapy does not appear to decrease LOS, time on oxygen, or escalations of care, such as ICU transfers, positive pressure ventilation, or intubation, when used as a primary therapy.^7,9,11,12 Future research will ideally use optimal flow rates to determine the effectiveness of HFNC on acute care units. Although not addressed in the above studies, additional benefits to be considered in future studies include: (1) increased critical care capacity by allowing patients to be supported on the floor and (2) the ability for patients to remain closer to home when HFNC is used in the community hospital setting.

In each of the large, randomized studies reviewed, most (66%-75%) patients treated with standard low flow oxygen were supported successfully and did not require escalation to HFNC.^9,10 Hospitalists should continue to use standard low flow oxygen as first-line respiratory support for patients with bronchiolitis.¹ No evidence supports the use of HFNC therapy early in a child’s inpatient course; rather, it should be used when standard oxygen therapy fails. Future research should focus on better elucidating which patients will benefit most from HFNC to prevent overuse.

Viral bronchiolitis is the most common indication for infant hospitalization in the United States.¹ The treatment mainstay remains supportive care, including supplemental oxygen when indicated.¹ High flow nasal cannula (HFNC) therapy delivers humidified, heated air blended with oxygen, allowing much higher flow rates than standard nasal cannula therapy and is being used more frequently in inpatient settings.

Infants and toddlers with bronchiolitis develop increased work of breathing to preserve oxygenation and ventilation in the setting of altered airway resistance and lung compliance.^2,3 In addition to oxygen supplementation, HFNC is used to reduce work of breathing through several mechanisms:^2-6 (1) Nasopharyngeal dead space washout clears oxygen-depleted gas at the end of expiration, facilitating alveolar ventilation (ie, carbon dioxide retention improves); (2) High flow rates match increased inspiratory flow demands of acutely ill patients, reducing nasopharyngeal inspiratory resistance and optimizing dead space washout, thus decreasing work of breathing; (3) Adequate flow rates generate distending pressure, which prevents pharyngeal collapse, supports lung recruitment, and reduces respiratory effort (demonstrated in younger infants); and (4) HFNC systems heat and humidify the breathing gas, reducing the metabolic work required to condition cool, dry gas and improving conductance and pulmonary compliance.^2-5

HFNC therapy is used more commonly in acute care units despite limited literature on its effectiveness outside the intensive care unit (ICU).^7,8 We asked the question, “Does use of HFNC therapy for infants with bronchiolitis hospitalized in acute care units result in improved outcomes when compared with standard nasal cannula oxygen therapy, including length of stay (LOS), oxygen therapy duration, and preventing escalations of care such as ICU transfer, positive pressure ventilation, and intubation?” Also, do published studies provide guidance for the initiation and management of HFNC? We focused our search on studies published in the last five years that included patients with bronchiolitis treated with HFNC outside the ICU; here, we review those studies most relevant to pediatric hospitalists.

No guideline exists for initiating flow or fraction of inspired oxygen (FiO₂). HFNC may be initiated for hypoxia, increased work of breathing, or both in patients with bronchiolitis. To achieve optimal dead space washout, inspiratory flow, and distending pressure, initial flow rates should be 1.5 to 2 L/kg/min, particularly for infants and young children.^2-5 Weiler et al.³ evaluated the breathing effort of ICU patients at 0.5, 1, 1.5, and 2 L/kg/min and found optimal flow rates for improved work of breathing were 1.5-2 L/kg/min. The smallest patients, ≤8 kg, saw the greatest benefit, a finding likely explained by larger anatomic dead space in infants/small children compared with older children.³ For older/larger children (>20 kg), an initial flow closer to 1 L/kg/min is often appropriate.⁵ When used for hypoxia, initiating flow without supplemental FiO₂ may improve oxygenation by flushing nasopharyngeal dead space. FiO₂ should be titrated to achieve the goal set by the treatment team, often ≥90%. Improvement in heart rate and peripheral oxygen saturation (SpO₂) can be observed within 60 minutes of initiating HFNC in patients responsive to therapy.⁶

HFNC therapy is safe when used correctly.^6,9,10Potential adverse effects include pneumothorax, pressure injury, mucosal injury/bleeding, and delayed escalation to invasive ventilation. While difficult to quantify, recent studies report low rates or no serious HFNC complications. For example, only 2 of 1,127 patients supported with HFNC developed a pneumothorax and neither required evacuation.^2,9-12

Inclusion criteria and HFNC protocols vary among published studies. Most HFNC protocols reviewed may not have optimally supported all of the patients in their HFNC groups, often by limiting flow to <2 L/kg/min.^6-9,11,12 These variables may explain the disparate results, with some studies demonstrating apparent benefits and others no difference.^7,9,10,12

Two studies of infants with bronchiolitis showed HFNC therapy may prevent ICU transfer, but this benefit may be limited to rescue when standard oxygen therapy fails, rather than as a superior initial support modality.^7,9 Kepreotes et al.⁹ reported a single-center, randomized controlled trial comparing HFNC with standard oxygen therapy with 101 patients in each treatment arm. The primary outcome, median time to wean off oxygen, was not significantly different between the two groups: 24 hours (95% CI: 18-28) in the HFNC group versus 20 hours in the standard therapy group (95% CI: 17-34). The HFNC group had fewer treatment failures (abnormal heart rate, respiratory rate, SpO₂ <90%, or severe respiratory distress score while on maximum therapy) than the standard therapy group, and 20 (63%) of the 33 patients who failed standard therapy were rescued with HFNC, avoiding transfer to the ICU. Fourteen patients from the HFNC group and 12 from the standard oxygen group required transfer to the ICU for support escalation. Although this study did not show a significant difference in oxygen weaning time between groups, it appears to support HFNC use as a rescue modality to reduce or prevent ICU transfer.⁹ Franklin et al.¹⁰ conducted a multicenter, randomized, controlled trial to compare standard nasal cannula oxygen therapy with HFNC (2 L/kg/min) in 1,472 patients. Patients receiving HFNC had lower care escalation rates due to treatment failure, defined as the presence of at least three of four clinical criteria and the clinician determining escalation was indicated. Oxygen therapy duration, ICU admission rates, and LOS were not significantly different between groups. Similar to the previous study, a large portion of the standard therapy patients who failed treatment (102 of 167) crossed over to the HFNC arm in an attempt to avoid ICU transfer. Twelve patients required intubation: 8 (1%) receiving HFNC and 4 (0.5%) receiving the standard therapy.¹⁰

Two additional studies, both with study design limitations, did not demonstrate differences in ICU transfer rates and had variable differences in outcomes. Riese et al.⁷ retrospectively assessed HFNC use outside the ICU at one institution and included 936 patients admitted before and 1,001 patients admitted after HFNC guideline implementation on the wards. Flow rates were based on age and not weight. They found no difference in LOS, ICU transfer rate, ICU LOS, intubation rates, or 30-day readmission rates, though HFNC use increased over time. The HFNC guideline is a potentially significant limitation as it may not have provided optimal flow rates to all subjects given it was based on age rather than weight. Milani et al.¹² performed a single-center observational study of 36 infants aged <12 months, treated for bronchiolitis on the ward, who were informally assigned to HFNC or standard therapy based upon HFNC device availability. HFNC flow rate was determined by the equation: L/min = 8 mL/kg × respiratory rate × 0.3. Using mean weight and respiratory rate for patients in this group, it appears patients in the HFNC group were treated with flow rates less than the 1.5-2 L/kg/min recommended to be effective.^2,3,12 Despite this, clinical improvement was faster in the HFNC group, including respiratory rate and effort, ability to feed, days on oxygen supplementation, and hospital LOS. ICU admission was not different between the two groups.¹² The Table compares the four studies discussed above.

Many children’s hospitals have extended the use of HFNC outside the ICU for children with bronchiolitis despite the paucity of evidence demonstrating its benefit over standard flow oxygen. Given variation in protocols, study designs, outcomes, and number of patients studied, it is difficult to assess its efficacy outside the ICU. However, based on the studies reviewed herein, HFNC therapy does not appear to decrease LOS, time on oxygen, or escalations of care, such as ICU transfers, positive pressure ventilation, or intubation, when used as a primary therapy.^7,9,11,12 Future research will ideally use optimal flow rates to determine the effectiveness of HFNC on acute care units. Although not addressed in the above studies, additional benefits to be considered in future studies include: (1) increased critical care capacity by allowing patients to be supported on the floor and (2) the ability for patients to remain closer to home when HFNC is used in the community hospital setting.

In each of the large, randomized studies reviewed, most (66%-75%) patients treated with standard low flow oxygen were supported successfully and did not require escalation to HFNC.^9,10 Hospitalists should continue to use standard low flow oxygen as first-line respiratory support for patients with bronchiolitis.¹ No evidence supports the use of HFNC therapy early in a child’s inpatient course; rather, it should be used when standard oxygen therapy fails. Future research should focus on better elucidating which patients will benefit most from HFNC to prevent overuse.

Intravenous (IV) antibiotics are commonly used in hospitalized pediatric patients to treat bacterial infections. Antimicrobial stewardship guidelines published by the Infectious Diseases Society of America (IDSA) recommend institutions develop a systematic plan to convert from IV to enteral antibiotics, as early transition may reduce healthcare costs, decrease length of stay (LOS), and avoid prolonged IV access complications¹ such as extravasation, thrombosis, and catheter-associated infections.^2-5

Pediatric patients with community-acquired pneumonia (CAP) and mild skin and soft tissue infections (SSTI) may not require IV antibiotics, even if the patient is hospitalized.⁶ Although national guidelines for pediatric CAP and SSTI recommend IV antibiotics for hospitalized patients, these guidelines state that mild infections may be treated with enteral antibiotics and emphasize discontinuation of IV antibiotics when the patient meets discharge criteria.^7,8 Furthermore, several enteral antibiotics used for the treatment of CAP and SSTI, such as cephalexin and clindamycin,⁹ have excellent bioavailability (>90%) or can achieve sufficient concentrations to attain the pharmacodynamic target (ie, amoxicillin and trimethoprim–sulfamethoxazole).^10,11 Nonetheless, the guidelines do not explicitly outline criteria regarding the transition from IV to enteral antibiotics.^7,8

At our institution, patients admitted to Hospital Medicine (HM) often remained on IV antibiotics until discharge. Data review revealed that antibiotic treatment of CAP and SSTI posed the greatest opportunity for early conversion to enteral therapy based on the high frequency of admissions and the ability of commonly used enteral antibiotics to attain pharmacodynamic targets. We sought to change practice culture by decoupling transition to enteral antibiotics from discharge and use administration of other enteral medications as an objective indicator for transition. Our aim was to increase the proportion of enterally administered antibiotic doses for HM patients aged >60 days admitted with uncomplicated CAP or SSTI from 44% to 75% in eight months.

Context

Cincinnati Children’s Hospital Medical Center (CCHMC) is a large, urban, academic hospital. The HM division has 45 attendings and admits >8,000 general pediatric patients annually. The five HM teams at the main campus consist of attendings, fellows, residents, and medical students. One HM team serves as the resident quality improvement (QI) team where residents collaborate in a longitudinal study under the guidance of QI-trained coaches. The focus of this QI initiative was determined by resident consensus and aligned with a high-value care curriculum.¹²

To identify the target patient population, we investigated IV antimicrobials frequently used in HM patients. Ampicillin and clindamycin are commonly used IV antibiotics, most frequently corresponding with the diagnoses of CAP and SSTI, respectively, accounting for half of all antibiotic use on the HM service. Amoxicillin, the enteral equivalent of ampicillin, can achieve sufficient concentrations to attain the pharmacodynamic target at infection sites, and clindamycin has high bioavailability, making them ideal options for early transition. Our institution’s robust antimicrobial stewardship program has published local guidelines on using amoxicillin as the enteral antibiotic of choice for uncomplicated CAP, but it does not provide guidance on the timing of transition for either CAP or SSTI; the clinical team makes this decision.

HM attendings were surveyed to determine the criteria used to transition from IV to enteral antibiotics for patients with CAP or SSTI. The survey illustrated practice variability with providers using differing clinical criteria to signal the timing of transition. Additionally, only 49% of respondents (n = 37) rated themselves as “very comfortable” with residents making autonomous decisions to transition to enteral antibiotics. We chose to use the administration of other enteral medications, instead of discharge readiness, as an objective indicator of a patient’s readiness to transition to enteral antibiotics, given the low-risk patient population and the ability of the enteral antibiotics commonly used for CAP and SSTI to achieve pharmacodynamic targets.

The study population included patients aged >60 days admitted to HM with CAP or SSTI treated with any antibiotic. We excluded patients with potential complications or significant progression of their disease process, including patients with parapneumonic effusions or chest tubes, patients who underwent bronchoscopy, and patients with osteomyelitis, septic arthritis, or preseptal or orbital cellulitis. Past medical history and clinical status on admission were not used to exclude patients.

Interventions

Our multidisciplinary team, formed in January 2017, included HM attendings, HM fellows, pediatric residents, a critical care attending, a pharmacy resident, and an antimicrobial stewardship pharmacist. Under the guidance of QI coaches, the residents on the HM QI team developed and tested all interventions on their team and then determined which interventions would spread to the other four teams. The nursing director of our primary HM unit disseminated project updates to bedside nurses. A simplified failure mode and effects analysis identified areas for improvement and potential interventions. Interventions focused on the following key drivers (Figure 1): increased prescriber awareness of medication charge, standardization of conversion from IV to enteral antibiotics, clear definition of the patients ready for transition, ongoing evaluation of the antimicrobial plan, timely recognition by prescribers of patients ready for transition, culture shift regarding the appropriate administration route in the inpatient setting, and transparency of data. The team implemented sequential Plan-Do-Study-Act (PDSA) cycles¹³ to test the interventions.

Charge Table

To improve knowledge about the increased charge for commonly used IV medications compared with enteral formulations, a table comparing relative charges was shared during monthly resident morning conferences and at an HM faculty meeting. The table included charge comparisons between ampicillin and amoxicillin and IV and enteral clindamycin.

Standardized Language in Electronic Health Record (EHR) Antibiotic Plan on Rounds

Standardized language to document antibiotic transition plans was added to admission and progress note templates in the EHR. The standard template prompted residents to (1) define clinical transition criteria, (2) discuss attending comfort with transition overnight (based on survey results), and (3) document patient preference of solid or liquid dosage forms. Plans were reviewed and updated daily. We hypothesized that since residents use the information in the daily progress notes, including assessments and plans, to present on rounds, inclusion of the transition criteria in the note would prompt transition plan discussions.

Communication Bundle

To promote early transition to enteral antibiotics, we standardized the discussion about antibiotic transition between residents and attendings. During a weekly preexisting meeting, the resident QI team reviewed preferences for transitions with the new service attending. By identifying attending preferences early, residents were able to proactively transition patients who met the criteria (eg, antibiotic transition in the evening instead of waiting until morning rounds). This discussion also provided an opportunity to engage service attendings in the QI efforts, which were also shared at HM faculty meetings quarterly.

Recognizing that in times of high census, discussion of patient plans may be abbreviated during rounds, residents were asked to identify all patients on IV antibiotics while reviewing patient medication orders prior to rounds. As part of an existing daily prerounds huddle to discuss rounding logistics, residents listed all patients on IV antibiotics and discussed which patients were ready for transition. If patients could not be transitioned immediately, the team identified the transition criteria.

At preexisting evening huddles between overnight shift HM residents and the evening HM attending, residents identified patients who were prescribed IV antibiotics and discussed readiness for enteral transition. If a patient could be transitioned overnight, enteral antibiotic orders were placed. Overnight residents were also encouraged to review the transition criteria with families upon admission.

Real-time Identification of Failures and Feedback

For two weeks, the EHR was queried daily to identify patients admitted for uncomplicated CAP and SSTI who were on antibiotics as well as other enteral medications. A failure was defined as an IV antibiotic dose given to a patient who was administered any enteral medication. Residents on the QI team approached residents on other HM teams whenever patients were identified as a failed transition to learn about failure reasons.

Study of the Interventions

Data for HM patients who met the inclusion criteria were collected weekly from January 2016 through June 2018 via EHR query. We initially searched for diagnoses that fit under the disease categories of pneumonia and SSTI in the EHR, which generated a list of International Classification of Disease-9 and -10 Diagnosis codes (Appendix Figure 1). The query identified patients based on these codes and reported whether the identified patients took a dose of any enteral medication, excluding nystatin, sildenafil, tacrolimus, and mouthwashes, which are commonly continued during NPO status due to no need for absorption or limited parenteral options. It also reported the ordered route of administration for the queried antibiotics (Appendix Figure 1).

The 2016 calendar year established our baseline to account for seasonal variability. Data were reported weekly and reviewed to evaluate the impact of PDSA cycles and inform new interventions.

Measures

Our process measure was the total number of enteral antibiotic doses divided by all antibiotic doses in patients receiving any enteral medication. We reasoned that if patients were well enough to take medications enterally, they could be given an enteral antibiotic that is highly bioavailable or readily achieves concentrations that attain pharmacodynamic targets. This practice change was a culture shift, decoupling the switch to enteral antibiotics from discharge readiness. Our EHR query reported only the antibiotic doses given to patients who took an enteral medication on the day of antibiotic administration and excluded patients who received only IV medications.

Outcome measures included antimicrobial costs per patient encounter using average wholesale prices, which were reported in our EHR query, and LOS. To ensure that transitions of IV to enteral antibiotics were not negatively impacting patient outcomes, patient readmissions within seven days served as a balancing measure.

Analysis

An annotated statistical process control p-chart tracked the impact of interventions on the proportion of antibiotic doses that were enterally administered during hospitalization. An x-bar and an s-chart tracked the impact of interventions on antimicrobial costs per patient encounter and on LOS. A p-chart and an encounters-between g-chart were used to evaluate the impact of our interventions on readmissions. Control chart rules for identifying special cause were used for center line shifts.¹⁴

Ethical Considerations

This study was part of a larger study of the residency high-value care curriculum,¹² which was deemed exempt by the CCHMC IRB.

The baseline data collected included 372 patients and the postintervention period in 2017 included 326 patients (Table). Approximately two-thirds of patients had a diagnosis of CAP.

The percentage of antibiotic doses given enterally increased from 44% to 80% within eight months (Figure 2). When studying the impact of interventions, residents on the HM QI team found that the standard EHR template added to daily notes did not consistently prompt residents to discuss antibiotic plans and thus was abandoned. Initial improvement coincided with standardizing discussions between residents and attendings regarding transitions. Furthermore, discussion of all patients on IV antibiotics during the prerounds huddle allowed for reliable, daily communication about antibiotic plans and was subsequently spread to and adopted by all HM teams. The percentage of enterally administered antibiotic doses increased to >75% after the evening huddle, which involved all HM teams, and real-time identification of failures on all HM teams with provider feedback. Despite variability when the total number of antibiotic doses prescribed per week was low (<10), we demonstrated sustainability for 11 months (Figure 2), during which the prerounds and evening huddle discussions were continued and an updated control chart was shown monthly to residents during their educational conferences.

Summary

Using improvement science, we promoted earlier transition to enteral antibiotics for children hospitalized with uncomplicated CAP and SSTI by linking the decision for transition to the ability to take other enteral medications, rather than to discharge readiness. We increased the percentage of enterally administered antibiotic doses in this patient population from 44% to 80% in eight months. Although we did not observe a decrease in LOS as previously noted in a cost analysis study comparing pediatric patients with CAP treated with oral antibiotics versus those treated with IV antibiotics,¹⁵ we did find a decrease in LOS variability and in antimicrobial costs to our patients. These cost savings did not include potential savings from nursing or pharmacy labor. In addition, we noted a decrease in the variability in antibiotic prescribing practice, which demonstrates provider ability and willingness to couple antibiotic route transition to an objective characteristic (administration of other enteral medications).

A strength of our study was that residents, the most frequent prescribers of antibiotics on our HM service, were highly involved in the QI initiative, including defining the SMART aim, identifying key drivers, developing interventions, and completing sequential PDSA cycles. Under the guidance of QI-trained coaches, residents developed feasible interventions and assessed their success in real time. Consistent with other studies,^16,17 resident buy-in and involvement led to the success of our improvement study.

Interpretation

Despite emerging evidence regarding the timing of transition to enteral antibiotics, several factors impeded early transition at our institution, including physician culture, variable practice habits, and hospital workflow. Evidence supports the use of enteral antibiotics in immunocompetent children hospitalized for uncomplicated CAP who do not have chronic lung disease, are not in shock, and have oxygen saturations >85%.⁶ Although existing literature suggests that in pediatric patients admitted for SSTIs not involving the eye or bone, IV antibiotics may be transitioned when clinical improvement, evidenced by a reduction in fever or erythema, is noted,⁶ enteral antibiotics that achieve appropriate concentrations to attain pharmacodynamic targets should have the same efficacy as that of IV antibiotics.⁹ Using the criterion of administration of any medication enterally to identify a patient’s readiness to transition, we were able to overcome practice variation among providers who may have differing opinions of what constitutes clinical improvement. Of note, new evidence is emerging on predictors of enteral antibiotic treatment failure in patients with CAP and SSTI to guide transition timing, but these studies have largely focused on the adult population or were performed in the outpatient and emergency department (ED) settings.^18,19 Regardless, the stable number of encounters between readmissions in our patient population likely indicates that treatment failure in these patients was rare.

Rising healthcare costs have led to concerns around sustainability of the healthcare system;^20,21 tackling overuse in clinical practice, as in our study, is one mitigation strategy. Several studies have used QI methods to facilitate the provision of high-value care through the decrease of continuous monitor overuse and extraneous ordering of electrolytes.^22,23 Our QI study adds to the high-value care literature by safely decreasing the use of IV antibiotics. One retrospective study demonstrated that a one-day decrease in the use of IV antibiotics in pneumonia resulted in decreased costs without an increase in readmissions, similar to our findings.²⁴ In adults, QI initiatives aimed at improving early transition of antibiotics utilized electronic trigger tools.^25,26 Fischer et al. used active orders for scheduled enteral medications or an enteral diet as indication that a patient’s IV medications could be converted to enteral form.²⁶

Our work is not without limitations. The list of ICD-9 and -10 codes used to query the EHR did not capture all diagnoses that would be considered as uncomplicated CAP or SSTI. However, we included an extensive list of diagnoses to ensure that the majority of patients meeting our inclusion criteria were captured. Our process measure did not account for patients on IV antibiotics who were not administered other enteral medications but tolerating an enteral diet. These patients were not identified in our EHR query and were not included in our process measure as a failure. However, in latter interventions, residents identified all patients on IV antibiotics, so that patients not identified by our EHR query benefited from our work. Furthermore, this QI study was conducted at a single institution and several interventions took advantage of preexisting structured huddles and a resident QI curriculum, which may not exist at other institutions. Our study does highlight that engaging frontline providers, such as residents, to review antibiotic orders consistently and question the appropriateness of the administration route is key to making incremental changes in prescribing practices.

Through a partnership between HM and Pharmacy and with substantial resident involvement, we improved the transition of IV antibiotics in patients with CAP or SSTI by increasing the percentage of enterally administered antibiotic doses and reducing antimicrobial costs and variability in antibiotic prescribing practices. This work illustrates how reducing overuse of IV antibiotics promotes high-value care and aligns with initiatives to prevent avoidable harm.²⁷ Our work highlights that standardized discussions about medication orders to create consensus around enteral antibiotic transitions, real-time feedback, and challenging the status quo can influence practice habits and effect change.

Next steps include testing automated methods to notify providers of opportunities for transition from IV to enteral antibiotics through embedded clinical decision support, a method similar to the electronic trigger tools used in adult QI studies.^25,26 Since our prerounds huddle includes identifying all patients on IV antibiotics, studying the transition to enteral antibiotics and its effect on prescribing practices in other diagnoses (ie, urinary tract infection and osteomyelitis) may contribute to spreading these efforts. Partnering with our ED colleagues may be an important next step, as several patients admitted to HM on IV antibiotics are given their first dose in the ED.

Acknowledgments

The authors would like to thank the faculty of the James M. Anderson Center for Health Systems Excellence Intermediate Improvement Science Series for their guidance in the planning of this project. The authors would also like to thank Ms. Ursula Bradshaw and Mr. Michael Ponti-Zins for obtaining the hospital data on length of stay and readmissions. The authors acknowledge Dr. Philip Hagedorn for his assistance with the software that queries the electronic health record and Dr. Laura Brower and Dr. Joanna Thomson for their assistance with statistical analysis. The authors are grateful to all the residents and coaches on the QI Hospital Medicine team who contributed ideas on study design and interventions.

Intravenous (IV) antibiotics are commonly used in hospitalized pediatric patients to treat bacterial infections. Antimicrobial stewardship guidelines published by the Infectious Diseases Society of America (IDSA) recommend institutions develop a systematic plan to convert from IV to enteral antibiotics, as early transition may reduce healthcare costs, decrease length of stay (LOS), and avoid prolonged IV access complications¹ such as extravasation, thrombosis, and catheter-associated infections.^2-5

Pediatric patients with community-acquired pneumonia (CAP) and mild skin and soft tissue infections (SSTI) may not require IV antibiotics, even if the patient is hospitalized.⁶ Although national guidelines for pediatric CAP and SSTI recommend IV antibiotics for hospitalized patients, these guidelines state that mild infections may be treated with enteral antibiotics and emphasize discontinuation of IV antibiotics when the patient meets discharge criteria.^7,8 Furthermore, several enteral antibiotics used for the treatment of CAP and SSTI, such as cephalexin and clindamycin,⁹ have excellent bioavailability (>90%) or can achieve sufficient concentrations to attain the pharmacodynamic target (ie, amoxicillin and trimethoprim–sulfamethoxazole).^10,11 Nonetheless, the guidelines do not explicitly outline criteria regarding the transition from IV to enteral antibiotics.^7,8

At our institution, patients admitted to Hospital Medicine (HM) often remained on IV antibiotics until discharge. Data review revealed that antibiotic treatment of CAP and SSTI posed the greatest opportunity for early conversion to enteral therapy based on the high frequency of admissions and the ability of commonly used enteral antibiotics to attain pharmacodynamic targets. We sought to change practice culture by decoupling transition to enteral antibiotics from discharge and use administration of other enteral medications as an objective indicator for transition. Our aim was to increase the proportion of enterally administered antibiotic doses for HM patients aged >60 days admitted with uncomplicated CAP or SSTI from 44% to 75% in eight months.

Context

Cincinnati Children’s Hospital Medical Center (CCHMC) is a large, urban, academic hospital. The HM division has 45 attendings and admits >8,000 general pediatric patients annually. The five HM teams at the main campus consist of attendings, fellows, residents, and medical students. One HM team serves as the resident quality improvement (QI) team where residents collaborate in a longitudinal study under the guidance of QI-trained coaches. The focus of this QI initiative was determined by resident consensus and aligned with a high-value care curriculum.¹²

To identify the target patient population, we investigated IV antimicrobials frequently used in HM patients. Ampicillin and clindamycin are commonly used IV antibiotics, most frequently corresponding with the diagnoses of CAP and SSTI, respectively, accounting for half of all antibiotic use on the HM service. Amoxicillin, the enteral equivalent of ampicillin, can achieve sufficient concentrations to attain the pharmacodynamic target at infection sites, and clindamycin has high bioavailability, making them ideal options for early transition. Our institution’s robust antimicrobial stewardship program has published local guidelines on using amoxicillin as the enteral antibiotic of choice for uncomplicated CAP, but it does not provide guidance on the timing of transition for either CAP or SSTI; the clinical team makes this decision.

HM attendings were surveyed to determine the criteria used to transition from IV to enteral antibiotics for patients with CAP or SSTI. The survey illustrated practice variability with providers using differing clinical criteria to signal the timing of transition. Additionally, only 49% of respondents (n = 37) rated themselves as “very comfortable” with residents making autonomous decisions to transition to enteral antibiotics. We chose to use the administration of other enteral medications, instead of discharge readiness, as an objective indicator of a patient’s readiness to transition to enteral antibiotics, given the low-risk patient population and the ability of the enteral antibiotics commonly used for CAP and SSTI to achieve pharmacodynamic targets.

The study population included patients aged >60 days admitted to HM with CAP or SSTI treated with any antibiotic. We excluded patients with potential complications or significant progression of their disease process, including patients with parapneumonic effusions or chest tubes, patients who underwent bronchoscopy, and patients with osteomyelitis, septic arthritis, or preseptal or orbital cellulitis. Past medical history and clinical status on admission were not used to exclude patients.

Interventions

Our multidisciplinary team, formed in January 2017, included HM attendings, HM fellows, pediatric residents, a critical care attending, a pharmacy resident, and an antimicrobial stewardship pharmacist. Under the guidance of QI coaches, the residents on the HM QI team developed and tested all interventions on their team and then determined which interventions would spread to the other four teams. The nursing director of our primary HM unit disseminated project updates to bedside nurses. A simplified failure mode and effects analysis identified areas for improvement and potential interventions. Interventions focused on the following key drivers (Figure 1): increased prescriber awareness of medication charge, standardization of conversion from IV to enteral antibiotics, clear definition of the patients ready for transition, ongoing evaluation of the antimicrobial plan, timely recognition by prescribers of patients ready for transition, culture shift regarding the appropriate administration route in the inpatient setting, and transparency of data. The team implemented sequential Plan-Do-Study-Act (PDSA) cycles¹³ to test the interventions.

Charge Table

To improve knowledge about the increased charge for commonly used IV medications compared with enteral formulations, a table comparing relative charges was shared during monthly resident morning conferences and at an HM faculty meeting. The table included charge comparisons between ampicillin and amoxicillin and IV and enteral clindamycin.

Standardized Language in Electronic Health Record (EHR) Antibiotic Plan on Rounds

Standardized language to document antibiotic transition plans was added to admission and progress note templates in the EHR. The standard template prompted residents to (1) define clinical transition criteria, (2) discuss attending comfort with transition overnight (based on survey results), and (3) document patient preference of solid or liquid dosage forms. Plans were reviewed and updated daily. We hypothesized that since residents use the information in the daily progress notes, including assessments and plans, to present on rounds, inclusion of the transition criteria in the note would prompt transition plan discussions.

Communication Bundle

To promote early transition to enteral antibiotics, we standardized the discussion about antibiotic transition between residents and attendings. During a weekly preexisting meeting, the resident QI team reviewed preferences for transitions with the new service attending. By identifying attending preferences early, residents were able to proactively transition patients who met the criteria (eg, antibiotic transition in the evening instead of waiting until morning rounds). This discussion also provided an opportunity to engage service attendings in the QI efforts, which were also shared at HM faculty meetings quarterly.

Recognizing that in times of high census, discussion of patient plans may be abbreviated during rounds, residents were asked to identify all patients on IV antibiotics while reviewing patient medication orders prior to rounds. As part of an existing daily prerounds huddle to discuss rounding logistics, residents listed all patients on IV antibiotics and discussed which patients were ready for transition. If patients could not be transitioned immediately, the team identified the transition criteria.

At preexisting evening huddles between overnight shift HM residents and the evening HM attending, residents identified patients who were prescribed IV antibiotics and discussed readiness for enteral transition. If a patient could be transitioned overnight, enteral antibiotic orders were placed. Overnight residents were also encouraged to review the transition criteria with families upon admission.

Real-time Identification of Failures and Feedback

For two weeks, the EHR was queried daily to identify patients admitted for uncomplicated CAP and SSTI who were on antibiotics as well as other enteral medications. A failure was defined as an IV antibiotic dose given to a patient who was administered any enteral medication. Residents on the QI team approached residents on other HM teams whenever patients were identified as a failed transition to learn about failure reasons.

Study of the Interventions

Data for HM patients who met the inclusion criteria were collected weekly from January 2016 through June 2018 via EHR query. We initially searched for diagnoses that fit under the disease categories of pneumonia and SSTI in the EHR, which generated a list of International Classification of Disease-9 and -10 Diagnosis codes (Appendix Figure 1). The query identified patients based on these codes and reported whether the identified patients took a dose of any enteral medication, excluding nystatin, sildenafil, tacrolimus, and mouthwashes, which are commonly continued during NPO status due to no need for absorption or limited parenteral options. It also reported the ordered route of administration for the queried antibiotics (Appendix Figure 1).

The 2016 calendar year established our baseline to account for seasonal variability. Data were reported weekly and reviewed to evaluate the impact of PDSA cycles and inform new interventions.

Measures

Our process measure was the total number of enteral antibiotic doses divided by all antibiotic doses in patients receiving any enteral medication. We reasoned that if patients were well enough to take medications enterally, they could be given an enteral antibiotic that is highly bioavailable or readily achieves concentrations that attain pharmacodynamic targets. This practice change was a culture shift, decoupling the switch to enteral antibiotics from discharge readiness. Our EHR query reported only the antibiotic doses given to patients who took an enteral medication on the day of antibiotic administration and excluded patients who received only IV medications.

Outcome measures included antimicrobial costs per patient encounter using average wholesale prices, which were reported in our EHR query, and LOS. To ensure that transitions of IV to enteral antibiotics were not negatively impacting patient outcomes, patient readmissions within seven days served as a balancing measure.

Analysis

An annotated statistical process control p-chart tracked the impact of interventions on the proportion of antibiotic doses that were enterally administered during hospitalization. An x-bar and an s-chart tracked the impact of interventions on antimicrobial costs per patient encounter and on LOS. A p-chart and an encounters-between g-chart were used to evaluate the impact of our interventions on readmissions. Control chart rules for identifying special cause were used for center line shifts.¹⁴

Ethical Considerations

This study was part of a larger study of the residency high-value care curriculum,¹² which was deemed exempt by the CCHMC IRB.

The baseline data collected included 372 patients and the postintervention period in 2017 included 326 patients (Table). Approximately two-thirds of patients had a diagnosis of CAP.

The percentage of antibiotic doses given enterally increased from 44% to 80% within eight months (Figure 2). When studying the impact of interventions, residents on the HM QI team found that the standard EHR template added to daily notes did not consistently prompt residents to discuss antibiotic plans and thus was abandoned. Initial improvement coincided with standardizing discussions between residents and attendings regarding transitions. Furthermore, discussion of all patients on IV antibiotics during the prerounds huddle allowed for reliable, daily communication about antibiotic plans and was subsequently spread to and adopted by all HM teams. The percentage of enterally administered antibiotic doses increased to >75% after the evening huddle, which involved all HM teams, and real-time identification of failures on all HM teams with provider feedback. Despite variability when the total number of antibiotic doses prescribed per week was low (<10), we demonstrated sustainability for 11 months (Figure 2), during which the prerounds and evening huddle discussions were continued and an updated control chart was shown monthly to residents during their educational conferences.

Summary

Using improvement science, we promoted earlier transition to enteral antibiotics for children hospitalized with uncomplicated CAP and SSTI by linking the decision for transition to the ability to take other enteral medications, rather than to discharge readiness. We increased the percentage of enterally administered antibiotic doses in this patient population from 44% to 80% in eight months. Although we did not observe a decrease in LOS as previously noted in a cost analysis study comparing pediatric patients with CAP treated with oral antibiotics versus those treated with IV antibiotics,¹⁵ we did find a decrease in LOS variability and in antimicrobial costs to our patients. These cost savings did not include potential savings from nursing or pharmacy labor. In addition, we noted a decrease in the variability in antibiotic prescribing practice, which demonstrates provider ability and willingness to couple antibiotic route transition to an objective characteristic (administration of other enteral medications).

A strength of our study was that residents, the most frequent prescribers of antibiotics on our HM service, were highly involved in the QI initiative, including defining the SMART aim, identifying key drivers, developing interventions, and completing sequential PDSA cycles. Under the guidance of QI-trained coaches, residents developed feasible interventions and assessed their success in real time. Consistent with other studies,^16,17 resident buy-in and involvement led to the success of our improvement study.

Interpretation

Despite emerging evidence regarding the timing of transition to enteral antibiotics, several factors impeded early transition at our institution, including physician culture, variable practice habits, and hospital workflow. Evidence supports the use of enteral antibiotics in immunocompetent children hospitalized for uncomplicated CAP who do not have chronic lung disease, are not in shock, and have oxygen saturations >85%.⁶ Although existing literature suggests that in pediatric patients admitted for SSTIs not involving the eye or bone, IV antibiotics may be transitioned when clinical improvement, evidenced by a reduction in fever or erythema, is noted,⁶ enteral antibiotics that achieve appropriate concentrations to attain pharmacodynamic targets should have the same efficacy as that of IV antibiotics.⁹ Using the criterion of administration of any medication enterally to identify a patient’s readiness to transition, we were able to overcome practice variation among providers who may have differing opinions of what constitutes clinical improvement. Of note, new evidence is emerging on predictors of enteral antibiotic treatment failure in patients with CAP and SSTI to guide transition timing, but these studies have largely focused on the adult population or were performed in the outpatient and emergency department (ED) settings.^18,19 Regardless, the stable number of encounters between readmissions in our patient population likely indicates that treatment failure in these patients was rare.

Rising healthcare costs have led to concerns around sustainability of the healthcare system;^20,21 tackling overuse in clinical practice, as in our study, is one mitigation strategy. Several studies have used QI methods to facilitate the provision of high-value care through the decrease of continuous monitor overuse and extraneous ordering of electrolytes.^22,23 Our QI study adds to the high-value care literature by safely decreasing the use of IV antibiotics. One retrospective study demonstrated that a one-day decrease in the use of IV antibiotics in pneumonia resulted in decreased costs without an increase in readmissions, similar to our findings.²⁴ In adults, QI initiatives aimed at improving early transition of antibiotics utilized electronic trigger tools.^25,26 Fischer et al. used active orders for scheduled enteral medications or an enteral diet as indication that a patient’s IV medications could be converted to enteral form.²⁶

Our work is not without limitations. The list of ICD-9 and -10 codes used to query the EHR did not capture all diagnoses that would be considered as uncomplicated CAP or SSTI. However, we included an extensive list of diagnoses to ensure that the majority of patients meeting our inclusion criteria were captured. Our process measure did not account for patients on IV antibiotics who were not administered other enteral medications but tolerating an enteral diet. These patients were not identified in our EHR query and were not included in our process measure as a failure. However, in latter interventions, residents identified all patients on IV antibiotics, so that patients not identified by our EHR query benefited from our work. Furthermore, this QI study was conducted at a single institution and several interventions took advantage of preexisting structured huddles and a resident QI curriculum, which may not exist at other institutions. Our study does highlight that engaging frontline providers, such as residents, to review antibiotic orders consistently and question the appropriateness of the administration route is key to making incremental changes in prescribing practices.

Through a partnership between HM and Pharmacy and with substantial resident involvement, we improved the transition of IV antibiotics in patients with CAP or SSTI by increasing the percentage of enterally administered antibiotic doses and reducing antimicrobial costs and variability in antibiotic prescribing practices. This work illustrates how reducing overuse of IV antibiotics promotes high-value care and aligns with initiatives to prevent avoidable harm.²⁷ Our work highlights that standardized discussions about medication orders to create consensus around enteral antibiotic transitions, real-time feedback, and challenging the status quo can influence practice habits and effect change.

Next steps include testing automated methods to notify providers of opportunities for transition from IV to enteral antibiotics through embedded clinical decision support, a method similar to the electronic trigger tools used in adult QI studies.^25,26 Since our prerounds huddle includes identifying all patients on IV antibiotics, studying the transition to enteral antibiotics and its effect on prescribing practices in other diagnoses (ie, urinary tract infection and osteomyelitis) may contribute to spreading these efforts. Partnering with our ED colleagues may be an important next step, as several patients admitted to HM on IV antibiotics are given their first dose in the ED.

Acknowledgments

The authors would like to thank the faculty of the James M. Anderson Center for Health Systems Excellence Intermediate Improvement Science Series for their guidance in the planning of this project. The authors would also like to thank Ms. Ursula Bradshaw and Mr. Michael Ponti-Zins for obtaining the hospital data on length of stay and readmissions. The authors acknowledge Dr. Philip Hagedorn for his assistance with the software that queries the electronic health record and Dr. Laura Brower and Dr. Joanna Thomson for their assistance with statistical analysis. The authors are grateful to all the residents and coaches on the QI Hospital Medicine team who contributed ideas on study design and interventions.

Intravenous (IV) antibiotics are commonly used in hospitalized pediatric patients to treat bacterial infections. Antimicrobial stewardship guidelines published by the Infectious Diseases Society of America (IDSA) recommend institutions develop a systematic plan to convert from IV to enteral antibiotics, as early transition may reduce healthcare costs, decrease length of stay (LOS), and avoid prolonged IV access complications¹ such as extravasation, thrombosis, and catheter-associated infections.^2-5

Pediatric patients with community-acquired pneumonia (CAP) and mild skin and soft tissue infections (SSTI) may not require IV antibiotics, even if the patient is hospitalized.⁶ Although national guidelines for pediatric CAP and SSTI recommend IV antibiotics for hospitalized patients, these guidelines state that mild infections may be treated with enteral antibiotics and emphasize discontinuation of IV antibiotics when the patient meets discharge criteria.^7,8 Furthermore, several enteral antibiotics used for the treatment of CAP and SSTI, such as cephalexin and clindamycin,⁹ have excellent bioavailability (>90%) or can achieve sufficient concentrations to attain the pharmacodynamic target (ie, amoxicillin and trimethoprim–sulfamethoxazole).^10,11 Nonetheless, the guidelines do not explicitly outline criteria regarding the transition from IV to enteral antibiotics.^7,8

At our institution, patients admitted to Hospital Medicine (HM) often remained on IV antibiotics until discharge. Data review revealed that antibiotic treatment of CAP and SSTI posed the greatest opportunity for early conversion to enteral therapy based on the high frequency of admissions and the ability of commonly used enteral antibiotics to attain pharmacodynamic targets. We sought to change practice culture by decoupling transition to enteral antibiotics from discharge and use administration of other enteral medications as an objective indicator for transition. Our aim was to increase the proportion of enterally administered antibiotic doses for HM patients aged >60 days admitted with uncomplicated CAP or SSTI from 44% to 75% in eight months.

Context

Cincinnati Children’s Hospital Medical Center (CCHMC) is a large, urban, academic hospital. The HM division has 45 attendings and admits >8,000 general pediatric patients annually. The five HM teams at the main campus consist of attendings, fellows, residents, and medical students. One HM team serves as the resident quality improvement (QI) team where residents collaborate in a longitudinal study under the guidance of QI-trained coaches. The focus of this QI initiative was determined by resident consensus and aligned with a high-value care curriculum.¹²

To identify the target patient population, we investigated IV antimicrobials frequently used in HM patients. Ampicillin and clindamycin are commonly used IV antibiotics, most frequently corresponding with the diagnoses of CAP and SSTI, respectively, accounting for half of all antibiotic use on the HM service. Amoxicillin, the enteral equivalent of ampicillin, can achieve sufficient concentrations to attain the pharmacodynamic target at infection sites, and clindamycin has high bioavailability, making them ideal options for early transition. Our institution’s robust antimicrobial stewardship program has published local guidelines on using amoxicillin as the enteral antibiotic of choice for uncomplicated CAP, but it does not provide guidance on the timing of transition for either CAP or SSTI; the clinical team makes this decision.

HM attendings were surveyed to determine the criteria used to transition from IV to enteral antibiotics for patients with CAP or SSTI. The survey illustrated practice variability with providers using differing clinical criteria to signal the timing of transition. Additionally, only 49% of respondents (n = 37) rated themselves as “very comfortable” with residents making autonomous decisions to transition to enteral antibiotics. We chose to use the administration of other enteral medications, instead of discharge readiness, as an objective indicator of a patient’s readiness to transition to enteral antibiotics, given the low-risk patient population and the ability of the enteral antibiotics commonly used for CAP and SSTI to achieve pharmacodynamic targets.

The study population included patients aged >60 days admitted to HM with CAP or SSTI treated with any antibiotic. We excluded patients with potential complications or significant progression of their disease process, including patients with parapneumonic effusions or chest tubes, patients who underwent bronchoscopy, and patients with osteomyelitis, septic arthritis, or preseptal or orbital cellulitis. Past medical history and clinical status on admission were not used to exclude patients.

Interventions

Our multidisciplinary team, formed in January 2017, included HM attendings, HM fellows, pediatric residents, a critical care attending, a pharmacy resident, and an antimicrobial stewardship pharmacist. Under the guidance of QI coaches, the residents on the HM QI team developed and tested all interventions on their team and then determined which interventions would spread to the other four teams. The nursing director of our primary HM unit disseminated project updates to bedside nurses. A simplified failure mode and effects analysis identified areas for improvement and potential interventions. Interventions focused on the following key drivers (Figure 1): increased prescriber awareness of medication charge, standardization of conversion from IV to enteral antibiotics, clear definition of the patients ready for transition, ongoing evaluation of the antimicrobial plan, timely recognition by prescribers of patients ready for transition, culture shift regarding the appropriate administration route in the inpatient setting, and transparency of data. The team implemented sequential Plan-Do-Study-Act (PDSA) cycles¹³ to test the interventions.

Charge Table

To improve knowledge about the increased charge for commonly used IV medications compared with enteral formulations, a table comparing relative charges was shared during monthly resident morning conferences and at an HM faculty meeting. The table included charge comparisons between ampicillin and amoxicillin and IV and enteral clindamycin.

Standardized Language in Electronic Health Record (EHR) Antibiotic Plan on Rounds

Standardized language to document antibiotic transition plans was added to admission and progress note templates in the EHR. The standard template prompted residents to (1) define clinical transition criteria, (2) discuss attending comfort with transition overnight (based on survey results), and (3) document patient preference of solid or liquid dosage forms. Plans were reviewed and updated daily. We hypothesized that since residents use the information in the daily progress notes, including assessments and plans, to present on rounds, inclusion of the transition criteria in the note would prompt transition plan discussions.

Communication Bundle

To promote early transition to enteral antibiotics, we standardized the discussion about antibiotic transition between residents and attendings. During a weekly preexisting meeting, the resident QI team reviewed preferences for transitions with the new service attending. By identifying attending preferences early, residents were able to proactively transition patients who met the criteria (eg, antibiotic transition in the evening instead of waiting until morning rounds). This discussion also provided an opportunity to engage service attendings in the QI efforts, which were also shared at HM faculty meetings quarterly.

Recognizing that in times of high census, discussion of patient plans may be abbreviated during rounds, residents were asked to identify all patients on IV antibiotics while reviewing patient medication orders prior to rounds. As part of an existing daily prerounds huddle to discuss rounding logistics, residents listed all patients on IV antibiotics and discussed which patients were ready for transition. If patients could not be transitioned immediately, the team identified the transition criteria.

At preexisting evening huddles between overnight shift HM residents and the evening HM attending, residents identified patients who were prescribed IV antibiotics and discussed readiness for enteral transition. If a patient could be transitioned overnight, enteral antibiotic orders were placed. Overnight residents were also encouraged to review the transition criteria with families upon admission.

Real-time Identification of Failures and Feedback

For two weeks, the EHR was queried daily to identify patients admitted for uncomplicated CAP and SSTI who were on antibiotics as well as other enteral medications. A failure was defined as an IV antibiotic dose given to a patient who was administered any enteral medication. Residents on the QI team approached residents on other HM teams whenever patients were identified as a failed transition to learn about failure reasons.

Study of the Interventions

Data for HM patients who met the inclusion criteria were collected weekly from January 2016 through June 2018 via EHR query. We initially searched for diagnoses that fit under the disease categories of pneumonia and SSTI in the EHR, which generated a list of International Classification of Disease-9 and -10 Diagnosis codes (Appendix Figure 1). The query identified patients based on these codes and reported whether the identified patients took a dose of any enteral medication, excluding nystatin, sildenafil, tacrolimus, and mouthwashes, which are commonly continued during NPO status due to no need for absorption or limited parenteral options. It also reported the ordered route of administration for the queried antibiotics (Appendix Figure 1).

The 2016 calendar year established our baseline to account for seasonal variability. Data were reported weekly and reviewed to evaluate the impact of PDSA cycles and inform new interventions.

Measures

Our process measure was the total number of enteral antibiotic doses divided by all antibiotic doses in patients receiving any enteral medication. We reasoned that if patients were well enough to take medications enterally, they could be given an enteral antibiotic that is highly bioavailable or readily achieves concentrations that attain pharmacodynamic targets. This practice change was a culture shift, decoupling the switch to enteral antibiotics from discharge readiness. Our EHR query reported only the antibiotic doses given to patients who took an enteral medication on the day of antibiotic administration and excluded patients who received only IV medications.

Outcome measures included antimicrobial costs per patient encounter using average wholesale prices, which were reported in our EHR query, and LOS. To ensure that transitions of IV to enteral antibiotics were not negatively impacting patient outcomes, patient readmissions within seven days served as a balancing measure.

Analysis

An annotated statistical process control p-chart tracked the impact of interventions on the proportion of antibiotic doses that were enterally administered during hospitalization. An x-bar and an s-chart tracked the impact of interventions on antimicrobial costs per patient encounter and on LOS. A p-chart and an encounters-between g-chart were used to evaluate the impact of our interventions on readmissions. Control chart rules for identifying special cause were used for center line shifts.¹⁴

Ethical Considerations

This study was part of a larger study of the residency high-value care curriculum,¹² which was deemed exempt by the CCHMC IRB.

The baseline data collected included 372 patients and the postintervention period in 2017 included 326 patients (Table). Approximately two-thirds of patients had a diagnosis of CAP.

The percentage of antibiotic doses given enterally increased from 44% to 80% within eight months (Figure 2). When studying the impact of interventions, residents on the HM QI team found that the standard EHR template added to daily notes did not consistently prompt residents to discuss antibiotic plans and thus was abandoned. Initial improvement coincided with standardizing discussions between residents and attendings regarding transitions. Furthermore, discussion of all patients on IV antibiotics during the prerounds huddle allowed for reliable, daily communication about antibiotic plans and was subsequently spread to and adopted by all HM teams. The percentage of enterally administered antibiotic doses increased to >75% after the evening huddle, which involved all HM teams, and real-time identification of failures on all HM teams with provider feedback. Despite variability when the total number of antibiotic doses prescribed per week was low (<10), we demonstrated sustainability for 11 months (Figure 2), during which the prerounds and evening huddle discussions were continued and an updated control chart was shown monthly to residents during their educational conferences.

Summary

Using improvement science, we promoted earlier transition to enteral antibiotics for children hospitalized with uncomplicated CAP and SSTI by linking the decision for transition to the ability to take other enteral medications, rather than to discharge readiness. We increased the percentage of enterally administered antibiotic doses in this patient population from 44% to 80% in eight months. Although we did not observe a decrease in LOS as previously noted in a cost analysis study comparing pediatric patients with CAP treated with oral antibiotics versus those treated with IV antibiotics,¹⁵ we did find a decrease in LOS variability and in antimicrobial costs to our patients. These cost savings did not include potential savings from nursing or pharmacy labor. In addition, we noted a decrease in the variability in antibiotic prescribing practice, which demonstrates provider ability and willingness to couple antibiotic route transition to an objective characteristic (administration of other enteral medications).

A strength of our study was that residents, the most frequent prescribers of antibiotics on our HM service, were highly involved in the QI initiative, including defining the SMART aim, identifying key drivers, developing interventions, and completing sequential PDSA cycles. Under the guidance of QI-trained coaches, residents developed feasible interventions and assessed their success in real time. Consistent with other studies,^16,17 resident buy-in and involvement led to the success of our improvement study.

Interpretation

Despite emerging evidence regarding the timing of transition to enteral antibiotics, several factors impeded early transition at our institution, including physician culture, variable practice habits, and hospital workflow. Evidence supports the use of enteral antibiotics in immunocompetent children hospitalized for uncomplicated CAP who do not have chronic lung disease, are not in shock, and have oxygen saturations >85%.⁶ Although existing literature suggests that in pediatric patients admitted for SSTIs not involving the eye or bone, IV antibiotics may be transitioned when clinical improvement, evidenced by a reduction in fever or erythema, is noted,⁶ enteral antibiotics that achieve appropriate concentrations to attain pharmacodynamic targets should have the same efficacy as that of IV antibiotics.⁹ Using the criterion of administration of any medication enterally to identify a patient’s readiness to transition, we were able to overcome practice variation among providers who may have differing opinions of what constitutes clinical improvement. Of note, new evidence is emerging on predictors of enteral antibiotic treatment failure in patients with CAP and SSTI to guide transition timing, but these studies have largely focused on the adult population or were performed in the outpatient and emergency department (ED) settings.^18,19 Regardless, the stable number of encounters between readmissions in our patient population likely indicates that treatment failure in these patients was rare.

Rising healthcare costs have led to concerns around sustainability of the healthcare system;^20,21 tackling overuse in clinical practice, as in our study, is one mitigation strategy. Several studies have used QI methods to facilitate the provision of high-value care through the decrease of continuous monitor overuse and extraneous ordering of electrolytes.^22,23 Our QI study adds to the high-value care literature by safely decreasing the use of IV antibiotics. One retrospective study demonstrated that a one-day decrease in the use of IV antibiotics in pneumonia resulted in decreased costs without an increase in readmissions, similar to our findings.²⁴ In adults, QI initiatives aimed at improving early transition of antibiotics utilized electronic trigger tools.^25,26 Fischer et al. used active orders for scheduled enteral medications or an enteral diet as indication that a patient’s IV medications could be converted to enteral form.²⁶

Our work is not without limitations. The list of ICD-9 and -10 codes used to query the EHR did not capture all diagnoses that would be considered as uncomplicated CAP or SSTI. However, we included an extensive list of diagnoses to ensure that the majority of patients meeting our inclusion criteria were captured. Our process measure did not account for patients on IV antibiotics who were not administered other enteral medications but tolerating an enteral diet. These patients were not identified in our EHR query and were not included in our process measure as a failure. However, in latter interventions, residents identified all patients on IV antibiotics, so that patients not identified by our EHR query benefited from our work. Furthermore, this QI study was conducted at a single institution and several interventions took advantage of preexisting structured huddles and a resident QI curriculum, which may not exist at other institutions. Our study does highlight that engaging frontline providers, such as residents, to review antibiotic orders consistently and question the appropriateness of the administration route is key to making incremental changes in prescribing practices.

Through a partnership between HM and Pharmacy and with substantial resident involvement, we improved the transition of IV antibiotics in patients with CAP or SSTI by increasing the percentage of enterally administered antibiotic doses and reducing antimicrobial costs and variability in antibiotic prescribing practices. This work illustrates how reducing overuse of IV antibiotics promotes high-value care and aligns with initiatives to prevent avoidable harm.²⁷ Our work highlights that standardized discussions about medication orders to create consensus around enteral antibiotic transitions, real-time feedback, and challenging the status quo can influence practice habits and effect change.

Next steps include testing automated methods to notify providers of opportunities for transition from IV to enteral antibiotics through embedded clinical decision support, a method similar to the electronic trigger tools used in adult QI studies.^25,26 Since our prerounds huddle includes identifying all patients on IV antibiotics, studying the transition to enteral antibiotics and its effect on prescribing practices in other diagnoses (ie, urinary tract infection and osteomyelitis) may contribute to spreading these efforts. Partnering with our ED colleagues may be an important next step, as several patients admitted to HM on IV antibiotics are given their first dose in the ED.

Acknowledgments

The authors would like to thank the faculty of the James M. Anderson Center for Health Systems Excellence Intermediate Improvement Science Series for their guidance in the planning of this project. The authors would also like to thank Ms. Ursula Bradshaw and Mr. Michael Ponti-Zins for obtaining the hospital data on length of stay and readmissions. The authors acknowledge Dr. Philip Hagedorn for his assistance with the software that queries the electronic health record and Dr. Laura Brower and Dr. Joanna Thomson for their assistance with statistical analysis. The authors are grateful to all the residents and coaches on the QI Hospital Medicine team who contributed ideas on study design and interventions.

Prior healthcare utilization predicts future utilization;¹ thus, providers should know when a child has had a recent healthcare visit. Healthcare providers typically obtain this information from parents and caregivers, who may not always provide accurate information.^2-4

The Hospital to Home Outcomes study (H2O) was a randomized controlled trial conducted to assess the effects of a one-time home nurse visit following discharge on unplanned healthcare reutilization.⁵ We assessed reutilization through two sources: parent report via a postdischarge telephone call and administrative data. In this analysis, we sought to understand differences in reutilization rates by source by comparing parent report with administrative data.

The H2O trial included children (<18 years) hospitalized on the hospital medicine (HM) or neuroscience (Neurology/Neurosurgery) services at Cincinnati Children’s Hospital Medical Center (CCHMC) from February 2015 to April 2016; they had an English-speaking parent and were discharged to home without skilled nursing care.⁶ For this analysis, we restricted the sample to children randomized to the control arm (discharge without a home visit), which reflects typical clinical care.

We used administrative data to capture 14-day reutilization (unplanned hospital readmissions, emergency department [ED] visits, or urgent care visits). CCHMC is the only pediatric admitting facility in the region and includes two pediatric EDs and five urgent care centers. We supplemented hospital data with a dataset (The Health Collaborative⁷) that included utilization at other regional facilities. Parent report was assessed via a research coordinator phone call 14-23 days after discharge. Parents were asked: “I’m going to [ask] about your child’s health since [discharge date]. Has s/he been hospitalized overnight? Has s/he been taken to the Emergency Room/Emergency Department (didn’t stay overnight)? Has s/he been taken to an urgent care?” We report 14-day reutilization rates by source (parent and/or administrative) and visit type.

We considered administrative data the gold standard for documentation of reutilization events for two reasons. First, all healthcare encounters generate billing and are therefore documented with verifiable coding. Second, we had access to data from our center and other regional healthcare facilities. Any parent-reported utilization to a facility not documented in either dataset was considered an unverifiable event (eg, outside our catchment region). Agreement between administrative and parent report of 14-day reutilization was summarized as positive agreement (reutilization documented in both administrative and parent report), negative agreement (no reutilization reported in either administrative or parent report), and overall agreement (combination of positive and negative agreement). We classified discrepancies as reutilization events in administrative data without parent report of reutilization or vice versa. We performed medical record review of discrepancies in our institutional data.

We summarized agreement by using the Cohen’s kappa statistic by reuse type (hospital readmission, ED, and urgent care visit) and overall (any reutilization event). Strength of agreement based on the kappa statistics was classified as poor (<0.20), fair (0.21-0.40), moderate (0.41-0.60), good (0.61-0.80), and very good (0.81-1.00).⁸ We used McNemar’s test to evaluate marginal homogeneity.

Of 749 children randomized to the standard of care arm, 723 parents completed the 14-day follow-up call and were included in this analysis. The median child age was two years (interquartile range: 0.4, 6.9), the median length of stay (LOS) was two days (1, 3), and the majority were white (62%). Payer mix varied, with 44% privately insured and 54% publicly insured. Most patients (83%) were admitted to the HM service, and the most common diagnoses groups for index admission were respiratory (35%), neurologic (14%), and gastrointestinal (9%) diseases.

Administrative data showed 63 children with any reutilization event; parents reported 63 with any reutilization event; 48 children had events reported by both sources. The overall agreement was high, ranging from 95.9% to 98.5% (Table 1) depending on visit type. The positive agreement (ie, parent and administrative data indicated reutilization) ranged from 47.6% to 76.2%. Negative agreement (ie, parent and administrative data agreed no reutilization) was very high, 97.7% to 99.2%. Parents reported three ED visits and four urgent care visits that were unverifiable due to lack of access to administrative data (sites of care reported were not included in our datasets).

By comparing parent report of reutilization after hospital discharge through postdischarge phone calls with administrative data, we demonstrated high overall agreement between sources (95.9%); this finding is similar to prior research investigating the relationship between an established medical home and reutilization.⁹ However, this agreement is largely due to both sources reporting no reutilization. When revisits did occur, the agreement was notably lower, especially with regard to urgent care visits.

Discrepancies between sources have several possible explanations. First, parents may be confused by the framing of reutilization questions, perhaps lacking clarity around which visit we were referencing. Second, parents may experience limitations in health literacy^10,11 with a lack of familiarity with healthcare language, such as the ability to delineate location types (for example, a parent may identify an urgent care visit as an ED visit, given their close proximity at our facility). Finally, our prior work identified that the “fog” of hospitalization,¹² which is often a stressful and disruptive time for families, may linger after admission and could lead to difficulty in recalling detailed events.

Our findings have implications for effective care in a complex healthcare system where parent report may be the most practical method to obtain historical information, both within clinical care and in the context of research or quality measures, such as postdischarge utilization. Given that one of the greatest risk factors for readmission is prior utilization,¹ the knowledge that a patient experienced a reutilization after a prior discharge might prompt the inpatient provider to better prepare families for subsequent transition to home.

To apply our findings practically, it is important to realize that a parent report may be sufficient when reporting that no revisit occurred, if there is also no record of a visit in accessible administrative data (such as an electronic health record). However, further questions or investigation should be considered when parents report a visit did occur or when administrative data indicate a visit occurred that the parent does not recall. Providers and researchers alike should remember to use health literacy universal precautions with all families, employing plain language without medical jargon.¹³ As linked electronic health record use becomes more prevalent, administrative data may be accessible in real-time, allowing for verification of family interview information. Administrative data beyond a single hospital system should be considered to effectively capture reutilization for research or quality efforts.

Our study has several limitations. Similar to most studies using reutilization outcomes, our data may miss a few unverifiable reuse events. By supplementing with additional regional data,⁷ we likely captured most events. Second, we did not include patients with limited English proficiency, although it is unclear how this might have biased our results. Third, while relatively few families did not complete the calls, it is possible that more discrepancies would have been noted in nonresponders. Fourth, research coordinators administering the calls followed a script to determine reutilization information; in clinical practice, a practitioner might not ask questions as clearly, which could negatively impact recall or might add clarifying follow-up questions to enhance recall. Finally, the analysis occurred in the setting of a randomized controlled trial that included children with relatively noncomplex health conditions with short LOS;⁶ thus, the results may not apply to other populations.

In conclusion, parent report and administrative data of reutilization following hospital discharge were usually in agreement when no reutilization occurred; however, discrepancies were noted more often when reutilizations occurred and may have care implications.

Collaborators

On behalf of the H2O Trial study group including: Joanne Bachus, BSN, RN; Andrew F. Beck, MD, MPH; Monica L. Borell, BSN, RN; Lenisa V. Chang, MA, PhD; Patricia Crawford, RN; Jennifer M. Gold, MSN, RN; Judy A. Heilman BSN, RN; Jane C. Khoury, PhD; Pierce Kuhnell, MS; Karen Lawley, BSN, RN; Allison Loechtenfeldt, BS; Colleen Mangeot, MS; Lynn O’Donnell, BSN, RN; Rita H. Pickler, PhD, RN; Hadley S. Sauers-Ford, MPH; Anita N. Shah, DO, MPH; Susan N. Sherman, DPA; Lauren G. Solan, MD, MEd; Karen P. Sullivan, BSN, RN; Susan Wade-Murphy, MSN, RN

Disclosures

Hospital to Home Outcomes team reports grants from the Patient Centered Outcomes Research Institute during the conduct of the study. Dr. White reports personal fees from the Institute for Health Care Improvement, outside the submitted work.

Funding

This work was supported by the Patient Centered Outcomes Research Institute (IHS-1306-0081 to Dr. S. Shah). All statements in this report, including findings and conclusions, are solely those of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute, its Board of Governors, or the Methodology Committee. Dr Auger’s research is funded by the Agency for Healthcare Research and Quality (1K08HS024735).

Prior healthcare utilization predicts future utilization;¹ thus, providers should know when a child has had a recent healthcare visit. Healthcare providers typically obtain this information from parents and caregivers, who may not always provide accurate information.^2-4

The Hospital to Home Outcomes study (H2O) was a randomized controlled trial conducted to assess the effects of a one-time home nurse visit following discharge on unplanned healthcare reutilization.⁵ We assessed reutilization through two sources: parent report via a postdischarge telephone call and administrative data. In this analysis, we sought to understand differences in reutilization rates by source by comparing parent report with administrative data.

The H2O trial included children (<18 years) hospitalized on the hospital medicine (HM) or neuroscience (Neurology/Neurosurgery) services at Cincinnati Children’s Hospital Medical Center (CCHMC) from February 2015 to April 2016; they had an English-speaking parent and were discharged to home without skilled nursing care.⁶ For this analysis, we restricted the sample to children randomized to the control arm (discharge without a home visit), which reflects typical clinical care.

We used administrative data to capture 14-day reutilization (unplanned hospital readmissions, emergency department [ED] visits, or urgent care visits). CCHMC is the only pediatric admitting facility in the region and includes two pediatric EDs and five urgent care centers. We supplemented hospital data with a dataset (The Health Collaborative⁷) that included utilization at other regional facilities. Parent report was assessed via a research coordinator phone call 14-23 days after discharge. Parents were asked: “I’m going to [ask] about your child’s health since [discharge date]. Has s/he been hospitalized overnight? Has s/he been taken to the Emergency Room/Emergency Department (didn’t stay overnight)? Has s/he been taken to an urgent care?” We report 14-day reutilization rates by source (parent and/or administrative) and visit type.

We considered administrative data the gold standard for documentation of reutilization events for two reasons. First, all healthcare encounters generate billing and are therefore documented with verifiable coding. Second, we had access to data from our center and other regional healthcare facilities. Any parent-reported utilization to a facility not documented in either dataset was considered an unverifiable event (eg, outside our catchment region). Agreement between administrative and parent report of 14-day reutilization was summarized as positive agreement (reutilization documented in both administrative and parent report), negative agreement (no reutilization reported in either administrative or parent report), and overall agreement (combination of positive and negative agreement). We classified discrepancies as reutilization events in administrative data without parent report of reutilization or vice versa. We performed medical record review of discrepancies in our institutional data.

We summarized agreement by using the Cohen’s kappa statistic by reuse type (hospital readmission, ED, and urgent care visit) and overall (any reutilization event). Strength of agreement based on the kappa statistics was classified as poor (<0.20), fair (0.21-0.40), moderate (0.41-0.60), good (0.61-0.80), and very good (0.81-1.00).⁸ We used McNemar’s test to evaluate marginal homogeneity.

Of 749 children randomized to the standard of care arm, 723 parents completed the 14-day follow-up call and were included in this analysis. The median child age was two years (interquartile range: 0.4, 6.9), the median length of stay (LOS) was two days (1, 3), and the majority were white (62%). Payer mix varied, with 44% privately insured and 54% publicly insured. Most patients (83%) were admitted to the HM service, and the most common diagnoses groups for index admission were respiratory (35%), neurologic (14%), and gastrointestinal (9%) diseases.

Administrative data showed 63 children with any reutilization event; parents reported 63 with any reutilization event; 48 children had events reported by both sources. The overall agreement was high, ranging from 95.9% to 98.5% (Table 1) depending on visit type. The positive agreement (ie, parent and administrative data indicated reutilization) ranged from 47.6% to 76.2%. Negative agreement (ie, parent and administrative data agreed no reutilization) was very high, 97.7% to 99.2%. Parents reported three ED visits and four urgent care visits that were unverifiable due to lack of access to administrative data (sites of care reported were not included in our datasets).

By comparing parent report of reutilization after hospital discharge through postdischarge phone calls with administrative data, we demonstrated high overall agreement between sources (95.9%); this finding is similar to prior research investigating the relationship between an established medical home and reutilization.⁹ However, this agreement is largely due to both sources reporting no reutilization. When revisits did occur, the agreement was notably lower, especially with regard to urgent care visits.

Discrepancies between sources have several possible explanations. First, parents may be confused by the framing of reutilization questions, perhaps lacking clarity around which visit we were referencing. Second, parents may experience limitations in health literacy^10,11 with a lack of familiarity with healthcare language, such as the ability to delineate location types (for example, a parent may identify an urgent care visit as an ED visit, given their close proximity at our facility). Finally, our prior work identified that the “fog” of hospitalization,¹² which is often a stressful and disruptive time for families, may linger after admission and could lead to difficulty in recalling detailed events.

Our findings have implications for effective care in a complex healthcare system where parent report may be the most practical method to obtain historical information, both within clinical care and in the context of research or quality measures, such as postdischarge utilization. Given that one of the greatest risk factors for readmission is prior utilization,¹ the knowledge that a patient experienced a reutilization after a prior discharge might prompt the inpatient provider to better prepare families for subsequent transition to home.

To apply our findings practically, it is important to realize that a parent report may be sufficient when reporting that no revisit occurred, if there is also no record of a visit in accessible administrative data (such as an electronic health record). However, further questions or investigation should be considered when parents report a visit did occur or when administrative data indicate a visit occurred that the parent does not recall. Providers and researchers alike should remember to use health literacy universal precautions with all families, employing plain language without medical jargon.¹³ As linked electronic health record use becomes more prevalent, administrative data may be accessible in real-time, allowing for verification of family interview information. Administrative data beyond a single hospital system should be considered to effectively capture reutilization for research or quality efforts.

Our study has several limitations. Similar to most studies using reutilization outcomes, our data may miss a few unverifiable reuse events. By supplementing with additional regional data,⁷ we likely captured most events. Second, we did not include patients with limited English proficiency, although it is unclear how this might have biased our results. Third, while relatively few families did not complete the calls, it is possible that more discrepancies would have been noted in nonresponders. Fourth, research coordinators administering the calls followed a script to determine reutilization information; in clinical practice, a practitioner might not ask questions as clearly, which could negatively impact recall or might add clarifying follow-up questions to enhance recall. Finally, the analysis occurred in the setting of a randomized controlled trial that included children with relatively noncomplex health conditions with short LOS;⁶ thus, the results may not apply to other populations.

In conclusion, parent report and administrative data of reutilization following hospital discharge were usually in agreement when no reutilization occurred; however, discrepancies were noted more often when reutilizations occurred and may have care implications.

Collaborators

On behalf of the H2O Trial study group including: Joanne Bachus, BSN, RN; Andrew F. Beck, MD, MPH; Monica L. Borell, BSN, RN; Lenisa V. Chang, MA, PhD; Patricia Crawford, RN; Jennifer M. Gold, MSN, RN; Judy A. Heilman BSN, RN; Jane C. Khoury, PhD; Pierce Kuhnell, MS; Karen Lawley, BSN, RN; Allison Loechtenfeldt, BS; Colleen Mangeot, MS; Lynn O’Donnell, BSN, RN; Rita H. Pickler, PhD, RN; Hadley S. Sauers-Ford, MPH; Anita N. Shah, DO, MPH; Susan N. Sherman, DPA; Lauren G. Solan, MD, MEd; Karen P. Sullivan, BSN, RN; Susan Wade-Murphy, MSN, RN

Disclosures

Hospital to Home Outcomes team reports grants from the Patient Centered Outcomes Research Institute during the conduct of the study. Dr. White reports personal fees from the Institute for Health Care Improvement, outside the submitted work.

Funding

This work was supported by the Patient Centered Outcomes Research Institute (IHS-1306-0081 to Dr. S. Shah). All statements in this report, including findings and conclusions, are solely those of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute, its Board of Governors, or the Methodology Committee. Dr Auger’s research is funded by the Agency for Healthcare Research and Quality (1K08HS024735).

Prior healthcare utilization predicts future utilization;¹ thus, providers should know when a child has had a recent healthcare visit. Healthcare providers typically obtain this information from parents and caregivers, who may not always provide accurate information.^2-4

The Hospital to Home Outcomes study (H2O) was a randomized controlled trial conducted to assess the effects of a one-time home nurse visit following discharge on unplanned healthcare reutilization.⁵ We assessed reutilization through two sources: parent report via a postdischarge telephone call and administrative data. In this analysis, we sought to understand differences in reutilization rates by source by comparing parent report with administrative data.

The H2O trial included children (<18 years) hospitalized on the hospital medicine (HM) or neuroscience (Neurology/Neurosurgery) services at Cincinnati Children’s Hospital Medical Center (CCHMC) from February 2015 to April 2016; they had an English-speaking parent and were discharged to home without skilled nursing care.⁶ For this analysis, we restricted the sample to children randomized to the control arm (discharge without a home visit), which reflects typical clinical care.

We used administrative data to capture 14-day reutilization (unplanned hospital readmissions, emergency department [ED] visits, or urgent care visits). CCHMC is the only pediatric admitting facility in the region and includes two pediatric EDs and five urgent care centers. We supplemented hospital data with a dataset (The Health Collaborative⁷) that included utilization at other regional facilities. Parent report was assessed via a research coordinator phone call 14-23 days after discharge. Parents were asked: “I’m going to [ask] about your child’s health since [discharge date]. Has s/he been hospitalized overnight? Has s/he been taken to the Emergency Room/Emergency Department (didn’t stay overnight)? Has s/he been taken to an urgent care?” We report 14-day reutilization rates by source (parent and/or administrative) and visit type.

We considered administrative data the gold standard for documentation of reutilization events for two reasons. First, all healthcare encounters generate billing and are therefore documented with verifiable coding. Second, we had access to data from our center and other regional healthcare facilities. Any parent-reported utilization to a facility not documented in either dataset was considered an unverifiable event (eg, outside our catchment region). Agreement between administrative and parent report of 14-day reutilization was summarized as positive agreement (reutilization documented in both administrative and parent report), negative agreement (no reutilization reported in either administrative or parent report), and overall agreement (combination of positive and negative agreement). We classified discrepancies as reutilization events in administrative data without parent report of reutilization or vice versa. We performed medical record review of discrepancies in our institutional data.

We summarized agreement by using the Cohen’s kappa statistic by reuse type (hospital readmission, ED, and urgent care visit) and overall (any reutilization event). Strength of agreement based on the kappa statistics was classified as poor (<0.20), fair (0.21-0.40), moderate (0.41-0.60), good (0.61-0.80), and very good (0.81-1.00).⁸ We used McNemar’s test to evaluate marginal homogeneity.

Of 749 children randomized to the standard of care arm, 723 parents completed the 14-day follow-up call and were included in this analysis. The median child age was two years (interquartile range: 0.4, 6.9), the median length of stay (LOS) was two days (1, 3), and the majority were white (62%). Payer mix varied, with 44% privately insured and 54% publicly insured. Most patients (83%) were admitted to the HM service, and the most common diagnoses groups for index admission were respiratory (35%), neurologic (14%), and gastrointestinal (9%) diseases.

Administrative data showed 63 children with any reutilization event; parents reported 63 with any reutilization event; 48 children had events reported by both sources. The overall agreement was high, ranging from 95.9% to 98.5% (Table 1) depending on visit type. The positive agreement (ie, parent and administrative data indicated reutilization) ranged from 47.6% to 76.2%. Negative agreement (ie, parent and administrative data agreed no reutilization) was very high, 97.7% to 99.2%. Parents reported three ED visits and four urgent care visits that were unverifiable due to lack of access to administrative data (sites of care reported were not included in our datasets).

By comparing parent report of reutilization after hospital discharge through postdischarge phone calls with administrative data, we demonstrated high overall agreement between sources (95.9%); this finding is similar to prior research investigating the relationship between an established medical home and reutilization.⁹ However, this agreement is largely due to both sources reporting no reutilization. When revisits did occur, the agreement was notably lower, especially with regard to urgent care visits.

Discrepancies between sources have several possible explanations. First, parents may be confused by the framing of reutilization questions, perhaps lacking clarity around which visit we were referencing. Second, parents may experience limitations in health literacy^10,11 with a lack of familiarity with healthcare language, such as the ability to delineate location types (for example, a parent may identify an urgent care visit as an ED visit, given their close proximity at our facility). Finally, our prior work identified that the “fog” of hospitalization,¹² which is often a stressful and disruptive time for families, may linger after admission and could lead to difficulty in recalling detailed events.

Our findings have implications for effective care in a complex healthcare system where parent report may be the most practical method to obtain historical information, both within clinical care and in the context of research or quality measures, such as postdischarge utilization. Given that one of the greatest risk factors for readmission is prior utilization,¹ the knowledge that a patient experienced a reutilization after a prior discharge might prompt the inpatient provider to better prepare families for subsequent transition to home.

To apply our findings practically, it is important to realize that a parent report may be sufficient when reporting that no revisit occurred, if there is also no record of a visit in accessible administrative data (such as an electronic health record). However, further questions or investigation should be considered when parents report a visit did occur or when administrative data indicate a visit occurred that the parent does not recall. Providers and researchers alike should remember to use health literacy universal precautions with all families, employing plain language without medical jargon.¹³ As linked electronic health record use becomes more prevalent, administrative data may be accessible in real-time, allowing for verification of family interview information. Administrative data beyond a single hospital system should be considered to effectively capture reutilization for research or quality efforts.

Our study has several limitations. Similar to most studies using reutilization outcomes, our data may miss a few unverifiable reuse events. By supplementing with additional regional data,⁷ we likely captured most events. Second, we did not include patients with limited English proficiency, although it is unclear how this might have biased our results. Third, while relatively few families did not complete the calls, it is possible that more discrepancies would have been noted in nonresponders. Fourth, research coordinators administering the calls followed a script to determine reutilization information; in clinical practice, a practitioner might not ask questions as clearly, which could negatively impact recall or might add clarifying follow-up questions to enhance recall. Finally, the analysis occurred in the setting of a randomized controlled trial that included children with relatively noncomplex health conditions with short LOS;⁶ thus, the results may not apply to other populations.

In conclusion, parent report and administrative data of reutilization following hospital discharge were usually in agreement when no reutilization occurred; however, discrepancies were noted more often when reutilizations occurred and may have care implications.

Collaborators

On behalf of the H2O Trial study group including: Joanne Bachus, BSN, RN; Andrew F. Beck, MD, MPH; Monica L. Borell, BSN, RN; Lenisa V. Chang, MA, PhD; Patricia Crawford, RN; Jennifer M. Gold, MSN, RN; Judy A. Heilman BSN, RN; Jane C. Khoury, PhD; Pierce Kuhnell, MS; Karen Lawley, BSN, RN; Allison Loechtenfeldt, BS; Colleen Mangeot, MS; Lynn O’Donnell, BSN, RN; Rita H. Pickler, PhD, RN; Hadley S. Sauers-Ford, MPH; Anita N. Shah, DO, MPH; Susan N. Sherman, DPA; Lauren G. Solan, MD, MEd; Karen P. Sullivan, BSN, RN; Susan Wade-Murphy, MSN, RN

Disclosures

Hospital to Home Outcomes team reports grants from the Patient Centered Outcomes Research Institute during the conduct of the study. Dr. White reports personal fees from the Institute for Health Care Improvement, outside the submitted work.

Funding

This work was supported by the Patient Centered Outcomes Research Institute (IHS-1306-0081 to Dr. S. Shah). All statements in this report, including findings and conclusions, are solely those of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute, its Board of Governors, or the Methodology Committee. Dr Auger’s research is funded by the Agency for Healthcare Research and Quality (1K08HS024735).