Matt Hall, PhD

Payors have been refining reimbursement policies for observation and inpatient stays over the past decade, and the effects on the healthcare payment system are significant.^1-4 Advocates claim that observation status could improve efficiency in the use of healthcare resources by reducing emergency department (ED) crowding and lowering costs for inpatient care.^5,6 Critics consider observation status to be a cost-shifting strategy that could lead to financial burdens for patients and hospitals.^7,8

Although reimbursement policies for observation stays traditionally have been set by the Centers for Medicare and Medicaid Services (CMS) in a uniform manner,^4,8 state Medicaid programs and commercial health insurers have developed a variety of policies for using observation status in broader populations and hospitals.^9-15 Coverage criteria and implementation timelines of these policies vary by states and commercial insurers.^11-15 For example, the California Department of Health Care Services did not have a specific reimbursement rate for observation stays in 2020, while some state Medicaid programs have had reimbursement policies for observation services in place since 2010.^11-15 These inconsistencies likely result in greater variation in use of observation stays across children’s hospitals than general hospitals.

Previous studies have shown rising trends in use of observation stays among adult patient populations and related implications for patients and general hospitals,^16-19 but few studies have reported the trends for pediatric populations. In this study, we sought to (1) describe recent trends of observation stays for pediatric populations at children’s hospitals from 2010 through 2019 and (2) investigate features of this shifting pattern for pediatric populations and hospital-level use of observation stays.

Study Design, Data, and Populations

We performed a retrospective analysis of the Pediatric Health Information System (PHIS), an administrative database that contains inpatient, observation, ambulatory, and ED encounter-level data from 50 not-for-profit, tertiary care children’s hospitals affiliated with the Children’s Hospital Association (CHA).²⁰ PHIS has an indicator to classify patient types (inpatient, observation, ED visits, ambulatory surgery, clinic visit, and others). The data are de-identified at the time of submission and subjected to validity and reliability checks by CHA and Truven Health Analytics (Ann Arbor, MI) before being included in PHIS. Each encounter in PHIS has only one patient type; therefore, encounters that transition to a higher level of care are assigned to their highest level of care (eg, a patient transitions from observation to inpatient status is classified as an inpatient encounter) to avoid duplicate counting.

To ensure consistent evaluations over time, we included 29 children’s hospitals that consistently reported both inpatient and observation data to PHIS across all quarters from 2010 through 2019. We identified the 20 most common clinical conditions using the All Patients Refined Diagnosis Related Groups (APR-DRGs; 3M Corporation) based upon their total frequencies of observation and inpatient stays over the study period. Regression analyses were conducted using all encounters within the 20 most common APR-DRGs.

Because all data have been de-identified in the PHIS database, the institutional review board at Ann and Robert H. Lurie Children’s Hospital of Chicago granted this study institutional review board–exempt status.

Main Outcome and Measures

We first presented longitudinal trends of observation stays for children’s hospitals using annual percentage of observation stays defined as:

To determine whether different pediatric populations have different trends of observation stays, we measured the growth rates of observation stays for each APR-DRG. Specifically, we first calculated the percentage of observation stays by APR-DRGs and years as described, and then calculated the growth rate of observation stays for each APR-DRG:

Next, we employed prolonged length of stay (LOS) and hospitalization resource-intensity scores for kids (H-RISK) to further investigate the shifting pattern of observation stays. Because most state Medicaid and commercial policies dictate that observation stays should not last longer than 48 hours, we defined prolonged LOS as >2 days.^11-15 We defined the annual percentage of observation stays with prolonged LOS for each year as:

Numerators and denominators of the three measures were obtained by pooling all children’s hospitals included in this study. H-RISK is a continuous variable developed by CHA to measure use of intensive care for children, which is comparable across various APR-DRGs.²¹ Changes in the empirical distribution of H-RISK from observation stays were presented over years using percentiles.

Other measures included sex, age, race, payor, and LOS. To investigate the use of observation stays among payors, we categorized payors into five groups: private, in-state Medicaid (managed care), in-state Medicaid (Children’s Health Insurance Program [CHIP]/others), other government, and all others, according to the data availability. The “private” group consisted of commercial preferred provider organizations, commercial health maintenance organizations, and commercial others. We combined both CHIP and in-state Medicaid (others), including Medicaid fee-for-service or unspecified Medicaid together as “in-state Medicaid (CHIP/others).” Detailed categorization information is summarized in Appendix Table 1. LOS was classified into four groups: 1 day (24 hours), 2 days (48 hours), 3 to 4 days, and >4 days.

Statistical Analysis

Descriptive statistics were stratified by inpatient and observation status and were summarized using frequency, percent, median, and interquartile range (IQR). Chi-square or Wilcoxon rank-sum tests were performed to examine differences between observation and inpatient status. Trends in annual percentage of observation stays and annual percentage of observation stays with prolonged LOS were estimated using first-order autoregressive models, in which year was considered a continuous variable. A nonparametric measure of rank correlation (Spearman’s rank correlation coefficient) was employed to evaluate the correlation between year and H-RISK from observation stays.

The risk-adjusted probability of being admitted as an observation stay was estimated using generalized linear mixed models by adjusting for year, age, sex, race, payor, LOS, H-RISK, and a random intercept for each hospital to control for patient clustering within a hospital (Appendix Model). Hospital-level use of observation stays was measured by risk-adjusted percent use of observation stays for each hospital using the predicted values from generalized linear mixed models. All analyses were performed using SAS software, version 9.4 (SAS Institute) and R (R Core Team, 2019), and P < .05 was considered statistically significant.

Increasing Trend of Observation Stays

Over the study period, there were 5,611,001 encounters, including 3,901,873 (69.5%) inpatient and 1,709,128 (30.5%) observation stays (Appendix Table 1). The number of observation stays increased from 117,246 in 2010 to 207,842 in 2019, and the number of inpatient stays slightly increased from 378,433 to 397,994 over the 10 years (Appendix Table 1). Because of different growth rates between observation and inpatient status, the annual percentage of observation stays increased from 23.7% in 2010 to 34.3% in 2019, while the annual percentage of inpatient stays decreased from 76.3% in 2010 to 65.7% in 2019 (Appendix Table 1; Figure 1, P < .001).

Different Growth Rates of Observation Stays for Various Pediatric Populations

As shown in the Table, growth rates of observation stays increased for 19 of the 20 most common APR-DRGs. The four APR-DRGs having the highest growth rates in observation stays were appendectomy, diabetes mellitus, kidney and urinary tract infections, and cellulitis and other bacterial skin infections (Appendix Figure). In particular, the annual percentage of observation stays for appendectomy increased from 19.8% in 2010 to 54.7% in 2019, with the number of observation stays growing from 2,321 to 7,876, while the number of inpatient stays decreased from 9,384 to 6,535 (Appendix Figure). The annual percentage of observation stays for diabetes mellitus increased from 8.16% in 2010 to 22.74% in 2019. Tonsil and adenoid procedures consistently held the largest numbers of observation stays across the 10 years among all the APR-DRGs, with 115,207 and 31,125 total observation and inpatient stays, respectively (Table).

Characteristics of Observation and Inpatient Stays

Patient characteristics are summarized in Appendix Table 1. There were 542,344 (32.9%) observation stays among patients with in-state Medicaid (managed care), and 241,157 (27.4%) observation stays among in-state Medicaid (CHIP/others). The percentages of observation and inpatient stays were 29.8% and 70.2% for private payor, as well as 29.6% and 70.4% for other government payor. Overall, the median (IQR) of H-RISK among observation stays was 0.79 (0.57-1.19) vs 1.23 (0.72-2.43) for inpatient stays. There were 1,410,694 (82.5%) observation stays discharged within 1 day and 243,972 (14.3%) observation stays discharged within 2 days. However, there were 47,413 (2.8%) and 7,049 (0.4%) observation stays with LOS 3 to 4 days or >4 days, respectively.

Shifting Pattern in Observation Stays

The annual percentage of observation stays with prolonged LOS (>2 days) rose from 1.1% in 2010 to 4.6% in 2019 (P < .001; Figure 2). The empirical distribution of H-RISK from observation stays by years further suggests a slightly increasing trend in intensity of care under observation stays. As shown in Appendix Table 2, although the 1st, 5th, 10th, 25th, and 99th percentiles of H-RISK were relatively stable, the 50th, 75th, 90th, and 95th percentiles of H-RISK were increasing over time. The correlation between year and intensity of care used under observation stays (H-RISK from observation stays) was found to be weak but significantly positive (Spearman correlation coefficients = 0.04; P < .001).

Interaction coefficients from our regression model demonstrate that the existing inverse association between H-RISK and odds of admission as an observation stay became less negative over the years. In 2010, the adjusted odds ratio (OR) of H-RISK was 0.57 (95% CI, 0.55-0.59). By 2017, the adjusted OR had increased to 0.65 (95% CI, 0.64-0.66). Compared with 2010, the seven adjusted ORs of H-RISK at years 2012 through 2018 were observed to be higher and statistically significant (P < .001, Appendix Table 3).

Hospitals-Level Use of Observation Stays

After adjusting for all covariates and hospital random effects, hospital-level use of observation stays increased between 2010 and 2019 for 26 out of 29 children’s hospitals. Although observation status essentially was not used at two children’s hospitals over the study period, the median hospital-level use of observation stays was 26% in 2010 (IQR, 3%-36%) and increased to 46% (IQR: 39%; 55%) in 2019. As shown in Figure 3, the number of hospitals with a low percentage of observation stays (<26%) decreased from 15 in 2010 to 4 in 2019. The number of hospitals with a high percentage of observation stays (≥51%) increased from 5 in 2010 to 10 in 2019. Nevertheless, there remained significant variation in the use of observation stays, and the hospital-level use ranged from 0% to 67% in 2019.

By 2020, observation status has become a key component of healthcare for pediatric patients, and its relevance for children’s hospitals recently has been described.^22,23 However, trends in observation stays for pediatric populations are not known. This represents the first study showing temporal trends of observation stays at children’s hospitals after 2010. Our results confirm that the increase in observation stays for pediatric populations is not attributable to decreasing patient acuity at children’s hospitals. We found a weak but significantly positive correlation between year and intensity of care used under observation stays. Although this correlation might not be clinically important, it demonstrates that patient acuity in observation stays is not decreasing. Regression results suggest that observation stays now encompass patients who need relatively higher intensity of care compared with those admitted under observation status in 2010.

This study also identifies a unique pattern in the use of observation stays among pediatric populations. Earlier studies exclusively focused on observation stays that were admitted from EDs.²⁴ Our results indicate that observation status has been used beyond a bridge from ED care to inpatient admission. In particular, observation status has expanded to include pediatric populations with more diverse clinical conditions (eg, appendicitis and diabetes mellitus), and has become a substantial component of postprocedural admissions (Appendix Figure). Looking forward, it is likely that the use of observation stays might surpass inpatient admissions for more conditions that primarily involve short-term stays.

Observation status originally was designed as a reimbursement strategy for patients who needed short stays in dedicated ED units or hospitals, but did not qualify for inpatient services.^5,25 After several changes in reimbursement policies, CMS released the “two midnight rule” for Medicare beneficiaries in 2013, which replaced condition-based criteria with time-based criteria to determine an inpatient or observation stay.¹ Some Medicaid programs and commercial payors have developed similar policies. Unlike the universal policy for Medicare populations, the regulations for pediatric populations vary by states and health insurers.^11-15,26-28 This might partially explain the wide variation observed among children’s hospital-level use of observation stays. For example, the California Medicaid program did not have a reimbursement rate for observation services as of 2020, while the Texas Medicaid program has had a policy for observation stays since 2010.^12,13 We found that two children’s hospitals in California had the lowest use of observation stays (almost zero), whereas the hospital-level use of observation stays was more than 50% for three out of four children’s hospitals in Texas. In addition to reimbursement policies, individual hospitals also might have different strategies for observation status designation. An earlier survey showed that there was lack of consistency in billing and payor-based designations of observation status at children’s hospitals.²⁹ These findings suggest that children’s hospital-level use of observation stays likely is influenced by reimbursement policy and practical strategy for observation status determination.

Earlier studies reported that observation status could be a more efficient use of healthcare resources.^5,6 However, there are still at least two concerns relevant to children’s hospitals during the last decade. The first is whether the use of observation stays can promote cost-saving or if it is just a cost-shifting strategy. An earlier study demonstrated that observation stays with prolonged LOS might increase risk of cost-sharing among adult patients.²⁹ Our study reveals an increasing trend of observation stays with prolonged LOS for pediatric patients. Similar to adult patients, LOS exceeding 24 or 48 hours could lead to uncovered healthcare costs and financial burdens on families.^30-32 Meanwhile, children’s hospitals also might take on a higher financial liability by implementing observation status. Earlier studies have indicated that resource use between observation and inpatient stays at children’s hospitals is similar, and increasing use of observation stays might lead to financial risk rather than cost effectiveness.³³ Further, administrative costs of observation determination are considerably high.³⁴ Medicaid is the major payor for pediatric patients in children’s hospitals. In this study, more than 50% of encounters were paid through Medicaid programs. It is well known that Medicaid reimbursement rates are lower than Medicare and commercial plans.³⁵ Therefore, the cost-saving conclusion drawn from Medicare patients cannot be generalized to pediatric populations at children’s hospitals without cautious reevaluation.

A second concern with increasing use of observation stays is selection bias in public reporting and comparisons of hospital performance. Presently, four main categories of quality indicators established by the Agency for Healthcare Research and Quality rely heavily on inpatient encounters.³⁶ In this study, we found that the range of hospital-level use of observation stays was large. In 2019, the risk-adjusted percent use of observation stays was less than 5% at three hospitals, while the percent use was greater than 60% in another three hospitals. Therefore, comparisons made without uniform accounting of observation stays might have significant implications for national rankings of children’s hospitals across the United States. These consequences have been investigated in several published studies.^22,23,37-39

There are several limitations to our study. First, the study sample was limited to children’s hospitals that consistently reported inpatient and observation data over the entire study period. Eighteen hospitals (86%) excluded from this study did not consistently submit inpatient and observation data to PHIS from 2010 through 2019. The primary purpose of this study was to present temporal trends of observation stays for children’s hospitals, and it was important to build the hospital cohort based on valid and consistent data during the study period. Appendix Table 4 presents differences of hospital characteristics by included and excluded groups of hospitals. Excluded hospitals might have fewer resources (eg, fewer pediatric intensive care beds). Nonetheless, the selection of hospitals was optimized based on data availability. Second, this study was a retrospective review of an administrative database of children’s hospitals and units. The sample does not represent all children’s hospitals or pediatric patients in the United States, but there are no available data sources—that we know of—that can generate national estimates for both inpatient and observation stays. Third, we did not attempt to conclusively infer any causal effects, and several factors could explain the increasing trends, such as reimbursement policies, hospital-level implementation strategies, determination guidelines for observation status designation, as well as changes in clinical care. Further studies should investigate impact of these factors on the use of observation stays for pediatric patients and children’s hospitals.

Observation status has been increasingly used for pediatric patients with more diverse clinical conditions, and there is a rising trend of prolonged LOS among observation stays since 2010. Considerable variation exists in hospital-level use of observation stays across children’s hospitals. Observation status could be an opportunity to improve efficiency of healthcare resource use or could lead to a financial risk for patients with prolonged LOS. Future studies should explore appropriateness of observation care in clinical practice through leveraging efficient care and alleviating financial risk.

Payors have been refining reimbursement policies for observation and inpatient stays over the past decade, and the effects on the healthcare payment system are significant.^1-4 Advocates claim that observation status could improve efficiency in the use of healthcare resources by reducing emergency department (ED) crowding and lowering costs for inpatient care.^5,6 Critics consider observation status to be a cost-shifting strategy that could lead to financial burdens for patients and hospitals.^7,8

Although reimbursement policies for observation stays traditionally have been set by the Centers for Medicare and Medicaid Services (CMS) in a uniform manner,^4,8 state Medicaid programs and commercial health insurers have developed a variety of policies for using observation status in broader populations and hospitals.^9-15 Coverage criteria and implementation timelines of these policies vary by states and commercial insurers.^11-15 For example, the California Department of Health Care Services did not have a specific reimbursement rate for observation stays in 2020, while some state Medicaid programs have had reimbursement policies for observation services in place since 2010.^11-15 These inconsistencies likely result in greater variation in use of observation stays across children’s hospitals than general hospitals.

Previous studies have shown rising trends in use of observation stays among adult patient populations and related implications for patients and general hospitals,^16-19 but few studies have reported the trends for pediatric populations. In this study, we sought to (1) describe recent trends of observation stays for pediatric populations at children’s hospitals from 2010 through 2019 and (2) investigate features of this shifting pattern for pediatric populations and hospital-level use of observation stays.

Study Design, Data, and Populations

We performed a retrospective analysis of the Pediatric Health Information System (PHIS), an administrative database that contains inpatient, observation, ambulatory, and ED encounter-level data from 50 not-for-profit, tertiary care children’s hospitals affiliated with the Children’s Hospital Association (CHA).²⁰ PHIS has an indicator to classify patient types (inpatient, observation, ED visits, ambulatory surgery, clinic visit, and others). The data are de-identified at the time of submission and subjected to validity and reliability checks by CHA and Truven Health Analytics (Ann Arbor, MI) before being included in PHIS. Each encounter in PHIS has only one patient type; therefore, encounters that transition to a higher level of care are assigned to their highest level of care (eg, a patient transitions from observation to inpatient status is classified as an inpatient encounter) to avoid duplicate counting.

To ensure consistent evaluations over time, we included 29 children’s hospitals that consistently reported both inpatient and observation data to PHIS across all quarters from 2010 through 2019. We identified the 20 most common clinical conditions using the All Patients Refined Diagnosis Related Groups (APR-DRGs; 3M Corporation) based upon their total frequencies of observation and inpatient stays over the study period. Regression analyses were conducted using all encounters within the 20 most common APR-DRGs.

Because all data have been de-identified in the PHIS database, the institutional review board at Ann and Robert H. Lurie Children’s Hospital of Chicago granted this study institutional review board–exempt status.

Main Outcome and Measures

We first presented longitudinal trends of observation stays for children’s hospitals using annual percentage of observation stays defined as:

To determine whether different pediatric populations have different trends of observation stays, we measured the growth rates of observation stays for each APR-DRG. Specifically, we first calculated the percentage of observation stays by APR-DRGs and years as described, and then calculated the growth rate of observation stays for each APR-DRG:

Next, we employed prolonged length of stay (LOS) and hospitalization resource-intensity scores for kids (H-RISK) to further investigate the shifting pattern of observation stays. Because most state Medicaid and commercial policies dictate that observation stays should not last longer than 48 hours, we defined prolonged LOS as >2 days.^11-15 We defined the annual percentage of observation stays with prolonged LOS for each year as:

Numerators and denominators of the three measures were obtained by pooling all children’s hospitals included in this study. H-RISK is a continuous variable developed by CHA to measure use of intensive care for children, which is comparable across various APR-DRGs.²¹ Changes in the empirical distribution of H-RISK from observation stays were presented over years using percentiles.

Other measures included sex, age, race, payor, and LOS. To investigate the use of observation stays among payors, we categorized payors into five groups: private, in-state Medicaid (managed care), in-state Medicaid (Children’s Health Insurance Program [CHIP]/others), other government, and all others, according to the data availability. The “private” group consisted of commercial preferred provider organizations, commercial health maintenance organizations, and commercial others. We combined both CHIP and in-state Medicaid (others), including Medicaid fee-for-service or unspecified Medicaid together as “in-state Medicaid (CHIP/others).” Detailed categorization information is summarized in Appendix Table 1. LOS was classified into four groups: 1 day (24 hours), 2 days (48 hours), 3 to 4 days, and >4 days.

Statistical Analysis

Descriptive statistics were stratified by inpatient and observation status and were summarized using frequency, percent, median, and interquartile range (IQR). Chi-square or Wilcoxon rank-sum tests were performed to examine differences between observation and inpatient status. Trends in annual percentage of observation stays and annual percentage of observation stays with prolonged LOS were estimated using first-order autoregressive models, in which year was considered a continuous variable. A nonparametric measure of rank correlation (Spearman’s rank correlation coefficient) was employed to evaluate the correlation between year and H-RISK from observation stays.

The risk-adjusted probability of being admitted as an observation stay was estimated using generalized linear mixed models by adjusting for year, age, sex, race, payor, LOS, H-RISK, and a random intercept for each hospital to control for patient clustering within a hospital (Appendix Model). Hospital-level use of observation stays was measured by risk-adjusted percent use of observation stays for each hospital using the predicted values from generalized linear mixed models. All analyses were performed using SAS software, version 9.4 (SAS Institute) and R (R Core Team, 2019), and P < .05 was considered statistically significant.

Increasing Trend of Observation Stays

Over the study period, there were 5,611,001 encounters, including 3,901,873 (69.5%) inpatient and 1,709,128 (30.5%) observation stays (Appendix Table 1). The number of observation stays increased from 117,246 in 2010 to 207,842 in 2019, and the number of inpatient stays slightly increased from 378,433 to 397,994 over the 10 years (Appendix Table 1). Because of different growth rates between observation and inpatient status, the annual percentage of observation stays increased from 23.7% in 2010 to 34.3% in 2019, while the annual percentage of inpatient stays decreased from 76.3% in 2010 to 65.7% in 2019 (Appendix Table 1; Figure 1, P < .001).

Different Growth Rates of Observation Stays for Various Pediatric Populations

As shown in the Table, growth rates of observation stays increased for 19 of the 20 most common APR-DRGs. The four APR-DRGs having the highest growth rates in observation stays were appendectomy, diabetes mellitus, kidney and urinary tract infections, and cellulitis and other bacterial skin infections (Appendix Figure). In particular, the annual percentage of observation stays for appendectomy increased from 19.8% in 2010 to 54.7% in 2019, with the number of observation stays growing from 2,321 to 7,876, while the number of inpatient stays decreased from 9,384 to 6,535 (Appendix Figure). The annual percentage of observation stays for diabetes mellitus increased from 8.16% in 2010 to 22.74% in 2019. Tonsil and adenoid procedures consistently held the largest numbers of observation stays across the 10 years among all the APR-DRGs, with 115,207 and 31,125 total observation and inpatient stays, respectively (Table).

Characteristics of Observation and Inpatient Stays

Patient characteristics are summarized in Appendix Table 1. There were 542,344 (32.9%) observation stays among patients with in-state Medicaid (managed care), and 241,157 (27.4%) observation stays among in-state Medicaid (CHIP/others). The percentages of observation and inpatient stays were 29.8% and 70.2% for private payor, as well as 29.6% and 70.4% for other government payor. Overall, the median (IQR) of H-RISK among observation stays was 0.79 (0.57-1.19) vs 1.23 (0.72-2.43) for inpatient stays. There were 1,410,694 (82.5%) observation stays discharged within 1 day and 243,972 (14.3%) observation stays discharged within 2 days. However, there were 47,413 (2.8%) and 7,049 (0.4%) observation stays with LOS 3 to 4 days or >4 days, respectively.

Shifting Pattern in Observation Stays

The annual percentage of observation stays with prolonged LOS (>2 days) rose from 1.1% in 2010 to 4.6% in 2019 (P < .001; Figure 2). The empirical distribution of H-RISK from observation stays by years further suggests a slightly increasing trend in intensity of care under observation stays. As shown in Appendix Table 2, although the 1st, 5th, 10th, 25th, and 99th percentiles of H-RISK were relatively stable, the 50th, 75th, 90th, and 95th percentiles of H-RISK were increasing over time. The correlation between year and intensity of care used under observation stays (H-RISK from observation stays) was found to be weak but significantly positive (Spearman correlation coefficients = 0.04; P < .001).

Interaction coefficients from our regression model demonstrate that the existing inverse association between H-RISK and odds of admission as an observation stay became less negative over the years. In 2010, the adjusted odds ratio (OR) of H-RISK was 0.57 (95% CI, 0.55-0.59). By 2017, the adjusted OR had increased to 0.65 (95% CI, 0.64-0.66). Compared with 2010, the seven adjusted ORs of H-RISK at years 2012 through 2018 were observed to be higher and statistically significant (P < .001, Appendix Table 3).

Hospitals-Level Use of Observation Stays

After adjusting for all covariates and hospital random effects, hospital-level use of observation stays increased between 2010 and 2019 for 26 out of 29 children’s hospitals. Although observation status essentially was not used at two children’s hospitals over the study period, the median hospital-level use of observation stays was 26% in 2010 (IQR, 3%-36%) and increased to 46% (IQR: 39%; 55%) in 2019. As shown in Figure 3, the number of hospitals with a low percentage of observation stays (<26%) decreased from 15 in 2010 to 4 in 2019. The number of hospitals with a high percentage of observation stays (≥51%) increased from 5 in 2010 to 10 in 2019. Nevertheless, there remained significant variation in the use of observation stays, and the hospital-level use ranged from 0% to 67% in 2019.

By 2020, observation status has become a key component of healthcare for pediatric patients, and its relevance for children’s hospitals recently has been described.^22,23 However, trends in observation stays for pediatric populations are not known. This represents the first study showing temporal trends of observation stays at children’s hospitals after 2010. Our results confirm that the increase in observation stays for pediatric populations is not attributable to decreasing patient acuity at children’s hospitals. We found a weak but significantly positive correlation between year and intensity of care used under observation stays. Although this correlation might not be clinically important, it demonstrates that patient acuity in observation stays is not decreasing. Regression results suggest that observation stays now encompass patients who need relatively higher intensity of care compared with those admitted under observation status in 2010.

This study also identifies a unique pattern in the use of observation stays among pediatric populations. Earlier studies exclusively focused on observation stays that were admitted from EDs.²⁴ Our results indicate that observation status has been used beyond a bridge from ED care to inpatient admission. In particular, observation status has expanded to include pediatric populations with more diverse clinical conditions (eg, appendicitis and diabetes mellitus), and has become a substantial component of postprocedural admissions (Appendix Figure). Looking forward, it is likely that the use of observation stays might surpass inpatient admissions for more conditions that primarily involve short-term stays.

Observation status originally was designed as a reimbursement strategy for patients who needed short stays in dedicated ED units or hospitals, but did not qualify for inpatient services.^5,25 After several changes in reimbursement policies, CMS released the “two midnight rule” for Medicare beneficiaries in 2013, which replaced condition-based criteria with time-based criteria to determine an inpatient or observation stay.¹ Some Medicaid programs and commercial payors have developed similar policies. Unlike the universal policy for Medicare populations, the regulations for pediatric populations vary by states and health insurers.^11-15,26-28 This might partially explain the wide variation observed among children’s hospital-level use of observation stays. For example, the California Medicaid program did not have a reimbursement rate for observation services as of 2020, while the Texas Medicaid program has had a policy for observation stays since 2010.^12,13 We found that two children’s hospitals in California had the lowest use of observation stays (almost zero), whereas the hospital-level use of observation stays was more than 50% for three out of four children’s hospitals in Texas. In addition to reimbursement policies, individual hospitals also might have different strategies for observation status designation. An earlier survey showed that there was lack of consistency in billing and payor-based designations of observation status at children’s hospitals.²⁹ These findings suggest that children’s hospital-level use of observation stays likely is influenced by reimbursement policy and practical strategy for observation status determination.

Earlier studies reported that observation status could be a more efficient use of healthcare resources.^5,6 However, there are still at least two concerns relevant to children’s hospitals during the last decade. The first is whether the use of observation stays can promote cost-saving or if it is just a cost-shifting strategy. An earlier study demonstrated that observation stays with prolonged LOS might increase risk of cost-sharing among adult patients.²⁹ Our study reveals an increasing trend of observation stays with prolonged LOS for pediatric patients. Similar to adult patients, LOS exceeding 24 or 48 hours could lead to uncovered healthcare costs and financial burdens on families.^30-32 Meanwhile, children’s hospitals also might take on a higher financial liability by implementing observation status. Earlier studies have indicated that resource use between observation and inpatient stays at children’s hospitals is similar, and increasing use of observation stays might lead to financial risk rather than cost effectiveness.³³ Further, administrative costs of observation determination are considerably high.³⁴ Medicaid is the major payor for pediatric patients in children’s hospitals. In this study, more than 50% of encounters were paid through Medicaid programs. It is well known that Medicaid reimbursement rates are lower than Medicare and commercial plans.³⁵ Therefore, the cost-saving conclusion drawn from Medicare patients cannot be generalized to pediatric populations at children’s hospitals without cautious reevaluation.

A second concern with increasing use of observation stays is selection bias in public reporting and comparisons of hospital performance. Presently, four main categories of quality indicators established by the Agency for Healthcare Research and Quality rely heavily on inpatient encounters.³⁶ In this study, we found that the range of hospital-level use of observation stays was large. In 2019, the risk-adjusted percent use of observation stays was less than 5% at three hospitals, while the percent use was greater than 60% in another three hospitals. Therefore, comparisons made without uniform accounting of observation stays might have significant implications for national rankings of children’s hospitals across the United States. These consequences have been investigated in several published studies.^22,23,37-39

There are several limitations to our study. First, the study sample was limited to children’s hospitals that consistently reported inpatient and observation data over the entire study period. Eighteen hospitals (86%) excluded from this study did not consistently submit inpatient and observation data to PHIS from 2010 through 2019. The primary purpose of this study was to present temporal trends of observation stays for children’s hospitals, and it was important to build the hospital cohort based on valid and consistent data during the study period. Appendix Table 4 presents differences of hospital characteristics by included and excluded groups of hospitals. Excluded hospitals might have fewer resources (eg, fewer pediatric intensive care beds). Nonetheless, the selection of hospitals was optimized based on data availability. Second, this study was a retrospective review of an administrative database of children’s hospitals and units. The sample does not represent all children’s hospitals or pediatric patients in the United States, but there are no available data sources—that we know of—that can generate national estimates for both inpatient and observation stays. Third, we did not attempt to conclusively infer any causal effects, and several factors could explain the increasing trends, such as reimbursement policies, hospital-level implementation strategies, determination guidelines for observation status designation, as well as changes in clinical care. Further studies should investigate impact of these factors on the use of observation stays for pediatric patients and children’s hospitals.

Observation status has been increasingly used for pediatric patients with more diverse clinical conditions, and there is a rising trend of prolonged LOS among observation stays since 2010. Considerable variation exists in hospital-level use of observation stays across children’s hospitals. Observation status could be an opportunity to improve efficiency of healthcare resource use or could lead to a financial risk for patients with prolonged LOS. Future studies should explore appropriateness of observation care in clinical practice through leveraging efficient care and alleviating financial risk.

Payors have been refining reimbursement policies for observation and inpatient stays over the past decade, and the effects on the healthcare payment system are significant.^1-4 Advocates claim that observation status could improve efficiency in the use of healthcare resources by reducing emergency department (ED) crowding and lowering costs for inpatient care.^5,6 Critics consider observation status to be a cost-shifting strategy that could lead to financial burdens for patients and hospitals.^7,8

Although reimbursement policies for observation stays traditionally have been set by the Centers for Medicare and Medicaid Services (CMS) in a uniform manner,^4,8 state Medicaid programs and commercial health insurers have developed a variety of policies for using observation status in broader populations and hospitals.^9-15 Coverage criteria and implementation timelines of these policies vary by states and commercial insurers.^11-15 For example, the California Department of Health Care Services did not have a specific reimbursement rate for observation stays in 2020, while some state Medicaid programs have had reimbursement policies for observation services in place since 2010.^11-15 These inconsistencies likely result in greater variation in use of observation stays across children’s hospitals than general hospitals.

Previous studies have shown rising trends in use of observation stays among adult patient populations and related implications for patients and general hospitals,^16-19 but few studies have reported the trends for pediatric populations. In this study, we sought to (1) describe recent trends of observation stays for pediatric populations at children’s hospitals from 2010 through 2019 and (2) investigate features of this shifting pattern for pediatric populations and hospital-level use of observation stays.

Study Design, Data, and Populations

We performed a retrospective analysis of the Pediatric Health Information System (PHIS), an administrative database that contains inpatient, observation, ambulatory, and ED encounter-level data from 50 not-for-profit, tertiary care children’s hospitals affiliated with the Children’s Hospital Association (CHA).²⁰ PHIS has an indicator to classify patient types (inpatient, observation, ED visits, ambulatory surgery, clinic visit, and others). The data are de-identified at the time of submission and subjected to validity and reliability checks by CHA and Truven Health Analytics (Ann Arbor, MI) before being included in PHIS. Each encounter in PHIS has only one patient type; therefore, encounters that transition to a higher level of care are assigned to their highest level of care (eg, a patient transitions from observation to inpatient status is classified as an inpatient encounter) to avoid duplicate counting.

To ensure consistent evaluations over time, we included 29 children’s hospitals that consistently reported both inpatient and observation data to PHIS across all quarters from 2010 through 2019. We identified the 20 most common clinical conditions using the All Patients Refined Diagnosis Related Groups (APR-DRGs; 3M Corporation) based upon their total frequencies of observation and inpatient stays over the study period. Regression analyses were conducted using all encounters within the 20 most common APR-DRGs.

Because all data have been de-identified in the PHIS database, the institutional review board at Ann and Robert H. Lurie Children’s Hospital of Chicago granted this study institutional review board–exempt status.

Main Outcome and Measures

We first presented longitudinal trends of observation stays for children’s hospitals using annual percentage of observation stays defined as:

To determine whether different pediatric populations have different trends of observation stays, we measured the growth rates of observation stays for each APR-DRG. Specifically, we first calculated the percentage of observation stays by APR-DRGs and years as described, and then calculated the growth rate of observation stays for each APR-DRG:

Next, we employed prolonged length of stay (LOS) and hospitalization resource-intensity scores for kids (H-RISK) to further investigate the shifting pattern of observation stays. Because most state Medicaid and commercial policies dictate that observation stays should not last longer than 48 hours, we defined prolonged LOS as >2 days.^11-15 We defined the annual percentage of observation stays with prolonged LOS for each year as:

Numerators and denominators of the three measures were obtained by pooling all children’s hospitals included in this study. H-RISK is a continuous variable developed by CHA to measure use of intensive care for children, which is comparable across various APR-DRGs.²¹ Changes in the empirical distribution of H-RISK from observation stays were presented over years using percentiles.

Other measures included sex, age, race, payor, and LOS. To investigate the use of observation stays among payors, we categorized payors into five groups: private, in-state Medicaid (managed care), in-state Medicaid (Children’s Health Insurance Program [CHIP]/others), other government, and all others, according to the data availability. The “private” group consisted of commercial preferred provider organizations, commercial health maintenance organizations, and commercial others. We combined both CHIP and in-state Medicaid (others), including Medicaid fee-for-service or unspecified Medicaid together as “in-state Medicaid (CHIP/others).” Detailed categorization information is summarized in Appendix Table 1. LOS was classified into four groups: 1 day (24 hours), 2 days (48 hours), 3 to 4 days, and >4 days.

Statistical Analysis

Descriptive statistics were stratified by inpatient and observation status and were summarized using frequency, percent, median, and interquartile range (IQR). Chi-square or Wilcoxon rank-sum tests were performed to examine differences between observation and inpatient status. Trends in annual percentage of observation stays and annual percentage of observation stays with prolonged LOS were estimated using first-order autoregressive models, in which year was considered a continuous variable. A nonparametric measure of rank correlation (Spearman’s rank correlation coefficient) was employed to evaluate the correlation between year and H-RISK from observation stays.

The risk-adjusted probability of being admitted as an observation stay was estimated using generalized linear mixed models by adjusting for year, age, sex, race, payor, LOS, H-RISK, and a random intercept for each hospital to control for patient clustering within a hospital (Appendix Model). Hospital-level use of observation stays was measured by risk-adjusted percent use of observation stays for each hospital using the predicted values from generalized linear mixed models. All analyses were performed using SAS software, version 9.4 (SAS Institute) and R (R Core Team, 2019), and P < .05 was considered statistically significant.

Increasing Trend of Observation Stays

Over the study period, there were 5,611,001 encounters, including 3,901,873 (69.5%) inpatient and 1,709,128 (30.5%) observation stays (Appendix Table 1). The number of observation stays increased from 117,246 in 2010 to 207,842 in 2019, and the number of inpatient stays slightly increased from 378,433 to 397,994 over the 10 years (Appendix Table 1). Because of different growth rates between observation and inpatient status, the annual percentage of observation stays increased from 23.7% in 2010 to 34.3% in 2019, while the annual percentage of inpatient stays decreased from 76.3% in 2010 to 65.7% in 2019 (Appendix Table 1; Figure 1, P < .001).

Different Growth Rates of Observation Stays for Various Pediatric Populations

As shown in the Table, growth rates of observation stays increased for 19 of the 20 most common APR-DRGs. The four APR-DRGs having the highest growth rates in observation stays were appendectomy, diabetes mellitus, kidney and urinary tract infections, and cellulitis and other bacterial skin infections (Appendix Figure). In particular, the annual percentage of observation stays for appendectomy increased from 19.8% in 2010 to 54.7% in 2019, with the number of observation stays growing from 2,321 to 7,876, while the number of inpatient stays decreased from 9,384 to 6,535 (Appendix Figure). The annual percentage of observation stays for diabetes mellitus increased from 8.16% in 2010 to 22.74% in 2019. Tonsil and adenoid procedures consistently held the largest numbers of observation stays across the 10 years among all the APR-DRGs, with 115,207 and 31,125 total observation and inpatient stays, respectively (Table).

Characteristics of Observation and Inpatient Stays

Patient characteristics are summarized in Appendix Table 1. There were 542,344 (32.9%) observation stays among patients with in-state Medicaid (managed care), and 241,157 (27.4%) observation stays among in-state Medicaid (CHIP/others). The percentages of observation and inpatient stays were 29.8% and 70.2% for private payor, as well as 29.6% and 70.4% for other government payor. Overall, the median (IQR) of H-RISK among observation stays was 0.79 (0.57-1.19) vs 1.23 (0.72-2.43) for inpatient stays. There were 1,410,694 (82.5%) observation stays discharged within 1 day and 243,972 (14.3%) observation stays discharged within 2 days. However, there were 47,413 (2.8%) and 7,049 (0.4%) observation stays with LOS 3 to 4 days or >4 days, respectively.

Shifting Pattern in Observation Stays

The annual percentage of observation stays with prolonged LOS (>2 days) rose from 1.1% in 2010 to 4.6% in 2019 (P < .001; Figure 2). The empirical distribution of H-RISK from observation stays by years further suggests a slightly increasing trend in intensity of care under observation stays. As shown in Appendix Table 2, although the 1st, 5th, 10th, 25th, and 99th percentiles of H-RISK were relatively stable, the 50th, 75th, 90th, and 95th percentiles of H-RISK were increasing over time. The correlation between year and intensity of care used under observation stays (H-RISK from observation stays) was found to be weak but significantly positive (Spearman correlation coefficients = 0.04; P < .001).

Interaction coefficients from our regression model demonstrate that the existing inverse association between H-RISK and odds of admission as an observation stay became less negative over the years. In 2010, the adjusted odds ratio (OR) of H-RISK was 0.57 (95% CI, 0.55-0.59). By 2017, the adjusted OR had increased to 0.65 (95% CI, 0.64-0.66). Compared with 2010, the seven adjusted ORs of H-RISK at years 2012 through 2018 were observed to be higher and statistically significant (P < .001, Appendix Table 3).

Hospitals-Level Use of Observation Stays

After adjusting for all covariates and hospital random effects, hospital-level use of observation stays increased between 2010 and 2019 for 26 out of 29 children’s hospitals. Although observation status essentially was not used at two children’s hospitals over the study period, the median hospital-level use of observation stays was 26% in 2010 (IQR, 3%-36%) and increased to 46% (IQR: 39%; 55%) in 2019. As shown in Figure 3, the number of hospitals with a low percentage of observation stays (<26%) decreased from 15 in 2010 to 4 in 2019. The number of hospitals with a high percentage of observation stays (≥51%) increased from 5 in 2010 to 10 in 2019. Nevertheless, there remained significant variation in the use of observation stays, and the hospital-level use ranged from 0% to 67% in 2019.

By 2020, observation status has become a key component of healthcare for pediatric patients, and its relevance for children’s hospitals recently has been described.^22,23 However, trends in observation stays for pediatric populations are not known. This represents the first study showing temporal trends of observation stays at children’s hospitals after 2010. Our results confirm that the increase in observation stays for pediatric populations is not attributable to decreasing patient acuity at children’s hospitals. We found a weak but significantly positive correlation between year and intensity of care used under observation stays. Although this correlation might not be clinically important, it demonstrates that patient acuity in observation stays is not decreasing. Regression results suggest that observation stays now encompass patients who need relatively higher intensity of care compared with those admitted under observation status in 2010.

This study also identifies a unique pattern in the use of observation stays among pediatric populations. Earlier studies exclusively focused on observation stays that were admitted from EDs.²⁴ Our results indicate that observation status has been used beyond a bridge from ED care to inpatient admission. In particular, observation status has expanded to include pediatric populations with more diverse clinical conditions (eg, appendicitis and diabetes mellitus), and has become a substantial component of postprocedural admissions (Appendix Figure). Looking forward, it is likely that the use of observation stays might surpass inpatient admissions for more conditions that primarily involve short-term stays.

Observation status originally was designed as a reimbursement strategy for patients who needed short stays in dedicated ED units or hospitals, but did not qualify for inpatient services.^5,25 After several changes in reimbursement policies, CMS released the “two midnight rule” for Medicare beneficiaries in 2013, which replaced condition-based criteria with time-based criteria to determine an inpatient or observation stay.¹ Some Medicaid programs and commercial payors have developed similar policies. Unlike the universal policy for Medicare populations, the regulations for pediatric populations vary by states and health insurers.^11-15,26-28 This might partially explain the wide variation observed among children’s hospital-level use of observation stays. For example, the California Medicaid program did not have a reimbursement rate for observation services as of 2020, while the Texas Medicaid program has had a policy for observation stays since 2010.^12,13 We found that two children’s hospitals in California had the lowest use of observation stays (almost zero), whereas the hospital-level use of observation stays was more than 50% for three out of four children’s hospitals in Texas. In addition to reimbursement policies, individual hospitals also might have different strategies for observation status designation. An earlier survey showed that there was lack of consistency in billing and payor-based designations of observation status at children’s hospitals.²⁹ These findings suggest that children’s hospital-level use of observation stays likely is influenced by reimbursement policy and practical strategy for observation status determination.

Earlier studies reported that observation status could be a more efficient use of healthcare resources.^5,6 However, there are still at least two concerns relevant to children’s hospitals during the last decade. The first is whether the use of observation stays can promote cost-saving or if it is just a cost-shifting strategy. An earlier study demonstrated that observation stays with prolonged LOS might increase risk of cost-sharing among adult patients.²⁹ Our study reveals an increasing trend of observation stays with prolonged LOS for pediatric patients. Similar to adult patients, LOS exceeding 24 or 48 hours could lead to uncovered healthcare costs and financial burdens on families.^30-32 Meanwhile, children’s hospitals also might take on a higher financial liability by implementing observation status. Earlier studies have indicated that resource use between observation and inpatient stays at children’s hospitals is similar, and increasing use of observation stays might lead to financial risk rather than cost effectiveness.³³ Further, administrative costs of observation determination are considerably high.³⁴ Medicaid is the major payor for pediatric patients in children’s hospitals. In this study, more than 50% of encounters were paid through Medicaid programs. It is well known that Medicaid reimbursement rates are lower than Medicare and commercial plans.³⁵ Therefore, the cost-saving conclusion drawn from Medicare patients cannot be generalized to pediatric populations at children’s hospitals without cautious reevaluation.

A second concern with increasing use of observation stays is selection bias in public reporting and comparisons of hospital performance. Presently, four main categories of quality indicators established by the Agency for Healthcare Research and Quality rely heavily on inpatient encounters.³⁶ In this study, we found that the range of hospital-level use of observation stays was large. In 2019, the risk-adjusted percent use of observation stays was less than 5% at three hospitals, while the percent use was greater than 60% in another three hospitals. Therefore, comparisons made without uniform accounting of observation stays might have significant implications for national rankings of children’s hospitals across the United States. These consequences have been investigated in several published studies.^22,23,37-39

There are several limitations to our study. First, the study sample was limited to children’s hospitals that consistently reported inpatient and observation data over the entire study period. Eighteen hospitals (86%) excluded from this study did not consistently submit inpatient and observation data to PHIS from 2010 through 2019. The primary purpose of this study was to present temporal trends of observation stays for children’s hospitals, and it was important to build the hospital cohort based on valid and consistent data during the study period. Appendix Table 4 presents differences of hospital characteristics by included and excluded groups of hospitals. Excluded hospitals might have fewer resources (eg, fewer pediatric intensive care beds). Nonetheless, the selection of hospitals was optimized based on data availability. Second, this study was a retrospective review of an administrative database of children’s hospitals and units. The sample does not represent all children’s hospitals or pediatric patients in the United States, but there are no available data sources—that we know of—that can generate national estimates for both inpatient and observation stays. Third, we did not attempt to conclusively infer any causal effects, and several factors could explain the increasing trends, such as reimbursement policies, hospital-level implementation strategies, determination guidelines for observation status designation, as well as changes in clinical care. Further studies should investigate impact of these factors on the use of observation stays for pediatric patients and children’s hospitals.

Observation status has been increasingly used for pediatric patients with more diverse clinical conditions, and there is a rising trend of prolonged LOS among observation stays since 2010. Considerable variation exists in hospital-level use of observation stays across children’s hospitals. Observation status could be an opportunity to improve efficiency of healthcare resource use or could lead to a financial risk for patients with prolonged LOS. Future studies should explore appropriateness of observation care in clinical practice through leveraging efficient care and alleviating financial risk.

To benefit patients and the public health of their communities, children’s hospitals across the United States prepared for and responded to COVID-19 by conserving personal protective equipment, suspending noncritical in-person healthcare encounters (including outpatient visits and elective surgeries), and implementing socially distanced essential care.^1,2 These measures were promptly instituted during a time of both substantial uncertainty about the pandemic’s behavior in children—including its severity and duration—and extreme variation in local and state governments’ responses to the pandemic.

Congruent with other healthcare institutions, children’s hospitals calibrated their clinical operations to the evolving nature of the pandemic, prioritizing the safety of patients and staff while striving to maintain financial viability in the setting of increased costs and decreased revenue. In some cases, children’s hospitals aided adult hospitals and health systems by admitting young and middle-aged adult patients and by centralizing all pediatric patients requiring intensive care within a region. These efforts occurred while many children’s hospitals remained the sole source of specialized pediatric care, including care for rare complex health problems.

As the COVID-19 pandemic continues, there is a critical need to assess how the initial phase of the pandemic affected healthcare encounters and related finances in children’s hospitals. Understanding these trends will position children’s hospitals to project and prepare for subsequent COVID-19 surges, as well as future related public health crises that necessitate widespread social distancing. Therefore, we compared year-over-year trends in healthcare encounters and hospital charges across US children’s hospitals before and during the COVID-19 pandemic, focusing on the beginning of COVID-19 in the United States, which was defined as February through June 2020.

This is a retrospective analysis of 26 children’s hospitals (22 freestanding, 4 nonfreestanding) from all US regions (12 South, 7 Midwest, 5 West, 2 Northeast) contributing encounter and financial data to the PROSPECT database (Children’s Hospital Association, Lenexa, Kansas) from February 1 to June 30 in both 2019 (before COVID-19) and 2020 (during COVID-19). In response to COVID-19, hospitals participating in PROSPECT increased the efficiency of data centralization and reporting in 2020 during the period February 1 to June 30 to expedite analysis and dissemination of findings.

The main outcome measures were the percentage of change in weekly encounters (inpatient bed-days, emergency department [ED] visits, and surgeries) and inflation-adjusted charges (categorized as inpatient care and outpatient care, such as ambulatory surgery, clinics, and ED visits) before vs during COVID-19. Number of encounters and charges were compared using the Wilcoxon signed-rank test. The distribution of weekly change in outcome measures from 2019 vs 2020 across hospitals was reported with medians and interquartile ranges (IQRs). The threshold of statistical significance was set at P < .05. All analyses were performed with SAS version 9.4 (SAS Institute). This study was considered exempt from human subjects research by the Institutional Review Board of Children’s Mercy Hospital (Kansas City, Missouri).

All 26 children’s hospitals experienced similar trends in healthcare encounters and charges during the study period (Figure and Table). From February 1 to March 10, 2020, the volume of healthcare encounters in the children’s hospitals remained the same as that for the same period in 2019 (P > .1) (Figure).

Charges that accrued from February 1 to June 30 were lower in 2020 by a median 23.6% (IQR, –28.7% to –19.1%) per children’s hospital than they were in 2019, corresponding to a median decrease of $276.3 million (IQR, $404.0-$126.0 million) in charges per hospital (Table). Forty percent of this decrease was attributable to decreased charges resulting from fewer inpatient healthcare encounters.

During the initial phase of the COVID-19 pandemic in the United States, children’s hospitals experienced a substantial decrease in healthcare encounters and charges. Greater decreases were observed for ED visits and surgery encounters than for inpatient bed-days. Nonetheless, inpatient bed-days decreased by more than one-third, consistent with the decrease in inpatient resource use reported for adult hospitals.³ Remarkably, these trends were consistent across children’s hospitals, despite variation in the content and installation of and adherence with social distancing policies in their surrounding local areas.

These findings beg the question of how well children’s hospitals are positioned to weather a recurrent surge in COVID-19. Because the severity of illness of COVID-19 has been lower to date in the pediatric vs adult populations, an increase in COVID-19-related visits to EDs and admissions to offset the decreased resource use of other pediatric healthcare problems is not anticipated. Existing hospital financial reserves as well as federal aid from the Coronavirus Aid, Relief, and Economic Security Act that helped mitigate the initial encounter and financial losses during the beginning of COVID-19 may not be readily available over time.^4,5 Certainly, the findings from the current study support continued lobbying for additional state and federal funds allocated through future relief packages to children’s hospitals.

Additional approaches to financial solvency in children’s hospitals during the sustained COVID-19 pandemic include addressing surgical backlogs and sharing best practices for safe and sustained reopening of clinical operations and financial practices across institutions. Although the PROSPECT database does not contain information on the types of surgeries present within this backlog, our experiences suggest that both same-day and inpatient elective surgeries have been affected, especially lengthy procedures (eg, spinal fusion for neuromuscular scoliosis). Spread and scale of feasible and efficient solutions to reengineer and expand patient capacities and throughput for operating rooms, postanesthesia recovery areas, and intensive care and floor units are needed. Enhanced analytics that accurately predict postoperative length of hospital stay, coupled with early recovery after surgery clinical protocols, could help optimize hospital bed management. Effective ways to convert hospital rooms from single to double occupancy, to manage family visitation, and to proactively test asymptomatic patients, family, and hospital staff will mitigate continued COVID-19 penetration through children’s hospitals.

One important limitation of the current study is the measurement of hospitals’ charges. The charge data were not positioned to comprehensively measure each hospital’s financial state during the COVID-19 pandemic. However, the decrease in hospital charges reported by the children’s hospitals in the current study is comparable with the financial losses reported for many adult hospitals during the pandemic.^6,7 It is important to recognize that the amount of the charges may not be equivalent to the cost of care or revenue collected by the hospitals. PROSPECT does not contain information on cost, and current cost-to-charge ratios are based on historical (ie, pre-COVID-19) data; therefore, they do not account for increased cost of personal protective equipment and other related costs that occurred during the pandemic, which makes use of these ratios challenging. Nevertheless, it is possible that the relative difference in costs incurred and revenue collected before and during COVID-19 may have been similar to the differences observed in hospital charges.

Children’s hospitals’ ability to serve the nation’s pediatric patients depends on the success of the hospitals’ plans to manage current and future COVID-19 surges and to reopen and recover from the surges that have passed. Additional investigation is needed to identify best operational and financial practices among children’s hospitals that have enabled them to endure the COVID-19 pandemic.

To benefit patients and the public health of their communities, children’s hospitals across the United States prepared for and responded to COVID-19 by conserving personal protective equipment, suspending noncritical in-person healthcare encounters (including outpatient visits and elective surgeries), and implementing socially distanced essential care.^1,2 These measures were promptly instituted during a time of both substantial uncertainty about the pandemic’s behavior in children—including its severity and duration—and extreme variation in local and state governments’ responses to the pandemic.

Congruent with other healthcare institutions, children’s hospitals calibrated their clinical operations to the evolving nature of the pandemic, prioritizing the safety of patients and staff while striving to maintain financial viability in the setting of increased costs and decreased revenue. In some cases, children’s hospitals aided adult hospitals and health systems by admitting young and middle-aged adult patients and by centralizing all pediatric patients requiring intensive care within a region. These efforts occurred while many children’s hospitals remained the sole source of specialized pediatric care, including care for rare complex health problems.

As the COVID-19 pandemic continues, there is a critical need to assess how the initial phase of the pandemic affected healthcare encounters and related finances in children’s hospitals. Understanding these trends will position children’s hospitals to project and prepare for subsequent COVID-19 surges, as well as future related public health crises that necessitate widespread social distancing. Therefore, we compared year-over-year trends in healthcare encounters and hospital charges across US children’s hospitals before and during the COVID-19 pandemic, focusing on the beginning of COVID-19 in the United States, which was defined as February through June 2020.

This is a retrospective analysis of 26 children’s hospitals (22 freestanding, 4 nonfreestanding) from all US regions (12 South, 7 Midwest, 5 West, 2 Northeast) contributing encounter and financial data to the PROSPECT database (Children’s Hospital Association, Lenexa, Kansas) from February 1 to June 30 in both 2019 (before COVID-19) and 2020 (during COVID-19). In response to COVID-19, hospitals participating in PROSPECT increased the efficiency of data centralization and reporting in 2020 during the period February 1 to June 30 to expedite analysis and dissemination of findings.

The main outcome measures were the percentage of change in weekly encounters (inpatient bed-days, emergency department [ED] visits, and surgeries) and inflation-adjusted charges (categorized as inpatient care and outpatient care, such as ambulatory surgery, clinics, and ED visits) before vs during COVID-19. Number of encounters and charges were compared using the Wilcoxon signed-rank test. The distribution of weekly change in outcome measures from 2019 vs 2020 across hospitals was reported with medians and interquartile ranges (IQRs). The threshold of statistical significance was set at P < .05. All analyses were performed with SAS version 9.4 (SAS Institute). This study was considered exempt from human subjects research by the Institutional Review Board of Children’s Mercy Hospital (Kansas City, Missouri).

All 26 children’s hospitals experienced similar trends in healthcare encounters and charges during the study period (Figure and Table). From February 1 to March 10, 2020, the volume of healthcare encounters in the children’s hospitals remained the same as that for the same period in 2019 (P > .1) (Figure).

Charges that accrued from February 1 to June 30 were lower in 2020 by a median 23.6% (IQR, –28.7% to –19.1%) per children’s hospital than they were in 2019, corresponding to a median decrease of $276.3 million (IQR, $404.0-$126.0 million) in charges per hospital (Table). Forty percent of this decrease was attributable to decreased charges resulting from fewer inpatient healthcare encounters.

During the initial phase of the COVID-19 pandemic in the United States, children’s hospitals experienced a substantial decrease in healthcare encounters and charges. Greater decreases were observed for ED visits and surgery encounters than for inpatient bed-days. Nonetheless, inpatient bed-days decreased by more than one-third, consistent with the decrease in inpatient resource use reported for adult hospitals.³ Remarkably, these trends were consistent across children’s hospitals, despite variation in the content and installation of and adherence with social distancing policies in their surrounding local areas.

These findings beg the question of how well children’s hospitals are positioned to weather a recurrent surge in COVID-19. Because the severity of illness of COVID-19 has been lower to date in the pediatric vs adult populations, an increase in COVID-19-related visits to EDs and admissions to offset the decreased resource use of other pediatric healthcare problems is not anticipated. Existing hospital financial reserves as well as federal aid from the Coronavirus Aid, Relief, and Economic Security Act that helped mitigate the initial encounter and financial losses during the beginning of COVID-19 may not be readily available over time.^4,5 Certainly, the findings from the current study support continued lobbying for additional state and federal funds allocated through future relief packages to children’s hospitals.

Additional approaches to financial solvency in children’s hospitals during the sustained COVID-19 pandemic include addressing surgical backlogs and sharing best practices for safe and sustained reopening of clinical operations and financial practices across institutions. Although the PROSPECT database does not contain information on the types of surgeries present within this backlog, our experiences suggest that both same-day and inpatient elective surgeries have been affected, especially lengthy procedures (eg, spinal fusion for neuromuscular scoliosis). Spread and scale of feasible and efficient solutions to reengineer and expand patient capacities and throughput for operating rooms, postanesthesia recovery areas, and intensive care and floor units are needed. Enhanced analytics that accurately predict postoperative length of hospital stay, coupled with early recovery after surgery clinical protocols, could help optimize hospital bed management. Effective ways to convert hospital rooms from single to double occupancy, to manage family visitation, and to proactively test asymptomatic patients, family, and hospital staff will mitigate continued COVID-19 penetration through children’s hospitals.

One important limitation of the current study is the measurement of hospitals’ charges. The charge data were not positioned to comprehensively measure each hospital’s financial state during the COVID-19 pandemic. However, the decrease in hospital charges reported by the children’s hospitals in the current study is comparable with the financial losses reported for many adult hospitals during the pandemic.^6,7 It is important to recognize that the amount of the charges may not be equivalent to the cost of care or revenue collected by the hospitals. PROSPECT does not contain information on cost, and current cost-to-charge ratios are based on historical (ie, pre-COVID-19) data; therefore, they do not account for increased cost of personal protective equipment and other related costs that occurred during the pandemic, which makes use of these ratios challenging. Nevertheless, it is possible that the relative difference in costs incurred and revenue collected before and during COVID-19 may have been similar to the differences observed in hospital charges.

Children’s hospitals’ ability to serve the nation’s pediatric patients depends on the success of the hospitals’ plans to manage current and future COVID-19 surges and to reopen and recover from the surges that have passed. Additional investigation is needed to identify best operational and financial practices among children’s hospitals that have enabled them to endure the COVID-19 pandemic.

To benefit patients and the public health of their communities, children’s hospitals across the United States prepared for and responded to COVID-19 by conserving personal protective equipment, suspending noncritical in-person healthcare encounters (including outpatient visits and elective surgeries), and implementing socially distanced essential care.^1,2 These measures were promptly instituted during a time of both substantial uncertainty about the pandemic’s behavior in children—including its severity and duration—and extreme variation in local and state governments’ responses to the pandemic.

Congruent with other healthcare institutions, children’s hospitals calibrated their clinical operations to the evolving nature of the pandemic, prioritizing the safety of patients and staff while striving to maintain financial viability in the setting of increased costs and decreased revenue. In some cases, children’s hospitals aided adult hospitals and health systems by admitting young and middle-aged adult patients and by centralizing all pediatric patients requiring intensive care within a region. These efforts occurred while many children’s hospitals remained the sole source of specialized pediatric care, including care for rare complex health problems.

As the COVID-19 pandemic continues, there is a critical need to assess how the initial phase of the pandemic affected healthcare encounters and related finances in children’s hospitals. Understanding these trends will position children’s hospitals to project and prepare for subsequent COVID-19 surges, as well as future related public health crises that necessitate widespread social distancing. Therefore, we compared year-over-year trends in healthcare encounters and hospital charges across US children’s hospitals before and during the COVID-19 pandemic, focusing on the beginning of COVID-19 in the United States, which was defined as February through June 2020.

This is a retrospective analysis of 26 children’s hospitals (22 freestanding, 4 nonfreestanding) from all US regions (12 South, 7 Midwest, 5 West, 2 Northeast) contributing encounter and financial data to the PROSPECT database (Children’s Hospital Association, Lenexa, Kansas) from February 1 to June 30 in both 2019 (before COVID-19) and 2020 (during COVID-19). In response to COVID-19, hospitals participating in PROSPECT increased the efficiency of data centralization and reporting in 2020 during the period February 1 to June 30 to expedite analysis and dissemination of findings.

The main outcome measures were the percentage of change in weekly encounters (inpatient bed-days, emergency department [ED] visits, and surgeries) and inflation-adjusted charges (categorized as inpatient care and outpatient care, such as ambulatory surgery, clinics, and ED visits) before vs during COVID-19. Number of encounters and charges were compared using the Wilcoxon signed-rank test. The distribution of weekly change in outcome measures from 2019 vs 2020 across hospitals was reported with medians and interquartile ranges (IQRs). The threshold of statistical significance was set at P < .05. All analyses were performed with SAS version 9.4 (SAS Institute). This study was considered exempt from human subjects research by the Institutional Review Board of Children’s Mercy Hospital (Kansas City, Missouri).

All 26 children’s hospitals experienced similar trends in healthcare encounters and charges during the study period (Figure and Table). From February 1 to March 10, 2020, the volume of healthcare encounters in the children’s hospitals remained the same as that for the same period in 2019 (P > .1) (Figure).

Charges that accrued from February 1 to June 30 were lower in 2020 by a median 23.6% (IQR, –28.7% to –19.1%) per children’s hospital than they were in 2019, corresponding to a median decrease of $276.3 million (IQR, $404.0-$126.0 million) in charges per hospital (Table). Forty percent of this decrease was attributable to decreased charges resulting from fewer inpatient healthcare encounters.

During the initial phase of the COVID-19 pandemic in the United States, children’s hospitals experienced a substantial decrease in healthcare encounters and charges. Greater decreases were observed for ED visits and surgery encounters than for inpatient bed-days. Nonetheless, inpatient bed-days decreased by more than one-third, consistent with the decrease in inpatient resource use reported for adult hospitals.³ Remarkably, these trends were consistent across children’s hospitals, despite variation in the content and installation of and adherence with social distancing policies in their surrounding local areas.

These findings beg the question of how well children’s hospitals are positioned to weather a recurrent surge in COVID-19. Because the severity of illness of COVID-19 has been lower to date in the pediatric vs adult populations, an increase in COVID-19-related visits to EDs and admissions to offset the decreased resource use of other pediatric healthcare problems is not anticipated. Existing hospital financial reserves as well as federal aid from the Coronavirus Aid, Relief, and Economic Security Act that helped mitigate the initial encounter and financial losses during the beginning of COVID-19 may not be readily available over time.^4,5 Certainly, the findings from the current study support continued lobbying for additional state and federal funds allocated through future relief packages to children’s hospitals.

Additional approaches to financial solvency in children’s hospitals during the sustained COVID-19 pandemic include addressing surgical backlogs and sharing best practices for safe and sustained reopening of clinical operations and financial practices across institutions. Although the PROSPECT database does not contain information on the types of surgeries present within this backlog, our experiences suggest that both same-day and inpatient elective surgeries have been affected, especially lengthy procedures (eg, spinal fusion for neuromuscular scoliosis). Spread and scale of feasible and efficient solutions to reengineer and expand patient capacities and throughput for operating rooms, postanesthesia recovery areas, and intensive care and floor units are needed. Enhanced analytics that accurately predict postoperative length of hospital stay, coupled with early recovery after surgery clinical protocols, could help optimize hospital bed management. Effective ways to convert hospital rooms from single to double occupancy, to manage family visitation, and to proactively test asymptomatic patients, family, and hospital staff will mitigate continued COVID-19 penetration through children’s hospitals.

One important limitation of the current study is the measurement of hospitals’ charges. The charge data were not positioned to comprehensively measure each hospital’s financial state during the COVID-19 pandemic. However, the decrease in hospital charges reported by the children’s hospitals in the current study is comparable with the financial losses reported for many adult hospitals during the pandemic.^6,7 It is important to recognize that the amount of the charges may not be equivalent to the cost of care or revenue collected by the hospitals. PROSPECT does not contain information on cost, and current cost-to-charge ratios are based on historical (ie, pre-COVID-19) data; therefore, they do not account for increased cost of personal protective equipment and other related costs that occurred during the pandemic, which makes use of these ratios challenging. Nevertheless, it is possible that the relative difference in costs incurred and revenue collected before and during COVID-19 may have been similar to the differences observed in hospital charges.

Children’s hospitals’ ability to serve the nation’s pediatric patients depends on the success of the hospitals’ plans to manage current and future COVID-19 surges and to reopen and recover from the surges that have passed. Additional investigation is needed to identify best operational and financial practices among children’s hospitals that have enabled them to endure the COVID-19 pandemic.

Hospitalizations for infections are common in children, with respiratory illnesses, including pneumonia and bronchiolitis, among the most prevalent indications for hospitalization.^1,2 Infections are also among the most frequent indications for all-cause readmissions and for potentially preventable readmissions in children.^3-5 Beyond hospital resource use, infection hospitalizations and readmissions represent a considerable cause of life disruption for patients and their families.^6,7 While emerging evidence supports shortened durations of parenteral antibiotics before transitioning to oral therapy for some infections (eg, pyelonephritis, osteomyelitis),^8-10 other infections may require extended treatment courses for weeks. The risk of adverse outcomes (eg, complications of medical treatment, readmission risk) and burdens placed on patients and their families may therefore differ across infection types and extend well beyond the immediate hospitalization.

Although infections are common and pediatric providers are expected to have proficiency in managing infections, substantial variation in the management of common pediatric infections exists and is associated with adverse hospitalization outcomes, including increased readmission risk and healthcare costs.^11-18 Potentially avoidable resource use associated with hospital readmission from infection has led to adoption of hospital-level readmission metrics as indicators of the quality of healthcare delivery. For example, the Pediatric Quality Measures Program, established by the Children’s Health Insurance Program Reauthorization Act of 2009, has prioritized measurement of readmissions following hospitalization for lower respiratory tract infection.¹⁹ With government agencies increasingly using readmission metrics to assess quality of healthcare delivery, developing metrics that focus on these resource-intensive conditions is essential.

Because infections are a common and costly indication for hospital resource use and because substantial variation in management has been observed, promoting a broader understanding of infection-specific readmission rates is important for prioritizing readmission-reduction opportunities in children. This study’s objectives were the following: (1) to describe the prevalence and characteristics of infection hospitalizations in children and their associated readmissions and (2) to estimate the number of readmissions avoided and costs saved if all hospitals achieved the 10th percentile of the hospitals’ risk-adjusted readmission rate (ie, readmission benchmark).

Study Design and Data Source

We performed a retrospective cohort analysis using the 2014 Agency for Healthcare Research and Quality (AHRQ) Nationwide Readmissions Database (NRD).²⁰ The 2014 NRD is an administrative database that contains information on inpatient stays from January 1, 2014, to December 31, 2014, for all payers and allows for weighted national estimates of readmissions for all US individuals. Data within NRD are aggregated from 22 geographically diverse states representing approximately one-half of the US population. NRD contains deidentified patient-level data with unique verified patient identifiers to track individuals within and across hospitals in a state. However, AHRQ guidelines specify that NRD cannot be used for reporting hospital-specific readmission rates. Thus, for the current study, the Inpatient Essentials (Children’s Hospital Association), or IE, database was used to measure hospital-level readmission rates and to distinguish benchmark readmission rates for individual infection diagnoses.²¹ The IE database is composed of 90 children’s hospitals distributed throughout all regions of the United States. The inclusion of free-standing children’s hospitals and children’s hospitals within adult hospitals allows for comparisons and benchmarking across hospitals on multiple metrics, including readmissions.

Study Population

Children 0 to 17 years of age with a primary diagnosis at the index admission for infection between January 1, 2014, and November 30, 2014, were included. The end date of November 30, 2014, allowed for a full 30-day readmission window for all index admissions. We excluded index admissions that resulted in transfer to another acute care hospital or in-hospital mortality. Additionally, we excluded index admissions of children who had hematologic or immunologic conditions, malignancy, or history of bone marrow and solid-organ transplant, using the classification system for complex chronic conditions (CCCs) from Feudtner et al.²² Due to the high likelihood of immunosuppression in patients with these conditions, children may have nuanced experiences with illness severity, trajectory, and treatment associated with infection that place them at increased risk for nonpreventable readmission.

Main Exposure

The main exposure was infection type during the index admission. Condition-specific index admissions were identified using AHRQ’s Clinical Classifications Software (CCS) categories.²³ CCS is a classification schema that categorizes the greater than 14,000 International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes and 3,900 ICD-9-CM procedure codes into clinically meaningful categories of 295 diagnosis (including mental health codes and E-codes) and 231 procedural groupings. Twenty-two groupings indicative of infection were distinguished and used for the current study. Examples of infections included aspiration pneumonia, pneumonia, bronchiolitis, and sexually transmitted infection. We combined related CCS categories when possible for ease of interpretation and presentation of data (Appendix Table 1).

Main Outcome Measure

The main outcome measure was 30-day hospital readmission. Readmission was defined as all-cause, unplanned admission within 30 days following discharge from a preceding hospitalization. Planned hospital readmissions were identified and excluded using methods from AHRQ’s Pediatric All-Condition Readmission Measure.²⁴ We defined a same-cause return as a return with the same CCS infection category as the index admission. Costs associated with readmissions were estimated from charges using hospital-specific cost-to-charge ratios provided with NRD.

Patient Demographic and Clinical Characteristics

Patient demographic characteristics included age at index admission (<1 year, 1-5 years, 6-9 years, 10-14 years, and 15-18 years), sex, payer (ie, government, private, other), and discharge disposition (ie, routine, home health, other). We assessed all patients for medical complexity, as defined by the presence of at least one CCC, and we reported the categories of CCCs by organ system involved.²² Otherwise, patients were identified as without medical complexity.

Statistical Analysis

We summarized continuous variables with medians and interquartile ranges and categorical variables with frequencies and percentages. To develop benchmark readmission rates for each infection type, we used generalized linear mixed models with random intercepts for each hospital in the IE database. For each infection type, the benchmark readmission rate was defined as the 10th percentile of hospitals’ risk-adjusted readmission rates. The 10th percentile was chosen to identify the best performing 10% of hospitals (ie, hospitals with the lowest readmission rates). Because children with medical complexity account for a large proportion of hospital resource use and are at high risk for readmission,^4,25 we developed benchmarks stratified by presence/absence of a CCC (ie, with complexity vs without complexity). Models were adjusted for severity of illness using the Hospitalization Resource Intensity Score for Kids (H-RISK),²⁶ a scoring system that assigns relative weights for each All Patient Refined Diagnosis-Related Group (3M Corporation) and severity of illness level, and each hospital’s risk-adjusted readmission rate was determined.

With use of weights to achieve national estimates of index admissions and readmissions, we determined the number of potentially avoidable readmissions by calculating the number of readmissions observed in the NRD that would not occur if all hospitals achieved readmission rates equal to the 10th percentile. Avoidable costs were estimated by multiplying the number of potentially avoidable readmissions by the mean cost of a readmission for infections of that type. Estimates of avoidable readmissions and costs were stratified by medical complexity. In addition to describing estimates at the 10th percentile benchmark, we similarly developed estimates of potentially avoidable readmissions and avoidable costs for the 5th and 25th percentiles, which are presented within Appendix Table 2 (children without complexity) and Appendix Table 3 (children with complexity).

All statistical analyses were performed using SAS version 9.4 (SAS Institute), and P values <.001 were considered statistically significant due to the large sample size. The Office of Research Integrity at Children’s Mercy Hospital deemed this study exempt from institutional board review.

Characteristics of the Study Population

The study included 380,067 index admissions for infection and an accompanying 18,469 unplanned all-cause readmissions over the 30 days following discharge (readmission rate, 4.9%; Table 1). Children ages 1 to 5 years accounted for the largest percentage (32.9%) of index hospitalizations, followed by infants younger than 1 year (30.3%). The readmission rate by age group was highest for infants younger than 1 year, compared with rates among all other age groups (5.6% among infants < 1 year vs 4.4%-4.7% for other age groups; P < .001). In the overall cohort, 16.2% of admissions included patients with a CCC. Children with medical complexity had higher readmission rates than those without medical complexity (no CCC, 3.2%; 1 CCC, 9.2%; 2+ CCCs, 18.9%). A greater percentage of children experiencing a readmission had government insurance (63.0% vs 59.2%; P < .001) and received home health nursing (10.1% vs 2.7%; P < .001) following the index encounter.

Children Without Complexity

Index Admissions and 30-day Readmissions

Among patients without medical complexity, index admissions occurred most frequently for pneumonia (n = 54,717), bronchiolitis (n = 53,959), and appendicitis (n = 45,036) (Figure 1). The median length of stay (LOS) for index admissions ranged from 1 to 5 days (Table 2), with septic arthritis and osteomyelitis having the longest median LOS at 5 (IQR, 3-7) days.

Thirty-day readmission rates varied substantially by infection at the index admission (range, 1.5% for sexually transmitted infection to 8.6% for peritonitis) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 7 days (Table 2), while the median number of days to readmission varied substantially by infection type (range, 4 days for bacterial infection [site unspecified] to 24 days for sexually transmitted infections). Among the top five indications for admission for children without complexity, 14.9% to 52.8% of readmissions were for the same cause as the index admission; however, many additional returns were likely related to the index admission (Appendix Table 4). For example, among other return reasons, an additional 992 (61.7%) readmissions following appendicitis hospitalizations were for complications of surgical procedures or medical care, peritonitis, intestinal obstruction, and abdominal pain.

Impact of Achieving Readmission Benchmarks

Among children without complexity, readmission benchmarks (ie, the 10th percentile of readmission rates across hospitals) ranged from 0% to 26.7% (Figure 2). An estimated 54.7% of readmissions (n = 5,507) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $44.5 million in savings. Pneumonia, bronchiolitis, gastroenteritis, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if a 10th percentile benchmark was achieved.

Children With Medical Complexity

Index Admissions and 30-day Readmissions

Among patients with complexity, index admissions occurred most frequently for pneumonia (n = 14,344), bronchiolitis (n = 8,618), and upper respiratory tract infection (n = 6,407) (Figure 1). The median LOS for index admissions ranged from 1 to 9 days (Table 2), with septicemia and CNS infections having the longest median LOS at 9 days.

Thirty-day readmission rates varied substantially by the type of infection at the index admission (range, 0% for sexually transmitted infection to 21.6% for aspiration pneumonia) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 14 days (Table 2), and the median number of days to readmission varied substantially by infection type (range, 6 days for tonsillitis to 23 days for other infection). Among the top five indications for admission for medically complex children, 8% to 40.4% of readmissions were for the same cause as the index admission (Appendix Table 4). As with the children without complexity, additional returns were likely related to the index admission.

Impact of Achieving Readmission Benchmarks

Among children with medical complexity, readmission benchmarks ranged from 0% to 30.3% (Figure 2). An estimated 42.6% of readmissions (n = 3,576) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $70.8 million in savings. Pneumonia, bronchiolitis, septicemia, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if the benchmarks were achieved.

The current study uncovered new findings regarding unplanned readmissions following index infection hospitalizations for children. In particular, readmission rates and time to readmission varied substantially by infection subtype. The study also identified priority infections and unique considerations for children with CCCs, all of which may help maximize the value of readmission metrics aimed at advancing hospital quality and reducing costs for infection hospitalizations in children. If all hospitals achieved the readmission rates of the best performing hospitals, 42.6% to 54.7% fewer readmissions could be realized with associated cost savings.

Nationally, studies have reported 30-day, all-cause unplanned readmission rates of 6.2% to 10.3%.^5,27 In our current study we observed an overall readmission rate of 4.9% across all infectious conditions; however, readmission rates varied substantially by condition, with upper and lower respiratory tract infections, septicemia, and gastroenteritis among infections with the greatest number of potentially avoidable readmissions based on achievement of readmission benchmarks. Currently, pediatric-specific all-cause and lower respiratory tract infection readmission metrics have been developed with the aim of improving quality of care and reducing healthcare expenditures.²⁸ Future readmission studies and metrics may prioritize conditions such as septicemia, gastroenteritis, and other respiratory tract infections. Our current study demonstrated that many readmissions following infection hospitalizations were associated with the same CCS category or within a related CCS category (eg, complications of surgical procedures or medical care, appendicitis, peritonitis, intestinal obstruction, and abdominal pain constituted the top five indications for readmission for appendicitis, whereas respiratory illnesses constituted the top five indications for readmissions for pneumonia). While this current study cannot clarify this relationship further, and the “avoidability” of unplanned readmissions is debated,^29-31 our findings suggest that future investigations might focus on identifying whether condition-specific interventions during the index admission could mitigate some readmissions.

While the LOS of the index admission and the readmission were similar for most infection subtypes, we observed substantial variability in the temporal risk for readmission by infection subtype. Our observations complement studies exploring the timing of readmissions for other pediatric conditions.^32-34 In particular, our findings further highlight that the composition and interaction of factors influencing infection readmissions may vary by condition. Infections represent a diverse group of conditions, with treatment courses that vary in need for parenteral antibiotics, ability to tailor treatment based on organism and susceptibilities, and length of treatment. While treatment for some infections may be accomplished, or nearly accomplished, prior to discharge, other infections (eg, osteomyelitis) may require prolonged treatment, shifting the burden of management and monitoring to patients and their families, which along with the timeliness and adequacy of outpatient follow-up, may modify an individual’s readmission risk. Consequently, a “one-size fits all” approach to discharge counseling may not be successful across all conditions. Our study lays the groundwork for how these temporal relationships may be used by clinicians to counsel families regarding the likely readmission timeframe, based on infection, and to establish follow-up appointments ahead of the infection-specific “readmission window,” which may allow outpatient providers to intervene when readmission risk is greatest.

Consistent with prior literature, children with medical complexity in our study had increased frequency of 30-day, all-cause unplanned readmissions and LOS, compared with peers who did not have complexity.⁵ Readmission rates following hospitalizations for aspiration pneumonia were comparable to those reported by Thompson et al in their study examining rates of pneumonia in children with neurologic impairment.³⁵ In our current study, nearly 96% of readmissions following aspiration pneumonia hospitalizations were for children with medical complexity, and more than 58% of these readmissions were for aspiration pneumonia or respiratory illness. Future investigations should seek to explore factors contributing to readmissions in children with medical complexity and to evaluate whether interventions such as postdischarge coaching or discharge bundles could assist with reductions in healthcare resource use.^36,37

Despite a lack of clear association between readmissions and quality of care for children,³⁸ readmissions rates continue to be used as a quality measure for hospitalized patients. Within our present study, we found that achieving benchmark readmission rates for infection hospitalizations could lead to substantial reductions in readmissions; however, these reductions were seen across this relatively similar group of infection diagnoses, and hospitals may face greater challenges when attempting to achieve a 10th percentile readmission benchmark across a more expansive set of diagnoses. Despite these challenges, understanding the intricacies of readmissions may lead to improved readmission metrics and the systematic identification of avoidable readmissions, the goal of which is to enhance the quality of healthcare for hospitalized children.

Our findings should be interpreted in the context of several limitations. We defined our readmission benchmark at the 10th percentile using the IE database. Because an estimated 70% of hospitalizations for children occur within general hospitals,³⁹ we believe that our use of the IE database allowed for increased generalizability, though the broadening of our findings to nonacademic hospital settings may be less reliable. While we describe the 10th percentile readmission benchmark here, alternative benchmarks would result in different estimates of avoidable readmissions and associated readmission costs (Appendix Tables 2 and 3). The IE and NRD databases do not distinguish intensive care use. We tried to address this limitation through model adjustments using H-RISK, which is particularly helpful for adjusting for NICU admissions through use of the 27 All Patient Refined Diagnosis-Related Groups for neonatal conditions. Additionally, the NRD uses state-level data to derive national estimates and is not equipped to measure readmissions to hospitals in a different state, distinguish patient deaths occurring after discharge, or to examine the specific indication for readmission (ie, unable to assess if the readmission is related to a complication of the treatment plan, progression of the illness course, or for an unrelated reason). While sociodemographic and socioeconomic factors may affect readmissions, the NRD does not contain information on patients’ race/ethnicity, family/social attributes, or postdischarge follow-up health services, which are known to influence the need for readmission.

Despite these limitations, this study highlights future areas for research and potential opportunities for reducing readmission following infection hospitalizations. First, identifying hospital- and systems-based factors that contribute to readmission reductions at the best-performing hospitals may identify opportunities for hospitals with the highest readmission rates to achieve the rates of the best-performing hospitals. Second, conditions such as upper and lower respiratory tract infections, along with septicemia and gastroenteritis, may serve as prime targets for future investigation based on the estimated number of avoidable readmissions and potential cost savings associated with these conditions. Finally, future investigations that explore discharge counseling and follow-up tailored to the infection-specific temporal risk and patient complexity may identify opportunities for further readmission reductions.

Our study provides a broad look at readmissions following infection hospitalizations and highlights substantial variation in readmissions based on infection type. To improve hospital resource use for infections, future preventive measures could prioritize children with complex chronic conditions and those with specific diagnoses (eg, upper and lower respiratory tract infections).

This information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by, NIH or the US government.

Hospitalizations for infections are common in children, with respiratory illnesses, including pneumonia and bronchiolitis, among the most prevalent indications for hospitalization.^1,2 Infections are also among the most frequent indications for all-cause readmissions and for potentially preventable readmissions in children.^3-5 Beyond hospital resource use, infection hospitalizations and readmissions represent a considerable cause of life disruption for patients and their families.^6,7 While emerging evidence supports shortened durations of parenteral antibiotics before transitioning to oral therapy for some infections (eg, pyelonephritis, osteomyelitis),^8-10 other infections may require extended treatment courses for weeks. The risk of adverse outcomes (eg, complications of medical treatment, readmission risk) and burdens placed on patients and their families may therefore differ across infection types and extend well beyond the immediate hospitalization.

Although infections are common and pediatric providers are expected to have proficiency in managing infections, substantial variation in the management of common pediatric infections exists and is associated with adverse hospitalization outcomes, including increased readmission risk and healthcare costs.^11-18 Potentially avoidable resource use associated with hospital readmission from infection has led to adoption of hospital-level readmission metrics as indicators of the quality of healthcare delivery. For example, the Pediatric Quality Measures Program, established by the Children’s Health Insurance Program Reauthorization Act of 2009, has prioritized measurement of readmissions following hospitalization for lower respiratory tract infection.¹⁹ With government agencies increasingly using readmission metrics to assess quality of healthcare delivery, developing metrics that focus on these resource-intensive conditions is essential.

Because infections are a common and costly indication for hospital resource use and because substantial variation in management has been observed, promoting a broader understanding of infection-specific readmission rates is important for prioritizing readmission-reduction opportunities in children. This study’s objectives were the following: (1) to describe the prevalence and characteristics of infection hospitalizations in children and their associated readmissions and (2) to estimate the number of readmissions avoided and costs saved if all hospitals achieved the 10th percentile of the hospitals’ risk-adjusted readmission rate (ie, readmission benchmark).

Study Design and Data Source

We performed a retrospective cohort analysis using the 2014 Agency for Healthcare Research and Quality (AHRQ) Nationwide Readmissions Database (NRD).²⁰ The 2014 NRD is an administrative database that contains information on inpatient stays from January 1, 2014, to December 31, 2014, for all payers and allows for weighted national estimates of readmissions for all US individuals. Data within NRD are aggregated from 22 geographically diverse states representing approximately one-half of the US population. NRD contains deidentified patient-level data with unique verified patient identifiers to track individuals within and across hospitals in a state. However, AHRQ guidelines specify that NRD cannot be used for reporting hospital-specific readmission rates. Thus, for the current study, the Inpatient Essentials (Children’s Hospital Association), or IE, database was used to measure hospital-level readmission rates and to distinguish benchmark readmission rates for individual infection diagnoses.²¹ The IE database is composed of 90 children’s hospitals distributed throughout all regions of the United States. The inclusion of free-standing children’s hospitals and children’s hospitals within adult hospitals allows for comparisons and benchmarking across hospitals on multiple metrics, including readmissions.

Study Population

Children 0 to 17 years of age with a primary diagnosis at the index admission for infection between January 1, 2014, and November 30, 2014, were included. The end date of November 30, 2014, allowed for a full 30-day readmission window for all index admissions. We excluded index admissions that resulted in transfer to another acute care hospital or in-hospital mortality. Additionally, we excluded index admissions of children who had hematologic or immunologic conditions, malignancy, or history of bone marrow and solid-organ transplant, using the classification system for complex chronic conditions (CCCs) from Feudtner et al.²² Due to the high likelihood of immunosuppression in patients with these conditions, children may have nuanced experiences with illness severity, trajectory, and treatment associated with infection that place them at increased risk for nonpreventable readmission.

Main Exposure

The main exposure was infection type during the index admission. Condition-specific index admissions were identified using AHRQ’s Clinical Classifications Software (CCS) categories.²³ CCS is a classification schema that categorizes the greater than 14,000 International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes and 3,900 ICD-9-CM procedure codes into clinically meaningful categories of 295 diagnosis (including mental health codes and E-codes) and 231 procedural groupings. Twenty-two groupings indicative of infection were distinguished and used for the current study. Examples of infections included aspiration pneumonia, pneumonia, bronchiolitis, and sexually transmitted infection. We combined related CCS categories when possible for ease of interpretation and presentation of data (Appendix Table 1).

Main Outcome Measure

The main outcome measure was 30-day hospital readmission. Readmission was defined as all-cause, unplanned admission within 30 days following discharge from a preceding hospitalization. Planned hospital readmissions were identified and excluded using methods from AHRQ’s Pediatric All-Condition Readmission Measure.²⁴ We defined a same-cause return as a return with the same CCS infection category as the index admission. Costs associated with readmissions were estimated from charges using hospital-specific cost-to-charge ratios provided with NRD.

Patient Demographic and Clinical Characteristics

Patient demographic characteristics included age at index admission (<1 year, 1-5 years, 6-9 years, 10-14 years, and 15-18 years), sex, payer (ie, government, private, other), and discharge disposition (ie, routine, home health, other). We assessed all patients for medical complexity, as defined by the presence of at least one CCC, and we reported the categories of CCCs by organ system involved.²² Otherwise, patients were identified as without medical complexity.

Statistical Analysis

We summarized continuous variables with medians and interquartile ranges and categorical variables with frequencies and percentages. To develop benchmark readmission rates for each infection type, we used generalized linear mixed models with random intercepts for each hospital in the IE database. For each infection type, the benchmark readmission rate was defined as the 10th percentile of hospitals’ risk-adjusted readmission rates. The 10th percentile was chosen to identify the best performing 10% of hospitals (ie, hospitals with the lowest readmission rates). Because children with medical complexity account for a large proportion of hospital resource use and are at high risk for readmission,^4,25 we developed benchmarks stratified by presence/absence of a CCC (ie, with complexity vs without complexity). Models were adjusted for severity of illness using the Hospitalization Resource Intensity Score for Kids (H-RISK),²⁶ a scoring system that assigns relative weights for each All Patient Refined Diagnosis-Related Group (3M Corporation) and severity of illness level, and each hospital’s risk-adjusted readmission rate was determined.

With use of weights to achieve national estimates of index admissions and readmissions, we determined the number of potentially avoidable readmissions by calculating the number of readmissions observed in the NRD that would not occur if all hospitals achieved readmission rates equal to the 10th percentile. Avoidable costs were estimated by multiplying the number of potentially avoidable readmissions by the mean cost of a readmission for infections of that type. Estimates of avoidable readmissions and costs were stratified by medical complexity. In addition to describing estimates at the 10th percentile benchmark, we similarly developed estimates of potentially avoidable readmissions and avoidable costs for the 5th and 25th percentiles, which are presented within Appendix Table 2 (children without complexity) and Appendix Table 3 (children with complexity).

All statistical analyses were performed using SAS version 9.4 (SAS Institute), and P values <.001 were considered statistically significant due to the large sample size. The Office of Research Integrity at Children’s Mercy Hospital deemed this study exempt from institutional board review.

Characteristics of the Study Population

The study included 380,067 index admissions for infection and an accompanying 18,469 unplanned all-cause readmissions over the 30 days following discharge (readmission rate, 4.9%; Table 1). Children ages 1 to 5 years accounted for the largest percentage (32.9%) of index hospitalizations, followed by infants younger than 1 year (30.3%). The readmission rate by age group was highest for infants younger than 1 year, compared with rates among all other age groups (5.6% among infants < 1 year vs 4.4%-4.7% for other age groups; P < .001). In the overall cohort, 16.2% of admissions included patients with a CCC. Children with medical complexity had higher readmission rates than those without medical complexity (no CCC, 3.2%; 1 CCC, 9.2%; 2+ CCCs, 18.9%). A greater percentage of children experiencing a readmission had government insurance (63.0% vs 59.2%; P < .001) and received home health nursing (10.1% vs 2.7%; P < .001) following the index encounter.

Children Without Complexity

Index Admissions and 30-day Readmissions

Among patients without medical complexity, index admissions occurred most frequently for pneumonia (n = 54,717), bronchiolitis (n = 53,959), and appendicitis (n = 45,036) (Figure 1). The median length of stay (LOS) for index admissions ranged from 1 to 5 days (Table 2), with septic arthritis and osteomyelitis having the longest median LOS at 5 (IQR, 3-7) days.

Thirty-day readmission rates varied substantially by infection at the index admission (range, 1.5% for sexually transmitted infection to 8.6% for peritonitis) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 7 days (Table 2), while the median number of days to readmission varied substantially by infection type (range, 4 days for bacterial infection [site unspecified] to 24 days for sexually transmitted infections). Among the top five indications for admission for children without complexity, 14.9% to 52.8% of readmissions were for the same cause as the index admission; however, many additional returns were likely related to the index admission (Appendix Table 4). For example, among other return reasons, an additional 992 (61.7%) readmissions following appendicitis hospitalizations were for complications of surgical procedures or medical care, peritonitis, intestinal obstruction, and abdominal pain.

Impact of Achieving Readmission Benchmarks

Among children without complexity, readmission benchmarks (ie, the 10th percentile of readmission rates across hospitals) ranged from 0% to 26.7% (Figure 2). An estimated 54.7% of readmissions (n = 5,507) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $44.5 million in savings. Pneumonia, bronchiolitis, gastroenteritis, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if a 10th percentile benchmark was achieved.

Children With Medical Complexity

Index Admissions and 30-day Readmissions

Among patients with complexity, index admissions occurred most frequently for pneumonia (n = 14,344), bronchiolitis (n = 8,618), and upper respiratory tract infection (n = 6,407) (Figure 1). The median LOS for index admissions ranged from 1 to 9 days (Table 2), with septicemia and CNS infections having the longest median LOS at 9 days.

Thirty-day readmission rates varied substantially by the type of infection at the index admission (range, 0% for sexually transmitted infection to 21.6% for aspiration pneumonia) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 14 days (Table 2), and the median number of days to readmission varied substantially by infection type (range, 6 days for tonsillitis to 23 days for other infection). Among the top five indications for admission for medically complex children, 8% to 40.4% of readmissions were for the same cause as the index admission (Appendix Table 4). As with the children without complexity, additional returns were likely related to the index admission.

Impact of Achieving Readmission Benchmarks

Among children with medical complexity, readmission benchmarks ranged from 0% to 30.3% (Figure 2). An estimated 42.6% of readmissions (n = 3,576) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $70.8 million in savings. Pneumonia, bronchiolitis, septicemia, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if the benchmarks were achieved.

The current study uncovered new findings regarding unplanned readmissions following index infection hospitalizations for children. In particular, readmission rates and time to readmission varied substantially by infection subtype. The study also identified priority infections and unique considerations for children with CCCs, all of which may help maximize the value of readmission metrics aimed at advancing hospital quality and reducing costs for infection hospitalizations in children. If all hospitals achieved the readmission rates of the best performing hospitals, 42.6% to 54.7% fewer readmissions could be realized with associated cost savings.

Nationally, studies have reported 30-day, all-cause unplanned readmission rates of 6.2% to 10.3%.^5,27 In our current study we observed an overall readmission rate of 4.9% across all infectious conditions; however, readmission rates varied substantially by condition, with upper and lower respiratory tract infections, septicemia, and gastroenteritis among infections with the greatest number of potentially avoidable readmissions based on achievement of readmission benchmarks. Currently, pediatric-specific all-cause and lower respiratory tract infection readmission metrics have been developed with the aim of improving quality of care and reducing healthcare expenditures.²⁸ Future readmission studies and metrics may prioritize conditions such as septicemia, gastroenteritis, and other respiratory tract infections. Our current study demonstrated that many readmissions following infection hospitalizations were associated with the same CCS category or within a related CCS category (eg, complications of surgical procedures or medical care, appendicitis, peritonitis, intestinal obstruction, and abdominal pain constituted the top five indications for readmission for appendicitis, whereas respiratory illnesses constituted the top five indications for readmissions for pneumonia). While this current study cannot clarify this relationship further, and the “avoidability” of unplanned readmissions is debated,^29-31 our findings suggest that future investigations might focus on identifying whether condition-specific interventions during the index admission could mitigate some readmissions.

While the LOS of the index admission and the readmission were similar for most infection subtypes, we observed substantial variability in the temporal risk for readmission by infection subtype. Our observations complement studies exploring the timing of readmissions for other pediatric conditions.^32-34 In particular, our findings further highlight that the composition and interaction of factors influencing infection readmissions may vary by condition. Infections represent a diverse group of conditions, with treatment courses that vary in need for parenteral antibiotics, ability to tailor treatment based on organism and susceptibilities, and length of treatment. While treatment for some infections may be accomplished, or nearly accomplished, prior to discharge, other infections (eg, osteomyelitis) may require prolonged treatment, shifting the burden of management and monitoring to patients and their families, which along with the timeliness and adequacy of outpatient follow-up, may modify an individual’s readmission risk. Consequently, a “one-size fits all” approach to discharge counseling may not be successful across all conditions. Our study lays the groundwork for how these temporal relationships may be used by clinicians to counsel families regarding the likely readmission timeframe, based on infection, and to establish follow-up appointments ahead of the infection-specific “readmission window,” which may allow outpatient providers to intervene when readmission risk is greatest.

Consistent with prior literature, children with medical complexity in our study had increased frequency of 30-day, all-cause unplanned readmissions and LOS, compared with peers who did not have complexity.⁵ Readmission rates following hospitalizations for aspiration pneumonia were comparable to those reported by Thompson et al in their study examining rates of pneumonia in children with neurologic impairment.³⁵ In our current study, nearly 96% of readmissions following aspiration pneumonia hospitalizations were for children with medical complexity, and more than 58% of these readmissions were for aspiration pneumonia or respiratory illness. Future investigations should seek to explore factors contributing to readmissions in children with medical complexity and to evaluate whether interventions such as postdischarge coaching or discharge bundles could assist with reductions in healthcare resource use.^36,37

Despite a lack of clear association between readmissions and quality of care for children,³⁸ readmissions rates continue to be used as a quality measure for hospitalized patients. Within our present study, we found that achieving benchmark readmission rates for infection hospitalizations could lead to substantial reductions in readmissions; however, these reductions were seen across this relatively similar group of infection diagnoses, and hospitals may face greater challenges when attempting to achieve a 10th percentile readmission benchmark across a more expansive set of diagnoses. Despite these challenges, understanding the intricacies of readmissions may lead to improved readmission metrics and the systematic identification of avoidable readmissions, the goal of which is to enhance the quality of healthcare for hospitalized children.

Our findings should be interpreted in the context of several limitations. We defined our readmission benchmark at the 10th percentile using the IE database. Because an estimated 70% of hospitalizations for children occur within general hospitals,³⁹ we believe that our use of the IE database allowed for increased generalizability, though the broadening of our findings to nonacademic hospital settings may be less reliable. While we describe the 10th percentile readmission benchmark here, alternative benchmarks would result in different estimates of avoidable readmissions and associated readmission costs (Appendix Tables 2 and 3). The IE and NRD databases do not distinguish intensive care use. We tried to address this limitation through model adjustments using H-RISK, which is particularly helpful for adjusting for NICU admissions through use of the 27 All Patient Refined Diagnosis-Related Groups for neonatal conditions. Additionally, the NRD uses state-level data to derive national estimates and is not equipped to measure readmissions to hospitals in a different state, distinguish patient deaths occurring after discharge, or to examine the specific indication for readmission (ie, unable to assess if the readmission is related to a complication of the treatment plan, progression of the illness course, or for an unrelated reason). While sociodemographic and socioeconomic factors may affect readmissions, the NRD does not contain information on patients’ race/ethnicity, family/social attributes, or postdischarge follow-up health services, which are known to influence the need for readmission.

Despite these limitations, this study highlights future areas for research and potential opportunities for reducing readmission following infection hospitalizations. First, identifying hospital- and systems-based factors that contribute to readmission reductions at the best-performing hospitals may identify opportunities for hospitals with the highest readmission rates to achieve the rates of the best-performing hospitals. Second, conditions such as upper and lower respiratory tract infections, along with septicemia and gastroenteritis, may serve as prime targets for future investigation based on the estimated number of avoidable readmissions and potential cost savings associated with these conditions. Finally, future investigations that explore discharge counseling and follow-up tailored to the infection-specific temporal risk and patient complexity may identify opportunities for further readmission reductions.

Our study provides a broad look at readmissions following infection hospitalizations and highlights substantial variation in readmissions based on infection type. To improve hospital resource use for infections, future preventive measures could prioritize children with complex chronic conditions and those with specific diagnoses (eg, upper and lower respiratory tract infections).

This information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by, NIH or the US government.

Hospitalizations for infections are common in children, with respiratory illnesses, including pneumonia and bronchiolitis, among the most prevalent indications for hospitalization.^1,2 Infections are also among the most frequent indications for all-cause readmissions and for potentially preventable readmissions in children.^3-5 Beyond hospital resource use, infection hospitalizations and readmissions represent a considerable cause of life disruption for patients and their families.^6,7 While emerging evidence supports shortened durations of parenteral antibiotics before transitioning to oral therapy for some infections (eg, pyelonephritis, osteomyelitis),^8-10 other infections may require extended treatment courses for weeks. The risk of adverse outcomes (eg, complications of medical treatment, readmission risk) and burdens placed on patients and their families may therefore differ across infection types and extend well beyond the immediate hospitalization.

Although infections are common and pediatric providers are expected to have proficiency in managing infections, substantial variation in the management of common pediatric infections exists and is associated with adverse hospitalization outcomes, including increased readmission risk and healthcare costs.^11-18 Potentially avoidable resource use associated with hospital readmission from infection has led to adoption of hospital-level readmission metrics as indicators of the quality of healthcare delivery. For example, the Pediatric Quality Measures Program, established by the Children’s Health Insurance Program Reauthorization Act of 2009, has prioritized measurement of readmissions following hospitalization for lower respiratory tract infection.¹⁹ With government agencies increasingly using readmission metrics to assess quality of healthcare delivery, developing metrics that focus on these resource-intensive conditions is essential.

Because infections are a common and costly indication for hospital resource use and because substantial variation in management has been observed, promoting a broader understanding of infection-specific readmission rates is important for prioritizing readmission-reduction opportunities in children. This study’s objectives were the following: (1) to describe the prevalence and characteristics of infection hospitalizations in children and their associated readmissions and (2) to estimate the number of readmissions avoided and costs saved if all hospitals achieved the 10th percentile of the hospitals’ risk-adjusted readmission rate (ie, readmission benchmark).

Study Design and Data Source

We performed a retrospective cohort analysis using the 2014 Agency for Healthcare Research and Quality (AHRQ) Nationwide Readmissions Database (NRD).²⁰ The 2014 NRD is an administrative database that contains information on inpatient stays from January 1, 2014, to December 31, 2014, for all payers and allows for weighted national estimates of readmissions for all US individuals. Data within NRD are aggregated from 22 geographically diverse states representing approximately one-half of the US population. NRD contains deidentified patient-level data with unique verified patient identifiers to track individuals within and across hospitals in a state. However, AHRQ guidelines specify that NRD cannot be used for reporting hospital-specific readmission rates. Thus, for the current study, the Inpatient Essentials (Children’s Hospital Association), or IE, database was used to measure hospital-level readmission rates and to distinguish benchmark readmission rates for individual infection diagnoses.²¹ The IE database is composed of 90 children’s hospitals distributed throughout all regions of the United States. The inclusion of free-standing children’s hospitals and children’s hospitals within adult hospitals allows for comparisons and benchmarking across hospitals on multiple metrics, including readmissions.

Study Population

Children 0 to 17 years of age with a primary diagnosis at the index admission for infection between January 1, 2014, and November 30, 2014, were included. The end date of November 30, 2014, allowed for a full 30-day readmission window for all index admissions. We excluded index admissions that resulted in transfer to another acute care hospital or in-hospital mortality. Additionally, we excluded index admissions of children who had hematologic or immunologic conditions, malignancy, or history of bone marrow and solid-organ transplant, using the classification system for complex chronic conditions (CCCs) from Feudtner et al.²² Due to the high likelihood of immunosuppression in patients with these conditions, children may have nuanced experiences with illness severity, trajectory, and treatment associated with infection that place them at increased risk for nonpreventable readmission.

Main Exposure

The main exposure was infection type during the index admission. Condition-specific index admissions were identified using AHRQ’s Clinical Classifications Software (CCS) categories.²³ CCS is a classification schema that categorizes the greater than 14,000 International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) diagnosis codes and 3,900 ICD-9-CM procedure codes into clinically meaningful categories of 295 diagnosis (including mental health codes and E-codes) and 231 procedural groupings. Twenty-two groupings indicative of infection were distinguished and used for the current study. Examples of infections included aspiration pneumonia, pneumonia, bronchiolitis, and sexually transmitted infection. We combined related CCS categories when possible for ease of interpretation and presentation of data (Appendix Table 1).

Main Outcome Measure

The main outcome measure was 30-day hospital readmission. Readmission was defined as all-cause, unplanned admission within 30 days following discharge from a preceding hospitalization. Planned hospital readmissions were identified and excluded using methods from AHRQ’s Pediatric All-Condition Readmission Measure.²⁴ We defined a same-cause return as a return with the same CCS infection category as the index admission. Costs associated with readmissions were estimated from charges using hospital-specific cost-to-charge ratios provided with NRD.

Patient Demographic and Clinical Characteristics

Patient demographic characteristics included age at index admission (<1 year, 1-5 years, 6-9 years, 10-14 years, and 15-18 years), sex, payer (ie, government, private, other), and discharge disposition (ie, routine, home health, other). We assessed all patients for medical complexity, as defined by the presence of at least one CCC, and we reported the categories of CCCs by organ system involved.²² Otherwise, patients were identified as without medical complexity.

Statistical Analysis

We summarized continuous variables with medians and interquartile ranges and categorical variables with frequencies and percentages. To develop benchmark readmission rates for each infection type, we used generalized linear mixed models with random intercepts for each hospital in the IE database. For each infection type, the benchmark readmission rate was defined as the 10th percentile of hospitals’ risk-adjusted readmission rates. The 10th percentile was chosen to identify the best performing 10% of hospitals (ie, hospitals with the lowest readmission rates). Because children with medical complexity account for a large proportion of hospital resource use and are at high risk for readmission,^4,25 we developed benchmarks stratified by presence/absence of a CCC (ie, with complexity vs without complexity). Models were adjusted for severity of illness using the Hospitalization Resource Intensity Score for Kids (H-RISK),²⁶ a scoring system that assigns relative weights for each All Patient Refined Diagnosis-Related Group (3M Corporation) and severity of illness level, and each hospital’s risk-adjusted readmission rate was determined.

With use of weights to achieve national estimates of index admissions and readmissions, we determined the number of potentially avoidable readmissions by calculating the number of readmissions observed in the NRD that would not occur if all hospitals achieved readmission rates equal to the 10th percentile. Avoidable costs were estimated by multiplying the number of potentially avoidable readmissions by the mean cost of a readmission for infections of that type. Estimates of avoidable readmissions and costs were stratified by medical complexity. In addition to describing estimates at the 10th percentile benchmark, we similarly developed estimates of potentially avoidable readmissions and avoidable costs for the 5th and 25th percentiles, which are presented within Appendix Table 2 (children without complexity) and Appendix Table 3 (children with complexity).

All statistical analyses were performed using SAS version 9.4 (SAS Institute), and P values <.001 were considered statistically significant due to the large sample size. The Office of Research Integrity at Children’s Mercy Hospital deemed this study exempt from institutional board review.

Characteristics of the Study Population

The study included 380,067 index admissions for infection and an accompanying 18,469 unplanned all-cause readmissions over the 30 days following discharge (readmission rate, 4.9%; Table 1). Children ages 1 to 5 years accounted for the largest percentage (32.9%) of index hospitalizations, followed by infants younger than 1 year (30.3%). The readmission rate by age group was highest for infants younger than 1 year, compared with rates among all other age groups (5.6% among infants < 1 year vs 4.4%-4.7% for other age groups; P < .001). In the overall cohort, 16.2% of admissions included patients with a CCC. Children with medical complexity had higher readmission rates than those without medical complexity (no CCC, 3.2%; 1 CCC, 9.2%; 2+ CCCs, 18.9%). A greater percentage of children experiencing a readmission had government insurance (63.0% vs 59.2%; P < .001) and received home health nursing (10.1% vs 2.7%; P < .001) following the index encounter.

Children Without Complexity

Index Admissions and 30-day Readmissions

Among patients without medical complexity, index admissions occurred most frequently for pneumonia (n = 54,717), bronchiolitis (n = 53,959), and appendicitis (n = 45,036) (Figure 1). The median length of stay (LOS) for index admissions ranged from 1 to 5 days (Table 2), with septic arthritis and osteomyelitis having the longest median LOS at 5 (IQR, 3-7) days.

Thirty-day readmission rates varied substantially by infection at the index admission (range, 1.5% for sexually transmitted infection to 8.6% for peritonitis) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 7 days (Table 2), while the median number of days to readmission varied substantially by infection type (range, 4 days for bacterial infection [site unspecified] to 24 days for sexually transmitted infections). Among the top five indications for admission for children without complexity, 14.9% to 52.8% of readmissions were for the same cause as the index admission; however, many additional returns were likely related to the index admission (Appendix Table 4). For example, among other return reasons, an additional 992 (61.7%) readmissions following appendicitis hospitalizations were for complications of surgical procedures or medical care, peritonitis, intestinal obstruction, and abdominal pain.

Impact of Achieving Readmission Benchmarks

Among children without complexity, readmission benchmarks (ie, the 10th percentile of readmission rates across hospitals) ranged from 0% to 26.7% (Figure 2). An estimated 54.7% of readmissions (n = 5,507) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $44.5 million in savings. Pneumonia, bronchiolitis, gastroenteritis, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if a 10th percentile benchmark was achieved.

Children With Medical Complexity

Index Admissions and 30-day Readmissions

Among patients with complexity, index admissions occurred most frequently for pneumonia (n = 14,344), bronchiolitis (n = 8,618), and upper respiratory tract infection (n = 6,407) (Figure 1). The median LOS for index admissions ranged from 1 to 9 days (Table 2), with septicemia and CNS infections having the longest median LOS at 9 days.

Thirty-day readmission rates varied substantially by the type of infection at the index admission (range, 0% for sexually transmitted infection to 21.6% for aspiration pneumonia) (Figure 1). The median LOS for 30-day readmissions varied from 2 to 14 days (Table 2), and the median number of days to readmission varied substantially by infection type (range, 6 days for tonsillitis to 23 days for other infection). Among the top five indications for admission for medically complex children, 8% to 40.4% of readmissions were for the same cause as the index admission (Appendix Table 4). As with the children without complexity, additional returns were likely related to the index admission.

Impact of Achieving Readmission Benchmarks

Among children with medical complexity, readmission benchmarks ranged from 0% to 30.3% (Figure 2). An estimated 42.6% of readmissions (n = 3,576) could potentially be reduced if hospitals achieved infection-specific benchmark readmission rates, which could result in an estimated $70.8 million in savings. Pneumonia, bronchiolitis, septicemia, and upper respiratory tract infections were among conditions with the greatest potential reductions in readmissions and costs if the benchmarks were achieved.

The current study uncovered new findings regarding unplanned readmissions following index infection hospitalizations for children. In particular, readmission rates and time to readmission varied substantially by infection subtype. The study also identified priority infections and unique considerations for children with CCCs, all of which may help maximize the value of readmission metrics aimed at advancing hospital quality and reducing costs for infection hospitalizations in children. If all hospitals achieved the readmission rates of the best performing hospitals, 42.6% to 54.7% fewer readmissions could be realized with associated cost savings.

Nationally, studies have reported 30-day, all-cause unplanned readmission rates of 6.2% to 10.3%.^5,27 In our current study we observed an overall readmission rate of 4.9% across all infectious conditions; however, readmission rates varied substantially by condition, with upper and lower respiratory tract infections, septicemia, and gastroenteritis among infections with the greatest number of potentially avoidable readmissions based on achievement of readmission benchmarks. Currently, pediatric-specific all-cause and lower respiratory tract infection readmission metrics have been developed with the aim of improving quality of care and reducing healthcare expenditures.²⁸ Future readmission studies and metrics may prioritize conditions such as septicemia, gastroenteritis, and other respiratory tract infections. Our current study demonstrated that many readmissions following infection hospitalizations were associated with the same CCS category or within a related CCS category (eg, complications of surgical procedures or medical care, appendicitis, peritonitis, intestinal obstruction, and abdominal pain constituted the top five indications for readmission for appendicitis, whereas respiratory illnesses constituted the top five indications for readmissions for pneumonia). While this current study cannot clarify this relationship further, and the “avoidability” of unplanned readmissions is debated,^29-31 our findings suggest that future investigations might focus on identifying whether condition-specific interventions during the index admission could mitigate some readmissions.

While the LOS of the index admission and the readmission were similar for most infection subtypes, we observed substantial variability in the temporal risk for readmission by infection subtype. Our observations complement studies exploring the timing of readmissions for other pediatric conditions.^32-34 In particular, our findings further highlight that the composition and interaction of factors influencing infection readmissions may vary by condition. Infections represent a diverse group of conditions, with treatment courses that vary in need for parenteral antibiotics, ability to tailor treatment based on organism and susceptibilities, and length of treatment. While treatment for some infections may be accomplished, or nearly accomplished, prior to discharge, other infections (eg, osteomyelitis) may require prolonged treatment, shifting the burden of management and monitoring to patients and their families, which along with the timeliness and adequacy of outpatient follow-up, may modify an individual’s readmission risk. Consequently, a “one-size fits all” approach to discharge counseling may not be successful across all conditions. Our study lays the groundwork for how these temporal relationships may be used by clinicians to counsel families regarding the likely readmission timeframe, based on infection, and to establish follow-up appointments ahead of the infection-specific “readmission window,” which may allow outpatient providers to intervene when readmission risk is greatest.

Consistent with prior literature, children with medical complexity in our study had increased frequency of 30-day, all-cause unplanned readmissions and LOS, compared with peers who did not have complexity.⁵ Readmission rates following hospitalizations for aspiration pneumonia were comparable to those reported by Thompson et al in their study examining rates of pneumonia in children with neurologic impairment.³⁵ In our current study, nearly 96% of readmissions following aspiration pneumonia hospitalizations were for children with medical complexity, and more than 58% of these readmissions were for aspiration pneumonia or respiratory illness. Future investigations should seek to explore factors contributing to readmissions in children with medical complexity and to evaluate whether interventions such as postdischarge coaching or discharge bundles could assist with reductions in healthcare resource use.^36,37

Despite a lack of clear association between readmissions and quality of care for children,³⁸ readmissions rates continue to be used as a quality measure for hospitalized patients. Within our present study, we found that achieving benchmark readmission rates for infection hospitalizations could lead to substantial reductions in readmissions; however, these reductions were seen across this relatively similar group of infection diagnoses, and hospitals may face greater challenges when attempting to achieve a 10th percentile readmission benchmark across a more expansive set of diagnoses. Despite these challenges, understanding the intricacies of readmissions may lead to improved readmission metrics and the systematic identification of avoidable readmissions, the goal of which is to enhance the quality of healthcare for hospitalized children.

Our findings should be interpreted in the context of several limitations. We defined our readmission benchmark at the 10th percentile using the IE database. Because an estimated 70% of hospitalizations for children occur within general hospitals,³⁹ we believe that our use of the IE database allowed for increased generalizability, though the broadening of our findings to nonacademic hospital settings may be less reliable. While we describe the 10th percentile readmission benchmark here, alternative benchmarks would result in different estimates of avoidable readmissions and associated readmission costs (Appendix Tables 2 and 3). The IE and NRD databases do not distinguish intensive care use. We tried to address this limitation through model adjustments using H-RISK, which is particularly helpful for adjusting for NICU admissions through use of the 27 All Patient Refined Diagnosis-Related Groups for neonatal conditions. Additionally, the NRD uses state-level data to derive national estimates and is not equipped to measure readmissions to hospitals in a different state, distinguish patient deaths occurring after discharge, or to examine the specific indication for readmission (ie, unable to assess if the readmission is related to a complication of the treatment plan, progression of the illness course, or for an unrelated reason). While sociodemographic and socioeconomic factors may affect readmissions, the NRD does not contain information on patients’ race/ethnicity, family/social attributes, or postdischarge follow-up health services, which are known to influence the need for readmission.

Despite these limitations, this study highlights future areas for research and potential opportunities for reducing readmission following infection hospitalizations. First, identifying hospital- and systems-based factors that contribute to readmission reductions at the best-performing hospitals may identify opportunities for hospitals with the highest readmission rates to achieve the rates of the best-performing hospitals. Second, conditions such as upper and lower respiratory tract infections, along with septicemia and gastroenteritis, may serve as prime targets for future investigation based on the estimated number of avoidable readmissions and potential cost savings associated with these conditions. Finally, future investigations that explore discharge counseling and follow-up tailored to the infection-specific temporal risk and patient complexity may identify opportunities for further readmission reductions.

Our study provides a broad look at readmissions following infection hospitalizations and highlights substantial variation in readmissions based on infection type. To improve hospital resource use for infections, future preventive measures could prioritize children with complex chronic conditions and those with specific diagnoses (eg, upper and lower respiratory tract infections).

This information or content and conclusions are those of the authors and should not be construed as the official position or policy of, nor should any endorsements be inferred by, NIH or the US government.

Bacterial infections are a common reason for pediatric hospital admissions in the United States.¹ Antibiotics are the mainstay of treatment, and whether to administer them intravenously (IV) or enterally is an important and, at times, challenging decision. Not all hospitalized patients with infections require IV antibiotics, and safe, effective early transitions to enteral therapy have been described for numerous infections.^2-7 However, guidelines describing the ideal initial route of antibiotic administration and when to transition to oral therapy are lacking.^5,7,8 This lack of high-quality evidence-based guidance may contribute to overuse of IV antibiotics for many hospitalized pediatric patients, even when safe and effective enteral options exist.⁹

Significant costs and harms are associated with the use of IV antibiotics. In particular, studies have demonstrated longer length of stay (LOS), increased costs, and worsened pain or anxiety related to complications (eg, phlebitis, extravasation injury, thrombosis, catheter-associated bloodstream infections) associated with IV antibiotics.^3,4,10-13 Earlier transition to enteral therapy, however, can mitigate these increased risks and costs.

The Centers for Disease Control and Prevention lists the transition from IV to oral antibiotics as a key stewardship intervention for improving antibiotic use.¹⁴ The Infectious Diseases Society of America (IDSA) antibiotic stewardship program guidelines strongly recommend the timely conversion from IV to oral antibiotics, stating that efforts focusing on this transition should be integrated into routine practice.¹⁵ There are a few metrics in the literature to measure this intervention, but none is universally used, and a modified delphi process could not reach consensus on IV-to-oral transition metrics.¹⁶

Few studies describe the opportunity to transition to enteral antibiotics in hospitalized patients with common bacterial infections or explore variation across hospitals. It is critical to understand current practice of antibiotic administration in order to identify opportunities to optimize patient outcomes and promote high-value care. Furthermore, few studies have evaluated the feasibility of IV-to-oral transition metrics using an administrative database. Thus, the aims of this study were to (1) determine opportunities to transition from IV to enteral antibiotics for pediatric patients hospitalized with common bacterial infections based on their ability to tolerate other enteral medications, (2) describe variation in transition practices among children’s hospitals, and (3) evaluate the feasibility of novel IV-to-oral transition metrics using an administrative database to inform stewardship efforts.

Study Design and Setting

This multicenter, retrospective cohort study used data from the Pediatric Health Information System (PHIS), an administrative and billing database containing encounter-level data from 52 tertiary care pediatric hospitals across the United States affiliated with the Children’s Hospital Association (Lenexa, Kansas). Hospitals submit encounter-level data, including demographics, medications, and diagnoses based on International Classification of Disease, Tenth Revision, Clinical Modification (ICD-10-CM) codes. Data were de-identified at the time of submission, and data quality and reliability were assured by joint efforts between the Children’s Hospital Association and participating hospitals.

Study Population

This study included pediatric patients aged 60 days to 18 years who were hospitalized (inpatient or observation status) at one of the participating hospitals between January 1, 2017, and December 31, 2018, for one of the following seven common bacterial infections: community-acquired pneumonia (CAP), neck infection (superficial and deep), periorbital/orbital infection, urinary tract infection (UTI), osteomyelitis, septic arthritis, or skin and soft tissue infection (SSTI). The diagnosis cohorts were defined based on ICD-10-CM discharge diagnoses adapted from previous studies (Appendix Table 1).^3,17-23 To define a cohort of generally healthy pediatric patients with an acute infection, we excluded patients hospitalized in the intensive care unit, patients with nonhome discharges, and patients with complex chronic conditions.²⁴ We also excluded hospitals with incomplete data during the study period (n=1). The Institutional Review Board at Cincinnati Children’s Hospital Medical Center determined this study to be non–human-subjects research.

Outcomes

The primary outcomes were the number of opportunity days and the percent of days with opportunity to transition from IV to enteral therapy. Opportunity days, or days in which there was a potential opportunity to transition from IV to enteral antibiotics, were defined as days patients received only IV antibiotic doses and at least one enteral nonantibiotic medication, suggesting an ability to take enteral medications.¹³ We excluded days patients received IV antibiotics for which there was no enteral alternative (eg, vancomycin, Appendix Table 2). When measuring opportunity, to be conservative (ie, to underestimate rather than overestimate opportunity), we did not count as an opportunity day any day in which patients received both IV and enteral antibiotics. Percent opportunity, or the percent of days patients received antibiotics in which there was potential opportunity to transition from IV to enteral antibiotics, was defined as the number of opportunity days divided by number of inpatient days patients received enteral antibiotics or IV antibiotics with at least one enteral nonantibiotic medication (antibiotic days). Similar to opportunity days, antibiotic days excluded days patients were on IV antibiotics for which there was no enteral alternative. Based on our definition, a lower percent opportunity indicates that a hospital is using enteral antibiotics earlier during the hospitalization (earlier transition), while a higher percent opportunity represents later enteral antibiotic use (later transition).

Statistical Analysis

Demographic and clinical characteristics were summarized by diagnosis with descriptive statistics, including frequency with percentage, mean with standard deviation, and median with interquartile range (IQR). For each diagnosis, we evaluated aggregate opportunity days (sum of opportunity days among all hospitals), opportunity days per encounter, and aggregate percent opportunity using frequencies, mean with standard deviation, and percentages, respectively. We also calculated aggregate opportunity days for diagnosis-antibiotic combinations. To visually show variation in the percent opportunity across hospitals, we displayed the percent opportunity on a heat map, and evaluated percent opportunity across hospitals using chi-square tests. To compare the variability in the percent opportunity across and within hospitals, we used a generalized linear model with two fixed effects (hospital and diagnosis), and parsed the variability using the sum of squares. We performed a sensitivity analysis and excluded days that patients received antiemetic medications (eg, ondansetron, granisetron, prochlorperazine, promethazine), as these suggest potential intolerance of enteral medications. All statistical analyses were performed using SAS v.9.4 (SAS Institute Inc, Cary, North Carolina) and GraphPad Prism 8.0 (GraphPad Software Inc., San Diego, California), and P values < .05 were considered statistically significant.

During the 2-year study period, 100,103 hospitalizations met our inclusion criteria across 51 hospitals and seven diagnosis categories (Table 1). Diagnosis cohorts ranged in size from 1,462 encounters for septic arthritis to 35,665 encounters for neck infections. Overall, we identified 88,522 aggregate opportunity days on which there was an opportunity to switch from IV to enteral treatment in the majority of participants (percent opportunity, 57%).

Opportunity by Diagnosis

The number of opportunity days (aggregate and mean per encounter) and percent opportunity varied by diagnosis (Table 2). The aggregate number of opportunity days ranged from 3,693 in patients with septic arthritis to 25,359 in patients with SSTI, and mean opportunity days per encounter ranged from 0.9 in CAP to 2.8 in septic arthritis. Percent opportunity was highest for septic arthritis at 72.7% and lowest for CAP at 39.7%.

Variation in Opportunity Among Hospitals

The variation in the percent opportunity across hospitals was statistically significant for all diagnoses (Figure). Within hospitals, we observed similar practice patterns across diagnoses. For example, hospitals with a higher percent opportunity for one diagnosis tended to have higher percent opportunity for the other diagnoses (as noted in the top portion of the Figure), and those with lower percent opportunity for one diagnosis tended to also have lower percent opportunity for the other diagnoses studied (as noted in the bottom portion of the Figure). When evaluating variability in the percent opportunity, 45% of the variability was attributable to the hospital-effect and 35% to the diagnosis; the remainder was unexplained variability. Sensitivity analysis excluding days when patients received an antiemetic medication yielded no differences in our results.

Opportunity by Antibiotic

The aggregate number of opportunity days varied by antibiotic (Table 3). Intravenous antibiotics with the largest number of opportunity days included clindamycin (44,293), ceftriaxone (23,896), and ampicillin-sulbactam (15,484). Antibiotic-diagnosis combinations with the largest number of opportunity days for each diagnosis included ceftriaxone and ampicillin in CAP; clindamycin in cellulitis, SSTI, and neck infections; ceftriaxone in UTI; and cefazolin in osteomyelitis and septic arthritis.

In this multicenter study of pediatric patients hospitalized with common bacterial infections, there was the potential to transition from IV to enteral treatment in over half of the antibiotic days. The degree of opportunity varied by infection, antibiotic, and hospital. Antibiotics with a large aggregate number of opportunity days for enteral transition included clindamycin, which has excellent bioavailability; and ampicillin and ampicillin-sulbactam, which can achieve pharmacodynamic targets with oral equivalents.^25-29 The across-hospital variation for a given diagnosis suggests that certain hospitals have strategies in place which permit an earlier transition to enteral antibiotics compared to other institutions in which there were likely missed opportunities to do so. This variability is likely due to limited evidence, emphasizing the need for robust studies to better understand the optimal initial antibiotic route and transition time. Our findings highlight the need for, and large potential impact of, stewardship efforts to promote earlier transition for specific drug targets. This study also demonstrates the feasibility of obtaining two metrics—percent opportunity and opportunity days—from administrative databases to inform stewardship efforts within and across hospitals.

Opportunity days and percent opportunity varied among diagnoses. The variation in aggregate opportunity days was largely a reflection of the number of encounters: Diagnoses such as SSTI, neck infections, and CAP had a large number of both aggregate opportunity days and encounters. The range of opportunity days per encounter (0.9-2.5) suggests potential missed opportunities to transition to enteral antibiotics across all diagnoses (Table 2). The higher opportunity days per encounter in osteomyelitis and septic arthritis may be related to longer LOS and higher percent opportunity. Percent opportunity likely varied among diagnoses due to differences in admission and discharge readiness criteria, diagnostic evaluation, frequency of antibiotic administration, and evidence on the optimal route of initial antibiotics and when to transition to oral formulations. For example, we hypothesize that certain diagnoses, such as osteomyelitis and septic arthritis, have admission and discharge readiness criteria directly tied to the perceived need for IV antibiotics, which may limit in-hospital days on enteral antibiotics and explain the high percent opportunity that we observed. The high percent opportunity seen in musculoskeletal infections also may be due to delays in initiating targeted treatment until culture results were available. Encounters for CAP had the lowest percent opportunity; we hypothesize that this is because admission and discharge readiness may be determined by factors other than the need for IV antibiotics (eg, need for supplemental oxygen), which may increase days on enteral antibiotics and lead to a lower percent opportunity.³⁰

Urinary tract infection encounters had a high percent opportunity. As with musculoskeletal infection, this may be related to delays in initiating targeted treatment until culture results became available. Another reason for the high percent opportunity in UTI could be the common use of ceftriaxone, which, dosed every 24 hours, likely reduced the opportunity to transition to enteral antibiotics. There is strong evidence demonstrating no difference in outcomes based on antibiotic routes for UTI, and we would expect this to result in a low percent opportunity.^2,31 While the observed high opportunity in UTI may relate to an initial unknown diagnosis or concern for systemic infection, this highlights potential opportunities for quality improvement initiatives to promote empiric oral antibiotics in clinically stable patients hospitalized with suspected UTI.

There was substantial variation in percent opportunity across hospitals for a given diagnosis, with less variation across diagnoses for a given hospital. Variation across hospitals but consistency within individual hospitals suggests that some hospitals may promote earlier transition from IV to enteral antibiotics as standard practice for all diagnoses, while other hospitals continue IV antibiotics for the entire hospitalization, highlighting potential missed opportunities at some institutions. While emerging data suggest that traditional long durations of IV antibiotics are not necessary for many infections, the limited evidence on the optimal time to switch to oral antibiotics may have influenced this variation.^2-7 Many guidelines recommend initial IV antibiotics for hospitalized pediatric patients, but there are few studies comparing IV and enteral therapy.^2,5,9 Limited evidence leaves significant room for hospital culture, antibiotic stewardship efforts, reimbursement considerations, and/or hospital workflow to influence transition timing and overall opportunity at individual hospitals.^7,8,32-34 These findings emphasize the importance of research to identify optimal transition time and comparative effectiveness studies to evaluate whether initial IV antibiotics are truly needed for mild—and even severe—disease presentations. Since many patients are admitted for the perceived need for IV antibiotics, earlier use of enteral antibiotics could reduce rates of hospitalizations, LOS, healthcare costs, and resource utilization.

Antibiotics with a high number of opportunity days included clindamycin, ceftriaxone, ampicillin-sublactam, and ampicillin. Our findings are consistent with another study which found that most bioavailable drugs, including clindamycin, were administered via the IV route and accounted for a large number of antibiotic days.³⁵ The Infectious Diseases Society of America recommends that hospitals promote earlier transition to oral formulations for highly bioavailable drugs.⁷ Given the high bioavailability of clindamycin, its common use in high-frequency encounters such as SSTI and neck infections, and the fact that it accounted for a large number of opportunity days, quality improvement initiatives promoting earlier transition to oral clindamycin could have a large impact across health systems.^25,26 Additionally, although beta-lactam antibiotics such as amoxicillin and amoxicillin-sulbactam are not highly bioavailable, oral dosing can achieve sufficient serum concentrations to reach pharmacodynamic targets for common clinical indications; this could be an important quality improvement initiative.^27-29 Several single-site studies have successfully implemented quality improvement initiatives to promote earlier IV-to-enteral transition, with resulting reductions in costs and no adverse events noted, highlighting the feasibility and impact of such efforts.^13,36-38

This study also demonstrates the feasibility of collecting two metrics (percent opportunity and opportunity days) from administrative databases to inform IV-to-oral transition benchmarking and stewardship efforts. While there are several metrics in the literature for evaluating antibiotic transition (eg, days of IV or oral therapy, percentage of antibiotics given via the oral route, time to switch from IV to oral, and acceptance rate of suggested changes to antibiotic route), none are universally used or agreed upon.^15,16,39 The opportunity metrics used in this study have several strengths, including the feasibility of obtaining them from existing databases and the ability to account for intake of other enteral medications; the latter is not evaluated in other metrics. These opportunity metrics can be used together to identify the percent of time in which there is opportunity to transition and total number of days to understand the full extent of potential opportunity for future interventions. As demonstrated in this study, these metrics can be measured by diagnosis, antibiotic, or diagnosis-antibiotic combination, and they can be used to evaluate stewardship efforts at a single institution over time or compare efforts across hospitals.

These findings should be interpreted in the context of important limitations. First, we attempted to characterize potential opportunity to transition to enteral medications based on a patient’s ability to tolerate nonenteral medications. However, there are other factors that could limit the opportunity to transition that we could not account for with an administrative dataset, including the use of antibiotics prior to admission, disease progression, severity of illness, and malabsorptive concerns. Thus, though we may have overestimated the true opportunity to transition to enteral antibiotics, it is unlikely that this would account for all of the variation in transition times that we observed across hospitals. Second, while our study required patients to have one of seven types of infection, we did not exclude any additional infectious diagnoses (eg, concurrent bacteremia, Clostridioides difficile, otitis media) that could have driven the choice of antibiotic type and modality. Although emerging evidence is supporting earlier transitions to oral therapy, bacteremia is typically treated with IV antibiotics; this may have led to an overestimation of true opportunity.⁴⁰ “Clostridioides” difficile and otitis media are typically treated with enteral therapy; concurrent infections such as these may have led to an underestimation of opportunity given the fact that, based on our definition, the days on which patients received both IV and enteral antibiotics were not counted as opportunity days. Third, because PHIS uses billing days to capture medication use, we were unable to distinguish transitions that occurred early in the day vs those that took place later in the day. This could have led to an underestimation of percent opportunity, particularly for diagnoses with a short LOS; it also likely led to an underestimation of the variability observed across hospitals. Fourth, because we used an administrative dataset, we are unable to understand reasoning behind transitioning time from IV to oral antibiotics, as well as provider, patient, and institutional level factors that influenced these decisions.

Children hospitalized with bacterial infections often receive IV antibiotics, and the timing of transition from IV to enteral antibiotics varies significantly across hospitals. Further research is needed to compare the effectiveness of IV and enteral antibiotics and better define criteria for transition to enteral therapy. We identified ample opportunities for quality improvement initiatives to promote earlier transition, which have the potential to reduce healthcare utilization and promote optimal patient-directed high-value care.

Bacterial infections are a common reason for pediatric hospital admissions in the United States.¹ Antibiotics are the mainstay of treatment, and whether to administer them intravenously (IV) or enterally is an important and, at times, challenging decision. Not all hospitalized patients with infections require IV antibiotics, and safe, effective early transitions to enteral therapy have been described for numerous infections.^2-7 However, guidelines describing the ideal initial route of antibiotic administration and when to transition to oral therapy are lacking.^5,7,8 This lack of high-quality evidence-based guidance may contribute to overuse of IV antibiotics for many hospitalized pediatric patients, even when safe and effective enteral options exist.⁹

Significant costs and harms are associated with the use of IV antibiotics. In particular, studies have demonstrated longer length of stay (LOS), increased costs, and worsened pain or anxiety related to complications (eg, phlebitis, extravasation injury, thrombosis, catheter-associated bloodstream infections) associated with IV antibiotics.^3,4,10-13 Earlier transition to enteral therapy, however, can mitigate these increased risks and costs.

The Centers for Disease Control and Prevention lists the transition from IV to oral antibiotics as a key stewardship intervention for improving antibiotic use.¹⁴ The Infectious Diseases Society of America (IDSA) antibiotic stewardship program guidelines strongly recommend the timely conversion from IV to oral antibiotics, stating that efforts focusing on this transition should be integrated into routine practice.¹⁵ There are a few metrics in the literature to measure this intervention, but none is universally used, and a modified delphi process could not reach consensus on IV-to-oral transition metrics.¹⁶

Few studies describe the opportunity to transition to enteral antibiotics in hospitalized patients with common bacterial infections or explore variation across hospitals. It is critical to understand current practice of antibiotic administration in order to identify opportunities to optimize patient outcomes and promote high-value care. Furthermore, few studies have evaluated the feasibility of IV-to-oral transition metrics using an administrative database. Thus, the aims of this study were to (1) determine opportunities to transition from IV to enteral antibiotics for pediatric patients hospitalized with common bacterial infections based on their ability to tolerate other enteral medications, (2) describe variation in transition practices among children’s hospitals, and (3) evaluate the feasibility of novel IV-to-oral transition metrics using an administrative database to inform stewardship efforts.

Study Design and Setting

This multicenter, retrospective cohort study used data from the Pediatric Health Information System (PHIS), an administrative and billing database containing encounter-level data from 52 tertiary care pediatric hospitals across the United States affiliated with the Children’s Hospital Association (Lenexa, Kansas). Hospitals submit encounter-level data, including demographics, medications, and diagnoses based on International Classification of Disease, Tenth Revision, Clinical Modification (ICD-10-CM) codes. Data were de-identified at the time of submission, and data quality and reliability were assured by joint efforts between the Children’s Hospital Association and participating hospitals.

Study Population

This study included pediatric patients aged 60 days to 18 years who were hospitalized (inpatient or observation status) at one of the participating hospitals between January 1, 2017, and December 31, 2018, for one of the following seven common bacterial infections: community-acquired pneumonia (CAP), neck infection (superficial and deep), periorbital/orbital infection, urinary tract infection (UTI), osteomyelitis, septic arthritis, or skin and soft tissue infection (SSTI). The diagnosis cohorts were defined based on ICD-10-CM discharge diagnoses adapted from previous studies (Appendix Table 1).^3,17-23 To define a cohort of generally healthy pediatric patients with an acute infection, we excluded patients hospitalized in the intensive care unit, patients with nonhome discharges, and patients with complex chronic conditions.²⁴ We also excluded hospitals with incomplete data during the study period (n=1). The Institutional Review Board at Cincinnati Children’s Hospital Medical Center determined this study to be non–human-subjects research.

Outcomes

The primary outcomes were the number of opportunity days and the percent of days with opportunity to transition from IV to enteral therapy. Opportunity days, or days in which there was a potential opportunity to transition from IV to enteral antibiotics, were defined as days patients received only IV antibiotic doses and at least one enteral nonantibiotic medication, suggesting an ability to take enteral medications.¹³ We excluded days patients received IV antibiotics for which there was no enteral alternative (eg, vancomycin, Appendix Table 2). When measuring opportunity, to be conservative (ie, to underestimate rather than overestimate opportunity), we did not count as an opportunity day any day in which patients received both IV and enteral antibiotics. Percent opportunity, or the percent of days patients received antibiotics in which there was potential opportunity to transition from IV to enteral antibiotics, was defined as the number of opportunity days divided by number of inpatient days patients received enteral antibiotics or IV antibiotics with at least one enteral nonantibiotic medication (antibiotic days). Similar to opportunity days, antibiotic days excluded days patients were on IV antibiotics for which there was no enteral alternative. Based on our definition, a lower percent opportunity indicates that a hospital is using enteral antibiotics earlier during the hospitalization (earlier transition), while a higher percent opportunity represents later enteral antibiotic use (later transition).

Statistical Analysis

Demographic and clinical characteristics were summarized by diagnosis with descriptive statistics, including frequency with percentage, mean with standard deviation, and median with interquartile range (IQR). For each diagnosis, we evaluated aggregate opportunity days (sum of opportunity days among all hospitals), opportunity days per encounter, and aggregate percent opportunity using frequencies, mean with standard deviation, and percentages, respectively. We also calculated aggregate opportunity days for diagnosis-antibiotic combinations. To visually show variation in the percent opportunity across hospitals, we displayed the percent opportunity on a heat map, and evaluated percent opportunity across hospitals using chi-square tests. To compare the variability in the percent opportunity across and within hospitals, we used a generalized linear model with two fixed effects (hospital and diagnosis), and parsed the variability using the sum of squares. We performed a sensitivity analysis and excluded days that patients received antiemetic medications (eg, ondansetron, granisetron, prochlorperazine, promethazine), as these suggest potential intolerance of enteral medications. All statistical analyses were performed using SAS v.9.4 (SAS Institute Inc, Cary, North Carolina) and GraphPad Prism 8.0 (GraphPad Software Inc., San Diego, California), and P values < .05 were considered statistically significant.

During the 2-year study period, 100,103 hospitalizations met our inclusion criteria across 51 hospitals and seven diagnosis categories (Table 1). Diagnosis cohorts ranged in size from 1,462 encounters for septic arthritis to 35,665 encounters for neck infections. Overall, we identified 88,522 aggregate opportunity days on which there was an opportunity to switch from IV to enteral treatment in the majority of participants (percent opportunity, 57%).

Opportunity by Diagnosis

The number of opportunity days (aggregate and mean per encounter) and percent opportunity varied by diagnosis (Table 2). The aggregate number of opportunity days ranged from 3,693 in patients with septic arthritis to 25,359 in patients with SSTI, and mean opportunity days per encounter ranged from 0.9 in CAP to 2.8 in septic arthritis. Percent opportunity was highest for septic arthritis at 72.7% and lowest for CAP at 39.7%.

Variation in Opportunity Among Hospitals

The variation in the percent opportunity across hospitals was statistically significant for all diagnoses (Figure). Within hospitals, we observed similar practice patterns across diagnoses. For example, hospitals with a higher percent opportunity for one diagnosis tended to have higher percent opportunity for the other diagnoses (as noted in the top portion of the Figure), and those with lower percent opportunity for one diagnosis tended to also have lower percent opportunity for the other diagnoses studied (as noted in the bottom portion of the Figure). When evaluating variability in the percent opportunity, 45% of the variability was attributable to the hospital-effect and 35% to the diagnosis; the remainder was unexplained variability. Sensitivity analysis excluding days when patients received an antiemetic medication yielded no differences in our results.

Opportunity by Antibiotic

The aggregate number of opportunity days varied by antibiotic (Table 3). Intravenous antibiotics with the largest number of opportunity days included clindamycin (44,293), ceftriaxone (23,896), and ampicillin-sulbactam (15,484). Antibiotic-diagnosis combinations with the largest number of opportunity days for each diagnosis included ceftriaxone and ampicillin in CAP; clindamycin in cellulitis, SSTI, and neck infections; ceftriaxone in UTI; and cefazolin in osteomyelitis and septic arthritis.

In this multicenter study of pediatric patients hospitalized with common bacterial infections, there was the potential to transition from IV to enteral treatment in over half of the antibiotic days. The degree of opportunity varied by infection, antibiotic, and hospital. Antibiotics with a large aggregate number of opportunity days for enteral transition included clindamycin, which has excellent bioavailability; and ampicillin and ampicillin-sulbactam, which can achieve pharmacodynamic targets with oral equivalents.^25-29 The across-hospital variation for a given diagnosis suggests that certain hospitals have strategies in place which permit an earlier transition to enteral antibiotics compared to other institutions in which there were likely missed opportunities to do so. This variability is likely due to limited evidence, emphasizing the need for robust studies to better understand the optimal initial antibiotic route and transition time. Our findings highlight the need for, and large potential impact of, stewardship efforts to promote earlier transition for specific drug targets. This study also demonstrates the feasibility of obtaining two metrics—percent opportunity and opportunity days—from administrative databases to inform stewardship efforts within and across hospitals.

Opportunity days and percent opportunity varied among diagnoses. The variation in aggregate opportunity days was largely a reflection of the number of encounters: Diagnoses such as SSTI, neck infections, and CAP had a large number of both aggregate opportunity days and encounters. The range of opportunity days per encounter (0.9-2.5) suggests potential missed opportunities to transition to enteral antibiotics across all diagnoses (Table 2). The higher opportunity days per encounter in osteomyelitis and septic arthritis may be related to longer LOS and higher percent opportunity. Percent opportunity likely varied among diagnoses due to differences in admission and discharge readiness criteria, diagnostic evaluation, frequency of antibiotic administration, and evidence on the optimal route of initial antibiotics and when to transition to oral formulations. For example, we hypothesize that certain diagnoses, such as osteomyelitis and septic arthritis, have admission and discharge readiness criteria directly tied to the perceived need for IV antibiotics, which may limit in-hospital days on enteral antibiotics and explain the high percent opportunity that we observed. The high percent opportunity seen in musculoskeletal infections also may be due to delays in initiating targeted treatment until culture results were available. Encounters for CAP had the lowest percent opportunity; we hypothesize that this is because admission and discharge readiness may be determined by factors other than the need for IV antibiotics (eg, need for supplemental oxygen), which may increase days on enteral antibiotics and lead to a lower percent opportunity.³⁰

Urinary tract infection encounters had a high percent opportunity. As with musculoskeletal infection, this may be related to delays in initiating targeted treatment until culture results became available. Another reason for the high percent opportunity in UTI could be the common use of ceftriaxone, which, dosed every 24 hours, likely reduced the opportunity to transition to enteral antibiotics. There is strong evidence demonstrating no difference in outcomes based on antibiotic routes for UTI, and we would expect this to result in a low percent opportunity.^2,31 While the observed high opportunity in UTI may relate to an initial unknown diagnosis or concern for systemic infection, this highlights potential opportunities for quality improvement initiatives to promote empiric oral antibiotics in clinically stable patients hospitalized with suspected UTI.

There was substantial variation in percent opportunity across hospitals for a given diagnosis, with less variation across diagnoses for a given hospital. Variation across hospitals but consistency within individual hospitals suggests that some hospitals may promote earlier transition from IV to enteral antibiotics as standard practice for all diagnoses, while other hospitals continue IV antibiotics for the entire hospitalization, highlighting potential missed opportunities at some institutions. While emerging data suggest that traditional long durations of IV antibiotics are not necessary for many infections, the limited evidence on the optimal time to switch to oral antibiotics may have influenced this variation.^2-7 Many guidelines recommend initial IV antibiotics for hospitalized pediatric patients, but there are few studies comparing IV and enteral therapy.^2,5,9 Limited evidence leaves significant room for hospital culture, antibiotic stewardship efforts, reimbursement considerations, and/or hospital workflow to influence transition timing and overall opportunity at individual hospitals.^7,8,32-34 These findings emphasize the importance of research to identify optimal transition time and comparative effectiveness studies to evaluate whether initial IV antibiotics are truly needed for mild—and even severe—disease presentations. Since many patients are admitted for the perceived need for IV antibiotics, earlier use of enteral antibiotics could reduce rates of hospitalizations, LOS, healthcare costs, and resource utilization.

Antibiotics with a high number of opportunity days included clindamycin, ceftriaxone, ampicillin-sublactam, and ampicillin. Our findings are consistent with another study which found that most bioavailable drugs, including clindamycin, were administered via the IV route and accounted for a large number of antibiotic days.³⁵ The Infectious Diseases Society of America recommends that hospitals promote earlier transition to oral formulations for highly bioavailable drugs.⁷ Given the high bioavailability of clindamycin, its common use in high-frequency encounters such as SSTI and neck infections, and the fact that it accounted for a large number of opportunity days, quality improvement initiatives promoting earlier transition to oral clindamycin could have a large impact across health systems.^25,26 Additionally, although beta-lactam antibiotics such as amoxicillin and amoxicillin-sulbactam are not highly bioavailable, oral dosing can achieve sufficient serum concentrations to reach pharmacodynamic targets for common clinical indications; this could be an important quality improvement initiative.^27-29 Several single-site studies have successfully implemented quality improvement initiatives to promote earlier IV-to-enteral transition, with resulting reductions in costs and no adverse events noted, highlighting the feasibility and impact of such efforts.^13,36-38

This study also demonstrates the feasibility of collecting two metrics (percent opportunity and opportunity days) from administrative databases to inform IV-to-oral transition benchmarking and stewardship efforts. While there are several metrics in the literature for evaluating antibiotic transition (eg, days of IV or oral therapy, percentage of antibiotics given via the oral route, time to switch from IV to oral, and acceptance rate of suggested changes to antibiotic route), none are universally used or agreed upon.^15,16,39 The opportunity metrics used in this study have several strengths, including the feasibility of obtaining them from existing databases and the ability to account for intake of other enteral medications; the latter is not evaluated in other metrics. These opportunity metrics can be used together to identify the percent of time in which there is opportunity to transition and total number of days to understand the full extent of potential opportunity for future interventions. As demonstrated in this study, these metrics can be measured by diagnosis, antibiotic, or diagnosis-antibiotic combination, and they can be used to evaluate stewardship efforts at a single institution over time or compare efforts across hospitals.

These findings should be interpreted in the context of important limitations. First, we attempted to characterize potential opportunity to transition to enteral medications based on a patient’s ability to tolerate nonenteral medications. However, there are other factors that could limit the opportunity to transition that we could not account for with an administrative dataset, including the use of antibiotics prior to admission, disease progression, severity of illness, and malabsorptive concerns. Thus, though we may have overestimated the true opportunity to transition to enteral antibiotics, it is unlikely that this would account for all of the variation in transition times that we observed across hospitals. Second, while our study required patients to have one of seven types of infection, we did not exclude any additional infectious diagnoses (eg, concurrent bacteremia, Clostridioides difficile, otitis media) that could have driven the choice of antibiotic type and modality. Although emerging evidence is supporting earlier transitions to oral therapy, bacteremia is typically treated with IV antibiotics; this may have led to an overestimation of true opportunity.⁴⁰ “Clostridioides” difficile and otitis media are typically treated with enteral therapy; concurrent infections such as these may have led to an underestimation of opportunity given the fact that, based on our definition, the days on which patients received both IV and enteral antibiotics were not counted as opportunity days. Third, because PHIS uses billing days to capture medication use, we were unable to distinguish transitions that occurred early in the day vs those that took place later in the day. This could have led to an underestimation of percent opportunity, particularly for diagnoses with a short LOS; it also likely led to an underestimation of the variability observed across hospitals. Fourth, because we used an administrative dataset, we are unable to understand reasoning behind transitioning time from IV to oral antibiotics, as well as provider, patient, and institutional level factors that influenced these decisions.

Children hospitalized with bacterial infections often receive IV antibiotics, and the timing of transition from IV to enteral antibiotics varies significantly across hospitals. Further research is needed to compare the effectiveness of IV and enteral antibiotics and better define criteria for transition to enteral therapy. We identified ample opportunities for quality improvement initiatives to promote earlier transition, which have the potential to reduce healthcare utilization and promote optimal patient-directed high-value care.

Bacterial infections are a common reason for pediatric hospital admissions in the United States.¹ Antibiotics are the mainstay of treatment, and whether to administer them intravenously (IV) or enterally is an important and, at times, challenging decision. Not all hospitalized patients with infections require IV antibiotics, and safe, effective early transitions to enteral therapy have been described for numerous infections.^2-7 However, guidelines describing the ideal initial route of antibiotic administration and when to transition to oral therapy are lacking.^5,7,8 This lack of high-quality evidence-based guidance may contribute to overuse of IV antibiotics for many hospitalized pediatric patients, even when safe and effective enteral options exist.⁹

Significant costs and harms are associated with the use of IV antibiotics. In particular, studies have demonstrated longer length of stay (LOS), increased costs, and worsened pain or anxiety related to complications (eg, phlebitis, extravasation injury, thrombosis, catheter-associated bloodstream infections) associated with IV antibiotics.^3,4,10-13 Earlier transition to enteral therapy, however, can mitigate these increased risks and costs.

The Centers for Disease Control and Prevention lists the transition from IV to oral antibiotics as a key stewardship intervention for improving antibiotic use.¹⁴ The Infectious Diseases Society of America (IDSA) antibiotic stewardship program guidelines strongly recommend the timely conversion from IV to oral antibiotics, stating that efforts focusing on this transition should be integrated into routine practice.¹⁵ There are a few metrics in the literature to measure this intervention, but none is universally used, and a modified delphi process could not reach consensus on IV-to-oral transition metrics.¹⁶

Few studies describe the opportunity to transition to enteral antibiotics in hospitalized patients with common bacterial infections or explore variation across hospitals. It is critical to understand current practice of antibiotic administration in order to identify opportunities to optimize patient outcomes and promote high-value care. Furthermore, few studies have evaluated the feasibility of IV-to-oral transition metrics using an administrative database. Thus, the aims of this study were to (1) determine opportunities to transition from IV to enteral antibiotics for pediatric patients hospitalized with common bacterial infections based on their ability to tolerate other enteral medications, (2) describe variation in transition practices among children’s hospitals, and (3) evaluate the feasibility of novel IV-to-oral transition metrics using an administrative database to inform stewardship efforts.

Study Design and Setting

This multicenter, retrospective cohort study used data from the Pediatric Health Information System (PHIS), an administrative and billing database containing encounter-level data from 52 tertiary care pediatric hospitals across the United States affiliated with the Children’s Hospital Association (Lenexa, Kansas). Hospitals submit encounter-level data, including demographics, medications, and diagnoses based on International Classification of Disease, Tenth Revision, Clinical Modification (ICD-10-CM) codes. Data were de-identified at the time of submission, and data quality and reliability were assured by joint efforts between the Children’s Hospital Association and participating hospitals.

Study Population

This study included pediatric patients aged 60 days to 18 years who were hospitalized (inpatient or observation status) at one of the participating hospitals between January 1, 2017, and December 31, 2018, for one of the following seven common bacterial infections: community-acquired pneumonia (CAP), neck infection (superficial and deep), periorbital/orbital infection, urinary tract infection (UTI), osteomyelitis, septic arthritis, or skin and soft tissue infection (SSTI). The diagnosis cohorts were defined based on ICD-10-CM discharge diagnoses adapted from previous studies (Appendix Table 1).^3,17-23 To define a cohort of generally healthy pediatric patients with an acute infection, we excluded patients hospitalized in the intensive care unit, patients with nonhome discharges, and patients with complex chronic conditions.²⁴ We also excluded hospitals with incomplete data during the study period (n=1). The Institutional Review Board at Cincinnati Children’s Hospital Medical Center determined this study to be non–human-subjects research.

Outcomes

The primary outcomes were the number of opportunity days and the percent of days with opportunity to transition from IV to enteral therapy. Opportunity days, or days in which there was a potential opportunity to transition from IV to enteral antibiotics, were defined as days patients received only IV antibiotic doses and at least one enteral nonantibiotic medication, suggesting an ability to take enteral medications.¹³ We excluded days patients received IV antibiotics for which there was no enteral alternative (eg, vancomycin, Appendix Table 2). When measuring opportunity, to be conservative (ie, to underestimate rather than overestimate opportunity), we did not count as an opportunity day any day in which patients received both IV and enteral antibiotics. Percent opportunity, or the percent of days patients received antibiotics in which there was potential opportunity to transition from IV to enteral antibiotics, was defined as the number of opportunity days divided by number of inpatient days patients received enteral antibiotics or IV antibiotics with at least one enteral nonantibiotic medication (antibiotic days). Similar to opportunity days, antibiotic days excluded days patients were on IV antibiotics for which there was no enteral alternative. Based on our definition, a lower percent opportunity indicates that a hospital is using enteral antibiotics earlier during the hospitalization (earlier transition), while a higher percent opportunity represents later enteral antibiotic use (later transition).

Statistical Analysis

Demographic and clinical characteristics were summarized by diagnosis with descriptive statistics, including frequency with percentage, mean with standard deviation, and median with interquartile range (IQR). For each diagnosis, we evaluated aggregate opportunity days (sum of opportunity days among all hospitals), opportunity days per encounter, and aggregate percent opportunity using frequencies, mean with standard deviation, and percentages, respectively. We also calculated aggregate opportunity days for diagnosis-antibiotic combinations. To visually show variation in the percent opportunity across hospitals, we displayed the percent opportunity on a heat map, and evaluated percent opportunity across hospitals using chi-square tests. To compare the variability in the percent opportunity across and within hospitals, we used a generalized linear model with two fixed effects (hospital and diagnosis), and parsed the variability using the sum of squares. We performed a sensitivity analysis and excluded days that patients received antiemetic medications (eg, ondansetron, granisetron, prochlorperazine, promethazine), as these suggest potential intolerance of enteral medications. All statistical analyses were performed using SAS v.9.4 (SAS Institute Inc, Cary, North Carolina) and GraphPad Prism 8.0 (GraphPad Software Inc., San Diego, California), and P values < .05 were considered statistically significant.

During the 2-year study period, 100,103 hospitalizations met our inclusion criteria across 51 hospitals and seven diagnosis categories (Table 1). Diagnosis cohorts ranged in size from 1,462 encounters for septic arthritis to 35,665 encounters for neck infections. Overall, we identified 88,522 aggregate opportunity days on which there was an opportunity to switch from IV to enteral treatment in the majority of participants (percent opportunity, 57%).

Opportunity by Diagnosis

The number of opportunity days (aggregate and mean per encounter) and percent opportunity varied by diagnosis (Table 2). The aggregate number of opportunity days ranged from 3,693 in patients with septic arthritis to 25,359 in patients with SSTI, and mean opportunity days per encounter ranged from 0.9 in CAP to 2.8 in septic arthritis. Percent opportunity was highest for septic arthritis at 72.7% and lowest for CAP at 39.7%.

Variation in Opportunity Among Hospitals

The variation in the percent opportunity across hospitals was statistically significant for all diagnoses (Figure). Within hospitals, we observed similar practice patterns across diagnoses. For example, hospitals with a higher percent opportunity for one diagnosis tended to have higher percent opportunity for the other diagnoses (as noted in the top portion of the Figure), and those with lower percent opportunity for one diagnosis tended to also have lower percent opportunity for the other diagnoses studied (as noted in the bottom portion of the Figure). When evaluating variability in the percent opportunity, 45% of the variability was attributable to the hospital-effect and 35% to the diagnosis; the remainder was unexplained variability. Sensitivity analysis excluding days when patients received an antiemetic medication yielded no differences in our results.

Opportunity by Antibiotic

The aggregate number of opportunity days varied by antibiotic (Table 3). Intravenous antibiotics with the largest number of opportunity days included clindamycin (44,293), ceftriaxone (23,896), and ampicillin-sulbactam (15,484). Antibiotic-diagnosis combinations with the largest number of opportunity days for each diagnosis included ceftriaxone and ampicillin in CAP; clindamycin in cellulitis, SSTI, and neck infections; ceftriaxone in UTI; and cefazolin in osteomyelitis and septic arthritis.

In this multicenter study of pediatric patients hospitalized with common bacterial infections, there was the potential to transition from IV to enteral treatment in over half of the antibiotic days. The degree of opportunity varied by infection, antibiotic, and hospital. Antibiotics with a large aggregate number of opportunity days for enteral transition included clindamycin, which has excellent bioavailability; and ampicillin and ampicillin-sulbactam, which can achieve pharmacodynamic targets with oral equivalents.^25-29 The across-hospital variation for a given diagnosis suggests that certain hospitals have strategies in place which permit an earlier transition to enteral antibiotics compared to other institutions in which there were likely missed opportunities to do so. This variability is likely due to limited evidence, emphasizing the need for robust studies to better understand the optimal initial antibiotic route and transition time. Our findings highlight the need for, and large potential impact of, stewardship efforts to promote earlier transition for specific drug targets. This study also demonstrates the feasibility of obtaining two metrics—percent opportunity and opportunity days—from administrative databases to inform stewardship efforts within and across hospitals.

Opportunity days and percent opportunity varied among diagnoses. The variation in aggregate opportunity days was largely a reflection of the number of encounters: Diagnoses such as SSTI, neck infections, and CAP had a large number of both aggregate opportunity days and encounters. The range of opportunity days per encounter (0.9-2.5) suggests potential missed opportunities to transition to enteral antibiotics across all diagnoses (Table 2). The higher opportunity days per encounter in osteomyelitis and septic arthritis may be related to longer LOS and higher percent opportunity. Percent opportunity likely varied among diagnoses due to differences in admission and discharge readiness criteria, diagnostic evaluation, frequency of antibiotic administration, and evidence on the optimal route of initial antibiotics and when to transition to oral formulations. For example, we hypothesize that certain diagnoses, such as osteomyelitis and septic arthritis, have admission and discharge readiness criteria directly tied to the perceived need for IV antibiotics, which may limit in-hospital days on enteral antibiotics and explain the high percent opportunity that we observed. The high percent opportunity seen in musculoskeletal infections also may be due to delays in initiating targeted treatment until culture results were available. Encounters for CAP had the lowest percent opportunity; we hypothesize that this is because admission and discharge readiness may be determined by factors other than the need for IV antibiotics (eg, need for supplemental oxygen), which may increase days on enteral antibiotics and lead to a lower percent opportunity.³⁰

Urinary tract infection encounters had a high percent opportunity. As with musculoskeletal infection, this may be related to delays in initiating targeted treatment until culture results became available. Another reason for the high percent opportunity in UTI could be the common use of ceftriaxone, which, dosed every 24 hours, likely reduced the opportunity to transition to enteral antibiotics. There is strong evidence demonstrating no difference in outcomes based on antibiotic routes for UTI, and we would expect this to result in a low percent opportunity.^2,31 While the observed high opportunity in UTI may relate to an initial unknown diagnosis or concern for systemic infection, this highlights potential opportunities for quality improvement initiatives to promote empiric oral antibiotics in clinically stable patients hospitalized with suspected UTI.

There was substantial variation in percent opportunity across hospitals for a given diagnosis, with less variation across diagnoses for a given hospital. Variation across hospitals but consistency within individual hospitals suggests that some hospitals may promote earlier transition from IV to enteral antibiotics as standard practice for all diagnoses, while other hospitals continue IV antibiotics for the entire hospitalization, highlighting potential missed opportunities at some institutions. While emerging data suggest that traditional long durations of IV antibiotics are not necessary for many infections, the limited evidence on the optimal time to switch to oral antibiotics may have influenced this variation.^2-7 Many guidelines recommend initial IV antibiotics for hospitalized pediatric patients, but there are few studies comparing IV and enteral therapy.^2,5,9 Limited evidence leaves significant room for hospital culture, antibiotic stewardship efforts, reimbursement considerations, and/or hospital workflow to influence transition timing and overall opportunity at individual hospitals.^7,8,32-34 These findings emphasize the importance of research to identify optimal transition time and comparative effectiveness studies to evaluate whether initial IV antibiotics are truly needed for mild—and even severe—disease presentations. Since many patients are admitted for the perceived need for IV antibiotics, earlier use of enteral antibiotics could reduce rates of hospitalizations, LOS, healthcare costs, and resource utilization.

Antibiotics with a high number of opportunity days included clindamycin, ceftriaxone, ampicillin-sublactam, and ampicillin. Our findings are consistent with another study which found that most bioavailable drugs, including clindamycin, were administered via the IV route and accounted for a large number of antibiotic days.³⁵ The Infectious Diseases Society of America recommends that hospitals promote earlier transition to oral formulations for highly bioavailable drugs.⁷ Given the high bioavailability of clindamycin, its common use in high-frequency encounters such as SSTI and neck infections, and the fact that it accounted for a large number of opportunity days, quality improvement initiatives promoting earlier transition to oral clindamycin could have a large impact across health systems.^25,26 Additionally, although beta-lactam antibiotics such as amoxicillin and amoxicillin-sulbactam are not highly bioavailable, oral dosing can achieve sufficient serum concentrations to reach pharmacodynamic targets for common clinical indications; this could be an important quality improvement initiative.^27-29 Several single-site studies have successfully implemented quality improvement initiatives to promote earlier IV-to-enteral transition, with resulting reductions in costs and no adverse events noted, highlighting the feasibility and impact of such efforts.^13,36-38

This study also demonstrates the feasibility of collecting two metrics (percent opportunity and opportunity days) from administrative databases to inform IV-to-oral transition benchmarking and stewardship efforts. While there are several metrics in the literature for evaluating antibiotic transition (eg, days of IV or oral therapy, percentage of antibiotics given via the oral route, time to switch from IV to oral, and acceptance rate of suggested changes to antibiotic route), none are universally used or agreed upon.^15,16,39 The opportunity metrics used in this study have several strengths, including the feasibility of obtaining them from existing databases and the ability to account for intake of other enteral medications; the latter is not evaluated in other metrics. These opportunity metrics can be used together to identify the percent of time in which there is opportunity to transition and total number of days to understand the full extent of potential opportunity for future interventions. As demonstrated in this study, these metrics can be measured by diagnosis, antibiotic, or diagnosis-antibiotic combination, and they can be used to evaluate stewardship efforts at a single institution over time or compare efforts across hospitals.

These findings should be interpreted in the context of important limitations. First, we attempted to characterize potential opportunity to transition to enteral medications based on a patient’s ability to tolerate nonenteral medications. However, there are other factors that could limit the opportunity to transition that we could not account for with an administrative dataset, including the use of antibiotics prior to admission, disease progression, severity of illness, and malabsorptive concerns. Thus, though we may have overestimated the true opportunity to transition to enteral antibiotics, it is unlikely that this would account for all of the variation in transition times that we observed across hospitals. Second, while our study required patients to have one of seven types of infection, we did not exclude any additional infectious diagnoses (eg, concurrent bacteremia, Clostridioides difficile, otitis media) that could have driven the choice of antibiotic type and modality. Although emerging evidence is supporting earlier transitions to oral therapy, bacteremia is typically treated with IV antibiotics; this may have led to an overestimation of true opportunity.⁴⁰ “Clostridioides” difficile and otitis media are typically treated with enteral therapy; concurrent infections such as these may have led to an underestimation of opportunity given the fact that, based on our definition, the days on which patients received both IV and enteral antibiotics were not counted as opportunity days. Third, because PHIS uses billing days to capture medication use, we were unable to distinguish transitions that occurred early in the day vs those that took place later in the day. This could have led to an underestimation of percent opportunity, particularly for diagnoses with a short LOS; it also likely led to an underestimation of the variability observed across hospitals. Fourth, because we used an administrative dataset, we are unable to understand reasoning behind transitioning time from IV to oral antibiotics, as well as provider, patient, and institutional level factors that influenced these decisions.

Children hospitalized with bacterial infections often receive IV antibiotics, and the timing of transition from IV to enteral antibiotics varies significantly across hospitals. Further research is needed to compare the effectiveness of IV and enteral antibiotics and better define criteria for transition to enteral therapy. We identified ample opportunities for quality improvement initiatives to promote earlier transition, which have the potential to reduce healthcare utilization and promote optimal patient-directed high-value care.

For severe asthma exacerbations unresponsive to initial treatment, expert consensus guidelines from 2007 recommend consideration for adjunct treatments (magnesium or heliox) to decrease the likelihood of intubation.¹ Over the last decade, data have emerged suggesting that intravenous (IV) magnesium may be more effective for reduction of hospital admission rates.² Pooled meta-analyses have demonstrated improved pulmonary function and reduction of hospital admission by as much as 68% in children when IV magnesium is administered in the emergency department (ED), although the evidence is extremely limited because of a small number of studies (three) and small sample size (115 children).^2-5

Though these data suggest that use of IV magnesium may reduce admission rates, a study of pediatric emergency medicine (PEM) physicians in US and Canada reported reluctance regarding use for this purpose. While PEM physicians reported awareness of the literature on admission prevention, they estimated that fewer than 5% of their patients receiving IV magnesium were discharged home.⁶ Their practice was generally limited to using IV magnesium in children with impending respiratory failure for the purpose of reducing intensive care unit (ICU) admission and not hospitalization.⁶ PEM physicians’ reluctance to use IV magnesium was related to the lack of strong available evidence supporting the impact of IV magnesium on outcomes, such as admission, and gaps in the literature about its dosing and safety profile.

The goal of this study was to assess the prevailing trends in IV magnesium use across US children’s hospitals and to assess the relationship of IV magnesium use to admission rate, length of stay (LOS), readmission rate, and ICU admission rate. We hypothesized that IV magnesium use might have increased following publication of studies demonstrating an association between IV magnesium use and fewer admissions.

Study Design, Setting, and Participants

This is a retrospective cohort study of asthma (All Patient Refined Diagnosis Related Group 141) hospitalizations for patients less than 18 years old presenting to 35 tertiary care children’s hospitals from January 1, 2010, to December 31, 2017, included in the Pediatric Health Information System (PHIS; Children’s Hospital Association, Lenexa, Kansas) database. The PHIS database is an administrative database that contains demographics, International Classification of Diseases 9th and 10th Revision diagnoses and procedures, and daily billing records for all inpatient, observation, ED, and ambulatory surgery encounters. All data were deidentified prior to inclusion in the database and tracking of patients across ED and inpatient visits was achieved through an encrypted and unique patient identifier. Children transferred from other hospitals were excluded because we could not verify IV magnesium use prior to transfer. For hospitals to be included, they were required to provide continuous data throughout the study period.

Main Outcome Measure

The main outcome was exposure to IV magnesium as determined by billing information available in the PHIS database.

Patient Demographics

We assessed patients’ demographic characteristics, including age (younger than 5 years, 5-11 years, and 12-17 years), sex, race/ethnicity, and insurance status.

Healthcare Utilization and Hospital Characteristics

We assessed healthcare utilization using geometric mean LOS, proportion of patients admitted to the hospital and to the ICU, and the proportion of patients with a 7-day all-cause readmission. In addition, we divided hospitals into three equal groups based on their annual inpatient asthma volume (<300, 300-850, >850 cases per year).

Statistical Analysis

We compared demographic and clinical characteristics across patients receiving IV magnesium with those who did not receive it with use of chi-square tests for categorical variables and Wilcoxon rank sum test for continuous variables. We calculated annual IV magnesium use rates for each hospital and modeled the average annual rate with a general linear model in order to assess change over time. We used Pearson product moment correlation to compare the annual proportion of magnesium use and healthcare utilization measures, including geometric mean LOS, the proportion of patients using the inpatient wards or the ICU, and the proportion of cases with a 7-day all-cause readmission. Geometric mean LOS was used to normalize the compounding effect of non–normally distributed arithmetic mean LOS. A sensitivity analysis was performed stratifying IV magnesium use over time by hospital inpatient volume. Data were analyzed using SAS version 9.4 (SAS Institute, Cary, North Carolina), and P values < .05 were considered statistically significant.

Study Population

A total of 878,188 encounters with acute asthma exacerbation met the inclusion criteria, with 65,558 (7.5%) receiving IV magnesium (Table). Of those receiving IV magnesium, 90% were admitted to the hospital. There were statistically significant differences in IV magnesium use when compared by age, race/ethnicity, and payer type, but not gender. IV magnesium use was significantly associated with older age (more than 5 years old), non-Hispanic black race, ED visit in the year prior to admission, longer hospital LOS, and higher ICU admission rate.

Trends in Intravenous Magnesium Use

IV magnesium use among hospitalized children more than doubled from 2010 to 2017 (17% vs 36%). Low-volume hospitals had a lower frequency of IV magnesium use, compared with the moderate- and high-volume hospitals. The growth rate per year of IV magnesium use was greater in high- and moderate-volume hospitals (3.4% and 2.9% per year, respectively), compared with the low-volume hospitals (1.2% per year; P = .04).

Trends in Intravenous Magnesium Use and Hospital Outcomes

The trend in IV magnesium use was not associated with a statistically significant change in the inpatient and ICU admission rate or in the 7-day all-cause readmission rate (Figure and Appendix Figure). Although IV magnesium use increased over time, LOS decreased significantly during the same period (1.6 days in 2010 vs 1.4 days in 2017; P < .001). When analyzed by hospital volume, no significant associations were found in the inpatient admission, ICU admission, and 7-day readmission rate.

The use of IV magnesium has significantly increased in US children’s hospitals over the last 8 years, especially among those hospitalized following an ED evaluation. Over this interval, trends in inpatient and ICU admission rate, as well as 7-day all-cause readmission rate, for asthma did not change, while LOS decreased. These findings contrast with a recent Cochrane review that summarized the efficacy of IV magnesium for reducing admission rate in few small trials.²

Our study findings are more consistent with prior survey findings that IV magnesium does not reduce hospitalization and that ED physicians tend to use IV magnesium in severe asthma exacerbation for its potential therapeutic benefits because of bronchodilator and anti-inflammatory effect.^6,7 Similar to PEM physicians’ estimates, only 10% of patients receiving IV magnesium were discharged home in our study.

IV magnesium use is higher in high-volume hospitals than in moderate- and low-volume ones. One potential explanation is that high- and moderate-volume hospitals may see a higher volume of children with severe or impending respiratory failure and, therefore, are more likely to use IV magnesium than the low-volume hospitals are. Alternatively, physician adoption of magnesium use for lower-acuity asthma exacerbations could vary by hospital volume.

Trend analyses of outcomes suggest that increase in IV magnesium use was not associated with an increase in inpatient and ICU admission rate or with 7-day all-cause readmission rate, although LOS reduced. LOS might be reduced because of various quality improvement initiatives (eg, discharging patients after every 3 hours albuterol treatments or respiratory therapist–driven protocols) and might not be related to IV magnesium use.^8,9 To this point, a recent study of a respiratory assessment score–matched cohort did not find any therapeutic benefit of IV magnesium with severe asthma exacerbation when receiving continuous albuterol therapy on a pediatric ward.⁵ Perhaps future studies could explore estimating the outcome by performing comparative effectiveness studies between those with severe asthma exacerbation who did or did not receive IV magnesium. Additionally, randomized controlled trials comparing IV magnesium and standard therapy and its effects on outcomes, such as hospitalization, LOS, association with asthma chronicity, and previous oral steroid use, might provide further insight to inform clinical practice.

Certain study limitations should be noted. The study cohort included children’s hospitals only, and it is possible that care at nonchildren’s hospitals for asthma differs. PHIS dataset used in this study does not allow determination of where and when IV magnesium was given, the severity of asthma exacerbation, or the chronicity of baseline disease. Moreover, PHIS hospitals include centers in large cities, and other competing children’s hospitals may provide other tertiary care that could affect the readmission data calculation. Lastly, the temporal associations between IV magnesium use and outcomes reported in this study should not be used as evidence or lack of evidence for the effectiveness of magnesium given the limitations of the observational study design and dataset used.

In conclusion, IV magnesium use in management of asthma exacerbation in children across the United States has significantly increased. The increase occurred disproportionately in high-volume hospitals and was not associated with changes in admission rate, ICU admission rate, or 7-day all-cause readmission rate, although LOS has decreased over time.

The authors have no financial relationships relevant to this article or conflicts of interest to disclose. 

This paper was a platform presentation at annual meetings of Pediatric Academic Societies 2019; accepted for presentation at annual meeting of Pediatric Hospital Medicine, July 2019.

No funding was secured for this study. 

For severe asthma exacerbations unresponsive to initial treatment, expert consensus guidelines from 2007 recommend consideration for adjunct treatments (magnesium or heliox) to decrease the likelihood of intubation.¹ Over the last decade, data have emerged suggesting that intravenous (IV) magnesium may be more effective for reduction of hospital admission rates.² Pooled meta-analyses have demonstrated improved pulmonary function and reduction of hospital admission by as much as 68% in children when IV magnesium is administered in the emergency department (ED), although the evidence is extremely limited because of a small number of studies (three) and small sample size (115 children).^2-5

Though these data suggest that use of IV magnesium may reduce admission rates, a study of pediatric emergency medicine (PEM) physicians in US and Canada reported reluctance regarding use for this purpose. While PEM physicians reported awareness of the literature on admission prevention, they estimated that fewer than 5% of their patients receiving IV magnesium were discharged home.⁶ Their practice was generally limited to using IV magnesium in children with impending respiratory failure for the purpose of reducing intensive care unit (ICU) admission and not hospitalization.⁶ PEM physicians’ reluctance to use IV magnesium was related to the lack of strong available evidence supporting the impact of IV magnesium on outcomes, such as admission, and gaps in the literature about its dosing and safety profile.

The goal of this study was to assess the prevailing trends in IV magnesium use across US children’s hospitals and to assess the relationship of IV magnesium use to admission rate, length of stay (LOS), readmission rate, and ICU admission rate. We hypothesized that IV magnesium use might have increased following publication of studies demonstrating an association between IV magnesium use and fewer admissions.

Study Design, Setting, and Participants

This is a retrospective cohort study of asthma (All Patient Refined Diagnosis Related Group 141) hospitalizations for patients less than 18 years old presenting to 35 tertiary care children’s hospitals from January 1, 2010, to December 31, 2017, included in the Pediatric Health Information System (PHIS; Children’s Hospital Association, Lenexa, Kansas) database. The PHIS database is an administrative database that contains demographics, International Classification of Diseases 9th and 10th Revision diagnoses and procedures, and daily billing records for all inpatient, observation, ED, and ambulatory surgery encounters. All data were deidentified prior to inclusion in the database and tracking of patients across ED and inpatient visits was achieved through an encrypted and unique patient identifier. Children transferred from other hospitals were excluded because we could not verify IV magnesium use prior to transfer. For hospitals to be included, they were required to provide continuous data throughout the study period.

Main Outcome Measure

The main outcome was exposure to IV magnesium as determined by billing information available in the PHIS database.

Patient Demographics

We assessed patients’ demographic characteristics, including age (younger than 5 years, 5-11 years, and 12-17 years), sex, race/ethnicity, and insurance status.

Healthcare Utilization and Hospital Characteristics

We assessed healthcare utilization using geometric mean LOS, proportion of patients admitted to the hospital and to the ICU, and the proportion of patients with a 7-day all-cause readmission. In addition, we divided hospitals into three equal groups based on their annual inpatient asthma volume (<300, 300-850, >850 cases per year).

Statistical Analysis

We compared demographic and clinical characteristics across patients receiving IV magnesium with those who did not receive it with use of chi-square tests for categorical variables and Wilcoxon rank sum test for continuous variables. We calculated annual IV magnesium use rates for each hospital and modeled the average annual rate with a general linear model in order to assess change over time. We used Pearson product moment correlation to compare the annual proportion of magnesium use and healthcare utilization measures, including geometric mean LOS, the proportion of patients using the inpatient wards or the ICU, and the proportion of cases with a 7-day all-cause readmission. Geometric mean LOS was used to normalize the compounding effect of non–normally distributed arithmetic mean LOS. A sensitivity analysis was performed stratifying IV magnesium use over time by hospital inpatient volume. Data were analyzed using SAS version 9.4 (SAS Institute, Cary, North Carolina), and P values < .05 were considered statistically significant.

Study Population

A total of 878,188 encounters with acute asthma exacerbation met the inclusion criteria, with 65,558 (7.5%) receiving IV magnesium (Table). Of those receiving IV magnesium, 90% were admitted to the hospital. There were statistically significant differences in IV magnesium use when compared by age, race/ethnicity, and payer type, but not gender. IV magnesium use was significantly associated with older age (more than 5 years old), non-Hispanic black race, ED visit in the year prior to admission, longer hospital LOS, and higher ICU admission rate.

Trends in Intravenous Magnesium Use

IV magnesium use among hospitalized children more than doubled from 2010 to 2017 (17% vs 36%). Low-volume hospitals had a lower frequency of IV magnesium use, compared with the moderate- and high-volume hospitals. The growth rate per year of IV magnesium use was greater in high- and moderate-volume hospitals (3.4% and 2.9% per year, respectively), compared with the low-volume hospitals (1.2% per year; P = .04).

Trends in Intravenous Magnesium Use and Hospital Outcomes

The trend in IV magnesium use was not associated with a statistically significant change in the inpatient and ICU admission rate or in the 7-day all-cause readmission rate (Figure and Appendix Figure). Although IV magnesium use increased over time, LOS decreased significantly during the same period (1.6 days in 2010 vs 1.4 days in 2017; P < .001). When analyzed by hospital volume, no significant associations were found in the inpatient admission, ICU admission, and 7-day readmission rate.

The use of IV magnesium has significantly increased in US children’s hospitals over the last 8 years, especially among those hospitalized following an ED evaluation. Over this interval, trends in inpatient and ICU admission rate, as well as 7-day all-cause readmission rate, for asthma did not change, while LOS decreased. These findings contrast with a recent Cochrane review that summarized the efficacy of IV magnesium for reducing admission rate in few small trials.²

Our study findings are more consistent with prior survey findings that IV magnesium does not reduce hospitalization and that ED physicians tend to use IV magnesium in severe asthma exacerbation for its potential therapeutic benefits because of bronchodilator and anti-inflammatory effect.^6,7 Similar to PEM physicians’ estimates, only 10% of patients receiving IV magnesium were discharged home in our study.

IV magnesium use is higher in high-volume hospitals than in moderate- and low-volume ones. One potential explanation is that high- and moderate-volume hospitals may see a higher volume of children with severe or impending respiratory failure and, therefore, are more likely to use IV magnesium than the low-volume hospitals are. Alternatively, physician adoption of magnesium use for lower-acuity asthma exacerbations could vary by hospital volume.

Trend analyses of outcomes suggest that increase in IV magnesium use was not associated with an increase in inpatient and ICU admission rate or with 7-day all-cause readmission rate, although LOS reduced. LOS might be reduced because of various quality improvement initiatives (eg, discharging patients after every 3 hours albuterol treatments or respiratory therapist–driven protocols) and might not be related to IV magnesium use.^8,9 To this point, a recent study of a respiratory assessment score–matched cohort did not find any therapeutic benefit of IV magnesium with severe asthma exacerbation when receiving continuous albuterol therapy on a pediatric ward.⁵ Perhaps future studies could explore estimating the outcome by performing comparative effectiveness studies between those with severe asthma exacerbation who did or did not receive IV magnesium. Additionally, randomized controlled trials comparing IV magnesium and standard therapy and its effects on outcomes, such as hospitalization, LOS, association with asthma chronicity, and previous oral steroid use, might provide further insight to inform clinical practice.

Certain study limitations should be noted. The study cohort included children’s hospitals only, and it is possible that care at nonchildren’s hospitals for asthma differs. PHIS dataset used in this study does not allow determination of where and when IV magnesium was given, the severity of asthma exacerbation, or the chronicity of baseline disease. Moreover, PHIS hospitals include centers in large cities, and other competing children’s hospitals may provide other tertiary care that could affect the readmission data calculation. Lastly, the temporal associations between IV magnesium use and outcomes reported in this study should not be used as evidence or lack of evidence for the effectiveness of magnesium given the limitations of the observational study design and dataset used.

In conclusion, IV magnesium use in management of asthma exacerbation in children across the United States has significantly increased. The increase occurred disproportionately in high-volume hospitals and was not associated with changes in admission rate, ICU admission rate, or 7-day all-cause readmission rate, although LOS has decreased over time.

The authors have no financial relationships relevant to this article or conflicts of interest to disclose. 

This paper was a platform presentation at annual meetings of Pediatric Academic Societies 2019; accepted for presentation at annual meeting of Pediatric Hospital Medicine, July 2019.

No funding was secured for this study. 

For severe asthma exacerbations unresponsive to initial treatment, expert consensus guidelines from 2007 recommend consideration for adjunct treatments (magnesium or heliox) to decrease the likelihood of intubation.¹ Over the last decade, data have emerged suggesting that intravenous (IV) magnesium may be more effective for reduction of hospital admission rates.² Pooled meta-analyses have demonstrated improved pulmonary function and reduction of hospital admission by as much as 68% in children when IV magnesium is administered in the emergency department (ED), although the evidence is extremely limited because of a small number of studies (three) and small sample size (115 children).^2-5

Though these data suggest that use of IV magnesium may reduce admission rates, a study of pediatric emergency medicine (PEM) physicians in US and Canada reported reluctance regarding use for this purpose. While PEM physicians reported awareness of the literature on admission prevention, they estimated that fewer than 5% of their patients receiving IV magnesium were discharged home.⁶ Their practice was generally limited to using IV magnesium in children with impending respiratory failure for the purpose of reducing intensive care unit (ICU) admission and not hospitalization.⁶ PEM physicians’ reluctance to use IV magnesium was related to the lack of strong available evidence supporting the impact of IV magnesium on outcomes, such as admission, and gaps in the literature about its dosing and safety profile.

The goal of this study was to assess the prevailing trends in IV magnesium use across US children’s hospitals and to assess the relationship of IV magnesium use to admission rate, length of stay (LOS), readmission rate, and ICU admission rate. We hypothesized that IV magnesium use might have increased following publication of studies demonstrating an association between IV magnesium use and fewer admissions.

Study Design, Setting, and Participants

This is a retrospective cohort study of asthma (All Patient Refined Diagnosis Related Group 141) hospitalizations for patients less than 18 years old presenting to 35 tertiary care children’s hospitals from January 1, 2010, to December 31, 2017, included in the Pediatric Health Information System (PHIS; Children’s Hospital Association, Lenexa, Kansas) database. The PHIS database is an administrative database that contains demographics, International Classification of Diseases 9th and 10th Revision diagnoses and procedures, and daily billing records for all inpatient, observation, ED, and ambulatory surgery encounters. All data were deidentified prior to inclusion in the database and tracking of patients across ED and inpatient visits was achieved through an encrypted and unique patient identifier. Children transferred from other hospitals were excluded because we could not verify IV magnesium use prior to transfer. For hospitals to be included, they were required to provide continuous data throughout the study period.

Main Outcome Measure

The main outcome was exposure to IV magnesium as determined by billing information available in the PHIS database.

Patient Demographics

We assessed patients’ demographic characteristics, including age (younger than 5 years, 5-11 years, and 12-17 years), sex, race/ethnicity, and insurance status.

Healthcare Utilization and Hospital Characteristics

We assessed healthcare utilization using geometric mean LOS, proportion of patients admitted to the hospital and to the ICU, and the proportion of patients with a 7-day all-cause readmission. In addition, we divided hospitals into three equal groups based on their annual inpatient asthma volume (<300, 300-850, >850 cases per year).

Statistical Analysis

We compared demographic and clinical characteristics across patients receiving IV magnesium with those who did not receive it with use of chi-square tests for categorical variables and Wilcoxon rank sum test for continuous variables. We calculated annual IV magnesium use rates for each hospital and modeled the average annual rate with a general linear model in order to assess change over time. We used Pearson product moment correlation to compare the annual proportion of magnesium use and healthcare utilization measures, including geometric mean LOS, the proportion of patients using the inpatient wards or the ICU, and the proportion of cases with a 7-day all-cause readmission. Geometric mean LOS was used to normalize the compounding effect of non–normally distributed arithmetic mean LOS. A sensitivity analysis was performed stratifying IV magnesium use over time by hospital inpatient volume. Data were analyzed using SAS version 9.4 (SAS Institute, Cary, North Carolina), and P values < .05 were considered statistically significant.

Study Population

A total of 878,188 encounters with acute asthma exacerbation met the inclusion criteria, with 65,558 (7.5%) receiving IV magnesium (Table). Of those receiving IV magnesium, 90% were admitted to the hospital. There were statistically significant differences in IV magnesium use when compared by age, race/ethnicity, and payer type, but not gender. IV magnesium use was significantly associated with older age (more than 5 years old), non-Hispanic black race, ED visit in the year prior to admission, longer hospital LOS, and higher ICU admission rate.

Trends in Intravenous Magnesium Use

IV magnesium use among hospitalized children more than doubled from 2010 to 2017 (17% vs 36%). Low-volume hospitals had a lower frequency of IV magnesium use, compared with the moderate- and high-volume hospitals. The growth rate per year of IV magnesium use was greater in high- and moderate-volume hospitals (3.4% and 2.9% per year, respectively), compared with the low-volume hospitals (1.2% per year; P = .04).

Trends in Intravenous Magnesium Use and Hospital Outcomes

The trend in IV magnesium use was not associated with a statistically significant change in the inpatient and ICU admission rate or in the 7-day all-cause readmission rate (Figure and Appendix Figure). Although IV magnesium use increased over time, LOS decreased significantly during the same period (1.6 days in 2010 vs 1.4 days in 2017; P < .001). When analyzed by hospital volume, no significant associations were found in the inpatient admission, ICU admission, and 7-day readmission rate.

The use of IV magnesium has significantly increased in US children’s hospitals over the last 8 years, especially among those hospitalized following an ED evaluation. Over this interval, trends in inpatient and ICU admission rate, as well as 7-day all-cause readmission rate, for asthma did not change, while LOS decreased. These findings contrast with a recent Cochrane review that summarized the efficacy of IV magnesium for reducing admission rate in few small trials.²

Our study findings are more consistent with prior survey findings that IV magnesium does not reduce hospitalization and that ED physicians tend to use IV magnesium in severe asthma exacerbation for its potential therapeutic benefits because of bronchodilator and anti-inflammatory effect.^6,7 Similar to PEM physicians’ estimates, only 10% of patients receiving IV magnesium were discharged home in our study.

IV magnesium use is higher in high-volume hospitals than in moderate- and low-volume ones. One potential explanation is that high- and moderate-volume hospitals may see a higher volume of children with severe or impending respiratory failure and, therefore, are more likely to use IV magnesium than the low-volume hospitals are. Alternatively, physician adoption of magnesium use for lower-acuity asthma exacerbations could vary by hospital volume.

Trend analyses of outcomes suggest that increase in IV magnesium use was not associated with an increase in inpatient and ICU admission rate or with 7-day all-cause readmission rate, although LOS reduced. LOS might be reduced because of various quality improvement initiatives (eg, discharging patients after every 3 hours albuterol treatments or respiratory therapist–driven protocols) and might not be related to IV magnesium use.^8,9 To this point, a recent study of a respiratory assessment score–matched cohort did not find any therapeutic benefit of IV magnesium with severe asthma exacerbation when receiving continuous albuterol therapy on a pediatric ward.⁵ Perhaps future studies could explore estimating the outcome by performing comparative effectiveness studies between those with severe asthma exacerbation who did or did not receive IV magnesium. Additionally, randomized controlled trials comparing IV magnesium and standard therapy and its effects on outcomes, such as hospitalization, LOS, association with asthma chronicity, and previous oral steroid use, might provide further insight to inform clinical practice.

Certain study limitations should be noted. The study cohort included children’s hospitals only, and it is possible that care at nonchildren’s hospitals for asthma differs. PHIS dataset used in this study does not allow determination of where and when IV magnesium was given, the severity of asthma exacerbation, or the chronicity of baseline disease. Moreover, PHIS hospitals include centers in large cities, and other competing children’s hospitals may provide other tertiary care that could affect the readmission data calculation. Lastly, the temporal associations between IV magnesium use and outcomes reported in this study should not be used as evidence or lack of evidence for the effectiveness of magnesium given the limitations of the observational study design and dataset used.

In conclusion, IV magnesium use in management of asthma exacerbation in children across the United States has significantly increased. The increase occurred disproportionately in high-volume hospitals and was not associated with changes in admission rate, ICU admission rate, or 7-day all-cause readmission rate, although LOS has decreased over time.

The authors have no financial relationships relevant to this article or conflicts of interest to disclose. 

This paper was a platform presentation at annual meetings of Pediatric Academic Societies 2019; accepted for presentation at annual meeting of Pediatric Hospital Medicine, July 2019.

No funding was secured for this study. 

Increasing numbers of children and adolescents are presenting to children’s hospitals with acute mental health crises requiring emergent or inpatient treatment.^1-5 As a result, children’s hospitals are experiencing additional financial challenges because specialty mental health services are often reimbursed at lower rates than other medical services.^6-9 Poor reimbursement has also been cited as a deterrent to the provision of mental health specialty care, including emergency mental health crisis services.¹⁰ The cumulative financial impact of recent trends in the provision of mental health crisis services at children’s hospitals, however, is unknown. We conducted this study to assess children’s hospitals’ costs, reimbursement, and net profits or losses when delivering inpatient mental health care.

We conducted a retrospective cohort study of the Children’s Hospital Association’s Pediatric Health Information System (PHIS) and Revenue Management Program (RMP) databases. PHIS is an administrative and billing database that collects International Classification of Disease, 10th Revision (ICD-10) diagnoses, procedure codes, and hospital charges from encounters at 52 US children’s hospitals. Costs are estimated from charges using hospital-, year-, and department-specific cost-to-charge ratios. The RMP database is an add-on module to the PHIS database that captures reimbursement data submitted quarterly from 17 participating hospitals based on actual reimbursement amounts collected for each encounter.

Among the 17 participating hospitals, we included all medical (ie, not surgical or intensive care) encounters during calendar year 2017 for children older than 6 years. We stratified encounters into three diagnosis types: primary mental health diagnosis,⁵ suicide attempt,¹¹ or other medical hospitalizations. We separated suicide attempts since these encounters often require care for both mental health concerns and medical complications. Eating disorders were excluded because these programs at children’s hospitals primarily focus on medical complications, require complex multispecialty support, have significantly longer hospitalizations and made up a small volume of overall mental health hospitalizations.

We stratified all analyses by inpatient or observation encounter and determined the proportion of encounters and hospital days attributed to primary mental health, suicide attempt, and other medical conditions at each hospital. One of the 17 children’s hospitals does not use observation status billing, so the observation encounters dataset includes 16 hospitals.

We summarized patients’ demographic and clinical characteristics using frequencies and percentages, comparing across diagnosis groups using chi-square tests. We calculated mean cost per day as (total cost) ÷ (total length of stay [LOS]), reimbursement per day as (total reimbursement) ÷ (total LOS) for each hospital and patient group, and margin per day as (reimbursement per day) – (cost per day). We then determined the total margin difference of caring for mental health vs caring for other medical encounters as ([margin per day for mental health] – [margin per day other medical]) × (number of mental health days). Similarly, we calculated the total margin loss for suicide attempts vs other medical encounters. After calculating profits and losses at individual hospitals, we summed total annual profits and losses to calculate cumulative annual differences. We summarized these profits and losses across all hospitals with medians and interquartile ranges (IQR).

This study of deidentified administrative data was approved by the Internal Review Board at Vanderbilt University as non-human subjects research. All statistical analyses were performed using SAS v.9.4 (SAS Institute, Cary, North Carolina), and P values < .05 were considered statistically significant.

Study Population

Across the 17 included children’s hospitals, there were 8,521 (7.6%) mental health encounters, 3,247 (2.9%) suicide attempt encounters, and 99,937 (89.5%) other medical encounters. LOS was significantly longer for mental health hospitalizations than for suicide attempts and for other medical hospitalizations.

Hospital Characteristics

All 17 free-standing children’s hospitals in the study had an inpatient behavioral health/psychiatric consultation service, and 7 of the 17 had an inpatient behavioral health/psychiatric unit. The total number of discharges for mental health, suicide attempt, and other medical conditions per year varied (range, 2,868-13,214) across the hospitals.

Hospital Daily Profits and Losses for Mental Health, Suicide Attempt, and Other Medical Admissions

For inpatient status mental health hospitalizations, the median margin was $376/day (IQR, $23-$618). For inpatient status suicide attempt hospitalizations, the median margin was $685/day (IQR, $3-$1,117), and for other medical hospitalizations the median margin was $603/day (IQR, $240-$991). With regard to observation status admissions, mental health hospitalizations had a median margin of –$453/day (IQR, –$806 to $362), suicide attempts of –$103/day (IQR, –$639 to $264), and other medical conditions of $353/day (IQR, –$616 to $658; Figure).

Hospital Annual Profits and Losses for Mental Health and Suicide Attempt Admissions, Compared With Other Medical Admissions

The Table shows daily and annual profits and losses for inpatient and observation status. The total annual loss across all hospitals for mental health and suicide attempt hospitalizations, compared with other medical hospitalizations, including both inpatient and observation status, was –$26,658,255 when taking both profits and losses into account. For the seven hospitals with net profits for mental health and suicide attempt hospitalizations, compared with other medical hospitalizations, the median net profit for combined inpatient and observation status encounters was $119,361 (IQR, $82,818-$195,543), and the total net profit was $5,872,665. For the 10 hospitals with net losses for mental health and suicide attempt hospitalizations, compared with other medical hospitalizations, the median net loss for combined inpatient and observation status was –$2,169,357 (IQR, –$4,034,085 to –$511,755), and the total net loss was –$27,419,379.

Hospitalizations for mental health disorders and suicide attempts accounted for 10.5% of hospitalizations at 17 US children’s hospitals in 2017. Overall, mental health and suicide attempt hospitalizations had lower financial margins than did other medical hospitalizations, and they accounted for a total margin loss of more than $26 million across 17 hospitals. Seven hospitals generated a profit for mental health and suicide attempt admissions; 10 hospitals reported losses. Only three hospitals generated a higher net profit for mental health admissions than for other medical admissions. More hospitals had net profits for inpatient status mental health and suicide attempt admissions than for observation status mental health and suicide attempt admissions.

For a minority of children’s hospitals, mental health hospitalizations had higher profit margins than for other medical hospitalizations. This raises questions about patient outcomes and the type of care models employed. One potential explanation is that these hospitals have negotiated favorable agreements with payers. Another possibility could be variations in case-mix and payer mix. Certain mental health services, such as crisis response teams, social workers, and child life specialists, may also be funded from nonpayer sources, so estimates may not fully reflect the cost of providing mental health services. A worst-case view is that hospitals with higher profit margins are providing less or poorer care because of lower reimbursement.

Mental health and suicide attempt hospitalizations were associated with smaller margins but counterintuitively generally wider IQRs for cost. This might be related to variation in care models, but our study was not positioned to examine reasons for this variation. The relationship between reimbursement or margins and patient outcomes, as well as specific mechanisms which may drive costs and outcomes, are areas for future research.

Health insurance plays a crucial role in mental health care. In our study, hospitals were more likely to report positive margins from inpatient status mental health hospitalizations rather than from observation status ones. This is unsurprising because payments for observation status are generally lower than for inpatient status.¹² Less is known about what influences billing and payment for inpatient versus observation at individual hospitals, particularly for mental health hospitalizations. In many cases, billing status is not strictly under the hospital’s control and may be determined by payers during or after the hospitalization. Significant variability in the percentage of patients billed as observation status and the impact of lower, often negative, margins for observation mental health encounters, will have a disproportionate effect on some hospitals. Future work could investigate how these differences may influence overall costs and delivery of care.

This study has several limitations that deserve attention. Costs reported are based on cost to charge ratios, which may generate imperfect estimates. Data was limited to 17 freestanding children’s hospitals, and our findings may not generalize to other hospitals. We also compared mental health and suicide attempt hospitalizations with “other medical” hospitalizations. This broad group contains certain medical conditions that may have higher or lower profit margins than average, and estimates of the margins could be over- or underestimated. We assumed that mental health and suicide attempt admissions were displacing admissions with non–mental health medical conditions (ie, not an empty bed). If those beds would otherwise be unoccupied, raw margins are better estimates of the financial impact than margin differences between mental health/suicide attempt and other medical hospitalizations.

Children’s hospitals are more likely to have significantly lower financial margins for mental health and suicide attempt hospitalizations than for other medical hospitalizations. Future work to investigate how quality of care is associated with reimbursement can help ensure that funding for children’s acute mental health care services is commensurate with resources required to provide high quality services.

The authors had no financial relationships relevant to this article to disclose.

Research reported in this publication was supported by the National Institute of Mental Health of the National Institutes of Health under Award Number K23MH115162 (Doupnik).

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Increasing numbers of children and adolescents are presenting to children’s hospitals with acute mental health crises requiring emergent or inpatient treatment.^1-5 As a result, children’s hospitals are experiencing additional financial challenges because specialty mental health services are often reimbursed at lower rates than other medical services.^6-9 Poor reimbursement has also been cited as a deterrent to the provision of mental health specialty care, including emergency mental health crisis services.¹⁰ The cumulative financial impact of recent trends in the provision of mental health crisis services at children’s hospitals, however, is unknown. We conducted this study to assess children’s hospitals’ costs, reimbursement, and net profits or losses when delivering inpatient mental health care.

We conducted a retrospective cohort study of the Children’s Hospital Association’s Pediatric Health Information System (PHIS) and Revenue Management Program (RMP) databases. PHIS is an administrative and billing database that collects International Classification of Disease, 10th Revision (ICD-10) diagnoses, procedure codes, and hospital charges from encounters at 52 US children’s hospitals. Costs are estimated from charges using hospital-, year-, and department-specific cost-to-charge ratios. The RMP database is an add-on module to the PHIS database that captures reimbursement data submitted quarterly from 17 participating hospitals based on actual reimbursement amounts collected for each encounter.

Among the 17 participating hospitals, we included all medical (ie, not surgical or intensive care) encounters during calendar year 2017 for children older than 6 years. We stratified encounters into three diagnosis types: primary mental health diagnosis,⁵ suicide attempt,¹¹ or other medical hospitalizations. We separated suicide attempts since these encounters often require care for both mental health concerns and medical complications. Eating disorders were excluded because these programs at children’s hospitals primarily focus on medical complications, require complex multispecialty support, have significantly longer hospitalizations and made up a small volume of overall mental health hospitalizations.

We stratified all analyses by inpatient or observation encounter and determined the proportion of encounters and hospital days attributed to primary mental health, suicide attempt, and other medical conditions at each hospital. One of the 17 children’s hospitals does not use observation status billing, so the observation encounters dataset includes 16 hospitals.

We summarized patients’ demographic and clinical characteristics using frequencies and percentages, comparing across diagnosis groups using chi-square tests. We calculated mean cost per day as (total cost) ÷ (total length of stay [LOS]), reimbursement per day as (total reimbursement) ÷ (total LOS) for each hospital and patient group, and margin per day as (reimbursement per day) – (cost per day). We then determined the total margin difference of caring for mental health vs caring for other medical encounters as ([margin per day for mental health] – [margin per day other medical]) × (number of mental health days). Similarly, we calculated the total margin loss for suicide attempts vs other medical encounters. After calculating profits and losses at individual hospitals, we summed total annual profits and losses to calculate cumulative annual differences. We summarized these profits and losses across all hospitals with medians and interquartile ranges (IQR).

This study of deidentified administrative data was approved by the Internal Review Board at Vanderbilt University as non-human subjects research. All statistical analyses were performed using SAS v.9.4 (SAS Institute, Cary, North Carolina), and P values < .05 were considered statistically significant.

Study Population

Across the 17 included children’s hospitals, there were 8,521 (7.6%) mental health encounters, 3,247 (2.9%) suicide attempt encounters, and 99,937 (89.5%) other medical encounters. LOS was significantly longer for mental health hospitalizations than for suicide attempts and for other medical hospitalizations.

Hospital Characteristics

All 17 free-standing children’s hospitals in the study had an inpatient behavioral health/psychiatric consultation service, and 7 of the 17 had an inpatient behavioral health/psychiatric unit. The total number of discharges for mental health, suicide attempt, and other medical conditions per year varied (range, 2,868-13,214) across the hospitals.

Hospital Daily Profits and Losses for Mental Health, Suicide Attempt, and Other Medical Admissions

For inpatient status mental health hospitalizations, the median margin was $376/day (IQR, $23-$618). For inpatient status suicide attempt hospitalizations, the median margin was $685/day (IQR, $3-$1,117), and for other medical hospitalizations the median margin was $603/day (IQR, $240-$991). With regard to observation status admissions, mental health hospitalizations had a median margin of –$453/day (IQR, –$806 to $362), suicide attempts of –$103/day (IQR, –$639 to $264), and other medical conditions of $353/day (IQR, –$616 to $658; Figure).

Hospital Annual Profits and Losses for Mental Health and Suicide Attempt Admissions, Compared With Other Medical Admissions

The Table shows daily and annual profits and losses for inpatient and observation status. The total annual loss across all hospitals for mental health and suicide attempt hospitalizations, compared with other medical hospitalizations, including both inpatient and observation status, was –$26,658,255 when taking both profits and losses into account. For the seven hospitals with net profits for mental health and suicide attempt hospitalizations, compared with other medical hospitalizations, the median net profit for combined inpatient and observation status encounters was $119,361 (IQR, $82,818-$195,543), and the total net profit was $5,872,665. For the 10 hospitals with net losses for mental health and suicide attempt hospitalizations, compared with other medical hospitalizations, the median net loss for combined inpatient and observation status was –$2,169,357 (IQR, –$4,034,085 to –$511,755), and the total net loss was –$27,419,379.

Hospitalizations for mental health disorders and suicide attempts accounted for 10.5% of hospitalizations at 17 US children’s hospitals in 2017. Overall, mental health and suicide attempt hospitalizations had lower financial margins than did other medical hospitalizations, and they accounted for a total margin loss of more than $26 million across 17 hospitals. Seven hospitals generated a profit for mental health and suicide attempt admissions; 10 hospitals reported losses. Only three hospitals generated a higher net profit for mental health admissions than for other medical admissions. More hospitals had net profits for inpatient status mental health and suicide attempt admissions than for observation status mental health and suicide attempt admissions.

For a minority of children’s hospitals, mental health hospitalizations had higher profit margins than for other medical hospitalizations. This raises questions about patient outcomes and the type of care models employed. One potential explanation is that these hospitals have negotiated favorable agreements with payers. Another possibility could be variations in case-mix and payer mix. Certain mental health services, such as crisis response teams, social workers, and child life specialists, may also be funded from nonpayer sources, so estimates may not fully reflect the cost of providing mental health services. A worst-case view is that hospitals with higher profit margins are providing less or poorer care because of lower reimbursement.

Mental health and suicide attempt hospitalizations were associated with smaller margins but counterintuitively generally wider IQRs for cost. This might be related to variation in care models, but our study was not positioned to examine reasons for this variation. The relationship between reimbursement or margins and patient outcomes, as well as specific mechanisms which may drive costs and outcomes, are areas for future research.

Health insurance plays a crucial role in mental health care. In our study, hospitals were more likely to report positive margins from inpatient status mental health hospitalizations rather than from observation status ones. This is unsurprising because payments for observation status are generally lower than for inpatient status.¹² Less is known about what influences billing and payment for inpatient versus observation at individual hospitals, particularly for mental health hospitalizations. In many cases, billing status is not strictly under the hospital’s control and may be determined by payers during or after the hospitalization. Significant variability in the percentage of patients billed as observation status and the impact of lower, often negative, margins for observation mental health encounters, will have a disproportionate effect on some hospitals. Future work could investigate how these differences may influence overall costs and delivery of care.

This study has several limitations that deserve attention. Costs reported are based on cost to charge ratios, which may generate imperfect estimates. Data was limited to 17 freestanding children’s hospitals, and our findings may not generalize to other hospitals. We also compared mental health and suicide attempt hospitalizations with “other medical” hospitalizations. This broad group contains certain medical conditions that may have higher or lower profit margins than average, and estimates of the margins could be over- or underestimated. We assumed that mental health and suicide attempt admissions were displacing admissions with non–mental health medical conditions (ie, not an empty bed). If those beds would otherwise be unoccupied, raw margins are better estimates of the financial impact than margin differences between mental health/suicide attempt and other medical hospitalizations.

Children’s hospitals are more likely to have significantly lower financial margins for mental health and suicide attempt hospitalizations than for other medical hospitalizations. Future work to investigate how quality of care is associated with reimbursement can help ensure that funding for children’s acute mental health care services is commensurate with resources required to provide high quality services.

The authors had no financial relationships relevant to this article to disclose.

Research reported in this publication was supported by the National Institute of Mental Health of the National Institutes of Health under Award Number K23MH115162 (Doupnik).

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Increasing numbers of children and adolescents are presenting to children’s hospitals with acute mental health crises requiring emergent or inpatient treatment.^1-5 As a result, children’s hospitals are experiencing additional financial challenges because specialty mental health services are often reimbursed at lower rates than other medical services.^6-9 Poor reimbursement has also been cited as a deterrent to the provision of mental health specialty care, including emergency mental health crisis services.¹⁰ The cumulative financial impact of recent trends in the provision of mental health crisis services at children’s hospitals, however, is unknown. We conducted this study to assess children’s hospitals’ costs, reimbursement, and net profits or losses when delivering inpatient mental health care.

We conducted a retrospective cohort study of the Children’s Hospital Association’s Pediatric Health Information System (PHIS) and Revenue Management Program (RMP) databases. PHIS is an administrative and billing database that collects International Classification of Disease, 10th Revision (ICD-10) diagnoses, procedure codes, and hospital charges from encounters at 52 US children’s hospitals. Costs are estimated from charges using hospital-, year-, and department-specific cost-to-charge ratios. The RMP database is an add-on module to the PHIS database that captures reimbursement data submitted quarterly from 17 participating hospitals based on actual reimbursement amounts collected for each encounter.

Among the 17 participating hospitals, we included all medical (ie, not surgical or intensive care) encounters during calendar year 2017 for children older than 6 years. We stratified encounters into three diagnosis types: primary mental health diagnosis,⁵ suicide attempt,¹¹ or other medical hospitalizations. We separated suicide attempts since these encounters often require care for both mental health concerns and medical complications. Eating disorders were excluded because these programs at children’s hospitals primarily focus on medical complications, require complex multispecialty support, have significantly longer hospitalizations and made up a small volume of overall mental health hospitalizations.

We stratified all analyses by inpatient or observation encounter and determined the proportion of encounters and hospital days attributed to primary mental health, suicide attempt, and other medical conditions at each hospital. One of the 17 children’s hospitals does not use observation status billing, so the observation encounters dataset includes 16 hospitals.

We summarized patients’ demographic and clinical characteristics using frequencies and percentages, comparing across diagnosis groups using chi-square tests. We calculated mean cost per day as (total cost) ÷ (total length of stay [LOS]), reimbursement per day as (total reimbursement) ÷ (total LOS) for each hospital and patient group, and margin per day as (reimbursement per day) – (cost per day). We then determined the total margin difference of caring for mental health vs caring for other medical encounters as ([margin per day for mental health] – [margin per day other medical]) × (number of mental health days). Similarly, we calculated the total margin loss for suicide attempts vs other medical encounters. After calculating profits and losses at individual hospitals, we summed total annual profits and losses to calculate cumulative annual differences. We summarized these profits and losses across all hospitals with medians and interquartile ranges (IQR).

This study of deidentified administrative data was approved by the Internal Review Board at Vanderbilt University as non-human subjects research. All statistical analyses were performed using SAS v.9.4 (SAS Institute, Cary, North Carolina), and P values < .05 were considered statistically significant.

Study Population

Across the 17 included children’s hospitals, there were 8,521 (7.6%) mental health encounters, 3,247 (2.9%) suicide attempt encounters, and 99,937 (89.5%) other medical encounters. LOS was significantly longer for mental health hospitalizations than for suicide attempts and for other medical hospitalizations.

Hospital Characteristics

All 17 free-standing children’s hospitals in the study had an inpatient behavioral health/psychiatric consultation service, and 7 of the 17 had an inpatient behavioral health/psychiatric unit. The total number of discharges for mental health, suicide attempt, and other medical conditions per year varied (range, 2,868-13,214) across the hospitals.

Hospital Daily Profits and Losses for Mental Health, Suicide Attempt, and Other Medical Admissions

For inpatient status mental health hospitalizations, the median margin was $376/day (IQR, $23-$618). For inpatient status suicide attempt hospitalizations, the median margin was $685/day (IQR, $3-$1,117), and for other medical hospitalizations the median margin was $603/day (IQR, $240-$991). With regard to observation status admissions, mental health hospitalizations had a median margin of –$453/day (IQR, –$806 to $362), suicide attempts of –$103/day (IQR, –$639 to $264), and other medical conditions of $353/day (IQR, –$616 to $658; Figure).

Hospital Annual Profits and Losses for Mental Health and Suicide Attempt Admissions, Compared With Other Medical Admissions

The Table shows daily and annual profits and losses for inpatient and observation status. The total annual loss across all hospitals for mental health and suicide attempt hospitalizations, compared with other medical hospitalizations, including both inpatient and observation status, was –$26,658,255 when taking both profits and losses into account. For the seven hospitals with net profits for mental health and suicide attempt hospitalizations, compared with other medical hospitalizations, the median net profit for combined inpatient and observation status encounters was $119,361 (IQR, $82,818-$195,543), and the total net profit was $5,872,665. For the 10 hospitals with net losses for mental health and suicide attempt hospitalizations, compared with other medical hospitalizations, the median net loss for combined inpatient and observation status was –$2,169,357 (IQR, –$4,034,085 to –$511,755), and the total net loss was –$27,419,379.

Hospitalizations for mental health disorders and suicide attempts accounted for 10.5% of hospitalizations at 17 US children’s hospitals in 2017. Overall, mental health and suicide attempt hospitalizations had lower financial margins than did other medical hospitalizations, and they accounted for a total margin loss of more than $26 million across 17 hospitals. Seven hospitals generated a profit for mental health and suicide attempt admissions; 10 hospitals reported losses. Only three hospitals generated a higher net profit for mental health admissions than for other medical admissions. More hospitals had net profits for inpatient status mental health and suicide attempt admissions than for observation status mental health and suicide attempt admissions.

For a minority of children’s hospitals, mental health hospitalizations had higher profit margins than for other medical hospitalizations. This raises questions about patient outcomes and the type of care models employed. One potential explanation is that these hospitals have negotiated favorable agreements with payers. Another possibility could be variations in case-mix and payer mix. Certain mental health services, such as crisis response teams, social workers, and child life specialists, may also be funded from nonpayer sources, so estimates may not fully reflect the cost of providing mental health services. A worst-case view is that hospitals with higher profit margins are providing less or poorer care because of lower reimbursement.

Mental health and suicide attempt hospitalizations were associated with smaller margins but counterintuitively generally wider IQRs for cost. This might be related to variation in care models, but our study was not positioned to examine reasons for this variation. The relationship between reimbursement or margins and patient outcomes, as well as specific mechanisms which may drive costs and outcomes, are areas for future research.

Health insurance plays a crucial role in mental health care. In our study, hospitals were more likely to report positive margins from inpatient status mental health hospitalizations rather than from observation status ones. This is unsurprising because payments for observation status are generally lower than for inpatient status.¹² Less is known about what influences billing and payment for inpatient versus observation at individual hospitals, particularly for mental health hospitalizations. In many cases, billing status is not strictly under the hospital’s control and may be determined by payers during or after the hospitalization. Significant variability in the percentage of patients billed as observation status and the impact of lower, often negative, margins for observation mental health encounters, will have a disproportionate effect on some hospitals. Future work could investigate how these differences may influence overall costs and delivery of care.

This study has several limitations that deserve attention. Costs reported are based on cost to charge ratios, which may generate imperfect estimates. Data was limited to 17 freestanding children’s hospitals, and our findings may not generalize to other hospitals. We also compared mental health and suicide attempt hospitalizations with “other medical” hospitalizations. This broad group contains certain medical conditions that may have higher or lower profit margins than average, and estimates of the margins could be over- or underestimated. We assumed that mental health and suicide attempt admissions were displacing admissions with non–mental health medical conditions (ie, not an empty bed). If those beds would otherwise be unoccupied, raw margins are better estimates of the financial impact than margin differences between mental health/suicide attempt and other medical hospitalizations.

Children’s hospitals are more likely to have significantly lower financial margins for mental health and suicide attempt hospitalizations than for other medical hospitalizations. Future work to investigate how quality of care is associated with reimbursement can help ensure that funding for children’s acute mental health care services is commensurate with resources required to provide high quality services.

The authors had no financial relationships relevant to this article to disclose.

Research reported in this publication was supported by the National Institute of Mental Health of the National Institutes of Health under Award Number K23MH115162 (Doupnik).

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Cervical lymphadenitis is a common superficial neck infection in childhood. While most children with cervical lymphadenitis recover with antibiotic therapy, a subset can develop an abscess that may require surgical drainage. Radiologic imaging, most commonly ultrasound or computed tomography (CT), is often performed to identify such an abscess.^1-3 However, no national standards exist to guide clinician decision making around imaging in this population. In the absence of evidence-based guidelines, variability in frequency, timing, and modality of imaging likely exists in children hospitalized with cervical lymphadenitis.

As demonstrated for several other common pediatric conditions,^4,5 variability in imaging practices may contribute to overutilization of resources in children with cervical lymphadenitis. In particular, routinely conducting imaging on presentation may constitute overuse, as children with cervical lymphadenitis who present with less than 72 hours of neck swelling rarely undergo surgical drainage within the first 24 hours of hospitalization.^1,6,7 Imaging performed on presentation is often repeated later during hospitalization, particularly if the patient has not improved with antibiotic therapy. The net result may be unnecessary, redundant radiologic studies. Furthermore, serious complications such as bacteremia, extension of infection into the retropharyngeal space, or involvement of the airway or vasculature rarely occur in children with cervical lymphadenitis.^6,8 In this context, deferring initial imaging in this population is unlikely to lead to adverse outcomes and may reduce radiation exposure.

The overall objectives of this study are to describe hospital-level variation in imaging practices for pediatric cervical lymphadenitis and to examine the association between early imaging and outcomes in this population.

Study Design and Data Source

We conducted a multicenter, cross-sectional study using the Pediatric Health Information Systems (PHIS) database, which contains administrative and billing data from 49 geographically diverse children’s hospitals across the United States (US) affiliated with the Children’s Hospital Association (Lenexa, Kansas). PHIS includes data on patient demographics, discharge diagnoses, and procedures using the International Classification of Diseases, 9th (ICD-9) and 10th Revision (ICD-10) diagnosis codes, as well as daily billed resource utilization for laboratory tests, imaging studies, and medications. Encrypted medical record numbers permit longitudinal identification of children across multiple visits to the same hospital. Use of de-identified PHIS data was deemed to be nonhuman subjects research; our approach to validation of ICD codes using local electronic medical record review was reviewed and approved by the Cincinnati Children’s Hospital Medical Center Institutional Review Board.

Study Population

Our study team developed an algorithm to identify children with cervical lymphadenitis and minimize misclassification using PHIS (Appendix A). All children with lymphadenitis-related ICD-9 and ICD-10 discharge diagnosis codes were eligible for inclusion. Codes were validated at a single center via electronic medical record review; clinician-documented discharge diagnosis of cervical lymphadenitis or the presence of fever and unilateral or asymmetrical neck swelling with overlying skin changes was used as the reference standard. We then excluded children who did not receive antibiotics, children who received radiologic imaging not involving the head or neck (which suggested noncervical lymphadenitis or other illness), and children who had discharge diagnosis codes for other specified conditions that are sometimes associated with enlarged cervical lymph nodes but warrant different evaluation or treatment (eg, Kawasaki disease, retropharyngeal abscess, and dental abscess; Appendix A). Our final algorithm yielded a positive predictive value of 87.5% (95% CI: 79.2%-93.4%) when ICD-9 codes were considered, and 95.1% (95% CI: 88.9%-98.4%) when ICD-10 codes were considered (Appendix A).

This algorithm was subsequently applied to the PHIS database. Children ages two months to 18 years hospitalized at participating PHIS institutions between July 2013 and December 2017 with a diagnosis of cervical lymphadenitis as per the algorithm (Appendix A) were eligible for inclusion. For children with multiple eligible admissions during the study period, we only included the first hospitalization. Children with complex chronic condition diagnosis codes⁹ were excluded as their clinical complexity could influence decisions around timing and modality of diagnostic imaging. In addition, we excluded children who did not have an emergency department (ED) visit associated with their hospitalization. This step was intended to exclude children who were transferred from another institution, as imaging performed at outside institutions prior to transfer is not available in PHIS. To avoid overinflating hospital-level variation in the setting of a small sample size, we also excluded all children admitted to the five hospitals with fewer than 50 cases of cervical lymphadenitis during the study period. Our final cohort consisted of 44 PHIS hospitals.

Measures of Interest

To examine hospital-level variation in imaging practices, we measured the proportion of children at each hospital who underwent any neck imaging study, CT or ultrasound imaging, early imaging, and multiple imaging studies within a single hospitalization. Neck imaging was defined as the presence of a billing code for ultrasound, CT, or magnetic resonance imaging (MRI) study of the neck (Appendix B). Early imaging was defined as neck imaging conducted on day 0 of hospitalization (ie, calendar day of admission and ending at midnight). Multiple imaging studies were defined as the receipt of more than one imaging study, regardless of timing or modality. We also measured the proportion of children by hospital who received surgical drainage, defined by the presence of procedure codes for incision and drainage of abscess of the neck (Appendix B).

In examining patient-level association between early imaging and clinical outcomes, our primary outcome of interest was the receipt of multiple imaging studies. Secondary outcomes included rates of surgical drainage, length of stay (in hospital days), and rates of lymphadenitis-related hospital readmission within 30 days of index discharge.

Covariates

Baseline demographic characteristics included age, gender, race/ethnicity, and insurance type. We measured ED visits associated with lymphadenitis-related diagnosis codes in the 30 days prior to admission as a proxy measure for illness duration prior to presentation. To approximate illness severity, we included the following covariates: rates of intensive care unit admission on presentation, rates of receipt of intravenous (IV) analgesia (Appendix B) on hospital days prior to surgical drainage, and rates of receipt of broad-spectrum antibiotics on day 0 or 1 of hospitalization. Broad-spectrum antibiotics (Appendix B) were defined by an independent three-person review of available antibiotic codes (SD, SSS, and JT); differences were resolved by group consensus.

Analysis

Categorical variables were described using frequencies and percentages, while continuous data were described using median and interquartile range. We described hospital-level variation in imaging practices by calculating and comparing the proportion of children at each hospital who underwent any neck imaging study, CT imaging, ultrasound imaging, early imaging, multiple imaging studies, and surgical drainage.

Patient-level demographics and clinical characteristics were compared across groups using chi-square test. To examine the association between early imaging and outcomes, we used generalized linear or logistic mixed effects models to control for patient demographic characteristics and clinical markers of illness duration and severity, with a random effect for hospital to account for clustering. Patient demographics in the model defined a priori included age, race/ethnicity, and insurance type; clinical characteristics included prior ED visit for lymphadenitis, initial intensive care unit (ICU) admission, use of IV analgesia, and use of broad-spectrum antibiotics on day 0 or 1 of hospitalization. To assess the potential for misclassification related to the availability of calendar day but not time of imaging in PHIS, we conducted a secondary analysis to examine the patient-level association between early imaging and outcomes using an alternative definition for early imaging (defined as imaging conducted on day 0 or day 1 of hospitalization).

We identified 19,785 PHIS hospitalizations with lymphadenitis-related discharge diagnosis codes between July 1, 2013 and December 31, 2017. Applying our algorithm and exclusion criteria, we assembled a cohort of 10,014 children hospitalized with cervical lymphadenitis (Figure 1). Two-thirds of the children in our cohort were <4 years old, 42% were non-Hispanic white, and 63% had a government payor (Table 1). Neck imaging (ultrasound, CT, or MRI) was conducted in 8,103 (81%) children. CT imaging was performed in 4,097 (41%) of children, and early imaging was conducted in 6,111 (61%) of children with cervical lymphadenitis.

We noted hospital-level variation in rates of any neck imaging (median: 82.1%, interquartile range [IQR]: 77.7%-85.5%, full range: 68.7%-93.1%), CT imaging (median: 42.3%, IQR: 26.7%-55.2%, full range: 12.0%-81.5%), early imaging (median: 64.4%, IQR: 59.8%-68.4%, full range: 13.8%-76.9%), and multiple imaging studies (median: 23.7%, IQR: 18.6%-28.9%, full range: 1.2%-40.7%; Figure 2). Rates of surgical drainage also varied by hospital (median: 35.1%, IQR: 31.3%-42.0%, full range: 17.1%-54.5%).

In this large multicenter study of children with cervical lymphadenitis, we found variation in imaging practices across 44 US children’s hospitals. Children with cervical lymphadenitis who underwent early imaging were more likely to receive multiple imaging studies during a single hospitalization than those who did not receive early imaging. At the patient level, early imaging was also associated with higher rates of surgical drainage, more frequent 30-day readmission, and longer lengths of stay.

To our knowledge, imaging practices in the population of children hospitalized with cervical lymphadenitis have not been previously characterized in the US; one study from Atlanta, Georgia, describes imaging practices in all children evaluated in the ED.¹ Single-center studies of children hospitalized with cervical lymphadenitis have been previously conducted in Canada⁶ and New Zealand,⁸ in which 42%-51% of children received imaging. In our study, most (81%) children hospitalized with lymphadenitis received some form of imaging, with 61% of all children receiving early imaging. Furthermore, 41% received CT imaging, as compared with 8%-10% of children in the aforementioned studies from Canada and New Zealand.^6,8 This finding is consistent with a pattern of imaging overuse in the US, which has amongst the highest utilization rates globally for advanced imaging such as CT and MRI.^10,11 Identifying opportunities to safely reduce routine imaging, particularly CT imaging, in this population could decrease unnecessary radiation exposure without compromising outcomes.

We also noted variability in imaging practices across PHIS hospitals. Some of this variability may be partially explained by differences in the patient population or illness severity across hospitals. However, given the absence of evidence-based best practices for children with cervical lymphadenitis, clinicians may rely on anecdotal experience or local practice culture to guide their decision making,¹² leading to variability in frequency, timing, and modality of imaging.

At the patient level, we found that children who received early imaging were more likely to receive multiple imaging studies. This finding supports our hypothesis that clinicians often order a second imaging study when the initial imaging study does not clearly demonstrate an abscess, and the child subsequently fails to demonstrate clear improvement after 24-48 hours of antibiotics.

Furthermore, early imaging was associated with overall increased utilization in our cohort, including increased likelihood of surgical drainage, 30-day readmission for lymphadenitis, as well as longer lengths of stay. Confounding may be one explanation for this finding. For instance, clinicians may pursue early imaging in children who present with longer duration of symptoms or more severe illness on presentation, as these factors may be associated with abscess formation.^1,6,7 These clinical covariates are not available in PHIS. Thus, we used prior ED visits for lymphadenitis to approximate illness duration, and initial admission to ICU, receipt of IV analgesia, and receipt of broad-spectrum antibiotics to approximate illness severity in an attempt to mitigate confounding. However, our proxy measures may not appropriately estimate illness duration and severity. For instance, children who had urgent care or outpatient visits for lymphadenitis would not be captured using the proxy of prior ED visit for lymphadenitis. Similarly, use of broad-spectrum antibiotics and IV analgesia may be influenced by provider or institutional preference rather than illness severity. Thus, residual confounding may exist despite adjusting for these measures.

On the other hand, it is also possible that a proportion of children with a small fluid collection on imaging may have improved with antibiotics alone. There is a growing body of evidence in children with other head and neck infections (eg, retropharyngeal abscess and orbital cellulitis with periosteal abscess)^13-15 that suggests that children with small abscesses often improve with antibiotic therapy alone. In children with cervical lymphadenitis who have small or developing abscesses identified via routine imaging on presentation, clinicians may be driven to pursue a surgical intervention with uncertain benefit. Deferring routine imaging in this population may provide an opportunity to improve the value of care in children with lymphadenitis without adversely affecting outcomes.

Upon closer examination of readmissions, children who received early imaging during index hospitalization were more likely to have a 30-day readmission when only evaluating the subset of patients who did not receive surgical drainage during the index admission. This suggests that readmissions are less likely attributable to surgical complications and more likely a reflection of the natural history of lymphadenitis in which a subset of patients eventually develop an abscess. Further supporting this, 61% of children who had a 30-day readmission for lymphadenitis underwent surgical drainage during readmission. Given that lymphadenitis is a slow-brewing infection in which serious complications are rare, patients who demonstrate gradual clinical improvement do not need to remain hospitalized and serially imaged to identify a possible abscess. Outpatient expectant management and readmission as needed for drainage may be an acceptable approach.

This study has several limitations given our use of an administrative database. Children with lymphadenitis may have been misclassified as these patients were identified using discharge diagnosis codes. To mitigate this potential misclassification, we conducted a structured validation process and found that the included codes had high positive predictive values (Appendix A). This validation process was conducted at a single hospital, and coding may vary across hospitals. To approximate sensitivity, we also sampled children without our included codes but with neck imaging and antibiotic use, and found that rates of cervical lymphadenitis were very low among children without our included diagnosis codes.

Furthermore, we were unable to measure the exact time of imaging study in PHIS; we used imaging conducted on hospital day 0 as a proxy measure for imaging conducted within the first 24 hours of presentation. With this definition, some children who had early imaging were likely misclassified as not having received early imaging. For example, a patient who arrived in the ED at 9 pm on day 0 of admission and had a neck ultrasound performed at 1 am would be classified as having had an imaging study on day 1 of hospitalization even though the imaging study was conducted within 4 hours of presentation. Using an alternative definition of early imaging as imaging conducted on hospital day 0 and day 1, we found a much higher adjusted OR for multiple imaging studies, with similar associations for secondary outcomes. As such, our definition of early imaging as day 0 likely biases the results toward the null; the true increase in likelihood of multiple imaging for those who receive early imaging is probably greater than our conservative estimation.

Additionally, there may be a subset of children who underwent imaging prior to presentation at the PHIS hospital ED for further workup and admission. Imaging conducted outside a PHIS hospital was not captured in this database. Similarly, children who had a readmission at a different hospital than their index admission would not be captured using PHIS. Finally, PHIS captures data from children’s hospitals; practices at these hospitals may not be generalizable to practices in the community hospital setting.

In conclusion, we found that imaging practices in children hospitalized with cervical lymphadenitis were widely variable across hospitals. Children receiving early imaging had more resource utilization and intervention when compared with children who did not receive early imaging. Our findings may represent a cascade effect, in which routinely conducted early imaging prompts clinicians to pursue more testing and interventions in this population. Future studies should obtain more detailed patient level covariates to further characterize clinical factors that may impact decisions around imaging and clinical outcomes for children with cervical lymphadenitis.

Acknowledgments

The authors would like to acknowledge the following investigators for their contributions to data interpretation and review of the final manuscript: Angela Choe MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Margaret Rush MD, Children’s National Medical Center, Washington, DC; Ryosuke Takei MD, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; Wallis Molchen DO, Texas Children’s Hospital, Houston, Texas; Stephanie Royer Moss MD, Cleveland Clinic, Cleveland, Ohio; Rebecca Dang, MD, Lucile Packard Children’s Hospital Stanford, Palo Alto, California; Joy Solano MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas; Nathaniel P. Goodrich MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Ngozi Eboh MD, Texas Tech University Health Sciences Center, Dallas, Texas; Ashley Jenkins MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Rebecca Steuart MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sonya Tang Girdwood MD, PhD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Alissa McInerney MD, Maria Fareri Children’s Hospital at Westchester Medical Center, Valhalla, New York; Sumeet Banker MD, MPH, New York Presbyterian Morgan Stanley Children’s Hospital, New York, New York; Corrie McDaniel DO, Seattle Children’s Hospital, Seattle, Washington; Christiane Lenzen MD, Rady Children’s Hospital, San Diego, California; Aleisha Nabower MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Waheeda Samady MD, Ann & Robert H. Lurie Children’s Hospital, Chicago, Illinois; Jennifer Chen MD, Rady Children’s Hospital, San Diego, California; Marquita Genies MD, MPH, John’s Hopkins Children’s Center, Baltimore, Maryland; Justin Lockwood MD, Children’s Hospital Colorado, Aurora, Colorado; David Synhorst MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas.

Cervical lymphadenitis is a common superficial neck infection in childhood. While most children with cervical lymphadenitis recover with antibiotic therapy, a subset can develop an abscess that may require surgical drainage. Radiologic imaging, most commonly ultrasound or computed tomography (CT), is often performed to identify such an abscess.^1-3 However, no national standards exist to guide clinician decision making around imaging in this population. In the absence of evidence-based guidelines, variability in frequency, timing, and modality of imaging likely exists in children hospitalized with cervical lymphadenitis.

As demonstrated for several other common pediatric conditions,^4,5 variability in imaging practices may contribute to overutilization of resources in children with cervical lymphadenitis. In particular, routinely conducting imaging on presentation may constitute overuse, as children with cervical lymphadenitis who present with less than 72 hours of neck swelling rarely undergo surgical drainage within the first 24 hours of hospitalization.^1,6,7 Imaging performed on presentation is often repeated later during hospitalization, particularly if the patient has not improved with antibiotic therapy. The net result may be unnecessary, redundant radiologic studies. Furthermore, serious complications such as bacteremia, extension of infection into the retropharyngeal space, or involvement of the airway or vasculature rarely occur in children with cervical lymphadenitis.^6,8 In this context, deferring initial imaging in this population is unlikely to lead to adverse outcomes and may reduce radiation exposure.

The overall objectives of this study are to describe hospital-level variation in imaging practices for pediatric cervical lymphadenitis and to examine the association between early imaging and outcomes in this population.

Study Design and Data Source

We conducted a multicenter, cross-sectional study using the Pediatric Health Information Systems (PHIS) database, which contains administrative and billing data from 49 geographically diverse children’s hospitals across the United States (US) affiliated with the Children’s Hospital Association (Lenexa, Kansas). PHIS includes data on patient demographics, discharge diagnoses, and procedures using the International Classification of Diseases, 9th (ICD-9) and 10th Revision (ICD-10) diagnosis codes, as well as daily billed resource utilization for laboratory tests, imaging studies, and medications. Encrypted medical record numbers permit longitudinal identification of children across multiple visits to the same hospital. Use of de-identified PHIS data was deemed to be nonhuman subjects research; our approach to validation of ICD codes using local electronic medical record review was reviewed and approved by the Cincinnati Children’s Hospital Medical Center Institutional Review Board.

Study Population

Our study team developed an algorithm to identify children with cervical lymphadenitis and minimize misclassification using PHIS (Appendix A). All children with lymphadenitis-related ICD-9 and ICD-10 discharge diagnosis codes were eligible for inclusion. Codes were validated at a single center via electronic medical record review; clinician-documented discharge diagnosis of cervical lymphadenitis or the presence of fever and unilateral or asymmetrical neck swelling with overlying skin changes was used as the reference standard. We then excluded children who did not receive antibiotics, children who received radiologic imaging not involving the head or neck (which suggested noncervical lymphadenitis or other illness), and children who had discharge diagnosis codes for other specified conditions that are sometimes associated with enlarged cervical lymph nodes but warrant different evaluation or treatment (eg, Kawasaki disease, retropharyngeal abscess, and dental abscess; Appendix A). Our final algorithm yielded a positive predictive value of 87.5% (95% CI: 79.2%-93.4%) when ICD-9 codes were considered, and 95.1% (95% CI: 88.9%-98.4%) when ICD-10 codes were considered (Appendix A).

This algorithm was subsequently applied to the PHIS database. Children ages two months to 18 years hospitalized at participating PHIS institutions between July 2013 and December 2017 with a diagnosis of cervical lymphadenitis as per the algorithm (Appendix A) were eligible for inclusion. For children with multiple eligible admissions during the study period, we only included the first hospitalization. Children with complex chronic condition diagnosis codes⁹ were excluded as their clinical complexity could influence decisions around timing and modality of diagnostic imaging. In addition, we excluded children who did not have an emergency department (ED) visit associated with their hospitalization. This step was intended to exclude children who were transferred from another institution, as imaging performed at outside institutions prior to transfer is not available in PHIS. To avoid overinflating hospital-level variation in the setting of a small sample size, we also excluded all children admitted to the five hospitals with fewer than 50 cases of cervical lymphadenitis during the study period. Our final cohort consisted of 44 PHIS hospitals.

Measures of Interest

To examine hospital-level variation in imaging practices, we measured the proportion of children at each hospital who underwent any neck imaging study, CT or ultrasound imaging, early imaging, and multiple imaging studies within a single hospitalization. Neck imaging was defined as the presence of a billing code for ultrasound, CT, or magnetic resonance imaging (MRI) study of the neck (Appendix B). Early imaging was defined as neck imaging conducted on day 0 of hospitalization (ie, calendar day of admission and ending at midnight). Multiple imaging studies were defined as the receipt of more than one imaging study, regardless of timing or modality. We also measured the proportion of children by hospital who received surgical drainage, defined by the presence of procedure codes for incision and drainage of abscess of the neck (Appendix B).

In examining patient-level association between early imaging and clinical outcomes, our primary outcome of interest was the receipt of multiple imaging studies. Secondary outcomes included rates of surgical drainage, length of stay (in hospital days), and rates of lymphadenitis-related hospital readmission within 30 days of index discharge.

Covariates

Baseline demographic characteristics included age, gender, race/ethnicity, and insurance type. We measured ED visits associated with lymphadenitis-related diagnosis codes in the 30 days prior to admission as a proxy measure for illness duration prior to presentation. To approximate illness severity, we included the following covariates: rates of intensive care unit admission on presentation, rates of receipt of intravenous (IV) analgesia (Appendix B) on hospital days prior to surgical drainage, and rates of receipt of broad-spectrum antibiotics on day 0 or 1 of hospitalization. Broad-spectrum antibiotics (Appendix B) were defined by an independent three-person review of available antibiotic codes (SD, SSS, and JT); differences were resolved by group consensus.

Analysis

Categorical variables were described using frequencies and percentages, while continuous data were described using median and interquartile range. We described hospital-level variation in imaging practices by calculating and comparing the proportion of children at each hospital who underwent any neck imaging study, CT imaging, ultrasound imaging, early imaging, multiple imaging studies, and surgical drainage.

Patient-level demographics and clinical characteristics were compared across groups using chi-square test. To examine the association between early imaging and outcomes, we used generalized linear or logistic mixed effects models to control for patient demographic characteristics and clinical markers of illness duration and severity, with a random effect for hospital to account for clustering. Patient demographics in the model defined a priori included age, race/ethnicity, and insurance type; clinical characteristics included prior ED visit for lymphadenitis, initial intensive care unit (ICU) admission, use of IV analgesia, and use of broad-spectrum antibiotics on day 0 or 1 of hospitalization. To assess the potential for misclassification related to the availability of calendar day but not time of imaging in PHIS, we conducted a secondary analysis to examine the patient-level association between early imaging and outcomes using an alternative definition for early imaging (defined as imaging conducted on day 0 or day 1 of hospitalization).

We identified 19,785 PHIS hospitalizations with lymphadenitis-related discharge diagnosis codes between July 1, 2013 and December 31, 2017. Applying our algorithm and exclusion criteria, we assembled a cohort of 10,014 children hospitalized with cervical lymphadenitis (Figure 1). Two-thirds of the children in our cohort were <4 years old, 42% were non-Hispanic white, and 63% had a government payor (Table 1). Neck imaging (ultrasound, CT, or MRI) was conducted in 8,103 (81%) children. CT imaging was performed in 4,097 (41%) of children, and early imaging was conducted in 6,111 (61%) of children with cervical lymphadenitis.

We noted hospital-level variation in rates of any neck imaging (median: 82.1%, interquartile range [IQR]: 77.7%-85.5%, full range: 68.7%-93.1%), CT imaging (median: 42.3%, IQR: 26.7%-55.2%, full range: 12.0%-81.5%), early imaging (median: 64.4%, IQR: 59.8%-68.4%, full range: 13.8%-76.9%), and multiple imaging studies (median: 23.7%, IQR: 18.6%-28.9%, full range: 1.2%-40.7%; Figure 2). Rates of surgical drainage also varied by hospital (median: 35.1%, IQR: 31.3%-42.0%, full range: 17.1%-54.5%).

In this large multicenter study of children with cervical lymphadenitis, we found variation in imaging practices across 44 US children’s hospitals. Children with cervical lymphadenitis who underwent early imaging were more likely to receive multiple imaging studies during a single hospitalization than those who did not receive early imaging. At the patient level, early imaging was also associated with higher rates of surgical drainage, more frequent 30-day readmission, and longer lengths of stay.

To our knowledge, imaging practices in the population of children hospitalized with cervical lymphadenitis have not been previously characterized in the US; one study from Atlanta, Georgia, describes imaging practices in all children evaluated in the ED.¹ Single-center studies of children hospitalized with cervical lymphadenitis have been previously conducted in Canada⁶ and New Zealand,⁸ in which 42%-51% of children received imaging. In our study, most (81%) children hospitalized with lymphadenitis received some form of imaging, with 61% of all children receiving early imaging. Furthermore, 41% received CT imaging, as compared with 8%-10% of children in the aforementioned studies from Canada and New Zealand.^6,8 This finding is consistent with a pattern of imaging overuse in the US, which has amongst the highest utilization rates globally for advanced imaging such as CT and MRI.^10,11 Identifying opportunities to safely reduce routine imaging, particularly CT imaging, in this population could decrease unnecessary radiation exposure without compromising outcomes.

We also noted variability in imaging practices across PHIS hospitals. Some of this variability may be partially explained by differences in the patient population or illness severity across hospitals. However, given the absence of evidence-based best practices for children with cervical lymphadenitis, clinicians may rely on anecdotal experience or local practice culture to guide their decision making,¹² leading to variability in frequency, timing, and modality of imaging.

At the patient level, we found that children who received early imaging were more likely to receive multiple imaging studies. This finding supports our hypothesis that clinicians often order a second imaging study when the initial imaging study does not clearly demonstrate an abscess, and the child subsequently fails to demonstrate clear improvement after 24-48 hours of antibiotics.

Furthermore, early imaging was associated with overall increased utilization in our cohort, including increased likelihood of surgical drainage, 30-day readmission for lymphadenitis, as well as longer lengths of stay. Confounding may be one explanation for this finding. For instance, clinicians may pursue early imaging in children who present with longer duration of symptoms or more severe illness on presentation, as these factors may be associated with abscess formation.^1,6,7 These clinical covariates are not available in PHIS. Thus, we used prior ED visits for lymphadenitis to approximate illness duration, and initial admission to ICU, receipt of IV analgesia, and receipt of broad-spectrum antibiotics to approximate illness severity in an attempt to mitigate confounding. However, our proxy measures may not appropriately estimate illness duration and severity. For instance, children who had urgent care or outpatient visits for lymphadenitis would not be captured using the proxy of prior ED visit for lymphadenitis. Similarly, use of broad-spectrum antibiotics and IV analgesia may be influenced by provider or institutional preference rather than illness severity. Thus, residual confounding may exist despite adjusting for these measures.

On the other hand, it is also possible that a proportion of children with a small fluid collection on imaging may have improved with antibiotics alone. There is a growing body of evidence in children with other head and neck infections (eg, retropharyngeal abscess and orbital cellulitis with periosteal abscess)^13-15 that suggests that children with small abscesses often improve with antibiotic therapy alone. In children with cervical lymphadenitis who have small or developing abscesses identified via routine imaging on presentation, clinicians may be driven to pursue a surgical intervention with uncertain benefit. Deferring routine imaging in this population may provide an opportunity to improve the value of care in children with lymphadenitis without adversely affecting outcomes.

Upon closer examination of readmissions, children who received early imaging during index hospitalization were more likely to have a 30-day readmission when only evaluating the subset of patients who did not receive surgical drainage during the index admission. This suggests that readmissions are less likely attributable to surgical complications and more likely a reflection of the natural history of lymphadenitis in which a subset of patients eventually develop an abscess. Further supporting this, 61% of children who had a 30-day readmission for lymphadenitis underwent surgical drainage during readmission. Given that lymphadenitis is a slow-brewing infection in which serious complications are rare, patients who demonstrate gradual clinical improvement do not need to remain hospitalized and serially imaged to identify a possible abscess. Outpatient expectant management and readmission as needed for drainage may be an acceptable approach.

This study has several limitations given our use of an administrative database. Children with lymphadenitis may have been misclassified as these patients were identified using discharge diagnosis codes. To mitigate this potential misclassification, we conducted a structured validation process and found that the included codes had high positive predictive values (Appendix A). This validation process was conducted at a single hospital, and coding may vary across hospitals. To approximate sensitivity, we also sampled children without our included codes but with neck imaging and antibiotic use, and found that rates of cervical lymphadenitis were very low among children without our included diagnosis codes.

Furthermore, we were unable to measure the exact time of imaging study in PHIS; we used imaging conducted on hospital day 0 as a proxy measure for imaging conducted within the first 24 hours of presentation. With this definition, some children who had early imaging were likely misclassified as not having received early imaging. For example, a patient who arrived in the ED at 9 pm on day 0 of admission and had a neck ultrasound performed at 1 am would be classified as having had an imaging study on day 1 of hospitalization even though the imaging study was conducted within 4 hours of presentation. Using an alternative definition of early imaging as imaging conducted on hospital day 0 and day 1, we found a much higher adjusted OR for multiple imaging studies, with similar associations for secondary outcomes. As such, our definition of early imaging as day 0 likely biases the results toward the null; the true increase in likelihood of multiple imaging for those who receive early imaging is probably greater than our conservative estimation.

Additionally, there may be a subset of children who underwent imaging prior to presentation at the PHIS hospital ED for further workup and admission. Imaging conducted outside a PHIS hospital was not captured in this database. Similarly, children who had a readmission at a different hospital than their index admission would not be captured using PHIS. Finally, PHIS captures data from children’s hospitals; practices at these hospitals may not be generalizable to practices in the community hospital setting.

In conclusion, we found that imaging practices in children hospitalized with cervical lymphadenitis were widely variable across hospitals. Children receiving early imaging had more resource utilization and intervention when compared with children who did not receive early imaging. Our findings may represent a cascade effect, in which routinely conducted early imaging prompts clinicians to pursue more testing and interventions in this population. Future studies should obtain more detailed patient level covariates to further characterize clinical factors that may impact decisions around imaging and clinical outcomes for children with cervical lymphadenitis.

Acknowledgments

The authors would like to acknowledge the following investigators for their contributions to data interpretation and review of the final manuscript: Angela Choe MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Margaret Rush MD, Children’s National Medical Center, Washington, DC; Ryosuke Takei MD, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; Wallis Molchen DO, Texas Children’s Hospital, Houston, Texas; Stephanie Royer Moss MD, Cleveland Clinic, Cleveland, Ohio; Rebecca Dang, MD, Lucile Packard Children’s Hospital Stanford, Palo Alto, California; Joy Solano MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas; Nathaniel P. Goodrich MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Ngozi Eboh MD, Texas Tech University Health Sciences Center, Dallas, Texas; Ashley Jenkins MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Rebecca Steuart MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sonya Tang Girdwood MD, PhD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Alissa McInerney MD, Maria Fareri Children’s Hospital at Westchester Medical Center, Valhalla, New York; Sumeet Banker MD, MPH, New York Presbyterian Morgan Stanley Children’s Hospital, New York, New York; Corrie McDaniel DO, Seattle Children’s Hospital, Seattle, Washington; Christiane Lenzen MD, Rady Children’s Hospital, San Diego, California; Aleisha Nabower MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Waheeda Samady MD, Ann & Robert H. Lurie Children’s Hospital, Chicago, Illinois; Jennifer Chen MD, Rady Children’s Hospital, San Diego, California; Marquita Genies MD, MPH, John’s Hopkins Children’s Center, Baltimore, Maryland; Justin Lockwood MD, Children’s Hospital Colorado, Aurora, Colorado; David Synhorst MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas.

Cervical lymphadenitis is a common superficial neck infection in childhood. While most children with cervical lymphadenitis recover with antibiotic therapy, a subset can develop an abscess that may require surgical drainage. Radiologic imaging, most commonly ultrasound or computed tomography (CT), is often performed to identify such an abscess.^1-3 However, no national standards exist to guide clinician decision making around imaging in this population. In the absence of evidence-based guidelines, variability in frequency, timing, and modality of imaging likely exists in children hospitalized with cervical lymphadenitis.

As demonstrated for several other common pediatric conditions,^4,5 variability in imaging practices may contribute to overutilization of resources in children with cervical lymphadenitis. In particular, routinely conducting imaging on presentation may constitute overuse, as children with cervical lymphadenitis who present with less than 72 hours of neck swelling rarely undergo surgical drainage within the first 24 hours of hospitalization.^1,6,7 Imaging performed on presentation is often repeated later during hospitalization, particularly if the patient has not improved with antibiotic therapy. The net result may be unnecessary, redundant radiologic studies. Furthermore, serious complications such as bacteremia, extension of infection into the retropharyngeal space, or involvement of the airway or vasculature rarely occur in children with cervical lymphadenitis.^6,8 In this context, deferring initial imaging in this population is unlikely to lead to adverse outcomes and may reduce radiation exposure.

The overall objectives of this study are to describe hospital-level variation in imaging practices for pediatric cervical lymphadenitis and to examine the association between early imaging and outcomes in this population.

Study Design and Data Source

We conducted a multicenter, cross-sectional study using the Pediatric Health Information Systems (PHIS) database, which contains administrative and billing data from 49 geographically diverse children’s hospitals across the United States (US) affiliated with the Children’s Hospital Association (Lenexa, Kansas). PHIS includes data on patient demographics, discharge diagnoses, and procedures using the International Classification of Diseases, 9th (ICD-9) and 10th Revision (ICD-10) diagnosis codes, as well as daily billed resource utilization for laboratory tests, imaging studies, and medications. Encrypted medical record numbers permit longitudinal identification of children across multiple visits to the same hospital. Use of de-identified PHIS data was deemed to be nonhuman subjects research; our approach to validation of ICD codes using local electronic medical record review was reviewed and approved by the Cincinnati Children’s Hospital Medical Center Institutional Review Board.

Study Population

Our study team developed an algorithm to identify children with cervical lymphadenitis and minimize misclassification using PHIS (Appendix A). All children with lymphadenitis-related ICD-9 and ICD-10 discharge diagnosis codes were eligible for inclusion. Codes were validated at a single center via electronic medical record review; clinician-documented discharge diagnosis of cervical lymphadenitis or the presence of fever and unilateral or asymmetrical neck swelling with overlying skin changes was used as the reference standard. We then excluded children who did not receive antibiotics, children who received radiologic imaging not involving the head or neck (which suggested noncervical lymphadenitis or other illness), and children who had discharge diagnosis codes for other specified conditions that are sometimes associated with enlarged cervical lymph nodes but warrant different evaluation or treatment (eg, Kawasaki disease, retropharyngeal abscess, and dental abscess; Appendix A). Our final algorithm yielded a positive predictive value of 87.5% (95% CI: 79.2%-93.4%) when ICD-9 codes were considered, and 95.1% (95% CI: 88.9%-98.4%) when ICD-10 codes were considered (Appendix A).

This algorithm was subsequently applied to the PHIS database. Children ages two months to 18 years hospitalized at participating PHIS institutions between July 2013 and December 2017 with a diagnosis of cervical lymphadenitis as per the algorithm (Appendix A) were eligible for inclusion. For children with multiple eligible admissions during the study period, we only included the first hospitalization. Children with complex chronic condition diagnosis codes⁹ were excluded as their clinical complexity could influence decisions around timing and modality of diagnostic imaging. In addition, we excluded children who did not have an emergency department (ED) visit associated with their hospitalization. This step was intended to exclude children who were transferred from another institution, as imaging performed at outside institutions prior to transfer is not available in PHIS. To avoid overinflating hospital-level variation in the setting of a small sample size, we also excluded all children admitted to the five hospitals with fewer than 50 cases of cervical lymphadenitis during the study period. Our final cohort consisted of 44 PHIS hospitals.

Measures of Interest

To examine hospital-level variation in imaging practices, we measured the proportion of children at each hospital who underwent any neck imaging study, CT or ultrasound imaging, early imaging, and multiple imaging studies within a single hospitalization. Neck imaging was defined as the presence of a billing code for ultrasound, CT, or magnetic resonance imaging (MRI) study of the neck (Appendix B). Early imaging was defined as neck imaging conducted on day 0 of hospitalization (ie, calendar day of admission and ending at midnight). Multiple imaging studies were defined as the receipt of more than one imaging study, regardless of timing or modality. We also measured the proportion of children by hospital who received surgical drainage, defined by the presence of procedure codes for incision and drainage of abscess of the neck (Appendix B).

In examining patient-level association between early imaging and clinical outcomes, our primary outcome of interest was the receipt of multiple imaging studies. Secondary outcomes included rates of surgical drainage, length of stay (in hospital days), and rates of lymphadenitis-related hospital readmission within 30 days of index discharge.

Covariates

Baseline demographic characteristics included age, gender, race/ethnicity, and insurance type. We measured ED visits associated with lymphadenitis-related diagnosis codes in the 30 days prior to admission as a proxy measure for illness duration prior to presentation. To approximate illness severity, we included the following covariates: rates of intensive care unit admission on presentation, rates of receipt of intravenous (IV) analgesia (Appendix B) on hospital days prior to surgical drainage, and rates of receipt of broad-spectrum antibiotics on day 0 or 1 of hospitalization. Broad-spectrum antibiotics (Appendix B) were defined by an independent three-person review of available antibiotic codes (SD, SSS, and JT); differences were resolved by group consensus.

Analysis

Categorical variables were described using frequencies and percentages, while continuous data were described using median and interquartile range. We described hospital-level variation in imaging practices by calculating and comparing the proportion of children at each hospital who underwent any neck imaging study, CT imaging, ultrasound imaging, early imaging, multiple imaging studies, and surgical drainage.

Patient-level demographics and clinical characteristics were compared across groups using chi-square test. To examine the association between early imaging and outcomes, we used generalized linear or logistic mixed effects models to control for patient demographic characteristics and clinical markers of illness duration and severity, with a random effect for hospital to account for clustering. Patient demographics in the model defined a priori included age, race/ethnicity, and insurance type; clinical characteristics included prior ED visit for lymphadenitis, initial intensive care unit (ICU) admission, use of IV analgesia, and use of broad-spectrum antibiotics on day 0 or 1 of hospitalization. To assess the potential for misclassification related to the availability of calendar day but not time of imaging in PHIS, we conducted a secondary analysis to examine the patient-level association between early imaging and outcomes using an alternative definition for early imaging (defined as imaging conducted on day 0 or day 1 of hospitalization).

We identified 19,785 PHIS hospitalizations with lymphadenitis-related discharge diagnosis codes between July 1, 2013 and December 31, 2017. Applying our algorithm and exclusion criteria, we assembled a cohort of 10,014 children hospitalized with cervical lymphadenitis (Figure 1). Two-thirds of the children in our cohort were <4 years old, 42% were non-Hispanic white, and 63% had a government payor (Table 1). Neck imaging (ultrasound, CT, or MRI) was conducted in 8,103 (81%) children. CT imaging was performed in 4,097 (41%) of children, and early imaging was conducted in 6,111 (61%) of children with cervical lymphadenitis.

We noted hospital-level variation in rates of any neck imaging (median: 82.1%, interquartile range [IQR]: 77.7%-85.5%, full range: 68.7%-93.1%), CT imaging (median: 42.3%, IQR: 26.7%-55.2%, full range: 12.0%-81.5%), early imaging (median: 64.4%, IQR: 59.8%-68.4%, full range: 13.8%-76.9%), and multiple imaging studies (median: 23.7%, IQR: 18.6%-28.9%, full range: 1.2%-40.7%; Figure 2). Rates of surgical drainage also varied by hospital (median: 35.1%, IQR: 31.3%-42.0%, full range: 17.1%-54.5%).

In this large multicenter study of children with cervical lymphadenitis, we found variation in imaging practices across 44 US children’s hospitals. Children with cervical lymphadenitis who underwent early imaging were more likely to receive multiple imaging studies during a single hospitalization than those who did not receive early imaging. At the patient level, early imaging was also associated with higher rates of surgical drainage, more frequent 30-day readmission, and longer lengths of stay.

To our knowledge, imaging practices in the population of children hospitalized with cervical lymphadenitis have not been previously characterized in the US; one study from Atlanta, Georgia, describes imaging practices in all children evaluated in the ED.¹ Single-center studies of children hospitalized with cervical lymphadenitis have been previously conducted in Canada⁶ and New Zealand,⁸ in which 42%-51% of children received imaging. In our study, most (81%) children hospitalized with lymphadenitis received some form of imaging, with 61% of all children receiving early imaging. Furthermore, 41% received CT imaging, as compared with 8%-10% of children in the aforementioned studies from Canada and New Zealand.^6,8 This finding is consistent with a pattern of imaging overuse in the US, which has amongst the highest utilization rates globally for advanced imaging such as CT and MRI.^10,11 Identifying opportunities to safely reduce routine imaging, particularly CT imaging, in this population could decrease unnecessary radiation exposure without compromising outcomes.

We also noted variability in imaging practices across PHIS hospitals. Some of this variability may be partially explained by differences in the patient population or illness severity across hospitals. However, given the absence of evidence-based best practices for children with cervical lymphadenitis, clinicians may rely on anecdotal experience or local practice culture to guide their decision making,¹² leading to variability in frequency, timing, and modality of imaging.

At the patient level, we found that children who received early imaging were more likely to receive multiple imaging studies. This finding supports our hypothesis that clinicians often order a second imaging study when the initial imaging study does not clearly demonstrate an abscess, and the child subsequently fails to demonstrate clear improvement after 24-48 hours of antibiotics.

Furthermore, early imaging was associated with overall increased utilization in our cohort, including increased likelihood of surgical drainage, 30-day readmission for lymphadenitis, as well as longer lengths of stay. Confounding may be one explanation for this finding. For instance, clinicians may pursue early imaging in children who present with longer duration of symptoms or more severe illness on presentation, as these factors may be associated with abscess formation.^1,6,7 These clinical covariates are not available in PHIS. Thus, we used prior ED visits for lymphadenitis to approximate illness duration, and initial admission to ICU, receipt of IV analgesia, and receipt of broad-spectrum antibiotics to approximate illness severity in an attempt to mitigate confounding. However, our proxy measures may not appropriately estimate illness duration and severity. For instance, children who had urgent care or outpatient visits for lymphadenitis would not be captured using the proxy of prior ED visit for lymphadenitis. Similarly, use of broad-spectrum antibiotics and IV analgesia may be influenced by provider or institutional preference rather than illness severity. Thus, residual confounding may exist despite adjusting for these measures.

On the other hand, it is also possible that a proportion of children with a small fluid collection on imaging may have improved with antibiotics alone. There is a growing body of evidence in children with other head and neck infections (eg, retropharyngeal abscess and orbital cellulitis with periosteal abscess)^13-15 that suggests that children with small abscesses often improve with antibiotic therapy alone. In children with cervical lymphadenitis who have small or developing abscesses identified via routine imaging on presentation, clinicians may be driven to pursue a surgical intervention with uncertain benefit. Deferring routine imaging in this population may provide an opportunity to improve the value of care in children with lymphadenitis without adversely affecting outcomes.

Upon closer examination of readmissions, children who received early imaging during index hospitalization were more likely to have a 30-day readmission when only evaluating the subset of patients who did not receive surgical drainage during the index admission. This suggests that readmissions are less likely attributable to surgical complications and more likely a reflection of the natural history of lymphadenitis in which a subset of patients eventually develop an abscess. Further supporting this, 61% of children who had a 30-day readmission for lymphadenitis underwent surgical drainage during readmission. Given that lymphadenitis is a slow-brewing infection in which serious complications are rare, patients who demonstrate gradual clinical improvement do not need to remain hospitalized and serially imaged to identify a possible abscess. Outpatient expectant management and readmission as needed for drainage may be an acceptable approach.

This study has several limitations given our use of an administrative database. Children with lymphadenitis may have been misclassified as these patients were identified using discharge diagnosis codes. To mitigate this potential misclassification, we conducted a structured validation process and found that the included codes had high positive predictive values (Appendix A). This validation process was conducted at a single hospital, and coding may vary across hospitals. To approximate sensitivity, we also sampled children without our included codes but with neck imaging and antibiotic use, and found that rates of cervical lymphadenitis were very low among children without our included diagnosis codes.

Furthermore, we were unable to measure the exact time of imaging study in PHIS; we used imaging conducted on hospital day 0 as a proxy measure for imaging conducted within the first 24 hours of presentation. With this definition, some children who had early imaging were likely misclassified as not having received early imaging. For example, a patient who arrived in the ED at 9 pm on day 0 of admission and had a neck ultrasound performed at 1 am would be classified as having had an imaging study on day 1 of hospitalization even though the imaging study was conducted within 4 hours of presentation. Using an alternative definition of early imaging as imaging conducted on hospital day 0 and day 1, we found a much higher adjusted OR for multiple imaging studies, with similar associations for secondary outcomes. As such, our definition of early imaging as day 0 likely biases the results toward the null; the true increase in likelihood of multiple imaging for those who receive early imaging is probably greater than our conservative estimation.

Additionally, there may be a subset of children who underwent imaging prior to presentation at the PHIS hospital ED for further workup and admission. Imaging conducted outside a PHIS hospital was not captured in this database. Similarly, children who had a readmission at a different hospital than their index admission would not be captured using PHIS. Finally, PHIS captures data from children’s hospitals; practices at these hospitals may not be generalizable to practices in the community hospital setting.

In conclusion, we found that imaging practices in children hospitalized with cervical lymphadenitis were widely variable across hospitals. Children receiving early imaging had more resource utilization and intervention when compared with children who did not receive early imaging. Our findings may represent a cascade effect, in which routinely conducted early imaging prompts clinicians to pursue more testing and interventions in this population. Future studies should obtain more detailed patient level covariates to further characterize clinical factors that may impact decisions around imaging and clinical outcomes for children with cervical lymphadenitis.

Acknowledgments

The authors would like to acknowledge the following investigators for their contributions to data interpretation and review of the final manuscript: Angela Choe MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Margaret Rush MD, Children’s National Medical Center, Washington, DC; Ryosuke Takei MD, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; Wallis Molchen DO, Texas Children’s Hospital, Houston, Texas; Stephanie Royer Moss MD, Cleveland Clinic, Cleveland, Ohio; Rebecca Dang, MD, Lucile Packard Children’s Hospital Stanford, Palo Alto, California; Joy Solano MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas; Nathaniel P. Goodrich MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Ngozi Eboh MD, Texas Tech University Health Sciences Center, Dallas, Texas; Ashley Jenkins MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Rebecca Steuart MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sonya Tang Girdwood MD, PhD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Alissa McInerney MD, Maria Fareri Children’s Hospital at Westchester Medical Center, Valhalla, New York; Sumeet Banker MD, MPH, New York Presbyterian Morgan Stanley Children’s Hospital, New York, New York; Corrie McDaniel DO, Seattle Children’s Hospital, Seattle, Washington; Christiane Lenzen MD, Rady Children’s Hospital, San Diego, California; Aleisha Nabower MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Waheeda Samady MD, Ann & Robert H. Lurie Children’s Hospital, Chicago, Illinois; Jennifer Chen MD, Rady Children’s Hospital, San Diego, California; Marquita Genies MD, MPH, John’s Hopkins Children’s Center, Baltimore, Maryland; Justin Lockwood MD, Children’s Hospital Colorado, Aurora, Colorado; David Synhorst MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas.

Neurologic impairment (NI) encompasses static and progressive diseases of the central and/or peripheral nervous systems that result in functional and intellectual impairments.¹ While a variety of neurologic diseases are responsible for NI (eg, hypoxic-ischemic encephalopathy, muscular dystrophy), consequences of these diseases extend beyond neurologic manifestations.¹ These children are at an increased risk for aspiration of oral and gastric contents given their common comorbidities of dysphagia, gastroesophageal reflux, impaired cough, and respiratory muscle weakness.² While aspiration may manifest as a self-resolving pneumonitis, the presence of oral or enteric bacteria in aspirated material may result in the development of bacterial pneumonia. Children with NI hospitalized with aspiration pneumonia have higher complication rates, longer and costlier hospitalizations, and higher readmission rates when compared with children with nonaspiration pneumonia.³

While pediatric aspiration pneumonia is commonly attributed to anaerobic bacteria, this is largely based on extrapolation from epidemiologic studies that were conducted in past decades.^4-8 A single randomized controlled trial found that penicillin and clindamycin, antimicrobials with similar antimicrobial activity against anaerobes, to be equally effective.⁹ However, the recent literature emphasizes the polymicrobial nature of aspiration pneumonia in adults, with the common isolation of Gram-negative enteric bacteria.¹⁰ Further, while Pseudomonas aeruginosa is often identified in respiratory cultures from children with NI and chronic respiratory insufficiency,^11,12 the significance of P. aeruginosa in lower airways remains unclear.

We designed this study to compare hospital outcomes associated with the most commonly prescribed empiric antimicrobial therapies for aspiration pneumonia in children with NI.

Study Design and Data Source

This multicenter, retrospective cohort study used the Pediatric Health Information System (PHIS) database. PHIS, an administrative database of 50 not-for-profit tertiary care pediatric hospitals, contains data regarding patient demographics, diagnoses and procedures, and daily billed resource utilization, including laboratory and imaging studies. Data quality and reliability are assured through the Children’s Hospital Association (CHA; Lenexa, Kansas) and participating hospitals. Due to incomplete data through the study period and data quality issues, six hospitals were excluded.

Inclusion Criteria

Children 1-18 years of age who were discharged between July 1, 2007 and June 30, 2015 were included if they had a NI diagnosis,¹ a principal diagnosis indicative of aspiration pneumonia (507.x),^3,13,14 and received antibiotics in the first two calendar days of admission. NI was determined using previously defined International Classification of Diseases, Ninth Revision-Clinical Modification (ICD-9-CM) diagnosis codes.¹ We only included children who received antibiotics in the first two calendar days of admission to minimize the likelihood of including children admitted for other reasons who acquired aspiration pneumonia after hospitalization. For children with multiple hospitalizations, one admission was randomly selected for inclusion to minimize weighting results toward repeat visits.

Exclusion Criteria

Children transferred from another hospital were excluded as records from their initial presentation, including treatment and outcomes, were not available. We also excluded children with tracheostomy^15,16 or chronic ventilator dependence,¹⁷ those with a diagnosis of human immunodeficiency virus or tuberculosis, and children who received chemotherapy during hospitalization given expected differences in etiology, treatment, and outcomes.¹⁸

Exposure

The primary exposure was antibiotic therapy received in the first two days of admission. Antibiotics were classified by their antimicrobial spectra of activity as defined by The Sanford Guide to Antimicrobial Therapy¹⁹ against the most commonly recognized pathogens of aspiration pneumonia: anaerobes, Gram-negatives, and P. aeruginosa (Appendix Table 1).^10,20 For example, penicillin G and clindamycin were among the antibiotics classified as providing anaerobic coverage alone, whereas ceftriaxone was classified as providing Gram-negative coverage alone and ampicillin-sulbactam or as combination therapy with clindamycin and ceftriaxone were classified as providing anaerobic and Gram-negative coverage. Piperacillin-tazobactam and meropenem were classified as providing anaerobic, Gram-negative, and P. aeruginosa coverage. We excluded antibiotics that do not provide coverage against anaerobes, Gram-negative, or P. aeruginosa (eg, ampicillin, azithromycin) or that provide coverage against Gram-negative and P. aeruginosa, but not anaerobes (eg, cefepime, tobramycin), as these therapies were prescribed for <5% of the cohort. We chose not to examine the coverage for Streptococcus pneumonia or Staphylococcus aureus as antibiotics included in this analysis covered these bacteria for 99.9% of our cohort.

Outcomes included acute respiratory failure during hospitalization, intensive care unit (ICU) transfer, and hospital length of stay (LOS). Acute respiratory failure during hospitalization was defined as the presence of Clinical Transaction Classification (CTC) or ICD-9 procedure code for noninvasive or invasive mechanical ventilation on day two or later of hospitalization, with or without the need for respiratory support on day 0 or day 1 (Appendix Table 2). Given the variability in hospital policies that may drive ICU admission criteria for complex patients, our outcome of ICU transfer was defined as the requirement for ICU level care on day two or later of hospitalization without ICU admission. Acute respiratory failure and ICU care occurring within the first two hospital days were not classified as outcomes because these early events likely reflect illness severity at presentation rather than outcomes attributable to treatment failure; these were included as markers of severity in the models.

Patient Demographics and Clinical Characteristics

Demographic and clinical characteristics that might influence antibiotic choice and/or hospital outcomes were assessed. Clinical characteristics included complex chronic conditions,^21-23 medical technology assistance,²⁴ performance of diagnostic testing, and markers of severe illness on presentation. Diagnostic testing included bacterial cultures (blood, respiratory, urine) and chest radiograph performance in the first two days of hospitalization. Results of diagnostic testing are not available in the PHIS. Illness severity on presentation included acute respiratory failure, pleural drainage, receipt of vasoactive agents, and transfusion of blood products in the first two days of hospitalization (Appendix Table 2).^17,25,26

Continuous data were described with median and interquartile ranges (IQR) due to nonnormal distribution. Categorical data were described with frequencies and percentages. Patient demographics, clinical characteristics, and hospital outcomes were stratified by empiric antimicrobial coverage and compared using chi-square and Kruskal–Wallis tests as appropriate.

Generalized linear mixed-effects models with random hospital intercepts were derived to assess the independent effect of antimicrobial spectra of activity on outcomes of acute respiratory failure, ICU transfer, and LOS while adjusting for important differences in demographic and clinical characteristics. LOS had a nonnormal distribution. Thus, we used an exponential distribution. Covariates were chosen a priori given the clinical and biological relevance to exposure and outcomes—age, presence of complex chronic condition diagnoses, the number of complex chronic conditions, technology dependence, the performance of diagnostic tests on presentation, and illness severity on presentation. ICU admission was included as a covariate in acute respiratory failure and LOS outcome models. The results of the model for acute respiratory failure and ICU transfer are presented as adjusted odds ratios (OR) with a 95% CI. LOS results are presented as adjusted rate ratios (RR) with 95% CI.

All analyses were performed with SAS 9.3 (SAS Institute, Cary, North Carolina). P values <.05 were considered statistically significant. Cincinnati Children’s Hospital Medical Center Institutional Review Board considered this deidentified dataset study as not human subjects research.

Study Cohort

At the 44 hospitals included, 4,812 children with NI hospitalized with the diagnosis of aspiration pneumonia met the eligibility criteria. However, 79 received antibiotics with the spectra of activity not examined, leaving 4,733 children in our final analysis (Appendix Figure). Demographic and clinical characteristics of the study cohort are shown in Table 1. Median age was five years (interquartile range [IQR]: 2-11 years). Most subjects were male (53.9%), non-Hispanic white (47.9%), and publicly insured (63.6%). There was a slight variation in the distribution of admissions across seasons (spring 31.6%, summer 19.2%, fall 21.3%, and winter 27.9%). One-third of children had four or more comorbid CCCs (complex chronic conditions; 34.2%). The three most common nonneurologic CCC diagnosis categories were gastrointestinal (63.1%), congenital and/or genetic defects (36.9%), and respiratory (8.9%). Assistance with medical technologies was also common (82%)—particularly gastrointestinal (63.1%) and neurologic/neuromuscular (9.8%) technologies. The vast majority of children (92.5%) had either a chest radiograph (90.5%), respiratory viral study (33.7%), or respiratory culture (10.0%) obtained on presentation. A minority required noninvasive or invasive respiratory support (25.4%), vasoactive agents (8.9%), blood products (1.2%), or pleural drainage (0.3%) in the first two hospital days.

Spectrum of Antimicrobial Coverage

Most children (57.9%) received anaerobic and Gram-negative coverage; 16.2% received anaerobic, Gram-negative and P. aeruginosa coverage; 15.3% received anaerobic coverage alone; and 10.6% received Gram-negative coverage alone. Empiric antimicrobial coverage varied substantially across hospitals: anaerobic coverage was prescribed for 0%-44% of patients; Gram-negative coverage was prescribed for 3%-26% of patients; anaerobic and Gram-negative coverage was prescribed for 25%-90% of patients; and anaerobic, Gram-negative, and P. aeruginosa coverage was prescribed for 0%-65% of patients (Figure 1).

Outcomes

Acute Respiratory Failure

One-quarter (25.4%) of patients had acute respiratory failure on presentation; 22.5% required respiratory support (continued from presentation or were new) on day two or later of hospitalization (Table 2). In the adjusted analysis, children receiving Gram-negative coverage alone had two-fold greater odds (OR 2.15, 95% CI: 1.41-3.27) and children receiving anaerobic and Gram-negative coverage had 1.6-fold greater odds (OR 1.65, 95% CI: 1.19-2.28), of respiratory failure during hospitalization compared with those receiving anaerobic coverage alone (Figure 2). Odds of respiratory failure during hospitalization did not significantly differ for children receiving anaerobic, Gram-negative, and P. aeruginosa coverage compared with those receiving anaerobic coverage alone.

ICU Transfer

Nearly thirty percent (29.0%) of children required ICU admission, with an additional 3.8% requiring ICU transfer following admission (Table 2). In the multivariable analysis, the odds of an ICU transfer were greater for children receiving Gram-negative coverage alone (OR 1.80, 95% CI: 1.03-3.14) compared with those receiving anaerobic coverage alone. There was no statistical difference in ICU transfer for those receiving anaerobic and Gram-negative coverage (with or without P. aeruginosa coverage) compared with those receiving anaerobic coverage alone (Figure 2).

Length of Stay

Median hospital LOS for the total cohort was five days (IQR: 3-9 days; Table 2). In the multivariable analysis, children receiving Gram-negative coverage alone had a longer LOS (RR 1.28; 95% CI: 1.16-1.41) compared with those receiving anaerobic coverage alone, whereas children receiving anaerobic, Gram-negative, and P. aeruginosa coverage had a shorter LOS (RR 0.83; 95% CI: 0.76-0.90) than those receiving anaerobic coverage alone (Figure 2). There was no statistical difference in the LOS between children receiving anaerobic and Gram-negative coverage and those receiving anaerobic coverage alone.

In this multicenter study of children with NI hospitalized with aspiration pneumonia, we found substantial variation in empiric antimicrobial coverage for children with aspiration pneumonia. When comparing outcomes across groups, children who received anaerobic and Gram-negative coverage had outcomes similar to children who received anaerobic therapy alone. However, children who did not receive anaerobic coverage (ie, Gram-negative coverage alone) had worse outcomes, most notably a greater than two-fold increase in the odds of experiencing acute respiratory failure during hospitalization when compared with children receiving anaerobic therapy. These findings support prior literature that has highlighted the importance of anaerobic therapy in the treatment of aspiration pneumonia. The benefit of antibiotics targeting Gram-negative organisms, in addition to anaerobes, remains uncertain.

The variability in empiric antimicrobial coverage likely reflects the paucity of available information on oral and/or enteric bacteria required to identify them as causative organisms in aspiration pneumonia. In part, this problem is due to the difficulty in obtaining adequate sputum for culture from pediatric patients.²⁷ While it may be more feasible to obtain tracheal aspirates for respiratory culture in children with a tracheostomy, interpretation of culture results remains challenging because the lower airways of children with tracheostomy are commonly colonized with bacterial pathogens.²⁸ Thus, physicians are often left to choose empiric antimicrobial coverage with inadequate supporting evidence.²⁹ Although the polymicrobial nature of aspiration pneumonia is well recognized in adult and pediatric literature,^10,30 it is less clear which organisms are of pathological significance and require treatment.

The treatment standard for aspiration pneumonia has long included anaerobic therapy.²⁹ The worse outcomes of children not receiving anaerobic therapy (ie, Gram-negative coverage alone) compared with children who received anaerobic therapy support the continued importance of anaerobic therapy in the treatment of aspiration pneumonia for hospitalized children with NI. The role of antibiotics covering Gram-negative organisms is less clear. Recent studies suggest the role of anaerobes is overemphasized in the etiology and treatment of aspiration pneumonia.^10,29,31-38 Multiple studies on aspiration pneumonia bacteriology in hospitalized adults have demonstrated a predominance of Gram-negative organisms (ranging from 37%-71% of isolates identified on respiratory culture) and a relative scarcity of anaerobes (ranging from 0%-16% of isolates).^31-37 A prospective study of 50 children hospitalized with clinical and radiographic evidence of pneumonia with known aspiration risk (eg, neuromuscular disease or dysphagia) found that ~80% of 163 bacterial isolates were Gram-negative.³⁸ However, this study included repeat cultures from the same children, and thus, may overestimate the prevalence of Gram-negative organisms. In our study, children who received both anaerobic and Gram-negative therapy had no differences in ICU transfer or LOS but did experience higher odds of acute respiratory failure. As these results may be due to unmeasured confounding, future studies should further explore the necessity of Gram-negative coverage in addition to anaerobic coverage in this population.

While these recent studies may seem to suggest that anaerobic coverage is not necessary for aspiration pneumonia, there are important limitations worth noting. First, these studies used a variety of sampling techniques. While organisms grown from samples obtained via bronchoalveolar lavage^31-34,36 are likely pathogenic, those grown from tracheal or oral samples obtained via percutaneous transtracheal aspiration,³⁴ a protected specimen brush,^34,36,37 or expectorated sputum^35,38 may not represent lower airway organisms. Second, anaerobic cultures were not obtained in all studies.^31,34,38 Anaerobic organisms are difficult to isolate using traditional clinical specimen collection techniques and aerobic culture media.¹⁸ Furthermore, anaerobes are not easily recovered from lung infections after the receipt of antibiotic therapy.³⁹ Details regarding pretreatment, which are largely lacking from these studies, are necessary to interpret the relative scarcity of anaerobes on respiratory culture. Finally, caution should be taken when extrapolating the results of studies focused on the etiology and treatment of aspiration pneumonia in elderly adults to children. Our results, particularly in the context of the limitation of these more recent studies, suggest that the role of anaerobes has been underestimated.

Recent studies examining populations of children with cerebral palsy and/or tracheostomy have emphasized the high rates of carriage and infection rates with Gram-negative and drug-resistant bacteria; in particular, P. aeruginosa accounts for 50%-72% of pathogenic bacteria.^11,12,38,40These studies note the generally poor outcomes of children with P. aeruginosa—including multiple and longer hospitalizations, frequent readmissions, and the increased severity of pneumonia, including the need for ICU admission, pleural effusions, the need for intubation, and mortality.^{11,12,38,40,41} In our study, nearly 35% of children who received anaerobic, Gram-negative, and P. aeruginosa coverage experienced acute respiratory failure during hospitalization compared with 20% of children who received other therapies. While these results might seem to suggest that broader spectrum therapy is harmful, they must be interpreted in the context of important population differences; children who received a combination of anaerobic, Gram-negative, and P. aeruginosa coverage had greater medical complexity and greater severity of illness on presentation. Such factors may provide the reason for the appropriate prescription of antipseudomonal antibiotics (eg, history of tracheostomy colonization or infection, long-term care facility resident).⁴² When we controlled for population differences, children who received antipseudomonal therapy had a significantly shorter LOS and no differences in outcomes of acute respiratory failure or ICU transfer compared with those receiving anaerobic therapy alone. This result suggests that worse outcomes were associated with antipseudomonal therapy on unadjusted analyses resulting from underlying medical complexity and illness severity rather than from colonization or infection with P. aeruginosa.

Our multicenter observational study has several limitations. We used diagnosis codes to identify patients with aspiration pneumonia. As validated clinical criteria for the diagnosis of aspiration pneumonia do not exist, clinicians may assign a diagnosis of and treatment for aspiration pneumonia by subjective suspicion based on a child’s severe NI or illness severity on presentation leading to selection bias. Although administrative data are not able to verify pneumonia type with absolute certainty, we previously demonstrated that the differences in the outcomes of children with aspiration and nonaspiration pneumonia diagnosis codes persist after accounting for the complexity that might influence the diagnosis.³It is also possible that the diagnosis of aspiration pneumonia was not made upon admission for a subset of patients leading to misclassification of exposure. Some children may have had aspiration pneumonia on admission but were not assigned that diagnosis or treated for presumed aspiration pneumonia until later in the hospital course as they demonstrated treatment failure or clinical worsening. It is also possible that some children had an aspiration event during hospitalization that developed into aspiration pneumonia. We attempted to adjust for medical complexity and illness severity through multivariable adjustment based on the diagnosis and procedure codes, as well as the laboratory testing performed. However, unmeasured or residual confounding may remain as administrative data are not equipped to distinguish detailed functional status (eg, ability to cough, chest wall strength) or illness severity (eg, respiratory distress) that might influence antibiotic selection and/or outcomes.

Frthermore, we were unable to account for laboratory, microbiology, or radiology test results, and other management practices (eg, frequency of airway clearance, previous antimicrobial therapy) that may influence outcomes. Future studies should certainly include an examination of the concordance of the antibiotics prescribed with causative organisms, as this undoubtedly affects patient outcomes. Other outcomes are important to examine (eg, time to return to respiratory baseline), but we were unable to do so, given the lack of clinical detail in our database. We randomly selected a single hospitalization for children with multiple admissions; alternative methods could have different results. Although children with NI predominately use children’s hospitals,¹ results may not be generalizable.

These findings support prior literature that has highlighted the important role anaerobic therapy plays in the treatment of aspiration pneumonia in children with NI. In light of the limitations of our study design, we believe that rigorous clinical trials comparing anaerobic with anaerobic and Gram-negative therapy are an important and necessary next step to determine the optimal treatment for aspiration pneumonia in this population.

Disclosures

The authors do not have any financial relationships relevant to this article to disclose.

Funding

Dr. Thomson was supported by the Agency for Healthcare Research and Quality (AHRQ) under award number K08HS025138. Dr. Ambroggio was supported by the National Institute for Allergy and Infectious Diseases (NIAID) under award number K01AI125413. The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ or NIAID.

Neurologic impairment (NI) encompasses static and progressive diseases of the central and/or peripheral nervous systems that result in functional and intellectual impairments.¹ While a variety of neurologic diseases are responsible for NI (eg, hypoxic-ischemic encephalopathy, muscular dystrophy), consequences of these diseases extend beyond neurologic manifestations.¹ These children are at an increased risk for aspiration of oral and gastric contents given their common comorbidities of dysphagia, gastroesophageal reflux, impaired cough, and respiratory muscle weakness.² While aspiration may manifest as a self-resolving pneumonitis, the presence of oral or enteric bacteria in aspirated material may result in the development of bacterial pneumonia. Children with NI hospitalized with aspiration pneumonia have higher complication rates, longer and costlier hospitalizations, and higher readmission rates when compared with children with nonaspiration pneumonia.³

While pediatric aspiration pneumonia is commonly attributed to anaerobic bacteria, this is largely based on extrapolation from epidemiologic studies that were conducted in past decades.^4-8 A single randomized controlled trial found that penicillin and clindamycin, antimicrobials with similar antimicrobial activity against anaerobes, to be equally effective.⁹ However, the recent literature emphasizes the polymicrobial nature of aspiration pneumonia in adults, with the common isolation of Gram-negative enteric bacteria.¹⁰ Further, while Pseudomonas aeruginosa is often identified in respiratory cultures from children with NI and chronic respiratory insufficiency,^11,12 the significance of P. aeruginosa in lower airways remains unclear.

We designed this study to compare hospital outcomes associated with the most commonly prescribed empiric antimicrobial therapies for aspiration pneumonia in children with NI.

Study Design and Data Source

This multicenter, retrospective cohort study used the Pediatric Health Information System (PHIS) database. PHIS, an administrative database of 50 not-for-profit tertiary care pediatric hospitals, contains data regarding patient demographics, diagnoses and procedures, and daily billed resource utilization, including laboratory and imaging studies. Data quality and reliability are assured through the Children’s Hospital Association (CHA; Lenexa, Kansas) and participating hospitals. Due to incomplete data through the study period and data quality issues, six hospitals were excluded.

Inclusion Criteria

Children 1-18 years of age who were discharged between July 1, 2007 and June 30, 2015 were included if they had a NI diagnosis,¹ a principal diagnosis indicative of aspiration pneumonia (507.x),^3,13,14 and received antibiotics in the first two calendar days of admission. NI was determined using previously defined International Classification of Diseases, Ninth Revision-Clinical Modification (ICD-9-CM) diagnosis codes.¹ We only included children who received antibiotics in the first two calendar days of admission to minimize the likelihood of including children admitted for other reasons who acquired aspiration pneumonia after hospitalization. For children with multiple hospitalizations, one admission was randomly selected for inclusion to minimize weighting results toward repeat visits.

Exclusion Criteria

Children transferred from another hospital were excluded as records from their initial presentation, including treatment and outcomes, were not available. We also excluded children with tracheostomy^15,16 or chronic ventilator dependence,¹⁷ those with a diagnosis of human immunodeficiency virus or tuberculosis, and children who received chemotherapy during hospitalization given expected differences in etiology, treatment, and outcomes.¹⁸

Exposure

The primary exposure was antibiotic therapy received in the first two days of admission. Antibiotics were classified by their antimicrobial spectra of activity as defined by The Sanford Guide to Antimicrobial Therapy¹⁹ against the most commonly recognized pathogens of aspiration pneumonia: anaerobes, Gram-negatives, and P. aeruginosa (Appendix Table 1).^10,20 For example, penicillin G and clindamycin were among the antibiotics classified as providing anaerobic coverage alone, whereas ceftriaxone was classified as providing Gram-negative coverage alone and ampicillin-sulbactam or as combination therapy with clindamycin and ceftriaxone were classified as providing anaerobic and Gram-negative coverage. Piperacillin-tazobactam and meropenem were classified as providing anaerobic, Gram-negative, and P. aeruginosa coverage. We excluded antibiotics that do not provide coverage against anaerobes, Gram-negative, or P. aeruginosa (eg, ampicillin, azithromycin) or that provide coverage against Gram-negative and P. aeruginosa, but not anaerobes (eg, cefepime, tobramycin), as these therapies were prescribed for <5% of the cohort. We chose not to examine the coverage for Streptococcus pneumonia or Staphylococcus aureus as antibiotics included in this analysis covered these bacteria for 99.9% of our cohort.

Outcomes included acute respiratory failure during hospitalization, intensive care unit (ICU) transfer, and hospital length of stay (LOS). Acute respiratory failure during hospitalization was defined as the presence of Clinical Transaction Classification (CTC) or ICD-9 procedure code for noninvasive or invasive mechanical ventilation on day two or later of hospitalization, with or without the need for respiratory support on day 0 or day 1 (Appendix Table 2). Given the variability in hospital policies that may drive ICU admission criteria for complex patients, our outcome of ICU transfer was defined as the requirement for ICU level care on day two or later of hospitalization without ICU admission. Acute respiratory failure and ICU care occurring within the first two hospital days were not classified as outcomes because these early events likely reflect illness severity at presentation rather than outcomes attributable to treatment failure; these were included as markers of severity in the models.

Patient Demographics and Clinical Characteristics

Demographic and clinical characteristics that might influence antibiotic choice and/or hospital outcomes were assessed. Clinical characteristics included complex chronic conditions,^21-23 medical technology assistance,²⁴ performance of diagnostic testing, and markers of severe illness on presentation. Diagnostic testing included bacterial cultures (blood, respiratory, urine) and chest radiograph performance in the first two days of hospitalization. Results of diagnostic testing are not available in the PHIS. Illness severity on presentation included acute respiratory failure, pleural drainage, receipt of vasoactive agents, and transfusion of blood products in the first two days of hospitalization (Appendix Table 2).^17,25,26

Continuous data were described with median and interquartile ranges (IQR) due to nonnormal distribution. Categorical data were described with frequencies and percentages. Patient demographics, clinical characteristics, and hospital outcomes were stratified by empiric antimicrobial coverage and compared using chi-square and Kruskal–Wallis tests as appropriate.

Generalized linear mixed-effects models with random hospital intercepts were derived to assess the independent effect of antimicrobial spectra of activity on outcomes of acute respiratory failure, ICU transfer, and LOS while adjusting for important differences in demographic and clinical characteristics. LOS had a nonnormal distribution. Thus, we used an exponential distribution. Covariates were chosen a priori given the clinical and biological relevance to exposure and outcomes—age, presence of complex chronic condition diagnoses, the number of complex chronic conditions, technology dependence, the performance of diagnostic tests on presentation, and illness severity on presentation. ICU admission was included as a covariate in acute respiratory failure and LOS outcome models. The results of the model for acute respiratory failure and ICU transfer are presented as adjusted odds ratios (OR) with a 95% CI. LOS results are presented as adjusted rate ratios (RR) with 95% CI.

All analyses were performed with SAS 9.3 (SAS Institute, Cary, North Carolina). P values <.05 were considered statistically significant. Cincinnati Children’s Hospital Medical Center Institutional Review Board considered this deidentified dataset study as not human subjects research.

Study Cohort

At the 44 hospitals included, 4,812 children with NI hospitalized with the diagnosis of aspiration pneumonia met the eligibility criteria. However, 79 received antibiotics with the spectra of activity not examined, leaving 4,733 children in our final analysis (Appendix Figure). Demographic and clinical characteristics of the study cohort are shown in Table 1. Median age was five years (interquartile range [IQR]: 2-11 years). Most subjects were male (53.9%), non-Hispanic white (47.9%), and publicly insured (63.6%). There was a slight variation in the distribution of admissions across seasons (spring 31.6%, summer 19.2%, fall 21.3%, and winter 27.9%). One-third of children had four or more comorbid CCCs (complex chronic conditions; 34.2%). The three most common nonneurologic CCC diagnosis categories were gastrointestinal (63.1%), congenital and/or genetic defects (36.9%), and respiratory (8.9%). Assistance with medical technologies was also common (82%)—particularly gastrointestinal (63.1%) and neurologic/neuromuscular (9.8%) technologies. The vast majority of children (92.5%) had either a chest radiograph (90.5%), respiratory viral study (33.7%), or respiratory culture (10.0%) obtained on presentation. A minority required noninvasive or invasive respiratory support (25.4%), vasoactive agents (8.9%), blood products (1.2%), or pleural drainage (0.3%) in the first two hospital days.

Spectrum of Antimicrobial Coverage

Most children (57.9%) received anaerobic and Gram-negative coverage; 16.2% received anaerobic, Gram-negative and P. aeruginosa coverage; 15.3% received anaerobic coverage alone; and 10.6% received Gram-negative coverage alone. Empiric antimicrobial coverage varied substantially across hospitals: anaerobic coverage was prescribed for 0%-44% of patients; Gram-negative coverage was prescribed for 3%-26% of patients; anaerobic and Gram-negative coverage was prescribed for 25%-90% of patients; and anaerobic, Gram-negative, and P. aeruginosa coverage was prescribed for 0%-65% of patients (Figure 1).

Outcomes

Acute Respiratory Failure

One-quarter (25.4%) of patients had acute respiratory failure on presentation; 22.5% required respiratory support (continued from presentation or were new) on day two or later of hospitalization (Table 2). In the adjusted analysis, children receiving Gram-negative coverage alone had two-fold greater odds (OR 2.15, 95% CI: 1.41-3.27) and children receiving anaerobic and Gram-negative coverage had 1.6-fold greater odds (OR 1.65, 95% CI: 1.19-2.28), of respiratory failure during hospitalization compared with those receiving anaerobic coverage alone (Figure 2). Odds of respiratory failure during hospitalization did not significantly differ for children receiving anaerobic, Gram-negative, and P. aeruginosa coverage compared with those receiving anaerobic coverage alone.

ICU Transfer

Nearly thirty percent (29.0%) of children required ICU admission, with an additional 3.8% requiring ICU transfer following admission (Table 2). In the multivariable analysis, the odds of an ICU transfer were greater for children receiving Gram-negative coverage alone (OR 1.80, 95% CI: 1.03-3.14) compared with those receiving anaerobic coverage alone. There was no statistical difference in ICU transfer for those receiving anaerobic and Gram-negative coverage (with or without P. aeruginosa coverage) compared with those receiving anaerobic coverage alone (Figure 2).

Length of Stay

Median hospital LOS for the total cohort was five days (IQR: 3-9 days; Table 2). In the multivariable analysis, children receiving Gram-negative coverage alone had a longer LOS (RR 1.28; 95% CI: 1.16-1.41) compared with those receiving anaerobic coverage alone, whereas children receiving anaerobic, Gram-negative, and P. aeruginosa coverage had a shorter LOS (RR 0.83; 95% CI: 0.76-0.90) than those receiving anaerobic coverage alone (Figure 2). There was no statistical difference in the LOS between children receiving anaerobic and Gram-negative coverage and those receiving anaerobic coverage alone.

In this multicenter study of children with NI hospitalized with aspiration pneumonia, we found substantial variation in empiric antimicrobial coverage for children with aspiration pneumonia. When comparing outcomes across groups, children who received anaerobic and Gram-negative coverage had outcomes similar to children who received anaerobic therapy alone. However, children who did not receive anaerobic coverage (ie, Gram-negative coverage alone) had worse outcomes, most notably a greater than two-fold increase in the odds of experiencing acute respiratory failure during hospitalization when compared with children receiving anaerobic therapy. These findings support prior literature that has highlighted the importance of anaerobic therapy in the treatment of aspiration pneumonia. The benefit of antibiotics targeting Gram-negative organisms, in addition to anaerobes, remains uncertain.

The variability in empiric antimicrobial coverage likely reflects the paucity of available information on oral and/or enteric bacteria required to identify them as causative organisms in aspiration pneumonia. In part, this problem is due to the difficulty in obtaining adequate sputum for culture from pediatric patients.²⁷ While it may be more feasible to obtain tracheal aspirates for respiratory culture in children with a tracheostomy, interpretation of culture results remains challenging because the lower airways of children with tracheostomy are commonly colonized with bacterial pathogens.²⁸ Thus, physicians are often left to choose empiric antimicrobial coverage with inadequate supporting evidence.²⁹ Although the polymicrobial nature of aspiration pneumonia is well recognized in adult and pediatric literature,^10,30 it is less clear which organisms are of pathological significance and require treatment.

The treatment standard for aspiration pneumonia has long included anaerobic therapy.²⁹ The worse outcomes of children not receiving anaerobic therapy (ie, Gram-negative coverage alone) compared with children who received anaerobic therapy support the continued importance of anaerobic therapy in the treatment of aspiration pneumonia for hospitalized children with NI. The role of antibiotics covering Gram-negative organisms is less clear. Recent studies suggest the role of anaerobes is overemphasized in the etiology and treatment of aspiration pneumonia.^10,29,31-38 Multiple studies on aspiration pneumonia bacteriology in hospitalized adults have demonstrated a predominance of Gram-negative organisms (ranging from 37%-71% of isolates identified on respiratory culture) and a relative scarcity of anaerobes (ranging from 0%-16% of isolates).^31-37 A prospective study of 50 children hospitalized with clinical and radiographic evidence of pneumonia with known aspiration risk (eg, neuromuscular disease or dysphagia) found that ~80% of 163 bacterial isolates were Gram-negative.³⁸ However, this study included repeat cultures from the same children, and thus, may overestimate the prevalence of Gram-negative organisms. In our study, children who received both anaerobic and Gram-negative therapy had no differences in ICU transfer or LOS but did experience higher odds of acute respiratory failure. As these results may be due to unmeasured confounding, future studies should further explore the necessity of Gram-negative coverage in addition to anaerobic coverage in this population.

While these recent studies may seem to suggest that anaerobic coverage is not necessary for aspiration pneumonia, there are important limitations worth noting. First, these studies used a variety of sampling techniques. While organisms grown from samples obtained via bronchoalveolar lavage^31-34,36 are likely pathogenic, those grown from tracheal or oral samples obtained via percutaneous transtracheal aspiration,³⁴ a protected specimen brush,^34,36,37 or expectorated sputum^35,38 may not represent lower airway organisms. Second, anaerobic cultures were not obtained in all studies.^31,34,38 Anaerobic organisms are difficult to isolate using traditional clinical specimen collection techniques and aerobic culture media.¹⁸ Furthermore, anaerobes are not easily recovered from lung infections after the receipt of antibiotic therapy.³⁹ Details regarding pretreatment, which are largely lacking from these studies, are necessary to interpret the relative scarcity of anaerobes on respiratory culture. Finally, caution should be taken when extrapolating the results of studies focused on the etiology and treatment of aspiration pneumonia in elderly adults to children. Our results, particularly in the context of the limitation of these more recent studies, suggest that the role of anaerobes has been underestimated.

Recent studies examining populations of children with cerebral palsy and/or tracheostomy have emphasized the high rates of carriage and infection rates with Gram-negative and drug-resistant bacteria; in particular, P. aeruginosa accounts for 50%-72% of pathogenic bacteria.^11,12,38,40These studies note the generally poor outcomes of children with P. aeruginosa—including multiple and longer hospitalizations, frequent readmissions, and the increased severity of pneumonia, including the need for ICU admission, pleural effusions, the need for intubation, and mortality.^{11,12,38,40,41} In our study, nearly 35% of children who received anaerobic, Gram-negative, and P. aeruginosa coverage experienced acute respiratory failure during hospitalization compared with 20% of children who received other therapies. While these results might seem to suggest that broader spectrum therapy is harmful, they must be interpreted in the context of important population differences; children who received a combination of anaerobic, Gram-negative, and P. aeruginosa coverage had greater medical complexity and greater severity of illness on presentation. Such factors may provide the reason for the appropriate prescription of antipseudomonal antibiotics (eg, history of tracheostomy colonization or infection, long-term care facility resident).⁴² When we controlled for population differences, children who received antipseudomonal therapy had a significantly shorter LOS and no differences in outcomes of acute respiratory failure or ICU transfer compared with those receiving anaerobic therapy alone. This result suggests that worse outcomes were associated with antipseudomonal therapy on unadjusted analyses resulting from underlying medical complexity and illness severity rather than from colonization or infection with P. aeruginosa.

Our multicenter observational study has several limitations. We used diagnosis codes to identify patients with aspiration pneumonia. As validated clinical criteria for the diagnosis of aspiration pneumonia do not exist, clinicians may assign a diagnosis of and treatment for aspiration pneumonia by subjective suspicion based on a child’s severe NI or illness severity on presentation leading to selection bias. Although administrative data are not able to verify pneumonia type with absolute certainty, we previously demonstrated that the differences in the outcomes of children with aspiration and nonaspiration pneumonia diagnosis codes persist after accounting for the complexity that might influence the diagnosis.³It is also possible that the diagnosis of aspiration pneumonia was not made upon admission for a subset of patients leading to misclassification of exposure. Some children may have had aspiration pneumonia on admission but were not assigned that diagnosis or treated for presumed aspiration pneumonia until later in the hospital course as they demonstrated treatment failure or clinical worsening. It is also possible that some children had an aspiration event during hospitalization that developed into aspiration pneumonia. We attempted to adjust for medical complexity and illness severity through multivariable adjustment based on the diagnosis and procedure codes, as well as the laboratory testing performed. However, unmeasured or residual confounding may remain as administrative data are not equipped to distinguish detailed functional status (eg, ability to cough, chest wall strength) or illness severity (eg, respiratory distress) that might influence antibiotic selection and/or outcomes.

Frthermore, we were unable to account for laboratory, microbiology, or radiology test results, and other management practices (eg, frequency of airway clearance, previous antimicrobial therapy) that may influence outcomes. Future studies should certainly include an examination of the concordance of the antibiotics prescribed with causative organisms, as this undoubtedly affects patient outcomes. Other outcomes are important to examine (eg, time to return to respiratory baseline), but we were unable to do so, given the lack of clinical detail in our database. We randomly selected a single hospitalization for children with multiple admissions; alternative methods could have different results. Although children with NI predominately use children’s hospitals,¹ results may not be generalizable.

These findings support prior literature that has highlighted the important role anaerobic therapy plays in the treatment of aspiration pneumonia in children with NI. In light of the limitations of our study design, we believe that rigorous clinical trials comparing anaerobic with anaerobic and Gram-negative therapy are an important and necessary next step to determine the optimal treatment for aspiration pneumonia in this population.

Disclosures

The authors do not have any financial relationships relevant to this article to disclose.

Funding

Dr. Thomson was supported by the Agency for Healthcare Research and Quality (AHRQ) under award number K08HS025138. Dr. Ambroggio was supported by the National Institute for Allergy and Infectious Diseases (NIAID) under award number K01AI125413. The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ or NIAID.

Neurologic impairment (NI) encompasses static and progressive diseases of the central and/or peripheral nervous systems that result in functional and intellectual impairments.¹ While a variety of neurologic diseases are responsible for NI (eg, hypoxic-ischemic encephalopathy, muscular dystrophy), consequences of these diseases extend beyond neurologic manifestations.¹ These children are at an increased risk for aspiration of oral and gastric contents given their common comorbidities of dysphagia, gastroesophageal reflux, impaired cough, and respiratory muscle weakness.² While aspiration may manifest as a self-resolving pneumonitis, the presence of oral or enteric bacteria in aspirated material may result in the development of bacterial pneumonia. Children with NI hospitalized with aspiration pneumonia have higher complication rates, longer and costlier hospitalizations, and higher readmission rates when compared with children with nonaspiration pneumonia.³

While pediatric aspiration pneumonia is commonly attributed to anaerobic bacteria, this is largely based on extrapolation from epidemiologic studies that were conducted in past decades.^4-8 A single randomized controlled trial found that penicillin and clindamycin, antimicrobials with similar antimicrobial activity against anaerobes, to be equally effective.⁹ However, the recent literature emphasizes the polymicrobial nature of aspiration pneumonia in adults, with the common isolation of Gram-negative enteric bacteria.¹⁰ Further, while Pseudomonas aeruginosa is often identified in respiratory cultures from children with NI and chronic respiratory insufficiency,^11,12 the significance of P. aeruginosa in lower airways remains unclear.

We designed this study to compare hospital outcomes associated with the most commonly prescribed empiric antimicrobial therapies for aspiration pneumonia in children with NI.

Study Design and Data Source

This multicenter, retrospective cohort study used the Pediatric Health Information System (PHIS) database. PHIS, an administrative database of 50 not-for-profit tertiary care pediatric hospitals, contains data regarding patient demographics, diagnoses and procedures, and daily billed resource utilization, including laboratory and imaging studies. Data quality and reliability are assured through the Children’s Hospital Association (CHA; Lenexa, Kansas) and participating hospitals. Due to incomplete data through the study period and data quality issues, six hospitals were excluded.

Inclusion Criteria

Children 1-18 years of age who were discharged between July 1, 2007 and June 30, 2015 were included if they had a NI diagnosis,¹ a principal diagnosis indicative of aspiration pneumonia (507.x),^3,13,14 and received antibiotics in the first two calendar days of admission. NI was determined using previously defined International Classification of Diseases, Ninth Revision-Clinical Modification (ICD-9-CM) diagnosis codes.¹ We only included children who received antibiotics in the first two calendar days of admission to minimize the likelihood of including children admitted for other reasons who acquired aspiration pneumonia after hospitalization. For children with multiple hospitalizations, one admission was randomly selected for inclusion to minimize weighting results toward repeat visits.

Exclusion Criteria

Children transferred from another hospital were excluded as records from their initial presentation, including treatment and outcomes, were not available. We also excluded children with tracheostomy^15,16 or chronic ventilator dependence,¹⁷ those with a diagnosis of human immunodeficiency virus or tuberculosis, and children who received chemotherapy during hospitalization given expected differences in etiology, treatment, and outcomes.¹⁸

Exposure

The primary exposure was antibiotic therapy received in the first two days of admission. Antibiotics were classified by their antimicrobial spectra of activity as defined by The Sanford Guide to Antimicrobial Therapy¹⁹ against the most commonly recognized pathogens of aspiration pneumonia: anaerobes, Gram-negatives, and P. aeruginosa (Appendix Table 1).^10,20 For example, penicillin G and clindamycin were among the antibiotics classified as providing anaerobic coverage alone, whereas ceftriaxone was classified as providing Gram-negative coverage alone and ampicillin-sulbactam or as combination therapy with clindamycin and ceftriaxone were classified as providing anaerobic and Gram-negative coverage. Piperacillin-tazobactam and meropenem were classified as providing anaerobic, Gram-negative, and P. aeruginosa coverage. We excluded antibiotics that do not provide coverage against anaerobes, Gram-negative, or P. aeruginosa (eg, ampicillin, azithromycin) or that provide coverage against Gram-negative and P. aeruginosa, but not anaerobes (eg, cefepime, tobramycin), as these therapies were prescribed for <5% of the cohort. We chose not to examine the coverage for Streptococcus pneumonia or Staphylococcus aureus as antibiotics included in this analysis covered these bacteria for 99.9% of our cohort.

Outcomes included acute respiratory failure during hospitalization, intensive care unit (ICU) transfer, and hospital length of stay (LOS). Acute respiratory failure during hospitalization was defined as the presence of Clinical Transaction Classification (CTC) or ICD-9 procedure code for noninvasive or invasive mechanical ventilation on day two or later of hospitalization, with or without the need for respiratory support on day 0 or day 1 (Appendix Table 2). Given the variability in hospital policies that may drive ICU admission criteria for complex patients, our outcome of ICU transfer was defined as the requirement for ICU level care on day two or later of hospitalization without ICU admission. Acute respiratory failure and ICU care occurring within the first two hospital days were not classified as outcomes because these early events likely reflect illness severity at presentation rather than outcomes attributable to treatment failure; these were included as markers of severity in the models.

Patient Demographics and Clinical Characteristics

Demographic and clinical characteristics that might influence antibiotic choice and/or hospital outcomes were assessed. Clinical characteristics included complex chronic conditions,^21-23 medical technology assistance,²⁴ performance of diagnostic testing, and markers of severe illness on presentation. Diagnostic testing included bacterial cultures (blood, respiratory, urine) and chest radiograph performance in the first two days of hospitalization. Results of diagnostic testing are not available in the PHIS. Illness severity on presentation included acute respiratory failure, pleural drainage, receipt of vasoactive agents, and transfusion of blood products in the first two days of hospitalization (Appendix Table 2).^17,25,26

Continuous data were described with median and interquartile ranges (IQR) due to nonnormal distribution. Categorical data were described with frequencies and percentages. Patient demographics, clinical characteristics, and hospital outcomes were stratified by empiric antimicrobial coverage and compared using chi-square and Kruskal–Wallis tests as appropriate.

Generalized linear mixed-effects models with random hospital intercepts were derived to assess the independent effect of antimicrobial spectra of activity on outcomes of acute respiratory failure, ICU transfer, and LOS while adjusting for important differences in demographic and clinical characteristics. LOS had a nonnormal distribution. Thus, we used an exponential distribution. Covariates were chosen a priori given the clinical and biological relevance to exposure and outcomes—age, presence of complex chronic condition diagnoses, the number of complex chronic conditions, technology dependence, the performance of diagnostic tests on presentation, and illness severity on presentation. ICU admission was included as a covariate in acute respiratory failure and LOS outcome models. The results of the model for acute respiratory failure and ICU transfer are presented as adjusted odds ratios (OR) with a 95% CI. LOS results are presented as adjusted rate ratios (RR) with 95% CI.

All analyses were performed with SAS 9.3 (SAS Institute, Cary, North Carolina). P values <.05 were considered statistically significant. Cincinnati Children’s Hospital Medical Center Institutional Review Board considered this deidentified dataset study as not human subjects research.

Study Cohort

At the 44 hospitals included, 4,812 children with NI hospitalized with the diagnosis of aspiration pneumonia met the eligibility criteria. However, 79 received antibiotics with the spectra of activity not examined, leaving 4,733 children in our final analysis (Appendix Figure). Demographic and clinical characteristics of the study cohort are shown in Table 1. Median age was five years (interquartile range [IQR]: 2-11 years). Most subjects were male (53.9%), non-Hispanic white (47.9%), and publicly insured (63.6%). There was a slight variation in the distribution of admissions across seasons (spring 31.6%, summer 19.2%, fall 21.3%, and winter 27.9%). One-third of children had four or more comorbid CCCs (complex chronic conditions; 34.2%). The three most common nonneurologic CCC diagnosis categories were gastrointestinal (63.1%), congenital and/or genetic defects (36.9%), and respiratory (8.9%). Assistance with medical technologies was also common (82%)—particularly gastrointestinal (63.1%) and neurologic/neuromuscular (9.8%) technologies. The vast majority of children (92.5%) had either a chest radiograph (90.5%), respiratory viral study (33.7%), or respiratory culture (10.0%) obtained on presentation. A minority required noninvasive or invasive respiratory support (25.4%), vasoactive agents (8.9%), blood products (1.2%), or pleural drainage (0.3%) in the first two hospital days.

Spectrum of Antimicrobial Coverage

Most children (57.9%) received anaerobic and Gram-negative coverage; 16.2% received anaerobic, Gram-negative and P. aeruginosa coverage; 15.3% received anaerobic coverage alone; and 10.6% received Gram-negative coverage alone. Empiric antimicrobial coverage varied substantially across hospitals: anaerobic coverage was prescribed for 0%-44% of patients; Gram-negative coverage was prescribed for 3%-26% of patients; anaerobic and Gram-negative coverage was prescribed for 25%-90% of patients; and anaerobic, Gram-negative, and P. aeruginosa coverage was prescribed for 0%-65% of patients (Figure 1).

Outcomes

Acute Respiratory Failure

One-quarter (25.4%) of patients had acute respiratory failure on presentation; 22.5% required respiratory support (continued from presentation or were new) on day two or later of hospitalization (Table 2). In the adjusted analysis, children receiving Gram-negative coverage alone had two-fold greater odds (OR 2.15, 95% CI: 1.41-3.27) and children receiving anaerobic and Gram-negative coverage had 1.6-fold greater odds (OR 1.65, 95% CI: 1.19-2.28), of respiratory failure during hospitalization compared with those receiving anaerobic coverage alone (Figure 2). Odds of respiratory failure during hospitalization did not significantly differ for children receiving anaerobic, Gram-negative, and P. aeruginosa coverage compared with those receiving anaerobic coverage alone.

ICU Transfer

Nearly thirty percent (29.0%) of children required ICU admission, with an additional 3.8% requiring ICU transfer following admission (Table 2). In the multivariable analysis, the odds of an ICU transfer were greater for children receiving Gram-negative coverage alone (OR 1.80, 95% CI: 1.03-3.14) compared with those receiving anaerobic coverage alone. There was no statistical difference in ICU transfer for those receiving anaerobic and Gram-negative coverage (with or without P. aeruginosa coverage) compared with those receiving anaerobic coverage alone (Figure 2).

Length of Stay

Median hospital LOS for the total cohort was five days (IQR: 3-9 days; Table 2). In the multivariable analysis, children receiving Gram-negative coverage alone had a longer LOS (RR 1.28; 95% CI: 1.16-1.41) compared with those receiving anaerobic coverage alone, whereas children receiving anaerobic, Gram-negative, and P. aeruginosa coverage had a shorter LOS (RR 0.83; 95% CI: 0.76-0.90) than those receiving anaerobic coverage alone (Figure 2). There was no statistical difference in the LOS between children receiving anaerobic and Gram-negative coverage and those receiving anaerobic coverage alone.

In this multicenter study of children with NI hospitalized with aspiration pneumonia, we found substantial variation in empiric antimicrobial coverage for children with aspiration pneumonia. When comparing outcomes across groups, children who received anaerobic and Gram-negative coverage had outcomes similar to children who received anaerobic therapy alone. However, children who did not receive anaerobic coverage (ie, Gram-negative coverage alone) had worse outcomes, most notably a greater than two-fold increase in the odds of experiencing acute respiratory failure during hospitalization when compared with children receiving anaerobic therapy. These findings support prior literature that has highlighted the importance of anaerobic therapy in the treatment of aspiration pneumonia. The benefit of antibiotics targeting Gram-negative organisms, in addition to anaerobes, remains uncertain.

The variability in empiric antimicrobial coverage likely reflects the paucity of available information on oral and/or enteric bacteria required to identify them as causative organisms in aspiration pneumonia. In part, this problem is due to the difficulty in obtaining adequate sputum for culture from pediatric patients.²⁷ While it may be more feasible to obtain tracheal aspirates for respiratory culture in children with a tracheostomy, interpretation of culture results remains challenging because the lower airways of children with tracheostomy are commonly colonized with bacterial pathogens.²⁸ Thus, physicians are often left to choose empiric antimicrobial coverage with inadequate supporting evidence.²⁹ Although the polymicrobial nature of aspiration pneumonia is well recognized in adult and pediatric literature,^10,30 it is less clear which organisms are of pathological significance and require treatment.

The treatment standard for aspiration pneumonia has long included anaerobic therapy.²⁹ The worse outcomes of children not receiving anaerobic therapy (ie, Gram-negative coverage alone) compared with children who received anaerobic therapy support the continued importance of anaerobic therapy in the treatment of aspiration pneumonia for hospitalized children with NI. The role of antibiotics covering Gram-negative organisms is less clear. Recent studies suggest the role of anaerobes is overemphasized in the etiology and treatment of aspiration pneumonia.^10,29,31-38 Multiple studies on aspiration pneumonia bacteriology in hospitalized adults have demonstrated a predominance of Gram-negative organisms (ranging from 37%-71% of isolates identified on respiratory culture) and a relative scarcity of anaerobes (ranging from 0%-16% of isolates).^31-37 A prospective study of 50 children hospitalized with clinical and radiographic evidence of pneumonia with known aspiration risk (eg, neuromuscular disease or dysphagia) found that ~80% of 163 bacterial isolates were Gram-negative.³⁸ However, this study included repeat cultures from the same children, and thus, may overestimate the prevalence of Gram-negative organisms. In our study, children who received both anaerobic and Gram-negative therapy had no differences in ICU transfer or LOS but did experience higher odds of acute respiratory failure. As these results may be due to unmeasured confounding, future studies should further explore the necessity of Gram-negative coverage in addition to anaerobic coverage in this population.

While these recent studies may seem to suggest that anaerobic coverage is not necessary for aspiration pneumonia, there are important limitations worth noting. First, these studies used a variety of sampling techniques. While organisms grown from samples obtained via bronchoalveolar lavage^31-34,36 are likely pathogenic, those grown from tracheal or oral samples obtained via percutaneous transtracheal aspiration,³⁴ a protected specimen brush,^34,36,37 or expectorated sputum^35,38 may not represent lower airway organisms. Second, anaerobic cultures were not obtained in all studies.^31,34,38 Anaerobic organisms are difficult to isolate using traditional clinical specimen collection techniques and aerobic culture media.¹⁸ Furthermore, anaerobes are not easily recovered from lung infections after the receipt of antibiotic therapy.³⁹ Details regarding pretreatment, which are largely lacking from these studies, are necessary to interpret the relative scarcity of anaerobes on respiratory culture. Finally, caution should be taken when extrapolating the results of studies focused on the etiology and treatment of aspiration pneumonia in elderly adults to children. Our results, particularly in the context of the limitation of these more recent studies, suggest that the role of anaerobes has been underestimated.

Recent studies examining populations of children with cerebral palsy and/or tracheostomy have emphasized the high rates of carriage and infection rates with Gram-negative and drug-resistant bacteria; in particular, P. aeruginosa accounts for 50%-72% of pathogenic bacteria.^11,12,38,40These studies note the generally poor outcomes of children with P. aeruginosa—including multiple and longer hospitalizations, frequent readmissions, and the increased severity of pneumonia, including the need for ICU admission, pleural effusions, the need for intubation, and mortality.^{11,12,38,40,41} In our study, nearly 35% of children who received anaerobic, Gram-negative, and P. aeruginosa coverage experienced acute respiratory failure during hospitalization compared with 20% of children who received other therapies. While these results might seem to suggest that broader spectrum therapy is harmful, they must be interpreted in the context of important population differences; children who received a combination of anaerobic, Gram-negative, and P. aeruginosa coverage had greater medical complexity and greater severity of illness on presentation. Such factors may provide the reason for the appropriate prescription of antipseudomonal antibiotics (eg, history of tracheostomy colonization or infection, long-term care facility resident).⁴² When we controlled for population differences, children who received antipseudomonal therapy had a significantly shorter LOS and no differences in outcomes of acute respiratory failure or ICU transfer compared with those receiving anaerobic therapy alone. This result suggests that worse outcomes were associated with antipseudomonal therapy on unadjusted analyses resulting from underlying medical complexity and illness severity rather than from colonization or infection with P. aeruginosa.

Our multicenter observational study has several limitations. We used diagnosis codes to identify patients with aspiration pneumonia. As validated clinical criteria for the diagnosis of aspiration pneumonia do not exist, clinicians may assign a diagnosis of and treatment for aspiration pneumonia by subjective suspicion based on a child’s severe NI or illness severity on presentation leading to selection bias. Although administrative data are not able to verify pneumonia type with absolute certainty, we previously demonstrated that the differences in the outcomes of children with aspiration and nonaspiration pneumonia diagnosis codes persist after accounting for the complexity that might influence the diagnosis.³It is also possible that the diagnosis of aspiration pneumonia was not made upon admission for a subset of patients leading to misclassification of exposure. Some children may have had aspiration pneumonia on admission but were not assigned that diagnosis or treated for presumed aspiration pneumonia until later in the hospital course as they demonstrated treatment failure or clinical worsening. It is also possible that some children had an aspiration event during hospitalization that developed into aspiration pneumonia. We attempted to adjust for medical complexity and illness severity through multivariable adjustment based on the diagnosis and procedure codes, as well as the laboratory testing performed. However, unmeasured or residual confounding may remain as administrative data are not equipped to distinguish detailed functional status (eg, ability to cough, chest wall strength) or illness severity (eg, respiratory distress) that might influence antibiotic selection and/or outcomes.

Frthermore, we were unable to account for laboratory, microbiology, or radiology test results, and other management practices (eg, frequency of airway clearance, previous antimicrobial therapy) that may influence outcomes. Future studies should certainly include an examination of the concordance of the antibiotics prescribed with causative organisms, as this undoubtedly affects patient outcomes. Other outcomes are important to examine (eg, time to return to respiratory baseline), but we were unable to do so, given the lack of clinical detail in our database. We randomly selected a single hospitalization for children with multiple admissions; alternative methods could have different results. Although children with NI predominately use children’s hospitals,¹ results may not be generalizable.

These findings support prior literature that has highlighted the important role anaerobic therapy plays in the treatment of aspiration pneumonia in children with NI. In light of the limitations of our study design, we believe that rigorous clinical trials comparing anaerobic with anaerobic and Gram-negative therapy are an important and necessary next step to determine the optimal treatment for aspiration pneumonia in this population.

Disclosures

The authors do not have any financial relationships relevant to this article to disclose.

Funding

Dr. Thomson was supported by the Agency for Healthcare Research and Quality (AHRQ) under award number K08HS025138. Dr. Ambroggio was supported by the National Institute for Allergy and Infectious Diseases (NIAID) under award number K01AI125413. The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ or NIAID.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

Hospital readmission rates are a common metric for defining, evaluating, and benchmarking quality of care. The Centers for Medicare and Medicaid Services (CMS) publicly report hospital readmission rates for common adult conditions and reduces payments to hospitals with excessive readmissions.¹ Recently, the focus on pediatric readmission rates has increased and the National Quality Forum (NQF) has endorsed at least two pediatric readmission-specific quality indicators which could be used by public and private payers in pay-for-performance programs aimed at institutions caring for children.² While preventability of readmissions and their value as a marker of quality remains debated, their acceptance by the NQF and CMS has led public and private payers to propose readmission-related penalties for hospitals caring for children. ^3-5

All-cause 30-day same-hospital readmission rates for pediatric conditions are half of the adult readmission rates, around 6% in most studies, compared to 12% in adults.^6,7 The lower rates of pediatric readmissions makes it difficult to only use mean readmission rates to stratify hospitals into high- or low-performers and set target goals for improvement.⁸ While adult readmissions have been studied in depth, there are no consistent measures used to benchmark pediatric readmissions across hospital types.

Given the emphasis placed on readmissions, it is essential to understand patterns in pediatric readmission rates to determine optimal and achievable targets for improvement. Achievable Benchmarks of Care (ABCs) are one approach to understanding readmission rates and have an advantage over using mean or medians in performance improvement as they can stratify performance for conditions with low readmission rates and low volumes.⁹ When creating benchmarks, it is important that hospitals performance is evaluated among peer hospitals with similar patient populations, not just a cumulative average from all hospital types which may punish hospitals with a more complex patient case mix.¹⁰ The goal of this study was to calculate the readmission rates and the ABCs for common pediatric diagnoses by hospital type to identify priority conditions for quality improvement efforts using a previously published methodology.^11-13

Data Source

We conducted a retrospective analysis of patients less than 18 years of age in the Healthcare Utilization Project 2014 Nationwide Readmissions Database (NRD). The NRD includes public hospitals; academic medical centers; and specialty hospitals in obstetrics and gynecology, otolaryngology, orthopedics, and cancer; and pediatric, public, and academic medical hospitals. Excluded are long-term care facilities such as rehabilitation, long-term acute care, psychiatric, alcoholism, and chemical dependency hospitals. The readmissions data contains information from hospitals grouped by region, population census, and teaching status.¹⁴ Three hospital type classifications used in this study were metropolitan teaching hospitals, metropolitan nonteaching hospitals, and nonmetropolitan hospitals. These three hospital type classifications follow the reporting format in the NRD.

Study Population

Patients less than 18 years old were included if they were discharged from January 1, 2014 through November 30, 2014 and had a readmission to the index hospital within 30 days. We limited inclusion to discharges through November 30 so we could identify patients with a 30-day readmission as patient identifiers do not link across years in the NRD.

Exposure

We included 30-day, all-cause, same-hospital readmissions to the index acute care hospital, excluding labor and delivery, normal newborn care, chemotherapy, transfers, and mortalities. Intrahospital discharge and admissions within the same hospital system were not defined as a readmission, but rather as a “same-day event.”¹⁵ For example, institutions with inpatient mental health facilities, medical unit discharges and admission to the mental health unit were not identified as a readmission in this dataset.

Outcome

For each hospital type, we measured same-hospital, all-cause, 30-day readmission rates and achievable benchmark of care for the 17 most commonly readmitted pediatric discharge diagnoses. To identify the target readmission diagnoses and all-cause, 30-day readmissions based on their index hospitalizations, All-Patient Refined Diagnosis-Related Groups (APR-DRG), version 25 (3M Health Information Systems, Salt Lake City, Utah) were ordered by frequency for each hospital type. The 20 most common APR-DRGs were the same across all hospital types. The authors then evaluated these 20 APR-DRGs for clinical consistency of included diagnoses identified by the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes within each APR-DRG. Three diagnosis-related groups were excluded from the analysis (major hematologic/immunologic disease except for sickle cell, other anemia and disorders of blood and blood forming organs, and other digestive system diagnoses) due to the heterogeneity of the diagnoses identified by the ICD-9-CM codes within each APR-DRG. We refer to each APR-DRG as a “diagnosis” throughout the article.

Analysis

The demographic characteristics of the patients seen at the three hospital types were summarized using frequencies and percentages. Reports were generated for patient age, gender, payer source, patient residence, median household income, patient complexity, and discharge disposition. Patient complexity was defined using complex chronic condition (CCC) and the number of chronic conditions (CCI).^16,17 As previously defined in the literature, a complex chronic condition is “any medical condition that can be reasonably expected to last at least 12 months (unless death intervenes) and to involve either several different organ systems or one organ system severely enough to require specialty pediatric care and probably some period of hospitalization in a tertiary care center.”¹⁶ Whereas, the Agency for Healthcare Research and Quality’s Chronic Condition Indicator (CCI) defines single, non-CCCs (eg, allergic rhinitis).¹⁷

For each diagnosis, we calculated the mean readmission rate for hospitals in each hospital type category. We then calculated an ABC for each diagnosis in each hospital type using a four-step process.^13,18 First, to control for hospitals with small sample sizes, we adjusted all readmission rates using an adjusted performance fraction ([numerator+1]/[denominator +2]), where the numerator is the number of all-cause 30-day readmissions and the denominator is the number of discharges for the selected diagnosis. Then the hospitals were ordered from lowest (best performing) to highest (worst performing) using the adjusted readmission rate. Third, the number of discharges from the best performing hospital to the worst performing hospital was summed until at least 10% of the total discharges had been accounted for. Finally, we computed the ABC as the average of these best performing hospitals. We only report ABCs for which at least three hospitals were included as best performers in the calculation.¹³

To evaluate hospital performance on ABCs for each diagnosis, we identified the percent of hospitals in each setting that were outliers. We defined an outlier as any hospital whose 95% confidence interval for their readmission rate for a given diagnosis did not contain the ABC for their hospital type. All the statistical analyses were performed using SAS version 9.3 (SAS Institute, Inc, Cary, North Carolina).

This project was reviewed by the Cincinnati Children’s Hospital Medical Center Institutional Review Board and determined to be nonhuman subjects research.

Hospital-Type Demographics

The 690,949 discharges from 1,664 hospitals were categorized into 525 metropolitan teaching (550,039 discharges, 79.6% of discharges), 552 metropolitan nonteaching (97,207 discharges, 14% of discharges), and 587 nonmetropolitan hospitals (43,703 discharges, 6.3% of discharges; Table 1). There were significant differences in the patient composition among the three hospital settings. Nonmetropolitan hospitals had a larger percentage of younger patients (aged 0-4 years, P < .001), prominence of first and second quartile median household income, and fewer medically complex patients (48.3% No CCC/No CCI versus 25.5% metropolitan teaching and 33.7% nonteaching, P < .001). Disposition home was over 96% in all three hospital types; however, the metropolitan teaching had a greater percentage of patients discharged to home health versus metropolitan nonteaching and nonmetropolitan hospitals (2.3% versus 0.5%; P < .001).

Readmission Rates

The 17 most common diagnoses based on the number of all-cause 30-day same-hospital readmissions, were categorized into two surgical, seven acute/infectious, four chronic, and four mental health diagnoses (Table 2). Readmission rates varied based on diagnosis and hospital type (Table 2). Overall, mean readmission rates were low, especially in acute respiratory tract related diseases. For chronic diseases, asthma readmissions were consistently low in all three hospital types, whereas sickle cell disease had the highest readmission rate in all three hospital types.

Achievable Benchmarks of Care by Hospital Type

The diagnoses for which ABC could be calculated across all three hospital types included appendectomy and four acute conditions (bronchiolitis, pneumonia, nonbacterial gastroenteritis, and kidney/urinary tract infections). For these conditions, metropolitan teaching hospitals had a more significant percentage of outlier hospitals compared to metropolitan nonteaching and nonmetropolitan hospitals. The percent of outlier hospitals varied by diagnosis and hospital type (Figure).

Metropolitan Teaching

The readmission ABC was calculated for all 17 diagnoses (Table 2). The ABC ranged from 0.4% in acute kidney and urinary tract infection to 7.0% in sickle cell anemia crisis. Bipolar disorder, major depressive disorders and other psychoses, and sickle cell disease (SCD) had the highest percent of outlier hospitals whose mean readmission rates confidence interval did not contain the ABC; tonsil and adenoid procedures and viral illness had the lowest.¹

Metropolitan Nonteaching

The ABC was calculated for 13 of the 17 diagnoses because ABCs were not calculated when there were fewer than three best practicing hospitals. This was the case for tonsil and adenoid procedures, diabetes, seizures, and depression except for major depressive disorder (Table 2). Seven of the 13 diagnoses had an ABC of 0.0%: viral illness, infections of the upper respiratory tract, bronchiolitis, gastroenteritis, hypovolemia and electrolyte disorders, asthma, and childhood behavioral disorders. Like the findings at the metropolitan teaching hospitals, ABCs were lowest for surgical and acute conditions while bipolar disorder, major depressive disorders and other psychoses, and SCD had the highest percent of outlier hospitals with readmission rates beyond the 95% confidence interval of their hospital type’s ABC.

Nonmetropolitan

There was a sufficient number of best practicing hospitals to calculate the ABC for six of the 17 diagnoses (Table 2). For conditions where readmission ABCs could be calculated, they were low: 0.0% for appendectomy, bronchiolitis, gastroenteritis, and seizure; 0.3% for pneumonia; and 1.3% in kidney and urinary tract disorders. None of the conditions with the highest ABCs in other hospital settings (bipolar disease, sickle cell anemia crisis, and major depressive disorders and other psychoses) could be calculated in this setting. Seizure-related readmissions exhibited the most outlier hospitals yet were less than 5%.¹

Among a nationally representative sample of different hospital types that deliver care to children, we report the mean readmission rates and ABCs for 30-day all-cause, same-hospital readmissions for the most commonly readmitted pediatric diagnoses based on hospital type. Previous studies have shown patient variables such as race, ethnicity, and insurance type influencing readmission rates.^19,20 However, hospital type has also been associated with a higher risk of readmission due to the varying complexity of patients at different hospital types.^21,22 Our analyses provide hospital-type specific national estimates of pediatric readmission ABCs for medical and surgical conditions, many less than 1%. While commonly encountered pediatric conditions like asthma and bronchiolitis had low mean readmission rates and ABCs across all hospital types, the mean rates and ABCs for SCD and mental health disorders were much higher with more hospitals performing far from the ABCs.

Diagnoses with a larger percentage of outlier hospitals may represent a national opportunity to improve care for children. Conditions such as SCD and mental illnesses have the highest percentage of hospitals whose readmission rates fall outside of the ABCs in both metropolitan teaching and metropolitan nonteaching hospitals. Hospital performance on SCD and mental health disorders may not reflect deficits in hospital quality or poor adherence to evidence-based best practices, but rather the complex interplay of factors on various levels from government policy and insurance plans, to patient and family resources, to access and availability of medical and mental health specific care. Most importantly, these diseases may represent a significant opportunity for quality improvementin hospitals across the United States.

Sickle cell disease is predominantly a disease among African-Americans, a demographic risk factor for decreased access to care and limited patient and family resources.^23-26 In previous studies evaluating the disparity in readmission rates for Black children with asthma, socioeconomic variables explained 53% of the observed disparity and readmission rates were inversely related to the childhood opportunity index of the patient’s census tract and positively related with geographic social risk.^27,28 Likewise, with SCD affecting a specific demographic and being a chronic disease, best practice policies need to account for the child’s medical needs and include the patient and family resources to ensure access to care and enhanced case management for chronic disease if we aim to improve performance among the outlier hospitals.

Similarly, barriers to care for children with mental illnesses in the United States need attention.^29,30 While there is a paucity of data on the prevalence of mental health disorders in children, one national report estimates that one in 10 American adolescents have depression.^29,31 The American Academy of Pediatrics has developed a policy statement on mental health competencies and a mental health tool-kit for primary care pediatricians; however, no such guidelines or policy statements exist for hospitalized patients with acute or chronic psychiatric conditions.^32,33 Moreover, hospitals are increasingly facing “boarding” of children with acute psychiatric illness in inpatient units and emergency departments.³⁴ The American Medical Association and the American College of Emergency Physicians have expressed concerns regarding the boarding of children with acute psychiatric illness because nonpsychiatric hospitals do not have adequate resources to evaluate, manage, and place these children who deserve appropriate facilities for further management. Coordinated case management and “bundled” discharge planning in other chronic illnesses have shown benefit in cost reduction and readmission.^35-37 Evidence-based practices around pediatric readmissions in other diagnoses should be explored as possible interventions in these conditions.³⁸

There are several limitations to this study. Our data is limited to one calendar year; therefore, admissions in January do not account for potential readmissions from December of the previous year, as patient identifiers do not link across years in the NRD. We also limited our evaluation to the conventional 30-day readmission window, but recent publications may indicate that readmission windows with different timelines could be a more accurate reflection of medically preventable readmissions versus a reflection of social determinants of health leading to readmissions.²⁴ Newborn index admissions were not an allowable index admission; therefore, we may be underreporting readmissions in the neonatal age group. We also chose to include all-cause readmissions, a conventional method to evaluate readmission within an institution, but which may not reflect the quality of care delivered in the index admission. For example, an asthmatic discharged after an acute exacerbation readmitted for dehydration secondary to gastroenteritis may not reflect a lack of quality in asthma inpatient care. Readmissions were limited to the same hospital; therefore, this study cannot account for readmissions at other institutions, which may cause us to underestimate readmission rates. However, end-users of our findings most likely have access only to their own institution’s data. The inclusion of observation status admissions in the database varies from state to state; therefore, this percent of admissions in the database is unknown.

The use of the ABC methodology has some inherent limitations. One hospital with a significant volume diagnosis and low readmission rate within a hospital type may prohibit the reporting of an ABC if less than three hospitals composed the total of the ‘best performing’ hospitals. This was a significant limitation leading to the exclusion of many ABCs in nonmetropolitan institutions. The limitation of calculating and reporting an ABC then prohibits the calculation of outlier hospitals within a hospital type for a given diagnosis. However, when the ABCs are not available, we do provide the mean readmission rate for the diagnosis within the hospital type. While the hospital groupings by population and teaching status for ABCs provide meaningful comparisons for within each hospital setting, it should be noted that there may be vast differences among hospitals within each type (eg, tertiary children’s hospitals compared to teaching hospitals with a pediatric floor in the metropolitan teaching hospital category).^39,40

As healthcare moves from a fee-for-service model to a population-health centered, value-based model, reduction in readmission rates will be more than a quality measure and will have potential financial implications.⁴¹ In the Medicare fee-for-service patients, the Hospital Readmission Reduction Program (HRRP) penalize hospitals with excess readmissions for acute myocardial infarction, heart failure, and pneumonia. The hospitals subject to penalties in the HRRP had greater reduction in readmission rates in the targeted, and even nontargeted conditions, compared with hospitals not subject to penalties.⁴² Similarly, we believe that our data on low readmission rates and ABCs for conditions such as asthma, bronchiolitis, and appendicitis could represent decades of quality improvement work for the most common pediatric conditions among hospitalized children. Sickle cell disease and mental health problems remain as outliers and merit further attention. To move to a true population-health model, hospitals will need to explore outlier conditions including evaluating patient-level readmission patterns across institutions. This moves readmission from a hospital quality measure to a patient-centric quality measure, and perhaps will provide value to the patient and the healthcare system alike.

The readmission ABCs for the most commonly readmitted pediatric diagnoses are low, regardless of the hospital setting. The highest pediatric readmission rates in SCD, bipolar disorders, and major depressive disorder were lower than the most common adult readmission diagnoses. However, mental health conditions and SCD remain as outliers for pediatric readmissions, burden hospital systems, and perhaps warrant national-level attention. The ABCs stratified by hospital type in this study facilitate comparisons and identify opportunities for population-level interventions to meaningfully improve patient care.

Disclosures

The authors have nothing to disclose.

Hospital readmission rates are a common metric for defining, evaluating, and benchmarking quality of care. The Centers for Medicare and Medicaid Services (CMS) publicly report hospital readmission rates for common adult conditions and reduces payments to hospitals with excessive readmissions.¹ Recently, the focus on pediatric readmission rates has increased and the National Quality Forum (NQF) has endorsed at least two pediatric readmission-specific quality indicators which could be used by public and private payers in pay-for-performance programs aimed at institutions caring for children.² While preventability of readmissions and their value as a marker of quality remains debated, their acceptance by the NQF and CMS has led public and private payers to propose readmission-related penalties for hospitals caring for children. ^3-5

All-cause 30-day same-hospital readmission rates for pediatric conditions are half of the adult readmission rates, around 6% in most studies, compared to 12% in adults.^6,7 The lower rates of pediatric readmissions makes it difficult to only use mean readmission rates to stratify hospitals into high- or low-performers and set target goals for improvement.⁸ While adult readmissions have been studied in depth, there are no consistent measures used to benchmark pediatric readmissions across hospital types.

Given the emphasis placed on readmissions, it is essential to understand patterns in pediatric readmission rates to determine optimal and achievable targets for improvement. Achievable Benchmarks of Care (ABCs) are one approach to understanding readmission rates and have an advantage over using mean or medians in performance improvement as they can stratify performance for conditions with low readmission rates and low volumes.⁹ When creating benchmarks, it is important that hospitals performance is evaluated among peer hospitals with similar patient populations, not just a cumulative average from all hospital types which may punish hospitals with a more complex patient case mix.¹⁰ The goal of this study was to calculate the readmission rates and the ABCs for common pediatric diagnoses by hospital type to identify priority conditions for quality improvement efforts using a previously published methodology.^11-13

Data Source

We conducted a retrospective analysis of patients less than 18 years of age in the Healthcare Utilization Project 2014 Nationwide Readmissions Database (NRD). The NRD includes public hospitals; academic medical centers; and specialty hospitals in obstetrics and gynecology, otolaryngology, orthopedics, and cancer; and pediatric, public, and academic medical hospitals. Excluded are long-term care facilities such as rehabilitation, long-term acute care, psychiatric, alcoholism, and chemical dependency hospitals. The readmissions data contains information from hospitals grouped by region, population census, and teaching status.¹⁴ Three hospital type classifications used in this study were metropolitan teaching hospitals, metropolitan nonteaching hospitals, and nonmetropolitan hospitals. These three hospital type classifications follow the reporting format in the NRD.

Study Population

Patients less than 18 years old were included if they were discharged from January 1, 2014 through November 30, 2014 and had a readmission to the index hospital within 30 days. We limited inclusion to discharges through November 30 so we could identify patients with a 30-day readmission as patient identifiers do not link across years in the NRD.

Exposure

We included 30-day, all-cause, same-hospital readmissions to the index acute care hospital, excluding labor and delivery, normal newborn care, chemotherapy, transfers, and mortalities. Intrahospital discharge and admissions within the same hospital system were not defined as a readmission, but rather as a “same-day event.”¹⁵ For example, institutions with inpatient mental health facilities, medical unit discharges and admission to the mental health unit were not identified as a readmission in this dataset.

Outcome

For each hospital type, we measured same-hospital, all-cause, 30-day readmission rates and achievable benchmark of care for the 17 most commonly readmitted pediatric discharge diagnoses. To identify the target readmission diagnoses and all-cause, 30-day readmissions based on their index hospitalizations, All-Patient Refined Diagnosis-Related Groups (APR-DRG), version 25 (3M Health Information Systems, Salt Lake City, Utah) were ordered by frequency for each hospital type. The 20 most common APR-DRGs were the same across all hospital types. The authors then evaluated these 20 APR-DRGs for clinical consistency of included diagnoses identified by the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes within each APR-DRG. Three diagnosis-related groups were excluded from the analysis (major hematologic/immunologic disease except for sickle cell, other anemia and disorders of blood and blood forming organs, and other digestive system diagnoses) due to the heterogeneity of the diagnoses identified by the ICD-9-CM codes within each APR-DRG. We refer to each APR-DRG as a “diagnosis” throughout the article.

Analysis

The demographic characteristics of the patients seen at the three hospital types were summarized using frequencies and percentages. Reports were generated for patient age, gender, payer source, patient residence, median household income, patient complexity, and discharge disposition. Patient complexity was defined using complex chronic condition (CCC) and the number of chronic conditions (CCI).^16,17 As previously defined in the literature, a complex chronic condition is “any medical condition that can be reasonably expected to last at least 12 months (unless death intervenes) and to involve either several different organ systems or one organ system severely enough to require specialty pediatric care and probably some period of hospitalization in a tertiary care center.”¹⁶ Whereas, the Agency for Healthcare Research and Quality’s Chronic Condition Indicator (CCI) defines single, non-CCCs (eg, allergic rhinitis).¹⁷

For each diagnosis, we calculated the mean readmission rate for hospitals in each hospital type category. We then calculated an ABC for each diagnosis in each hospital type using a four-step process.^13,18 First, to control for hospitals with small sample sizes, we adjusted all readmission rates using an adjusted performance fraction ([numerator+1]/[denominator +2]), where the numerator is the number of all-cause 30-day readmissions and the denominator is the number of discharges for the selected diagnosis. Then the hospitals were ordered from lowest (best performing) to highest (worst performing) using the adjusted readmission rate. Third, the number of discharges from the best performing hospital to the worst performing hospital was summed until at least 10% of the total discharges had been accounted for. Finally, we computed the ABC as the average of these best performing hospitals. We only report ABCs for which at least three hospitals were included as best performers in the calculation.¹³

To evaluate hospital performance on ABCs for each diagnosis, we identified the percent of hospitals in each setting that were outliers. We defined an outlier as any hospital whose 95% confidence interval for their readmission rate for a given diagnosis did not contain the ABC for their hospital type. All the statistical analyses were performed using SAS version 9.3 (SAS Institute, Inc, Cary, North Carolina).

This project was reviewed by the Cincinnati Children’s Hospital Medical Center Institutional Review Board and determined to be nonhuman subjects research.

Hospital-Type Demographics

The 690,949 discharges from 1,664 hospitals were categorized into 525 metropolitan teaching (550,039 discharges, 79.6% of discharges), 552 metropolitan nonteaching (97,207 discharges, 14% of discharges), and 587 nonmetropolitan hospitals (43,703 discharges, 6.3% of discharges; Table 1). There were significant differences in the patient composition among the three hospital settings. Nonmetropolitan hospitals had a larger percentage of younger patients (aged 0-4 years, P < .001), prominence of first and second quartile median household income, and fewer medically complex patients (48.3% No CCC/No CCI versus 25.5% metropolitan teaching and 33.7% nonteaching, P < .001). Disposition home was over 96% in all three hospital types; however, the metropolitan teaching had a greater percentage of patients discharged to home health versus metropolitan nonteaching and nonmetropolitan hospitals (2.3% versus 0.5%; P < .001).

Readmission Rates

The 17 most common diagnoses based on the number of all-cause 30-day same-hospital readmissions, were categorized into two surgical, seven acute/infectious, four chronic, and four mental health diagnoses (Table 2). Readmission rates varied based on diagnosis and hospital type (Table 2). Overall, mean readmission rates were low, especially in acute respiratory tract related diseases. For chronic diseases, asthma readmissions were consistently low in all three hospital types, whereas sickle cell disease had the highest readmission rate in all three hospital types.

Achievable Benchmarks of Care by Hospital Type

The diagnoses for which ABC could be calculated across all three hospital types included appendectomy and four acute conditions (bronchiolitis, pneumonia, nonbacterial gastroenteritis, and kidney/urinary tract infections). For these conditions, metropolitan teaching hospitals had a more significant percentage of outlier hospitals compared to metropolitan nonteaching and nonmetropolitan hospitals. The percent of outlier hospitals varied by diagnosis and hospital type (Figure).

Metropolitan Teaching

The readmission ABC was calculated for all 17 diagnoses (Table 2). The ABC ranged from 0.4% in acute kidney and urinary tract infection to 7.0% in sickle cell anemia crisis. Bipolar disorder, major depressive disorders and other psychoses, and sickle cell disease (SCD) had the highest percent of outlier hospitals whose mean readmission rates confidence interval did not contain the ABC; tonsil and adenoid procedures and viral illness had the lowest.¹

Metropolitan Nonteaching

The ABC was calculated for 13 of the 17 diagnoses because ABCs were not calculated when there were fewer than three best practicing hospitals. This was the case for tonsil and adenoid procedures, diabetes, seizures, and depression except for major depressive disorder (Table 2). Seven of the 13 diagnoses had an ABC of 0.0%: viral illness, infections of the upper respiratory tract, bronchiolitis, gastroenteritis, hypovolemia and electrolyte disorders, asthma, and childhood behavioral disorders. Like the findings at the metropolitan teaching hospitals, ABCs were lowest for surgical and acute conditions while bipolar disorder, major depressive disorders and other psychoses, and SCD had the highest percent of outlier hospitals with readmission rates beyond the 95% confidence interval of their hospital type’s ABC.

Nonmetropolitan

There was a sufficient number of best practicing hospitals to calculate the ABC for six of the 17 diagnoses (Table 2). For conditions where readmission ABCs could be calculated, they were low: 0.0% for appendectomy, bronchiolitis, gastroenteritis, and seizure; 0.3% for pneumonia; and 1.3% in kidney and urinary tract disorders. None of the conditions with the highest ABCs in other hospital settings (bipolar disease, sickle cell anemia crisis, and major depressive disorders and other psychoses) could be calculated in this setting. Seizure-related readmissions exhibited the most outlier hospitals yet were less than 5%.¹

Among a nationally representative sample of different hospital types that deliver care to children, we report the mean readmission rates and ABCs for 30-day all-cause, same-hospital readmissions for the most commonly readmitted pediatric diagnoses based on hospital type. Previous studies have shown patient variables such as race, ethnicity, and insurance type influencing readmission rates.^19,20 However, hospital type has also been associated with a higher risk of readmission due to the varying complexity of patients at different hospital types.^21,22 Our analyses provide hospital-type specific national estimates of pediatric readmission ABCs for medical and surgical conditions, many less than 1%. While commonly encountered pediatric conditions like asthma and bronchiolitis had low mean readmission rates and ABCs across all hospital types, the mean rates and ABCs for SCD and mental health disorders were much higher with more hospitals performing far from the ABCs.

Diagnoses with a larger percentage of outlier hospitals may represent a national opportunity to improve care for children. Conditions such as SCD and mental illnesses have the highest percentage of hospitals whose readmission rates fall outside of the ABCs in both metropolitan teaching and metropolitan nonteaching hospitals. Hospital performance on SCD and mental health disorders may not reflect deficits in hospital quality or poor adherence to evidence-based best practices, but rather the complex interplay of factors on various levels from government policy and insurance plans, to patient and family resources, to access and availability of medical and mental health specific care. Most importantly, these diseases may represent a significant opportunity for quality improvementin hospitals across the United States.

Sickle cell disease is predominantly a disease among African-Americans, a demographic risk factor for decreased access to care and limited patient and family resources.^23-26 In previous studies evaluating the disparity in readmission rates for Black children with asthma, socioeconomic variables explained 53% of the observed disparity and readmission rates were inversely related to the childhood opportunity index of the patient’s census tract and positively related with geographic social risk.^27,28 Likewise, with SCD affecting a specific demographic and being a chronic disease, best practice policies need to account for the child’s medical needs and include the patient and family resources to ensure access to care and enhanced case management for chronic disease if we aim to improve performance among the outlier hospitals.

Similarly, barriers to care for children with mental illnesses in the United States need attention.^29,30 While there is a paucity of data on the prevalence of mental health disorders in children, one national report estimates that one in 10 American adolescents have depression.^29,31 The American Academy of Pediatrics has developed a policy statement on mental health competencies and a mental health tool-kit for primary care pediatricians; however, no such guidelines or policy statements exist for hospitalized patients with acute or chronic psychiatric conditions.^32,33 Moreover, hospitals are increasingly facing “boarding” of children with acute psychiatric illness in inpatient units and emergency departments.³⁴ The American Medical Association and the American College of Emergency Physicians have expressed concerns regarding the boarding of children with acute psychiatric illness because nonpsychiatric hospitals do not have adequate resources to evaluate, manage, and place these children who deserve appropriate facilities for further management. Coordinated case management and “bundled” discharge planning in other chronic illnesses have shown benefit in cost reduction and readmission.^35-37 Evidence-based practices around pediatric readmissions in other diagnoses should be explored as possible interventions in these conditions.³⁸

There are several limitations to this study. Our data is limited to one calendar year; therefore, admissions in January do not account for potential readmissions from December of the previous year, as patient identifiers do not link across years in the NRD. We also limited our evaluation to the conventional 30-day readmission window, but recent publications may indicate that readmission windows with different timelines could be a more accurate reflection of medically preventable readmissions versus a reflection of social determinants of health leading to readmissions.²⁴ Newborn index admissions were not an allowable index admission; therefore, we may be underreporting readmissions in the neonatal age group. We also chose to include all-cause readmissions, a conventional method to evaluate readmission within an institution, but which may not reflect the quality of care delivered in the index admission. For example, an asthmatic discharged after an acute exacerbation readmitted for dehydration secondary to gastroenteritis may not reflect a lack of quality in asthma inpatient care. Readmissions were limited to the same hospital; therefore, this study cannot account for readmissions at other institutions, which may cause us to underestimate readmission rates. However, end-users of our findings most likely have access only to their own institution’s data. The inclusion of observation status admissions in the database varies from state to state; therefore, this percent of admissions in the database is unknown.

The use of the ABC methodology has some inherent limitations. One hospital with a significant volume diagnosis and low readmission rate within a hospital type may prohibit the reporting of an ABC if less than three hospitals composed the total of the ‘best performing’ hospitals. This was a significant limitation leading to the exclusion of many ABCs in nonmetropolitan institutions. The limitation of calculating and reporting an ABC then prohibits the calculation of outlier hospitals within a hospital type for a given diagnosis. However, when the ABCs are not available, we do provide the mean readmission rate for the diagnosis within the hospital type. While the hospital groupings by population and teaching status for ABCs provide meaningful comparisons for within each hospital setting, it should be noted that there may be vast differences among hospitals within each type (eg, tertiary children’s hospitals compared to teaching hospitals with a pediatric floor in the metropolitan teaching hospital category).^39,40

As healthcare moves from a fee-for-service model to a population-health centered, value-based model, reduction in readmission rates will be more than a quality measure and will have potential financial implications.⁴¹ In the Medicare fee-for-service patients, the Hospital Readmission Reduction Program (HRRP) penalize hospitals with excess readmissions for acute myocardial infarction, heart failure, and pneumonia. The hospitals subject to penalties in the HRRP had greater reduction in readmission rates in the targeted, and even nontargeted conditions, compared with hospitals not subject to penalties.⁴² Similarly, we believe that our data on low readmission rates and ABCs for conditions such as asthma, bronchiolitis, and appendicitis could represent decades of quality improvement work for the most common pediatric conditions among hospitalized children. Sickle cell disease and mental health problems remain as outliers and merit further attention. To move to a true population-health model, hospitals will need to explore outlier conditions including evaluating patient-level readmission patterns across institutions. This moves readmission from a hospital quality measure to a patient-centric quality measure, and perhaps will provide value to the patient and the healthcare system alike.

The readmission ABCs for the most commonly readmitted pediatric diagnoses are low, regardless of the hospital setting. The highest pediatric readmission rates in SCD, bipolar disorders, and major depressive disorder were lower than the most common adult readmission diagnoses. However, mental health conditions and SCD remain as outliers for pediatric readmissions, burden hospital systems, and perhaps warrant national-level attention. The ABCs stratified by hospital type in this study facilitate comparisons and identify opportunities for population-level interventions to meaningfully improve patient care.

Disclosures

The authors have nothing to disclose.

Hospital readmission rates are a common metric for defining, evaluating, and benchmarking quality of care. The Centers for Medicare and Medicaid Services (CMS) publicly report hospital readmission rates for common adult conditions and reduces payments to hospitals with excessive readmissions.¹ Recently, the focus on pediatric readmission rates has increased and the National Quality Forum (NQF) has endorsed at least two pediatric readmission-specific quality indicators which could be used by public and private payers in pay-for-performance programs aimed at institutions caring for children.² While preventability of readmissions and their value as a marker of quality remains debated, their acceptance by the NQF and CMS has led public and private payers to propose readmission-related penalties for hospitals caring for children. ^3-5

All-cause 30-day same-hospital readmission rates for pediatric conditions are half of the adult readmission rates, around 6% in most studies, compared to 12% in adults.^6,7 The lower rates of pediatric readmissions makes it difficult to only use mean readmission rates to stratify hospitals into high- or low-performers and set target goals for improvement.⁸ While adult readmissions have been studied in depth, there are no consistent measures used to benchmark pediatric readmissions across hospital types.

Given the emphasis placed on readmissions, it is essential to understand patterns in pediatric readmission rates to determine optimal and achievable targets for improvement. Achievable Benchmarks of Care (ABCs) are one approach to understanding readmission rates and have an advantage over using mean or medians in performance improvement as they can stratify performance for conditions with low readmission rates and low volumes.⁹ When creating benchmarks, it is important that hospitals performance is evaluated among peer hospitals with similar patient populations, not just a cumulative average from all hospital types which may punish hospitals with a more complex patient case mix.¹⁰ The goal of this study was to calculate the readmission rates and the ABCs for common pediatric diagnoses by hospital type to identify priority conditions for quality improvement efforts using a previously published methodology.^11-13

Data Source

We conducted a retrospective analysis of patients less than 18 years of age in the Healthcare Utilization Project 2014 Nationwide Readmissions Database (NRD). The NRD includes public hospitals; academic medical centers; and specialty hospitals in obstetrics and gynecology, otolaryngology, orthopedics, and cancer; and pediatric, public, and academic medical hospitals. Excluded are long-term care facilities such as rehabilitation, long-term acute care, psychiatric, alcoholism, and chemical dependency hospitals. The readmissions data contains information from hospitals grouped by region, population census, and teaching status.¹⁴ Three hospital type classifications used in this study were metropolitan teaching hospitals, metropolitan nonteaching hospitals, and nonmetropolitan hospitals. These three hospital type classifications follow the reporting format in the NRD.

Study Population

Patients less than 18 years old were included if they were discharged from January 1, 2014 through November 30, 2014 and had a readmission to the index hospital within 30 days. We limited inclusion to discharges through November 30 so we could identify patients with a 30-day readmission as patient identifiers do not link across years in the NRD.

Exposure

We included 30-day, all-cause, same-hospital readmissions to the index acute care hospital, excluding labor and delivery, normal newborn care, chemotherapy, transfers, and mortalities. Intrahospital discharge and admissions within the same hospital system were not defined as a readmission, but rather as a “same-day event.”¹⁵ For example, institutions with inpatient mental health facilities, medical unit discharges and admission to the mental health unit were not identified as a readmission in this dataset.

Outcome

For each hospital type, we measured same-hospital, all-cause, 30-day readmission rates and achievable benchmark of care for the 17 most commonly readmitted pediatric discharge diagnoses. To identify the target readmission diagnoses and all-cause, 30-day readmissions based on their index hospitalizations, All-Patient Refined Diagnosis-Related Groups (APR-DRG), version 25 (3M Health Information Systems, Salt Lake City, Utah) were ordered by frequency for each hospital type. The 20 most common APR-DRGs were the same across all hospital types. The authors then evaluated these 20 APR-DRGs for clinical consistency of included diagnoses identified by the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes within each APR-DRG. Three diagnosis-related groups were excluded from the analysis (major hematologic/immunologic disease except for sickle cell, other anemia and disorders of blood and blood forming organs, and other digestive system diagnoses) due to the heterogeneity of the diagnoses identified by the ICD-9-CM codes within each APR-DRG. We refer to each APR-DRG as a “diagnosis” throughout the article.

Analysis

The demographic characteristics of the patients seen at the three hospital types were summarized using frequencies and percentages. Reports were generated for patient age, gender, payer source, patient residence, median household income, patient complexity, and discharge disposition. Patient complexity was defined using complex chronic condition (CCC) and the number of chronic conditions (CCI).^16,17 As previously defined in the literature, a complex chronic condition is “any medical condition that can be reasonably expected to last at least 12 months (unless death intervenes) and to involve either several different organ systems or one organ system severely enough to require specialty pediatric care and probably some period of hospitalization in a tertiary care center.”¹⁶ Whereas, the Agency for Healthcare Research and Quality’s Chronic Condition Indicator (CCI) defines single, non-CCCs (eg, allergic rhinitis).¹⁷

For each diagnosis, we calculated the mean readmission rate for hospitals in each hospital type category. We then calculated an ABC for each diagnosis in each hospital type using a four-step process.^13,18 First, to control for hospitals with small sample sizes, we adjusted all readmission rates using an adjusted performance fraction ([numerator+1]/[denominator +2]), where the numerator is the number of all-cause 30-day readmissions and the denominator is the number of discharges for the selected diagnosis. Then the hospitals were ordered from lowest (best performing) to highest (worst performing) using the adjusted readmission rate. Third, the number of discharges from the best performing hospital to the worst performing hospital was summed until at least 10% of the total discharges had been accounted for. Finally, we computed the ABC as the average of these best performing hospitals. We only report ABCs for which at least three hospitals were included as best performers in the calculation.¹³

To evaluate hospital performance on ABCs for each diagnosis, we identified the percent of hospitals in each setting that were outliers. We defined an outlier as any hospital whose 95% confidence interval for their readmission rate for a given diagnosis did not contain the ABC for their hospital type. All the statistical analyses were performed using SAS version 9.3 (SAS Institute, Inc, Cary, North Carolina).

This project was reviewed by the Cincinnati Children’s Hospital Medical Center Institutional Review Board and determined to be nonhuman subjects research.

Hospital-Type Demographics

The 690,949 discharges from 1,664 hospitals were categorized into 525 metropolitan teaching (550,039 discharges, 79.6% of discharges), 552 metropolitan nonteaching (97,207 discharges, 14% of discharges), and 587 nonmetropolitan hospitals (43,703 discharges, 6.3% of discharges; Table 1). There were significant differences in the patient composition among the three hospital settings. Nonmetropolitan hospitals had a larger percentage of younger patients (aged 0-4 years, P < .001), prominence of first and second quartile median household income, and fewer medically complex patients (48.3% No CCC/No CCI versus 25.5% metropolitan teaching and 33.7% nonteaching, P < .001). Disposition home was over 96% in all three hospital types; however, the metropolitan teaching had a greater percentage of patients discharged to home health versus metropolitan nonteaching and nonmetropolitan hospitals (2.3% versus 0.5%; P < .001).

Readmission Rates

The 17 most common diagnoses based on the number of all-cause 30-day same-hospital readmissions, were categorized into two surgical, seven acute/infectious, four chronic, and four mental health diagnoses (Table 2). Readmission rates varied based on diagnosis and hospital type (Table 2). Overall, mean readmission rates were low, especially in acute respiratory tract related diseases. For chronic diseases, asthma readmissions were consistently low in all three hospital types, whereas sickle cell disease had the highest readmission rate in all three hospital types.

Achievable Benchmarks of Care by Hospital Type

The diagnoses for which ABC could be calculated across all three hospital types included appendectomy and four acute conditions (bronchiolitis, pneumonia, nonbacterial gastroenteritis, and kidney/urinary tract infections). For these conditions, metropolitan teaching hospitals had a more significant percentage of outlier hospitals compared to metropolitan nonteaching and nonmetropolitan hospitals. The percent of outlier hospitals varied by diagnosis and hospital type (Figure).

Metropolitan Teaching

The readmission ABC was calculated for all 17 diagnoses (Table 2). The ABC ranged from 0.4% in acute kidney and urinary tract infection to 7.0% in sickle cell anemia crisis. Bipolar disorder, major depressive disorders and other psychoses, and sickle cell disease (SCD) had the highest percent of outlier hospitals whose mean readmission rates confidence interval did not contain the ABC; tonsil and adenoid procedures and viral illness had the lowest.¹

Metropolitan Nonteaching

The ABC was calculated for 13 of the 17 diagnoses because ABCs were not calculated when there were fewer than three best practicing hospitals. This was the case for tonsil and adenoid procedures, diabetes, seizures, and depression except for major depressive disorder (Table 2). Seven of the 13 diagnoses had an ABC of 0.0%: viral illness, infections of the upper respiratory tract, bronchiolitis, gastroenteritis, hypovolemia and electrolyte disorders, asthma, and childhood behavioral disorders. Like the findings at the metropolitan teaching hospitals, ABCs were lowest for surgical and acute conditions while bipolar disorder, major depressive disorders and other psychoses, and SCD had the highest percent of outlier hospitals with readmission rates beyond the 95% confidence interval of their hospital type’s ABC.

Nonmetropolitan

There was a sufficient number of best practicing hospitals to calculate the ABC for six of the 17 diagnoses (Table 2). For conditions where readmission ABCs could be calculated, they were low: 0.0% for appendectomy, bronchiolitis, gastroenteritis, and seizure; 0.3% for pneumonia; and 1.3% in kidney and urinary tract disorders. None of the conditions with the highest ABCs in other hospital settings (bipolar disease, sickle cell anemia crisis, and major depressive disorders and other psychoses) could be calculated in this setting. Seizure-related readmissions exhibited the most outlier hospitals yet were less than 5%.¹

Among a nationally representative sample of different hospital types that deliver care to children, we report the mean readmission rates and ABCs for 30-day all-cause, same-hospital readmissions for the most commonly readmitted pediatric diagnoses based on hospital type. Previous studies have shown patient variables such as race, ethnicity, and insurance type influencing readmission rates.^19,20 However, hospital type has also been associated with a higher risk of readmission due to the varying complexity of patients at different hospital types.^21,22 Our analyses provide hospital-type specific national estimates of pediatric readmission ABCs for medical and surgical conditions, many less than 1%. While commonly encountered pediatric conditions like asthma and bronchiolitis had low mean readmission rates and ABCs across all hospital types, the mean rates and ABCs for SCD and mental health disorders were much higher with more hospitals performing far from the ABCs.

Diagnoses with a larger percentage of outlier hospitals may represent a national opportunity to improve care for children. Conditions such as SCD and mental illnesses have the highest percentage of hospitals whose readmission rates fall outside of the ABCs in both metropolitan teaching and metropolitan nonteaching hospitals. Hospital performance on SCD and mental health disorders may not reflect deficits in hospital quality or poor adherence to evidence-based best practices, but rather the complex interplay of factors on various levels from government policy and insurance plans, to patient and family resources, to access and availability of medical and mental health specific care. Most importantly, these diseases may represent a significant opportunity for quality improvementin hospitals across the United States.

Sickle cell disease is predominantly a disease among African-Americans, a demographic risk factor for decreased access to care and limited patient and family resources.^23-26 In previous studies evaluating the disparity in readmission rates for Black children with asthma, socioeconomic variables explained 53% of the observed disparity and readmission rates were inversely related to the childhood opportunity index of the patient’s census tract and positively related with geographic social risk.^27,28 Likewise, with SCD affecting a specific demographic and being a chronic disease, best practice policies need to account for the child’s medical needs and include the patient and family resources to ensure access to care and enhanced case management for chronic disease if we aim to improve performance among the outlier hospitals.

Similarly, barriers to care for children with mental illnesses in the United States need attention.^29,30 While there is a paucity of data on the prevalence of mental health disorders in children, one national report estimates that one in 10 American adolescents have depression.^29,31 The American Academy of Pediatrics has developed a policy statement on mental health competencies and a mental health tool-kit for primary care pediatricians; however, no such guidelines or policy statements exist for hospitalized patients with acute or chronic psychiatric conditions.^32,33 Moreover, hospitals are increasingly facing “boarding” of children with acute psychiatric illness in inpatient units and emergency departments.³⁴ The American Medical Association and the American College of Emergency Physicians have expressed concerns regarding the boarding of children with acute psychiatric illness because nonpsychiatric hospitals do not have adequate resources to evaluate, manage, and place these children who deserve appropriate facilities for further management. Coordinated case management and “bundled” discharge planning in other chronic illnesses have shown benefit in cost reduction and readmission.^35-37 Evidence-based practices around pediatric readmissions in other diagnoses should be explored as possible interventions in these conditions.³⁸

There are several limitations to this study. Our data is limited to one calendar year; therefore, admissions in January do not account for potential readmissions from December of the previous year, as patient identifiers do not link across years in the NRD. We also limited our evaluation to the conventional 30-day readmission window, but recent publications may indicate that readmission windows with different timelines could be a more accurate reflection of medically preventable readmissions versus a reflection of social determinants of health leading to readmissions.²⁴ Newborn index admissions were not an allowable index admission; therefore, we may be underreporting readmissions in the neonatal age group. We also chose to include all-cause readmissions, a conventional method to evaluate readmission within an institution, but which may not reflect the quality of care delivered in the index admission. For example, an asthmatic discharged after an acute exacerbation readmitted for dehydration secondary to gastroenteritis may not reflect a lack of quality in asthma inpatient care. Readmissions were limited to the same hospital; therefore, this study cannot account for readmissions at other institutions, which may cause us to underestimate readmission rates. However, end-users of our findings most likely have access only to their own institution’s data. The inclusion of observation status admissions in the database varies from state to state; therefore, this percent of admissions in the database is unknown.

The use of the ABC methodology has some inherent limitations. One hospital with a significant volume diagnosis and low readmission rate within a hospital type may prohibit the reporting of an ABC if less than three hospitals composed the total of the ‘best performing’ hospitals. This was a significant limitation leading to the exclusion of many ABCs in nonmetropolitan institutions. The limitation of calculating and reporting an ABC then prohibits the calculation of outlier hospitals within a hospital type for a given diagnosis. However, when the ABCs are not available, we do provide the mean readmission rate for the diagnosis within the hospital type. While the hospital groupings by population and teaching status for ABCs provide meaningful comparisons for within each hospital setting, it should be noted that there may be vast differences among hospitals within each type (eg, tertiary children’s hospitals compared to teaching hospitals with a pediatric floor in the metropolitan teaching hospital category).^39,40

As healthcare moves from a fee-for-service model to a population-health centered, value-based model, reduction in readmission rates will be more than a quality measure and will have potential financial implications.⁴¹ In the Medicare fee-for-service patients, the Hospital Readmission Reduction Program (HRRP) penalize hospitals with excess readmissions for acute myocardial infarction, heart failure, and pneumonia. The hospitals subject to penalties in the HRRP had greater reduction in readmission rates in the targeted, and even nontargeted conditions, compared with hospitals not subject to penalties.⁴² Similarly, we believe that our data on low readmission rates and ABCs for conditions such as asthma, bronchiolitis, and appendicitis could represent decades of quality improvement work for the most common pediatric conditions among hospitalized children. Sickle cell disease and mental health problems remain as outliers and merit further attention. To move to a true population-health model, hospitals will need to explore outlier conditions including evaluating patient-level readmission patterns across institutions. This moves readmission from a hospital quality measure to a patient-centric quality measure, and perhaps will provide value to the patient and the healthcare system alike.

The readmission ABCs for the most commonly readmitted pediatric diagnoses are low, regardless of the hospital setting. The highest pediatric readmission rates in SCD, bipolar disorders, and major depressive disorder were lower than the most common adult readmission diagnoses. However, mental health conditions and SCD remain as outliers for pediatric readmissions, burden hospital systems, and perhaps warrant national-level attention. The ABCs stratified by hospital type in this study facilitate comparisons and identify opportunities for population-level interventions to meaningfully improve patient care.

Disclosures

The authors have nothing to disclose.