Ricardo Quinonez, MD

For severe asthma exacerbations unresponsive to initial treatment, expert consensus guidelines from 2007 recommend consideration for adjunct treatments (magnesium or heliox) to decrease the likelihood of intubation.¹ Over the last decade, data have emerged suggesting that intravenous (IV) magnesium may be more effective for reduction of hospital admission rates.² Pooled meta-analyses have demonstrated improved pulmonary function and reduction of hospital admission by as much as 68% in children when IV magnesium is administered in the emergency department (ED), although the evidence is extremely limited because of a small number of studies (three) and small sample size (115 children).^2-5

Though these data suggest that use of IV magnesium may reduce admission rates, a study of pediatric emergency medicine (PEM) physicians in US and Canada reported reluctance regarding use for this purpose. While PEM physicians reported awareness of the literature on admission prevention, they estimated that fewer than 5% of their patients receiving IV magnesium were discharged home.⁶ Their practice was generally limited to using IV magnesium in children with impending respiratory failure for the purpose of reducing intensive care unit (ICU) admission and not hospitalization.⁶ PEM physicians’ reluctance to use IV magnesium was related to the lack of strong available evidence supporting the impact of IV magnesium on outcomes, such as admission, and gaps in the literature about its dosing and safety profile.

The goal of this study was to assess the prevailing trends in IV magnesium use across US children’s hospitals and to assess the relationship of IV magnesium use to admission rate, length of stay (LOS), readmission rate, and ICU admission rate. We hypothesized that IV magnesium use might have increased following publication of studies demonstrating an association between IV magnesium use and fewer admissions.

Study Design, Setting, and Participants

This is a retrospective cohort study of asthma (All Patient Refined Diagnosis Related Group 141) hospitalizations for patients less than 18 years old presenting to 35 tertiary care children’s hospitals from January 1, 2010, to December 31, 2017, included in the Pediatric Health Information System (PHIS; Children’s Hospital Association, Lenexa, Kansas) database. The PHIS database is an administrative database that contains demographics, International Classification of Diseases 9th and 10th Revision diagnoses and procedures, and daily billing records for all inpatient, observation, ED, and ambulatory surgery encounters. All data were deidentified prior to inclusion in the database and tracking of patients across ED and inpatient visits was achieved through an encrypted and unique patient identifier. Children transferred from other hospitals were excluded because we could not verify IV magnesium use prior to transfer. For hospitals to be included, they were required to provide continuous data throughout the study period.

Main Outcome Measure

The main outcome was exposure to IV magnesium as determined by billing information available in the PHIS database.

Patient Demographics

We assessed patients’ demographic characteristics, including age (younger than 5 years, 5-11 years, and 12-17 years), sex, race/ethnicity, and insurance status.

Healthcare Utilization and Hospital Characteristics

We assessed healthcare utilization using geometric mean LOS, proportion of patients admitted to the hospital and to the ICU, and the proportion of patients with a 7-day all-cause readmission. In addition, we divided hospitals into three equal groups based on their annual inpatient asthma volume (<300, 300-850, >850 cases per year).

Statistical Analysis

We compared demographic and clinical characteristics across patients receiving IV magnesium with those who did not receive it with use of chi-square tests for categorical variables and Wilcoxon rank sum test for continuous variables. We calculated annual IV magnesium use rates for each hospital and modeled the average annual rate with a general linear model in order to assess change over time. We used Pearson product moment correlation to compare the annual proportion of magnesium use and healthcare utilization measures, including geometric mean LOS, the proportion of patients using the inpatient wards or the ICU, and the proportion of cases with a 7-day all-cause readmission. Geometric mean LOS was used to normalize the compounding effect of non–normally distributed arithmetic mean LOS. A sensitivity analysis was performed stratifying IV magnesium use over time by hospital inpatient volume. Data were analyzed using SAS version 9.4 (SAS Institute, Cary, North Carolina), and P values < .05 were considered statistically significant.

Study Population

A total of 878,188 encounters with acute asthma exacerbation met the inclusion criteria, with 65,558 (7.5%) receiving IV magnesium (Table). Of those receiving IV magnesium, 90% were admitted to the hospital. There were statistically significant differences in IV magnesium use when compared by age, race/ethnicity, and payer type, but not gender. IV magnesium use was significantly associated with older age (more than 5 years old), non-Hispanic black race, ED visit in the year prior to admission, longer hospital LOS, and higher ICU admission rate.

Trends in Intravenous Magnesium Use

IV magnesium use among hospitalized children more than doubled from 2010 to 2017 (17% vs 36%). Low-volume hospitals had a lower frequency of IV magnesium use, compared with the moderate- and high-volume hospitals. The growth rate per year of IV magnesium use was greater in high- and moderate-volume hospitals (3.4% and 2.9% per year, respectively), compared with the low-volume hospitals (1.2% per year; P = .04).

Trends in Intravenous Magnesium Use and Hospital Outcomes

The trend in IV magnesium use was not associated with a statistically significant change in the inpatient and ICU admission rate or in the 7-day all-cause readmission rate (Figure and Appendix Figure). Although IV magnesium use increased over time, LOS decreased significantly during the same period (1.6 days in 2010 vs 1.4 days in 2017; P < .001). When analyzed by hospital volume, no significant associations were found in the inpatient admission, ICU admission, and 7-day readmission rate.

The use of IV magnesium has significantly increased in US children’s hospitals over the last 8 years, especially among those hospitalized following an ED evaluation. Over this interval, trends in inpatient and ICU admission rate, as well as 7-day all-cause readmission rate, for asthma did not change, while LOS decreased. These findings contrast with a recent Cochrane review that summarized the efficacy of IV magnesium for reducing admission rate in few small trials.²

Our study findings are more consistent with prior survey findings that IV magnesium does not reduce hospitalization and that ED physicians tend to use IV magnesium in severe asthma exacerbation for its potential therapeutic benefits because of bronchodilator and anti-inflammatory effect.^6,7 Similar to PEM physicians’ estimates, only 10% of patients receiving IV magnesium were discharged home in our study.

IV magnesium use is higher in high-volume hospitals than in moderate- and low-volume ones. One potential explanation is that high- and moderate-volume hospitals may see a higher volume of children with severe or impending respiratory failure and, therefore, are more likely to use IV magnesium than the low-volume hospitals are. Alternatively, physician adoption of magnesium use for lower-acuity asthma exacerbations could vary by hospital volume.

Trend analyses of outcomes suggest that increase in IV magnesium use was not associated with an increase in inpatient and ICU admission rate or with 7-day all-cause readmission rate, although LOS reduced. LOS might be reduced because of various quality improvement initiatives (eg, discharging patients after every 3 hours albuterol treatments or respiratory therapist–driven protocols) and might not be related to IV magnesium use.^8,9 To this point, a recent study of a respiratory assessment score–matched cohort did not find any therapeutic benefit of IV magnesium with severe asthma exacerbation when receiving continuous albuterol therapy on a pediatric ward.⁵ Perhaps future studies could explore estimating the outcome by performing comparative effectiveness studies between those with severe asthma exacerbation who did or did not receive IV magnesium. Additionally, randomized controlled trials comparing IV magnesium and standard therapy and its effects on outcomes, such as hospitalization, LOS, association with asthma chronicity, and previous oral steroid use, might provide further insight to inform clinical practice.

Certain study limitations should be noted. The study cohort included children’s hospitals only, and it is possible that care at nonchildren’s hospitals for asthma differs. PHIS dataset used in this study does not allow determination of where and when IV magnesium was given, the severity of asthma exacerbation, or the chronicity of baseline disease. Moreover, PHIS hospitals include centers in large cities, and other competing children’s hospitals may provide other tertiary care that could affect the readmission data calculation. Lastly, the temporal associations between IV magnesium use and outcomes reported in this study should not be used as evidence or lack of evidence for the effectiveness of magnesium given the limitations of the observational study design and dataset used.

In conclusion, IV magnesium use in management of asthma exacerbation in children across the United States has significantly increased. The increase occurred disproportionately in high-volume hospitals and was not associated with changes in admission rate, ICU admission rate, or 7-day all-cause readmission rate, although LOS has decreased over time.

The authors have no financial relationships relevant to this article or conflicts of interest to disclose. 

This paper was a platform presentation at annual meetings of Pediatric Academic Societies 2019; accepted for presentation at annual meeting of Pediatric Hospital Medicine, July 2019.

No funding was secured for this study. 

For severe asthma exacerbations unresponsive to initial treatment, expert consensus guidelines from 2007 recommend consideration for adjunct treatments (magnesium or heliox) to decrease the likelihood of intubation.¹ Over the last decade, data have emerged suggesting that intravenous (IV) magnesium may be more effective for reduction of hospital admission rates.² Pooled meta-analyses have demonstrated improved pulmonary function and reduction of hospital admission by as much as 68% in children when IV magnesium is administered in the emergency department (ED), although the evidence is extremely limited because of a small number of studies (three) and small sample size (115 children).^2-5

Though these data suggest that use of IV magnesium may reduce admission rates, a study of pediatric emergency medicine (PEM) physicians in US and Canada reported reluctance regarding use for this purpose. While PEM physicians reported awareness of the literature on admission prevention, they estimated that fewer than 5% of their patients receiving IV magnesium were discharged home.⁶ Their practice was generally limited to using IV magnesium in children with impending respiratory failure for the purpose of reducing intensive care unit (ICU) admission and not hospitalization.⁶ PEM physicians’ reluctance to use IV magnesium was related to the lack of strong available evidence supporting the impact of IV magnesium on outcomes, such as admission, and gaps in the literature about its dosing and safety profile.

The goal of this study was to assess the prevailing trends in IV magnesium use across US children’s hospitals and to assess the relationship of IV magnesium use to admission rate, length of stay (LOS), readmission rate, and ICU admission rate. We hypothesized that IV magnesium use might have increased following publication of studies demonstrating an association between IV magnesium use and fewer admissions.

Study Design, Setting, and Participants

This is a retrospective cohort study of asthma (All Patient Refined Diagnosis Related Group 141) hospitalizations for patients less than 18 years old presenting to 35 tertiary care children’s hospitals from January 1, 2010, to December 31, 2017, included in the Pediatric Health Information System (PHIS; Children’s Hospital Association, Lenexa, Kansas) database. The PHIS database is an administrative database that contains demographics, International Classification of Diseases 9th and 10th Revision diagnoses and procedures, and daily billing records for all inpatient, observation, ED, and ambulatory surgery encounters. All data were deidentified prior to inclusion in the database and tracking of patients across ED and inpatient visits was achieved through an encrypted and unique patient identifier. Children transferred from other hospitals were excluded because we could not verify IV magnesium use prior to transfer. For hospitals to be included, they were required to provide continuous data throughout the study period.

Main Outcome Measure

The main outcome was exposure to IV magnesium as determined by billing information available in the PHIS database.

Patient Demographics

We assessed patients’ demographic characteristics, including age (younger than 5 years, 5-11 years, and 12-17 years), sex, race/ethnicity, and insurance status.

Healthcare Utilization and Hospital Characteristics

We assessed healthcare utilization using geometric mean LOS, proportion of patients admitted to the hospital and to the ICU, and the proportion of patients with a 7-day all-cause readmission. In addition, we divided hospitals into three equal groups based on their annual inpatient asthma volume (<300, 300-850, >850 cases per year).

Statistical Analysis

We compared demographic and clinical characteristics across patients receiving IV magnesium with those who did not receive it with use of chi-square tests for categorical variables and Wilcoxon rank sum test for continuous variables. We calculated annual IV magnesium use rates for each hospital and modeled the average annual rate with a general linear model in order to assess change over time. We used Pearson product moment correlation to compare the annual proportion of magnesium use and healthcare utilization measures, including geometric mean LOS, the proportion of patients using the inpatient wards or the ICU, and the proportion of cases with a 7-day all-cause readmission. Geometric mean LOS was used to normalize the compounding effect of non–normally distributed arithmetic mean LOS. A sensitivity analysis was performed stratifying IV magnesium use over time by hospital inpatient volume. Data were analyzed using SAS version 9.4 (SAS Institute, Cary, North Carolina), and P values < .05 were considered statistically significant.

Study Population

A total of 878,188 encounters with acute asthma exacerbation met the inclusion criteria, with 65,558 (7.5%) receiving IV magnesium (Table). Of those receiving IV magnesium, 90% were admitted to the hospital. There were statistically significant differences in IV magnesium use when compared by age, race/ethnicity, and payer type, but not gender. IV magnesium use was significantly associated with older age (more than 5 years old), non-Hispanic black race, ED visit in the year prior to admission, longer hospital LOS, and higher ICU admission rate.

Trends in Intravenous Magnesium Use

IV magnesium use among hospitalized children more than doubled from 2010 to 2017 (17% vs 36%). Low-volume hospitals had a lower frequency of IV magnesium use, compared with the moderate- and high-volume hospitals. The growth rate per year of IV magnesium use was greater in high- and moderate-volume hospitals (3.4% and 2.9% per year, respectively), compared with the low-volume hospitals (1.2% per year; P = .04).

Trends in Intravenous Magnesium Use and Hospital Outcomes

The trend in IV magnesium use was not associated with a statistically significant change in the inpatient and ICU admission rate or in the 7-day all-cause readmission rate (Figure and Appendix Figure). Although IV magnesium use increased over time, LOS decreased significantly during the same period (1.6 days in 2010 vs 1.4 days in 2017; P < .001). When analyzed by hospital volume, no significant associations were found in the inpatient admission, ICU admission, and 7-day readmission rate.

The use of IV magnesium has significantly increased in US children’s hospitals over the last 8 years, especially among those hospitalized following an ED evaluation. Over this interval, trends in inpatient and ICU admission rate, as well as 7-day all-cause readmission rate, for asthma did not change, while LOS decreased. These findings contrast with a recent Cochrane review that summarized the efficacy of IV magnesium for reducing admission rate in few small trials.²

Our study findings are more consistent with prior survey findings that IV magnesium does not reduce hospitalization and that ED physicians tend to use IV magnesium in severe asthma exacerbation for its potential therapeutic benefits because of bronchodilator and anti-inflammatory effect.^6,7 Similar to PEM physicians’ estimates, only 10% of patients receiving IV magnesium were discharged home in our study.

IV magnesium use is higher in high-volume hospitals than in moderate- and low-volume ones. One potential explanation is that high- and moderate-volume hospitals may see a higher volume of children with severe or impending respiratory failure and, therefore, are more likely to use IV magnesium than the low-volume hospitals are. Alternatively, physician adoption of magnesium use for lower-acuity asthma exacerbations could vary by hospital volume.

Trend analyses of outcomes suggest that increase in IV magnesium use was not associated with an increase in inpatient and ICU admission rate or with 7-day all-cause readmission rate, although LOS reduced. LOS might be reduced because of various quality improvement initiatives (eg, discharging patients after every 3 hours albuterol treatments or respiratory therapist–driven protocols) and might not be related to IV magnesium use.^8,9 To this point, a recent study of a respiratory assessment score–matched cohort did not find any therapeutic benefit of IV magnesium with severe asthma exacerbation when receiving continuous albuterol therapy on a pediatric ward.⁵ Perhaps future studies could explore estimating the outcome by performing comparative effectiveness studies between those with severe asthma exacerbation who did or did not receive IV magnesium. Additionally, randomized controlled trials comparing IV magnesium and standard therapy and its effects on outcomes, such as hospitalization, LOS, association with asthma chronicity, and previous oral steroid use, might provide further insight to inform clinical practice.

Certain study limitations should be noted. The study cohort included children’s hospitals only, and it is possible that care at nonchildren’s hospitals for asthma differs. PHIS dataset used in this study does not allow determination of where and when IV magnesium was given, the severity of asthma exacerbation, or the chronicity of baseline disease. Moreover, PHIS hospitals include centers in large cities, and other competing children’s hospitals may provide other tertiary care that could affect the readmission data calculation. Lastly, the temporal associations between IV magnesium use and outcomes reported in this study should not be used as evidence or lack of evidence for the effectiveness of magnesium given the limitations of the observational study design and dataset used.

In conclusion, IV magnesium use in management of asthma exacerbation in children across the United States has significantly increased. The increase occurred disproportionately in high-volume hospitals and was not associated with changes in admission rate, ICU admission rate, or 7-day all-cause readmission rate, although LOS has decreased over time.

The authors have no financial relationships relevant to this article or conflicts of interest to disclose. 

This paper was a platform presentation at annual meetings of Pediatric Academic Societies 2019; accepted for presentation at annual meeting of Pediatric Hospital Medicine, July 2019.

No funding was secured for this study. 

For severe asthma exacerbations unresponsive to initial treatment, expert consensus guidelines from 2007 recommend consideration for adjunct treatments (magnesium or heliox) to decrease the likelihood of intubation.¹ Over the last decade, data have emerged suggesting that intravenous (IV) magnesium may be more effective for reduction of hospital admission rates.² Pooled meta-analyses have demonstrated improved pulmonary function and reduction of hospital admission by as much as 68% in children when IV magnesium is administered in the emergency department (ED), although the evidence is extremely limited because of a small number of studies (three) and small sample size (115 children).^2-5

Though these data suggest that use of IV magnesium may reduce admission rates, a study of pediatric emergency medicine (PEM) physicians in US and Canada reported reluctance regarding use for this purpose. While PEM physicians reported awareness of the literature on admission prevention, they estimated that fewer than 5% of their patients receiving IV magnesium were discharged home.⁶ Their practice was generally limited to using IV magnesium in children with impending respiratory failure for the purpose of reducing intensive care unit (ICU) admission and not hospitalization.⁶ PEM physicians’ reluctance to use IV magnesium was related to the lack of strong available evidence supporting the impact of IV magnesium on outcomes, such as admission, and gaps in the literature about its dosing and safety profile.

The goal of this study was to assess the prevailing trends in IV magnesium use across US children’s hospitals and to assess the relationship of IV magnesium use to admission rate, length of stay (LOS), readmission rate, and ICU admission rate. We hypothesized that IV magnesium use might have increased following publication of studies demonstrating an association between IV magnesium use and fewer admissions.

Study Design, Setting, and Participants

This is a retrospective cohort study of asthma (All Patient Refined Diagnosis Related Group 141) hospitalizations for patients less than 18 years old presenting to 35 tertiary care children’s hospitals from January 1, 2010, to December 31, 2017, included in the Pediatric Health Information System (PHIS; Children’s Hospital Association, Lenexa, Kansas) database. The PHIS database is an administrative database that contains demographics, International Classification of Diseases 9th and 10th Revision diagnoses and procedures, and daily billing records for all inpatient, observation, ED, and ambulatory surgery encounters. All data were deidentified prior to inclusion in the database and tracking of patients across ED and inpatient visits was achieved through an encrypted and unique patient identifier. Children transferred from other hospitals were excluded because we could not verify IV magnesium use prior to transfer. For hospitals to be included, they were required to provide continuous data throughout the study period.

Main Outcome Measure

The main outcome was exposure to IV magnesium as determined by billing information available in the PHIS database.

Patient Demographics

We assessed patients’ demographic characteristics, including age (younger than 5 years, 5-11 years, and 12-17 years), sex, race/ethnicity, and insurance status.

Healthcare Utilization and Hospital Characteristics

We assessed healthcare utilization using geometric mean LOS, proportion of patients admitted to the hospital and to the ICU, and the proportion of patients with a 7-day all-cause readmission. In addition, we divided hospitals into three equal groups based on their annual inpatient asthma volume (<300, 300-850, >850 cases per year).

Statistical Analysis

We compared demographic and clinical characteristics across patients receiving IV magnesium with those who did not receive it with use of chi-square tests for categorical variables and Wilcoxon rank sum test for continuous variables. We calculated annual IV magnesium use rates for each hospital and modeled the average annual rate with a general linear model in order to assess change over time. We used Pearson product moment correlation to compare the annual proportion of magnesium use and healthcare utilization measures, including geometric mean LOS, the proportion of patients using the inpatient wards or the ICU, and the proportion of cases with a 7-day all-cause readmission. Geometric mean LOS was used to normalize the compounding effect of non–normally distributed arithmetic mean LOS. A sensitivity analysis was performed stratifying IV magnesium use over time by hospital inpatient volume. Data were analyzed using SAS version 9.4 (SAS Institute, Cary, North Carolina), and P values < .05 were considered statistically significant.

Study Population

A total of 878,188 encounters with acute asthma exacerbation met the inclusion criteria, with 65,558 (7.5%) receiving IV magnesium (Table). Of those receiving IV magnesium, 90% were admitted to the hospital. There were statistically significant differences in IV magnesium use when compared by age, race/ethnicity, and payer type, but not gender. IV magnesium use was significantly associated with older age (more than 5 years old), non-Hispanic black race, ED visit in the year prior to admission, longer hospital LOS, and higher ICU admission rate.

Trends in Intravenous Magnesium Use

IV magnesium use among hospitalized children more than doubled from 2010 to 2017 (17% vs 36%). Low-volume hospitals had a lower frequency of IV magnesium use, compared with the moderate- and high-volume hospitals. The growth rate per year of IV magnesium use was greater in high- and moderate-volume hospitals (3.4% and 2.9% per year, respectively), compared with the low-volume hospitals (1.2% per year; P = .04).

Trends in Intravenous Magnesium Use and Hospital Outcomes

The trend in IV magnesium use was not associated with a statistically significant change in the inpatient and ICU admission rate or in the 7-day all-cause readmission rate (Figure and Appendix Figure). Although IV magnesium use increased over time, LOS decreased significantly during the same period (1.6 days in 2010 vs 1.4 days in 2017; P < .001). When analyzed by hospital volume, no significant associations were found in the inpatient admission, ICU admission, and 7-day readmission rate.

The use of IV magnesium has significantly increased in US children’s hospitals over the last 8 years, especially among those hospitalized following an ED evaluation. Over this interval, trends in inpatient and ICU admission rate, as well as 7-day all-cause readmission rate, for asthma did not change, while LOS decreased. These findings contrast with a recent Cochrane review that summarized the efficacy of IV magnesium for reducing admission rate in few small trials.²

Our study findings are more consistent with prior survey findings that IV magnesium does not reduce hospitalization and that ED physicians tend to use IV magnesium in severe asthma exacerbation for its potential therapeutic benefits because of bronchodilator and anti-inflammatory effect.^6,7 Similar to PEM physicians’ estimates, only 10% of patients receiving IV magnesium were discharged home in our study.

IV magnesium use is higher in high-volume hospitals than in moderate- and low-volume ones. One potential explanation is that high- and moderate-volume hospitals may see a higher volume of children with severe or impending respiratory failure and, therefore, are more likely to use IV magnesium than the low-volume hospitals are. Alternatively, physician adoption of magnesium use for lower-acuity asthma exacerbations could vary by hospital volume.

Trend analyses of outcomes suggest that increase in IV magnesium use was not associated with an increase in inpatient and ICU admission rate or with 7-day all-cause readmission rate, although LOS reduced. LOS might be reduced because of various quality improvement initiatives (eg, discharging patients after every 3 hours albuterol treatments or respiratory therapist–driven protocols) and might not be related to IV magnesium use.^8,9 To this point, a recent study of a respiratory assessment score–matched cohort did not find any therapeutic benefit of IV magnesium with severe asthma exacerbation when receiving continuous albuterol therapy on a pediatric ward.⁵ Perhaps future studies could explore estimating the outcome by performing comparative effectiveness studies between those with severe asthma exacerbation who did or did not receive IV magnesium. Additionally, randomized controlled trials comparing IV magnesium and standard therapy and its effects on outcomes, such as hospitalization, LOS, association with asthma chronicity, and previous oral steroid use, might provide further insight to inform clinical practice.

Certain study limitations should be noted. The study cohort included children’s hospitals only, and it is possible that care at nonchildren’s hospitals for asthma differs. PHIS dataset used in this study does not allow determination of where and when IV magnesium was given, the severity of asthma exacerbation, or the chronicity of baseline disease. Moreover, PHIS hospitals include centers in large cities, and other competing children’s hospitals may provide other tertiary care that could affect the readmission data calculation. Lastly, the temporal associations between IV magnesium use and outcomes reported in this study should not be used as evidence or lack of evidence for the effectiveness of magnesium given the limitations of the observational study design and dataset used.

In conclusion, IV magnesium use in management of asthma exacerbation in children across the United States has significantly increased. The increase occurred disproportionately in high-volume hospitals and was not associated with changes in admission rate, ICU admission rate, or 7-day all-cause readmission rate, although LOS has decreased over time.

The authors have no financial relationships relevant to this article or conflicts of interest to disclose. 

This paper was a platform presentation at annual meetings of Pediatric Academic Societies 2019; accepted for presentation at annual meeting of Pediatric Hospital Medicine, July 2019.

No funding was secured for this study. 

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.