John Pope, MD, MPH

Currently, 3%5% of infants under a year of age will be admitted to a hospital for acute viral bronchiolitis each year, making it the leading cause of hospitalization in children.15 The American Academy of Pediatrics guideline on the diagnosis and management of bronchiolitis advocates primarily supportive care for this self‐limited disease.6 Specifically, the routine use of therapies such as bronchodilators and corticosteroids are not recommended, nor is routine evaluation with diagnostic testing.6 Numerous studies have established the presence of unwarranted variation in most aspects of bronchiolitis care,713 and the current evidence does not support the routine usage of specific interventions in inpatients.1418

Acute bronchiolitis accounts for direct inpatient medical costs of over $500 million per year.19 Based on estimates from the Healthcare Utilization Project Kids' Inpatient Database, acute bronchiolitis is second only to respiratory distress syndrome as the most expensive disease of hospitalized children.1 Although charges may not correlate directly with costs or even the actual intensity of resource utilization, the national bill, based on charges, is approximately 1.4 billion dollars per year.1 Either way, the leading cause of hospitalization in children is expensive and suffers from dramatic variation in care characterized by overutilization of ineffective interventions.

Evidence‐based guidelines for bronchiolitis are readily available and their successful adoption within larger, academic children's hospitals has been demonstrated.2028 However, upwards of 70% of all children in this country are cared for outside of freestanding children's hospitals,1 and very little has been published about wide dissemination of evidence‐based guidelines in these settings.29 In 2008, the Value in Inpatient Pediatrics (VIP) network was created, as an inclusive pediatric inpatient quality improvement collaborative with a focus on linking academic and community‐based hospitalist groups, to disseminate evidence‐based management strategies for bronchiolitis. We hypothesized that group norming, through benchmarking and public goal setting at the level of the hospitalist group, would decrease overall utilization of nonevidence‐based therapies. Specifically, we were trying to decrease the utilization of bronchodilators, steroids, chest physiotherapy, chest radiography, and viral testing in hospitalized children diagnosed with uncomplicated bronchiolitis.

METHODS

Beginning in early 2008, we recruited pediatric hospitalists into a voluntary bronchiolitis quality improvement collaborative from within the community of hospitalists created by the American Academy of Pediatrics Section on Hospital Medicine. Participants were recruited through open calls at national conferences and mass e‐mails to the section membership through the listserve. The guiding principle for the collaborative was the idea that institutional adoption of evidence‐based disease‐management strategies would result in higher value of care, and that this process could be facilitated by benchmarking local performance against norms created within the larger community. We used group consensus to identify the therapies and tests to benchmark, although the chosen measures meshed with those addressed in the American Academy of Pediatrics (AAP) clinical practice guideline. Use of bronchodilators, corticosteroids, chest physiotherapy, chest radiography, and viral testing were all felt to be significantly overutilized in participating clinical sites. We were unaware of any published national targets for utilization of these therapies or tests, and none of the participating hospitalist groups was actively benchmarking their utilization against any peer group at the start of the project. Length of stay, rates of readmission within 72 hours of discharge, and variable direct costs were chosen as balancing measures for the project.

We collected data on hospitalizations for bronchiolitis for 4 calendar years, from 2007 through 2010, based on the following inclusion criteria: children under 24 months of age, hospitalized for the primary diagnosis of acute viral bronchiolitis as defined by International Classification of Diseases, Ninth Revision (ICD‐9) codes 466.11 and 466.19. We specifically included patients who were in observation status as well as those in inpatient status, and excluded all intensive care unit admissions. Other exclusions were specific ICD‐9 codes for: chronic lung diseases, asthma, chromosomal abnormalities, heart disease, and neurological diseases. We then tracked overall utilization of any bronchodilator (albuterol, levalbuterol, epinephrine, or ipratropium) during the hospitalization, including the emergency department; total number of bronchodilator doses per patient; utilization of any corticosteroids (inhaled or systemic); chest radiography; respiratory syncytial virus (RSV) testing; and chest physiotherapy; as well as variable direct costs per hospitalization for each center. A standardized toolkit was provided to participating centers to facilitate data collection. Data was sought from administrative sources, collected in aggregate form and not at the patient level, and no protected health information was collected as part of the project. The project was categorized as exempt by the University of Texas Health Science Center San Antonio Institutional Review Board, the location of the data repository.

The project began in 2008, though we requested that centers provide 2007 data to supplement our baseline. We held the first group meeting in July 2009 and began the facilitated sharing of resources to promote evidence‐based care, such as guidelines, protocols, respiratory scores, and patient handouts, across sites using data from 2007 and 2008 as our baseline for benchmarking and later assessing any improvement. Centers adopted guidelines at their own pace and we did not require guideline adoption for continued participation. We provided summaries of the available literature by topic, in the event that site leaders wished to give institutional grand rounds or other presentations. All dissemination of guidelines or protocols was done based on the request of the center, and no specific resource was created or sanctioned by the group, though the AAP Guideline for the Diagnosis and Management of Bronchiolitis6 remained a guiding document. Some of our centers participated in more extensive collaborative projects which involved small‐group goal setting, adoption of similar protocols, and conference calls, though this never encompassed more than 25% of the network.

The main product of the project was a yearly report benchmarking each hospital against the network average on each of our chosen utilization measures. The first report was disseminated in July 2009 and included data on calendar year 2007 and 2008, which we considered our group baseline. Most institutions began local Plan‐Do‐Study‐Act (PDSA) cycles by mid‐2009 using the data we provided as they benchmarked their performance against other members of the collaborative, and these continued through 2010. Hospitals were coded and remained anonymous. However, we publicly honored the high performers within the network at a yearly meeting, and urged these centers to share their tools and strategies, which was facilitated through a project Web site.30 All participation was voluntary, and all costs were borne by individuals or their respective centers.

In order to assess data quality, we undertook a validation project for calendar year 2009. We requested local direct chart review of a 10% sample, or a minimum of 10 charts, to confirm reported utilization rates for the therapies and tests we tracked. Any center with less than 80% accuracy was then asked to review data collection methods and make adjustments accordingly. One center identified and resolved a significant data discrepancy and 2 centers refused to participate in the validation project, citing their participation in a large national database for which there was already a very rigorous data validation process (Child Health Corporation of America's Pediatric Health Information System database). Given that we did not uncover major discrepancies in data quality within our network, we did not request further data validation but rather promoted year‐to‐year consistency of collection methods, seeking to collect the same type/quality of data that hospitals use in their own internal performance assessments.

Statistical analyses were performed using GraphPad InStat, version 3.0 (GraphPad Software, San Diego, CA). Descriptive statistics (including interquartile range ([IQR], the range from 25th to 75th percentile of the data) are provided. Analysis of process measures over the series of years was performed using repeated measures analysis of variance (ANOVA), as were intra‐hospital comparisons for all measures. Hospitals were not weighted by volume of admissions, ie, the unit of analysis was the hospital and not individual hospitalizations. Data were analyzed for normality using the method of Kolmogorov and Smirnoff, and in cases where normality was not satisfied (steroids and chest physiotherapy), the data were transformed and nonparametric methods were used. Post‐test adjustment for multiple comparisons was done using the TukeyKramer test in cases where ANOVA P values were <0.05. Fisher's exact test was used to analyze contingency tables for categorical variables such as presence or absence of a protocol.

RESULTS

Data encompassing 11,568 bronchiolitis hospitalizations in 17 centers, for calendar years 2007 to 2010, were analyzed for this report. A total of 31 centers ever participated in the project; however, this report is restricted to centers who participated for the entirety of the project from 2008 through 2010, and who consented to have their data reported. Specifically, 18 centers met inclusion criteria and 1 center opted out of the project, leaving the 17 centers described in Table 1. The overall network makeup shifted each year, but was always more than 80% non‐freestanding children's hospitals and approximately 30% urban, as defined as located in a population center of more than 1 million. A large majority of the participants did not have a local bronchiolitis protocol or guideline at the start of the project, although 88% of participants adopted some form of protocolized care by 2010. Calendar years 2007 and 2008 served as our network baseline, with most interventions (in institutions where they occurred) begun by calendar year 2009. The level of intervention varied greatly among institutions, with a few institutions doing nothing more than benchmarking their performance.

Mean length of stay (LOS), readmission rates, and variable direct costs did not differ significantly during the project time period. Mean LOS for the network ranged from a low of 2.4 days (IQR, 2.22.8 days) to a high of 2.7 days (IQR, 2.43.1 days), and mean readmission rates ranged from 1.2% (IQR, 0.7%1.8%) to 1.7% (IQR, 0.7%2.5%) during the project. Mean variable direct costs ranged from $1639 (IQR, $1383$1864) to $1767 (IQR, $1365$2320).

Table 2 describes the mean overall utilization of bronchodilators, chest radiography, RSV testing, steroids, and chest physiotherapy among the group from 2007 to 2010. By 2010, we saw a 46% decline in the volume of bronchodilator used within the network, a 3.6 (95% confidence interval [CI] 1.45.8) dose per patient absolute decrease (P < 0.01). We also saw a 12% (95% CI 5%25%) absolute decline in the overall percentage of patients exposed to any bronchodilator (P < 0.01). Finally, there was a 10% (95% CI 3%18%) absolute decline in the overall utilization of any chest physiotherapy (P < 0.01). The project did not demonstrate a significant impact on utilization of corticosteroids, chest radiography, or viral testing, although several centers achieved significant decreases on a local level (data not shown).

We analyzed within‐hospital trends as well. Figure 1 describes intra‐hospital change over the course of the project for overall bronchodilator usage. In this analysis, 15 of 17 hospitals (88%) achieved a significant decrease in overall bronchodilator utilization by 2010. (Hospitals 27 and 29 were unable to provide 2007 baseline data.) For doses per patient, 15 of 17 institutions provided data on this measure, and 12 of 15 (80%) achieved significant decreases (Figure 2). Of note, the institutions failing to achieve significant decreases in bronchodilator utilization entered the project with utilization rates that were already significantly below network mean at the start of the project. (Institutions failing to improve are denoted with an asterisk in Figures 1 and 2.) Since most institutions made significant improvements in bronchodilator utilization over time, we looked for correlates of failure to decrease utilization. The strongest association for failure to improve during the project period was use of a protocol prior to joining the network (odds ratio [OR] = 11, 95% CI 261).

Figure 1

Figure 2

DISCUSSION

We demonstrated a significant decline in utilization of bronchodilators and chest physiotherapy in inpatient bronchiolitis within a voluntary quality collaborative focused on benchmarking without employing intensive interventions. This observation is important in that it demonstrates real‐world efficacy for our methods. Prior literature has clearly demonstrated that local bronchiolitis guidelines are effective; however, our data on over 11,000 hospitalizations from a broad array of inpatient settings continue to show a high rate of overutilization. We facilitated dissemination and sharing of guideline‐related tools primarily electronically, and capitalized on perceived peer‐group frustration with inefficient management of a high‐volume, high‐utilization disease. While the project leadership had varying degrees of advanced training in quality improvement methodology, the majority of the site leaders were self‐taught and trained while on the job. Our inclusive collaborative had some success using pragmatic and low‐resource methods which we believe is a novel approach to the issue of overutilization.

These considerations are highlighted given the pressing need to find more efficient and scalable means of bending the cost curve of healthcare in the United States. Learning collaboratives are a relatively new model for improvement, with some history in pediatrics,31, 32 and are attractive because of their potential to generate both widespread capacity for change as well as direct improvement. Both cystic fibrosis31 and neonatology collaboratives32 have been celebrated for their positive impacts on children's healthcare, and both are testaments to the power inherent in creating a community of like‐minded individuals. One of the most popular models for learning collaboratives remains the Institute for Healthcare Improvement's Breakthrough Series; however, this model is resource intensive in that it typically involves large teams and several yearly face‐to‐face meetings, with significant monetary investment on the part of hospitals. On the other hand, virtual collaboratives have produced mixed results with respect to quality improvement,33 so there is a continued need to maximize our learning about what works efficiently. Our collaborative was able to successfully disseminate tools developed in large academic institutions to be applied in smaller and more varied settings, where resources for quality improvement activities were limited.

One possible reason for any successes in this project was the existence of a well‐known guideline for the management of bronchiolitis published by the American Academy of Pediatrics in 2006. This guideline recommends primarily symptomatic care, and has a statement supporting the contention that routine use of our targeted therapies is unnecessary. It allows for a trial of bronchodilator, but specifically states that all trials should be accompanied by the use of an objective measure of improvement (typically interpreted to mean a respiratory distress score). A guideline sanctioned by an important national organization of pediatricians was invaluable, and we believe that it should serve as a basis for any nationally promoted inpatient quality measure for this very common pediatric illness. The existence of the AAP guideline also highlights the possibility that our results are merely representative of secular trends in utilization in bronchiolitis care, since we had no control group. The available literature on national guidelines has shown mixed and quite modest impacts in other countries.28, 34 Most of our group took active steps to operationalize the guidelines as part of their participation in this collaborative, though they might have done similar work anyway due to the increasing importance of quality improvement in hospitalist culture over the years of the project.

The project did not demonstrate any impact on steroid utilization, or on rates of obtaining chest radiography or viral testing, despite expressly targeting these widely overused interventions. These modalities are often employed in the emergency department and, as a collaborative of pediatric hospitalists, we did not have specific emergency department participation which we recognize as a major weakness and potential impediment to further progress. We hope to collaborate with our respective emergency departments in the future on these particular measures. We also noted that many institutions were inflexible about foregoing viral testing, due to infection control issues arising from the need to cohort patients in shared rooms based on RSV positivity during the busy winter months. A few institutions were able to alter their infection control policies using the strategy of assuming all children with bronchiolitis had RSV (ie, choosing to use both contact precautions and to wear a mask when entering rooms), though this was not universally popular. Finally, we recognize a missed opportunity in not collecting dose per patient level data for steroids, which might have allowed us to distinguish hospitals with ongoing inpatient utilization of steroids from those with only emergency department usage.

Another significant limitation of this project was the lack of annual assessments of data quality. However, we believe our findings are still useful and important, even with this obvious limitation. Most quality improvement work is done using hospital‐supplied data gleaned from administrative databases, exactly the sources used in this project. Key decisions are made in most hospitals in the country based on data of similar quality. Further limitations of the project relate to the issue of replicability. The disease process we addressed is a major source of frustration to pediatric hospitalists, and our sample likely consisted of the most highly motivated individuals, as they sought out and joined a group with the express purpose of decreasing unnecessary utilization in bronchiolitis. We believe this limitation highlights the likely need for quality measures to emerge organically out of a community of practice when resources are limited, ie, we do not believe we would have had significant success using our methods with an unpopular or externally imposed quality measure.

Although a detailed analysis of costs was beyond the scope of the current project, it is possible that decreased utilization resulted in overall cost savings, despite the fact that our data did not demonstrate a significant change in network‐level average variable direct costs related to bronchiolitis. It has been suggested that such savings may be particularly difficult to demonstrate objectively, especially when the principal costs targeted are labor‐based.35 LOS did not significantly vary during the project, whereas the use of labor‐intensive therapies like nebulized bronchodilators and chest physiotherapy declined. It is, however, quite possible that the decreased utilization we demonstrated was accompanied by a concomitant increase in utilization of other unmeasured therapies.

CONCLUSIONS

A volunteer, peer‐group collaborative focused on benchmarking decreased utilization of bronchodilators and chest physiotherapy in bronchiolitis, though had no impact on overuse of other unnecessary therapies and tests.

Currently, 3%5% of infants under a year of age will be admitted to a hospital for acute viral bronchiolitis each year, making it the leading cause of hospitalization in children.15 The American Academy of Pediatrics guideline on the diagnosis and management of bronchiolitis advocates primarily supportive care for this self‐limited disease.6 Specifically, the routine use of therapies such as bronchodilators and corticosteroids are not recommended, nor is routine evaluation with diagnostic testing.6 Numerous studies have established the presence of unwarranted variation in most aspects of bronchiolitis care,713 and the current evidence does not support the routine usage of specific interventions in inpatients.1418

Acute bronchiolitis accounts for direct inpatient medical costs of over $500 million per year.19 Based on estimates from the Healthcare Utilization Project Kids' Inpatient Database, acute bronchiolitis is second only to respiratory distress syndrome as the most expensive disease of hospitalized children.1 Although charges may not correlate directly with costs or even the actual intensity of resource utilization, the national bill, based on charges, is approximately 1.4 billion dollars per year.1 Either way, the leading cause of hospitalization in children is expensive and suffers from dramatic variation in care characterized by overutilization of ineffective interventions.

Evidence‐based guidelines for bronchiolitis are readily available and their successful adoption within larger, academic children's hospitals has been demonstrated.2028 However, upwards of 70% of all children in this country are cared for outside of freestanding children's hospitals,1 and very little has been published about wide dissemination of evidence‐based guidelines in these settings.29 In 2008, the Value in Inpatient Pediatrics (VIP) network was created, as an inclusive pediatric inpatient quality improvement collaborative with a focus on linking academic and community‐based hospitalist groups, to disseminate evidence‐based management strategies for bronchiolitis. We hypothesized that group norming, through benchmarking and public goal setting at the level of the hospitalist group, would decrease overall utilization of nonevidence‐based therapies. Specifically, we were trying to decrease the utilization of bronchodilators, steroids, chest physiotherapy, chest radiography, and viral testing in hospitalized children diagnosed with uncomplicated bronchiolitis.

METHODS

Beginning in early 2008, we recruited pediatric hospitalists into a voluntary bronchiolitis quality improvement collaborative from within the community of hospitalists created by the American Academy of Pediatrics Section on Hospital Medicine. Participants were recruited through open calls at national conferences and mass e‐mails to the section membership through the listserve. The guiding principle for the collaborative was the idea that institutional adoption of evidence‐based disease‐management strategies would result in higher value of care, and that this process could be facilitated by benchmarking local performance against norms created within the larger community. We used group consensus to identify the therapies and tests to benchmark, although the chosen measures meshed with those addressed in the American Academy of Pediatrics (AAP) clinical practice guideline. Use of bronchodilators, corticosteroids, chest physiotherapy, chest radiography, and viral testing were all felt to be significantly overutilized in participating clinical sites. We were unaware of any published national targets for utilization of these therapies or tests, and none of the participating hospitalist groups was actively benchmarking their utilization against any peer group at the start of the project. Length of stay, rates of readmission within 72 hours of discharge, and variable direct costs were chosen as balancing measures for the project.

We collected data on hospitalizations for bronchiolitis for 4 calendar years, from 2007 through 2010, based on the following inclusion criteria: children under 24 months of age, hospitalized for the primary diagnosis of acute viral bronchiolitis as defined by International Classification of Diseases, Ninth Revision (ICD‐9) codes 466.11 and 466.19. We specifically included patients who were in observation status as well as those in inpatient status, and excluded all intensive care unit admissions. Other exclusions were specific ICD‐9 codes for: chronic lung diseases, asthma, chromosomal abnormalities, heart disease, and neurological diseases. We then tracked overall utilization of any bronchodilator (albuterol, levalbuterol, epinephrine, or ipratropium) during the hospitalization, including the emergency department; total number of bronchodilator doses per patient; utilization of any corticosteroids (inhaled or systemic); chest radiography; respiratory syncytial virus (RSV) testing; and chest physiotherapy; as well as variable direct costs per hospitalization for each center. A standardized toolkit was provided to participating centers to facilitate data collection. Data was sought from administrative sources, collected in aggregate form and not at the patient level, and no protected health information was collected as part of the project. The project was categorized as exempt by the University of Texas Health Science Center San Antonio Institutional Review Board, the location of the data repository.

The project began in 2008, though we requested that centers provide 2007 data to supplement our baseline. We held the first group meeting in July 2009 and began the facilitated sharing of resources to promote evidence‐based care, such as guidelines, protocols, respiratory scores, and patient handouts, across sites using data from 2007 and 2008 as our baseline for benchmarking and later assessing any improvement. Centers adopted guidelines at their own pace and we did not require guideline adoption for continued participation. We provided summaries of the available literature by topic, in the event that site leaders wished to give institutional grand rounds or other presentations. All dissemination of guidelines or protocols was done based on the request of the center, and no specific resource was created or sanctioned by the group, though the AAP Guideline for the Diagnosis and Management of Bronchiolitis6 remained a guiding document. Some of our centers participated in more extensive collaborative projects which involved small‐group goal setting, adoption of similar protocols, and conference calls, though this never encompassed more than 25% of the network.

The main product of the project was a yearly report benchmarking each hospital against the network average on each of our chosen utilization measures. The first report was disseminated in July 2009 and included data on calendar year 2007 and 2008, which we considered our group baseline. Most institutions began local Plan‐Do‐Study‐Act (PDSA) cycles by mid‐2009 using the data we provided as they benchmarked their performance against other members of the collaborative, and these continued through 2010. Hospitals were coded and remained anonymous. However, we publicly honored the high performers within the network at a yearly meeting, and urged these centers to share their tools and strategies, which was facilitated through a project Web site.30 All participation was voluntary, and all costs were borne by individuals or their respective centers.

In order to assess data quality, we undertook a validation project for calendar year 2009. We requested local direct chart review of a 10% sample, or a minimum of 10 charts, to confirm reported utilization rates for the therapies and tests we tracked. Any center with less than 80% accuracy was then asked to review data collection methods and make adjustments accordingly. One center identified and resolved a significant data discrepancy and 2 centers refused to participate in the validation project, citing their participation in a large national database for which there was already a very rigorous data validation process (Child Health Corporation of America's Pediatric Health Information System database). Given that we did not uncover major discrepancies in data quality within our network, we did not request further data validation but rather promoted year‐to‐year consistency of collection methods, seeking to collect the same type/quality of data that hospitals use in their own internal performance assessments.

Statistical analyses were performed using GraphPad InStat, version 3.0 (GraphPad Software, San Diego, CA). Descriptive statistics (including interquartile range ([IQR], the range from 25th to 75th percentile of the data) are provided. Analysis of process measures over the series of years was performed using repeated measures analysis of variance (ANOVA), as were intra‐hospital comparisons for all measures. Hospitals were not weighted by volume of admissions, ie, the unit of analysis was the hospital and not individual hospitalizations. Data were analyzed for normality using the method of Kolmogorov and Smirnoff, and in cases where normality was not satisfied (steroids and chest physiotherapy), the data were transformed and nonparametric methods were used. Post‐test adjustment for multiple comparisons was done using the TukeyKramer test in cases where ANOVA P values were <0.05. Fisher's exact test was used to analyze contingency tables for categorical variables such as presence or absence of a protocol.

RESULTS

Data encompassing 11,568 bronchiolitis hospitalizations in 17 centers, for calendar years 2007 to 2010, were analyzed for this report. A total of 31 centers ever participated in the project; however, this report is restricted to centers who participated for the entirety of the project from 2008 through 2010, and who consented to have their data reported. Specifically, 18 centers met inclusion criteria and 1 center opted out of the project, leaving the 17 centers described in Table 1. The overall network makeup shifted each year, but was always more than 80% non‐freestanding children's hospitals and approximately 30% urban, as defined as located in a population center of more than 1 million. A large majority of the participants did not have a local bronchiolitis protocol or guideline at the start of the project, although 88% of participants adopted some form of protocolized care by 2010. Calendar years 2007 and 2008 served as our network baseline, with most interventions (in institutions where they occurred) begun by calendar year 2009. The level of intervention varied greatly among institutions, with a few institutions doing nothing more than benchmarking their performance.

Mean length of stay (LOS), readmission rates, and variable direct costs did not differ significantly during the project time period. Mean LOS for the network ranged from a low of 2.4 days (IQR, 2.22.8 days) to a high of 2.7 days (IQR, 2.43.1 days), and mean readmission rates ranged from 1.2% (IQR, 0.7%1.8%) to 1.7% (IQR, 0.7%2.5%) during the project. Mean variable direct costs ranged from $1639 (IQR, $1383$1864) to $1767 (IQR, $1365$2320).

Table 2 describes the mean overall utilization of bronchodilators, chest radiography, RSV testing, steroids, and chest physiotherapy among the group from 2007 to 2010. By 2010, we saw a 46% decline in the volume of bronchodilator used within the network, a 3.6 (95% confidence interval [CI] 1.45.8) dose per patient absolute decrease (P < 0.01). We also saw a 12% (95% CI 5%25%) absolute decline in the overall percentage of patients exposed to any bronchodilator (P < 0.01). Finally, there was a 10% (95% CI 3%18%) absolute decline in the overall utilization of any chest physiotherapy (P < 0.01). The project did not demonstrate a significant impact on utilization of corticosteroids, chest radiography, or viral testing, although several centers achieved significant decreases on a local level (data not shown).

We analyzed within‐hospital trends as well. Figure 1 describes intra‐hospital change over the course of the project for overall bronchodilator usage. In this analysis, 15 of 17 hospitals (88%) achieved a significant decrease in overall bronchodilator utilization by 2010. (Hospitals 27 and 29 were unable to provide 2007 baseline data.) For doses per patient, 15 of 17 institutions provided data on this measure, and 12 of 15 (80%) achieved significant decreases (Figure 2). Of note, the institutions failing to achieve significant decreases in bronchodilator utilization entered the project with utilization rates that were already significantly below network mean at the start of the project. (Institutions failing to improve are denoted with an asterisk in Figures 1 and 2.) Since most institutions made significant improvements in bronchodilator utilization over time, we looked for correlates of failure to decrease utilization. The strongest association for failure to improve during the project period was use of a protocol prior to joining the network (odds ratio [OR] = 11, 95% CI 261).

Figure 1

Figure 2

DISCUSSION

We demonstrated a significant decline in utilization of bronchodilators and chest physiotherapy in inpatient bronchiolitis within a voluntary quality collaborative focused on benchmarking without employing intensive interventions. This observation is important in that it demonstrates real‐world efficacy for our methods. Prior literature has clearly demonstrated that local bronchiolitis guidelines are effective; however, our data on over 11,000 hospitalizations from a broad array of inpatient settings continue to show a high rate of overutilization. We facilitated dissemination and sharing of guideline‐related tools primarily electronically, and capitalized on perceived peer‐group frustration with inefficient management of a high‐volume, high‐utilization disease. While the project leadership had varying degrees of advanced training in quality improvement methodology, the majority of the site leaders were self‐taught and trained while on the job. Our inclusive collaborative had some success using pragmatic and low‐resource methods which we believe is a novel approach to the issue of overutilization.

These considerations are highlighted given the pressing need to find more efficient and scalable means of bending the cost curve of healthcare in the United States. Learning collaboratives are a relatively new model for improvement, with some history in pediatrics,31, 32 and are attractive because of their potential to generate both widespread capacity for change as well as direct improvement. Both cystic fibrosis31 and neonatology collaboratives32 have been celebrated for their positive impacts on children's healthcare, and both are testaments to the power inherent in creating a community of like‐minded individuals. One of the most popular models for learning collaboratives remains the Institute for Healthcare Improvement's Breakthrough Series; however, this model is resource intensive in that it typically involves large teams and several yearly face‐to‐face meetings, with significant monetary investment on the part of hospitals. On the other hand, virtual collaboratives have produced mixed results with respect to quality improvement,33 so there is a continued need to maximize our learning about what works efficiently. Our collaborative was able to successfully disseminate tools developed in large academic institutions to be applied in smaller and more varied settings, where resources for quality improvement activities were limited.

One possible reason for any successes in this project was the existence of a well‐known guideline for the management of bronchiolitis published by the American Academy of Pediatrics in 2006. This guideline recommends primarily symptomatic care, and has a statement supporting the contention that routine use of our targeted therapies is unnecessary. It allows for a trial of bronchodilator, but specifically states that all trials should be accompanied by the use of an objective measure of improvement (typically interpreted to mean a respiratory distress score). A guideline sanctioned by an important national organization of pediatricians was invaluable, and we believe that it should serve as a basis for any nationally promoted inpatient quality measure for this very common pediatric illness. The existence of the AAP guideline also highlights the possibility that our results are merely representative of secular trends in utilization in bronchiolitis care, since we had no control group. The available literature on national guidelines has shown mixed and quite modest impacts in other countries.28, 34 Most of our group took active steps to operationalize the guidelines as part of their participation in this collaborative, though they might have done similar work anyway due to the increasing importance of quality improvement in hospitalist culture over the years of the project.

The project did not demonstrate any impact on steroid utilization, or on rates of obtaining chest radiography or viral testing, despite expressly targeting these widely overused interventions. These modalities are often employed in the emergency department and, as a collaborative of pediatric hospitalists, we did not have specific emergency department participation which we recognize as a major weakness and potential impediment to further progress. We hope to collaborate with our respective emergency departments in the future on these particular measures. We also noted that many institutions were inflexible about foregoing viral testing, due to infection control issues arising from the need to cohort patients in shared rooms based on RSV positivity during the busy winter months. A few institutions were able to alter their infection control policies using the strategy of assuming all children with bronchiolitis had RSV (ie, choosing to use both contact precautions and to wear a mask when entering rooms), though this was not universally popular. Finally, we recognize a missed opportunity in not collecting dose per patient level data for steroids, which might have allowed us to distinguish hospitals with ongoing inpatient utilization of steroids from those with only emergency department usage.

Another significant limitation of this project was the lack of annual assessments of data quality. However, we believe our findings are still useful and important, even with this obvious limitation. Most quality improvement work is done using hospital‐supplied data gleaned from administrative databases, exactly the sources used in this project. Key decisions are made in most hospitals in the country based on data of similar quality. Further limitations of the project relate to the issue of replicability. The disease process we addressed is a major source of frustration to pediatric hospitalists, and our sample likely consisted of the most highly motivated individuals, as they sought out and joined a group with the express purpose of decreasing unnecessary utilization in bronchiolitis. We believe this limitation highlights the likely need for quality measures to emerge organically out of a community of practice when resources are limited, ie, we do not believe we would have had significant success using our methods with an unpopular or externally imposed quality measure.

Although a detailed analysis of costs was beyond the scope of the current project, it is possible that decreased utilization resulted in overall cost savings, despite the fact that our data did not demonstrate a significant change in network‐level average variable direct costs related to bronchiolitis. It has been suggested that such savings may be particularly difficult to demonstrate objectively, especially when the principal costs targeted are labor‐based.35 LOS did not significantly vary during the project, whereas the use of labor‐intensive therapies like nebulized bronchodilators and chest physiotherapy declined. It is, however, quite possible that the decreased utilization we demonstrated was accompanied by a concomitant increase in utilization of other unmeasured therapies.

CONCLUSIONS

A volunteer, peer‐group collaborative focused on benchmarking decreased utilization of bronchodilators and chest physiotherapy in bronchiolitis, though had no impact on overuse of other unnecessary therapies and tests.

Currently, 3%5% of infants under a year of age will be admitted to a hospital for acute viral bronchiolitis each year, making it the leading cause of hospitalization in children.15 The American Academy of Pediatrics guideline on the diagnosis and management of bronchiolitis advocates primarily supportive care for this self‐limited disease.6 Specifically, the routine use of therapies such as bronchodilators and corticosteroids are not recommended, nor is routine evaluation with diagnostic testing.6 Numerous studies have established the presence of unwarranted variation in most aspects of bronchiolitis care,713 and the current evidence does not support the routine usage of specific interventions in inpatients.1418

Acute bronchiolitis accounts for direct inpatient medical costs of over $500 million per year.19 Based on estimates from the Healthcare Utilization Project Kids' Inpatient Database, acute bronchiolitis is second only to respiratory distress syndrome as the most expensive disease of hospitalized children.1 Although charges may not correlate directly with costs or even the actual intensity of resource utilization, the national bill, based on charges, is approximately 1.4 billion dollars per year.1 Either way, the leading cause of hospitalization in children is expensive and suffers from dramatic variation in care characterized by overutilization of ineffective interventions.

Evidence‐based guidelines for bronchiolitis are readily available and their successful adoption within larger, academic children's hospitals has been demonstrated.2028 However, upwards of 70% of all children in this country are cared for outside of freestanding children's hospitals,1 and very little has been published about wide dissemination of evidence‐based guidelines in these settings.29 In 2008, the Value in Inpatient Pediatrics (VIP) network was created, as an inclusive pediatric inpatient quality improvement collaborative with a focus on linking academic and community‐based hospitalist groups, to disseminate evidence‐based management strategies for bronchiolitis. We hypothesized that group norming, through benchmarking and public goal setting at the level of the hospitalist group, would decrease overall utilization of nonevidence‐based therapies. Specifically, we were trying to decrease the utilization of bronchodilators, steroids, chest physiotherapy, chest radiography, and viral testing in hospitalized children diagnosed with uncomplicated bronchiolitis.

METHODS

Beginning in early 2008, we recruited pediatric hospitalists into a voluntary bronchiolitis quality improvement collaborative from within the community of hospitalists created by the American Academy of Pediatrics Section on Hospital Medicine. Participants were recruited through open calls at national conferences and mass e‐mails to the section membership through the listserve. The guiding principle for the collaborative was the idea that institutional adoption of evidence‐based disease‐management strategies would result in higher value of care, and that this process could be facilitated by benchmarking local performance against norms created within the larger community. We used group consensus to identify the therapies and tests to benchmark, although the chosen measures meshed with those addressed in the American Academy of Pediatrics (AAP) clinical practice guideline. Use of bronchodilators, corticosteroids, chest physiotherapy, chest radiography, and viral testing were all felt to be significantly overutilized in participating clinical sites. We were unaware of any published national targets for utilization of these therapies or tests, and none of the participating hospitalist groups was actively benchmarking their utilization against any peer group at the start of the project. Length of stay, rates of readmission within 72 hours of discharge, and variable direct costs were chosen as balancing measures for the project.

We collected data on hospitalizations for bronchiolitis for 4 calendar years, from 2007 through 2010, based on the following inclusion criteria: children under 24 months of age, hospitalized for the primary diagnosis of acute viral bronchiolitis as defined by International Classification of Diseases, Ninth Revision (ICD‐9) codes 466.11 and 466.19. We specifically included patients who were in observation status as well as those in inpatient status, and excluded all intensive care unit admissions. Other exclusions were specific ICD‐9 codes for: chronic lung diseases, asthma, chromosomal abnormalities, heart disease, and neurological diseases. We then tracked overall utilization of any bronchodilator (albuterol, levalbuterol, epinephrine, or ipratropium) during the hospitalization, including the emergency department; total number of bronchodilator doses per patient; utilization of any corticosteroids (inhaled or systemic); chest radiography; respiratory syncytial virus (RSV) testing; and chest physiotherapy; as well as variable direct costs per hospitalization for each center. A standardized toolkit was provided to participating centers to facilitate data collection. Data was sought from administrative sources, collected in aggregate form and not at the patient level, and no protected health information was collected as part of the project. The project was categorized as exempt by the University of Texas Health Science Center San Antonio Institutional Review Board, the location of the data repository.

The project began in 2008, though we requested that centers provide 2007 data to supplement our baseline. We held the first group meeting in July 2009 and began the facilitated sharing of resources to promote evidence‐based care, such as guidelines, protocols, respiratory scores, and patient handouts, across sites using data from 2007 and 2008 as our baseline for benchmarking and later assessing any improvement. Centers adopted guidelines at their own pace and we did not require guideline adoption for continued participation. We provided summaries of the available literature by topic, in the event that site leaders wished to give institutional grand rounds or other presentations. All dissemination of guidelines or protocols was done based on the request of the center, and no specific resource was created or sanctioned by the group, though the AAP Guideline for the Diagnosis and Management of Bronchiolitis6 remained a guiding document. Some of our centers participated in more extensive collaborative projects which involved small‐group goal setting, adoption of similar protocols, and conference calls, though this never encompassed more than 25% of the network.

The main product of the project was a yearly report benchmarking each hospital against the network average on each of our chosen utilization measures. The first report was disseminated in July 2009 and included data on calendar year 2007 and 2008, which we considered our group baseline. Most institutions began local Plan‐Do‐Study‐Act (PDSA) cycles by mid‐2009 using the data we provided as they benchmarked their performance against other members of the collaborative, and these continued through 2010. Hospitals were coded and remained anonymous. However, we publicly honored the high performers within the network at a yearly meeting, and urged these centers to share their tools and strategies, which was facilitated through a project Web site.30 All participation was voluntary, and all costs were borne by individuals or their respective centers.

In order to assess data quality, we undertook a validation project for calendar year 2009. We requested local direct chart review of a 10% sample, or a minimum of 10 charts, to confirm reported utilization rates for the therapies and tests we tracked. Any center with less than 80% accuracy was then asked to review data collection methods and make adjustments accordingly. One center identified and resolved a significant data discrepancy and 2 centers refused to participate in the validation project, citing their participation in a large national database for which there was already a very rigorous data validation process (Child Health Corporation of America's Pediatric Health Information System database). Given that we did not uncover major discrepancies in data quality within our network, we did not request further data validation but rather promoted year‐to‐year consistency of collection methods, seeking to collect the same type/quality of data that hospitals use in their own internal performance assessments.

Statistical analyses were performed using GraphPad InStat, version 3.0 (GraphPad Software, San Diego, CA). Descriptive statistics (including interquartile range ([IQR], the range from 25th to 75th percentile of the data) are provided. Analysis of process measures over the series of years was performed using repeated measures analysis of variance (ANOVA), as were intra‐hospital comparisons for all measures. Hospitals were not weighted by volume of admissions, ie, the unit of analysis was the hospital and not individual hospitalizations. Data were analyzed for normality using the method of Kolmogorov and Smirnoff, and in cases where normality was not satisfied (steroids and chest physiotherapy), the data were transformed and nonparametric methods were used. Post‐test adjustment for multiple comparisons was done using the TukeyKramer test in cases where ANOVA P values were <0.05. Fisher's exact test was used to analyze contingency tables for categorical variables such as presence or absence of a protocol.

RESULTS

Data encompassing 11,568 bronchiolitis hospitalizations in 17 centers, for calendar years 2007 to 2010, were analyzed for this report. A total of 31 centers ever participated in the project; however, this report is restricted to centers who participated for the entirety of the project from 2008 through 2010, and who consented to have their data reported. Specifically, 18 centers met inclusion criteria and 1 center opted out of the project, leaving the 17 centers described in Table 1. The overall network makeup shifted each year, but was always more than 80% non‐freestanding children's hospitals and approximately 30% urban, as defined as located in a population center of more than 1 million. A large majority of the participants did not have a local bronchiolitis protocol or guideline at the start of the project, although 88% of participants adopted some form of protocolized care by 2010. Calendar years 2007 and 2008 served as our network baseline, with most interventions (in institutions where they occurred) begun by calendar year 2009. The level of intervention varied greatly among institutions, with a few institutions doing nothing more than benchmarking their performance.

Mean length of stay (LOS), readmission rates, and variable direct costs did not differ significantly during the project time period. Mean LOS for the network ranged from a low of 2.4 days (IQR, 2.22.8 days) to a high of 2.7 days (IQR, 2.43.1 days), and mean readmission rates ranged from 1.2% (IQR, 0.7%1.8%) to 1.7% (IQR, 0.7%2.5%) during the project. Mean variable direct costs ranged from $1639 (IQR, $1383$1864) to $1767 (IQR, $1365$2320).

Table 2 describes the mean overall utilization of bronchodilators, chest radiography, RSV testing, steroids, and chest physiotherapy among the group from 2007 to 2010. By 2010, we saw a 46% decline in the volume of bronchodilator used within the network, a 3.6 (95% confidence interval [CI] 1.45.8) dose per patient absolute decrease (P < 0.01). We also saw a 12% (95% CI 5%25%) absolute decline in the overall percentage of patients exposed to any bronchodilator (P < 0.01). Finally, there was a 10% (95% CI 3%18%) absolute decline in the overall utilization of any chest physiotherapy (P < 0.01). The project did not demonstrate a significant impact on utilization of corticosteroids, chest radiography, or viral testing, although several centers achieved significant decreases on a local level (data not shown).

We analyzed within‐hospital trends as well. Figure 1 describes intra‐hospital change over the course of the project for overall bronchodilator usage. In this analysis, 15 of 17 hospitals (88%) achieved a significant decrease in overall bronchodilator utilization by 2010. (Hospitals 27 and 29 were unable to provide 2007 baseline data.) For doses per patient, 15 of 17 institutions provided data on this measure, and 12 of 15 (80%) achieved significant decreases (Figure 2). Of note, the institutions failing to achieve significant decreases in bronchodilator utilization entered the project with utilization rates that were already significantly below network mean at the start of the project. (Institutions failing to improve are denoted with an asterisk in Figures 1 and 2.) Since most institutions made significant improvements in bronchodilator utilization over time, we looked for correlates of failure to decrease utilization. The strongest association for failure to improve during the project period was use of a protocol prior to joining the network (odds ratio [OR] = 11, 95% CI 261).

Figure 1

Figure 2

DISCUSSION

We demonstrated a significant decline in utilization of bronchodilators and chest physiotherapy in inpatient bronchiolitis within a voluntary quality collaborative focused on benchmarking without employing intensive interventions. This observation is important in that it demonstrates real‐world efficacy for our methods. Prior literature has clearly demonstrated that local bronchiolitis guidelines are effective; however, our data on over 11,000 hospitalizations from a broad array of inpatient settings continue to show a high rate of overutilization. We facilitated dissemination and sharing of guideline‐related tools primarily electronically, and capitalized on perceived peer‐group frustration with inefficient management of a high‐volume, high‐utilization disease. While the project leadership had varying degrees of advanced training in quality improvement methodology, the majority of the site leaders were self‐taught and trained while on the job. Our inclusive collaborative had some success using pragmatic and low‐resource methods which we believe is a novel approach to the issue of overutilization.

These considerations are highlighted given the pressing need to find more efficient and scalable means of bending the cost curve of healthcare in the United States. Learning collaboratives are a relatively new model for improvement, with some history in pediatrics,31, 32 and are attractive because of their potential to generate both widespread capacity for change as well as direct improvement. Both cystic fibrosis31 and neonatology collaboratives32 have been celebrated for their positive impacts on children's healthcare, and both are testaments to the power inherent in creating a community of like‐minded individuals. One of the most popular models for learning collaboratives remains the Institute for Healthcare Improvement's Breakthrough Series; however, this model is resource intensive in that it typically involves large teams and several yearly face‐to‐face meetings, with significant monetary investment on the part of hospitals. On the other hand, virtual collaboratives have produced mixed results with respect to quality improvement,33 so there is a continued need to maximize our learning about what works efficiently. Our collaborative was able to successfully disseminate tools developed in large academic institutions to be applied in smaller and more varied settings, where resources for quality improvement activities were limited.

One possible reason for any successes in this project was the existence of a well‐known guideline for the management of bronchiolitis published by the American Academy of Pediatrics in 2006. This guideline recommends primarily symptomatic care, and has a statement supporting the contention that routine use of our targeted therapies is unnecessary. It allows for a trial of bronchodilator, but specifically states that all trials should be accompanied by the use of an objective measure of improvement (typically interpreted to mean a respiratory distress score). A guideline sanctioned by an important national organization of pediatricians was invaluable, and we believe that it should serve as a basis for any nationally promoted inpatient quality measure for this very common pediatric illness. The existence of the AAP guideline also highlights the possibility that our results are merely representative of secular trends in utilization in bronchiolitis care, since we had no control group. The available literature on national guidelines has shown mixed and quite modest impacts in other countries.28, 34 Most of our group took active steps to operationalize the guidelines as part of their participation in this collaborative, though they might have done similar work anyway due to the increasing importance of quality improvement in hospitalist culture over the years of the project.

The project did not demonstrate any impact on steroid utilization, or on rates of obtaining chest radiography or viral testing, despite expressly targeting these widely overused interventions. These modalities are often employed in the emergency department and, as a collaborative of pediatric hospitalists, we did not have specific emergency department participation which we recognize as a major weakness and potential impediment to further progress. We hope to collaborate with our respective emergency departments in the future on these particular measures. We also noted that many institutions were inflexible about foregoing viral testing, due to infection control issues arising from the need to cohort patients in shared rooms based on RSV positivity during the busy winter months. A few institutions were able to alter their infection control policies using the strategy of assuming all children with bronchiolitis had RSV (ie, choosing to use both contact precautions and to wear a mask when entering rooms), though this was not universally popular. Finally, we recognize a missed opportunity in not collecting dose per patient level data for steroids, which might have allowed us to distinguish hospitals with ongoing inpatient utilization of steroids from those with only emergency department usage.

Another significant limitation of this project was the lack of annual assessments of data quality. However, we believe our findings are still useful and important, even with this obvious limitation. Most quality improvement work is done using hospital‐supplied data gleaned from administrative databases, exactly the sources used in this project. Key decisions are made in most hospitals in the country based on data of similar quality. Further limitations of the project relate to the issue of replicability. The disease process we addressed is a major source of frustration to pediatric hospitalists, and our sample likely consisted of the most highly motivated individuals, as they sought out and joined a group with the express purpose of decreasing unnecessary utilization in bronchiolitis. We believe this limitation highlights the likely need for quality measures to emerge organically out of a community of practice when resources are limited, ie, we do not believe we would have had significant success using our methods with an unpopular or externally imposed quality measure.

Although a detailed analysis of costs was beyond the scope of the current project, it is possible that decreased utilization resulted in overall cost savings, despite the fact that our data did not demonstrate a significant change in network‐level average variable direct costs related to bronchiolitis. It has been suggested that such savings may be particularly difficult to demonstrate objectively, especially when the principal costs targeted are labor‐based.35 LOS did not significantly vary during the project, whereas the use of labor‐intensive therapies like nebulized bronchodilators and chest physiotherapy declined. It is, however, quite possible that the decreased utilization we demonstrated was accompanied by a concomitant increase in utilization of other unmeasured therapies.

CONCLUSIONS

A volunteer, peer‐group collaborative focused on benchmarking decreased utilization of bronchodilators and chest physiotherapy in bronchiolitis, though had no impact on overuse of other unnecessary therapies and tests.