Ryan J. Coller, MD, MPH

BACKGROUND: Hospital charges and lengths of stay may be greater when adults with chronic conditions are admitted to children’s hospitals. Despite multiple efforts to improve pediatric-adult health care transitions, little guidance exists for transitioning inpatient care.

OBJECTIVE: This study sought to characterize pediatric-adult inpatient care transitions across general pediatric services at U.S. children’s hospitals.

DESIGN and SETTING: National survey of inpatient general pediatric service leaders at U.S. children’s hospitals from January 2016 to July 2016.

MEASUREMENT: Questionnaires assessed institutional characteristics, presence of inpatient transition initiatives (having a specific process and/or leader), and 22 inpatient transition activities. Scales of highly correlated activities were created using exploratory factor analysis. Logistic regression identified associations among institutional characteristics, transition activities, and presence of an inpatient transition initiative.

RESULTS: Of 195 children’s hospitals, 96 responded (49.2% response rate). Transition initiatives were present at 38% of children’s hospitals, more often where there were providers who were trained in both internal medicine and pediatrics or where there were outpatient transition processes. Specific activities were infrequent and varied widely from 2.1% (systems to track youth in transition) to 40.5% (addressing potential insurance problems). Institutions with initiatives more often consistently performed the majority of activities, including using checklists and creating patient-centered transition care plans. Of remaining activities, half involved transition planning, the essential step between readiness and transfer.

CONCLUSION: Relatively few inpatient general pediatric services at U.S. children’s hospitals have leaders or dedicated processes to shepherd transitions to adult-oriented inpatient care. Across institutions, there is wide variability in performance of activities to facilitate this transition. Feasible process and outcome measures are needed.

Also in JHM this month

Characterizing hospitalist practice and perceptions of critical care delivery

AUTHORS: Joseph R. Sweigart, MD, FACP, FHM; David Aymond, MD; Alfred Burger, MD, FACP, SFHM; Andy Kelly, MAS, MS; Nick Marzano, Med; Thomas McIlraith, MD, SFHM; Peter Morris, MD; Mark V. Williams, MD, FACP, MHM; and Eric M. Siegal, MD, SFHM, FCCM

Clinical decision making: Observing the smartphone user an observational study in predicting acute surgical patients’ suitability for discharge

AUTHORS: Richard Hoffmann, MBBS; Simon Harley, MBBS; Samuel Ellison, MBBS; and Peter G. Devitt, MBBS, FRACS

BACKGROUND: Hospital charges and lengths of stay may be greater when adults with chronic conditions are admitted to children’s hospitals. Despite multiple efforts to improve pediatric-adult health care transitions, little guidance exists for transitioning inpatient care.

OBJECTIVE: This study sought to characterize pediatric-adult inpatient care transitions across general pediatric services at U.S. children’s hospitals.

DESIGN and SETTING: National survey of inpatient general pediatric service leaders at U.S. children’s hospitals from January 2016 to July 2016.

MEASUREMENT: Questionnaires assessed institutional characteristics, presence of inpatient transition initiatives (having a specific process and/or leader), and 22 inpatient transition activities. Scales of highly correlated activities were created using exploratory factor analysis. Logistic regression identified associations among institutional characteristics, transition activities, and presence of an inpatient transition initiative.

RESULTS: Of 195 children’s hospitals, 96 responded (49.2% response rate). Transition initiatives were present at 38% of children’s hospitals, more often where there were providers who were trained in both internal medicine and pediatrics or where there were outpatient transition processes. Specific activities were infrequent and varied widely from 2.1% (systems to track youth in transition) to 40.5% (addressing potential insurance problems). Institutions with initiatives more often consistently performed the majority of activities, including using checklists and creating patient-centered transition care plans. Of remaining activities, half involved transition planning, the essential step between readiness and transfer.

CONCLUSION: Relatively few inpatient general pediatric services at U.S. children’s hospitals have leaders or dedicated processes to shepherd transitions to adult-oriented inpatient care. Across institutions, there is wide variability in performance of activities to facilitate this transition. Feasible process and outcome measures are needed.

Also in JHM this month

Characterizing hospitalist practice and perceptions of critical care delivery

AUTHORS: Joseph R. Sweigart, MD, FACP, FHM; David Aymond, MD; Alfred Burger, MD, FACP, SFHM; Andy Kelly, MAS, MS; Nick Marzano, Med; Thomas McIlraith, MD, SFHM; Peter Morris, MD; Mark V. Williams, MD, FACP, MHM; and Eric M. Siegal, MD, SFHM, FCCM

Clinical decision making: Observing the smartphone user an observational study in predicting acute surgical patients’ suitability for discharge

AUTHORS: Richard Hoffmann, MBBS; Simon Harley, MBBS; Samuel Ellison, MBBS; and Peter G. Devitt, MBBS, FRACS

BACKGROUND: Hospital charges and lengths of stay may be greater when adults with chronic conditions are admitted to children’s hospitals. Despite multiple efforts to improve pediatric-adult health care transitions, little guidance exists for transitioning inpatient care.

OBJECTIVE: This study sought to characterize pediatric-adult inpatient care transitions across general pediatric services at U.S. children’s hospitals.

DESIGN and SETTING: National survey of inpatient general pediatric service leaders at U.S. children’s hospitals from January 2016 to July 2016.

MEASUREMENT: Questionnaires assessed institutional characteristics, presence of inpatient transition initiatives (having a specific process and/or leader), and 22 inpatient transition activities. Scales of highly correlated activities were created using exploratory factor analysis. Logistic regression identified associations among institutional characteristics, transition activities, and presence of an inpatient transition initiative.

RESULTS: Of 195 children’s hospitals, 96 responded (49.2% response rate). Transition initiatives were present at 38% of children’s hospitals, more often where there were providers who were trained in both internal medicine and pediatrics or where there were outpatient transition processes. Specific activities were infrequent and varied widely from 2.1% (systems to track youth in transition) to 40.5% (addressing potential insurance problems). Institutions with initiatives more often consistently performed the majority of activities, including using checklists and creating patient-centered transition care plans. Of remaining activities, half involved transition planning, the essential step between readiness and transfer.

CONCLUSION: Relatively few inpatient general pediatric services at U.S. children’s hospitals have leaders or dedicated processes to shepherd transitions to adult-oriented inpatient care. Across institutions, there is wide variability in performance of activities to facilitate this transition. Feasible process and outcome measures are needed.

Also in JHM this month

Characterizing hospitalist practice and perceptions of critical care delivery

AUTHORS: Joseph R. Sweigart, MD, FACP, FHM; David Aymond, MD; Alfred Burger, MD, FACP, SFHM; Andy Kelly, MAS, MS; Nick Marzano, Med; Thomas McIlraith, MD, SFHM; Peter Morris, MD; Mark V. Williams, MD, FACP, MHM; and Eric M. Siegal, MD, SFHM, FCCM

Clinical decision making: Observing the smartphone user an observational study in predicting acute surgical patients’ suitability for discharge

AUTHORS: Richard Hoffmann, MBBS; Simon Harley, MBBS; Samuel Ellison, MBBS; and Peter G. Devitt, MBBS, FRACS

Over 90% of children with chronic diseases now survive into adulthood.^1,2 Clinical advances overcoming diseases previously fatal in childhood create new challenges for health systems with limited capacity to manage young adults with complicated and unfamiliar childhood-onset conditions. Consequently, improving the transition from pediatric to adult-oriented care has become a national priority.

Although major pediatric-adult transition initiatives—such as the Six Core Elements Framework,³ a technical brief from the Agency for Healthcare Research and Quality,⁴ and joint statements from major medical societies^5,6—outline key transition recommendations generally and for outpatients, they contain limited or no guidance specifically devoted to transitioning inpatient hospital care from pediatric to adult-oriented settings. Key unknowns include whether, when, and how to transition inpatient care from children’s to nonchildren’s hospitals and how this can be integrated into comprehensive youth-adult transition care.

Nevertheless, the number of discharges of 18- to 21-year-old patients with chronic conditions admitted to children’s hospitals is increasing at a faster rate than discharges of other age groups,⁷ suggesting both that the population is growing in size and that there are important barriers to transitioning these patients into nonchildren’s hospital settings. Spending on adult patients 18 years or older admitted to children’s hospitals has grown to $1 billion annually.⁸ Hospitalizations are a commonly proposed outcome measure of pediatric-adult transition work.^1,9,10 For example, higher rates of avoidable hospitalizations during early adulthood have been observed for 15- to 22-year-olds with kidney failure cared for exclusively in adult-oriented facilities and during the years immediately after transfer to adult care.¹¹

While research is beginning to describe outcomes of adult-aged patients with childhood-onset chronic conditions admitted to children’s hospitals,^7,12,13 there has been no comprehensive description of efforts within children’s hospitals to transition such patients into adult-oriented inpatient settings. This information is necessary to outline institutional needs, delineate opportunities for improvement, and help clinicians strategically organize services for patients requiring this transition.

We sought to characterize the current state of the transition from pediatric- to adult-oriented inpatient care across general pediatric inpatient services at US children’s hospitals. We hypothesized that only a limited and inconsistent set of activities would be practiced. We also hypothesized that institutions having formal outpatient transition processes or providers with specialization to care for this age group, such as dual-trained internal medicine–pediatrics (med–peds) physicians, would report performing more activities.

Study Design, Setting, Participants

We conducted a national survey of leaders of inpatient general pediatrics services at US children’s hospitals from January 2016 to July 2016. Hospitals were identified using the online Children’s Hospital Association directory. Hospitals without inpatient general pediatrics services (eg, rehabilitation or subspecialty-only facilities) were excluded.

We identified a single respondent from each of the 195 remaining children’s hospitals using a structured protocol. Phone numbers and e-mail addresses of potential respondents were gathered from hospital or medical school directories. Following a standard script, study team members contacted potential respondents to describe the purpose of the study and to confirm their contact information. Hospitals were also allowed to designate a different individual with more specific expertise to participate, when relevant (eg, specific faculty member leading a related quality improvement initiative). The goal was to identify a leader of inpatient care with the most knowledge of institutional practices related to the transition to adult inpatient care. Examples of respondent roles included director of inpatient pediatrics, chief of hospital medicine or general pediatrics, medical director, and similar titles.

Survey Elements

As part of a larger quality improvement initiative at our institution, a multidisciplinary team of pediatric and internal medicine healthcare providers (physicians, nurse practitioners, nurses, case managers, social workers, child life specialists), as well as parents and patients, developed an “ideal state” with this transition and a consensus-based conceptual framework of key patient and institutional determinants of a formal inpatient transition initiative for children with chronic conditions within a children’s hospital (Figure).

Institutional Context and Factors Influencing Inpatient Transitions

The following hospital characteristics were assessed: administrative structure (free-standing, hospital-within-hospital, or “free-leaning,” ie, separate physical structure but same administrative structure as a general hospital), urban versus rural, academic versus nonacademic, presence of an inpatient adolescent unit, presence of subspecialty admitting services, and providers with med–peds or family medicine training. The following provider group characteristics were assessed: number of full-time equivalents (FTEs), scope of practice (inpatient only, combination inpatient/outpatient), proportion of providers at a “senior” level (ie, at least 7 years posttraining or at an associate professor rank), estimated number of discharges per week, and proportion of patients cared for without resident physicians.

Inpatient Transition Initiative

Each institution was categorized as having or not having an inpatient transition initiative by whether they indicated having either (1) an institutional leader of the transition from pediatric to adult-oriented inpatient settings or (2) an inpatient transition process, for which “process” was defined as “a standard, organized, and predictable set of transition activities that may or may not be documented, but the steps are generally agreed upon.”

Specific Inpatient Transition Activities

Respondents indicated whether 22 activities occurred consistently, defined as at least 50% of the time. To facilitate description, activities were grouped into categories using the labels from the Six Core Elements framework³ (Table 1): Policy, Tracking and Monitoring, Readiness, Planning, Transfer of Care, and Transfer Completion. Respondents were also asked whether outpatient pediatric-adult transition activities existed at their institution and whether they were linked to inpatient transition activities.

Data Collection

After verifying contact information, respondents received an advanced priming phone call followed by a mailed request to participate with a printed uniform resource locator (URL) to the web survey. Two email reminders containing the URL were sent to nonresponders at 5 and 10 days after the initial mailing. Remaining nonresponders then received a reminder phone call, followed by a mailed paper copy of the survey questionnaire to be completed by hand approximately 2 weeks after the last emailed request. The survey was administered using the Qualtrics web survey platform (www.qualtrics.com). Data collection occurred between January 2016 and July 2016. Participants received a $20 incentive.

Statistical Analysis

Descriptive statistics summarized the current state of inpatient transition at general pediatrics services across US children’s hospitals. Exploratory factor analysis assessed whether individual activities were sufficiently correlated to allow grouping items and constructing scales. Differences in institutional or respondent characteristics between hospitals that did and did not report having an inpatient initiative were compared using t tests for continuous data. Fisher’s exact test was used for categorical data because some cell sizes were ≤5. Bivariate logistic regression quantified associations between presence versus absence of specific transition activities and presence versus absence of an inpatient transition initiative. Analyses were completed in STATA (SE version 14.0; StataCorp, College Station, Texas). The institutional review board at our institution approved this study.

Responses were received from 96 of 195 children’s hospitals (49.2% response rate). Responding institution characteristics are summarized in Table 2. Free-standing children’s hospitals made up just over one-third of the sample (36%), while the remaining were free-leaning (22%) or hospital-within-hospital (43%). Most children’s hospitals (58%) did not have a specific adult-oriented hospital identified to receive transitioning patients. Slightly more than 10% had an inpatient adolescent unit. The majority of institutions were academic medical centers (78%) in urban locations (88%). Respondents represented small (<5 FTE, 21%), medium (6-10 FTE, 36%), and large provider groups (11+ FTE, 44%). Although 70% of respondents described their groups as “hospitalist only,” meaning providers only practiced inpatient general pediatrics, nearly 30% had providers practicing inpatient and outpatient general pediatrics. Just over 40% of respondents reported having med–peds providers. Pediatric-adult transition processes for outpatient care were present at 45% of institutions.

Transition Activities

Thirty-eight percent of children’s hospitals had an inpatient transition initiative using our study definition—31% by having a set of generally agreed upon activities, 19% by having a leader, and 11% having both. Inpatient transition leaders included pediatric hospitalists (43%), pediatric subspecialists and primary care providers (14% each), med–peds providers (11%), or case managers (7%). Respondent and institutional characteristics were similar at institutions that did and did not have an inpatient transition initiative (Table 2); however, children’s hospitals with inpatient transition initiatives more often had med–peds providers (P = .04). Institutions with pediatric-adult outpatient care transition processes more often had an inpatient initiative (71% and 29%, respectively; P = .001).

Exploratory factor analysis identified 2 groups of well-correlated items, which we grouped into “preparation” and “transfer initiation” scales (supplementary Appendix). The preparation scale was composed of the following 5 items (Cronbach α = 0.84): proactive identification of patients anticipated to need transition, proactive identification of patients overdue for transition, readiness formally assessed, timing discussed with family, and patient and/or family informed that the next stay would be at the adult facility. The transfer initiation scale comprised the following 6 items (Cronbach α = 0.72): transition education provided to families, primary care–subspecialist agreement on timing, subspecialist–subspecialist agreement on timing, patient decision-making ability established, adult facility tour, and standardized handoff communication between healthcare providers. While these items were analyzed only in this scale, other activities were analyzed as independent variables. In this analysis, 40.9% of institutions had a preparation scale score of 0 (no items performed), while 13% had all 5 items performed. Transfer initiation scale scores ranged from 0 (47%) to 6 (2%).

Specific activities varied widely across institutions, and none of the activities occurred at a majority of children’s hospitals (Table 3). Only 11% of children’s hospital transition policies referenced transitions of inpatient care. The activity most commonly reported across children’s hospitals was addressing potential insurance problems (41%). The least common inpatient transition activities were having child life consult during the first adult hospital stay (6%) or having a system to track and monitor youth in the inpatient transition process (2%). Transition processes and policies were relatively new among institutions that had them—average years an inpatient transition process had been in place was 1.2 (SD 0.4), and average years with a transition policy, including inpatient care, was 1.3 (SD 0.4).

Transition Activities at Hospitals With and Without an Inpatient Transition Initiative

Most activities assessed in this study (both scales plus 5 of 11 individual activities) were significantly more common in children’s hospitals with an inpatient transition initiative (Table 3). The most common activity was addressing potential insurance problems (46%), and the least common activity was having a system to track and monitor youth in the inpatient transition process (3%). The majority of institutions without an inpatient transition initiative (53%) performed 0 transfer initiation scale items. Large effect sizes between hospitals with and without a transition initiative were observed for use of a checklist to complete tasks (odds ratio [OR] 9.6, P = .04) and creation of a transition care plan (OR 9.0, P = .008). Of the 6 activities performed at similarly low frequencies at institutions with and without an initiative, half involved transition planning, the essential step after readiness but before actual transfer of care.

We conducted the first national survey describing the policies and procedures of the transition of general inpatient care from children’s to adult-oriented hospitals for youth and young adults with chronic conditions. Our main findings demonstrate that a relatively small number of general inpatient services at children’s hospitals have leaders or dedicated processes to shepherd this transition, and a minority have a specific adult hospital identified to receive their patients. Even among institutions with inpatient transition initiatives, there is wide variability in the performance of activities to facilitate transitioning out of US children’s hospitals. In these institutions, performance seems to be more lacking in later links of the transition chain. Results from this work can serve as a baseline and identify organizational needs and opportunities for future work.

Children’s hospital general services with and without an inpatient pediatric-adult transition initiative had largely similar characteristics; however, the limited sample size may lack power to detect some differences. Perhaps not surprisingly, having med–peds providers and outpatient transition processes were the characteristics most associated with having an inpatient pediatric-adult transition initiative. The observation that over 70% of hospitals with an outpatient process had an inpatient transition leader or dedicated process makes us optimistic that as general transition efforts expand, more robust inpatient transition activities may be achievable.

We appreciate that the most appropriate location to care for hospitalized young adults with childhood-onset chronic conditions is neither known nor answered with this study. Both options face challenges—adult-oriented hospitals may not be equipped to care for adult manifestations of childhood-onset conditions,^14,15 while children’s hospitals may lack the resources and expertise to provide comprehensive care to adults.⁷ Although hospital charges and lengths of stay may be greater when adults with childhood-onset chronic conditions are admitted to children’s compared with adult hospitals,^12,13,16 important confounders such as severity of illness could explain why adult-aged patients may both remain in children’s hospitals at older ages and simultaneously have worse outcomes than peers. Regardless, at some point, transitioning care into an adult-oriented hospital may be in patients’ best interests. If so, families and providers need guidance on (1) the important aspects of this transition and (2) how to effectively implement the transition.

Because the most important inpatient transition care activities are not empirically known, we designed our survey to assess a broad set of desirable activities emerging from our multidisciplinary quality improvement work. We mapped these activities to the categories used by the Six Core Elements framework.³ Addressing insurance issues was one of the most commonly reported activities, although still fewer than 50% of hospitals reported addressing these problems. It was notable that the majority of institutions without a transition initiative performed none of the transfer initiation scale items. In addition, 2 features of transition efforts highlighted by advocates nationally—use of a checklist and creation of a transition care plan— were 9 times more likely when sites had transition initiatives. Such findings may be motivating for institutions that are considering establishing a transition initiative. Overall, we were not surprised with hospitals’ relatively low performance across most transition activities because only about 40% of US families of children with special healthcare needs report receiving the general services they need to transition to adult healthcare.¹⁷

We suspect that a number of the studied inpatient transition activities may be uncommon for structural reasons. For example, having child life consultation during an initial adult stay was rare. In fact, we observed post hoc that it occurred only in hospital-within-hospital systems, an expected finding because adult-only facilities are unlikely to have child life personnel. Other barriers, however, are less obviously structural. Almost no respondents indicated providing a tour of an adult facility, which was true whether the children’s hospital was free-standing or hospital-within-hospital. Given that hospitals with med–peds providers more often had inpatient transition initiatives, it would be interesting to examine whether institutions with med–peds training programs are able to overcome more of these barriers because of the bridges inherently created between departments even when at physically separated sites.

Having a system to track and/or monitor youth going through the transition process was also uncommon. This presumably valuable activity is one of the Six Core Elements³ and is reminiscent of population management strategies increasingly common in primary care.¹⁸ Pediatric hospitalists might benefit from adopting a similar philosophy for certain patient populations. Determining whether this activity would be most appropriately managed by inpatient providers versus being integrated into a comprehensive tracking and/or monitoring strategy (ie, inpatient care plus primary care, subspecialty care, school, employment, insurance, etc.) is worth continued consideration.

Although the activities we studied spanned many important dimensions, the most important transition activities in any given context may differ based on institutional resources and those of nearby adult healthcare providers.¹⁶ For example, an activity may be absent at a children’s hospital because it is already readily handled in primary care within that health system. Understanding how local resources and patient needs influence the relationship between transition activities and outcomes is an important next step in this line of work. Such research could inform how institutions adapt effective transition activities (eg, developing care plans) to most efficiently meet the needs of their patients and families.

Our findings align with and advance the limited work published on this aspect of transition. A systematic literature review of general healthcare transition interventions found that meeting adult providers prior to transitioning out of the pediatric system was associated with less concern about admission to the adult hospital floor.⁹ Formally recognizing inpatient care as a part of a comprehensive approach to transition may help adults with childhood-onset chronic conditions progress into adult-oriented hospitals. Inpatient and outpatient providers can educate one another on critical aspects of transition that span across settings. The Cystic Fibrosis (CF) Foundation has established a set of processes to facilitate the transition to adult care and specifically articulates the transfer to adult inpatient settings.^19,20 Perhaps as a result, CF is also one of few conditions with fewer adult patients being admitted to children’s hospitals⁷ despite the increasing number of adults living with the condition.¹⁹ Adapting the CF Foundation approach to other chronic conditions may be an effective approach.

Our study has important limitations. Most pertinently, the list of transition activities was developed at a single institution. Although drawing on accepted national guidelines and a diverse local quality improvement group, our listed activities could not be exhaustive. Care plan development and posttransition follow-up activities may benefit from ongoing development in subsequent work. Continuing to identify and integrate approaches taken at other children’s hospitals will also be informative. For example, some children’s hospitals have introduced adult medicine consultative services to focus on transition, attending children’s hospital safety rounds, and sharing standard care protocols for adult patients still cared for in pediatric settings (eg, stroke and myocardial infarction).¹⁶

In addition, our findings are limited to generalist teams at children’s hospitals and may not be applicable to inpatient subspecialty services. We could not compare differences in respondents versus nonrespondents to determine whether important selection bias exists. Respondent answers could not be verified. Despite our attempt to identify the most informed respondent at each hospital, responses may have differed with other hospital respondents. We used a novel instrument with unknown psychometric properties. Our data provide only the children’s hospital perspective, and perspectives of others (eg, families, primary care pediatricians or internists, subspecialists, etc.) will be valuable to explore in subsequent research. Subsequent research should investigate the relative importance and feasibility of specific inpatient transition activities, ideal timing, as well as the expected outcomes of high-quality inpatient transition. An important question for future work is to identify which patients are most likely to benefit by having inpatient care as part of their transition plan.

Nevertheless, the clinical and health services implications of this facet of transition appear to be substantial.¹⁶ To meet the Maternal and Child Health Bureau (MCHB) core outcome for children with special healthcare needs to receive “the services necessary to make transitions to adult healthcare,”²¹ development, validation, and implementation of effective inpatient-specific transition activities and a set of measurable processes and outcomes are needed. A key direction for the healthcare transitions field, with respect to inpatient care, is to determine the activities most effective at improving relevant patient and family outcomes. Ultimately, we advocate that the transition of inpatient care be integrated into comprehensive approaches to transitional care.

Disclosure: The project described was supported in part by the Clinical and Translational Science Award (CTSA) program, through the National Institutes of Health (NIH) National Center for Advancing Translational Sciences (NCATS), grant UL1TR000427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The project was also supported by the University of Wisconsin Departments of Pediatrics and Medicine. The authors have no financial or other relationships relevant to this article to disclose.

Over 90% of children with chronic diseases now survive into adulthood.^1,2 Clinical advances overcoming diseases previously fatal in childhood create new challenges for health systems with limited capacity to manage young adults with complicated and unfamiliar childhood-onset conditions. Consequently, improving the transition from pediatric to adult-oriented care has become a national priority.

Although major pediatric-adult transition initiatives—such as the Six Core Elements Framework,³ a technical brief from the Agency for Healthcare Research and Quality,⁴ and joint statements from major medical societies^5,6—outline key transition recommendations generally and for outpatients, they contain limited or no guidance specifically devoted to transitioning inpatient hospital care from pediatric to adult-oriented settings. Key unknowns include whether, when, and how to transition inpatient care from children’s to nonchildren’s hospitals and how this can be integrated into comprehensive youth-adult transition care.

Nevertheless, the number of discharges of 18- to 21-year-old patients with chronic conditions admitted to children’s hospitals is increasing at a faster rate than discharges of other age groups,⁷ suggesting both that the population is growing in size and that there are important barriers to transitioning these patients into nonchildren’s hospital settings. Spending on adult patients 18 years or older admitted to children’s hospitals has grown to $1 billion annually.⁸ Hospitalizations are a commonly proposed outcome measure of pediatric-adult transition work.^1,9,10 For example, higher rates of avoidable hospitalizations during early adulthood have been observed for 15- to 22-year-olds with kidney failure cared for exclusively in adult-oriented facilities and during the years immediately after transfer to adult care.¹¹

While research is beginning to describe outcomes of adult-aged patients with childhood-onset chronic conditions admitted to children’s hospitals,^7,12,13 there has been no comprehensive description of efforts within children’s hospitals to transition such patients into adult-oriented inpatient settings. This information is necessary to outline institutional needs, delineate opportunities for improvement, and help clinicians strategically organize services for patients requiring this transition.

We sought to characterize the current state of the transition from pediatric- to adult-oriented inpatient care across general pediatric inpatient services at US children’s hospitals. We hypothesized that only a limited and inconsistent set of activities would be practiced. We also hypothesized that institutions having formal outpatient transition processes or providers with specialization to care for this age group, such as dual-trained internal medicine–pediatrics (med–peds) physicians, would report performing more activities.

Study Design, Setting, Participants

We conducted a national survey of leaders of inpatient general pediatrics services at US children’s hospitals from January 2016 to July 2016. Hospitals were identified using the online Children’s Hospital Association directory. Hospitals without inpatient general pediatrics services (eg, rehabilitation or subspecialty-only facilities) were excluded.

We identified a single respondent from each of the 195 remaining children’s hospitals using a structured protocol. Phone numbers and e-mail addresses of potential respondents were gathered from hospital or medical school directories. Following a standard script, study team members contacted potential respondents to describe the purpose of the study and to confirm their contact information. Hospitals were also allowed to designate a different individual with more specific expertise to participate, when relevant (eg, specific faculty member leading a related quality improvement initiative). The goal was to identify a leader of inpatient care with the most knowledge of institutional practices related to the transition to adult inpatient care. Examples of respondent roles included director of inpatient pediatrics, chief of hospital medicine or general pediatrics, medical director, and similar titles.

Survey Elements

As part of a larger quality improvement initiative at our institution, a multidisciplinary team of pediatric and internal medicine healthcare providers (physicians, nurse practitioners, nurses, case managers, social workers, child life specialists), as well as parents and patients, developed an “ideal state” with this transition and a consensus-based conceptual framework of key patient and institutional determinants of a formal inpatient transition initiative for children with chronic conditions within a children’s hospital (Figure).

Institutional Context and Factors Influencing Inpatient Transitions

The following hospital characteristics were assessed: administrative structure (free-standing, hospital-within-hospital, or “free-leaning,” ie, separate physical structure but same administrative structure as a general hospital), urban versus rural, academic versus nonacademic, presence of an inpatient adolescent unit, presence of subspecialty admitting services, and providers with med–peds or family medicine training. The following provider group characteristics were assessed: number of full-time equivalents (FTEs), scope of practice (inpatient only, combination inpatient/outpatient), proportion of providers at a “senior” level (ie, at least 7 years posttraining or at an associate professor rank), estimated number of discharges per week, and proportion of patients cared for without resident physicians.

Inpatient Transition Initiative

Each institution was categorized as having or not having an inpatient transition initiative by whether they indicated having either (1) an institutional leader of the transition from pediatric to adult-oriented inpatient settings or (2) an inpatient transition process, for which “process” was defined as “a standard, organized, and predictable set of transition activities that may or may not be documented, but the steps are generally agreed upon.”

Specific Inpatient Transition Activities

Respondents indicated whether 22 activities occurred consistently, defined as at least 50% of the time. To facilitate description, activities were grouped into categories using the labels from the Six Core Elements framework³ (Table 1): Policy, Tracking and Monitoring, Readiness, Planning, Transfer of Care, and Transfer Completion. Respondents were also asked whether outpatient pediatric-adult transition activities existed at their institution and whether they were linked to inpatient transition activities.

Data Collection

After verifying contact information, respondents received an advanced priming phone call followed by a mailed request to participate with a printed uniform resource locator (URL) to the web survey. Two email reminders containing the URL were sent to nonresponders at 5 and 10 days after the initial mailing. Remaining nonresponders then received a reminder phone call, followed by a mailed paper copy of the survey questionnaire to be completed by hand approximately 2 weeks after the last emailed request. The survey was administered using the Qualtrics web survey platform (www.qualtrics.com). Data collection occurred between January 2016 and July 2016. Participants received a $20 incentive.

Statistical Analysis

Descriptive statistics summarized the current state of inpatient transition at general pediatrics services across US children’s hospitals. Exploratory factor analysis assessed whether individual activities were sufficiently correlated to allow grouping items and constructing scales. Differences in institutional or respondent characteristics between hospitals that did and did not report having an inpatient initiative were compared using t tests for continuous data. Fisher’s exact test was used for categorical data because some cell sizes were ≤5. Bivariate logistic regression quantified associations between presence versus absence of specific transition activities and presence versus absence of an inpatient transition initiative. Analyses were completed in STATA (SE version 14.0; StataCorp, College Station, Texas). The institutional review board at our institution approved this study.

Responses were received from 96 of 195 children’s hospitals (49.2% response rate). Responding institution characteristics are summarized in Table 2. Free-standing children’s hospitals made up just over one-third of the sample (36%), while the remaining were free-leaning (22%) or hospital-within-hospital (43%). Most children’s hospitals (58%) did not have a specific adult-oriented hospital identified to receive transitioning patients. Slightly more than 10% had an inpatient adolescent unit. The majority of institutions were academic medical centers (78%) in urban locations (88%). Respondents represented small (<5 FTE, 21%), medium (6-10 FTE, 36%), and large provider groups (11+ FTE, 44%). Although 70% of respondents described their groups as “hospitalist only,” meaning providers only practiced inpatient general pediatrics, nearly 30% had providers practicing inpatient and outpatient general pediatrics. Just over 40% of respondents reported having med–peds providers. Pediatric-adult transition processes for outpatient care were present at 45% of institutions.

Transition Activities

Thirty-eight percent of children’s hospitals had an inpatient transition initiative using our study definition—31% by having a set of generally agreed upon activities, 19% by having a leader, and 11% having both. Inpatient transition leaders included pediatric hospitalists (43%), pediatric subspecialists and primary care providers (14% each), med–peds providers (11%), or case managers (7%). Respondent and institutional characteristics were similar at institutions that did and did not have an inpatient transition initiative (Table 2); however, children’s hospitals with inpatient transition initiatives more often had med–peds providers (P = .04). Institutions with pediatric-adult outpatient care transition processes more often had an inpatient initiative (71% and 29%, respectively; P = .001).

Exploratory factor analysis identified 2 groups of well-correlated items, which we grouped into “preparation” and “transfer initiation” scales (supplementary Appendix). The preparation scale was composed of the following 5 items (Cronbach α = 0.84): proactive identification of patients anticipated to need transition, proactive identification of patients overdue for transition, readiness formally assessed, timing discussed with family, and patient and/or family informed that the next stay would be at the adult facility. The transfer initiation scale comprised the following 6 items (Cronbach α = 0.72): transition education provided to families, primary care–subspecialist agreement on timing, subspecialist–subspecialist agreement on timing, patient decision-making ability established, adult facility tour, and standardized handoff communication between healthcare providers. While these items were analyzed only in this scale, other activities were analyzed as independent variables. In this analysis, 40.9% of institutions had a preparation scale score of 0 (no items performed), while 13% had all 5 items performed. Transfer initiation scale scores ranged from 0 (47%) to 6 (2%).

Specific activities varied widely across institutions, and none of the activities occurred at a majority of children’s hospitals (Table 3). Only 11% of children’s hospital transition policies referenced transitions of inpatient care. The activity most commonly reported across children’s hospitals was addressing potential insurance problems (41%). The least common inpatient transition activities were having child life consult during the first adult hospital stay (6%) or having a system to track and monitor youth in the inpatient transition process (2%). Transition processes and policies were relatively new among institutions that had them—average years an inpatient transition process had been in place was 1.2 (SD 0.4), and average years with a transition policy, including inpatient care, was 1.3 (SD 0.4).

Transition Activities at Hospitals With and Without an Inpatient Transition Initiative

Most activities assessed in this study (both scales plus 5 of 11 individual activities) were significantly more common in children’s hospitals with an inpatient transition initiative (Table 3). The most common activity was addressing potential insurance problems (46%), and the least common activity was having a system to track and monitor youth in the inpatient transition process (3%). The majority of institutions without an inpatient transition initiative (53%) performed 0 transfer initiation scale items. Large effect sizes between hospitals with and without a transition initiative were observed for use of a checklist to complete tasks (odds ratio [OR] 9.6, P = .04) and creation of a transition care plan (OR 9.0, P = .008). Of the 6 activities performed at similarly low frequencies at institutions with and without an initiative, half involved transition planning, the essential step after readiness but before actual transfer of care.

We conducted the first national survey describing the policies and procedures of the transition of general inpatient care from children’s to adult-oriented hospitals for youth and young adults with chronic conditions. Our main findings demonstrate that a relatively small number of general inpatient services at children’s hospitals have leaders or dedicated processes to shepherd this transition, and a minority have a specific adult hospital identified to receive their patients. Even among institutions with inpatient transition initiatives, there is wide variability in the performance of activities to facilitate transitioning out of US children’s hospitals. In these institutions, performance seems to be more lacking in later links of the transition chain. Results from this work can serve as a baseline and identify organizational needs and opportunities for future work.

Children’s hospital general services with and without an inpatient pediatric-adult transition initiative had largely similar characteristics; however, the limited sample size may lack power to detect some differences. Perhaps not surprisingly, having med–peds providers and outpatient transition processes were the characteristics most associated with having an inpatient pediatric-adult transition initiative. The observation that over 70% of hospitals with an outpatient process had an inpatient transition leader or dedicated process makes us optimistic that as general transition efforts expand, more robust inpatient transition activities may be achievable.

We appreciate that the most appropriate location to care for hospitalized young adults with childhood-onset chronic conditions is neither known nor answered with this study. Both options face challenges—adult-oriented hospitals may not be equipped to care for adult manifestations of childhood-onset conditions,^14,15 while children’s hospitals may lack the resources and expertise to provide comprehensive care to adults.⁷ Although hospital charges and lengths of stay may be greater when adults with childhood-onset chronic conditions are admitted to children’s compared with adult hospitals,^12,13,16 important confounders such as severity of illness could explain why adult-aged patients may both remain in children’s hospitals at older ages and simultaneously have worse outcomes than peers. Regardless, at some point, transitioning care into an adult-oriented hospital may be in patients’ best interests. If so, families and providers need guidance on (1) the important aspects of this transition and (2) how to effectively implement the transition.

Because the most important inpatient transition care activities are not empirically known, we designed our survey to assess a broad set of desirable activities emerging from our multidisciplinary quality improvement work. We mapped these activities to the categories used by the Six Core Elements framework.³ Addressing insurance issues was one of the most commonly reported activities, although still fewer than 50% of hospitals reported addressing these problems. It was notable that the majority of institutions without a transition initiative performed none of the transfer initiation scale items. In addition, 2 features of transition efforts highlighted by advocates nationally—use of a checklist and creation of a transition care plan— were 9 times more likely when sites had transition initiatives. Such findings may be motivating for institutions that are considering establishing a transition initiative. Overall, we were not surprised with hospitals’ relatively low performance across most transition activities because only about 40% of US families of children with special healthcare needs report receiving the general services they need to transition to adult healthcare.¹⁷

We suspect that a number of the studied inpatient transition activities may be uncommon for structural reasons. For example, having child life consultation during an initial adult stay was rare. In fact, we observed post hoc that it occurred only in hospital-within-hospital systems, an expected finding because adult-only facilities are unlikely to have child life personnel. Other barriers, however, are less obviously structural. Almost no respondents indicated providing a tour of an adult facility, which was true whether the children’s hospital was free-standing or hospital-within-hospital. Given that hospitals with med–peds providers more often had inpatient transition initiatives, it would be interesting to examine whether institutions with med–peds training programs are able to overcome more of these barriers because of the bridges inherently created between departments even when at physically separated sites.

Having a system to track and/or monitor youth going through the transition process was also uncommon. This presumably valuable activity is one of the Six Core Elements³ and is reminiscent of population management strategies increasingly common in primary care.¹⁸ Pediatric hospitalists might benefit from adopting a similar philosophy for certain patient populations. Determining whether this activity would be most appropriately managed by inpatient providers versus being integrated into a comprehensive tracking and/or monitoring strategy (ie, inpatient care plus primary care, subspecialty care, school, employment, insurance, etc.) is worth continued consideration.

Although the activities we studied spanned many important dimensions, the most important transition activities in any given context may differ based on institutional resources and those of nearby adult healthcare providers.¹⁶ For example, an activity may be absent at a children’s hospital because it is already readily handled in primary care within that health system. Understanding how local resources and patient needs influence the relationship between transition activities and outcomes is an important next step in this line of work. Such research could inform how institutions adapt effective transition activities (eg, developing care plans) to most efficiently meet the needs of their patients and families.

Our findings align with and advance the limited work published on this aspect of transition. A systematic literature review of general healthcare transition interventions found that meeting adult providers prior to transitioning out of the pediatric system was associated with less concern about admission to the adult hospital floor.⁹ Formally recognizing inpatient care as a part of a comprehensive approach to transition may help adults with childhood-onset chronic conditions progress into adult-oriented hospitals. Inpatient and outpatient providers can educate one another on critical aspects of transition that span across settings. The Cystic Fibrosis (CF) Foundation has established a set of processes to facilitate the transition to adult care and specifically articulates the transfer to adult inpatient settings.^19,20 Perhaps as a result, CF is also one of few conditions with fewer adult patients being admitted to children’s hospitals⁷ despite the increasing number of adults living with the condition.¹⁹ Adapting the CF Foundation approach to other chronic conditions may be an effective approach.

Our study has important limitations. Most pertinently, the list of transition activities was developed at a single institution. Although drawing on accepted national guidelines and a diverse local quality improvement group, our listed activities could not be exhaustive. Care plan development and posttransition follow-up activities may benefit from ongoing development in subsequent work. Continuing to identify and integrate approaches taken at other children’s hospitals will also be informative. For example, some children’s hospitals have introduced adult medicine consultative services to focus on transition, attending children’s hospital safety rounds, and sharing standard care protocols for adult patients still cared for in pediatric settings (eg, stroke and myocardial infarction).¹⁶

In addition, our findings are limited to generalist teams at children’s hospitals and may not be applicable to inpatient subspecialty services. We could not compare differences in respondents versus nonrespondents to determine whether important selection bias exists. Respondent answers could not be verified. Despite our attempt to identify the most informed respondent at each hospital, responses may have differed with other hospital respondents. We used a novel instrument with unknown psychometric properties. Our data provide only the children’s hospital perspective, and perspectives of others (eg, families, primary care pediatricians or internists, subspecialists, etc.) will be valuable to explore in subsequent research. Subsequent research should investigate the relative importance and feasibility of specific inpatient transition activities, ideal timing, as well as the expected outcomes of high-quality inpatient transition. An important question for future work is to identify which patients are most likely to benefit by having inpatient care as part of their transition plan.

Nevertheless, the clinical and health services implications of this facet of transition appear to be substantial.¹⁶ To meet the Maternal and Child Health Bureau (MCHB) core outcome for children with special healthcare needs to receive “the services necessary to make transitions to adult healthcare,”²¹ development, validation, and implementation of effective inpatient-specific transition activities and a set of measurable processes and outcomes are needed. A key direction for the healthcare transitions field, with respect to inpatient care, is to determine the activities most effective at improving relevant patient and family outcomes. Ultimately, we advocate that the transition of inpatient care be integrated into comprehensive approaches to transitional care.

Disclosure: The project described was supported in part by the Clinical and Translational Science Award (CTSA) program, through the National Institutes of Health (NIH) National Center for Advancing Translational Sciences (NCATS), grant UL1TR000427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The project was also supported by the University of Wisconsin Departments of Pediatrics and Medicine. The authors have no financial or other relationships relevant to this article to disclose.

Over 90% of children with chronic diseases now survive into adulthood.^1,2 Clinical advances overcoming diseases previously fatal in childhood create new challenges for health systems with limited capacity to manage young adults with complicated and unfamiliar childhood-onset conditions. Consequently, improving the transition from pediatric to adult-oriented care has become a national priority.

Although major pediatric-adult transition initiatives—such as the Six Core Elements Framework,³ a technical brief from the Agency for Healthcare Research and Quality,⁴ and joint statements from major medical societies^5,6—outline key transition recommendations generally and for outpatients, they contain limited or no guidance specifically devoted to transitioning inpatient hospital care from pediatric to adult-oriented settings. Key unknowns include whether, when, and how to transition inpatient care from children’s to nonchildren’s hospitals and how this can be integrated into comprehensive youth-adult transition care.

Nevertheless, the number of discharges of 18- to 21-year-old patients with chronic conditions admitted to children’s hospitals is increasing at a faster rate than discharges of other age groups,⁷ suggesting both that the population is growing in size and that there are important barriers to transitioning these patients into nonchildren’s hospital settings. Spending on adult patients 18 years or older admitted to children’s hospitals has grown to $1 billion annually.⁸ Hospitalizations are a commonly proposed outcome measure of pediatric-adult transition work.^1,9,10 For example, higher rates of avoidable hospitalizations during early adulthood have been observed for 15- to 22-year-olds with kidney failure cared for exclusively in adult-oriented facilities and during the years immediately after transfer to adult care.¹¹

While research is beginning to describe outcomes of adult-aged patients with childhood-onset chronic conditions admitted to children’s hospitals,^7,12,13 there has been no comprehensive description of efforts within children’s hospitals to transition such patients into adult-oriented inpatient settings. This information is necessary to outline institutional needs, delineate opportunities for improvement, and help clinicians strategically organize services for patients requiring this transition.

We sought to characterize the current state of the transition from pediatric- to adult-oriented inpatient care across general pediatric inpatient services at US children’s hospitals. We hypothesized that only a limited and inconsistent set of activities would be practiced. We also hypothesized that institutions having formal outpatient transition processes or providers with specialization to care for this age group, such as dual-trained internal medicine–pediatrics (med–peds) physicians, would report performing more activities.

Study Design, Setting, Participants

We conducted a national survey of leaders of inpatient general pediatrics services at US children’s hospitals from January 2016 to July 2016. Hospitals were identified using the online Children’s Hospital Association directory. Hospitals without inpatient general pediatrics services (eg, rehabilitation or subspecialty-only facilities) were excluded.

We identified a single respondent from each of the 195 remaining children’s hospitals using a structured protocol. Phone numbers and e-mail addresses of potential respondents were gathered from hospital or medical school directories. Following a standard script, study team members contacted potential respondents to describe the purpose of the study and to confirm their contact information. Hospitals were also allowed to designate a different individual with more specific expertise to participate, when relevant (eg, specific faculty member leading a related quality improvement initiative). The goal was to identify a leader of inpatient care with the most knowledge of institutional practices related to the transition to adult inpatient care. Examples of respondent roles included director of inpatient pediatrics, chief of hospital medicine or general pediatrics, medical director, and similar titles.

Survey Elements

As part of a larger quality improvement initiative at our institution, a multidisciplinary team of pediatric and internal medicine healthcare providers (physicians, nurse practitioners, nurses, case managers, social workers, child life specialists), as well as parents and patients, developed an “ideal state” with this transition and a consensus-based conceptual framework of key patient and institutional determinants of a formal inpatient transition initiative for children with chronic conditions within a children’s hospital (Figure).

Institutional Context and Factors Influencing Inpatient Transitions

The following hospital characteristics were assessed: administrative structure (free-standing, hospital-within-hospital, or “free-leaning,” ie, separate physical structure but same administrative structure as a general hospital), urban versus rural, academic versus nonacademic, presence of an inpatient adolescent unit, presence of subspecialty admitting services, and providers with med–peds or family medicine training. The following provider group characteristics were assessed: number of full-time equivalents (FTEs), scope of practice (inpatient only, combination inpatient/outpatient), proportion of providers at a “senior” level (ie, at least 7 years posttraining or at an associate professor rank), estimated number of discharges per week, and proportion of patients cared for without resident physicians.

Inpatient Transition Initiative

Each institution was categorized as having or not having an inpatient transition initiative by whether they indicated having either (1) an institutional leader of the transition from pediatric to adult-oriented inpatient settings or (2) an inpatient transition process, for which “process” was defined as “a standard, organized, and predictable set of transition activities that may or may not be documented, but the steps are generally agreed upon.”

Specific Inpatient Transition Activities

Respondents indicated whether 22 activities occurred consistently, defined as at least 50% of the time. To facilitate description, activities were grouped into categories using the labels from the Six Core Elements framework³ (Table 1): Policy, Tracking and Monitoring, Readiness, Planning, Transfer of Care, and Transfer Completion. Respondents were also asked whether outpatient pediatric-adult transition activities existed at their institution and whether they were linked to inpatient transition activities.

Data Collection

After verifying contact information, respondents received an advanced priming phone call followed by a mailed request to participate with a printed uniform resource locator (URL) to the web survey. Two email reminders containing the URL were sent to nonresponders at 5 and 10 days after the initial mailing. Remaining nonresponders then received a reminder phone call, followed by a mailed paper copy of the survey questionnaire to be completed by hand approximately 2 weeks after the last emailed request. The survey was administered using the Qualtrics web survey platform (www.qualtrics.com). Data collection occurred between January 2016 and July 2016. Participants received a $20 incentive.

Statistical Analysis

Descriptive statistics summarized the current state of inpatient transition at general pediatrics services across US children’s hospitals. Exploratory factor analysis assessed whether individual activities were sufficiently correlated to allow grouping items and constructing scales. Differences in institutional or respondent characteristics between hospitals that did and did not report having an inpatient initiative were compared using t tests for continuous data. Fisher’s exact test was used for categorical data because some cell sizes were ≤5. Bivariate logistic regression quantified associations between presence versus absence of specific transition activities and presence versus absence of an inpatient transition initiative. Analyses were completed in STATA (SE version 14.0; StataCorp, College Station, Texas). The institutional review board at our institution approved this study.

Responses were received from 96 of 195 children’s hospitals (49.2% response rate). Responding institution characteristics are summarized in Table 2. Free-standing children’s hospitals made up just over one-third of the sample (36%), while the remaining were free-leaning (22%) or hospital-within-hospital (43%). Most children’s hospitals (58%) did not have a specific adult-oriented hospital identified to receive transitioning patients. Slightly more than 10% had an inpatient adolescent unit. The majority of institutions were academic medical centers (78%) in urban locations (88%). Respondents represented small (<5 FTE, 21%), medium (6-10 FTE, 36%), and large provider groups (11+ FTE, 44%). Although 70% of respondents described their groups as “hospitalist only,” meaning providers only practiced inpatient general pediatrics, nearly 30% had providers practicing inpatient and outpatient general pediatrics. Just over 40% of respondents reported having med–peds providers. Pediatric-adult transition processes for outpatient care were present at 45% of institutions.

Transition Activities

Thirty-eight percent of children’s hospitals had an inpatient transition initiative using our study definition—31% by having a set of generally agreed upon activities, 19% by having a leader, and 11% having both. Inpatient transition leaders included pediatric hospitalists (43%), pediatric subspecialists and primary care providers (14% each), med–peds providers (11%), or case managers (7%). Respondent and institutional characteristics were similar at institutions that did and did not have an inpatient transition initiative (Table 2); however, children’s hospitals with inpatient transition initiatives more often had med–peds providers (P = .04). Institutions with pediatric-adult outpatient care transition processes more often had an inpatient initiative (71% and 29%, respectively; P = .001).

Exploratory factor analysis identified 2 groups of well-correlated items, which we grouped into “preparation” and “transfer initiation” scales (supplementary Appendix). The preparation scale was composed of the following 5 items (Cronbach α = 0.84): proactive identification of patients anticipated to need transition, proactive identification of patients overdue for transition, readiness formally assessed, timing discussed with family, and patient and/or family informed that the next stay would be at the adult facility. The transfer initiation scale comprised the following 6 items (Cronbach α = 0.72): transition education provided to families, primary care–subspecialist agreement on timing, subspecialist–subspecialist agreement on timing, patient decision-making ability established, adult facility tour, and standardized handoff communication between healthcare providers. While these items were analyzed only in this scale, other activities were analyzed as independent variables. In this analysis, 40.9% of institutions had a preparation scale score of 0 (no items performed), while 13% had all 5 items performed. Transfer initiation scale scores ranged from 0 (47%) to 6 (2%).

Specific activities varied widely across institutions, and none of the activities occurred at a majority of children’s hospitals (Table 3). Only 11% of children’s hospital transition policies referenced transitions of inpatient care. The activity most commonly reported across children’s hospitals was addressing potential insurance problems (41%). The least common inpatient transition activities were having child life consult during the first adult hospital stay (6%) or having a system to track and monitor youth in the inpatient transition process (2%). Transition processes and policies were relatively new among institutions that had them—average years an inpatient transition process had been in place was 1.2 (SD 0.4), and average years with a transition policy, including inpatient care, was 1.3 (SD 0.4).

Transition Activities at Hospitals With and Without an Inpatient Transition Initiative

Most activities assessed in this study (both scales plus 5 of 11 individual activities) were significantly more common in children’s hospitals with an inpatient transition initiative (Table 3). The most common activity was addressing potential insurance problems (46%), and the least common activity was having a system to track and monitor youth in the inpatient transition process (3%). The majority of institutions without an inpatient transition initiative (53%) performed 0 transfer initiation scale items. Large effect sizes between hospitals with and without a transition initiative were observed for use of a checklist to complete tasks (odds ratio [OR] 9.6, P = .04) and creation of a transition care plan (OR 9.0, P = .008). Of the 6 activities performed at similarly low frequencies at institutions with and without an initiative, half involved transition planning, the essential step after readiness but before actual transfer of care.

We conducted the first national survey describing the policies and procedures of the transition of general inpatient care from children’s to adult-oriented hospitals for youth and young adults with chronic conditions. Our main findings demonstrate that a relatively small number of general inpatient services at children’s hospitals have leaders or dedicated processes to shepherd this transition, and a minority have a specific adult hospital identified to receive their patients. Even among institutions with inpatient transition initiatives, there is wide variability in the performance of activities to facilitate transitioning out of US children’s hospitals. In these institutions, performance seems to be more lacking in later links of the transition chain. Results from this work can serve as a baseline and identify organizational needs and opportunities for future work.

Children’s hospital general services with and without an inpatient pediatric-adult transition initiative had largely similar characteristics; however, the limited sample size may lack power to detect some differences. Perhaps not surprisingly, having med–peds providers and outpatient transition processes were the characteristics most associated with having an inpatient pediatric-adult transition initiative. The observation that over 70% of hospitals with an outpatient process had an inpatient transition leader or dedicated process makes us optimistic that as general transition efforts expand, more robust inpatient transition activities may be achievable.

We appreciate that the most appropriate location to care for hospitalized young adults with childhood-onset chronic conditions is neither known nor answered with this study. Both options face challenges—adult-oriented hospitals may not be equipped to care for adult manifestations of childhood-onset conditions,^14,15 while children’s hospitals may lack the resources and expertise to provide comprehensive care to adults.⁷ Although hospital charges and lengths of stay may be greater when adults with childhood-onset chronic conditions are admitted to children’s compared with adult hospitals,^12,13,16 important confounders such as severity of illness could explain why adult-aged patients may both remain in children’s hospitals at older ages and simultaneously have worse outcomes than peers. Regardless, at some point, transitioning care into an adult-oriented hospital may be in patients’ best interests. If so, families and providers need guidance on (1) the important aspects of this transition and (2) how to effectively implement the transition.

Because the most important inpatient transition care activities are not empirically known, we designed our survey to assess a broad set of desirable activities emerging from our multidisciplinary quality improvement work. We mapped these activities to the categories used by the Six Core Elements framework.³ Addressing insurance issues was one of the most commonly reported activities, although still fewer than 50% of hospitals reported addressing these problems. It was notable that the majority of institutions without a transition initiative performed none of the transfer initiation scale items. In addition, 2 features of transition efforts highlighted by advocates nationally—use of a checklist and creation of a transition care plan— were 9 times more likely when sites had transition initiatives. Such findings may be motivating for institutions that are considering establishing a transition initiative. Overall, we were not surprised with hospitals’ relatively low performance across most transition activities because only about 40% of US families of children with special healthcare needs report receiving the general services they need to transition to adult healthcare.¹⁷

We suspect that a number of the studied inpatient transition activities may be uncommon for structural reasons. For example, having child life consultation during an initial adult stay was rare. In fact, we observed post hoc that it occurred only in hospital-within-hospital systems, an expected finding because adult-only facilities are unlikely to have child life personnel. Other barriers, however, are less obviously structural. Almost no respondents indicated providing a tour of an adult facility, which was true whether the children’s hospital was free-standing or hospital-within-hospital. Given that hospitals with med–peds providers more often had inpatient transition initiatives, it would be interesting to examine whether institutions with med–peds training programs are able to overcome more of these barriers because of the bridges inherently created between departments even when at physically separated sites.

Having a system to track and/or monitor youth going through the transition process was also uncommon. This presumably valuable activity is one of the Six Core Elements³ and is reminiscent of population management strategies increasingly common in primary care.¹⁸ Pediatric hospitalists might benefit from adopting a similar philosophy for certain patient populations. Determining whether this activity would be most appropriately managed by inpatient providers versus being integrated into a comprehensive tracking and/or monitoring strategy (ie, inpatient care plus primary care, subspecialty care, school, employment, insurance, etc.) is worth continued consideration.

Although the activities we studied spanned many important dimensions, the most important transition activities in any given context may differ based on institutional resources and those of nearby adult healthcare providers.¹⁶ For example, an activity may be absent at a children’s hospital because it is already readily handled in primary care within that health system. Understanding how local resources and patient needs influence the relationship between transition activities and outcomes is an important next step in this line of work. Such research could inform how institutions adapt effective transition activities (eg, developing care plans) to most efficiently meet the needs of their patients and families.

Our findings align with and advance the limited work published on this aspect of transition. A systematic literature review of general healthcare transition interventions found that meeting adult providers prior to transitioning out of the pediatric system was associated with less concern about admission to the adult hospital floor.⁹ Formally recognizing inpatient care as a part of a comprehensive approach to transition may help adults with childhood-onset chronic conditions progress into adult-oriented hospitals. Inpatient and outpatient providers can educate one another on critical aspects of transition that span across settings. The Cystic Fibrosis (CF) Foundation has established a set of processes to facilitate the transition to adult care and specifically articulates the transfer to adult inpatient settings.^19,20 Perhaps as a result, CF is also one of few conditions with fewer adult patients being admitted to children’s hospitals⁷ despite the increasing number of adults living with the condition.¹⁹ Adapting the CF Foundation approach to other chronic conditions may be an effective approach.

Our study has important limitations. Most pertinently, the list of transition activities was developed at a single institution. Although drawing on accepted national guidelines and a diverse local quality improvement group, our listed activities could not be exhaustive. Care plan development and posttransition follow-up activities may benefit from ongoing development in subsequent work. Continuing to identify and integrate approaches taken at other children’s hospitals will also be informative. For example, some children’s hospitals have introduced adult medicine consultative services to focus on transition, attending children’s hospital safety rounds, and sharing standard care protocols for adult patients still cared for in pediatric settings (eg, stroke and myocardial infarction).¹⁶

In addition, our findings are limited to generalist teams at children’s hospitals and may not be applicable to inpatient subspecialty services. We could not compare differences in respondents versus nonrespondents to determine whether important selection bias exists. Respondent answers could not be verified. Despite our attempt to identify the most informed respondent at each hospital, responses may have differed with other hospital respondents. We used a novel instrument with unknown psychometric properties. Our data provide only the children’s hospital perspective, and perspectives of others (eg, families, primary care pediatricians or internists, subspecialists, etc.) will be valuable to explore in subsequent research. Subsequent research should investigate the relative importance and feasibility of specific inpatient transition activities, ideal timing, as well as the expected outcomes of high-quality inpatient transition. An important question for future work is to identify which patients are most likely to benefit by having inpatient care as part of their transition plan.

Nevertheless, the clinical and health services implications of this facet of transition appear to be substantial.¹⁶ To meet the Maternal and Child Health Bureau (MCHB) core outcome for children with special healthcare needs to receive “the services necessary to make transitions to adult healthcare,”²¹ development, validation, and implementation of effective inpatient-specific transition activities and a set of measurable processes and outcomes are needed. A key direction for the healthcare transitions field, with respect to inpatient care, is to determine the activities most effective at improving relevant patient and family outcomes. Ultimately, we advocate that the transition of inpatient care be integrated into comprehensive approaches to transitional care.

Disclosure: The project described was supported in part by the Clinical and Translational Science Award (CTSA) program, through the National Institutes of Health (NIH) National Center for Advancing Translational Sciences (NCATS), grant UL1TR000427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The project was also supported by the University of Wisconsin Departments of Pediatrics and Medicine. The authors have no financial or other relationships relevant to this article to disclose.

Engaging patients and their caregivers in care improves health outcomes^1-3 and is endorsed by leading healthcare organizations as essential to improving care quality and safety.^4-6 Patient engagement emphasizes that patients, caregivers, and healthcare providers work together to “promote and support active patient and public involvement in health and healthcare and to strengthen their influence on healthcare decisions.”⁷ Patient portals, web-based personal health records linked to electronic health record (EHR) data, are intended to promote engagement by providing patients and their caregivers with timely electronic access to their healthcare information and supporting communication through secure messaging with their healthcare team.⁸ The use of patient portals has also been suggested as a way for patients and/or caregivers to identify and intercept medical errors, thus having the potential to also improve patient safety.^8,9

As a requirement for meaningful use, access to health information through patient portals in the ambulatory setting has increased dramatically.¹⁰ Studies evaluating the use of these patient portals to promote patient-centered care are growing, but evidence supporting their impact on improved health outcomes is currently insufficient.^11-15 Although research and policy focus on the use of patient portals in the ambulatory setting, recent literature suggests that patient portals may be used to share inpatient clinical information to engage patients and their caregivers during their hospitalization.^16-18 Before the widespread use of patient portals in the inpatient setting is endorsed, systematic research is needed to understand optimal portal design requirements, if and how these portals are used, and whether their use provides value to the hospitalized patient and/or caregiver.⁸

Prior literature summarized early findings regarding the use of various technologies designed to engage hospitalized patients.^17,19,20 In this systematic review, we describe the emerging literature examining the design, use, and impact of inpatient portals for hospitalized patients and/or caregivers over the last 10 years. Inpatient portals are defined here as electronic patient portals tethered to EHRs that are designed to provide hospitalized patients and/or caregivers secure access to personalized, inpatient clinical information with the intent of engaging them in their hospital care. After analyzing and summarizing these data, we then identify knowledge gaps and potential future research directions.

Search Strategy, Study Selection, and Analysis

This systematic review included available, peer-reviewed, and grey literature published from January 1, 2006, to August 8, 2017, in PubMed, Web of Science (including the Institute of Electrical and Electronics Engineers Xplore), Cochrane, CINAHLPlus, and Scopus databases. Terms and phrases, including those found in the Medical Subject Heading (MeSH) index, were used to identify studies evaluating (1) patient portals (“health record, personal [MeSH],” “personal health record,” “patient portal,” “inpatient portal,” “ipad,” “tablet,” or “bedside information technology”), (2) engagement (“engagement,” “empowerment,” “participation,” “activation,” or “self-efficacy”), and (3) in the hospital (“inpatient [MeSH],” “hospital [MeSH],” “hospitalized patient [MeSH],” or “unit”). MeSH terms were used when applicable. Based on previous literature, free-text terms were also used when subject headings were not applied consistently, such as with terms related to engagement.^17,21 Studies were excluded if they were not written in English, if they evaluated portals exclusively in the emergency department or ambulatory setting, and/or if they described future study protocols. Studies describing general inpatient technology or evaluating portals used in the hospital but not tethered to inpatient EHR clinical data were also excluded.

By using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines,²² 2 researchers (M.K. and P.H.) completed the literature search and potential article screening. Results were aggregated and studies were screened and excluded from full review based on title and abstract information. Additional studies were included after reference list review. During a full review of included studies, 2 researchers independently extracted data, including the study objective, design, setting, sample, data collection instruments, outcomes, and a description of results. Guided by our study objective, findings were reconciled by consensus and analyzed and described according to the following 3 themes: (1) inpatient portal design, (2) inpatient portal use and usability, and (3) the impact of inpatient portal use on patient or caregiver and healthcare team outcomes as defined by retrieved studies.

The quality of studies was evaluated by the same 2 researchers independently by using the Downs and Black checklist for assessing the methodological quality of randomized and nonrandomized healthcare interventions.²³ Qualitative studies describing the development of portal prototypes and/or portal redesign efforts were excluded from these analyses. Discrepancies were resolved by consensus.Because of the wide variability in study designs, populations, and outcomes, a meta-analysis of pooled data was not performed.

Of the 731 studies identified through database searching and reference review, 36 were included for full-text review and 17 met inclusion criteria (Figure; Table 1). Studies excluded after full-text review described portal use outside of the inpatient setting, portals not linked to hospital EHR clinical data, portals not designed for inpatients, and/or inpatient technology in general. The inpatient portal platforms, hardware used, and functionalities varied within included studies (Table 2). The majority of studies used custom, web-based inpatient portal applications on tablet computers. Most provided information about the patients’ hospital medications, healthcare team, and education about their condition and/or a medical glossary. Many included the patient’s schedule, hospital problem list, discharge information, and a way to keep notes.

There has been a recent increase in inpatient portal study publication, with 9 studies published during or after 2016. Five were conducted in the pediatric setting and all but 1³⁰ with English-speaking participants. Twelve studies were qualitative, many of which were conducted in multiple phases by using semi-structured interviews and/or focus groups to develop or redesign inpatient portals. Of the remaining studies, 3 used a cross-sectional design, 1 used a before and after design without a control group, and 1 was a nonrandomized trial. Studies were rated as having medium-to-high risk of bias because of design flaws (Table 1 in supplementary Appendix). Because many studies were small pilot studies and all were single-centered studies, the generalizability of findings to different healthcare settings or patient populations is limited.

Inpatient Portal Design

Most included studies evaluated patient and/or caregiver information needs to design and/or enhance inpatient portals.^16,24-37 In 1 study, patients described an overall lack of information provided in the hospital and insufficient time to understand and remember information, which, when shared, was often presented by using medical terminology.³⁰ They wanted information to help them understand their daily hospital routine, confirm and compare medications and test results, learn about care, and prepare for discharge. Participants in multiple studies echoed these results, indicating the need for a schedule of upcoming clinical events (eg, medication administration, procedures, imaging), secure and timely clinical information (eg, list of diagnoses and medications, test results), personalized education, a medical glossary, discharge information, and a way to take notes and recognize and communicate with providers.

Patients also requested further information transparency,^34,37 including physicians’ notes, radiology results, operative reports, and billing information, along with general hospital information,¹⁶ meal ordering,³³ and video conferencing.²⁷ ln designing and refining an inpatient medication-tracking tool, participants identified the need for information about medication dosage, frequency, timing, administration method, criticality, alternative medications or forms, and education.^26,36 Patients and/or caregivers also indicated interest in communicating with inpatient providers by using the portal.^{16,27,28,30-37} In 1 study, patients highlighted the need to be involved in care plan development,²⁷ which led to portal refinement to allow for patient-generated data entry, including care goals and a way to communicate real-time concerns and feedback.²⁸

Studies also considered healthcare team perspectives to inform portal design.^{25,26,28,30,35,37} Although information needs usually overlapped, patient and healthcare team priorities differed in some areas. Although patients wanted to “know what was going to happen to them,” nurses in 1 study were more concerned about providing information to protect patients, such as safety and precaution materials.²⁵ Similarly, when designing a medication-tracking tool, patients sought information that helped them understand what to expect, while pharmacists focused on medication safety and providing information that fit their workflow (eg, abstract medication schedules).³⁶

Identified study data raised important portal interface design considerations. Results suggested clinical data should be presented by using simple displays,²⁸ accommodating real-time information. Participants recommended links^16,29 to personalized patient-friendly³⁷ education accessed with minimal steps.²⁶ Interfaces may be personalized for target users, such as patient or proxy and younger or older individuals. For example, older patients reported less familiarity with touch screens, internal keyboards, and handwriting recognition, favoring voice recognition for recording notes.²⁷ This raised questions about how portals can be designed to best maintain patient privacy.²⁵ Interface design, such as navigation, also relied heavily on hardware choice, such as tablet versus mobile phone.²⁸

Inpatient Portal Use and Usability

Most patient and/or caregiver participants in included studies were interested in using an inpatient portal, used it when offered, found it easy to use, useful, and/or were satisfied with it.^16,18,24-37 Most used and liked functionalities that provided healthcare team, test result, and medication information.^22,33,37 In the 1 identified controlled trial,¹⁸ researchers evaluated an inpatient portal given to adult inpatients that included a problem list, schedule, medication list, and healthcare team information. Of the intervention unit patients, 80% used the portal, 76% indicated it was easy to use, and 71% thought it provided useful information. When a portal was given to 239 adult patients and caregivers in another study, 66% sent a total of 291 messages to the healthcare team.³¹ Of these, 153 provided feedback, 76 expressed preferences, and 16 communicated concerns. In a pediatric study, an inpatient portal was given to 296 parents who sent a total of 36 messages and 176 requests.³³ Messages sent included information regarding caregiver needs, questions, updates, and/or positive endorsements of the healthcare team and/or care.

Impact of Inpatient Portal Use

Multiple studies evaluated the impact of inpatient portal use on patient and/or caregiver engagement, empowerment, activation, and/or knowledge, which had mixed results. Most adult patients interviewed in one study had positive experiences using a portal to answer their questions between physician visits and learn about, remember, and engage in care.³⁷ A majority of adult inpatient portal users in another study agreed that portal use helped them feel in control and understand their condition; however, they did not report having improved discharge timing knowledge.²⁹ In a pediatric study, most parent inpatient portal users agreed use improved their ability to monitor, understand, and make decisions about their child’s care.³³ In the controlled trial,¹⁸ a higher percentage of portal intervention patients could identify their physician or role; however, patient activation was not statistically different between intervention and control patients.

Results from included studies also evaluated the impact of portal use on communication. Some suggest inpatient portal use may replace and/or facilitate verbal communication between patients, caregivers, and providers.³⁵ In a pediatric study, 51% of parent portal users reported it gave them the information they needed, reducing the amount of questions they had for their healthcare team.³³ Similarly 43% of 14 adult inpatient portal users in another study thought the portal could replace at least some face-to-face communication.³⁷ Some providers indicated portal use enhanced rounding discussion quality.³⁵ Another study suggested that patient-provider communication via electronic messaging may provide benefits for some patients and not others.³⁷

Multiple studies evaluated patient, caregiver, and/or healthcare team perceptions of the impact of inpatient portal use on detection of errors and patient safety.^29,31,33,35 In adult inpatients, 6% agreed portal use could help them find errors.²⁹ In a pediatric study, 8% reported finding at least 1 medication error by using the portal, and 89% thought use reduced errors in their child’s care.³³ One patient in a qualitative study of adult inpatients cited an example of a dosing error discovered by using the portal.³⁷ Healthcare providers in another study also reported that use facilitated patient error identification.³⁵

Included studies evaluated the potential impact of portal use on patient anxiety, confusion, and/or worry, and the work of healthcare teams. In 1 study, nurses voiced concerns about giving information subject to change or that couldn’t always be achieved because of competing hospital priorities, such as discharge timing.²⁵ They also worried about giving medical information that would create cognitive overload for patients and/or require professional interpretation. Although providers in another study perceived little negative impact on their workflow after portal implementation, they worried about the potential of adding other information to the portal.³⁵ For example, they were concerned that the future release of abnormal test results or sensitive data would lead to confusion and more time spent answering patient questions. Physicians also worried that secure messaging could be overused by patients, would be used to inappropriately express acute concerns, or might adversely affect verbal communication. Providers in 2 studies expressed concerns about potential negative implications of portal use on their work before implementation, which were subsequently reduced after portal implementation.^29,38 Conversely, no parent portal users in another study thought portal information was confusing.³³ One parent participant noted portal use may actually decrease anxiety: “Access to their medical information gives patients and their caregivers perspective and insight into their hospital care and empowers them with knowledge about [what is going on], which reduces anxiety.”³⁷

We identified multiple studies evaluating the design, use, and impact of inpatient patient portals for hospitalized patients and caregivers. Based on the information needs identified by patients and healthcare team participants, multiple key content and design recommendations are suggested, including presenting (1) timely, personalized clinical and educational information in lay terms, (2) the care trajectory, including care plan and patient schedule, and (3) a way to recognize and communicate with the inpatient healthcare team. Design challenges still exist, such as translating medical terminology from EHRs into patient-friendly language, proxy access, and portal integration across transitions. Data from identified studies suggest hospitalized patients and caregivers are interested in and willing to use inpatient portals, but there is less information about the use of each functionality. Evidence supporting the role of inpatient portal use in improving patient and/or caregiver engagement, knowledge, communication, and the quality and safety of care is currently limited. Included studies indicate that healthcare team members had concerns about using portals to share clinical information and communicate electronically in the hospital. The extent to which these concerns translate to demonstrable problems remains to be seen.

Early studies focus on patient and caregiver information needs and portal interface design. Although the necessity for certain core functionalities and design requirements are becoming clear,²⁰ best practices regarding the amount and timing of information released (eg, physician notes, lab results), optimal hardware decisions (eg, large-screen displays, hospital-owned tablets, bring-your-own-device model), and details around secure-messaging implementation in the acute hospital setting are still lacking. Future work is needed to understand optimal patient-provider communication architectures that support improved synchronous and asynchronous messaging and privacy-preserving approaches to the design of these systems to handle patient-generated data as it becomes more commonplace. Although patient participants in these studies were generally satisfied using inpatient portals, many indicated the need for even more transparency, such as the release of results in real time and inclusion of physician notes (even if they could not be fully comprehended).³⁷ As the movement of sharing notes with patients in the ambulatory setting grows,³⁹ it will inevitably extend to the inpatient setting.⁴⁰ Further research is needed to understand the impact of increased transparency on health outcomes, patient anxiety, and inpatient healthcare team workload. Although the majority of studies described the design and/or use of custom portal platforms, EHR vendors are now developing inpatient portals that integrate into preexisting systems (eg, MyChart Bedside, Epic Systems). This will increase the likelihood of broad inpatient portal adoption and may facilitate multicenter trials evaluating the impact of their use.

The next steps will need to focus on the evaluation of specific inpatient portal functionalities and the impact of their use on objective process and outcome measures by using rigorous, experimental study designs. Akin to ambulatory portal research, measures of interest will include patient activation,^41,42 patient and/or caregiver satisfaction,⁴³ care processes (eg, length of stay, readmissions), and patient safety (eg, safety perceptions, adverse drug events, hospital-acquired conditions, and diagnostic errors). More than a mechanism for unidirectional sharing information from providers to the patient, inpatient portals will also provide a platform for the reciprocal exchange of information from the patient to the provider through patient-generated data, such as goal setting and feedback. Patients may play a larger role in reporting hospital satisfaction in real time, reconciling medications, contributing to the treatment plan, and identifying medical errors. As portals are integrated across the care continuum,²⁰ our understanding of their impact may become more clear.

In this review, only 5 studies were conducted in the pediatric hospital setting.^24,32-34,38 With hospitalized children experiencing 3 times more harm from medical errors than adults,⁴⁴ engaging parents in inpatient care to improve safety has become a national priority.⁴⁵ Giving patient portals, or “parent portals,” to parents of hospitalized children may provide a unique opportunity to share healthcare information and promote engagement, a direction for future study. There is also a research gap in evaluating adolescent inpatient portal use. Future portals may be designed to incentivize young children to learn about their hospitalization through games linked to health-related education.

Finally, as patients and caregivers begin using inpatient portals, there will almost certainly be consequences for healthcare teams. Understanding and anticipating human and work system factors influencing inpatient portal adoption and use from the perspectives of both patients and healthcare teams are needed.^46,47 Engaging healthcare team members as valuable stakeholders during implementation and measuring the impact of portal use on their workload is necessary, especially as portal use spreads beyond pilot units. The success of inpatient portals is dependent upon both the positive benefits for patients and their acceptance by healthcare teams.⁴⁸

Limitations exist in conducting a systematic literature review.⁴⁹ The conceptual definition of a portal for hospitalized patients and patient/caregiver engagement is evolving; therefore, our definition may not have captured all relevant studies. We intentionally did not include all inpatient technology, as we were interested in a narrow definition of portals designed for inpatients that provided clinical information from the inpatient EHR. Because of rapid technology changes, we also limited our search to studies published within the last 10 years; prior literature has been described elsewhere.¹⁷ We excluded non-English language studies, limiting our ability to capture the full scope of inpatient portal research. These patients already experience healthcare delivery disparities, widened by the inaccessibility of innovative health information technologies.⁵⁰ Future studies would be enhanced with the inclusion of these participants.

Inpatient portal research is in its infancy but growing rapidly. Studies to date are primarily focused on portal design and have small sample sizes. Early findings suggest that patients and caregivers are, in general, enthusiastic about using inpatient portals. Further research is needed, however, to determine the impact of inpatient portal use on patient engagement and hospital-care quality, safety, and cost.

Disclosure

This work was supported by a Department of Pediatrics Research and Development Grant at the University of Wisconsin School of Medicine and Public Health. This publication was also supported by the Clinical and Translational Science Award program through the National Center for Advancing Translational Sciences, grant UL1TR000427. Dr. Hoonakker’s involvement was also partially supported by the National Science Foundation, grant CMMI 1536987. Funding sources had no involvement in study design, analysis, or interpretation of data. The authors have no conflicts of interest to declare.

Engaging patients and their caregivers in care improves health outcomes^1-3 and is endorsed by leading healthcare organizations as essential to improving care quality and safety.^4-6 Patient engagement emphasizes that patients, caregivers, and healthcare providers work together to “promote and support active patient and public involvement in health and healthcare and to strengthen their influence on healthcare decisions.”⁷ Patient portals, web-based personal health records linked to electronic health record (EHR) data, are intended to promote engagement by providing patients and their caregivers with timely electronic access to their healthcare information and supporting communication through secure messaging with their healthcare team.⁸ The use of patient portals has also been suggested as a way for patients and/or caregivers to identify and intercept medical errors, thus having the potential to also improve patient safety.^8,9

As a requirement for meaningful use, access to health information through patient portals in the ambulatory setting has increased dramatically.¹⁰ Studies evaluating the use of these patient portals to promote patient-centered care are growing, but evidence supporting their impact on improved health outcomes is currently insufficient.^11-15 Although research and policy focus on the use of patient portals in the ambulatory setting, recent literature suggests that patient portals may be used to share inpatient clinical information to engage patients and their caregivers during their hospitalization.^16-18 Before the widespread use of patient portals in the inpatient setting is endorsed, systematic research is needed to understand optimal portal design requirements, if and how these portals are used, and whether their use provides value to the hospitalized patient and/or caregiver.⁸

Prior literature summarized early findings regarding the use of various technologies designed to engage hospitalized patients.^17,19,20 In this systematic review, we describe the emerging literature examining the design, use, and impact of inpatient portals for hospitalized patients and/or caregivers over the last 10 years. Inpatient portals are defined here as electronic patient portals tethered to EHRs that are designed to provide hospitalized patients and/or caregivers secure access to personalized, inpatient clinical information with the intent of engaging them in their hospital care. After analyzing and summarizing these data, we then identify knowledge gaps and potential future research directions.

Search Strategy, Study Selection, and Analysis

This systematic review included available, peer-reviewed, and grey literature published from January 1, 2006, to August 8, 2017, in PubMed, Web of Science (including the Institute of Electrical and Electronics Engineers Xplore), Cochrane, CINAHLPlus, and Scopus databases. Terms and phrases, including those found in the Medical Subject Heading (MeSH) index, were used to identify studies evaluating (1) patient portals (“health record, personal [MeSH],” “personal health record,” “patient portal,” “inpatient portal,” “ipad,” “tablet,” or “bedside information technology”), (2) engagement (“engagement,” “empowerment,” “participation,” “activation,” or “self-efficacy”), and (3) in the hospital (“inpatient [MeSH],” “hospital [MeSH],” “hospitalized patient [MeSH],” or “unit”). MeSH terms were used when applicable. Based on previous literature, free-text terms were also used when subject headings were not applied consistently, such as with terms related to engagement.^17,21 Studies were excluded if they were not written in English, if they evaluated portals exclusively in the emergency department or ambulatory setting, and/or if they described future study protocols. Studies describing general inpatient technology or evaluating portals used in the hospital but not tethered to inpatient EHR clinical data were also excluded.

By using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines,²² 2 researchers (M.K. and P.H.) completed the literature search and potential article screening. Results were aggregated and studies were screened and excluded from full review based on title and abstract information. Additional studies were included after reference list review. During a full review of included studies, 2 researchers independently extracted data, including the study objective, design, setting, sample, data collection instruments, outcomes, and a description of results. Guided by our study objective, findings were reconciled by consensus and analyzed and described according to the following 3 themes: (1) inpatient portal design, (2) inpatient portal use and usability, and (3) the impact of inpatient portal use on patient or caregiver and healthcare team outcomes as defined by retrieved studies.

The quality of studies was evaluated by the same 2 researchers independently by using the Downs and Black checklist for assessing the methodological quality of randomized and nonrandomized healthcare interventions.²³ Qualitative studies describing the development of portal prototypes and/or portal redesign efforts were excluded from these analyses. Discrepancies were resolved by consensus.Because of the wide variability in study designs, populations, and outcomes, a meta-analysis of pooled data was not performed.

Of the 731 studies identified through database searching and reference review, 36 were included for full-text review and 17 met inclusion criteria (Figure; Table 1). Studies excluded after full-text review described portal use outside of the inpatient setting, portals not linked to hospital EHR clinical data, portals not designed for inpatients, and/or inpatient technology in general. The inpatient portal platforms, hardware used, and functionalities varied within included studies (Table 2). The majority of studies used custom, web-based inpatient portal applications on tablet computers. Most provided information about the patients’ hospital medications, healthcare team, and education about their condition and/or a medical glossary. Many included the patient’s schedule, hospital problem list, discharge information, and a way to keep notes.

There has been a recent increase in inpatient portal study publication, with 9 studies published during or after 2016. Five were conducted in the pediatric setting and all but 1³⁰ with English-speaking participants. Twelve studies were qualitative, many of which were conducted in multiple phases by using semi-structured interviews and/or focus groups to develop or redesign inpatient portals. Of the remaining studies, 3 used a cross-sectional design, 1 used a before and after design without a control group, and 1 was a nonrandomized trial. Studies were rated as having medium-to-high risk of bias because of design flaws (Table 1 in supplementary Appendix). Because many studies were small pilot studies and all were single-centered studies, the generalizability of findings to different healthcare settings or patient populations is limited.

Inpatient Portal Design

Most included studies evaluated patient and/or caregiver information needs to design and/or enhance inpatient portals.^16,24-37 In 1 study, patients described an overall lack of information provided in the hospital and insufficient time to understand and remember information, which, when shared, was often presented by using medical terminology.³⁰ They wanted information to help them understand their daily hospital routine, confirm and compare medications and test results, learn about care, and prepare for discharge. Participants in multiple studies echoed these results, indicating the need for a schedule of upcoming clinical events (eg, medication administration, procedures, imaging), secure and timely clinical information (eg, list of diagnoses and medications, test results), personalized education, a medical glossary, discharge information, and a way to take notes and recognize and communicate with providers.

Patients also requested further information transparency,^34,37 including physicians’ notes, radiology results, operative reports, and billing information, along with general hospital information,¹⁶ meal ordering,³³ and video conferencing.²⁷ ln designing and refining an inpatient medication-tracking tool, participants identified the need for information about medication dosage, frequency, timing, administration method, criticality, alternative medications or forms, and education.^26,36 Patients and/or caregivers also indicated interest in communicating with inpatient providers by using the portal.^{16,27,28,30-37} In 1 study, patients highlighted the need to be involved in care plan development,²⁷ which led to portal refinement to allow for patient-generated data entry, including care goals and a way to communicate real-time concerns and feedback.²⁸

Studies also considered healthcare team perspectives to inform portal design.^{25,26,28,30,35,37} Although information needs usually overlapped, patient and healthcare team priorities differed in some areas. Although patients wanted to “know what was going to happen to them,” nurses in 1 study were more concerned about providing information to protect patients, such as safety and precaution materials.²⁵ Similarly, when designing a medication-tracking tool, patients sought information that helped them understand what to expect, while pharmacists focused on medication safety and providing information that fit their workflow (eg, abstract medication schedules).³⁶

Identified study data raised important portal interface design considerations. Results suggested clinical data should be presented by using simple displays,²⁸ accommodating real-time information. Participants recommended links^16,29 to personalized patient-friendly³⁷ education accessed with minimal steps.²⁶ Interfaces may be personalized for target users, such as patient or proxy and younger or older individuals. For example, older patients reported less familiarity with touch screens, internal keyboards, and handwriting recognition, favoring voice recognition for recording notes.²⁷ This raised questions about how portals can be designed to best maintain patient privacy.²⁵ Interface design, such as navigation, also relied heavily on hardware choice, such as tablet versus mobile phone.²⁸

Inpatient Portal Use and Usability

Most patient and/or caregiver participants in included studies were interested in using an inpatient portal, used it when offered, found it easy to use, useful, and/or were satisfied with it.^16,18,24-37 Most used and liked functionalities that provided healthcare team, test result, and medication information.^22,33,37 In the 1 identified controlled trial,¹⁸ researchers evaluated an inpatient portal given to adult inpatients that included a problem list, schedule, medication list, and healthcare team information. Of the intervention unit patients, 80% used the portal, 76% indicated it was easy to use, and 71% thought it provided useful information. When a portal was given to 239 adult patients and caregivers in another study, 66% sent a total of 291 messages to the healthcare team.³¹ Of these, 153 provided feedback, 76 expressed preferences, and 16 communicated concerns. In a pediatric study, an inpatient portal was given to 296 parents who sent a total of 36 messages and 176 requests.³³ Messages sent included information regarding caregiver needs, questions, updates, and/or positive endorsements of the healthcare team and/or care.

Impact of Inpatient Portal Use

Multiple studies evaluated the impact of inpatient portal use on patient and/or caregiver engagement, empowerment, activation, and/or knowledge, which had mixed results. Most adult patients interviewed in one study had positive experiences using a portal to answer their questions between physician visits and learn about, remember, and engage in care.³⁷ A majority of adult inpatient portal users in another study agreed that portal use helped them feel in control and understand their condition; however, they did not report having improved discharge timing knowledge.²⁹ In a pediatric study, most parent inpatient portal users agreed use improved their ability to monitor, understand, and make decisions about their child’s care.³³ In the controlled trial,¹⁸ a higher percentage of portal intervention patients could identify their physician or role; however, patient activation was not statistically different between intervention and control patients.

Results from included studies also evaluated the impact of portal use on communication. Some suggest inpatient portal use may replace and/or facilitate verbal communication between patients, caregivers, and providers.³⁵ In a pediatric study, 51% of parent portal users reported it gave them the information they needed, reducing the amount of questions they had for their healthcare team.³³ Similarly 43% of 14 adult inpatient portal users in another study thought the portal could replace at least some face-to-face communication.³⁷ Some providers indicated portal use enhanced rounding discussion quality.³⁵ Another study suggested that patient-provider communication via electronic messaging may provide benefits for some patients and not others.³⁷

Multiple studies evaluated patient, caregiver, and/or healthcare team perceptions of the impact of inpatient portal use on detection of errors and patient safety.^29,31,33,35 In adult inpatients, 6% agreed portal use could help them find errors.²⁹ In a pediatric study, 8% reported finding at least 1 medication error by using the portal, and 89% thought use reduced errors in their child’s care.³³ One patient in a qualitative study of adult inpatients cited an example of a dosing error discovered by using the portal.³⁷ Healthcare providers in another study also reported that use facilitated patient error identification.³⁵

Included studies evaluated the potential impact of portal use on patient anxiety, confusion, and/or worry, and the work of healthcare teams. In 1 study, nurses voiced concerns about giving information subject to change or that couldn’t always be achieved because of competing hospital priorities, such as discharge timing.²⁵ They also worried about giving medical information that would create cognitive overload for patients and/or require professional interpretation. Although providers in another study perceived little negative impact on their workflow after portal implementation, they worried about the potential of adding other information to the portal.³⁵ For example, they were concerned that the future release of abnormal test results or sensitive data would lead to confusion and more time spent answering patient questions. Physicians also worried that secure messaging could be overused by patients, would be used to inappropriately express acute concerns, or might adversely affect verbal communication. Providers in 2 studies expressed concerns about potential negative implications of portal use on their work before implementation, which were subsequently reduced after portal implementation.^29,38 Conversely, no parent portal users in another study thought portal information was confusing.³³ One parent participant noted portal use may actually decrease anxiety: “Access to their medical information gives patients and their caregivers perspective and insight into their hospital care and empowers them with knowledge about [what is going on], which reduces anxiety.”³⁷

We identified multiple studies evaluating the design, use, and impact of inpatient patient portals for hospitalized patients and caregivers. Based on the information needs identified by patients and healthcare team participants, multiple key content and design recommendations are suggested, including presenting (1) timely, personalized clinical and educational information in lay terms, (2) the care trajectory, including care plan and patient schedule, and (3) a way to recognize and communicate with the inpatient healthcare team. Design challenges still exist, such as translating medical terminology from EHRs into patient-friendly language, proxy access, and portal integration across transitions. Data from identified studies suggest hospitalized patients and caregivers are interested in and willing to use inpatient portals, but there is less information about the use of each functionality. Evidence supporting the role of inpatient portal use in improving patient and/or caregiver engagement, knowledge, communication, and the quality and safety of care is currently limited. Included studies indicate that healthcare team members had concerns about using portals to share clinical information and communicate electronically in the hospital. The extent to which these concerns translate to demonstrable problems remains to be seen.

Early studies focus on patient and caregiver information needs and portal interface design. Although the necessity for certain core functionalities and design requirements are becoming clear,²⁰ best practices regarding the amount and timing of information released (eg, physician notes, lab results), optimal hardware decisions (eg, large-screen displays, hospital-owned tablets, bring-your-own-device model), and details around secure-messaging implementation in the acute hospital setting are still lacking. Future work is needed to understand optimal patient-provider communication architectures that support improved synchronous and asynchronous messaging and privacy-preserving approaches to the design of these systems to handle patient-generated data as it becomes more commonplace. Although patient participants in these studies were generally satisfied using inpatient portals, many indicated the need for even more transparency, such as the release of results in real time and inclusion of physician notes (even if they could not be fully comprehended).³⁷ As the movement of sharing notes with patients in the ambulatory setting grows,³⁹ it will inevitably extend to the inpatient setting.⁴⁰ Further research is needed to understand the impact of increased transparency on health outcomes, patient anxiety, and inpatient healthcare team workload. Although the majority of studies described the design and/or use of custom portal platforms, EHR vendors are now developing inpatient portals that integrate into preexisting systems (eg, MyChart Bedside, Epic Systems). This will increase the likelihood of broad inpatient portal adoption and may facilitate multicenter trials evaluating the impact of their use.

The next steps will need to focus on the evaluation of specific inpatient portal functionalities and the impact of their use on objective process and outcome measures by using rigorous, experimental study designs. Akin to ambulatory portal research, measures of interest will include patient activation,^41,42 patient and/or caregiver satisfaction,⁴³ care processes (eg, length of stay, readmissions), and patient safety (eg, safety perceptions, adverse drug events, hospital-acquired conditions, and diagnostic errors). More than a mechanism for unidirectional sharing information from providers to the patient, inpatient portals will also provide a platform for the reciprocal exchange of information from the patient to the provider through patient-generated data, such as goal setting and feedback. Patients may play a larger role in reporting hospital satisfaction in real time, reconciling medications, contributing to the treatment plan, and identifying medical errors. As portals are integrated across the care continuum,²⁰ our understanding of their impact may become more clear.

In this review, only 5 studies were conducted in the pediatric hospital setting.^24,32-34,38 With hospitalized children experiencing 3 times more harm from medical errors than adults,⁴⁴ engaging parents in inpatient care to improve safety has become a national priority.⁴⁵ Giving patient portals, or “parent portals,” to parents of hospitalized children may provide a unique opportunity to share healthcare information and promote engagement, a direction for future study. There is also a research gap in evaluating adolescent inpatient portal use. Future portals may be designed to incentivize young children to learn about their hospitalization through games linked to health-related education.

Finally, as patients and caregivers begin using inpatient portals, there will almost certainly be consequences for healthcare teams. Understanding and anticipating human and work system factors influencing inpatient portal adoption and use from the perspectives of both patients and healthcare teams are needed.^46,47 Engaging healthcare team members as valuable stakeholders during implementation and measuring the impact of portal use on their workload is necessary, especially as portal use spreads beyond pilot units. The success of inpatient portals is dependent upon both the positive benefits for patients and their acceptance by healthcare teams.⁴⁸

Limitations exist in conducting a systematic literature review.⁴⁹ The conceptual definition of a portal for hospitalized patients and patient/caregiver engagement is evolving; therefore, our definition may not have captured all relevant studies. We intentionally did not include all inpatient technology, as we were interested in a narrow definition of portals designed for inpatients that provided clinical information from the inpatient EHR. Because of rapid technology changes, we also limited our search to studies published within the last 10 years; prior literature has been described elsewhere.¹⁷ We excluded non-English language studies, limiting our ability to capture the full scope of inpatient portal research. These patients already experience healthcare delivery disparities, widened by the inaccessibility of innovative health information technologies.⁵⁰ Future studies would be enhanced with the inclusion of these participants.

Inpatient portal research is in its infancy but growing rapidly. Studies to date are primarily focused on portal design and have small sample sizes. Early findings suggest that patients and caregivers are, in general, enthusiastic about using inpatient portals. Further research is needed, however, to determine the impact of inpatient portal use on patient engagement and hospital-care quality, safety, and cost.

Disclosure

This work was supported by a Department of Pediatrics Research and Development Grant at the University of Wisconsin School of Medicine and Public Health. This publication was also supported by the Clinical and Translational Science Award program through the National Center for Advancing Translational Sciences, grant UL1TR000427. Dr. Hoonakker’s involvement was also partially supported by the National Science Foundation, grant CMMI 1536987. Funding sources had no involvement in study design, analysis, or interpretation of data. The authors have no conflicts of interest to declare.

Engaging patients and their caregivers in care improves health outcomes^1-3 and is endorsed by leading healthcare organizations as essential to improving care quality and safety.^4-6 Patient engagement emphasizes that patients, caregivers, and healthcare providers work together to “promote and support active patient and public involvement in health and healthcare and to strengthen their influence on healthcare decisions.”⁷ Patient portals, web-based personal health records linked to electronic health record (EHR) data, are intended to promote engagement by providing patients and their caregivers with timely electronic access to their healthcare information and supporting communication through secure messaging with their healthcare team.⁸ The use of patient portals has also been suggested as a way for patients and/or caregivers to identify and intercept medical errors, thus having the potential to also improve patient safety.^8,9

As a requirement for meaningful use, access to health information through patient portals in the ambulatory setting has increased dramatically.¹⁰ Studies evaluating the use of these patient portals to promote patient-centered care are growing, but evidence supporting their impact on improved health outcomes is currently insufficient.^11-15 Although research and policy focus on the use of patient portals in the ambulatory setting, recent literature suggests that patient portals may be used to share inpatient clinical information to engage patients and their caregivers during their hospitalization.^16-18 Before the widespread use of patient portals in the inpatient setting is endorsed, systematic research is needed to understand optimal portal design requirements, if and how these portals are used, and whether their use provides value to the hospitalized patient and/or caregiver.⁸

Prior literature summarized early findings regarding the use of various technologies designed to engage hospitalized patients.^17,19,20 In this systematic review, we describe the emerging literature examining the design, use, and impact of inpatient portals for hospitalized patients and/or caregivers over the last 10 years. Inpatient portals are defined here as electronic patient portals tethered to EHRs that are designed to provide hospitalized patients and/or caregivers secure access to personalized, inpatient clinical information with the intent of engaging them in their hospital care. After analyzing and summarizing these data, we then identify knowledge gaps and potential future research directions.

Search Strategy, Study Selection, and Analysis

This systematic review included available, peer-reviewed, and grey literature published from January 1, 2006, to August 8, 2017, in PubMed, Web of Science (including the Institute of Electrical and Electronics Engineers Xplore), Cochrane, CINAHLPlus, and Scopus databases. Terms and phrases, including those found in the Medical Subject Heading (MeSH) index, were used to identify studies evaluating (1) patient portals (“health record, personal [MeSH],” “personal health record,” “patient portal,” “inpatient portal,” “ipad,” “tablet,” or “bedside information technology”), (2) engagement (“engagement,” “empowerment,” “participation,” “activation,” or “self-efficacy”), and (3) in the hospital (“inpatient [MeSH],” “hospital [MeSH],” “hospitalized patient [MeSH],” or “unit”). MeSH terms were used when applicable. Based on previous literature, free-text terms were also used when subject headings were not applied consistently, such as with terms related to engagement.^17,21 Studies were excluded if they were not written in English, if they evaluated portals exclusively in the emergency department or ambulatory setting, and/or if they described future study protocols. Studies describing general inpatient technology or evaluating portals used in the hospital but not tethered to inpatient EHR clinical data were also excluded.

By using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines,²² 2 researchers (M.K. and P.H.) completed the literature search and potential article screening. Results were aggregated and studies were screened and excluded from full review based on title and abstract information. Additional studies were included after reference list review. During a full review of included studies, 2 researchers independently extracted data, including the study objective, design, setting, sample, data collection instruments, outcomes, and a description of results. Guided by our study objective, findings were reconciled by consensus and analyzed and described according to the following 3 themes: (1) inpatient portal design, (2) inpatient portal use and usability, and (3) the impact of inpatient portal use on patient or caregiver and healthcare team outcomes as defined by retrieved studies.

The quality of studies was evaluated by the same 2 researchers independently by using the Downs and Black checklist for assessing the methodological quality of randomized and nonrandomized healthcare interventions.²³ Qualitative studies describing the development of portal prototypes and/or portal redesign efforts were excluded from these analyses. Discrepancies were resolved by consensus.Because of the wide variability in study designs, populations, and outcomes, a meta-analysis of pooled data was not performed.

Of the 731 studies identified through database searching and reference review, 36 were included for full-text review and 17 met inclusion criteria (Figure; Table 1). Studies excluded after full-text review described portal use outside of the inpatient setting, portals not linked to hospital EHR clinical data, portals not designed for inpatients, and/or inpatient technology in general. The inpatient portal platforms, hardware used, and functionalities varied within included studies (Table 2). The majority of studies used custom, web-based inpatient portal applications on tablet computers. Most provided information about the patients’ hospital medications, healthcare team, and education about their condition and/or a medical glossary. Many included the patient’s schedule, hospital problem list, discharge information, and a way to keep notes.

There has been a recent increase in inpatient portal study publication, with 9 studies published during or after 2016. Five were conducted in the pediatric setting and all but 1³⁰ with English-speaking participants. Twelve studies were qualitative, many of which were conducted in multiple phases by using semi-structured interviews and/or focus groups to develop or redesign inpatient portals. Of the remaining studies, 3 used a cross-sectional design, 1 used a before and after design without a control group, and 1 was a nonrandomized trial. Studies were rated as having medium-to-high risk of bias because of design flaws (Table 1 in supplementary Appendix). Because many studies were small pilot studies and all were single-centered studies, the generalizability of findings to different healthcare settings or patient populations is limited.

Inpatient Portal Design

Most included studies evaluated patient and/or caregiver information needs to design and/or enhance inpatient portals.^16,24-37 In 1 study, patients described an overall lack of information provided in the hospital and insufficient time to understand and remember information, which, when shared, was often presented by using medical terminology.³⁰ They wanted information to help them understand their daily hospital routine, confirm and compare medications and test results, learn about care, and prepare for discharge. Participants in multiple studies echoed these results, indicating the need for a schedule of upcoming clinical events (eg, medication administration, procedures, imaging), secure and timely clinical information (eg, list of diagnoses and medications, test results), personalized education, a medical glossary, discharge information, and a way to take notes and recognize and communicate with providers.

Patients also requested further information transparency,^34,37 including physicians’ notes, radiology results, operative reports, and billing information, along with general hospital information,¹⁶ meal ordering,³³ and video conferencing.²⁷ ln designing and refining an inpatient medication-tracking tool, participants identified the need for information about medication dosage, frequency, timing, administration method, criticality, alternative medications or forms, and education.^26,36 Patients and/or caregivers also indicated interest in communicating with inpatient providers by using the portal.^{16,27,28,30-37} In 1 study, patients highlighted the need to be involved in care plan development,²⁷ which led to portal refinement to allow for patient-generated data entry, including care goals and a way to communicate real-time concerns and feedback.²⁸

Studies also considered healthcare team perspectives to inform portal design.^{25,26,28,30,35,37} Although information needs usually overlapped, patient and healthcare team priorities differed in some areas. Although patients wanted to “know what was going to happen to them,” nurses in 1 study were more concerned about providing information to protect patients, such as safety and precaution materials.²⁵ Similarly, when designing a medication-tracking tool, patients sought information that helped them understand what to expect, while pharmacists focused on medication safety and providing information that fit their workflow (eg, abstract medication schedules).³⁶

Identified study data raised important portal interface design considerations. Results suggested clinical data should be presented by using simple displays,²⁸ accommodating real-time information. Participants recommended links^16,29 to personalized patient-friendly³⁷ education accessed with minimal steps.²⁶ Interfaces may be personalized for target users, such as patient or proxy and younger or older individuals. For example, older patients reported less familiarity with touch screens, internal keyboards, and handwriting recognition, favoring voice recognition for recording notes.²⁷ This raised questions about how portals can be designed to best maintain patient privacy.²⁵ Interface design, such as navigation, also relied heavily on hardware choice, such as tablet versus mobile phone.²⁸

Inpatient Portal Use and Usability

Most patient and/or caregiver participants in included studies were interested in using an inpatient portal, used it when offered, found it easy to use, useful, and/or were satisfied with it.^16,18,24-37 Most used and liked functionalities that provided healthcare team, test result, and medication information.^22,33,37 In the 1 identified controlled trial,¹⁸ researchers evaluated an inpatient portal given to adult inpatients that included a problem list, schedule, medication list, and healthcare team information. Of the intervention unit patients, 80% used the portal, 76% indicated it was easy to use, and 71% thought it provided useful information. When a portal was given to 239 adult patients and caregivers in another study, 66% sent a total of 291 messages to the healthcare team.³¹ Of these, 153 provided feedback, 76 expressed preferences, and 16 communicated concerns. In a pediatric study, an inpatient portal was given to 296 parents who sent a total of 36 messages and 176 requests.³³ Messages sent included information regarding caregiver needs, questions, updates, and/or positive endorsements of the healthcare team and/or care.

Impact of Inpatient Portal Use

Multiple studies evaluated the impact of inpatient portal use on patient and/or caregiver engagement, empowerment, activation, and/or knowledge, which had mixed results. Most adult patients interviewed in one study had positive experiences using a portal to answer their questions between physician visits and learn about, remember, and engage in care.³⁷ A majority of adult inpatient portal users in another study agreed that portal use helped them feel in control and understand their condition; however, they did not report having improved discharge timing knowledge.²⁹ In a pediatric study, most parent inpatient portal users agreed use improved their ability to monitor, understand, and make decisions about their child’s care.³³ In the controlled trial,¹⁸ a higher percentage of portal intervention patients could identify their physician or role; however, patient activation was not statistically different between intervention and control patients.

Results from included studies also evaluated the impact of portal use on communication. Some suggest inpatient portal use may replace and/or facilitate verbal communication between patients, caregivers, and providers.³⁵ In a pediatric study, 51% of parent portal users reported it gave them the information they needed, reducing the amount of questions they had for their healthcare team.³³ Similarly 43% of 14 adult inpatient portal users in another study thought the portal could replace at least some face-to-face communication.³⁷ Some providers indicated portal use enhanced rounding discussion quality.³⁵ Another study suggested that patient-provider communication via electronic messaging may provide benefits for some patients and not others.³⁷

Multiple studies evaluated patient, caregiver, and/or healthcare team perceptions of the impact of inpatient portal use on detection of errors and patient safety.^29,31,33,35 In adult inpatients, 6% agreed portal use could help them find errors.²⁹ In a pediatric study, 8% reported finding at least 1 medication error by using the portal, and 89% thought use reduced errors in their child’s care.³³ One patient in a qualitative study of adult inpatients cited an example of a dosing error discovered by using the portal.³⁷ Healthcare providers in another study also reported that use facilitated patient error identification.³⁵

Included studies evaluated the potential impact of portal use on patient anxiety, confusion, and/or worry, and the work of healthcare teams. In 1 study, nurses voiced concerns about giving information subject to change or that couldn’t always be achieved because of competing hospital priorities, such as discharge timing.²⁵ They also worried about giving medical information that would create cognitive overload for patients and/or require professional interpretation. Although providers in another study perceived little negative impact on their workflow after portal implementation, they worried about the potential of adding other information to the portal.³⁵ For example, they were concerned that the future release of abnormal test results or sensitive data would lead to confusion and more time spent answering patient questions. Physicians also worried that secure messaging could be overused by patients, would be used to inappropriately express acute concerns, or might adversely affect verbal communication. Providers in 2 studies expressed concerns about potential negative implications of portal use on their work before implementation, which were subsequently reduced after portal implementation.^29,38 Conversely, no parent portal users in another study thought portal information was confusing.³³ One parent participant noted portal use may actually decrease anxiety: “Access to their medical information gives patients and their caregivers perspective and insight into their hospital care and empowers them with knowledge about [what is going on], which reduces anxiety.”³⁷

We identified multiple studies evaluating the design, use, and impact of inpatient patient portals for hospitalized patients and caregivers. Based on the information needs identified by patients and healthcare team participants, multiple key content and design recommendations are suggested, including presenting (1) timely, personalized clinical and educational information in lay terms, (2) the care trajectory, including care plan and patient schedule, and (3) a way to recognize and communicate with the inpatient healthcare team. Design challenges still exist, such as translating medical terminology from EHRs into patient-friendly language, proxy access, and portal integration across transitions. Data from identified studies suggest hospitalized patients and caregivers are interested in and willing to use inpatient portals, but there is less information about the use of each functionality. Evidence supporting the role of inpatient portal use in improving patient and/or caregiver engagement, knowledge, communication, and the quality and safety of care is currently limited. Included studies indicate that healthcare team members had concerns about using portals to share clinical information and communicate electronically in the hospital. The extent to which these concerns translate to demonstrable problems remains to be seen.

Early studies focus on patient and caregiver information needs and portal interface design. Although the necessity for certain core functionalities and design requirements are becoming clear,²⁰ best practices regarding the amount and timing of information released (eg, physician notes, lab results), optimal hardware decisions (eg, large-screen displays, hospital-owned tablets, bring-your-own-device model), and details around secure-messaging implementation in the acute hospital setting are still lacking. Future work is needed to understand optimal patient-provider communication architectures that support improved synchronous and asynchronous messaging and privacy-preserving approaches to the design of these systems to handle patient-generated data as it becomes more commonplace. Although patient participants in these studies were generally satisfied using inpatient portals, many indicated the need for even more transparency, such as the release of results in real time and inclusion of physician notes (even if they could not be fully comprehended).³⁷ As the movement of sharing notes with patients in the ambulatory setting grows,³⁹ it will inevitably extend to the inpatient setting.⁴⁰ Further research is needed to understand the impact of increased transparency on health outcomes, patient anxiety, and inpatient healthcare team workload. Although the majority of studies described the design and/or use of custom portal platforms, EHR vendors are now developing inpatient portals that integrate into preexisting systems (eg, MyChart Bedside, Epic Systems). This will increase the likelihood of broad inpatient portal adoption and may facilitate multicenter trials evaluating the impact of their use.

The next steps will need to focus on the evaluation of specific inpatient portal functionalities and the impact of their use on objective process and outcome measures by using rigorous, experimental study designs. Akin to ambulatory portal research, measures of interest will include patient activation,^41,42 patient and/or caregiver satisfaction,⁴³ care processes (eg, length of stay, readmissions), and patient safety (eg, safety perceptions, adverse drug events, hospital-acquired conditions, and diagnostic errors). More than a mechanism for unidirectional sharing information from providers to the patient, inpatient portals will also provide a platform for the reciprocal exchange of information from the patient to the provider through patient-generated data, such as goal setting and feedback. Patients may play a larger role in reporting hospital satisfaction in real time, reconciling medications, contributing to the treatment plan, and identifying medical errors. As portals are integrated across the care continuum,²⁰ our understanding of their impact may become more clear.

In this review, only 5 studies were conducted in the pediatric hospital setting.^24,32-34,38 With hospitalized children experiencing 3 times more harm from medical errors than adults,⁴⁴ engaging parents in inpatient care to improve safety has become a national priority.⁴⁵ Giving patient portals, or “parent portals,” to parents of hospitalized children may provide a unique opportunity to share healthcare information and promote engagement, a direction for future study. There is also a research gap in evaluating adolescent inpatient portal use. Future portals may be designed to incentivize young children to learn about their hospitalization through games linked to health-related education.

Finally, as patients and caregivers begin using inpatient portals, there will almost certainly be consequences for healthcare teams. Understanding and anticipating human and work system factors influencing inpatient portal adoption and use from the perspectives of both patients and healthcare teams are needed.^46,47 Engaging healthcare team members as valuable stakeholders during implementation and measuring the impact of portal use on their workload is necessary, especially as portal use spreads beyond pilot units. The success of inpatient portals is dependent upon both the positive benefits for patients and their acceptance by healthcare teams.⁴⁸

Limitations exist in conducting a systematic literature review.⁴⁹ The conceptual definition of a portal for hospitalized patients and patient/caregiver engagement is evolving; therefore, our definition may not have captured all relevant studies. We intentionally did not include all inpatient technology, as we were interested in a narrow definition of portals designed for inpatients that provided clinical information from the inpatient EHR. Because of rapid technology changes, we also limited our search to studies published within the last 10 years; prior literature has been described elsewhere.¹⁷ We excluded non-English language studies, limiting our ability to capture the full scope of inpatient portal research. These patients already experience healthcare delivery disparities, widened by the inaccessibility of innovative health information technologies.⁵⁰ Future studies would be enhanced with the inclusion of these participants.

Inpatient portal research is in its infancy but growing rapidly. Studies to date are primarily focused on portal design and have small sample sizes. Early findings suggest that patients and caregivers are, in general, enthusiastic about using inpatient portals. Further research is needed, however, to determine the impact of inpatient portal use on patient engagement and hospital-care quality, safety, and cost.

Disclosure

This work was supported by a Department of Pediatrics Research and Development Grant at the University of Wisconsin School of Medicine and Public Health. This publication was also supported by the Clinical and Translational Science Award program through the National Center for Advancing Translational Sciences, grant UL1TR000427. Dr. Hoonakker’s involvement was also partially supported by the National Science Foundation, grant CMMI 1536987. Funding sources had no involvement in study design, analysis, or interpretation of data. The authors have no conflicts of interest to declare.

BACKGROUND AND OBJECTIVES: Adherence to American Academy of Pediatrics (AAP) bronchiolitis clinical practice guideline recommendations improved significantly through the AAP’s multi-institutional collaborative the Bronchiolitis Quality Improvement Project (BQIP). We assessed sustainability of improvements at participating institutions for 1 year following completion of the collaborative.

METHODS: Twenty-one multidisciplinary hospital-based teams provided monthly data for key inpatient bronchiolitis measures during baseline and intervention bronchiolitis seasons. Nine sites provided data in the season following completion of the collaborative. Encounters included children younger than 24 months who were hospitalized for bronchiolitis without comorbid chronic illness, prematurity, or intensive care. Changes between baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with site-specific random effects. Differences between hospital characteristics, baseline performance, and initial improvement among sites that did and did not participate in the sustainability season were compared.

RESULTS: A total of 2,275 discharges were reviewed, comprising 995 baseline, 877 intervention, and 403 sustainability-season encounters. Improvements in all key bronchiolitis quality measures achieved during the intervention season were maintained during the sustainability season, and orders for intermittent pulse oximetry increased from 40.6% (95% confidence interval, 22.8-61.1) to 79.2% (95% CI, 58.0-91.3). Sites that did and did not participate in the sustainability season had similar characteristics.

DISCUSSION: BQIP participating sites maintained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. This approach, which provided an evidence-based best-practice toolkit while building the quality-improvement capacity of local interdisciplinary teams, may support performance gains that persist beyond the active phase of the collaborative.
 

Also in JHM this month

The effect of an inpatient smoking cessation treatment program on hospital readmissions and length of stayAUTHORS: Eline M. van den Broek-Altenburg, MS, MA, Adam J. Atherly, PhD

Treatment trends and outcomes in healthcare-associated pneumoniaAUTHORS: Sarah Haessler, MD; Tara Lagu, MD, MPH; Peter K. Lindenauer, MD, MSc; Daniel J. Skiest, MD; Aruna Priya, MA, MSc; Penelope S. Pekow, PhD; Marya D. Zilberberg, MD, MPH; Thomas L. Higgins, MD, MBA; Michael B. Rothberg, MD, MPH

What’s the purpose of rounds? A qualitative study examining the perceptions of faculty and studentsAUTHORS: Oliver Hulland; Jeanne Farnan, MD, MHPE; Raphael Rabinowitz; Lisa Kearns, MD, MS; Michele Long, MD; Bradley Monash, MD; Priti Bhansali, MD; H. Barrett Fromme, MD, MHPE

Association between anemia and fatigue in hospitalized patients: does the measure of anemia matter?AUTHORS: Micah T. Prochaska, MD, MS; Richard Newcomb, BA; Graham Block, BA; Brian Park, BA; David O. Meltzer MD, PhD

Helping seniors plan for posthospital discharge needs before a hospitalization occurs: Results from the randomized control trial of planyourlifespan.orgAUTHORS: Lee A. Lindquist, MD, MPH, MBA; Vanessa Ramirez-Zohfeld, MPH; Priya D. Sunkara, MA; Chris Forcucci, RN, BSN; Dianne S. Campbell, BS; Phyllis Mitzen, MA; Jody D. Ciolino, PhD; Gayle Kricke, MSW; Anne Seltzer, LSW; Ana V. Ramirez, BA; Kenzie A. Cameron, PhD, MPH

BACKGROUND AND OBJECTIVES: Adherence to American Academy of Pediatrics (AAP) bronchiolitis clinical practice guideline recommendations improved significantly through the AAP’s multi-institutional collaborative the Bronchiolitis Quality Improvement Project (BQIP). We assessed sustainability of improvements at participating institutions for 1 year following completion of the collaborative.

METHODS: Twenty-one multidisciplinary hospital-based teams provided monthly data for key inpatient bronchiolitis measures during baseline and intervention bronchiolitis seasons. Nine sites provided data in the season following completion of the collaborative. Encounters included children younger than 24 months who were hospitalized for bronchiolitis without comorbid chronic illness, prematurity, or intensive care. Changes between baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with site-specific random effects. Differences between hospital characteristics, baseline performance, and initial improvement among sites that did and did not participate in the sustainability season were compared.

RESULTS: A total of 2,275 discharges were reviewed, comprising 995 baseline, 877 intervention, and 403 sustainability-season encounters. Improvements in all key bronchiolitis quality measures achieved during the intervention season were maintained during the sustainability season, and orders for intermittent pulse oximetry increased from 40.6% (95% confidence interval, 22.8-61.1) to 79.2% (95% CI, 58.0-91.3). Sites that did and did not participate in the sustainability season had similar characteristics.

DISCUSSION: BQIP participating sites maintained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. This approach, which provided an evidence-based best-practice toolkit while building the quality-improvement capacity of local interdisciplinary teams, may support performance gains that persist beyond the active phase of the collaborative.
 

Also in JHM this month

The effect of an inpatient smoking cessation treatment program on hospital readmissions and length of stayAUTHORS: Eline M. van den Broek-Altenburg, MS, MA, Adam J. Atherly, PhD

Treatment trends and outcomes in healthcare-associated pneumoniaAUTHORS: Sarah Haessler, MD; Tara Lagu, MD, MPH; Peter K. Lindenauer, MD, MSc; Daniel J. Skiest, MD; Aruna Priya, MA, MSc; Penelope S. Pekow, PhD; Marya D. Zilberberg, MD, MPH; Thomas L. Higgins, MD, MBA; Michael B. Rothberg, MD, MPH

What’s the purpose of rounds? A qualitative study examining the perceptions of faculty and studentsAUTHORS: Oliver Hulland; Jeanne Farnan, MD, MHPE; Raphael Rabinowitz; Lisa Kearns, MD, MS; Michele Long, MD; Bradley Monash, MD; Priti Bhansali, MD; H. Barrett Fromme, MD, MHPE

Association between anemia and fatigue in hospitalized patients: does the measure of anemia matter?AUTHORS: Micah T. Prochaska, MD, MS; Richard Newcomb, BA; Graham Block, BA; Brian Park, BA; David O. Meltzer MD, PhD

Helping seniors plan for posthospital discharge needs before a hospitalization occurs: Results from the randomized control trial of planyourlifespan.orgAUTHORS: Lee A. Lindquist, MD, MPH, MBA; Vanessa Ramirez-Zohfeld, MPH; Priya D. Sunkara, MA; Chris Forcucci, RN, BSN; Dianne S. Campbell, BS; Phyllis Mitzen, MA; Jody D. Ciolino, PhD; Gayle Kricke, MSW; Anne Seltzer, LSW; Ana V. Ramirez, BA; Kenzie A. Cameron, PhD, MPH

BACKGROUND AND OBJECTIVES: Adherence to American Academy of Pediatrics (AAP) bronchiolitis clinical practice guideline recommendations improved significantly through the AAP’s multi-institutional collaborative the Bronchiolitis Quality Improvement Project (BQIP). We assessed sustainability of improvements at participating institutions for 1 year following completion of the collaborative.

METHODS: Twenty-one multidisciplinary hospital-based teams provided monthly data for key inpatient bronchiolitis measures during baseline and intervention bronchiolitis seasons. Nine sites provided data in the season following completion of the collaborative. Encounters included children younger than 24 months who were hospitalized for bronchiolitis without comorbid chronic illness, prematurity, or intensive care. Changes between baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with site-specific random effects. Differences between hospital characteristics, baseline performance, and initial improvement among sites that did and did not participate in the sustainability season were compared.

RESULTS: A total of 2,275 discharges were reviewed, comprising 995 baseline, 877 intervention, and 403 sustainability-season encounters. Improvements in all key bronchiolitis quality measures achieved during the intervention season were maintained during the sustainability season, and orders for intermittent pulse oximetry increased from 40.6% (95% confidence interval, 22.8-61.1) to 79.2% (95% CI, 58.0-91.3). Sites that did and did not participate in the sustainability season had similar characteristics.

DISCUSSION: BQIP participating sites maintained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. This approach, which provided an evidence-based best-practice toolkit while building the quality-improvement capacity of local interdisciplinary teams, may support performance gains that persist beyond the active phase of the collaborative.
 

Also in JHM this month

The effect of an inpatient smoking cessation treatment program on hospital readmissions and length of stayAUTHORS: Eline M. van den Broek-Altenburg, MS, MA, Adam J. Atherly, PhD

Treatment trends and outcomes in healthcare-associated pneumoniaAUTHORS: Sarah Haessler, MD; Tara Lagu, MD, MPH; Peter K. Lindenauer, MD, MSc; Daniel J. Skiest, MD; Aruna Priya, MA, MSc; Penelope S. Pekow, PhD; Marya D. Zilberberg, MD, MPH; Thomas L. Higgins, MD, MBA; Michael B. Rothberg, MD, MPH

What’s the purpose of rounds? A qualitative study examining the perceptions of faculty and studentsAUTHORS: Oliver Hulland; Jeanne Farnan, MD, MHPE; Raphael Rabinowitz; Lisa Kearns, MD, MS; Michele Long, MD; Bradley Monash, MD; Priti Bhansali, MD; H. Barrett Fromme, MD, MHPE

Association between anemia and fatigue in hospitalized patients: does the measure of anemia matter?AUTHORS: Micah T. Prochaska, MD, MS; Richard Newcomb, BA; Graham Block, BA; Brian Park, BA; David O. Meltzer MD, PhD

Helping seniors plan for posthospital discharge needs before a hospitalization occurs: Results from the randomized control trial of planyourlifespan.orgAUTHORS: Lee A. Lindquist, MD, MPH, MBA; Vanessa Ramirez-Zohfeld, MPH; Priya D. Sunkara, MA; Chris Forcucci, RN, BSN; Dianne S. Campbell, BS; Phyllis Mitzen, MA; Jody D. Ciolino, PhD; Gayle Kricke, MSW; Anne Seltzer, LSW; Ana V. Ramirez, BA; Kenzie A. Cameron, PhD, MPH

Acute viral bronchiolitis is the most common cause of hospitalization for children less than 1 year of age.¹ Overuse of ineffective therapies has persisted despite the existence of the evidence-based American Academy of Pediatrics (AAP) clinical practice guideline (CPG), which recommends primarily supportive care.^2-8 Adherence to the AAP CPG recommendations for management of bronchiolitis improved significantly through the AAP’s Bronchiolitis Quality Improvement Project (BQIP), a 12-month, multiinstitutional collaborative of community and free-standing children’s hospitals.⁹ This subsequent study investigates if these improvements were sustained after completion of the formal 12-month project.

Published multiinstitutional bronchiolitis quality improvement (QI) work is limited to 1 study⁵ that describes the results of a single intervention season at academic medical centers. Multiyear bronchiolitis QI projects are limited to single-center studies, and results have been mixed.^5,6,8,10-13 One study¹¹ observed continued improvement in bronchodilator use in subsequent seasons, whereas a second study¹⁰ observed a return to baseline bronchodilator use in the following season. Mittal⁶ observed inconsistent improvements in key bronchiolitis measures during postintervention seasons.

Our specific aim was to assess the sustainability of improvements in bronchiolitis management at participating institutions 1 year following completion of the AAP BQIP collaborative.⁹ Because no studies demonstrate the most effective way to support long-term improvement through a QI collaborative, we hypothesized that the initial collaborative activities, which were designed to build the capacity of local interdisciplinary teams while providing standardized evidence-based care pathways, would lead to performance in the subsequent season at levels similar to or better than those observed during the active phase of the collaborative, without additional project interventions.

Study Design and Setting

This was a follow-up study of the AAP Quality Improvement Innovation Networks project entitled “A Quality Collaborative for Improving Hospital Compliance with the AAP Bronchiolitis Guideline” (BQIP).⁹ The AAP Institutional Review Board approved this project.

Twenty-one multidisciplinary, hospital-based teams participated in the BQIP collaborative and provided monthly data during the January through March bronchiolitis season. Teams submitted 2013 baseline data and 2014 intervention data. Nine sites provided 2015 sustainability data following the completion of the collaborative.

Participants

Hospital encounters with a primary diagnosis of acute viral bronchiolitis were eligible for inclusion among patients from 1 month to 2 years of age. Encounters were excluded for prematurity (<35 weeks gestational age), congenital heart disease, bronchopulmonary dysplasia, genetic, congenital or neuromuscular abnormalities, and pediatric intensive-care admission.

Data Collection

Hospital characteristics were collected, including hospital type (academic, community), bed size, location (urban, rural), hospital distributions of race/ethnicity and public payer, cases of bronchiolitis per year, presence of an electronic medical record and a pediatric respiratory therapist, and self-rated QI knowledge of the multidisciplinary team (very knowledgeable, knowledgeable, and somewhat knowledgeable). A trained member at each site collected data through structured chart review in baseline, intervention, and sustainability bronchiolitis seasons for January, February, and March. Site members reviewed the first 20 charts per month that met the inclusion criteria or all charts if there were fewer than 20 eligible encounters. Sites input data about key quality measures into the AAP’s Quality Improvement Data Aggregator, a web-based data repository.

Intervention

The BQIP project was designed as a virtual collaborative consisting of monthly education webinars about QI methods and bronchiolitis management, opportunities for collaboration via teleconference and e-mail listserv, and individual site-coaching by e-mail or telephone.⁹ A change package was shared with sites that included examples of evidence-based pathways, ordersets, a respiratory scoring tool, communication tools for parents and referring physicians, and slide sets for individual site education efforts. Following completion of the collaborative, written resources remained available to participants, although virtual collaboration ceased and no additional project interventions to promote sustainability were introduced.

Bronchiolitis Process and Outcome Measures

Process measures following admission included the following: severity assessment using a respiratory score, respiratory score use to assess response to bronchodilators, bronchodilator use, bronchodilator doses, steroid doses per patient encounter, chest radiographs per encounter, and presence of an order to transition to intermittent pulse oximetry monitoring. Outcome measures included length of stay and readmissions within 72 hours.

Analysis

Changes among baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with random effect for study sites. Negative binomial models were used for count variables to allow for overdispersion. Length of stay was log-transformed to achieve a normal distribution. We also analyzed each site individually to assess whether sustained improvements were the result of broad sustainability across all sites or whether they represented an aggregation of some sites that continued to improve while other sites actually worsened.

To address any bias introduced by the voluntary and incomplete participation of sites in the sustainability season, we planned a priori to conduct 3 additional analyses. First, we compared the characteristics of sites that did participate in the sustainability season with those that did not participate by using Chi-squared tests for differences in proportions and t tests for differences in means. Second, we determined whether the baseline-season process and outcome measures were different between sites that did and did not participate using descriptive statistics. Third, we assessed whether improvements between the baseline and intervention seasons were different between sites that did and did not participate using a linear mixed-effects model for normally distributed outcomes and generalized linear mixed-effects model with site-specific random effects for nonnormally distributed outcomes. All study outcomes were summarized in terms of model-adjusted means along with the corresponding 95% confidence intervals. All P values are 2-sided, and P < 0.05 was used to define statistical significance. Data analyses were conducted using SAS software (SAS Institute Inc., Cary, North Carolina) version 9.4.

To our knowledge, this is the first report of sustained improvements in care achieved through a multiinstitutional QI collaborative of community and academic hospitals focused on bronchiolitis care. We found that overall sites participating in a national bronchiolitis QI project sustained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. For the aggregate group no measures worsened, and one measure, orders for intermittent pulse oximetry monitoring, continued to increase during the sustainability season. Furthermore, the sustained improvements were primarily the result of consistent sustained performance of each individual site, as opposed to averages wherein some sites worsened while others improved (Appendix 1a). These findings suggest that designing a collaborative approach, which provides an evidence-based best-practice toolkit while building the QI capacity of local interdisciplinary teams, can support performance gains that persist beyond the project’s active phase.

There are a number of possible reasons why improvements were sustained following the collaborative. The BQIP requirement for institutional leadership buy-in may have motivated accountability to local leaders in subsequent bronchiolitis seasons at each site. We suspect that culture change such as flattened hierarchies through multidisciplinary teams,¹⁴ which empowered nurse and respiratory therapy staff, may have facilitated consistent use of tools created locally. The synergy of interdisciplinary teams composed of physician, nurse, and respiratory therapy champions may have created accountability to perpetuate the previous year’s efforts.¹⁵ In addition, the sites adopted elements of the evidence-based toolkit, such as pathways,^16,17 forcing function tools^13,18 and order sets that limited management decision options and bronchodilator use contingent on respiratory scores,^9,19 which may have driven desired behaviors.

Moreover, the 2014 AAP CPG for the management of bronchiolitis,²⁰ released prior to the sustainability bronchiolitis season, may have underscored the key concepts of the collaborative. Similarly, national exposure of best practices for bronchiolitis management, including the 3 widespread Choosing Wisely recommendations related to bronchiolitis,²¹ might have been a compelling reason for sites to maintain their improvement efforts and contribute to secular trends toward decreasing interventions in bronchiolitis management nationally.³ Lastly, the mechanisms developed for local data collection may have created opportunities at each site to conduct ongoing evaluation of performance on key bronchiolitis quality measures through data-driven feedback systems.²² Our study highlights the need for additional research in order to understand why improvements are or are not sustained.

Even with substantial, sustained improvements in this initiative, further reduction in unnecessary care may be possible. Findings from previous studies suggest that even multifaceted QI interventions, including provider education, guidelines and use of respiratory scores, may only modestly reduce bronchodilators, steroids, and chest radiograph use.^8,13 To achieve continued improvements in bronchiolitis care, additional active efforts may be needed to develop new interventions that target root causes for areas of overuse at individual sites.

Future multiinstitutional collaboratives might benefit their participants if they include a focus on helping sites develop skills to ensure that local improvement activities continue after the collaborative phases are completed. Proactively scheduling intermittent check-ins with collaborative members to discuss experiences with both sustainability and ongoing improvement may be valuable and likely needs to be incorporated into the initial collaborative planning.

As these sustainability data represent a subset of 9 of the original 21 BQIP sites, there is concern for potential selection bias related to factors that could have motivated sites to participate in the sustainability season’s data collection and simultaneously influenced their performance. These concerns were mitigated to some extent through 3 specific analyses: finding limited differences in hospital characteristics, baseline performance in key bronchiolitis measures, and performance change from baseline to intervention seasons between sites that did and did not participate in the sustainability season.

Notably, sites that participated in the sustainability phase actually had lower baseline respiratory score use and fewer orders for intermittent pulse oximetry at baseline. Theoretically, if participation in the collaborative highlighted this disparity for these sites, it could have been a motivating factor for their continued participation and sustained performance across these measures. Similarly, sites that recognized their higher baseline performance through participation in the collaborative might have felt less motivation to participate in ongoing data collection during the sustainability season. Whether they might have also sustained, declined, or continued improving is not known. Additionally, the magnitude of improvement in the collaborative period might have also motivated ongoing participation during the sustainability phase. For example, although all sites improved in score use during the collaborative, sites participating in the sustainability season demonstrated significantly more improvement in these measures. Sites with a higher magnitude of improvement in collaborative measures might have more enthusiasm about the project, more commitment to the project activities, or feel a sense of obligation to respond to requests for additional data collection.

This work has several limitations. Selection bias may limit generalizability of the results, as sites that did not participate in the sustainability season may have had different results than those that did participate. It is unknown whether sites that regressed toward their baseline were deterred from participating in the sustainability season. The analyses that we were able to preform, however, suggest that the 2 groups were similar in their characteristics as well as in their baseline and improvement performance.

We have limited knowledge of the local improvement work that sites conducted between the completion of the collaborative and the sustainability season. Site-specific factors may have influenced improvement sustainability. For example, qualitative research with the original group found that team engagement had a quantitative association with better performance, but only for the bronchodilator use measure.²³ Sites were responsible for their own data collection, and despite attempts to centralize and standardize the process, data collection inconsistencies may have occurred. For instance, it is unknown how closely that orders for intermittent pulse oximetry correlate with intermittent use at the bedside. Lastly, the absence of a control group limits examination of the causal relationships of interventions and the influence of secular trends.

Improvements gained during the BQIP collaborative were sustained at 1 year following completion of the collaborative. These findings are encouraging, as national QI collaborative efforts are increasingly common. Our findings suggest that opportunities exist to even further reduce unnecessary care in the management of bronchiolitis. Such opportunities highlight the importance of integrating strategies to both measure sustainability and plan for ongoing independent local activities after completion of the collaborative. Future efforts should focus on supporting local sites to continue individual practice-improvement as they transition from collaborative to independent quality initiatives.

Acknowledgments

The authors thank the 21 hospitals that participated in the BQIP collaborative, and in particular the 9 hospital teams that contributed sustainability data for their ongoing dedication. There was no external funding for this manuscript.

Disclosure

The authors report no financial conflicts of interest.

Acute viral bronchiolitis is the most common cause of hospitalization for children less than 1 year of age.¹ Overuse of ineffective therapies has persisted despite the existence of the evidence-based American Academy of Pediatrics (AAP) clinical practice guideline (CPG), which recommends primarily supportive care.^2-8 Adherence to the AAP CPG recommendations for management of bronchiolitis improved significantly through the AAP’s Bronchiolitis Quality Improvement Project (BQIP), a 12-month, multiinstitutional collaborative of community and free-standing children’s hospitals.⁹ This subsequent study investigates if these improvements were sustained after completion of the formal 12-month project.

Published multiinstitutional bronchiolitis quality improvement (QI) work is limited to 1 study⁵ that describes the results of a single intervention season at academic medical centers. Multiyear bronchiolitis QI projects are limited to single-center studies, and results have been mixed.^5,6,8,10-13 One study¹¹ observed continued improvement in bronchodilator use in subsequent seasons, whereas a second study¹⁰ observed a return to baseline bronchodilator use in the following season. Mittal⁶ observed inconsistent improvements in key bronchiolitis measures during postintervention seasons.

Our specific aim was to assess the sustainability of improvements in bronchiolitis management at participating institutions 1 year following completion of the AAP BQIP collaborative.⁹ Because no studies demonstrate the most effective way to support long-term improvement through a QI collaborative, we hypothesized that the initial collaborative activities, which were designed to build the capacity of local interdisciplinary teams while providing standardized evidence-based care pathways, would lead to performance in the subsequent season at levels similar to or better than those observed during the active phase of the collaborative, without additional project interventions.

Study Design and Setting

This was a follow-up study of the AAP Quality Improvement Innovation Networks project entitled “A Quality Collaborative for Improving Hospital Compliance with the AAP Bronchiolitis Guideline” (BQIP).⁹ The AAP Institutional Review Board approved this project.

Twenty-one multidisciplinary, hospital-based teams participated in the BQIP collaborative and provided monthly data during the January through March bronchiolitis season. Teams submitted 2013 baseline data and 2014 intervention data. Nine sites provided 2015 sustainability data following the completion of the collaborative.

Participants

Hospital encounters with a primary diagnosis of acute viral bronchiolitis were eligible for inclusion among patients from 1 month to 2 years of age. Encounters were excluded for prematurity (<35 weeks gestational age), congenital heart disease, bronchopulmonary dysplasia, genetic, congenital or neuromuscular abnormalities, and pediatric intensive-care admission.

Data Collection

Hospital characteristics were collected, including hospital type (academic, community), bed size, location (urban, rural), hospital distributions of race/ethnicity and public payer, cases of bronchiolitis per year, presence of an electronic medical record and a pediatric respiratory therapist, and self-rated QI knowledge of the multidisciplinary team (very knowledgeable, knowledgeable, and somewhat knowledgeable). A trained member at each site collected data through structured chart review in baseline, intervention, and sustainability bronchiolitis seasons for January, February, and March. Site members reviewed the first 20 charts per month that met the inclusion criteria or all charts if there were fewer than 20 eligible encounters. Sites input data about key quality measures into the AAP’s Quality Improvement Data Aggregator, a web-based data repository.

Intervention

The BQIP project was designed as a virtual collaborative consisting of monthly education webinars about QI methods and bronchiolitis management, opportunities for collaboration via teleconference and e-mail listserv, and individual site-coaching by e-mail or telephone.⁹ A change package was shared with sites that included examples of evidence-based pathways, ordersets, a respiratory scoring tool, communication tools for parents and referring physicians, and slide sets for individual site education efforts. Following completion of the collaborative, written resources remained available to participants, although virtual collaboration ceased and no additional project interventions to promote sustainability were introduced.

Bronchiolitis Process and Outcome Measures

Process measures following admission included the following: severity assessment using a respiratory score, respiratory score use to assess response to bronchodilators, bronchodilator use, bronchodilator doses, steroid doses per patient encounter, chest radiographs per encounter, and presence of an order to transition to intermittent pulse oximetry monitoring. Outcome measures included length of stay and readmissions within 72 hours.

Analysis

Changes among baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with random effect for study sites. Negative binomial models were used for count variables to allow for overdispersion. Length of stay was log-transformed to achieve a normal distribution. We also analyzed each site individually to assess whether sustained improvements were the result of broad sustainability across all sites or whether they represented an aggregation of some sites that continued to improve while other sites actually worsened.

To address any bias introduced by the voluntary and incomplete participation of sites in the sustainability season, we planned a priori to conduct 3 additional analyses. First, we compared the characteristics of sites that did participate in the sustainability season with those that did not participate by using Chi-squared tests for differences in proportions and t tests for differences in means. Second, we determined whether the baseline-season process and outcome measures were different between sites that did and did not participate using descriptive statistics. Third, we assessed whether improvements between the baseline and intervention seasons were different between sites that did and did not participate using a linear mixed-effects model for normally distributed outcomes and generalized linear mixed-effects model with site-specific random effects for nonnormally distributed outcomes. All study outcomes were summarized in terms of model-adjusted means along with the corresponding 95% confidence intervals. All P values are 2-sided, and P < 0.05 was used to define statistical significance. Data analyses were conducted using SAS software (SAS Institute Inc., Cary, North Carolina) version 9.4.

To our knowledge, this is the first report of sustained improvements in care achieved through a multiinstitutional QI collaborative of community and academic hospitals focused on bronchiolitis care. We found that overall sites participating in a national bronchiolitis QI project sustained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. For the aggregate group no measures worsened, and one measure, orders for intermittent pulse oximetry monitoring, continued to increase during the sustainability season. Furthermore, the sustained improvements were primarily the result of consistent sustained performance of each individual site, as opposed to averages wherein some sites worsened while others improved (Appendix 1a). These findings suggest that designing a collaborative approach, which provides an evidence-based best-practice toolkit while building the QI capacity of local interdisciplinary teams, can support performance gains that persist beyond the project’s active phase.

There are a number of possible reasons why improvements were sustained following the collaborative. The BQIP requirement for institutional leadership buy-in may have motivated accountability to local leaders in subsequent bronchiolitis seasons at each site. We suspect that culture change such as flattened hierarchies through multidisciplinary teams,¹⁴ which empowered nurse and respiratory therapy staff, may have facilitated consistent use of tools created locally. The synergy of interdisciplinary teams composed of physician, nurse, and respiratory therapy champions may have created accountability to perpetuate the previous year’s efforts.¹⁵ In addition, the sites adopted elements of the evidence-based toolkit, such as pathways,^16,17 forcing function tools^13,18 and order sets that limited management decision options and bronchodilator use contingent on respiratory scores,^9,19 which may have driven desired behaviors.

Moreover, the 2014 AAP CPG for the management of bronchiolitis,²⁰ released prior to the sustainability bronchiolitis season, may have underscored the key concepts of the collaborative. Similarly, national exposure of best practices for bronchiolitis management, including the 3 widespread Choosing Wisely recommendations related to bronchiolitis,²¹ might have been a compelling reason for sites to maintain their improvement efforts and contribute to secular trends toward decreasing interventions in bronchiolitis management nationally.³ Lastly, the mechanisms developed for local data collection may have created opportunities at each site to conduct ongoing evaluation of performance on key bronchiolitis quality measures through data-driven feedback systems.²² Our study highlights the need for additional research in order to understand why improvements are or are not sustained.

Even with substantial, sustained improvements in this initiative, further reduction in unnecessary care may be possible. Findings from previous studies suggest that even multifaceted QI interventions, including provider education, guidelines and use of respiratory scores, may only modestly reduce bronchodilators, steroids, and chest radiograph use.^8,13 To achieve continued improvements in bronchiolitis care, additional active efforts may be needed to develop new interventions that target root causes for areas of overuse at individual sites.

Future multiinstitutional collaboratives might benefit their participants if they include a focus on helping sites develop skills to ensure that local improvement activities continue after the collaborative phases are completed. Proactively scheduling intermittent check-ins with collaborative members to discuss experiences with both sustainability and ongoing improvement may be valuable and likely needs to be incorporated into the initial collaborative planning.

As these sustainability data represent a subset of 9 of the original 21 BQIP sites, there is concern for potential selection bias related to factors that could have motivated sites to participate in the sustainability season’s data collection and simultaneously influenced their performance. These concerns were mitigated to some extent through 3 specific analyses: finding limited differences in hospital characteristics, baseline performance in key bronchiolitis measures, and performance change from baseline to intervention seasons between sites that did and did not participate in the sustainability season.

Notably, sites that participated in the sustainability phase actually had lower baseline respiratory score use and fewer orders for intermittent pulse oximetry at baseline. Theoretically, if participation in the collaborative highlighted this disparity for these sites, it could have been a motivating factor for their continued participation and sustained performance across these measures. Similarly, sites that recognized their higher baseline performance through participation in the collaborative might have felt less motivation to participate in ongoing data collection during the sustainability season. Whether they might have also sustained, declined, or continued improving is not known. Additionally, the magnitude of improvement in the collaborative period might have also motivated ongoing participation during the sustainability phase. For example, although all sites improved in score use during the collaborative, sites participating in the sustainability season demonstrated significantly more improvement in these measures. Sites with a higher magnitude of improvement in collaborative measures might have more enthusiasm about the project, more commitment to the project activities, or feel a sense of obligation to respond to requests for additional data collection.

This work has several limitations. Selection bias may limit generalizability of the results, as sites that did not participate in the sustainability season may have had different results than those that did participate. It is unknown whether sites that regressed toward their baseline were deterred from participating in the sustainability season. The analyses that we were able to preform, however, suggest that the 2 groups were similar in their characteristics as well as in their baseline and improvement performance.

We have limited knowledge of the local improvement work that sites conducted between the completion of the collaborative and the sustainability season. Site-specific factors may have influenced improvement sustainability. For example, qualitative research with the original group found that team engagement had a quantitative association with better performance, but only for the bronchodilator use measure.²³ Sites were responsible for their own data collection, and despite attempts to centralize and standardize the process, data collection inconsistencies may have occurred. For instance, it is unknown how closely that orders for intermittent pulse oximetry correlate with intermittent use at the bedside. Lastly, the absence of a control group limits examination of the causal relationships of interventions and the influence of secular trends.

Improvements gained during the BQIP collaborative were sustained at 1 year following completion of the collaborative. These findings are encouraging, as national QI collaborative efforts are increasingly common. Our findings suggest that opportunities exist to even further reduce unnecessary care in the management of bronchiolitis. Such opportunities highlight the importance of integrating strategies to both measure sustainability and plan for ongoing independent local activities after completion of the collaborative. Future efforts should focus on supporting local sites to continue individual practice-improvement as they transition from collaborative to independent quality initiatives.

Acknowledgments

The authors thank the 21 hospitals that participated in the BQIP collaborative, and in particular the 9 hospital teams that contributed sustainability data for their ongoing dedication. There was no external funding for this manuscript.

Disclosure

The authors report no financial conflicts of interest.

Acute viral bronchiolitis is the most common cause of hospitalization for children less than 1 year of age.¹ Overuse of ineffective therapies has persisted despite the existence of the evidence-based American Academy of Pediatrics (AAP) clinical practice guideline (CPG), which recommends primarily supportive care.^2-8 Adherence to the AAP CPG recommendations for management of bronchiolitis improved significantly through the AAP’s Bronchiolitis Quality Improvement Project (BQIP), a 12-month, multiinstitutional collaborative of community and free-standing children’s hospitals.⁹ This subsequent study investigates if these improvements were sustained after completion of the formal 12-month project.

Published multiinstitutional bronchiolitis quality improvement (QI) work is limited to 1 study⁵ that describes the results of a single intervention season at academic medical centers. Multiyear bronchiolitis QI projects are limited to single-center studies, and results have been mixed.^5,6,8,10-13 One study¹¹ observed continued improvement in bronchodilator use in subsequent seasons, whereas a second study¹⁰ observed a return to baseline bronchodilator use in the following season. Mittal⁶ observed inconsistent improvements in key bronchiolitis measures during postintervention seasons.

Our specific aim was to assess the sustainability of improvements in bronchiolitis management at participating institutions 1 year following completion of the AAP BQIP collaborative.⁹ Because no studies demonstrate the most effective way to support long-term improvement through a QI collaborative, we hypothesized that the initial collaborative activities, which were designed to build the capacity of local interdisciplinary teams while providing standardized evidence-based care pathways, would lead to performance in the subsequent season at levels similar to or better than those observed during the active phase of the collaborative, without additional project interventions.

Study Design and Setting

This was a follow-up study of the AAP Quality Improvement Innovation Networks project entitled “A Quality Collaborative for Improving Hospital Compliance with the AAP Bronchiolitis Guideline” (BQIP).⁹ The AAP Institutional Review Board approved this project.

Twenty-one multidisciplinary, hospital-based teams participated in the BQIP collaborative and provided monthly data during the January through March bronchiolitis season. Teams submitted 2013 baseline data and 2014 intervention data. Nine sites provided 2015 sustainability data following the completion of the collaborative.

Participants

Hospital encounters with a primary diagnosis of acute viral bronchiolitis were eligible for inclusion among patients from 1 month to 2 years of age. Encounters were excluded for prematurity (<35 weeks gestational age), congenital heart disease, bronchopulmonary dysplasia, genetic, congenital or neuromuscular abnormalities, and pediatric intensive-care admission.

Data Collection

Hospital characteristics were collected, including hospital type (academic, community), bed size, location (urban, rural), hospital distributions of race/ethnicity and public payer, cases of bronchiolitis per year, presence of an electronic medical record and a pediatric respiratory therapist, and self-rated QI knowledge of the multidisciplinary team (very knowledgeable, knowledgeable, and somewhat knowledgeable). A trained member at each site collected data through structured chart review in baseline, intervention, and sustainability bronchiolitis seasons for January, February, and March. Site members reviewed the first 20 charts per month that met the inclusion criteria or all charts if there were fewer than 20 eligible encounters. Sites input data about key quality measures into the AAP’s Quality Improvement Data Aggregator, a web-based data repository.

Intervention

The BQIP project was designed as a virtual collaborative consisting of monthly education webinars about QI methods and bronchiolitis management, opportunities for collaboration via teleconference and e-mail listserv, and individual site-coaching by e-mail or telephone.⁹ A change package was shared with sites that included examples of evidence-based pathways, ordersets, a respiratory scoring tool, communication tools for parents and referring physicians, and slide sets for individual site education efforts. Following completion of the collaborative, written resources remained available to participants, although virtual collaboration ceased and no additional project interventions to promote sustainability were introduced.

Bronchiolitis Process and Outcome Measures

Process measures following admission included the following: severity assessment using a respiratory score, respiratory score use to assess response to bronchodilators, bronchodilator use, bronchodilator doses, steroid doses per patient encounter, chest radiographs per encounter, and presence of an order to transition to intermittent pulse oximetry monitoring. Outcome measures included length of stay and readmissions within 72 hours.

Analysis

Changes among baseline-, intervention-, and sustainability-season data were assessed using generalized linear mixed-effects models with random effect for study sites. Negative binomial models were used for count variables to allow for overdispersion. Length of stay was log-transformed to achieve a normal distribution. We also analyzed each site individually to assess whether sustained improvements were the result of broad sustainability across all sites or whether they represented an aggregation of some sites that continued to improve while other sites actually worsened.

To address any bias introduced by the voluntary and incomplete participation of sites in the sustainability season, we planned a priori to conduct 3 additional analyses. First, we compared the characteristics of sites that did participate in the sustainability season with those that did not participate by using Chi-squared tests for differences in proportions and t tests for differences in means. Second, we determined whether the baseline-season process and outcome measures were different between sites that did and did not participate using descriptive statistics. Third, we assessed whether improvements between the baseline and intervention seasons were different between sites that did and did not participate using a linear mixed-effects model for normally distributed outcomes and generalized linear mixed-effects model with site-specific random effects for nonnormally distributed outcomes. All study outcomes were summarized in terms of model-adjusted means along with the corresponding 95% confidence intervals. All P values are 2-sided, and P < 0.05 was used to define statistical significance. Data analyses were conducted using SAS software (SAS Institute Inc., Cary, North Carolina) version 9.4.

To our knowledge, this is the first report of sustained improvements in care achieved through a multiinstitutional QI collaborative of community and academic hospitals focused on bronchiolitis care. We found that overall sites participating in a national bronchiolitis QI project sustained improvements in key bronchiolitis quality measures for 1 year following the project’s completion. For the aggregate group no measures worsened, and one measure, orders for intermittent pulse oximetry monitoring, continued to increase during the sustainability season. Furthermore, the sustained improvements were primarily the result of consistent sustained performance of each individual site, as opposed to averages wherein some sites worsened while others improved (Appendix 1a). These findings suggest that designing a collaborative approach, which provides an evidence-based best-practice toolkit while building the QI capacity of local interdisciplinary teams, can support performance gains that persist beyond the project’s active phase.

There are a number of possible reasons why improvements were sustained following the collaborative. The BQIP requirement for institutional leadership buy-in may have motivated accountability to local leaders in subsequent bronchiolitis seasons at each site. We suspect that culture change such as flattened hierarchies through multidisciplinary teams,¹⁴ which empowered nurse and respiratory therapy staff, may have facilitated consistent use of tools created locally. The synergy of interdisciplinary teams composed of physician, nurse, and respiratory therapy champions may have created accountability to perpetuate the previous year’s efforts.¹⁵ In addition, the sites adopted elements of the evidence-based toolkit, such as pathways,^16,17 forcing function tools^13,18 and order sets that limited management decision options and bronchodilator use contingent on respiratory scores,^9,19 which may have driven desired behaviors.

Moreover, the 2014 AAP CPG for the management of bronchiolitis,²⁰ released prior to the sustainability bronchiolitis season, may have underscored the key concepts of the collaborative. Similarly, national exposure of best practices for bronchiolitis management, including the 3 widespread Choosing Wisely recommendations related to bronchiolitis,²¹ might have been a compelling reason for sites to maintain their improvement efforts and contribute to secular trends toward decreasing interventions in bronchiolitis management nationally.³ Lastly, the mechanisms developed for local data collection may have created opportunities at each site to conduct ongoing evaluation of performance on key bronchiolitis quality measures through data-driven feedback systems.²² Our study highlights the need for additional research in order to understand why improvements are or are not sustained.

Even with substantial, sustained improvements in this initiative, further reduction in unnecessary care may be possible. Findings from previous studies suggest that even multifaceted QI interventions, including provider education, guidelines and use of respiratory scores, may only modestly reduce bronchodilators, steroids, and chest radiograph use.^8,13 To achieve continued improvements in bronchiolitis care, additional active efforts may be needed to develop new interventions that target root causes for areas of overuse at individual sites.

Future multiinstitutional collaboratives might benefit their participants if they include a focus on helping sites develop skills to ensure that local improvement activities continue after the collaborative phases are completed. Proactively scheduling intermittent check-ins with collaborative members to discuss experiences with both sustainability and ongoing improvement may be valuable and likely needs to be incorporated into the initial collaborative planning.

As these sustainability data represent a subset of 9 of the original 21 BQIP sites, there is concern for potential selection bias related to factors that could have motivated sites to participate in the sustainability season’s data collection and simultaneously influenced their performance. These concerns were mitigated to some extent through 3 specific analyses: finding limited differences in hospital characteristics, baseline performance in key bronchiolitis measures, and performance change from baseline to intervention seasons between sites that did and did not participate in the sustainability season.

Notably, sites that participated in the sustainability phase actually had lower baseline respiratory score use and fewer orders for intermittent pulse oximetry at baseline. Theoretically, if participation in the collaborative highlighted this disparity for these sites, it could have been a motivating factor for their continued participation and sustained performance across these measures. Similarly, sites that recognized their higher baseline performance through participation in the collaborative might have felt less motivation to participate in ongoing data collection during the sustainability season. Whether they might have also sustained, declined, or continued improving is not known. Additionally, the magnitude of improvement in the collaborative period might have also motivated ongoing participation during the sustainability phase. For example, although all sites improved in score use during the collaborative, sites participating in the sustainability season demonstrated significantly more improvement in these measures. Sites with a higher magnitude of improvement in collaborative measures might have more enthusiasm about the project, more commitment to the project activities, or feel a sense of obligation to respond to requests for additional data collection.

This work has several limitations. Selection bias may limit generalizability of the results, as sites that did not participate in the sustainability season may have had different results than those that did participate. It is unknown whether sites that regressed toward their baseline were deterred from participating in the sustainability season. The analyses that we were able to preform, however, suggest that the 2 groups were similar in their characteristics as well as in their baseline and improvement performance.

We have limited knowledge of the local improvement work that sites conducted between the completion of the collaborative and the sustainability season. Site-specific factors may have influenced improvement sustainability. For example, qualitative research with the original group found that team engagement had a quantitative association with better performance, but only for the bronchodilator use measure.²³ Sites were responsible for their own data collection, and despite attempts to centralize and standardize the process, data collection inconsistencies may have occurred. For instance, it is unknown how closely that orders for intermittent pulse oximetry correlate with intermittent use at the bedside. Lastly, the absence of a control group limits examination of the causal relationships of interventions and the influence of secular trends.

Improvements gained during the BQIP collaborative were sustained at 1 year following completion of the collaborative. These findings are encouraging, as national QI collaborative efforts are increasingly common. Our findings suggest that opportunities exist to even further reduce unnecessary care in the management of bronchiolitis. Such opportunities highlight the importance of integrating strategies to both measure sustainability and plan for ongoing independent local activities after completion of the collaborative. Future efforts should focus on supporting local sites to continue individual practice-improvement as they transition from collaborative to independent quality initiatives.

Acknowledgments

The authors thank the 21 hospitals that participated in the BQIP collaborative, and in particular the 9 hospital teams that contributed sustainability data for their ongoing dedication. There was no external funding for this manuscript.

Disclosure

The authors report no financial conflicts of interest.