Original Research

Bronchiolitis is the most common cause of nonbirth hospitalization in children in the United States less than one year of age.¹ For children with severe bronchiolitis, high-flow nasal cannula (HFNC) is increasingly used^2-4 to reduce work of breathing and prevent the need for further escalation of ventilatory support.^5,6 Although previous studies suggest that enteral feeding is recommended in the management of patients hospitalized with bronchiolitis,^7-9 limited evidence exists to guide feeding practices for patients receiving HFNC support.^5,10,11

Respiratory support with HFNC has been associated with prolonged periods without enteral hydration/nutrition (ie, nil per os [NPO])¹² primarily due to anticipation of further escalation of respiratory support or concern for increased risk of aspiration. The majority of patients with bronchiolitis managed with HFNC, however, do not require escalation of care.^5,13 When feeding is attempted during HFNC support, it is frequently interrupted.⁵ Moreover, keeping all children NPO when receiving HFNC may be associated with weight loss and longer length of stay (LOS).^12,14 Two small studies found that children admitted to the intensive care unit who received HFNC support for bronchiolitis did not have increased rates of emesis, worsening respiratory distress or aspiration pneumonia when enterally fed.^10,11 However, no comparison of adverse events or LOS has been made between patients who were fed and those who were not fed during HFNC therapy, and previous studies have included only patients who have received HFNC in the intensive care setting.

Supporting safe feeding early in hospitalizations for bronchiolitis may facilitate expedited clinical improvement and discharge. As part of an ongoing bronchiolitis quality improvement initiative at our hospital, we sought to characterize feeding practices during HFNC therapy and assess whether feeding exposure was associated with (1) time to discharge after HFNC or (2) feeding-related adverse events. We hypothesized that feeding during HFNC therapy would be associated with a shorter time to discharge after HFNC cessation.

Study Design, Setting, Participants

This was a retrospective cohort study of patients aged 1-24 months receiving HFNC support for respiratory failure due to bronchiolitis at an academic children’s hospital between January 1, 2015 and March 1, 2017. Our institution has had a clinical practice guideline, associated order set, and respiratory therapy protocol for general care patients with bronchiolitis since 2009. Patients with bronchiolitis who were weaning HFNC have been cared for in both the intensive and general care settings since 2013. A formal process for initiation of HFNC on general care units was instituted in January of 2017. During the study period, no patients with HFNC support for bronchiolitis had all their care entirely outside the intensive care unit at our institution. However, initiation and subsequent use of HFNC may have occurred in either the intensive care or general care setting. No specific guidance for feeding during HFNC existed during this period.

Patients were identified using the Virtual PICU Systems database, (VPS LLC, myvps.org, Los Angeles, California) and, by definition, all patients received at least some of their care in the intensive care unit. Patients with comorbid conditions of prematurity (<35 weeks) and those with cardiopulmonary, neuromuscular, and genetic diseases were included. Patients with preexisting dysphagia, defined as ongoing outpatient speech therapy for swallowing concerns, an admission diagnosis of aspiration pneumonia or on home respiratory support, were excluded. Children (n = 7) were excluded if they had more than one period of HFNC during admission. This study was determined to be exempt by the University of Wisconsin School of Medicine and Public Health’s Institutional Review Board.

Data Collection and Study Variables

The following variables were collected from VPS administrative data: patient gender, age, admission and discharge date and time, type and total hours of respiratory support, intensive care admission, and LOS (in hours). Additional demographic, clinical, and feeding exposure variables were abstracted manually from the electronic medical record (Epic, Verona, Wisconsin) using a structured data collection tool and stored in REDCap (Research Electronic Data Capture)¹⁵ including prematurity, race/ethnicity, insurance status, primary language, and passive tobacco smoke exposure. Clinical variables included duration of illness (days) at the time of admission, unit of HFNC initiation (emergency department, general care, intensive care, respiratory rate and oxygen saturation at HFNC initiation (<90%, 91%-92%, or >92%), acquisition of blood gas at HFNC admission, duration of time on HFNC (hours) and need for intubation or noninvasive ventilation prior to HFNC. The Pediatric Index of Mortality 2 Risk of Mortality (PIM 2 ROM)¹⁶ was used to estimate the severity of illness. The PIM2ROM uses clinical variables (systolic blood pressure, fixed pupils, measure of hypoxia using PaO₂/FiO₂ ratio, base excess, mechanical ventilation, elective admission, recovery from surgery, cardiac bypass, high-risk diagnosis, low-risk diagnosis) collected at the time of intensive care admission to generate a score that predicts the risk of mortality for an individual patient.¹⁷

Feeding exposures were documented in three-hour intervals from HFNC initiation to completion using a structured protocol. At each interval the following feeding information was abstracted from a review of nursing and physician documentation and relevant clinical flowsheets: presence or absence of feeding during the interval, route of feeding (oral, nasogastric [NG] or nasojejunal [NJ]). Feeding exposure was categorized a priori as fed at any point during HFNC (vs not fed at any point). Fed children were further characterized as (1) mixed feeding consisting of oral and tube feeds (NG or NJ) or (2) exclusive oral feeding throughout HFNC support (Appendix 1).

The primary outcome was the number of hours to discharge from HFNC cessation. Secondary outcomes were time to discharge from HFNC initiation, all-cause readmissions within seven days of discharge, and potential feeding-related adverse events. Potential adverse events included: (1) aspiration, defined as initiation of antibiotic AND either chest radiograph official interpreted as evidence for aspiration and/or documented concern for aspiration from the treating physician, or (2) intubation after feeding during HFNC.

Statistical Analysis

Descriptive statistics evaluated differences in demographics and clinical variables for feeding exposure groups. We used chi-squared tests for differences in proportions and t-tests or Wilcoxon Rank-Sum tests for differences in means or medians for continuous variables, respectively. Associations between feeding exposure during HFNC and time to discharge (measured in hours) after HFNC completion were modeled with Cox proportional hazards regression. Using this approach, hazard ratios (HR)>1 indicate a higher hazard (rate) of discharge for children with a feeding exposure than for children without the exposure. For example, a hazard ratio equal to two indicates that the exposed population is discharged at twice the rate per unit time as the nonexposed population. Death or censoring events did not occur. Feeding exposure was first modeled dichotomously as not fed or fed. To further explore associations between feeding modality and our outcome, we then modeled feeding exposure categorically as not fed (reference), mixed (oral and tube) feeding, or exclusive oral feeding throughout HFNC.

After constructing a set of unadjusted models, we then adjusted the models for variables having independent (bivariate P < .10) associations with time to discharge: age, unit of HFNC initiation, highest respiratory support required before HFNC, and HFNC duration. Finally, to attempt to account for residual confounding from latent constructs, we also created a set of propensity-weighted Cox proportional hazards models. Propensity weights¹⁸ reflecting the probability of being fed or never being fed during HFNC were created using logistic regression with predictors we hypothesized a priori that may have influenced the clinical decision to feed during HFNC: age, day of illness on admission, prematurity, PIM2 ROM score, respiratory rate, oxygen saturation and blood gas acquisition at HFNC initiation, and highest respiratory support required before HFNC. All analyses were conducted using STATA 14.0 (StataCorp, College Station, Texas), and adjusted hazard ratios (aHR) with 95% confidence intervals (95% CIs) were reported.

Patients (n = 123) had a mean age of 7.3 months (standard deviation [SD] 7.1) and presented on day of illness 4.8 (SD 2.3). Prior to HFNC, 10% required higher respiratory support (3% mechanical ventilation). Former preterm children were 12% of the overall sample.

During HFNC, 37% of patients were never fed, 41% were exclusively orally fed, and 23% had tube or mixed oral and tube feedings (Table 1 and Appendix 2). Children who were not fed were older, but groups were otherwise similar in terms of gender, race/ethnicity, passive smoke exposure, day of illness, unit of HFNC initiation, respiratory support required prior to HFNC, and respiratory rate at HFNC initiation.

This observational study found that being fed during HFNC was associated with shorter time to discharge after HFNC support was completed. Exclusive oral feeding was associated with the shortest time to discharge, and these results were consistent across a variety of analytical approaches. Adverse events were rare and occurred in both fed and unfed children.

These findings advance research on relationships between nutrition and bronchiolitis outcomes. Studies of general care patients with bronchiolitis without HFNC have observed associations between poor nutrition and prolonged LOS.^19,20 Two previous studies of patients receiving HFNC therapy for bronchiolitis concluded that frequent interruption¹¹ and later initiation of enteral nutrition¹⁰ during ICU stay was associated with longer time to discharge.¹¹ To our knowledge, this is the first study of patients with bronchiolitis treated with HFNC in both general care and ICU settings that compared outcomes according to whether children were fed during HFNC therapy. Our results extend previous work demonstrating that delays in feeding may be associated with longer LOS.

Children fed while receiving HFNC for bronchiolitis may have shorter time to discharge than those who are not fed. Feeding-related adverse events were rare regardless of the feeding method. Controlled prospective studies addressing residual confounding are needed to justify a change in the current practice.

Acknowledgments

The authors would like to acknowledge the valuable feedback on earlier drafts from members of the University of Wisconsin Division of Pediatric Hospital Medicine CREATE writing group.

Bronchiolitis is the most common cause of nonbirth hospitalization in children in the United States less than one year of age.¹ For children with severe bronchiolitis, high-flow nasal cannula (HFNC) is increasingly used^2-4 to reduce work of breathing and prevent the need for further escalation of ventilatory support.^5,6 Although previous studies suggest that enteral feeding is recommended in the management of patients hospitalized with bronchiolitis,^7-9 limited evidence exists to guide feeding practices for patients receiving HFNC support.^5,10,11

Respiratory support with HFNC has been associated with prolonged periods without enteral hydration/nutrition (ie, nil per os [NPO])¹² primarily due to anticipation of further escalation of respiratory support or concern for increased risk of aspiration. The majority of patients with bronchiolitis managed with HFNC, however, do not require escalation of care.^5,13 When feeding is attempted during HFNC support, it is frequently interrupted.⁵ Moreover, keeping all children NPO when receiving HFNC may be associated with weight loss and longer length of stay (LOS).^12,14 Two small studies found that children admitted to the intensive care unit who received HFNC support for bronchiolitis did not have increased rates of emesis, worsening respiratory distress or aspiration pneumonia when enterally fed.^10,11 However, no comparison of adverse events or LOS has been made between patients who were fed and those who were not fed during HFNC therapy, and previous studies have included only patients who have received HFNC in the intensive care setting.

Supporting safe feeding early in hospitalizations for bronchiolitis may facilitate expedited clinical improvement and discharge. As part of an ongoing bronchiolitis quality improvement initiative at our hospital, we sought to characterize feeding practices during HFNC therapy and assess whether feeding exposure was associated with (1) time to discharge after HFNC or (2) feeding-related adverse events. We hypothesized that feeding during HFNC therapy would be associated with a shorter time to discharge after HFNC cessation.

Study Design, Setting, Participants

This was a retrospective cohort study of patients aged 1-24 months receiving HFNC support for respiratory failure due to bronchiolitis at an academic children’s hospital between January 1, 2015 and March 1, 2017. Our institution has had a clinical practice guideline, associated order set, and respiratory therapy protocol for general care patients with bronchiolitis since 2009. Patients with bronchiolitis who were weaning HFNC have been cared for in both the intensive and general care settings since 2013. A formal process for initiation of HFNC on general care units was instituted in January of 2017. During the study period, no patients with HFNC support for bronchiolitis had all their care entirely outside the intensive care unit at our institution. However, initiation and subsequent use of HFNC may have occurred in either the intensive care or general care setting. No specific guidance for feeding during HFNC existed during this period.

Patients were identified using the Virtual PICU Systems database, (VPS LLC, myvps.org, Los Angeles, California) and, by definition, all patients received at least some of their care in the intensive care unit. Patients with comorbid conditions of prematurity (<35 weeks) and those with cardiopulmonary, neuromuscular, and genetic diseases were included. Patients with preexisting dysphagia, defined as ongoing outpatient speech therapy for swallowing concerns, an admission diagnosis of aspiration pneumonia or on home respiratory support, were excluded. Children (n = 7) were excluded if they had more than one period of HFNC during admission. This study was determined to be exempt by the University of Wisconsin School of Medicine and Public Health’s Institutional Review Board.

Data Collection and Study Variables

The following variables were collected from VPS administrative data: patient gender, age, admission and discharge date and time, type and total hours of respiratory support, intensive care admission, and LOS (in hours). Additional demographic, clinical, and feeding exposure variables were abstracted manually from the electronic medical record (Epic, Verona, Wisconsin) using a structured data collection tool and stored in REDCap (Research Electronic Data Capture)¹⁵ including prematurity, race/ethnicity, insurance status, primary language, and passive tobacco smoke exposure. Clinical variables included duration of illness (days) at the time of admission, unit of HFNC initiation (emergency department, general care, intensive care, respiratory rate and oxygen saturation at HFNC initiation (<90%, 91%-92%, or >92%), acquisition of blood gas at HFNC admission, duration of time on HFNC (hours) and need for intubation or noninvasive ventilation prior to HFNC. The Pediatric Index of Mortality 2 Risk of Mortality (PIM 2 ROM)¹⁶ was used to estimate the severity of illness. The PIM2ROM uses clinical variables (systolic blood pressure, fixed pupils, measure of hypoxia using PaO₂/FiO₂ ratio, base excess, mechanical ventilation, elective admission, recovery from surgery, cardiac bypass, high-risk diagnosis, low-risk diagnosis) collected at the time of intensive care admission to generate a score that predicts the risk of mortality for an individual patient.¹⁷

Feeding exposures were documented in three-hour intervals from HFNC initiation to completion using a structured protocol. At each interval the following feeding information was abstracted from a review of nursing and physician documentation and relevant clinical flowsheets: presence or absence of feeding during the interval, route of feeding (oral, nasogastric [NG] or nasojejunal [NJ]). Feeding exposure was categorized a priori as fed at any point during HFNC (vs not fed at any point). Fed children were further characterized as (1) mixed feeding consisting of oral and tube feeds (NG or NJ) or (2) exclusive oral feeding throughout HFNC support (Appendix 1).

The primary outcome was the number of hours to discharge from HFNC cessation. Secondary outcomes were time to discharge from HFNC initiation, all-cause readmissions within seven days of discharge, and potential feeding-related adverse events. Potential adverse events included: (1) aspiration, defined as initiation of antibiotic AND either chest radiograph official interpreted as evidence for aspiration and/or documented concern for aspiration from the treating physician, or (2) intubation after feeding during HFNC.

Statistical Analysis

Descriptive statistics evaluated differences in demographics and clinical variables for feeding exposure groups. We used chi-squared tests for differences in proportions and t-tests or Wilcoxon Rank-Sum tests for differences in means or medians for continuous variables, respectively. Associations between feeding exposure during HFNC and time to discharge (measured in hours) after HFNC completion were modeled with Cox proportional hazards regression. Using this approach, hazard ratios (HR)>1 indicate a higher hazard (rate) of discharge for children with a feeding exposure than for children without the exposure. For example, a hazard ratio equal to two indicates that the exposed population is discharged at twice the rate per unit time as the nonexposed population. Death or censoring events did not occur. Feeding exposure was first modeled dichotomously as not fed or fed. To further explore associations between feeding modality and our outcome, we then modeled feeding exposure categorically as not fed (reference), mixed (oral and tube) feeding, or exclusive oral feeding throughout HFNC.

After constructing a set of unadjusted models, we then adjusted the models for variables having independent (bivariate P < .10) associations with time to discharge: age, unit of HFNC initiation, highest respiratory support required before HFNC, and HFNC duration. Finally, to attempt to account for residual confounding from latent constructs, we also created a set of propensity-weighted Cox proportional hazards models. Propensity weights¹⁸ reflecting the probability of being fed or never being fed during HFNC were created using logistic regression with predictors we hypothesized a priori that may have influenced the clinical decision to feed during HFNC: age, day of illness on admission, prematurity, PIM2 ROM score, respiratory rate, oxygen saturation and blood gas acquisition at HFNC initiation, and highest respiratory support required before HFNC. All analyses were conducted using STATA 14.0 (StataCorp, College Station, Texas), and adjusted hazard ratios (aHR) with 95% confidence intervals (95% CIs) were reported.

Patients (n = 123) had a mean age of 7.3 months (standard deviation [SD] 7.1) and presented on day of illness 4.8 (SD 2.3). Prior to HFNC, 10% required higher respiratory support (3% mechanical ventilation). Former preterm children were 12% of the overall sample.

During HFNC, 37% of patients were never fed, 41% were exclusively orally fed, and 23% had tube or mixed oral and tube feedings (Table 1 and Appendix 2). Children who were not fed were older, but groups were otherwise similar in terms of gender, race/ethnicity, passive smoke exposure, day of illness, unit of HFNC initiation, respiratory support required prior to HFNC, and respiratory rate at HFNC initiation.

This observational study found that being fed during HFNC was associated with shorter time to discharge after HFNC support was completed. Exclusive oral feeding was associated with the shortest time to discharge, and these results were consistent across a variety of analytical approaches. Adverse events were rare and occurred in both fed and unfed children.

These findings advance research on relationships between nutrition and bronchiolitis outcomes. Studies of general care patients with bronchiolitis without HFNC have observed associations between poor nutrition and prolonged LOS.^19,20 Two previous studies of patients receiving HFNC therapy for bronchiolitis concluded that frequent interruption¹¹ and later initiation of enteral nutrition¹⁰ during ICU stay was associated with longer time to discharge.¹¹ To our knowledge, this is the first study of patients with bronchiolitis treated with HFNC in both general care and ICU settings that compared outcomes according to whether children were fed during HFNC therapy. Our results extend previous work demonstrating that delays in feeding may be associated with longer LOS.

Children fed while receiving HFNC for bronchiolitis may have shorter time to discharge than those who are not fed. Feeding-related adverse events were rare regardless of the feeding method. Controlled prospective studies addressing residual confounding are needed to justify a change in the current practice.

Acknowledgments

The authors would like to acknowledge the valuable feedback on earlier drafts from members of the University of Wisconsin Division of Pediatric Hospital Medicine CREATE writing group.

Bronchiolitis is the most common cause of nonbirth hospitalization in children in the United States less than one year of age.¹ For children with severe bronchiolitis, high-flow nasal cannula (HFNC) is increasingly used^2-4 to reduce work of breathing and prevent the need for further escalation of ventilatory support.^5,6 Although previous studies suggest that enteral feeding is recommended in the management of patients hospitalized with bronchiolitis,^7-9 limited evidence exists to guide feeding practices for patients receiving HFNC support.^5,10,11

Respiratory support with HFNC has been associated with prolonged periods without enteral hydration/nutrition (ie, nil per os [NPO])¹² primarily due to anticipation of further escalation of respiratory support or concern for increased risk of aspiration. The majority of patients with bronchiolitis managed with HFNC, however, do not require escalation of care.^5,13 When feeding is attempted during HFNC support, it is frequently interrupted.⁵ Moreover, keeping all children NPO when receiving HFNC may be associated with weight loss and longer length of stay (LOS).^12,14 Two small studies found that children admitted to the intensive care unit who received HFNC support for bronchiolitis did not have increased rates of emesis, worsening respiratory distress or aspiration pneumonia when enterally fed.^10,11 However, no comparison of adverse events or LOS has been made between patients who were fed and those who were not fed during HFNC therapy, and previous studies have included only patients who have received HFNC in the intensive care setting.

Supporting safe feeding early in hospitalizations for bronchiolitis may facilitate expedited clinical improvement and discharge. As part of an ongoing bronchiolitis quality improvement initiative at our hospital, we sought to characterize feeding practices during HFNC therapy and assess whether feeding exposure was associated with (1) time to discharge after HFNC or (2) feeding-related adverse events. We hypothesized that feeding during HFNC therapy would be associated with a shorter time to discharge after HFNC cessation.

Study Design, Setting, Participants

This was a retrospective cohort study of patients aged 1-24 months receiving HFNC support for respiratory failure due to bronchiolitis at an academic children’s hospital between January 1, 2015 and March 1, 2017. Our institution has had a clinical practice guideline, associated order set, and respiratory therapy protocol for general care patients with bronchiolitis since 2009. Patients with bronchiolitis who were weaning HFNC have been cared for in both the intensive and general care settings since 2013. A formal process for initiation of HFNC on general care units was instituted in January of 2017. During the study period, no patients with HFNC support for bronchiolitis had all their care entirely outside the intensive care unit at our institution. However, initiation and subsequent use of HFNC may have occurred in either the intensive care or general care setting. No specific guidance for feeding during HFNC existed during this period.

Patients were identified using the Virtual PICU Systems database, (VPS LLC, myvps.org, Los Angeles, California) and, by definition, all patients received at least some of their care in the intensive care unit. Patients with comorbid conditions of prematurity (<35 weeks) and those with cardiopulmonary, neuromuscular, and genetic diseases were included. Patients with preexisting dysphagia, defined as ongoing outpatient speech therapy for swallowing concerns, an admission diagnosis of aspiration pneumonia or on home respiratory support, were excluded. Children (n = 7) were excluded if they had more than one period of HFNC during admission. This study was determined to be exempt by the University of Wisconsin School of Medicine and Public Health’s Institutional Review Board.

Data Collection and Study Variables

The following variables were collected from VPS administrative data: patient gender, age, admission and discharge date and time, type and total hours of respiratory support, intensive care admission, and LOS (in hours). Additional demographic, clinical, and feeding exposure variables were abstracted manually from the electronic medical record (Epic, Verona, Wisconsin) using a structured data collection tool and stored in REDCap (Research Electronic Data Capture)¹⁵ including prematurity, race/ethnicity, insurance status, primary language, and passive tobacco smoke exposure. Clinical variables included duration of illness (days) at the time of admission, unit of HFNC initiation (emergency department, general care, intensive care, respiratory rate and oxygen saturation at HFNC initiation (<90%, 91%-92%, or >92%), acquisition of blood gas at HFNC admission, duration of time on HFNC (hours) and need for intubation or noninvasive ventilation prior to HFNC. The Pediatric Index of Mortality 2 Risk of Mortality (PIM 2 ROM)¹⁶ was used to estimate the severity of illness. The PIM2ROM uses clinical variables (systolic blood pressure, fixed pupils, measure of hypoxia using PaO₂/FiO₂ ratio, base excess, mechanical ventilation, elective admission, recovery from surgery, cardiac bypass, high-risk diagnosis, low-risk diagnosis) collected at the time of intensive care admission to generate a score that predicts the risk of mortality for an individual patient.¹⁷

Feeding exposures were documented in three-hour intervals from HFNC initiation to completion using a structured protocol. At each interval the following feeding information was abstracted from a review of nursing and physician documentation and relevant clinical flowsheets: presence or absence of feeding during the interval, route of feeding (oral, nasogastric [NG] or nasojejunal [NJ]). Feeding exposure was categorized a priori as fed at any point during HFNC (vs not fed at any point). Fed children were further characterized as (1) mixed feeding consisting of oral and tube feeds (NG or NJ) or (2) exclusive oral feeding throughout HFNC support (Appendix 1).

The primary outcome was the number of hours to discharge from HFNC cessation. Secondary outcomes were time to discharge from HFNC initiation, all-cause readmissions within seven days of discharge, and potential feeding-related adverse events. Potential adverse events included: (1) aspiration, defined as initiation of antibiotic AND either chest radiograph official interpreted as evidence for aspiration and/or documented concern for aspiration from the treating physician, or (2) intubation after feeding during HFNC.

Statistical Analysis

Descriptive statistics evaluated differences in demographics and clinical variables for feeding exposure groups. We used chi-squared tests for differences in proportions and t-tests or Wilcoxon Rank-Sum tests for differences in means or medians for continuous variables, respectively. Associations between feeding exposure during HFNC and time to discharge (measured in hours) after HFNC completion were modeled with Cox proportional hazards regression. Using this approach, hazard ratios (HR)>1 indicate a higher hazard (rate) of discharge for children with a feeding exposure than for children without the exposure. For example, a hazard ratio equal to two indicates that the exposed population is discharged at twice the rate per unit time as the nonexposed population. Death or censoring events did not occur. Feeding exposure was first modeled dichotomously as not fed or fed. To further explore associations between feeding modality and our outcome, we then modeled feeding exposure categorically as not fed (reference), mixed (oral and tube) feeding, or exclusive oral feeding throughout HFNC.

After constructing a set of unadjusted models, we then adjusted the models for variables having independent (bivariate P < .10) associations with time to discharge: age, unit of HFNC initiation, highest respiratory support required before HFNC, and HFNC duration. Finally, to attempt to account for residual confounding from latent constructs, we also created a set of propensity-weighted Cox proportional hazards models. Propensity weights¹⁸ reflecting the probability of being fed or never being fed during HFNC were created using logistic regression with predictors we hypothesized a priori that may have influenced the clinical decision to feed during HFNC: age, day of illness on admission, prematurity, PIM2 ROM score, respiratory rate, oxygen saturation and blood gas acquisition at HFNC initiation, and highest respiratory support required before HFNC. All analyses were conducted using STATA 14.0 (StataCorp, College Station, Texas), and adjusted hazard ratios (aHR) with 95% confidence intervals (95% CIs) were reported.

Patients (n = 123) had a mean age of 7.3 months (standard deviation [SD] 7.1) and presented on day of illness 4.8 (SD 2.3). Prior to HFNC, 10% required higher respiratory support (3% mechanical ventilation). Former preterm children were 12% of the overall sample.

During HFNC, 37% of patients were never fed, 41% were exclusively orally fed, and 23% had tube or mixed oral and tube feedings (Table 1 and Appendix 2). Children who were not fed were older, but groups were otherwise similar in terms of gender, race/ethnicity, passive smoke exposure, day of illness, unit of HFNC initiation, respiratory support required prior to HFNC, and respiratory rate at HFNC initiation.

This observational study found that being fed during HFNC was associated with shorter time to discharge after HFNC support was completed. Exclusive oral feeding was associated with the shortest time to discharge, and these results were consistent across a variety of analytical approaches. Adverse events were rare and occurred in both fed and unfed children.

These findings advance research on relationships between nutrition and bronchiolitis outcomes. Studies of general care patients with bronchiolitis without HFNC have observed associations between poor nutrition and prolonged LOS.^19,20 Two previous studies of patients receiving HFNC therapy for bronchiolitis concluded that frequent interruption¹¹ and later initiation of enteral nutrition¹⁰ during ICU stay was associated with longer time to discharge.¹¹ To our knowledge, this is the first study of patients with bronchiolitis treated with HFNC in both general care and ICU settings that compared outcomes according to whether children were fed during HFNC therapy. Our results extend previous work demonstrating that delays in feeding may be associated with longer LOS.

Children fed while receiving HFNC for bronchiolitis may have shorter time to discharge than those who are not fed. Feeding-related adverse events were rare regardless of the feeding method. Controlled prospective studies addressing residual confounding are needed to justify a change in the current practice.

Acknowledgments

The authors would like to acknowledge the valuable feedback on earlier drafts from members of the University of Wisconsin Division of Pediatric Hospital Medicine CREATE writing group.

Under the Hospital Readmission Reduction Program (HRRP), hospitals with higher than expected readmissions for select conditions receive a financial penalty. In 2017, hospitals were penalized a total of $528 million.^1,2 In an effort to deter readmissions, hospitals have focused on the transition from inpatient to outpatient care with particular emphasis on timely follow-up with a primary care physician (PCP).^3-7 Medicare has also introduced transitional care codes, which reimburse physicians for follow-up care after a hospitalization.

Most observational studies have found an association among patients discharged from the hospital between early follow-up with a PCP and fewer readmissions. One study found that patients without timely PCP follow-up after hospitalization on medical wards had a 10-fold increase in the likelihood of readmission.⁵ This association between early PCP follow-up and readmissions has been echoed in studies of all general admissions,⁵ as well as hospitalizations specific to heart failure,^7,8 chronic obstructive pulmonary disease,³ high-risk surgery,⁹ and sickle cell disease.¹⁰ One potential concern with this prior literature is that unmeasured patient characteristics might be confounders; for example, patients with more social support may be both more likely to have follow-up visits and less likely to have readmissions. Also, there are several studies showing no association between early PCP appointments and readmission rates.^6,9,11-13

Several prior interventional studies to improve the care transition from hospital to outpatient care have successfully deterred readmissions.¹⁴ In these trials, facilitating early PCP follow-up is just one component of a larger intervention,¹⁵ and a systemic review noted that the interventions were heterogeneous and often consisted of multiple complex steps.⁶ It is less clear whether interventions to facilitate early PCP follow-up alone are successful.

In this study, we evaluated the impact of an intervention that focused on facilitating early follow-up of PCPs. We assessed the impact of this intervention on the likelihood of having a PCP appointment within seven days of discharge and being readmitted within 30 days of discharge.

Postdischarge Appointment Service

In the fall of 2009, Beth Israel Deaconess introduced a postdischarge appointment intervention to facilitate follow-up with PCPs and specialty physicians after discharge from the hospital. Within the provider order entry system, attending and resident physicians enter a discharge appointment request for specified providers within and outside of the medical center and a specified time period. For example, a physician may enter a request to schedule a PCP appointment within 2-3, 4-8, 9-15, 16-30, or >30 days of discharge. Physicians are asked to submit this request on the day of discharge. The request is transmitted to dedicated staff (four full-time administrative staff and four part-time registered nurses) who verify the PCP, process the orders, and call the relevant practices to book the appointments. The date and time of the follow-up appointments are set without input from the patient. The details of the appointment, location, phone number of the clinic, and any other relevant instructions are automatically entered into the discharge instructions and discharge summary. The service is available Monday through Friday, and the turnaround on appointment creation is typically within one to three hours of the request. For patients who do not have a PCP or want to switch their PCP, the discharging physician can request a new PCP within the health system, and the service will schedule an appointment in this new PCP’s practice. Anecdotally, physicians are more likely to order the postdischarge appointment service for patients with more complex illnesses and longer lengths of stay and for those who come from underserved populations, as they perceive that it is more important for the patient to have this follow-up appointment, and/or the patient may have a harder time navigating the system and scheduling an appointment. Because of funding limitations, the hospital limited the intervention to hospitalizations on the general medicine and cardiology services. It was expanded in late 2011 to include the trauma surgery service.

Study Population

We conducted a retrospective, cohort study at Beth Israel Deaconess Medical Center, a tertiary care hospital, using data derived from electronic health records for all hospitalizations from September 2008 to October 2015. At this medical center, the vast majority of patients on the general medicine service are cared for by hospitalists and not their PCPs. We focused on patients 18 years of age or older discharged home and excluded hospitalizations where the patient died, was transferred to another hospital, or was discharged to a skilled nursing facility or inpatient rehabilitation hospital. We excluded patients who were kept under observation in the emergency department (ED), but our data did include patients cared for on a hospital ward under observation. To measure whether patients attended a follow-up visit, we used internal scheduling data and therefore only included hospitalizations for patients with a PCP affiliated with the Beth Israel Deaconess medical system. This includes patients previously without a PCP whose first PCP appointment was after discharge. Finally, we limited our sample to hospitalizations on the general medicine and cardiology services because, as previously discussed, these are the services where the intervention was available. To address transfers within the hospital from one service to another, we categorized hospitalizations by the service on the date of discharge.

Outcomes

The primary outcomes of this study were kept PCP follow-up visits within seven days and readmission within 30 days of discharge. We focused on PCP visits within seven days, as this has been the measure used in prior research,^5,7 but conducted a sensitivity analysis of PCP follow-up within 14 days. No-shows for the scheduled follow-up PCP appointments were not included. We focused on readmissions within 30 days of discharge, given this is the measure used in the HRRP,¹⁶ but conducted a sensitivity analysis of 14 days. Secondary outcomes included ED revisit within the 30 days. Given the data available, we only observed physician visits and hospitalizations that occurred within the Beth Israel Deaconess system.

Analyses

We conducted two analyses to assess whether the implementation of the postdischarge appointment service was associated with an increase in PCP follow-up and a decrease in the readmission rate.

In the first analysis, we focused only on hospitalizations from the medical and cardiology services during the postintervention period between January 2011 and September 2015 (n = 17,582). We compared the PCP follow-up rate and the readmission rate among hospitalizations where the postdischarge appointment service was used versus those where it was not used. We used a multivariable logistic regression, and the covariates included in the model were age, gender, hospital length of stay, and diagnosis-related group (DRG) cost weight. The DRG cost weight captures the average resources used to treat Medicare patients’ hospitalizations within a given DRG category and was used as a surrogate marker for the complexity of hospitalization.¹⁷ Instead of presenting odds ratios, we used predictive margins to generate adjusted percentage point estimates of the differences in our outcomes associated with the use of the postdischarge appointment service.¹⁸

In our second analysis, we examined the period before and after the introduction of the postdischarge appointment service (September 2008 through October 2015, n = 20,918). Among these hospitalizations, we conducted an instrumental variable analysis to address the concern that there are unmeasured differences between those patients who receive the discharge appointment service and those who do not. Instrumental variable analyses are used to estimate causal relationships in observational studies.¹⁹ A valid instrument is associated with the explanatory variable (use of the postdischarge appointment service) but has no independent effect on the outcomes (follow-up visits, readmissions). In this analysis, our set of instruments was the day of the week of admission (indicator variable for each day) interacted with the time period (pre- vs postintervention period).

This instrumental variable exploits the fact that the postdischarge appointment service was only available on weekdays and that physicians are asked to only submit the order for follow-up appointments on the day of discharge. We focused on the day of the week of admission (versus discharge) because of concerns that patients with more complicated hospital courses might be kept in the hospital over the weekend (eg, to facilitate testing available only on weekdays or to consult with regular physicians only available on weekdays). This would create a relationship between the day of discharge and the outcomes (follow-up visits, readmissions). The day of admission is less likely to be impacted by this bias. Given concerns that admissions on different days of the week might be different, our instrument is the day of the week interacted with the time period. Therefore, to create bias, there must be a systematic change in the nature of admissions on a given day of the week during this time period. We provide more details on this analysis, testing of the instrument, and results in the Appendix.

Analyses were conducted in Stata, version 14.2 (StataCorp LP, College Station, Texas). Statistical testing was two-sided, with a significance level of 0.05, and the project was judged exempt by the Committee on Clinical Investigations for Beth Israel Deaconess Medical Center.

Overall, there were 17,582 hospitalizations on the medicine and cardiology services following implementation of the postdischarge appointment service. The use of the postdischarge appointment service rose rapidly after it was introduced (Figure) and then plateaued at roughly 50%. Of the hospitalizations where the postdischarge appointment service was used, the physician requested a new PCP for 1.2% of the patients. Among hospitalizations where the intervention was used, the average age was 65.5 years, 55.7% were female, the length of stay was 3.52 days, the DRG cost weight was 1.26 and 20.4% were patients on the cardiology service. Characteristics were similar in hospitalizations where the services were not used (Table 1).

Multivariable Logistic Regression

In this analysis, we focused on the 17,582 hospitalizations from January 2011 to September 2015 on the general medicine and cardiology services that occurred after the postdischarge appointment service was introduced. Among these hospitalizations, the postdischarge appointment service was used in 51.8% of discharges.

In an unadjusted analysis, patients discharged using the tool had higher rates of seven-day PCP follow-up (60.2% vs 29.2%, P < .001) and lower 30-day readmission rates (14.7% vs 16.7%; P < .001) than those who were not (Table 2). There was no significant difference in 30-day ED revisit between hospitalizations with and without use of the postdischarge appointment service (22.3% vs 23.1%; P = .23).

Instrumental Variable Analysis

In our instrumental variable analysis, we used all hospitalizations both before and after the introduction of the intervention. In this analysis, we estimate that use of the postdischarge appointment service increases the probability of visiting a PCP within seven days by 33.4 percentage points (95% CI: 7.9%, 58.9%; P = .01) (Table 3). The use of the postdischarge appointment was associated with a 2.5 percentage point (95% CI: −22.0%, 17.1%; P = .80) reduction in readmissions and a 4.8 percentage point (95% CI; −27.5%, 17.9%; P = .68) reduction in an ED visit within 30 days (Table 3). Neither of these differences were statistically significant with wide confidence intervals.

In sensitivity analyses, we obtained similar results when we considered PCP visits and readmissions within 14 days.

The hospital introduced the postdischarge appointment service to facilitate postdischarge appointments and to deter readmissions. In our analyses the use of the postdischarge appointment service was associated with a substantial 30 percentage point increase in the likelihood of a PCP follow-up visit within seven days after hospital discharge. There was a roughly 2% reduction in 30-day readmissions, but this difference was not consistently statistically significant across our analyses. Together, our evaluation implies that this type of intervention may make it much easier for patients to attend a PCP appointment, but scheduling an appointment alone may have a modest impact on deterring a readmission.

Our findings are inconsistent with prior studies that described a strong association between early PCP follow-up and readmissions. However, our results were consistent with research where follow-up visits were not clearly protective against readmissions.²⁰ One potential explanation of the discrepant findings is that there are unmeasured socioeconomic differences between patients who have a PCP follow-up appointment and those who do not.

We advance the literature by studying an intervention focused only on increasing early PCP follow-up. Most successful readmission programs that have been studied in randomized, controlled trials take a multipronged approach, including transitional care management with dedicated staff and medication reconciliation.^{3-7,9,15,21-23} For example, Coleman and colleagues randomized 750 hospitalized patients to a care-transitions intervention, which led to a substantial decrease in readmissions.¹⁵ Their care-transitions intervention included four components: (1) timely PCP or specialist follow-up, (2) educating patients on how best to take their medications, (3) a patient-centered record that allowed them to track their own disease and care, and (4) disease-specific patient education. The relative importance of each of these components in deterring readmissions is unclear. Instead of this multipronged strategy, we focused on a single component—timely follow-up. Together, our study and these prior studies are broadly consistent with a meta-analysis that suggests that transitional care programs with a narrow focus are less successful at reducing readmissions.²⁴ Facilitating early PCP follow-up alone is not a panacea and can be undermined by the incomplete or inexistent transmission of the discharge paperwork.^{25, 26} Moreover, the impact of interventions may only be seen among the highest-risk populations, and ongoing work by others seeks to identify these patients.²⁷

Regardless of the impact on readmissions, it is important to acknowledge that early PCP follow-up offers many potential benefits. Continuing to evaluate and treat new diagnoses, adjusting and reconciling medications, reconnecting with outpatient providers, capturing new incidental findings, and ensuring stability through regular follow-up are just a few of the potential benefits. We believe the dramatic increase observed in PCP follow-up reflects the administrative complexity required for a patient to call their PCP’s office and to schedule a follow-up appointment soon after they are discharged from the hospital. Our study implies that simply requesting that a patient call their PCP to schedule a timely appointment is often impossible, and this may be particularly true for those who need to obtain a new PCP.

Our study has many limitations. The study was limited to a single academic center, and the intervention was limited to patients cared for by the general medicine and cardiology services. Our multivariable regression analysis comparing outcomes among patients where the postdischarge appointment service was used and not used may be biased by unmeasured differences in these patients. We attempted to address this limitation by exploiting the fact that the intervention was only available on weekdays through an instrumental variable analysis, but the instrument we used itself is subject to bias. Also, in the instrumental variable analysis, our estimates were imprecise and therefore not powered to identify smaller but still clinically important reductions in readmissions. Given the data limitations, we could not compare the no-show rates among appointments made by the discharge appointment service versus those made by patients. Finally, we were only able to observe follow-up visits and hospitalizations within the health system, and it is possible that our results were biased by patients preferentially going to other hospitals for readmission.

In summary, we found that the introduction of a postdischarge appointment service resulted in substantially increased rates of early PCP follow-up but less clear benefits in preventing readmissions.

Under the Hospital Readmission Reduction Program (HRRP), hospitals with higher than expected readmissions for select conditions receive a financial penalty. In 2017, hospitals were penalized a total of $528 million.^1,2 In an effort to deter readmissions, hospitals have focused on the transition from inpatient to outpatient care with particular emphasis on timely follow-up with a primary care physician (PCP).^3-7 Medicare has also introduced transitional care codes, which reimburse physicians for follow-up care after a hospitalization.

Most observational studies have found an association among patients discharged from the hospital between early follow-up with a PCP and fewer readmissions. One study found that patients without timely PCP follow-up after hospitalization on medical wards had a 10-fold increase in the likelihood of readmission.⁵ This association between early PCP follow-up and readmissions has been echoed in studies of all general admissions,⁵ as well as hospitalizations specific to heart failure,^7,8 chronic obstructive pulmonary disease,³ high-risk surgery,⁹ and sickle cell disease.¹⁰ One potential concern with this prior literature is that unmeasured patient characteristics might be confounders; for example, patients with more social support may be both more likely to have follow-up visits and less likely to have readmissions. Also, there are several studies showing no association between early PCP appointments and readmission rates.^6,9,11-13

Several prior interventional studies to improve the care transition from hospital to outpatient care have successfully deterred readmissions.¹⁴ In these trials, facilitating early PCP follow-up is just one component of a larger intervention,¹⁵ and a systemic review noted that the interventions were heterogeneous and often consisted of multiple complex steps.⁶ It is less clear whether interventions to facilitate early PCP follow-up alone are successful.

In this study, we evaluated the impact of an intervention that focused on facilitating early follow-up of PCPs. We assessed the impact of this intervention on the likelihood of having a PCP appointment within seven days of discharge and being readmitted within 30 days of discharge.

Postdischarge Appointment Service

In the fall of 2009, Beth Israel Deaconess introduced a postdischarge appointment intervention to facilitate follow-up with PCPs and specialty physicians after discharge from the hospital. Within the provider order entry system, attending and resident physicians enter a discharge appointment request for specified providers within and outside of the medical center and a specified time period. For example, a physician may enter a request to schedule a PCP appointment within 2-3, 4-8, 9-15, 16-30, or >30 days of discharge. Physicians are asked to submit this request on the day of discharge. The request is transmitted to dedicated staff (four full-time administrative staff and four part-time registered nurses) who verify the PCP, process the orders, and call the relevant practices to book the appointments. The date and time of the follow-up appointments are set without input from the patient. The details of the appointment, location, phone number of the clinic, and any other relevant instructions are automatically entered into the discharge instructions and discharge summary. The service is available Monday through Friday, and the turnaround on appointment creation is typically within one to three hours of the request. For patients who do not have a PCP or want to switch their PCP, the discharging physician can request a new PCP within the health system, and the service will schedule an appointment in this new PCP’s practice. Anecdotally, physicians are more likely to order the postdischarge appointment service for patients with more complex illnesses and longer lengths of stay and for those who come from underserved populations, as they perceive that it is more important for the patient to have this follow-up appointment, and/or the patient may have a harder time navigating the system and scheduling an appointment. Because of funding limitations, the hospital limited the intervention to hospitalizations on the general medicine and cardiology services. It was expanded in late 2011 to include the trauma surgery service.

Study Population

We conducted a retrospective, cohort study at Beth Israel Deaconess Medical Center, a tertiary care hospital, using data derived from electronic health records for all hospitalizations from September 2008 to October 2015. At this medical center, the vast majority of patients on the general medicine service are cared for by hospitalists and not their PCPs. We focused on patients 18 years of age or older discharged home and excluded hospitalizations where the patient died, was transferred to another hospital, or was discharged to a skilled nursing facility or inpatient rehabilitation hospital. We excluded patients who were kept under observation in the emergency department (ED), but our data did include patients cared for on a hospital ward under observation. To measure whether patients attended a follow-up visit, we used internal scheduling data and therefore only included hospitalizations for patients with a PCP affiliated with the Beth Israel Deaconess medical system. This includes patients previously without a PCP whose first PCP appointment was after discharge. Finally, we limited our sample to hospitalizations on the general medicine and cardiology services because, as previously discussed, these are the services where the intervention was available. To address transfers within the hospital from one service to another, we categorized hospitalizations by the service on the date of discharge.

Outcomes

The primary outcomes of this study were kept PCP follow-up visits within seven days and readmission within 30 days of discharge. We focused on PCP visits within seven days, as this has been the measure used in prior research,^5,7 but conducted a sensitivity analysis of PCP follow-up within 14 days. No-shows for the scheduled follow-up PCP appointments were not included. We focused on readmissions within 30 days of discharge, given this is the measure used in the HRRP,¹⁶ but conducted a sensitivity analysis of 14 days. Secondary outcomes included ED revisit within the 30 days. Given the data available, we only observed physician visits and hospitalizations that occurred within the Beth Israel Deaconess system.

Analyses

We conducted two analyses to assess whether the implementation of the postdischarge appointment service was associated with an increase in PCP follow-up and a decrease in the readmission rate.

In the first analysis, we focused only on hospitalizations from the medical and cardiology services during the postintervention period between January 2011 and September 2015 (n = 17,582). We compared the PCP follow-up rate and the readmission rate among hospitalizations where the postdischarge appointment service was used versus those where it was not used. We used a multivariable logistic regression, and the covariates included in the model were age, gender, hospital length of stay, and diagnosis-related group (DRG) cost weight. The DRG cost weight captures the average resources used to treat Medicare patients’ hospitalizations within a given DRG category and was used as a surrogate marker for the complexity of hospitalization.¹⁷ Instead of presenting odds ratios, we used predictive margins to generate adjusted percentage point estimates of the differences in our outcomes associated with the use of the postdischarge appointment service.¹⁸

In our second analysis, we examined the period before and after the introduction of the postdischarge appointment service (September 2008 through October 2015, n = 20,918). Among these hospitalizations, we conducted an instrumental variable analysis to address the concern that there are unmeasured differences between those patients who receive the discharge appointment service and those who do not. Instrumental variable analyses are used to estimate causal relationships in observational studies.¹⁹ A valid instrument is associated with the explanatory variable (use of the postdischarge appointment service) but has no independent effect on the outcomes (follow-up visits, readmissions). In this analysis, our set of instruments was the day of the week of admission (indicator variable for each day) interacted with the time period (pre- vs postintervention period).

This instrumental variable exploits the fact that the postdischarge appointment service was only available on weekdays and that physicians are asked to only submit the order for follow-up appointments on the day of discharge. We focused on the day of the week of admission (versus discharge) because of concerns that patients with more complicated hospital courses might be kept in the hospital over the weekend (eg, to facilitate testing available only on weekdays or to consult with regular physicians only available on weekdays). This would create a relationship between the day of discharge and the outcomes (follow-up visits, readmissions). The day of admission is less likely to be impacted by this bias. Given concerns that admissions on different days of the week might be different, our instrument is the day of the week interacted with the time period. Therefore, to create bias, there must be a systematic change in the nature of admissions on a given day of the week during this time period. We provide more details on this analysis, testing of the instrument, and results in the Appendix.

Analyses were conducted in Stata, version 14.2 (StataCorp LP, College Station, Texas). Statistical testing was two-sided, with a significance level of 0.05, and the project was judged exempt by the Committee on Clinical Investigations for Beth Israel Deaconess Medical Center.

Overall, there were 17,582 hospitalizations on the medicine and cardiology services following implementation of the postdischarge appointment service. The use of the postdischarge appointment service rose rapidly after it was introduced (Figure) and then plateaued at roughly 50%. Of the hospitalizations where the postdischarge appointment service was used, the physician requested a new PCP for 1.2% of the patients. Among hospitalizations where the intervention was used, the average age was 65.5 years, 55.7% were female, the length of stay was 3.52 days, the DRG cost weight was 1.26 and 20.4% were patients on the cardiology service. Characteristics were similar in hospitalizations where the services were not used (Table 1).

Multivariable Logistic Regression

In this analysis, we focused on the 17,582 hospitalizations from January 2011 to September 2015 on the general medicine and cardiology services that occurred after the postdischarge appointment service was introduced. Among these hospitalizations, the postdischarge appointment service was used in 51.8% of discharges.

In an unadjusted analysis, patients discharged using the tool had higher rates of seven-day PCP follow-up (60.2% vs 29.2%, P < .001) and lower 30-day readmission rates (14.7% vs 16.7%; P < .001) than those who were not (Table 2). There was no significant difference in 30-day ED revisit between hospitalizations with and without use of the postdischarge appointment service (22.3% vs 23.1%; P = .23).

Instrumental Variable Analysis

In our instrumental variable analysis, we used all hospitalizations both before and after the introduction of the intervention. In this analysis, we estimate that use of the postdischarge appointment service increases the probability of visiting a PCP within seven days by 33.4 percentage points (95% CI: 7.9%, 58.9%; P = .01) (Table 3). The use of the postdischarge appointment was associated with a 2.5 percentage point (95% CI: −22.0%, 17.1%; P = .80) reduction in readmissions and a 4.8 percentage point (95% CI; −27.5%, 17.9%; P = .68) reduction in an ED visit within 30 days (Table 3). Neither of these differences were statistically significant with wide confidence intervals.

In sensitivity analyses, we obtained similar results when we considered PCP visits and readmissions within 14 days.

The hospital introduced the postdischarge appointment service to facilitate postdischarge appointments and to deter readmissions. In our analyses the use of the postdischarge appointment service was associated with a substantial 30 percentage point increase in the likelihood of a PCP follow-up visit within seven days after hospital discharge. There was a roughly 2% reduction in 30-day readmissions, but this difference was not consistently statistically significant across our analyses. Together, our evaluation implies that this type of intervention may make it much easier for patients to attend a PCP appointment, but scheduling an appointment alone may have a modest impact on deterring a readmission.

Our findings are inconsistent with prior studies that described a strong association between early PCP follow-up and readmissions. However, our results were consistent with research where follow-up visits were not clearly protective against readmissions.²⁰ One potential explanation of the discrepant findings is that there are unmeasured socioeconomic differences between patients who have a PCP follow-up appointment and those who do not.

We advance the literature by studying an intervention focused only on increasing early PCP follow-up. Most successful readmission programs that have been studied in randomized, controlled trials take a multipronged approach, including transitional care management with dedicated staff and medication reconciliation.^{3-7,9,15,21-23} For example, Coleman and colleagues randomized 750 hospitalized patients to a care-transitions intervention, which led to a substantial decrease in readmissions.¹⁵ Their care-transitions intervention included four components: (1) timely PCP or specialist follow-up, (2) educating patients on how best to take their medications, (3) a patient-centered record that allowed them to track their own disease and care, and (4) disease-specific patient education. The relative importance of each of these components in deterring readmissions is unclear. Instead of this multipronged strategy, we focused on a single component—timely follow-up. Together, our study and these prior studies are broadly consistent with a meta-analysis that suggests that transitional care programs with a narrow focus are less successful at reducing readmissions.²⁴ Facilitating early PCP follow-up alone is not a panacea and can be undermined by the incomplete or inexistent transmission of the discharge paperwork.^{25, 26} Moreover, the impact of interventions may only be seen among the highest-risk populations, and ongoing work by others seeks to identify these patients.²⁷

Regardless of the impact on readmissions, it is important to acknowledge that early PCP follow-up offers many potential benefits. Continuing to evaluate and treat new diagnoses, adjusting and reconciling medications, reconnecting with outpatient providers, capturing new incidental findings, and ensuring stability through regular follow-up are just a few of the potential benefits. We believe the dramatic increase observed in PCP follow-up reflects the administrative complexity required for a patient to call their PCP’s office and to schedule a follow-up appointment soon after they are discharged from the hospital. Our study implies that simply requesting that a patient call their PCP to schedule a timely appointment is often impossible, and this may be particularly true for those who need to obtain a new PCP.

Our study has many limitations. The study was limited to a single academic center, and the intervention was limited to patients cared for by the general medicine and cardiology services. Our multivariable regression analysis comparing outcomes among patients where the postdischarge appointment service was used and not used may be biased by unmeasured differences in these patients. We attempted to address this limitation by exploiting the fact that the intervention was only available on weekdays through an instrumental variable analysis, but the instrument we used itself is subject to bias. Also, in the instrumental variable analysis, our estimates were imprecise and therefore not powered to identify smaller but still clinically important reductions in readmissions. Given the data limitations, we could not compare the no-show rates among appointments made by the discharge appointment service versus those made by patients. Finally, we were only able to observe follow-up visits and hospitalizations within the health system, and it is possible that our results were biased by patients preferentially going to other hospitals for readmission.

In summary, we found that the introduction of a postdischarge appointment service resulted in substantially increased rates of early PCP follow-up but less clear benefits in preventing readmissions.

Under the Hospital Readmission Reduction Program (HRRP), hospitals with higher than expected readmissions for select conditions receive a financial penalty. In 2017, hospitals were penalized a total of $528 million.^1,2 In an effort to deter readmissions, hospitals have focused on the transition from inpatient to outpatient care with particular emphasis on timely follow-up with a primary care physician (PCP).^3-7 Medicare has also introduced transitional care codes, which reimburse physicians for follow-up care after a hospitalization.

Most observational studies have found an association among patients discharged from the hospital between early follow-up with a PCP and fewer readmissions. One study found that patients without timely PCP follow-up after hospitalization on medical wards had a 10-fold increase in the likelihood of readmission.⁵ This association between early PCP follow-up and readmissions has been echoed in studies of all general admissions,⁵ as well as hospitalizations specific to heart failure,^7,8 chronic obstructive pulmonary disease,³ high-risk surgery,⁹ and sickle cell disease.¹⁰ One potential concern with this prior literature is that unmeasured patient characteristics might be confounders; for example, patients with more social support may be both more likely to have follow-up visits and less likely to have readmissions. Also, there are several studies showing no association between early PCP appointments and readmission rates.^6,9,11-13

Several prior interventional studies to improve the care transition from hospital to outpatient care have successfully deterred readmissions.¹⁴ In these trials, facilitating early PCP follow-up is just one component of a larger intervention,¹⁵ and a systemic review noted that the interventions were heterogeneous and often consisted of multiple complex steps.⁶ It is less clear whether interventions to facilitate early PCP follow-up alone are successful.

In this study, we evaluated the impact of an intervention that focused on facilitating early follow-up of PCPs. We assessed the impact of this intervention on the likelihood of having a PCP appointment within seven days of discharge and being readmitted within 30 days of discharge.

Postdischarge Appointment Service

In the fall of 2009, Beth Israel Deaconess introduced a postdischarge appointment intervention to facilitate follow-up with PCPs and specialty physicians after discharge from the hospital. Within the provider order entry system, attending and resident physicians enter a discharge appointment request for specified providers within and outside of the medical center and a specified time period. For example, a physician may enter a request to schedule a PCP appointment within 2-3, 4-8, 9-15, 16-30, or >30 days of discharge. Physicians are asked to submit this request on the day of discharge. The request is transmitted to dedicated staff (four full-time administrative staff and four part-time registered nurses) who verify the PCP, process the orders, and call the relevant practices to book the appointments. The date and time of the follow-up appointments are set without input from the patient. The details of the appointment, location, phone number of the clinic, and any other relevant instructions are automatically entered into the discharge instructions and discharge summary. The service is available Monday through Friday, and the turnaround on appointment creation is typically within one to three hours of the request. For patients who do not have a PCP or want to switch their PCP, the discharging physician can request a new PCP within the health system, and the service will schedule an appointment in this new PCP’s practice. Anecdotally, physicians are more likely to order the postdischarge appointment service for patients with more complex illnesses and longer lengths of stay and for those who come from underserved populations, as they perceive that it is more important for the patient to have this follow-up appointment, and/or the patient may have a harder time navigating the system and scheduling an appointment. Because of funding limitations, the hospital limited the intervention to hospitalizations on the general medicine and cardiology services. It was expanded in late 2011 to include the trauma surgery service.

Study Population

We conducted a retrospective, cohort study at Beth Israel Deaconess Medical Center, a tertiary care hospital, using data derived from electronic health records for all hospitalizations from September 2008 to October 2015. At this medical center, the vast majority of patients on the general medicine service are cared for by hospitalists and not their PCPs. We focused on patients 18 years of age or older discharged home and excluded hospitalizations where the patient died, was transferred to another hospital, or was discharged to a skilled nursing facility or inpatient rehabilitation hospital. We excluded patients who were kept under observation in the emergency department (ED), but our data did include patients cared for on a hospital ward under observation. To measure whether patients attended a follow-up visit, we used internal scheduling data and therefore only included hospitalizations for patients with a PCP affiliated with the Beth Israel Deaconess medical system. This includes patients previously without a PCP whose first PCP appointment was after discharge. Finally, we limited our sample to hospitalizations on the general medicine and cardiology services because, as previously discussed, these are the services where the intervention was available. To address transfers within the hospital from one service to another, we categorized hospitalizations by the service on the date of discharge.

Outcomes

The primary outcomes of this study were kept PCP follow-up visits within seven days and readmission within 30 days of discharge. We focused on PCP visits within seven days, as this has been the measure used in prior research,^5,7 but conducted a sensitivity analysis of PCP follow-up within 14 days. No-shows for the scheduled follow-up PCP appointments were not included. We focused on readmissions within 30 days of discharge, given this is the measure used in the HRRP,¹⁶ but conducted a sensitivity analysis of 14 days. Secondary outcomes included ED revisit within the 30 days. Given the data available, we only observed physician visits and hospitalizations that occurred within the Beth Israel Deaconess system.

Analyses

We conducted two analyses to assess whether the implementation of the postdischarge appointment service was associated with an increase in PCP follow-up and a decrease in the readmission rate.

In the first analysis, we focused only on hospitalizations from the medical and cardiology services during the postintervention period between January 2011 and September 2015 (n = 17,582). We compared the PCP follow-up rate and the readmission rate among hospitalizations where the postdischarge appointment service was used versus those where it was not used. We used a multivariable logistic regression, and the covariates included in the model were age, gender, hospital length of stay, and diagnosis-related group (DRG) cost weight. The DRG cost weight captures the average resources used to treat Medicare patients’ hospitalizations within a given DRG category and was used as a surrogate marker for the complexity of hospitalization.¹⁷ Instead of presenting odds ratios, we used predictive margins to generate adjusted percentage point estimates of the differences in our outcomes associated with the use of the postdischarge appointment service.¹⁸

In our second analysis, we examined the period before and after the introduction of the postdischarge appointment service (September 2008 through October 2015, n = 20,918). Among these hospitalizations, we conducted an instrumental variable analysis to address the concern that there are unmeasured differences between those patients who receive the discharge appointment service and those who do not. Instrumental variable analyses are used to estimate causal relationships in observational studies.¹⁹ A valid instrument is associated with the explanatory variable (use of the postdischarge appointment service) but has no independent effect on the outcomes (follow-up visits, readmissions). In this analysis, our set of instruments was the day of the week of admission (indicator variable for each day) interacted with the time period (pre- vs postintervention period).

This instrumental variable exploits the fact that the postdischarge appointment service was only available on weekdays and that physicians are asked to only submit the order for follow-up appointments on the day of discharge. We focused on the day of the week of admission (versus discharge) because of concerns that patients with more complicated hospital courses might be kept in the hospital over the weekend (eg, to facilitate testing available only on weekdays or to consult with regular physicians only available on weekdays). This would create a relationship between the day of discharge and the outcomes (follow-up visits, readmissions). The day of admission is less likely to be impacted by this bias. Given concerns that admissions on different days of the week might be different, our instrument is the day of the week interacted with the time period. Therefore, to create bias, there must be a systematic change in the nature of admissions on a given day of the week during this time period. We provide more details on this analysis, testing of the instrument, and results in the Appendix.

Analyses were conducted in Stata, version 14.2 (StataCorp LP, College Station, Texas). Statistical testing was two-sided, with a significance level of 0.05, and the project was judged exempt by the Committee on Clinical Investigations for Beth Israel Deaconess Medical Center.

Overall, there were 17,582 hospitalizations on the medicine and cardiology services following implementation of the postdischarge appointment service. The use of the postdischarge appointment service rose rapidly after it was introduced (Figure) and then plateaued at roughly 50%. Of the hospitalizations where the postdischarge appointment service was used, the physician requested a new PCP for 1.2% of the patients. Among hospitalizations where the intervention was used, the average age was 65.5 years, 55.7% were female, the length of stay was 3.52 days, the DRG cost weight was 1.26 and 20.4% were patients on the cardiology service. Characteristics were similar in hospitalizations where the services were not used (Table 1).

Multivariable Logistic Regression

In this analysis, we focused on the 17,582 hospitalizations from January 2011 to September 2015 on the general medicine and cardiology services that occurred after the postdischarge appointment service was introduced. Among these hospitalizations, the postdischarge appointment service was used in 51.8% of discharges.

In an unadjusted analysis, patients discharged using the tool had higher rates of seven-day PCP follow-up (60.2% vs 29.2%, P < .001) and lower 30-day readmission rates (14.7% vs 16.7%; P < .001) than those who were not (Table 2). There was no significant difference in 30-day ED revisit between hospitalizations with and without use of the postdischarge appointment service (22.3% vs 23.1%; P = .23).

Instrumental Variable Analysis

In our instrumental variable analysis, we used all hospitalizations both before and after the introduction of the intervention. In this analysis, we estimate that use of the postdischarge appointment service increases the probability of visiting a PCP within seven days by 33.4 percentage points (95% CI: 7.9%, 58.9%; P = .01) (Table 3). The use of the postdischarge appointment was associated with a 2.5 percentage point (95% CI: −22.0%, 17.1%; P = .80) reduction in readmissions and a 4.8 percentage point (95% CI; −27.5%, 17.9%; P = .68) reduction in an ED visit within 30 days (Table 3). Neither of these differences were statistically significant with wide confidence intervals.

In sensitivity analyses, we obtained similar results when we considered PCP visits and readmissions within 14 days.

The hospital introduced the postdischarge appointment service to facilitate postdischarge appointments and to deter readmissions. In our analyses the use of the postdischarge appointment service was associated with a substantial 30 percentage point increase in the likelihood of a PCP follow-up visit within seven days after hospital discharge. There was a roughly 2% reduction in 30-day readmissions, but this difference was not consistently statistically significant across our analyses. Together, our evaluation implies that this type of intervention may make it much easier for patients to attend a PCP appointment, but scheduling an appointment alone may have a modest impact on deterring a readmission.

Our findings are inconsistent with prior studies that described a strong association between early PCP follow-up and readmissions. However, our results were consistent with research where follow-up visits were not clearly protective against readmissions.²⁰ One potential explanation of the discrepant findings is that there are unmeasured socioeconomic differences between patients who have a PCP follow-up appointment and those who do not.

We advance the literature by studying an intervention focused only on increasing early PCP follow-up. Most successful readmission programs that have been studied in randomized, controlled trials take a multipronged approach, including transitional care management with dedicated staff and medication reconciliation.^{3-7,9,15,21-23} For example, Coleman and colleagues randomized 750 hospitalized patients to a care-transitions intervention, which led to a substantial decrease in readmissions.¹⁵ Their care-transitions intervention included four components: (1) timely PCP or specialist follow-up, (2) educating patients on how best to take their medications, (3) a patient-centered record that allowed them to track their own disease and care, and (4) disease-specific patient education. The relative importance of each of these components in deterring readmissions is unclear. Instead of this multipronged strategy, we focused on a single component—timely follow-up. Together, our study and these prior studies are broadly consistent with a meta-analysis that suggests that transitional care programs with a narrow focus are less successful at reducing readmissions.²⁴ Facilitating early PCP follow-up alone is not a panacea and can be undermined by the incomplete or inexistent transmission of the discharge paperwork.^{25, 26} Moreover, the impact of interventions may only be seen among the highest-risk populations, and ongoing work by others seeks to identify these patients.²⁷

Regardless of the impact on readmissions, it is important to acknowledge that early PCP follow-up offers many potential benefits. Continuing to evaluate and treat new diagnoses, adjusting and reconciling medications, reconnecting with outpatient providers, capturing new incidental findings, and ensuring stability through regular follow-up are just a few of the potential benefits. We believe the dramatic increase observed in PCP follow-up reflects the administrative complexity required for a patient to call their PCP’s office and to schedule a follow-up appointment soon after they are discharged from the hospital. Our study implies that simply requesting that a patient call their PCP to schedule a timely appointment is often impossible, and this may be particularly true for those who need to obtain a new PCP.

Our study has many limitations. The study was limited to a single academic center, and the intervention was limited to patients cared for by the general medicine and cardiology services. Our multivariable regression analysis comparing outcomes among patients where the postdischarge appointment service was used and not used may be biased by unmeasured differences in these patients. We attempted to address this limitation by exploiting the fact that the intervention was only available on weekdays through an instrumental variable analysis, but the instrument we used itself is subject to bias. Also, in the instrumental variable analysis, our estimates were imprecise and therefore not powered to identify smaller but still clinically important reductions in readmissions. Given the data limitations, we could not compare the no-show rates among appointments made by the discharge appointment service versus those made by patients. Finally, we were only able to observe follow-up visits and hospitalizations within the health system, and it is possible that our results were biased by patients preferentially going to other hospitals for readmission.

In summary, we found that the introduction of a postdischarge appointment service resulted in substantially increased rates of early PCP follow-up but less clear benefits in preventing readmissions.

Accidental falls are among the most common incidents reported in hospitals, complicating approximately 2% of hospital stays.^1-3 Approximately 25% of falls in hospitalized patients result in injury, and 2% involve fractures.⁴ Substantial costs are associated with falls, including patient care costs associated with increased length of stay and liability.^5-7

Beginning October 1, 2008, the Centers for Medicare & Medicaid Services (CMS) stopped reimbursing hospitals for the additional care associated with eight hospital-acquired conditions (HACs), including serious fall-related injury, which were believed to be “reasonably preventable.”^8,9 Before this change, hospitals recovered the costs of these “never events” by assigning a higher level MS-DRG (Medicare Severity Diagnosis-Related Group) code for patients experiencing such an event. This is no longer allowed under the revised CMS Prospective Payment System rules.

Although the financial penalty for iatrogenic injury was modest, the payment change placed pressure on hospital staff to decrease falls, and some nurses reported changing practice to be more restrictive of patient mobility.¹⁰ Increased use of physical restraints is a potential unintended consequence of this rule change.¹¹ Restraints are known to cause agitation, delirium, decubiti, deconditioning, strangulation, and death.¹² Not surprisingly, use of restraints is discouraged in hospitals and is a CMS quality of care indicator.^13,14 Although there is no evidence that restraint use prevents patients from falling,^15,16 there is a perception among both health professionals and patients that restraints reduce the risk of falling, and they are often used as a “last resort” method of fall prevention.^17-19

The aim of this longitudinal study was to determine whether this payment change was associated with changes in short-, intermediate-, and long-term rates of falls, injurious falls, and physical restraint use in acute care hospitals. The CMS has included fall-related hip fracture in newer value-based purchasing programs by adding Patient Safety Indictor (PSI) 90 to both the HACs Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ in FY2015. However, the HACs Initiative remains the only Medicare value program that directly penalizes all injurious inpatient falls.

Study Units

As previously described,²² the National Database of Nursing Quality Indicators (NDNQI) is a data collection project initiated by the American Nurses Association (ANA). The NDNQI provides national comparative data at the unit and facility levels on nursing-sensitive indicators endorsed by the National Quality Forum. More than 2,000 hospitals voluntarily participate in the NDNQI, including virtually all ANA Magnet-recognized hospitals, and more than 90% of nursing units participate in the fall measures (NDNQI, personal communication). At the start of study data collection, the project was administered by the School of Nursing at the University of Kansas Medical Center. In 2014, the ownership of the NDNQI was transferred from the ANA to Press Ganey Associates, Inc. In addition to standardized data on unit, facility, and staffing characteristics, the NDNQI member hospitals can elect to submit monthly data on falls and quarterly data on physical restraint use prevalence.

We examined the data collected from adult medical, medical-surgical, and surgical units in United States acute care hospitals that elected to participate in the fall and physical restraint use data collection within the NDNQI for the 27 months before and the 87 months after the implementation of the CMS rule change. Eligible units contributed at least one fall and physical restraint use data point during both the 27 months preceding October 1, 2008, and the 87 months immediately after. The Institutional Review Board at the University of Kansas Medical Center reviewed and approved the study before its implementation.

Endpoints

Fall Events

The NDNQI defines a patient fall as an unplanned descent to the floor, regardless of whether the fall results in injury and regardless of whether the patient was assisted to the floor by a member of the hospital staff. Events in which a patient lands on a surface where one would not expect to find a patient (eg, on a mat next to a low bed) are also counted as falls.

Using internal data sources (eg, medical records, incident reports), participating hospitals report the number of inpatient falls each month to the NDNQI. We analyzed the falls data for the period July 1, 2006, through December 31, 2015. Thus, each unit could contribute 114 months (27 months before the rule change and 87 months after the rule change) of falls data.

Hospitals classify the injury level of each fall as none, minor (resulting in bruise, pain, abrasion, wound cleaning, or limb elevation, or in the use of ice, dressing, or topical medication), moderate (resulting in suturing, splinting, muscle or joint strain, or application of steri-strips or skin glue), major (resulting in surgery, casting, traction, any type of fracture, consultation for neurological or internal injury, or receipt of blood products for patients with coagulopathy), or death (resulting from injuries sustained from falling). For this study, a fall resulting in any injury (including minor) was considered as an injurious fall. The NDNQI data have been validated for falls and fall injury.^23,24

Based on patient counts from unit censuses and/or internal data on actual patient hours on the unit, hospitals also report to the NDNQI the monthly number of patient days for each unit for which falls data are reported. The NDNQI uses these data to calculate each unit’s total and injurious fall rate per 1,000 patient days.

Physical Restraint Use

The NDNQI follows the CMS definition of restraint, which is “any manual method, physical or mechanical device, material, or equipment that immobilizes or reduces the ability of a patient to move his or her arms, legs, body, or head freely”.¹³ The NDNQI restraint use data are collected quarterly. Participating hospitals choose one day each quarter to conduct a restraint use prevalence survey on participating units. On the selected day, designated RNs within these hospitals visually assess each patient on the unit for restraint use. Based on this survey, hospitals report to the NDNQI the total count of patients surveyed and whether each was restrained. For restrained patients, hospitals also report the type of restraint as limb, vest, or other (eg, four side rails, net beds, mitts not attached to the bed).

We analyzed the restraint use data for the period October 1, 2006, through December 31, 2015. Thus, 37 quarters of data (eight pre- and 29 postrule change) were available. For this study, we computed for each unit the quarterly proportion of surveyed patients who were physically restrained by dividing the total count of restrained patients (regardless of the type of restraint) on the day of the survey by the total count of surveyed patients.

Covariates

Unit- and facility-level covariates were included in several model specifications to determine whether patient or facility characteristics affected the results. The unit-level covariates included the type of nursing unit (medical, medical and surgical, or surgical), monthly rates of total nursing hours per patient day, and nursing skill mix (percent registered nurses/total nursing personnel). The three facility-level variables included urban–rural location (defined as metropolitan [located in an area containing an urban core with a population of at least 50,000], micropolitan [located in an area containing an urban core with a population of 10,000-49,999], or neither), bed size (<300 beds or ≥300 beds), and teaching status (academic health center, major teaching hospital, or nonteaching hospital).

Because larger, academically affiliated hospitals are overrepresented in the NDNQI, we conducted stratified analyses of these variables to explore how change in the rates of falls and restraint use in the entire sample might differ between hospitals according to bed size (<300 beds, ≥300 beds) and teaching status (nonteaching versus teaching and academic health center).

Statistical Methods

We compared the mean annual rates of change in falls, injurious falls, and physical restraint use prevalence during the two years before the HACs Initiative went into effect (October 2006-September 2008) with the mean annual rates of change following the implementation of the payment rule. Short-term (one-year) change was the slope from October 2008 to September 2009, intermediate-term (four-year) change was the slope from October 2008 to September 2012, and long-term (seven-year) change was the slope from October 2008 to September 2015.

Monthly rates of falls and injurious falls over the 114-month period were modeled using negative binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). We also fit the models that included the terms in the base model and facility-level covariates, unit-level covariates, both individually and combined. All models included terms for seasonality.

Quarterly prevalence rates of restraint use over the 37 quarters were modeled using beta-binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). Similar to the one specified for falls, models were also fitted that included facility- and unit-level covariates as described above.

To adjust for multiple comparisons of the three postimplementation slopes, all confidence intervals were Bonferroni corrected.

Nursing Units

We included nursing units with one or more months of falls data and one or more quarters of restraint use data before and after the rule change. Of the 11,117 nursing units that submitted data to the NDNQI, 2,862 units (983 medical, 1,219 medical-surgical, and 660 surgical) with the requisite demographic, falls, and restraint use data were considered for inclusion in the study. The characteristics of the nursing units (ie, the type of unit, total nursing hours per patient day, and nursing skill mix) and hospitals (ie, location, bed size, teaching status) included in the study were similar to those of the overall NDNQI member units.

Baseline Characteristics

In the first study month (July 2006), 1,941 sample nursing units reported 5,101 falls during 1,401,652 patient-days of observation. Of these, 1,502 (29%) resulted in injury (1,281 minor, 144 moderate, 75 major, and two deaths). Across falls, the median (interquartile range [IQR]) patient age was 70 (55-80) years, with males accounting for 51% of falls. Most of the falls, 4,328 (85%), were documented as unassisted. A total of 209 (4%) falls occurred while physical restraints were in use.

In the first quarterly restraint use prevalence survey (October 2006), the 829 participating nursing units surveyed 19,979 patients (median [IQR] = 23 [20-23] patients per nursing unit). The median (IQR) age was 66 (51-78) years, and 54% of them were females. At the time of the survey, restraints were in use for 326 (1.6%) patients. Restrained patients were older than unrestrained patients (median age: 78 vs 65 years) and more likely to be male (56% vs 46%). Limb restraints were used for 139 patients, vest restraints for 66, both limb and vest restraints for 24, and other restraint types were used for 113 patients (including 11 in limb restraints and 5 in limb and vest restraints).

Change in Endpoints after Implementation of the HACs Initiative

Stratified Analysis

At baseline, fall rates and restraint use prevalence were slightly higher, whereas the rate of injurious falls was slightly lower, among teaching and academic medical centers compared to those in nonteaching hospitals. Declines in falls rate and restraint use prevalence were higher in teaching hospitals than in nonteaching hospitals (data not included).

We examined the rates of falls and fall injuries among 2,862 hospital units before and after the implementation of the HACs Initiative. Implementation of the CMS payment change was associated with a modest improvement in the rate of decrease for falls; a statistically significant effect on the rate of decrease for injurious falls was detectable only at seven years postchange. Fall reductions were the greatest among teaching and larger hospitals. These findings are consistent with our previous analysis of NDNQI data that found no short-term effect of the rule change on the rate of injurious falls.²⁵

We found no evidence indicating that restraint use prevalence increased because of this payment change. Physical -restraint use prevalence showed a rapidly decreasing trend before 2008, and although the rate of decline was attenuated seven years after the rule change, the overall physical restraint use prevalence in these units in 2015 was less than half of that in 2006. Unlike falls, the steepest declines in restraint use prevalence occurred in smaller hospitals.

The CMS decision to include falls with injury among the “reasonably preventable” HACs was controversial.¹¹ Inpatient falls are largely attributable to individual patient risk factors and are unusual among HACs in the extent to which patient behavior plays a role in their occurrence. Although hospital fall prevention guidelines have been published, only a few controlled trials have been conducted, with little evidence supporting the recommendations.^1,26 A quantitative review found no evidence of benefit in published hospital fall prevention studies using concurrent controls (internal rate of return = 0.92; 95% CI: 0.65-1.30),²⁶ and a recent, well-executed, cluster randomized trial of multifactorial fall prevention interventions found no change in fall rates compared with controls.²⁷ Current hospital fall prevention guidelines are limited to unproven and time-consuming nursing-level (eg, toileting schedules and use of alarms) or organizational-level strategies (eg, changing staff attitudes regarding the inevitability of falls or “leadership support”).^1,28

Despite the large sample size and the use of nurse-reported data that include patient falls from all age groups and not subject to bias due to the regulation itself (eg, ICD coding changes), our findings should be interpreted taking into account several limitations.

First, hospitals participating in the NDNQI self-select to participate and are larger and disproportionately urban compared with nonparticipating hospitals.²⁹ Although our findings were unchanged when hospital-level covariates were included in modeling, analyses stratified by teaching status and bed size demonstrated important differences. Larger teaching hospitals experienced greater fall reductions, whereas restraint use prevalence decreased more rapidly in smaller hospitals.

Second, the absence of a control group prevents us from conclusively attributing changes in falls rate and restraint use prevalence to the 2008 CMS payment change.³⁰ Our findings may have been influenced by other policy changes. For example, in October 2014, the CMS implemented the Hospital-Acquired Condition Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ Program. Under these programs, falls with hip fractures were an indicator that could alter hospital payment.

Third, we did not ascertain the use of all available fall prevention measures such as companions, bed rails, very low beds, bed alarms, and restricted activity.³¹ Nor could the study address changes in patient functional status or discharge location. In a before- and after-study of four hospitals in a single hospital system, we found that bed alarm use increased, restraint orders decreased, and the use of room change or sitters remained stable after the implementation of the CMS payment.³²

Nevertheless, we believe that these findings are consistent with the hypothesis that the HACs Initiative increased the cost of patient falls to hospitals, and, in response, some hospitals were able to modestly reduce the rate of falls. We found no evidence that physical restraint use prevalence increased.

In summary, our findings suggest modest impact of the HACs Initiative on falls and injurious falls, but no unintended impact on restraint use. These results highlight the importance of ensuring that pay-for-performance initiatives target outcomes where there are evidence-based approaches to prevention. The creation or identification of prevention tools and guidelines does not make an outcome preventable. Despite interval improvement in these self-selected hospital units in fall rates and physical restraint use prevalence, falls remain a difficult patient safety problem for hospitals, and further research is required to develop cost-effective, generalizable strategies for their prevention.

Accidental falls are among the most common incidents reported in hospitals, complicating approximately 2% of hospital stays.^1-3 Approximately 25% of falls in hospitalized patients result in injury, and 2% involve fractures.⁴ Substantial costs are associated with falls, including patient care costs associated with increased length of stay and liability.^5-7

Beginning October 1, 2008, the Centers for Medicare & Medicaid Services (CMS) stopped reimbursing hospitals for the additional care associated with eight hospital-acquired conditions (HACs), including serious fall-related injury, which were believed to be “reasonably preventable.”^8,9 Before this change, hospitals recovered the costs of these “never events” by assigning a higher level MS-DRG (Medicare Severity Diagnosis-Related Group) code for patients experiencing such an event. This is no longer allowed under the revised CMS Prospective Payment System rules.

Although the financial penalty for iatrogenic injury was modest, the payment change placed pressure on hospital staff to decrease falls, and some nurses reported changing practice to be more restrictive of patient mobility.¹⁰ Increased use of physical restraints is a potential unintended consequence of this rule change.¹¹ Restraints are known to cause agitation, delirium, decubiti, deconditioning, strangulation, and death.¹² Not surprisingly, use of restraints is discouraged in hospitals and is a CMS quality of care indicator.^13,14 Although there is no evidence that restraint use prevents patients from falling,^15,16 there is a perception among both health professionals and patients that restraints reduce the risk of falling, and they are often used as a “last resort” method of fall prevention.^17-19

The aim of this longitudinal study was to determine whether this payment change was associated with changes in short-, intermediate-, and long-term rates of falls, injurious falls, and physical restraint use in acute care hospitals. The CMS has included fall-related hip fracture in newer value-based purchasing programs by adding Patient Safety Indictor (PSI) 90 to both the HACs Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ in FY2015. However, the HACs Initiative remains the only Medicare value program that directly penalizes all injurious inpatient falls.

Study Units

As previously described,²² the National Database of Nursing Quality Indicators (NDNQI) is a data collection project initiated by the American Nurses Association (ANA). The NDNQI provides national comparative data at the unit and facility levels on nursing-sensitive indicators endorsed by the National Quality Forum. More than 2,000 hospitals voluntarily participate in the NDNQI, including virtually all ANA Magnet-recognized hospitals, and more than 90% of nursing units participate in the fall measures (NDNQI, personal communication). At the start of study data collection, the project was administered by the School of Nursing at the University of Kansas Medical Center. In 2014, the ownership of the NDNQI was transferred from the ANA to Press Ganey Associates, Inc. In addition to standardized data on unit, facility, and staffing characteristics, the NDNQI member hospitals can elect to submit monthly data on falls and quarterly data on physical restraint use prevalence.

We examined the data collected from adult medical, medical-surgical, and surgical units in United States acute care hospitals that elected to participate in the fall and physical restraint use data collection within the NDNQI for the 27 months before and the 87 months after the implementation of the CMS rule change. Eligible units contributed at least one fall and physical restraint use data point during both the 27 months preceding October 1, 2008, and the 87 months immediately after. The Institutional Review Board at the University of Kansas Medical Center reviewed and approved the study before its implementation.

Endpoints

Fall Events

The NDNQI defines a patient fall as an unplanned descent to the floor, regardless of whether the fall results in injury and regardless of whether the patient was assisted to the floor by a member of the hospital staff. Events in which a patient lands on a surface where one would not expect to find a patient (eg, on a mat next to a low bed) are also counted as falls.

Using internal data sources (eg, medical records, incident reports), participating hospitals report the number of inpatient falls each month to the NDNQI. We analyzed the falls data for the period July 1, 2006, through December 31, 2015. Thus, each unit could contribute 114 months (27 months before the rule change and 87 months after the rule change) of falls data.

Hospitals classify the injury level of each fall as none, minor (resulting in bruise, pain, abrasion, wound cleaning, or limb elevation, or in the use of ice, dressing, or topical medication), moderate (resulting in suturing, splinting, muscle or joint strain, or application of steri-strips or skin glue), major (resulting in surgery, casting, traction, any type of fracture, consultation for neurological or internal injury, or receipt of blood products for patients with coagulopathy), or death (resulting from injuries sustained from falling). For this study, a fall resulting in any injury (including minor) was considered as an injurious fall. The NDNQI data have been validated for falls and fall injury.^23,24

Based on patient counts from unit censuses and/or internal data on actual patient hours on the unit, hospitals also report to the NDNQI the monthly number of patient days for each unit for which falls data are reported. The NDNQI uses these data to calculate each unit’s total and injurious fall rate per 1,000 patient days.

Physical Restraint Use

The NDNQI follows the CMS definition of restraint, which is “any manual method, physical or mechanical device, material, or equipment that immobilizes or reduces the ability of a patient to move his or her arms, legs, body, or head freely”.¹³ The NDNQI restraint use data are collected quarterly. Participating hospitals choose one day each quarter to conduct a restraint use prevalence survey on participating units. On the selected day, designated RNs within these hospitals visually assess each patient on the unit for restraint use. Based on this survey, hospitals report to the NDNQI the total count of patients surveyed and whether each was restrained. For restrained patients, hospitals also report the type of restraint as limb, vest, or other (eg, four side rails, net beds, mitts not attached to the bed).

We analyzed the restraint use data for the period October 1, 2006, through December 31, 2015. Thus, 37 quarters of data (eight pre- and 29 postrule change) were available. For this study, we computed for each unit the quarterly proportion of surveyed patients who were physically restrained by dividing the total count of restrained patients (regardless of the type of restraint) on the day of the survey by the total count of surveyed patients.

Covariates

Unit- and facility-level covariates were included in several model specifications to determine whether patient or facility characteristics affected the results. The unit-level covariates included the type of nursing unit (medical, medical and surgical, or surgical), monthly rates of total nursing hours per patient day, and nursing skill mix (percent registered nurses/total nursing personnel). The three facility-level variables included urban–rural location (defined as metropolitan [located in an area containing an urban core with a population of at least 50,000], micropolitan [located in an area containing an urban core with a population of 10,000-49,999], or neither), bed size (<300 beds or ≥300 beds), and teaching status (academic health center, major teaching hospital, or nonteaching hospital).

Because larger, academically affiliated hospitals are overrepresented in the NDNQI, we conducted stratified analyses of these variables to explore how change in the rates of falls and restraint use in the entire sample might differ between hospitals according to bed size (<300 beds, ≥300 beds) and teaching status (nonteaching versus teaching and academic health center).

Statistical Methods

We compared the mean annual rates of change in falls, injurious falls, and physical restraint use prevalence during the two years before the HACs Initiative went into effect (October 2006-September 2008) with the mean annual rates of change following the implementation of the payment rule. Short-term (one-year) change was the slope from October 2008 to September 2009, intermediate-term (four-year) change was the slope from October 2008 to September 2012, and long-term (seven-year) change was the slope from October 2008 to September 2015.

Monthly rates of falls and injurious falls over the 114-month period were modeled using negative binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). We also fit the models that included the terms in the base model and facility-level covariates, unit-level covariates, both individually and combined. All models included terms for seasonality.

Quarterly prevalence rates of restraint use over the 37 quarters were modeled using beta-binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). Similar to the one specified for falls, models were also fitted that included facility- and unit-level covariates as described above.

To adjust for multiple comparisons of the three postimplementation slopes, all confidence intervals were Bonferroni corrected.

Nursing Units

We included nursing units with one or more months of falls data and one or more quarters of restraint use data before and after the rule change. Of the 11,117 nursing units that submitted data to the NDNQI, 2,862 units (983 medical, 1,219 medical-surgical, and 660 surgical) with the requisite demographic, falls, and restraint use data were considered for inclusion in the study. The characteristics of the nursing units (ie, the type of unit, total nursing hours per patient day, and nursing skill mix) and hospitals (ie, location, bed size, teaching status) included in the study were similar to those of the overall NDNQI member units.

Baseline Characteristics

In the first study month (July 2006), 1,941 sample nursing units reported 5,101 falls during 1,401,652 patient-days of observation. Of these, 1,502 (29%) resulted in injury (1,281 minor, 144 moderate, 75 major, and two deaths). Across falls, the median (interquartile range [IQR]) patient age was 70 (55-80) years, with males accounting for 51% of falls. Most of the falls, 4,328 (85%), were documented as unassisted. A total of 209 (4%) falls occurred while physical restraints were in use.

In the first quarterly restraint use prevalence survey (October 2006), the 829 participating nursing units surveyed 19,979 patients (median [IQR] = 23 [20-23] patients per nursing unit). The median (IQR) age was 66 (51-78) years, and 54% of them were females. At the time of the survey, restraints were in use for 326 (1.6%) patients. Restrained patients were older than unrestrained patients (median age: 78 vs 65 years) and more likely to be male (56% vs 46%). Limb restraints were used for 139 patients, vest restraints for 66, both limb and vest restraints for 24, and other restraint types were used for 113 patients (including 11 in limb restraints and 5 in limb and vest restraints).

Change in Endpoints after Implementation of the HACs Initiative

Stratified Analysis

At baseline, fall rates and restraint use prevalence were slightly higher, whereas the rate of injurious falls was slightly lower, among teaching and academic medical centers compared to those in nonteaching hospitals. Declines in falls rate and restraint use prevalence were higher in teaching hospitals than in nonteaching hospitals (data not included).

We examined the rates of falls and fall injuries among 2,862 hospital units before and after the implementation of the HACs Initiative. Implementation of the CMS payment change was associated with a modest improvement in the rate of decrease for falls; a statistically significant effect on the rate of decrease for injurious falls was detectable only at seven years postchange. Fall reductions were the greatest among teaching and larger hospitals. These findings are consistent with our previous analysis of NDNQI data that found no short-term effect of the rule change on the rate of injurious falls.²⁵

We found no evidence indicating that restraint use prevalence increased because of this payment change. Physical -restraint use prevalence showed a rapidly decreasing trend before 2008, and although the rate of decline was attenuated seven years after the rule change, the overall physical restraint use prevalence in these units in 2015 was less than half of that in 2006. Unlike falls, the steepest declines in restraint use prevalence occurred in smaller hospitals.

The CMS decision to include falls with injury among the “reasonably preventable” HACs was controversial.¹¹ Inpatient falls are largely attributable to individual patient risk factors and are unusual among HACs in the extent to which patient behavior plays a role in their occurrence. Although hospital fall prevention guidelines have been published, only a few controlled trials have been conducted, with little evidence supporting the recommendations.^1,26 A quantitative review found no evidence of benefit in published hospital fall prevention studies using concurrent controls (internal rate of return = 0.92; 95% CI: 0.65-1.30),²⁶ and a recent, well-executed, cluster randomized trial of multifactorial fall prevention interventions found no change in fall rates compared with controls.²⁷ Current hospital fall prevention guidelines are limited to unproven and time-consuming nursing-level (eg, toileting schedules and use of alarms) or organizational-level strategies (eg, changing staff attitudes regarding the inevitability of falls or “leadership support”).^1,28

Despite the large sample size and the use of nurse-reported data that include patient falls from all age groups and not subject to bias due to the regulation itself (eg, ICD coding changes), our findings should be interpreted taking into account several limitations.

First, hospitals participating in the NDNQI self-select to participate and are larger and disproportionately urban compared with nonparticipating hospitals.²⁹ Although our findings were unchanged when hospital-level covariates were included in modeling, analyses stratified by teaching status and bed size demonstrated important differences. Larger teaching hospitals experienced greater fall reductions, whereas restraint use prevalence decreased more rapidly in smaller hospitals.

Second, the absence of a control group prevents us from conclusively attributing changes in falls rate and restraint use prevalence to the 2008 CMS payment change.³⁰ Our findings may have been influenced by other policy changes. For example, in October 2014, the CMS implemented the Hospital-Acquired Condition Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ Program. Under these programs, falls with hip fractures were an indicator that could alter hospital payment.

Third, we did not ascertain the use of all available fall prevention measures such as companions, bed rails, very low beds, bed alarms, and restricted activity.³¹ Nor could the study address changes in patient functional status or discharge location. In a before- and after-study of four hospitals in a single hospital system, we found that bed alarm use increased, restraint orders decreased, and the use of room change or sitters remained stable after the implementation of the CMS payment.³²

Nevertheless, we believe that these findings are consistent with the hypothesis that the HACs Initiative increased the cost of patient falls to hospitals, and, in response, some hospitals were able to modestly reduce the rate of falls. We found no evidence that physical restraint use prevalence increased.

In summary, our findings suggest modest impact of the HACs Initiative on falls and injurious falls, but no unintended impact on restraint use. These results highlight the importance of ensuring that pay-for-performance initiatives target outcomes where there are evidence-based approaches to prevention. The creation or identification of prevention tools and guidelines does not make an outcome preventable. Despite interval improvement in these self-selected hospital units in fall rates and physical restraint use prevalence, falls remain a difficult patient safety problem for hospitals, and further research is required to develop cost-effective, generalizable strategies for their prevention.

Accidental falls are among the most common incidents reported in hospitals, complicating approximately 2% of hospital stays.^1-3 Approximately 25% of falls in hospitalized patients result in injury, and 2% involve fractures.⁴ Substantial costs are associated with falls, including patient care costs associated with increased length of stay and liability.^5-7

Beginning October 1, 2008, the Centers for Medicare & Medicaid Services (CMS) stopped reimbursing hospitals for the additional care associated with eight hospital-acquired conditions (HACs), including serious fall-related injury, which were believed to be “reasonably preventable.”^8,9 Before this change, hospitals recovered the costs of these “never events” by assigning a higher level MS-DRG (Medicare Severity Diagnosis-Related Group) code for patients experiencing such an event. This is no longer allowed under the revised CMS Prospective Payment System rules.

Although the financial penalty for iatrogenic injury was modest, the payment change placed pressure on hospital staff to decrease falls, and some nurses reported changing practice to be more restrictive of patient mobility.¹⁰ Increased use of physical restraints is a potential unintended consequence of this rule change.¹¹ Restraints are known to cause agitation, delirium, decubiti, deconditioning, strangulation, and death.¹² Not surprisingly, use of restraints is discouraged in hospitals and is a CMS quality of care indicator.^13,14 Although there is no evidence that restraint use prevents patients from falling,^15,16 there is a perception among both health professionals and patients that restraints reduce the risk of falling, and they are often used as a “last resort” method of fall prevention.^17-19

The aim of this longitudinal study was to determine whether this payment change was associated with changes in short-, intermediate-, and long-term rates of falls, injurious falls, and physical restraint use in acute care hospitals. The CMS has included fall-related hip fracture in newer value-based purchasing programs by adding Patient Safety Indictor (PSI) 90 to both the HACs Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ in FY2015. However, the HACs Initiative remains the only Medicare value program that directly penalizes all injurious inpatient falls.

Study Units

As previously described,²² the National Database of Nursing Quality Indicators (NDNQI) is a data collection project initiated by the American Nurses Association (ANA). The NDNQI provides national comparative data at the unit and facility levels on nursing-sensitive indicators endorsed by the National Quality Forum. More than 2,000 hospitals voluntarily participate in the NDNQI, including virtually all ANA Magnet-recognized hospitals, and more than 90% of nursing units participate in the fall measures (NDNQI, personal communication). At the start of study data collection, the project was administered by the School of Nursing at the University of Kansas Medical Center. In 2014, the ownership of the NDNQI was transferred from the ANA to Press Ganey Associates, Inc. In addition to standardized data on unit, facility, and staffing characteristics, the NDNQI member hospitals can elect to submit monthly data on falls and quarterly data on physical restraint use prevalence.

We examined the data collected from adult medical, medical-surgical, and surgical units in United States acute care hospitals that elected to participate in the fall and physical restraint use data collection within the NDNQI for the 27 months before and the 87 months after the implementation of the CMS rule change. Eligible units contributed at least one fall and physical restraint use data point during both the 27 months preceding October 1, 2008, and the 87 months immediately after. The Institutional Review Board at the University of Kansas Medical Center reviewed and approved the study before its implementation.

Endpoints

Fall Events

The NDNQI defines a patient fall as an unplanned descent to the floor, regardless of whether the fall results in injury and regardless of whether the patient was assisted to the floor by a member of the hospital staff. Events in which a patient lands on a surface where one would not expect to find a patient (eg, on a mat next to a low bed) are also counted as falls.

Using internal data sources (eg, medical records, incident reports), participating hospitals report the number of inpatient falls each month to the NDNQI. We analyzed the falls data for the period July 1, 2006, through December 31, 2015. Thus, each unit could contribute 114 months (27 months before the rule change and 87 months after the rule change) of falls data.

Hospitals classify the injury level of each fall as none, minor (resulting in bruise, pain, abrasion, wound cleaning, or limb elevation, or in the use of ice, dressing, or topical medication), moderate (resulting in suturing, splinting, muscle or joint strain, or application of steri-strips or skin glue), major (resulting in surgery, casting, traction, any type of fracture, consultation for neurological or internal injury, or receipt of blood products for patients with coagulopathy), or death (resulting from injuries sustained from falling). For this study, a fall resulting in any injury (including minor) was considered as an injurious fall. The NDNQI data have been validated for falls and fall injury.^23,24

Based on patient counts from unit censuses and/or internal data on actual patient hours on the unit, hospitals also report to the NDNQI the monthly number of patient days for each unit for which falls data are reported. The NDNQI uses these data to calculate each unit’s total and injurious fall rate per 1,000 patient days.

Physical Restraint Use

The NDNQI follows the CMS definition of restraint, which is “any manual method, physical or mechanical device, material, or equipment that immobilizes or reduces the ability of a patient to move his or her arms, legs, body, or head freely”.¹³ The NDNQI restraint use data are collected quarterly. Participating hospitals choose one day each quarter to conduct a restraint use prevalence survey on participating units. On the selected day, designated RNs within these hospitals visually assess each patient on the unit for restraint use. Based on this survey, hospitals report to the NDNQI the total count of patients surveyed and whether each was restrained. For restrained patients, hospitals also report the type of restraint as limb, vest, or other (eg, four side rails, net beds, mitts not attached to the bed).

We analyzed the restraint use data for the period October 1, 2006, through December 31, 2015. Thus, 37 quarters of data (eight pre- and 29 postrule change) were available. For this study, we computed for each unit the quarterly proportion of surveyed patients who were physically restrained by dividing the total count of restrained patients (regardless of the type of restraint) on the day of the survey by the total count of surveyed patients.

Covariates

Unit- and facility-level covariates were included in several model specifications to determine whether patient or facility characteristics affected the results. The unit-level covariates included the type of nursing unit (medical, medical and surgical, or surgical), monthly rates of total nursing hours per patient day, and nursing skill mix (percent registered nurses/total nursing personnel). The three facility-level variables included urban–rural location (defined as metropolitan [located in an area containing an urban core with a population of at least 50,000], micropolitan [located in an area containing an urban core with a population of 10,000-49,999], or neither), bed size (<300 beds or ≥300 beds), and teaching status (academic health center, major teaching hospital, or nonteaching hospital).

Because larger, academically affiliated hospitals are overrepresented in the NDNQI, we conducted stratified analyses of these variables to explore how change in the rates of falls and restraint use in the entire sample might differ between hospitals according to bed size (<300 beds, ≥300 beds) and teaching status (nonteaching versus teaching and academic health center).

Statistical Methods

We compared the mean annual rates of change in falls, injurious falls, and physical restraint use prevalence during the two years before the HACs Initiative went into effect (October 2006-September 2008) with the mean annual rates of change following the implementation of the payment rule. Short-term (one-year) change was the slope from October 2008 to September 2009, intermediate-term (four-year) change was the slope from October 2008 to September 2012, and long-term (seven-year) change was the slope from October 2008 to September 2015.

Monthly rates of falls and injurious falls over the 114-month period were modeled using negative binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). We also fit the models that included the terms in the base model and facility-level covariates, unit-level covariates, both individually and combined. All models included terms for seasonality.

Quarterly prevalence rates of restraint use over the 37 quarters were modeled using beta-binomial models with a random intercept to account for heterogeneity between units. Each base mean model included the preimplementation intercept and slope (over time), the postimplementation intercept, and slope (both linear and quadratic). Similar to the one specified for falls, models were also fitted that included facility- and unit-level covariates as described above.

To adjust for multiple comparisons of the three postimplementation slopes, all confidence intervals were Bonferroni corrected.

Nursing Units

We included nursing units with one or more months of falls data and one or more quarters of restraint use data before and after the rule change. Of the 11,117 nursing units that submitted data to the NDNQI, 2,862 units (983 medical, 1,219 medical-surgical, and 660 surgical) with the requisite demographic, falls, and restraint use data were considered for inclusion in the study. The characteristics of the nursing units (ie, the type of unit, total nursing hours per patient day, and nursing skill mix) and hospitals (ie, location, bed size, teaching status) included in the study were similar to those of the overall NDNQI member units.

Baseline Characteristics

In the first study month (July 2006), 1,941 sample nursing units reported 5,101 falls during 1,401,652 patient-days of observation. Of these, 1,502 (29%) resulted in injury (1,281 minor, 144 moderate, 75 major, and two deaths). Across falls, the median (interquartile range [IQR]) patient age was 70 (55-80) years, with males accounting for 51% of falls. Most of the falls, 4,328 (85%), were documented as unassisted. A total of 209 (4%) falls occurred while physical restraints were in use.

In the first quarterly restraint use prevalence survey (October 2006), the 829 participating nursing units surveyed 19,979 patients (median [IQR] = 23 [20-23] patients per nursing unit). The median (IQR) age was 66 (51-78) years, and 54% of them were females. At the time of the survey, restraints were in use for 326 (1.6%) patients. Restrained patients were older than unrestrained patients (median age: 78 vs 65 years) and more likely to be male (56% vs 46%). Limb restraints were used for 139 patients, vest restraints for 66, both limb and vest restraints for 24, and other restraint types were used for 113 patients (including 11 in limb restraints and 5 in limb and vest restraints).

Change in Endpoints after Implementation of the HACs Initiative

Stratified Analysis

At baseline, fall rates and restraint use prevalence were slightly higher, whereas the rate of injurious falls was slightly lower, among teaching and academic medical centers compared to those in nonteaching hospitals. Declines in falls rate and restraint use prevalence were higher in teaching hospitals than in nonteaching hospitals (data not included).

We examined the rates of falls and fall injuries among 2,862 hospital units before and after the implementation of the HACs Initiative. Implementation of the CMS payment change was associated with a modest improvement in the rate of decrease for falls; a statistically significant effect on the rate of decrease for injurious falls was detectable only at seven years postchange. Fall reductions were the greatest among teaching and larger hospitals. These findings are consistent with our previous analysis of NDNQI data that found no short-term effect of the rule change on the rate of injurious falls.²⁵

We found no evidence indicating that restraint use prevalence increased because of this payment change. Physical -restraint use prevalence showed a rapidly decreasing trend before 2008, and although the rate of decline was attenuated seven years after the rule change, the overall physical restraint use prevalence in these units in 2015 was less than half of that in 2006. Unlike falls, the steepest declines in restraint use prevalence occurred in smaller hospitals.

The CMS decision to include falls with injury among the “reasonably preventable” HACs was controversial.¹¹ Inpatient falls are largely attributable to individual patient risk factors and are unusual among HACs in the extent to which patient behavior plays a role in their occurrence. Although hospital fall prevention guidelines have been published, only a few controlled trials have been conducted, with little evidence supporting the recommendations.^1,26 A quantitative review found no evidence of benefit in published hospital fall prevention studies using concurrent controls (internal rate of return = 0.92; 95% CI: 0.65-1.30),²⁶ and a recent, well-executed, cluster randomized trial of multifactorial fall prevention interventions found no change in fall rates compared with controls.²⁷ Current hospital fall prevention guidelines are limited to unproven and time-consuming nursing-level (eg, toileting schedules and use of alarms) or organizational-level strategies (eg, changing staff attitudes regarding the inevitability of falls or “leadership support”).^1,28

Despite the large sample size and the use of nurse-reported data that include patient falls from all age groups and not subject to bias due to the regulation itself (eg, ICD coding changes), our findings should be interpreted taking into account several limitations.

First, hospitals participating in the NDNQI self-select to participate and are larger and disproportionately urban compared with nonparticipating hospitals.²⁹ Although our findings were unchanged when hospital-level covariates were included in modeling, analyses stratified by teaching status and bed size demonstrated important differences. Larger teaching hospitals experienced greater fall reductions, whereas restraint use prevalence decreased more rapidly in smaller hospitals.

Second, the absence of a control group prevents us from conclusively attributing changes in falls rate and restraint use prevalence to the 2008 CMS payment change.³⁰ Our findings may have been influenced by other policy changes. For example, in October 2014, the CMS implemented the Hospital-Acquired Condition Reduction Program (HACRP)²⁰ and the Hospital Value-Based Purchasing (VBP)²¹ Program. Under these programs, falls with hip fractures were an indicator that could alter hospital payment.

Third, we did not ascertain the use of all available fall prevention measures such as companions, bed rails, very low beds, bed alarms, and restricted activity.³¹ Nor could the study address changes in patient functional status or discharge location. In a before- and after-study of four hospitals in a single hospital system, we found that bed alarm use increased, restraint orders decreased, and the use of room change or sitters remained stable after the implementation of the CMS payment.³²

Nevertheless, we believe that these findings are consistent with the hypothesis that the HACs Initiative increased the cost of patient falls to hospitals, and, in response, some hospitals were able to modestly reduce the rate of falls. We found no evidence that physical restraint use prevalence increased.

In summary, our findings suggest modest impact of the HACs Initiative on falls and injurious falls, but no unintended impact on restraint use. These results highlight the importance of ensuring that pay-for-performance initiatives target outcomes where there are evidence-based approaches to prevention. The creation or identification of prevention tools and guidelines does not make an outcome preventable. Despite interval improvement in these self-selected hospital units in fall rates and physical restraint use prevalence, falls remain a difficult patient safety problem for hospitals, and further research is required to develop cost-effective, generalizable strategies for their prevention.

Chronic obstructive pulmonary disease (COPD) is recognized as the third leading cause of death nationally. Globally, it has been estimated that 10% of the population has COPD; in the United States, approximately 15 million people are affected.^1,2 The annual estimated cost of COPD management in the United States is approximately $50 billion, one-third of which is directly related to inpatient hospitalization for COPD exacerbation.^3,4,5 The 30-day readmission rate after hospitalization for acute exacerbation of COPD (AECOPD) is approximately 21% with an approximate cost of $13 billion per year.^6,7 To reduce the cost and to improve patient outcomes, the Centers for Medicare and Medicaid Services (CMS) has designed several interventions with little effect.⁸

In October 2012, the Affordable Care Act added section 1886(q) to the Social Security Act and established the Hospital Readmission Reduction Program (HRRP), an initiative to decrease hospitalization costs by penalizing hospitals with high 30-day readmission rates. Under this program, hospitals received up to 3% penalty for excess readmissions after the index hospitalization with acute myocardial infarction (AMI), heart failure (HF), and pneumonia.^9-11 Hospitals are penalized if their annual readmission rates are significantly above the average national readmission rate. In 2014, the HRRP was extended to include AECOPD for the FY 2015.

Since the implementation of readmission penalties, data have shown a significant decrease in the 30-day readmission rates for all conditions.^12,13 On the other hand, studies have suggested that, at least for some conditions, the decrease in the 30-day readmission rate is associated with higher adverse patients outcomes, including higher mortality.^14,15 However, whether a decrease in readmission rates after an AECOPD hospitalization is associated with a concomitant increase in mortality has not been examined. Therefore, our objective was to examine the association of the 30-day risk-adjusted hospital readmission rate with the 30-day risk-adjusted hospital mortality rate for patients discharged with a diagnosis of AECOPD.

Data Sources

Publicly available data from three sources were used. The all-cause 30-day risk-standardized readmission rate (RSRR) and the 30-day risk-standardized mortality rate (RSMR) of each hospital for patients with AECOPD were obtained from the Hospital Compare database; a database maintained by the CMS.^16,17 In 2014, the CMS started reporting three-year running average of 30-day mortality and readmission rate data on hospitals for AECOPD hospitalizations; the data start date was July 2010.^18-22 We examined data from the FY 2010-2013 to 2014-2017 cycles on readmission and mortality reported by the CMS; this included data before and after the implementation of penalties.

Hospital characteristics were also obtained from the CMS website. Hospital ownership was defined as government (owned by Federal or state), for-profit (owned by physicians or another proprietary), or nonprofit (owned by a nonprofit organization such as a church). A hospital was considered as a teaching hospital if it obtained graduate medical education funding from the CMS.

Data on local population characteristics according to ZIP codes were obtained from the 2010 decennial census and the American Community Survey five-year (2009-2013) data files available at the United States Census Bureau website.²³ For each ZIP code, we obtained data on the total population, percentage of African Americans in the population, median income, poverty level, and insurance status.

We used Hospital service area (HSA) information obtained from the Dartmouth Atlas of Health Care crosswalk files to link local population characteristics to hospitals. The Dartmouth Atlas defined 3,436 HSAs by assigning the ZIP codes to the hospital area where the greatest proportion of their Medicare residents was hospitalized.^24,25

Hospital Compare data and Census Bureau population data were matched to the HSAs from the Dartmouth Atlas of Healthcare data at the ZIP code level. First, the ZIP code-level data from the Census Bureau were pooled by the HSAs obtained from the Dartmouth Atlas of Healthcare, followed by matching these data by the HSAs to the Hospital Compare data. Merging data from these three sources generated a dataset that contained information about readmission and mortality rates from a particular hospital and the population characteristics of the local healthcare market or neighborhood. Our final dataset included hospitals that had readmission and mortality information available at the Hospital Compare website and were included in the crosswalk files of the Dartmouth Atlas of Healthcare.

Statistical Analysis

Data are summarized as mean and standard deviation (SD), median with interquartile range, or frequencies as appropriate. To model the dependence of observations from the same hospital over time, we used mixed linear models with random intercept and slope. A strength of this modeling approach is that it incorporates information from all hospitals even when some hospitals are missing data for some time periods. We reached our final model through stages with increasing model complexity at each stage. In the first stage, we developed an empty model without any covariates to determine the unconditional variance components so that we can partition mortality variance into between- and within-hospital components. In the second stage, we developed an unconditional growth curve model to determine the shape of time trend in mortality over time using linear and quadratic (by including squared time in the model) growth curves. In the third stage, we added baseline readmission rates (from 2010 to 2013) to the model to determine the effect of baseline readmission rate on mortality trends and also examined its interaction with time and squared time. We generated a change in the readmission rate variable by subtracting the last readmission rate from the baseline readmission rate (readmission rate in 2010-2013 − readmission rate in 2014-2017). In the fourth stage, we included this change in readmission rate into the third-stage model to examine how changes in the readmission rate affected the time trends of mortality and also examined its interaction with time and squared time. In the final model, we included the following potential confounding variables to the fourth stage model: African American percentage in the HSA, HSA median income, percentage of people living in poverty in the HSA, median age, ownership of hospital (government, for profit), teaching status (teaching vs nonteaching), and acute care hospital beds in the HSA. Within each stage, the models were compared using the Akaike information criterion (AIC) and the Bayesian information criterion (BIC), and the model with the lowest value of each was moved to the next stage of model development. All analyses were performed in Stata 14.1 for Windows (College Station, Texas).

Of the 3,685 acute care hospitals analyzed in the 2010-2013 data cycle for COPD, the 30-day RSRR was 20.7% (1.28), which decreased to 19.6% (1.11) in 2014-2017 (Table 1). During the same period, the 30-day all-cause RSMR increased from 7.8% (1.03) in 2010-2013 to 8.4% (1.11) in 2014-2017. The partitioning of variance showed that 57% of variation in the mortality rate over the study period was due to between-hospital differences.

The unconditional growth model examining the linear time trend revealed a 0.13% per year (95% CI = 0.12 to 0.14; P < .0001) increase in mortality rate over the five data cycles. When the squared time variable was added to the model to examine a quadratic trend, both time and squared trend were statistically significant (Table 2) and the AIC and BIC were lower for the quadratic model. Thus, the unconditional growth curve model suggested that the mortality trend was nonlinear and the coefficients demonstrated that not only the mortality rate increased, but the rate of change in the mortality rate was also increasing during the study period.

When we added the baseline readmission rate to the abovementioned quadratic growth model, we found an inverse association; each 1% increase in baseline readmission rate was associated with 0.03% (95% CI = −0.05 to −0.005; P = .02) decrease in mortality rate. These findings suggest that hospitals with higher baseline readmission rates also had lower mortality rates. To examine whether the effect of baseline readmission rate on mortality varied over time, we included the interaction term with time in the model and then added the interaction term with squared time. As the AIC and BIC were the lowest for the model with interaction between time and baseline readmission (and not when interaction between squared time and baseline readmission were included), we accepted this model. In this model, although there was no difference in mortality according to readmissions at baseline, each 1% increase in baseline readmission rate was associated with a smaller increase in mortality rate by 0.015% (95% CI = −0.02 to −0.01; P < .0001; Table 2 and Figure 1). These findings suggest that hospitals with higher readmission rates at baseline had a smaller increase in mortality rate during the study period than those with lower readmission rates.

As efforts to decrease readmission rates continue as a part of the HRRP implementation by the CMS, our study shows that among hospitals that discharged patients with AECOPD during 2010-2017, the all-cause 30-day RSRR was decreased, whereas the all-cause 30-day RSMR was increased. Of particular concern is that the rate of increase in mortality also increased. We also found that hospitals with higher readmission rates in 2010-2013 had a lower rate of increase in mortality than hospitals with lower readmission rates. In addition, hospitals that had a larger decrease in readmission rates during the study period had a larger rate of increase in mortality than hospitals with a smaller decrease in readmission rates. Our findings were robust to potential confounders such as hospital characteristics and local population characteristics in which hospitals operate.

Our study findings raise the question whether the implementation of the HRRP resulted in unintentional patient harm by forcing hospitals to make changes that may affect overall patient care. This question is particularly important as other studies on hospitalized patients with HF have found similar results.^13,14 On the other hand, a similar association between readmission and mortality rates has not been observed in patients with pneumonia or AMI.¹⁴ Several possible explanations can be given for the observed discrepancy between the diseases and their effect on the relationship between readmission rate and mortality rate. Both COPD and HF are chronic diseases and characterized by exacerbations, whereas AMI and pneumonia are episodic diseases that are treatable. As the number of patients hospitalized with AECOPD and HF is much larger, hospitals may have a greater focus on reducing the 30-day readmission rates and may attempt to game the process, such as by delaying admissions through the emergency department within the 30-day period or by admitting patients for observation. In fact, a study found a 3% reduction in the within-hospital readmission rate with a concurrent 0.8% increase in observation unit use since the implementation of the HRRP.²⁶ Such approaches to patient care may lead to adverse outcomes.

It is possible that readmissions and mortality act as competing risks and hence hospitals with higher mortality rates are left with fewer patients and thus have fewer readmissions, whereas those with lower mortality rates have more patients and a higher readmission rate.²⁷ Such studies are not possible with hospital-level data, and patient-level studies will be required to examine this competing risk hypothesis. Our study results provide some support to the competing risk hypothesis (hospitals with lower baseline readmission rates had a steeper increase in mortality); however, it is not possible to draw any conclusions due to the high risk of ecological fallacy bias.

This study has important potential implications for healthcare policy, public health, and research. We found that an important national intervention aimed at decreasing readmission rates and improving the quality of care for patients with AECOPD may be associated with higher mortality rates in these patients. There may be a need to redefine measures for determining the performance of an institution. Our study supports research into the underlying mechanisms resulting in an inverse association between readmissions and mortality. In particular, health policy researchers may need to examine how incentives and penalties affect the allocation of resources within hospitals.

This study has several strengths and some potential weaknesses. We used a national dataset to examine readmission and mortality rates that include the majority of hospitals in the United States. We also included data from the local population for each hospital, thus allowing us to examine hospital performance within the context of its target population. One potential limitation is that we used hospital-level data and not patient-level data; however, the readmission penalties are designed for hospitals, which justifies our use of hospital-level data. Furthermore, data were not available for shorter time intervals; data from shorter time intervals may be associated with greater variability. Being an observational study, it is difficult to establish a causal relationship; the longitudinal nature of the study does establish temporality, an important factor in establishing causality.

In conclusion, we found that although the readmission rates decreased, there was an increase in the mortality rate within the 30 days of discharge from the hospital in patients with AECOPD. The rate of increase in mortality was higher in hospitals with lower readmission rates than in hospitals with higher readmission rates. Further research for determining the mechanism responsible for this association is needed. Future health policy interventions may need to consider the potential for adverse outcomes.

Chronic obstructive pulmonary disease (COPD) is recognized as the third leading cause of death nationally. Globally, it has been estimated that 10% of the population has COPD; in the United States, approximately 15 million people are affected.^1,2 The annual estimated cost of COPD management in the United States is approximately $50 billion, one-third of which is directly related to inpatient hospitalization for COPD exacerbation.^3,4,5 The 30-day readmission rate after hospitalization for acute exacerbation of COPD (AECOPD) is approximately 21% with an approximate cost of $13 billion per year.^6,7 To reduce the cost and to improve patient outcomes, the Centers for Medicare and Medicaid Services (CMS) has designed several interventions with little effect.⁸

In October 2012, the Affordable Care Act added section 1886(q) to the Social Security Act and established the Hospital Readmission Reduction Program (HRRP), an initiative to decrease hospitalization costs by penalizing hospitals with high 30-day readmission rates. Under this program, hospitals received up to 3% penalty for excess readmissions after the index hospitalization with acute myocardial infarction (AMI), heart failure (HF), and pneumonia.^9-11 Hospitals are penalized if their annual readmission rates are significantly above the average national readmission rate. In 2014, the HRRP was extended to include AECOPD for the FY 2015.

Since the implementation of readmission penalties, data have shown a significant decrease in the 30-day readmission rates for all conditions.^12,13 On the other hand, studies have suggested that, at least for some conditions, the decrease in the 30-day readmission rate is associated with higher adverse patients outcomes, including higher mortality.^14,15 However, whether a decrease in readmission rates after an AECOPD hospitalization is associated with a concomitant increase in mortality has not been examined. Therefore, our objective was to examine the association of the 30-day risk-adjusted hospital readmission rate with the 30-day risk-adjusted hospital mortality rate for patients discharged with a diagnosis of AECOPD.

Data Sources

Publicly available data from three sources were used. The all-cause 30-day risk-standardized readmission rate (RSRR) and the 30-day risk-standardized mortality rate (RSMR) of each hospital for patients with AECOPD were obtained from the Hospital Compare database; a database maintained by the CMS.^16,17 In 2014, the CMS started reporting three-year running average of 30-day mortality and readmission rate data on hospitals for AECOPD hospitalizations; the data start date was July 2010.^18-22 We examined data from the FY 2010-2013 to 2014-2017 cycles on readmission and mortality reported by the CMS; this included data before and after the implementation of penalties.

Hospital characteristics were also obtained from the CMS website. Hospital ownership was defined as government (owned by Federal or state), for-profit (owned by physicians or another proprietary), or nonprofit (owned by a nonprofit organization such as a church). A hospital was considered as a teaching hospital if it obtained graduate medical education funding from the CMS.

Data on local population characteristics according to ZIP codes were obtained from the 2010 decennial census and the American Community Survey five-year (2009-2013) data files available at the United States Census Bureau website.²³ For each ZIP code, we obtained data on the total population, percentage of African Americans in the population, median income, poverty level, and insurance status.

We used Hospital service area (HSA) information obtained from the Dartmouth Atlas of Health Care crosswalk files to link local population characteristics to hospitals. The Dartmouth Atlas defined 3,436 HSAs by assigning the ZIP codes to the hospital area where the greatest proportion of their Medicare residents was hospitalized.^24,25

Hospital Compare data and Census Bureau population data were matched to the HSAs from the Dartmouth Atlas of Healthcare data at the ZIP code level. First, the ZIP code-level data from the Census Bureau were pooled by the HSAs obtained from the Dartmouth Atlas of Healthcare, followed by matching these data by the HSAs to the Hospital Compare data. Merging data from these three sources generated a dataset that contained information about readmission and mortality rates from a particular hospital and the population characteristics of the local healthcare market or neighborhood. Our final dataset included hospitals that had readmission and mortality information available at the Hospital Compare website and were included in the crosswalk files of the Dartmouth Atlas of Healthcare.

Statistical Analysis

Data are summarized as mean and standard deviation (SD), median with interquartile range, or frequencies as appropriate. To model the dependence of observations from the same hospital over time, we used mixed linear models with random intercept and slope. A strength of this modeling approach is that it incorporates information from all hospitals even when some hospitals are missing data for some time periods. We reached our final model through stages with increasing model complexity at each stage. In the first stage, we developed an empty model without any covariates to determine the unconditional variance components so that we can partition mortality variance into between- and within-hospital components. In the second stage, we developed an unconditional growth curve model to determine the shape of time trend in mortality over time using linear and quadratic (by including squared time in the model) growth curves. In the third stage, we added baseline readmission rates (from 2010 to 2013) to the model to determine the effect of baseline readmission rate on mortality trends and also examined its interaction with time and squared time. We generated a change in the readmission rate variable by subtracting the last readmission rate from the baseline readmission rate (readmission rate in 2010-2013 − readmission rate in 2014-2017). In the fourth stage, we included this change in readmission rate into the third-stage model to examine how changes in the readmission rate affected the time trends of mortality and also examined its interaction with time and squared time. In the final model, we included the following potential confounding variables to the fourth stage model: African American percentage in the HSA, HSA median income, percentage of people living in poverty in the HSA, median age, ownership of hospital (government, for profit), teaching status (teaching vs nonteaching), and acute care hospital beds in the HSA. Within each stage, the models were compared using the Akaike information criterion (AIC) and the Bayesian information criterion (BIC), and the model with the lowest value of each was moved to the next stage of model development. All analyses were performed in Stata 14.1 for Windows (College Station, Texas).

Of the 3,685 acute care hospitals analyzed in the 2010-2013 data cycle for COPD, the 30-day RSRR was 20.7% (1.28), which decreased to 19.6% (1.11) in 2014-2017 (Table 1). During the same period, the 30-day all-cause RSMR increased from 7.8% (1.03) in 2010-2013 to 8.4% (1.11) in 2014-2017. The partitioning of variance showed that 57% of variation in the mortality rate over the study period was due to between-hospital differences.

The unconditional growth model examining the linear time trend revealed a 0.13% per year (95% CI = 0.12 to 0.14; P < .0001) increase in mortality rate over the five data cycles. When the squared time variable was added to the model to examine a quadratic trend, both time and squared trend were statistically significant (Table 2) and the AIC and BIC were lower for the quadratic model. Thus, the unconditional growth curve model suggested that the mortality trend was nonlinear and the coefficients demonstrated that not only the mortality rate increased, but the rate of change in the mortality rate was also increasing during the study period.

When we added the baseline readmission rate to the abovementioned quadratic growth model, we found an inverse association; each 1% increase in baseline readmission rate was associated with 0.03% (95% CI = −0.05 to −0.005; P = .02) decrease in mortality rate. These findings suggest that hospitals with higher baseline readmission rates also had lower mortality rates. To examine whether the effect of baseline readmission rate on mortality varied over time, we included the interaction term with time in the model and then added the interaction term with squared time. As the AIC and BIC were the lowest for the model with interaction between time and baseline readmission (and not when interaction between squared time and baseline readmission were included), we accepted this model. In this model, although there was no difference in mortality according to readmissions at baseline, each 1% increase in baseline readmission rate was associated with a smaller increase in mortality rate by 0.015% (95% CI = −0.02 to −0.01; P < .0001; Table 2 and Figure 1). These findings suggest that hospitals with higher readmission rates at baseline had a smaller increase in mortality rate during the study period than those with lower readmission rates.

As efforts to decrease readmission rates continue as a part of the HRRP implementation by the CMS, our study shows that among hospitals that discharged patients with AECOPD during 2010-2017, the all-cause 30-day RSRR was decreased, whereas the all-cause 30-day RSMR was increased. Of particular concern is that the rate of increase in mortality also increased. We also found that hospitals with higher readmission rates in 2010-2013 had a lower rate of increase in mortality than hospitals with lower readmission rates. In addition, hospitals that had a larger decrease in readmission rates during the study period had a larger rate of increase in mortality than hospitals with a smaller decrease in readmission rates. Our findings were robust to potential confounders such as hospital characteristics and local population characteristics in which hospitals operate.

Our study findings raise the question whether the implementation of the HRRP resulted in unintentional patient harm by forcing hospitals to make changes that may affect overall patient care. This question is particularly important as other studies on hospitalized patients with HF have found similar results.^13,14 On the other hand, a similar association between readmission and mortality rates has not been observed in patients with pneumonia or AMI.¹⁴ Several possible explanations can be given for the observed discrepancy between the diseases and their effect on the relationship between readmission rate and mortality rate. Both COPD and HF are chronic diseases and characterized by exacerbations, whereas AMI and pneumonia are episodic diseases that are treatable. As the number of patients hospitalized with AECOPD and HF is much larger, hospitals may have a greater focus on reducing the 30-day readmission rates and may attempt to game the process, such as by delaying admissions through the emergency department within the 30-day period or by admitting patients for observation. In fact, a study found a 3% reduction in the within-hospital readmission rate with a concurrent 0.8% increase in observation unit use since the implementation of the HRRP.²⁶ Such approaches to patient care may lead to adverse outcomes.

It is possible that readmissions and mortality act as competing risks and hence hospitals with higher mortality rates are left with fewer patients and thus have fewer readmissions, whereas those with lower mortality rates have more patients and a higher readmission rate.²⁷ Such studies are not possible with hospital-level data, and patient-level studies will be required to examine this competing risk hypothesis. Our study results provide some support to the competing risk hypothesis (hospitals with lower baseline readmission rates had a steeper increase in mortality); however, it is not possible to draw any conclusions due to the high risk of ecological fallacy bias.

This study has important potential implications for healthcare policy, public health, and research. We found that an important national intervention aimed at decreasing readmission rates and improving the quality of care for patients with AECOPD may be associated with higher mortality rates in these patients. There may be a need to redefine measures for determining the performance of an institution. Our study supports research into the underlying mechanisms resulting in an inverse association between readmissions and mortality. In particular, health policy researchers may need to examine how incentives and penalties affect the allocation of resources within hospitals.

This study has several strengths and some potential weaknesses. We used a national dataset to examine readmission and mortality rates that include the majority of hospitals in the United States. We also included data from the local population for each hospital, thus allowing us to examine hospital performance within the context of its target population. One potential limitation is that we used hospital-level data and not patient-level data; however, the readmission penalties are designed for hospitals, which justifies our use of hospital-level data. Furthermore, data were not available for shorter time intervals; data from shorter time intervals may be associated with greater variability. Being an observational study, it is difficult to establish a causal relationship; the longitudinal nature of the study does establish temporality, an important factor in establishing causality.

In conclusion, we found that although the readmission rates decreased, there was an increase in the mortality rate within the 30 days of discharge from the hospital in patients with AECOPD. The rate of increase in mortality was higher in hospitals with lower readmission rates than in hospitals with higher readmission rates. Further research for determining the mechanism responsible for this association is needed. Future health policy interventions may need to consider the potential for adverse outcomes.

Chronic obstructive pulmonary disease (COPD) is recognized as the third leading cause of death nationally. Globally, it has been estimated that 10% of the population has COPD; in the United States, approximately 15 million people are affected.^1,2 The annual estimated cost of COPD management in the United States is approximately $50 billion, one-third of which is directly related to inpatient hospitalization for COPD exacerbation.^3,4,5 The 30-day readmission rate after hospitalization for acute exacerbation of COPD (AECOPD) is approximately 21% with an approximate cost of $13 billion per year.^6,7 To reduce the cost and to improve patient outcomes, the Centers for Medicare and Medicaid Services (CMS) has designed several interventions with little effect.⁸

In October 2012, the Affordable Care Act added section 1886(q) to the Social Security Act and established the Hospital Readmission Reduction Program (HRRP), an initiative to decrease hospitalization costs by penalizing hospitals with high 30-day readmission rates. Under this program, hospitals received up to 3% penalty for excess readmissions after the index hospitalization with acute myocardial infarction (AMI), heart failure (HF), and pneumonia.^9-11 Hospitals are penalized if their annual readmission rates are significantly above the average national readmission rate. In 2014, the HRRP was extended to include AECOPD for the FY 2015.

Since the implementation of readmission penalties, data have shown a significant decrease in the 30-day readmission rates for all conditions.^12,13 On the other hand, studies have suggested that, at least for some conditions, the decrease in the 30-day readmission rate is associated with higher adverse patients outcomes, including higher mortality.^14,15 However, whether a decrease in readmission rates after an AECOPD hospitalization is associated with a concomitant increase in mortality has not been examined. Therefore, our objective was to examine the association of the 30-day risk-adjusted hospital readmission rate with the 30-day risk-adjusted hospital mortality rate for patients discharged with a diagnosis of AECOPD.

Data Sources

Publicly available data from three sources were used. The all-cause 30-day risk-standardized readmission rate (RSRR) and the 30-day risk-standardized mortality rate (RSMR) of each hospital for patients with AECOPD were obtained from the Hospital Compare database; a database maintained by the CMS.^16,17 In 2014, the CMS started reporting three-year running average of 30-day mortality and readmission rate data on hospitals for AECOPD hospitalizations; the data start date was July 2010.^18-22 We examined data from the FY 2010-2013 to 2014-2017 cycles on readmission and mortality reported by the CMS; this included data before and after the implementation of penalties.

Hospital characteristics were also obtained from the CMS website. Hospital ownership was defined as government (owned by Federal or state), for-profit (owned by physicians or another proprietary), or nonprofit (owned by a nonprofit organization such as a church). A hospital was considered as a teaching hospital if it obtained graduate medical education funding from the CMS.

Data on local population characteristics according to ZIP codes were obtained from the 2010 decennial census and the American Community Survey five-year (2009-2013) data files available at the United States Census Bureau website.²³ For each ZIP code, we obtained data on the total population, percentage of African Americans in the population, median income, poverty level, and insurance status.

We used Hospital service area (HSA) information obtained from the Dartmouth Atlas of Health Care crosswalk files to link local population characteristics to hospitals. The Dartmouth Atlas defined 3,436 HSAs by assigning the ZIP codes to the hospital area where the greatest proportion of their Medicare residents was hospitalized.^24,25

Hospital Compare data and Census Bureau population data were matched to the HSAs from the Dartmouth Atlas of Healthcare data at the ZIP code level. First, the ZIP code-level data from the Census Bureau were pooled by the HSAs obtained from the Dartmouth Atlas of Healthcare, followed by matching these data by the HSAs to the Hospital Compare data. Merging data from these three sources generated a dataset that contained information about readmission and mortality rates from a particular hospital and the population characteristics of the local healthcare market or neighborhood. Our final dataset included hospitals that had readmission and mortality information available at the Hospital Compare website and were included in the crosswalk files of the Dartmouth Atlas of Healthcare.

Statistical Analysis

Data are summarized as mean and standard deviation (SD), median with interquartile range, or frequencies as appropriate. To model the dependence of observations from the same hospital over time, we used mixed linear models with random intercept and slope. A strength of this modeling approach is that it incorporates information from all hospitals even when some hospitals are missing data for some time periods. We reached our final model through stages with increasing model complexity at each stage. In the first stage, we developed an empty model without any covariates to determine the unconditional variance components so that we can partition mortality variance into between- and within-hospital components. In the second stage, we developed an unconditional growth curve model to determine the shape of time trend in mortality over time using linear and quadratic (by including squared time in the model) growth curves. In the third stage, we added baseline readmission rates (from 2010 to 2013) to the model to determine the effect of baseline readmission rate on mortality trends and also examined its interaction with time and squared time. We generated a change in the readmission rate variable by subtracting the last readmission rate from the baseline readmission rate (readmission rate in 2010-2013 − readmission rate in 2014-2017). In the fourth stage, we included this change in readmission rate into the third-stage model to examine how changes in the readmission rate affected the time trends of mortality and also examined its interaction with time and squared time. In the final model, we included the following potential confounding variables to the fourth stage model: African American percentage in the HSA, HSA median income, percentage of people living in poverty in the HSA, median age, ownership of hospital (government, for profit), teaching status (teaching vs nonteaching), and acute care hospital beds in the HSA. Within each stage, the models were compared using the Akaike information criterion (AIC) and the Bayesian information criterion (BIC), and the model with the lowest value of each was moved to the next stage of model development. All analyses were performed in Stata 14.1 for Windows (College Station, Texas).

Of the 3,685 acute care hospitals analyzed in the 2010-2013 data cycle for COPD, the 30-day RSRR was 20.7% (1.28), which decreased to 19.6% (1.11) in 2014-2017 (Table 1). During the same period, the 30-day all-cause RSMR increased from 7.8% (1.03) in 2010-2013 to 8.4% (1.11) in 2014-2017. The partitioning of variance showed that 57% of variation in the mortality rate over the study period was due to between-hospital differences.

The unconditional growth model examining the linear time trend revealed a 0.13% per year (95% CI = 0.12 to 0.14; P < .0001) increase in mortality rate over the five data cycles. When the squared time variable was added to the model to examine a quadratic trend, both time and squared trend were statistically significant (Table 2) and the AIC and BIC were lower for the quadratic model. Thus, the unconditional growth curve model suggested that the mortality trend was nonlinear and the coefficients demonstrated that not only the mortality rate increased, but the rate of change in the mortality rate was also increasing during the study period.

When we added the baseline readmission rate to the abovementioned quadratic growth model, we found an inverse association; each 1% increase in baseline readmission rate was associated with 0.03% (95% CI = −0.05 to −0.005; P = .02) decrease in mortality rate. These findings suggest that hospitals with higher baseline readmission rates also had lower mortality rates. To examine whether the effect of baseline readmission rate on mortality varied over time, we included the interaction term with time in the model and then added the interaction term with squared time. As the AIC and BIC were the lowest for the model with interaction between time and baseline readmission (and not when interaction between squared time and baseline readmission were included), we accepted this model. In this model, although there was no difference in mortality according to readmissions at baseline, each 1% increase in baseline readmission rate was associated with a smaller increase in mortality rate by 0.015% (95% CI = −0.02 to −0.01; P < .0001; Table 2 and Figure 1). These findings suggest that hospitals with higher readmission rates at baseline had a smaller increase in mortality rate during the study period than those with lower readmission rates.

As efforts to decrease readmission rates continue as a part of the HRRP implementation by the CMS, our study shows that among hospitals that discharged patients with AECOPD during 2010-2017, the all-cause 30-day RSRR was decreased, whereas the all-cause 30-day RSMR was increased. Of particular concern is that the rate of increase in mortality also increased. We also found that hospitals with higher readmission rates in 2010-2013 had a lower rate of increase in mortality than hospitals with lower readmission rates. In addition, hospitals that had a larger decrease in readmission rates during the study period had a larger rate of increase in mortality than hospitals with a smaller decrease in readmission rates. Our findings were robust to potential confounders such as hospital characteristics and local population characteristics in which hospitals operate.

Our study findings raise the question whether the implementation of the HRRP resulted in unintentional patient harm by forcing hospitals to make changes that may affect overall patient care. This question is particularly important as other studies on hospitalized patients with HF have found similar results.^13,14 On the other hand, a similar association between readmission and mortality rates has not been observed in patients with pneumonia or AMI.¹⁴ Several possible explanations can be given for the observed discrepancy between the diseases and their effect on the relationship between readmission rate and mortality rate. Both COPD and HF are chronic diseases and characterized by exacerbations, whereas AMI and pneumonia are episodic diseases that are treatable. As the number of patients hospitalized with AECOPD and HF is much larger, hospitals may have a greater focus on reducing the 30-day readmission rates and may attempt to game the process, such as by delaying admissions through the emergency department within the 30-day period or by admitting patients for observation. In fact, a study found a 3% reduction in the within-hospital readmission rate with a concurrent 0.8% increase in observation unit use since the implementation of the HRRP.²⁶ Such approaches to patient care may lead to adverse outcomes.

It is possible that readmissions and mortality act as competing risks and hence hospitals with higher mortality rates are left with fewer patients and thus have fewer readmissions, whereas those with lower mortality rates have more patients and a higher readmission rate.²⁷ Such studies are not possible with hospital-level data, and patient-level studies will be required to examine this competing risk hypothesis. Our study results provide some support to the competing risk hypothesis (hospitals with lower baseline readmission rates had a steeper increase in mortality); however, it is not possible to draw any conclusions due to the high risk of ecological fallacy bias.

This study has important potential implications for healthcare policy, public health, and research. We found that an important national intervention aimed at decreasing readmission rates and improving the quality of care for patients with AECOPD may be associated with higher mortality rates in these patients. There may be a need to redefine measures for determining the performance of an institution. Our study supports research into the underlying mechanisms resulting in an inverse association between readmissions and mortality. In particular, health policy researchers may need to examine how incentives and penalties affect the allocation of resources within hospitals.

This study has several strengths and some potential weaknesses. We used a national dataset to examine readmission and mortality rates that include the majority of hospitals in the United States. We also included data from the local population for each hospital, thus allowing us to examine hospital performance within the context of its target population. One potential limitation is that we used hospital-level data and not patient-level data; however, the readmission penalties are designed for hospitals, which justifies our use of hospital-level data. Furthermore, data were not available for shorter time intervals; data from shorter time intervals may be associated with greater variability. Being an observational study, it is difficult to establish a causal relationship; the longitudinal nature of the study does establish temporality, an important factor in establishing causality.

In conclusion, we found that although the readmission rates decreased, there was an increase in the mortality rate within the 30 days of discharge from the hospital in patients with AECOPD. The rate of increase in mortality was higher in hospitals with lower readmission rates than in hospitals with higher readmission rates. Further research for determining the mechanism responsible for this association is needed. Future health policy interventions may need to consider the potential for adverse outcomes.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

The Choosing Wisely^® Campaign (CWC) was launched in 2012. This ongoing national initiative encourages conversations among patients and clinicians about the need —or the lack thereof—for frequent tests, treatments, and procedures in healthcare. More than 80 professional societies have developed short lists of evidence-based recommendations aimed at avoiding unnecessary, “low-value” care. More than 550 recommendations are currently available.¹ The Society of Hospital Medicine (SHM) Pediatric Committee published a list of five recommendations for the CWC in 2013.²

After seven years, the campaign has posted several success stories highlighting the increase in clinicians’ awareness about the recommendations. Several local, regional, and national initiatives and quality improvement (QI) projects have been inspired by the CWC and its tenants.^1,3 However, limited research has been performed on the true impact of these recommendations on avoiding “low-value” services. A more comprehensive approach is required to “measure wisely” the impact of the campaign on bedside clinical practice.⁴ Stakeholders in healthcare value have been challenged to collaborate in creating high-impact lists of “low-value” interventions and designing effective tools to measure their impact on clinical practice and costs.⁵

We initially developed a report card with five metrics derived from the CWC-SHM pediatric recommendations to help individual institutions and group practices to measure their performance and benchmark their results with peers.⁶ The report card is available for hospital members of the Children’s Hospital Association (CHA).⁷

The current study analyzes the frequency of utilization and trends of five metrics included in the CHA/Pediatric Health Information System^® (PHIS) CWC report card in tertiary children’s hospitals in the United States. We analyzed data from five years before and five years after the CWC-PHM recommendations were published in 2013. We hypothesize that the publication and dissemination of the CWC-PHM recommendations—the intervention—will result in either an immediate decrease in the use of the “low-value” services studied and/or a change in the trend of utilization over time.

Study Design

We conducted an observational, longitudinal retrospective study aimed at evaluating the impact of the CWC-PHM recommendations on clinical practice in tertiary children’s hospitals in the US.

Study Population

The population included inpatient and observation stays for children aged 0-18 years admitted to the 36 children’s hospitals consistently providing data from 2008 to 2017 to the PHIS administrative database (CHA, Lenexa, Kansas). This database contains inpatient, emergency department, ambulatory, and observation encounter–level data from more than 50 not-for-profit, tertiary care pediatric hospitals and accounts for ~20% of all pediatric hospitalizations in the US every year.

A joint effort between the CHA and the participating hospitals ensures the quality of the data submitted, as previously described.⁸ These data are subjected to a routine quality check with each submission and within each report. Data were fully deidentified for this study. In total, 36 PHIS hospitals met the strict quality standards for inclusion of submitted data. The remaining hospitals were excluded because they did not have complete data or had incomplete billing information.

For external benchmarking purposes, PHIS participating hospitals provide encounter data, including demographics, diagnoses, and procedures (International Classification of Diseases versions 9 and 10).^9,10 The transition from ICD-9 to ICD-10 in the US took place during the study period. However, the CHA completed a process of translating and mapping all ICD-9 codes to every possible equivalent ICD-10 code in the PHIS database. Thus, the change from ICD-9 to ICD-10 should not have had any significant effect on population definition and data analytics, including trend analysis.

For each condition, the study population was divided into the following two cohorts for comparison of the trends: all admissions from January 1, 2008 to December 31, 2012 (before) and all admissions from January 1, 2013 to December 31, 2017 (after) the CWC-PHM recommendations were published.

This study was determined to be nonhuman subject research and was therefore exempted by Nicklaus Children’s Hospital Human Research Protection Program.

Outcomes

The outcomes for this study were the percentages of patients receiving the not-recommended “low-value” services targeted by the CWC-PHM recommendations. For this purpose, four of the five recommendations were translated into the following five metrics, operationalized in the PHIS database and displayed in the “Choosing Wisely” report card:⁶

1. Percentage of patients with uncomplicated asthma receiving chest radiograph (CXR).

2. Percentage of patients with uncomplicated bronchiolitis receiving CXR.

3. Percentage of patients with uncomplicated bronchiolitis receiving bronchodilators.

4. Percentage of patients with lower respiratory tract infection (LRTI) receiving systemic corticosteroids (relievers).

5. Percentage of patients with uncomplicated gastroesophageal reflux (GER) receiving acid suppressor therapy.

The fifth recommendation—limiting the use of continuous pulse oximetry unless the patient is receiving supplemental oxygen—could not be operationalized in the PHIS database because of inconsistent reporting of these resources.⁶

The resulting percentages represent nonadherence to the recommendations, suggesting overuse of the specific “low-value” intervention. As such, a decreasing trend over time is the desired direction of improvement.

The definition of “uncomplicated” conditions and the metrics are presented in Table 1. A complete list of the inclusion and exclusion criteria to define “uncomplicated” conditions and the complete list of the clinical translation codes used in PHIS to identify the “low-value” services are presented as an electronic supplement.

Statistical Analyses

We compared the demographic and clinical characteristics of the various cohorts before and after the release of the CWC-PHM recommendations—the intervention—using chi-square statistics. To assess the individual hospital-level trends over time for each measure, we modeled the patient-level data of each hospital using generalized linear mixed effects models with a binomial distribution. These models were adjusted for patient demographic and clinical factors that were found to be significantly different (P < .01) before and after the intervention on bivariate analyses. From these models, we generated adjusted estimates for the quarterly percentages for each hospital. We then conducted an interrupted time series (ITS) using these estimates to compare trends in the five years before (2008-2012) and five years after (2013-2017) the publication of the CWC-PHM recommendations. For the ITS analysis, we used a generalized linear mixed effects model with the quarterly adjusted hospital-level utilization rates of “low-value” services for each cohort as the unit of analysis and a random intercept for each hospital. The model used an autoregressive(1) covariance structure to account for autocorrelation. The ITS allowed us to test our hypothesis by assessing the following two important features: (a) if a significant decrease occurred right after the CWC-PHM recommendations were published (level-change) and/or (b) if the intervention altered the secular trend (slope-change). All statistical analyses were performed using SAS v. 9.4 (SAS Institute, Cary, North Carolina), and P values <.01 were considered to be statistically significant.

Table 2 presents the demographic characteristics of the cohorts before (2008-2012) and after (2013-2017) the publication of the CWC-PHM recommendations. Hospitalizations due to asthma represented the largest cohort with 142,067 cases, followed by hospitalizations due to bronchiolitis with 94,253 cases. Hospitalizations due to GER comprised the smallest cohort with 13,635 cases. Most of the children had government insurance and had “minor” severity according to the All Patient Revised Diagnosis Related Group (APR-DRG) system.

We found statistically significant differences in most of the demographic characteristics for the cohorts when comparing cases before and after the introduction of the CWC-PHM recommendations.

After adjusting for demographic characteristics, we estimated the percentages of the utilization of the “low-value” services from 2008 to 2017. We observed a steady decrease in overutilization of all services over time. The absolute percentage decrease was more evident in the reduction of the utilization of relievers by 36.6% and that of CXR by 31.5% for bronchiolitis. We also observed a 20.8% absolute reduction in the use of CXR for asthma.

The use of systemic steroids in LRTI revealed the lowest utilization among the “low-value” services studied, with 15.1% in 2008 and 12.2% in 2017, a 2.9% absolute reduction. However, the prescription of acid suppressors for GER showed the highest utilization among all the overuse metrics studied, ie, 63% in 2008 and 48.9% in 2017, with an absolute decrease of 24.1%. The yearly adjusted estimated percentages of utilization for each “low-value” service are presented in Appendix Table A.

Table 3 and the Figure (attached as supplemental online graphic) respectively present the risk-adjusted ITS parameter estimates and the graphic representation before and after the inception of the CWC-PHM recommendations for the trend analysis.

We found a steady reduction in the frequency of overutilization of five “low-value” services described in the CWC-PHM recommendations from 2008 to 2017 in 36 tertiary children’s hospitals in the US. This trend was more evident in the utilization of relievers and CXR for bronchiolitis. The ITS analysis demonstrated that immediately after the publication of the CWC-PHM recommendations, only the use of CXR for asthma decreased significantly. Then, only the use of relievers for bronchiolitis decreased significantly over time in comparison with the secular trend.

These results support our hypothesis for two of the five metrics studied, suggesting that the publication of the CWC-PHM recommendations had a modest impact in clinical practices related to those services in tertiary children’s hospitals.

These findings align with a limited number of published studies that have consistently found a modest decrease in the use of “low-value” services before 2012^11-13 and a limited impact of the CWC in clinical practices on the use of “low-value” services after the inception of the campaign.^14-17

For instance, in a cross-sectional analysis of the 1999 and 2009 samples of ambulatory care practices in the US, only two of 11 overuse quality indicators showed improvement.¹¹ The authors recognized that reducing inappropriate care will require the same attention to guideline development and performance measurement that was directed at reducing the underuse of needed therapies. However, determining whether a patient received inappropriate care generally requires a much more detailed analysis of clinical information than what is required for assessments of underuse.¹¹

Another study designed claims-based algorithms to measure the prevalence of 11 Choosing Wisely-identified “low-value” services in fee-for-service Medicare patients aged >65 years from 2006 to 2011.¹² The annual prevalence of selected CWC “low-value” services ranged from 1.2% (upper urinary tract imaging in men with benign prostatic hyperplasia) to 46.5% (preoperative cardiac testing for low-risk, noncardiac procedures). The study concluded that identifying and measuring “low-value” health services is a prerequisite for improving quality and eliminating waste.¹²

In pediatric medicine, the authors investigated a large cohort of infants aged one to 24 months hospitalized with bronchiolitis to 41 tertiary children’s hospitals reporting data to the PHIS database from 2004 to 2012.¹³ The trend analysis revealed a decrease in the utilization of diagnostics and treatment interventions before the publication of the American Academy of Pediatrics 2006 Bronchiolitis Guidelines.¹⁸ There was an additional reduction in the use of CXR, steroids, and bronchodilators after the publication of the guidelines.¹³

After the CWC was launched in 2012, several surveys have demonstrated a tangible increase in awareness of the CWC and its goals, mostly among primary care physicians and subspecialists. Clinicians who were aware of the campaign found the recommendations to be useful as a legitimate source of guidance and were more likely to reduce the indication of unnecessary care and “low-value” clinical services included in the CWC.^1,3,19,20

Few studies in adults have focused on measuring the trends in overuse metrics derived from the CWC recommendations.^14-16 The initial studies have found limited reduction on the use of “low-value” care after the inception of the CWC. They suggest that clinician education, awareness, and public promotion alone do not appear to be sufficient to achieve widespread changes in clinical practice. Additional interventions are necessary for the wider implementation and success of the CWC recommendations.^{11,14,15,19,21,22}

However, a more recent study was conducted in 91 academic centers from 2013 through 2016, before and after the publication of a CWC recommendation on the use of troponin-only testing for the diagnosis of acute myocardial infarction. Hospitals with low rates of troponin-only testing before the publication of the recommendation demonstrated a statistically significant increase over time in the rate of adherence. The authors postulated that the impact of the CWC might have been significant because of the increase in the institutional and provider attention to “high-value” care as a result of the campaign.¹⁶

In pediatrics, a cross-sectional study defined 20 “low-value” services from a list of more than 400 items from the CWC and other sources of highly regarded, evidence-based pediatrics healthcare recommendations. The list included six diagnostic tests, five imaging tests, and nine prescription drugs ordered in a robust cohort of 4.4 million children nationwide in 2014. The study concluded that approximately one in 10 children received a “low-value” service. The majority (59.4%) were related to prescription drugs, specifically the inappropriate use of antibiotics for a variety of conditions. The estimated combined cost of these unnecessary services was approximately $27 million, with one-third of the cost being paid out of pocket, arguing for significant financial harm. However, this study did not perform a trend analysis.¹⁷

Our results are comparable with these studies, reporting an initial increase in awareness and beliefs, followed by progressive changes in clinical practice among pediatric hospital-based clinicians in delivering evidence-based, high-value care after the CWC.

The attribution of the steady reduction in the absolute percentages of overuse/waste in the five metrics related to the CWC observed in this study, including the significant changes noted in two of the overuse indicators after the publication of the CWC-PHM recommendations, should be interpreted with caution. For example, the significant decrease in the use of “low-value” services in bronchiolitis could be attributed to multiple factors such as national guidelines released in 2014 after the campaign,²³ national multicenter QI collaborative projects,^24,25 and multiple local QI efforts.^26,27 The increase in the awareness and impact of the CWC recommendations among pediatric providers could also be a contributing factor, but this association cannot be established in the light of our findings.

On the other hand, despite extensive evidence for the lack of efficacy and the potential harm associated with the use of acid suppressors for uncomplicated GER in infants,^28-30 the frequency of this “low-value” therapeutic intervention remains high (~50%). The trend in utilization was not impacted by the CWC-PHM recommendations. This finding could be explained by several factors, including the possibility that several hospitalized patients may suffer from GER disease requiring acid suppressors. Another possibility is that acid suppressors are generally prescribed as an outpatient medication, and physicians treating inpatients may be reluctant to discontinue it during hospitalization. Nevertheless, this recommendation represents a target for review, update, and QI interventions in the near future.

The delivery of inappropriate “low-value” care represents the most significant dimension of waste in healthcare.³¹ The development of quality measures of “low-value” services representing overuse and waste is the most needed step toward assessing the magnitude of the problem. Overuse metrics could be incorporated into QI interventions to decrease the provision of such services. However, systematic efforts aimed at developing quality indicators of overuse based on the CWC recommendations have been limited. To our knowledge, this is the first study on the trends of metrics derived from the CWC recommendations in pediatric medicine.

Future research is needed to develop overuse metrics further to assess the specific outcomes related to the implementation of the CWC. How much has clinical practice changed as a result of the campaign? What are the outcomes and savings attributable to these efforts? These are critical questions for the immediate future that should be answered to sustain the ongoing efforts and results and to validate that the efforts are worthwhile.

This study has several limitations. First, this is a retrospective and observational study. It cannot prove a direct causal relationship between the publication of the CWC-PHM and the observed trends, as other potential factors may have contributed to the outcomes. Second, in administrative databases, the data quality is dependent on proper documentation and coding that may vary among reporting institutions. These data lack clinical information, and a fair assessment of “appropriateness” could be questioned. In addition, the study included only 36 academic, tertiary children’s hospitals. Because approximately two-thirds of all pediatric hospitalizations in the US occur in community settings,³² this study may not fully represent clinical practice in the majority of pediatric hospitalizations in the US. Finally, the validity of the ITS analysis has inherent limitations due to the variability of the data in some metrics that may affect the power of the analysis. This fact could lead to inaccurate conclusions regarding intervention effectiveness due to the data-driven model applied, as well as the lack of control for other time-varying confounders.³³

After seven years, the CWC faces important challenges. Critical to the success of the campaign is to “measure wisely” by developing quality indicators of overuse and operationalizing them into administrative and clinical data sources to assess the impact on clinical practice. Our study highlights some limited but steady reduction in the use of some “low-value” services before the campaign. It also demonstrates a modest impact of the campaign on clinical practices in tertiary care children’s hospitals in the US. Clinicians and institutions still have a long way to go in reducing the use of “low-value” interventions in pediatric medicine. These observations challenge us to step up our efforts to implement QI interventions aimed at incorporating these professional, society-endorsed recommendations into our clinical practice.

Acknowledgments

The authors thank Dr. Kristine De La Torre and Dr. Jennifer McCafferty-Fernandez and the Research Institute of Nicklaus Children’s Hospital for medical writing assistance. They also acknowledge Tatiana Consuegra, library technician, for her clerical assistance in the preparation and submission of this article.

Despite the widespread availability of evidence-based guidelines,¹ there is inappropriate variation in the care and outcomes for children with asthma in both the emergency department (ED) and the inpatient setting.^2-6 Operational versions of evidence-based guidelines known as “pathways” have been shown to improve adoption of evidence-based guidelines, quality of care, and health outcomes for children with asthma.^7-14 However, little is known about how to successfully implement pathways outside of free-standing children’s hospitals.^15-19

The majority of children with asthma in the United States are cared for in community hospitals, which provide services for both adults and children.²⁰ However, prior studies of pediatric asthma pathways have largely excluded community hospitals. These studies primarily focused on determining clinical effectiveness, rather than detailing the implementation process. These approaches have left critical gaps that hinder our ability to implement pathways and improve care in community hospitals, which have unique barriers and less resources.^21,22 Therefore, understanding the process of pathway implementation in community hospitals is critical to improving care for children.²² Our objective was to identify the key determinants of successful pediatric asthma pathway implementation using a national sample of community hospitals. This knowledge can guide hospital leaders and healthcare providers in efforts to improve pediatric care and outcomes in these settings.

Study Setting, Design, and Population

In Fall 2017, the Value in Inpatient Pediatrics (VIP) network launched PIPA, Pathways to Improving Pediatric Asthma care.²³ The VIP network, a part of the American Academy of Pediatrics (AAP), aims to improve the value of care delivered to any pediatric patient in a hospital bed, from rural to free-standing children’s hospitals.²⁴ PIPA used a learning collaborative model²⁵ and recruited local project leaders (physicians, nurses, respiratory therapists (RT), and pharmacists) from 89 hospitals around the country. PIPA provided these hospital teams with asthma pathways and several resources for implementation support, including educational meetings, quality improvement (QI) training, audit and feedback, and facilitation. Facilitation is a process of interactive problem-solving and support that occurs in the context of a supportive interpersonal relationship and a recognized need for improvement.²⁶ A facilitator, or a “coach”, is an external expert who provides project mentorship and assists the process of making meaningful changes to improve patient care.²⁶ Facilitation was provided by external consultants with QI expertise.

For this qualitative study, facilitators conducted semi-structured interviews with a convenience sample of project leaders from community hospitals participating in PIPA, with some interviews including multiple project leaders (eg, nursing, inpatient, and Emergency Department [ED] leaders). Verbal consent was obtained from all participants. No incentives were provided. This study was approved by the AAP institutional review board.

Data Collection

We used the constructs described in the Consolidated Framework for Implementation Research (CFIR)²⁷ and adapted those salient to pediatric asthma pathways to develop an interview guide that was used with all participants (Appendix 1). The CFIR offers an overarching typology to understand what works where and why across five major domains that influence implementation: intervention characteristics, inner setting (hospital), outer setting (economic, political, and social context of the hospital), characteristics of the individuals involved, and the process of implementation. Data were collected across these domains to inform our analysis of the key determinants of pediatric asthma pathway implementation in community hospitals.

Interviews were conducted by phone from December 2017 to April 2018 (first four months of pathway implementation). Interviews lasted 30-60 minutes and were recorded and transcribed verbatim. Transcripts were edited for accuracy using the audio recordings. As data collection occurred concurrently with analysis, the interview guide was iteratively revised to reflect new insights and patterns that emerged from our analysis. All sites were anonymized in the data analysis. New interviews were coded until thematic saturation was reached.

Analysis

We conducted an inductive thematic analysis using the CFIR as our conceptual framework.^28,29 Four investigators (CM, MJ, ES, and SK) performed the initial open coding of the data. Investigators met twice during the open coding process to develop and then finalize a codebook of standard definitions for codes. This codebook facilitated coding consistency through the remainder of the analytic process. Two investigators (CM and MJ) then independently read and coded all data to ensure intercoder reliability. During this process, CM and MJ met every two weeks to compare coding consistency, resolve discrepancies, and discuss preliminary findings. When the coding was complete, all investigators met to explore and develop themes that encompassed related common codes.

The CFIR was used at two stages of the study: (1) developing the interview guide and (2) cross-checking for any potentially important codes that were missing/needed to be explored further. Thus, the investigators maintained an inductive approach grounded in the data. To assure study rigor, we employed investigator triangulation (use of multiple investigators and participants from multiple clinical roles) and reflexivity (ongoing critique and critical reflection of the individual biases of the investigators).³⁰ Coding was performed using Dedoose (version 7.0.23; Los Angeles, California).

A total of 34 community hospitals completed the PIPA project, of which the project leaders of 25 hospitals connected with the facilitators and were approached to participate; 18 (72%) hospitals’ project leaders participated in the study. We analyzed 18 interviews conducted between facilitators and project leaders, which included a total of 32 project leaders (one to five leaders per interview). The hospitals represented were diverse in geographic location and size (range 4-50 pediatric beds per hospital), and the majority of sites (78%) supported the trainees (Table 1).

We identified four overarching themes that described the key determinants of pathway implementation in community hospitals. These themes are presented in order of their frequency of occurrence in the data. They included (1) building an implementation infrastructure, (2) engaging and motivating providers, (3) addressing organizational and resource limitations, and (4) devising implementation solutions with facilitators. Descriptions and exemplary quotations for each theme are provided in Table 2 and Appendix Figure 1.

Building an Implementation Infrastructure

“So, I’m going to sit down with the primary nursing staff and the other four physicians in the group to go over the expectations…We’re not going to have the actual EMR [changes] and we’re not going to have the nursing documentation field built right away but [we want to] make sure that people are documenting the respiratory score in their generic nursing note so that the information is easily accessible.” (Physician leader, Hospital G)

Participants also described the need to deliver education on the evidence supporting changes in practice and skills training specific to pediatric asthma care:

“Once we realized that we were going to be doing this pathway, we started training our nurses on the inpatient side on [pediatric respiratory scoring].” (Nursing leader, Hospital P)

In addition, pathway implementation required modification of clinical workflows via changes to hospital policies or guidelines, electronic medical records (EMR), and/or the physical environment (eg, placing supplies in proximity to care delivery):

“I think it can help if we could get an order set or a nursing protocol where asthmatics over a certain severity can just get steroids in triage.” (Physician leader, Hospital A)

Engaging and Motivating Providers

Another crucial step in pathway implementation was engaging and motivating providers. This included overcoming inertia to practice change, facilitating multidisciplinary collaboration, and handling conflicts regarding practice changes. Participants discussed the excitement of participating in a national collaborative as especially motivating to help drive engagement and overcome barriers to change, particularly the ability to compare local hospital performance to national peers.

“I think everyone is a little competitive. So I think that when we see how we compare to other institutions—both our group and the ER—I think it also adds a little oomph…I think for our nurses too; we’re able to say, ‘[look how we compare to] most of the other hospitals.’ I think that helps.” (Physician leader, Hospital B)

Multidisciplinary collaboration across a wide variety of frontline pediatric and nonpediatric providers was key to understanding current workflows and identifying needed modifications for pathway implementation:

“I do think clearly our biggest obstacles are the fact that we have adult ED providers. We have the opportunity on the inpatient side [with nursing and respiratory therapy], who really do awesome with pediatric changes, to take our wins where we can and make the changes with the ED. In the ED we have an RN educator. She’s very on board with doing the respiratory scoring and getting this whole thing started.” (Physician leader, Hospital L)

Intentional communication and leadership skills also played key roles in engaging hesitant providers and handling conflict:

“Just sitting and talking with our respiratory therapist about the ability to provide this type of service or support and seeing what their reservations have been, at least it’s open to conversation so that we could provide these types of therapies in the future and we’re able to see like what people’s concerns are. I think just basically increasing familiarity with not only these processes, but different types of therapy will hopefully in the future help us provide better care to our patients.” (Physician leader, Hospital Q)

Addressing Organizational and Resource Limitations

Participants recognized organizational and resource limitations, some of which may be unique to community hospitals that prioritize resources for adult care. The limitations described included EMR staff support, healthcare provider staffing/capacity, navigating IRBs, and addressing administrative processes. Competing demands for information technology staff support and lack of prioritization of pediatric-specific initiatives often hindered efforts to modify the EMR.

“Resource wise, we are hoping to implement an order set in our Epic EMR, [but] finding the availability from the Epic team may be a challenge.” (Physician Leader, Hospital A)

Participants also reported that limited staff capacity (eg, nursing, RT) hindered pathway implementation efforts. This limited capacity hindered workflow changes and limited the time available for education and training on pathways:

“[Respiratory scoring for asthma is] an added responsibility for the [nursing] staff and we don’t have patient technicians. So they’re doing everything from changing the sheets to bringing water to all of the medical patients. So, that I think may be a barrier.” (Physician leader, Hospital B)

Across sites, navigating the IRB posed various challenges. Some sites were required to obtain approval from regional IRBs, which did not have resources to devote to pediatric projects. Other sites did not have IRBs at all, but instead required separate approvals for the project from hospital leadership or other entities:

“On the IRB, I contacted the manager of the IRB and she’s said, ‘No, it’s not an IRB project,’ but she sent it to another director for review, and it took forever to be able to get a data agreement with [the local university hospital] so that we can pull the data. I just couldn’t believe it took months to get done.” (Physician Leader, Hospital K)

Finally, administrative barriers such as addressing formulary changes in the context of adult-focused settings were challenging. For example, at one hospital, metered dose inhalers (MDIs) were not used for adult patients, and the hospital administration was resistant to incorporate their use into practice for pediatric patients due to the cost of such changes.

“The [general hospital] didn’t have MDI’s anymore because of cost reasons, and when we started the pediatric work, we really made it a point to get the MDI’s for pediatric patients back in the formulary.” (Physician leader, Hospital A)

Devising Implementation Solutions with Practice Facilitators

Participants often devised pathway implementation solutions with facilitators in-the-moment during meetings. This problem-solving included figuring out work-arounds, proactive coaching by external facilitators, and just-in-time solution building. Furthermore, in meetings that included more than one project leader, leaders would often work with each other to devise solutions. Meetings provided forums that stimulated identification of implementation barriers, brainstorming, and subsequently solution building.

Physician leader: I’m wondering if we could, as an interim solution, try out an algorithm on paper, I don’t know if that’s allowed, until we get Epic approval. Do you know?

Nurse Leader: You mean having an algorithm posted in triage? Yeah, I don’t see why not. (Hospital A)

Next, problem solving was often driven by the facilitator’s experience and knowledge, drawn from their interactions with other collaborative sites or their own prior experiences with asthma, QI, or pathway implementation. The facilitators brought an outside perspective, not bound by that particular hospital’s local culture or structural intricacies. This proactive coaching spurred the identification of creative, yet practical solutions:

Project Leader: We’re still trying to get all our templates [for the EMR]…because [currently they are] all adult templates.

Facilitator: If you’re making templates right now, could you also add the three asterisks? Like smoking or exposure to second hand tobacco smoke or marijuana…then have the three asterisks there and then “Referral made?***”. That would force people to document in a certain place in the template as well.Project Leader: That’s definitely something we could add right now. (Hospital O)

Check-in meetings with facilitators offered an opportunity to trouble shoot, brainstorm work-arounds, devise in-the-moment site-specific solutions to enable successful pathway implementation, and provide ongoing support throughout implementation.

Pathways can improve the quality of care for children with asthma.³¹ However, there is little evidence-based guidance on how to implement pathways and improve pediatric care in community hospitals,^17-20 where the majority of children are cared for nationally. This is the first study to our knowledge that details the key determinants of pediatric asthma pathway implementation in community hospital settings. We identified four key determinants of implementation that can help guide others in similar settings. These include building an implementation infrastructure, engaging and motivating multidisciplinary providers, addressing organizational and resource limitations, and using external facilitators to devise implementation solutions.

Existing frameworks such as the CFIR outline the potential determinants of implementation success but do not provide population- or setting-specific guidance.²⁷ There have been prior studies detailing pathway implementation for pediatric populations, but these studies did not focus on community hospitals.^32,33 Our findings align with these prior studies, which highlight the importance of identifying implementation champions, engaging and motivating multidisciplinary providers, establishing a QI infrastructure, and addressing organizational and resource limitations, such as EMR integration.^32,33 However, our study provides unique insights into issues that are important to successful pathway implementation in community hospitals, including engagement of adult-focused healthcare providers, reprioritization of resources toward the care of children, and the potentially critical role of external facilitators.

Our findings indicate that community hospitals seeking to improve care for children may particularly benefit from using external facilitators and/or partnering with external organizations. We found that external facilitators played a significant and proactive role in community hospitals’ efforts to improve care for children. Facilitators helped devise work-arounds and engaged in just-in-time solution building with local project leaders. For instance, facilitators helped develop strategies for training healthcare providers in performing new clinical tasks, building reminders of pathway recommendations into clinical workflows, and overcoming resource barriers. Thus, community hospitals may uniquely benefit from participation in national learning collaboratives, which often provide avenues for external facilitation.^25,34,35 National networks, such as the VIP network, lead national learning collaboratives that provide external facilitation as well as other resources (eg, educational materials, data analysis support) to community hospitals seeking to improve pediatric care.²⁴ Previous work by McDaniel et al. identified that intentional partnerships between children’s and community hospitals can also potentially provide access to resources for education and training in pediatric care and support in navigating organizational and resource challenges.²²

Our results characterize the key determinants of pediatric asthma pathway implementation using a national sample of community hospitals that were diverse in geography, size, and structure. This imparts greater transferability of our findings. We also used strategies to promote the rigor of our findings, including triangulation and reflexivity. However, our study has several limitations. First, we analyzed only the meetings that occurred during the early months of pathway implementation. As such, we did not capture any key determinants that may have arisen later in implementation. However, process analyses of implementation indicate that the majority of implementation efforts occurred within these first three to four months.³⁶ Second, we did not elicit input from hospital administration or leadership. The lack of administrative/leadership input probably affected the CFIR themes we found, as no themes from the outer setting were elicited. However, the goal of our study was to characterize the experiences of those leading implementation efforts, and focusing on these leaders allows our work to better guide those doing similar work in the future. Third, we used CFIR to guide the development of our interview guide and as a reference during analysis, which may have skewed our findings to preferentially reflect CFIR constructs. However, our overall analysis was grounded in the primary data and we employed reflexivity during all stages of our analysis. In addition, having the facilitators conduct the qualitative interviews may have biased our findings toward the perspectives of the facilitators; however, the facilitators represented quite diverse clinical and QI backgrounds. Finally, our findings do not necessarily correlate with improvements in clinical outcomes. As such, they are not meant to serve as explicit recommendations for improving patient outcomes, but rather as a characterization of the context, processes, and experiences of implementing pathways in the community setting to inform others doing this important work.

CONCLUSIONS

We identified the key determinants of pediatric asthma pathway implementation in community hospitals, which may help inform QI efforts in these settings. We also identified organizational and resource limitations that are probably unique to these adult-focused hospitals. Participating in national learning collaboratives and/or working with facilitators may support pathway implementation and improved quality of care for children with asthma in community hospitals.

Future work should seek to correlate these and other determinants of pathway implementation with health outcomes for hospitalized children, as well as integrate broader and more diverse samples of community hospitals.

Despite the widespread availability of evidence-based guidelines,¹ there is inappropriate variation in the care and outcomes for children with asthma in both the emergency department (ED) and the inpatient setting.^2-6 Operational versions of evidence-based guidelines known as “pathways” have been shown to improve adoption of evidence-based guidelines, quality of care, and health outcomes for children with asthma.^7-14 However, little is known about how to successfully implement pathways outside of free-standing children’s hospitals.^15-19

The majority of children with asthma in the United States are cared for in community hospitals, which provide services for both adults and children.²⁰ However, prior studies of pediatric asthma pathways have largely excluded community hospitals. These studies primarily focused on determining clinical effectiveness, rather than detailing the implementation process. These approaches have left critical gaps that hinder our ability to implement pathways and improve care in community hospitals, which have unique barriers and less resources.^21,22 Therefore, understanding the process of pathway implementation in community hospitals is critical to improving care for children.²² Our objective was to identify the key determinants of successful pediatric asthma pathway implementation using a national sample of community hospitals. This knowledge can guide hospital leaders and healthcare providers in efforts to improve pediatric care and outcomes in these settings.

Study Setting, Design, and Population

In Fall 2017, the Value in Inpatient Pediatrics (VIP) network launched PIPA, Pathways to Improving Pediatric Asthma care.²³ The VIP network, a part of the American Academy of Pediatrics (AAP), aims to improve the value of care delivered to any pediatric patient in a hospital bed, from rural to free-standing children’s hospitals.²⁴ PIPA used a learning collaborative model²⁵ and recruited local project leaders (physicians, nurses, respiratory therapists (RT), and pharmacists) from 89 hospitals around the country. PIPA provided these hospital teams with asthma pathways and several resources for implementation support, including educational meetings, quality improvement (QI) training, audit and feedback, and facilitation. Facilitation is a process of interactive problem-solving and support that occurs in the context of a supportive interpersonal relationship and a recognized need for improvement.²⁶ A facilitator, or a “coach”, is an external expert who provides project mentorship and assists the process of making meaningful changes to improve patient care.²⁶ Facilitation was provided by external consultants with QI expertise.

For this qualitative study, facilitators conducted semi-structured interviews with a convenience sample of project leaders from community hospitals participating in PIPA, with some interviews including multiple project leaders (eg, nursing, inpatient, and Emergency Department [ED] leaders). Verbal consent was obtained from all participants. No incentives were provided. This study was approved by the AAP institutional review board.

Data Collection

We used the constructs described in the Consolidated Framework for Implementation Research (CFIR)²⁷ and adapted those salient to pediatric asthma pathways to develop an interview guide that was used with all participants (Appendix 1). The CFIR offers an overarching typology to understand what works where and why across five major domains that influence implementation: intervention characteristics, inner setting (hospital), outer setting (economic, political, and social context of the hospital), characteristics of the individuals involved, and the process of implementation. Data were collected across these domains to inform our analysis of the key determinants of pediatric asthma pathway implementation in community hospitals.

Interviews were conducted by phone from December 2017 to April 2018 (first four months of pathway implementation). Interviews lasted 30-60 minutes and were recorded and transcribed verbatim. Transcripts were edited for accuracy using the audio recordings. As data collection occurred concurrently with analysis, the interview guide was iteratively revised to reflect new insights and patterns that emerged from our analysis. All sites were anonymized in the data analysis. New interviews were coded until thematic saturation was reached.

Analysis

We conducted an inductive thematic analysis using the CFIR as our conceptual framework.^28,29 Four investigators (CM, MJ, ES, and SK) performed the initial open coding of the data. Investigators met twice during the open coding process to develop and then finalize a codebook of standard definitions for codes. This codebook facilitated coding consistency through the remainder of the analytic process. Two investigators (CM and MJ) then independently read and coded all data to ensure intercoder reliability. During this process, CM and MJ met every two weeks to compare coding consistency, resolve discrepancies, and discuss preliminary findings. When the coding was complete, all investigators met to explore and develop themes that encompassed related common codes.

The CFIR was used at two stages of the study: (1) developing the interview guide and (2) cross-checking for any potentially important codes that were missing/needed to be explored further. Thus, the investigators maintained an inductive approach grounded in the data. To assure study rigor, we employed investigator triangulation (use of multiple investigators and participants from multiple clinical roles) and reflexivity (ongoing critique and critical reflection of the individual biases of the investigators).³⁰ Coding was performed using Dedoose (version 7.0.23; Los Angeles, California).

A total of 34 community hospitals completed the PIPA project, of which the project leaders of 25 hospitals connected with the facilitators and were approached to participate; 18 (72%) hospitals’ project leaders participated in the study. We analyzed 18 interviews conducted between facilitators and project leaders, which included a total of 32 project leaders (one to five leaders per interview). The hospitals represented were diverse in geographic location and size (range 4-50 pediatric beds per hospital), and the majority of sites (78%) supported the trainees (Table 1).

We identified four overarching themes that described the key determinants of pathway implementation in community hospitals. These themes are presented in order of their frequency of occurrence in the data. They included (1) building an implementation infrastructure, (2) engaging and motivating providers, (3) addressing organizational and resource limitations, and (4) devising implementation solutions with facilitators. Descriptions and exemplary quotations for each theme are provided in Table 2 and Appendix Figure 1.

Building an Implementation Infrastructure

“So, I’m going to sit down with the primary nursing staff and the other four physicians in the group to go over the expectations…We’re not going to have the actual EMR [changes] and we’re not going to have the nursing documentation field built right away but [we want to] make sure that people are documenting the respiratory score in their generic nursing note so that the information is easily accessible.” (Physician leader, Hospital G)

Participants also described the need to deliver education on the evidence supporting changes in practice and skills training specific to pediatric asthma care:

“Once we realized that we were going to be doing this pathway, we started training our nurses on the inpatient side on [pediatric respiratory scoring].” (Nursing leader, Hospital P)

In addition, pathway implementation required modification of clinical workflows via changes to hospital policies or guidelines, electronic medical records (EMR), and/or the physical environment (eg, placing supplies in proximity to care delivery):

“I think it can help if we could get an order set or a nursing protocol where asthmatics over a certain severity can just get steroids in triage.” (Physician leader, Hospital A)

Engaging and Motivating Providers

Another crucial step in pathway implementation was engaging and motivating providers. This included overcoming inertia to practice change, facilitating multidisciplinary collaboration, and handling conflicts regarding practice changes. Participants discussed the excitement of participating in a national collaborative as especially motivating to help drive engagement and overcome barriers to change, particularly the ability to compare local hospital performance to national peers.

“I think everyone is a little competitive. So I think that when we see how we compare to other institutions—both our group and the ER—I think it also adds a little oomph…I think for our nurses too; we’re able to say, ‘[look how we compare to] most of the other hospitals.’ I think that helps.” (Physician leader, Hospital B)

Multidisciplinary collaboration across a wide variety of frontline pediatric and nonpediatric providers was key to understanding current workflows and identifying needed modifications for pathway implementation:

“I do think clearly our biggest obstacles are the fact that we have adult ED providers. We have the opportunity on the inpatient side [with nursing and respiratory therapy], who really do awesome with pediatric changes, to take our wins where we can and make the changes with the ED. In the ED we have an RN educator. She’s very on board with doing the respiratory scoring and getting this whole thing started.” (Physician leader, Hospital L)

Intentional communication and leadership skills also played key roles in engaging hesitant providers and handling conflict:

“Just sitting and talking with our respiratory therapist about the ability to provide this type of service or support and seeing what their reservations have been, at least it’s open to conversation so that we could provide these types of therapies in the future and we’re able to see like what people’s concerns are. I think just basically increasing familiarity with not only these processes, but different types of therapy will hopefully in the future help us provide better care to our patients.” (Physician leader, Hospital Q)

Addressing Organizational and Resource Limitations

Participants recognized organizational and resource limitations, some of which may be unique to community hospitals that prioritize resources for adult care. The limitations described included EMR staff support, healthcare provider staffing/capacity, navigating IRBs, and addressing administrative processes. Competing demands for information technology staff support and lack of prioritization of pediatric-specific initiatives often hindered efforts to modify the EMR.

“Resource wise, we are hoping to implement an order set in our Epic EMR, [but] finding the availability from the Epic team may be a challenge.” (Physician Leader, Hospital A)

Participants also reported that limited staff capacity (eg, nursing, RT) hindered pathway implementation efforts. This limited capacity hindered workflow changes and limited the time available for education and training on pathways:

“[Respiratory scoring for asthma is] an added responsibility for the [nursing] staff and we don’t have patient technicians. So they’re doing everything from changing the sheets to bringing water to all of the medical patients. So, that I think may be a barrier.” (Physician leader, Hospital B)

Across sites, navigating the IRB posed various challenges. Some sites were required to obtain approval from regional IRBs, which did not have resources to devote to pediatric projects. Other sites did not have IRBs at all, but instead required separate approvals for the project from hospital leadership or other entities:

“On the IRB, I contacted the manager of the IRB and she’s said, ‘No, it’s not an IRB project,’ but she sent it to another director for review, and it took forever to be able to get a data agreement with [the local university hospital] so that we can pull the data. I just couldn’t believe it took months to get done.” (Physician Leader, Hospital K)

Finally, administrative barriers such as addressing formulary changes in the context of adult-focused settings were challenging. For example, at one hospital, metered dose inhalers (MDIs) were not used for adult patients, and the hospital administration was resistant to incorporate their use into practice for pediatric patients due to the cost of such changes.

“The [general hospital] didn’t have MDI’s anymore because of cost reasons, and when we started the pediatric work, we really made it a point to get the MDI’s for pediatric patients back in the formulary.” (Physician leader, Hospital A)

Devising Implementation Solutions with Practice Facilitators

Participants often devised pathway implementation solutions with facilitators in-the-moment during meetings. This problem-solving included figuring out work-arounds, proactive coaching by external facilitators, and just-in-time solution building. Furthermore, in meetings that included more than one project leader, leaders would often work with each other to devise solutions. Meetings provided forums that stimulated identification of implementation barriers, brainstorming, and subsequently solution building.

Physician leader: I’m wondering if we could, as an interim solution, try out an algorithm on paper, I don’t know if that’s allowed, until we get Epic approval. Do you know?

Nurse Leader: You mean having an algorithm posted in triage? Yeah, I don’t see why not. (Hospital A)

Next, problem solving was often driven by the facilitator’s experience and knowledge, drawn from their interactions with other collaborative sites or their own prior experiences with asthma, QI, or pathway implementation. The facilitators brought an outside perspective, not bound by that particular hospital’s local culture or structural intricacies. This proactive coaching spurred the identification of creative, yet practical solutions:

Project Leader: We’re still trying to get all our templates [for the EMR]…because [currently they are] all adult templates.

Facilitator: If you’re making templates right now, could you also add the three asterisks? Like smoking or exposure to second hand tobacco smoke or marijuana…then have the three asterisks there and then “Referral made?***”. That would force people to document in a certain place in the template as well.Project Leader: That’s definitely something we could add right now. (Hospital O)

Check-in meetings with facilitators offered an opportunity to trouble shoot, brainstorm work-arounds, devise in-the-moment site-specific solutions to enable successful pathway implementation, and provide ongoing support throughout implementation.

Pathways can improve the quality of care for children with asthma.³¹ However, there is little evidence-based guidance on how to implement pathways and improve pediatric care in community hospitals,^17-20 where the majority of children are cared for nationally. This is the first study to our knowledge that details the key determinants of pediatric asthma pathway implementation in community hospital settings. We identified four key determinants of implementation that can help guide others in similar settings. These include building an implementation infrastructure, engaging and motivating multidisciplinary providers, addressing organizational and resource limitations, and using external facilitators to devise implementation solutions.

Existing frameworks such as the CFIR outline the potential determinants of implementation success but do not provide population- or setting-specific guidance.²⁷ There have been prior studies detailing pathway implementation for pediatric populations, but these studies did not focus on community hospitals.^32,33 Our findings align with these prior studies, which highlight the importance of identifying implementation champions, engaging and motivating multidisciplinary providers, establishing a QI infrastructure, and addressing organizational and resource limitations, such as EMR integration.^32,33 However, our study provides unique insights into issues that are important to successful pathway implementation in community hospitals, including engagement of adult-focused healthcare providers, reprioritization of resources toward the care of children, and the potentially critical role of external facilitators.

Our findings indicate that community hospitals seeking to improve care for children may particularly benefit from using external facilitators and/or partnering with external organizations. We found that external facilitators played a significant and proactive role in community hospitals’ efforts to improve care for children. Facilitators helped devise work-arounds and engaged in just-in-time solution building with local project leaders. For instance, facilitators helped develop strategies for training healthcare providers in performing new clinical tasks, building reminders of pathway recommendations into clinical workflows, and overcoming resource barriers. Thus, community hospitals may uniquely benefit from participation in national learning collaboratives, which often provide avenues for external facilitation.^25,34,35 National networks, such as the VIP network, lead national learning collaboratives that provide external facilitation as well as other resources (eg, educational materials, data analysis support) to community hospitals seeking to improve pediatric care.²⁴ Previous work by McDaniel et al. identified that intentional partnerships between children’s and community hospitals can also potentially provide access to resources for education and training in pediatric care and support in navigating organizational and resource challenges.²²

Our results characterize the key determinants of pediatric asthma pathway implementation using a national sample of community hospitals that were diverse in geography, size, and structure. This imparts greater transferability of our findings. We also used strategies to promote the rigor of our findings, including triangulation and reflexivity. However, our study has several limitations. First, we analyzed only the meetings that occurred during the early months of pathway implementation. As such, we did not capture any key determinants that may have arisen later in implementation. However, process analyses of implementation indicate that the majority of implementation efforts occurred within these first three to four months.³⁶ Second, we did not elicit input from hospital administration or leadership. The lack of administrative/leadership input probably affected the CFIR themes we found, as no themes from the outer setting were elicited. However, the goal of our study was to characterize the experiences of those leading implementation efforts, and focusing on these leaders allows our work to better guide those doing similar work in the future. Third, we used CFIR to guide the development of our interview guide and as a reference during analysis, which may have skewed our findings to preferentially reflect CFIR constructs. However, our overall analysis was grounded in the primary data and we employed reflexivity during all stages of our analysis. In addition, having the facilitators conduct the qualitative interviews may have biased our findings toward the perspectives of the facilitators; however, the facilitators represented quite diverse clinical and QI backgrounds. Finally, our findings do not necessarily correlate with improvements in clinical outcomes. As such, they are not meant to serve as explicit recommendations for improving patient outcomes, but rather as a characterization of the context, processes, and experiences of implementing pathways in the community setting to inform others doing this important work.

CONCLUSIONS

We identified the key determinants of pediatric asthma pathway implementation in community hospitals, which may help inform QI efforts in these settings. We also identified organizational and resource limitations that are probably unique to these adult-focused hospitals. Participating in national learning collaboratives and/or working with facilitators may support pathway implementation and improved quality of care for children with asthma in community hospitals.

Future work should seek to correlate these and other determinants of pathway implementation with health outcomes for hospitalized children, as well as integrate broader and more diverse samples of community hospitals.

Despite the widespread availability of evidence-based guidelines,¹ there is inappropriate variation in the care and outcomes for children with asthma in both the emergency department (ED) and the inpatient setting.^2-6 Operational versions of evidence-based guidelines known as “pathways” have been shown to improve adoption of evidence-based guidelines, quality of care, and health outcomes for children with asthma.^7-14 However, little is known about how to successfully implement pathways outside of free-standing children’s hospitals.^15-19

The majority of children with asthma in the United States are cared for in community hospitals, which provide services for both adults and children.²⁰ However, prior studies of pediatric asthma pathways have largely excluded community hospitals. These studies primarily focused on determining clinical effectiveness, rather than detailing the implementation process. These approaches have left critical gaps that hinder our ability to implement pathways and improve care in community hospitals, which have unique barriers and less resources.^21,22 Therefore, understanding the process of pathway implementation in community hospitals is critical to improving care for children.²² Our objective was to identify the key determinants of successful pediatric asthma pathway implementation using a national sample of community hospitals. This knowledge can guide hospital leaders and healthcare providers in efforts to improve pediatric care and outcomes in these settings.

Study Setting, Design, and Population

In Fall 2017, the Value in Inpatient Pediatrics (VIP) network launched PIPA, Pathways to Improving Pediatric Asthma care.²³ The VIP network, a part of the American Academy of Pediatrics (AAP), aims to improve the value of care delivered to any pediatric patient in a hospital bed, from rural to free-standing children’s hospitals.²⁴ PIPA used a learning collaborative model²⁵ and recruited local project leaders (physicians, nurses, respiratory therapists (RT), and pharmacists) from 89 hospitals around the country. PIPA provided these hospital teams with asthma pathways and several resources for implementation support, including educational meetings, quality improvement (QI) training, audit and feedback, and facilitation. Facilitation is a process of interactive problem-solving and support that occurs in the context of a supportive interpersonal relationship and a recognized need for improvement.²⁶ A facilitator, or a “coach”, is an external expert who provides project mentorship and assists the process of making meaningful changes to improve patient care.²⁶ Facilitation was provided by external consultants with QI expertise.

For this qualitative study, facilitators conducted semi-structured interviews with a convenience sample of project leaders from community hospitals participating in PIPA, with some interviews including multiple project leaders (eg, nursing, inpatient, and Emergency Department [ED] leaders). Verbal consent was obtained from all participants. No incentives were provided. This study was approved by the AAP institutional review board.

Data Collection

We used the constructs described in the Consolidated Framework for Implementation Research (CFIR)²⁷ and adapted those salient to pediatric asthma pathways to develop an interview guide that was used with all participants (Appendix 1). The CFIR offers an overarching typology to understand what works where and why across five major domains that influence implementation: intervention characteristics, inner setting (hospital), outer setting (economic, political, and social context of the hospital), characteristics of the individuals involved, and the process of implementation. Data were collected across these domains to inform our analysis of the key determinants of pediatric asthma pathway implementation in community hospitals.

Interviews were conducted by phone from December 2017 to April 2018 (first four months of pathway implementation). Interviews lasted 30-60 minutes and were recorded and transcribed verbatim. Transcripts were edited for accuracy using the audio recordings. As data collection occurred concurrently with analysis, the interview guide was iteratively revised to reflect new insights and patterns that emerged from our analysis. All sites were anonymized in the data analysis. New interviews were coded until thematic saturation was reached.

Analysis

We conducted an inductive thematic analysis using the CFIR as our conceptual framework.^28,29 Four investigators (CM, MJ, ES, and SK) performed the initial open coding of the data. Investigators met twice during the open coding process to develop and then finalize a codebook of standard definitions for codes. This codebook facilitated coding consistency through the remainder of the analytic process. Two investigators (CM and MJ) then independently read and coded all data to ensure intercoder reliability. During this process, CM and MJ met every two weeks to compare coding consistency, resolve discrepancies, and discuss preliminary findings. When the coding was complete, all investigators met to explore and develop themes that encompassed related common codes.

The CFIR was used at two stages of the study: (1) developing the interview guide and (2) cross-checking for any potentially important codes that were missing/needed to be explored further. Thus, the investigators maintained an inductive approach grounded in the data. To assure study rigor, we employed investigator triangulation (use of multiple investigators and participants from multiple clinical roles) and reflexivity (ongoing critique and critical reflection of the individual biases of the investigators).³⁰ Coding was performed using Dedoose (version 7.0.23; Los Angeles, California).

A total of 34 community hospitals completed the PIPA project, of which the project leaders of 25 hospitals connected with the facilitators and were approached to participate; 18 (72%) hospitals’ project leaders participated in the study. We analyzed 18 interviews conducted between facilitators and project leaders, which included a total of 32 project leaders (one to five leaders per interview). The hospitals represented were diverse in geographic location and size (range 4-50 pediatric beds per hospital), and the majority of sites (78%) supported the trainees (Table 1).

We identified four overarching themes that described the key determinants of pathway implementation in community hospitals. These themes are presented in order of their frequency of occurrence in the data. They included (1) building an implementation infrastructure, (2) engaging and motivating providers, (3) addressing organizational and resource limitations, and (4) devising implementation solutions with facilitators. Descriptions and exemplary quotations for each theme are provided in Table 2 and Appendix Figure 1.

Building an Implementation Infrastructure

“So, I’m going to sit down with the primary nursing staff and the other four physicians in the group to go over the expectations…We’re not going to have the actual EMR [changes] and we’re not going to have the nursing documentation field built right away but [we want to] make sure that people are documenting the respiratory score in their generic nursing note so that the information is easily accessible.” (Physician leader, Hospital G)

Participants also described the need to deliver education on the evidence supporting changes in practice and skills training specific to pediatric asthma care:

“Once we realized that we were going to be doing this pathway, we started training our nurses on the inpatient side on [pediatric respiratory scoring].” (Nursing leader, Hospital P)

In addition, pathway implementation required modification of clinical workflows via changes to hospital policies or guidelines, electronic medical records (EMR), and/or the physical environment (eg, placing supplies in proximity to care delivery):

“I think it can help if we could get an order set or a nursing protocol where asthmatics over a certain severity can just get steroids in triage.” (Physician leader, Hospital A)

Engaging and Motivating Providers

Another crucial step in pathway implementation was engaging and motivating providers. This included overcoming inertia to practice change, facilitating multidisciplinary collaboration, and handling conflicts regarding practice changes. Participants discussed the excitement of participating in a national collaborative as especially motivating to help drive engagement and overcome barriers to change, particularly the ability to compare local hospital performance to national peers.

“I think everyone is a little competitive. So I think that when we see how we compare to other institutions—both our group and the ER—I think it also adds a little oomph…I think for our nurses too; we’re able to say, ‘[look how we compare to] most of the other hospitals.’ I think that helps.” (Physician leader, Hospital B)

Multidisciplinary collaboration across a wide variety of frontline pediatric and nonpediatric providers was key to understanding current workflows and identifying needed modifications for pathway implementation:

“I do think clearly our biggest obstacles are the fact that we have adult ED providers. We have the opportunity on the inpatient side [with nursing and respiratory therapy], who really do awesome with pediatric changes, to take our wins where we can and make the changes with the ED. In the ED we have an RN educator. She’s very on board with doing the respiratory scoring and getting this whole thing started.” (Physician leader, Hospital L)

Intentional communication and leadership skills also played key roles in engaging hesitant providers and handling conflict:

“Just sitting and talking with our respiratory therapist about the ability to provide this type of service or support and seeing what their reservations have been, at least it’s open to conversation so that we could provide these types of therapies in the future and we’re able to see like what people’s concerns are. I think just basically increasing familiarity with not only these processes, but different types of therapy will hopefully in the future help us provide better care to our patients.” (Physician leader, Hospital Q)

Addressing Organizational and Resource Limitations

Participants recognized organizational and resource limitations, some of which may be unique to community hospitals that prioritize resources for adult care. The limitations described included EMR staff support, healthcare provider staffing/capacity, navigating IRBs, and addressing administrative processes. Competing demands for information technology staff support and lack of prioritization of pediatric-specific initiatives often hindered efforts to modify the EMR.

“Resource wise, we are hoping to implement an order set in our Epic EMR, [but] finding the availability from the Epic team may be a challenge.” (Physician Leader, Hospital A)

Participants also reported that limited staff capacity (eg, nursing, RT) hindered pathway implementation efforts. This limited capacity hindered workflow changes and limited the time available for education and training on pathways:

“[Respiratory scoring for asthma is] an added responsibility for the [nursing] staff and we don’t have patient technicians. So they’re doing everything from changing the sheets to bringing water to all of the medical patients. So, that I think may be a barrier.” (Physician leader, Hospital B)

Across sites, navigating the IRB posed various challenges. Some sites were required to obtain approval from regional IRBs, which did not have resources to devote to pediatric projects. Other sites did not have IRBs at all, but instead required separate approvals for the project from hospital leadership or other entities:

“On the IRB, I contacted the manager of the IRB and she’s said, ‘No, it’s not an IRB project,’ but she sent it to another director for review, and it took forever to be able to get a data agreement with [the local university hospital] so that we can pull the data. I just couldn’t believe it took months to get done.” (Physician Leader, Hospital K)

Finally, administrative barriers such as addressing formulary changes in the context of adult-focused settings were challenging. For example, at one hospital, metered dose inhalers (MDIs) were not used for adult patients, and the hospital administration was resistant to incorporate their use into practice for pediatric patients due to the cost of such changes.

“The [general hospital] didn’t have MDI’s anymore because of cost reasons, and when we started the pediatric work, we really made it a point to get the MDI’s for pediatric patients back in the formulary.” (Physician leader, Hospital A)

Devising Implementation Solutions with Practice Facilitators

Participants often devised pathway implementation solutions with facilitators in-the-moment during meetings. This problem-solving included figuring out work-arounds, proactive coaching by external facilitators, and just-in-time solution building. Furthermore, in meetings that included more than one project leader, leaders would often work with each other to devise solutions. Meetings provided forums that stimulated identification of implementation barriers, brainstorming, and subsequently solution building.

Physician leader: I’m wondering if we could, as an interim solution, try out an algorithm on paper, I don’t know if that’s allowed, until we get Epic approval. Do you know?

Nurse Leader: You mean having an algorithm posted in triage? Yeah, I don’t see why not. (Hospital A)

Next, problem solving was often driven by the facilitator’s experience and knowledge, drawn from their interactions with other collaborative sites or their own prior experiences with asthma, QI, or pathway implementation. The facilitators brought an outside perspective, not bound by that particular hospital’s local culture or structural intricacies. This proactive coaching spurred the identification of creative, yet practical solutions:

Project Leader: We’re still trying to get all our templates [for the EMR]…because [currently they are] all adult templates.

Facilitator: If you’re making templates right now, could you also add the three asterisks? Like smoking or exposure to second hand tobacco smoke or marijuana…then have the three asterisks there and then “Referral made?***”. That would force people to document in a certain place in the template as well.Project Leader: That’s definitely something we could add right now. (Hospital O)

Check-in meetings with facilitators offered an opportunity to trouble shoot, brainstorm work-arounds, devise in-the-moment site-specific solutions to enable successful pathway implementation, and provide ongoing support throughout implementation.

Pathways can improve the quality of care for children with asthma.³¹ However, there is little evidence-based guidance on how to implement pathways and improve pediatric care in community hospitals,^17-20 where the majority of children are cared for nationally. This is the first study to our knowledge that details the key determinants of pediatric asthma pathway implementation in community hospital settings. We identified four key determinants of implementation that can help guide others in similar settings. These include building an implementation infrastructure, engaging and motivating multidisciplinary providers, addressing organizational and resource limitations, and using external facilitators to devise implementation solutions.

Existing frameworks such as the CFIR outline the potential determinants of implementation success but do not provide population- or setting-specific guidance.²⁷ There have been prior studies detailing pathway implementation for pediatric populations, but these studies did not focus on community hospitals.^32,33 Our findings align with these prior studies, which highlight the importance of identifying implementation champions, engaging and motivating multidisciplinary providers, establishing a QI infrastructure, and addressing organizational and resource limitations, such as EMR integration.^32,33 However, our study provides unique insights into issues that are important to successful pathway implementation in community hospitals, including engagement of adult-focused healthcare providers, reprioritization of resources toward the care of children, and the potentially critical role of external facilitators.

Our findings indicate that community hospitals seeking to improve care for children may particularly benefit from using external facilitators and/or partnering with external organizations. We found that external facilitators played a significant and proactive role in community hospitals’ efforts to improve care for children. Facilitators helped devise work-arounds and engaged in just-in-time solution building with local project leaders. For instance, facilitators helped develop strategies for training healthcare providers in performing new clinical tasks, building reminders of pathway recommendations into clinical workflows, and overcoming resource barriers. Thus, community hospitals may uniquely benefit from participation in national learning collaboratives, which often provide avenues for external facilitation.^25,34,35 National networks, such as the VIP network, lead national learning collaboratives that provide external facilitation as well as other resources (eg, educational materials, data analysis support) to community hospitals seeking to improve pediatric care.²⁴ Previous work by McDaniel et al. identified that intentional partnerships between children’s and community hospitals can also potentially provide access to resources for education and training in pediatric care and support in navigating organizational and resource challenges.²²

Our results characterize the key determinants of pediatric asthma pathway implementation using a national sample of community hospitals that were diverse in geography, size, and structure. This imparts greater transferability of our findings. We also used strategies to promote the rigor of our findings, including triangulation and reflexivity. However, our study has several limitations. First, we analyzed only the meetings that occurred during the early months of pathway implementation. As such, we did not capture any key determinants that may have arisen later in implementation. However, process analyses of implementation indicate that the majority of implementation efforts occurred within these first three to four months.³⁶ Second, we did not elicit input from hospital administration or leadership. The lack of administrative/leadership input probably affected the CFIR themes we found, as no themes from the outer setting were elicited. However, the goal of our study was to characterize the experiences of those leading implementation efforts, and focusing on these leaders allows our work to better guide those doing similar work in the future. Third, we used CFIR to guide the development of our interview guide and as a reference during analysis, which may have skewed our findings to preferentially reflect CFIR constructs. However, our overall analysis was grounded in the primary data and we employed reflexivity during all stages of our analysis. In addition, having the facilitators conduct the qualitative interviews may have biased our findings toward the perspectives of the facilitators; however, the facilitators represented quite diverse clinical and QI backgrounds. Finally, our findings do not necessarily correlate with improvements in clinical outcomes. As such, they are not meant to serve as explicit recommendations for improving patient outcomes, but rather as a characterization of the context, processes, and experiences of implementing pathways in the community setting to inform others doing this important work.

CONCLUSIONS

We identified the key determinants of pediatric asthma pathway implementation in community hospitals, which may help inform QI efforts in these settings. We also identified organizational and resource limitations that are probably unique to these adult-focused hospitals. Participating in national learning collaboratives and/or working with facilitators may support pathway implementation and improved quality of care for children with asthma in community hospitals.

Future work should seek to correlate these and other determinants of pathway implementation with health outcomes for hospitalized children, as well as integrate broader and more diverse samples of community hospitals.

The following is a lightly edited transcript of a teleconference recorded in July 2018. The teleconference brought together health care providers from the Greater Los Angeles VA Health Care System (GLAVAHCS) to discuss the real-world processes for managing the treatment of patients with prostate cancer as they move between primary and specialist care.

William J. Aronson, MD. We are fortunate in having a superb medical record system at the Department of Veterans Affairs (VA) where we can all communicate with each other through a number of methods. Let’s start our discussion by reviewing an index patient that we see in our practice who has been treated with either radical prostatectomy or radiation therapy. One question to address is: Is there a point when the Urology or Radiation Oncology service can transition the patient’s entire care back to the primary care team? And if so, what would be the optimal way to accomplish this?

Nick, is there some point at which you discharge the patient from the radiation oncology service and give specific directions to primary care, or is it primarily just back to urology in your case?

Nicholas G. Nickols, MD, PhD. I have not discharged any patient from my clinic after definitive prostate cancer treatment. During treatment, patients are seen every week. Subsequently, I see them 6 weeks posttreatment, and then every 4 months for the first year, then every 6 months for the next 4 years, and then yearly after that. Although I never formally discharged a patient from my clinic, you can see based on the frequency of visits, that the patient will see more often than their primary care provider (PCP) toward the beginning. And then, after some years, the patient sees their primary more than they me. So it’s not an immediate hand off but rather a gradual transition. It’s important that the PCP is aware of what to look for especially for the late recurrences, late potential side effects, probably more significantly than the early side effects, how to manage them when appropriate, and when to ask the patient to see our team more frequently in follow-up.

William Aronson. We have a number of patients who travel tremendous distances to see us, and I tend to think that many of our follow-up patients, once things are stabilized with regards to management of their side effects, really could see their primary care doctors if we can give them specific instructions on, for example, when to get a prostate-specific antigen (PSA) test and when to refer back to us.

Alison, can you think of some specific cases where you feel like we’ve successfully done that?

Alison Neymark, MS. For the most part we haven’t discharged people, either. What we have done is transitioned them over to a phone clinic. In our department, we have 4 nurse practitioners (NPs) who each have a half-day of phone clinic where they call patients with their test results. Some of those patients are prostate cancer patients that we have been following for years. We schedule them for a phone call, whether it’s every 3 months, every 6 months or every year, to review the updated PSA level and to just check in with them by phone. It’s a win-win because it’s a really quick phone call to reassure the veteran that the PSA level is being followed, and it frees up an in-person appointment slot for another veteran.

We still have patients that prefer face-to-face visits, even though they know we’re not doing anything except discussing a PSA level with them—they just want that security of seeing our face. Some patients are very nervous, and they don’t necessarily want to be discharged, so to speak, back to primary care. Also, for those patients that travel a long distance to clinic, we offer an appointment in the video chat clinic, with the community-based outpatient clinics in Bakersfield and Santa Maria, California.

PSA Levels

William Aronson. I probably see a patient about every 4 to 6 weeks who has a low PSA after about 10 years and has a long distance to travel and mobility and other problems that make it difficult to come in. 

The challenge that I have is, what is that specific guideline to give with regards to the rise in PSA? I think it all depends on the patients prostate cancer clinical features and comorbidities.

Nicholas Nickols. If a patient has been seen by me in follow-up a number of times and there’s really no active issues and there’s a low suspicion of recurrence, then I offer the patient the option of a phone follow-up as an alternative to face to face. Some of them accept that, but I ask that they agree to also see either urology or their PCP face to face. I will also remotely ensure that they’re getting the right laboratory tests, and if not, I’ll put those orders in.

With regard to when to refer a patient back for a suspected recurrence after definitive radiation therapy, there is an accepted definition of biochemical failure called the Phoenix definition, which is an absolute rise in 2 ng/mL of PSA over their posttreatment nadir. Often the posttreatment nadir, especially if they were on hormone therapy, will be close to 0. If the PSA gets to 2, that is a good trigger for a referral back to me and/or urology to discuss restaging and workup for a suspected recurrence.

For patients that are postsurgery and then subsequently get salvage radiation, it is not as clear when a restaging workup should be initiated. Currently, the imaging that is routine care is not very sensitive for detecting PSA in that setting until the PSA is around 0.8 ng/mL, and that’s with the most modern imaging available. Over time that may improve.

William Aronson. The other index patient to think about would be the patient who is on watchful waiting for their prostate cancer, which is to be distinguished from active surveillance. If someone’s on active surveillance, we’re regularly doing prostate biopsies and doing very close monitoring; but we also have patients who have multiple other medical problems, have a limited life expectancy, don’t have aggressive prostate cancer, and it’s extremely reasonable not to do a biopsy in those patients.

Again, those are patients where we do follow the PSA generally every 6 months. And I think there’s also scenarios there where it’s reasonable to refer back to primary care with specific instructions. These, again, are patients who had difficulty getting in to see us or have mobility issues, but it is also a way to limit patient visits if that’s their desire.

Peter Glassman, MBBS, MSc: I’m trained as both a general internist and board certified in hospice and palliative medicine. I currently provide primary care as well as palliative care. I view prostate cancer from the diagnosis through the treatment spectrum as a continuum. It starts with the PCP with an elevated PSA level or if the digital rectal exam has an abnormality, and then the role of the genitourinary (GU) practitioner becomes more significant during the active treatment and diagnostic phases.

Primary care doesn’t disappear, and I think there are 2 major issues that go along with that. First of all, we in primary care, because we take care of patients that often have other comorbidities, need to work with the patient on those comorbidities. Secondly, we need the information shared between the GU and primary care providers so that we can answer questions from our patients and have an understanding of what they’re going through and when.

As time goes on, we go through various phases: We may reach a cure, a quiescent period, active therapy, watchful waiting, or recurrence. Primary care gets involved as time goes on when the disease either becomes quiescent, is just being followed, or is considered cured. Clearly when you have watchful waiting, active treatment, or are in a recurrence, then GU takes the forefront.

I view it as a wave function. Primary care to GU with primary in smaller letters and then primary, if you will, in larger letters, GU becomes a lesser participant unless there is active therapy, watchful waiting or recurrence.

In doing a little bit of research, I found 2 very good and very helpful documents. One is the American Cancer Society (ACS) prostate cancer survivorship care guidelines (Box). And the other is a synopsis of the guidelines. What I liked was that the guidelines focused not only on what should be done for the initial period of prostate cancer, but also for many of the ancillary issues which we often don’t give voice to. The guidelines provide a structure, a foundation to work with our patients over time on their prostate cancer-related issues while, at the same time, being cognizant that we need to deal with their other comorbid conditions.

Modes of Communication

Alison Neymark. We find that including parameters for PSA monitoring in our Progress Notes in the electronic health record (EHR) the best way to communicate with other providers. We’ll say, “If PSA gets to this level, please refer back.” We try to make it clear because with the VA being a training facility, it could be a different resident/attending physician team that’s going to see the patient the next time he is in primary care.

Peter Glassman. Yes, we’re very lucky, as Bill talked about earlier and Alison just mentioned. We have the EHR, and Bill may remember this. Before the EHR, we were constantly fishing to find the most relevant notes. If a patient saw a GU practitioner the day before they saw me, I was often asking the patient what was said. Now we can just review the notes.

It’s a double-edged sword though because there are, of course, many notes in a medical record; and you have to look for the specific items. The EHR and documenting the medical record probably plays the primary role in getting information across. When you want to have an active handoff, or you need to communicate with each other, we have a variety of mechanisms, ranging from the phone to the Microsoft Skype Link (Redmond, WA) system that allows us to tap a message to a colleague.

And I’ve been here long enough that I’ve seen most permutations of how prostate cancer is diagnosed as well as shared among providers. Bill and I have shared patients. Alison and I have shared patients, not necessarily with prostate cancer, although that too. But we know how to communicate with each other. And of course, there’s paging if you need something more urgently.

William Aronson. We also use Microsoft Outlook e-mail, and encrypt the messages to keep them confidential and private. The other nice thing we have is there is a nationwide urology Outlook e-mail, so if any of us have any specific questions, through one e-mail we can send it around the country; and there’s usually multiple very useful responses. That’s another real strength of our system within the VA that helps patient care enormously.

Nicholas Nickols. Sometimes, if there’s a critical note that I absolutely want someone on the care team to read, I’ll add them as a cosigner; and that will pop up when they log in to the Computerized Patient Record System (CPRS) as something that they need to read.

If the patient lives particularly far or gets his care at another VA medical center and laboratory tests are needed, then I will reach out to their PCP via e-mail. If contact is not confirmed, I will reach out via phone or Skype.

Peter Glassman. The most helpful notes are those that are very specific as to what primary care is being asked to do and/or what urology is going to be doing. So, the more specific we get in the notes as to what is being addressed, I think that’s very helpful.

I have been here long enough that I’ve known both Alison and Bill; and if they have an issue, they will tap me a message. It wasn’t long ago that Bill sent a message to me, and we worked on a patient with prostate cancer who was going to be on long-term hormone therapy. We talked about osteoporosis management, and between us we worked out who was going to do what. Those are the kind of shared decision-making situations that are very, very helpful.
 

Alison Neymark. Also, GLAVAHCS has a home-based primary care team (HBPC), and a lot of the PCPs for that team are NPs. They know that they can contact me for their patients because a lot of those patients are on watchful waiting, and we do not necessarily need to see them face to face in clinic. Our urology team just needs to review updated lab results and how they are doing clinically. The HBPC NP who knows them best can contact me every 6 months or so, and we’ll discuss the case, which avoids making the patient come in, especially when they’re homebound. Those of us that have been working at the VA for many years have established good relationships. We feel very comfortable reaching out and talking to each other about these patients

Peter Glassman. Alison, I agree. When I can talk to my patients and say, “You know, we had that question about,” whatever the question might be, “and I contacted urology, and this is what they said.” It gives the patient confidence that we’re following up on the issues that they have and that we’re communicating with each other in a way that is to their benefit. And I think it’s very appreciated both by the provider as well as the patient.

William Aronson. Not infrequently I’ll have patients who have nonurologic issues, which I may first detect, or who have specific issues with their prostate cancer that can be comanaged. And I have found that when I send an encrypted e-mail to the PCP, it has been an extremely satisfying interaction; and we really get to the heart of the matter quickly for the sake of the veteran.

Veterans With Comorbidities

William Aronson. Posttraumatic stress disorder (PTSD) is a very significant and unique aspect of our patients, which is enormously important to recognize. For example, the side effects of prostate treatments can be very significant, whether radiation or surgery. Our patients understandably can be very fearful of the prostate cancer diagnosis and treatment side effects.

We know, for example, after a patient gets a diagnosis of prostate cancer, they’re at increased risk of cardiac death. That’s an especially important issue for our patients that there be an ongoing interaction between urology and primary care.

The ACS guidelines that Dr. Glassman referred to were enlightening. In many cases, primary care can look at the whole patient and their circumstances better than we can and may detect, for example, specific psychological issues that either they can manage or refer to other specialists.

Peter Glassman. One of the things that was highlighted in the ACS guideline is that in any population of men who have this disease, there’s going to be distress, anxiety, and full-fledged depression. Of course, there are psychosocial aspects of prostate cancer, such as sexual activity and intimacy with a partner that we often don’t explore but are probably playing an important role in the overall health of our patients. We need to be mindful of these psychosocial aspects and at least periodically ask them, “How are you doing with this? How are things at home?” And of course, we already use screeners for depression. As the article noted, distress and anxiety and other factors can make somebody’s life less optimal with poorer quality of life.

Dual Care Patients

Alison Neymark. Many patients whether they have Medicare, insurance through their spouse, or Kaiser Permanente through their job, choose to go to both places. The challenge is communicating with the non-VA providers because here at the VA we can communicate easily through Skype, Outlook e-mail, or CPRS, but for dual care patients who’s in charge? I encourage the veterans to choose whom they want to manage their care; we’re always here and happy to treat them, but they need to decide who’s in charge because I don’t want them to get into a situation where the differing opinions lead to a delay in care.

Nicholas Nickols. The communication when the patient is receiving care outside VA, either on a continuous basis or temporarily, is more of a challenge. We obviously can’t rely upon the messaging system, face-to-face contact is difficult, and they may not be able to use e-mail as well. So in those situations, usually a phone call is the best approach. I have found that the outside providers are happy to speak on the phone to coordinate care.

Peter Glassman. I agree, it does add a layer of complexity because we don’t readily have the notes, any information in front of us. That said, a lot of our patients can and do bring in information from outside specialists, and I’m hopeful that they share the information that we provide back to their outside doctors as well.

William Aronson. Some patient get nervous. They might decide they want care elsewhere, but they still want the VA available for them. I always let them know they should proceed in whatever way they prefer, but we’re always available and here for them. I try to empower them to make their own decisions and feel comfortable with them.

Nicholas Nickols. Notes from the outside, if they’re being referred for VA Choice or community care, do get uploaded into VistA Imaging and can be accessed, although it’s not instantaneous. Sometimes there’s a delay, but I have been able to access outside notes most of the time. If a patient goes through a clinic at the VA, the note is written in real time, and you can read it immediately.

Peter Glassman. That is true for patients that are within the VA system who receive contracted care either through Choice or through non-VA care that is contracted through VA. For somebody who is choosing to use 2 health care systems, that can provide more of a challenge because those notes don’t come to us. Over time, most of my patients have brought test results to me.

The thing with oncologic care, of course, is it’s a lot more complex. And it’s hard to know without reasonable documentation what’s been going on. At some level, you have to trust that the outside provider is doing whatever they need to do, or you have to take it upon yourself to do it within the system.

Alison Neymark. In my experience with the Choice Program, it really depends on the outside providers and how comfortable they are with the system that has been established to share records. Not all providers are going into that system and accessing it. I have had cases where I will see the non-VA provider’s note and it’ll say, “No documentation available for this consultation.” It just happens that they didn’t go into the system to review it. So it can be a challenge.

I’ve had good communication with the providers who use the system correctly. In some cases, just to make it easier, I will go ahead and communicate with them through encrypted e-mail, or I’ll talk to their care coordinators directly by phone. 

Peter Glassman. Many, if not most, PCPs are going to take care of these patients, certainly within the VA, with their GU colleagues. And most of us feel comfortable using the current documentation system in a way that allows us to share information or at least to gather information about these patients.

One of the things that I think came out for me in looking at this was that there are guidelines or there are ideas out there on how to take better care of these patients. And I for one learned a fair bit just by going through these documents, which I’m very appreciative of. But it does highlight to me that we can give good care and provide good shared care for prostate cancer survivors. I think that is something that perhaps this discussion will highlight that not only are people doing that, but there are resources they can utilize that will help them get a more comprehensive picture of taking care of prostate cancer survivors in the primary care clinic.

The beauty of the VA system as a system is that as these issues come up that might affect the overall health of the veteran with prostate cancer, for example, psychosocial issues, we have many people that can address this that are experts in their area. And one of the great beauties of having an all-encompassing healthcare system is being able to use resources within the system, whether that be for other medical problems or other social or other psychological issues, that we ourselves are not expert in. We can reach out to our other colleagues and ask them for assistance. We have that available to help the patients. It’s really holistic.

We even have integrated medicine where we can help patients, hopefully, get back into a healthy lifestyle, for example, whereas we may not have that expertise or knowledge. We often think of this as sort of a shared decision between GU and primary care. But, in fact, it’s really the responsibility of many, many people of the system at large. We are very lucky to have that.

The following is a lightly edited transcript of a teleconference recorded in July 2018. The teleconference brought together health care providers from the Greater Los Angeles VA Health Care System (GLAVAHCS) to discuss the real-world processes for managing the treatment of patients with prostate cancer as they move between primary and specialist care.

William J. Aronson, MD. We are fortunate in having a superb medical record system at the Department of Veterans Affairs (VA) where we can all communicate with each other through a number of methods. Let’s start our discussion by reviewing an index patient that we see in our practice who has been treated with either radical prostatectomy or radiation therapy. One question to address is: Is there a point when the Urology or Radiation Oncology service can transition the patient’s entire care back to the primary care team? And if so, what would be the optimal way to accomplish this?

Nick, is there some point at which you discharge the patient from the radiation oncology service and give specific directions to primary care, or is it primarily just back to urology in your case?

Nicholas G. Nickols, MD, PhD. I have not discharged any patient from my clinic after definitive prostate cancer treatment. During treatment, patients are seen every week. Subsequently, I see them 6 weeks posttreatment, and then every 4 months for the first year, then every 6 months for the next 4 years, and then yearly after that. Although I never formally discharged a patient from my clinic, you can see based on the frequency of visits, that the patient will see more often than their primary care provider (PCP) toward the beginning. And then, after some years, the patient sees their primary more than they me. So it’s not an immediate hand off but rather a gradual transition. It’s important that the PCP is aware of what to look for especially for the late recurrences, late potential side effects, probably more significantly than the early side effects, how to manage them when appropriate, and when to ask the patient to see our team more frequently in follow-up.

William Aronson. We have a number of patients who travel tremendous distances to see us, and I tend to think that many of our follow-up patients, once things are stabilized with regards to management of their side effects, really could see their primary care doctors if we can give them specific instructions on, for example, when to get a prostate-specific antigen (PSA) test and when to refer back to us.

Alison, can you think of some specific cases where you feel like we’ve successfully done that?

Alison Neymark, MS. For the most part we haven’t discharged people, either. What we have done is transitioned them over to a phone clinic. In our department, we have 4 nurse practitioners (NPs) who each have a half-day of phone clinic where they call patients with their test results. Some of those patients are prostate cancer patients that we have been following for years. We schedule them for a phone call, whether it’s every 3 months, every 6 months or every year, to review the updated PSA level and to just check in with them by phone. It’s a win-win because it’s a really quick phone call to reassure the veteran that the PSA level is being followed, and it frees up an in-person appointment slot for another veteran.

We still have patients that prefer face-to-face visits, even though they know we’re not doing anything except discussing a PSA level with them—they just want that security of seeing our face. Some patients are very nervous, and they don’t necessarily want to be discharged, so to speak, back to primary care. Also, for those patients that travel a long distance to clinic, we offer an appointment in the video chat clinic, with the community-based outpatient clinics in Bakersfield and Santa Maria, California.

PSA Levels

William Aronson. I probably see a patient about every 4 to 6 weeks who has a low PSA after about 10 years and has a long distance to travel and mobility and other problems that make it difficult to come in. 

The challenge that I have is, what is that specific guideline to give with regards to the rise in PSA? I think it all depends on the patients prostate cancer clinical features and comorbidities.

Nicholas Nickols. If a patient has been seen by me in follow-up a number of times and there’s really no active issues and there’s a low suspicion of recurrence, then I offer the patient the option of a phone follow-up as an alternative to face to face. Some of them accept that, but I ask that they agree to also see either urology or their PCP face to face. I will also remotely ensure that they’re getting the right laboratory tests, and if not, I’ll put those orders in.

With regard to when to refer a patient back for a suspected recurrence after definitive radiation therapy, there is an accepted definition of biochemical failure called the Phoenix definition, which is an absolute rise in 2 ng/mL of PSA over their posttreatment nadir. Often the posttreatment nadir, especially if they were on hormone therapy, will be close to 0. If the PSA gets to 2, that is a good trigger for a referral back to me and/or urology to discuss restaging and workup for a suspected recurrence.

For patients that are postsurgery and then subsequently get salvage radiation, it is not as clear when a restaging workup should be initiated. Currently, the imaging that is routine care is not very sensitive for detecting PSA in that setting until the PSA is around 0.8 ng/mL, and that’s with the most modern imaging available. Over time that may improve.

William Aronson. The other index patient to think about would be the patient who is on watchful waiting for their prostate cancer, which is to be distinguished from active surveillance. If someone’s on active surveillance, we’re regularly doing prostate biopsies and doing very close monitoring; but we also have patients who have multiple other medical problems, have a limited life expectancy, don’t have aggressive prostate cancer, and it’s extremely reasonable not to do a biopsy in those patients.

Again, those are patients where we do follow the PSA generally every 6 months. And I think there’s also scenarios there where it’s reasonable to refer back to primary care with specific instructions. These, again, are patients who had difficulty getting in to see us or have mobility issues, but it is also a way to limit patient visits if that’s their desire.

Peter Glassman, MBBS, MSc: I’m trained as both a general internist and board certified in hospice and palliative medicine. I currently provide primary care as well as palliative care. I view prostate cancer from the diagnosis through the treatment spectrum as a continuum. It starts with the PCP with an elevated PSA level or if the digital rectal exam has an abnormality, and then the role of the genitourinary (GU) practitioner becomes more significant during the active treatment and diagnostic phases.

Primary care doesn’t disappear, and I think there are 2 major issues that go along with that. First of all, we in primary care, because we take care of patients that often have other comorbidities, need to work with the patient on those comorbidities. Secondly, we need the information shared between the GU and primary care providers so that we can answer questions from our patients and have an understanding of what they’re going through and when.

As time goes on, we go through various phases: We may reach a cure, a quiescent period, active therapy, watchful waiting, or recurrence. Primary care gets involved as time goes on when the disease either becomes quiescent, is just being followed, or is considered cured. Clearly when you have watchful waiting, active treatment, or are in a recurrence, then GU takes the forefront.

I view it as a wave function. Primary care to GU with primary in smaller letters and then primary, if you will, in larger letters, GU becomes a lesser participant unless there is active therapy, watchful waiting or recurrence.

In doing a little bit of research, I found 2 very good and very helpful documents. One is the American Cancer Society (ACS) prostate cancer survivorship care guidelines (Box). And the other is a synopsis of the guidelines. What I liked was that the guidelines focused not only on what should be done for the initial period of prostate cancer, but also for many of the ancillary issues which we often don’t give voice to. The guidelines provide a structure, a foundation to work with our patients over time on their prostate cancer-related issues while, at the same time, being cognizant that we need to deal with their other comorbid conditions.

Modes of Communication

Alison Neymark. We find that including parameters for PSA monitoring in our Progress Notes in the electronic health record (EHR) the best way to communicate with other providers. We’ll say, “If PSA gets to this level, please refer back.” We try to make it clear because with the VA being a training facility, it could be a different resident/attending physician team that’s going to see the patient the next time he is in primary care.

Peter Glassman. Yes, we’re very lucky, as Bill talked about earlier and Alison just mentioned. We have the EHR, and Bill may remember this. Before the EHR, we were constantly fishing to find the most relevant notes. If a patient saw a GU practitioner the day before they saw me, I was often asking the patient what was said. Now we can just review the notes.

It’s a double-edged sword though because there are, of course, many notes in a medical record; and you have to look for the specific items. The EHR and documenting the medical record probably plays the primary role in getting information across. When you want to have an active handoff, or you need to communicate with each other, we have a variety of mechanisms, ranging from the phone to the Microsoft Skype Link (Redmond, WA) system that allows us to tap a message to a colleague.

And I’ve been here long enough that I’ve seen most permutations of how prostate cancer is diagnosed as well as shared among providers. Bill and I have shared patients. Alison and I have shared patients, not necessarily with prostate cancer, although that too. But we know how to communicate with each other. And of course, there’s paging if you need something more urgently.

William Aronson. We also use Microsoft Outlook e-mail, and encrypt the messages to keep them confidential and private. The other nice thing we have is there is a nationwide urology Outlook e-mail, so if any of us have any specific questions, through one e-mail we can send it around the country; and there’s usually multiple very useful responses. That’s another real strength of our system within the VA that helps patient care enormously.

Nicholas Nickols. Sometimes, if there’s a critical note that I absolutely want someone on the care team to read, I’ll add them as a cosigner; and that will pop up when they log in to the Computerized Patient Record System (CPRS) as something that they need to read.

If the patient lives particularly far or gets his care at another VA medical center and laboratory tests are needed, then I will reach out to their PCP via e-mail. If contact is not confirmed, I will reach out via phone or Skype.

Peter Glassman. The most helpful notes are those that are very specific as to what primary care is being asked to do and/or what urology is going to be doing. So, the more specific we get in the notes as to what is being addressed, I think that’s very helpful.

I have been here long enough that I’ve known both Alison and Bill; and if they have an issue, they will tap me a message. It wasn’t long ago that Bill sent a message to me, and we worked on a patient with prostate cancer who was going to be on long-term hormone therapy. We talked about osteoporosis management, and between us we worked out who was going to do what. Those are the kind of shared decision-making situations that are very, very helpful.
 

Alison Neymark. Also, GLAVAHCS has a home-based primary care team (HBPC), and a lot of the PCPs for that team are NPs. They know that they can contact me for their patients because a lot of those patients are on watchful waiting, and we do not necessarily need to see them face to face in clinic. Our urology team just needs to review updated lab results and how they are doing clinically. The HBPC NP who knows them best can contact me every 6 months or so, and we’ll discuss the case, which avoids making the patient come in, especially when they’re homebound. Those of us that have been working at the VA for many years have established good relationships. We feel very comfortable reaching out and talking to each other about these patients

Peter Glassman. Alison, I agree. When I can talk to my patients and say, “You know, we had that question about,” whatever the question might be, “and I contacted urology, and this is what they said.” It gives the patient confidence that we’re following up on the issues that they have and that we’re communicating with each other in a way that is to their benefit. And I think it’s very appreciated both by the provider as well as the patient.

William Aronson. Not infrequently I’ll have patients who have nonurologic issues, which I may first detect, or who have specific issues with their prostate cancer that can be comanaged. And I have found that when I send an encrypted e-mail to the PCP, it has been an extremely satisfying interaction; and we really get to the heart of the matter quickly for the sake of the veteran.

Veterans With Comorbidities

William Aronson. Posttraumatic stress disorder (PTSD) is a very significant and unique aspect of our patients, which is enormously important to recognize. For example, the side effects of prostate treatments can be very significant, whether radiation or surgery. Our patients understandably can be very fearful of the prostate cancer diagnosis and treatment side effects.

We know, for example, after a patient gets a diagnosis of prostate cancer, they’re at increased risk of cardiac death. That’s an especially important issue for our patients that there be an ongoing interaction between urology and primary care.

The ACS guidelines that Dr. Glassman referred to were enlightening. In many cases, primary care can look at the whole patient and their circumstances better than we can and may detect, for example, specific psychological issues that either they can manage or refer to other specialists.

Peter Glassman. One of the things that was highlighted in the ACS guideline is that in any population of men who have this disease, there’s going to be distress, anxiety, and full-fledged depression. Of course, there are psychosocial aspects of prostate cancer, such as sexual activity and intimacy with a partner that we often don’t explore but are probably playing an important role in the overall health of our patients. We need to be mindful of these psychosocial aspects and at least periodically ask them, “How are you doing with this? How are things at home?” And of course, we already use screeners for depression. As the article noted, distress and anxiety and other factors can make somebody’s life less optimal with poorer quality of life.

Dual Care Patients

Alison Neymark. Many patients whether they have Medicare, insurance through their spouse, or Kaiser Permanente through their job, choose to go to both places. The challenge is communicating with the non-VA providers because here at the VA we can communicate easily through Skype, Outlook e-mail, or CPRS, but for dual care patients who’s in charge? I encourage the veterans to choose whom they want to manage their care; we’re always here and happy to treat them, but they need to decide who’s in charge because I don’t want them to get into a situation where the differing opinions lead to a delay in care.

Nicholas Nickols. The communication when the patient is receiving care outside VA, either on a continuous basis or temporarily, is more of a challenge. We obviously can’t rely upon the messaging system, face-to-face contact is difficult, and they may not be able to use e-mail as well. So in those situations, usually a phone call is the best approach. I have found that the outside providers are happy to speak on the phone to coordinate care.

Peter Glassman. I agree, it does add a layer of complexity because we don’t readily have the notes, any information in front of us. That said, a lot of our patients can and do bring in information from outside specialists, and I’m hopeful that they share the information that we provide back to their outside doctors as well.

William Aronson. Some patient get nervous. They might decide they want care elsewhere, but they still want the VA available for them. I always let them know they should proceed in whatever way they prefer, but we’re always available and here for them. I try to empower them to make their own decisions and feel comfortable with them.

Nicholas Nickols. Notes from the outside, if they’re being referred for VA Choice or community care, do get uploaded into VistA Imaging and can be accessed, although it’s not instantaneous. Sometimes there’s a delay, but I have been able to access outside notes most of the time. If a patient goes through a clinic at the VA, the note is written in real time, and you can read it immediately.

Peter Glassman. That is true for patients that are within the VA system who receive contracted care either through Choice or through non-VA care that is contracted through VA. For somebody who is choosing to use 2 health care systems, that can provide more of a challenge because those notes don’t come to us. Over time, most of my patients have brought test results to me.

The thing with oncologic care, of course, is it’s a lot more complex. And it’s hard to know without reasonable documentation what’s been going on. At some level, you have to trust that the outside provider is doing whatever they need to do, or you have to take it upon yourself to do it within the system.

Alison Neymark. In my experience with the Choice Program, it really depends on the outside providers and how comfortable they are with the system that has been established to share records. Not all providers are going into that system and accessing it. I have had cases where I will see the non-VA provider’s note and it’ll say, “No documentation available for this consultation.” It just happens that they didn’t go into the system to review it. So it can be a challenge.

I’ve had good communication with the providers who use the system correctly. In some cases, just to make it easier, I will go ahead and communicate with them through encrypted e-mail, or I’ll talk to their care coordinators directly by phone. 

Peter Glassman. Many, if not most, PCPs are going to take care of these patients, certainly within the VA, with their GU colleagues. And most of us feel comfortable using the current documentation system in a way that allows us to share information or at least to gather information about these patients.

One of the things that I think came out for me in looking at this was that there are guidelines or there are ideas out there on how to take better care of these patients. And I for one learned a fair bit just by going through these documents, which I’m very appreciative of. But it does highlight to me that we can give good care and provide good shared care for prostate cancer survivors. I think that is something that perhaps this discussion will highlight that not only are people doing that, but there are resources they can utilize that will help them get a more comprehensive picture of taking care of prostate cancer survivors in the primary care clinic.

The beauty of the VA system as a system is that as these issues come up that might affect the overall health of the veteran with prostate cancer, for example, psychosocial issues, we have many people that can address this that are experts in their area. And one of the great beauties of having an all-encompassing healthcare system is being able to use resources within the system, whether that be for other medical problems or other social or other psychological issues, that we ourselves are not expert in. We can reach out to our other colleagues and ask them for assistance. We have that available to help the patients. It’s really holistic.

We even have integrated medicine where we can help patients, hopefully, get back into a healthy lifestyle, for example, whereas we may not have that expertise or knowledge. We often think of this as sort of a shared decision between GU and primary care. But, in fact, it’s really the responsibility of many, many people of the system at large. We are very lucky to have that.

The following is a lightly edited transcript of a teleconference recorded in July 2018. The teleconference brought together health care providers from the Greater Los Angeles VA Health Care System (GLAVAHCS) to discuss the real-world processes for managing the treatment of patients with prostate cancer as they move between primary and specialist care.

William J. Aronson, MD. We are fortunate in having a superb medical record system at the Department of Veterans Affairs (VA) where we can all communicate with each other through a number of methods. Let’s start our discussion by reviewing an index patient that we see in our practice who has been treated with either radical prostatectomy or radiation therapy. One question to address is: Is there a point when the Urology or Radiation Oncology service can transition the patient’s entire care back to the primary care team? And if so, what would be the optimal way to accomplish this?

Nick, is there some point at which you discharge the patient from the radiation oncology service and give specific directions to primary care, or is it primarily just back to urology in your case?

Nicholas G. Nickols, MD, PhD. I have not discharged any patient from my clinic after definitive prostate cancer treatment. During treatment, patients are seen every week. Subsequently, I see them 6 weeks posttreatment, and then every 4 months for the first year, then every 6 months for the next 4 years, and then yearly after that. Although I never formally discharged a patient from my clinic, you can see based on the frequency of visits, that the patient will see more often than their primary care provider (PCP) toward the beginning. And then, after some years, the patient sees their primary more than they me. So it’s not an immediate hand off but rather a gradual transition. It’s important that the PCP is aware of what to look for especially for the late recurrences, late potential side effects, probably more significantly than the early side effects, how to manage them when appropriate, and when to ask the patient to see our team more frequently in follow-up.

William Aronson. We have a number of patients who travel tremendous distances to see us, and I tend to think that many of our follow-up patients, once things are stabilized with regards to management of their side effects, really could see their primary care doctors if we can give them specific instructions on, for example, when to get a prostate-specific antigen (PSA) test and when to refer back to us.

Alison, can you think of some specific cases where you feel like we’ve successfully done that?

Alison Neymark, MS. For the most part we haven’t discharged people, either. What we have done is transitioned them over to a phone clinic. In our department, we have 4 nurse practitioners (NPs) who each have a half-day of phone clinic where they call patients with their test results. Some of those patients are prostate cancer patients that we have been following for years. We schedule them for a phone call, whether it’s every 3 months, every 6 months or every year, to review the updated PSA level and to just check in with them by phone. It’s a win-win because it’s a really quick phone call to reassure the veteran that the PSA level is being followed, and it frees up an in-person appointment slot for another veteran.

We still have patients that prefer face-to-face visits, even though they know we’re not doing anything except discussing a PSA level with them—they just want that security of seeing our face. Some patients are very nervous, and they don’t necessarily want to be discharged, so to speak, back to primary care. Also, for those patients that travel a long distance to clinic, we offer an appointment in the video chat clinic, with the community-based outpatient clinics in Bakersfield and Santa Maria, California.

PSA Levels

William Aronson. I probably see a patient about every 4 to 6 weeks who has a low PSA after about 10 years and has a long distance to travel and mobility and other problems that make it difficult to come in. 

The challenge that I have is, what is that specific guideline to give with regards to the rise in PSA? I think it all depends on the patients prostate cancer clinical features and comorbidities.

Nicholas Nickols. If a patient has been seen by me in follow-up a number of times and there’s really no active issues and there’s a low suspicion of recurrence, then I offer the patient the option of a phone follow-up as an alternative to face to face. Some of them accept that, but I ask that they agree to also see either urology or their PCP face to face. I will also remotely ensure that they’re getting the right laboratory tests, and if not, I’ll put those orders in.

With regard to when to refer a patient back for a suspected recurrence after definitive radiation therapy, there is an accepted definition of biochemical failure called the Phoenix definition, which is an absolute rise in 2 ng/mL of PSA over their posttreatment nadir. Often the posttreatment nadir, especially if they were on hormone therapy, will be close to 0. If the PSA gets to 2, that is a good trigger for a referral back to me and/or urology to discuss restaging and workup for a suspected recurrence.

For patients that are postsurgery and then subsequently get salvage radiation, it is not as clear when a restaging workup should be initiated. Currently, the imaging that is routine care is not very sensitive for detecting PSA in that setting until the PSA is around 0.8 ng/mL, and that’s with the most modern imaging available. Over time that may improve.

William Aronson. The other index patient to think about would be the patient who is on watchful waiting for their prostate cancer, which is to be distinguished from active surveillance. If someone’s on active surveillance, we’re regularly doing prostate biopsies and doing very close monitoring; but we also have patients who have multiple other medical problems, have a limited life expectancy, don’t have aggressive prostate cancer, and it’s extremely reasonable not to do a biopsy in those patients.

Again, those are patients where we do follow the PSA generally every 6 months. And I think there’s also scenarios there where it’s reasonable to refer back to primary care with specific instructions. These, again, are patients who had difficulty getting in to see us or have mobility issues, but it is also a way to limit patient visits if that’s their desire.

Peter Glassman, MBBS, MSc: I’m trained as both a general internist and board certified in hospice and palliative medicine. I currently provide primary care as well as palliative care. I view prostate cancer from the diagnosis through the treatment spectrum as a continuum. It starts with the PCP with an elevated PSA level or if the digital rectal exam has an abnormality, and then the role of the genitourinary (GU) practitioner becomes more significant during the active treatment and diagnostic phases.

Primary care doesn’t disappear, and I think there are 2 major issues that go along with that. First of all, we in primary care, because we take care of patients that often have other comorbidities, need to work with the patient on those comorbidities. Secondly, we need the information shared between the GU and primary care providers so that we can answer questions from our patients and have an understanding of what they’re going through and when.

As time goes on, we go through various phases: We may reach a cure, a quiescent period, active therapy, watchful waiting, or recurrence. Primary care gets involved as time goes on when the disease either becomes quiescent, is just being followed, or is considered cured. Clearly when you have watchful waiting, active treatment, or are in a recurrence, then GU takes the forefront.

I view it as a wave function. Primary care to GU with primary in smaller letters and then primary, if you will, in larger letters, GU becomes a lesser participant unless there is active therapy, watchful waiting or recurrence.

In doing a little bit of research, I found 2 very good and very helpful documents. One is the American Cancer Society (ACS) prostate cancer survivorship care guidelines (Box). And the other is a synopsis of the guidelines. What I liked was that the guidelines focused not only on what should be done for the initial period of prostate cancer, but also for many of the ancillary issues which we often don’t give voice to. The guidelines provide a structure, a foundation to work with our patients over time on their prostate cancer-related issues while, at the same time, being cognizant that we need to deal with their other comorbid conditions.

Modes of Communication

Alison Neymark. We find that including parameters for PSA monitoring in our Progress Notes in the electronic health record (EHR) the best way to communicate with other providers. We’ll say, “If PSA gets to this level, please refer back.” We try to make it clear because with the VA being a training facility, it could be a different resident/attending physician team that’s going to see the patient the next time he is in primary care.

Peter Glassman. Yes, we’re very lucky, as Bill talked about earlier and Alison just mentioned. We have the EHR, and Bill may remember this. Before the EHR, we were constantly fishing to find the most relevant notes. If a patient saw a GU practitioner the day before they saw me, I was often asking the patient what was said. Now we can just review the notes.

It’s a double-edged sword though because there are, of course, many notes in a medical record; and you have to look for the specific items. The EHR and documenting the medical record probably plays the primary role in getting information across. When you want to have an active handoff, or you need to communicate with each other, we have a variety of mechanisms, ranging from the phone to the Microsoft Skype Link (Redmond, WA) system that allows us to tap a message to a colleague.

And I’ve been here long enough that I’ve seen most permutations of how prostate cancer is diagnosed as well as shared among providers. Bill and I have shared patients. Alison and I have shared patients, not necessarily with prostate cancer, although that too. But we know how to communicate with each other. And of course, there’s paging if you need something more urgently.

William Aronson. We also use Microsoft Outlook e-mail, and encrypt the messages to keep them confidential and private. The other nice thing we have is there is a nationwide urology Outlook e-mail, so if any of us have any specific questions, through one e-mail we can send it around the country; and there’s usually multiple very useful responses. That’s another real strength of our system within the VA that helps patient care enormously.

Nicholas Nickols. Sometimes, if there’s a critical note that I absolutely want someone on the care team to read, I’ll add them as a cosigner; and that will pop up when they log in to the Computerized Patient Record System (CPRS) as something that they need to read.

If the patient lives particularly far or gets his care at another VA medical center and laboratory tests are needed, then I will reach out to their PCP via e-mail. If contact is not confirmed, I will reach out via phone or Skype.

Peter Glassman. The most helpful notes are those that are very specific as to what primary care is being asked to do and/or what urology is going to be doing. So, the more specific we get in the notes as to what is being addressed, I think that’s very helpful.

I have been here long enough that I’ve known both Alison and Bill; and if they have an issue, they will tap me a message. It wasn’t long ago that Bill sent a message to me, and we worked on a patient with prostate cancer who was going to be on long-term hormone therapy. We talked about osteoporosis management, and between us we worked out who was going to do what. Those are the kind of shared decision-making situations that are very, very helpful.
 

Alison Neymark. Also, GLAVAHCS has a home-based primary care team (HBPC), and a lot of the PCPs for that team are NPs. They know that they can contact me for their patients because a lot of those patients are on watchful waiting, and we do not necessarily need to see them face to face in clinic. Our urology team just needs to review updated lab results and how they are doing clinically. The HBPC NP who knows them best can contact me every 6 months or so, and we’ll discuss the case, which avoids making the patient come in, especially when they’re homebound. Those of us that have been working at the VA for many years have established good relationships. We feel very comfortable reaching out and talking to each other about these patients

Peter Glassman. Alison, I agree. When I can talk to my patients and say, “You know, we had that question about,” whatever the question might be, “and I contacted urology, and this is what they said.” It gives the patient confidence that we’re following up on the issues that they have and that we’re communicating with each other in a way that is to their benefit. And I think it’s very appreciated both by the provider as well as the patient.

William Aronson. Not infrequently I’ll have patients who have nonurologic issues, which I may first detect, or who have specific issues with their prostate cancer that can be comanaged. And I have found that when I send an encrypted e-mail to the PCP, it has been an extremely satisfying interaction; and we really get to the heart of the matter quickly for the sake of the veteran.

Veterans With Comorbidities

William Aronson. Posttraumatic stress disorder (PTSD) is a very significant and unique aspect of our patients, which is enormously important to recognize. For example, the side effects of prostate treatments can be very significant, whether radiation or surgery. Our patients understandably can be very fearful of the prostate cancer diagnosis and treatment side effects.

We know, for example, after a patient gets a diagnosis of prostate cancer, they’re at increased risk of cardiac death. That’s an especially important issue for our patients that there be an ongoing interaction between urology and primary care.

The ACS guidelines that Dr. Glassman referred to were enlightening. In many cases, primary care can look at the whole patient and their circumstances better than we can and may detect, for example, specific psychological issues that either they can manage or refer to other specialists.

Peter Glassman. One of the things that was highlighted in the ACS guideline is that in any population of men who have this disease, there’s going to be distress, anxiety, and full-fledged depression. Of course, there are psychosocial aspects of prostate cancer, such as sexual activity and intimacy with a partner that we often don’t explore but are probably playing an important role in the overall health of our patients. We need to be mindful of these psychosocial aspects and at least periodically ask them, “How are you doing with this? How are things at home?” And of course, we already use screeners for depression. As the article noted, distress and anxiety and other factors can make somebody’s life less optimal with poorer quality of life.

Dual Care Patients

Alison Neymark. Many patients whether they have Medicare, insurance through their spouse, or Kaiser Permanente through their job, choose to go to both places. The challenge is communicating with the non-VA providers because here at the VA we can communicate easily through Skype, Outlook e-mail, or CPRS, but for dual care patients who’s in charge? I encourage the veterans to choose whom they want to manage their care; we’re always here and happy to treat them, but they need to decide who’s in charge because I don’t want them to get into a situation where the differing opinions lead to a delay in care.

Nicholas Nickols. The communication when the patient is receiving care outside VA, either on a continuous basis or temporarily, is more of a challenge. We obviously can’t rely upon the messaging system, face-to-face contact is difficult, and they may not be able to use e-mail as well. So in those situations, usually a phone call is the best approach. I have found that the outside providers are happy to speak on the phone to coordinate care.

Peter Glassman. I agree, it does add a layer of complexity because we don’t readily have the notes, any information in front of us. That said, a lot of our patients can and do bring in information from outside specialists, and I’m hopeful that they share the information that we provide back to their outside doctors as well.

William Aronson. Some patient get nervous. They might decide they want care elsewhere, but they still want the VA available for them. I always let them know they should proceed in whatever way they prefer, but we’re always available and here for them. I try to empower them to make their own decisions and feel comfortable with them.

Nicholas Nickols. Notes from the outside, if they’re being referred for VA Choice or community care, do get uploaded into VistA Imaging and can be accessed, although it’s not instantaneous. Sometimes there’s a delay, but I have been able to access outside notes most of the time. If a patient goes through a clinic at the VA, the note is written in real time, and you can read it immediately.

Peter Glassman. That is true for patients that are within the VA system who receive contracted care either through Choice or through non-VA care that is contracted through VA. For somebody who is choosing to use 2 health care systems, that can provide more of a challenge because those notes don’t come to us. Over time, most of my patients have brought test results to me.

The thing with oncologic care, of course, is it’s a lot more complex. And it’s hard to know without reasonable documentation what’s been going on. At some level, you have to trust that the outside provider is doing whatever they need to do, or you have to take it upon yourself to do it within the system.

Alison Neymark. In my experience with the Choice Program, it really depends on the outside providers and how comfortable they are with the system that has been established to share records. Not all providers are going into that system and accessing it. I have had cases where I will see the non-VA provider’s note and it’ll say, “No documentation available for this consultation.” It just happens that they didn’t go into the system to review it. So it can be a challenge.

I’ve had good communication with the providers who use the system correctly. In some cases, just to make it easier, I will go ahead and communicate with them through encrypted e-mail, or I’ll talk to their care coordinators directly by phone. 

Peter Glassman. Many, if not most, PCPs are going to take care of these patients, certainly within the VA, with their GU colleagues. And most of us feel comfortable using the current documentation system in a way that allows us to share information or at least to gather information about these patients.

One of the things that I think came out for me in looking at this was that there are guidelines or there are ideas out there on how to take better care of these patients. And I for one learned a fair bit just by going through these documents, which I’m very appreciative of. But it does highlight to me that we can give good care and provide good shared care for prostate cancer survivors. I think that is something that perhaps this discussion will highlight that not only are people doing that, but there are resources they can utilize that will help them get a more comprehensive picture of taking care of prostate cancer survivors in the primary care clinic.

The beauty of the VA system as a system is that as these issues come up that might affect the overall health of the veteran with prostate cancer, for example, psychosocial issues, we have many people that can address this that are experts in their area. And one of the great beauties of having an all-encompassing healthcare system is being able to use resources within the system, whether that be for other medical problems or other social or other psychological issues, that we ourselves are not expert in. We can reach out to our other colleagues and ask them for assistance. We have that available to help the patients. It’s really holistic.

We even have integrated medicine where we can help patients, hopefully, get back into a healthy lifestyle, for example, whereas we may not have that expertise or knowledge. We often think of this as sort of a shared decision between GU and primary care. But, in fact, it’s really the responsibility of many, many people of the system at large. We are very lucky to have that.

The combination of decreasing hospital lengths of stay and increasing age and comorbidity of the United States population is a principal driver of the increased use of postacute care in the US.^1-3 Postacute care refers to care in long-term acute care hospitals, inpatient rehabilitation facilities, skilled nursing facilities (SNFs), and care provided by home health agencies after an acute hospitalization. In 2016, 43% of Medicare beneficiaries received postacute care after hospital discharge at the cost of $60 billion annually; nearly half of these received care in an SNF.⁴ Increasing recognition of the significant cost and poor outcomes of postacute care led to payment reforms, such as bundled payments, that incentivized less expensive forms of postacute care and improvements in outcomes.^5-9 Early evaluations suggested that hospitals are sensitive to these reforms and responded by significantly decreasing SNF utilization.^10,11 It remains unclear whether this was safe and effective.

In this context, increased attention to how hospital clinicians and hospitalized patients decide whether to use postacute care (and what form to use) is appropriate since the effect of payment reforms could negatively impact vulnerable populations of older adults without adequate protection.¹² Suboptimal decision-making can drive both overuse and inappropriate underuse of this expensive medical resource. Initial evidence suggests that patients and clinicians are poorly equipped to make high-quality decisions about postacute care, with significant deficits in both the decision-making process and content.^13-16 While these gaps are important to address, they may only be part of the problem. The fields of cognitive psychology and behavioral economics have revealed new insights into decision-making, demonstrating that people deviate from rational decision-making in predictable ways, termed decision heuristics, or cognitive biases.¹⁷ This growing field of research suggests heuristics or biases play important roles in decision-making and determining behavior, particularly in situations where there may be little information provided and the patient is stressed, tired, and ill—precisely like deciding on postacute care.¹⁸ However, it is currently unknown whether cognitive biases are at play when making hospital discharge decisions.

We sought to identify the most salient heuristics or cognitive biases patients may utilize when making decisions about postacute care at the end of their hospitalization and ways clinicians may contribute to these biases. The overall goal was to derive insights for improving postacute care decision-making.

Study Design

We conducted a secondary analysis on interviews with hospital and SNF clinicians as well as patients and their caregivers who were either leaving the hospital for an SNF or newly arrived in an SNF from the hospital to understand if cognitive biases were present and how they manifested themselves in a real-world clinical context.¹⁹ These interviews were part of a larger qualitative study that sought to understand how clinicians, patients, and their caregivers made decisions about postacute care, particularly related to SNFs.^13,14 This study represents the analysis of all our interviews, specifically examining decision-making bias. Participating sites, clinical roles, and both patient and caregiver characteristics (Table 1) in our cohort have been previously described.^13,14

Analysis

We used a team-based approach to framework analysis, which has been used in other decision-making studies¹⁴, including those measuring cognitive bias.²⁰ A limitation in cognitive bias research is the lack of a standardized list or categorization of cognitive biases. We reviewed prior systematic^17,21 and narrative reviews^18,22, as well as prior studies describing examples of cognitive biases playing a role in decision-making about therapy²⁰ to construct a list of possible cognitive biases to evaluate and narrow these a priori to potential biases relevant to the decision about postacute care based on our prior work (Table 2).

We applied this framework to analyze transcripts through an iterative process of deductive coding and reviewing across four reviewers (ML, RA, AL, CL) and a hospitalist physician with expertise leading qualitative studies (REB).

Intercoder consensus was built through team discussion by resolving points of disagreement.²³ Consistency of coding was regularly checked by having more than one investigator code individual manuscripts and comparing coding, and discrepancies were resolved through team discussion. We triangulated the data (shared our preliminary results) using a larger study team, including an expert in behavioral economics (SRG), physicians at study sites (EC, RA), and an anthropologist with expertise in qualitative methods (CL). We did this to ensure credibility (to what extent the findings are credible or believable) and confirmability of findings (ensuring the findings are based on participant narratives rather than researcher biases).

We reviewed a total of 105 interviews with 25 hospital clinicians, 20 SNF clinicians, 21 patients and 14 caregivers in the hospital, and 15 patients and 10 caregivers in the SNF setting (Table 1). We found authority bias/halo effect; default/status quo bias, anchoring bias, and framing was commonly present in decision-making about postacute care in a SNF, whereas there were few if any examples of ambiguity aversion, availability heuristic, confirmation bias, optimism bias, or false consensus effect (Table 2).

Authority Bias/Halo Effect

While most patients deferred to their inpatient teams when it came to decision-making, this effect seemed to differ across VA and non-VA settings. Veterans expressed a higher degree of potential authority bias regarding the VA as an institution, whereas older adults in non-VA settings saw physicians as the authority figure making decisions in their best interests.

Veterans expressed confidence in the VA regarding both whether to go to a SNF and where to go:

“The VA wouldn’t license [an SNF] if they didn’t have a good reputation for care, cleanliness, things of that nature” (Veteran, VA CLC)

“I just knew the VA would have my best interests at heart” (Veteran, VA CLC)

Their caregivers expressed similar confidence:

“I’m not gonna decide [on whether the patient they care for goes to postacute care], like I told you, that’s totally up to the VA. I have trust and faith in them…so wherever they send him, that’s where he’s going” (Caregiver, VA hospital)

In some cases, this perspective was closer to the halo effect: a positive experience with the care provider or the care team led the decision-makers to believe that their recommendations about postacute care would be similarly positive.

“I think we were very trusting in the sense that whatever happened the last time around, he survived it…they took care of him…he got back home, and he started his life again, you know, so why would we question what they’re telling us to do? (Caregiver, VA hospital)

In contrast to Veterans, non-Veteran patients seemed to experience authority bias when it came to the inpatient team.

“Well, I’d like to know more about the PTs [Physical Therapists] there, but I assume since they were recommended, they will be good.” (Patient, University hospital)

This perspective was especially apparent when it came to physicians:

“The level of trust that they [patients] put in their doctor is gonna outweigh what anyone else would say.” (Clinical liaison, SNF)

“[In response to a question about influences on the decision to go to rehab] I don’t…that’s not my decision to make, that’s the doctor’s decision.” (Patient, University hospital)

“They said so…[the doctor] said I needed to go to rehab, so I guess I do because it’s the doctor’s decision.” (Patient, University hospital)

Default/Status quo Bias

In a related way, patients and caregivers with exposure to a SNF seemed to default to the same SNF with which they had previous experience. This bias seems to be primarily related to knowing what to expect.

“He thinks it’s [a particular SNF] the right place for him now…he was there before and he knew, again, it was the right place for him to be” (Caregiver, VA hospital)

“It’s the only one I’ve ever been in…but they have a lot of activities; you have a lot of freedom, staff was good” (Patient, VA hospital)

“I’ve been [to this SNF] before and I kind of know what the program involves…so it was kind of like going home, not, going home is the wrong way to put it…I mean coming here is like something I know, you know, I didn’t need anybody to explain it to me.” (Patient, VA hospital)

“Anybody that’s been to [SNF], that would be their choice to go back to, and I guess I must’ve liked it that first time because I asked to go back again.” (Patient, University hospital)

Anchoring Bias

While anchoring bias was less frequent, it came up in two domains: first, related to costs of care, and second, related to facility characteristics. Costs came up most frequently for Veterans who preferred to move their care to the VA for cost reasons, which appeared in these cases to overshadow other considerations:

“I kept emphasizing that the VA could do all the same things at a lot more reasonable price. The whole purpose of having the VA is for the Veteran, so that…we can get the healthcare that we need at a more reasonable [sic] or a reasonable price.” (Veteran, CLC)

“I think the CLC [VA SNF] is going to take care of her probably the same way any other facility of its type would, unless she were in a private facility, but you know, that costs a lot more money.” (Caregiver, VA hospital)

Patients occasionally had striking responses to particular characteristics of SNFs, regardless of whether this was a central feature or related to their rehabilitation:

“The social worker comes and talks to me about the nursing home where cats are running around, you know, to infect my leg or spin their little cat hairs into my lungs and make my asthma worse…I’m going to have to beg the nurses or the aides or the family or somebody to clean the cat…” (Veteran, VA hospital)

Framing

Framing was the strongest theme among clinician interviews in our sample. Clinicians most frequently described the SNF as a place where patients could recover function (a positive frame), explaining risks (eg, rehospitalization) associated with alternative postacute care options besides the SNF in great detail.

“Aside from explaining the benefits of going and…having that 24-hour care, having the therapies provided to them [the patients], talking about them getting stronger, phrasing it in such a way that patients sometimes are more agreeable, like not calling it a skilled nursing facility, calling it a rehab you know, for them to get physically stronger so they can be the most independent that they can once they do go home, and also explaining … we think that this would be the best plan to prevent them from coming back to the hospital, so those are some of the things that we’ll mention to patients to try and educate them and get them to be agreeable for placement.” (Social worker, University hospital)

Clinicians avoided negative associations with “nursing home” (even though all SNFs are nursing homes) and tended to use more positive frames such as “rehabilitation facility.”

“Use the word rehab….we definitely use the word rehab, to get more therapy, to go home; it’s not a, we really emphasize it’s not a nursing home, it’s not to go to stay forever.” (Physical therapist, safety-net hospital)

Clinicians used a frame of “safety” when discussing the SNF and used a frame of “risk” when discussing alternative postacute care options such as returning home. We did not find examples of clinicians discussing similar risks in going to a SNF even for risks, such as falling, which exist in both settings.

“I’ve talked to them primarily on an avenue of safety because I think people want and they value independence, they value making sure they can get home, but you know, a lot of the times they understand safety is, it can be a concern and outlining that our goal is to make sure that they’re safe and they stay home, and I tend to broach the subject saying that our therapists believe that they might not be safe at home in the moment, but they have potential goals to be safe later on if we continue therapy. I really highlight safety being the major driver of our discussion.” (Physician, VA hospital)

In some cases, framing was so overt that other risk-mitigating options (eg, home healthcare) are not discussed.

“I definitely tend to explain the ideal first. I’m not going to bring up home care when we really think somebody should go to rehab, however, once people say I don’t want to do that, I’m not going, then that’s when I’m like OK, well, let’s talk to the doctors, but we can see about other supports in the home.” (Social worker, VA hospital)

In a large sample of patients and their caregivers, as well as multidisciplinary clinicians at three different hospitals and three SNFs, we found authority bias/halo effect and framing biases were most common and seemed most impactful. Default/status quo bias and anchoring bias were also present in decision-making about a SNF. The combination of authority bias/halo effect and framing biases could synergistically interact to augment the likelihood of patients accepting a SNF for postacute care. Patients who had been to a SNF before seemed more likely to choose the SNF they had experienced previously even if they had no other postacute care experiences, and could be highly influenced by isolated characteristics of that facility (such as the physical environment or cost of care).

It is important to mention that cognitive biases do not necessarily have a negative impact: indeed, as Kahneman and Tversky point out, these are useful heuristics from “fast” thinking that are often effective.²⁴ For example, clinicians may be trying to act in the best interests of the patient when framing the decision in terms of regaining function and averting loss of safety and independence. However, the evidence base regarding the outcomes of an SNF versus other postacute options is not robust, and this decision-making is complex. While this decision was most commonly framed in terms of rehabilitation and returning home, the fact that only about half of patients have returned to the community by 100 days⁴ was not discussed in any interview. In fact, initial evidence suggests replacing the SNF with home healthcare in patients with hip and knee arthroplasty may reduce costs without worsening clinical outcomes.⁶ However, across a broader population, SNFs significantly reduce 30-day readmissions when directly compared with home healthcare, but other clinical outcomes are similar.²⁵ This evidence suggests that the “right” postacute care option for an individual patient is not clear, highlighting a key role biases may play in decision-making. Further, the nebulous concept of “safety” could introduce potential disparities related to social determinants of health.¹² The observed inclination to accept an SNF with which the individual had prior experience may be influenced by the acceptability of this choice because of personal factors or prior research, even if it also represents a bias by limiting the consideration of current alternatives.

Our findings complement those of others in the literature which have also identified profound gaps in discharge decision-making among patients and clinicians,^13-16,26-31 though to our knowledge the role of cognitive biases in these decisions has not been explored. This study also addresses gaps in the cognitive bias literature, including the need for real-world data rather than hypothetical vignettes,¹⁷ and evaluation of treatment and management decisions rather than diagnoses, which have been more commonly studied.²¹

These findings have implications for both individual clinicians and healthcare institutions. In the immediate term, these findings may serve as a call to discharging clinicians to modulate language and “debias” their conversations with patients about care after discharge.^18,22 Shared decision-making requires an informed choice by patients based on their goals and values; framing a decision in a way that puts the clinician’s goals or values (eg, safety) ahead of patient values (eg, independence and autonomy) or limits disclosure (eg, a “rehab” is a nursing home) in the hope of influencing choice may be more consistent with framing bias and less with shared decision-making.¹⁴ Although controversy exists about the best way to “debias” oneself,³² self-awareness of bias is increasingly recognized across healthcare venues as critical to improving care for vulnerable populations.³³ The use of data rather than vignettes may be a useful debiasing strategy, although the limitations of currently available data (eg, capturing nursing home quality) are increasingly recognized.³⁴ From a policy and health system perspective, cognitive biases should be integrated into the development of decision aids to facilitate informed, shared, and high-quality decision-making that incorporates patient values, and perhaps “nudges” from behavioral economics to assist patients in choosing the right postdischarge care for them. Such nudges use principles of framing to influence care without restricting choice.³⁵ As the science informing best practice regarding postacute care improves, identifying the “right” postdischarge care may become easier and recommendations more evidence-based.³⁶

Strengths of the study include a large, diverse sample of patients, caregivers, and clinicians in both the hospital and SNF setting. Also, we used a team-based analysis with an experienced team and a deep knowledge of the data, including triangulation with clinicians to verify results. However, all hospitals and SNFs were located in a single metropolitan area, and responses may vary by region or population density. All three hospitals have housestaff teaching programs, and at the time of the interviews all three community SNFs were “five-star” facilities on the Nursing Home Compare website; results may be different at community hospitals or other SNFs. Hospitalists were the only physician group sampled in the hospital as they provide the majority of inpatient care to older adults; geriatricians, in particular, may have had different perspectives. Since we intended to explore whether cognitive biases were present overall, we did not evaluate whether cognitive biases differed by role or subgroup (by clinician type, patient, or caregiver), but this may be a promising area to explore in future work. Many cognitive biases have been described, and there are likely additional biases we did not identify. To confirm the generalizability of these findings, they should be studied in a larger, more generalizable sample of respondents in future work.

Cognitive biases play an important role in patient decision-making about postacute care, particularly regarding SNF care. As postacute care undergoes a transformation spurred by payment reforms, it is more important than ever to ensure that patients understand their choices at hospital discharge and can make a high-quality decision consistent with their goals.

The combination of decreasing hospital lengths of stay and increasing age and comorbidity of the United States population is a principal driver of the increased use of postacute care in the US.^1-3 Postacute care refers to care in long-term acute care hospitals, inpatient rehabilitation facilities, skilled nursing facilities (SNFs), and care provided by home health agencies after an acute hospitalization. In 2016, 43% of Medicare beneficiaries received postacute care after hospital discharge at the cost of $60 billion annually; nearly half of these received care in an SNF.⁴ Increasing recognition of the significant cost and poor outcomes of postacute care led to payment reforms, such as bundled payments, that incentivized less expensive forms of postacute care and improvements in outcomes.^5-9 Early evaluations suggested that hospitals are sensitive to these reforms and responded by significantly decreasing SNF utilization.^10,11 It remains unclear whether this was safe and effective.

In this context, increased attention to how hospital clinicians and hospitalized patients decide whether to use postacute care (and what form to use) is appropriate since the effect of payment reforms could negatively impact vulnerable populations of older adults without adequate protection.¹² Suboptimal decision-making can drive both overuse and inappropriate underuse of this expensive medical resource. Initial evidence suggests that patients and clinicians are poorly equipped to make high-quality decisions about postacute care, with significant deficits in both the decision-making process and content.^13-16 While these gaps are important to address, they may only be part of the problem. The fields of cognitive psychology and behavioral economics have revealed new insights into decision-making, demonstrating that people deviate from rational decision-making in predictable ways, termed decision heuristics, or cognitive biases.¹⁷ This growing field of research suggests heuristics or biases play important roles in decision-making and determining behavior, particularly in situations where there may be little information provided and the patient is stressed, tired, and ill—precisely like deciding on postacute care.¹⁸ However, it is currently unknown whether cognitive biases are at play when making hospital discharge decisions.

We sought to identify the most salient heuristics or cognitive biases patients may utilize when making decisions about postacute care at the end of their hospitalization and ways clinicians may contribute to these biases. The overall goal was to derive insights for improving postacute care decision-making.

Study Design

We conducted a secondary analysis on interviews with hospital and SNF clinicians as well as patients and their caregivers who were either leaving the hospital for an SNF or newly arrived in an SNF from the hospital to understand if cognitive biases were present and how they manifested themselves in a real-world clinical context.¹⁹ These interviews were part of a larger qualitative study that sought to understand how clinicians, patients, and their caregivers made decisions about postacute care, particularly related to SNFs.^13,14 This study represents the analysis of all our interviews, specifically examining decision-making bias. Participating sites, clinical roles, and both patient and caregiver characteristics (Table 1) in our cohort have been previously described.^13,14

Analysis

We used a team-based approach to framework analysis, which has been used in other decision-making studies¹⁴, including those measuring cognitive bias.²⁰ A limitation in cognitive bias research is the lack of a standardized list or categorization of cognitive biases. We reviewed prior systematic^17,21 and narrative reviews^18,22, as well as prior studies describing examples of cognitive biases playing a role in decision-making about therapy²⁰ to construct a list of possible cognitive biases to evaluate and narrow these a priori to potential biases relevant to the decision about postacute care based on our prior work (Table 2).

We applied this framework to analyze transcripts through an iterative process of deductive coding and reviewing across four reviewers (ML, RA, AL, CL) and a hospitalist physician with expertise leading qualitative studies (REB).

Intercoder consensus was built through team discussion by resolving points of disagreement.²³ Consistency of coding was regularly checked by having more than one investigator code individual manuscripts and comparing coding, and discrepancies were resolved through team discussion. We triangulated the data (shared our preliminary results) using a larger study team, including an expert in behavioral economics (SRG), physicians at study sites (EC, RA), and an anthropologist with expertise in qualitative methods (CL). We did this to ensure credibility (to what extent the findings are credible or believable) and confirmability of findings (ensuring the findings are based on participant narratives rather than researcher biases).

We reviewed a total of 105 interviews with 25 hospital clinicians, 20 SNF clinicians, 21 patients and 14 caregivers in the hospital, and 15 patients and 10 caregivers in the SNF setting (Table 1). We found authority bias/halo effect; default/status quo bias, anchoring bias, and framing was commonly present in decision-making about postacute care in a SNF, whereas there were few if any examples of ambiguity aversion, availability heuristic, confirmation bias, optimism bias, or false consensus effect (Table 2).

Authority Bias/Halo Effect

While most patients deferred to their inpatient teams when it came to decision-making, this effect seemed to differ across VA and non-VA settings. Veterans expressed a higher degree of potential authority bias regarding the VA as an institution, whereas older adults in non-VA settings saw physicians as the authority figure making decisions in their best interests.

Veterans expressed confidence in the VA regarding both whether to go to a SNF and where to go:

“The VA wouldn’t license [an SNF] if they didn’t have a good reputation for care, cleanliness, things of that nature” (Veteran, VA CLC)

“I just knew the VA would have my best interests at heart” (Veteran, VA CLC)

Their caregivers expressed similar confidence:

“I’m not gonna decide [on whether the patient they care for goes to postacute care], like I told you, that’s totally up to the VA. I have trust and faith in them…so wherever they send him, that’s where he’s going” (Caregiver, VA hospital)

In some cases, this perspective was closer to the halo effect: a positive experience with the care provider or the care team led the decision-makers to believe that their recommendations about postacute care would be similarly positive.

“I think we were very trusting in the sense that whatever happened the last time around, he survived it…they took care of him…he got back home, and he started his life again, you know, so why would we question what they’re telling us to do? (Caregiver, VA hospital)

In contrast to Veterans, non-Veteran patients seemed to experience authority bias when it came to the inpatient team.

“Well, I’d like to know more about the PTs [Physical Therapists] there, but I assume since they were recommended, they will be good.” (Patient, University hospital)

This perspective was especially apparent when it came to physicians:

“The level of trust that they [patients] put in their doctor is gonna outweigh what anyone else would say.” (Clinical liaison, SNF)

“[In response to a question about influences on the decision to go to rehab] I don’t…that’s not my decision to make, that’s the doctor’s decision.” (Patient, University hospital)

“They said so…[the doctor] said I needed to go to rehab, so I guess I do because it’s the doctor’s decision.” (Patient, University hospital)

Default/Status quo Bias

In a related way, patients and caregivers with exposure to a SNF seemed to default to the same SNF with which they had previous experience. This bias seems to be primarily related to knowing what to expect.

“He thinks it’s [a particular SNF] the right place for him now…he was there before and he knew, again, it was the right place for him to be” (Caregiver, VA hospital)

“It’s the only one I’ve ever been in…but they have a lot of activities; you have a lot of freedom, staff was good” (Patient, VA hospital)

“I’ve been [to this SNF] before and I kind of know what the program involves…so it was kind of like going home, not, going home is the wrong way to put it…I mean coming here is like something I know, you know, I didn’t need anybody to explain it to me.” (Patient, VA hospital)

“Anybody that’s been to [SNF], that would be their choice to go back to, and I guess I must’ve liked it that first time because I asked to go back again.” (Patient, University hospital)

Anchoring Bias

While anchoring bias was less frequent, it came up in two domains: first, related to costs of care, and second, related to facility characteristics. Costs came up most frequently for Veterans who preferred to move their care to the VA for cost reasons, which appeared in these cases to overshadow other considerations:

“I kept emphasizing that the VA could do all the same things at a lot more reasonable price. The whole purpose of having the VA is for the Veteran, so that…we can get the healthcare that we need at a more reasonable [sic] or a reasonable price.” (Veteran, CLC)

“I think the CLC [VA SNF] is going to take care of her probably the same way any other facility of its type would, unless she were in a private facility, but you know, that costs a lot more money.” (Caregiver, VA hospital)

Patients occasionally had striking responses to particular characteristics of SNFs, regardless of whether this was a central feature or related to their rehabilitation:

“The social worker comes and talks to me about the nursing home where cats are running around, you know, to infect my leg or spin their little cat hairs into my lungs and make my asthma worse…I’m going to have to beg the nurses or the aides or the family or somebody to clean the cat…” (Veteran, VA hospital)

Framing

Framing was the strongest theme among clinician interviews in our sample. Clinicians most frequently described the SNF as a place where patients could recover function (a positive frame), explaining risks (eg, rehospitalization) associated with alternative postacute care options besides the SNF in great detail.

“Aside from explaining the benefits of going and…having that 24-hour care, having the therapies provided to them [the patients], talking about them getting stronger, phrasing it in such a way that patients sometimes are more agreeable, like not calling it a skilled nursing facility, calling it a rehab you know, for them to get physically stronger so they can be the most independent that they can once they do go home, and also explaining … we think that this would be the best plan to prevent them from coming back to the hospital, so those are some of the things that we’ll mention to patients to try and educate them and get them to be agreeable for placement.” (Social worker, University hospital)

Clinicians avoided negative associations with “nursing home” (even though all SNFs are nursing homes) and tended to use more positive frames such as “rehabilitation facility.”

“Use the word rehab….we definitely use the word rehab, to get more therapy, to go home; it’s not a, we really emphasize it’s not a nursing home, it’s not to go to stay forever.” (Physical therapist, safety-net hospital)

Clinicians used a frame of “safety” when discussing the SNF and used a frame of “risk” when discussing alternative postacute care options such as returning home. We did not find examples of clinicians discussing similar risks in going to a SNF even for risks, such as falling, which exist in both settings.

“I’ve talked to them primarily on an avenue of safety because I think people want and they value independence, they value making sure they can get home, but you know, a lot of the times they understand safety is, it can be a concern and outlining that our goal is to make sure that they’re safe and they stay home, and I tend to broach the subject saying that our therapists believe that they might not be safe at home in the moment, but they have potential goals to be safe later on if we continue therapy. I really highlight safety being the major driver of our discussion.” (Physician, VA hospital)

In some cases, framing was so overt that other risk-mitigating options (eg, home healthcare) are not discussed.

“I definitely tend to explain the ideal first. I’m not going to bring up home care when we really think somebody should go to rehab, however, once people say I don’t want to do that, I’m not going, then that’s when I’m like OK, well, let’s talk to the doctors, but we can see about other supports in the home.” (Social worker, VA hospital)

In a large sample of patients and their caregivers, as well as multidisciplinary clinicians at three different hospitals and three SNFs, we found authority bias/halo effect and framing biases were most common and seemed most impactful. Default/status quo bias and anchoring bias were also present in decision-making about a SNF. The combination of authority bias/halo effect and framing biases could synergistically interact to augment the likelihood of patients accepting a SNF for postacute care. Patients who had been to a SNF before seemed more likely to choose the SNF they had experienced previously even if they had no other postacute care experiences, and could be highly influenced by isolated characteristics of that facility (such as the physical environment or cost of care).

It is important to mention that cognitive biases do not necessarily have a negative impact: indeed, as Kahneman and Tversky point out, these are useful heuristics from “fast” thinking that are often effective.²⁴ For example, clinicians may be trying to act in the best interests of the patient when framing the decision in terms of regaining function and averting loss of safety and independence. However, the evidence base regarding the outcomes of an SNF versus other postacute options is not robust, and this decision-making is complex. While this decision was most commonly framed in terms of rehabilitation and returning home, the fact that only about half of patients have returned to the community by 100 days⁴ was not discussed in any interview. In fact, initial evidence suggests replacing the SNF with home healthcare in patients with hip and knee arthroplasty may reduce costs without worsening clinical outcomes.⁶ However, across a broader population, SNFs significantly reduce 30-day readmissions when directly compared with home healthcare, but other clinical outcomes are similar.²⁵ This evidence suggests that the “right” postacute care option for an individual patient is not clear, highlighting a key role biases may play in decision-making. Further, the nebulous concept of “safety” could introduce potential disparities related to social determinants of health.¹² The observed inclination to accept an SNF with which the individual had prior experience may be influenced by the acceptability of this choice because of personal factors or prior research, even if it also represents a bias by limiting the consideration of current alternatives.

Our findings complement those of others in the literature which have also identified profound gaps in discharge decision-making among patients and clinicians,^13-16,26-31 though to our knowledge the role of cognitive biases in these decisions has not been explored. This study also addresses gaps in the cognitive bias literature, including the need for real-world data rather than hypothetical vignettes,¹⁷ and evaluation of treatment and management decisions rather than diagnoses, which have been more commonly studied.²¹

These findings have implications for both individual clinicians and healthcare institutions. In the immediate term, these findings may serve as a call to discharging clinicians to modulate language and “debias” their conversations with patients about care after discharge.^18,22 Shared decision-making requires an informed choice by patients based on their goals and values; framing a decision in a way that puts the clinician’s goals or values (eg, safety) ahead of patient values (eg, independence and autonomy) or limits disclosure (eg, a “rehab” is a nursing home) in the hope of influencing choice may be more consistent with framing bias and less with shared decision-making.¹⁴ Although controversy exists about the best way to “debias” oneself,³² self-awareness of bias is increasingly recognized across healthcare venues as critical to improving care for vulnerable populations.³³ The use of data rather than vignettes may be a useful debiasing strategy, although the limitations of currently available data (eg, capturing nursing home quality) are increasingly recognized.³⁴ From a policy and health system perspective, cognitive biases should be integrated into the development of decision aids to facilitate informed, shared, and high-quality decision-making that incorporates patient values, and perhaps “nudges” from behavioral economics to assist patients in choosing the right postdischarge care for them. Such nudges use principles of framing to influence care without restricting choice.³⁵ As the science informing best practice regarding postacute care improves, identifying the “right” postdischarge care may become easier and recommendations more evidence-based.³⁶

Strengths of the study include a large, diverse sample of patients, caregivers, and clinicians in both the hospital and SNF setting. Also, we used a team-based analysis with an experienced team and a deep knowledge of the data, including triangulation with clinicians to verify results. However, all hospitals and SNFs were located in a single metropolitan area, and responses may vary by region or population density. All three hospitals have housestaff teaching programs, and at the time of the interviews all three community SNFs were “five-star” facilities on the Nursing Home Compare website; results may be different at community hospitals or other SNFs. Hospitalists were the only physician group sampled in the hospital as they provide the majority of inpatient care to older adults; geriatricians, in particular, may have had different perspectives. Since we intended to explore whether cognitive biases were present overall, we did not evaluate whether cognitive biases differed by role or subgroup (by clinician type, patient, or caregiver), but this may be a promising area to explore in future work. Many cognitive biases have been described, and there are likely additional biases we did not identify. To confirm the generalizability of these findings, they should be studied in a larger, more generalizable sample of respondents in future work.

Cognitive biases play an important role in patient decision-making about postacute care, particularly regarding SNF care. As postacute care undergoes a transformation spurred by payment reforms, it is more important than ever to ensure that patients understand their choices at hospital discharge and can make a high-quality decision consistent with their goals.

The combination of decreasing hospital lengths of stay and increasing age and comorbidity of the United States population is a principal driver of the increased use of postacute care in the US.^1-3 Postacute care refers to care in long-term acute care hospitals, inpatient rehabilitation facilities, skilled nursing facilities (SNFs), and care provided by home health agencies after an acute hospitalization. In 2016, 43% of Medicare beneficiaries received postacute care after hospital discharge at the cost of $60 billion annually; nearly half of these received care in an SNF.⁴ Increasing recognition of the significant cost and poor outcomes of postacute care led to payment reforms, such as bundled payments, that incentivized less expensive forms of postacute care and improvements in outcomes.^5-9 Early evaluations suggested that hospitals are sensitive to these reforms and responded by significantly decreasing SNF utilization.^10,11 It remains unclear whether this was safe and effective.

In this context, increased attention to how hospital clinicians and hospitalized patients decide whether to use postacute care (and what form to use) is appropriate since the effect of payment reforms could negatively impact vulnerable populations of older adults without adequate protection.¹² Suboptimal decision-making can drive both overuse and inappropriate underuse of this expensive medical resource. Initial evidence suggests that patients and clinicians are poorly equipped to make high-quality decisions about postacute care, with significant deficits in both the decision-making process and content.^13-16 While these gaps are important to address, they may only be part of the problem. The fields of cognitive psychology and behavioral economics have revealed new insights into decision-making, demonstrating that people deviate from rational decision-making in predictable ways, termed decision heuristics, or cognitive biases.¹⁷ This growing field of research suggests heuristics or biases play important roles in decision-making and determining behavior, particularly in situations where there may be little information provided and the patient is stressed, tired, and ill—precisely like deciding on postacute care.¹⁸ However, it is currently unknown whether cognitive biases are at play when making hospital discharge decisions.

We sought to identify the most salient heuristics or cognitive biases patients may utilize when making decisions about postacute care at the end of their hospitalization and ways clinicians may contribute to these biases. The overall goal was to derive insights for improving postacute care decision-making.

Study Design

We conducted a secondary analysis on interviews with hospital and SNF clinicians as well as patients and their caregivers who were either leaving the hospital for an SNF or newly arrived in an SNF from the hospital to understand if cognitive biases were present and how they manifested themselves in a real-world clinical context.¹⁹ These interviews were part of a larger qualitative study that sought to understand how clinicians, patients, and their caregivers made decisions about postacute care, particularly related to SNFs.^13,14 This study represents the analysis of all our interviews, specifically examining decision-making bias. Participating sites, clinical roles, and both patient and caregiver characteristics (Table 1) in our cohort have been previously described.^13,14

Analysis

We used a team-based approach to framework analysis, which has been used in other decision-making studies¹⁴, including those measuring cognitive bias.²⁰ A limitation in cognitive bias research is the lack of a standardized list or categorization of cognitive biases. We reviewed prior systematic^17,21 and narrative reviews^18,22, as well as prior studies describing examples of cognitive biases playing a role in decision-making about therapy²⁰ to construct a list of possible cognitive biases to evaluate and narrow these a priori to potential biases relevant to the decision about postacute care based on our prior work (Table 2).

We applied this framework to analyze transcripts through an iterative process of deductive coding and reviewing across four reviewers (ML, RA, AL, CL) and a hospitalist physician with expertise leading qualitative studies (REB).

Intercoder consensus was built through team discussion by resolving points of disagreement.²³ Consistency of coding was regularly checked by having more than one investigator code individual manuscripts and comparing coding, and discrepancies were resolved through team discussion. We triangulated the data (shared our preliminary results) using a larger study team, including an expert in behavioral economics (SRG), physicians at study sites (EC, RA), and an anthropologist with expertise in qualitative methods (CL). We did this to ensure credibility (to what extent the findings are credible or believable) and confirmability of findings (ensuring the findings are based on participant narratives rather than researcher biases).

We reviewed a total of 105 interviews with 25 hospital clinicians, 20 SNF clinicians, 21 patients and 14 caregivers in the hospital, and 15 patients and 10 caregivers in the SNF setting (Table 1). We found authority bias/halo effect; default/status quo bias, anchoring bias, and framing was commonly present in decision-making about postacute care in a SNF, whereas there were few if any examples of ambiguity aversion, availability heuristic, confirmation bias, optimism bias, or false consensus effect (Table 2).

Authority Bias/Halo Effect

While most patients deferred to their inpatient teams when it came to decision-making, this effect seemed to differ across VA and non-VA settings. Veterans expressed a higher degree of potential authority bias regarding the VA as an institution, whereas older adults in non-VA settings saw physicians as the authority figure making decisions in their best interests.

Veterans expressed confidence in the VA regarding both whether to go to a SNF and where to go:

“The VA wouldn’t license [an SNF] if they didn’t have a good reputation for care, cleanliness, things of that nature” (Veteran, VA CLC)

“I just knew the VA would have my best interests at heart” (Veteran, VA CLC)

Their caregivers expressed similar confidence:

“I’m not gonna decide [on whether the patient they care for goes to postacute care], like I told you, that’s totally up to the VA. I have trust and faith in them…so wherever they send him, that’s where he’s going” (Caregiver, VA hospital)

In some cases, this perspective was closer to the halo effect: a positive experience with the care provider or the care team led the decision-makers to believe that their recommendations about postacute care would be similarly positive.

“I think we were very trusting in the sense that whatever happened the last time around, he survived it…they took care of him…he got back home, and he started his life again, you know, so why would we question what they’re telling us to do? (Caregiver, VA hospital)

In contrast to Veterans, non-Veteran patients seemed to experience authority bias when it came to the inpatient team.

“Well, I’d like to know more about the PTs [Physical Therapists] there, but I assume since they were recommended, they will be good.” (Patient, University hospital)

This perspective was especially apparent when it came to physicians:

“The level of trust that they [patients] put in their doctor is gonna outweigh what anyone else would say.” (Clinical liaison, SNF)

“[In response to a question about influences on the decision to go to rehab] I don’t…that’s not my decision to make, that’s the doctor’s decision.” (Patient, University hospital)

“They said so…[the doctor] said I needed to go to rehab, so I guess I do because it’s the doctor’s decision.” (Patient, University hospital)

Default/Status quo Bias

In a related way, patients and caregivers with exposure to a SNF seemed to default to the same SNF with which they had previous experience. This bias seems to be primarily related to knowing what to expect.

“He thinks it’s [a particular SNF] the right place for him now…he was there before and he knew, again, it was the right place for him to be” (Caregiver, VA hospital)

“It’s the only one I’ve ever been in…but they have a lot of activities; you have a lot of freedom, staff was good” (Patient, VA hospital)

“I’ve been [to this SNF] before and I kind of know what the program involves…so it was kind of like going home, not, going home is the wrong way to put it…I mean coming here is like something I know, you know, I didn’t need anybody to explain it to me.” (Patient, VA hospital)

“Anybody that’s been to [SNF], that would be their choice to go back to, and I guess I must’ve liked it that first time because I asked to go back again.” (Patient, University hospital)

Anchoring Bias

While anchoring bias was less frequent, it came up in two domains: first, related to costs of care, and second, related to facility characteristics. Costs came up most frequently for Veterans who preferred to move their care to the VA for cost reasons, which appeared in these cases to overshadow other considerations:

“I kept emphasizing that the VA could do all the same things at a lot more reasonable price. The whole purpose of having the VA is for the Veteran, so that…we can get the healthcare that we need at a more reasonable [sic] or a reasonable price.” (Veteran, CLC)

“I think the CLC [VA SNF] is going to take care of her probably the same way any other facility of its type would, unless she were in a private facility, but you know, that costs a lot more money.” (Caregiver, VA hospital)

Patients occasionally had striking responses to particular characteristics of SNFs, regardless of whether this was a central feature or related to their rehabilitation:

“The social worker comes and talks to me about the nursing home where cats are running around, you know, to infect my leg or spin their little cat hairs into my lungs and make my asthma worse…I’m going to have to beg the nurses or the aides or the family or somebody to clean the cat…” (Veteran, VA hospital)

Framing

Framing was the strongest theme among clinician interviews in our sample. Clinicians most frequently described the SNF as a place where patients could recover function (a positive frame), explaining risks (eg, rehospitalization) associated with alternative postacute care options besides the SNF in great detail.

“Aside from explaining the benefits of going and…having that 24-hour care, having the therapies provided to them [the patients], talking about them getting stronger, phrasing it in such a way that patients sometimes are more agreeable, like not calling it a skilled nursing facility, calling it a rehab you know, for them to get physically stronger so they can be the most independent that they can once they do go home, and also explaining … we think that this would be the best plan to prevent them from coming back to the hospital, so those are some of the things that we’ll mention to patients to try and educate them and get them to be agreeable for placement.” (Social worker, University hospital)

Clinicians avoided negative associations with “nursing home” (even though all SNFs are nursing homes) and tended to use more positive frames such as “rehabilitation facility.”

“Use the word rehab….we definitely use the word rehab, to get more therapy, to go home; it’s not a, we really emphasize it’s not a nursing home, it’s not to go to stay forever.” (Physical therapist, safety-net hospital)

Clinicians used a frame of “safety” when discussing the SNF and used a frame of “risk” when discussing alternative postacute care options such as returning home. We did not find examples of clinicians discussing similar risks in going to a SNF even for risks, such as falling, which exist in both settings.

“I’ve talked to them primarily on an avenue of safety because I think people want and they value independence, they value making sure they can get home, but you know, a lot of the times they understand safety is, it can be a concern and outlining that our goal is to make sure that they’re safe and they stay home, and I tend to broach the subject saying that our therapists believe that they might not be safe at home in the moment, but they have potential goals to be safe later on if we continue therapy. I really highlight safety being the major driver of our discussion.” (Physician, VA hospital)

In some cases, framing was so overt that other risk-mitigating options (eg, home healthcare) are not discussed.

“I definitely tend to explain the ideal first. I’m not going to bring up home care when we really think somebody should go to rehab, however, once people say I don’t want to do that, I’m not going, then that’s when I’m like OK, well, let’s talk to the doctors, but we can see about other supports in the home.” (Social worker, VA hospital)

In a large sample of patients and their caregivers, as well as multidisciplinary clinicians at three different hospitals and three SNFs, we found authority bias/halo effect and framing biases were most common and seemed most impactful. Default/status quo bias and anchoring bias were also present in decision-making about a SNF. The combination of authority bias/halo effect and framing biases could synergistically interact to augment the likelihood of patients accepting a SNF for postacute care. Patients who had been to a SNF before seemed more likely to choose the SNF they had experienced previously even if they had no other postacute care experiences, and could be highly influenced by isolated characteristics of that facility (such as the physical environment or cost of care).

It is important to mention that cognitive biases do not necessarily have a negative impact: indeed, as Kahneman and Tversky point out, these are useful heuristics from “fast” thinking that are often effective.²⁴ For example, clinicians may be trying to act in the best interests of the patient when framing the decision in terms of regaining function and averting loss of safety and independence. However, the evidence base regarding the outcomes of an SNF versus other postacute options is not robust, and this decision-making is complex. While this decision was most commonly framed in terms of rehabilitation and returning home, the fact that only about half of patients have returned to the community by 100 days⁴ was not discussed in any interview. In fact, initial evidence suggests replacing the SNF with home healthcare in patients with hip and knee arthroplasty may reduce costs without worsening clinical outcomes.⁶ However, across a broader population, SNFs significantly reduce 30-day readmissions when directly compared with home healthcare, but other clinical outcomes are similar.²⁵ This evidence suggests that the “right” postacute care option for an individual patient is not clear, highlighting a key role biases may play in decision-making. Further, the nebulous concept of “safety” could introduce potential disparities related to social determinants of health.¹² The observed inclination to accept an SNF with which the individual had prior experience may be influenced by the acceptability of this choice because of personal factors or prior research, even if it also represents a bias by limiting the consideration of current alternatives.

Our findings complement those of others in the literature which have also identified profound gaps in discharge decision-making among patients and clinicians,^13-16,26-31 though to our knowledge the role of cognitive biases in these decisions has not been explored. This study also addresses gaps in the cognitive bias literature, including the need for real-world data rather than hypothetical vignettes,¹⁷ and evaluation of treatment and management decisions rather than diagnoses, which have been more commonly studied.²¹

These findings have implications for both individual clinicians and healthcare institutions. In the immediate term, these findings may serve as a call to discharging clinicians to modulate language and “debias” their conversations with patients about care after discharge.^18,22 Shared decision-making requires an informed choice by patients based on their goals and values; framing a decision in a way that puts the clinician’s goals or values (eg, safety) ahead of patient values (eg, independence and autonomy) or limits disclosure (eg, a “rehab” is a nursing home) in the hope of influencing choice may be more consistent with framing bias and less with shared decision-making.¹⁴ Although controversy exists about the best way to “debias” oneself,³² self-awareness of bias is increasingly recognized across healthcare venues as critical to improving care for vulnerable populations.³³ The use of data rather than vignettes may be a useful debiasing strategy, although the limitations of currently available data (eg, capturing nursing home quality) are increasingly recognized.³⁴ From a policy and health system perspective, cognitive biases should be integrated into the development of decision aids to facilitate informed, shared, and high-quality decision-making that incorporates patient values, and perhaps “nudges” from behavioral economics to assist patients in choosing the right postdischarge care for them. Such nudges use principles of framing to influence care without restricting choice.³⁵ As the science informing best practice regarding postacute care improves, identifying the “right” postdischarge care may become easier and recommendations more evidence-based.³⁶

Strengths of the study include a large, diverse sample of patients, caregivers, and clinicians in both the hospital and SNF setting. Also, we used a team-based analysis with an experienced team and a deep knowledge of the data, including triangulation with clinicians to verify results. However, all hospitals and SNFs were located in a single metropolitan area, and responses may vary by region or population density. All three hospitals have housestaff teaching programs, and at the time of the interviews all three community SNFs were “five-star” facilities on the Nursing Home Compare website; results may be different at community hospitals or other SNFs. Hospitalists were the only physician group sampled in the hospital as they provide the majority of inpatient care to older adults; geriatricians, in particular, may have had different perspectives. Since we intended to explore whether cognitive biases were present overall, we did not evaluate whether cognitive biases differed by role or subgroup (by clinician type, patient, or caregiver), but this may be a promising area to explore in future work. Many cognitive biases have been described, and there are likely additional biases we did not identify. To confirm the generalizability of these findings, they should be studied in a larger, more generalizable sample of respondents in future work.

Cognitive biases play an important role in patient decision-making about postacute care, particularly regarding SNF care. As postacute care undergoes a transformation spurred by payment reforms, it is more important than ever to ensure that patients understand their choices at hospital discharge and can make a high-quality decision consistent with their goals.

Venous thromboembolism (VTE), comprising deep venous thrombosis and pulmonary embolism (PE),¹ is a serious medical condition that results in preventable morbidity and mortality.^1-5 VTE affects all age groups, all races/ethnicities, and both genders, but there are known factors that increase the risk of developing VTE (eg, advanced age, undergoing surgery, hospitalization, and immobility).^1-3,5-7 Prevention of VTE among hospitalized patients is of paramount importance to avoid preventable death, chronic illness/long-term complications,⁸ longer hospital stays, and increased hospital costs.⁹ Fortunately, there is clear evidence that provision of appropriate prophylaxis can decrease the risk of a VTE event occurring, and broadly accepted best-practice guidelines reflect this evidence.^3,5

Given the inadequacy of current VTE-related quality measures to identify actionable failures in the provision of VTE prophylaxis, our group created a VTE process-of-care measure to assess adherence to the components of VTE prophylaxis: (1) early ambulation, (2) mechanical prophylaxis (sequential compression devices [SCDs]), and (3) chemoprophylaxis administered at the correct dose and frequency for the duration of the patient’s hospital stay.^3,10,11 This quality measure was conceived, created, and iteratively revised to measure whether optimal care is provided to patients throughout their hospitalization and identify actionable areas in which failures of care occur, in order to decrease the risk of a VTE event. Data from our institution provided evidence that while ambulation and SCD component measure adherence is high, chemoprophylaxis adherence required significant improvement.¹⁰ When chemoprophylaxis process measure adherence data were analyzed further, a major failure mode was patient refusal of one or more doses. However, the drivers of patient refusal are not well defined in the literature, and previous studies have called for a greater focus on developing interventions to improve VTE chemoprophylaxis administration.¹²

Previous research has shown that nurses can influence patient compliance with VTE prophylaxis.^13-15 A mixed-methods study by Elder et al. found that nurses in units with high rates of failure to provide optimal chemoprophylaxis offered the medication as optional, leading researchers to conclude that nurses perceived chemoprophylaxis as discretionary.¹³ Another study by Lee et al., conducted a survey of bedside registered nurses and identified nurses’ lack of education on VTE prevention as a significant barrier to providing care.¹⁴ These studies show that multiple levels of influence impact how nurses provide VTE chemoprophylaxis, particularly when they encounter patients who refuse chemoprophylaxis.

To explore the nuance and interplay of multiple influences, we used the Theoretical Domains Framework (TDF), an integrative framework that applies theoretical approaches to interventions aimed at behavior change.^15-18 The framework contains 14 interrelated domains that characterize the behavior being studied, in this case, administration of VTE chemoprophylaxis. Consequently, we designed a nurse-focused, qualitative evaluation with the objective to identify nursing-related barriers to administration of VTE chemoprophylaxis.

Inpatient Unit Selection

The study team accessed data from the hospital’s Enterprise Data Warehouse to review patient refusal rates of VTE chemoprophylaxis for each inpatient unit in the hospital. Patient refusal was utilized as a proxy measure for the behavior of nurses attempting to administer VTE chemoprophylaxis. Of the 14 medical and surgical units in the hospital, two medical and two surgical units were selected to participate in the qualitative evaluation based on having the highest patient refusal rates. One unit (surgical) was also selected to serve as a benchmark because it had the lowest patient refusal rate. Table 1 includes the refusal rates for the five units. Given the low refusal rate for the best performing unit, we suspected that it would be possible to decrease the patient refusal rate for other units with similar patient populations and interprofessional teams at the institution.

Observations

We observed chemoprophylaxis administration on the five units to understand the process for ordering and administering chemoprophylaxis. An observation protocol was utilized to document the date, time, and location of the observation as well as descriptive notes including accounts of particular events.^19,20 Observations occurred in May 2016 and informed the creation of a process map outlining the procedure for ordering and administering VTE chemoprophylaxis. The process map was utilized to create the focus group interview guide and ensure the interview guide included pertinent questions for each step of the process (Appendix A).

Focus Group Interviews

We conducted focus group interviews with day and night shift nurses on the five units to assess nurses’ understanding of VTE chemoprophylaxis and nurses’ perceptions of barriers to administration of VTE chemoprophylaxis. The study team chose to conduct focus group interviews in an effort to maximize participation and to speak with multiple nurses within a shorter period of time. The focus group structure allowed the study team to speak with nurses during their shifts, as one could briefly step out, if required, for patient care and return to rejoin the discussion.

We developed a semistructured interview guide²¹ with questions focused on identifying nurses’ perceptions of guideline-recommended care for VTE chemoprophylaxis, where they learned these guidelines, how nurses discuss chemoprophylaxis with patients, how they handle the conversation with patients who refuse, and if there are times when chemoprophylaxis is not necessary. The interview guide was vetted by a multidisciplinary team consisting of clinical nursing coordinators and nurse managers from medical and surgical units, hospital quality leaders, surgeons and general internists, and qualitative research experts. The interview guide is included as Appendix B.

The unit clinical coordinators and nurse managers identified dates and times for the focus groups that would be minimally disruptive to the unit. For each of the four units with a high patient refusal rate, two focus groups were conducted during the lunch hour and one was conducted at the end of the night shift to ensure that both day and night shift nurses were included in the study. Two focus groups were conducted with the best-practice unit during the lunch hour. For each focus group, the clinical coordinator identified two to eight nurses who could step away from patient care to participate or who had completed their shifts. In total, approximately 67 nurses participated in the focus groups.

The focus groups (n = 14) lasted approximately 40 minutes during May and June 2016. Two members of the study team cofacilitated interviews, which were recorded and transcribed verbatim.

Coding and Data Analysis

To develop the code book, the study team, consisting of three qualitative researchers, independently read one focus group transcript and applied the TDF domains to the nurses’ perceptions of barriers to administration of VTE chemoprophylaxis.^21-24 In addition to coding by domain, the study team also coded nursing perceptions as barriers or facilitators. The study team reviewed the coded transcript and reconciled any differences in coding. This process was repeated for a second transcript, and then all remaining transcripts were assigned to two out of three study team members for coding, with the entire study team meeting to reconcile any differences. If necessary, the team member who did not code a transcript acted as the tie-breaker if there were discrepancies in codes that could not be reconciled.

Once coding was completed, we identified the TDF domains that were most relevant to the administration of VTE chemoprophylaxis.¹⁶ Member checking (testing the analysis, interpretations, and conclusions with members of those groups from whom the data were originally obtained) was performed with the four clinical nursing coordinators and four nurse managers from the participating units to establish face validity of the themes identified from the focus group interviews.²⁵

The study team used MaxQDA, V12 (Berlin, Germany) to support data coding and analysis.²⁶ The Northwestern University institutional review board office deemed this project research on nonhuman-subjects because it focused on the process of providing VTE chemoprophylaxis and not about the patients themselves. The purpose of the study was explained at the beginning of each focus group, and nurses gave verbal consent to have the focus group recorded.

We conducted 14 focus groups with day and night shift nurses from five units (two medical and three surgical) at a single institution. All nurses invited to participate in a focus group agreed to participate. The data were coded and grouped by domain and identified as barriers or facilitators. The findings included below are for the domains most relevant to the provision of VTE prophylaxis. Table 2 provides illustrative verbatim quotes for each domain that was represented in the focus groups.

Knowledge

All interviewees recognized that providing some form of prophylaxis to mitigate the risk of a VTE event is essential. Some nurses stated that seeing a patient ambulating meant they would consider not administering prescribed chemoprophylaxis, while others would try to negotiate with patients by asking the patient to allow one dose of chemoprophylaxis prescribed two to three times daily because it was better than receiving no doses.

Environmental Context and Resources

Multiple barriers to providing optimal care were associated with the environmental context and a lack of resources. There was a lack of accessible, comprehensive, patient-centered education materials on VTE chemoprophylaxis to supplement a nurse’s explanation about the importance of chemoprophylaxis. Furthermore, many nurses cited the perceived patient pain of chemoprophylaxis injections as the main deterrent to patient compliance, especially subcutaneous heparin injections, which occur up to three times in 24 hours and often cause more pain at the site of injection than low-molecular-weight heparin. Nurses felt that transitioning patients from receiving subcutaneous heparin injections to receiving low-molecular-weight heparin could be a main driver to reduce patient refusals.

Skills

Nurses felt inadequately equipped to handle patient refusals. Many said that patient refusal of treatments was never discussed in nursing school. As a result, when patients refused treatments, the nurses did not know how to handle the situation. They felt that they lacked the tools and techniques to persuade the patient to comply.

Beliefs about Capabilities

Nurses did not know their own patient refusal rate or benchmarks of an acceptable refusal rate in contrast to one that is too high. Without this feedback, they were unable to assess their own behavior or performance related to providing VTE chemoprophylaxis.

Nurses play a critical role in providing VTE chemoprophylaxis to patients throughout their hospitalization. This study provided a unique opportunity to perform an in-depth, qualitative analysis of the barriers nurses face in providing patients with VTE chemoprophylaxis as part of their daily work caring for patients. We discovered several nursing-related barriers to the provision of VTE chemoprophylaxis, including lack of knowledge, resources, skill, and misconceptions of their capability to provide VTE chemoprophylaxis. We used a bottom-up approach by incorporating the voices of unit nurses, clinical coordinators, and nurse managers to understand potential barriers. Our findings brought to light the challenge of delivering standardized care in an area of care that is generally agreed upon, yet not fully followed. Some nurses display greater proficiency than others at communicating with patients who do not understand their risk for VTE and need for chemoprophylaxis. Furthermore, there is a pronounced misconception around the delivery of VTE chemoprophylaxis. Nurses have the inaccurate belief that even if ordered, chemoprophylaxis is not required. This misconception was widespread among nurses taking care of both medical and surgical patients. These factors appear to be modifiable targets for quality improvement and highlight the need for a skills-based education during the new hire onboarding process, as well as ongoing reeducation to ensure nursing staff have the skills to appropriately provide best-practice care for VTE chemoprophylaxis. Nurses felt ownership of the results of the qualitative evaluation because they were included in every aspect from the beginning.²⁷ This sense of ownership will support future quality improvement efforts to develop a skills-based intervention to improve the provision of VTE chemoprophylaxis.^18,27

This study has certain limitations. First, it was a qualitative study assessing nursing-related barriers to providing VTE chemoprophylaxis at a single institution, and the results cannot be generalized broadly. However, the techniques and results are transferable to other hospital settings and other clinical care situations. Thus, we believe that other institutions can utilize our methods and that similar lessons can be learned and applied. Furthermore, the validity of our study is bolstered by concordance between the results of this study and those of other studies conducted on the topic of provision of VTE prophylaxis by nurses.^13-15,21 Other studies utilized observations and surveys to determine potential nurse-related barriers to the provision of VTE prophylaxis, such as lack of knowledge and the belief that the need for prophylaxis can be determined based on whether or not the patient is ambulating;^13,14 however, by utilizing focus group interviews, we allowed nurses to speak in their own voices about their experiences with VTE prophylaxis, and we were able to delve deeper and identify additional barriers that emerged from discussions with nurses, such as the lack of skill and misconceptions of capability.^28,29 Second, the study focused solely on nurses. Additional initiatives are underway to assess the roles of resident physicians, attending physicians, and patients in the provision of VTE prophylaxis.

Nursing-related barriers to the provision of VTE chemoprophylaxis include a lack of knowledge, resources, skills, and misconceptions of the consequences of missed elements of VTE prophylaxis. Future initiatives will focus on equipping nurses to have meaningful conversations with patients and engaging patients in their care through development of a multifaceted bundle of interventions. Furthermore, similar methods of qualitative inquiry will be used to identify the role of resident and attending physicians and patients in the provision of VTE chemoprophylaxis.

Acknowledgments

The authors thank Sonali Oberoi, Joanne Prinz, Nancy Tomaska, and Kate Paredes, as well as all the nurses who participated in focus group interviews for this study and the nurse managers and clinical coordinators who helped to schedule the focus group interviews.

Disclosures

The authors declare that they have no competing interests.

Funding

This study was funded by the Surgical Outcomes and Quality Improvement Center at Northwestern University.

Venous thromboembolism (VTE), comprising deep venous thrombosis and pulmonary embolism (PE),¹ is a serious medical condition that results in preventable morbidity and mortality.^1-5 VTE affects all age groups, all races/ethnicities, and both genders, but there are known factors that increase the risk of developing VTE (eg, advanced age, undergoing surgery, hospitalization, and immobility).^1-3,5-7 Prevention of VTE among hospitalized patients is of paramount importance to avoid preventable death, chronic illness/long-term complications,⁸ longer hospital stays, and increased hospital costs.⁹ Fortunately, there is clear evidence that provision of appropriate prophylaxis can decrease the risk of a VTE event occurring, and broadly accepted best-practice guidelines reflect this evidence.^3,5

Given the inadequacy of current VTE-related quality measures to identify actionable failures in the provision of VTE prophylaxis, our group created a VTE process-of-care measure to assess adherence to the components of VTE prophylaxis: (1) early ambulation, (2) mechanical prophylaxis (sequential compression devices [SCDs]), and (3) chemoprophylaxis administered at the correct dose and frequency for the duration of the patient’s hospital stay.^3,10,11 This quality measure was conceived, created, and iteratively revised to measure whether optimal care is provided to patients throughout their hospitalization and identify actionable areas in which failures of care occur, in order to decrease the risk of a VTE event. Data from our institution provided evidence that while ambulation and SCD component measure adherence is high, chemoprophylaxis adherence required significant improvement.¹⁰ When chemoprophylaxis process measure adherence data were analyzed further, a major failure mode was patient refusal of one or more doses. However, the drivers of patient refusal are not well defined in the literature, and previous studies have called for a greater focus on developing interventions to improve VTE chemoprophylaxis administration.¹²

Previous research has shown that nurses can influence patient compliance with VTE prophylaxis.^13-15 A mixed-methods study by Elder et al. found that nurses in units with high rates of failure to provide optimal chemoprophylaxis offered the medication as optional, leading researchers to conclude that nurses perceived chemoprophylaxis as discretionary.¹³ Another study by Lee et al., conducted a survey of bedside registered nurses and identified nurses’ lack of education on VTE prevention as a significant barrier to providing care.¹⁴ These studies show that multiple levels of influence impact how nurses provide VTE chemoprophylaxis, particularly when they encounter patients who refuse chemoprophylaxis.

To explore the nuance and interplay of multiple influences, we used the Theoretical Domains Framework (TDF), an integrative framework that applies theoretical approaches to interventions aimed at behavior change.^15-18 The framework contains 14 interrelated domains that characterize the behavior being studied, in this case, administration of VTE chemoprophylaxis. Consequently, we designed a nurse-focused, qualitative evaluation with the objective to identify nursing-related barriers to administration of VTE chemoprophylaxis.

Inpatient Unit Selection

The study team accessed data from the hospital’s Enterprise Data Warehouse to review patient refusal rates of VTE chemoprophylaxis for each inpatient unit in the hospital. Patient refusal was utilized as a proxy measure for the behavior of nurses attempting to administer VTE chemoprophylaxis. Of the 14 medical and surgical units in the hospital, two medical and two surgical units were selected to participate in the qualitative evaluation based on having the highest patient refusal rates. One unit (surgical) was also selected to serve as a benchmark because it had the lowest patient refusal rate. Table 1 includes the refusal rates for the five units. Given the low refusal rate for the best performing unit, we suspected that it would be possible to decrease the patient refusal rate for other units with similar patient populations and interprofessional teams at the institution.

Observations

We observed chemoprophylaxis administration on the five units to understand the process for ordering and administering chemoprophylaxis. An observation protocol was utilized to document the date, time, and location of the observation as well as descriptive notes including accounts of particular events.^19,20 Observations occurred in May 2016 and informed the creation of a process map outlining the procedure for ordering and administering VTE chemoprophylaxis. The process map was utilized to create the focus group interview guide and ensure the interview guide included pertinent questions for each step of the process (Appendix A).

Focus Group Interviews

We conducted focus group interviews with day and night shift nurses on the five units to assess nurses’ understanding of VTE chemoprophylaxis and nurses’ perceptions of barriers to administration of VTE chemoprophylaxis. The study team chose to conduct focus group interviews in an effort to maximize participation and to speak with multiple nurses within a shorter period of time. The focus group structure allowed the study team to speak with nurses during their shifts, as one could briefly step out, if required, for patient care and return to rejoin the discussion.

We developed a semistructured interview guide²¹ with questions focused on identifying nurses’ perceptions of guideline-recommended care for VTE chemoprophylaxis, where they learned these guidelines, how nurses discuss chemoprophylaxis with patients, how they handle the conversation with patients who refuse, and if there are times when chemoprophylaxis is not necessary. The interview guide was vetted by a multidisciplinary team consisting of clinical nursing coordinators and nurse managers from medical and surgical units, hospital quality leaders, surgeons and general internists, and qualitative research experts. The interview guide is included as Appendix B.

The unit clinical coordinators and nurse managers identified dates and times for the focus groups that would be minimally disruptive to the unit. For each of the four units with a high patient refusal rate, two focus groups were conducted during the lunch hour and one was conducted at the end of the night shift to ensure that both day and night shift nurses were included in the study. Two focus groups were conducted with the best-practice unit during the lunch hour. For each focus group, the clinical coordinator identified two to eight nurses who could step away from patient care to participate or who had completed their shifts. In total, approximately 67 nurses participated in the focus groups.

The focus groups (n = 14) lasted approximately 40 minutes during May and June 2016. Two members of the study team cofacilitated interviews, which were recorded and transcribed verbatim.

Coding and Data Analysis

To develop the code book, the study team, consisting of three qualitative researchers, independently read one focus group transcript and applied the TDF domains to the nurses’ perceptions of barriers to administration of VTE chemoprophylaxis.^21-24 In addition to coding by domain, the study team also coded nursing perceptions as barriers or facilitators. The study team reviewed the coded transcript and reconciled any differences in coding. This process was repeated for a second transcript, and then all remaining transcripts were assigned to two out of three study team members for coding, with the entire study team meeting to reconcile any differences. If necessary, the team member who did not code a transcript acted as the tie-breaker if there were discrepancies in codes that could not be reconciled.

Once coding was completed, we identified the TDF domains that were most relevant to the administration of VTE chemoprophylaxis.¹⁶ Member checking (testing the analysis, interpretations, and conclusions with members of those groups from whom the data were originally obtained) was performed with the four clinical nursing coordinators and four nurse managers from the participating units to establish face validity of the themes identified from the focus group interviews.²⁵

The study team used MaxQDA, V12 (Berlin, Germany) to support data coding and analysis.²⁶ The Northwestern University institutional review board office deemed this project research on nonhuman-subjects because it focused on the process of providing VTE chemoprophylaxis and not about the patients themselves. The purpose of the study was explained at the beginning of each focus group, and nurses gave verbal consent to have the focus group recorded.

We conducted 14 focus groups with day and night shift nurses from five units (two medical and three surgical) at a single institution. All nurses invited to participate in a focus group agreed to participate. The data were coded and grouped by domain and identified as barriers or facilitators. The findings included below are for the domains most relevant to the provision of VTE prophylaxis. Table 2 provides illustrative verbatim quotes for each domain that was represented in the focus groups.

Knowledge

All interviewees recognized that providing some form of prophylaxis to mitigate the risk of a VTE event is essential. Some nurses stated that seeing a patient ambulating meant they would consider not administering prescribed chemoprophylaxis, while others would try to negotiate with patients by asking the patient to allow one dose of chemoprophylaxis prescribed two to three times daily because it was better than receiving no doses.

Environmental Context and Resources

Multiple barriers to providing optimal care were associated with the environmental context and a lack of resources. There was a lack of accessible, comprehensive, patient-centered education materials on VTE chemoprophylaxis to supplement a nurse’s explanation about the importance of chemoprophylaxis. Furthermore, many nurses cited the perceived patient pain of chemoprophylaxis injections as the main deterrent to patient compliance, especially subcutaneous heparin injections, which occur up to three times in 24 hours and often cause more pain at the site of injection than low-molecular-weight heparin. Nurses felt that transitioning patients from receiving subcutaneous heparin injections to receiving low-molecular-weight heparin could be a main driver to reduce patient refusals.

Skills

Nurses felt inadequately equipped to handle patient refusals. Many said that patient refusal of treatments was never discussed in nursing school. As a result, when patients refused treatments, the nurses did not know how to handle the situation. They felt that they lacked the tools and techniques to persuade the patient to comply.

Beliefs about Capabilities

Nurses did not know their own patient refusal rate or benchmarks of an acceptable refusal rate in contrast to one that is too high. Without this feedback, they were unable to assess their own behavior or performance related to providing VTE chemoprophylaxis.

Nurses play a critical role in providing VTE chemoprophylaxis to patients throughout their hospitalization. This study provided a unique opportunity to perform an in-depth, qualitative analysis of the barriers nurses face in providing patients with VTE chemoprophylaxis as part of their daily work caring for patients. We discovered several nursing-related barriers to the provision of VTE chemoprophylaxis, including lack of knowledge, resources, skill, and misconceptions of their capability to provide VTE chemoprophylaxis. We used a bottom-up approach by incorporating the voices of unit nurses, clinical coordinators, and nurse managers to understand potential barriers. Our findings brought to light the challenge of delivering standardized care in an area of care that is generally agreed upon, yet not fully followed. Some nurses display greater proficiency than others at communicating with patients who do not understand their risk for VTE and need for chemoprophylaxis. Furthermore, there is a pronounced misconception around the delivery of VTE chemoprophylaxis. Nurses have the inaccurate belief that even if ordered, chemoprophylaxis is not required. This misconception was widespread among nurses taking care of both medical and surgical patients. These factors appear to be modifiable targets for quality improvement and highlight the need for a skills-based education during the new hire onboarding process, as well as ongoing reeducation to ensure nursing staff have the skills to appropriately provide best-practice care for VTE chemoprophylaxis. Nurses felt ownership of the results of the qualitative evaluation because they were included in every aspect from the beginning.²⁷ This sense of ownership will support future quality improvement efforts to develop a skills-based intervention to improve the provision of VTE chemoprophylaxis.^18,27

This study has certain limitations. First, it was a qualitative study assessing nursing-related barriers to providing VTE chemoprophylaxis at a single institution, and the results cannot be generalized broadly. However, the techniques and results are transferable to other hospital settings and other clinical care situations. Thus, we believe that other institutions can utilize our methods and that similar lessons can be learned and applied. Furthermore, the validity of our study is bolstered by concordance between the results of this study and those of other studies conducted on the topic of provision of VTE prophylaxis by nurses.^13-15,21 Other studies utilized observations and surveys to determine potential nurse-related barriers to the provision of VTE prophylaxis, such as lack of knowledge and the belief that the need for prophylaxis can be determined based on whether or not the patient is ambulating;^13,14 however, by utilizing focus group interviews, we allowed nurses to speak in their own voices about their experiences with VTE prophylaxis, and we were able to delve deeper and identify additional barriers that emerged from discussions with nurses, such as the lack of skill and misconceptions of capability.^28,29 Second, the study focused solely on nurses. Additional initiatives are underway to assess the roles of resident physicians, attending physicians, and patients in the provision of VTE prophylaxis.

Nursing-related barriers to the provision of VTE chemoprophylaxis include a lack of knowledge, resources, skills, and misconceptions of the consequences of missed elements of VTE prophylaxis. Future initiatives will focus on equipping nurses to have meaningful conversations with patients and engaging patients in their care through development of a multifaceted bundle of interventions. Furthermore, similar methods of qualitative inquiry will be used to identify the role of resident and attending physicians and patients in the provision of VTE chemoprophylaxis.

Acknowledgments

The authors thank Sonali Oberoi, Joanne Prinz, Nancy Tomaska, and Kate Paredes, as well as all the nurses who participated in focus group interviews for this study and the nurse managers and clinical coordinators who helped to schedule the focus group interviews.

Disclosures

The authors declare that they have no competing interests.

Funding

This study was funded by the Surgical Outcomes and Quality Improvement Center at Northwestern University.

Venous thromboembolism (VTE), comprising deep venous thrombosis and pulmonary embolism (PE),¹ is a serious medical condition that results in preventable morbidity and mortality.^1-5 VTE affects all age groups, all races/ethnicities, and both genders, but there are known factors that increase the risk of developing VTE (eg, advanced age, undergoing surgery, hospitalization, and immobility).^1-3,5-7 Prevention of VTE among hospitalized patients is of paramount importance to avoid preventable death, chronic illness/long-term complications,⁸ longer hospital stays, and increased hospital costs.⁹ Fortunately, there is clear evidence that provision of appropriate prophylaxis can decrease the risk of a VTE event occurring, and broadly accepted best-practice guidelines reflect this evidence.^3,5

Given the inadequacy of current VTE-related quality measures to identify actionable failures in the provision of VTE prophylaxis, our group created a VTE process-of-care measure to assess adherence to the components of VTE prophylaxis: (1) early ambulation, (2) mechanical prophylaxis (sequential compression devices [SCDs]), and (3) chemoprophylaxis administered at the correct dose and frequency for the duration of the patient’s hospital stay.^3,10,11 This quality measure was conceived, created, and iteratively revised to measure whether optimal care is provided to patients throughout their hospitalization and identify actionable areas in which failures of care occur, in order to decrease the risk of a VTE event. Data from our institution provided evidence that while ambulation and SCD component measure adherence is high, chemoprophylaxis adherence required significant improvement.¹⁰ When chemoprophylaxis process measure adherence data were analyzed further, a major failure mode was patient refusal of one or more doses. However, the drivers of patient refusal are not well defined in the literature, and previous studies have called for a greater focus on developing interventions to improve VTE chemoprophylaxis administration.¹²

Previous research has shown that nurses can influence patient compliance with VTE prophylaxis.^13-15 A mixed-methods study by Elder et al. found that nurses in units with high rates of failure to provide optimal chemoprophylaxis offered the medication as optional, leading researchers to conclude that nurses perceived chemoprophylaxis as discretionary.¹³ Another study by Lee et al., conducted a survey of bedside registered nurses and identified nurses’ lack of education on VTE prevention as a significant barrier to providing care.¹⁴ These studies show that multiple levels of influence impact how nurses provide VTE chemoprophylaxis, particularly when they encounter patients who refuse chemoprophylaxis.

To explore the nuance and interplay of multiple influences, we used the Theoretical Domains Framework (TDF), an integrative framework that applies theoretical approaches to interventions aimed at behavior change.^15-18 The framework contains 14 interrelated domains that characterize the behavior being studied, in this case, administration of VTE chemoprophylaxis. Consequently, we designed a nurse-focused, qualitative evaluation with the objective to identify nursing-related barriers to administration of VTE chemoprophylaxis.

Inpatient Unit Selection

The study team accessed data from the hospital’s Enterprise Data Warehouse to review patient refusal rates of VTE chemoprophylaxis for each inpatient unit in the hospital. Patient refusal was utilized as a proxy measure for the behavior of nurses attempting to administer VTE chemoprophylaxis. Of the 14 medical and surgical units in the hospital, two medical and two surgical units were selected to participate in the qualitative evaluation based on having the highest patient refusal rates. One unit (surgical) was also selected to serve as a benchmark because it had the lowest patient refusal rate. Table 1 includes the refusal rates for the five units. Given the low refusal rate for the best performing unit, we suspected that it would be possible to decrease the patient refusal rate for other units with similar patient populations and interprofessional teams at the institution.

Observations

We observed chemoprophylaxis administration on the five units to understand the process for ordering and administering chemoprophylaxis. An observation protocol was utilized to document the date, time, and location of the observation as well as descriptive notes including accounts of particular events.^19,20 Observations occurred in May 2016 and informed the creation of a process map outlining the procedure for ordering and administering VTE chemoprophylaxis. The process map was utilized to create the focus group interview guide and ensure the interview guide included pertinent questions for each step of the process (Appendix A).

Focus Group Interviews

We conducted focus group interviews with day and night shift nurses on the five units to assess nurses’ understanding of VTE chemoprophylaxis and nurses’ perceptions of barriers to administration of VTE chemoprophylaxis. The study team chose to conduct focus group interviews in an effort to maximize participation and to speak with multiple nurses within a shorter period of time. The focus group structure allowed the study team to speak with nurses during their shifts, as one could briefly step out, if required, for patient care and return to rejoin the discussion.

We developed a semistructured interview guide²¹ with questions focused on identifying nurses’ perceptions of guideline-recommended care for VTE chemoprophylaxis, where they learned these guidelines, how nurses discuss chemoprophylaxis with patients, how they handle the conversation with patients who refuse, and if there are times when chemoprophylaxis is not necessary. The interview guide was vetted by a multidisciplinary team consisting of clinical nursing coordinators and nurse managers from medical and surgical units, hospital quality leaders, surgeons and general internists, and qualitative research experts. The interview guide is included as Appendix B.

The unit clinical coordinators and nurse managers identified dates and times for the focus groups that would be minimally disruptive to the unit. For each of the four units with a high patient refusal rate, two focus groups were conducted during the lunch hour and one was conducted at the end of the night shift to ensure that both day and night shift nurses were included in the study. Two focus groups were conducted with the best-practice unit during the lunch hour. For each focus group, the clinical coordinator identified two to eight nurses who could step away from patient care to participate or who had completed their shifts. In total, approximately 67 nurses participated in the focus groups.

The focus groups (n = 14) lasted approximately 40 minutes during May and June 2016. Two members of the study team cofacilitated interviews, which were recorded and transcribed verbatim.

Coding and Data Analysis

To develop the code book, the study team, consisting of three qualitative researchers, independently read one focus group transcript and applied the TDF domains to the nurses’ perceptions of barriers to administration of VTE chemoprophylaxis.^21-24 In addition to coding by domain, the study team also coded nursing perceptions as barriers or facilitators. The study team reviewed the coded transcript and reconciled any differences in coding. This process was repeated for a second transcript, and then all remaining transcripts were assigned to two out of three study team members for coding, with the entire study team meeting to reconcile any differences. If necessary, the team member who did not code a transcript acted as the tie-breaker if there were discrepancies in codes that could not be reconciled.

Once coding was completed, we identified the TDF domains that were most relevant to the administration of VTE chemoprophylaxis.¹⁶ Member checking (testing the analysis, interpretations, and conclusions with members of those groups from whom the data were originally obtained) was performed with the four clinical nursing coordinators and four nurse managers from the participating units to establish face validity of the themes identified from the focus group interviews.²⁵

The study team used MaxQDA, V12 (Berlin, Germany) to support data coding and analysis.²⁶ The Northwestern University institutional review board office deemed this project research on nonhuman-subjects because it focused on the process of providing VTE chemoprophylaxis and not about the patients themselves. The purpose of the study was explained at the beginning of each focus group, and nurses gave verbal consent to have the focus group recorded.

We conducted 14 focus groups with day and night shift nurses from five units (two medical and three surgical) at a single institution. All nurses invited to participate in a focus group agreed to participate. The data were coded and grouped by domain and identified as barriers or facilitators. The findings included below are for the domains most relevant to the provision of VTE prophylaxis. Table 2 provides illustrative verbatim quotes for each domain that was represented in the focus groups.

Knowledge

All interviewees recognized that providing some form of prophylaxis to mitigate the risk of a VTE event is essential. Some nurses stated that seeing a patient ambulating meant they would consider not administering prescribed chemoprophylaxis, while others would try to negotiate with patients by asking the patient to allow one dose of chemoprophylaxis prescribed two to three times daily because it was better than receiving no doses.

Environmental Context and Resources

Multiple barriers to providing optimal care were associated with the environmental context and a lack of resources. There was a lack of accessible, comprehensive, patient-centered education materials on VTE chemoprophylaxis to supplement a nurse’s explanation about the importance of chemoprophylaxis. Furthermore, many nurses cited the perceived patient pain of chemoprophylaxis injections as the main deterrent to patient compliance, especially subcutaneous heparin injections, which occur up to three times in 24 hours and often cause more pain at the site of injection than low-molecular-weight heparin. Nurses felt that transitioning patients from receiving subcutaneous heparin injections to receiving low-molecular-weight heparin could be a main driver to reduce patient refusals.

Skills

Nurses felt inadequately equipped to handle patient refusals. Many said that patient refusal of treatments was never discussed in nursing school. As a result, when patients refused treatments, the nurses did not know how to handle the situation. They felt that they lacked the tools and techniques to persuade the patient to comply.

Beliefs about Capabilities

Nurses did not know their own patient refusal rate or benchmarks of an acceptable refusal rate in contrast to one that is too high. Without this feedback, they were unable to assess their own behavior or performance related to providing VTE chemoprophylaxis.

Nurses play a critical role in providing VTE chemoprophylaxis to patients throughout their hospitalization. This study provided a unique opportunity to perform an in-depth, qualitative analysis of the barriers nurses face in providing patients with VTE chemoprophylaxis as part of their daily work caring for patients. We discovered several nursing-related barriers to the provision of VTE chemoprophylaxis, including lack of knowledge, resources, skill, and misconceptions of their capability to provide VTE chemoprophylaxis. We used a bottom-up approach by incorporating the voices of unit nurses, clinical coordinators, and nurse managers to understand potential barriers. Our findings brought to light the challenge of delivering standardized care in an area of care that is generally agreed upon, yet not fully followed. Some nurses display greater proficiency than others at communicating with patients who do not understand their risk for VTE and need for chemoprophylaxis. Furthermore, there is a pronounced misconception around the delivery of VTE chemoprophylaxis. Nurses have the inaccurate belief that even if ordered, chemoprophylaxis is not required. This misconception was widespread among nurses taking care of both medical and surgical patients. These factors appear to be modifiable targets for quality improvement and highlight the need for a skills-based education during the new hire onboarding process, as well as ongoing reeducation to ensure nursing staff have the skills to appropriately provide best-practice care for VTE chemoprophylaxis. Nurses felt ownership of the results of the qualitative evaluation because they were included in every aspect from the beginning.²⁷ This sense of ownership will support future quality improvement efforts to develop a skills-based intervention to improve the provision of VTE chemoprophylaxis.^18,27

This study has certain limitations. First, it was a qualitative study assessing nursing-related barriers to providing VTE chemoprophylaxis at a single institution, and the results cannot be generalized broadly. However, the techniques and results are transferable to other hospital settings and other clinical care situations. Thus, we believe that other institutions can utilize our methods and that similar lessons can be learned and applied. Furthermore, the validity of our study is bolstered by concordance between the results of this study and those of other studies conducted on the topic of provision of VTE prophylaxis by nurses.^13-15,21 Other studies utilized observations and surveys to determine potential nurse-related barriers to the provision of VTE prophylaxis, such as lack of knowledge and the belief that the need for prophylaxis can be determined based on whether or not the patient is ambulating;^13,14 however, by utilizing focus group interviews, we allowed nurses to speak in their own voices about their experiences with VTE prophylaxis, and we were able to delve deeper and identify additional barriers that emerged from discussions with nurses, such as the lack of skill and misconceptions of capability.^28,29 Second, the study focused solely on nurses. Additional initiatives are underway to assess the roles of resident physicians, attending physicians, and patients in the provision of VTE prophylaxis.

Nursing-related barriers to the provision of VTE chemoprophylaxis include a lack of knowledge, resources, skills, and misconceptions of the consequences of missed elements of VTE prophylaxis. Future initiatives will focus on equipping nurses to have meaningful conversations with patients and engaging patients in their care through development of a multifaceted bundle of interventions. Furthermore, similar methods of qualitative inquiry will be used to identify the role of resident and attending physicians and patients in the provision of VTE chemoprophylaxis.

Acknowledgments

The authors thank Sonali Oberoi, Joanne Prinz, Nancy Tomaska, and Kate Paredes, as well as all the nurses who participated in focus group interviews for this study and the nurse managers and clinical coordinators who helped to schedule the focus group interviews.

Disclosures

The authors declare that they have no competing interests.

Funding

This study was funded by the Surgical Outcomes and Quality Improvement Center at Northwestern University.