Patrick W. Brady, MD, MSc

Alarm fatigue is a patient safety hazard in hospitals¹ that occurs when exposure to high rates of alarms leads clinicians to ignore or delay their responses to the alarms.^2,3 To date, most studies of physiologic monitor alarms in hospitalized children have used data from single institutions and often only a few units within each institution.⁴ These limited studies have found that alarms in pediatric units are rarely actionable.² They have also shown that physiologic monitor alarms occur frequently in children’s hospitals and that alarm rates can vary widely within a single institution,⁵ but the extent of variation between children’s hospitals is unknown. In this study, we aimed to describe and compare physiologic monitor alarm characteristics and the proportion of patients monitored in the inpatient units of 5 children’s hospitals.

We performed a cross-sectional study using a point-prevalence design of physiologic monitor alarms and monitoring during a 24-hour period at 5 large, freestanding tertiary-care children’s hospitals. At the time of the study, each hospital had an alarm management committee in place and was working to address alarm fatigue. Each hospital’s institutional review board reviewed and approved the study.

We collected 24 consecutive hours of data from the inpatient units of each hospital between March 24, 2015, and May 1, 2015. Each hospital selected the data collection date within that window based on the availability of staff to perform data collection.⁶ We excluded emergency departments, procedural areas, and inpatient psychiatry and rehabilitation units. By using existing central alarm-collection software that interfaced with bedside physiologic monitors, we collected data on audible alarms generated for apnea, arrhythmia, low and high oxygen saturation, heart rate, respiratory rate, blood pressure, and exhaled carbon dioxide. Bedside alarm systems and alarm collection software differed between centers; therefore, alarm types that were not consistently collected at every institution (eg, alarms for electrode and device malfunction, ventilators, intracranial and central venous pressure monitors, and temperatures probes) were excluded. To estimate alarm rates and to account for fluctuations in hospital census throughout the day,⁷ we collected census (to calculate the number of alarms per patient day) and the number of monitored patients (to calculate the number of alarms per monitored-patient day, including only monitored patients in the denominator) on each unit at 3 time points, 8 hours apart. Patients were considered continuously monitored if they had presence of a waveform and data for pulse oximetry, respiratory rate, and/or heart rate at the time of data collection. We then determined the rate of alarms by unit type—medical-surgical unit (MSU), neonatal intensive care unit (NICU), or pediatric intensive care unit (PICU)—and the alarm types. Based on prior literature demonstrating up to 95% of alarms contributed by a minority of patients on a single unit,⁸ we also calculated the percentage of alarms contributed by beds in the highest quartile of alarms. We also assessed the percentage of patients monitored by unit type. The Supplementary Appendix shows the alarm parameter thresholds in use at the time of the study.

Physiologic monitor alarm rates and the proportion of patients monitored varied widely between unit types and among the tertiary-care children’s hospitals in our study. We found that among MSUs, the hospital with the lowest proportion of beds monitored had the highest alarm rate, with over triple the rate seen at the hospital with the lowest alarm rate. Regardless of unit type, a small subgroup of patients at each hospital contributed a disproportionate share of alarms. These findings are concerning because of the patient morbidity and mortality associated with alarm fatigue¹ and the studies suggesting that higher alarm rates may lead to delays in response to potentially critical alarms.²

We previously described alarm rates at a single children’s hospital and found that alarm rates were high both in and outside of the ICU areas.⁵ This study supports those findings and goes further to show that alarm rates on some MSUs approached rates seen in the ICU areas at other centers.⁴ However, our results should be considered in the context of several limitations. First, the 5 study hospitals utilized different bedside monitors, equipment, and software to collect alarm data. It is possible that this impacted how alarms were counted, though there were no technical specifications to suggest that results should have been biased in a specific way. Second, our data did not reflect alarm validity (ie, whether an alarm accurately reflected the physiologic state of the patient) or factors outside of the number of patients monitored—such as practices around ICU admission and transfer as well as monitor practices such as lead changes, the type of leads employed, and the degree to which alarm parameter thresholds could be customized, which may have also affected alarm rates. Finally, we excluded alarm types that were not consistently collected at all hospitals. We were also unable to capture alarms from other alarm-generating devices, including ventilators and infusion pumps, which have also been identified as sources of alarm-related safety issues in hospitals.^9-11 This suggests that the alarm rates reported here underestimate the total number of audible alarms experienced by staff and by hospitalized patients and families.

While our data collection was limited in scope, the striking differences in alarm rates between hospitals and between similar units in the same hospitals suggest that unit- and hospital-level factors—including default alarm parameter threshold settings, types of monitors used, and monitoring practices such as the degree to which alarm parameters are customized to the patient’s physiologic state—likely contribute to the variability. It is also important to note that while there were clear outlier hospitals, no single hospital had the lowest alarm rate across all unit types. And while we found that a small number of patients contributed disproportionately to alarms, monitoring fewer patients overall was not consistently associated with lower alarm rates. While it is difficult to draw conclusions based on a limited study, these findings suggest that solutions to meaningfully lower alarm rates may be multifaceted. Standardization of care in multiple areas of medicine has shown the potential to decrease unnecessary utilization of testing and therapies while maintaining good patient outcomes.^12-15 Our findings suggest that the concept of positive deviance,¹⁶ by which some organizations produce better outcomes than others despite similar limitations, may help identify successful alarm reduction strategies for further testing. Larger quantitative studies of alarm rates and ethnographic or qualitative studies of monitoring practices may reveal practices and policies that are associated with lower alarm rates with similar or improved monitoring outcomes.

We found wide variability in physiologic monitor alarm rates and the proportion of patients monitored across 5 children’s hospitals. Because alarm fatigue remains a pressing patient safety concern, further study of the features of high-performing (low-alarm) hospital systems may help identify barriers and facilitators of safe, effective monitoring and develop targeted interventions to reduce alarms.

The authors thank Melinda Egan, Matt MacMurchy, and Shannon Stemler for their assistance with data collection.

Disclosure

Dr. Bonafide is supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number K23HL116427. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Agency for Healthcare Research and Quality. There was no external funding obtained for this study. The authors have no conflicts of interest to disclose.

Alarm fatigue is a patient safety hazard in hospitals¹ that occurs when exposure to high rates of alarms leads clinicians to ignore or delay their responses to the alarms.^2,3 To date, most studies of physiologic monitor alarms in hospitalized children have used data from single institutions and often only a few units within each institution.⁴ These limited studies have found that alarms in pediatric units are rarely actionable.² They have also shown that physiologic monitor alarms occur frequently in children’s hospitals and that alarm rates can vary widely within a single institution,⁵ but the extent of variation between children’s hospitals is unknown. In this study, we aimed to describe and compare physiologic monitor alarm characteristics and the proportion of patients monitored in the inpatient units of 5 children’s hospitals.

We performed a cross-sectional study using a point-prevalence design of physiologic monitor alarms and monitoring during a 24-hour period at 5 large, freestanding tertiary-care children’s hospitals. At the time of the study, each hospital had an alarm management committee in place and was working to address alarm fatigue. Each hospital’s institutional review board reviewed and approved the study.

We collected 24 consecutive hours of data from the inpatient units of each hospital between March 24, 2015, and May 1, 2015. Each hospital selected the data collection date within that window based on the availability of staff to perform data collection.⁶ We excluded emergency departments, procedural areas, and inpatient psychiatry and rehabilitation units. By using existing central alarm-collection software that interfaced with bedside physiologic monitors, we collected data on audible alarms generated for apnea, arrhythmia, low and high oxygen saturation, heart rate, respiratory rate, blood pressure, and exhaled carbon dioxide. Bedside alarm systems and alarm collection software differed between centers; therefore, alarm types that were not consistently collected at every institution (eg, alarms for electrode and device malfunction, ventilators, intracranial and central venous pressure monitors, and temperatures probes) were excluded. To estimate alarm rates and to account for fluctuations in hospital census throughout the day,⁷ we collected census (to calculate the number of alarms per patient day) and the number of monitored patients (to calculate the number of alarms per monitored-patient day, including only monitored patients in the denominator) on each unit at 3 time points, 8 hours apart. Patients were considered continuously monitored if they had presence of a waveform and data for pulse oximetry, respiratory rate, and/or heart rate at the time of data collection. We then determined the rate of alarms by unit type—medical-surgical unit (MSU), neonatal intensive care unit (NICU), or pediatric intensive care unit (PICU)—and the alarm types. Based on prior literature demonstrating up to 95% of alarms contributed by a minority of patients on a single unit,⁸ we also calculated the percentage of alarms contributed by beds in the highest quartile of alarms. We also assessed the percentage of patients monitored by unit type. The Supplementary Appendix shows the alarm parameter thresholds in use at the time of the study.

Physiologic monitor alarm rates and the proportion of patients monitored varied widely between unit types and among the tertiary-care children’s hospitals in our study. We found that among MSUs, the hospital with the lowest proportion of beds monitored had the highest alarm rate, with over triple the rate seen at the hospital with the lowest alarm rate. Regardless of unit type, a small subgroup of patients at each hospital contributed a disproportionate share of alarms. These findings are concerning because of the patient morbidity and mortality associated with alarm fatigue¹ and the studies suggesting that higher alarm rates may lead to delays in response to potentially critical alarms.²

We previously described alarm rates at a single children’s hospital and found that alarm rates were high both in and outside of the ICU areas.⁵ This study supports those findings and goes further to show that alarm rates on some MSUs approached rates seen in the ICU areas at other centers.⁴ However, our results should be considered in the context of several limitations. First, the 5 study hospitals utilized different bedside monitors, equipment, and software to collect alarm data. It is possible that this impacted how alarms were counted, though there were no technical specifications to suggest that results should have been biased in a specific way. Second, our data did not reflect alarm validity (ie, whether an alarm accurately reflected the physiologic state of the patient) or factors outside of the number of patients monitored—such as practices around ICU admission and transfer as well as monitor practices such as lead changes, the type of leads employed, and the degree to which alarm parameter thresholds could be customized, which may have also affected alarm rates. Finally, we excluded alarm types that were not consistently collected at all hospitals. We were also unable to capture alarms from other alarm-generating devices, including ventilators and infusion pumps, which have also been identified as sources of alarm-related safety issues in hospitals.^9-11 This suggests that the alarm rates reported here underestimate the total number of audible alarms experienced by staff and by hospitalized patients and families.

While our data collection was limited in scope, the striking differences in alarm rates between hospitals and between similar units in the same hospitals suggest that unit- and hospital-level factors—including default alarm parameter threshold settings, types of monitors used, and monitoring practices such as the degree to which alarm parameters are customized to the patient’s physiologic state—likely contribute to the variability. It is also important to note that while there were clear outlier hospitals, no single hospital had the lowest alarm rate across all unit types. And while we found that a small number of patients contributed disproportionately to alarms, monitoring fewer patients overall was not consistently associated with lower alarm rates. While it is difficult to draw conclusions based on a limited study, these findings suggest that solutions to meaningfully lower alarm rates may be multifaceted. Standardization of care in multiple areas of medicine has shown the potential to decrease unnecessary utilization of testing and therapies while maintaining good patient outcomes.^12-15 Our findings suggest that the concept of positive deviance,¹⁶ by which some organizations produce better outcomes than others despite similar limitations, may help identify successful alarm reduction strategies for further testing. Larger quantitative studies of alarm rates and ethnographic or qualitative studies of monitoring practices may reveal practices and policies that are associated with lower alarm rates with similar or improved monitoring outcomes.

We found wide variability in physiologic monitor alarm rates and the proportion of patients monitored across 5 children’s hospitals. Because alarm fatigue remains a pressing patient safety concern, further study of the features of high-performing (low-alarm) hospital systems may help identify barriers and facilitators of safe, effective monitoring and develop targeted interventions to reduce alarms.

The authors thank Melinda Egan, Matt MacMurchy, and Shannon Stemler for their assistance with data collection.

Disclosure

Dr. Bonafide is supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number K23HL116427. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Agency for Healthcare Research and Quality. There was no external funding obtained for this study. The authors have no conflicts of interest to disclose.

Alarm fatigue is a patient safety hazard in hospitals¹ that occurs when exposure to high rates of alarms leads clinicians to ignore or delay their responses to the alarms.^2,3 To date, most studies of physiologic monitor alarms in hospitalized children have used data from single institutions and often only a few units within each institution.⁴ These limited studies have found that alarms in pediatric units are rarely actionable.² They have also shown that physiologic monitor alarms occur frequently in children’s hospitals and that alarm rates can vary widely within a single institution,⁵ but the extent of variation between children’s hospitals is unknown. In this study, we aimed to describe and compare physiologic monitor alarm characteristics and the proportion of patients monitored in the inpatient units of 5 children’s hospitals.

We performed a cross-sectional study using a point-prevalence design of physiologic monitor alarms and monitoring during a 24-hour period at 5 large, freestanding tertiary-care children’s hospitals. At the time of the study, each hospital had an alarm management committee in place and was working to address alarm fatigue. Each hospital’s institutional review board reviewed and approved the study.

We collected 24 consecutive hours of data from the inpatient units of each hospital between March 24, 2015, and May 1, 2015. Each hospital selected the data collection date within that window based on the availability of staff to perform data collection.⁶ We excluded emergency departments, procedural areas, and inpatient psychiatry and rehabilitation units. By using existing central alarm-collection software that interfaced with bedside physiologic monitors, we collected data on audible alarms generated for apnea, arrhythmia, low and high oxygen saturation, heart rate, respiratory rate, blood pressure, and exhaled carbon dioxide. Bedside alarm systems and alarm collection software differed between centers; therefore, alarm types that were not consistently collected at every institution (eg, alarms for electrode and device malfunction, ventilators, intracranial and central venous pressure monitors, and temperatures probes) were excluded. To estimate alarm rates and to account for fluctuations in hospital census throughout the day,⁷ we collected census (to calculate the number of alarms per patient day) and the number of monitored patients (to calculate the number of alarms per monitored-patient day, including only monitored patients in the denominator) on each unit at 3 time points, 8 hours apart. Patients were considered continuously monitored if they had presence of a waveform and data for pulse oximetry, respiratory rate, and/or heart rate at the time of data collection. We then determined the rate of alarms by unit type—medical-surgical unit (MSU), neonatal intensive care unit (NICU), or pediatric intensive care unit (PICU)—and the alarm types. Based on prior literature demonstrating up to 95% of alarms contributed by a minority of patients on a single unit,⁸ we also calculated the percentage of alarms contributed by beds in the highest quartile of alarms. We also assessed the percentage of patients monitored by unit type. The Supplementary Appendix shows the alarm parameter thresholds in use at the time of the study.

Physiologic monitor alarm rates and the proportion of patients monitored varied widely between unit types and among the tertiary-care children’s hospitals in our study. We found that among MSUs, the hospital with the lowest proportion of beds monitored had the highest alarm rate, with over triple the rate seen at the hospital with the lowest alarm rate. Regardless of unit type, a small subgroup of patients at each hospital contributed a disproportionate share of alarms. These findings are concerning because of the patient morbidity and mortality associated with alarm fatigue¹ and the studies suggesting that higher alarm rates may lead to delays in response to potentially critical alarms.²

We previously described alarm rates at a single children’s hospital and found that alarm rates were high both in and outside of the ICU areas.⁵ This study supports those findings and goes further to show that alarm rates on some MSUs approached rates seen in the ICU areas at other centers.⁴ However, our results should be considered in the context of several limitations. First, the 5 study hospitals utilized different bedside monitors, equipment, and software to collect alarm data. It is possible that this impacted how alarms were counted, though there were no technical specifications to suggest that results should have been biased in a specific way. Second, our data did not reflect alarm validity (ie, whether an alarm accurately reflected the physiologic state of the patient) or factors outside of the number of patients monitored—such as practices around ICU admission and transfer as well as monitor practices such as lead changes, the type of leads employed, and the degree to which alarm parameter thresholds could be customized, which may have also affected alarm rates. Finally, we excluded alarm types that were not consistently collected at all hospitals. We were also unable to capture alarms from other alarm-generating devices, including ventilators and infusion pumps, which have also been identified as sources of alarm-related safety issues in hospitals.^9-11 This suggests that the alarm rates reported here underestimate the total number of audible alarms experienced by staff and by hospitalized patients and families.

While our data collection was limited in scope, the striking differences in alarm rates between hospitals and between similar units in the same hospitals suggest that unit- and hospital-level factors—including default alarm parameter threshold settings, types of monitors used, and monitoring practices such as the degree to which alarm parameters are customized to the patient’s physiologic state—likely contribute to the variability. It is also important to note that while there were clear outlier hospitals, no single hospital had the lowest alarm rate across all unit types. And while we found that a small number of patients contributed disproportionately to alarms, monitoring fewer patients overall was not consistently associated with lower alarm rates. While it is difficult to draw conclusions based on a limited study, these findings suggest that solutions to meaningfully lower alarm rates may be multifaceted. Standardization of care in multiple areas of medicine has shown the potential to decrease unnecessary utilization of testing and therapies while maintaining good patient outcomes.^12-15 Our findings suggest that the concept of positive deviance,¹⁶ by which some organizations produce better outcomes than others despite similar limitations, may help identify successful alarm reduction strategies for further testing. Larger quantitative studies of alarm rates and ethnographic or qualitative studies of monitoring practices may reveal practices and policies that are associated with lower alarm rates with similar or improved monitoring outcomes.

We found wide variability in physiologic monitor alarm rates and the proportion of patients monitored across 5 children’s hospitals. Because alarm fatigue remains a pressing patient safety concern, further study of the features of high-performing (low-alarm) hospital systems may help identify barriers and facilitators of safe, effective monitoring and develop targeted interventions to reduce alarms.

The authors thank Melinda Egan, Matt MacMurchy, and Shannon Stemler for their assistance with data collection.

Disclosure

Dr. Bonafide is supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number K23HL116427. Dr. Brady is supported by the Agency for Healthcare Research and Quality under Award Number K08HS23827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Agency for Healthcare Research and Quality. There was no external funding obtained for this study. The authors have no conflicts of interest to disclose.

Physicians, researchers, and policymakers aspire to improve the value of healthcare, with reduced overall costs of care and improved outcomes. An important component of increasing healthcare costs in the United States is the rising cost of prescription medications, accounting for an estimated 17% of all spending in healthcare services.¹ One potentially modifiable driver of low-value prescribing is poor awareness of medication cost.² While displaying price to the ordering physician has reduced laboratory order volume and associated testing costs,^3,4 applying cost transparency to medication ordering has produced variable results, perhaps reflecting conceptual differences in decision making regarding diagnosis and treatment.^4-6

In this issue of the Journal of Hospital Medicine, Conway et al.⁷ performed a retrospective analysis applying interrupted times series models to measure the impact of passive cost display on the ordering frequency of 9 high-cost intravenous (IV) or inhaled medications that were identified as likely overused. For 7 of the IV medications, lower-cost oral alternatives were available; 2 study medications had no clear therapeutic alternatives. It was expected that lower-cost oral alternatives would have a concomitant increase in ordering rate as the order rate of the study medications decreased (eg, oral linezolid use would increase as IV linezolid use decreased). Order rate was the primary outcome, reported each week as treatment orders per 10,000 patient days, and was compared for both the pre- and postimplementation time periods. The particular methodology of segmented regressions allowed the research team to control for preintervention trends in medication ordering, as well as to analyze both immediate and delayed effects of the cost-display intervention. The research team framed the cost display as a passive approach. The intervention displayed average wholesale cost data and lower-cost oral alternatives on the ordering screen, which did not significantly reduce the ordering rate. Over the course of the study, outside influences led to 2 more active approaches to higher-cost medications, and Conway et al. wisely measured their effect as well. Specifically, the IV pantoprazole ordering rate decreased after restrictions secondary to a national medication shortage, and the oral voriconazole ordering rate decreased following an oncology order set change from oral voriconazole to oral posaconazole. These ordering-rate decreases were not temporally related to the implementation of the cost display intervention.

It is important to note several limitations of this study, some of which the authors discuss in the manuscript. Because 2 of the medications studied (eculizumab and calcitonin) do not have direct therapeutic alternatives, it is not surprising that price display alone would have no effect. The ordering providers who received this cost information had a more complex decision to make than they would in a scenario with a lower-cost alternative, essentially requiring them to ask “Does this patient need this class of medications at all?” rather than simply, “Is a lower-cost alternative appropriate?” Similarly, choosing medication alternatives that would require different routes of administration (ie, IV and oral) may have limited the effectiveness of a price intervention, given that factors such as illness severity also may influence the decision between IV and oral agents. Thus, the lack of an effect for the price display intervention for these specific medications may not be generalizable to all other medication decisions. Additionally, this manuscript offers limited data on the context in which the intervention was implemented and what adaptations, if any, were made based on early findings. The results may have varied greatly based on the visual design and how the cost display was presented within the electronic medical record. The wider organizational context may also have affected the intervention’s impact. A cost-display intervention appearing in isolation could understandably have a different impact, compared with an intervention within the context of a broader cost/value curriculum directed at house staff and faculty.

In summary, Conway et al. found that just displaying cost data did little to change prescribing patterns, but that more active approaches were quite efficacious. So where does this leave value-minded hospitalists looking to reduce overuse? Relatedly, what are the next steps for research and improvement science? We think there are 3 key strategic areas on which to focus. First, behavioral economics offers a critically important middle ground between the passive approaches studied here and more heavy-handed approaches that may limit provider autonomy, such as restricting drug use at the formulary.⁸ An improved choice architecture that presents the preferred higher-value option as the default selection may result in improved adoption of the high-value choice while also preserving provider autonomy and expertise required when clinical circumstances make the higher-cost drug the better choice.^9,10 The second consideration is to minimize ethical tensions between cost displays that discourage use and a provider’s belief that a treatment is beneficial. Using available ethical frameworks for high-value care that engage both patient and societal concerns may help us choose and design interventions with more successful outcomes.¹¹ Finally, research has shown that providers have poor knowledge of both cost and the relative benefits and harms of treatments and testing.¹² Thus, the third opportunity for improvement is to provide appropriate clinical information (ie, relative therapeutic equivalency or adverse effects in alternative therapies) to support decision making at the point of order entry. Encouraging data already exists regarding how drug facts boxes can help patients understand benefits and side effects.¹³ A similar approach may aid physicians and may prove an easier task than improving patient understanding, given physicians’ substantial existing knowledge. These strategies may help guide providers to make a more informed value determination and obviate some ethical concerns related to clinical decisions based on cost alone. Despite their negative results, Conway et al.⁷ provided additional evidence that influencing complex decision making is not easy. However, we believe that continuing research into the factors that lead to successful value interventions has incredible potential for supporting high-value decision making in the future.

Disclosure 

Nothing to report.

Physicians, researchers, and policymakers aspire to improve the value of healthcare, with reduced overall costs of care and improved outcomes. An important component of increasing healthcare costs in the United States is the rising cost of prescription medications, accounting for an estimated 17% of all spending in healthcare services.¹ One potentially modifiable driver of low-value prescribing is poor awareness of medication cost.² While displaying price to the ordering physician has reduced laboratory order volume and associated testing costs,^3,4 applying cost transparency to medication ordering has produced variable results, perhaps reflecting conceptual differences in decision making regarding diagnosis and treatment.^4-6

In this issue of the Journal of Hospital Medicine, Conway et al.⁷ performed a retrospective analysis applying interrupted times series models to measure the impact of passive cost display on the ordering frequency of 9 high-cost intravenous (IV) or inhaled medications that were identified as likely overused. For 7 of the IV medications, lower-cost oral alternatives were available; 2 study medications had no clear therapeutic alternatives. It was expected that lower-cost oral alternatives would have a concomitant increase in ordering rate as the order rate of the study medications decreased (eg, oral linezolid use would increase as IV linezolid use decreased). Order rate was the primary outcome, reported each week as treatment orders per 10,000 patient days, and was compared for both the pre- and postimplementation time periods. The particular methodology of segmented regressions allowed the research team to control for preintervention trends in medication ordering, as well as to analyze both immediate and delayed effects of the cost-display intervention. The research team framed the cost display as a passive approach. The intervention displayed average wholesale cost data and lower-cost oral alternatives on the ordering screen, which did not significantly reduce the ordering rate. Over the course of the study, outside influences led to 2 more active approaches to higher-cost medications, and Conway et al. wisely measured their effect as well. Specifically, the IV pantoprazole ordering rate decreased after restrictions secondary to a national medication shortage, and the oral voriconazole ordering rate decreased following an oncology order set change from oral voriconazole to oral posaconazole. These ordering-rate decreases were not temporally related to the implementation of the cost display intervention.

It is important to note several limitations of this study, some of which the authors discuss in the manuscript. Because 2 of the medications studied (eculizumab and calcitonin) do not have direct therapeutic alternatives, it is not surprising that price display alone would have no effect. The ordering providers who received this cost information had a more complex decision to make than they would in a scenario with a lower-cost alternative, essentially requiring them to ask “Does this patient need this class of medications at all?” rather than simply, “Is a lower-cost alternative appropriate?” Similarly, choosing medication alternatives that would require different routes of administration (ie, IV and oral) may have limited the effectiveness of a price intervention, given that factors such as illness severity also may influence the decision between IV and oral agents. Thus, the lack of an effect for the price display intervention for these specific medications may not be generalizable to all other medication decisions. Additionally, this manuscript offers limited data on the context in which the intervention was implemented and what adaptations, if any, were made based on early findings. The results may have varied greatly based on the visual design and how the cost display was presented within the electronic medical record. The wider organizational context may also have affected the intervention’s impact. A cost-display intervention appearing in isolation could understandably have a different impact, compared with an intervention within the context of a broader cost/value curriculum directed at house staff and faculty.

In summary, Conway et al. found that just displaying cost data did little to change prescribing patterns, but that more active approaches were quite efficacious. So where does this leave value-minded hospitalists looking to reduce overuse? Relatedly, what are the next steps for research and improvement science? We think there are 3 key strategic areas on which to focus. First, behavioral economics offers a critically important middle ground between the passive approaches studied here and more heavy-handed approaches that may limit provider autonomy, such as restricting drug use at the formulary.⁸ An improved choice architecture that presents the preferred higher-value option as the default selection may result in improved adoption of the high-value choice while also preserving provider autonomy and expertise required when clinical circumstances make the higher-cost drug the better choice.^9,10 The second consideration is to minimize ethical tensions between cost displays that discourage use and a provider’s belief that a treatment is beneficial. Using available ethical frameworks for high-value care that engage both patient and societal concerns may help us choose and design interventions with more successful outcomes.¹¹ Finally, research has shown that providers have poor knowledge of both cost and the relative benefits and harms of treatments and testing.¹² Thus, the third opportunity for improvement is to provide appropriate clinical information (ie, relative therapeutic equivalency or adverse effects in alternative therapies) to support decision making at the point of order entry. Encouraging data already exists regarding how drug facts boxes can help patients understand benefits and side effects.¹³ A similar approach may aid physicians and may prove an easier task than improving patient understanding, given physicians’ substantial existing knowledge. These strategies may help guide providers to make a more informed value determination and obviate some ethical concerns related to clinical decisions based on cost alone. Despite their negative results, Conway et al.⁷ provided additional evidence that influencing complex decision making is not easy. However, we believe that continuing research into the factors that lead to successful value interventions has incredible potential for supporting high-value decision making in the future.

Disclosure 

Nothing to report.

Physicians, researchers, and policymakers aspire to improve the value of healthcare, with reduced overall costs of care and improved outcomes. An important component of increasing healthcare costs in the United States is the rising cost of prescription medications, accounting for an estimated 17% of all spending in healthcare services.¹ One potentially modifiable driver of low-value prescribing is poor awareness of medication cost.² While displaying price to the ordering physician has reduced laboratory order volume and associated testing costs,^3,4 applying cost transparency to medication ordering has produced variable results, perhaps reflecting conceptual differences in decision making regarding diagnosis and treatment.^4-6

In this issue of the Journal of Hospital Medicine, Conway et al.⁷ performed a retrospective analysis applying interrupted times series models to measure the impact of passive cost display on the ordering frequency of 9 high-cost intravenous (IV) or inhaled medications that were identified as likely overused. For 7 of the IV medications, lower-cost oral alternatives were available; 2 study medications had no clear therapeutic alternatives. It was expected that lower-cost oral alternatives would have a concomitant increase in ordering rate as the order rate of the study medications decreased (eg, oral linezolid use would increase as IV linezolid use decreased). Order rate was the primary outcome, reported each week as treatment orders per 10,000 patient days, and was compared for both the pre- and postimplementation time periods. The particular methodology of segmented regressions allowed the research team to control for preintervention trends in medication ordering, as well as to analyze both immediate and delayed effects of the cost-display intervention. The research team framed the cost display as a passive approach. The intervention displayed average wholesale cost data and lower-cost oral alternatives on the ordering screen, which did not significantly reduce the ordering rate. Over the course of the study, outside influences led to 2 more active approaches to higher-cost medications, and Conway et al. wisely measured their effect as well. Specifically, the IV pantoprazole ordering rate decreased after restrictions secondary to a national medication shortage, and the oral voriconazole ordering rate decreased following an oncology order set change from oral voriconazole to oral posaconazole. These ordering-rate decreases were not temporally related to the implementation of the cost display intervention.

It is important to note several limitations of this study, some of which the authors discuss in the manuscript. Because 2 of the medications studied (eculizumab and calcitonin) do not have direct therapeutic alternatives, it is not surprising that price display alone would have no effect. The ordering providers who received this cost information had a more complex decision to make than they would in a scenario with a lower-cost alternative, essentially requiring them to ask “Does this patient need this class of medications at all?” rather than simply, “Is a lower-cost alternative appropriate?” Similarly, choosing medication alternatives that would require different routes of administration (ie, IV and oral) may have limited the effectiveness of a price intervention, given that factors such as illness severity also may influence the decision between IV and oral agents. Thus, the lack of an effect for the price display intervention for these specific medications may not be generalizable to all other medication decisions. Additionally, this manuscript offers limited data on the context in which the intervention was implemented and what adaptations, if any, were made based on early findings. The results may have varied greatly based on the visual design and how the cost display was presented within the electronic medical record. The wider organizational context may also have affected the intervention’s impact. A cost-display intervention appearing in isolation could understandably have a different impact, compared with an intervention within the context of a broader cost/value curriculum directed at house staff and faculty.

In summary, Conway et al. found that just displaying cost data did little to change prescribing patterns, but that more active approaches were quite efficacious. So where does this leave value-minded hospitalists looking to reduce overuse? Relatedly, what are the next steps for research and improvement science? We think there are 3 key strategic areas on which to focus. First, behavioral economics offers a critically important middle ground between the passive approaches studied here and more heavy-handed approaches that may limit provider autonomy, such as restricting drug use at the formulary.⁸ An improved choice architecture that presents the preferred higher-value option as the default selection may result in improved adoption of the high-value choice while also preserving provider autonomy and expertise required when clinical circumstances make the higher-cost drug the better choice.^9,10 The second consideration is to minimize ethical tensions between cost displays that discourage use and a provider’s belief that a treatment is beneficial. Using available ethical frameworks for high-value care that engage both patient and societal concerns may help us choose and design interventions with more successful outcomes.¹¹ Finally, research has shown that providers have poor knowledge of both cost and the relative benefits and harms of treatments and testing.¹² Thus, the third opportunity for improvement is to provide appropriate clinical information (ie, relative therapeutic equivalency or adverse effects in alternative therapies) to support decision making at the point of order entry. Encouraging data already exists regarding how drug facts boxes can help patients understand benefits and side effects.¹³ A similar approach may aid physicians and may prove an easier task than improving patient understanding, given physicians’ substantial existing knowledge. These strategies may help guide providers to make a more informed value determination and obviate some ethical concerns related to clinical decisions based on cost alone. Despite their negative results, Conway et al.⁷ provided additional evidence that influencing complex decision making is not easy. However, we believe that continuing research into the factors that lead to successful value interventions has incredible potential for supporting high-value decision making in the future.

Disclosure 

Nothing to report.

The transition from hospital to home can be overwhelming for caregivers.¹ Stress of hospitalization coupled with the expectation of families to execute postdischarge care plans make understandable discharge communication critical. Communication failures, inadequate education, absence of caregiver confidence, and lack of clarity regarding care plans may prohibit smooth transitions and lead to adverse postdischarge outcomes.^2-4

Health literacy plays a pivotal role in caregivers’ capacity to navigate the healthcare system, comprehend, and execute care plans. An estimated 90 million Americans have limited health literacy that may negatively impact the provision of safe and quality care^5,6 and be a risk factor for poor outcomes, including increased emergency department (ED) utilization and readmission rates.^7-9 Readability strongly influences the effectiveness of written materials.¹⁰ However, written medical information for patients and families are frequently between the 10^th and 12^th grade reading levels; more than 75% of all pediatric health information is written at or above 10^th grade reading level.¹¹ Government agencies recommend between a 6^th and 8^th grade reading level, for written material;^5,12,13 written discharge instructions have been identified as an important quality metric for hospital-to-home transitions.^14-16

At our center, we found that discharge instructions were commonly written at high reading levels and often incomplete.¹⁷ Poor discharge instructions may contribute to increased readmission rates and unnecessary ED visits.^9,18 Our global aim targeted improved health-literate written information, including understandability and completeness.

Our specific aim was to increase the percentage of discharge instructions written at or below the 7th grade level for hospital medicine (HM) patients on a community hospital pediatric unit from 13% to 80% in 6 months.

Context

The improvement work took place at a 42-bed inpatient pediatric unit at a community satellite of our large, urban, academic hospital. The unit is staffed by medical providers including attendings, fellows, nurse practitioners (NPs), and senior pediatric residents, and had more than 1000 HM discharges in fiscal year 2016. Children with common general pediatric diagnoses are admitted to this service; postsurgical patients are not admitted primarily to the HM service. In Cincinnati, the neighborhood-level high school drop-out rates are as high as 64%.¹⁹ Discharge instructions are written by medical providers in the electronic health record (EHR). A printed copy is given to families and verbally reviewed by a bedside nurse prior to discharge. Quality improvement (QI) efforts focused on discharge instructions were ignited by a prior review of 200 discharge instructions that showed they were difficult to read (median reading level of 10^th grade), poorly understandable (36% of instructions met the threshold of understandability as measured by the Patient Education Materials Assessment Tool²⁰) and were missing key elements of information.¹⁷

Improvement Team

The improvement team consisted of 4 pediatric hospitalists, 2 NPs, 1 nurse educator with health literacy expertise, 1 pediatric resident, 1 fourth-year medical student, 1 QI consultant, and 2 parents who had first-hand experience on the HM service. The improvement team observed the discharge process, including roles of the provider, nurse and family, outlined a process map, and created a modified failure mode and effect analysis.²¹ Prior to our work, discharge instructions written by providers often occurred as a last step, and the content was created as free text or from nonstandardized templates. Key drivers that informed interventions were determined and revised over time (Figure 1). The study was reviewed by our institutional review board and deemed not human subjects research.

Improvement Activities

Key drivers were identified, and interventions were executed using Plan-Do Study-Act cycles.²² The key drivers thought to be critical for the success of the QI efforts were family engagement; standardization of discharge instructions; medical staff engagement; and audit and feedback of data. The corresponding interventions were as follows:

Family Engagement

Understanding the discharge information families desired. Prior to testing, 10 families admitted to the HM service were asked about the discharge experience. We asked families about information they wanted in written discharge instructions: 1) reasons to call your primary doctor or return to the hospital; 2) when to see your primary doctor for a follow-up visit; 3) the phone number to reach your child’s doctor; 4) more information about why your child was admitted; 5) information about new medications; and 6) what to do to help your child continue to recover at home.

Development of templates. We engaged families throughout the process of creating general and disease-specific discharge templates. After a specific template was created and reviewed by the parents on our team, it was sent to members of the institutional Patient Education Committee, which includes parents and local health literacy experts, to review and critique. Feedback from the reviewers was incorporated into the templates prior to use in the EHR.

Postdischarge phone calls.A convenience sample of families discharged from the satellite campus was called 24 to 48 hours after discharge over a 2-week period in January, 2016. A member of our improvement team solicited feedback from families about the quality of the discharge instructions. Families were asked if discharge instructions were reviewed with them prior to going home, if they were given a copy of the instructions, how they would rate the ability to read and use the information, and if there were additional pieces of information that would have improved the instructions.

Standardization of Instructions

Education. A presentation was created and shared with medical providers; it was re-disseminated monthly to new residents rotating onto the service and to the attendings, fellows, and NPs scheduled for shifts during the month. This education continued for the duration of the study. The presentation included the definition of health literacy, scope of the problem, examples of poorly written discharge instructions, and tips on how to write readable and understandable instructions. Laminated cards that included tips on how to write instructions were also placed on work stations.

While the general template was an important first step, the content relied heavily on free text by providers, which could still lead to instructions written at a high reading level. Thus, disease-specific discharge instruction templates were created with prepopulated information that was written at a reading level at or below 7^th grade level (Figure 2). The diseases were prioritized based on the most common diagnoses on our HM service. Each template included information under each of the subheadings noted in the general template. Twelve disease-specific templates were tested and ultimately embedded in the EHR; the general template remained for use when the discharge diagnosis was not covered by a disease-specific template.

Medical Staff Engagement

Previously described tests of change also aimed to enhance staff engagement. These included frequent e-mails, discussion of the QI efforts at specific team meetings, and the creation of visual cues posted at computer work stations, which prompted staff to begin to work on discharge instructions soon after admission.

Audit and Feedback of Data

Weekly phone calls. One team updated clinicians through a regularly scheduled bi-weekly phone conference. The phone conference was established prior to our work and was designed to relay pertinent information to attendings and NPs who work at the satellite hospital. During the phone conferences, clinicians were notified of current performance on discharge instruction readability and specific tests of change for the week. Additionally, providers gave feedback about the improvement efforts. These updates continued for the first 6 months of the project until sustained improvements were observed.

E-mails. Weekly e-mails were sent to all providers scheduled for clinical time at the satellite campus. The e-mail contained information on current tests of change, a list of discharge instruction templates that were available in the EHR, and the annotated run chart illustrating readability levels over time.

Additionally, individual e-mails were sent to each provider after review of the written discharge instructions for the week. Providers were given information on the number of discharge instructions they personally composed, the percentage of those instructions that were written at or below 7^th grade level, and specific feedback on how their written instructions could be improved. We also encouraged feedback from each provider to better identify barriers to achieving our goal.

Study of the Interventions

Baseline data included a review of all instructions for patients discharged from the satellite campus from the end of April 2015 through mid-September 2015. The time period for testing of interventions during the fall and winter months allowed for rapid cycle learning due to higher patient census and predictability of admissions for specific diagnosis (ie, asthma and bronchiolitis). An automated report was generated from the EHR weekly with specific demographics and identifiers for patient discharged over the past 7 days, including patient age, gender, length of stay, discharge diagnosis, and insurance classification. Data was collected during the intervention period via structured review of the discharge instructions in the EHR by the principal investigator or a trained research coordinator. Discharge instructions for medically cleared mental health patients admitted to hospital medicine while awaiting psychiatric bed availability and patients and parents who were non-English speaking were excluded from review. All other instructions for patients discharged from the HM service at our Liberty Campus were included for review.

Measures

Readability, our primary measure of interest, was calculated using the mean score from the following formulas: Flesch Kincaid Grade Level,²³ Simple Measure of Gobbledygook Index,²⁴ Coleman-Liau Index,²⁵ Gunning-Fog Index,²⁶ and Automated Readability Index²⁷ by means of an online platform (https://readability-score.com).²⁸ This platform was chosen because it incorporated a variety of formulas, was user-friendly, and required minimal data cleaning. Each of the readability formulas have been used to assesses readability of health information given to patients and families.^29,30 The threshold of 7th grade is in alignment with our institutional policy for educational materials and with recommendations from several government agencies.^5,12

Analysis

A statistical process control p-chart was used to analyze our primary measure of readability, dichotomized as percent discharge instructions written at or below 7th grade level. Run charts were used to follow mean reading level of discharge instructions and our process measure of percent of discharge instruction written with a general or disease-specific standardized template. Run chart and control chart rules for identifying special cause were used for midline shifts.³¹

The Table includes the demographic and clinical information of patients included in our analyses. Through sequential interventions, the percentage of discharge instructions written at or below 7th grade readability level increased from a mean of 13% to more than 80% in 3 months (Figure 3). Furthermore, the mean was sustained above 90% for 10 months and at 98% for the last 4 months. The use of 1 of the 13 EHR templates increased from 0% to 96% and was associated with the largest impact on the overall improvements (Supplemental Figure 1). Additionally, the average reading level of the discharge instructions decreased from 10th grade to 6th grade level (Supplemental Figure 2).

Qualitative comments from providers about the discharge instructions included:

“Are these [discharge instructions] available at base??  Great resource for interns.”
“These [discharge] instructions make the [discharge] process so easy!!! Love these...”
“Also feel like they have helped my discharge teaching in the room!”

Qualitative comments from families postdischarge included:
“I thought the instructions were very clear and easy to read. I especially thought that highlighting the important areas really helped.”
“I think this form looks great, and I really like the idea of having your child’s name on it.”

Through sequential Plan-Do Study-Act cycles, we increased the percentage of discharge instructions written at or below 7^th grade reading level from 13% to 98%. Our most impactful intervention was the creation and dissemination of standardized disease-specific discharge instruction templates. Our findings complement evidence in the adult and pediatric literature that the use of standardized, disease-specific discharge instruction templates may improve readability of instructions.^32,33 And, while quality improvement efforts have been employed to improve the discharge process for patients,^34-36 this is the first study in the inpatient setting that, to our knowledge, specifically addresses discharge instructions using quality improvement methods.

Our work targeted the critical intersection between individual health literacy, an individual’s capacity to acquire, interpret, and use health information, and the necessary changes needed within our healthcare system to ensure that appropriately written instructions are given to patients and families.^17,37 Our efforts focused on improving discharge instructions answer the call to consider health literacy a modifiable clinical risk factor.³⁷ Furthermore, we address the 6 aims for quality healthcare delivery: 1) safe, timely, efficient and equitable delivery of care through the creation and dissemination of standardized instructions that are written at the appropriate reading level for families to ease hospital-to-home transitions and streamline the workflow of medical providers; 2) effective education of medical providers on health literacy concepts; and 3) family-centeredness through the involvement of families in our QI efforts. While previous QI efforts to improve hospital-to-home transitions have focused on medication reconciliation, communication with primary care physicians, follow-up appointments, and timely discharges of patients, none have specifically focused on the quality of discharge instructions.^34-36

Most physicians do not receive education about how to write information that is readable and understandable; more than half of providers desired more education in this area.³⁸ Furthermore, pediatric providers may overestimate parental health literacy levels,³⁹ which may contribute to variability in the readability of written health materials. While education alone can contribute to a provider’s ability to create readable instructions, we note the improvement after the introduction of disease templates to demonstrate the importance of workflow-integrated higher reliability interventions to sustain improvements.

Our baseline poor readability rates were due to limited knowledge by frontline providers composing the instructions and a system in which an important element for successful hospital-to-home transitions was not tackled until patients were ready for discharge. Streamlining of the discharge process, including the creation of discharge instructions, may lead to improved efficiency, fewer discrepancies, more effective communication, and an enhanced family experience. Moreover, the success of our improvement work was due to key stakeholders, including parents, being a part of the team and the notable buy-in from providers.

Our work was not without limitations. We excluded non-English speaking families from the study. We were unable to measure reading level of our population directly and instead based our goals on national estimates. Our primary measure was readability, which is only 1 piece contributing to quality discharge instructions. Understandability and actionability are also important considerations; ^17,20,29,40 however, improvements in these areas were limited by our design options within the EHR. Our efforts focused on children with common general pediatric diagnoses, and it is unclear how our interventions would generalize to medically complex patients with more volume of information to communicate at discharge and with uncommon diagnoses that are less readily incorporated into standardized templates. Relatedly, our work occurred at the satellite campus of our tertiary care center and may not represent generalizable material or methods to implement templates at our main campus location or at other hospitals. To begin to better understand this, we have spread to HM patients at our main campus, including medically complex patients with technology dependence and/or neurological impairments. Standardized, disease-specific templates most relevant to this population as well as several patient specific templates, for those with frequent readmissions due to medical complexity, have been created and are actively being tested.

In conclusion, in using interventions targeted at standardization of discharge instructions and timely feedback to staff, we saw rapid, dramatic, and sustained improvement in the readability of discharge instructions. Next steps include adaptation and spread to other patient populations and care teams, collaborations with other centers, and assessing the impact of effectively written discharge instructions on patient outcomes, such as adverse drug events, readmission rates, and family experience.

Disclosure

No external funding was secured for this study. Dr. Brady is supported by a Patient-Centered Outcomes Research Mentored Clinical Investigator Award from the Agency for Healthcare Research and Quality, Award Number K08HS023827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations. The funding organization had no role in the design, preparation, review, or approval of this paper; nor the decision to submit the manuscript for publication. The authors have no financial relationships relevant to this article to disclose.

The transition from hospital to home can be overwhelming for caregivers.¹ Stress of hospitalization coupled with the expectation of families to execute postdischarge care plans make understandable discharge communication critical. Communication failures, inadequate education, absence of caregiver confidence, and lack of clarity regarding care plans may prohibit smooth transitions and lead to adverse postdischarge outcomes.^2-4

Health literacy plays a pivotal role in caregivers’ capacity to navigate the healthcare system, comprehend, and execute care plans. An estimated 90 million Americans have limited health literacy that may negatively impact the provision of safe and quality care^5,6 and be a risk factor for poor outcomes, including increased emergency department (ED) utilization and readmission rates.^7-9 Readability strongly influences the effectiveness of written materials.¹⁰ However, written medical information for patients and families are frequently between the 10^th and 12^th grade reading levels; more than 75% of all pediatric health information is written at or above 10^th grade reading level.¹¹ Government agencies recommend between a 6^th and 8^th grade reading level, for written material;^5,12,13 written discharge instructions have been identified as an important quality metric for hospital-to-home transitions.^14-16

At our center, we found that discharge instructions were commonly written at high reading levels and often incomplete.¹⁷ Poor discharge instructions may contribute to increased readmission rates and unnecessary ED visits.^9,18 Our global aim targeted improved health-literate written information, including understandability and completeness.

Our specific aim was to increase the percentage of discharge instructions written at or below the 7th grade level for hospital medicine (HM) patients on a community hospital pediatric unit from 13% to 80% in 6 months.

Context

The improvement work took place at a 42-bed inpatient pediatric unit at a community satellite of our large, urban, academic hospital. The unit is staffed by medical providers including attendings, fellows, nurse practitioners (NPs), and senior pediatric residents, and had more than 1000 HM discharges in fiscal year 2016. Children with common general pediatric diagnoses are admitted to this service; postsurgical patients are not admitted primarily to the HM service. In Cincinnati, the neighborhood-level high school drop-out rates are as high as 64%.¹⁹ Discharge instructions are written by medical providers in the electronic health record (EHR). A printed copy is given to families and verbally reviewed by a bedside nurse prior to discharge. Quality improvement (QI) efforts focused on discharge instructions were ignited by a prior review of 200 discharge instructions that showed they were difficult to read (median reading level of 10^th grade), poorly understandable (36% of instructions met the threshold of understandability as measured by the Patient Education Materials Assessment Tool²⁰) and were missing key elements of information.¹⁷

Improvement Team

The improvement team consisted of 4 pediatric hospitalists, 2 NPs, 1 nurse educator with health literacy expertise, 1 pediatric resident, 1 fourth-year medical student, 1 QI consultant, and 2 parents who had first-hand experience on the HM service. The improvement team observed the discharge process, including roles of the provider, nurse and family, outlined a process map, and created a modified failure mode and effect analysis.²¹ Prior to our work, discharge instructions written by providers often occurred as a last step, and the content was created as free text or from nonstandardized templates. Key drivers that informed interventions were determined and revised over time (Figure 1). The study was reviewed by our institutional review board and deemed not human subjects research.

Improvement Activities

Key drivers were identified, and interventions were executed using Plan-Do Study-Act cycles.²² The key drivers thought to be critical for the success of the QI efforts were family engagement; standardization of discharge instructions; medical staff engagement; and audit and feedback of data. The corresponding interventions were as follows:

Family Engagement

Understanding the discharge information families desired. Prior to testing, 10 families admitted to the HM service were asked about the discharge experience. We asked families about information they wanted in written discharge instructions: 1) reasons to call your primary doctor or return to the hospital; 2) when to see your primary doctor for a follow-up visit; 3) the phone number to reach your child’s doctor; 4) more information about why your child was admitted; 5) information about new medications; and 6) what to do to help your child continue to recover at home.

Development of templates. We engaged families throughout the process of creating general and disease-specific discharge templates. After a specific template was created and reviewed by the parents on our team, it was sent to members of the institutional Patient Education Committee, which includes parents and local health literacy experts, to review and critique. Feedback from the reviewers was incorporated into the templates prior to use in the EHR.

Postdischarge phone calls.A convenience sample of families discharged from the satellite campus was called 24 to 48 hours after discharge over a 2-week period in January, 2016. A member of our improvement team solicited feedback from families about the quality of the discharge instructions. Families were asked if discharge instructions were reviewed with them prior to going home, if they were given a copy of the instructions, how they would rate the ability to read and use the information, and if there were additional pieces of information that would have improved the instructions.

Standardization of Instructions

Education. A presentation was created and shared with medical providers; it was re-disseminated monthly to new residents rotating onto the service and to the attendings, fellows, and NPs scheduled for shifts during the month. This education continued for the duration of the study. The presentation included the definition of health literacy, scope of the problem, examples of poorly written discharge instructions, and tips on how to write readable and understandable instructions. Laminated cards that included tips on how to write instructions were also placed on work stations.

While the general template was an important first step, the content relied heavily on free text by providers, which could still lead to instructions written at a high reading level. Thus, disease-specific discharge instruction templates were created with prepopulated information that was written at a reading level at or below 7^th grade level (Figure 2). The diseases were prioritized based on the most common diagnoses on our HM service. Each template included information under each of the subheadings noted in the general template. Twelve disease-specific templates were tested and ultimately embedded in the EHR; the general template remained for use when the discharge diagnosis was not covered by a disease-specific template.

Medical Staff Engagement

Previously described tests of change also aimed to enhance staff engagement. These included frequent e-mails, discussion of the QI efforts at specific team meetings, and the creation of visual cues posted at computer work stations, which prompted staff to begin to work on discharge instructions soon after admission.

Audit and Feedback of Data

Weekly phone calls. One team updated clinicians through a regularly scheduled bi-weekly phone conference. The phone conference was established prior to our work and was designed to relay pertinent information to attendings and NPs who work at the satellite hospital. During the phone conferences, clinicians were notified of current performance on discharge instruction readability and specific tests of change for the week. Additionally, providers gave feedback about the improvement efforts. These updates continued for the first 6 months of the project until sustained improvements were observed.

E-mails. Weekly e-mails were sent to all providers scheduled for clinical time at the satellite campus. The e-mail contained information on current tests of change, a list of discharge instruction templates that were available in the EHR, and the annotated run chart illustrating readability levels over time.

Additionally, individual e-mails were sent to each provider after review of the written discharge instructions for the week. Providers were given information on the number of discharge instructions they personally composed, the percentage of those instructions that were written at or below 7^th grade level, and specific feedback on how their written instructions could be improved. We also encouraged feedback from each provider to better identify barriers to achieving our goal.

Study of the Interventions

Baseline data included a review of all instructions for patients discharged from the satellite campus from the end of April 2015 through mid-September 2015. The time period for testing of interventions during the fall and winter months allowed for rapid cycle learning due to higher patient census and predictability of admissions for specific diagnosis (ie, asthma and bronchiolitis). An automated report was generated from the EHR weekly with specific demographics and identifiers for patient discharged over the past 7 days, including patient age, gender, length of stay, discharge diagnosis, and insurance classification. Data was collected during the intervention period via structured review of the discharge instructions in the EHR by the principal investigator or a trained research coordinator. Discharge instructions for medically cleared mental health patients admitted to hospital medicine while awaiting psychiatric bed availability and patients and parents who were non-English speaking were excluded from review. All other instructions for patients discharged from the HM service at our Liberty Campus were included for review.

Measures

Readability, our primary measure of interest, was calculated using the mean score from the following formulas: Flesch Kincaid Grade Level,²³ Simple Measure of Gobbledygook Index,²⁴ Coleman-Liau Index,²⁵ Gunning-Fog Index,²⁶ and Automated Readability Index²⁷ by means of an online platform (https://readability-score.com).²⁸ This platform was chosen because it incorporated a variety of formulas, was user-friendly, and required minimal data cleaning. Each of the readability formulas have been used to assesses readability of health information given to patients and families.^29,30 The threshold of 7th grade is in alignment with our institutional policy for educational materials and with recommendations from several government agencies.^5,12

Analysis

A statistical process control p-chart was used to analyze our primary measure of readability, dichotomized as percent discharge instructions written at or below 7th grade level. Run charts were used to follow mean reading level of discharge instructions and our process measure of percent of discharge instruction written with a general or disease-specific standardized template. Run chart and control chart rules for identifying special cause were used for midline shifts.³¹

The Table includes the demographic and clinical information of patients included in our analyses. Through sequential interventions, the percentage of discharge instructions written at or below 7th grade readability level increased from a mean of 13% to more than 80% in 3 months (Figure 3). Furthermore, the mean was sustained above 90% for 10 months and at 98% for the last 4 months. The use of 1 of the 13 EHR templates increased from 0% to 96% and was associated with the largest impact on the overall improvements (Supplemental Figure 1). Additionally, the average reading level of the discharge instructions decreased from 10th grade to 6th grade level (Supplemental Figure 2).

Qualitative comments from providers about the discharge instructions included:

“Are these [discharge instructions] available at base??  Great resource for interns.”
“These [discharge] instructions make the [discharge] process so easy!!! Love these...”
“Also feel like they have helped my discharge teaching in the room!”

Qualitative comments from families postdischarge included:
“I thought the instructions were very clear and easy to read. I especially thought that highlighting the important areas really helped.”
“I think this form looks great, and I really like the idea of having your child’s name on it.”

Through sequential Plan-Do Study-Act cycles, we increased the percentage of discharge instructions written at or below 7^th grade reading level from 13% to 98%. Our most impactful intervention was the creation and dissemination of standardized disease-specific discharge instruction templates. Our findings complement evidence in the adult and pediatric literature that the use of standardized, disease-specific discharge instruction templates may improve readability of instructions.^32,33 And, while quality improvement efforts have been employed to improve the discharge process for patients,^34-36 this is the first study in the inpatient setting that, to our knowledge, specifically addresses discharge instructions using quality improvement methods.

Our work targeted the critical intersection between individual health literacy, an individual’s capacity to acquire, interpret, and use health information, and the necessary changes needed within our healthcare system to ensure that appropriately written instructions are given to patients and families.^17,37 Our efforts focused on improving discharge instructions answer the call to consider health literacy a modifiable clinical risk factor.³⁷ Furthermore, we address the 6 aims for quality healthcare delivery: 1) safe, timely, efficient and equitable delivery of care through the creation and dissemination of standardized instructions that are written at the appropriate reading level for families to ease hospital-to-home transitions and streamline the workflow of medical providers; 2) effective education of medical providers on health literacy concepts; and 3) family-centeredness through the involvement of families in our QI efforts. While previous QI efforts to improve hospital-to-home transitions have focused on medication reconciliation, communication with primary care physicians, follow-up appointments, and timely discharges of patients, none have specifically focused on the quality of discharge instructions.^34-36

Most physicians do not receive education about how to write information that is readable and understandable; more than half of providers desired more education in this area.³⁸ Furthermore, pediatric providers may overestimate parental health literacy levels,³⁹ which may contribute to variability in the readability of written health materials. While education alone can contribute to a provider’s ability to create readable instructions, we note the improvement after the introduction of disease templates to demonstrate the importance of workflow-integrated higher reliability interventions to sustain improvements.

Our baseline poor readability rates were due to limited knowledge by frontline providers composing the instructions and a system in which an important element for successful hospital-to-home transitions was not tackled until patients were ready for discharge. Streamlining of the discharge process, including the creation of discharge instructions, may lead to improved efficiency, fewer discrepancies, more effective communication, and an enhanced family experience. Moreover, the success of our improvement work was due to key stakeholders, including parents, being a part of the team and the notable buy-in from providers.

Our work was not without limitations. We excluded non-English speaking families from the study. We were unable to measure reading level of our population directly and instead based our goals on national estimates. Our primary measure was readability, which is only 1 piece contributing to quality discharge instructions. Understandability and actionability are also important considerations; ^17,20,29,40 however, improvements in these areas were limited by our design options within the EHR. Our efforts focused on children with common general pediatric diagnoses, and it is unclear how our interventions would generalize to medically complex patients with more volume of information to communicate at discharge and with uncommon diagnoses that are less readily incorporated into standardized templates. Relatedly, our work occurred at the satellite campus of our tertiary care center and may not represent generalizable material or methods to implement templates at our main campus location or at other hospitals. To begin to better understand this, we have spread to HM patients at our main campus, including medically complex patients with technology dependence and/or neurological impairments. Standardized, disease-specific templates most relevant to this population as well as several patient specific templates, for those with frequent readmissions due to medical complexity, have been created and are actively being tested.

In conclusion, in using interventions targeted at standardization of discharge instructions and timely feedback to staff, we saw rapid, dramatic, and sustained improvement in the readability of discharge instructions. Next steps include adaptation and spread to other patient populations and care teams, collaborations with other centers, and assessing the impact of effectively written discharge instructions on patient outcomes, such as adverse drug events, readmission rates, and family experience.

Disclosure

No external funding was secured for this study. Dr. Brady is supported by a Patient-Centered Outcomes Research Mentored Clinical Investigator Award from the Agency for Healthcare Research and Quality, Award Number K08HS023827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations. The funding organization had no role in the design, preparation, review, or approval of this paper; nor the decision to submit the manuscript for publication. The authors have no financial relationships relevant to this article to disclose.

The transition from hospital to home can be overwhelming for caregivers.¹ Stress of hospitalization coupled with the expectation of families to execute postdischarge care plans make understandable discharge communication critical. Communication failures, inadequate education, absence of caregiver confidence, and lack of clarity regarding care plans may prohibit smooth transitions and lead to adverse postdischarge outcomes.^2-4

Health literacy plays a pivotal role in caregivers’ capacity to navigate the healthcare system, comprehend, and execute care plans. An estimated 90 million Americans have limited health literacy that may negatively impact the provision of safe and quality care^5,6 and be a risk factor for poor outcomes, including increased emergency department (ED) utilization and readmission rates.^7-9 Readability strongly influences the effectiveness of written materials.¹⁰ However, written medical information for patients and families are frequently between the 10^th and 12^th grade reading levels; more than 75% of all pediatric health information is written at or above 10^th grade reading level.¹¹ Government agencies recommend between a 6^th and 8^th grade reading level, for written material;^5,12,13 written discharge instructions have been identified as an important quality metric for hospital-to-home transitions.^14-16

At our center, we found that discharge instructions were commonly written at high reading levels and often incomplete.¹⁷ Poor discharge instructions may contribute to increased readmission rates and unnecessary ED visits.^9,18 Our global aim targeted improved health-literate written information, including understandability and completeness.

Our specific aim was to increase the percentage of discharge instructions written at or below the 7th grade level for hospital medicine (HM) patients on a community hospital pediatric unit from 13% to 80% in 6 months.

Context

The improvement work took place at a 42-bed inpatient pediatric unit at a community satellite of our large, urban, academic hospital. The unit is staffed by medical providers including attendings, fellows, nurse practitioners (NPs), and senior pediatric residents, and had more than 1000 HM discharges in fiscal year 2016. Children with common general pediatric diagnoses are admitted to this service; postsurgical patients are not admitted primarily to the HM service. In Cincinnati, the neighborhood-level high school drop-out rates are as high as 64%.¹⁹ Discharge instructions are written by medical providers in the electronic health record (EHR). A printed copy is given to families and verbally reviewed by a bedside nurse prior to discharge. Quality improvement (QI) efforts focused on discharge instructions were ignited by a prior review of 200 discharge instructions that showed they were difficult to read (median reading level of 10^th grade), poorly understandable (36% of instructions met the threshold of understandability as measured by the Patient Education Materials Assessment Tool²⁰) and were missing key elements of information.¹⁷

Improvement Team

The improvement team consisted of 4 pediatric hospitalists, 2 NPs, 1 nurse educator with health literacy expertise, 1 pediatric resident, 1 fourth-year medical student, 1 QI consultant, and 2 parents who had first-hand experience on the HM service. The improvement team observed the discharge process, including roles of the provider, nurse and family, outlined a process map, and created a modified failure mode and effect analysis.²¹ Prior to our work, discharge instructions written by providers often occurred as a last step, and the content was created as free text or from nonstandardized templates. Key drivers that informed interventions were determined and revised over time (Figure 1). The study was reviewed by our institutional review board and deemed not human subjects research.

Improvement Activities

Key drivers were identified, and interventions were executed using Plan-Do Study-Act cycles.²² The key drivers thought to be critical for the success of the QI efforts were family engagement; standardization of discharge instructions; medical staff engagement; and audit and feedback of data. The corresponding interventions were as follows:

Family Engagement

Understanding the discharge information families desired. Prior to testing, 10 families admitted to the HM service were asked about the discharge experience. We asked families about information they wanted in written discharge instructions: 1) reasons to call your primary doctor or return to the hospital; 2) when to see your primary doctor for a follow-up visit; 3) the phone number to reach your child’s doctor; 4) more information about why your child was admitted; 5) information about new medications; and 6) what to do to help your child continue to recover at home.

Development of templates. We engaged families throughout the process of creating general and disease-specific discharge templates. After a specific template was created and reviewed by the parents on our team, it was sent to members of the institutional Patient Education Committee, which includes parents and local health literacy experts, to review and critique. Feedback from the reviewers was incorporated into the templates prior to use in the EHR.

Postdischarge phone calls.A convenience sample of families discharged from the satellite campus was called 24 to 48 hours after discharge over a 2-week period in January, 2016. A member of our improvement team solicited feedback from families about the quality of the discharge instructions. Families were asked if discharge instructions were reviewed with them prior to going home, if they were given a copy of the instructions, how they would rate the ability to read and use the information, and if there were additional pieces of information that would have improved the instructions.

Standardization of Instructions

Education. A presentation was created and shared with medical providers; it was re-disseminated monthly to new residents rotating onto the service and to the attendings, fellows, and NPs scheduled for shifts during the month. This education continued for the duration of the study. The presentation included the definition of health literacy, scope of the problem, examples of poorly written discharge instructions, and tips on how to write readable and understandable instructions. Laminated cards that included tips on how to write instructions were also placed on work stations.

While the general template was an important first step, the content relied heavily on free text by providers, which could still lead to instructions written at a high reading level. Thus, disease-specific discharge instruction templates were created with prepopulated information that was written at a reading level at or below 7^th grade level (Figure 2). The diseases were prioritized based on the most common diagnoses on our HM service. Each template included information under each of the subheadings noted in the general template. Twelve disease-specific templates were tested and ultimately embedded in the EHR; the general template remained for use when the discharge diagnosis was not covered by a disease-specific template.

Medical Staff Engagement

Previously described tests of change also aimed to enhance staff engagement. These included frequent e-mails, discussion of the QI efforts at specific team meetings, and the creation of visual cues posted at computer work stations, which prompted staff to begin to work on discharge instructions soon after admission.

Audit and Feedback of Data

Weekly phone calls. One team updated clinicians through a regularly scheduled bi-weekly phone conference. The phone conference was established prior to our work and was designed to relay pertinent information to attendings and NPs who work at the satellite hospital. During the phone conferences, clinicians were notified of current performance on discharge instruction readability and specific tests of change for the week. Additionally, providers gave feedback about the improvement efforts. These updates continued for the first 6 months of the project until sustained improvements were observed.

E-mails. Weekly e-mails were sent to all providers scheduled for clinical time at the satellite campus. The e-mail contained information on current tests of change, a list of discharge instruction templates that were available in the EHR, and the annotated run chart illustrating readability levels over time.

Additionally, individual e-mails were sent to each provider after review of the written discharge instructions for the week. Providers were given information on the number of discharge instructions they personally composed, the percentage of those instructions that were written at or below 7^th grade level, and specific feedback on how their written instructions could be improved. We also encouraged feedback from each provider to better identify barriers to achieving our goal.

Study of the Interventions

Baseline data included a review of all instructions for patients discharged from the satellite campus from the end of April 2015 through mid-September 2015. The time period for testing of interventions during the fall and winter months allowed for rapid cycle learning due to higher patient census and predictability of admissions for specific diagnosis (ie, asthma and bronchiolitis). An automated report was generated from the EHR weekly with specific demographics and identifiers for patient discharged over the past 7 days, including patient age, gender, length of stay, discharge diagnosis, and insurance classification. Data was collected during the intervention period via structured review of the discharge instructions in the EHR by the principal investigator or a trained research coordinator. Discharge instructions for medically cleared mental health patients admitted to hospital medicine while awaiting psychiatric bed availability and patients and parents who were non-English speaking were excluded from review. All other instructions for patients discharged from the HM service at our Liberty Campus were included for review.

Measures

Readability, our primary measure of interest, was calculated using the mean score from the following formulas: Flesch Kincaid Grade Level,²³ Simple Measure of Gobbledygook Index,²⁴ Coleman-Liau Index,²⁵ Gunning-Fog Index,²⁶ and Automated Readability Index²⁷ by means of an online platform (https://readability-score.com).²⁸ This platform was chosen because it incorporated a variety of formulas, was user-friendly, and required minimal data cleaning. Each of the readability formulas have been used to assesses readability of health information given to patients and families.^29,30 The threshold of 7th grade is in alignment with our institutional policy for educational materials and with recommendations from several government agencies.^5,12

Analysis

A statistical process control p-chart was used to analyze our primary measure of readability, dichotomized as percent discharge instructions written at or below 7th grade level. Run charts were used to follow mean reading level of discharge instructions and our process measure of percent of discharge instruction written with a general or disease-specific standardized template. Run chart and control chart rules for identifying special cause were used for midline shifts.³¹

The Table includes the demographic and clinical information of patients included in our analyses. Through sequential interventions, the percentage of discharge instructions written at or below 7th grade readability level increased from a mean of 13% to more than 80% in 3 months (Figure 3). Furthermore, the mean was sustained above 90% for 10 months and at 98% for the last 4 months. The use of 1 of the 13 EHR templates increased from 0% to 96% and was associated with the largest impact on the overall improvements (Supplemental Figure 1). Additionally, the average reading level of the discharge instructions decreased from 10th grade to 6th grade level (Supplemental Figure 2).

Qualitative comments from providers about the discharge instructions included:

“Are these [discharge instructions] available at base??  Great resource for interns.”
“These [discharge] instructions make the [discharge] process so easy!!! Love these...”
“Also feel like they have helped my discharge teaching in the room!”

Qualitative comments from families postdischarge included:
“I thought the instructions were very clear and easy to read. I especially thought that highlighting the important areas really helped.”
“I think this form looks great, and I really like the idea of having your child’s name on it.”

Through sequential Plan-Do Study-Act cycles, we increased the percentage of discharge instructions written at or below 7^th grade reading level from 13% to 98%. Our most impactful intervention was the creation and dissemination of standardized disease-specific discharge instruction templates. Our findings complement evidence in the adult and pediatric literature that the use of standardized, disease-specific discharge instruction templates may improve readability of instructions.^32,33 And, while quality improvement efforts have been employed to improve the discharge process for patients,^34-36 this is the first study in the inpatient setting that, to our knowledge, specifically addresses discharge instructions using quality improvement methods.

Our work targeted the critical intersection between individual health literacy, an individual’s capacity to acquire, interpret, and use health information, and the necessary changes needed within our healthcare system to ensure that appropriately written instructions are given to patients and families.^17,37 Our efforts focused on improving discharge instructions answer the call to consider health literacy a modifiable clinical risk factor.³⁷ Furthermore, we address the 6 aims for quality healthcare delivery: 1) safe, timely, efficient and equitable delivery of care through the creation and dissemination of standardized instructions that are written at the appropriate reading level for families to ease hospital-to-home transitions and streamline the workflow of medical providers; 2) effective education of medical providers on health literacy concepts; and 3) family-centeredness through the involvement of families in our QI efforts. While previous QI efforts to improve hospital-to-home transitions have focused on medication reconciliation, communication with primary care physicians, follow-up appointments, and timely discharges of patients, none have specifically focused on the quality of discharge instructions.^34-36

Most physicians do not receive education about how to write information that is readable and understandable; more than half of providers desired more education in this area.³⁸ Furthermore, pediatric providers may overestimate parental health literacy levels,³⁹ which may contribute to variability in the readability of written health materials. While education alone can contribute to a provider’s ability to create readable instructions, we note the improvement after the introduction of disease templates to demonstrate the importance of workflow-integrated higher reliability interventions to sustain improvements.

Our baseline poor readability rates were due to limited knowledge by frontline providers composing the instructions and a system in which an important element for successful hospital-to-home transitions was not tackled until patients were ready for discharge. Streamlining of the discharge process, including the creation of discharge instructions, may lead to improved efficiency, fewer discrepancies, more effective communication, and an enhanced family experience. Moreover, the success of our improvement work was due to key stakeholders, including parents, being a part of the team and the notable buy-in from providers.

Our work was not without limitations. We excluded non-English speaking families from the study. We were unable to measure reading level of our population directly and instead based our goals on national estimates. Our primary measure was readability, which is only 1 piece contributing to quality discharge instructions. Understandability and actionability are also important considerations; ^17,20,29,40 however, improvements in these areas were limited by our design options within the EHR. Our efforts focused on children with common general pediatric diagnoses, and it is unclear how our interventions would generalize to medically complex patients with more volume of information to communicate at discharge and with uncommon diagnoses that are less readily incorporated into standardized templates. Relatedly, our work occurred at the satellite campus of our tertiary care center and may not represent generalizable material or methods to implement templates at our main campus location or at other hospitals. To begin to better understand this, we have spread to HM patients at our main campus, including medically complex patients with technology dependence and/or neurological impairments. Standardized, disease-specific templates most relevant to this population as well as several patient specific templates, for those with frequent readmissions due to medical complexity, have been created and are actively being tested.

In conclusion, in using interventions targeted at standardization of discharge instructions and timely feedback to staff, we saw rapid, dramatic, and sustained improvement in the readability of discharge instructions. Next steps include adaptation and spread to other patient populations and care teams, collaborations with other centers, and assessing the impact of effectively written discharge instructions on patient outcomes, such as adverse drug events, readmission rates, and family experience.

Disclosure

No external funding was secured for this study. Dr. Brady is supported by a Patient-Centered Outcomes Research Mentored Clinical Investigator Award from the Agency for Healthcare Research and Quality, Award Number K08HS023827. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations. The funding organization had no role in the design, preparation, review, or approval of this paper; nor the decision to submit the manuscript for publication. The authors have no financial relationships relevant to this article to disclose.

The average American adult reads at an 8th-grade level.¹ Limited general literacy can affect health literacy, which is defined as the “degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions.”^2,3 Adults with limited health literacy are at risk for poorer outcomes, including overuse of the emergency department and lower adherence to preventive care recommendations.⁴

Children transitioning from hospital to home depend on their adult caregivers (and their caregivers’ health literacy) to carry out discharge instructions. During the immediate postdischarge period, complex care needs can involve new or changed medications, follow-up instructions, home care instructions, and suggestions regarding when and why to seek additional care.

The discharge education provided to patients in the hospital is often subpar because of lack of standardization and divided responsibility among providers.⁵ Communication of vital information to patients with low health literacy has been noted to be particularly poor,⁶ as many patient education materials are written at 10th-, 11th-, and 12th-grade reading levels.⁴ Evidence supports providing materials written at 6th-grade level or lower to increase comprehension.⁷ Several studies have evaluated the quality and readability of discharge instructions for hospitalized adults,^8,9 and one study found a link between poorly written instructions for adult patients and readmission risk.¹⁰ Less is known about readability in pediatrics, in which education may be more important for families of children most commonly hospitalized for acute illness.

We conducted a study to describe readability levels, understandability scores, and completeness of written instructions given to families at hospital discharge.

Study Design and Setting

In this study, we performed a cross-sectional review of discharge instructions within electronic health records at Cincinnati Children’s Hospital Medical Center (CCHMC). The study was reviewed and approved by CCHMC’s Institutional Review Board. Charts were randomly selected from all hospital medicine service discharges during two 3-month periods of high patient volume: January-March 2014 and January-March 2015.

CCHMC is a large urban academic referral center that is the sole provider of general, subspecialty, and critical pediatric inpatient care for a large geographical area. CCHMC, which has 600 beds, provides cares for many children who live in impoverished settings. Its hospital medicine service consists of 4 teams that care for approximately 7000 children hospitalized with general pediatric illnesses each year. Each team consists of 5 or 6 pediatric residents supervised by a hospital medicine attending.

Providers, most commonly pediatric interns, generate discharge instructions in electronic health records. In this nonautomated process, they use free-text or nonstandardized templates to create content. At discharge, instructions are printed as part of the postvisit summary, which includes updates on medications and scheduled follow-up appointments. Bedside nurses verbally review the instructions with families and provide printed copies for home use.

Data Collection and Analysis

A random sequence generator was used to select charts for review. Instructions written in a language other than English were excluded. Written discharge instructions and clinical information, including age, sex, primary diagnosis, insurance type, number of discharge medications, number of scheduled appointments at discharge, and hospital length of stay, were abstracted from electronic health records and anonymized before analysis. The primary outcomes assessed were discharge instruction readability, understandability, and completeness. Readability was calculated with Fry Readability Scale (FRS) scores,¹¹ which range from 1 to 17 and correspond to reading levels (score 1 = 1st-grade reading level). Health literacy experts have used the FRS to assess readability in health care environments.¹²

Understandability was measured with the Patient Education Materials Assessment Tool (PEMAT), a validated scoring system provided by the Agency for Healthcare Research and Quality.¹³ The PEMAT measures the understandability of print materials on a scale ranging from 0% to 100%. Higher scores indicate increased understandability, and scores under 70% indicate instructions are difficult to understand.

Although recent efforts have focused on the development of quality metrics for hospital-to-home transitions of pediatric patients,¹⁴ during our study there were no standard items to include in pediatric discharge instructions. Five criteria for completeness were determined by consensus of 3 pediatric hospital medicine faculty and were informed by qualitative results of work performed at our institution—work in which families noted challenges with information overload and a desire for pertinent and usable information that would enhance caregiver confidence and discharge preparedness.¹⁵ The criteria included statement of diagnosis, description of diagnosis, signs and symptoms indicative of the need for escalation of care (warning signs), the person caregivers should call if worried, and contact information for the primary care provider, subspecialist, and/or emergency department. Each set of discharge instructions was manually evaluated for completeness (presence of each individual component, number of components present, presence of all components). All charts were scored by the same investigator. A convenience sample of 20 charts was evaluated by a different investigator to ensure rating parameters were clear and classification was consistent (defined as perfect agreement). If the primary rater was undecided on a discharge instruction score, the secondary rater rated the instruction, and consensus was reached.

Means, medians, and ranges were calculated to enumerate the distribution of readability levels, understandability scores, and completeness of discharge instructions. Instructions were classified as readable if the FRS score was 6 or under, as understandable if the PEMAT score was under 70%, and as complete if all 5 criteria were satisfied. Descriptive statistics were generated for all demographic and clinical variables.

Of the study period’s 3819 discharges, 200 were randomly selected for review. Table 1 lists the demographic and clinical information of patients included in the analyses. Median FRS score was 10, indicating a 10th-grade reading level (interquartile range, 8-12; range, 1-13) (Table 2). Only 14 (7%) of 200 discharge instructions had a score of 6 or under. Median PEMAT understandability score was 73% (interquartile range, 64%-82%), and 36% of instructions had a PEMAT score under 70%. No instruction satisfied all 5 of the defined characteristics of complete discharge instructions (Table 2).

To our knowledge, this is the first study of the readability, understandability, and completeness of discharge instructions in a pediatric population. We found that the majority of discharge instruction readability levels were 10th grade or higher, that many instructions were difficult to understand, and that important information was missing from many instructions.

Discharge instruction readability levels were higher than the literacy level of many families in surrounding communities. The high school dropout rates in Cincinnati are staggering; they range from 22% to 64% in the 10 neighborhoods with the largest proportion of residents not completing high school.¹⁶ However, such findings are not unique to Cincinnati; low literacy is prevalent throughout the United States. Caregivers with limited literacy skills may struggle to navigate complex health systems, understand medical instructions and anticipatory guidance, perform child care and self-care tasks, and understand issues related to consent, medical authorization, and risk communication.¹⁷

Although readability is important, other factors also correlate with comprehension and execution of discharge tasks.¹⁸ Information must be understandable, or presented in a way that makes sense and can inform appropriate action. In many cases in our study, instructions were incomplete, despite previous investigators’ emphasizing caregivers’ desire and need for written instructions that are complete, informative, and inclusive of clearly outlined contingency plans.^15,19 In addition, families may differ in the level of support needed after discharge; standardizing elements and including families in the development of discharge instructions may improve communication.⁸

This study had several limitations. First, the discharge instructions randomly selected for review were all written during the winter months. As the census on the hospital medicine teams is particularly high during that time, authors with competing responsibilities may not have had enough time to write effective discharge instructions then. We selected the winter period in order to capture real-world instructions written during a busy clinical time, when providers care for a high volume of patients. Second, caregiver health literacy and English-language proficiency were not assessed, and information regarding caregivers’ race/ethnicity, educational attainment, and socioeconomic status was unavailable. Third, interrater agreement was not formally evaluated. Fourth, this was a single-center study with results that may not be generalizable.

In conclusion, discharge instructions for pediatric patients are often difficult to read and understand, and incomplete. Efforts to address these communication gaps—including educational initiatives for physician trainees focused on health literacy, and quality improvement work directed at standardization and creation of readable, understandable, and complete discharge instructions—are crucial in providing safe, high-value care. Researchers need to evaluate the relationship between discharge instruction quality and outcomes, including unplanned office visits, emergency department visits, and readmissions.

Disclosure

Nothing to report.

The average American adult reads at an 8th-grade level.¹ Limited general literacy can affect health literacy, which is defined as the “degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions.”^2,3 Adults with limited health literacy are at risk for poorer outcomes, including overuse of the emergency department and lower adherence to preventive care recommendations.⁴

Children transitioning from hospital to home depend on their adult caregivers (and their caregivers’ health literacy) to carry out discharge instructions. During the immediate postdischarge period, complex care needs can involve new or changed medications, follow-up instructions, home care instructions, and suggestions regarding when and why to seek additional care.

The discharge education provided to patients in the hospital is often subpar because of lack of standardization and divided responsibility among providers.⁵ Communication of vital information to patients with low health literacy has been noted to be particularly poor,⁶ as many patient education materials are written at 10th-, 11th-, and 12th-grade reading levels.⁴ Evidence supports providing materials written at 6th-grade level or lower to increase comprehension.⁷ Several studies have evaluated the quality and readability of discharge instructions for hospitalized adults,^8,9 and one study found a link between poorly written instructions for adult patients and readmission risk.¹⁰ Less is known about readability in pediatrics, in which education may be more important for families of children most commonly hospitalized for acute illness.

We conducted a study to describe readability levels, understandability scores, and completeness of written instructions given to families at hospital discharge.

Study Design and Setting

In this study, we performed a cross-sectional review of discharge instructions within electronic health records at Cincinnati Children’s Hospital Medical Center (CCHMC). The study was reviewed and approved by CCHMC’s Institutional Review Board. Charts were randomly selected from all hospital medicine service discharges during two 3-month periods of high patient volume: January-March 2014 and January-March 2015.

CCHMC is a large urban academic referral center that is the sole provider of general, subspecialty, and critical pediatric inpatient care for a large geographical area. CCHMC, which has 600 beds, provides cares for many children who live in impoverished settings. Its hospital medicine service consists of 4 teams that care for approximately 7000 children hospitalized with general pediatric illnesses each year. Each team consists of 5 or 6 pediatric residents supervised by a hospital medicine attending.

Providers, most commonly pediatric interns, generate discharge instructions in electronic health records. In this nonautomated process, they use free-text or nonstandardized templates to create content. At discharge, instructions are printed as part of the postvisit summary, which includes updates on medications and scheduled follow-up appointments. Bedside nurses verbally review the instructions with families and provide printed copies for home use.

Data Collection and Analysis

A random sequence generator was used to select charts for review. Instructions written in a language other than English were excluded. Written discharge instructions and clinical information, including age, sex, primary diagnosis, insurance type, number of discharge medications, number of scheduled appointments at discharge, and hospital length of stay, were abstracted from electronic health records and anonymized before analysis. The primary outcomes assessed were discharge instruction readability, understandability, and completeness. Readability was calculated with Fry Readability Scale (FRS) scores,¹¹ which range from 1 to 17 and correspond to reading levels (score 1 = 1st-grade reading level). Health literacy experts have used the FRS to assess readability in health care environments.¹²

Understandability was measured with the Patient Education Materials Assessment Tool (PEMAT), a validated scoring system provided by the Agency for Healthcare Research and Quality.¹³ The PEMAT measures the understandability of print materials on a scale ranging from 0% to 100%. Higher scores indicate increased understandability, and scores under 70% indicate instructions are difficult to understand.

Although recent efforts have focused on the development of quality metrics for hospital-to-home transitions of pediatric patients,¹⁴ during our study there were no standard items to include in pediatric discharge instructions. Five criteria for completeness were determined by consensus of 3 pediatric hospital medicine faculty and were informed by qualitative results of work performed at our institution—work in which families noted challenges with information overload and a desire for pertinent and usable information that would enhance caregiver confidence and discharge preparedness.¹⁵ The criteria included statement of diagnosis, description of diagnosis, signs and symptoms indicative of the need for escalation of care (warning signs), the person caregivers should call if worried, and contact information for the primary care provider, subspecialist, and/or emergency department. Each set of discharge instructions was manually evaluated for completeness (presence of each individual component, number of components present, presence of all components). All charts were scored by the same investigator. A convenience sample of 20 charts was evaluated by a different investigator to ensure rating parameters were clear and classification was consistent (defined as perfect agreement). If the primary rater was undecided on a discharge instruction score, the secondary rater rated the instruction, and consensus was reached.

Means, medians, and ranges were calculated to enumerate the distribution of readability levels, understandability scores, and completeness of discharge instructions. Instructions were classified as readable if the FRS score was 6 or under, as understandable if the PEMAT score was under 70%, and as complete if all 5 criteria were satisfied. Descriptive statistics were generated for all demographic and clinical variables.

Of the study period’s 3819 discharges, 200 were randomly selected for review. Table 1 lists the demographic and clinical information of patients included in the analyses. Median FRS score was 10, indicating a 10th-grade reading level (interquartile range, 8-12; range, 1-13) (Table 2). Only 14 (7%) of 200 discharge instructions had a score of 6 or under. Median PEMAT understandability score was 73% (interquartile range, 64%-82%), and 36% of instructions had a PEMAT score under 70%. No instruction satisfied all 5 of the defined characteristics of complete discharge instructions (Table 2).

To our knowledge, this is the first study of the readability, understandability, and completeness of discharge instructions in a pediatric population. We found that the majority of discharge instruction readability levels were 10th grade or higher, that many instructions were difficult to understand, and that important information was missing from many instructions.

Discharge instruction readability levels were higher than the literacy level of many families in surrounding communities. The high school dropout rates in Cincinnati are staggering; they range from 22% to 64% in the 10 neighborhoods with the largest proportion of residents not completing high school.¹⁶ However, such findings are not unique to Cincinnati; low literacy is prevalent throughout the United States. Caregivers with limited literacy skills may struggle to navigate complex health systems, understand medical instructions and anticipatory guidance, perform child care and self-care tasks, and understand issues related to consent, medical authorization, and risk communication.¹⁷

Although readability is important, other factors also correlate with comprehension and execution of discharge tasks.¹⁸ Information must be understandable, or presented in a way that makes sense and can inform appropriate action. In many cases in our study, instructions were incomplete, despite previous investigators’ emphasizing caregivers’ desire and need for written instructions that are complete, informative, and inclusive of clearly outlined contingency plans.^15,19 In addition, families may differ in the level of support needed after discharge; standardizing elements and including families in the development of discharge instructions may improve communication.⁸

This study had several limitations. First, the discharge instructions randomly selected for review were all written during the winter months. As the census on the hospital medicine teams is particularly high during that time, authors with competing responsibilities may not have had enough time to write effective discharge instructions then. We selected the winter period in order to capture real-world instructions written during a busy clinical time, when providers care for a high volume of patients. Second, caregiver health literacy and English-language proficiency were not assessed, and information regarding caregivers’ race/ethnicity, educational attainment, and socioeconomic status was unavailable. Third, interrater agreement was not formally evaluated. Fourth, this was a single-center study with results that may not be generalizable.

In conclusion, discharge instructions for pediatric patients are often difficult to read and understand, and incomplete. Efforts to address these communication gaps—including educational initiatives for physician trainees focused on health literacy, and quality improvement work directed at standardization and creation of readable, understandable, and complete discharge instructions—are crucial in providing safe, high-value care. Researchers need to evaluate the relationship between discharge instruction quality and outcomes, including unplanned office visits, emergency department visits, and readmissions.

Disclosure

Nothing to report.

The average American adult reads at an 8th-grade level.¹ Limited general literacy can affect health literacy, which is defined as the “degree to which individuals have the capacity to obtain, process and understand basic health information and services needed to make appropriate health decisions.”^2,3 Adults with limited health literacy are at risk for poorer outcomes, including overuse of the emergency department and lower adherence to preventive care recommendations.⁴

Children transitioning from hospital to home depend on their adult caregivers (and their caregivers’ health literacy) to carry out discharge instructions. During the immediate postdischarge period, complex care needs can involve new or changed medications, follow-up instructions, home care instructions, and suggestions regarding when and why to seek additional care.

The discharge education provided to patients in the hospital is often subpar because of lack of standardization and divided responsibility among providers.⁵ Communication of vital information to patients with low health literacy has been noted to be particularly poor,⁶ as many patient education materials are written at 10th-, 11th-, and 12th-grade reading levels.⁴ Evidence supports providing materials written at 6th-grade level or lower to increase comprehension.⁷ Several studies have evaluated the quality and readability of discharge instructions for hospitalized adults,^8,9 and one study found a link between poorly written instructions for adult patients and readmission risk.¹⁰ Less is known about readability in pediatrics, in which education may be more important for families of children most commonly hospitalized for acute illness.

We conducted a study to describe readability levels, understandability scores, and completeness of written instructions given to families at hospital discharge.

Study Design and Setting

In this study, we performed a cross-sectional review of discharge instructions within electronic health records at Cincinnati Children’s Hospital Medical Center (CCHMC). The study was reviewed and approved by CCHMC’s Institutional Review Board. Charts were randomly selected from all hospital medicine service discharges during two 3-month periods of high patient volume: January-March 2014 and January-March 2015.

CCHMC is a large urban academic referral center that is the sole provider of general, subspecialty, and critical pediatric inpatient care for a large geographical area. CCHMC, which has 600 beds, provides cares for many children who live in impoverished settings. Its hospital medicine service consists of 4 teams that care for approximately 7000 children hospitalized with general pediatric illnesses each year. Each team consists of 5 or 6 pediatric residents supervised by a hospital medicine attending.

Providers, most commonly pediatric interns, generate discharge instructions in electronic health records. In this nonautomated process, they use free-text or nonstandardized templates to create content. At discharge, instructions are printed as part of the postvisit summary, which includes updates on medications and scheduled follow-up appointments. Bedside nurses verbally review the instructions with families and provide printed copies for home use.

Data Collection and Analysis

A random sequence generator was used to select charts for review. Instructions written in a language other than English were excluded. Written discharge instructions and clinical information, including age, sex, primary diagnosis, insurance type, number of discharge medications, number of scheduled appointments at discharge, and hospital length of stay, were abstracted from electronic health records and anonymized before analysis. The primary outcomes assessed were discharge instruction readability, understandability, and completeness. Readability was calculated with Fry Readability Scale (FRS) scores,¹¹ which range from 1 to 17 and correspond to reading levels (score 1 = 1st-grade reading level). Health literacy experts have used the FRS to assess readability in health care environments.¹²

Understandability was measured with the Patient Education Materials Assessment Tool (PEMAT), a validated scoring system provided by the Agency for Healthcare Research and Quality.¹³ The PEMAT measures the understandability of print materials on a scale ranging from 0% to 100%. Higher scores indicate increased understandability, and scores under 70% indicate instructions are difficult to understand.

Although recent efforts have focused on the development of quality metrics for hospital-to-home transitions of pediatric patients,¹⁴ during our study there were no standard items to include in pediatric discharge instructions. Five criteria for completeness were determined by consensus of 3 pediatric hospital medicine faculty and were informed by qualitative results of work performed at our institution—work in which families noted challenges with information overload and a desire for pertinent and usable information that would enhance caregiver confidence and discharge preparedness.¹⁵ The criteria included statement of diagnosis, description of diagnosis, signs and symptoms indicative of the need for escalation of care (warning signs), the person caregivers should call if worried, and contact information for the primary care provider, subspecialist, and/or emergency department. Each set of discharge instructions was manually evaluated for completeness (presence of each individual component, number of components present, presence of all components). All charts were scored by the same investigator. A convenience sample of 20 charts was evaluated by a different investigator to ensure rating parameters were clear and classification was consistent (defined as perfect agreement). If the primary rater was undecided on a discharge instruction score, the secondary rater rated the instruction, and consensus was reached.

Means, medians, and ranges were calculated to enumerate the distribution of readability levels, understandability scores, and completeness of discharge instructions. Instructions were classified as readable if the FRS score was 6 or under, as understandable if the PEMAT score was under 70%, and as complete if all 5 criteria were satisfied. Descriptive statistics were generated for all demographic and clinical variables.

Of the study period’s 3819 discharges, 200 were randomly selected for review. Table 1 lists the demographic and clinical information of patients included in the analyses. Median FRS score was 10, indicating a 10th-grade reading level (interquartile range, 8-12; range, 1-13) (Table 2). Only 14 (7%) of 200 discharge instructions had a score of 6 or under. Median PEMAT understandability score was 73% (interquartile range, 64%-82%), and 36% of instructions had a PEMAT score under 70%. No instruction satisfied all 5 of the defined characteristics of complete discharge instructions (Table 2).

To our knowledge, this is the first study of the readability, understandability, and completeness of discharge instructions in a pediatric population. We found that the majority of discharge instruction readability levels were 10th grade or higher, that many instructions were difficult to understand, and that important information was missing from many instructions.

Discharge instruction readability levels were higher than the literacy level of many families in surrounding communities. The high school dropout rates in Cincinnati are staggering; they range from 22% to 64% in the 10 neighborhoods with the largest proportion of residents not completing high school.¹⁶ However, such findings are not unique to Cincinnati; low literacy is prevalent throughout the United States. Caregivers with limited literacy skills may struggle to navigate complex health systems, understand medical instructions and anticipatory guidance, perform child care and self-care tasks, and understand issues related to consent, medical authorization, and risk communication.¹⁷

Although readability is important, other factors also correlate with comprehension and execution of discharge tasks.¹⁸ Information must be understandable, or presented in a way that makes sense and can inform appropriate action. In many cases in our study, instructions were incomplete, despite previous investigators’ emphasizing caregivers’ desire and need for written instructions that are complete, informative, and inclusive of clearly outlined contingency plans.^15,19 In addition, families may differ in the level of support needed after discharge; standardizing elements and including families in the development of discharge instructions may improve communication.⁸

This study had several limitations. First, the discharge instructions randomly selected for review were all written during the winter months. As the census on the hospital medicine teams is particularly high during that time, authors with competing responsibilities may not have had enough time to write effective discharge instructions then. We selected the winter period in order to capture real-world instructions written during a busy clinical time, when providers care for a high volume of patients. Second, caregiver health literacy and English-language proficiency were not assessed, and information regarding caregivers’ race/ethnicity, educational attainment, and socioeconomic status was unavailable. Third, interrater agreement was not formally evaluated. Fourth, this was a single-center study with results that may not be generalizable.

In conclusion, discharge instructions for pediatric patients are often difficult to read and understand, and incomplete. Efforts to address these communication gaps—including educational initiatives for physician trainees focused on health literacy, and quality improvement work directed at standardization and creation of readable, understandable, and complete discharge instructions—are crucial in providing safe, high-value care. Researchers need to evaluate the relationship between discharge instruction quality and outcomes, including unplanned office visits, emergency department visits, and readmissions.

Disclosure

Nothing to report.