Exposure to clindamycin and doxycycline during the first trimester may be associated with an increased risk of organ-specific malformations such as ventricular/atrial septal defect, new research suggests.

In a population-based cohort study in 139,938 liveborn singletons, there was an 81% increased risk of ventricular/septal defect (adjusted odds ratio, 1.81) associated with clindamycin exposure during the first trimester, compared with no exposure. Similarly, there was a 67% greater risk of musculoskeletal system malformations (aOR, 1.67) associated with clindamycin exposure (Br J Clin Pharmacol. 2017 Jul 19. doi: 10.1111/bcp.13364).

Kathy Scarbeck/Frontline Medical News

Exposure to clindamycin and doxycycline during the first trimester may be associated with an increased risk of organ-specific malformations such as ventricular/atrial septal defect, new research suggests.

In a population-based cohort study in 139,938 liveborn singletons, there was an 81% increased risk of ventricular/septal defect (adjusted odds ratio, 1.81) associated with clindamycin exposure during the first trimester, compared with no exposure. Similarly, there was a 67% greater risk of musculoskeletal system malformations (aOR, 1.67) associated with clindamycin exposure (Br J Clin Pharmacol. 2017 Jul 19. doi: 10.1111/bcp.13364).

Kathy Scarbeck/Frontline Medical News

Exposure to clindamycin and doxycycline during the first trimester may be associated with an increased risk of organ-specific malformations such as ventricular/atrial septal defect, new research suggests.

In a population-based cohort study in 139,938 liveborn singletons, there was an 81% increased risk of ventricular/septal defect (adjusted odds ratio, 1.81) associated with clindamycin exposure during the first trimester, compared with no exposure. Similarly, there was a 67% greater risk of musculoskeletal system malformations (aOR, 1.67) associated with clindamycin exposure (Br J Clin Pharmacol. 2017 Jul 19. doi: 10.1111/bcp.13364).

Kathy Scarbeck/Frontline Medical News

Whether robbing banks or reducing healthcare spending, it makes sense to go where the money is. In the case of healthcare, 32% of spending goes to inpatient care, so hospitals represent a logical target for cost-reduction efforts. Because most hospital costs are fixed, there are basically 2 approaches to reducing spending—shorten length of stay or keep patients out of the hospital altogether. The government has tried both, using the power of financial incentives to spur adoption.

Faced with soaring hospital costs in the 1980s, Medicare introduced its prospective payment system, offering hospitals a fixed payment for each specific Diagnosis-Related Group. Hospitals responded by discharging patients sooner, with a resultant rise in admissions to skilled nursing facilities (SNFs) and rapid growth of the home care industry. Length of stay fell dramatically, dropping 9% in 1984 alone.¹ It continued to decline through the 1990s, falling by almost 20% between 1993 and 2000. In the following decade, despite the rise of hospital medicine, the rate of decrease slowed to 0.2% per year.²

Attention then turned to readmissions. In 2008, the Medicare Payment Advisory Committee proposed that hospitals with high risk-adjusted readmission rates receive lower payments, arguing that readmissions accounted for $15 billion in Medicare spending and that many were preventable. Thus the Hospital Readmissions Reduction Program was born, introducing readmission penalties in 2012.

Numerous interventions emerged from government and nongovernment parties to reduce readmissions. Many used intensive transitional care programs focusing on early follow-up or medication safety, and some even went as far as providing transitional housing.³ Shortly after passage of the Affordable Care Act, readmission rates fell rapidly. Within a few years, however, the rate of decline slowed dramatically and may have reached a plateau.⁴ Many have argued that only a small proportion of readmissions are preventable and that there are more direct ways to promote improved discharge planning without diverting resources from other areas.⁵ It seems that readmissions may not be feasibly reduced much further.

With the advent of accountable care organizations, health systems are now turning their focus to the small population of patients who consume a disproportionate share of healthcare dollars. Because the top 1% of patients—so-called super-utilizers—account for 21% of spending, efforts to reduce their utilization could produce outsized returns.⁶ Initial anecdotal reports described patients with complex physical, behavior, and social needs receiving fragmented care resulting in myriad expensive admissions. The response comprised teams of social workers and community health workers coupled with robust primary care, formulating individualized solutions. However, data supporting the effectiveness of this common-sense approach are lacking. In addition, our understanding of high-cost patients is evolving. For one thing, being a super-utilizer is often temporary, as just over one-quarter are still in that category a year later.⁷ Moreover, not all high-cost patients are frequently admitted.⁸

In this issue of The Journal of Hospital Medicine, Wick et al.⁹ provide additional insight into high utilizers of hospital services. The authors compare definitions of high utilizers based on cost, number of admissions, or cumulative length of stay over one year. Only 10 percent of high utilizers met all 3 definitions. The overlap between high utilizers by cost and length of stay was twice the overlap between high utilizers by number of admissions and either group. This finding is not surprising because hospitals have high fixed costs, so total cost tends to mirror length of stay.

The study was performed in Canada, and the overlap among these groups may be different in the US. The Canadian patients were hospitalized less frequently than their American counterparts, perhaps reflecting better access to primary care in the Canadian system. Regardless, Wick et al.⁹ add to the growing literature suggesting that the terms “high utilization” and “high cost” do not always describe the same population. This finding is important because strategies aimed at patients who are frequently admitted may not be effective for those who generate the highest costs. In trying to reduce overall costs, it may be time to revisit length of stay.

Given the long history of prospective payment in the US and the stagnation in length of stay over the past decade, it is reasonable to wonder whether further reductions are possible. In the study by Wick et al.,⁹ patients with longer lengths of stay were discharged to long-term care facilities. This observation is consistent with others’ reports. Studies of delays in care show that at least 10% of all hospital days can be attributed to delays in discharge, especially to SNFs. In the most recent study, 11% of hospital days were deemed unnecessary by hospitalists, with one-third of those delays due to lack of availability at an extended care facility.¹⁰ Six years earlier, Carey et al. found that 13.5% of inpatient days were unnecessary, with more than 60% of delays attributable to waiting for discharge to a SNF.¹¹

How, then, might we curtail unnecessary waiting, and whose job is it to solve the problem? The prospective payment system should reward hospitals for eliminating waiting—particularly those hospitals operating at capacity, for which the opportunity costs of occupied beds are most acute. Hospitalists, per se, have no incentive to discharge patients who are waiting; these patients are easy to round on, rarely have emergencies, and generate daily bills. Even when hospitalists are employed by the hospital and incentives for both are aligned, hospitalists may still be powerless to discharge waiting patients, summon busy consultants, or create extra slots in the endoscopy suite.

The move to value at the system level may offer hope. As health systems become responsible for the total cost of care, their focus must shift from the individual areas where care is provided to the transitions of care between treatment areas. It is in these transitions that US healthcare has failed most spectacularly, and consequently, it is where the greatest opportunity lies.

To date, most discharge interventions have focused on communication, with a goal of improving patient safety and, to a lesser extent, preventing readmissions. Partnering with SNFs can reduce the rate of readmissions,¹² but for the most part, the incentives for hospitals and post-acute care facilities remain misaligned. Because post-acute care facilities are paid per diem, they have little incentive to reduce patients’ stays or to admit new patients, who are more expensive to care for than existing ones. Physicians round on SNF patients infrequently and have no incentive to discharge patients, exacerbating the problem. Because post-acute care represents a growing proportion of costs for both medical and surgical patients, health systems will need to either have their own facilities or enter into contracts that align the incentives.

What can hospitalists do? As the predominant coordinators of hospitalized patients’ care both for medical and surgical teams, hospitalists meaningfully impact readmissions and lengths of stay through the care they provide.¹³ More important, as their roles in optimizing hospital throughput¹⁴ continue to expand, hospitalists are perhaps best positioned to observe a diverse range of inefficiencies and inadequacies in inpatient practice and translate those observations into new systems of care. Through thoughtful participation in hospital operations, administration, and health services research, hospitalists hold the key to improving the value of care we provide.

Disclosure 

Nothing to report.

Whether robbing banks or reducing healthcare spending, it makes sense to go where the money is. In the case of healthcare, 32% of spending goes to inpatient care, so hospitals represent a logical target for cost-reduction efforts. Because most hospital costs are fixed, there are basically 2 approaches to reducing spending—shorten length of stay or keep patients out of the hospital altogether. The government has tried both, using the power of financial incentives to spur adoption.

Faced with soaring hospital costs in the 1980s, Medicare introduced its prospective payment system, offering hospitals a fixed payment for each specific Diagnosis-Related Group. Hospitals responded by discharging patients sooner, with a resultant rise in admissions to skilled nursing facilities (SNFs) and rapid growth of the home care industry. Length of stay fell dramatically, dropping 9% in 1984 alone.¹ It continued to decline through the 1990s, falling by almost 20% between 1993 and 2000. In the following decade, despite the rise of hospital medicine, the rate of decrease slowed to 0.2% per year.²

Attention then turned to readmissions. In 2008, the Medicare Payment Advisory Committee proposed that hospitals with high risk-adjusted readmission rates receive lower payments, arguing that readmissions accounted for $15 billion in Medicare spending and that many were preventable. Thus the Hospital Readmissions Reduction Program was born, introducing readmission penalties in 2012.

Numerous interventions emerged from government and nongovernment parties to reduce readmissions. Many used intensive transitional care programs focusing on early follow-up or medication safety, and some even went as far as providing transitional housing.³ Shortly after passage of the Affordable Care Act, readmission rates fell rapidly. Within a few years, however, the rate of decline slowed dramatically and may have reached a plateau.⁴ Many have argued that only a small proportion of readmissions are preventable and that there are more direct ways to promote improved discharge planning without diverting resources from other areas.⁵ It seems that readmissions may not be feasibly reduced much further.

With the advent of accountable care organizations, health systems are now turning their focus to the small population of patients who consume a disproportionate share of healthcare dollars. Because the top 1% of patients—so-called super-utilizers—account for 21% of spending, efforts to reduce their utilization could produce outsized returns.⁶ Initial anecdotal reports described patients with complex physical, behavior, and social needs receiving fragmented care resulting in myriad expensive admissions. The response comprised teams of social workers and community health workers coupled with robust primary care, formulating individualized solutions. However, data supporting the effectiveness of this common-sense approach are lacking. In addition, our understanding of high-cost patients is evolving. For one thing, being a super-utilizer is often temporary, as just over one-quarter are still in that category a year later.⁷ Moreover, not all high-cost patients are frequently admitted.⁸

In this issue of The Journal of Hospital Medicine, Wick et al.⁹ provide additional insight into high utilizers of hospital services. The authors compare definitions of high utilizers based on cost, number of admissions, or cumulative length of stay over one year. Only 10 percent of high utilizers met all 3 definitions. The overlap between high utilizers by cost and length of stay was twice the overlap between high utilizers by number of admissions and either group. This finding is not surprising because hospitals have high fixed costs, so total cost tends to mirror length of stay.

The study was performed in Canada, and the overlap among these groups may be different in the US. The Canadian patients were hospitalized less frequently than their American counterparts, perhaps reflecting better access to primary care in the Canadian system. Regardless, Wick et al.⁹ add to the growing literature suggesting that the terms “high utilization” and “high cost” do not always describe the same population. This finding is important because strategies aimed at patients who are frequently admitted may not be effective for those who generate the highest costs. In trying to reduce overall costs, it may be time to revisit length of stay.

Given the long history of prospective payment in the US and the stagnation in length of stay over the past decade, it is reasonable to wonder whether further reductions are possible. In the study by Wick et al.,⁹ patients with longer lengths of stay were discharged to long-term care facilities. This observation is consistent with others’ reports. Studies of delays in care show that at least 10% of all hospital days can be attributed to delays in discharge, especially to SNFs. In the most recent study, 11% of hospital days were deemed unnecessary by hospitalists, with one-third of those delays due to lack of availability at an extended care facility.¹⁰ Six years earlier, Carey et al. found that 13.5% of inpatient days were unnecessary, with more than 60% of delays attributable to waiting for discharge to a SNF.¹¹

How, then, might we curtail unnecessary waiting, and whose job is it to solve the problem? The prospective payment system should reward hospitals for eliminating waiting—particularly those hospitals operating at capacity, for which the opportunity costs of occupied beds are most acute. Hospitalists, per se, have no incentive to discharge patients who are waiting; these patients are easy to round on, rarely have emergencies, and generate daily bills. Even when hospitalists are employed by the hospital and incentives for both are aligned, hospitalists may still be powerless to discharge waiting patients, summon busy consultants, or create extra slots in the endoscopy suite.

The move to value at the system level may offer hope. As health systems become responsible for the total cost of care, their focus must shift from the individual areas where care is provided to the transitions of care between treatment areas. It is in these transitions that US healthcare has failed most spectacularly, and consequently, it is where the greatest opportunity lies.

To date, most discharge interventions have focused on communication, with a goal of improving patient safety and, to a lesser extent, preventing readmissions. Partnering with SNFs can reduce the rate of readmissions,¹² but for the most part, the incentives for hospitals and post-acute care facilities remain misaligned. Because post-acute care facilities are paid per diem, they have little incentive to reduce patients’ stays or to admit new patients, who are more expensive to care for than existing ones. Physicians round on SNF patients infrequently and have no incentive to discharge patients, exacerbating the problem. Because post-acute care represents a growing proportion of costs for both medical and surgical patients, health systems will need to either have their own facilities or enter into contracts that align the incentives.

What can hospitalists do? As the predominant coordinators of hospitalized patients’ care both for medical and surgical teams, hospitalists meaningfully impact readmissions and lengths of stay through the care they provide.¹³ More important, as their roles in optimizing hospital throughput¹⁴ continue to expand, hospitalists are perhaps best positioned to observe a diverse range of inefficiencies and inadequacies in inpatient practice and translate those observations into new systems of care. Through thoughtful participation in hospital operations, administration, and health services research, hospitalists hold the key to improving the value of care we provide.

Disclosure 

Nothing to report.

Whether robbing banks or reducing healthcare spending, it makes sense to go where the money is. In the case of healthcare, 32% of spending goes to inpatient care, so hospitals represent a logical target for cost-reduction efforts. Because most hospital costs are fixed, there are basically 2 approaches to reducing spending—shorten length of stay or keep patients out of the hospital altogether. The government has tried both, using the power of financial incentives to spur adoption.

Faced with soaring hospital costs in the 1980s, Medicare introduced its prospective payment system, offering hospitals a fixed payment for each specific Diagnosis-Related Group. Hospitals responded by discharging patients sooner, with a resultant rise in admissions to skilled nursing facilities (SNFs) and rapid growth of the home care industry. Length of stay fell dramatically, dropping 9% in 1984 alone.¹ It continued to decline through the 1990s, falling by almost 20% between 1993 and 2000. In the following decade, despite the rise of hospital medicine, the rate of decrease slowed to 0.2% per year.²

Attention then turned to readmissions. In 2008, the Medicare Payment Advisory Committee proposed that hospitals with high risk-adjusted readmission rates receive lower payments, arguing that readmissions accounted for $15 billion in Medicare spending and that many were preventable. Thus the Hospital Readmissions Reduction Program was born, introducing readmission penalties in 2012.

Numerous interventions emerged from government and nongovernment parties to reduce readmissions. Many used intensive transitional care programs focusing on early follow-up or medication safety, and some even went as far as providing transitional housing.³ Shortly after passage of the Affordable Care Act, readmission rates fell rapidly. Within a few years, however, the rate of decline slowed dramatically and may have reached a plateau.⁴ Many have argued that only a small proportion of readmissions are preventable and that there are more direct ways to promote improved discharge planning without diverting resources from other areas.⁵ It seems that readmissions may not be feasibly reduced much further.

With the advent of accountable care organizations, health systems are now turning their focus to the small population of patients who consume a disproportionate share of healthcare dollars. Because the top 1% of patients—so-called super-utilizers—account for 21% of spending, efforts to reduce their utilization could produce outsized returns.⁶ Initial anecdotal reports described patients with complex physical, behavior, and social needs receiving fragmented care resulting in myriad expensive admissions. The response comprised teams of social workers and community health workers coupled with robust primary care, formulating individualized solutions. However, data supporting the effectiveness of this common-sense approach are lacking. In addition, our understanding of high-cost patients is evolving. For one thing, being a super-utilizer is often temporary, as just over one-quarter are still in that category a year later.⁷ Moreover, not all high-cost patients are frequently admitted.⁸

In this issue of The Journal of Hospital Medicine, Wick et al.⁹ provide additional insight into high utilizers of hospital services. The authors compare definitions of high utilizers based on cost, number of admissions, or cumulative length of stay over one year. Only 10 percent of high utilizers met all 3 definitions. The overlap between high utilizers by cost and length of stay was twice the overlap between high utilizers by number of admissions and either group. This finding is not surprising because hospitals have high fixed costs, so total cost tends to mirror length of stay.

The study was performed in Canada, and the overlap among these groups may be different in the US. The Canadian patients were hospitalized less frequently than their American counterparts, perhaps reflecting better access to primary care in the Canadian system. Regardless, Wick et al.⁹ add to the growing literature suggesting that the terms “high utilization” and “high cost” do not always describe the same population. This finding is important because strategies aimed at patients who are frequently admitted may not be effective for those who generate the highest costs. In trying to reduce overall costs, it may be time to revisit length of stay.

Given the long history of prospective payment in the US and the stagnation in length of stay over the past decade, it is reasonable to wonder whether further reductions are possible. In the study by Wick et al.,⁹ patients with longer lengths of stay were discharged to long-term care facilities. This observation is consistent with others’ reports. Studies of delays in care show that at least 10% of all hospital days can be attributed to delays in discharge, especially to SNFs. In the most recent study, 11% of hospital days were deemed unnecessary by hospitalists, with one-third of those delays due to lack of availability at an extended care facility.¹⁰ Six years earlier, Carey et al. found that 13.5% of inpatient days were unnecessary, with more than 60% of delays attributable to waiting for discharge to a SNF.¹¹

How, then, might we curtail unnecessary waiting, and whose job is it to solve the problem? The prospective payment system should reward hospitals for eliminating waiting—particularly those hospitals operating at capacity, for which the opportunity costs of occupied beds are most acute. Hospitalists, per se, have no incentive to discharge patients who are waiting; these patients are easy to round on, rarely have emergencies, and generate daily bills. Even when hospitalists are employed by the hospital and incentives for both are aligned, hospitalists may still be powerless to discharge waiting patients, summon busy consultants, or create extra slots in the endoscopy suite.

The move to value at the system level may offer hope. As health systems become responsible for the total cost of care, their focus must shift from the individual areas where care is provided to the transitions of care between treatment areas. It is in these transitions that US healthcare has failed most spectacularly, and consequently, it is where the greatest opportunity lies.

To date, most discharge interventions have focused on communication, with a goal of improving patient safety and, to a lesser extent, preventing readmissions. Partnering with SNFs can reduce the rate of readmissions,¹² but for the most part, the incentives for hospitals and post-acute care facilities remain misaligned. Because post-acute care facilities are paid per diem, they have little incentive to reduce patients’ stays or to admit new patients, who are more expensive to care for than existing ones. Physicians round on SNF patients infrequently and have no incentive to discharge patients, exacerbating the problem. Because post-acute care represents a growing proportion of costs for both medical and surgical patients, health systems will need to either have their own facilities or enter into contracts that align the incentives.

What can hospitalists do? As the predominant coordinators of hospitalized patients’ care both for medical and surgical teams, hospitalists meaningfully impact readmissions and lengths of stay through the care they provide.¹³ More important, as their roles in optimizing hospital throughput¹⁴ continue to expand, hospitalists are perhaps best positioned to observe a diverse range of inefficiencies and inadequacies in inpatient practice and translate those observations into new systems of care. Through thoughtful participation in hospital operations, administration, and health services research, hospitalists hold the key to improving the value of care we provide.

Disclosure 

Nothing to report.

Physicians, researchers, and policymakers aspire to improve the value of healthcare, with reduced overall costs of care and improved outcomes. An important component of increasing healthcare costs in the United States is the rising cost of prescription medications, accounting for an estimated 17% of all spending in healthcare services.¹ One potentially modifiable driver of low-value prescribing is poor awareness of medication cost.² While displaying price to the ordering physician has reduced laboratory order volume and associated testing costs,^3,4 applying cost transparency to medication ordering has produced variable results, perhaps reflecting conceptual differences in decision making regarding diagnosis and treatment.^4-6

In this issue of the Journal of Hospital Medicine, Conway et al.⁷ performed a retrospective analysis applying interrupted times series models to measure the impact of passive cost display on the ordering frequency of 9 high-cost intravenous (IV) or inhaled medications that were identified as likely overused. For 7 of the IV medications, lower-cost oral alternatives were available; 2 study medications had no clear therapeutic alternatives. It was expected that lower-cost oral alternatives would have a concomitant increase in ordering rate as the order rate of the study medications decreased (eg, oral linezolid use would increase as IV linezolid use decreased). Order rate was the primary outcome, reported each week as treatment orders per 10,000 patient days, and was compared for both the pre- and postimplementation time periods. The particular methodology of segmented regressions allowed the research team to control for preintervention trends in medication ordering, as well as to analyze both immediate and delayed effects of the cost-display intervention. The research team framed the cost display as a passive approach. The intervention displayed average wholesale cost data and lower-cost oral alternatives on the ordering screen, which did not significantly reduce the ordering rate. Over the course of the study, outside influences led to 2 more active approaches to higher-cost medications, and Conway et al. wisely measured their effect as well. Specifically, the IV pantoprazole ordering rate decreased after restrictions secondary to a national medication shortage, and the oral voriconazole ordering rate decreased following an oncology order set change from oral voriconazole to oral posaconazole. These ordering-rate decreases were not temporally related to the implementation of the cost display intervention.

It is important to note several limitations of this study, some of which the authors discuss in the manuscript. Because 2 of the medications studied (eculizumab and calcitonin) do not have direct therapeutic alternatives, it is not surprising that price display alone would have no effect. The ordering providers who received this cost information had a more complex decision to make than they would in a scenario with a lower-cost alternative, essentially requiring them to ask “Does this patient need this class of medications at all?” rather than simply, “Is a lower-cost alternative appropriate?” Similarly, choosing medication alternatives that would require different routes of administration (ie, IV and oral) may have limited the effectiveness of a price intervention, given that factors such as illness severity also may influence the decision between IV and oral agents. Thus, the lack of an effect for the price display intervention for these specific medications may not be generalizable to all other medication decisions. Additionally, this manuscript offers limited data on the context in which the intervention was implemented and what adaptations, if any, were made based on early findings. The results may have varied greatly based on the visual design and how the cost display was presented within the electronic medical record. The wider organizational context may also have affected the intervention’s impact. A cost-display intervention appearing in isolation could understandably have a different impact, compared with an intervention within the context of a broader cost/value curriculum directed at house staff and faculty.

In summary, Conway et al. found that just displaying cost data did little to change prescribing patterns, but that more active approaches were quite efficacious. So where does this leave value-minded hospitalists looking to reduce overuse? Relatedly, what are the next steps for research and improvement science? We think there are 3 key strategic areas on which to focus. First, behavioral economics offers a critically important middle ground between the passive approaches studied here and more heavy-handed approaches that may limit provider autonomy, such as restricting drug use at the formulary.⁸ An improved choice architecture that presents the preferred higher-value option as the default selection may result in improved adoption of the high-value choice while also preserving provider autonomy and expertise required when clinical circumstances make the higher-cost drug the better choice.^9,10 The second consideration is to minimize ethical tensions between cost displays that discourage use and a provider’s belief that a treatment is beneficial. Using available ethical frameworks for high-value care that engage both patient and societal concerns may help us choose and design interventions with more successful outcomes.¹¹ Finally, research has shown that providers have poor knowledge of both cost and the relative benefits and harms of treatments and testing.¹² Thus, the third opportunity for improvement is to provide appropriate clinical information (ie, relative therapeutic equivalency or adverse effects in alternative therapies) to support decision making at the point of order entry. Encouraging data already exists regarding how drug facts boxes can help patients understand benefits and side effects.¹³ A similar approach may aid physicians and may prove an easier task than improving patient understanding, given physicians’ substantial existing knowledge. These strategies may help guide providers to make a more informed value determination and obviate some ethical concerns related to clinical decisions based on cost alone. Despite their negative results, Conway et al.⁷ provided additional evidence that influencing complex decision making is not easy. However, we believe that continuing research into the factors that lead to successful value interventions has incredible potential for supporting high-value decision making in the future.

Disclosure 

Nothing to report.

Physicians, researchers, and policymakers aspire to improve the value of healthcare, with reduced overall costs of care and improved outcomes. An important component of increasing healthcare costs in the United States is the rising cost of prescription medications, accounting for an estimated 17% of all spending in healthcare services.¹ One potentially modifiable driver of low-value prescribing is poor awareness of medication cost.² While displaying price to the ordering physician has reduced laboratory order volume and associated testing costs,^3,4 applying cost transparency to medication ordering has produced variable results, perhaps reflecting conceptual differences in decision making regarding diagnosis and treatment.^4-6

In this issue of the Journal of Hospital Medicine, Conway et al.⁷ performed a retrospective analysis applying interrupted times series models to measure the impact of passive cost display on the ordering frequency of 9 high-cost intravenous (IV) or inhaled medications that were identified as likely overused. For 7 of the IV medications, lower-cost oral alternatives were available; 2 study medications had no clear therapeutic alternatives. It was expected that lower-cost oral alternatives would have a concomitant increase in ordering rate as the order rate of the study medications decreased (eg, oral linezolid use would increase as IV linezolid use decreased). Order rate was the primary outcome, reported each week as treatment orders per 10,000 patient days, and was compared for both the pre- and postimplementation time periods. The particular methodology of segmented regressions allowed the research team to control for preintervention trends in medication ordering, as well as to analyze both immediate and delayed effects of the cost-display intervention. The research team framed the cost display as a passive approach. The intervention displayed average wholesale cost data and lower-cost oral alternatives on the ordering screen, which did not significantly reduce the ordering rate. Over the course of the study, outside influences led to 2 more active approaches to higher-cost medications, and Conway et al. wisely measured their effect as well. Specifically, the IV pantoprazole ordering rate decreased after restrictions secondary to a national medication shortage, and the oral voriconazole ordering rate decreased following an oncology order set change from oral voriconazole to oral posaconazole. These ordering-rate decreases were not temporally related to the implementation of the cost display intervention.

It is important to note several limitations of this study, some of which the authors discuss in the manuscript. Because 2 of the medications studied (eculizumab and calcitonin) do not have direct therapeutic alternatives, it is not surprising that price display alone would have no effect. The ordering providers who received this cost information had a more complex decision to make than they would in a scenario with a lower-cost alternative, essentially requiring them to ask “Does this patient need this class of medications at all?” rather than simply, “Is a lower-cost alternative appropriate?” Similarly, choosing medication alternatives that would require different routes of administration (ie, IV and oral) may have limited the effectiveness of a price intervention, given that factors such as illness severity also may influence the decision between IV and oral agents. Thus, the lack of an effect for the price display intervention for these specific medications may not be generalizable to all other medication decisions. Additionally, this manuscript offers limited data on the context in which the intervention was implemented and what adaptations, if any, were made based on early findings. The results may have varied greatly based on the visual design and how the cost display was presented within the electronic medical record. The wider organizational context may also have affected the intervention’s impact. A cost-display intervention appearing in isolation could understandably have a different impact, compared with an intervention within the context of a broader cost/value curriculum directed at house staff and faculty.

In summary, Conway et al. found that just displaying cost data did little to change prescribing patterns, but that more active approaches were quite efficacious. So where does this leave value-minded hospitalists looking to reduce overuse? Relatedly, what are the next steps for research and improvement science? We think there are 3 key strategic areas on which to focus. First, behavioral economics offers a critically important middle ground between the passive approaches studied here and more heavy-handed approaches that may limit provider autonomy, such as restricting drug use at the formulary.⁸ An improved choice architecture that presents the preferred higher-value option as the default selection may result in improved adoption of the high-value choice while also preserving provider autonomy and expertise required when clinical circumstances make the higher-cost drug the better choice.^9,10 The second consideration is to minimize ethical tensions between cost displays that discourage use and a provider’s belief that a treatment is beneficial. Using available ethical frameworks for high-value care that engage both patient and societal concerns may help us choose and design interventions with more successful outcomes.¹¹ Finally, research has shown that providers have poor knowledge of both cost and the relative benefits and harms of treatments and testing.¹² Thus, the third opportunity for improvement is to provide appropriate clinical information (ie, relative therapeutic equivalency or adverse effects in alternative therapies) to support decision making at the point of order entry. Encouraging data already exists regarding how drug facts boxes can help patients understand benefits and side effects.¹³ A similar approach may aid physicians and may prove an easier task than improving patient understanding, given physicians’ substantial existing knowledge. These strategies may help guide providers to make a more informed value determination and obviate some ethical concerns related to clinical decisions based on cost alone. Despite their negative results, Conway et al.⁷ provided additional evidence that influencing complex decision making is not easy. However, we believe that continuing research into the factors that lead to successful value interventions has incredible potential for supporting high-value decision making in the future.

Disclosure 

Nothing to report.

Physicians, researchers, and policymakers aspire to improve the value of healthcare, with reduced overall costs of care and improved outcomes. An important component of increasing healthcare costs in the United States is the rising cost of prescription medications, accounting for an estimated 17% of all spending in healthcare services.¹ One potentially modifiable driver of low-value prescribing is poor awareness of medication cost.² While displaying price to the ordering physician has reduced laboratory order volume and associated testing costs,^3,4 applying cost transparency to medication ordering has produced variable results, perhaps reflecting conceptual differences in decision making regarding diagnosis and treatment.^4-6

In this issue of the Journal of Hospital Medicine, Conway et al.⁷ performed a retrospective analysis applying interrupted times series models to measure the impact of passive cost display on the ordering frequency of 9 high-cost intravenous (IV) or inhaled medications that were identified as likely overused. For 7 of the IV medications, lower-cost oral alternatives were available; 2 study medications had no clear therapeutic alternatives. It was expected that lower-cost oral alternatives would have a concomitant increase in ordering rate as the order rate of the study medications decreased (eg, oral linezolid use would increase as IV linezolid use decreased). Order rate was the primary outcome, reported each week as treatment orders per 10,000 patient days, and was compared for both the pre- and postimplementation time periods. The particular methodology of segmented regressions allowed the research team to control for preintervention trends in medication ordering, as well as to analyze both immediate and delayed effects of the cost-display intervention. The research team framed the cost display as a passive approach. The intervention displayed average wholesale cost data and lower-cost oral alternatives on the ordering screen, which did not significantly reduce the ordering rate. Over the course of the study, outside influences led to 2 more active approaches to higher-cost medications, and Conway et al. wisely measured their effect as well. Specifically, the IV pantoprazole ordering rate decreased after restrictions secondary to a national medication shortage, and the oral voriconazole ordering rate decreased following an oncology order set change from oral voriconazole to oral posaconazole. These ordering-rate decreases were not temporally related to the implementation of the cost display intervention.

It is important to note several limitations of this study, some of which the authors discuss in the manuscript. Because 2 of the medications studied (eculizumab and calcitonin) do not have direct therapeutic alternatives, it is not surprising that price display alone would have no effect. The ordering providers who received this cost information had a more complex decision to make than they would in a scenario with a lower-cost alternative, essentially requiring them to ask “Does this patient need this class of medications at all?” rather than simply, “Is a lower-cost alternative appropriate?” Similarly, choosing medication alternatives that would require different routes of administration (ie, IV and oral) may have limited the effectiveness of a price intervention, given that factors such as illness severity also may influence the decision between IV and oral agents. Thus, the lack of an effect for the price display intervention for these specific medications may not be generalizable to all other medication decisions. Additionally, this manuscript offers limited data on the context in which the intervention was implemented and what adaptations, if any, were made based on early findings. The results may have varied greatly based on the visual design and how the cost display was presented within the electronic medical record. The wider organizational context may also have affected the intervention’s impact. A cost-display intervention appearing in isolation could understandably have a different impact, compared with an intervention within the context of a broader cost/value curriculum directed at house staff and faculty.

In summary, Conway et al. found that just displaying cost data did little to change prescribing patterns, but that more active approaches were quite efficacious. So where does this leave value-minded hospitalists looking to reduce overuse? Relatedly, what are the next steps for research and improvement science? We think there are 3 key strategic areas on which to focus. First, behavioral economics offers a critically important middle ground between the passive approaches studied here and more heavy-handed approaches that may limit provider autonomy, such as restricting drug use at the formulary.⁸ An improved choice architecture that presents the preferred higher-value option as the default selection may result in improved adoption of the high-value choice while also preserving provider autonomy and expertise required when clinical circumstances make the higher-cost drug the better choice.^9,10 The second consideration is to minimize ethical tensions between cost displays that discourage use and a provider’s belief that a treatment is beneficial. Using available ethical frameworks for high-value care that engage both patient and societal concerns may help us choose and design interventions with more successful outcomes.¹¹ Finally, research has shown that providers have poor knowledge of both cost and the relative benefits and harms of treatments and testing.¹² Thus, the third opportunity for improvement is to provide appropriate clinical information (ie, relative therapeutic equivalency or adverse effects in alternative therapies) to support decision making at the point of order entry. Encouraging data already exists regarding how drug facts boxes can help patients understand benefits and side effects.¹³ A similar approach may aid physicians and may prove an easier task than improving patient understanding, given physicians’ substantial existing knowledge. These strategies may help guide providers to make a more informed value determination and obviate some ethical concerns related to clinical decisions based on cost alone. Despite their negative results, Conway et al.⁷ provided additional evidence that influencing complex decision making is not easy. However, we believe that continuing research into the factors that lead to successful value interventions has incredible potential for supporting high-value decision making in the future.

Disclosure 

Nothing to report.

In response to widely publicized reports highlighting the challenges of suboptimal quality of healthcare, improving patient safety has been a leading healthcare initiative for more than 10 years.^1-4 Numerous strategies to improve patient safety have been proposed,^5-9 but improvements have been limited, which raises questions about whether the right approaches are being employed.^10,11

Checklists have served as a foundation for the standardization and safety of aviation and nuclear power^12,13 and are advocated as simple and effective instruments for ensuring safe care.^7,14,15 Systematic reviews of observational studies suggest that checklists can reduce medical errors and adverse events,^15-19 but these reviews are at risk of bias due to the limitations of observational methods. Furthermore, discordant results of recent high-profile evaluations of the World Health Organization (WHO) Surgical Safety Checklist highlight the need for checklist evaluations using rigorous study designs.^20-22 Therefore, we sought to conduct a systematic review of RCTs (randomized controlled trials) to determine whether checklists, as a type of decision-support tool, are effective at improving patient safety outcomes in hospitalized patients.

The study protocol was registered with the PROSPERO Register of Systematic Reviews (registration number: CRD42016037441) and developed according to the Preferred Reporting Items in Systematic Reviews and Meta-analyses (PRISMA) statement.²³

Search Strategy

On December 8, 2016, we systematically searched Ovid MEDLINE, Ovid EMBASE, PubMed, and the Cochrane Central Register of Controlled Trials. The search was performed using no language or publication date restrictions and included 2 groups of terms (key words with similar characteristics): ‘checklists’ and ‘patient outcomes assessment’. We restricted our search to patient outcomes because these are more patient-oriented than the proximal processes of care that may not translate into outcomes. The search was restricted to RCTs using the Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials from the Cochrane Collaborative.²⁴ The MEDLINE search strategy is depicted in Appendix I (Supplementary File 1). Reference lists of included articles were manually searched for additional publications. The search strategy was designed with the help of an information scientist (DL). EndNote X7 (Thomas Reuters, Philadelphia, PA, USA) was the reference software used for the management of citations.

Eligibility Criteria

We selected all studies reporting patient safety outcomes of a checklist intervention, using the following inclusion criteria: 1) acute care hospital inpatient population, 2) checklist intervention, 3) contain a control group (ie, no checklist), 4) report one or more patient safety outcome, as defined by the authors (eg, medical errors, adverse events, mortality), and 5) RCT design. We restricted our focus to inpatient populations given the heterogeneity of illness and patient care between acute and community settings. We defined a checklist as a tool that details the essential steps of a task, requiring the target provider to indicate whether an item was completed or not.^1,7 Tools that included only 1 item (eg, electronic prompts) or did not require acknowledgement of the items (eg, guidelines) were excluded. We defined patient safety outcomes as the authors’ definition of patient safety (eg, medical error, adverse event, provider compliance with safety regulations).

Study Selection

Two reviewers (JMB, GW) independently, and in duplicate, reviewed the titles and abstracts of the retrieved citations against the eligibility criteria. The same 2 reviewers subsequently reviewed the full text of relevant articles for inclusion. Eligibility disagreements were resolved by consensus. A Kappa statistic was calculated for reviewer agreement of full-text screening.²⁵ Reviewers were not blinded to author or journal names.²⁶

Data Extraction

The structured data extraction form was calibrated using the first 2 articles. The 2 reviewers (JMB, GW) independently, and in duplicate, extracted data from included studies on the study characteristics, setting, study population, sample size, intervention used, outcomes examined, analytic method, and study quality. The data extraction form is depicted in Appendix II (Supplementary File 2). Coding discrepancies were resolved by consensus.

Quality Assessment

The 2 reviewers (JMB, GW) extracted data on study quality independently and in duplicate using 2 approaches. First, reviewers assessed study quality using a component method derived from the Cochrane Collaboration criteria.²⁴ For each included study, the reviewers documented if the authors had adequately described inclusion/exclusion criteria, randomization, allocation concealment, blinding of participants/outcome assessors, attrition, cross over, baseline characteristics, and power calculation. Second, the reviewers calculated and reported the Jadad score for each included study, a validated assessment scale that assigns points (1 to 5) based on randomization, blinding, and attrition.²⁷

Analysis

Owing to the heterogeneity of the data and the small number of studies that satisfied the inclusion criteria, the data were analyzed using guidelines for the narrative synthesis of a systematic review.²⁸ Descriptive statistical findings from each included study were reported. The DerSimonian and Laird method for random-effects models was used to calculate a pooled estimate of 30-day all-cause mortality from the raw data available from a subset of studies (number of events, study population).²⁹ Stata SE version 13.1 (Stata Corp, LP, College Station, TX) was used to perform the statistical analyses.

The literature search identified 11,225 unique citations from which 83 abstracts were eligible for full-text review. We identified 9 full-text articles for inclusion in the review (Figure 1 [Supplementary File 3]). The main reasons for citation exclusion during the full-text review were that the study design was not an RCT (39%) or there was no checklist intervention (34%). Inter-rater agreement for full-text inclusion was fair (K=0.660, 95% confidence interval[CI],0.414-0.828).

Study Characteristics

Characteristics of the included studies are summarized in Tables 1 and 2. Six of the studies were conducted in at least 1teaching hospital.^30-35 The studies varied in target populations for both the checklist user and patients. The outcomes reported varied; 3 studies examined 30-day mortality,^21,30,36 4 studies examined hospital length of stay,^21,30,33,36 and 2 studies reported user compliance with the checklist.^21,31 Five of the studies reported patient outcomes,^{21,30,33,35,36} and 5 studies reported provider-level outcomes related to patient safety (eg, compliance with checklist items such as communication of medications, isolation precautions, etc.).^31-34,37

Description of Checklists

Supplementary File 4 (Table 3) provides a detailed breakdown of the checklists’ purpose and components. Six of the checklists were designed to directly reduce patient safety events,^{21,30,33,35-37} whereas 3 of the checklists were designed to indirectly reduce patient safety events by increasing compliance with processes of care.^31,32,34 Six checklists were constructed and pilot tested by the research team conducting the RCT^30-35 and the 3 remaining studies used modified versions of previously validated checklists.^21,36,37 The number of items included in the checklist ranged from 2 to 54.

Impact of the Checklist

Table 4 summarizes the adverse events, medical errors, resource utilization and/or compliance reported for each checklist. Chaudhary et al. reported significant decreases in Grade III (requiring intervention)³⁸ and IV (life-threatening)³⁸ postoperative complications (23% v. 33%, P = 0.04) and 30-day mortality (5.7% vs 10.0%, P = 0.04) for patients assigned to the Modified WHO Surgical Safety Checklist compared to controls.²¹ Conversely, Haugen et al. reported a nonsignificant reduction in 30-day mortality between the WHO Surgical Safety Checklist group and controls (1.0% vs 1.6%, P = 0.151).³⁶ Bassor et al. reported no significant difference in 30-day hospital readmission for decompensated heart failure for the heart failure discharge checklist group when compared to controls (6% vs. 4%, P = NS); however, an exploratory analysis that excluded patients who died during the follow-up period found a significant difference in 30-day readmission rates (2% vs. 20%, P = 0.02).³⁰ Gentili et al. reported a higher proportion of patients with pain control in the checklist group compared to the controls (67.6% vs. 54.8%), as well as fewer incidents of analgesic therapy–related uncontrolled adverse events (25.9% vs. 49.9%); however, the statistical significance of these differences were not reported.³⁵ The Writing Group for CHECKLIST-ICU reported no significant difference for in-hospital mortality between the checklist and control groups (adjusted odds ratio [AOR] 1.02, 95% CI, 0.82-1.26, P = 0.88), nor for the secondary clinical outcomes examined (Table 4).³³ However, there was a significant difference between the checklist group and control group for 3 of the 7 outcomes related to processes of patient care, including a reduction in the use of both urinary catheters (adjusted rate ratio [ARR] 0.86, 95% CI, 0.80-0.93, P < 0.001) and central venous catheters (ARR 0.90, 95% CI 0.83-0.98, P = 0.02). Masson et al. reported that when using the FASTHUG-MAIDENS checklist, more drug-related problems were identified by pharmacy residents (in relation to the number identified by the ICU pharmacist) both per patient encounter (P = 0.008) and overall (P < 0.001).³⁷ Ong et al. reported higher rates of compliance with isolation precautions for infectious diseases in the checklist group (71% vs. 38%, P < 0.01); however, compliance with the checklist was low (40%) and qualitative analyses found participants were dissatisfied with the checklist.³¹ Salzwedel et al. reported the number of items handed over by anesthesia residents postoperatively to be higher in the checklist group than the control group (48.7% vs. 32.4%, P < 0.001).³² In a more recent study, Salzwedel et al. reported that proportion of items deemed by the attending anesthesiologist as “must be handed over” were more often actually handed over by the anesthesia residents assigned to the checklist group when compared to controls (87.1% vs. 75.0%, P = 0.005).³⁴

30-day Mortality

A random-effects model pooling data from the 3 studies that reported data for 30-day all cause mortality suggested a significant reduction with use of a checklist (OR 0.60, 95% CI, 0.41-0.89; P = 0.01, I² = 0.0%, P = 0.573).

Study Quality

Supplementary File 5 (Table 5) summarizes the quality assessment of the 9 studies. The clarity of description for each intervention varied. All studies reported inclusion/exclusion criteria and randomization procedures. Three studies indicated that outcome assessors were blinded to intervention allocation;^32,34,36 while this was unclear in 2 studies.^21,30 Three studies reported baseline characteristics.^21,30,36 Two studies reported power calculations;^33,37 however, one study had a sample size that was less than that required to achieve the target power.³⁷ The Jadad scores ranged from 1to 5.

This systematic review identified 9 RCTs that examined the impact of a checklist on patient safety outcomes in hospitalized patients. The studies employed checklists with different purposes and elements and measured different patient safety outcomes. The methodological quality of the included studies was moderate. In aggregate, the results suggest that checklists may be effective at improving patient safety outcomes, but the small number of moderate quality studies and the heterogeneity of interventions and outcome measures suggests that there is an urgent need for further evaluation.

The most important observation from our systematic review is the paucity of high quality evidence evaluating checklists’ impact on patient safety outcomes in acute inpatient care. The implementation of checklists is increasingly common as they are relatively low cost to develop and implement, and intuitively make sense. This is particularly true in an era of increasing efforts to standardize care as a means for improving quality and minimizing cost (ie, previous systematic reviews cite 38 unique studies).³⁹ However, implementation of an inadequately tested checklist risks unintended consequences (eg, inefficient resource utilization).¹⁸ The small number of RCTs identified might be owing to quality improvement efforts traditionally focusing on ‘real life’ applicability over rigorous research methodology.⁴⁰ The translation of evidence into clinical practice is known to be slow;⁴¹ however, these more rigorous methodologies reduce the risk of biases and generate high-quality evidence, which help to fulfill the necessity to identify best practices while avoiding these unintended consequences.

The studies varied both in the approaches used to develop checklists and in the number of items included (ranging from 2 to 54). What is the optimal method for developing a checklist and how does this impact their effectiveness?⁴² The answers to these questions are not known. However, this review highlights some important issues to consider when developing a checklist. As the number of items or complexity of a task increases, our ability to efficiently perform the task without aid decreases.^43-45 As such, a well-designed checklist should detail explicit instructions on the what, where, when, and how of a given task in a fashion that ensures a consistent accuracy for completing the work.⁵ It is recommended that construction of a checklist follow the principles of human factors engineering: engage stakeholders and human factors experts in the design; are developed based on user needs and realities; list items in order of importance; are concise and subgroup sections of checklists by task or chronological order; ensure usability and evaluate potential negative consequences (eg time to complete); are pilot tested and validated before implementation; are updated as needed based the on generation of new findings or changes in operational procedures.⁴⁶ These general principles of human factors engineering⁴⁶ provide a practical approach for the development and evaluation of a checklist. In addition, standardization of operational definitions (ie, process, outcome, compliance) is important for study replication and robust meta-analyses.

Checklists used in aviation are perhaps best known¹² and the evidence of their effectiveness is derived from the attribution of aviation errors to incomplete checklists.¹² Although more recently implemented in medicine, checklists have the potential to guide the successful completion of complex tasks in healthcare.⁷ Systematic reviews of observational studies have been conducted for specific checklists (eg, WHO Surgical Safety Checklist) and for select patient populations (eg, surgical patients), and the number of included studies ranges from 7-27 (n = 38 unique studies).^15,16,18,19 For example, Gillespie et al. in a systematic review and meta-analysis reported the implementation of Surgical Safety Checklists to be associated with a reduction in postoperative complications (relative risk [RR] 0.63, 95% CI, 0.58-0.72, P = < 0.001), but not mortality (RR 1.03, 95% CI, 0.73-1.4, P = 0.857).¹⁹ Similarly, Treadwell et al. reported in a systematic review of Surgical Safety Checklists that while data are promising, more evaluation of their impact on clinical outcomes is needed.¹⁸ These recommendations are nicely illustrated by Urbach et al.’s²⁰ and O’Leary et al.’s⁴⁷ evaluations of the mandatory adoption of Surgical Safety Checklists across all hospitals in Ontario, Canada, which respectively demonstrated no significant reductions in 30-day perioperatively conplications for both adult (OR 0.97, 95% CI, 0.90-1.03, P = 0.29) and pediatric (AOR 1.01, 95% CI, 0.90-1.14, P = 0.9) patients. These data not only highlight the need for further evaluation of checklists but are also a reminder that checklists and their associated implementation strategies are complex interventions for which there may be important differences between the efficacy reported in clinical trials and the effectiveness reported in implementation studies.⁴⁸ This all suggests that if checklists are to be effective in improving patient safety, process evaluations of implementation⁴⁹ and realist reviews of published studies⁵⁰ may be important to determine optimal approaches for implementation. We believe that, based on the limited currently available evidence, there is urgency for further robust evaluations of checklists before their widespread implementation. If effective, they should be widely implemented. If ineffective, they should be abandoned to minimize unintended consequences and inefficient use of resources.

There are 4 primary limitations to this review that should be considered when interpreting the findings. First, the RCT design is not the study design employed by most quality improvement initiatives.⁴⁰ While some quality improvement experts may argue that an RCT design is insufficiently flexible for applied settings, it does minimize the risk of biased assessments of intervention effectiveness. Second, our search strategy included an RCT filter. The filter helped restrict the number of citations to be reviewed (n = 11,225) but could have resulted in improperly indexed studies being excluded. To guard against this risk, we used the validated Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials,²⁴ reviewed reference lists of citations included in the review, and solicited suggestions for missing studies from quality improvement experts. Third, our review was restricted to hospitalized patients. Although the studies evaluated commonly reported safety outcomes across patients with diverse clinical conditions, care settings, and providers that broadly reflect hospital-based care, evaluations of checklists in additional patient and provider groups are needed (eg, hospitalists). Furthermore, the effectiveness of checklists for improving patient safety outcomes in outpatients is important; however, the organizational and patient characteristics of these 2 settings (hospitalized vs outpatient) are sufficiently different to warrant separate systematic reviews. Finally, owing to the heterogeneity of the checklists used and outcomes measured, we were unable to perform a robust meta-analysis. Heterogeneity, combined with the small number of studies identified in our search, prevented us from applying statistical methods to assess for publication bias. This limitation of our systematic review highlights an important gap in the literature and emphasizes the importance of additional primary research to evaluate checklists.

In summary, we identified few RCTs that examined checklists designed to improve patient safety outcomes. The small number of existing studies suggests that checklists may improve patient safety outcomes; however, these observations were not reported for all outcomes examined and the studies were heterogeneous and of limited methodological quality. There is an urgent need for high-quality evaluations of the effectiveness of patient safety checklists in inpatient healthcare settings to substantiate their perceived benefits.

Acknowledgments

We would like to thank Diane Lorenzetti for her help with the development of the search strategy.

Disclosure: The authors have no known conflicts of interest to declare.

Jamie Boyd was supported by a W21C – Alberta Innovates-Health Solutions (AIHS) Collaborative Research and Innovation Opportunities (CRIO) Health Services Research graduate studentship. Guosong Wu was supported by a Western Regional Training Centre (WRTC) for Health Services Research graduate studentship. Dr. Stelfox was supported by a Population Health Investigator Award from Alberta Innovates Health Solutions.

Authors’ Contributions

HTS was responsible for the study’s conception. All 3 authors contributed to the study’s design and interpretation. JB and GW were responsible for searching the literature, reviewing abstracts, selecting full-text articles and critically appraising them. All 3 authors performed the analyses. JB drafted the manuscript and all 3 authors assisted in the successive revisions of the final manuscript. All authors have read and approved the final manuscript.

In response to widely publicized reports highlighting the challenges of suboptimal quality of healthcare, improving patient safety has been a leading healthcare initiative for more than 10 years.^1-4 Numerous strategies to improve patient safety have been proposed,^5-9 but improvements have been limited, which raises questions about whether the right approaches are being employed.^10,11

Checklists have served as a foundation for the standardization and safety of aviation and nuclear power^12,13 and are advocated as simple and effective instruments for ensuring safe care.^7,14,15 Systematic reviews of observational studies suggest that checklists can reduce medical errors and adverse events,^15-19 but these reviews are at risk of bias due to the limitations of observational methods. Furthermore, discordant results of recent high-profile evaluations of the World Health Organization (WHO) Surgical Safety Checklist highlight the need for checklist evaluations using rigorous study designs.^20-22 Therefore, we sought to conduct a systematic review of RCTs (randomized controlled trials) to determine whether checklists, as a type of decision-support tool, are effective at improving patient safety outcomes in hospitalized patients.

The study protocol was registered with the PROSPERO Register of Systematic Reviews (registration number: CRD42016037441) and developed according to the Preferred Reporting Items in Systematic Reviews and Meta-analyses (PRISMA) statement.²³

Search Strategy

On December 8, 2016, we systematically searched Ovid MEDLINE, Ovid EMBASE, PubMed, and the Cochrane Central Register of Controlled Trials. The search was performed using no language or publication date restrictions and included 2 groups of terms (key words with similar characteristics): ‘checklists’ and ‘patient outcomes assessment’. We restricted our search to patient outcomes because these are more patient-oriented than the proximal processes of care that may not translate into outcomes. The search was restricted to RCTs using the Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials from the Cochrane Collaborative.²⁴ The MEDLINE search strategy is depicted in Appendix I (Supplementary File 1). Reference lists of included articles were manually searched for additional publications. The search strategy was designed with the help of an information scientist (DL). EndNote X7 (Thomas Reuters, Philadelphia, PA, USA) was the reference software used for the management of citations.

Eligibility Criteria

We selected all studies reporting patient safety outcomes of a checklist intervention, using the following inclusion criteria: 1) acute care hospital inpatient population, 2) checklist intervention, 3) contain a control group (ie, no checklist), 4) report one or more patient safety outcome, as defined by the authors (eg, medical errors, adverse events, mortality), and 5) RCT design. We restricted our focus to inpatient populations given the heterogeneity of illness and patient care between acute and community settings. We defined a checklist as a tool that details the essential steps of a task, requiring the target provider to indicate whether an item was completed or not.^1,7 Tools that included only 1 item (eg, electronic prompts) or did not require acknowledgement of the items (eg, guidelines) were excluded. We defined patient safety outcomes as the authors’ definition of patient safety (eg, medical error, adverse event, provider compliance with safety regulations).

Study Selection

Two reviewers (JMB, GW) independently, and in duplicate, reviewed the titles and abstracts of the retrieved citations against the eligibility criteria. The same 2 reviewers subsequently reviewed the full text of relevant articles for inclusion. Eligibility disagreements were resolved by consensus. A Kappa statistic was calculated for reviewer agreement of full-text screening.²⁵ Reviewers were not blinded to author or journal names.²⁶

Data Extraction

The structured data extraction form was calibrated using the first 2 articles. The 2 reviewers (JMB, GW) independently, and in duplicate, extracted data from included studies on the study characteristics, setting, study population, sample size, intervention used, outcomes examined, analytic method, and study quality. The data extraction form is depicted in Appendix II (Supplementary File 2). Coding discrepancies were resolved by consensus.

Quality Assessment

The 2 reviewers (JMB, GW) extracted data on study quality independently and in duplicate using 2 approaches. First, reviewers assessed study quality using a component method derived from the Cochrane Collaboration criteria.²⁴ For each included study, the reviewers documented if the authors had adequately described inclusion/exclusion criteria, randomization, allocation concealment, blinding of participants/outcome assessors, attrition, cross over, baseline characteristics, and power calculation. Second, the reviewers calculated and reported the Jadad score for each included study, a validated assessment scale that assigns points (1 to 5) based on randomization, blinding, and attrition.²⁷

Analysis

Owing to the heterogeneity of the data and the small number of studies that satisfied the inclusion criteria, the data were analyzed using guidelines for the narrative synthesis of a systematic review.²⁸ Descriptive statistical findings from each included study were reported. The DerSimonian and Laird method for random-effects models was used to calculate a pooled estimate of 30-day all-cause mortality from the raw data available from a subset of studies (number of events, study population).²⁹ Stata SE version 13.1 (Stata Corp, LP, College Station, TX) was used to perform the statistical analyses.

The literature search identified 11,225 unique citations from which 83 abstracts were eligible for full-text review. We identified 9 full-text articles for inclusion in the review (Figure 1 [Supplementary File 3]). The main reasons for citation exclusion during the full-text review were that the study design was not an RCT (39%) or there was no checklist intervention (34%). Inter-rater agreement for full-text inclusion was fair (K=0.660, 95% confidence interval[CI],0.414-0.828).

Study Characteristics

Characteristics of the included studies are summarized in Tables 1 and 2. Six of the studies were conducted in at least 1teaching hospital.^30-35 The studies varied in target populations for both the checklist user and patients. The outcomes reported varied; 3 studies examined 30-day mortality,^21,30,36 4 studies examined hospital length of stay,^21,30,33,36 and 2 studies reported user compliance with the checklist.^21,31 Five of the studies reported patient outcomes,^{21,30,33,35,36} and 5 studies reported provider-level outcomes related to patient safety (eg, compliance with checklist items such as communication of medications, isolation precautions, etc.).^31-34,37

Description of Checklists

Supplementary File 4 (Table 3) provides a detailed breakdown of the checklists’ purpose and components. Six of the checklists were designed to directly reduce patient safety events,^{21,30,33,35-37} whereas 3 of the checklists were designed to indirectly reduce patient safety events by increasing compliance with processes of care.^31,32,34 Six checklists were constructed and pilot tested by the research team conducting the RCT^30-35 and the 3 remaining studies used modified versions of previously validated checklists.^21,36,37 The number of items included in the checklist ranged from 2 to 54.

Impact of the Checklist

Table 4 summarizes the adverse events, medical errors, resource utilization and/or compliance reported for each checklist. Chaudhary et al. reported significant decreases in Grade III (requiring intervention)³⁸ and IV (life-threatening)³⁸ postoperative complications (23% v. 33%, P = 0.04) and 30-day mortality (5.7% vs 10.0%, P = 0.04) for patients assigned to the Modified WHO Surgical Safety Checklist compared to controls.²¹ Conversely, Haugen et al. reported a nonsignificant reduction in 30-day mortality between the WHO Surgical Safety Checklist group and controls (1.0% vs 1.6%, P = 0.151).³⁶ Bassor et al. reported no significant difference in 30-day hospital readmission for decompensated heart failure for the heart failure discharge checklist group when compared to controls (6% vs. 4%, P = NS); however, an exploratory analysis that excluded patients who died during the follow-up period found a significant difference in 30-day readmission rates (2% vs. 20%, P = 0.02).³⁰ Gentili et al. reported a higher proportion of patients with pain control in the checklist group compared to the controls (67.6% vs. 54.8%), as well as fewer incidents of analgesic therapy–related uncontrolled adverse events (25.9% vs. 49.9%); however, the statistical significance of these differences were not reported.³⁵ The Writing Group for CHECKLIST-ICU reported no significant difference for in-hospital mortality between the checklist and control groups (adjusted odds ratio [AOR] 1.02, 95% CI, 0.82-1.26, P = 0.88), nor for the secondary clinical outcomes examined (Table 4).³³ However, there was a significant difference between the checklist group and control group for 3 of the 7 outcomes related to processes of patient care, including a reduction in the use of both urinary catheters (adjusted rate ratio [ARR] 0.86, 95% CI, 0.80-0.93, P < 0.001) and central venous catheters (ARR 0.90, 95% CI 0.83-0.98, P = 0.02). Masson et al. reported that when using the FASTHUG-MAIDENS checklist, more drug-related problems were identified by pharmacy residents (in relation to the number identified by the ICU pharmacist) both per patient encounter (P = 0.008) and overall (P < 0.001).³⁷ Ong et al. reported higher rates of compliance with isolation precautions for infectious diseases in the checklist group (71% vs. 38%, P < 0.01); however, compliance with the checklist was low (40%) and qualitative analyses found participants were dissatisfied with the checklist.³¹ Salzwedel et al. reported the number of items handed over by anesthesia residents postoperatively to be higher in the checklist group than the control group (48.7% vs. 32.4%, P < 0.001).³² In a more recent study, Salzwedel et al. reported that proportion of items deemed by the attending anesthesiologist as “must be handed over” were more often actually handed over by the anesthesia residents assigned to the checklist group when compared to controls (87.1% vs. 75.0%, P = 0.005).³⁴

30-day Mortality

A random-effects model pooling data from the 3 studies that reported data for 30-day all cause mortality suggested a significant reduction with use of a checklist (OR 0.60, 95% CI, 0.41-0.89; P = 0.01, I² = 0.0%, P = 0.573).

Study Quality

Supplementary File 5 (Table 5) summarizes the quality assessment of the 9 studies. The clarity of description for each intervention varied. All studies reported inclusion/exclusion criteria and randomization procedures. Three studies indicated that outcome assessors were blinded to intervention allocation;^32,34,36 while this was unclear in 2 studies.^21,30 Three studies reported baseline characteristics.^21,30,36 Two studies reported power calculations;^33,37 however, one study had a sample size that was less than that required to achieve the target power.³⁷ The Jadad scores ranged from 1to 5.

This systematic review identified 9 RCTs that examined the impact of a checklist on patient safety outcomes in hospitalized patients. The studies employed checklists with different purposes and elements and measured different patient safety outcomes. The methodological quality of the included studies was moderate. In aggregate, the results suggest that checklists may be effective at improving patient safety outcomes, but the small number of moderate quality studies and the heterogeneity of interventions and outcome measures suggests that there is an urgent need for further evaluation.

The most important observation from our systematic review is the paucity of high quality evidence evaluating checklists’ impact on patient safety outcomes in acute inpatient care. The implementation of checklists is increasingly common as they are relatively low cost to develop and implement, and intuitively make sense. This is particularly true in an era of increasing efforts to standardize care as a means for improving quality and minimizing cost (ie, previous systematic reviews cite 38 unique studies).³⁹ However, implementation of an inadequately tested checklist risks unintended consequences (eg, inefficient resource utilization).¹⁸ The small number of RCTs identified might be owing to quality improvement efforts traditionally focusing on ‘real life’ applicability over rigorous research methodology.⁴⁰ The translation of evidence into clinical practice is known to be slow;⁴¹ however, these more rigorous methodologies reduce the risk of biases and generate high-quality evidence, which help to fulfill the necessity to identify best practices while avoiding these unintended consequences.

The studies varied both in the approaches used to develop checklists and in the number of items included (ranging from 2 to 54). What is the optimal method for developing a checklist and how does this impact their effectiveness?⁴² The answers to these questions are not known. However, this review highlights some important issues to consider when developing a checklist. As the number of items or complexity of a task increases, our ability to efficiently perform the task without aid decreases.^43-45 As such, a well-designed checklist should detail explicit instructions on the what, where, when, and how of a given task in a fashion that ensures a consistent accuracy for completing the work.⁵ It is recommended that construction of a checklist follow the principles of human factors engineering: engage stakeholders and human factors experts in the design; are developed based on user needs and realities; list items in order of importance; are concise and subgroup sections of checklists by task or chronological order; ensure usability and evaluate potential negative consequences (eg time to complete); are pilot tested and validated before implementation; are updated as needed based the on generation of new findings or changes in operational procedures.⁴⁶ These general principles of human factors engineering⁴⁶ provide a practical approach for the development and evaluation of a checklist. In addition, standardization of operational definitions (ie, process, outcome, compliance) is important for study replication and robust meta-analyses.

Checklists used in aviation are perhaps best known¹² and the evidence of their effectiveness is derived from the attribution of aviation errors to incomplete checklists.¹² Although more recently implemented in medicine, checklists have the potential to guide the successful completion of complex tasks in healthcare.⁷ Systematic reviews of observational studies have been conducted for specific checklists (eg, WHO Surgical Safety Checklist) and for select patient populations (eg, surgical patients), and the number of included studies ranges from 7-27 (n = 38 unique studies).^15,16,18,19 For example, Gillespie et al. in a systematic review and meta-analysis reported the implementation of Surgical Safety Checklists to be associated with a reduction in postoperative complications (relative risk [RR] 0.63, 95% CI, 0.58-0.72, P = < 0.001), but not mortality (RR 1.03, 95% CI, 0.73-1.4, P = 0.857).¹⁹ Similarly, Treadwell et al. reported in a systematic review of Surgical Safety Checklists that while data are promising, more evaluation of their impact on clinical outcomes is needed.¹⁸ These recommendations are nicely illustrated by Urbach et al.’s²⁰ and O’Leary et al.’s⁴⁷ evaluations of the mandatory adoption of Surgical Safety Checklists across all hospitals in Ontario, Canada, which respectively demonstrated no significant reductions in 30-day perioperatively conplications for both adult (OR 0.97, 95% CI, 0.90-1.03, P = 0.29) and pediatric (AOR 1.01, 95% CI, 0.90-1.14, P = 0.9) patients. These data not only highlight the need for further evaluation of checklists but are also a reminder that checklists and their associated implementation strategies are complex interventions for which there may be important differences between the efficacy reported in clinical trials and the effectiveness reported in implementation studies.⁴⁸ This all suggests that if checklists are to be effective in improving patient safety, process evaluations of implementation⁴⁹ and realist reviews of published studies⁵⁰ may be important to determine optimal approaches for implementation. We believe that, based on the limited currently available evidence, there is urgency for further robust evaluations of checklists before their widespread implementation. If effective, they should be widely implemented. If ineffective, they should be abandoned to minimize unintended consequences and inefficient use of resources.

There are 4 primary limitations to this review that should be considered when interpreting the findings. First, the RCT design is not the study design employed by most quality improvement initiatives.⁴⁰ While some quality improvement experts may argue that an RCT design is insufficiently flexible for applied settings, it does minimize the risk of biased assessments of intervention effectiveness. Second, our search strategy included an RCT filter. The filter helped restrict the number of citations to be reviewed (n = 11,225) but could have resulted in improperly indexed studies being excluded. To guard against this risk, we used the validated Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials,²⁴ reviewed reference lists of citations included in the review, and solicited suggestions for missing studies from quality improvement experts. Third, our review was restricted to hospitalized patients. Although the studies evaluated commonly reported safety outcomes across patients with diverse clinical conditions, care settings, and providers that broadly reflect hospital-based care, evaluations of checklists in additional patient and provider groups are needed (eg, hospitalists). Furthermore, the effectiveness of checklists for improving patient safety outcomes in outpatients is important; however, the organizational and patient characteristics of these 2 settings (hospitalized vs outpatient) are sufficiently different to warrant separate systematic reviews. Finally, owing to the heterogeneity of the checklists used and outcomes measured, we were unable to perform a robust meta-analysis. Heterogeneity, combined with the small number of studies identified in our search, prevented us from applying statistical methods to assess for publication bias. This limitation of our systematic review highlights an important gap in the literature and emphasizes the importance of additional primary research to evaluate checklists.

In summary, we identified few RCTs that examined checklists designed to improve patient safety outcomes. The small number of existing studies suggests that checklists may improve patient safety outcomes; however, these observations were not reported for all outcomes examined and the studies were heterogeneous and of limited methodological quality. There is an urgent need for high-quality evaluations of the effectiveness of patient safety checklists in inpatient healthcare settings to substantiate their perceived benefits.

Acknowledgments

We would like to thank Diane Lorenzetti for her help with the development of the search strategy.

Disclosure: The authors have no known conflicts of interest to declare.

Jamie Boyd was supported by a W21C – Alberta Innovates-Health Solutions (AIHS) Collaborative Research and Innovation Opportunities (CRIO) Health Services Research graduate studentship. Guosong Wu was supported by a Western Regional Training Centre (WRTC) for Health Services Research graduate studentship. Dr. Stelfox was supported by a Population Health Investigator Award from Alberta Innovates Health Solutions.

Authors’ Contributions

HTS was responsible for the study’s conception. All 3 authors contributed to the study’s design and interpretation. JB and GW were responsible for searching the literature, reviewing abstracts, selecting full-text articles and critically appraising them. All 3 authors performed the analyses. JB drafted the manuscript and all 3 authors assisted in the successive revisions of the final manuscript. All authors have read and approved the final manuscript.

In response to widely publicized reports highlighting the challenges of suboptimal quality of healthcare, improving patient safety has been a leading healthcare initiative for more than 10 years.^1-4 Numerous strategies to improve patient safety have been proposed,^5-9 but improvements have been limited, which raises questions about whether the right approaches are being employed.^10,11

Checklists have served as a foundation for the standardization and safety of aviation and nuclear power^12,13 and are advocated as simple and effective instruments for ensuring safe care.^7,14,15 Systematic reviews of observational studies suggest that checklists can reduce medical errors and adverse events,^15-19 but these reviews are at risk of bias due to the limitations of observational methods. Furthermore, discordant results of recent high-profile evaluations of the World Health Organization (WHO) Surgical Safety Checklist highlight the need for checklist evaluations using rigorous study designs.^20-22 Therefore, we sought to conduct a systematic review of RCTs (randomized controlled trials) to determine whether checklists, as a type of decision-support tool, are effective at improving patient safety outcomes in hospitalized patients.

The study protocol was registered with the PROSPERO Register of Systematic Reviews (registration number: CRD42016037441) and developed according to the Preferred Reporting Items in Systematic Reviews and Meta-analyses (PRISMA) statement.²³

Search Strategy

On December 8, 2016, we systematically searched Ovid MEDLINE, Ovid EMBASE, PubMed, and the Cochrane Central Register of Controlled Trials. The search was performed using no language or publication date restrictions and included 2 groups of terms (key words with similar characteristics): ‘checklists’ and ‘patient outcomes assessment’. We restricted our search to patient outcomes because these are more patient-oriented than the proximal processes of care that may not translate into outcomes. The search was restricted to RCTs using the Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials from the Cochrane Collaborative.²⁴ The MEDLINE search strategy is depicted in Appendix I (Supplementary File 1). Reference lists of included articles were manually searched for additional publications. The search strategy was designed with the help of an information scientist (DL). EndNote X7 (Thomas Reuters, Philadelphia, PA, USA) was the reference software used for the management of citations.

Eligibility Criteria

We selected all studies reporting patient safety outcomes of a checklist intervention, using the following inclusion criteria: 1) acute care hospital inpatient population, 2) checklist intervention, 3) contain a control group (ie, no checklist), 4) report one or more patient safety outcome, as defined by the authors (eg, medical errors, adverse events, mortality), and 5) RCT design. We restricted our focus to inpatient populations given the heterogeneity of illness and patient care between acute and community settings. We defined a checklist as a tool that details the essential steps of a task, requiring the target provider to indicate whether an item was completed or not.^1,7 Tools that included only 1 item (eg, electronic prompts) or did not require acknowledgement of the items (eg, guidelines) were excluded. We defined patient safety outcomes as the authors’ definition of patient safety (eg, medical error, adverse event, provider compliance with safety regulations).

Study Selection

Two reviewers (JMB, GW) independently, and in duplicate, reviewed the titles and abstracts of the retrieved citations against the eligibility criteria. The same 2 reviewers subsequently reviewed the full text of relevant articles for inclusion. Eligibility disagreements were resolved by consensus. A Kappa statistic was calculated for reviewer agreement of full-text screening.²⁵ Reviewers were not blinded to author or journal names.²⁶

Data Extraction

The structured data extraction form was calibrated using the first 2 articles. The 2 reviewers (JMB, GW) independently, and in duplicate, extracted data from included studies on the study characteristics, setting, study population, sample size, intervention used, outcomes examined, analytic method, and study quality. The data extraction form is depicted in Appendix II (Supplementary File 2). Coding discrepancies were resolved by consensus.

Quality Assessment

The 2 reviewers (JMB, GW) extracted data on study quality independently and in duplicate using 2 approaches. First, reviewers assessed study quality using a component method derived from the Cochrane Collaboration criteria.²⁴ For each included study, the reviewers documented if the authors had adequately described inclusion/exclusion criteria, randomization, allocation concealment, blinding of participants/outcome assessors, attrition, cross over, baseline characteristics, and power calculation. Second, the reviewers calculated and reported the Jadad score for each included study, a validated assessment scale that assigns points (1 to 5) based on randomization, blinding, and attrition.²⁷

Analysis

Owing to the heterogeneity of the data and the small number of studies that satisfied the inclusion criteria, the data were analyzed using guidelines for the narrative synthesis of a systematic review.²⁸ Descriptive statistical findings from each included study were reported. The DerSimonian and Laird method for random-effects models was used to calculate a pooled estimate of 30-day all-cause mortality from the raw data available from a subset of studies (number of events, study population).²⁹ Stata SE version 13.1 (Stata Corp, LP, College Station, TX) was used to perform the statistical analyses.

The literature search identified 11,225 unique citations from which 83 abstracts were eligible for full-text review. We identified 9 full-text articles for inclusion in the review (Figure 1 [Supplementary File 3]). The main reasons for citation exclusion during the full-text review were that the study design was not an RCT (39%) or there was no checklist intervention (34%). Inter-rater agreement for full-text inclusion was fair (K=0.660, 95% confidence interval[CI],0.414-0.828).

Study Characteristics

Characteristics of the included studies are summarized in Tables 1 and 2. Six of the studies were conducted in at least 1teaching hospital.^30-35 The studies varied in target populations for both the checklist user and patients. The outcomes reported varied; 3 studies examined 30-day mortality,^21,30,36 4 studies examined hospital length of stay,^21,30,33,36 and 2 studies reported user compliance with the checklist.^21,31 Five of the studies reported patient outcomes,^{21,30,33,35,36} and 5 studies reported provider-level outcomes related to patient safety (eg, compliance with checklist items such as communication of medications, isolation precautions, etc.).^31-34,37

Description of Checklists

Supplementary File 4 (Table 3) provides a detailed breakdown of the checklists’ purpose and components. Six of the checklists were designed to directly reduce patient safety events,^{21,30,33,35-37} whereas 3 of the checklists were designed to indirectly reduce patient safety events by increasing compliance with processes of care.^31,32,34 Six checklists were constructed and pilot tested by the research team conducting the RCT^30-35 and the 3 remaining studies used modified versions of previously validated checklists.^21,36,37 The number of items included in the checklist ranged from 2 to 54.

Impact of the Checklist

Table 4 summarizes the adverse events, medical errors, resource utilization and/or compliance reported for each checklist. Chaudhary et al. reported significant decreases in Grade III (requiring intervention)³⁸ and IV (life-threatening)³⁸ postoperative complications (23% v. 33%, P = 0.04) and 30-day mortality (5.7% vs 10.0%, P = 0.04) for patients assigned to the Modified WHO Surgical Safety Checklist compared to controls.²¹ Conversely, Haugen et al. reported a nonsignificant reduction in 30-day mortality between the WHO Surgical Safety Checklist group and controls (1.0% vs 1.6%, P = 0.151).³⁶ Bassor et al. reported no significant difference in 30-day hospital readmission for decompensated heart failure for the heart failure discharge checklist group when compared to controls (6% vs. 4%, P = NS); however, an exploratory analysis that excluded patients who died during the follow-up period found a significant difference in 30-day readmission rates (2% vs. 20%, P = 0.02).³⁰ Gentili et al. reported a higher proportion of patients with pain control in the checklist group compared to the controls (67.6% vs. 54.8%), as well as fewer incidents of analgesic therapy–related uncontrolled adverse events (25.9% vs. 49.9%); however, the statistical significance of these differences were not reported.³⁵ The Writing Group for CHECKLIST-ICU reported no significant difference for in-hospital mortality between the checklist and control groups (adjusted odds ratio [AOR] 1.02, 95% CI, 0.82-1.26, P = 0.88), nor for the secondary clinical outcomes examined (Table 4).³³ However, there was a significant difference between the checklist group and control group for 3 of the 7 outcomes related to processes of patient care, including a reduction in the use of both urinary catheters (adjusted rate ratio [ARR] 0.86, 95% CI, 0.80-0.93, P < 0.001) and central venous catheters (ARR 0.90, 95% CI 0.83-0.98, P = 0.02). Masson et al. reported that when using the FASTHUG-MAIDENS checklist, more drug-related problems were identified by pharmacy residents (in relation to the number identified by the ICU pharmacist) both per patient encounter (P = 0.008) and overall (P < 0.001).³⁷ Ong et al. reported higher rates of compliance with isolation precautions for infectious diseases in the checklist group (71% vs. 38%, P < 0.01); however, compliance with the checklist was low (40%) and qualitative analyses found participants were dissatisfied with the checklist.³¹ Salzwedel et al. reported the number of items handed over by anesthesia residents postoperatively to be higher in the checklist group than the control group (48.7% vs. 32.4%, P < 0.001).³² In a more recent study, Salzwedel et al. reported that proportion of items deemed by the attending anesthesiologist as “must be handed over” were more often actually handed over by the anesthesia residents assigned to the checklist group when compared to controls (87.1% vs. 75.0%, P = 0.005).³⁴

30-day Mortality

A random-effects model pooling data from the 3 studies that reported data for 30-day all cause mortality suggested a significant reduction with use of a checklist (OR 0.60, 95% CI, 0.41-0.89; P = 0.01, I² = 0.0%, P = 0.573).

Study Quality

Supplementary File 5 (Table 5) summarizes the quality assessment of the 9 studies. The clarity of description for each intervention varied. All studies reported inclusion/exclusion criteria and randomization procedures. Three studies indicated that outcome assessors were blinded to intervention allocation;^32,34,36 while this was unclear in 2 studies.^21,30 Three studies reported baseline characteristics.^21,30,36 Two studies reported power calculations;^33,37 however, one study had a sample size that was less than that required to achieve the target power.³⁷ The Jadad scores ranged from 1to 5.

This systematic review identified 9 RCTs that examined the impact of a checklist on patient safety outcomes in hospitalized patients. The studies employed checklists with different purposes and elements and measured different patient safety outcomes. The methodological quality of the included studies was moderate. In aggregate, the results suggest that checklists may be effective at improving patient safety outcomes, but the small number of moderate quality studies and the heterogeneity of interventions and outcome measures suggests that there is an urgent need for further evaluation.

The most important observation from our systematic review is the paucity of high quality evidence evaluating checklists’ impact on patient safety outcomes in acute inpatient care. The implementation of checklists is increasingly common as they are relatively low cost to develop and implement, and intuitively make sense. This is particularly true in an era of increasing efforts to standardize care as a means for improving quality and minimizing cost (ie, previous systematic reviews cite 38 unique studies).³⁹ However, implementation of an inadequately tested checklist risks unintended consequences (eg, inefficient resource utilization).¹⁸ The small number of RCTs identified might be owing to quality improvement efforts traditionally focusing on ‘real life’ applicability over rigorous research methodology.⁴⁰ The translation of evidence into clinical practice is known to be slow;⁴¹ however, these more rigorous methodologies reduce the risk of biases and generate high-quality evidence, which help to fulfill the necessity to identify best practices while avoiding these unintended consequences.

The studies varied both in the approaches used to develop checklists and in the number of items included (ranging from 2 to 54). What is the optimal method for developing a checklist and how does this impact their effectiveness?⁴² The answers to these questions are not known. However, this review highlights some important issues to consider when developing a checklist. As the number of items or complexity of a task increases, our ability to efficiently perform the task without aid decreases.^43-45 As such, a well-designed checklist should detail explicit instructions on the what, where, when, and how of a given task in a fashion that ensures a consistent accuracy for completing the work.⁵ It is recommended that construction of a checklist follow the principles of human factors engineering: engage stakeholders and human factors experts in the design; are developed based on user needs and realities; list items in order of importance; are concise and subgroup sections of checklists by task or chronological order; ensure usability and evaluate potential negative consequences (eg time to complete); are pilot tested and validated before implementation; are updated as needed based the on generation of new findings or changes in operational procedures.⁴⁶ These general principles of human factors engineering⁴⁶ provide a practical approach for the development and evaluation of a checklist. In addition, standardization of operational definitions (ie, process, outcome, compliance) is important for study replication and robust meta-analyses.

Checklists used in aviation are perhaps best known¹² and the evidence of their effectiveness is derived from the attribution of aviation errors to incomplete checklists.¹² Although more recently implemented in medicine, checklists have the potential to guide the successful completion of complex tasks in healthcare.⁷ Systematic reviews of observational studies have been conducted for specific checklists (eg, WHO Surgical Safety Checklist) and for select patient populations (eg, surgical patients), and the number of included studies ranges from 7-27 (n = 38 unique studies).^15,16,18,19 For example, Gillespie et al. in a systematic review and meta-analysis reported the implementation of Surgical Safety Checklists to be associated with a reduction in postoperative complications (relative risk [RR] 0.63, 95% CI, 0.58-0.72, P = < 0.001), but not mortality (RR 1.03, 95% CI, 0.73-1.4, P = 0.857).¹⁹ Similarly, Treadwell et al. reported in a systematic review of Surgical Safety Checklists that while data are promising, more evaluation of their impact on clinical outcomes is needed.¹⁸ These recommendations are nicely illustrated by Urbach et al.’s²⁰ and O’Leary et al.’s⁴⁷ evaluations of the mandatory adoption of Surgical Safety Checklists across all hospitals in Ontario, Canada, which respectively demonstrated no significant reductions in 30-day perioperatively conplications for both adult (OR 0.97, 95% CI, 0.90-1.03, P = 0.29) and pediatric (AOR 1.01, 95% CI, 0.90-1.14, P = 0.9) patients. These data not only highlight the need for further evaluation of checklists but are also a reminder that checklists and their associated implementation strategies are complex interventions for which there may be important differences between the efficacy reported in clinical trials and the effectiveness reported in implementation studies.⁴⁸ This all suggests that if checklists are to be effective in improving patient safety, process evaluations of implementation⁴⁹ and realist reviews of published studies⁵⁰ may be important to determine optimal approaches for implementation. We believe that, based on the limited currently available evidence, there is urgency for further robust evaluations of checklists before their widespread implementation. If effective, they should be widely implemented. If ineffective, they should be abandoned to minimize unintended consequences and inefficient use of resources.

There are 4 primary limitations to this review that should be considered when interpreting the findings. First, the RCT design is not the study design employed by most quality improvement initiatives.⁴⁰ While some quality improvement experts may argue that an RCT design is insufficiently flexible for applied settings, it does minimize the risk of biased assessments of intervention effectiveness. Second, our search strategy included an RCT filter. The filter helped restrict the number of citations to be reviewed (n = 11,225) but could have resulted in improperly indexed studies being excluded. To guard against this risk, we used the validated Cochrane Highly Sensitive Search Strategy for Identifying Randomized Trials,²⁴ reviewed reference lists of citations included in the review, and solicited suggestions for missing studies from quality improvement experts. Third, our review was restricted to hospitalized patients. Although the studies evaluated commonly reported safety outcomes across patients with diverse clinical conditions, care settings, and providers that broadly reflect hospital-based care, evaluations of checklists in additional patient and provider groups are needed (eg, hospitalists). Furthermore, the effectiveness of checklists for improving patient safety outcomes in outpatients is important; however, the organizational and patient characteristics of these 2 settings (hospitalized vs outpatient) are sufficiently different to warrant separate systematic reviews. Finally, owing to the heterogeneity of the checklists used and outcomes measured, we were unable to perform a robust meta-analysis. Heterogeneity, combined with the small number of studies identified in our search, prevented us from applying statistical methods to assess for publication bias. This limitation of our systematic review highlights an important gap in the literature and emphasizes the importance of additional primary research to evaluate checklists.

In summary, we identified few RCTs that examined checklists designed to improve patient safety outcomes. The small number of existing studies suggests that checklists may improve patient safety outcomes; however, these observations were not reported for all outcomes examined and the studies were heterogeneous and of limited methodological quality. There is an urgent need for high-quality evaluations of the effectiveness of patient safety checklists in inpatient healthcare settings to substantiate their perceived benefits.

Acknowledgments

We would like to thank Diane Lorenzetti for her help with the development of the search strategy.

Disclosure: The authors have no known conflicts of interest to declare.

Jamie Boyd was supported by a W21C – Alberta Innovates-Health Solutions (AIHS) Collaborative Research and Innovation Opportunities (CRIO) Health Services Research graduate studentship. Guosong Wu was supported by a Western Regional Training Centre (WRTC) for Health Services Research graduate studentship. Dr. Stelfox was supported by a Population Health Investigator Award from Alberta Innovates Health Solutions.

Authors’ Contributions

HTS was responsible for the study’s conception. All 3 authors contributed to the study’s design and interpretation. JB and GW were responsible for searching the literature, reviewing abstracts, selecting full-text articles and critically appraising them. All 3 authors performed the analyses. JB drafted the manuscript and all 3 authors assisted in the successive revisions of the final manuscript. All authors have read and approved the final manuscript.

The approach to clinical conundrums by an expert clinician is revealed through the presentation of an actual patient’s case in an approach typical of a morning report. Similarly to patient care, sequential pieces of information are provided to the clinician, who is unfamiliar with the case. The focus is on the thought processes of both the clinical team caring for the patient and the discussant.

A 78-year-old woman presented to her primary care physician with a 2-month history of progressive leg weakness. She reported walking difficulty caused by occasional “buckling” of the knees.

The knee buckling may be a clue to the quadriceps muscle weakness. The quadriceps straightens and locks the knee when the foot is being planted. Weakness of this muscle can result in the knee giving way. Isolated quadriceps weakness, which is uncommon, typically is caused by lower motor neuron issues, such as femoral neuropathy, L4–L5 radiculopathy, lumbosacral plexopathy, and primary muscle diseases, including inclusion body myositis.

The patient had diabetes mellitus and hypertension. Her medications were insulin glargine, metformin, glipizide, lisinopril, atorvastatin, and aspirin, and she was taking vitamin D and calcium. None of these was recently changed or added. Aside from having the weakness, the patient was in her usual state of health and had no other complaints. She denied weight changes, fevers, night sweats, and fatigue. She was widowed, lived with her daughter, had no pets, never used tobacco, and did not drink alcohol or use illicit drugs. There was no family history of neuromuscular disorders, and both of her parents died of natural causes at advanced ages.

The physical examination revealed no knee deformities, and the patient had good active range of motion of both knees and normal strength throughout her limbs. Plain radiographs of the knees showed mild medial compartment osteoarthritis. The patient was advised to stop atorvastatin.

Patients who take atorvastatin and other statins (3-hydroxy-3-methyl-glutaryl-co-enzyme A reductase inhibitors) can experience a spectrum of muscle disease, from myalgias and weakness to (rare) overt myositis with rhabdomyolysis. Statin-induced myopathy tends to affect larger proximal muscles, can occur at any time during the period the medication is being used, and usually resolves within weeks after discontinuation. Given this patient’s preserved strength, it was reasonable to manage her conservatively.

One month later, she presented to another hospital’s emergency department with increasing weakness in the lower extremities and new loss of balance requiring use of a walker for ambulation. She thought the weakness was confined to her legs and was affecting her thigh muscles more than her calves or feet. She reported fatigue, decreased appetite, and an unintentional 15-pound weight loss. She denied diarrhea, back pain, bowel and bladder function problems, sensation changes, myalgias, and arthralgias. She reported no swallowing or vision problems, rashes, Raynaud disease symptoms, photosensitivity, dry eyes or mouth, recent falls or trauma, fevers, night sweats, recent illness, or travel.

On physical examination, the patient’s temperature was 98.2°F, blood pressure 120/84 mm Hg, pulse 73 beats per minute, respiratory rate 16 breaths per minute, and oxygen saturation 99% with ambient air. The patient was obese and not in distress. She was alert, oriented, and able to follow multistep instructions. Cranial nerve examination was normal. The patient had mild weakness in her bilateral deltoids and bilateral hip flexors but full strength in all other muscle groups. Deep tendon reflexes were normal in the biceps and patella and reduced in the ankles. The Babinski sign was absent. Throughout the lower extremities, sensation was intact to light touch; there was no saddle anesthesia. Finger–nose–finger testing showed slight dysmetria in the left upper extremity. Because of her imbalance, the patient needed help to stand up; once upright, though, she was able to take 3 steps forward and backward with use of a walker. Her stride length was diminished, and her gait unsteady and wide based.

These exam findings suggest 2 separate localizations. Symmetric proximal weakness with preserved distal strength in the extremities indicate a lower motor neuron problem, further suggested by the absence of upper motor neuron findings of spasticity, and hyperreflexia. Intact patellar and biceps reflexes suggest either muscle or neuromuscular junction localization. Separately, the findings of wide-based gait and dysmetria raise the possibility of a problem in the ipsilateral cerebellum, in this case on the left. A cerebellar lesion would not explain the bilateral symmetric weakness. The combination of lower motor neuron leg weakness and cerebellar deficits suggests a systemic disorder, such as an infectious, inflammatory, or paraneoplastic process. A space-occupying lesion would not produce this constellation of deficits.

Serum chemistry panel was normal, creatinine level 0.47 mg/dL, and albumin level 4.0 g/dL. White blood cell (WBC) count was 8100/mm³, hemoglobin level 12 g/dL, and platelet count 287,000/mm³. Alanine aminotransferase (ALT) level was 74 U/L (reference range, 0-36 U/L), alkaline phosphatase level 41 U/L (reference range, 37-117 U/L), and total bilirubin level 0.4 mg/dL (reference range, 0.2-1.2 mg/dL). Prothrombin time and thyrotropin were normal. Creatine kinase (CK) level was 2328 U/L (reference range, <200 U/L). Erythrocyte sedimentation rate was 17 mm/h, and C-reactive protein level 0.1 mg/L. Urinalysis (dipstick testing) detected no myoglobin, and there were no casts. Plain radiograph of the chest was normal.

The elevated CK indicates muscle disease, and, in the absence of other findings of liver disease, the ALT elevation likely has a muscle origin as well. The differential diagnosis for elevated CK includes myopathy caused either by infection, trauma, ischemia, or a toxin (medication included) or by a hereditary, metabolic, endocrinologic, or inflammatory disorder. There is no history of trauma, strenuous exertion, or muscle toxin other than the statin, and the progression of symptoms after medication discontinuation argues against statin myopathy. The laboratory test results rule out derangement of potassium, calcium, phosphorus, magnesium, vitamin D, or thyroid function as the cause of the myopathy. The absence of fever, absence of diffuse organ involvement, and normal inflammatory markers point away from a systemic infection or vasculitis. The inflammatory myopathies dermatomyositis and polymyositis classically produce proximal muscle weakness and are possibilities in this case, but one would expect the inflammatory markers to be elevated in these conditions. Malignancy related to dermatomyositis or to paraneoplastic syndrome may account for the myopathy.

The data provided do not identify a unifying diagnosis. To look for an inflammatory myopathy, such as dermatomyositis or polymyositis, it is reasonable to perform electromyography (EMG) to delineate the location of muscle involvement and identify a site for tissue biopsy. As no obvious toxins or metabolic conditions explain the dysmetria, magnetic resonance imaging (MRI) of the brain should be performed to evaluate for lesions in the cerebellum.

The patient was admitted to the hospital. On T2-weighted and FLAIR (fluid attenuation inversion recovery) sequences, MRI of the brain showed a few scattered subcortical and periventricular white matter hyperintense foci bilaterally. Antibodies to human immunodeficiency virus 1 and 2, and Treponema pallidum immunoglobulins G and M, were not detected. Serum levels of lactate dehydrogenase, vitamin B ₁₂ , angiotensin-converting enzyme, antinuclear antibody, rheumatoid factor, and anti–cyclic citrullinated peptide IgG were normal.

The brain imaging excludes a space-occupying lesion in the cerebellum but does not identify the cause of dysmetria. Toxic-metabolic conditions, such as alcohol toxicity, vitamin B₁₂ deficiency, anoxia, and toxicity of certain medications, may impair cerebellar function (MRI findings may be normal), but none of these is present. Other disorders that attack the central nervous system (CNS) (again, brain imaging may show minimal abnormalities) include infections, early-stage neurodegenerative illnesses, and antibody-associated disorders (eg, autoimmune diseases, postinfectious and paraneoplastic conditions).

Four days after intravenous fluids were started, the patient’s CK level improved, but her weakness persisted. There was no evidence of peripheral neuropathy on lower extremity nerve conduction studies. EMG revealed few fibrillations and positive sharp waves in proximal limb muscles and thoracic paraspinal muscles. Deltoid, biceps, and tensor fasciae latae showed shorter duration complex motor units with early recruitment. The patient declined muscle biopsy. A rheumatologist was consulted, and prednisone 60 mg/d was started for possible inflammatory myopathy. The patient was discharged to a skilled nursing facility for physical therapy.

The fibrillations and positive sharp waves on EMG can be seen in both neuropathy (from denervation) and myopathy. The normal nerve conduction studies make localization to the nerve unlikely. In addition, the shorter duration motor units with early recruitment on EMG are characteristic of a myopathy. Despite the ongoing myopathy, the improved CK level suggests the muscle disease is playing a minimal role in the patient’s current illness. Prescribing corticosteroids for a presumed inflammatory myopathy without a clear diagnosis is risky, as steroids may render subsequent biopsy results unreliable, may themselves cause myopathy, and expose the patient to the side effects of immunosuppression.

One month later, the patient saw her rheumatologist. Although she had tapered the prednisone down to 10 mg/d, she had not returned to baseline strength, was still using a walker, and reported increased difficulty keeping her head raised. She also noted 2 new symptoms: speech slurring and, in both hands, a tremor that made it difficult to hold objects.

Examination revealed a fine tremor in both arms. There were no skin lesions, and the joints were normal. The patient was oriented to name, place, and date. Memory testing was 3 for 3 on register but 0 for 3 on recall. She was unable to perform serial 7s and able to spell backward only 3 of the 5 letters in the word world . Speech was dysarthric and scanning in quality. On extraocular movements, she demonstrated poor smooth pursuit. Examination of the head and neck was significant for nearly constant head titubation and weak neck flexors. Upper extremity strength was normal. Mild weakness was noted in both hip flexors. Deep tendon reflexes were preserved except at the ankle, where they were reduced. Finger–nose–finger testing revealed marked dysmetria, more pronounced on the left, and there was mild bilateral heel-to-shin dysmetria.

Diffuse myopathy cannot account for the patient’s impaired cognition or progressive cerebellar findings, which now include head titubation and scanning speech. As more than a month has elapsed since the brain imaging was performed, MRI could be repeated for evidence of infection, malignancy, inflammation, or demyelination. More important, lumbar puncture is indicated to exclude infection and, with flow cytometry, cytology, and measurement of oligoclonal bands and IgG index, to assess for autoimmune or paraneoplastic antibody-mediated disorders.

The patient was readmitted. On repeat brain MRI, there were no new significant findings. Complete blood cell count and chemistry panel results were unremarkable. Erythrocyte sedimentation rate and C-reactive protein level remained normal. CK level was 451 U/L, and ALT level 29 U/L.

Lumbar puncture was performed. Opening pressure was 14.5 cm of water, and cerebrospinal fluid (CSF) was clear and colorless. There were 3 red blood cells/mm ³ and no WBCs. Glucose level was 94 mg/dL, and protein level 74 mg/dL. CSF IgG synthesis rate was normal, flow cytometry revealed no abnormal clonal populations, and cytology was negative for malignancy. Two unique oligoclonal bands were found in the CSF.

The absence of WBCs in the CSF excludes CNS infection. The patient’s main problem is an inflammatory CNS process as defined by presence of oligoclonal bands in the CSF, compared with their absence in the serum. Autoimmune, neoplastic, and paraneoplastic disorders could explain these bands. There was no evidence of systemic autoimmune illness. The patient has not had a recent infection that could result in postinfectious demyelination, and her clinical and imaging features are not suggestive of a demyelinating disorder, such as multiple sclerosis. Of the neoplastic possibilities, lymphoma with CNS involvement may be difficult to detect initially; this diagnosis, however, is not supported by the unremarkable MRI, flow cytometry, and cytology findings. In paraneoplastic syndromes, the CSF may include antibodies that react to antigens in the brain or cerebellum.

At this point, evaluation for malignancy should involve mammography, imaging of the chest, abdomen, and pelvis, and colorectal screening. Testing should also include measurement of serum and CSF autoantibodies associated with paraneoplastic cerebellar degeneration. The expanding list of paraneoplastic antibodies that may attack the cerebellum includes anti-Hu (often associated with small cell lung cancer), anti-Yo (associated with ovarian or breast cancer), anti-aquaporin 4, antibodies to the voltage-gated potassium channel, and anti–glutamic acid decarboxylase (anti-GAD).

Mammography and breast examination findings were normal. Computed tomography (CT) of the chest showed no adenopathy, nodules, or masses. Abdomen CT showed nonspecific prominence of the gallbladder wall. Flexible sigmoidoscopy revealed no masses, only thickened folds in the sigmoid colon; results of multiple colon biopsy tests were normal. Carcinoembryonic antigen level was 2.0 μg/L, and CA-125 level 5 U/mL. Serum GAD-65 antibodies were elevated (>30 nmol/L).

Anti-GAD is mostly known as the antibody associated with type 1 diabetes mellitus (T1DM). In rare instances, even in patients without a history of diabetes, anti-GAD antibodies may lead to an autoimmune attack on the brain, particularly the cerebellum, as part of an idiopathic autoimmune disorder or as a paraneoplastic syndrome. In either case, treatment involves corticosteroids, intravenous Ig, or plasma exchange. When the autoimmune attack is associated with malignancy, treatment response is poorer, unless the malignancy is successfully managed. The next steps are intravenous Ig or plasma exchange and positron emission tomography–CT (PET-CT) assessing for an underlying neoplasm that may have been too small to be detected with routine CT.

Numerous antibodies to antigens of the nervous system have been identified in association with malignancy, resulting in a variety of paraneoplastic neurologic syndromes (PNSs). Collectively, these syndromes are seen in fewer than 1 in 10,000 patients with cancer. The malignancy may be diagnosed months to years before the paraneoplastic syndrome, or the syndrome may be the first clue to underlying malignancy.¹ Hospitalists should suspect a PNS if the patient has known cancer and has unexplained neurologic deficits, if the presentation fits a classic PNS pattern, or if evaluation for neurologic symptoms identifies an inflammatory CSF profile that cannot be explained by demyelinating disorder or infection.

• The differential diagnosis for cerebellar deficits associated with normal brain MRI includes infection, toxic-metabolic insults (alcohol toxicity, vitamin B₁₂ deficiency, medication toxicity), anoxia, early neurodegenerative illness, and antibody-mediated disorders, such as autoimmune, postinfectious, and paraneoplastic syndromes.
• Hospitalists should suspect a PNS when a patient with known cancer develops unexplained neurologic deficits or when evaluation for neurologic symptoms identifies an inflammatory CSF profile that cannot be explained by a demyelinating disorder or an infection.
• Hospitalists should familiarize themselves with the classic PNS presentations, including limbic encephalitis, cerebellar degeneration, stiff person syndrome, opsoclonus-myoclonus, NMDA receptor encephalitis, and encephalomyelitis.
• Suspicion for PNS may be confirmed by the presence of paraneoplastic antibodies in CSF or serum. When routine evaluation fails to identify cancer, PET-CT should be performed.

Disclosure

Nothing to report.

The approach to clinical conundrums by an expert clinician is revealed through the presentation of an actual patient’s case in an approach typical of a morning report. Similarly to patient care, sequential pieces of information are provided to the clinician, who is unfamiliar with the case. The focus is on the thought processes of both the clinical team caring for the patient and the discussant.

A 78-year-old woman presented to her primary care physician with a 2-month history of progressive leg weakness. She reported walking difficulty caused by occasional “buckling” of the knees.

The knee buckling may be a clue to the quadriceps muscle weakness. The quadriceps straightens and locks the knee when the foot is being planted. Weakness of this muscle can result in the knee giving way. Isolated quadriceps weakness, which is uncommon, typically is caused by lower motor neuron issues, such as femoral neuropathy, L4–L5 radiculopathy, lumbosacral plexopathy, and primary muscle diseases, including inclusion body myositis.

The patient had diabetes mellitus and hypertension. Her medications were insulin glargine, metformin, glipizide, lisinopril, atorvastatin, and aspirin, and she was taking vitamin D and calcium. None of these was recently changed or added. Aside from having the weakness, the patient was in her usual state of health and had no other complaints. She denied weight changes, fevers, night sweats, and fatigue. She was widowed, lived with her daughter, had no pets, never used tobacco, and did not drink alcohol or use illicit drugs. There was no family history of neuromuscular disorders, and both of her parents died of natural causes at advanced ages.

The physical examination revealed no knee deformities, and the patient had good active range of motion of both knees and normal strength throughout her limbs. Plain radiographs of the knees showed mild medial compartment osteoarthritis. The patient was advised to stop atorvastatin.

Patients who take atorvastatin and other statins (3-hydroxy-3-methyl-glutaryl-co-enzyme A reductase inhibitors) can experience a spectrum of muscle disease, from myalgias and weakness to (rare) overt myositis with rhabdomyolysis. Statin-induced myopathy tends to affect larger proximal muscles, can occur at any time during the period the medication is being used, and usually resolves within weeks after discontinuation. Given this patient’s preserved strength, it was reasonable to manage her conservatively.

One month later, she presented to another hospital’s emergency department with increasing weakness in the lower extremities and new loss of balance requiring use of a walker for ambulation. She thought the weakness was confined to her legs and was affecting her thigh muscles more than her calves or feet. She reported fatigue, decreased appetite, and an unintentional 15-pound weight loss. She denied diarrhea, back pain, bowel and bladder function problems, sensation changes, myalgias, and arthralgias. She reported no swallowing or vision problems, rashes, Raynaud disease symptoms, photosensitivity, dry eyes or mouth, recent falls or trauma, fevers, night sweats, recent illness, or travel.

On physical examination, the patient’s temperature was 98.2°F, blood pressure 120/84 mm Hg, pulse 73 beats per minute, respiratory rate 16 breaths per minute, and oxygen saturation 99% with ambient air. The patient was obese and not in distress. She was alert, oriented, and able to follow multistep instructions. Cranial nerve examination was normal. The patient had mild weakness in her bilateral deltoids and bilateral hip flexors but full strength in all other muscle groups. Deep tendon reflexes were normal in the biceps and patella and reduced in the ankles. The Babinski sign was absent. Throughout the lower extremities, sensation was intact to light touch; there was no saddle anesthesia. Finger–nose–finger testing showed slight dysmetria in the left upper extremity. Because of her imbalance, the patient needed help to stand up; once upright, though, she was able to take 3 steps forward and backward with use of a walker. Her stride length was diminished, and her gait unsteady and wide based.

These exam findings suggest 2 separate localizations. Symmetric proximal weakness with preserved distal strength in the extremities indicate a lower motor neuron problem, further suggested by the absence of upper motor neuron findings of spasticity, and hyperreflexia. Intact patellar and biceps reflexes suggest either muscle or neuromuscular junction localization. Separately, the findings of wide-based gait and dysmetria raise the possibility of a problem in the ipsilateral cerebellum, in this case on the left. A cerebellar lesion would not explain the bilateral symmetric weakness. The combination of lower motor neuron leg weakness and cerebellar deficits suggests a systemic disorder, such as an infectious, inflammatory, or paraneoplastic process. A space-occupying lesion would not produce this constellation of deficits.

Serum chemistry panel was normal, creatinine level 0.47 mg/dL, and albumin level 4.0 g/dL. White blood cell (WBC) count was 8100/mm³, hemoglobin level 12 g/dL, and platelet count 287,000/mm³. Alanine aminotransferase (ALT) level was 74 U/L (reference range, 0-36 U/L), alkaline phosphatase level 41 U/L (reference range, 37-117 U/L), and total bilirubin level 0.4 mg/dL (reference range, 0.2-1.2 mg/dL). Prothrombin time and thyrotropin were normal. Creatine kinase (CK) level was 2328 U/L (reference range, <200 U/L). Erythrocyte sedimentation rate was 17 mm/h, and C-reactive protein level 0.1 mg/L. Urinalysis (dipstick testing) detected no myoglobin, and there were no casts. Plain radiograph of the chest was normal.

The elevated CK indicates muscle disease, and, in the absence of other findings of liver disease, the ALT elevation likely has a muscle origin as well. The differential diagnosis for elevated CK includes myopathy caused either by infection, trauma, ischemia, or a toxin (medication included) or by a hereditary, metabolic, endocrinologic, or inflammatory disorder. There is no history of trauma, strenuous exertion, or muscle toxin other than the statin, and the progression of symptoms after medication discontinuation argues against statin myopathy. The laboratory test results rule out derangement of potassium, calcium, phosphorus, magnesium, vitamin D, or thyroid function as the cause of the myopathy. The absence of fever, absence of diffuse organ involvement, and normal inflammatory markers point away from a systemic infection or vasculitis. The inflammatory myopathies dermatomyositis and polymyositis classically produce proximal muscle weakness and are possibilities in this case, but one would expect the inflammatory markers to be elevated in these conditions. Malignancy related to dermatomyositis or to paraneoplastic syndrome may account for the myopathy.

The data provided do not identify a unifying diagnosis. To look for an inflammatory myopathy, such as dermatomyositis or polymyositis, it is reasonable to perform electromyography (EMG) to delineate the location of muscle involvement and identify a site for tissue biopsy. As no obvious toxins or metabolic conditions explain the dysmetria, magnetic resonance imaging (MRI) of the brain should be performed to evaluate for lesions in the cerebellum.

The patient was admitted to the hospital. On T2-weighted and FLAIR (fluid attenuation inversion recovery) sequences, MRI of the brain showed a few scattered subcortical and periventricular white matter hyperintense foci bilaterally. Antibodies to human immunodeficiency virus 1 and 2, and Treponema pallidum immunoglobulins G and M, were not detected. Serum levels of lactate dehydrogenase, vitamin B ₁₂ , angiotensin-converting enzyme, antinuclear antibody, rheumatoid factor, and anti–cyclic citrullinated peptide IgG were normal.

The brain imaging excludes a space-occupying lesion in the cerebellum but does not identify the cause of dysmetria. Toxic-metabolic conditions, such as alcohol toxicity, vitamin B₁₂ deficiency, anoxia, and toxicity of certain medications, may impair cerebellar function (MRI findings may be normal), but none of these is present. Other disorders that attack the central nervous system (CNS) (again, brain imaging may show minimal abnormalities) include infections, early-stage neurodegenerative illnesses, and antibody-associated disorders (eg, autoimmune diseases, postinfectious and paraneoplastic conditions).

Four days after intravenous fluids were started, the patient’s CK level improved, but her weakness persisted. There was no evidence of peripheral neuropathy on lower extremity nerve conduction studies. EMG revealed few fibrillations and positive sharp waves in proximal limb muscles and thoracic paraspinal muscles. Deltoid, biceps, and tensor fasciae latae showed shorter duration complex motor units with early recruitment. The patient declined muscle biopsy. A rheumatologist was consulted, and prednisone 60 mg/d was started for possible inflammatory myopathy. The patient was discharged to a skilled nursing facility for physical therapy.

The fibrillations and positive sharp waves on EMG can be seen in both neuropathy (from denervation) and myopathy. The normal nerve conduction studies make localization to the nerve unlikely. In addition, the shorter duration motor units with early recruitment on EMG are characteristic of a myopathy. Despite the ongoing myopathy, the improved CK level suggests the muscle disease is playing a minimal role in the patient’s current illness. Prescribing corticosteroids for a presumed inflammatory myopathy without a clear diagnosis is risky, as steroids may render subsequent biopsy results unreliable, may themselves cause myopathy, and expose the patient to the side effects of immunosuppression.

One month later, the patient saw her rheumatologist. Although she had tapered the prednisone down to 10 mg/d, she had not returned to baseline strength, was still using a walker, and reported increased difficulty keeping her head raised. She also noted 2 new symptoms: speech slurring and, in both hands, a tremor that made it difficult to hold objects.

Examination revealed a fine tremor in both arms. There were no skin lesions, and the joints were normal. The patient was oriented to name, place, and date. Memory testing was 3 for 3 on register but 0 for 3 on recall. She was unable to perform serial 7s and able to spell backward only 3 of the 5 letters in the word world . Speech was dysarthric and scanning in quality. On extraocular movements, she demonstrated poor smooth pursuit. Examination of the head and neck was significant for nearly constant head titubation and weak neck flexors. Upper extremity strength was normal. Mild weakness was noted in both hip flexors. Deep tendon reflexes were preserved except at the ankle, where they were reduced. Finger–nose–finger testing revealed marked dysmetria, more pronounced on the left, and there was mild bilateral heel-to-shin dysmetria.

Diffuse myopathy cannot account for the patient’s impaired cognition or progressive cerebellar findings, which now include head titubation and scanning speech. As more than a month has elapsed since the brain imaging was performed, MRI could be repeated for evidence of infection, malignancy, inflammation, or demyelination. More important, lumbar puncture is indicated to exclude infection and, with flow cytometry, cytology, and measurement of oligoclonal bands and IgG index, to assess for autoimmune or paraneoplastic antibody-mediated disorders.

The patient was readmitted. On repeat brain MRI, there were no new significant findings. Complete blood cell count and chemistry panel results were unremarkable. Erythrocyte sedimentation rate and C-reactive protein level remained normal. CK level was 451 U/L, and ALT level 29 U/L.

Lumbar puncture was performed. Opening pressure was 14.5 cm of water, and cerebrospinal fluid (CSF) was clear and colorless. There were 3 red blood cells/mm ³ and no WBCs. Glucose level was 94 mg/dL, and protein level 74 mg/dL. CSF IgG synthesis rate was normal, flow cytometry revealed no abnormal clonal populations, and cytology was negative for malignancy. Two unique oligoclonal bands were found in the CSF.

The absence of WBCs in the CSF excludes CNS infection. The patient’s main problem is an inflammatory CNS process as defined by presence of oligoclonal bands in the CSF, compared with their absence in the serum. Autoimmune, neoplastic, and paraneoplastic disorders could explain these bands. There was no evidence of systemic autoimmune illness. The patient has not had a recent infection that could result in postinfectious demyelination, and her clinical and imaging features are not suggestive of a demyelinating disorder, such as multiple sclerosis. Of the neoplastic possibilities, lymphoma with CNS involvement may be difficult to detect initially; this diagnosis, however, is not supported by the unremarkable MRI, flow cytometry, and cytology findings. In paraneoplastic syndromes, the CSF may include antibodies that react to antigens in the brain or cerebellum.

At this point, evaluation for malignancy should involve mammography, imaging of the chest, abdomen, and pelvis, and colorectal screening. Testing should also include measurement of serum and CSF autoantibodies associated with paraneoplastic cerebellar degeneration. The expanding list of paraneoplastic antibodies that may attack the cerebellum includes anti-Hu (often associated with small cell lung cancer), anti-Yo (associated with ovarian or breast cancer), anti-aquaporin 4, antibodies to the voltage-gated potassium channel, and anti–glutamic acid decarboxylase (anti-GAD).

Mammography and breast examination findings were normal. Computed tomography (CT) of the chest showed no adenopathy, nodules, or masses. Abdomen CT showed nonspecific prominence of the gallbladder wall. Flexible sigmoidoscopy revealed no masses, only thickened folds in the sigmoid colon; results of multiple colon biopsy tests were normal. Carcinoembryonic antigen level was 2.0 μg/L, and CA-125 level 5 U/mL. Serum GAD-65 antibodies were elevated (>30 nmol/L).

Anti-GAD is mostly known as the antibody associated with type 1 diabetes mellitus (T1DM). In rare instances, even in patients without a history of diabetes, anti-GAD antibodies may lead to an autoimmune attack on the brain, particularly the cerebellum, as part of an idiopathic autoimmune disorder or as a paraneoplastic syndrome. In either case, treatment involves corticosteroids, intravenous Ig, or plasma exchange. When the autoimmune attack is associated with malignancy, treatment response is poorer, unless the malignancy is successfully managed. The next steps are intravenous Ig or plasma exchange and positron emission tomography–CT (PET-CT) assessing for an underlying neoplasm that may have been too small to be detected with routine CT.

Numerous antibodies to antigens of the nervous system have been identified in association with malignancy, resulting in a variety of paraneoplastic neurologic syndromes (PNSs). Collectively, these syndromes are seen in fewer than 1 in 10,000 patients with cancer. The malignancy may be diagnosed months to years before the paraneoplastic syndrome, or the syndrome may be the first clue to underlying malignancy.¹ Hospitalists should suspect a PNS if the patient has known cancer and has unexplained neurologic deficits, if the presentation fits a classic PNS pattern, or if evaluation for neurologic symptoms identifies an inflammatory CSF profile that cannot be explained by demyelinating disorder or infection.

• The differential diagnosis for cerebellar deficits associated with normal brain MRI includes infection, toxic-metabolic insults (alcohol toxicity, vitamin B₁₂ deficiency, medication toxicity), anoxia, early neurodegenerative illness, and antibody-mediated disorders, such as autoimmune, postinfectious, and paraneoplastic syndromes.
• Hospitalists should suspect a PNS when a patient with known cancer develops unexplained neurologic deficits or when evaluation for neurologic symptoms identifies an inflammatory CSF profile that cannot be explained by a demyelinating disorder or an infection.
• Hospitalists should familiarize themselves with the classic PNS presentations, including limbic encephalitis, cerebellar degeneration, stiff person syndrome, opsoclonus-myoclonus, NMDA receptor encephalitis, and encephalomyelitis.
• Suspicion for PNS may be confirmed by the presence of paraneoplastic antibodies in CSF or serum. When routine evaluation fails to identify cancer, PET-CT should be performed.

Disclosure

Nothing to report.

The approach to clinical conundrums by an expert clinician is revealed through the presentation of an actual patient’s case in an approach typical of a morning report. Similarly to patient care, sequential pieces of information are provided to the clinician, who is unfamiliar with the case. The focus is on the thought processes of both the clinical team caring for the patient and the discussant.

A 78-year-old woman presented to her primary care physician with a 2-month history of progressive leg weakness. She reported walking difficulty caused by occasional “buckling” of the knees.

The knee buckling may be a clue to the quadriceps muscle weakness. The quadriceps straightens and locks the knee when the foot is being planted. Weakness of this muscle can result in the knee giving way. Isolated quadriceps weakness, which is uncommon, typically is caused by lower motor neuron issues, such as femoral neuropathy, L4–L5 radiculopathy, lumbosacral plexopathy, and primary muscle diseases, including inclusion body myositis.

The patient had diabetes mellitus and hypertension. Her medications were insulin glargine, metformin, glipizide, lisinopril, atorvastatin, and aspirin, and she was taking vitamin D and calcium. None of these was recently changed or added. Aside from having the weakness, the patient was in her usual state of health and had no other complaints. She denied weight changes, fevers, night sweats, and fatigue. She was widowed, lived with her daughter, had no pets, never used tobacco, and did not drink alcohol or use illicit drugs. There was no family history of neuromuscular disorders, and both of her parents died of natural causes at advanced ages.

The physical examination revealed no knee deformities, and the patient had good active range of motion of both knees and normal strength throughout her limbs. Plain radiographs of the knees showed mild medial compartment osteoarthritis. The patient was advised to stop atorvastatin.

Patients who take atorvastatin and other statins (3-hydroxy-3-methyl-glutaryl-co-enzyme A reductase inhibitors) can experience a spectrum of muscle disease, from myalgias and weakness to (rare) overt myositis with rhabdomyolysis. Statin-induced myopathy tends to affect larger proximal muscles, can occur at any time during the period the medication is being used, and usually resolves within weeks after discontinuation. Given this patient’s preserved strength, it was reasonable to manage her conservatively.

One month later, she presented to another hospital’s emergency department with increasing weakness in the lower extremities and new loss of balance requiring use of a walker for ambulation. She thought the weakness was confined to her legs and was affecting her thigh muscles more than her calves or feet. She reported fatigue, decreased appetite, and an unintentional 15-pound weight loss. She denied diarrhea, back pain, bowel and bladder function problems, sensation changes, myalgias, and arthralgias. She reported no swallowing or vision problems, rashes, Raynaud disease symptoms, photosensitivity, dry eyes or mouth, recent falls or trauma, fevers, night sweats, recent illness, or travel.

On physical examination, the patient’s temperature was 98.2°F, blood pressure 120/84 mm Hg, pulse 73 beats per minute, respiratory rate 16 breaths per minute, and oxygen saturation 99% with ambient air. The patient was obese and not in distress. She was alert, oriented, and able to follow multistep instructions. Cranial nerve examination was normal. The patient had mild weakness in her bilateral deltoids and bilateral hip flexors but full strength in all other muscle groups. Deep tendon reflexes were normal in the biceps and patella and reduced in the ankles. The Babinski sign was absent. Throughout the lower extremities, sensation was intact to light touch; there was no saddle anesthesia. Finger–nose–finger testing showed slight dysmetria in the left upper extremity. Because of her imbalance, the patient needed help to stand up; once upright, though, she was able to take 3 steps forward and backward with use of a walker. Her stride length was diminished, and her gait unsteady and wide based.

These exam findings suggest 2 separate localizations. Symmetric proximal weakness with preserved distal strength in the extremities indicate a lower motor neuron problem, further suggested by the absence of upper motor neuron findings of spasticity, and hyperreflexia. Intact patellar and biceps reflexes suggest either muscle or neuromuscular junction localization. Separately, the findings of wide-based gait and dysmetria raise the possibility of a problem in the ipsilateral cerebellum, in this case on the left. A cerebellar lesion would not explain the bilateral symmetric weakness. The combination of lower motor neuron leg weakness and cerebellar deficits suggests a systemic disorder, such as an infectious, inflammatory, or paraneoplastic process. A space-occupying lesion would not produce this constellation of deficits.

Serum chemistry panel was normal, creatinine level 0.47 mg/dL, and albumin level 4.0 g/dL. White blood cell (WBC) count was 8100/mm³, hemoglobin level 12 g/dL, and platelet count 287,000/mm³. Alanine aminotransferase (ALT) level was 74 U/L (reference range, 0-36 U/L), alkaline phosphatase level 41 U/L (reference range, 37-117 U/L), and total bilirubin level 0.4 mg/dL (reference range, 0.2-1.2 mg/dL). Prothrombin time and thyrotropin were normal. Creatine kinase (CK) level was 2328 U/L (reference range, <200 U/L). Erythrocyte sedimentation rate was 17 mm/h, and C-reactive protein level 0.1 mg/L. Urinalysis (dipstick testing) detected no myoglobin, and there were no casts. Plain radiograph of the chest was normal.

The elevated CK indicates muscle disease, and, in the absence of other findings of liver disease, the ALT elevation likely has a muscle origin as well. The differential diagnosis for elevated CK includes myopathy caused either by infection, trauma, ischemia, or a toxin (medication included) or by a hereditary, metabolic, endocrinologic, or inflammatory disorder. There is no history of trauma, strenuous exertion, or muscle toxin other than the statin, and the progression of symptoms after medication discontinuation argues against statin myopathy. The laboratory test results rule out derangement of potassium, calcium, phosphorus, magnesium, vitamin D, or thyroid function as the cause of the myopathy. The absence of fever, absence of diffuse organ involvement, and normal inflammatory markers point away from a systemic infection or vasculitis. The inflammatory myopathies dermatomyositis and polymyositis classically produce proximal muscle weakness and are possibilities in this case, but one would expect the inflammatory markers to be elevated in these conditions. Malignancy related to dermatomyositis or to paraneoplastic syndrome may account for the myopathy.

The data provided do not identify a unifying diagnosis. To look for an inflammatory myopathy, such as dermatomyositis or polymyositis, it is reasonable to perform electromyography (EMG) to delineate the location of muscle involvement and identify a site for tissue biopsy. As no obvious toxins or metabolic conditions explain the dysmetria, magnetic resonance imaging (MRI) of the brain should be performed to evaluate for lesions in the cerebellum.

The patient was admitted to the hospital. On T2-weighted and FLAIR (fluid attenuation inversion recovery) sequences, MRI of the brain showed a few scattered subcortical and periventricular white matter hyperintense foci bilaterally. Antibodies to human immunodeficiency virus 1 and 2, and Treponema pallidum immunoglobulins G and M, were not detected. Serum levels of lactate dehydrogenase, vitamin B ₁₂ , angiotensin-converting enzyme, antinuclear antibody, rheumatoid factor, and anti–cyclic citrullinated peptide IgG were normal.

The brain imaging excludes a space-occupying lesion in the cerebellum but does not identify the cause of dysmetria. Toxic-metabolic conditions, such as alcohol toxicity, vitamin B₁₂ deficiency, anoxia, and toxicity of certain medications, may impair cerebellar function (MRI findings may be normal), but none of these is present. Other disorders that attack the central nervous system (CNS) (again, brain imaging may show minimal abnormalities) include infections, early-stage neurodegenerative illnesses, and antibody-associated disorders (eg, autoimmune diseases, postinfectious and paraneoplastic conditions).

Four days after intravenous fluids were started, the patient’s CK level improved, but her weakness persisted. There was no evidence of peripheral neuropathy on lower extremity nerve conduction studies. EMG revealed few fibrillations and positive sharp waves in proximal limb muscles and thoracic paraspinal muscles. Deltoid, biceps, and tensor fasciae latae showed shorter duration complex motor units with early recruitment. The patient declined muscle biopsy. A rheumatologist was consulted, and prednisone 60 mg/d was started for possible inflammatory myopathy. The patient was discharged to a skilled nursing facility for physical therapy.

The fibrillations and positive sharp waves on EMG can be seen in both neuropathy (from denervation) and myopathy. The normal nerve conduction studies make localization to the nerve unlikely. In addition, the shorter duration motor units with early recruitment on EMG are characteristic of a myopathy. Despite the ongoing myopathy, the improved CK level suggests the muscle disease is playing a minimal role in the patient’s current illness. Prescribing corticosteroids for a presumed inflammatory myopathy without a clear diagnosis is risky, as steroids may render subsequent biopsy results unreliable, may themselves cause myopathy, and expose the patient to the side effects of immunosuppression.

One month later, the patient saw her rheumatologist. Although she had tapered the prednisone down to 10 mg/d, she had not returned to baseline strength, was still using a walker, and reported increased difficulty keeping her head raised. She also noted 2 new symptoms: speech slurring and, in both hands, a tremor that made it difficult to hold objects.

Examination revealed a fine tremor in both arms. There were no skin lesions, and the joints were normal. The patient was oriented to name, place, and date. Memory testing was 3 for 3 on register but 0 for 3 on recall. She was unable to perform serial 7s and able to spell backward only 3 of the 5 letters in the word world . Speech was dysarthric and scanning in quality. On extraocular movements, she demonstrated poor smooth pursuit. Examination of the head and neck was significant for nearly constant head titubation and weak neck flexors. Upper extremity strength was normal. Mild weakness was noted in both hip flexors. Deep tendon reflexes were preserved except at the ankle, where they were reduced. Finger–nose–finger testing revealed marked dysmetria, more pronounced on the left, and there was mild bilateral heel-to-shin dysmetria.

Diffuse myopathy cannot account for the patient’s impaired cognition or progressive cerebellar findings, which now include head titubation and scanning speech. As more than a month has elapsed since the brain imaging was performed, MRI could be repeated for evidence of infection, malignancy, inflammation, or demyelination. More important, lumbar puncture is indicated to exclude infection and, with flow cytometry, cytology, and measurement of oligoclonal bands and IgG index, to assess for autoimmune or paraneoplastic antibody-mediated disorders.

The patient was readmitted. On repeat brain MRI, there were no new significant findings. Complete blood cell count and chemistry panel results were unremarkable. Erythrocyte sedimentation rate and C-reactive protein level remained normal. CK level was 451 U/L, and ALT level 29 U/L.

Lumbar puncture was performed. Opening pressure was 14.5 cm of water, and cerebrospinal fluid (CSF) was clear and colorless. There were 3 red blood cells/mm ³ and no WBCs. Glucose level was 94 mg/dL, and protein level 74 mg/dL. CSF IgG synthesis rate was normal, flow cytometry revealed no abnormal clonal populations, and cytology was negative for malignancy. Two unique oligoclonal bands were found in the CSF.

The absence of WBCs in the CSF excludes CNS infection. The patient’s main problem is an inflammatory CNS process as defined by presence of oligoclonal bands in the CSF, compared with their absence in the serum. Autoimmune, neoplastic, and paraneoplastic disorders could explain these bands. There was no evidence of systemic autoimmune illness. The patient has not had a recent infection that could result in postinfectious demyelination, and her clinical and imaging features are not suggestive of a demyelinating disorder, such as multiple sclerosis. Of the neoplastic possibilities, lymphoma with CNS involvement may be difficult to detect initially; this diagnosis, however, is not supported by the unremarkable MRI, flow cytometry, and cytology findings. In paraneoplastic syndromes, the CSF may include antibodies that react to antigens in the brain or cerebellum.

At this point, evaluation for malignancy should involve mammography, imaging of the chest, abdomen, and pelvis, and colorectal screening. Testing should also include measurement of serum and CSF autoantibodies associated with paraneoplastic cerebellar degeneration. The expanding list of paraneoplastic antibodies that may attack the cerebellum includes anti-Hu (often associated with small cell lung cancer), anti-Yo (associated with ovarian or breast cancer), anti-aquaporin 4, antibodies to the voltage-gated potassium channel, and anti–glutamic acid decarboxylase (anti-GAD).

Mammography and breast examination findings were normal. Computed tomography (CT) of the chest showed no adenopathy, nodules, or masses. Abdomen CT showed nonspecific prominence of the gallbladder wall. Flexible sigmoidoscopy revealed no masses, only thickened folds in the sigmoid colon; results of multiple colon biopsy tests were normal. Carcinoembryonic antigen level was 2.0 μg/L, and CA-125 level 5 U/mL. Serum GAD-65 antibodies were elevated (>30 nmol/L).

Anti-GAD is mostly known as the antibody associated with type 1 diabetes mellitus (T1DM). In rare instances, even in patients without a history of diabetes, anti-GAD antibodies may lead to an autoimmune attack on the brain, particularly the cerebellum, as part of an idiopathic autoimmune disorder or as a paraneoplastic syndrome. In either case, treatment involves corticosteroids, intravenous Ig, or plasma exchange. When the autoimmune attack is associated with malignancy, treatment response is poorer, unless the malignancy is successfully managed. The next steps are intravenous Ig or plasma exchange and positron emission tomography–CT (PET-CT) assessing for an underlying neoplasm that may have been too small to be detected with routine CT.

Numerous antibodies to antigens of the nervous system have been identified in association with malignancy, resulting in a variety of paraneoplastic neurologic syndromes (PNSs). Collectively, these syndromes are seen in fewer than 1 in 10,000 patients with cancer. The malignancy may be diagnosed months to years before the paraneoplastic syndrome, or the syndrome may be the first clue to underlying malignancy.¹ Hospitalists should suspect a PNS if the patient has known cancer and has unexplained neurologic deficits, if the presentation fits a classic PNS pattern, or if evaluation for neurologic symptoms identifies an inflammatory CSF profile that cannot be explained by demyelinating disorder or infection.

• The differential diagnosis for cerebellar deficits associated with normal brain MRI includes infection, toxic-metabolic insults (alcohol toxicity, vitamin B₁₂ deficiency, medication toxicity), anoxia, early neurodegenerative illness, and antibody-mediated disorders, such as autoimmune, postinfectious, and paraneoplastic syndromes.
• Hospitalists should suspect a PNS when a patient with known cancer develops unexplained neurologic deficits or when evaluation for neurologic symptoms identifies an inflammatory CSF profile that cannot be explained by a demyelinating disorder or an infection.
• Hospitalists should familiarize themselves with the classic PNS presentations, including limbic encephalitis, cerebellar degeneration, stiff person syndrome, opsoclonus-myoclonus, NMDA receptor encephalitis, and encephalomyelitis.
• Suspicion for PNS may be confirmed by the presence of paraneoplastic antibodies in CSF or serum. When routine evaluation fails to identify cancer, PET-CT should be performed.

Disclosure

Nothing to report.

BOSTON—The incidence of serotonin syndrome ranged from 0.02% to 0.04% among all patients who were coprescribed triptans and an SSRI/SNRI during any calendar year from 2001 to 2014, according to a study presented at the 59th Annual Scientific Meeting of the American Headache Society. “Our data do not suggest a clinically meaningful risk of serotonin syndrome in patients coprescribed triptans with SSRI/SSNI antidepressants,” said Yulia Y. Orlova, MD, a clinical fellow at the John R. Graham Headache Center at Brigham and Women’s Faulkner Hospital in Boston.

Serotonin syndrome is a drug-induced group of symptoms that can be life-threatening. In 2006, the FDA issued an advisory concerning the risk of serotonin syndrome with concomitant use of triptans and SSRI/SNRI antidepressants. Since then, pharmacy systems and other decision support systems routinely have issued safety alerts when coprescription occurs. “However, all published reports of serotonin syndrome in patients receiving triptans alone or in combination with an SSRI/SNRI are case reports or case series that lack a denominator, so the true risk remains unknown,” said Dr. Orlova on behalf of her study collaborators.

Dr. Orlova and colleagues conducted a population-based study. For each year from 2001 to 2014, they used the Partners Healthcare System Research Patient Data Registry to identify patients receiving coprescriptions. The registry is a centralized data warehouse with clinical information about more than 6.5 million patients. The ICD-9 code for serotonin syndrome (333.99) is not reported separately in the database, but is part of a broader category of “other extrapyramidal diseases and abnormal movement disorders.” The researchers conservatively assumed that all reports of diagnostic code ICD-9 333.99 might represent serotonin syndrome. Among those patients receiving coprescriptions in the database, the researchers searched for those with the 333.99 code. The research team then reviewed detailed medical records to determine whether those patients met Sternbach or Hunter criteria for serotonin toxicity, or both, during the year in which concomitant prescription of a triptan antimigraine medication and SSRI/SNRI antidepressant may have occurred.

Over the 14-year study period, nearly 48,000 patients were prescribed triptans. Among these patients, about 19,000 were also coprescribed SSRI or SNRI antidepressants. A total of 229 received an ICD-9 diagnosis of 333.99. Detailed chart review revealed 17 cases where serotonin syndrome was reported as part of the differential diagnosis, past medical history, or main diagnosis. Seven of the 17 patients met Sternbach criteria (0.04% of all coprescription cases), four met Hunter criteria (0.02% of all coprescription cases), and all of the latter also satisfied Sternbach criteria.

Triptan use was reported in close temporal relation to the onset of symptoms in two cases. One case, involving eletriptan, was self-reported by the patient and recorded by the physician in the medical record. Chart information for this case did not allow assessment of whether the case met diagnostic criteria. The second case satisfied both sets of criteria for serotonin syndrome and involved the use of rizatriptan, although the onset of symptoms preceded rizatriptan use.

BOSTON—The incidence of serotonin syndrome ranged from 0.02% to 0.04% among all patients who were coprescribed triptans and an SSRI/SNRI during any calendar year from 2001 to 2014, according to a study presented at the 59th Annual Scientific Meeting of the American Headache Society. “Our data do not suggest a clinically meaningful risk of serotonin syndrome in patients coprescribed triptans with SSRI/SSNI antidepressants,” said Yulia Y. Orlova, MD, a clinical fellow at the John R. Graham Headache Center at Brigham and Women’s Faulkner Hospital in Boston.

Serotonin syndrome is a drug-induced group of symptoms that can be life-threatening. In 2006, the FDA issued an advisory concerning the risk of serotonin syndrome with concomitant use of triptans and SSRI/SNRI antidepressants. Since then, pharmacy systems and other decision support systems routinely have issued safety alerts when coprescription occurs. “However, all published reports of serotonin syndrome in patients receiving triptans alone or in combination with an SSRI/SNRI are case reports or case series that lack a denominator, so the true risk remains unknown,” said Dr. Orlova on behalf of her study collaborators.

Dr. Orlova and colleagues conducted a population-based study. For each year from 2001 to 2014, they used the Partners Healthcare System Research Patient Data Registry to identify patients receiving coprescriptions. The registry is a centralized data warehouse with clinical information about more than 6.5 million patients. The ICD-9 code for serotonin syndrome (333.99) is not reported separately in the database, but is part of a broader category of “other extrapyramidal diseases and abnormal movement disorders.” The researchers conservatively assumed that all reports of diagnostic code ICD-9 333.99 might represent serotonin syndrome. Among those patients receiving coprescriptions in the database, the researchers searched for those with the 333.99 code. The research team then reviewed detailed medical records to determine whether those patients met Sternbach or Hunter criteria for serotonin toxicity, or both, during the year in which concomitant prescription of a triptan antimigraine medication and SSRI/SNRI antidepressant may have occurred.

Over the 14-year study period, nearly 48,000 patients were prescribed triptans. Among these patients, about 19,000 were also coprescribed SSRI or SNRI antidepressants. A total of 229 received an ICD-9 diagnosis of 333.99. Detailed chart review revealed 17 cases where serotonin syndrome was reported as part of the differential diagnosis, past medical history, or main diagnosis. Seven of the 17 patients met Sternbach criteria (0.04% of all coprescription cases), four met Hunter criteria (0.02% of all coprescription cases), and all of the latter also satisfied Sternbach criteria.

Triptan use was reported in close temporal relation to the onset of symptoms in two cases. One case, involving eletriptan, was self-reported by the patient and recorded by the physician in the medical record. Chart information for this case did not allow assessment of whether the case met diagnostic criteria. The second case satisfied both sets of criteria for serotonin syndrome and involved the use of rizatriptan, although the onset of symptoms preceded rizatriptan use.

BOSTON—The incidence of serotonin syndrome ranged from 0.02% to 0.04% among all patients who were coprescribed triptans and an SSRI/SNRI during any calendar year from 2001 to 2014, according to a study presented at the 59th Annual Scientific Meeting of the American Headache Society. “Our data do not suggest a clinically meaningful risk of serotonin syndrome in patients coprescribed triptans with SSRI/SSNI antidepressants,” said Yulia Y. Orlova, MD, a clinical fellow at the John R. Graham Headache Center at Brigham and Women’s Faulkner Hospital in Boston.

Serotonin syndrome is a drug-induced group of symptoms that can be life-threatening. In 2006, the FDA issued an advisory concerning the risk of serotonin syndrome with concomitant use of triptans and SSRI/SNRI antidepressants. Since then, pharmacy systems and other decision support systems routinely have issued safety alerts when coprescription occurs. “However, all published reports of serotonin syndrome in patients receiving triptans alone or in combination with an SSRI/SNRI are case reports or case series that lack a denominator, so the true risk remains unknown,” said Dr. Orlova on behalf of her study collaborators.

Dr. Orlova and colleagues conducted a population-based study. For each year from 2001 to 2014, they used the Partners Healthcare System Research Patient Data Registry to identify patients receiving coprescriptions. The registry is a centralized data warehouse with clinical information about more than 6.5 million patients. The ICD-9 code for serotonin syndrome (333.99) is not reported separately in the database, but is part of a broader category of “other extrapyramidal diseases and abnormal movement disorders.” The researchers conservatively assumed that all reports of diagnostic code ICD-9 333.99 might represent serotonin syndrome. Among those patients receiving coprescriptions in the database, the researchers searched for those with the 333.99 code. The research team then reviewed detailed medical records to determine whether those patients met Sternbach or Hunter criteria for serotonin toxicity, or both, during the year in which concomitant prescription of a triptan antimigraine medication and SSRI/SNRI antidepressant may have occurred.

Over the 14-year study period, nearly 48,000 patients were prescribed triptans. Among these patients, about 19,000 were also coprescribed SSRI or SNRI antidepressants. A total of 229 received an ICD-9 diagnosis of 333.99. Detailed chart review revealed 17 cases where serotonin syndrome was reported as part of the differential diagnosis, past medical history, or main diagnosis. Seven of the 17 patients met Sternbach criteria (0.04% of all coprescription cases), four met Hunter criteria (0.02% of all coprescription cases), and all of the latter also satisfied Sternbach criteria.

Triptan use was reported in close temporal relation to the onset of symptoms in two cases. One case, involving eletriptan, was self-reported by the patient and recorded by the physician in the medical record. Chart information for this case did not allow assessment of whether the case met diagnostic criteria. The second case satisfied both sets of criteria for serotonin syndrome and involved the use of rizatriptan, although the onset of symptoms preceded rizatriptan use.

More than 44 million pelvic examinations are performed annually in the United States.¹ In March 2017, the United States Preventive Services Task Force (USPSTF) published an updated recommendation statement regarding the need for routine screening pelvic examinations in asymptomatic adult women (18 years and older) receiving primary care: “The USPSTF concludes that the current evidence is insufficient to assess the balance of benefits and harms of performing screening pelvic examinations in asymptomatic, nonpregnant adult women.”²

That statement, however, was assigned a grade of I, which means that evidence is lacking, of poor quality, or conflicting, and that the balance of benefits and harms cannot be determined. This USPSTF recommendation statement thus will not change practice for ObGyn providers but likely will renew our commitment to provide individualized well-woman care. There was inadequate or poor quality evidence for benefits related to all-cause mortality, disease-specific morbidity, and quality of life, as well as inadequate evidence on harms related to false-positive findings and anxiety stemming from screening pelvic exams.

Read about coding and billing for a standard pelvic exam

The pelvic examination and insurance coverage

---

Melanie Witt, RN, MA

Coding and billing for the care provided at a well-woman visit can be uncomplicated if you know the right codes for the right program. The information presented here concerns straightforward preventive care and assumes that the patient also has not presented with a significant problem at the same visit.

First, a patient who is not Medicare-eligible might have insurance coverage for an annual preventive care examination every year. Normally, this service would be billed using the Current Procedural Terminology (CPT) preventive medicine codes, but some insurers require the use of special codes for an annual gynecologic exam. These special codes are:

• S0610, Annual gynecological examination, new patient
• S0612, Annual gynecological examination, established patient
• S0613, Annual gynecological examination; clinical breast examination without pelvic evaluation.

Notably, Aetna, Cigna, and UnitedHealthcare require these codes to signify that a pelvic examination has been performed (except for code S0613), but many Blue Cross Blue Shield programs, for whom these codes were originally created, are now reverting to the CPT preventive medicine codes for all preventive care.

CPT outlines the requirements for use of the preventive medicine codes as: an initial or periodic comprehensive preventive medicine evaluation or reevaluation and management (E/M) service, which includes an age- and gender-appropriate history, examination, counseling/anticipatory guidance/risk factor reduction interventions, and the ordering of laboratory/diagnostic procedures. The codes are divided into new or established patient categories by age range as follows:

The Medicare E/M documentation guidelines do not apply to preventive services, and a head-to-toe examination also is not required. CPT recognizes the American College of Obstetricians and Gynecologists (ACOG) as an authoritative body to make recommendations for the expected preventive service for women, and if such a service is provided and documented, the preventive care codes are to be reported. The payers who use the S codes for a gynecologic exam will require that a pelvic examination has been performed, but such an examination would not be required when using the CPT codes or ACOG's guidelines if the physician and patient agreed that such an exam was not warranted every year. The other components of a preventive service applicable to the female patient's age, however, should be documented in order to report the CPT codes for preventive medicine services.

If a pelvic examination is not performed, say because the patient is young and not sexually active, but an examination of other areas is carried out, the diagnosis code would change from Z01.411, Encounter for gynecological examination (general) (routine) with abnormal findings, or Z01.419, Encounter for gynecological examination (general) (routine) without abnormal findings, to a general health exam: Z00.00, Encounter for general adult medical examination without abnormal findings, or Z00.01, Encounter for general adult medical examination with abnormal findings.  

What about Medicare?

Medicare requirements are somewhat different. First, Medicare covers only a small portion of the preventive care service; that is, it covers a physical examination of the genital organs and breasts and the collection and conveyance of a Pap specimen to the laboratory every 2 years for a low-risk patient. Second, the codes required to get reimbursed for the examination are:

• G0101, Cervical or vaginal cancer screening; pelvic and clinical breast examination
• Q0091, Screening Papanicolaou smear; obtaining, preparing, and conveyance of cervical or vaginal smear to laboratory.

It is not necessary to perform both of these services every 2 years (for instance, the patient may not need a Pap smear every 2 years based on her age and history), but the benefit is available if the service is performed. If the woman is at high risk for developing cervical or vaginal cancer, Medicare will cover this portion of the encounter every year so long as the Medicare-defined criteria for high risk have been documented at the time of the exam.

Related article:
GYN coding changes to note for your maximized reimbursement

Ms. Witt is an independent coding and documentation consultant and former program manager, department of coding and nomenclature, American Congress of Obstetricians and Gynecologists.

---

The author reports no financial relationships relevant to this article.

Read the authors’ interpretation of the new USPSTF statement

Interpreting the new USPSTF statement

We understand the USPSTF statement to mean that pelvic exams should not be abandoned, but rather should be individualized to each patient for her specific visit. We agree that for visits focused on counseling and routine screening in asymptomatic, nonpregnant women, pelvic exams likely will not increase the early detection and treatment of disease and more benefit likely would be derived by performing and discussing evidence-based and age-appropriate health services. A classic example would be for initiation or maintenance of oral contraception in an 18-year-old patient for whom an exam could cause unnecessary trauma, pain, or psychological distress leading to future avoidance or barriers to seeking health care. For long-acting reversible contraception placement, however, a pelvic exam clearly would be necessary for insertion of an intrauterine device.

Related article:
Women’s Preventive Services Initiative Guidelines provide consensus for practicing ObGyns

Indications for pelvic examination

Remember that the pelvic examination has 3 distinct parts (and that not all parts need to be routinely conducted)³:

• general inspection of the external genitalia and vulva
• speculum examination and evaluation of the vagina and cervix
• bimanual examination with possible rectovaginal examination in age-appropriate or symptomatic women.

According to the Well-Woman Task Force of the American College of Obstetricians and Gynecologists (ACOG), “For women 21 years and older, external exam may be performed annually and that inclusion of speculum examination, bimanual examination, or both in otherwise healthy women should be a shared, informed decision between patient and provider.”⁴

Indications for performing certain parts of the pelvic exam include⁴:

• routine screening for cervical cancer (Pap test)
• routine screening for gonorrhea, chlamydia infection, and other sexually transmitted infections
• evaluation of abnormal vaginal discharge
• evaluation of abnormal bleeding, pelvic pain, and pelvic floor disorders, such as prolapse, urinary incontinence, and accidental bowel leakage
• evaluation of menopausal symptoms, such as dryness, dyspareunia, and the genitourinary syndrome of menopause
• evaluation of women at increased risk for gynecologic malignancy, such as women with known hereditary breast–ovarian cancer syndromes.

In 2016, ACOG launched the Women’s Preventive Services Initiative (WPSI) in conjunction with the Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services. In this 5-year collaboration, the agencies are endeavoring to review and update the recommendations for women’s preventive health care services, including well-woman visits, human papillomavirus testing, and contraception, among many others.⁵ Once the HRSA adopts these recommendations, women will be able to access comprehensive preventive health services without incurring any out-of-pocket expenses.

The pediatric and adolescent gynecologist perspective

---

Roshanak Mansouri Zinn, MD, and Rebekah L. Williams, MD, MS

No literature addresses the utility of screening pelvic examination in the pediatric and adolescent population. According to the American College of Obstetricians and Gynecologists Committee on Adolescent Health Care opinion on the initial reproductive health visit for screening and preventive reproductive health care (reaffirmed in 2016), a screening internal exam is not necessary, but an external genital exam may be indicated and may vary depending on the patient's concerns and prior clinical encounters.¹ The American Academy of Pediatrics promotes annual screening external genital examination for all female patients as part of routine primary care, with internal examinations only as indicated.²

Age-appropriate pelvic examination for girls and nonsexually active adolescents usually is limited to an external genital exam to evaluate the anatomy and note the sexual maturity rating (Tanner stage), an important indicator of normal pubertal development. As in adults, the potential benefits of screening examination in this population include detection of benign gynecologic conditions (including vulvar skin conditions and abnormalities of hymenal or vaginal development). Additionally, early reproductive health visits are an important time for clinicians to build rapport with younger patients and to provide anticipatory education on menstruation, hygiene, and anatomy. These visits can destigmatize and demystify the pelvic examination and help young women seek care more appropriately and more comfortably if problems do arise.

Even when a pelvic exam is indicated, a patient's young age can give providers pause as to what type of exam to perform. Patients with vulvovaginal symptoms, abnormal vaginal bleeding, vaginal discharge, or pelvic or abdominal pain should receive complete evaluation with external genital examination. If external vaginal examination does not allow for complete assessment of the problem, the patient and provider can assess the likelihood of her tolerating an internal exam in the clinic versus undergoing vaginoscopy under sedation. Limited laboratory evaluation and transabdominal pelvic ultrasonography may provide sufficient information for appropriate clinical decision making and management without internal examination. If symptoms persist or do not respond to first-line treatment, an internal exam should be performed.

Patients of any age may experience anxiety or physical discomfort or may even delay or avoid seeking care because of fear of a pelvic exam. However, providers of reproductive health care for children and adolescents can offer early education, reassurance, and a more comfortable experience when pelvic examination is necessary in this population.

References

1. American College of Obstetricians and Gynecologists Committee on Adolescent Health Care. Committee Opinion No. 598: Committee on Adolescent Health Care: the initial reproductive health visit. Obstet Gynecol. 2014;123(5):1143-1147.
2. Braverman PK, Breech L; Committee on Adolescence. American Academy of Pediatrics. Clinical report: gynecologic examination for adolescents in the pediatric office setting. Pediatrics. 2010;126(3):583-590.

 

Dr. Mansouri Zinn is Assistant Professor, Department of Women's Health, University of Texas at Austin.

Dr. Williams is Assistant Professor, Clinical Pediatrics, Section of Adolescent Medicine, Indiana University School of Medicine, Indianapolis.

Developed in collaboration with the North American Society for Pediatric and Adolescent Gynecology

---

The authors report no financial relationships relevant to this article.

How will the USPSTF statement affect practice?

In an editorial in the Journal of the American Medical Association commenting on the USPSTF statement, McNicholas and Peipert stated, “Based on the recommendation from the task force, clinicians may ask whether the pelvic examination should be abandoned. The answer is not found in this recommendation statement, but instead in a renewed commitment to shared decision making.”⁶ We wholeheartedly agree with this statement. The health care provider and the patient should make the decision, taking into consideration the patient’s risk factors for gynecologic cancers and other conditions, her personal preferences, and her overall values.

This new USPSTF recommendation statement will not change how we currently practice, and the statement’s grade I rating should not impact insurance coverage for pelvic exams. Additionally, further research is needed to better elucidate the role of the pelvic exam at well-woman visits, with hopes of obtaining more precise guidelines from the USPSTF and ACOG.

Share your thoughts! Send your Letter to the Editor to [email protected]. Please include your name and the city and state in which you practice.

More than 44 million pelvic examinations are performed annually in the United States.¹ In March 2017, the United States Preventive Services Task Force (USPSTF) published an updated recommendation statement regarding the need for routine screening pelvic examinations in asymptomatic adult women (18 years and older) receiving primary care: “The USPSTF concludes that the current evidence is insufficient to assess the balance of benefits and harms of performing screening pelvic examinations in asymptomatic, nonpregnant adult women.”²

That statement, however, was assigned a grade of I, which means that evidence is lacking, of poor quality, or conflicting, and that the balance of benefits and harms cannot be determined. This USPSTF recommendation statement thus will not change practice for ObGyn providers but likely will renew our commitment to provide individualized well-woman care. There was inadequate or poor quality evidence for benefits related to all-cause mortality, disease-specific morbidity, and quality of life, as well as inadequate evidence on harms related to false-positive findings and anxiety stemming from screening pelvic exams.

Read about coding and billing for a standard pelvic exam

The pelvic examination and insurance coverage

---

Melanie Witt, RN, MA

Coding and billing for the care provided at a well-woman visit can be uncomplicated if you know the right codes for the right program. The information presented here concerns straightforward preventive care and assumes that the patient also has not presented with a significant problem at the same visit.

First, a patient who is not Medicare-eligible might have insurance coverage for an annual preventive care examination every year. Normally, this service would be billed using the Current Procedural Terminology (CPT) preventive medicine codes, but some insurers require the use of special codes for an annual gynecologic exam. These special codes are:

• S0610, Annual gynecological examination, new patient
• S0612, Annual gynecological examination, established patient
• S0613, Annual gynecological examination; clinical breast examination without pelvic evaluation.

Notably, Aetna, Cigna, and UnitedHealthcare require these codes to signify that a pelvic examination has been performed (except for code S0613), but many Blue Cross Blue Shield programs, for whom these codes were originally created, are now reverting to the CPT preventive medicine codes for all preventive care.

CPT outlines the requirements for use of the preventive medicine codes as: an initial or periodic comprehensive preventive medicine evaluation or reevaluation and management (E/M) service, which includes an age- and gender-appropriate history, examination, counseling/anticipatory guidance/risk factor reduction interventions, and the ordering of laboratory/diagnostic procedures. The codes are divided into new or established patient categories by age range as follows:

The Medicare E/M documentation guidelines do not apply to preventive services, and a head-to-toe examination also is not required. CPT recognizes the American College of Obstetricians and Gynecologists (ACOG) as an authoritative body to make recommendations for the expected preventive service for women, and if such a service is provided and documented, the preventive care codes are to be reported. The payers who use the S codes for a gynecologic exam will require that a pelvic examination has been performed, but such an examination would not be required when using the CPT codes or ACOG's guidelines if the physician and patient agreed that such an exam was not warranted every year. The other components of a preventive service applicable to the female patient's age, however, should be documented in order to report the CPT codes for preventive medicine services.

If a pelvic examination is not performed, say because the patient is young and not sexually active, but an examination of other areas is carried out, the diagnosis code would change from Z01.411, Encounter for gynecological examination (general) (routine) with abnormal findings, or Z01.419, Encounter for gynecological examination (general) (routine) without abnormal findings, to a general health exam: Z00.00, Encounter for general adult medical examination without abnormal findings, or Z00.01, Encounter for general adult medical examination with abnormal findings.  

What about Medicare?

Medicare requirements are somewhat different. First, Medicare covers only a small portion of the preventive care service; that is, it covers a physical examination of the genital organs and breasts and the collection and conveyance of a Pap specimen to the laboratory every 2 years for a low-risk patient. Second, the codes required to get reimbursed for the examination are:

• G0101, Cervical or vaginal cancer screening; pelvic and clinical breast examination
• Q0091, Screening Papanicolaou smear; obtaining, preparing, and conveyance of cervical or vaginal smear to laboratory.

It is not necessary to perform both of these services every 2 years (for instance, the patient may not need a Pap smear every 2 years based on her age and history), but the benefit is available if the service is performed. If the woman is at high risk for developing cervical or vaginal cancer, Medicare will cover this portion of the encounter every year so long as the Medicare-defined criteria for high risk have been documented at the time of the exam.

Related article:
GYN coding changes to note for your maximized reimbursement

Ms. Witt is an independent coding and documentation consultant and former program manager, department of coding and nomenclature, American Congress of Obstetricians and Gynecologists.

---

The author reports no financial relationships relevant to this article.

Read the authors’ interpretation of the new USPSTF statement

Interpreting the new USPSTF statement

We understand the USPSTF statement to mean that pelvic exams should not be abandoned, but rather should be individualized to each patient for her specific visit. We agree that for visits focused on counseling and routine screening in asymptomatic, nonpregnant women, pelvic exams likely will not increase the early detection and treatment of disease and more benefit likely would be derived by performing and discussing evidence-based and age-appropriate health services. A classic example would be for initiation or maintenance of oral contraception in an 18-year-old patient for whom an exam could cause unnecessary trauma, pain, or psychological distress leading to future avoidance or barriers to seeking health care. For long-acting reversible contraception placement, however, a pelvic exam clearly would be necessary for insertion of an intrauterine device.

Related article:
Women’s Preventive Services Initiative Guidelines provide consensus for practicing ObGyns

Indications for pelvic examination

Remember that the pelvic examination has 3 distinct parts (and that not all parts need to be routinely conducted)³:

• general inspection of the external genitalia and vulva
• speculum examination and evaluation of the vagina and cervix
• bimanual examination with possible rectovaginal examination in age-appropriate or symptomatic women.

According to the Well-Woman Task Force of the American College of Obstetricians and Gynecologists (ACOG), “For women 21 years and older, external exam may be performed annually and that inclusion of speculum examination, bimanual examination, or both in otherwise healthy women should be a shared, informed decision between patient and provider.”⁴

Indications for performing certain parts of the pelvic exam include⁴:

• routine screening for cervical cancer (Pap test)
• routine screening for gonorrhea, chlamydia infection, and other sexually transmitted infections
• evaluation of abnormal vaginal discharge
• evaluation of abnormal bleeding, pelvic pain, and pelvic floor disorders, such as prolapse, urinary incontinence, and accidental bowel leakage
• evaluation of menopausal symptoms, such as dryness, dyspareunia, and the genitourinary syndrome of menopause
• evaluation of women at increased risk for gynecologic malignancy, such as women with known hereditary breast–ovarian cancer syndromes.

In 2016, ACOG launched the Women’s Preventive Services Initiative (WPSI) in conjunction with the Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services. In this 5-year collaboration, the agencies are endeavoring to review and update the recommendations for women’s preventive health care services, including well-woman visits, human papillomavirus testing, and contraception, among many others.⁵ Once the HRSA adopts these recommendations, women will be able to access comprehensive preventive health services without incurring any out-of-pocket expenses.

The pediatric and adolescent gynecologist perspective

---

Roshanak Mansouri Zinn, MD, and Rebekah L. Williams, MD, MS

No literature addresses the utility of screening pelvic examination in the pediatric and adolescent population. According to the American College of Obstetricians and Gynecologists Committee on Adolescent Health Care opinion on the initial reproductive health visit for screening and preventive reproductive health care (reaffirmed in 2016), a screening internal exam is not necessary, but an external genital exam may be indicated and may vary depending on the patient's concerns and prior clinical encounters.¹ The American Academy of Pediatrics promotes annual screening external genital examination for all female patients as part of routine primary care, with internal examinations only as indicated.²

Age-appropriate pelvic examination for girls and nonsexually active adolescents usually is limited to an external genital exam to evaluate the anatomy and note the sexual maturity rating (Tanner stage), an important indicator of normal pubertal development. As in adults, the potential benefits of screening examination in this population include detection of benign gynecologic conditions (including vulvar skin conditions and abnormalities of hymenal or vaginal development). Additionally, early reproductive health visits are an important time for clinicians to build rapport with younger patients and to provide anticipatory education on menstruation, hygiene, and anatomy. These visits can destigmatize and demystify the pelvic examination and help young women seek care more appropriately and more comfortably if problems do arise.

Even when a pelvic exam is indicated, a patient's young age can give providers pause as to what type of exam to perform. Patients with vulvovaginal symptoms, abnormal vaginal bleeding, vaginal discharge, or pelvic or abdominal pain should receive complete evaluation with external genital examination. If external vaginal examination does not allow for complete assessment of the problem, the patient and provider can assess the likelihood of her tolerating an internal exam in the clinic versus undergoing vaginoscopy under sedation. Limited laboratory evaluation and transabdominal pelvic ultrasonography may provide sufficient information for appropriate clinical decision making and management without internal examination. If symptoms persist or do not respond to first-line treatment, an internal exam should be performed.

Patients of any age may experience anxiety or physical discomfort or may even delay or avoid seeking care because of fear of a pelvic exam. However, providers of reproductive health care for children and adolescents can offer early education, reassurance, and a more comfortable experience when pelvic examination is necessary in this population.

References

1. American College of Obstetricians and Gynecologists Committee on Adolescent Health Care. Committee Opinion No. 598: Committee on Adolescent Health Care: the initial reproductive health visit. Obstet Gynecol. 2014;123(5):1143-1147.
2. Braverman PK, Breech L; Committee on Adolescence. American Academy of Pediatrics. Clinical report: gynecologic examination for adolescents in the pediatric office setting. Pediatrics. 2010;126(3):583-590.

 

Dr. Mansouri Zinn is Assistant Professor, Department of Women's Health, University of Texas at Austin.

Dr. Williams is Assistant Professor, Clinical Pediatrics, Section of Adolescent Medicine, Indiana University School of Medicine, Indianapolis.

Developed in collaboration with the North American Society for Pediatric and Adolescent Gynecology

---

The authors report no financial relationships relevant to this article.

How will the USPSTF statement affect practice?

In an editorial in the Journal of the American Medical Association commenting on the USPSTF statement, McNicholas and Peipert stated, “Based on the recommendation from the task force, clinicians may ask whether the pelvic examination should be abandoned. The answer is not found in this recommendation statement, but instead in a renewed commitment to shared decision making.”⁶ We wholeheartedly agree with this statement. The health care provider and the patient should make the decision, taking into consideration the patient’s risk factors for gynecologic cancers and other conditions, her personal preferences, and her overall values.

This new USPSTF recommendation statement will not change how we currently practice, and the statement’s grade I rating should not impact insurance coverage for pelvic exams. Additionally, further research is needed to better elucidate the role of the pelvic exam at well-woman visits, with hopes of obtaining more precise guidelines from the USPSTF and ACOG.

Share your thoughts! Send your Letter to the Editor to [email protected]. Please include your name and the city and state in which you practice.

More than 44 million pelvic examinations are performed annually in the United States.¹ In March 2017, the United States Preventive Services Task Force (USPSTF) published an updated recommendation statement regarding the need for routine screening pelvic examinations in asymptomatic adult women (18 years and older) receiving primary care: “The USPSTF concludes that the current evidence is insufficient to assess the balance of benefits and harms of performing screening pelvic examinations in asymptomatic, nonpregnant adult women.”²

That statement, however, was assigned a grade of I, which means that evidence is lacking, of poor quality, or conflicting, and that the balance of benefits and harms cannot be determined. This USPSTF recommendation statement thus will not change practice for ObGyn providers but likely will renew our commitment to provide individualized well-woman care. There was inadequate or poor quality evidence for benefits related to all-cause mortality, disease-specific morbidity, and quality of life, as well as inadequate evidence on harms related to false-positive findings and anxiety stemming from screening pelvic exams.

Read about coding and billing for a standard pelvic exam

The pelvic examination and insurance coverage

---

Melanie Witt, RN, MA

Coding and billing for the care provided at a well-woman visit can be uncomplicated if you know the right codes for the right program. The information presented here concerns straightforward preventive care and assumes that the patient also has not presented with a significant problem at the same visit.

First, a patient who is not Medicare-eligible might have insurance coverage for an annual preventive care examination every year. Normally, this service would be billed using the Current Procedural Terminology (CPT) preventive medicine codes, but some insurers require the use of special codes for an annual gynecologic exam. These special codes are:

• S0610, Annual gynecological examination, new patient
• S0612, Annual gynecological examination, established patient
• S0613, Annual gynecological examination; clinical breast examination without pelvic evaluation.

Notably, Aetna, Cigna, and UnitedHealthcare require these codes to signify that a pelvic examination has been performed (except for code S0613), but many Blue Cross Blue Shield programs, for whom these codes were originally created, are now reverting to the CPT preventive medicine codes for all preventive care.

CPT outlines the requirements for use of the preventive medicine codes as: an initial or periodic comprehensive preventive medicine evaluation or reevaluation and management (E/M) service, which includes an age- and gender-appropriate history, examination, counseling/anticipatory guidance/risk factor reduction interventions, and the ordering of laboratory/diagnostic procedures. The codes are divided into new or established patient categories by age range as follows:

The Medicare E/M documentation guidelines do not apply to preventive services, and a head-to-toe examination also is not required. CPT recognizes the American College of Obstetricians and Gynecologists (ACOG) as an authoritative body to make recommendations for the expected preventive service for women, and if such a service is provided and documented, the preventive care codes are to be reported. The payers who use the S codes for a gynecologic exam will require that a pelvic examination has been performed, but such an examination would not be required when using the CPT codes or ACOG's guidelines if the physician and patient agreed that such an exam was not warranted every year. The other components of a preventive service applicable to the female patient's age, however, should be documented in order to report the CPT codes for preventive medicine services.

If a pelvic examination is not performed, say because the patient is young and not sexually active, but an examination of other areas is carried out, the diagnosis code would change from Z01.411, Encounter for gynecological examination (general) (routine) with abnormal findings, or Z01.419, Encounter for gynecological examination (general) (routine) without abnormal findings, to a general health exam: Z00.00, Encounter for general adult medical examination without abnormal findings, or Z00.01, Encounter for general adult medical examination with abnormal findings.  

What about Medicare?

Medicare requirements are somewhat different. First, Medicare covers only a small portion of the preventive care service; that is, it covers a physical examination of the genital organs and breasts and the collection and conveyance of a Pap specimen to the laboratory every 2 years for a low-risk patient. Second, the codes required to get reimbursed for the examination are:

• G0101, Cervical or vaginal cancer screening; pelvic and clinical breast examination
• Q0091, Screening Papanicolaou smear; obtaining, preparing, and conveyance of cervical or vaginal smear to laboratory.

It is not necessary to perform both of these services every 2 years (for instance, the patient may not need a Pap smear every 2 years based on her age and history), but the benefit is available if the service is performed. If the woman is at high risk for developing cervical or vaginal cancer, Medicare will cover this portion of the encounter every year so long as the Medicare-defined criteria for high risk have been documented at the time of the exam.

Related article:
GYN coding changes to note for your maximized reimbursement

Ms. Witt is an independent coding and documentation consultant and former program manager, department of coding and nomenclature, American Congress of Obstetricians and Gynecologists.

---

The author reports no financial relationships relevant to this article.

Read the authors’ interpretation of the new USPSTF statement

Interpreting the new USPSTF statement

We understand the USPSTF statement to mean that pelvic exams should not be abandoned, but rather should be individualized to each patient for her specific visit. We agree that for visits focused on counseling and routine screening in asymptomatic, nonpregnant women, pelvic exams likely will not increase the early detection and treatment of disease and more benefit likely would be derived by performing and discussing evidence-based and age-appropriate health services. A classic example would be for initiation or maintenance of oral contraception in an 18-year-old patient for whom an exam could cause unnecessary trauma, pain, or psychological distress leading to future avoidance or barriers to seeking health care. For long-acting reversible contraception placement, however, a pelvic exam clearly would be necessary for insertion of an intrauterine device.

Related article:
Women’s Preventive Services Initiative Guidelines provide consensus for practicing ObGyns

Indications for pelvic examination

Remember that the pelvic examination has 3 distinct parts (and that not all parts need to be routinely conducted)³:

• general inspection of the external genitalia and vulva
• speculum examination and evaluation of the vagina and cervix
• bimanual examination with possible rectovaginal examination in age-appropriate or symptomatic women.

According to the Well-Woman Task Force of the American College of Obstetricians and Gynecologists (ACOG), “For women 21 years and older, external exam may be performed annually and that inclusion of speculum examination, bimanual examination, or both in otherwise healthy women should be a shared, informed decision between patient and provider.”⁴

Indications for performing certain parts of the pelvic exam include⁴:

• routine screening for cervical cancer (Pap test)
• routine screening for gonorrhea, chlamydia infection, and other sexually transmitted infections
• evaluation of abnormal vaginal discharge
• evaluation of abnormal bleeding, pelvic pain, and pelvic floor disorders, such as prolapse, urinary incontinence, and accidental bowel leakage
• evaluation of menopausal symptoms, such as dryness, dyspareunia, and the genitourinary syndrome of menopause
• evaluation of women at increased risk for gynecologic malignancy, such as women with known hereditary breast–ovarian cancer syndromes.

In 2016, ACOG launched the Women’s Preventive Services Initiative (WPSI) in conjunction with the Health Resources and Services Administration (HRSA) of the US Department of Health and Human Services. In this 5-year collaboration, the agencies are endeavoring to review and update the recommendations for women’s preventive health care services, including well-woman visits, human papillomavirus testing, and contraception, among many others.⁵ Once the HRSA adopts these recommendations, women will be able to access comprehensive preventive health services without incurring any out-of-pocket expenses.

The pediatric and adolescent gynecologist perspective

---

Roshanak Mansouri Zinn, MD, and Rebekah L. Williams, MD, MS

No literature addresses the utility of screening pelvic examination in the pediatric and adolescent population. According to the American College of Obstetricians and Gynecologists Committee on Adolescent Health Care opinion on the initial reproductive health visit for screening and preventive reproductive health care (reaffirmed in 2016), a screening internal exam is not necessary, but an external genital exam may be indicated and may vary depending on the patient's concerns and prior clinical encounters.¹ The American Academy of Pediatrics promotes annual screening external genital examination for all female patients as part of routine primary care, with internal examinations only as indicated.²

Age-appropriate pelvic examination for girls and nonsexually active adolescents usually is limited to an external genital exam to evaluate the anatomy and note the sexual maturity rating (Tanner stage), an important indicator of normal pubertal development. As in adults, the potential benefits of screening examination in this population include detection of benign gynecologic conditions (including vulvar skin conditions and abnormalities of hymenal or vaginal development). Additionally, early reproductive health visits are an important time for clinicians to build rapport with younger patients and to provide anticipatory education on menstruation, hygiene, and anatomy. These visits can destigmatize and demystify the pelvic examination and help young women seek care more appropriately and more comfortably if problems do arise.

Even when a pelvic exam is indicated, a patient's young age can give providers pause as to what type of exam to perform. Patients with vulvovaginal symptoms, abnormal vaginal bleeding, vaginal discharge, or pelvic or abdominal pain should receive complete evaluation with external genital examination. If external vaginal examination does not allow for complete assessment of the problem, the patient and provider can assess the likelihood of her tolerating an internal exam in the clinic versus undergoing vaginoscopy under sedation. Limited laboratory evaluation and transabdominal pelvic ultrasonography may provide sufficient information for appropriate clinical decision making and management without internal examination. If symptoms persist or do not respond to first-line treatment, an internal exam should be performed.

Patients of any age may experience anxiety or physical discomfort or may even delay or avoid seeking care because of fear of a pelvic exam. However, providers of reproductive health care for children and adolescents can offer early education, reassurance, and a more comfortable experience when pelvic examination is necessary in this population.

References

1. American College of Obstetricians and Gynecologists Committee on Adolescent Health Care. Committee Opinion No. 598: Committee on Adolescent Health Care: the initial reproductive health visit. Obstet Gynecol. 2014;123(5):1143-1147.
2. Braverman PK, Breech L; Committee on Adolescence. American Academy of Pediatrics. Clinical report: gynecologic examination for adolescents in the pediatric office setting. Pediatrics. 2010;126(3):583-590.

 

Dr. Mansouri Zinn is Assistant Professor, Department of Women's Health, University of Texas at Austin.

Dr. Williams is Assistant Professor, Clinical Pediatrics, Section of Adolescent Medicine, Indiana University School of Medicine, Indianapolis.

Developed in collaboration with the North American Society for Pediatric and Adolescent Gynecology

---

The authors report no financial relationships relevant to this article.

How will the USPSTF statement affect practice?

In an editorial in the Journal of the American Medical Association commenting on the USPSTF statement, McNicholas and Peipert stated, “Based on the recommendation from the task force, clinicians may ask whether the pelvic examination should be abandoned. The answer is not found in this recommendation statement, but instead in a renewed commitment to shared decision making.”⁶ We wholeheartedly agree with this statement. The health care provider and the patient should make the decision, taking into consideration the patient’s risk factors for gynecologic cancers and other conditions, her personal preferences, and her overall values.

This new USPSTF recommendation statement will not change how we currently practice, and the statement’s grade I rating should not impact insurance coverage for pelvic exams. Additionally, further research is needed to better elucidate the role of the pelvic exam at well-woman visits, with hopes of obtaining more precise guidelines from the USPSTF and ACOG.

Share your thoughts! Send your Letter to the Editor to [email protected]. Please include your name and the city and state in which you practice.

© 2017 Society of Hospital Medicine

Strategies are needed to empower frontline clinicians to work with organizational leadership to reduce healthcare costs and improve high-value care. Caring Wisely^® is a program developed by the University of California, San Francisco’s (UCSF) Center for Healthcare Value (CHV), aimed at engaging frontline clinicians and staff, connecting them with implementation experts, and supporting the development of targeted interventions to improve value. Financial savings from the program more than cover program costs. Caring Wisely^® provides an institutional model for implementing robust interventions to address areas of low-value care.

Launched in 2013, the annual Caring Wisely^® program consists of 3 stages for identifying projects that meet the following criteria:

• Potential to measurably reduce UCSF Health’s costs of care without transferring costs to patients, insurers, or other providers
• Plan for ensuring that health outcomes are maintained or improved
• Envision disseminating the intervention within and beyond UCSF
• Demonstrate commitment and engagement of clinical leadership and frontline staff.

The first stage is the Ideas Contest, a UCSF Health-wide call (to learn more about UCSF Health: https://www.ucsf.edu/sites/default/files/052516_About_UCSF.pdf) to identify areas that may be targeted to reduce unnecessary services, inefficiencies, and healthcare costs. We use a crowdsourcing platform—Open Proposals—to solicit the best ideas from frontline clinicians and staff.¹ Open Proposals is a secure, web-based platform for transparent and collaborative proposal development that displays threads of comments, responses, and revisions, and allows submissions to be “liked.” Open Proposals is managed by the UCSF Clinical and Translational Science Institute, funded by the National Center for Advancing Translational Sciences (Grant Number UL1 TR000004) at the National Institutes of Health. Using institutional e-mail lists for faculty, staff and residents, as well as described at monthly managers and directors meetings, the Ideas Contest is announced each year by the Chief Medical Officer and the CHV leadership. The Caring Wisely^® Executive Steering Committee, which consists of CHV and senior UCSF Health system leaders, selects the top 5-10 ideas based on the above criteria. Each winning idea receives a $100 gift certificate for a popular restaurant in San Francisco, and the list of winners is announced to the entire UCSF community.

The second stage is the Request for Proposals. The Caring Wisely^® program solicits proposals that outline implementation plans to target specific areas identified through the Ideas Contest. Finalists from the Ideas Contest are encouraged to submit proposals that address the problem they identified, but anyone affiliated with UCSF Health may submit a proposal on a winning idea. There is an approximately 4-week open submission period during which applicants submit brief 2-page proposals on the Open Proposal platform. This is followed by a period of optimization that leverages the social media aspect of the Open Proposals platform in which the UCSF Health community asks clarifying questions, make suggestions, and modifications can be made to the proposals. All submissions receive written feedback from at least one Steering Committee member. In addition, the Caring Wisely^® Director directly invites relevant UCSF colleagues, administrators, or program leaders to comment on proposals and make suggestions for improvement. Plans for assessing financial and health care delivery impacts are developed in collaboration with the UCSF Health Finance department. UCSF Health managers and leaders who are stakeholders in project proposal areas are consulted to provide input and finalize proposal plans, including the identification of existing personnel who can support and drive the project forward. Proposers use this feedback to revise their applications throughout this stage.

The third stage is Project Implementation. The Caring Wisely^® Executive Steering Committee selects up to 3 winners from the submitted proposals. Using the program criteria above, each project is scored independently, discussed in committee, and rescored to identify the top proposals. Each selected project receives a maximum budget of $50,000 that can be used for project materials, activities, and salary support for project leaders or staff. In addition to funding, each project team receives input from the implementation science team to co-develop and implement the intervention with a goal of creating a first-test-of-change within 3-6 months. A key feature of Caring Wisely^® is the partnership between project teams and the Caring Wisely^® implementation team, which includes a director, program manager, data analysts, and implementation scientists (Table 1).

LIST OF ABBREVIATIONS

UCSF—University of California, San Francisco; PRECEDE—Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation; PROCEED—Policy, Regulatory and Organizational Constructs in Educational and Environmental Development

Acknowledgments

Other participants in blood transfusion reduction project (D. Johnson, K. Curcione); IV-to-PO Switch (C. Tsourounis, A. Pollock); Surgical Supply Cost Reduction (C. Zygourakis); Perioperative Efficiency (L. Hampson); CT for PE Risk Prediction (E. Weber); ERAS Pathways (L. Chen); Neonatal Sepsis Risk Prediction (T. Newman); Post-Operative Debrief (S. Imershein). Caring Wisely Executive Steering Committee (J. Adler, S. Antrum, A Auerbach, J. Bennan, M. Blum, C. Ritchie, C. Tsourounis). This Center for Healthcare Value is funded in part by a grant from the Grove Foundation. We appreciate additional review and comments to the manuscript provided by George Sawaya and Adams Dudley.

Disclosures 

Christopher Moriates has accepted royalties from McGraw-Hill for textbook, Understanding Value-Based Healthcare. Alvin Rajkomar has received fees as a research adviser from Google, Inc.

© 2017 Society of Hospital Medicine

Strategies are needed to empower frontline clinicians to work with organizational leadership to reduce healthcare costs and improve high-value care. Caring Wisely^® is a program developed by the University of California, San Francisco’s (UCSF) Center for Healthcare Value (CHV), aimed at engaging frontline clinicians and staff, connecting them with implementation experts, and supporting the development of targeted interventions to improve value. Financial savings from the program more than cover program costs. Caring Wisely^® provides an institutional model for implementing robust interventions to address areas of low-value care.

Launched in 2013, the annual Caring Wisely^® program consists of 3 stages for identifying projects that meet the following criteria:

• Potential to measurably reduce UCSF Health’s costs of care without transferring costs to patients, insurers, or other providers
• Plan for ensuring that health outcomes are maintained or improved
• Envision disseminating the intervention within and beyond UCSF
• Demonstrate commitment and engagement of clinical leadership and frontline staff.

The first stage is the Ideas Contest, a UCSF Health-wide call (to learn more about UCSF Health: https://www.ucsf.edu/sites/default/files/052516_About_UCSF.pdf) to identify areas that may be targeted to reduce unnecessary services, inefficiencies, and healthcare costs. We use a crowdsourcing platform—Open Proposals—to solicit the best ideas from frontline clinicians and staff.¹ Open Proposals is a secure, web-based platform for transparent and collaborative proposal development that displays threads of comments, responses, and revisions, and allows submissions to be “liked.” Open Proposals is managed by the UCSF Clinical and Translational Science Institute, funded by the National Center for Advancing Translational Sciences (Grant Number UL1 TR000004) at the National Institutes of Health. Using institutional e-mail lists for faculty, staff and residents, as well as described at monthly managers and directors meetings, the Ideas Contest is announced each year by the Chief Medical Officer and the CHV leadership. The Caring Wisely^® Executive Steering Committee, which consists of CHV and senior UCSF Health system leaders, selects the top 5-10 ideas based on the above criteria. Each winning idea receives a $100 gift certificate for a popular restaurant in San Francisco, and the list of winners is announced to the entire UCSF community.

The second stage is the Request for Proposals. The Caring Wisely^® program solicits proposals that outline implementation plans to target specific areas identified through the Ideas Contest. Finalists from the Ideas Contest are encouraged to submit proposals that address the problem they identified, but anyone affiliated with UCSF Health may submit a proposal on a winning idea. There is an approximately 4-week open submission period during which applicants submit brief 2-page proposals on the Open Proposal platform. This is followed by a period of optimization that leverages the social media aspect of the Open Proposals platform in which the UCSF Health community asks clarifying questions, make suggestions, and modifications can be made to the proposals. All submissions receive written feedback from at least one Steering Committee member. In addition, the Caring Wisely^® Director directly invites relevant UCSF colleagues, administrators, or program leaders to comment on proposals and make suggestions for improvement. Plans for assessing financial and health care delivery impacts are developed in collaboration with the UCSF Health Finance department. UCSF Health managers and leaders who are stakeholders in project proposal areas are consulted to provide input and finalize proposal plans, including the identification of existing personnel who can support and drive the project forward. Proposers use this feedback to revise their applications throughout this stage.

The third stage is Project Implementation. The Caring Wisely^® Executive Steering Committee selects up to 3 winners from the submitted proposals. Using the program criteria above, each project is scored independently, discussed in committee, and rescored to identify the top proposals. Each selected project receives a maximum budget of $50,000 that can be used for project materials, activities, and salary support for project leaders or staff. In addition to funding, each project team receives input from the implementation science team to co-develop and implement the intervention with a goal of creating a first-test-of-change within 3-6 months. A key feature of Caring Wisely^® is the partnership between project teams and the Caring Wisely^® implementation team, which includes a director, program manager, data analysts, and implementation scientists (Table 1).

LIST OF ABBREVIATIONS

UCSF—University of California, San Francisco; PRECEDE—Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation; PROCEED—Policy, Regulatory and Organizational Constructs in Educational and Environmental Development

Acknowledgments

Other participants in blood transfusion reduction project (D. Johnson, K. Curcione); IV-to-PO Switch (C. Tsourounis, A. Pollock); Surgical Supply Cost Reduction (C. Zygourakis); Perioperative Efficiency (L. Hampson); CT for PE Risk Prediction (E. Weber); ERAS Pathways (L. Chen); Neonatal Sepsis Risk Prediction (T. Newman); Post-Operative Debrief (S. Imershein). Caring Wisely Executive Steering Committee (J. Adler, S. Antrum, A Auerbach, J. Bennan, M. Blum, C. Ritchie, C. Tsourounis). This Center for Healthcare Value is funded in part by a grant from the Grove Foundation. We appreciate additional review and comments to the manuscript provided by George Sawaya and Adams Dudley.

Disclosures 

Christopher Moriates has accepted royalties from McGraw-Hill for textbook, Understanding Value-Based Healthcare. Alvin Rajkomar has received fees as a research adviser from Google, Inc.

© 2017 Society of Hospital Medicine

Strategies are needed to empower frontline clinicians to work with organizational leadership to reduce healthcare costs and improve high-value care. Caring Wisely^® is a program developed by the University of California, San Francisco’s (UCSF) Center for Healthcare Value (CHV), aimed at engaging frontline clinicians and staff, connecting them with implementation experts, and supporting the development of targeted interventions to improve value. Financial savings from the program more than cover program costs. Caring Wisely^® provides an institutional model for implementing robust interventions to address areas of low-value care.

Launched in 2013, the annual Caring Wisely^® program consists of 3 stages for identifying projects that meet the following criteria:

• Potential to measurably reduce UCSF Health’s costs of care without transferring costs to patients, insurers, or other providers
• Plan for ensuring that health outcomes are maintained or improved
• Envision disseminating the intervention within and beyond UCSF
• Demonstrate commitment and engagement of clinical leadership and frontline staff.

The first stage is the Ideas Contest, a UCSF Health-wide call (to learn more about UCSF Health: https://www.ucsf.edu/sites/default/files/052516_About_UCSF.pdf) to identify areas that may be targeted to reduce unnecessary services, inefficiencies, and healthcare costs. We use a crowdsourcing platform—Open Proposals—to solicit the best ideas from frontline clinicians and staff.¹ Open Proposals is a secure, web-based platform for transparent and collaborative proposal development that displays threads of comments, responses, and revisions, and allows submissions to be “liked.” Open Proposals is managed by the UCSF Clinical and Translational Science Institute, funded by the National Center for Advancing Translational Sciences (Grant Number UL1 TR000004) at the National Institutes of Health. Using institutional e-mail lists for faculty, staff and residents, as well as described at monthly managers and directors meetings, the Ideas Contest is announced each year by the Chief Medical Officer and the CHV leadership. The Caring Wisely^® Executive Steering Committee, which consists of CHV and senior UCSF Health system leaders, selects the top 5-10 ideas based on the above criteria. Each winning idea receives a $100 gift certificate for a popular restaurant in San Francisco, and the list of winners is announced to the entire UCSF community.

The second stage is the Request for Proposals. The Caring Wisely^® program solicits proposals that outline implementation plans to target specific areas identified through the Ideas Contest. Finalists from the Ideas Contest are encouraged to submit proposals that address the problem they identified, but anyone affiliated with UCSF Health may submit a proposal on a winning idea. There is an approximately 4-week open submission period during which applicants submit brief 2-page proposals on the Open Proposal platform. This is followed by a period of optimization that leverages the social media aspect of the Open Proposals platform in which the UCSF Health community asks clarifying questions, make suggestions, and modifications can be made to the proposals. All submissions receive written feedback from at least one Steering Committee member. In addition, the Caring Wisely^® Director directly invites relevant UCSF colleagues, administrators, or program leaders to comment on proposals and make suggestions for improvement. Plans for assessing financial and health care delivery impacts are developed in collaboration with the UCSF Health Finance department. UCSF Health managers and leaders who are stakeholders in project proposal areas are consulted to provide input and finalize proposal plans, including the identification of existing personnel who can support and drive the project forward. Proposers use this feedback to revise their applications throughout this stage.

The third stage is Project Implementation. The Caring Wisely^® Executive Steering Committee selects up to 3 winners from the submitted proposals. Using the program criteria above, each project is scored independently, discussed in committee, and rescored to identify the top proposals. Each selected project receives a maximum budget of $50,000 that can be used for project materials, activities, and salary support for project leaders or staff. In addition to funding, each project team receives input from the implementation science team to co-develop and implement the intervention with a goal of creating a first-test-of-change within 3-6 months. A key feature of Caring Wisely^® is the partnership between project teams and the Caring Wisely^® implementation team, which includes a director, program manager, data analysts, and implementation scientists (Table 1).

LIST OF ABBREVIATIONS

UCSF—University of California, San Francisco; PRECEDE—Predisposing, Reinforcing, and Enabling Constructs in Educational Diagnosis and Evaluation; PROCEED—Policy, Regulatory and Organizational Constructs in Educational and Environmental Development

Acknowledgments

Other participants in blood transfusion reduction project (D. Johnson, K. Curcione); IV-to-PO Switch (C. Tsourounis, A. Pollock); Surgical Supply Cost Reduction (C. Zygourakis); Perioperative Efficiency (L. Hampson); CT for PE Risk Prediction (E. Weber); ERAS Pathways (L. Chen); Neonatal Sepsis Risk Prediction (T. Newman); Post-Operative Debrief (S. Imershein). Caring Wisely Executive Steering Committee (J. Adler, S. Antrum, A Auerbach, J. Bennan, M. Blum, C. Ritchie, C. Tsourounis). This Center for Healthcare Value is funded in part by a grant from the Grove Foundation. We appreciate additional review and comments to the manuscript provided by George Sawaya and Adams Dudley.

Disclosures 

Christopher Moriates has accepted royalties from McGraw-Hill for textbook, Understanding Value-Based Healthcare. Alvin Rajkomar has received fees as a research adviser from Google, Inc.

© 2017 Society of Hospital Medicine

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care but which may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

Ammonia is predominantly generated in the gut by intestinal bacteria and enzymes and detoxified primarily in the liver. Since the 1930s, ammonia has been identified as the principal culprit in hepatic encephalopathy (HE). Many physicians utilize serum ammonia to diagnose, assess severity, and determine the resolution of HE in patients with chronic liver disease (CLD) despite research showing that ammonia levels are unhelpful in all of these clinical circumstances. HE in patients with CLD is a clinical diagnosis of exclusion that should not be based on ammonia levels.

A 62-year-old man diagnosed with cirrhosis due to Hepatitis C and alcoholism was brought to the emergency department for alteration in mentation. He had scant melenic stools 5 days preceding his admission and did not exhibit overt signs or symptoms of infection. His systemic examination was normal except for somnolence, disorientation to space and time, asterixis, and ascites. His lab parameters were within normal limits except for an elevated blood urea nitrogen and thrombocytopenia. His blood cultures did not grow any organisms, and paracentesis ruled out spontaneous bacterial peritonitis. During his hospital stay, he underwent esophageal variceal banding and was effectively managed with lactulose and rifaximin. The patient was alert, fully oriented, and without asterixis at the time of discharge 6 days later. Would an elevated venous ammonia level at admission alter management? If the ammonia level was elevated, would serial ammonia measurements affect management?

The colonic microbiome produces ammonia from dietary nitrogen. In health, approximately 85% of it is detoxified by the liver and excreted as urea in urine, while muscle and brain tissue metabolize the remaining 15%. The process of transamination and the urea cycle prevents this metabolic product from accumulating in the body. The elevated levels of nitrogenous toxins, including ammonia, in the systemic circulation of patients with CLD occur due to hepatocellular dysfunction and/or portosystemic shunting. This hyperammonemia is compounded by reduced peripheral metabolism of ammonia by muscle as a consequence of cachexia and muscle atrophy. Astrocytes synthesize glutamine excessively in the setting of hyperammonemia, resulting in astrocyte swelling and the generation of reactive oxygen species. Astrocyte swelling, free radical generation, and increased inhibitory function of gamma-Aminobutyric Acid result in cerebral dysfunction.^1,2 HE manifests as a broad spectrum of neurological or psychiatric abnormalities ranging from subclinical alterations to coma and was commonly graded on the West Haven Criteria (WHC) of 0 to 4 (Table).³ The Grade 0 from the previous WHC, referenced in many trials included in this article, has been replaced with minimal HE in the newly updated WHC by the American Association for the Study of Liver Diseases and the European Association for the Study of the Liver.^4,5

The ammonia hypothesis posits that ammonia is key in the pathogenesis of HE.^6-10 Some of the common precipitants of HE—gastrointestinal bleeding, infection, and renal failure—promote hyperammonemia.¹¹ HE is treated with nonabsorbable disaccharides (lactulose and lactitol) and rifaximin, which reduce the serum concentration of ammonia. Given these associations between HE and ammonia, physicians have for decades tested serum ammonia levels to diagnose HE and chart its resolution. In a study conducted by the Bavarian Society of Gastroenterology,¹² 60% of the respondents to an anonymous questionnaire regularly performed ammonia analysis in all their patients with liver cirrhosis, believing that it efficiently diagnosed HE.

Accuracy of Serum Ammonia

Multiple factors affect the accuracy of ammonia levels. First, fist clenching or the use of a tourniquet during the process of phlebotomy can falsely increase ammonia levels.¹³ Second, some authors have argued that the source of the ammonia sample matters. Kramer et al.¹⁴ reported that partial pressure of ammonia correlated closely with the degree of clinical and electrophysiological abnormalities of HE. However, Nicolao et al.¹⁵ and Ong et al.¹⁶ showed that the blood ammonia levels, whether measured by total venous, total arterial, or partial pressure methods, were equivalent. Third, ammonia levels are dependent on the time to processing of the specimen. Inaccurate results may occur if the blood sample is not immediately placed on ice after collection or if it is not centrifuged within 15 minutes of collection.^17,18

Ammonia Levels and Diagnosis of HE

Even with proper collection and processing, ammonia levels in patients with CLD do not reliably diagnose HE. Gundling et al.¹⁹ determined the sensitivity and specificity of venous ammonia levels ≥ 55 µmol/L to diagnose HE to be 47.2% and 78.3%, respectively, by using a gold standard of the WHC and the critical flicker frequency test (a psychophysiologic test). The positive predictive and negative predictive values of ammonia were 77.3% and 48.6%, with an overall diagnostic accuracy of 59.3%. Approximately 60% of the patients with Grade 3 WHC HE had a normal ammonia level in this study. Ong et al¹⁶ found that only 31% of patients with CLD and no evidence of HE had a normal ammonia level.In other words, CLD patients with normal ammonia levels can have HE, and patients with elevated ammonia levels may have normal cognitive functioning.

Furthermore, ammonia levels are not a valid tool to diagnose HE even with an oral glutamine challenge.²⁰ Most importantly, HE is a clinical diagnosis reached following the exclusion of other likely causes of cerebral dysfunction, independent of the ammonia level.

Ammonia Levels and Staging HE

The grading of HE was introduced to assess the response to an intervention in patients with HE enrolled in clinical trials.²¹ Tools like the WHC (Table) categorize the severity of HE. Nicolao et al.¹⁵ noted significant overlap in the levels of ammonia between patients with HE Grades 1 and 2 when compared with patients with Grades 3 and 4. This considerable overlap in levels of ammonia was more evident among patients with Grades 0 to 2 per Ong’s study.¹⁶ Most importantly, hospitalists do not need ammonia levels to determine that a patient has HE Grade 3 or HE Grade 4 symptoms, as the stage is graded on clinical grounds only. Once other causes for cerebral dysfunction have been ruled out, the ammonia level does not add to the clinical picture.

Serial Ammonia Levels and Resolution of HE

If the ammonia hypothesis is the sole explanation for the pathogenesis of HE, then the resolution of HE symptoms should be associated with normalization of ammonia levels. Physicians have commonly followed ammonia levels serially throughout a hospital stay. Nicolao et al.¹⁵ evaluated the association of ammonia with HE. They noted that some of the CLD patients had unchanged or increasing levels of ammonia despite overt neurological improvement from their HE.¹⁵ Some have argued that the normalization of ammonia levels lag behind the clinical improvement by 48 hours after resolution of symptoms. In the Nicolao et al.¹⁵ study, ammonia levels for almost all of the patients did not normalize 48 hours after resolution of neurologic symptoms. Moreover, 29% of the patients were noted to have higher venous ammonia levels 48 hours after the resolution of neurologic symptoms.¹⁵ These data underscore why serial measurements of ammonia in patients with CLD are not useful. For patients with overt symptoms, clinicians can determine improvement based on serial exams.

• HE is a diagnosis of exclusion and is made on clinical grounds.
• Do not check serum ammonia levels in patients with CLD to diagnose HE, to assess the severity of HE, or to determine whether HE is resolving.
• Use your clinical evaluation to determine the severity and course of HE.
• Treatment should be tailored according to clinical findings, not ammonia levels.

The attraction of the ammonia theory to explain HE continues to lead physicians to check and follow blood ammonia levels in patients with CLD and suspected HE. However, ammonia measurement, as in the clinical vignette, should be replaced by a thorough clinical evaluation to rule out other causes for altered mental status. Serial exams of the patient should guide management, not ammonia levels.

Disclosure 

The authors report no conflicts of interest.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason”? Let us know what you do in your practice and propose ideas for other “Things We Do for No Reason” topics. Please join in the conversation online at Twitter (#TWDFNR)/Facebook, and don’t forget to “Like It” on Facebook or retweet it on Twitter.

© 2017 Society of Hospital Medicine

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care but which may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

Ammonia is predominantly generated in the gut by intestinal bacteria and enzymes and detoxified primarily in the liver. Since the 1930s, ammonia has been identified as the principal culprit in hepatic encephalopathy (HE). Many physicians utilize serum ammonia to diagnose, assess severity, and determine the resolution of HE in patients with chronic liver disease (CLD) despite research showing that ammonia levels are unhelpful in all of these clinical circumstances. HE in patients with CLD is a clinical diagnosis of exclusion that should not be based on ammonia levels.

A 62-year-old man diagnosed with cirrhosis due to Hepatitis C and alcoholism was brought to the emergency department for alteration in mentation. He had scant melenic stools 5 days preceding his admission and did not exhibit overt signs or symptoms of infection. His systemic examination was normal except for somnolence, disorientation to space and time, asterixis, and ascites. His lab parameters were within normal limits except for an elevated blood urea nitrogen and thrombocytopenia. His blood cultures did not grow any organisms, and paracentesis ruled out spontaneous bacterial peritonitis. During his hospital stay, he underwent esophageal variceal banding and was effectively managed with lactulose and rifaximin. The patient was alert, fully oriented, and without asterixis at the time of discharge 6 days later. Would an elevated venous ammonia level at admission alter management? If the ammonia level was elevated, would serial ammonia measurements affect management?

The colonic microbiome produces ammonia from dietary nitrogen. In health, approximately 85% of it is detoxified by the liver and excreted as urea in urine, while muscle and brain tissue metabolize the remaining 15%. The process of transamination and the urea cycle prevents this metabolic product from accumulating in the body. The elevated levels of nitrogenous toxins, including ammonia, in the systemic circulation of patients with CLD occur due to hepatocellular dysfunction and/or portosystemic shunting. This hyperammonemia is compounded by reduced peripheral metabolism of ammonia by muscle as a consequence of cachexia and muscle atrophy. Astrocytes synthesize glutamine excessively in the setting of hyperammonemia, resulting in astrocyte swelling and the generation of reactive oxygen species. Astrocyte swelling, free radical generation, and increased inhibitory function of gamma-Aminobutyric Acid result in cerebral dysfunction.^1,2 HE manifests as a broad spectrum of neurological or psychiatric abnormalities ranging from subclinical alterations to coma and was commonly graded on the West Haven Criteria (WHC) of 0 to 4 (Table).³ The Grade 0 from the previous WHC, referenced in many trials included in this article, has been replaced with minimal HE in the newly updated WHC by the American Association for the Study of Liver Diseases and the European Association for the Study of the Liver.^4,5

The ammonia hypothesis posits that ammonia is key in the pathogenesis of HE.^6-10 Some of the common precipitants of HE—gastrointestinal bleeding, infection, and renal failure—promote hyperammonemia.¹¹ HE is treated with nonabsorbable disaccharides (lactulose and lactitol) and rifaximin, which reduce the serum concentration of ammonia. Given these associations between HE and ammonia, physicians have for decades tested serum ammonia levels to diagnose HE and chart its resolution. In a study conducted by the Bavarian Society of Gastroenterology,¹² 60% of the respondents to an anonymous questionnaire regularly performed ammonia analysis in all their patients with liver cirrhosis, believing that it efficiently diagnosed HE.

Accuracy of Serum Ammonia

Multiple factors affect the accuracy of ammonia levels. First, fist clenching or the use of a tourniquet during the process of phlebotomy can falsely increase ammonia levels.¹³ Second, some authors have argued that the source of the ammonia sample matters. Kramer et al.¹⁴ reported that partial pressure of ammonia correlated closely with the degree of clinical and electrophysiological abnormalities of HE. However, Nicolao et al.¹⁵ and Ong et al.¹⁶ showed that the blood ammonia levels, whether measured by total venous, total arterial, or partial pressure methods, were equivalent. Third, ammonia levels are dependent on the time to processing of the specimen. Inaccurate results may occur if the blood sample is not immediately placed on ice after collection or if it is not centrifuged within 15 minutes of collection.^17,18

Ammonia Levels and Diagnosis of HE

Even with proper collection and processing, ammonia levels in patients with CLD do not reliably diagnose HE. Gundling et al.¹⁹ determined the sensitivity and specificity of venous ammonia levels ≥ 55 µmol/L to diagnose HE to be 47.2% and 78.3%, respectively, by using a gold standard of the WHC and the critical flicker frequency test (a psychophysiologic test). The positive predictive and negative predictive values of ammonia were 77.3% and 48.6%, with an overall diagnostic accuracy of 59.3%. Approximately 60% of the patients with Grade 3 WHC HE had a normal ammonia level in this study. Ong et al¹⁶ found that only 31% of patients with CLD and no evidence of HE had a normal ammonia level.In other words, CLD patients with normal ammonia levels can have HE, and patients with elevated ammonia levels may have normal cognitive functioning.

Furthermore, ammonia levels are not a valid tool to diagnose HE even with an oral glutamine challenge.²⁰ Most importantly, HE is a clinical diagnosis reached following the exclusion of other likely causes of cerebral dysfunction, independent of the ammonia level.

Ammonia Levels and Staging HE

The grading of HE was introduced to assess the response to an intervention in patients with HE enrolled in clinical trials.²¹ Tools like the WHC (Table) categorize the severity of HE. Nicolao et al.¹⁵ noted significant overlap in the levels of ammonia between patients with HE Grades 1 and 2 when compared with patients with Grades 3 and 4. This considerable overlap in levels of ammonia was more evident among patients with Grades 0 to 2 per Ong’s study.¹⁶ Most importantly, hospitalists do not need ammonia levels to determine that a patient has HE Grade 3 or HE Grade 4 symptoms, as the stage is graded on clinical grounds only. Once other causes for cerebral dysfunction have been ruled out, the ammonia level does not add to the clinical picture.

Serial Ammonia Levels and Resolution of HE

If the ammonia hypothesis is the sole explanation for the pathogenesis of HE, then the resolution of HE symptoms should be associated with normalization of ammonia levels. Physicians have commonly followed ammonia levels serially throughout a hospital stay. Nicolao et al.¹⁵ evaluated the association of ammonia with HE. They noted that some of the CLD patients had unchanged or increasing levels of ammonia despite overt neurological improvement from their HE.¹⁵ Some have argued that the normalization of ammonia levels lag behind the clinical improvement by 48 hours after resolution of symptoms. In the Nicolao et al.¹⁵ study, ammonia levels for almost all of the patients did not normalize 48 hours after resolution of neurologic symptoms. Moreover, 29% of the patients were noted to have higher venous ammonia levels 48 hours after the resolution of neurologic symptoms.¹⁵ These data underscore why serial measurements of ammonia in patients with CLD are not useful. For patients with overt symptoms, clinicians can determine improvement based on serial exams.

• HE is a diagnosis of exclusion and is made on clinical grounds.
• Do not check serum ammonia levels in patients with CLD to diagnose HE, to assess the severity of HE, or to determine whether HE is resolving.
• Use your clinical evaluation to determine the severity and course of HE.
• Treatment should be tailored according to clinical findings, not ammonia levels.

The attraction of the ammonia theory to explain HE continues to lead physicians to check and follow blood ammonia levels in patients with CLD and suspected HE. However, ammonia measurement, as in the clinical vignette, should be replaced by a thorough clinical evaluation to rule out other causes for altered mental status. Serial exams of the patient should guide management, not ammonia levels.

Disclosure 

The authors report no conflicts of interest.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason”? Let us know what you do in your practice and propose ideas for other “Things We Do for No Reason” topics. Please join in the conversation online at Twitter (#TWDFNR)/Facebook, and don’t forget to “Like It” on Facebook or retweet it on Twitter.

© 2017 Society of Hospital Medicine

The “Things We Do for No Reason” series reviews practices which have become common parts of hospital care but which may provide little value to our patients. Practices reviewed in the TWDFNR series do not represent “black and white” conclusions or clinical practice standards, but are meant as a starting place for research and active discussions among hospitalists and patients. We invite you to be part of that discussion. https://www.choosingwisely.org/

Ammonia is predominantly generated in the gut by intestinal bacteria and enzymes and detoxified primarily in the liver. Since the 1930s, ammonia has been identified as the principal culprit in hepatic encephalopathy (HE). Many physicians utilize serum ammonia to diagnose, assess severity, and determine the resolution of HE in patients with chronic liver disease (CLD) despite research showing that ammonia levels are unhelpful in all of these clinical circumstances. HE in patients with CLD is a clinical diagnosis of exclusion that should not be based on ammonia levels.

A 62-year-old man diagnosed with cirrhosis due to Hepatitis C and alcoholism was brought to the emergency department for alteration in mentation. He had scant melenic stools 5 days preceding his admission and did not exhibit overt signs or symptoms of infection. His systemic examination was normal except for somnolence, disorientation to space and time, asterixis, and ascites. His lab parameters were within normal limits except for an elevated blood urea nitrogen and thrombocytopenia. His blood cultures did not grow any organisms, and paracentesis ruled out spontaneous bacterial peritonitis. During his hospital stay, he underwent esophageal variceal banding and was effectively managed with lactulose and rifaximin. The patient was alert, fully oriented, and without asterixis at the time of discharge 6 days later. Would an elevated venous ammonia level at admission alter management? If the ammonia level was elevated, would serial ammonia measurements affect management?

The colonic microbiome produces ammonia from dietary nitrogen. In health, approximately 85% of it is detoxified by the liver and excreted as urea in urine, while muscle and brain tissue metabolize the remaining 15%. The process of transamination and the urea cycle prevents this metabolic product from accumulating in the body. The elevated levels of nitrogenous toxins, including ammonia, in the systemic circulation of patients with CLD occur due to hepatocellular dysfunction and/or portosystemic shunting. This hyperammonemia is compounded by reduced peripheral metabolism of ammonia by muscle as a consequence of cachexia and muscle atrophy. Astrocytes synthesize glutamine excessively in the setting of hyperammonemia, resulting in astrocyte swelling and the generation of reactive oxygen species. Astrocyte swelling, free radical generation, and increased inhibitory function of gamma-Aminobutyric Acid result in cerebral dysfunction.^1,2 HE manifests as a broad spectrum of neurological or psychiatric abnormalities ranging from subclinical alterations to coma and was commonly graded on the West Haven Criteria (WHC) of 0 to 4 (Table).³ The Grade 0 from the previous WHC, referenced in many trials included in this article, has been replaced with minimal HE in the newly updated WHC by the American Association for the Study of Liver Diseases and the European Association for the Study of the Liver.^4,5

The ammonia hypothesis posits that ammonia is key in the pathogenesis of HE.^6-10 Some of the common precipitants of HE—gastrointestinal bleeding, infection, and renal failure—promote hyperammonemia.¹¹ HE is treated with nonabsorbable disaccharides (lactulose and lactitol) and rifaximin, which reduce the serum concentration of ammonia. Given these associations between HE and ammonia, physicians have for decades tested serum ammonia levels to diagnose HE and chart its resolution. In a study conducted by the Bavarian Society of Gastroenterology,¹² 60% of the respondents to an anonymous questionnaire regularly performed ammonia analysis in all their patients with liver cirrhosis, believing that it efficiently diagnosed HE.

Accuracy of Serum Ammonia

Multiple factors affect the accuracy of ammonia levels. First, fist clenching or the use of a tourniquet during the process of phlebotomy can falsely increase ammonia levels.¹³ Second, some authors have argued that the source of the ammonia sample matters. Kramer et al.¹⁴ reported that partial pressure of ammonia correlated closely with the degree of clinical and electrophysiological abnormalities of HE. However, Nicolao et al.¹⁵ and Ong et al.¹⁶ showed that the blood ammonia levels, whether measured by total venous, total arterial, or partial pressure methods, were equivalent. Third, ammonia levels are dependent on the time to processing of the specimen. Inaccurate results may occur if the blood sample is not immediately placed on ice after collection or if it is not centrifuged within 15 minutes of collection.^17,18

Ammonia Levels and Diagnosis of HE

Even with proper collection and processing, ammonia levels in patients with CLD do not reliably diagnose HE. Gundling et al.¹⁹ determined the sensitivity and specificity of venous ammonia levels ≥ 55 µmol/L to diagnose HE to be 47.2% and 78.3%, respectively, by using a gold standard of the WHC and the critical flicker frequency test (a psychophysiologic test). The positive predictive and negative predictive values of ammonia were 77.3% and 48.6%, with an overall diagnostic accuracy of 59.3%. Approximately 60% of the patients with Grade 3 WHC HE had a normal ammonia level in this study. Ong et al¹⁶ found that only 31% of patients with CLD and no evidence of HE had a normal ammonia level.In other words, CLD patients with normal ammonia levels can have HE, and patients with elevated ammonia levels may have normal cognitive functioning.

Furthermore, ammonia levels are not a valid tool to diagnose HE even with an oral glutamine challenge.²⁰ Most importantly, HE is a clinical diagnosis reached following the exclusion of other likely causes of cerebral dysfunction, independent of the ammonia level.

Ammonia Levels and Staging HE

The grading of HE was introduced to assess the response to an intervention in patients with HE enrolled in clinical trials.²¹ Tools like the WHC (Table) categorize the severity of HE. Nicolao et al.¹⁵ noted significant overlap in the levels of ammonia between patients with HE Grades 1 and 2 when compared with patients with Grades 3 and 4. This considerable overlap in levels of ammonia was more evident among patients with Grades 0 to 2 per Ong’s study.¹⁶ Most importantly, hospitalists do not need ammonia levels to determine that a patient has HE Grade 3 or HE Grade 4 symptoms, as the stage is graded on clinical grounds only. Once other causes for cerebral dysfunction have been ruled out, the ammonia level does not add to the clinical picture.

Serial Ammonia Levels and Resolution of HE

If the ammonia hypothesis is the sole explanation for the pathogenesis of HE, then the resolution of HE symptoms should be associated with normalization of ammonia levels. Physicians have commonly followed ammonia levels serially throughout a hospital stay. Nicolao et al.¹⁵ evaluated the association of ammonia with HE. They noted that some of the CLD patients had unchanged or increasing levels of ammonia despite overt neurological improvement from their HE.¹⁵ Some have argued that the normalization of ammonia levels lag behind the clinical improvement by 48 hours after resolution of symptoms. In the Nicolao et al.¹⁵ study, ammonia levels for almost all of the patients did not normalize 48 hours after resolution of neurologic symptoms. Moreover, 29% of the patients were noted to have higher venous ammonia levels 48 hours after the resolution of neurologic symptoms.¹⁵ These data underscore why serial measurements of ammonia in patients with CLD are not useful. For patients with overt symptoms, clinicians can determine improvement based on serial exams.

• HE is a diagnosis of exclusion and is made on clinical grounds.
• Do not check serum ammonia levels in patients with CLD to diagnose HE, to assess the severity of HE, or to determine whether HE is resolving.
• Use your clinical evaluation to determine the severity and course of HE.
• Treatment should be tailored according to clinical findings, not ammonia levels.

The attraction of the ammonia theory to explain HE continues to lead physicians to check and follow blood ammonia levels in patients with CLD and suspected HE. However, ammonia measurement, as in the clinical vignette, should be replaced by a thorough clinical evaluation to rule out other causes for altered mental status. Serial exams of the patient should guide management, not ammonia levels.

Disclosure 

The authors report no conflicts of interest.

Do you think this is a low-value practice? Is this truly a “Thing We Do for No Reason”? Let us know what you do in your practice and propose ideas for other “Things We Do for No Reason” topics. Please join in the conversation online at Twitter (#TWDFNR)/Facebook, and don’t forget to “Like It” on Facebook or retweet it on Twitter.

BACKGROUND

Physiologic monitors are intended to prevent cardiac and respiratory arrest by generating alarms to alert clinicians to signs of instability. To minimize the probability that monitors will miss signs of deterioration, alarm algorithms and default parameters are often set to maximize sensitivity while sacrificing specificity.¹ As a result, monitors generate large numbers of nonactionable alarms—alarms that are either invalid and do not accurately represent the physiologic status of the patient or are valid but do not warrant clinical intervention.² Prior research has demonstrated that the pediatric intensive care unit (PICU) is responsible for a higher proportion of alarms than pediatric wards³ and a large proportion of these alarms, 87% - 97%, are nonactionable.^4-8 In national surveys of healthcare staff, respondents report that high alarm rates interrupt patient care and can lead clinicians to disable alarms entirely.⁹ Recent research has supported this, demonstrating that nurses who are exposed to higher numbers of alarms have slower response times to alarms.^4,10 In an attempt to mitigate safety risks, the Joint Commission in 2012 issued recommendations for hospitals to (a) establish guidelines for tailoring alarm settings and limits for individual patients and (b) identify situations in which alarms are not clinically necessary.¹¹

In order to address these recommendations within our PICU, we sought to evaluate the impact of a focused physiologic monitor alarm reduction intervention integrated into safety huddles. Safety huddles are brief, structured discussions among physicians, nurses, and other staff aiming to identify safety concerns.¹² Huddles offer an appropriate forum for reviewing alarm data and identifying patients whose high alarm rates may necessitate safe tailoring of alarm limits. Pilot data demonstrating high alarm rates among low-acuity PICU patients led us to hypothesize that low-acuity, high-alarm PICU patients would be a safe and effective target for an alarm huddle-based intervention.

In this study, we aimed to measure the impact of a structured safety huddle review of low-acuity PICU patients with high rates of priority alarms who were randomized to intervention compared with other low-acuity, high-alarm, concurrent, and historical control patients in the PICU.

Study Definitions

Priority alarm activation rate. We conceptualized priority alarms as any alarm for a clinical condition that requires a timely response to determine if intervention is necessary to save a patient’s life,⁴ yet little empirical data support its existence in the hospital. We operationally defined these alarms on the General Electric Solar physiologic monitoring devices as any potentially life-threatening events including lethal arrhythmias (asystole, ventricular tachycardia, and ventricular fibrillation) and alarms for vital signs (heart rate, respiratory rate, and oxygen saturation) outside of the set parameter limits. These alarms produced audible tones in the patient room and automatically sent text messages to the nurse’s phone and had the potential to contribute to alarm fatigue regardless of the nurse’s location.

High-alarm patients. High-alarm patients were those who had more than 40 priority alarms in the preceding 4 hours, representing the top 20% of alarm rates in the PICU according to prior quality improvement projects completed in our PICU.

Low-acuity patients. Prior to and during this study, patient acuity was determined using the OptiLink Patient Classification System (OptiLink Healthcare Management Systems, Inc.; Tigard, OR; www.optilinkhealthcare.com; see Appendix 1) for the PICU twice daily. Low-acuity patients comprised on average 16% of the PICU patients.

Setting and Subjects

This study was performed in the PICU at The Children’s Hospital of Philadelphia.

The PICU is made up of 3 separate wings: east, south, and west. Bed availability was the only factor determining patient placement on the east, south, or west wing; the physical bed location was not preferentially assigned based on diagnosis or disease severity. The east wing was the intervention unit where the huddles occurred.

The PICU is composed of 3 different geographical teams. Two of the teams are composed of 4 to 5 pediatric or emergency medicine residents, 1 fellow, and 1 attending covering the south and west wings. The third team, located on the east wing, is composed of 1 to 2 pediatric residents, 2 to 3 nurse practitioners, 1 fellow, and 1 attending. Bedside family-centered rounds are held at each patient room, with the bedside nurse participating by reading a nursing rounding script that includes vital signs, vascular access, continuous medications, and additional questions or concerns.

Control subjects were any monitored patients on any of the 3 wings of the PICU between April 1, 2015, and October 31, 2015. The control patients were in 2 categories: historical controls from April 1, 2015, to May 31, 2015, and concurrent controls from June 1, 2015, to October 31, 2015, who were located anywhere in the PICU. On each nonholiday weekday beginning June 1, 2015, we randomly selected up to 2 patients to receive the intervention. These were high-alarm, low-acuity patients on the east wing to be discussed in the daily morning huddle. If more than 2 high-alarm, low-acuity patients were eligible for intervention, they were randomly selected by using the RAND function in Microsoft Excel. The other low-acuity, high-alarm patients in the PICU were included as control patients. Patients were eligible for the study if they were present for the 4 hours prior to huddle and present past noon on the day of huddle. If patients met criteria as high-alarm, low-acuity patients on multiple days, they could be enrolled as intervention or control patients multiple times. Patients’ alarm rates were calculated by dividing the number of alarms by their length of stay to the minute. There was no adjustment made for patients enrolled more than once.

Human Subjects Protection

The Institutional Review Board of The Children’s Hospital of Philadelphia approved this study with a waiver of informed consent.

Alarm Capture

We used BedMasterEx (Excel Medical Electronics; Jupiter, FL, http://excel-medical.com/products/bedmaster-ex) software connected to the General Electric monitor network to measure alarm rates. The software captured, in near real time, every alarm that occurred on every monitor in the PICU. Alarm rates over the preceding 4 hours for all PICU patients were exported and summarized by alarm type and level as set by hospital policy (crisis, warning, advisory, and system warning). Crisis and warning alarms were included as they represented potential life-threatening events meeting the definition of priority alarms. Physicians used an order within the PICU admission order-set to order monitoring based on preset age parameters (see online Appendix 1 for default settings). Physician orders were required for nurses to change alarm parameters. Daily electrode changes to reduce false alarms were standard of care.

Primary Outcome

The primary outcome was the change in priority alarm activation rate (the number of priority alarms per day) from prehuddle period (24 hours before morning huddle) to posthuddle period (the 24 hours following morning huddle) for intervention cases as compared with controls.

Primary Intervention

The intervention consisted of integrating a short script to facilitate the discussion of the alarm data during existing safety huddle and rounding workflows. The discussion and subsequent workflow proceeded as follows: A member of the research team who was not involved in patient care brought an alarm data sheet for each randomly selected intervention patient on the east wing to each safety huddle. The huddles were attended by the outgoing night charge nurse, the day charge nurse, and all bedside nurses working on the east wing that day. The alarm data sheet provided to the charge nurse displayed data on the 1 to 2 alarm parameters (respiratory rate, heart rate, or pulse oximetry) that generated the highest number of alarms. The charge nurse listed the high-alarm patients by room number during huddle, and the alarm data sheet was given to the bedside nurse responsible for the patient to facilitate further scripted discussion during bedside rounds with patient-specific information to reduce the alarm rates of individual patients throughout the adjustment of physiologic monitor parameters (see Appendix 2 for sample data sheet and script).

Data Collection

Intervention patients were high-alarm, low-acuity patients on the east wing from June 1, 2015, through October 31, 2015. Two months of baseline data were gathered prior to intervention on all 3 wings; therefore, control patients were high-alarm, low-acuity patients throughout the PICU from April 1, 2015, to May 31, 2015, as historical controls and from June 1, 2015, to October 31, 2015, as concurrent controls. Alarm rates for the 24 hours prior to huddle and the 24 hours following huddle were collected and analyzed. See Figure 1 for schematic of study design.

We collected data on patient characteristics, including patient location, age, sex, and intervention date. Information regarding changes to monitor alarm parameters for both intervention and control patients during the posthuddle period (the period following morning huddle until noon on intervention day) was also collected. We monitored for code blue events and unexpected changes in acuity until discharge or transfer out of the PICU.

Data Analysis

We compared the priority alarm activation rates of individual patients in the 24 hours before and the 24 hours after the huddle intervention and contrasted the differences in rates between intervention and control patients, both concurrent and historical controls. We also divided the intervention and control groups into 2 additional groups each—those patients whose alarm parameters were changed, compared with those whose parameters did not change. We evaluated for possible contamination by comparing alarm rates of historical and concurrent controls, as well as evaluating alarm rates by location. We used mixed-effects regression models to evaluate the effect of the intervention and control type (historical or concurrent) on alarm rates, adjusted for patient age and sex. Analysis was performed using Stata version 10.3 (StataCorp, LLC, College Station, TX) and SAS version 9.4 (SAS Institute Inc., Cary, NC).

Because patients could be enrolled more than once, we refer to the instances when they were included in the study as “events” (huddle discussions for intervention patients and huddle opportunities for controls) below. We identified 49 historical control events between April 1, 2015, and May 31, 2015. During the intervention period, we identified 88 intervention events and 163 concurrent control events between June 1, 2015, and October 31, 2015 (total n = 300; see Table 1 for event characteristics). A total of 6 patients were enrolled more than once as either intervention or control patients.

The average priority alarm activation rate for intervention patients was 433 alarms (95% confidence interval [CI], 392-472) per day in the 24 hours leading up to the intervention and 223 alarms (95% CI, 182-265) per day in the 24 hours following the intervention, a 48.5% unadjusted decrease (95% CI, 38.1%-58.9%). In contrast, priority alarm activation rates for concurrent control patients averaged 412 alarms (95% CI, 383-442) per day in the 24 hours leading up to the morning huddle and 323 alarms (95% CI, 270-375) per day in the 24 hours following huddle, a 21.6% unadjusted decrease (95% CI, 15.3%-27.9%). For historical controls, priority alarm activation rates averaged 369 alarms (95% CI, 339-399) per day in the 24 hours leading up to the morning huddle and 242 alarms (95% CI, 164-320) per day in the 24 hours following huddle, a 34.4% unadjusted decrease (95% CI, 13.5%-55.0%). When we compared historical versus concurrent controls in the unadjusted analysis, concurrent controls had 37 more alarms per day (95% CI, 59 fewer to 134 more; P = 0.45) than historical controls. There was no significant difference between concurrent and historical controls, demonstrating no evidence of contamination.

Adjusted Analysis of Changes in Alarm Rates

The overall estimate of the effect of the intervention adjusted for age and sex compared with concurrent controls was a reduction of 116 priority alarms per day (95% CI, 37-194; P = 0.004, Table 2). The adjusted percent decrease was 29.0% (95% CI, 12.1%-46.0%). There were no unexpected changes in patient acuity or code blue events related to the intervention.

Fidelity Analysis

We tracked changes in alarm parameter settings for evidence of intervention fidelity to determine if the team carried out the recommendations made. We found that 42% of intervention patients and 24% of combined control patients had alarm parameters changed during the posthuddle period (P = 0.002).

For those intervention patients who had parameters changed during the posthuddle period (N = 37), the mean effect was greater at a 54.9% decrease (95% CI, 38.8%-70.8%) in priority alarms as compared with control patients who had parameters adjusted during the posthuddle period (n = 50), having a mean decrease of only 12.2% (95% CI, –18.1%-42.3%). There was a 43.2% decrease (95% CI, 29.3%-57.0%) for intervention patients who were discussed but did not have parameters adjusted during the time window of observation (n = 51), as compared with combined control patients who did not have parameters adjusted (N = 162) who had a 28.1% decrease (95% CI, 16.8%-39.1%); see Figure 2.

This study is the first to demonstrate a successful and safe intervention to reduce the alarm rates of PICU patients. In addition, we observed a more significant reduction in priority alarm activation rates for intervention patients who had their alarm parameters changed during the monitored time period, leading us to hypothesize that providing patient-specific data regarding types of alarms was a key component of the intervention.

In control patients, we observed a reduction in alarm rates over time as well. There are 2 potential explanations for this. First, it is possible that as patients stabilize in the PICU, their vital signs become less extreme and generate fewer alarms even if the alarm parameters are not changed. The second is that parameters were changed within or outside of the time windows during which we evaluated for alarm parameter changes. Nevertheless, the decline over time observed in the intervention patients was greater than in both control groups. This change was even more noticeable in the intervention patients who had their alarm parameters changed during the posthuddle period as compared with controls who had their alarm parameters changed following the posthuddle period. This may have been due to the data provided during the huddle intervention, pointing the team to the cause of the high alarm rate.

Prior successful research regarding reduction of pediatric alarms has often shown decreased use of physiological monitors as 1 approach to reducing unnecessary alarms. The single prior pediatric alarm intervention study conducted on a pediatric ward involved instituting a cardiac monitor care process that included the ordering of age-based parameters, daily replacement of electrodes, individualized assessment of parameters, and a reliable method to discontinue monitoring.¹³ Because most patients in the PICU are critically ill, the reliance on monitor discontinuation as a main approach to decreasing alarms is not feasible in this setting. Instead, the use of targeted alarm parameter adjustments for low-acuity patients demonstrated a safe and feasible approach to decreasing alarms in PICU patients. The daily electrode change and age-based parameters were already in place at our institution.

There are a few limitations to this study. First, we focused only on low-acuity PICU patients. We believe that focusing on low-acuity patients allows for reduction in nonactionable alarms with limited potential for adverse events; however, this approach excludes many critically ill patients who might be at highest risk for harm from alarm fatigue if important alarms are ignored. Second, many of our patients were not present for the full 24 hours pre- and posthuddle due to their low acuity limiting our ability to follow alarm rates over time. Third, changes in alarm parameters were only monitored for a set period of 5 hours following the huddle to determine the effect of the recommended rounding script on changes to alarms. It is possible the changes to alarm parameters outside of the observed posthuddle period affected the alarm rates of both intervention and control patients. Lastly, the balancing metrics of unexpected changes in OptiLink status and code blue events are rare events, and therefore we may have been underpowered to find them. The effects of the huddle intervention on safety huddle length and rounding length were not measured.

Integrating a data-driven monitor alarm discussion into safety huddles was a safe and effective approach to reduce alarms in low-acuity, high-alarm PICU patients. Innovative approaches to make data-driven alarm decisions using informatics tools integrated into monitoring systems and electronic health records have the potential to facilitate cost-effective spread of this intervention.

Disclosure

This work was supported by a pilot grant from the Center for Pediatric Clinical Effectiveness, The Children’s Hospital of Philadelphia. Dr. Bonafide is supported by a Mentored Patient-Oriented Research Career Development Award from the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number K23HL116427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations or employers. The funding organizations had no role in the design, preparation, review, or approval of this paper, nor the decision to submit for publication.

BACKGROUND

Physiologic monitors are intended to prevent cardiac and respiratory arrest by generating alarms to alert clinicians to signs of instability. To minimize the probability that monitors will miss signs of deterioration, alarm algorithms and default parameters are often set to maximize sensitivity while sacrificing specificity.¹ As a result, monitors generate large numbers of nonactionable alarms—alarms that are either invalid and do not accurately represent the physiologic status of the patient or are valid but do not warrant clinical intervention.² Prior research has demonstrated that the pediatric intensive care unit (PICU) is responsible for a higher proportion of alarms than pediatric wards³ and a large proportion of these alarms, 87% - 97%, are nonactionable.^4-8 In national surveys of healthcare staff, respondents report that high alarm rates interrupt patient care and can lead clinicians to disable alarms entirely.⁹ Recent research has supported this, demonstrating that nurses who are exposed to higher numbers of alarms have slower response times to alarms.^4,10 In an attempt to mitigate safety risks, the Joint Commission in 2012 issued recommendations for hospitals to (a) establish guidelines for tailoring alarm settings and limits for individual patients and (b) identify situations in which alarms are not clinically necessary.¹¹

In order to address these recommendations within our PICU, we sought to evaluate the impact of a focused physiologic monitor alarm reduction intervention integrated into safety huddles. Safety huddles are brief, structured discussions among physicians, nurses, and other staff aiming to identify safety concerns.¹² Huddles offer an appropriate forum for reviewing alarm data and identifying patients whose high alarm rates may necessitate safe tailoring of alarm limits. Pilot data demonstrating high alarm rates among low-acuity PICU patients led us to hypothesize that low-acuity, high-alarm PICU patients would be a safe and effective target for an alarm huddle-based intervention.

In this study, we aimed to measure the impact of a structured safety huddle review of low-acuity PICU patients with high rates of priority alarms who were randomized to intervention compared with other low-acuity, high-alarm, concurrent, and historical control patients in the PICU.

Study Definitions

Priority alarm activation rate. We conceptualized priority alarms as any alarm for a clinical condition that requires a timely response to determine if intervention is necessary to save a patient’s life,⁴ yet little empirical data support its existence in the hospital. We operationally defined these alarms on the General Electric Solar physiologic monitoring devices as any potentially life-threatening events including lethal arrhythmias (asystole, ventricular tachycardia, and ventricular fibrillation) and alarms for vital signs (heart rate, respiratory rate, and oxygen saturation) outside of the set parameter limits. These alarms produced audible tones in the patient room and automatically sent text messages to the nurse’s phone and had the potential to contribute to alarm fatigue regardless of the nurse’s location.

High-alarm patients. High-alarm patients were those who had more than 40 priority alarms in the preceding 4 hours, representing the top 20% of alarm rates in the PICU according to prior quality improvement projects completed in our PICU.

Low-acuity patients. Prior to and during this study, patient acuity was determined using the OptiLink Patient Classification System (OptiLink Healthcare Management Systems, Inc.; Tigard, OR; www.optilinkhealthcare.com; see Appendix 1) for the PICU twice daily. Low-acuity patients comprised on average 16% of the PICU patients.

Setting and Subjects

This study was performed in the PICU at The Children’s Hospital of Philadelphia.

The PICU is made up of 3 separate wings: east, south, and west. Bed availability was the only factor determining patient placement on the east, south, or west wing; the physical bed location was not preferentially assigned based on diagnosis or disease severity. The east wing was the intervention unit where the huddles occurred.

The PICU is composed of 3 different geographical teams. Two of the teams are composed of 4 to 5 pediatric or emergency medicine residents, 1 fellow, and 1 attending covering the south and west wings. The third team, located on the east wing, is composed of 1 to 2 pediatric residents, 2 to 3 nurse practitioners, 1 fellow, and 1 attending. Bedside family-centered rounds are held at each patient room, with the bedside nurse participating by reading a nursing rounding script that includes vital signs, vascular access, continuous medications, and additional questions or concerns.

Control subjects were any monitored patients on any of the 3 wings of the PICU between April 1, 2015, and October 31, 2015. The control patients were in 2 categories: historical controls from April 1, 2015, to May 31, 2015, and concurrent controls from June 1, 2015, to October 31, 2015, who were located anywhere in the PICU. On each nonholiday weekday beginning June 1, 2015, we randomly selected up to 2 patients to receive the intervention. These were high-alarm, low-acuity patients on the east wing to be discussed in the daily morning huddle. If more than 2 high-alarm, low-acuity patients were eligible for intervention, they were randomly selected by using the RAND function in Microsoft Excel. The other low-acuity, high-alarm patients in the PICU were included as control patients. Patients were eligible for the study if they were present for the 4 hours prior to huddle and present past noon on the day of huddle. If patients met criteria as high-alarm, low-acuity patients on multiple days, they could be enrolled as intervention or control patients multiple times. Patients’ alarm rates were calculated by dividing the number of alarms by their length of stay to the minute. There was no adjustment made for patients enrolled more than once.

Human Subjects Protection

The Institutional Review Board of The Children’s Hospital of Philadelphia approved this study with a waiver of informed consent.

Alarm Capture

We used BedMasterEx (Excel Medical Electronics; Jupiter, FL, http://excel-medical.com/products/bedmaster-ex) software connected to the General Electric monitor network to measure alarm rates. The software captured, in near real time, every alarm that occurred on every monitor in the PICU. Alarm rates over the preceding 4 hours for all PICU patients were exported and summarized by alarm type and level as set by hospital policy (crisis, warning, advisory, and system warning). Crisis and warning alarms were included as they represented potential life-threatening events meeting the definition of priority alarms. Physicians used an order within the PICU admission order-set to order monitoring based on preset age parameters (see online Appendix 1 for default settings). Physician orders were required for nurses to change alarm parameters. Daily electrode changes to reduce false alarms were standard of care.

Primary Outcome

The primary outcome was the change in priority alarm activation rate (the number of priority alarms per day) from prehuddle period (24 hours before morning huddle) to posthuddle period (the 24 hours following morning huddle) for intervention cases as compared with controls.

Primary Intervention

The intervention consisted of integrating a short script to facilitate the discussion of the alarm data during existing safety huddle and rounding workflows. The discussion and subsequent workflow proceeded as follows: A member of the research team who was not involved in patient care brought an alarm data sheet for each randomly selected intervention patient on the east wing to each safety huddle. The huddles were attended by the outgoing night charge nurse, the day charge nurse, and all bedside nurses working on the east wing that day. The alarm data sheet provided to the charge nurse displayed data on the 1 to 2 alarm parameters (respiratory rate, heart rate, or pulse oximetry) that generated the highest number of alarms. The charge nurse listed the high-alarm patients by room number during huddle, and the alarm data sheet was given to the bedside nurse responsible for the patient to facilitate further scripted discussion during bedside rounds with patient-specific information to reduce the alarm rates of individual patients throughout the adjustment of physiologic monitor parameters (see Appendix 2 for sample data sheet and script).

Data Collection

Intervention patients were high-alarm, low-acuity patients on the east wing from June 1, 2015, through October 31, 2015. Two months of baseline data were gathered prior to intervention on all 3 wings; therefore, control patients were high-alarm, low-acuity patients throughout the PICU from April 1, 2015, to May 31, 2015, as historical controls and from June 1, 2015, to October 31, 2015, as concurrent controls. Alarm rates for the 24 hours prior to huddle and the 24 hours following huddle were collected and analyzed. See Figure 1 for schematic of study design.

We collected data on patient characteristics, including patient location, age, sex, and intervention date. Information regarding changes to monitor alarm parameters for both intervention and control patients during the posthuddle period (the period following morning huddle until noon on intervention day) was also collected. We monitored for code blue events and unexpected changes in acuity until discharge or transfer out of the PICU.

Data Analysis

We compared the priority alarm activation rates of individual patients in the 24 hours before and the 24 hours after the huddle intervention and contrasted the differences in rates between intervention and control patients, both concurrent and historical controls. We also divided the intervention and control groups into 2 additional groups each—those patients whose alarm parameters were changed, compared with those whose parameters did not change. We evaluated for possible contamination by comparing alarm rates of historical and concurrent controls, as well as evaluating alarm rates by location. We used mixed-effects regression models to evaluate the effect of the intervention and control type (historical or concurrent) on alarm rates, adjusted for patient age and sex. Analysis was performed using Stata version 10.3 (StataCorp, LLC, College Station, TX) and SAS version 9.4 (SAS Institute Inc., Cary, NC).

Because patients could be enrolled more than once, we refer to the instances when they were included in the study as “events” (huddle discussions for intervention patients and huddle opportunities for controls) below. We identified 49 historical control events between April 1, 2015, and May 31, 2015. During the intervention period, we identified 88 intervention events and 163 concurrent control events between June 1, 2015, and October 31, 2015 (total n = 300; see Table 1 for event characteristics). A total of 6 patients were enrolled more than once as either intervention or control patients.

The average priority alarm activation rate for intervention patients was 433 alarms (95% confidence interval [CI], 392-472) per day in the 24 hours leading up to the intervention and 223 alarms (95% CI, 182-265) per day in the 24 hours following the intervention, a 48.5% unadjusted decrease (95% CI, 38.1%-58.9%). In contrast, priority alarm activation rates for concurrent control patients averaged 412 alarms (95% CI, 383-442) per day in the 24 hours leading up to the morning huddle and 323 alarms (95% CI, 270-375) per day in the 24 hours following huddle, a 21.6% unadjusted decrease (95% CI, 15.3%-27.9%). For historical controls, priority alarm activation rates averaged 369 alarms (95% CI, 339-399) per day in the 24 hours leading up to the morning huddle and 242 alarms (95% CI, 164-320) per day in the 24 hours following huddle, a 34.4% unadjusted decrease (95% CI, 13.5%-55.0%). When we compared historical versus concurrent controls in the unadjusted analysis, concurrent controls had 37 more alarms per day (95% CI, 59 fewer to 134 more; P = 0.45) than historical controls. There was no significant difference between concurrent and historical controls, demonstrating no evidence of contamination.

Adjusted Analysis of Changes in Alarm Rates

The overall estimate of the effect of the intervention adjusted for age and sex compared with concurrent controls was a reduction of 116 priority alarms per day (95% CI, 37-194; P = 0.004, Table 2). The adjusted percent decrease was 29.0% (95% CI, 12.1%-46.0%). There were no unexpected changes in patient acuity or code blue events related to the intervention.

Fidelity Analysis

We tracked changes in alarm parameter settings for evidence of intervention fidelity to determine if the team carried out the recommendations made. We found that 42% of intervention patients and 24% of combined control patients had alarm parameters changed during the posthuddle period (P = 0.002).

For those intervention patients who had parameters changed during the posthuddle period (N = 37), the mean effect was greater at a 54.9% decrease (95% CI, 38.8%-70.8%) in priority alarms as compared with control patients who had parameters adjusted during the posthuddle period (n = 50), having a mean decrease of only 12.2% (95% CI, –18.1%-42.3%). There was a 43.2% decrease (95% CI, 29.3%-57.0%) for intervention patients who were discussed but did not have parameters adjusted during the time window of observation (n = 51), as compared with combined control patients who did not have parameters adjusted (N = 162) who had a 28.1% decrease (95% CI, 16.8%-39.1%); see Figure 2.

This study is the first to demonstrate a successful and safe intervention to reduce the alarm rates of PICU patients. In addition, we observed a more significant reduction in priority alarm activation rates for intervention patients who had their alarm parameters changed during the monitored time period, leading us to hypothesize that providing patient-specific data regarding types of alarms was a key component of the intervention.

In control patients, we observed a reduction in alarm rates over time as well. There are 2 potential explanations for this. First, it is possible that as patients stabilize in the PICU, their vital signs become less extreme and generate fewer alarms even if the alarm parameters are not changed. The second is that parameters were changed within or outside of the time windows during which we evaluated for alarm parameter changes. Nevertheless, the decline over time observed in the intervention patients was greater than in both control groups. This change was even more noticeable in the intervention patients who had their alarm parameters changed during the posthuddle period as compared with controls who had their alarm parameters changed following the posthuddle period. This may have been due to the data provided during the huddle intervention, pointing the team to the cause of the high alarm rate.

Prior successful research regarding reduction of pediatric alarms has often shown decreased use of physiological monitors as 1 approach to reducing unnecessary alarms. The single prior pediatric alarm intervention study conducted on a pediatric ward involved instituting a cardiac monitor care process that included the ordering of age-based parameters, daily replacement of electrodes, individualized assessment of parameters, and a reliable method to discontinue monitoring.¹³ Because most patients in the PICU are critically ill, the reliance on monitor discontinuation as a main approach to decreasing alarms is not feasible in this setting. Instead, the use of targeted alarm parameter adjustments for low-acuity patients demonstrated a safe and feasible approach to decreasing alarms in PICU patients. The daily electrode change and age-based parameters were already in place at our institution.

There are a few limitations to this study. First, we focused only on low-acuity PICU patients. We believe that focusing on low-acuity patients allows for reduction in nonactionable alarms with limited potential for adverse events; however, this approach excludes many critically ill patients who might be at highest risk for harm from alarm fatigue if important alarms are ignored. Second, many of our patients were not present for the full 24 hours pre- and posthuddle due to their low acuity limiting our ability to follow alarm rates over time. Third, changes in alarm parameters were only monitored for a set period of 5 hours following the huddle to determine the effect of the recommended rounding script on changes to alarms. It is possible the changes to alarm parameters outside of the observed posthuddle period affected the alarm rates of both intervention and control patients. Lastly, the balancing metrics of unexpected changes in OptiLink status and code blue events are rare events, and therefore we may have been underpowered to find them. The effects of the huddle intervention on safety huddle length and rounding length were not measured.

Integrating a data-driven monitor alarm discussion into safety huddles was a safe and effective approach to reduce alarms in low-acuity, high-alarm PICU patients. Innovative approaches to make data-driven alarm decisions using informatics tools integrated into monitoring systems and electronic health records have the potential to facilitate cost-effective spread of this intervention.

Disclosure

This work was supported by a pilot grant from the Center for Pediatric Clinical Effectiveness, The Children’s Hospital of Philadelphia. Dr. Bonafide is supported by a Mentored Patient-Oriented Research Career Development Award from the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number K23HL116427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations or employers. The funding organizations had no role in the design, preparation, review, or approval of this paper, nor the decision to submit for publication.

BACKGROUND

Physiologic monitors are intended to prevent cardiac and respiratory arrest by generating alarms to alert clinicians to signs of instability. To minimize the probability that monitors will miss signs of deterioration, alarm algorithms and default parameters are often set to maximize sensitivity while sacrificing specificity.¹ As a result, monitors generate large numbers of nonactionable alarms—alarms that are either invalid and do not accurately represent the physiologic status of the patient or are valid but do not warrant clinical intervention.² Prior research has demonstrated that the pediatric intensive care unit (PICU) is responsible for a higher proportion of alarms than pediatric wards³ and a large proportion of these alarms, 87% - 97%, are nonactionable.^4-8 In national surveys of healthcare staff, respondents report that high alarm rates interrupt patient care and can lead clinicians to disable alarms entirely.⁹ Recent research has supported this, demonstrating that nurses who are exposed to higher numbers of alarms have slower response times to alarms.^4,10 In an attempt to mitigate safety risks, the Joint Commission in 2012 issued recommendations for hospitals to (a) establish guidelines for tailoring alarm settings and limits for individual patients and (b) identify situations in which alarms are not clinically necessary.¹¹

In order to address these recommendations within our PICU, we sought to evaluate the impact of a focused physiologic monitor alarm reduction intervention integrated into safety huddles. Safety huddles are brief, structured discussions among physicians, nurses, and other staff aiming to identify safety concerns.¹² Huddles offer an appropriate forum for reviewing alarm data and identifying patients whose high alarm rates may necessitate safe tailoring of alarm limits. Pilot data demonstrating high alarm rates among low-acuity PICU patients led us to hypothesize that low-acuity, high-alarm PICU patients would be a safe and effective target for an alarm huddle-based intervention.

In this study, we aimed to measure the impact of a structured safety huddle review of low-acuity PICU patients with high rates of priority alarms who were randomized to intervention compared with other low-acuity, high-alarm, concurrent, and historical control patients in the PICU.

Study Definitions

Priority alarm activation rate. We conceptualized priority alarms as any alarm for a clinical condition that requires a timely response to determine if intervention is necessary to save a patient’s life,⁴ yet little empirical data support its existence in the hospital. We operationally defined these alarms on the General Electric Solar physiologic monitoring devices as any potentially life-threatening events including lethal arrhythmias (asystole, ventricular tachycardia, and ventricular fibrillation) and alarms for vital signs (heart rate, respiratory rate, and oxygen saturation) outside of the set parameter limits. These alarms produced audible tones in the patient room and automatically sent text messages to the nurse’s phone and had the potential to contribute to alarm fatigue regardless of the nurse’s location.

High-alarm patients. High-alarm patients were those who had more than 40 priority alarms in the preceding 4 hours, representing the top 20% of alarm rates in the PICU according to prior quality improvement projects completed in our PICU.

Low-acuity patients. Prior to and during this study, patient acuity was determined using the OptiLink Patient Classification System (OptiLink Healthcare Management Systems, Inc.; Tigard, OR; www.optilinkhealthcare.com; see Appendix 1) for the PICU twice daily. Low-acuity patients comprised on average 16% of the PICU patients.

Setting and Subjects

This study was performed in the PICU at The Children’s Hospital of Philadelphia.

The PICU is made up of 3 separate wings: east, south, and west. Bed availability was the only factor determining patient placement on the east, south, or west wing; the physical bed location was not preferentially assigned based on diagnosis or disease severity. The east wing was the intervention unit where the huddles occurred.

The PICU is composed of 3 different geographical teams. Two of the teams are composed of 4 to 5 pediatric or emergency medicine residents, 1 fellow, and 1 attending covering the south and west wings. The third team, located on the east wing, is composed of 1 to 2 pediatric residents, 2 to 3 nurse practitioners, 1 fellow, and 1 attending. Bedside family-centered rounds are held at each patient room, with the bedside nurse participating by reading a nursing rounding script that includes vital signs, vascular access, continuous medications, and additional questions or concerns.

Control subjects were any monitored patients on any of the 3 wings of the PICU between April 1, 2015, and October 31, 2015. The control patients were in 2 categories: historical controls from April 1, 2015, to May 31, 2015, and concurrent controls from June 1, 2015, to October 31, 2015, who were located anywhere in the PICU. On each nonholiday weekday beginning June 1, 2015, we randomly selected up to 2 patients to receive the intervention. These were high-alarm, low-acuity patients on the east wing to be discussed in the daily morning huddle. If more than 2 high-alarm, low-acuity patients were eligible for intervention, they were randomly selected by using the RAND function in Microsoft Excel. The other low-acuity, high-alarm patients in the PICU were included as control patients. Patients were eligible for the study if they were present for the 4 hours prior to huddle and present past noon on the day of huddle. If patients met criteria as high-alarm, low-acuity patients on multiple days, they could be enrolled as intervention or control patients multiple times. Patients’ alarm rates were calculated by dividing the number of alarms by their length of stay to the minute. There was no adjustment made for patients enrolled more than once.

Human Subjects Protection

The Institutional Review Board of The Children’s Hospital of Philadelphia approved this study with a waiver of informed consent.

Alarm Capture

We used BedMasterEx (Excel Medical Electronics; Jupiter, FL, http://excel-medical.com/products/bedmaster-ex) software connected to the General Electric monitor network to measure alarm rates. The software captured, in near real time, every alarm that occurred on every monitor in the PICU. Alarm rates over the preceding 4 hours for all PICU patients were exported and summarized by alarm type and level as set by hospital policy (crisis, warning, advisory, and system warning). Crisis and warning alarms were included as they represented potential life-threatening events meeting the definition of priority alarms. Physicians used an order within the PICU admission order-set to order monitoring based on preset age parameters (see online Appendix 1 for default settings). Physician orders were required for nurses to change alarm parameters. Daily electrode changes to reduce false alarms were standard of care.

Primary Outcome

The primary outcome was the change in priority alarm activation rate (the number of priority alarms per day) from prehuddle period (24 hours before morning huddle) to posthuddle period (the 24 hours following morning huddle) for intervention cases as compared with controls.

Primary Intervention

The intervention consisted of integrating a short script to facilitate the discussion of the alarm data during existing safety huddle and rounding workflows. The discussion and subsequent workflow proceeded as follows: A member of the research team who was not involved in patient care brought an alarm data sheet for each randomly selected intervention patient on the east wing to each safety huddle. The huddles were attended by the outgoing night charge nurse, the day charge nurse, and all bedside nurses working on the east wing that day. The alarm data sheet provided to the charge nurse displayed data on the 1 to 2 alarm parameters (respiratory rate, heart rate, or pulse oximetry) that generated the highest number of alarms. The charge nurse listed the high-alarm patients by room number during huddle, and the alarm data sheet was given to the bedside nurse responsible for the patient to facilitate further scripted discussion during bedside rounds with patient-specific information to reduce the alarm rates of individual patients throughout the adjustment of physiologic monitor parameters (see Appendix 2 for sample data sheet and script).

Data Collection

Intervention patients were high-alarm, low-acuity patients on the east wing from June 1, 2015, through October 31, 2015. Two months of baseline data were gathered prior to intervention on all 3 wings; therefore, control patients were high-alarm, low-acuity patients throughout the PICU from April 1, 2015, to May 31, 2015, as historical controls and from June 1, 2015, to October 31, 2015, as concurrent controls. Alarm rates for the 24 hours prior to huddle and the 24 hours following huddle were collected and analyzed. See Figure 1 for schematic of study design.

We collected data on patient characteristics, including patient location, age, sex, and intervention date. Information regarding changes to monitor alarm parameters for both intervention and control patients during the posthuddle period (the period following morning huddle until noon on intervention day) was also collected. We monitored for code blue events and unexpected changes in acuity until discharge or transfer out of the PICU.

Data Analysis

We compared the priority alarm activation rates of individual patients in the 24 hours before and the 24 hours after the huddle intervention and contrasted the differences in rates between intervention and control patients, both concurrent and historical controls. We also divided the intervention and control groups into 2 additional groups each—those patients whose alarm parameters were changed, compared with those whose parameters did not change. We evaluated for possible contamination by comparing alarm rates of historical and concurrent controls, as well as evaluating alarm rates by location. We used mixed-effects regression models to evaluate the effect of the intervention and control type (historical or concurrent) on alarm rates, adjusted for patient age and sex. Analysis was performed using Stata version 10.3 (StataCorp, LLC, College Station, TX) and SAS version 9.4 (SAS Institute Inc., Cary, NC).

Because patients could be enrolled more than once, we refer to the instances when they were included in the study as “events” (huddle discussions for intervention patients and huddle opportunities for controls) below. We identified 49 historical control events between April 1, 2015, and May 31, 2015. During the intervention period, we identified 88 intervention events and 163 concurrent control events between June 1, 2015, and October 31, 2015 (total n = 300; see Table 1 for event characteristics). A total of 6 patients were enrolled more than once as either intervention or control patients.

The average priority alarm activation rate for intervention patients was 433 alarms (95% confidence interval [CI], 392-472) per day in the 24 hours leading up to the intervention and 223 alarms (95% CI, 182-265) per day in the 24 hours following the intervention, a 48.5% unadjusted decrease (95% CI, 38.1%-58.9%). In contrast, priority alarm activation rates for concurrent control patients averaged 412 alarms (95% CI, 383-442) per day in the 24 hours leading up to the morning huddle and 323 alarms (95% CI, 270-375) per day in the 24 hours following huddle, a 21.6% unadjusted decrease (95% CI, 15.3%-27.9%). For historical controls, priority alarm activation rates averaged 369 alarms (95% CI, 339-399) per day in the 24 hours leading up to the morning huddle and 242 alarms (95% CI, 164-320) per day in the 24 hours following huddle, a 34.4% unadjusted decrease (95% CI, 13.5%-55.0%). When we compared historical versus concurrent controls in the unadjusted analysis, concurrent controls had 37 more alarms per day (95% CI, 59 fewer to 134 more; P = 0.45) than historical controls. There was no significant difference between concurrent and historical controls, demonstrating no evidence of contamination.

Adjusted Analysis of Changes in Alarm Rates

The overall estimate of the effect of the intervention adjusted for age and sex compared with concurrent controls was a reduction of 116 priority alarms per day (95% CI, 37-194; P = 0.004, Table 2). The adjusted percent decrease was 29.0% (95% CI, 12.1%-46.0%). There were no unexpected changes in patient acuity or code blue events related to the intervention.

Fidelity Analysis

We tracked changes in alarm parameter settings for evidence of intervention fidelity to determine if the team carried out the recommendations made. We found that 42% of intervention patients and 24% of combined control patients had alarm parameters changed during the posthuddle period (P = 0.002).

For those intervention patients who had parameters changed during the posthuddle period (N = 37), the mean effect was greater at a 54.9% decrease (95% CI, 38.8%-70.8%) in priority alarms as compared with control patients who had parameters adjusted during the posthuddle period (n = 50), having a mean decrease of only 12.2% (95% CI, –18.1%-42.3%). There was a 43.2% decrease (95% CI, 29.3%-57.0%) for intervention patients who were discussed but did not have parameters adjusted during the time window of observation (n = 51), as compared with combined control patients who did not have parameters adjusted (N = 162) who had a 28.1% decrease (95% CI, 16.8%-39.1%); see Figure 2.

This study is the first to demonstrate a successful and safe intervention to reduce the alarm rates of PICU patients. In addition, we observed a more significant reduction in priority alarm activation rates for intervention patients who had their alarm parameters changed during the monitored time period, leading us to hypothesize that providing patient-specific data regarding types of alarms was a key component of the intervention.

In control patients, we observed a reduction in alarm rates over time as well. There are 2 potential explanations for this. First, it is possible that as patients stabilize in the PICU, their vital signs become less extreme and generate fewer alarms even if the alarm parameters are not changed. The second is that parameters were changed within or outside of the time windows during which we evaluated for alarm parameter changes. Nevertheless, the decline over time observed in the intervention patients was greater than in both control groups. This change was even more noticeable in the intervention patients who had their alarm parameters changed during the posthuddle period as compared with controls who had their alarm parameters changed following the posthuddle period. This may have been due to the data provided during the huddle intervention, pointing the team to the cause of the high alarm rate.

Prior successful research regarding reduction of pediatric alarms has often shown decreased use of physiological monitors as 1 approach to reducing unnecessary alarms. The single prior pediatric alarm intervention study conducted on a pediatric ward involved instituting a cardiac monitor care process that included the ordering of age-based parameters, daily replacement of electrodes, individualized assessment of parameters, and a reliable method to discontinue monitoring.¹³ Because most patients in the PICU are critically ill, the reliance on monitor discontinuation as a main approach to decreasing alarms is not feasible in this setting. Instead, the use of targeted alarm parameter adjustments for low-acuity patients demonstrated a safe and feasible approach to decreasing alarms in PICU patients. The daily electrode change and age-based parameters were already in place at our institution.

There are a few limitations to this study. First, we focused only on low-acuity PICU patients. We believe that focusing on low-acuity patients allows for reduction in nonactionable alarms with limited potential for adverse events; however, this approach excludes many critically ill patients who might be at highest risk for harm from alarm fatigue if important alarms are ignored. Second, many of our patients were not present for the full 24 hours pre- and posthuddle due to their low acuity limiting our ability to follow alarm rates over time. Third, changes in alarm parameters were only monitored for a set period of 5 hours following the huddle to determine the effect of the recommended rounding script on changes to alarms. It is possible the changes to alarm parameters outside of the observed posthuddle period affected the alarm rates of both intervention and control patients. Lastly, the balancing metrics of unexpected changes in OptiLink status and code blue events are rare events, and therefore we may have been underpowered to find them. The effects of the huddle intervention on safety huddle length and rounding length were not measured.

Integrating a data-driven monitor alarm discussion into safety huddles was a safe and effective approach to reduce alarms in low-acuity, high-alarm PICU patients. Innovative approaches to make data-driven alarm decisions using informatics tools integrated into monitoring systems and electronic health records have the potential to facilitate cost-effective spread of this intervention.

Disclosure

This work was supported by a pilot grant from the Center for Pediatric Clinical Effectiveness, The Children’s Hospital of Philadelphia. Dr. Bonafide is supported by a Mentored Patient-Oriented Research Career Development Award from the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number K23HL116427. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding organizations or employers. The funding organizations had no role in the design, preparation, review, or approval of this paper, nor the decision to submit for publication.