Olufunmilayo Falade, MD

Many computerized provider order entry (CPOE) systems suffer from having too much of a good thing. Few would question the beneficial effect of CPOE on medication order clarity, completeness, and transmission.[1, 2] When mechanisms for basic decision support have been added, however, such as allergy, interaction, and duplicate warnings, reductions in medication errors and adverse events have not been consistently achieved.[3, 4, 5, 6, 7] This is likely due in part to the fact that ordering providers override medication warnings at staggeringly high rates.[8, 9] Clinicians acknowledge that they are ignoring potentially valuable warnings,[10, 11] but suffer from alert fatigue due to the sheer number of messages, many of them judged by clinicians to be of low‐value.[11, 12]

Redesign of medication alert systems to increase their signal‐to‐noise ratio is badly needed,[13, 14, 15, 16] and will need to consider the clinical significance of alerts, their presentation, and context‐specific factors that potentially contribute to warning effectiveness.[17, 18, 19] Relatively few studies, however, have objectively looked at context factors such as the characteristics of providers, patients, medications, and warnings that are associated with provider responses to warnings,[9, 20, 21, 22, 23, 24, 25] and only 2 have studied how warning acceptance is associated with medication risk.[18, 26] We wished to explore these factors further. Warning acceptance has been shown to be higher, at least in the outpatient setting, when orders are entered by low‐volume prescribers for infrequently encountered warnings,[24] and there is some evidence that patients receive higher‐quality care during the day.[27] Significant attention has been placed in recent years on inappropriate prescribing in older patients,[28] and on creating a culture of safety in healthcare.[29] We therefore hypothesized that our providers would be more cautious, and medication warning acceptance rates would be higher, when orders were entered for patients who were older or with more complex medical problems, when they were entered during the day by caregivers who entered few orders, when the medications ordered were potentially associated with greater risk, and when the warnings themselves were infrequently encountered.

METHODS

RESULTS

A total of 259,656 medication orders were placed for adult non‐ICU patients during the 7‐month study period. Of those orders, 45,835 generated some number of medication warnings.[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20] The median number of warnings per patient was 4 (interquartile range [IQR]=28; mean=5.9, standard deviation [SD]=6.2), with a range from 1 to 84. The median number of warnings generated per provider during the study period was 36 (IQR=6106, mean=87.4, SD=133.7), with a range of 1 to 1096.

There were 40,391 orders placed for 454 medications for adult non‐ICU patients, which generated a single‐medication warning (excluding those with the response replace, which was used 20 times) during the 7‐month study period. Data regarding the patients and providers associated with the orders generating single warnings are shown in Table 1. Most patients were on medicine units, and most orders were entered by residents. Patients' LOS at the time the orders were placed ranged from 0 to 118 days (median=1, IQR=04; mean=4.0, SD=7.2). The median number of single warnings per patient was 4 (IQR=28; mean=6.1, SD=6.5), with a range from 1 to 84. The median number of single warnings generated per provider during the study period was 15 (IQR=373; mean=61.7, SD=109.6), with a range of 1 to 1057.

Patient and caregiver characteristics for the medication orders that generated single warnings are shown in Table 2. The majority of medications were nonparenteral and not on the ISMP list (Table 3). Most warnings generated were either duplicate (47%) or interaction warnings (47%). Warnings of a particular type were repeated 14.5% of the time for a particular medication and patient (from 2 to 24 times, median=2, IQR=22, mean=2.7, SD=1.4), and 9.8% of the time for a particular caregiver, medication, and patient (from 2 to 18 times, median=2, IQR=22, mean=2.4, SD=1.1).

One thousand five hundred fifty‐four warnings were erased (ie, accepted by clinicians [4%]). In univariate analysis, only patient gender was not associated with warning acceptance. Patient age, LOS, hospital unit at the time of order entry, ordering caregiver type, day and time the medication was ordered, administration route, presence on the ISMP list, warning frequency, and warning type were all significantly associated with warning acceptance (Table 2).

Older patient age, longer LOS, presence of the medication on the ISMP list, and interaction warning type were all negatively associated with warning acceptance in multivariable analysis. Warning acceptance was positively associated with male patient gender, being on a service other than medicine, being a caregiver other than a resident, parenteral medications, lower warning frequency, and allergy or adverse reaction warning types (Table 3).

The 20 medications that generated the most single warnings are shown in Table 4. Medications on the ISMP list accounted for 8 of these top 20 medications. For most of them, duplicate and interaction warnings accounted for most of the warnings generated, except for parenteral hydromorphone, oral oxycodone, parenteral morphine, and oral hydromorphone, which each had more allergy than interaction warnings.

DISCUSSION

Medication warnings in our study were frequently overridden, particularly when encountered by residents, for patients with a long LOS and on the internal medicine service, and for medications generating the most warnings and on the ISMP list. Disturbingly, this means that potentially important warnings for medications with the highest potential for causing harm, for possibly the sickest and most complex patients, were those that were most often ignored by young physicians in training who should have had the most to gain from them. Of course, this is not entirely surprising. Despite our hope that a culture of safety would influence young physicians' actions when caring for these patients and prescribing these medications, these patients and medications are those for whom the most warnings are generated, and these physicians are the ones entering the most orders. Only 13% of the medications studied were on the ISMP list, but they generated 32% of the warnings. We controlled for number of warnings and ISMP list status, but not for warning validity. Most likely, high‐risk medications have been set up with more warnings, many of them of lower quality, in an errant but well‐intentioned effort to make them safer. If developers of CPOE systems want to gain serious traction in using decision support to promote prescribing safe medications, they must take substantial action to increase attention to important warnings and decrease the number of clinically insignificant, low‐value warnings encountered by active caregivers on a daily basis.

Only 2 prior studies, both by Seidling et al., have specifically looked at provider response to warnings for high risk medications. Interaction warnings were rarely accepted in 1,[18] as in our study; however, in contrast to our findings, warning acceptance in both studies was higher for drugs with dose‐dependent toxicity.[18, 26] The effect of physician experience on warning acceptance has been addressed in 2 prior studies. In Weingart et al., residents were more likely than staff physicians to erase medication orders when presented with allergy and interaction warnings in a primary care setting.[20] Long et al. found that physicians younger than 40 years were less likely than older physicians to accept duplicate warnings, but those who had been at the study hospital for a longer period of time were more likely to accept them.[23] The influence of patient LOS and service on warning acceptance has not previously been described. Further study is needed looking at each of these factors.

Individual hospitals tend to avoid making modifications to order entry warning systems, because monitoring and maintaining these changes is labor intensive. Some institutions may make the decision to turn off certain categories of alerts, such as intermediate interaction warnings, to minimize the noise their providers encounter. There are even tools for disabling individual alerts or groups of alerts, such as that available for purchase from our interaction database vendor.[31] However, institutions may fear litigation should an adverse event be attributed to a disabled warning.[15, 16] Clearly, a comprehensive, health system‐wide approach is warranted.[13, 15] To date, published efforts describing ways to improve the effectiveness of medication warning systems have focused on either heightening the clinical significance of alerts[14, 21, 22, 32, 33, 34, 35, 36] or altering their presentation and how providers experience them.[21, 36, 37, 38, 39, 40, 41, 42, 43] The single medication warnings our providers receive are all presented in an identical font, and presumably response to each would be different if they were better distinguished from each other. We also found that a small but significant number of warnings were repeated for a given patient and even a given provider. If the providers knew they would only be presented with warnings the first time they occurred for a given patient and medication, they might be more attuned to the remaining warnings. Previous studies describe context‐specific decision support for medication ordering[44, 45, 46]; however, only 1 has described the use of patient context factors to modify when or how warnings are presented to providers.[47] None have described tailoring allergy, duplicate, and interaction warnings according to medication or provider types. If further study confirms our findings, modulating basic warning systems according to severity of illness, provider experience, and medication risk could powerfully increase their effectiveness. Of course, this would be extremely challenging to achieve, and is likely outside the capabilities of most, if not all, CPOE systems, at least for now.

Our study has some limitations. First, it was limited to medications that generated a single warning. We did this for ease of analysis and so that we could ensure understanding of provider response to each warning type without bias from simultaneously occurring warnings; however, caregiver response to multiple warnings appearing simultaneously for a particular medication order might be quite different. Second, we did not include any assessment of the number of medications ordered by each provider type or for each patient, either of which could significantly affect provider response to warnings. Third, as previously noted, we did not include any assessment of the validity of the warnings, beyond the 4 main categories described, which could also significantly affect provider response. However, it should be noted that although the validity of interaction warnings varies significantly from 1 medication to another, the validity of duplicate, allergy, and adverse reaction warnings in the described system are essentially the same for all medications. Fourth, it is possible that providers did modify or even erase their orders even after selecting override in response to the warning; it is also possible that providers reentered the same order after choosing erase. Unfortunately auditing for actions such as these would be extremely laborious. Finally, the study was conducted at a single medical center using a single order‐entry system. The system in use at our medical center is in use at one‐third of the 6000 hospitals in the United States, though certainly not all are using our version. Even if a hospital was using the same CPOE version and interaction database as our institution, variations in patient population and local decisions modifying how the database interacts with the warning presentation system might affect reproducibility at that institution.

Commonly encountered medication warnings are overridden at extremely high rates, and in our study this was particularly so for medications on the ISMP list, when ordered by physicians in training. Warnings of little clinical significance must be identified and eliminated, the most important warnings need to be visually distinct to increase user attention, and further research should be done into the patient, provider, setting, and medication factors that affect user responses to warnings, so that they may be customized accordingly and their significance increased. Doing so will enable us to reap the maximum possible potential from our CPOE systems, and increase the CPOE's power to protect our most vulnerable patients from our most dangerous medications, particularly when cared for by our most inexperienced physicians.

Many computerized provider order entry (CPOE) systems suffer from having too much of a good thing. Few would question the beneficial effect of CPOE on medication order clarity, completeness, and transmission.[1, 2] When mechanisms for basic decision support have been added, however, such as allergy, interaction, and duplicate warnings, reductions in medication errors and adverse events have not been consistently achieved.[3, 4, 5, 6, 7] This is likely due in part to the fact that ordering providers override medication warnings at staggeringly high rates.[8, 9] Clinicians acknowledge that they are ignoring potentially valuable warnings,[10, 11] but suffer from alert fatigue due to the sheer number of messages, many of them judged by clinicians to be of low‐value.[11, 12]

Redesign of medication alert systems to increase their signal‐to‐noise ratio is badly needed,[13, 14, 15, 16] and will need to consider the clinical significance of alerts, their presentation, and context‐specific factors that potentially contribute to warning effectiveness.[17, 18, 19] Relatively few studies, however, have objectively looked at context factors such as the characteristics of providers, patients, medications, and warnings that are associated with provider responses to warnings,[9, 20, 21, 22, 23, 24, 25] and only 2 have studied how warning acceptance is associated with medication risk.[18, 26] We wished to explore these factors further. Warning acceptance has been shown to be higher, at least in the outpatient setting, when orders are entered by low‐volume prescribers for infrequently encountered warnings,[24] and there is some evidence that patients receive higher‐quality care during the day.[27] Significant attention has been placed in recent years on inappropriate prescribing in older patients,[28] and on creating a culture of safety in healthcare.[29] We therefore hypothesized that our providers would be more cautious, and medication warning acceptance rates would be higher, when orders were entered for patients who were older or with more complex medical problems, when they were entered during the day by caregivers who entered few orders, when the medications ordered were potentially associated with greater risk, and when the warnings themselves were infrequently encountered.

METHODS

RESULTS

A total of 259,656 medication orders were placed for adult non‐ICU patients during the 7‐month study period. Of those orders, 45,835 generated some number of medication warnings.[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20] The median number of warnings per patient was 4 (interquartile range [IQR]=28; mean=5.9, standard deviation [SD]=6.2), with a range from 1 to 84. The median number of warnings generated per provider during the study period was 36 (IQR=6106, mean=87.4, SD=133.7), with a range of 1 to 1096.

There were 40,391 orders placed for 454 medications for adult non‐ICU patients, which generated a single‐medication warning (excluding those with the response replace, which was used 20 times) during the 7‐month study period. Data regarding the patients and providers associated with the orders generating single warnings are shown in Table 1. Most patients were on medicine units, and most orders were entered by residents. Patients' LOS at the time the orders were placed ranged from 0 to 118 days (median=1, IQR=04; mean=4.0, SD=7.2). The median number of single warnings per patient was 4 (IQR=28; mean=6.1, SD=6.5), with a range from 1 to 84. The median number of single warnings generated per provider during the study period was 15 (IQR=373; mean=61.7, SD=109.6), with a range of 1 to 1057.

Patient and caregiver characteristics for the medication orders that generated single warnings are shown in Table 2. The majority of medications were nonparenteral and not on the ISMP list (Table 3). Most warnings generated were either duplicate (47%) or interaction warnings (47%). Warnings of a particular type were repeated 14.5% of the time for a particular medication and patient (from 2 to 24 times, median=2, IQR=22, mean=2.7, SD=1.4), and 9.8% of the time for a particular caregiver, medication, and patient (from 2 to 18 times, median=2, IQR=22, mean=2.4, SD=1.1).

One thousand five hundred fifty‐four warnings were erased (ie, accepted by clinicians [4%]). In univariate analysis, only patient gender was not associated with warning acceptance. Patient age, LOS, hospital unit at the time of order entry, ordering caregiver type, day and time the medication was ordered, administration route, presence on the ISMP list, warning frequency, and warning type were all significantly associated with warning acceptance (Table 2).

Older patient age, longer LOS, presence of the medication on the ISMP list, and interaction warning type were all negatively associated with warning acceptance in multivariable analysis. Warning acceptance was positively associated with male patient gender, being on a service other than medicine, being a caregiver other than a resident, parenteral medications, lower warning frequency, and allergy or adverse reaction warning types (Table 3).

The 20 medications that generated the most single warnings are shown in Table 4. Medications on the ISMP list accounted for 8 of these top 20 medications. For most of them, duplicate and interaction warnings accounted for most of the warnings generated, except for parenteral hydromorphone, oral oxycodone, parenteral morphine, and oral hydromorphone, which each had more allergy than interaction warnings.

DISCUSSION

Medication warnings in our study were frequently overridden, particularly when encountered by residents, for patients with a long LOS and on the internal medicine service, and for medications generating the most warnings and on the ISMP list. Disturbingly, this means that potentially important warnings for medications with the highest potential for causing harm, for possibly the sickest and most complex patients, were those that were most often ignored by young physicians in training who should have had the most to gain from them. Of course, this is not entirely surprising. Despite our hope that a culture of safety would influence young physicians' actions when caring for these patients and prescribing these medications, these patients and medications are those for whom the most warnings are generated, and these physicians are the ones entering the most orders. Only 13% of the medications studied were on the ISMP list, but they generated 32% of the warnings. We controlled for number of warnings and ISMP list status, but not for warning validity. Most likely, high‐risk medications have been set up with more warnings, many of them of lower quality, in an errant but well‐intentioned effort to make them safer. If developers of CPOE systems want to gain serious traction in using decision support to promote prescribing safe medications, they must take substantial action to increase attention to important warnings and decrease the number of clinically insignificant, low‐value warnings encountered by active caregivers on a daily basis.

Only 2 prior studies, both by Seidling et al., have specifically looked at provider response to warnings for high risk medications. Interaction warnings were rarely accepted in 1,[18] as in our study; however, in contrast to our findings, warning acceptance in both studies was higher for drugs with dose‐dependent toxicity.[18, 26] The effect of physician experience on warning acceptance has been addressed in 2 prior studies. In Weingart et al., residents were more likely than staff physicians to erase medication orders when presented with allergy and interaction warnings in a primary care setting.[20] Long et al. found that physicians younger than 40 years were less likely than older physicians to accept duplicate warnings, but those who had been at the study hospital for a longer period of time were more likely to accept them.[23] The influence of patient LOS and service on warning acceptance has not previously been described. Further study is needed looking at each of these factors.

Individual hospitals tend to avoid making modifications to order entry warning systems, because monitoring and maintaining these changes is labor intensive. Some institutions may make the decision to turn off certain categories of alerts, such as intermediate interaction warnings, to minimize the noise their providers encounter. There are even tools for disabling individual alerts or groups of alerts, such as that available for purchase from our interaction database vendor.[31] However, institutions may fear litigation should an adverse event be attributed to a disabled warning.[15, 16] Clearly, a comprehensive, health system‐wide approach is warranted.[13, 15] To date, published efforts describing ways to improve the effectiveness of medication warning systems have focused on either heightening the clinical significance of alerts[14, 21, 22, 32, 33, 34, 35, 36] or altering their presentation and how providers experience them.[21, 36, 37, 38, 39, 40, 41, 42, 43] The single medication warnings our providers receive are all presented in an identical font, and presumably response to each would be different if they were better distinguished from each other. We also found that a small but significant number of warnings were repeated for a given patient and even a given provider. If the providers knew they would only be presented with warnings the first time they occurred for a given patient and medication, they might be more attuned to the remaining warnings. Previous studies describe context‐specific decision support for medication ordering[44, 45, 46]; however, only 1 has described the use of patient context factors to modify when or how warnings are presented to providers.[47] None have described tailoring allergy, duplicate, and interaction warnings according to medication or provider types. If further study confirms our findings, modulating basic warning systems according to severity of illness, provider experience, and medication risk could powerfully increase their effectiveness. Of course, this would be extremely challenging to achieve, and is likely outside the capabilities of most, if not all, CPOE systems, at least for now.

Our study has some limitations. First, it was limited to medications that generated a single warning. We did this for ease of analysis and so that we could ensure understanding of provider response to each warning type without bias from simultaneously occurring warnings; however, caregiver response to multiple warnings appearing simultaneously for a particular medication order might be quite different. Second, we did not include any assessment of the number of medications ordered by each provider type or for each patient, either of which could significantly affect provider response to warnings. Third, as previously noted, we did not include any assessment of the validity of the warnings, beyond the 4 main categories described, which could also significantly affect provider response. However, it should be noted that although the validity of interaction warnings varies significantly from 1 medication to another, the validity of duplicate, allergy, and adverse reaction warnings in the described system are essentially the same for all medications. Fourth, it is possible that providers did modify or even erase their orders even after selecting override in response to the warning; it is also possible that providers reentered the same order after choosing erase. Unfortunately auditing for actions such as these would be extremely laborious. Finally, the study was conducted at a single medical center using a single order‐entry system. The system in use at our medical center is in use at one‐third of the 6000 hospitals in the United States, though certainly not all are using our version. Even if a hospital was using the same CPOE version and interaction database as our institution, variations in patient population and local decisions modifying how the database interacts with the warning presentation system might affect reproducibility at that institution.

Commonly encountered medication warnings are overridden at extremely high rates, and in our study this was particularly so for medications on the ISMP list, when ordered by physicians in training. Warnings of little clinical significance must be identified and eliminated, the most important warnings need to be visually distinct to increase user attention, and further research should be done into the patient, provider, setting, and medication factors that affect user responses to warnings, so that they may be customized accordingly and their significance increased. Doing so will enable us to reap the maximum possible potential from our CPOE systems, and increase the CPOE's power to protect our most vulnerable patients from our most dangerous medications, particularly when cared for by our most inexperienced physicians.

Many computerized provider order entry (CPOE) systems suffer from having too much of a good thing. Few would question the beneficial effect of CPOE on medication order clarity, completeness, and transmission.[1, 2] When mechanisms for basic decision support have been added, however, such as allergy, interaction, and duplicate warnings, reductions in medication errors and adverse events have not been consistently achieved.[3, 4, 5, 6, 7] This is likely due in part to the fact that ordering providers override medication warnings at staggeringly high rates.[8, 9] Clinicians acknowledge that they are ignoring potentially valuable warnings,[10, 11] but suffer from alert fatigue due to the sheer number of messages, many of them judged by clinicians to be of low‐value.[11, 12]

Redesign of medication alert systems to increase their signal‐to‐noise ratio is badly needed,[13, 14, 15, 16] and will need to consider the clinical significance of alerts, their presentation, and context‐specific factors that potentially contribute to warning effectiveness.[17, 18, 19] Relatively few studies, however, have objectively looked at context factors such as the characteristics of providers, patients, medications, and warnings that are associated with provider responses to warnings,[9, 20, 21, 22, 23, 24, 25] and only 2 have studied how warning acceptance is associated with medication risk.[18, 26] We wished to explore these factors further. Warning acceptance has been shown to be higher, at least in the outpatient setting, when orders are entered by low‐volume prescribers for infrequently encountered warnings,[24] and there is some evidence that patients receive higher‐quality care during the day.[27] Significant attention has been placed in recent years on inappropriate prescribing in older patients,[28] and on creating a culture of safety in healthcare.[29] We therefore hypothesized that our providers would be more cautious, and medication warning acceptance rates would be higher, when orders were entered for patients who were older or with more complex medical problems, when they were entered during the day by caregivers who entered few orders, when the medications ordered were potentially associated with greater risk, and when the warnings themselves were infrequently encountered.

METHODS

RESULTS

A total of 259,656 medication orders were placed for adult non‐ICU patients during the 7‐month study period. Of those orders, 45,835 generated some number of medication warnings.[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20] The median number of warnings per patient was 4 (interquartile range [IQR]=28; mean=5.9, standard deviation [SD]=6.2), with a range from 1 to 84. The median number of warnings generated per provider during the study period was 36 (IQR=6106, mean=87.4, SD=133.7), with a range of 1 to 1096.

There were 40,391 orders placed for 454 medications for adult non‐ICU patients, which generated a single‐medication warning (excluding those with the response replace, which was used 20 times) during the 7‐month study period. Data regarding the patients and providers associated with the orders generating single warnings are shown in Table 1. Most patients were on medicine units, and most orders were entered by residents. Patients' LOS at the time the orders were placed ranged from 0 to 118 days (median=1, IQR=04; mean=4.0, SD=7.2). The median number of single warnings per patient was 4 (IQR=28; mean=6.1, SD=6.5), with a range from 1 to 84. The median number of single warnings generated per provider during the study period was 15 (IQR=373; mean=61.7, SD=109.6), with a range of 1 to 1057.

Patient and caregiver characteristics for the medication orders that generated single warnings are shown in Table 2. The majority of medications were nonparenteral and not on the ISMP list (Table 3). Most warnings generated were either duplicate (47%) or interaction warnings (47%). Warnings of a particular type were repeated 14.5% of the time for a particular medication and patient (from 2 to 24 times, median=2, IQR=22, mean=2.7, SD=1.4), and 9.8% of the time for a particular caregiver, medication, and patient (from 2 to 18 times, median=2, IQR=22, mean=2.4, SD=1.1).

One thousand five hundred fifty‐four warnings were erased (ie, accepted by clinicians [4%]). In univariate analysis, only patient gender was not associated with warning acceptance. Patient age, LOS, hospital unit at the time of order entry, ordering caregiver type, day and time the medication was ordered, administration route, presence on the ISMP list, warning frequency, and warning type were all significantly associated with warning acceptance (Table 2).

Older patient age, longer LOS, presence of the medication on the ISMP list, and interaction warning type were all negatively associated with warning acceptance in multivariable analysis. Warning acceptance was positively associated with male patient gender, being on a service other than medicine, being a caregiver other than a resident, parenteral medications, lower warning frequency, and allergy or adverse reaction warning types (Table 3).

The 20 medications that generated the most single warnings are shown in Table 4. Medications on the ISMP list accounted for 8 of these top 20 medications. For most of them, duplicate and interaction warnings accounted for most of the warnings generated, except for parenteral hydromorphone, oral oxycodone, parenteral morphine, and oral hydromorphone, which each had more allergy than interaction warnings.

DISCUSSION

Medication warnings in our study were frequently overridden, particularly when encountered by residents, for patients with a long LOS and on the internal medicine service, and for medications generating the most warnings and on the ISMP list. Disturbingly, this means that potentially important warnings for medications with the highest potential for causing harm, for possibly the sickest and most complex patients, were those that were most often ignored by young physicians in training who should have had the most to gain from them. Of course, this is not entirely surprising. Despite our hope that a culture of safety would influence young physicians' actions when caring for these patients and prescribing these medications, these patients and medications are those for whom the most warnings are generated, and these physicians are the ones entering the most orders. Only 13% of the medications studied were on the ISMP list, but they generated 32% of the warnings. We controlled for number of warnings and ISMP list status, but not for warning validity. Most likely, high‐risk medications have been set up with more warnings, many of them of lower quality, in an errant but well‐intentioned effort to make them safer. If developers of CPOE systems want to gain serious traction in using decision support to promote prescribing safe medications, they must take substantial action to increase attention to important warnings and decrease the number of clinically insignificant, low‐value warnings encountered by active caregivers on a daily basis.

Only 2 prior studies, both by Seidling et al., have specifically looked at provider response to warnings for high risk medications. Interaction warnings were rarely accepted in 1,[18] as in our study; however, in contrast to our findings, warning acceptance in both studies was higher for drugs with dose‐dependent toxicity.[18, 26] The effect of physician experience on warning acceptance has been addressed in 2 prior studies. In Weingart et al., residents were more likely than staff physicians to erase medication orders when presented with allergy and interaction warnings in a primary care setting.[20] Long et al. found that physicians younger than 40 years were less likely than older physicians to accept duplicate warnings, but those who had been at the study hospital for a longer period of time were more likely to accept them.[23] The influence of patient LOS and service on warning acceptance has not previously been described. Further study is needed looking at each of these factors.

Individual hospitals tend to avoid making modifications to order entry warning systems, because monitoring and maintaining these changes is labor intensive. Some institutions may make the decision to turn off certain categories of alerts, such as intermediate interaction warnings, to minimize the noise their providers encounter. There are even tools for disabling individual alerts or groups of alerts, such as that available for purchase from our interaction database vendor.[31] However, institutions may fear litigation should an adverse event be attributed to a disabled warning.[15, 16] Clearly, a comprehensive, health system‐wide approach is warranted.[13, 15] To date, published efforts describing ways to improve the effectiveness of medication warning systems have focused on either heightening the clinical significance of alerts[14, 21, 22, 32, 33, 34, 35, 36] or altering their presentation and how providers experience them.[21, 36, 37, 38, 39, 40, 41, 42, 43] The single medication warnings our providers receive are all presented in an identical font, and presumably response to each would be different if they were better distinguished from each other. We also found that a small but significant number of warnings were repeated for a given patient and even a given provider. If the providers knew they would only be presented with warnings the first time they occurred for a given patient and medication, they might be more attuned to the remaining warnings. Previous studies describe context‐specific decision support for medication ordering[44, 45, 46]; however, only 1 has described the use of patient context factors to modify when or how warnings are presented to providers.[47] None have described tailoring allergy, duplicate, and interaction warnings according to medication or provider types. If further study confirms our findings, modulating basic warning systems according to severity of illness, provider experience, and medication risk could powerfully increase their effectiveness. Of course, this would be extremely challenging to achieve, and is likely outside the capabilities of most, if not all, CPOE systems, at least for now.

Our study has some limitations. First, it was limited to medications that generated a single warning. We did this for ease of analysis and so that we could ensure understanding of provider response to each warning type without bias from simultaneously occurring warnings; however, caregiver response to multiple warnings appearing simultaneously for a particular medication order might be quite different. Second, we did not include any assessment of the number of medications ordered by each provider type or for each patient, either of which could significantly affect provider response to warnings. Third, as previously noted, we did not include any assessment of the validity of the warnings, beyond the 4 main categories described, which could also significantly affect provider response. However, it should be noted that although the validity of interaction warnings varies significantly from 1 medication to another, the validity of duplicate, allergy, and adverse reaction warnings in the described system are essentially the same for all medications. Fourth, it is possible that providers did modify or even erase their orders even after selecting override in response to the warning; it is also possible that providers reentered the same order after choosing erase. Unfortunately auditing for actions such as these would be extremely laborious. Finally, the study was conducted at a single medical center using a single order‐entry system. The system in use at our medical center is in use at one‐third of the 6000 hospitals in the United States, though certainly not all are using our version. Even if a hospital was using the same CPOE version and interaction database as our institution, variations in patient population and local decisions modifying how the database interacts with the warning presentation system might affect reproducibility at that institution.

Commonly encountered medication warnings are overridden at extremely high rates, and in our study this was particularly so for medications on the ISMP list, when ordered by physicians in training. Warnings of little clinical significance must be identified and eliminated, the most important warnings need to be visually distinct to increase user attention, and further research should be done into the patient, provider, setting, and medication factors that affect user responses to warnings, so that they may be customized accordingly and their significance increased. Doing so will enable us to reap the maximum possible potential from our CPOE systems, and increase the CPOE's power to protect our most vulnerable patients from our most dangerous medications, particularly when cared for by our most inexperienced physicians.

Sepsis is a major cause of death in hospitalized patients.13 It is recommended that patients with sepsis be treated with early appropriate antibiotics, as well as early goal‐directed therapy including fluid and vasopressor support according to evidence‐based guidelines.46 Following such evidence‐based protocols and process‐of‐care interventions has been shown to be associated with better patient outcomes, including decreased mortality.7, 8

Most patients with severe sepsis are cared for in intensive care units (ICUs). At times, there are no beds available in the primary ICU and patients presenting to the hospital with sepsis are cared for in other units. Patients admitted to a non‐preferred clinical inpatient setting are sometimes referred to as overflow.9 ICUs can differ significantly in staffing patterns, equipment, and training.10 It is not known if overflow sepsis patients receive similar care when admitted to non‐primary ICUs.

At our hospital, we have an active bed management system led by the hospitalist division.11 This system includes protocols to place sepsis patients in the overflow ICU if the primary ICU is full. We hypothesized that process‐of‐care interventions would be more strictly adhered to when sepsis patients were in the primary ICU rather than in the overflow unit at our institution.

METHODS

RESULTS

There were 100 patients admitted to the MICU and 41 patients admitted to the CICU during the study period (Table 1). The majority of the patients were admitted to the ICUs directly from the emergency department (ED) (n = 129), with a small number of patients who were transferred from the Medicine floors (n = 12).

There were no significant differences between the 2 study groups in terms of age, sex, primary site of infection, mean APACHE II score, SOFA scores on day 1, chronic organ insufficiency, immune suppression, or need for mechanical ventilation (Table 1). The most common site of infection was lung. There were significantly more patients with severe sepsis in the MICU (88% vs 51%, P <0.001).

DISCUSSION

Since sepsis is more commonly treated in the medical ICU and some data suggests that specialty ICUs may be better at providing desired care,18, 19 we believed that patients treated in the MICU would be more likely to receive guideline‐concordant care. The study refutes our a priori hypothesis and reveals that evidence‐based processes‐of‐care associated with improved outcomes for sepsis are similarly implemented at our institution in the primary and overflow ICU. These findings are important, as ICU bed availability is a frequent problem and many hospitals overflow patients to non‐primary ICUs.9, 20

The observed equivalence in the care delivered may be a function of the relatively high number of patients with sepsis treated in the overflow unit, thereby giving the delivery teams enough experience to provide the desired care. An alternative explanation could be that the residents in CICU brought with them the experience from having previously trained in the MICU. Although, some of the care processes for sepsis patients are influenced by the CPOE (with embedded order sets and protocols), it is unlikely that CPOE can fully account for similarity in care because many processes and therapies (like use of steroids, amount of fluid delivered in first 24 hours, packed red blood cells [PRBC] transfusion, and spontaneous breathing trials) are not embedded within order sets.

The significant difference noted in the areas of deep vein thrombosis (DVT) and gastrointestinal (GI) prophylaxis within 24 hours of ICU admission was unexpected. These preventive therapies are included in initial order sets in the CPOE, which prompt physicians to order them as standard‐of‐care. With respect to DVT prophylaxis, we suspect that some of the difference might be attributable to specific contraindications to its use, which could have been more common in one of the units. There were more patients in MICU on mechanical ventilation (although not statistically significant) and with severe sepsis (statistically significant) at time of admission, which might have contributed to the difference noted in use of GI prophylaxis. It is also plausible that these differences might have disappeared if they were reassessed beyond 24 hours into the ICU admission. We cannot rule out the presence of unit‐ and physician‐level differences that contributed to this. Likewise, there was an unexpected trend towards significance, wherein more patients in CICU had spontaneous breathing trials within 24 hours of admission. This might also be explained by the higher number of patients with severe sepsis in the MICU (preempting any weaning attempts). These caveats aside, it is reassuring that, at our institution, admitting septic patients to the first available ICU bed does not adversely affect important processes‐of‐care.

One might ask whether this study's data should reassure other sites who are boarding septic patients in non‐primary ICUs. Irrespective of the number of patients studied or the degree of statistical significance of the associations, an observational study design cannot prove that boarding septic patients in non‐primary ICUs is either safe or unsafe. However, we hope that readers reflect on, and take inventory of, systems issues that may be different between unitswith an eye towards eliminating variation such that all units managing septic patients are primed to deliver guideline‐concordant care. Other hospitals that use CPOE with sepsis order sets, have protocols for sepsis care, and who train nursing and respiratory therapists to meet high standards might be pleased to see that the patients in our study received comparable, high‐quality care across the 2 units. While our data suggests that boarding patients in overflow units may be safe, these findings would need to be replicated at other sites using prospective designs to prove safety.

Length of emergency room stay prior to admission is associated with higher mortality rates.2123 At many hospitals, critical care beds are a scarce resource such that most hospitals have a policy for the triage of patients to critical care beds.24, 25 Lundberg and colleagues' study demonstrated that patients who developed septic shock on the medical wards experienced delays in receipt of intravenous fluids, inotropic agents and transfer to a critical care setting.26 Thus, rather than waiting in the ED or on the medical service for an MICU bed to become available, it may be most wise to admit a critically sick septic patient to the first available ICU bed, even to an overflow ICU. In a recent study by Sidlow and Aggarwal, 1104 patients discharged from the coronary care unit (CCU) with a non‐cardiac primary diagnosis were compared to patients admitted to the MICU in the same hospital.27 The study found no differences in patient mortality, 30‐day readmission rate, hospital LOS, ICU LOS, and safety outcomes of ventilator‐associated pneumonia and catheter‐associated bloodstream infections between ICUs. However, their study did not examine processes‐of‐care delivered between the primary ICU and the overflow unit, and did not validate the primary diagnoses of patients admitted to the ICU.

Several limitations of this study should be considered. First, this study was conducted at a single center. Second, we used a retrospective study design; however, a prospective study randomizing patients to 1 of the 2 units would likely never be possible. Third, the relatively small number of patients limited the power of the study to detect mortality differences between the units. However, this was a pilot study focused on processes of care as opposed to clinical outcomes. Fourth, it is possible that we did not capture every single patient with sepsis with our keyword search. Our use of a previously validated screening process should have limited the number of missed cases.15, 16 Fifth, although the 2 ICUs have exclusive nursing staff and attending physicians, the housestaff and respiratory therapists do rotate between the 2 ICUs and place orders in the common CPOE. The rotating housestaff may certainly represent a source for confounding, but the large numbers (>30) of evenly spread housestaff over the study period minimizes the potential for any trainee to be responsible for a large proportion of observed practice. Sixth, ICU attendings are the physicians of record and could influence the results. Because no attending physician was on service for more than 4 weeks during the study period, and patients were equally spread over this same time, concerns about clustering and biases this may have created should be minimal but cannot be ruled out. Seventh, some interventions and processes, such as antibiotic administration and measurement of lactate, may have been initiated in the ED, thereby decreasing the potential for differences between the groups. Additionally, we cannot rule out the possibility that factors other than bed availability drove the admission process (we found that the relative proportion of patients admitted to overflow ICU during hours of ambulance diversion was similar to the overflow ICU admissions during non‐ambulance diversion hours). It is possible that some selection bias by the hospitalist assigning patients to specific ICUs influenced their triage decisionsalthough all triaging doctors go through the same process of training in active bed management.11 While more patients admitted to the MICU had severe sepsis, there were no differences between groups in APACHE II or SOFA scores. However, we cannot rule out that there were other residual confounders. Finally, in a small number of cases (4/41, 10%), the CICU team consulted the MICU attending for assistance. This input had the potential to reduce disparities in care between the units.

Overflowing patients to non‐primary ICUs occurs in many hospitals. Our study demonstrates that sepsis treatment for overflow patients may be similar to that received in the primary ICU. While a large multicentered and randomized trial could determine whether significant management and outcome differences exist between primary and overflow ICUs, feasibility concerns make it unlikely that such a study will ever be conducted.

Sepsis is a major cause of death in hospitalized patients.13 It is recommended that patients with sepsis be treated with early appropriate antibiotics, as well as early goal‐directed therapy including fluid and vasopressor support according to evidence‐based guidelines.46 Following such evidence‐based protocols and process‐of‐care interventions has been shown to be associated with better patient outcomes, including decreased mortality.7, 8

Most patients with severe sepsis are cared for in intensive care units (ICUs). At times, there are no beds available in the primary ICU and patients presenting to the hospital with sepsis are cared for in other units. Patients admitted to a non‐preferred clinical inpatient setting are sometimes referred to as overflow.9 ICUs can differ significantly in staffing patterns, equipment, and training.10 It is not known if overflow sepsis patients receive similar care when admitted to non‐primary ICUs.

At our hospital, we have an active bed management system led by the hospitalist division.11 This system includes protocols to place sepsis patients in the overflow ICU if the primary ICU is full. We hypothesized that process‐of‐care interventions would be more strictly adhered to when sepsis patients were in the primary ICU rather than in the overflow unit at our institution.

METHODS

RESULTS

There were 100 patients admitted to the MICU and 41 patients admitted to the CICU during the study period (Table 1). The majority of the patients were admitted to the ICUs directly from the emergency department (ED) (n = 129), with a small number of patients who were transferred from the Medicine floors (n = 12).

There were no significant differences between the 2 study groups in terms of age, sex, primary site of infection, mean APACHE II score, SOFA scores on day 1, chronic organ insufficiency, immune suppression, or need for mechanical ventilation (Table 1). The most common site of infection was lung. There were significantly more patients with severe sepsis in the MICU (88% vs 51%, P <0.001).

DISCUSSION

Since sepsis is more commonly treated in the medical ICU and some data suggests that specialty ICUs may be better at providing desired care,18, 19 we believed that patients treated in the MICU would be more likely to receive guideline‐concordant care. The study refutes our a priori hypothesis and reveals that evidence‐based processes‐of‐care associated with improved outcomes for sepsis are similarly implemented at our institution in the primary and overflow ICU. These findings are important, as ICU bed availability is a frequent problem and many hospitals overflow patients to non‐primary ICUs.9, 20

The observed equivalence in the care delivered may be a function of the relatively high number of patients with sepsis treated in the overflow unit, thereby giving the delivery teams enough experience to provide the desired care. An alternative explanation could be that the residents in CICU brought with them the experience from having previously trained in the MICU. Although, some of the care processes for sepsis patients are influenced by the CPOE (with embedded order sets and protocols), it is unlikely that CPOE can fully account for similarity in care because many processes and therapies (like use of steroids, amount of fluid delivered in first 24 hours, packed red blood cells [PRBC] transfusion, and spontaneous breathing trials) are not embedded within order sets.

The significant difference noted in the areas of deep vein thrombosis (DVT) and gastrointestinal (GI) prophylaxis within 24 hours of ICU admission was unexpected. These preventive therapies are included in initial order sets in the CPOE, which prompt physicians to order them as standard‐of‐care. With respect to DVT prophylaxis, we suspect that some of the difference might be attributable to specific contraindications to its use, which could have been more common in one of the units. There were more patients in MICU on mechanical ventilation (although not statistically significant) and with severe sepsis (statistically significant) at time of admission, which might have contributed to the difference noted in use of GI prophylaxis. It is also plausible that these differences might have disappeared if they were reassessed beyond 24 hours into the ICU admission. We cannot rule out the presence of unit‐ and physician‐level differences that contributed to this. Likewise, there was an unexpected trend towards significance, wherein more patients in CICU had spontaneous breathing trials within 24 hours of admission. This might also be explained by the higher number of patients with severe sepsis in the MICU (preempting any weaning attempts). These caveats aside, it is reassuring that, at our institution, admitting septic patients to the first available ICU bed does not adversely affect important processes‐of‐care.

One might ask whether this study's data should reassure other sites who are boarding septic patients in non‐primary ICUs. Irrespective of the number of patients studied or the degree of statistical significance of the associations, an observational study design cannot prove that boarding septic patients in non‐primary ICUs is either safe or unsafe. However, we hope that readers reflect on, and take inventory of, systems issues that may be different between unitswith an eye towards eliminating variation such that all units managing septic patients are primed to deliver guideline‐concordant care. Other hospitals that use CPOE with sepsis order sets, have protocols for sepsis care, and who train nursing and respiratory therapists to meet high standards might be pleased to see that the patients in our study received comparable, high‐quality care across the 2 units. While our data suggests that boarding patients in overflow units may be safe, these findings would need to be replicated at other sites using prospective designs to prove safety.

Length of emergency room stay prior to admission is associated with higher mortality rates.2123 At many hospitals, critical care beds are a scarce resource such that most hospitals have a policy for the triage of patients to critical care beds.24, 25 Lundberg and colleagues' study demonstrated that patients who developed septic shock on the medical wards experienced delays in receipt of intravenous fluids, inotropic agents and transfer to a critical care setting.26 Thus, rather than waiting in the ED or on the medical service for an MICU bed to become available, it may be most wise to admit a critically sick septic patient to the first available ICU bed, even to an overflow ICU. In a recent study by Sidlow and Aggarwal, 1104 patients discharged from the coronary care unit (CCU) with a non‐cardiac primary diagnosis were compared to patients admitted to the MICU in the same hospital.27 The study found no differences in patient mortality, 30‐day readmission rate, hospital LOS, ICU LOS, and safety outcomes of ventilator‐associated pneumonia and catheter‐associated bloodstream infections between ICUs. However, their study did not examine processes‐of‐care delivered between the primary ICU and the overflow unit, and did not validate the primary diagnoses of patients admitted to the ICU.

Several limitations of this study should be considered. First, this study was conducted at a single center. Second, we used a retrospective study design; however, a prospective study randomizing patients to 1 of the 2 units would likely never be possible. Third, the relatively small number of patients limited the power of the study to detect mortality differences between the units. However, this was a pilot study focused on processes of care as opposed to clinical outcomes. Fourth, it is possible that we did not capture every single patient with sepsis with our keyword search. Our use of a previously validated screening process should have limited the number of missed cases.15, 16 Fifth, although the 2 ICUs have exclusive nursing staff and attending physicians, the housestaff and respiratory therapists do rotate between the 2 ICUs and place orders in the common CPOE. The rotating housestaff may certainly represent a source for confounding, but the large numbers (>30) of evenly spread housestaff over the study period minimizes the potential for any trainee to be responsible for a large proportion of observed practice. Sixth, ICU attendings are the physicians of record and could influence the results. Because no attending physician was on service for more than 4 weeks during the study period, and patients were equally spread over this same time, concerns about clustering and biases this may have created should be minimal but cannot be ruled out. Seventh, some interventions and processes, such as antibiotic administration and measurement of lactate, may have been initiated in the ED, thereby decreasing the potential for differences between the groups. Additionally, we cannot rule out the possibility that factors other than bed availability drove the admission process (we found that the relative proportion of patients admitted to overflow ICU during hours of ambulance diversion was similar to the overflow ICU admissions during non‐ambulance diversion hours). It is possible that some selection bias by the hospitalist assigning patients to specific ICUs influenced their triage decisionsalthough all triaging doctors go through the same process of training in active bed management.11 While more patients admitted to the MICU had severe sepsis, there were no differences between groups in APACHE II or SOFA scores. However, we cannot rule out that there were other residual confounders. Finally, in a small number of cases (4/41, 10%), the CICU team consulted the MICU attending for assistance. This input had the potential to reduce disparities in care between the units.

Overflowing patients to non‐primary ICUs occurs in many hospitals. Our study demonstrates that sepsis treatment for overflow patients may be similar to that received in the primary ICU. While a large multicentered and randomized trial could determine whether significant management and outcome differences exist between primary and overflow ICUs, feasibility concerns make it unlikely that such a study will ever be conducted.

Sepsis is a major cause of death in hospitalized patients.13 It is recommended that patients with sepsis be treated with early appropriate antibiotics, as well as early goal‐directed therapy including fluid and vasopressor support according to evidence‐based guidelines.46 Following such evidence‐based protocols and process‐of‐care interventions has been shown to be associated with better patient outcomes, including decreased mortality.7, 8

Most patients with severe sepsis are cared for in intensive care units (ICUs). At times, there are no beds available in the primary ICU and patients presenting to the hospital with sepsis are cared for in other units. Patients admitted to a non‐preferred clinical inpatient setting are sometimes referred to as overflow.9 ICUs can differ significantly in staffing patterns, equipment, and training.10 It is not known if overflow sepsis patients receive similar care when admitted to non‐primary ICUs.

At our hospital, we have an active bed management system led by the hospitalist division.11 This system includes protocols to place sepsis patients in the overflow ICU if the primary ICU is full. We hypothesized that process‐of‐care interventions would be more strictly adhered to when sepsis patients were in the primary ICU rather than in the overflow unit at our institution.

METHODS

RESULTS

There were 100 patients admitted to the MICU and 41 patients admitted to the CICU during the study period (Table 1). The majority of the patients were admitted to the ICUs directly from the emergency department (ED) (n = 129), with a small number of patients who were transferred from the Medicine floors (n = 12).

There were no significant differences between the 2 study groups in terms of age, sex, primary site of infection, mean APACHE II score, SOFA scores on day 1, chronic organ insufficiency, immune suppression, or need for mechanical ventilation (Table 1). The most common site of infection was lung. There were significantly more patients with severe sepsis in the MICU (88% vs 51%, P <0.001).

DISCUSSION

Since sepsis is more commonly treated in the medical ICU and some data suggests that specialty ICUs may be better at providing desired care,18, 19 we believed that patients treated in the MICU would be more likely to receive guideline‐concordant care. The study refutes our a priori hypothesis and reveals that evidence‐based processes‐of‐care associated with improved outcomes for sepsis are similarly implemented at our institution in the primary and overflow ICU. These findings are important, as ICU bed availability is a frequent problem and many hospitals overflow patients to non‐primary ICUs.9, 20

The observed equivalence in the care delivered may be a function of the relatively high number of patients with sepsis treated in the overflow unit, thereby giving the delivery teams enough experience to provide the desired care. An alternative explanation could be that the residents in CICU brought with them the experience from having previously trained in the MICU. Although, some of the care processes for sepsis patients are influenced by the CPOE (with embedded order sets and protocols), it is unlikely that CPOE can fully account for similarity in care because many processes and therapies (like use of steroids, amount of fluid delivered in first 24 hours, packed red blood cells [PRBC] transfusion, and spontaneous breathing trials) are not embedded within order sets.

The significant difference noted in the areas of deep vein thrombosis (DVT) and gastrointestinal (GI) prophylaxis within 24 hours of ICU admission was unexpected. These preventive therapies are included in initial order sets in the CPOE, which prompt physicians to order them as standard‐of‐care. With respect to DVT prophylaxis, we suspect that some of the difference might be attributable to specific contraindications to its use, which could have been more common in one of the units. There were more patients in MICU on mechanical ventilation (although not statistically significant) and with severe sepsis (statistically significant) at time of admission, which might have contributed to the difference noted in use of GI prophylaxis. It is also plausible that these differences might have disappeared if they were reassessed beyond 24 hours into the ICU admission. We cannot rule out the presence of unit‐ and physician‐level differences that contributed to this. Likewise, there was an unexpected trend towards significance, wherein more patients in CICU had spontaneous breathing trials within 24 hours of admission. This might also be explained by the higher number of patients with severe sepsis in the MICU (preempting any weaning attempts). These caveats aside, it is reassuring that, at our institution, admitting septic patients to the first available ICU bed does not adversely affect important processes‐of‐care.

One might ask whether this study's data should reassure other sites who are boarding septic patients in non‐primary ICUs. Irrespective of the number of patients studied or the degree of statistical significance of the associations, an observational study design cannot prove that boarding septic patients in non‐primary ICUs is either safe or unsafe. However, we hope that readers reflect on, and take inventory of, systems issues that may be different between unitswith an eye towards eliminating variation such that all units managing septic patients are primed to deliver guideline‐concordant care. Other hospitals that use CPOE with sepsis order sets, have protocols for sepsis care, and who train nursing and respiratory therapists to meet high standards might be pleased to see that the patients in our study received comparable, high‐quality care across the 2 units. While our data suggests that boarding patients in overflow units may be safe, these findings would need to be replicated at other sites using prospective designs to prove safety.

Length of emergency room stay prior to admission is associated with higher mortality rates.2123 At many hospitals, critical care beds are a scarce resource such that most hospitals have a policy for the triage of patients to critical care beds.24, 25 Lundberg and colleagues' study demonstrated that patients who developed septic shock on the medical wards experienced delays in receipt of intravenous fluids, inotropic agents and transfer to a critical care setting.26 Thus, rather than waiting in the ED or on the medical service for an MICU bed to become available, it may be most wise to admit a critically sick septic patient to the first available ICU bed, even to an overflow ICU. In a recent study by Sidlow and Aggarwal, 1104 patients discharged from the coronary care unit (CCU) with a non‐cardiac primary diagnosis were compared to patients admitted to the MICU in the same hospital.27 The study found no differences in patient mortality, 30‐day readmission rate, hospital LOS, ICU LOS, and safety outcomes of ventilator‐associated pneumonia and catheter‐associated bloodstream infections between ICUs. However, their study did not examine processes‐of‐care delivered between the primary ICU and the overflow unit, and did not validate the primary diagnoses of patients admitted to the ICU.

Several limitations of this study should be considered. First, this study was conducted at a single center. Second, we used a retrospective study design; however, a prospective study randomizing patients to 1 of the 2 units would likely never be possible. Third, the relatively small number of patients limited the power of the study to detect mortality differences between the units. However, this was a pilot study focused on processes of care as opposed to clinical outcomes. Fourth, it is possible that we did not capture every single patient with sepsis with our keyword search. Our use of a previously validated screening process should have limited the number of missed cases.15, 16 Fifth, although the 2 ICUs have exclusive nursing staff and attending physicians, the housestaff and respiratory therapists do rotate between the 2 ICUs and place orders in the common CPOE. The rotating housestaff may certainly represent a source for confounding, but the large numbers (>30) of evenly spread housestaff over the study period minimizes the potential for any trainee to be responsible for a large proportion of observed practice. Sixth, ICU attendings are the physicians of record and could influence the results. Because no attending physician was on service for more than 4 weeks during the study period, and patients were equally spread over this same time, concerns about clustering and biases this may have created should be minimal but cannot be ruled out. Seventh, some interventions and processes, such as antibiotic administration and measurement of lactate, may have been initiated in the ED, thereby decreasing the potential for differences between the groups. Additionally, we cannot rule out the possibility that factors other than bed availability drove the admission process (we found that the relative proportion of patients admitted to overflow ICU during hours of ambulance diversion was similar to the overflow ICU admissions during non‐ambulance diversion hours). It is possible that some selection bias by the hospitalist assigning patients to specific ICUs influenced their triage decisionsalthough all triaging doctors go through the same process of training in active bed management.11 While more patients admitted to the MICU had severe sepsis, there were no differences between groups in APACHE II or SOFA scores. However, we cannot rule out that there were other residual confounders. Finally, in a small number of cases (4/41, 10%), the CICU team consulted the MICU attending for assistance. This input had the potential to reduce disparities in care between the units.

Overflowing patients to non‐primary ICUs occurs in many hospitals. Our study demonstrates that sepsis treatment for overflow patients may be similar to that received in the primary ICU. While a large multicentered and randomized trial could determine whether significant management and outcome differences exist between primary and overflow ICUs, feasibility concerns make it unlikely that such a study will ever be conducted.