Frank J. Zadravecz, MPH

Patients on hospital wards may become critically ill due to worsening of the underlying condition that was the cause of their admission or acquisition of a new hospital‐acquired illness. Once physiologic deterioration occurs, some patients are evaluated and quickly transferred to the intensive care unit (ICU), whereas others are left on the wards until further deterioration occurs. Because many critical illness syndromes benefit from early intervention, such as sepsis and respiratory failure, early transfer to the ICU for treatment may improve patient outcomes, and conversely, delays in ICU transfer may lead to increased mortality and length of stay (LOS) in critically ill ward patients.[1, 2] However, the timeliness of that transfer is dependent on numerous changing variables, such as ICU bed availability, clinician identification of the deterioration, and clinical judgment regarding the appropriate transfer thresholds.[2, 3, 4, 5, 6, 7] As a result, there is a large degree of heterogeneity in the severity of illness of patients at the time of ICU transfer and in patient outcomes.[6, 8]

Previous studies investigating the association between delayed ICU transfer and patient outcomes have typically utilized the time of consultation by the ICU team to denote the onset of critical illness.[5, 6, 9, 10] However, the decision to transfer a patient to the ICU is often subjective, and previous studies have found an alarmingly high rate of errors in diagnosis and management of critically ill ward patients, including the failure to call for help.[2, 11] Therefore, a more objective tool for quantifying critical illness is necessary for determining the onset of critical illness and quantifying the association of transfer delay with patient outcomes.

Early warning scores, which are designed to detect critical illness on the wards, represent objective measures of critical illness that can be easily calculated in ward patients.[12] The aim of this study was to utilize the electronic Cardiac Arrest Risk Triage (eCART) score, a previously published, statistically derived early warning score that utilizes demographic, vital sign, and laboratory data, as an objective measure of critical illness to estimate the effect of delayed ICU transfer on patient outcomes in a large, multicenter database.[13] We chose 6 hours as the cutoff for delay in this study a priori because it is a threshold noted to be an important time period in critical illness syndromes, such as sepsis.[14, 15]

METHODS

All patients admitted to the medical‐surgical wards at 5 hospitals between November 2008 and January 2013 were eligible for inclusion in this observational cohort study. Further details of the hospital populations have been previously described.[13] A waiver of consent was granted by NorthShore University HealthSystem (IRB #EH11‐258) and the University of Chicago Institutional Review Board (IRB #16995A) based on general impracticability and minimal harm. Collection of patient information was designed to comply with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulations.

RESULTS

A total of 269,999 admissions had documented vital signs on the hospital wards during the study period, including 11,995 patients who were either transferred from the wards to the ICU (n=11,636) or who suffered a cardiac arrest on the wards (n=359) during their initial ward stay. Of these patients, 3789 reached an eCART score at the 95% specificity cutoff (critical eCART score of 60) within 24 hours of transfer. The median time from first critical eCART value to ICU transfer was 5.4 hours (interquartile range (IQR), 214 hours; mean, 8 hours). Compared to patients without delayed ICU transfer, those with delayed transfer were slightly older (median age, 73 [IQR, 6083] years vs 71 [IQR, 5882] years; P=0.002), whereas all other characteristics were similar (Table 1). Table 2 shows comparisons of vital sign and laboratory results for delayed and nondelayed transfers at the time of ICU transfer. As shown, patients with delayed transfer had lower median respiratory rate, blood pressure, heart rate, and hemoglobin, but higher median white blood cell count and creatinine.

Delayed transfer occurred in 46% of patients (n=1734) and was associated with increased in‐hospital mortality (33.2% vs 24.5%, P<0.001). This relationship was linear, with each 1‐hour increase in transfer delay associated with a 3% increase in the odds of in‐hospital death (P<0.001) (Figure 1). The association between length of transfer delay and hospital mortality remained unchanged after controlling for age, sex, surgical status, initial eCART score on the wards, vital signs, laboratory values, and whether the ICU transfer was due to a cardiac arrest (3% increase per hour, P<0.001). This association did not vary based on the initial eCART score on the wards (P=0.71 for interaction). Additionally, despite having similar median hospital lengths of stay prior to first critical eCART score (1.6 vs 1.5 days, P=0.04), patients experiencing delayed ICU transfer who survived to discharge had a longer median hospital LOS by 2 days compared to those with nondelayed transfer who survived to discharge (median LOS, 13 (821) days vs 11 (719) days, P=0.01). The change in eCART score over time in the 12 hours before first reaching the critical eCART score until ICU transfer is shown in Figure 2 for patients with delayed and nondelayed transfer. As shown, patients transferred within 6 hours had a more rapid rise in eCART score prior to ICU transfer compared to those with a delayed transfer. This difference in trajectories between delayed and nondelayed patients was similar in patients with low (<13), intermediate (1359), and high (60) initial eCART scores on the wards. A regression model investigating the association between eCART slope prior to ICU transfer and time to ICU transfer demonstrated that a steeper slope was significantly associated with a decreased time to ICU transfer (P<0.01).

Figure 1

Figure 2

DISCUSSION

We found that a delay in transfer to the ICU after reaching a predefined objective threshold of critical illness was associated with a significant increase in hospital mortality and hospital LOS. We also discovered a significant association between critical illness trajectory and delays in transfer, suggesting that caregivers may not recognize more subtle trends in critical illness. This work highlights the importance of timely transfer to the ICU for critically ill ward patients, which can be affected by several factors such as ICU bed availability and caregiver recognition and triage decisions. Our findings have significant implications for patient safety on the wards and provide further evidence for implementing early warning scores into practice to aid with clinical decision making.

Our findings of increased mortality with delayed ICU transfer are consistent with previous studies.[1, 5, 9] For example, Young et al. compared ICU mortality between delayed and nondelayed transfers in 91 consecutive patients with noncardiac diagnoses at a community hospital.[1] They also used predefined criteria for critical illness, and found that delayed transfers had a higher ICU mortality than nondelayed patients (41% vs 11%). However, their criteria for critical illness only had a specificity of 13% for predicting ICU transfer, compared to 95% in our study, suggesting that our threshold is more consistent with critical illness. Another study, by Cardoso and colleagues, investigated the impact of delayed ICU admission due to bed shortages on ICU mortality in 401 patients at a university hospital.[9] Of those patients deemed appropriate for transfer to the ICU but who had to wait for a bed to become available, the median wait time for a bed was 18 hours. They found that each hour of waiting was associated with a 1.5% increase in ICU death. A similar study by Robert and colleagues investigated the impact of delayed or refused ICU admission due to a lack of bed availability.[5] Patients deemed too sick (or too well) to benefit from ICU transfer were excluded. Twenty‐eightday and 60‐day mortality were higher in the admitted group compared to those not admitted, although this finding was not statistically significant. In addition, patients later admitted to the ICU once a bed became available (median wait time, 6 hours; n=89) had higher 28‐day mortality than those admitted immediately (adjusted odds ratio, 1.78; P=0.05). Several other studies have investigated the impact of ICU refusal for reasons that included bed shortages, and found increased mortality in those not admitted to the ICU.[16, 17] However, many of these studies included patients deemed too sick or too well to be transferred to the ICU in the group of nonadmitted patients. Our study adds to this literature by utilizing a highly specific objective measure of critical illness and by including all patients on the wards who reached this threshold, rather than only those for whom a consult was requested.

There are several potential explanations for our finding of increased mortality with delayed ICU transfer. First, those with delayed transfer might be different in some way from those transferred immediately. For example, we found that those with delayed transfer were older. The finding that increasing age is associated with a delay in ICU transfer is interesting, and may reflect physiologic differences in older patients compared to younger ones. For example, older patients have a lower maximum heart rate and thus may not develop the same level of vital sign abnormalities that younger patients do, causing them to be inappropriately left on the wards for too long.[18] In addition, patients with delayed transfer had more deranged renal function and lower blood pressure. It is unknown whether these organ dysfunctions would have been prevented by earlier transfer and to what degree they were related to chronic conditions. However, delayed transfer was still associated with increased mortality even after controlling for age, vital sign and laboratory values, and eCART on ward admission. It may also be possible that patients with delayed transfer received early and appropriate treatment on the wards but failed to improve and thus required ICU transfer. We did not have access to orders in this large database, so this theory will need to be investigated in future work. Finally, the most likely explanation for our findings is that earlier identification and treatment improves outcomes of critically ill patients on the wards, which is consistent with the findings of previous studies.[1, 5, 9, 10] Our study demonstrates that early identification of critical illness is crucial, and that delayed treatment can rapidly lead to increased mortality and LOS.

Our comparison of eCART score trajectory showed that patients transferred within 6 hours of onset of critical illness had a more rapid rise in eCART score over the preceding time period, whereas patients who experienced transfer delay showed a slower increase in eCART score. One explanation for this finding is that patients who decompensate more rapidly are in turn more readily recognizable to providers, whereas patients who experience a more insidious clinical deterioration are recognized later in the process, which then leads to a delay in escalation of care. This hypothesis underlines the importance of utilizing an objective marker of illness that is calculated longitudinally and in real time, as opposed to relying upon provider recognition alone. In fact, we have recently demonstrated that eCART is more accurate and identifies patients earlier than standard rapid response team activation.[19]

There are several important implications of our findings. First, it highlights the potential impact that early warning scores, particular those that are evidence based, can have on the outcomes of hospitalized patients. Second, it suggests that it is important to include age in early warning scores. Previous studies have been mixed as to whether the inclusion of age improves detection of outcomes on the wards, although the method of inclusion of age has been variable in terms of its weighting.[20, 21, 22] Our study found that older patients were more likely to be left on the wards longer prior to ICU transfer after becoming critically ill. By incorporating age into early warning scores, both accuracy and early recognition of critical illness may be improved. Finally, our finding that the trends of the eCART score differed among patients who were immediately transferred to the ICU, and who had a delay in their transfer, suggests that adding vital sign trends to early warning scores may further improve their accuracy and ability to serve as clinical decision support tools.

Our study is unique in that we used an objective measure of critical illness and then examined outcomes after patients reached this threshold on the wards. This overcomes the subjectivity of using evaluation by the ICU team or rapid response team as the starting point, as previous studies have shown a failure to call for help when patients become critically ill on the wards.[2, 11, 23] By using the eCART score, which contains commonly collected electronic health record data and can be calculated electronically in real time, we were able to calculate the score for patients on the wards and in the ICU. This allowed us to examine trends in the eCART score over time to find clues as to why some patients are transferred late to the ICU and why these late transfers have worse outcomes than those transferred earlier. Another strength is the large multicenter database used for the analysis, which included an urban tertiary care hospital, suburban teaching hospitals, and a community nonteaching hospital.

Our study has several limitations. First, we utilized just 1 of many potential measures of critical illness and a cutoff that only included one‐third of patients ultimately transferred to the ICU. However, by using the eCART score, we were able to track a patient's physiologic status over time and remove the variability that comes with using subjective definitions of critical illness. Furthermore, we utilized a high‐specificity cutoff for eCART to ensure that transferred patients had significantly deranged physiology and to avoid including planned transfers to the ICU. It is likely that some patients who were critically ill with less deranged physiology that would have benefitted from earlier transfer were excluded from the study. Second, we were unable to determine the cause of physiologic deterioration for patients in our study due to the large number of included patients. In addition, we did not have code status, comorbidities, or reason for ICU admission available in the dataset. It is likely that the impact of delayed transfer varies by the indication for ICU admission and chronic disease burden. It is also possible that controlling for these unmeasured factors could negate the beneficial association seen for earlier ICU admission. However, our finding of such a strong relationship between time to transfer and mortality after controlling for several important variables suggests that early recognition of critical illness is beneficial to many patients on the wards. Third, due to its observational nature, our study cannot estimate the true impact of timely ICU transfer on critically ill ward patient outcomes. Future clinical trials will be needed to determine the impact of electronic early warning scores on patient outcomes.

In conclusion, delayed ICU transfer is associated with significantly increased hospital LOS and mortality. This association highlights the need for ongoing work toward both the implementation of an evidence‐based risk stratification tool as well as development of effective critical care outreach resources for patients decompensating on the wards. Real‐time use of a validated early warning score, such as eCART, could potentially lead to more timely ICU transfer for critically ill patients and reduced rates of preventable in‐hospital death.

Patients on hospital wards may become critically ill due to worsening of the underlying condition that was the cause of their admission or acquisition of a new hospital‐acquired illness. Once physiologic deterioration occurs, some patients are evaluated and quickly transferred to the intensive care unit (ICU), whereas others are left on the wards until further deterioration occurs. Because many critical illness syndromes benefit from early intervention, such as sepsis and respiratory failure, early transfer to the ICU for treatment may improve patient outcomes, and conversely, delays in ICU transfer may lead to increased mortality and length of stay (LOS) in critically ill ward patients.[1, 2] However, the timeliness of that transfer is dependent on numerous changing variables, such as ICU bed availability, clinician identification of the deterioration, and clinical judgment regarding the appropriate transfer thresholds.[2, 3, 4, 5, 6, 7] As a result, there is a large degree of heterogeneity in the severity of illness of patients at the time of ICU transfer and in patient outcomes.[6, 8]

Previous studies investigating the association between delayed ICU transfer and patient outcomes have typically utilized the time of consultation by the ICU team to denote the onset of critical illness.[5, 6, 9, 10] However, the decision to transfer a patient to the ICU is often subjective, and previous studies have found an alarmingly high rate of errors in diagnosis and management of critically ill ward patients, including the failure to call for help.[2, 11] Therefore, a more objective tool for quantifying critical illness is necessary for determining the onset of critical illness and quantifying the association of transfer delay with patient outcomes.

Early warning scores, which are designed to detect critical illness on the wards, represent objective measures of critical illness that can be easily calculated in ward patients.[12] The aim of this study was to utilize the electronic Cardiac Arrest Risk Triage (eCART) score, a previously published, statistically derived early warning score that utilizes demographic, vital sign, and laboratory data, as an objective measure of critical illness to estimate the effect of delayed ICU transfer on patient outcomes in a large, multicenter database.[13] We chose 6 hours as the cutoff for delay in this study a priori because it is a threshold noted to be an important time period in critical illness syndromes, such as sepsis.[14, 15]

METHODS

All patients admitted to the medical‐surgical wards at 5 hospitals between November 2008 and January 2013 were eligible for inclusion in this observational cohort study. Further details of the hospital populations have been previously described.[13] A waiver of consent was granted by NorthShore University HealthSystem (IRB #EH11‐258) and the University of Chicago Institutional Review Board (IRB #16995A) based on general impracticability and minimal harm. Collection of patient information was designed to comply with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulations.

RESULTS

A total of 269,999 admissions had documented vital signs on the hospital wards during the study period, including 11,995 patients who were either transferred from the wards to the ICU (n=11,636) or who suffered a cardiac arrest on the wards (n=359) during their initial ward stay. Of these patients, 3789 reached an eCART score at the 95% specificity cutoff (critical eCART score of 60) within 24 hours of transfer. The median time from first critical eCART value to ICU transfer was 5.4 hours (interquartile range (IQR), 214 hours; mean, 8 hours). Compared to patients without delayed ICU transfer, those with delayed transfer were slightly older (median age, 73 [IQR, 6083] years vs 71 [IQR, 5882] years; P=0.002), whereas all other characteristics were similar (Table 1). Table 2 shows comparisons of vital sign and laboratory results for delayed and nondelayed transfers at the time of ICU transfer. As shown, patients with delayed transfer had lower median respiratory rate, blood pressure, heart rate, and hemoglobin, but higher median white blood cell count and creatinine.

Delayed transfer occurred in 46% of patients (n=1734) and was associated with increased in‐hospital mortality (33.2% vs 24.5%, P<0.001). This relationship was linear, with each 1‐hour increase in transfer delay associated with a 3% increase in the odds of in‐hospital death (P<0.001) (Figure 1). The association between length of transfer delay and hospital mortality remained unchanged after controlling for age, sex, surgical status, initial eCART score on the wards, vital signs, laboratory values, and whether the ICU transfer was due to a cardiac arrest (3% increase per hour, P<0.001). This association did not vary based on the initial eCART score on the wards (P=0.71 for interaction). Additionally, despite having similar median hospital lengths of stay prior to first critical eCART score (1.6 vs 1.5 days, P=0.04), patients experiencing delayed ICU transfer who survived to discharge had a longer median hospital LOS by 2 days compared to those with nondelayed transfer who survived to discharge (median LOS, 13 (821) days vs 11 (719) days, P=0.01). The change in eCART score over time in the 12 hours before first reaching the critical eCART score until ICU transfer is shown in Figure 2 for patients with delayed and nondelayed transfer. As shown, patients transferred within 6 hours had a more rapid rise in eCART score prior to ICU transfer compared to those with a delayed transfer. This difference in trajectories between delayed and nondelayed patients was similar in patients with low (<13), intermediate (1359), and high (60) initial eCART scores on the wards. A regression model investigating the association between eCART slope prior to ICU transfer and time to ICU transfer demonstrated that a steeper slope was significantly associated with a decreased time to ICU transfer (P<0.01).

Figure 1

Figure 2

DISCUSSION

We found that a delay in transfer to the ICU after reaching a predefined objective threshold of critical illness was associated with a significant increase in hospital mortality and hospital LOS. We also discovered a significant association between critical illness trajectory and delays in transfer, suggesting that caregivers may not recognize more subtle trends in critical illness. This work highlights the importance of timely transfer to the ICU for critically ill ward patients, which can be affected by several factors such as ICU bed availability and caregiver recognition and triage decisions. Our findings have significant implications for patient safety on the wards and provide further evidence for implementing early warning scores into practice to aid with clinical decision making.

Our findings of increased mortality with delayed ICU transfer are consistent with previous studies.[1, 5, 9] For example, Young et al. compared ICU mortality between delayed and nondelayed transfers in 91 consecutive patients with noncardiac diagnoses at a community hospital.[1] They also used predefined criteria for critical illness, and found that delayed transfers had a higher ICU mortality than nondelayed patients (41% vs 11%). However, their criteria for critical illness only had a specificity of 13% for predicting ICU transfer, compared to 95% in our study, suggesting that our threshold is more consistent with critical illness. Another study, by Cardoso and colleagues, investigated the impact of delayed ICU admission due to bed shortages on ICU mortality in 401 patients at a university hospital.[9] Of those patients deemed appropriate for transfer to the ICU but who had to wait for a bed to become available, the median wait time for a bed was 18 hours. They found that each hour of waiting was associated with a 1.5% increase in ICU death. A similar study by Robert and colleagues investigated the impact of delayed or refused ICU admission due to a lack of bed availability.[5] Patients deemed too sick (or too well) to benefit from ICU transfer were excluded. Twenty‐eightday and 60‐day mortality were higher in the admitted group compared to those not admitted, although this finding was not statistically significant. In addition, patients later admitted to the ICU once a bed became available (median wait time, 6 hours; n=89) had higher 28‐day mortality than those admitted immediately (adjusted odds ratio, 1.78; P=0.05). Several other studies have investigated the impact of ICU refusal for reasons that included bed shortages, and found increased mortality in those not admitted to the ICU.[16, 17] However, many of these studies included patients deemed too sick or too well to be transferred to the ICU in the group of nonadmitted patients. Our study adds to this literature by utilizing a highly specific objective measure of critical illness and by including all patients on the wards who reached this threshold, rather than only those for whom a consult was requested.

There are several potential explanations for our finding of increased mortality with delayed ICU transfer. First, those with delayed transfer might be different in some way from those transferred immediately. For example, we found that those with delayed transfer were older. The finding that increasing age is associated with a delay in ICU transfer is interesting, and may reflect physiologic differences in older patients compared to younger ones. For example, older patients have a lower maximum heart rate and thus may not develop the same level of vital sign abnormalities that younger patients do, causing them to be inappropriately left on the wards for too long.[18] In addition, patients with delayed transfer had more deranged renal function and lower blood pressure. It is unknown whether these organ dysfunctions would have been prevented by earlier transfer and to what degree they were related to chronic conditions. However, delayed transfer was still associated with increased mortality even after controlling for age, vital sign and laboratory values, and eCART on ward admission. It may also be possible that patients with delayed transfer received early and appropriate treatment on the wards but failed to improve and thus required ICU transfer. We did not have access to orders in this large database, so this theory will need to be investigated in future work. Finally, the most likely explanation for our findings is that earlier identification and treatment improves outcomes of critically ill patients on the wards, which is consistent with the findings of previous studies.[1, 5, 9, 10] Our study demonstrates that early identification of critical illness is crucial, and that delayed treatment can rapidly lead to increased mortality and LOS.

Our comparison of eCART score trajectory showed that patients transferred within 6 hours of onset of critical illness had a more rapid rise in eCART score over the preceding time period, whereas patients who experienced transfer delay showed a slower increase in eCART score. One explanation for this finding is that patients who decompensate more rapidly are in turn more readily recognizable to providers, whereas patients who experience a more insidious clinical deterioration are recognized later in the process, which then leads to a delay in escalation of care. This hypothesis underlines the importance of utilizing an objective marker of illness that is calculated longitudinally and in real time, as opposed to relying upon provider recognition alone. In fact, we have recently demonstrated that eCART is more accurate and identifies patients earlier than standard rapid response team activation.[19]

There are several important implications of our findings. First, it highlights the potential impact that early warning scores, particular those that are evidence based, can have on the outcomes of hospitalized patients. Second, it suggests that it is important to include age in early warning scores. Previous studies have been mixed as to whether the inclusion of age improves detection of outcomes on the wards, although the method of inclusion of age has been variable in terms of its weighting.[20, 21, 22] Our study found that older patients were more likely to be left on the wards longer prior to ICU transfer after becoming critically ill. By incorporating age into early warning scores, both accuracy and early recognition of critical illness may be improved. Finally, our finding that the trends of the eCART score differed among patients who were immediately transferred to the ICU, and who had a delay in their transfer, suggests that adding vital sign trends to early warning scores may further improve their accuracy and ability to serve as clinical decision support tools.

Our study is unique in that we used an objective measure of critical illness and then examined outcomes after patients reached this threshold on the wards. This overcomes the subjectivity of using evaluation by the ICU team or rapid response team as the starting point, as previous studies have shown a failure to call for help when patients become critically ill on the wards.[2, 11, 23] By using the eCART score, which contains commonly collected electronic health record data and can be calculated electronically in real time, we were able to calculate the score for patients on the wards and in the ICU. This allowed us to examine trends in the eCART score over time to find clues as to why some patients are transferred late to the ICU and why these late transfers have worse outcomes than those transferred earlier. Another strength is the large multicenter database used for the analysis, which included an urban tertiary care hospital, suburban teaching hospitals, and a community nonteaching hospital.

Our study has several limitations. First, we utilized just 1 of many potential measures of critical illness and a cutoff that only included one‐third of patients ultimately transferred to the ICU. However, by using the eCART score, we were able to track a patient's physiologic status over time and remove the variability that comes with using subjective definitions of critical illness. Furthermore, we utilized a high‐specificity cutoff for eCART to ensure that transferred patients had significantly deranged physiology and to avoid including planned transfers to the ICU. It is likely that some patients who were critically ill with less deranged physiology that would have benefitted from earlier transfer were excluded from the study. Second, we were unable to determine the cause of physiologic deterioration for patients in our study due to the large number of included patients. In addition, we did not have code status, comorbidities, or reason for ICU admission available in the dataset. It is likely that the impact of delayed transfer varies by the indication for ICU admission and chronic disease burden. It is also possible that controlling for these unmeasured factors could negate the beneficial association seen for earlier ICU admission. However, our finding of such a strong relationship between time to transfer and mortality after controlling for several important variables suggests that early recognition of critical illness is beneficial to many patients on the wards. Third, due to its observational nature, our study cannot estimate the true impact of timely ICU transfer on critically ill ward patient outcomes. Future clinical trials will be needed to determine the impact of electronic early warning scores on patient outcomes.

In conclusion, delayed ICU transfer is associated with significantly increased hospital LOS and mortality. This association highlights the need for ongoing work toward both the implementation of an evidence‐based risk stratification tool as well as development of effective critical care outreach resources for patients decompensating on the wards. Real‐time use of a validated early warning score, such as eCART, could potentially lead to more timely ICU transfer for critically ill patients and reduced rates of preventable in‐hospital death.

Patients on hospital wards may become critically ill due to worsening of the underlying condition that was the cause of their admission or acquisition of a new hospital‐acquired illness. Once physiologic deterioration occurs, some patients are evaluated and quickly transferred to the intensive care unit (ICU), whereas others are left on the wards until further deterioration occurs. Because many critical illness syndromes benefit from early intervention, such as sepsis and respiratory failure, early transfer to the ICU for treatment may improve patient outcomes, and conversely, delays in ICU transfer may lead to increased mortality and length of stay (LOS) in critically ill ward patients.[1, 2] However, the timeliness of that transfer is dependent on numerous changing variables, such as ICU bed availability, clinician identification of the deterioration, and clinical judgment regarding the appropriate transfer thresholds.[2, 3, 4, 5, 6, 7] As a result, there is a large degree of heterogeneity in the severity of illness of patients at the time of ICU transfer and in patient outcomes.[6, 8]

Previous studies investigating the association between delayed ICU transfer and patient outcomes have typically utilized the time of consultation by the ICU team to denote the onset of critical illness.[5, 6, 9, 10] However, the decision to transfer a patient to the ICU is often subjective, and previous studies have found an alarmingly high rate of errors in diagnosis and management of critically ill ward patients, including the failure to call for help.[2, 11] Therefore, a more objective tool for quantifying critical illness is necessary for determining the onset of critical illness and quantifying the association of transfer delay with patient outcomes.

Early warning scores, which are designed to detect critical illness on the wards, represent objective measures of critical illness that can be easily calculated in ward patients.[12] The aim of this study was to utilize the electronic Cardiac Arrest Risk Triage (eCART) score, a previously published, statistically derived early warning score that utilizes demographic, vital sign, and laboratory data, as an objective measure of critical illness to estimate the effect of delayed ICU transfer on patient outcomes in a large, multicenter database.[13] We chose 6 hours as the cutoff for delay in this study a priori because it is a threshold noted to be an important time period in critical illness syndromes, such as sepsis.[14, 15]

METHODS

All patients admitted to the medical‐surgical wards at 5 hospitals between November 2008 and January 2013 were eligible for inclusion in this observational cohort study. Further details of the hospital populations have been previously described.[13] A waiver of consent was granted by NorthShore University HealthSystem (IRB #EH11‐258) and the University of Chicago Institutional Review Board (IRB #16995A) based on general impracticability and minimal harm. Collection of patient information was designed to comply with the Health Insurance Portability and Accountability Act of 1996 (HIPAA) regulations.

RESULTS

A total of 269,999 admissions had documented vital signs on the hospital wards during the study period, including 11,995 patients who were either transferred from the wards to the ICU (n=11,636) or who suffered a cardiac arrest on the wards (n=359) during their initial ward stay. Of these patients, 3789 reached an eCART score at the 95% specificity cutoff (critical eCART score of 60) within 24 hours of transfer. The median time from first critical eCART value to ICU transfer was 5.4 hours (interquartile range (IQR), 214 hours; mean, 8 hours). Compared to patients without delayed ICU transfer, those with delayed transfer were slightly older (median age, 73 [IQR, 6083] years vs 71 [IQR, 5882] years; P=0.002), whereas all other characteristics were similar (Table 1). Table 2 shows comparisons of vital sign and laboratory results for delayed and nondelayed transfers at the time of ICU transfer. As shown, patients with delayed transfer had lower median respiratory rate, blood pressure, heart rate, and hemoglobin, but higher median white blood cell count and creatinine.

Delayed transfer occurred in 46% of patients (n=1734) and was associated with increased in‐hospital mortality (33.2% vs 24.5%, P<0.001). This relationship was linear, with each 1‐hour increase in transfer delay associated with a 3% increase in the odds of in‐hospital death (P<0.001) (Figure 1). The association between length of transfer delay and hospital mortality remained unchanged after controlling for age, sex, surgical status, initial eCART score on the wards, vital signs, laboratory values, and whether the ICU transfer was due to a cardiac arrest (3% increase per hour, P<0.001). This association did not vary based on the initial eCART score on the wards (P=0.71 for interaction). Additionally, despite having similar median hospital lengths of stay prior to first critical eCART score (1.6 vs 1.5 days, P=0.04), patients experiencing delayed ICU transfer who survived to discharge had a longer median hospital LOS by 2 days compared to those with nondelayed transfer who survived to discharge (median LOS, 13 (821) days vs 11 (719) days, P=0.01). The change in eCART score over time in the 12 hours before first reaching the critical eCART score until ICU transfer is shown in Figure 2 for patients with delayed and nondelayed transfer. As shown, patients transferred within 6 hours had a more rapid rise in eCART score prior to ICU transfer compared to those with a delayed transfer. This difference in trajectories between delayed and nondelayed patients was similar in patients with low (<13), intermediate (1359), and high (60) initial eCART scores on the wards. A regression model investigating the association between eCART slope prior to ICU transfer and time to ICU transfer demonstrated that a steeper slope was significantly associated with a decreased time to ICU transfer (P<0.01).

Figure 1

Figure 2

DISCUSSION

We found that a delay in transfer to the ICU after reaching a predefined objective threshold of critical illness was associated with a significant increase in hospital mortality and hospital LOS. We also discovered a significant association between critical illness trajectory and delays in transfer, suggesting that caregivers may not recognize more subtle trends in critical illness. This work highlights the importance of timely transfer to the ICU for critically ill ward patients, which can be affected by several factors such as ICU bed availability and caregiver recognition and triage decisions. Our findings have significant implications for patient safety on the wards and provide further evidence for implementing early warning scores into practice to aid with clinical decision making.

Our findings of increased mortality with delayed ICU transfer are consistent with previous studies.[1, 5, 9] For example, Young et al. compared ICU mortality between delayed and nondelayed transfers in 91 consecutive patients with noncardiac diagnoses at a community hospital.[1] They also used predefined criteria for critical illness, and found that delayed transfers had a higher ICU mortality than nondelayed patients (41% vs 11%). However, their criteria for critical illness only had a specificity of 13% for predicting ICU transfer, compared to 95% in our study, suggesting that our threshold is more consistent with critical illness. Another study, by Cardoso and colleagues, investigated the impact of delayed ICU admission due to bed shortages on ICU mortality in 401 patients at a university hospital.[9] Of those patients deemed appropriate for transfer to the ICU but who had to wait for a bed to become available, the median wait time for a bed was 18 hours. They found that each hour of waiting was associated with a 1.5% increase in ICU death. A similar study by Robert and colleagues investigated the impact of delayed or refused ICU admission due to a lack of bed availability.[5] Patients deemed too sick (or too well) to benefit from ICU transfer were excluded. Twenty‐eightday and 60‐day mortality were higher in the admitted group compared to those not admitted, although this finding was not statistically significant. In addition, patients later admitted to the ICU once a bed became available (median wait time, 6 hours; n=89) had higher 28‐day mortality than those admitted immediately (adjusted odds ratio, 1.78; P=0.05). Several other studies have investigated the impact of ICU refusal for reasons that included bed shortages, and found increased mortality in those not admitted to the ICU.[16, 17] However, many of these studies included patients deemed too sick or too well to be transferred to the ICU in the group of nonadmitted patients. Our study adds to this literature by utilizing a highly specific objective measure of critical illness and by including all patients on the wards who reached this threshold, rather than only those for whom a consult was requested.

There are several potential explanations for our finding of increased mortality with delayed ICU transfer. First, those with delayed transfer might be different in some way from those transferred immediately. For example, we found that those with delayed transfer were older. The finding that increasing age is associated with a delay in ICU transfer is interesting, and may reflect physiologic differences in older patients compared to younger ones. For example, older patients have a lower maximum heart rate and thus may not develop the same level of vital sign abnormalities that younger patients do, causing them to be inappropriately left on the wards for too long.[18] In addition, patients with delayed transfer had more deranged renal function and lower blood pressure. It is unknown whether these organ dysfunctions would have been prevented by earlier transfer and to what degree they were related to chronic conditions. However, delayed transfer was still associated with increased mortality even after controlling for age, vital sign and laboratory values, and eCART on ward admission. It may also be possible that patients with delayed transfer received early and appropriate treatment on the wards but failed to improve and thus required ICU transfer. We did not have access to orders in this large database, so this theory will need to be investigated in future work. Finally, the most likely explanation for our findings is that earlier identification and treatment improves outcomes of critically ill patients on the wards, which is consistent with the findings of previous studies.[1, 5, 9, 10] Our study demonstrates that early identification of critical illness is crucial, and that delayed treatment can rapidly lead to increased mortality and LOS.

Our comparison of eCART score trajectory showed that patients transferred within 6 hours of onset of critical illness had a more rapid rise in eCART score over the preceding time period, whereas patients who experienced transfer delay showed a slower increase in eCART score. One explanation for this finding is that patients who decompensate more rapidly are in turn more readily recognizable to providers, whereas patients who experience a more insidious clinical deterioration are recognized later in the process, which then leads to a delay in escalation of care. This hypothesis underlines the importance of utilizing an objective marker of illness that is calculated longitudinally and in real time, as opposed to relying upon provider recognition alone. In fact, we have recently demonstrated that eCART is more accurate and identifies patients earlier than standard rapid response team activation.[19]

There are several important implications of our findings. First, it highlights the potential impact that early warning scores, particular those that are evidence based, can have on the outcomes of hospitalized patients. Second, it suggests that it is important to include age in early warning scores. Previous studies have been mixed as to whether the inclusion of age improves detection of outcomes on the wards, although the method of inclusion of age has been variable in terms of its weighting.[20, 21, 22] Our study found that older patients were more likely to be left on the wards longer prior to ICU transfer after becoming critically ill. By incorporating age into early warning scores, both accuracy and early recognition of critical illness may be improved. Finally, our finding that the trends of the eCART score differed among patients who were immediately transferred to the ICU, and who had a delay in their transfer, suggests that adding vital sign trends to early warning scores may further improve their accuracy and ability to serve as clinical decision support tools.

Our study is unique in that we used an objective measure of critical illness and then examined outcomes after patients reached this threshold on the wards. This overcomes the subjectivity of using evaluation by the ICU team or rapid response team as the starting point, as previous studies have shown a failure to call for help when patients become critically ill on the wards.[2, 11, 23] By using the eCART score, which contains commonly collected electronic health record data and can be calculated electronically in real time, we were able to calculate the score for patients on the wards and in the ICU. This allowed us to examine trends in the eCART score over time to find clues as to why some patients are transferred late to the ICU and why these late transfers have worse outcomes than those transferred earlier. Another strength is the large multicenter database used for the analysis, which included an urban tertiary care hospital, suburban teaching hospitals, and a community nonteaching hospital.

Our study has several limitations. First, we utilized just 1 of many potential measures of critical illness and a cutoff that only included one‐third of patients ultimately transferred to the ICU. However, by using the eCART score, we were able to track a patient's physiologic status over time and remove the variability that comes with using subjective definitions of critical illness. Furthermore, we utilized a high‐specificity cutoff for eCART to ensure that transferred patients had significantly deranged physiology and to avoid including planned transfers to the ICU. It is likely that some patients who were critically ill with less deranged physiology that would have benefitted from earlier transfer were excluded from the study. Second, we were unable to determine the cause of physiologic deterioration for patients in our study due to the large number of included patients. In addition, we did not have code status, comorbidities, or reason for ICU admission available in the dataset. It is likely that the impact of delayed transfer varies by the indication for ICU admission and chronic disease burden. It is also possible that controlling for these unmeasured factors could negate the beneficial association seen for earlier ICU admission. However, our finding of such a strong relationship between time to transfer and mortality after controlling for several important variables suggests that early recognition of critical illness is beneficial to many patients on the wards. Third, due to its observational nature, our study cannot estimate the true impact of timely ICU transfer on critically ill ward patient outcomes. Future clinical trials will be needed to determine the impact of electronic early warning scores on patient outcomes.

In conclusion, delayed ICU transfer is associated with significantly increased hospital LOS and mortality. This association highlights the need for ongoing work toward both the implementation of an evidence‐based risk stratification tool as well as development of effective critical care outreach resources for patients decompensating on the wards. Real‐time use of a validated early warning score, such as eCART, could potentially lead to more timely ICU transfer for critically ill patients and reduced rates of preventable in‐hospital death.

Altered mental status (AMS), characterized by abnormal changes in a patient's arousal and/or cognition, is a significant predictor of hospital mortality.[1, 2, 3] Yet despite its prevalence[3, 4, 5] and importance, up to three‐quarters of AMS events go unrecognized by caregivers.[6, 7, 8] Acute changes in mental status, often caused by delirium in the hospitalized patient,[3] can present nonspecifically, making it difficult to detect and distinguish from other diagnoses such as depression or dementia.[7, 9] Further complicating the recognition of AMS, numerous and imprecise qualitative descriptors such as confused and alert and oriented are used in clinical practice to describe the mental status of patients.[10] Thus, more objective measures may result in improved detection of altered mental status and in earlier diagnostic and therapeutic interventions.

In critically ill patients, several scales have been widely adopted for quantifying mental status. The Richmond Agitation and Sedation Scale (RASS) was created to optimize sedation.[11] The Glasgow Coma Scale (GCS) was developed for head‐trauma patients[12] and is now a standardized assessment tool in intensive care units,[13] the emergency department,[14] and the prehospital setting.[15] In addition, a simplified scale, AVPU (Alert, responsive to Verbal stimuli, responsive to Painful stimuli, and Unresponsive) was initially used in the primary survey of trauma patients[16] but is now a common component of early‐warning scores and rapid response activation criteria, such as the Modified Early Warning Score (MEWS).[17, 18] In fact, in a systematic review of 72 distinct early‐warning scores, 89% of the scores used AVPU as the measure of mentation.[17] However, the utility of these 3 scales is not well established in the general‐ward setting. Our aim was therefore to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in hospitalized general‐ward patients to provide insight into the accuracy of these different scores for clinical deterioration.

METHODS

RESULTS

During the study period, 313,577 complete GCS and 305,177 RASS scores were recorded in the electronic health record by nursing staff. A total of 26,806 (17,603 GCS and 9203 RASS) observations were excluded due to nonsimultaneous measurement of the other score, resulting in 295,974 paired mental‐status observations. These observations were obtained from 26,873 admissions in 17,660 unique patients, with a median MEWS at ward admission of 1 (IQR 11). The mean patient age was 57 years (SD 17), and 23% were surgical patients (Table 1). Patients spent a median 63.9 hours (IQR 26.7118.6) on the wards per admission and contributed a median of 3 paired observations (IQR 24) per day, with 91% of patients having at least 2 observations per day. A total of 417 (1.6%) general‐ward admissions resulted in death during the hospitalization, with 354 mental‐status observations occurring within 24 hours of a death. In addition, 26,618 (99.9%) admissions had at least 1 paired mental‐status observation within the last 24 hours of their ward stay.

AVPU was moderately correlated with GCS (Spearman's =0.56) (Figure 1a) and weakly correlated with RASS (Spearman's =0.28) (Figure 1b). GCS scores were also weakly correlated to RASS (Spearman's =0.13, P<0.001). Notably, AVPU mapped to distinct levels of GCS, with Alert associated with a median GCS total score of 15, Voice a score of 12, Pain a score of 8, and Unresponsive a score of 5. Abnormal mental‐status scores on any scale were associated with significantly higher odds of death within 24 hours than normal mental‐status scores (Table 2). This association was consistent within the 3 subscales of GCS and for scores in both the sedation (<0) and agitation (>0) ranges of RASS.

Figure 1

AVPU was the least accurate predictor of mortality (AUC 0.73 [95% confidence interval {CI}: 0.710.76]), whereas simultaneous use of GCS and RASS was the most accurate predictor (AUC 0.85 [95% CI: 0.820.87] (Figure 2). The accuracies of GCS and RASS were not significantly different from one another in the total study population (AUC 0.80 [95% CI: 0.770.83] and 0.82 [0.790.84], respectively, P=0.13). Ten‐fold cross‐validation to estimate the internal validity of the RASS model resulted in a lower AUC (0.78 [95% CI: 0.750.81]) for RASS as a predictor of 24‐hour mortality. Subgroup analysis indicated that RASS was more accurate than GCS in younger patients (<57 years old) and in surgical patients (Figure 3).

Figure 2

Figure 3

Removal of the 255 admissions missing a paired mental‐status observation within the last 24 hours of their ward stay resulted in no change in the AUC values. A sensitivity analysis for prediction of a combined secondary outcome of 24‐hour intensive care unit ICU transfer or cardiac arrest yielded lower AUCs for each mental‐status scale, with no change in the association among scales.

DISCUSSION

To our knowledge, this study is the first to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in the general‐ward setting. Similar to McNarry and Goldhill, we demonstrated that AVPU scores mapped to distinct levels of GCS. Although our study reports the same median GCS scores of 15 and 8 for AVPU levels of Alert and Pain, respectively, we indicate slightly lower corresponding median GCS scores for AVPU scores of Voice (12 vs 13) and Unresponsive (5 vs 6) than their previous work.[21] We found that AVPU was the least accurate predictor of mortality within 24 hours of an observation, and the combination of GCS and RASS was the most accurate. RASS was at least as accurate a predictor for 24‐hour mortality in comparison to GCS total in the overall study population. However, the RASS score was the most accurate individual score in surgical and younger patients. These findings suggest that changing from the commonly used AVPU scale to the RASS and/or GCS would improve the prognostic ability of mental‐status assessments on the general wards.

Buist and colleagues have previously demonstrated altered mental status to be one of the strongest predictors of death on the wards. In that study, a GCS score of 3 and a decrease in GCS score by more than 2 points were independently associated with mortality (odds ratio 6.1 [95% CI: 3.111.8] and 5.5 [95% CI: 2.611.9], respectively).[22] We have also previously shown that after adjusting for vital signs, being unresponsive to pain was associated with a 4.5‐fold increase in the odds of death within 24 hours,[23]whereas Subbe and colleagues showed a relative risk ratio of 5.2 (95% CI: 1.518.1) for the combined endpoint of cardiac arrest, death at 60 days, or admission to the intensive care/high dependency unit.[19] In the current study, the magnitude of these associations was even stronger, with a GCS score <13 correlating with a 55‐fold increase in the odds of death, compared to a normal GCS, and not being alert being associated with a 33.8‐fold increase in the odds of death. This difference in magnitude is likely a product of the univariate nature of the current analysis, compared to both the Buist et al. and Churpek et al. studies, which adjusted for vital signs, thereby lessening the impact of any single predictor. Because this study was designed to compare mental‐status variables to one another for future model inclusion, and all the analyses were paired, confounding by additional predictors of death was not a concern.

One of the potential strengths of RASS over GCS and AVPU is its ability to measure agitation levels, in addition to depressed mentation, a feature that has been shown to be present in up to 60% of delirium episodes.[24] This may also explain why RASS was the most accurate predictor of mortality in our subset of younger patients and surgical patients, because hyperactive delirium is more common in younger and healthier patients, which surgical patients tend to be as compared to medical patients.[25, 26] In this study, we found negative RASS scores portending a worse prognosis than positive ones, which supports previous findings that hypoactive delirium had a higher association with mortality than hyperactive delirium at 6 months (hazard ratio 1.90 vs 1.37) and at 1 year (hazard ratio 1.60 vs 1.30) in elderly patients at postacute‐care facilities in 2 separate studies.[27, 28] However, a study of patients undergoing surgery for hip fracture found that patients with hyperactive delirium were more likely to die or be placed in a nursing home at 1 month follow‐up when compared to patients with purely hypoactive delirium (79% vs 32%, P=0.003).[29]

We found the assessment of RASS and GCS by ward nurses to be highly feasible. During the study period, nurses assessed mental status with the GCS and RASS scales at least once per 12‐hour shift in 91% of patients. GCS has been shown to be reliably and accurately recorded by experienced nurses (reliability coefficient=0.944 with 96.4% agreement with expert ratings).[30] RASS can take <30 seconds to administer, and in previous studies of the ICU setting has been shown to have over 94% nurse compliance for administration,[31] and good inter‐rater reliability (weighted kappa 0.66 and 0.89, respectively).[31, 32] Further, in a prior survey of 55 critical care nurses, 82% agreed that RASS was easy to score and clinically relevant.[31]

This study has several limitations. First, it was conducted in a single academic institution, which may limit generalizability to other hospitals. Second, baseline cognition and comorbidities were not available in the dataset, so we were unable to conduct additional subgroup analyses by these categories. However, we used age and hospital admission type as proxies. Third, the AVPU scores in this study were extracted from the Eye subset of the GCS scale, as AVPU was not directly assessed on our wards during the study period. Clinical assessment of mental status on the AVPU scale notes the presence of any active patient response (eg, eye opening, grunting, moaning, movement) to increasingly noxious stimuli. As such, our adaptation of AVPU using only eye‐opening criteria may underestimate the true number of patients correctly classified as alert, or responding to vocal/painful stimuli. However, a sensitivity analysis comparing directly assessed AVPU during a 3‐year period prior to the study implementation at our institution, and AVPU derived from the GCS Eye subscale for the study period, indicated no difference in predictive value for 24‐hour mortality. Fourth, we did not perform trend analyses for change from baseline mental status or evolution of AMS, which may more accurately predict 24‐hour mortality than discrete mental‐status observations. Finally, the 3 scales we compared differ in length, which may bias the AUC against AVPU, a 4‐point scale with a trapezoidal ROC curve compared to the smoother curve generated by the 15‐point GCS scale, for example. However, the lack of discrimination of the AVPU is the likely source of its lesser accuracy.

CONCLUSION

In the general‐ward setting, routine collection of GCS and RASS is feasible, and both are significantly more accurate for predicting mortality than the more commonly used AVPU scale. In addition, the combination of GCS and RASS has greater accuracy than any of the 3 individual scales. RASS may be particularly beneficial in the assessment of younger and/or surgical patients. Routine documentation and tracking of GCS and/or RASS by nurses may improve the detection of clinical deterioration in general‐ward patients. In addition, future early‐warning scores may benefit from the inclusion of GCS and/or RASS in lieu of AVPU.

Altered mental status (AMS), characterized by abnormal changes in a patient's arousal and/or cognition, is a significant predictor of hospital mortality.[1, 2, 3] Yet despite its prevalence[3, 4, 5] and importance, up to three‐quarters of AMS events go unrecognized by caregivers.[6, 7, 8] Acute changes in mental status, often caused by delirium in the hospitalized patient,[3] can present nonspecifically, making it difficult to detect and distinguish from other diagnoses such as depression or dementia.[7, 9] Further complicating the recognition of AMS, numerous and imprecise qualitative descriptors such as confused and alert and oriented are used in clinical practice to describe the mental status of patients.[10] Thus, more objective measures may result in improved detection of altered mental status and in earlier diagnostic and therapeutic interventions.

In critically ill patients, several scales have been widely adopted for quantifying mental status. The Richmond Agitation and Sedation Scale (RASS) was created to optimize sedation.[11] The Glasgow Coma Scale (GCS) was developed for head‐trauma patients[12] and is now a standardized assessment tool in intensive care units,[13] the emergency department,[14] and the prehospital setting.[15] In addition, a simplified scale, AVPU (Alert, responsive to Verbal stimuli, responsive to Painful stimuli, and Unresponsive) was initially used in the primary survey of trauma patients[16] but is now a common component of early‐warning scores and rapid response activation criteria, such as the Modified Early Warning Score (MEWS).[17, 18] In fact, in a systematic review of 72 distinct early‐warning scores, 89% of the scores used AVPU as the measure of mentation.[17] However, the utility of these 3 scales is not well established in the general‐ward setting. Our aim was therefore to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in hospitalized general‐ward patients to provide insight into the accuracy of these different scores for clinical deterioration.

METHODS

RESULTS

During the study period, 313,577 complete GCS and 305,177 RASS scores were recorded in the electronic health record by nursing staff. A total of 26,806 (17,603 GCS and 9203 RASS) observations were excluded due to nonsimultaneous measurement of the other score, resulting in 295,974 paired mental‐status observations. These observations were obtained from 26,873 admissions in 17,660 unique patients, with a median MEWS at ward admission of 1 (IQR 11). The mean patient age was 57 years (SD 17), and 23% were surgical patients (Table 1). Patients spent a median 63.9 hours (IQR 26.7118.6) on the wards per admission and contributed a median of 3 paired observations (IQR 24) per day, with 91% of patients having at least 2 observations per day. A total of 417 (1.6%) general‐ward admissions resulted in death during the hospitalization, with 354 mental‐status observations occurring within 24 hours of a death. In addition, 26,618 (99.9%) admissions had at least 1 paired mental‐status observation within the last 24 hours of their ward stay.

AVPU was moderately correlated with GCS (Spearman's =0.56) (Figure 1a) and weakly correlated with RASS (Spearman's =0.28) (Figure 1b). GCS scores were also weakly correlated to RASS (Spearman's =0.13, P<0.001). Notably, AVPU mapped to distinct levels of GCS, with Alert associated with a median GCS total score of 15, Voice a score of 12, Pain a score of 8, and Unresponsive a score of 5. Abnormal mental‐status scores on any scale were associated with significantly higher odds of death within 24 hours than normal mental‐status scores (Table 2). This association was consistent within the 3 subscales of GCS and for scores in both the sedation (<0) and agitation (>0) ranges of RASS.

Figure 1

AVPU was the least accurate predictor of mortality (AUC 0.73 [95% confidence interval {CI}: 0.710.76]), whereas simultaneous use of GCS and RASS was the most accurate predictor (AUC 0.85 [95% CI: 0.820.87] (Figure 2). The accuracies of GCS and RASS were not significantly different from one another in the total study population (AUC 0.80 [95% CI: 0.770.83] and 0.82 [0.790.84], respectively, P=0.13). Ten‐fold cross‐validation to estimate the internal validity of the RASS model resulted in a lower AUC (0.78 [95% CI: 0.750.81]) for RASS as a predictor of 24‐hour mortality. Subgroup analysis indicated that RASS was more accurate than GCS in younger patients (<57 years old) and in surgical patients (Figure 3).

Figure 2

Figure 3

Removal of the 255 admissions missing a paired mental‐status observation within the last 24 hours of their ward stay resulted in no change in the AUC values. A sensitivity analysis for prediction of a combined secondary outcome of 24‐hour intensive care unit ICU transfer or cardiac arrest yielded lower AUCs for each mental‐status scale, with no change in the association among scales.

DISCUSSION

To our knowledge, this study is the first to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in the general‐ward setting. Similar to McNarry and Goldhill, we demonstrated that AVPU scores mapped to distinct levels of GCS. Although our study reports the same median GCS scores of 15 and 8 for AVPU levels of Alert and Pain, respectively, we indicate slightly lower corresponding median GCS scores for AVPU scores of Voice (12 vs 13) and Unresponsive (5 vs 6) than their previous work.[21] We found that AVPU was the least accurate predictor of mortality within 24 hours of an observation, and the combination of GCS and RASS was the most accurate. RASS was at least as accurate a predictor for 24‐hour mortality in comparison to GCS total in the overall study population. However, the RASS score was the most accurate individual score in surgical and younger patients. These findings suggest that changing from the commonly used AVPU scale to the RASS and/or GCS would improve the prognostic ability of mental‐status assessments on the general wards.

Buist and colleagues have previously demonstrated altered mental status to be one of the strongest predictors of death on the wards. In that study, a GCS score of 3 and a decrease in GCS score by more than 2 points were independently associated with mortality (odds ratio 6.1 [95% CI: 3.111.8] and 5.5 [95% CI: 2.611.9], respectively).[22] We have also previously shown that after adjusting for vital signs, being unresponsive to pain was associated with a 4.5‐fold increase in the odds of death within 24 hours,[23]whereas Subbe and colleagues showed a relative risk ratio of 5.2 (95% CI: 1.518.1) for the combined endpoint of cardiac arrest, death at 60 days, or admission to the intensive care/high dependency unit.[19] In the current study, the magnitude of these associations was even stronger, with a GCS score <13 correlating with a 55‐fold increase in the odds of death, compared to a normal GCS, and not being alert being associated with a 33.8‐fold increase in the odds of death. This difference in magnitude is likely a product of the univariate nature of the current analysis, compared to both the Buist et al. and Churpek et al. studies, which adjusted for vital signs, thereby lessening the impact of any single predictor. Because this study was designed to compare mental‐status variables to one another for future model inclusion, and all the analyses were paired, confounding by additional predictors of death was not a concern.

One of the potential strengths of RASS over GCS and AVPU is its ability to measure agitation levels, in addition to depressed mentation, a feature that has been shown to be present in up to 60% of delirium episodes.[24] This may also explain why RASS was the most accurate predictor of mortality in our subset of younger patients and surgical patients, because hyperactive delirium is more common in younger and healthier patients, which surgical patients tend to be as compared to medical patients.[25, 26] In this study, we found negative RASS scores portending a worse prognosis than positive ones, which supports previous findings that hypoactive delirium had a higher association with mortality than hyperactive delirium at 6 months (hazard ratio 1.90 vs 1.37) and at 1 year (hazard ratio 1.60 vs 1.30) in elderly patients at postacute‐care facilities in 2 separate studies.[27, 28] However, a study of patients undergoing surgery for hip fracture found that patients with hyperactive delirium were more likely to die or be placed in a nursing home at 1 month follow‐up when compared to patients with purely hypoactive delirium (79% vs 32%, P=0.003).[29]

We found the assessment of RASS and GCS by ward nurses to be highly feasible. During the study period, nurses assessed mental status with the GCS and RASS scales at least once per 12‐hour shift in 91% of patients. GCS has been shown to be reliably and accurately recorded by experienced nurses (reliability coefficient=0.944 with 96.4% agreement with expert ratings).[30] RASS can take <30 seconds to administer, and in previous studies of the ICU setting has been shown to have over 94% nurse compliance for administration,[31] and good inter‐rater reliability (weighted kappa 0.66 and 0.89, respectively).[31, 32] Further, in a prior survey of 55 critical care nurses, 82% agreed that RASS was easy to score and clinically relevant.[31]

This study has several limitations. First, it was conducted in a single academic institution, which may limit generalizability to other hospitals. Second, baseline cognition and comorbidities were not available in the dataset, so we were unable to conduct additional subgroup analyses by these categories. However, we used age and hospital admission type as proxies. Third, the AVPU scores in this study were extracted from the Eye subset of the GCS scale, as AVPU was not directly assessed on our wards during the study period. Clinical assessment of mental status on the AVPU scale notes the presence of any active patient response (eg, eye opening, grunting, moaning, movement) to increasingly noxious stimuli. As such, our adaptation of AVPU using only eye‐opening criteria may underestimate the true number of patients correctly classified as alert, or responding to vocal/painful stimuli. However, a sensitivity analysis comparing directly assessed AVPU during a 3‐year period prior to the study implementation at our institution, and AVPU derived from the GCS Eye subscale for the study period, indicated no difference in predictive value for 24‐hour mortality. Fourth, we did not perform trend analyses for change from baseline mental status or evolution of AMS, which may more accurately predict 24‐hour mortality than discrete mental‐status observations. Finally, the 3 scales we compared differ in length, which may bias the AUC against AVPU, a 4‐point scale with a trapezoidal ROC curve compared to the smoother curve generated by the 15‐point GCS scale, for example. However, the lack of discrimination of the AVPU is the likely source of its lesser accuracy.

CONCLUSION

In the general‐ward setting, routine collection of GCS and RASS is feasible, and both are significantly more accurate for predicting mortality than the more commonly used AVPU scale. In addition, the combination of GCS and RASS has greater accuracy than any of the 3 individual scales. RASS may be particularly beneficial in the assessment of younger and/or surgical patients. Routine documentation and tracking of GCS and/or RASS by nurses may improve the detection of clinical deterioration in general‐ward patients. In addition, future early‐warning scores may benefit from the inclusion of GCS and/or RASS in lieu of AVPU.

Altered mental status (AMS), characterized by abnormal changes in a patient's arousal and/or cognition, is a significant predictor of hospital mortality.[1, 2, 3] Yet despite its prevalence[3, 4, 5] and importance, up to three‐quarters of AMS events go unrecognized by caregivers.[6, 7, 8] Acute changes in mental status, often caused by delirium in the hospitalized patient,[3] can present nonspecifically, making it difficult to detect and distinguish from other diagnoses such as depression or dementia.[7, 9] Further complicating the recognition of AMS, numerous and imprecise qualitative descriptors such as confused and alert and oriented are used in clinical practice to describe the mental status of patients.[10] Thus, more objective measures may result in improved detection of altered mental status and in earlier diagnostic and therapeutic interventions.

In critically ill patients, several scales have been widely adopted for quantifying mental status. The Richmond Agitation and Sedation Scale (RASS) was created to optimize sedation.[11] The Glasgow Coma Scale (GCS) was developed for head‐trauma patients[12] and is now a standardized assessment tool in intensive care units,[13] the emergency department,[14] and the prehospital setting.[15] In addition, a simplified scale, AVPU (Alert, responsive to Verbal stimuli, responsive to Painful stimuli, and Unresponsive) was initially used in the primary survey of trauma patients[16] but is now a common component of early‐warning scores and rapid response activation criteria, such as the Modified Early Warning Score (MEWS).[17, 18] In fact, in a systematic review of 72 distinct early‐warning scores, 89% of the scores used AVPU as the measure of mentation.[17] However, the utility of these 3 scales is not well established in the general‐ward setting. Our aim was therefore to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in hospitalized general‐ward patients to provide insight into the accuracy of these different scores for clinical deterioration.

METHODS

RESULTS

During the study period, 313,577 complete GCS and 305,177 RASS scores were recorded in the electronic health record by nursing staff. A total of 26,806 (17,603 GCS and 9203 RASS) observations were excluded due to nonsimultaneous measurement of the other score, resulting in 295,974 paired mental‐status observations. These observations were obtained from 26,873 admissions in 17,660 unique patients, with a median MEWS at ward admission of 1 (IQR 11). The mean patient age was 57 years (SD 17), and 23% were surgical patients (Table 1). Patients spent a median 63.9 hours (IQR 26.7118.6) on the wards per admission and contributed a median of 3 paired observations (IQR 24) per day, with 91% of patients having at least 2 observations per day. A total of 417 (1.6%) general‐ward admissions resulted in death during the hospitalization, with 354 mental‐status observations occurring within 24 hours of a death. In addition, 26,618 (99.9%) admissions had at least 1 paired mental‐status observation within the last 24 hours of their ward stay.

AVPU was moderately correlated with GCS (Spearman's =0.56) (Figure 1a) and weakly correlated with RASS (Spearman's =0.28) (Figure 1b). GCS scores were also weakly correlated to RASS (Spearman's =0.13, P<0.001). Notably, AVPU mapped to distinct levels of GCS, with Alert associated with a median GCS total score of 15, Voice a score of 12, Pain a score of 8, and Unresponsive a score of 5. Abnormal mental‐status scores on any scale were associated with significantly higher odds of death within 24 hours than normal mental‐status scores (Table 2). This association was consistent within the 3 subscales of GCS and for scores in both the sedation (<0) and agitation (>0) ranges of RASS.

Figure 1

AVPU was the least accurate predictor of mortality (AUC 0.73 [95% confidence interval {CI}: 0.710.76]), whereas simultaneous use of GCS and RASS was the most accurate predictor (AUC 0.85 [95% CI: 0.820.87] (Figure 2). The accuracies of GCS and RASS were not significantly different from one another in the total study population (AUC 0.80 [95% CI: 0.770.83] and 0.82 [0.790.84], respectively, P=0.13). Ten‐fold cross‐validation to estimate the internal validity of the RASS model resulted in a lower AUC (0.78 [95% CI: 0.750.81]) for RASS as a predictor of 24‐hour mortality. Subgroup analysis indicated that RASS was more accurate than GCS in younger patients (<57 years old) and in surgical patients (Figure 3).

Figure 2

Figure 3

Removal of the 255 admissions missing a paired mental‐status observation within the last 24 hours of their ward stay resulted in no change in the AUC values. A sensitivity analysis for prediction of a combined secondary outcome of 24‐hour intensive care unit ICU transfer or cardiac arrest yielded lower AUCs for each mental‐status scale, with no change in the association among scales.

DISCUSSION

To our knowledge, this study is the first to compare the accuracies of AVPU, GCS, and RASS for predicting mortality in the general‐ward setting. Similar to McNarry and Goldhill, we demonstrated that AVPU scores mapped to distinct levels of GCS. Although our study reports the same median GCS scores of 15 and 8 for AVPU levels of Alert and Pain, respectively, we indicate slightly lower corresponding median GCS scores for AVPU scores of Voice (12 vs 13) and Unresponsive (5 vs 6) than their previous work.[21] We found that AVPU was the least accurate predictor of mortality within 24 hours of an observation, and the combination of GCS and RASS was the most accurate. RASS was at least as accurate a predictor for 24‐hour mortality in comparison to GCS total in the overall study population. However, the RASS score was the most accurate individual score in surgical and younger patients. These findings suggest that changing from the commonly used AVPU scale to the RASS and/or GCS would improve the prognostic ability of mental‐status assessments on the general wards.

Buist and colleagues have previously demonstrated altered mental status to be one of the strongest predictors of death on the wards. In that study, a GCS score of 3 and a decrease in GCS score by more than 2 points were independently associated with mortality (odds ratio 6.1 [95% CI: 3.111.8] and 5.5 [95% CI: 2.611.9], respectively).[22] We have also previously shown that after adjusting for vital signs, being unresponsive to pain was associated with a 4.5‐fold increase in the odds of death within 24 hours,[23]whereas Subbe and colleagues showed a relative risk ratio of 5.2 (95% CI: 1.518.1) for the combined endpoint of cardiac arrest, death at 60 days, or admission to the intensive care/high dependency unit.[19] In the current study, the magnitude of these associations was even stronger, with a GCS score <13 correlating with a 55‐fold increase in the odds of death, compared to a normal GCS, and not being alert being associated with a 33.8‐fold increase in the odds of death. This difference in magnitude is likely a product of the univariate nature of the current analysis, compared to both the Buist et al. and Churpek et al. studies, which adjusted for vital signs, thereby lessening the impact of any single predictor. Because this study was designed to compare mental‐status variables to one another for future model inclusion, and all the analyses were paired, confounding by additional predictors of death was not a concern.

One of the potential strengths of RASS over GCS and AVPU is its ability to measure agitation levels, in addition to depressed mentation, a feature that has been shown to be present in up to 60% of delirium episodes.[24] This may also explain why RASS was the most accurate predictor of mortality in our subset of younger patients and surgical patients, because hyperactive delirium is more common in younger and healthier patients, which surgical patients tend to be as compared to medical patients.[25, 26] In this study, we found negative RASS scores portending a worse prognosis than positive ones, which supports previous findings that hypoactive delirium had a higher association with mortality than hyperactive delirium at 6 months (hazard ratio 1.90 vs 1.37) and at 1 year (hazard ratio 1.60 vs 1.30) in elderly patients at postacute‐care facilities in 2 separate studies.[27, 28] However, a study of patients undergoing surgery for hip fracture found that patients with hyperactive delirium were more likely to die or be placed in a nursing home at 1 month follow‐up when compared to patients with purely hypoactive delirium (79% vs 32%, P=0.003).[29]

We found the assessment of RASS and GCS by ward nurses to be highly feasible. During the study period, nurses assessed mental status with the GCS and RASS scales at least once per 12‐hour shift in 91% of patients. GCS has been shown to be reliably and accurately recorded by experienced nurses (reliability coefficient=0.944 with 96.4% agreement with expert ratings).[30] RASS can take <30 seconds to administer, and in previous studies of the ICU setting has been shown to have over 94% nurse compliance for administration,[31] and good inter‐rater reliability (weighted kappa 0.66 and 0.89, respectively).[31, 32] Further, in a prior survey of 55 critical care nurses, 82% agreed that RASS was easy to score and clinically relevant.[31]

This study has several limitations. First, it was conducted in a single academic institution, which may limit generalizability to other hospitals. Second, baseline cognition and comorbidities were not available in the dataset, so we were unable to conduct additional subgroup analyses by these categories. However, we used age and hospital admission type as proxies. Third, the AVPU scores in this study were extracted from the Eye subset of the GCS scale, as AVPU was not directly assessed on our wards during the study period. Clinical assessment of mental status on the AVPU scale notes the presence of any active patient response (eg, eye opening, grunting, moaning, movement) to increasingly noxious stimuli. As such, our adaptation of AVPU using only eye‐opening criteria may underestimate the true number of patients correctly classified as alert, or responding to vocal/painful stimuli. However, a sensitivity analysis comparing directly assessed AVPU during a 3‐year period prior to the study implementation at our institution, and AVPU derived from the GCS Eye subscale for the study period, indicated no difference in predictive value for 24‐hour mortality. Fourth, we did not perform trend analyses for change from baseline mental status or evolution of AMS, which may more accurately predict 24‐hour mortality than discrete mental‐status observations. Finally, the 3 scales we compared differ in length, which may bias the AUC against AVPU, a 4‐point scale with a trapezoidal ROC curve compared to the smoother curve generated by the 15‐point GCS scale, for example. However, the lack of discrimination of the AVPU is the likely source of its lesser accuracy.

CONCLUSION

In the general‐ward setting, routine collection of GCS and RASS is feasible, and both are significantly more accurate for predicting mortality than the more commonly used AVPU scale. In addition, the combination of GCS and RASS has greater accuracy than any of the 3 individual scales. RASS may be particularly beneficial in the assessment of younger and/or surgical patients. Routine documentation and tracking of GCS and/or RASS by nurses may improve the detection of clinical deterioration in general‐ward patients. In addition, future early‐warning scores may benefit from the inclusion of GCS and/or RASS in lieu of AVPU.

Obstructive sleep apnea (OSA) is an increasingly prevalent condition characterized by intermittent airway obstruction during sleep, which leads to hypoxemia, hypercapnia, and fragmented sleep. The current prevalence estimates of moderate to severe OSA (apnea‐hypopnea index 15, measured as events/hour) in middle‐aged adults are approximately 13% in men and 6% in women.[1] OSA is a well‐described independent risk factor for long‐term neurocognitive, cardiovascular, and cerebrovascular morbidity and mortality.[2, 3, 4, 5, 6]

Recent studies have also identified OSA as an independent risk factor for adverse perioperative outcomes, including endotracheal intubation, intensive care unit (ICU) transfer, and increased length of stay.[7, 8, 9, 10, 11] Paradoxically, despite an increase in the risk of complications, several of these studies did not find an association between in‐hospital death and OSA even after controlling for potential confounders.[9, 10, 11] Furthermore, a recent study of patients hospitalized for pneumonia reported increased rates of clinical deterioration and mechanical ventilation, but also lower odds of inpatient mortality in patients with OSA.[12]

These studies may have been limited by the absence of physiologic data, which prevented controlling for severity of illness. It is also unclear whether these previously described associations between OSA and adverse clinical outcomes hold true for general hospital inpatients. OSA may be worsened by medications frequently used in hospitals, such as narcotics and benzodiazepines. Opiate use contributes to both central and obstructive sleep apneas,[13, 14] and benzodiazepines are known to produce airway smooth muscle relaxation and can cause respiratory depression.[15] In fact, the use of benzodiazepines has been implicated in the unmasking of OSA in patients with previously undiagnosed sleep‐disordered breathing.[16] These findings suggest mechanisms by which OSA could contribute to an increased risk in hospital ward patients for rapid response team (RRT) activation, ICU transfer, cardiac arrest, and in‐hospital death.

The aim of this study was to determine the independent association between OSA and in‐hospital mortality in ward patients. We also aimed to investigate the association of OSA with clinical deterioration on the wards, while controlling for patient characteristics, initial physiology, and severity of illness.

MATERIALS AND METHODS

RESULTS

Complications and Adverse Outcomes

In the unadjusted analyses, the overall incidence of adverse outcomes was higher among patient admissions with a diagnosis of OSA compared to those without OSA (Table 3). Those with OSA were more likely to experience RRT activation (1.5% vs 1.1%), ICU transfer (8% vs 7%), and endotracheal intubation (3.9% vs 2.9%) than those without OSA diagnoses (P < 0.001 for all comparisons). There was no significant difference in the incidence of cardiac arrest between the 2 groups, nor was there a significant difference in length of stay. Unadjusted inpatient mortality for OSA patient admissions was lower than that for non‐OSA hospitalizations (1.1% vs 1.4%, P < 0.05). A diagnosis of OSA was associated with increased unadjusted odds for RRT activation (odds ratio [OR]: 1.36 [1.16‐1.59]) and ICU transfer (OR: 1.28 [1.20‐1.38]). However, after controlling for confounders, OSA was not associated with increased odds for RRT activation (OR: 1.14 [0.95‐1.36]) or intubation (OR: 1.06 [0.94‐1.19]), and was associated with slightly decreased odds for ICU transfer (OR: 0.91 [0.84‐0.99]) (Figure 1). Those with OSA had decreased adjusted odds of cardiac arrest (OR: 0.72 [0.55‐0.95]) compared to those without OSA. OSA was also associated with decreased odds of in‐hospital mortality before (OR: 0.83 [0.70‐0.99]) and after (OR: 0.70 [0.58‐0.85]) controlling for confounders.

Table 3.Unadjusted Outcomes for Patient Admissions With and Without OSA Diagnosis

Characteristic	Patient Admissions With OSA Diagnoses, n = 12,745	Patient Admissions Without OSA Diagnoses, n = 80,931	P Value
NOTE: Abbreviations: ICU, intensive care unit; OSA, obstructive sleep apnea. a Experiencing rapid response team call, ICU transfer, or in‐hospital death.
Outcomes, n (%)
Composite outcomea	1,137 (9%)	5,792 (7%)	<0.001
In‐hospital death	144 (1.1%)	1,095 (1.4%)	0.04
Rapid response team call	188 (1.5%)	881 (1.1%)	<0.001
ICU transfer	1,045 (8%)	5,260 (7%)	<0.001
Cardiac arrest	413 (0.5%)	73 (0.6%)	0.36

Figure 1

Subgroup Analyses

Surgical and Nonsurgical Patients

Subgroup analyses of surgical versus nonsurgical patients (Figure 2) revealed similarly decreased adjusted odds of in‐hospital death for OSA patients in both groups (surgical OR: 0.69 [0.49‐0.97]; nonsurgical OR: 0.72 [0.58‐0.91]). Surgical patients with OSA diagnoses had decreased adjusted odds for ICU transfer (surgical OR: 0.82 [0.73‐0.92], but this finding was not seen in nonsurgical patients (OR: 1.03 [0.92‐1.15]).

Figure 2

Patients Stratified by BMI

Examination across BMI categories (Figure 2) showed a significant decrease in adjusted odds of death for OSA patients with BMI 30 to 40 kg/m² (OR: 0.60 [0.43‐0.84]). A nonsignificant decrease in adjusted odds of death was seen for OSA patients in all other groups. Adjusted odds ratios for the risk of RRT activation and ICU transfer in OSA patients within the different BMI categories were not statistically significant.

DISCUSSION

In this large observational single‐center cohort study, we found that OSA was associated with increased odds of adverse events, such as ICU transfers and RRT calls, but this risk was no longer present after adjusting for demographics, comorbidities, and presenting vital signs. Interestingly, we also found that patients with OSA had decreased adjusted odds for cardiac arrest and mortality. This mortality finding was robust to multiple sensitivity analyses and subgroup analyses. These results have significant implications for our understanding of the short‐term risks of sleep‐disordered breathing in hospitalized patients, and suggest the possibility that OSA is associated with a protective effect with regard to inpatient mortality.

Our findings are in line with other recent work in this area. In 2 large observational cohorts of surgical populations drawn from the nationally representative Nationwide Inpatient Sample administrative database, our group reported decreased odds of in‐hospital postoperative mortality in OSA patients.[10, 11] Using the same Nationwide Inpatient Sample, Lindenauer et al. showed that among inpatients hospitalized with pneumonia, OSA diagnosis was associated with increased rates of clinical deterioration but lower rates of inpatient mortality.[12] Although these 3 studies have identified decreased inpatient mortality among certain surgical populations and patients hospitalized with pneumonia, they are limited by using administrative databases that do not provide specific data on vital signs, presenting physiology, BMI, or race. Another important limitation of the Nationwide Inpatient Sample is the lack of any information on RRT activations and ICU transfers. Moreover, the database does not include information on outpatient diagnoses, which may have led to a significantly lower prevalence of OSA than expected in these studies. Despite the important methodological differences, our study corroborates this finding among a diverse cohort of hospitalized patients. Unlike these previous studies of postoperative patients or those hospitalized with pneumonia, we did not find an increased risk of adverse events associated with OSA after controlling for potential confounders.

The decreased mortality seen in OSA patients could be explained by these patients receiving more vigilant care, showing earlier signs of deterioration, or displaying more easily treatable forms of distress than patients without OSA. For example, earlier identification of deterioration could lead to earlier interventions, which could decrease inpatient mortality. In 2 studies of postsurgical patients,[10, 11] those with OSA diagnosis who developed respiratory failure were intubated earlier and received mechanical ventilation for a shorter period of time, suggesting that the cause of respiratory failure was rapidly reversible (eg, upper airway complications due to oversedation or excessive analgesia). However, we did not find increased adjusted odds of RRT activation or ICU transfer for OSA patients in our study, and so it is less likely that earlier recognition of decompensation occurred in our sample. In addition, our hospital did not have standardized practices for monitoring or managing OSA patients during the study period, which makes systematic early recognition of clinical deterioration among the OSA population in our study less likely.

Alternatively, there may be a true physiologic phenomenon providing a short‐term mortality benefit in those with OSA. It has been observed that patients with obesity (but without severe obesity) often have better outcomes after acute illness, whether by earlier or more frequent contact with medical care or heightened levels of metabolic reserve.[21, 22] However, our findings of decreased mortality for OSA patients remained even after controlling for BMI. An additional important possibility to consider is ischemic preconditioning, a well‐described phenomenon in which episodes of sublethal ischemia confer protection on tissues from subsequent ischemia/reperfusion damage.[23] Ischemic preconditioning has been demonstrated in models of cardiac and neural tissue[24, 25, 26] and has been shown to enhance stem cell survival by providing resistance to necrosis and lending functional benefits to heart, brain, and kidney models after transplantation.[25, 26, 27, 28, 29, 30, 31] The fundamentals of this concept may have applications in transplant and cardiac surgery,[32, 33] in the management of acute coronary syndromes and stroke,[32, 34, 35] and in athletic training and performance.[35, 36] Although OSA has been associated with long‐term cardiovascular morbidity and mortality,[2, 3, 4, 5, 6] the intermittent hypoxemia OSA patients experience could actually improve their ability to survive clinical deterioration in the short‐term (ie, during a hospitalization).

Limitations of our study include its conduction at a single center, which may prevent generalization to populations different than ours. Furthermore, during the study period, our hospital did not have formal guidelines or standardized management or monitoring practices for patients with OSA. Additionally, practices for managing OSA may vary across institutions. Therefore, our results may not be generalizable to hospitals with such protocols in place. However, as mentioned above, similar findings have been noted in studies using large, nationally representative administrative databases. In addition, we identified OSA via ICD‐9‐CM codes, which are likely insensitive for estimating the true prevalence of OSA in our sample. Despite this, our reported OSA prevalence of over 10% falls within the prevalence range reported in large epidemiological studies.[37, 38, 39] Finally, we did not have data on polysomnograms or treatment received for patients with OSA, so we do not know the severity of OSA or adequacy of treatment for these patients.

Notwithstanding our limitations, our study has several strengths. First, we included a large number of hospitalized patients across a diverse range of medical and surgical ward admissions, which increases the generalizability of our results. We also addressed potential confounders by including a large number of comorbidities and controlling for severity of presenting physiology with the CART score. The CART score, which contains physiologic variables such as respiratory rate, heart rate, and diastolic blood pressure, is an accurate predictor of cardiac arrest, ICU transfer, and in‐hospital mortality in our population.[40] Finally, we were able to obtain information about these diagnoses from outpatient as well as inpatient data.

In conclusion, we found that after adjustment for important confounders, OSA was associated with a decrease in hospital mortality and cardiac arrest but not with other adverse events on the wards. These results may suggest a protective benefit from OSA with regard to mortality, an advantage that could be explained by ischemic preconditioning or a higher level of care or vigilance not reflected by the number of RRT activations or ICU transfers experienced by these patients. Further research is needed to confirm these findings across other populations, to investigate the physiologic pathways by which OSA may produce these effects, and to examine the mechanisms by which treatment of OSA could influence these outcomes.

Obstructive sleep apnea (OSA) is an increasingly prevalent condition characterized by intermittent airway obstruction during sleep, which leads to hypoxemia, hypercapnia, and fragmented sleep. The current prevalence estimates of moderate to severe OSA (apnea‐hypopnea index 15, measured as events/hour) in middle‐aged adults are approximately 13% in men and 6% in women.[1] OSA is a well‐described independent risk factor for long‐term neurocognitive, cardiovascular, and cerebrovascular morbidity and mortality.[2, 3, 4, 5, 6]

Recent studies have also identified OSA as an independent risk factor for adverse perioperative outcomes, including endotracheal intubation, intensive care unit (ICU) transfer, and increased length of stay.[7, 8, 9, 10, 11] Paradoxically, despite an increase in the risk of complications, several of these studies did not find an association between in‐hospital death and OSA even after controlling for potential confounders.[9, 10, 11] Furthermore, a recent study of patients hospitalized for pneumonia reported increased rates of clinical deterioration and mechanical ventilation, but also lower odds of inpatient mortality in patients with OSA.[12]

These studies may have been limited by the absence of physiologic data, which prevented controlling for severity of illness. It is also unclear whether these previously described associations between OSA and adverse clinical outcomes hold true for general hospital inpatients. OSA may be worsened by medications frequently used in hospitals, such as narcotics and benzodiazepines. Opiate use contributes to both central and obstructive sleep apneas,[13, 14] and benzodiazepines are known to produce airway smooth muscle relaxation and can cause respiratory depression.[15] In fact, the use of benzodiazepines has been implicated in the unmasking of OSA in patients with previously undiagnosed sleep‐disordered breathing.[16] These findings suggest mechanisms by which OSA could contribute to an increased risk in hospital ward patients for rapid response team (RRT) activation, ICU transfer, cardiac arrest, and in‐hospital death.

The aim of this study was to determine the independent association between OSA and in‐hospital mortality in ward patients. We also aimed to investigate the association of OSA with clinical deterioration on the wards, while controlling for patient characteristics, initial physiology, and severity of illness.

MATERIALS AND METHODS

RESULTS

Complications and Adverse Outcomes

In the unadjusted analyses, the overall incidence of adverse outcomes was higher among patient admissions with a diagnosis of OSA compared to those without OSA (Table 3). Those with OSA were more likely to experience RRT activation (1.5% vs 1.1%), ICU transfer (8% vs 7%), and endotracheal intubation (3.9% vs 2.9%) than those without OSA diagnoses (P < 0.001 for all comparisons). There was no significant difference in the incidence of cardiac arrest between the 2 groups, nor was there a significant difference in length of stay. Unadjusted inpatient mortality for OSA patient admissions was lower than that for non‐OSA hospitalizations (1.1% vs 1.4%, P < 0.05). A diagnosis of OSA was associated with increased unadjusted odds for RRT activation (odds ratio [OR]: 1.36 [1.16‐1.59]) and ICU transfer (OR: 1.28 [1.20‐1.38]). However, after controlling for confounders, OSA was not associated with increased odds for RRT activation (OR: 1.14 [0.95‐1.36]) or intubation (OR: 1.06 [0.94‐1.19]), and was associated with slightly decreased odds for ICU transfer (OR: 0.91 [0.84‐0.99]) (Figure 1). Those with OSA had decreased adjusted odds of cardiac arrest (OR: 0.72 [0.55‐0.95]) compared to those without OSA. OSA was also associated with decreased odds of in‐hospital mortality before (OR: 0.83 [0.70‐0.99]) and after (OR: 0.70 [0.58‐0.85]) controlling for confounders.

Table 3.Unadjusted Outcomes for Patient Admissions With and Without OSA Diagnosis

Characteristic	Patient Admissions With OSA Diagnoses, n = 12,745	Patient Admissions Without OSA Diagnoses, n = 80,931	P Value
NOTE: Abbreviations: ICU, intensive care unit; OSA, obstructive sleep apnea. a Experiencing rapid response team call, ICU transfer, or in‐hospital death.
Outcomes, n (%)
Composite outcomea	1,137 (9%)	5,792 (7%)	<0.001
In‐hospital death	144 (1.1%)	1,095 (1.4%)	0.04
Rapid response team call	188 (1.5%)	881 (1.1%)	<0.001
ICU transfer	1,045 (8%)	5,260 (7%)	<0.001
Cardiac arrest	413 (0.5%)	73 (0.6%)	0.36

Figure 1

Subgroup Analyses

Surgical and Nonsurgical Patients

Subgroup analyses of surgical versus nonsurgical patients (Figure 2) revealed similarly decreased adjusted odds of in‐hospital death for OSA patients in both groups (surgical OR: 0.69 [0.49‐0.97]; nonsurgical OR: 0.72 [0.58‐0.91]). Surgical patients with OSA diagnoses had decreased adjusted odds for ICU transfer (surgical OR: 0.82 [0.73‐0.92], but this finding was not seen in nonsurgical patients (OR: 1.03 [0.92‐1.15]).

Figure 2

Patients Stratified by BMI

Examination across BMI categories (Figure 2) showed a significant decrease in adjusted odds of death for OSA patients with BMI 30 to 40 kg/m² (OR: 0.60 [0.43‐0.84]). A nonsignificant decrease in adjusted odds of death was seen for OSA patients in all other groups. Adjusted odds ratios for the risk of RRT activation and ICU transfer in OSA patients within the different BMI categories were not statistically significant.

DISCUSSION

In this large observational single‐center cohort study, we found that OSA was associated with increased odds of adverse events, such as ICU transfers and RRT calls, but this risk was no longer present after adjusting for demographics, comorbidities, and presenting vital signs. Interestingly, we also found that patients with OSA had decreased adjusted odds for cardiac arrest and mortality. This mortality finding was robust to multiple sensitivity analyses and subgroup analyses. These results have significant implications for our understanding of the short‐term risks of sleep‐disordered breathing in hospitalized patients, and suggest the possibility that OSA is associated with a protective effect with regard to inpatient mortality.

Our findings are in line with other recent work in this area. In 2 large observational cohorts of surgical populations drawn from the nationally representative Nationwide Inpatient Sample administrative database, our group reported decreased odds of in‐hospital postoperative mortality in OSA patients.[10, 11] Using the same Nationwide Inpatient Sample, Lindenauer et al. showed that among inpatients hospitalized with pneumonia, OSA diagnosis was associated with increased rates of clinical deterioration but lower rates of inpatient mortality.[12] Although these 3 studies have identified decreased inpatient mortality among certain surgical populations and patients hospitalized with pneumonia, they are limited by using administrative databases that do not provide specific data on vital signs, presenting physiology, BMI, or race. Another important limitation of the Nationwide Inpatient Sample is the lack of any information on RRT activations and ICU transfers. Moreover, the database does not include information on outpatient diagnoses, which may have led to a significantly lower prevalence of OSA than expected in these studies. Despite the important methodological differences, our study corroborates this finding among a diverse cohort of hospitalized patients. Unlike these previous studies of postoperative patients or those hospitalized with pneumonia, we did not find an increased risk of adverse events associated with OSA after controlling for potential confounders.

The decreased mortality seen in OSA patients could be explained by these patients receiving more vigilant care, showing earlier signs of deterioration, or displaying more easily treatable forms of distress than patients without OSA. For example, earlier identification of deterioration could lead to earlier interventions, which could decrease inpatient mortality. In 2 studies of postsurgical patients,[10, 11] those with OSA diagnosis who developed respiratory failure were intubated earlier and received mechanical ventilation for a shorter period of time, suggesting that the cause of respiratory failure was rapidly reversible (eg, upper airway complications due to oversedation or excessive analgesia). However, we did not find increased adjusted odds of RRT activation or ICU transfer for OSA patients in our study, and so it is less likely that earlier recognition of decompensation occurred in our sample. In addition, our hospital did not have standardized practices for monitoring or managing OSA patients during the study period, which makes systematic early recognition of clinical deterioration among the OSA population in our study less likely.

Alternatively, there may be a true physiologic phenomenon providing a short‐term mortality benefit in those with OSA. It has been observed that patients with obesity (but without severe obesity) often have better outcomes after acute illness, whether by earlier or more frequent contact with medical care or heightened levels of metabolic reserve.[21, 22] However, our findings of decreased mortality for OSA patients remained even after controlling for BMI. An additional important possibility to consider is ischemic preconditioning, a well‐described phenomenon in which episodes of sublethal ischemia confer protection on tissues from subsequent ischemia/reperfusion damage.[23] Ischemic preconditioning has been demonstrated in models of cardiac and neural tissue[24, 25, 26] and has been shown to enhance stem cell survival by providing resistance to necrosis and lending functional benefits to heart, brain, and kidney models after transplantation.[25, 26, 27, 28, 29, 30, 31] The fundamentals of this concept may have applications in transplant and cardiac surgery,[32, 33] in the management of acute coronary syndromes and stroke,[32, 34, 35] and in athletic training and performance.[35, 36] Although OSA has been associated with long‐term cardiovascular morbidity and mortality,[2, 3, 4, 5, 6] the intermittent hypoxemia OSA patients experience could actually improve their ability to survive clinical deterioration in the short‐term (ie, during a hospitalization).

Limitations of our study include its conduction at a single center, which may prevent generalization to populations different than ours. Furthermore, during the study period, our hospital did not have formal guidelines or standardized management or monitoring practices for patients with OSA. Additionally, practices for managing OSA may vary across institutions. Therefore, our results may not be generalizable to hospitals with such protocols in place. However, as mentioned above, similar findings have been noted in studies using large, nationally representative administrative databases. In addition, we identified OSA via ICD‐9‐CM codes, which are likely insensitive for estimating the true prevalence of OSA in our sample. Despite this, our reported OSA prevalence of over 10% falls within the prevalence range reported in large epidemiological studies.[37, 38, 39] Finally, we did not have data on polysomnograms or treatment received for patients with OSA, so we do not know the severity of OSA or adequacy of treatment for these patients.

Notwithstanding our limitations, our study has several strengths. First, we included a large number of hospitalized patients across a diverse range of medical and surgical ward admissions, which increases the generalizability of our results. We also addressed potential confounders by including a large number of comorbidities and controlling for severity of presenting physiology with the CART score. The CART score, which contains physiologic variables such as respiratory rate, heart rate, and diastolic blood pressure, is an accurate predictor of cardiac arrest, ICU transfer, and in‐hospital mortality in our population.[40] Finally, we were able to obtain information about these diagnoses from outpatient as well as inpatient data.

In conclusion, we found that after adjustment for important confounders, OSA was associated with a decrease in hospital mortality and cardiac arrest but not with other adverse events on the wards. These results may suggest a protective benefit from OSA with regard to mortality, an advantage that could be explained by ischemic preconditioning or a higher level of care or vigilance not reflected by the number of RRT activations or ICU transfers experienced by these patients. Further research is needed to confirm these findings across other populations, to investigate the physiologic pathways by which OSA may produce these effects, and to examine the mechanisms by which treatment of OSA could influence these outcomes.

Obstructive sleep apnea (OSA) is an increasingly prevalent condition characterized by intermittent airway obstruction during sleep, which leads to hypoxemia, hypercapnia, and fragmented sleep. The current prevalence estimates of moderate to severe OSA (apnea‐hypopnea index 15, measured as events/hour) in middle‐aged adults are approximately 13% in men and 6% in women.[1] OSA is a well‐described independent risk factor for long‐term neurocognitive, cardiovascular, and cerebrovascular morbidity and mortality.[2, 3, 4, 5, 6]

Recent studies have also identified OSA as an independent risk factor for adverse perioperative outcomes, including endotracheal intubation, intensive care unit (ICU) transfer, and increased length of stay.[7, 8, 9, 10, 11] Paradoxically, despite an increase in the risk of complications, several of these studies did not find an association between in‐hospital death and OSA even after controlling for potential confounders.[9, 10, 11] Furthermore, a recent study of patients hospitalized for pneumonia reported increased rates of clinical deterioration and mechanical ventilation, but also lower odds of inpatient mortality in patients with OSA.[12]

These studies may have been limited by the absence of physiologic data, which prevented controlling for severity of illness. It is also unclear whether these previously described associations between OSA and adverse clinical outcomes hold true for general hospital inpatients. OSA may be worsened by medications frequently used in hospitals, such as narcotics and benzodiazepines. Opiate use contributes to both central and obstructive sleep apneas,[13, 14] and benzodiazepines are known to produce airway smooth muscle relaxation and can cause respiratory depression.[15] In fact, the use of benzodiazepines has been implicated in the unmasking of OSA in patients with previously undiagnosed sleep‐disordered breathing.[16] These findings suggest mechanisms by which OSA could contribute to an increased risk in hospital ward patients for rapid response team (RRT) activation, ICU transfer, cardiac arrest, and in‐hospital death.

The aim of this study was to determine the independent association between OSA and in‐hospital mortality in ward patients. We also aimed to investigate the association of OSA with clinical deterioration on the wards, while controlling for patient characteristics, initial physiology, and severity of illness.

MATERIALS AND METHODS

RESULTS

Complications and Adverse Outcomes

In the unadjusted analyses, the overall incidence of adverse outcomes was higher among patient admissions with a diagnosis of OSA compared to those without OSA (Table 3). Those with OSA were more likely to experience RRT activation (1.5% vs 1.1%), ICU transfer (8% vs 7%), and endotracheal intubation (3.9% vs 2.9%) than those without OSA diagnoses (P < 0.001 for all comparisons). There was no significant difference in the incidence of cardiac arrest between the 2 groups, nor was there a significant difference in length of stay. Unadjusted inpatient mortality for OSA patient admissions was lower than that for non‐OSA hospitalizations (1.1% vs 1.4%, P < 0.05). A diagnosis of OSA was associated with increased unadjusted odds for RRT activation (odds ratio [OR]: 1.36 [1.16‐1.59]) and ICU transfer (OR: 1.28 [1.20‐1.38]). However, after controlling for confounders, OSA was not associated with increased odds for RRT activation (OR: 1.14 [0.95‐1.36]) or intubation (OR: 1.06 [0.94‐1.19]), and was associated with slightly decreased odds for ICU transfer (OR: 0.91 [0.84‐0.99]) (Figure 1). Those with OSA had decreased adjusted odds of cardiac arrest (OR: 0.72 [0.55‐0.95]) compared to those without OSA. OSA was also associated with decreased odds of in‐hospital mortality before (OR: 0.83 [0.70‐0.99]) and after (OR: 0.70 [0.58‐0.85]) controlling for confounders.

Table 3.Unadjusted Outcomes for Patient Admissions With and Without OSA Diagnosis

Characteristic	Patient Admissions With OSA Diagnoses, n = 12,745	Patient Admissions Without OSA Diagnoses, n = 80,931	P Value
NOTE: Abbreviations: ICU, intensive care unit; OSA, obstructive sleep apnea. a Experiencing rapid response team call, ICU transfer, or in‐hospital death.
Outcomes, n (%)
Composite outcomea	1,137 (9%)	5,792 (7%)	<0.001
In‐hospital death	144 (1.1%)	1,095 (1.4%)	0.04
Rapid response team call	188 (1.5%)	881 (1.1%)	<0.001
ICU transfer	1,045 (8%)	5,260 (7%)	<0.001
Cardiac arrest	413 (0.5%)	73 (0.6%)	0.36

Figure 1

Subgroup Analyses

Surgical and Nonsurgical Patients

Subgroup analyses of surgical versus nonsurgical patients (Figure 2) revealed similarly decreased adjusted odds of in‐hospital death for OSA patients in both groups (surgical OR: 0.69 [0.49‐0.97]; nonsurgical OR: 0.72 [0.58‐0.91]). Surgical patients with OSA diagnoses had decreased adjusted odds for ICU transfer (surgical OR: 0.82 [0.73‐0.92], but this finding was not seen in nonsurgical patients (OR: 1.03 [0.92‐1.15]).

Figure 2

Patients Stratified by BMI

Examination across BMI categories (Figure 2) showed a significant decrease in adjusted odds of death for OSA patients with BMI 30 to 40 kg/m² (OR: 0.60 [0.43‐0.84]). A nonsignificant decrease in adjusted odds of death was seen for OSA patients in all other groups. Adjusted odds ratios for the risk of RRT activation and ICU transfer in OSA patients within the different BMI categories were not statistically significant.

DISCUSSION

In this large observational single‐center cohort study, we found that OSA was associated with increased odds of adverse events, such as ICU transfers and RRT calls, but this risk was no longer present after adjusting for demographics, comorbidities, and presenting vital signs. Interestingly, we also found that patients with OSA had decreased adjusted odds for cardiac arrest and mortality. This mortality finding was robust to multiple sensitivity analyses and subgroup analyses. These results have significant implications for our understanding of the short‐term risks of sleep‐disordered breathing in hospitalized patients, and suggest the possibility that OSA is associated with a protective effect with regard to inpatient mortality.

Our findings are in line with other recent work in this area. In 2 large observational cohorts of surgical populations drawn from the nationally representative Nationwide Inpatient Sample administrative database, our group reported decreased odds of in‐hospital postoperative mortality in OSA patients.[10, 11] Using the same Nationwide Inpatient Sample, Lindenauer et al. showed that among inpatients hospitalized with pneumonia, OSA diagnosis was associated with increased rates of clinical deterioration but lower rates of inpatient mortality.[12] Although these 3 studies have identified decreased inpatient mortality among certain surgical populations and patients hospitalized with pneumonia, they are limited by using administrative databases that do not provide specific data on vital signs, presenting physiology, BMI, or race. Another important limitation of the Nationwide Inpatient Sample is the lack of any information on RRT activations and ICU transfers. Moreover, the database does not include information on outpatient diagnoses, which may have led to a significantly lower prevalence of OSA than expected in these studies. Despite the important methodological differences, our study corroborates this finding among a diverse cohort of hospitalized patients. Unlike these previous studies of postoperative patients or those hospitalized with pneumonia, we did not find an increased risk of adverse events associated with OSA after controlling for potential confounders.

The decreased mortality seen in OSA patients could be explained by these patients receiving more vigilant care, showing earlier signs of deterioration, or displaying more easily treatable forms of distress than patients without OSA. For example, earlier identification of deterioration could lead to earlier interventions, which could decrease inpatient mortality. In 2 studies of postsurgical patients,[10, 11] those with OSA diagnosis who developed respiratory failure were intubated earlier and received mechanical ventilation for a shorter period of time, suggesting that the cause of respiratory failure was rapidly reversible (eg, upper airway complications due to oversedation or excessive analgesia). However, we did not find increased adjusted odds of RRT activation or ICU transfer for OSA patients in our study, and so it is less likely that earlier recognition of decompensation occurred in our sample. In addition, our hospital did not have standardized practices for monitoring or managing OSA patients during the study period, which makes systematic early recognition of clinical deterioration among the OSA population in our study less likely.

Alternatively, there may be a true physiologic phenomenon providing a short‐term mortality benefit in those with OSA. It has been observed that patients with obesity (but without severe obesity) often have better outcomes after acute illness, whether by earlier or more frequent contact with medical care or heightened levels of metabolic reserve.[21, 22] However, our findings of decreased mortality for OSA patients remained even after controlling for BMI. An additional important possibility to consider is ischemic preconditioning, a well‐described phenomenon in which episodes of sublethal ischemia confer protection on tissues from subsequent ischemia/reperfusion damage.[23] Ischemic preconditioning has been demonstrated in models of cardiac and neural tissue[24, 25, 26] and has been shown to enhance stem cell survival by providing resistance to necrosis and lending functional benefits to heart, brain, and kidney models after transplantation.[25, 26, 27, 28, 29, 30, 31] The fundamentals of this concept may have applications in transplant and cardiac surgery,[32, 33] in the management of acute coronary syndromes and stroke,[32, 34, 35] and in athletic training and performance.[35, 36] Although OSA has been associated with long‐term cardiovascular morbidity and mortality,[2, 3, 4, 5, 6] the intermittent hypoxemia OSA patients experience could actually improve their ability to survive clinical deterioration in the short‐term (ie, during a hospitalization).

Limitations of our study include its conduction at a single center, which may prevent generalization to populations different than ours. Furthermore, during the study period, our hospital did not have formal guidelines or standardized management or monitoring practices for patients with OSA. Additionally, practices for managing OSA may vary across institutions. Therefore, our results may not be generalizable to hospitals with such protocols in place. However, as mentioned above, similar findings have been noted in studies using large, nationally representative administrative databases. In addition, we identified OSA via ICD‐9‐CM codes, which are likely insensitive for estimating the true prevalence of OSA in our sample. Despite this, our reported OSA prevalence of over 10% falls within the prevalence range reported in large epidemiological studies.[37, 38, 39] Finally, we did not have data on polysomnograms or treatment received for patients with OSA, so we do not know the severity of OSA or adequacy of treatment for these patients.

Notwithstanding our limitations, our study has several strengths. First, we included a large number of hospitalized patients across a diverse range of medical and surgical ward admissions, which increases the generalizability of our results. We also addressed potential confounders by including a large number of comorbidities and controlling for severity of presenting physiology with the CART score. The CART score, which contains physiologic variables such as respiratory rate, heart rate, and diastolic blood pressure, is an accurate predictor of cardiac arrest, ICU transfer, and in‐hospital mortality in our population.[40] Finally, we were able to obtain information about these diagnoses from outpatient as well as inpatient data.

In conclusion, we found that after adjustment for important confounders, OSA was associated with a decrease in hospital mortality and cardiac arrest but not with other adverse events on the wards. These results may suggest a protective benefit from OSA with regard to mortality, an advantage that could be explained by ischemic preconditioning or a higher level of care or vigilance not reflected by the number of RRT activations or ICU transfers experienced by these patients. Further research is needed to confirm these findings across other populations, to investigate the physiologic pathways by which OSA may produce these effects, and to examine the mechanisms by which treatment of OSA could influence these outcomes.