Affiliations
Systems Research Initiative, Division of Research, Kaiser Permanente Medical Care Program, Oakland, California
Given name(s)
M. Kit
Family name
Delgado
Degrees
MD, MS

Risk Factors For Unplanned ICU Transfer

Article Type
Changed
Mon, 05/22/2017 - 18:05
Display Headline
Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system

Emergency Department (ED) patients who are hospitalized and require unplanned transfer to the intensive care unit (ICU) within 24 hours of arrival on the ward have higher mortality than direct ICU admissions.1, 2 Previous research found that 5% of ED admissions experienced unplanned ICU transfer during their hospitalization, yet these patients account for 25% of in‐hospital deaths and have a longer length of stay than direct ICU admissions.1, 3 For these reasons, inpatient rapid‐response teams and early warning systems have been studied to reduce the mortality of patients who rapidly deteriorate on the hospital ward.410 However, there is little conclusive evidence that these interventions decrease mortality.710 It is possible that with better recognition and intervention in the ED, a portion of these unplanned ICU transfers and their subsequent adverse outcomes could be prevented.11

Previous research on risk factors for unplanned ICU transfers among ED admissions is limited. While 2 previous studies from non‐US hospitals used administrative data to identify some general populations at risk for unplanned ICU transfer,12, 13 these studies did not differentiate between transfers shortly after admission and those that occurred during a prolonged hospital staya critical distinction since the outcomes between these groups differs substantially.1 Another limitation of these studies is the absence of physiologic measures at ED presentation, which have been shown to be highly predictive of mortality.14

In this study, we describe risk factors for unplanned transfer to the ICU within 24 hours of arrival on the ward, among a large cohort of ED hospitalizations across 13 community hospitals. Focusing on admitting diagnoses most at risk, our goal was to inform efforts to improve the triage of ED admissions and determine which patients may benefit from additional interventions, such as improved resuscitation, closer monitoring, or risk stratification tools. We also hypothesized that higher volume hospitals would have lower rates of unplanned ICU transfers, as these hospitals are more likely have more patient care resources on the hospital ward and a higher threshold to transfer to the ICU.

METHODS

Setting and Patients

The setting for this study was Kaiser Permanente Northern California (KPNC), a large integrated healthcare delivery system serving approximately 3.3 million members.1, 3, 15, 16 We extracted data on all adult ED admissions (18 years old) to the hospital between 2007 and 2009. We excluded patients who went directly to the operating room or the ICU, as well as gynecological/pregnancy‐related admissions, as these patients have substantially different mortality risks.14 ED admissions to hospital wards could either go to medicalsurgical units or transitional care units (TCU), an intermediate level of care between the medicalsurgical units and the ICU. We chose to focus on hospitals with similar inpatient structures. Thus, 8 hospitals without TCUs were excluded, leaving 13 hospitals for analysis. The KPNC Institutional Review Board approved this study.

Main Outcome Measure

The main outcome measure was unplanned transfer to the ICU within 24 hours of arrival to the hospital ward, based upon bed history data. As in previous research, we make the assumptionwhich is supported by the high observed‐to‐expected mortality ratios found in these patientsthat these transfers to the ICU were due to clinical deterioration, and thus were unplanned, rather than a planned transfer to the ICU as is more common after an elective surgical procedure.13 The comparison population was patients admitted from the ED to the ward who never experienced a transfer to the ICU.

Patient and Hospital Characteristics

We extracted patient data on age, sex, admitting diagnosis, chronic illness burden, acute physiologic derangement in the ED, and hospital unit length of stay. Chronic illness was measured using the Comorbidity Point Score (COPS), and physiologic derangement was measured using the Laboratory Acute Physiology Score (LAPS) calculated from labs collected in the ED.1, 14, 17 The derivation of these variables from the electronic medical record has been previously described.14 The COPS was derived from International Classification of Diseases, Ninth Revision (ICD‐9) codes for all Kaiser Permanente Medical Care Program (KPMCP) inpatient and outpatient encounters prior to hospitalization. The LAPS is based on 14 possible lab tests that could be drawn in the ED or in the 72 hours prior to hospitalization. The admitting diagnosis is the ICD‐9 code assigned for the primary diagnosis determined by the admitting physician at the time when hospital admission orders are entered. We further collapsed a previously used categorization of 44 primary condition diagnoses, based on admission ICD‐9 codes,14 into 25 broad diagnostic categories based on pathophysiologic plausibility and mortality rates. We tabulated inpatient admissions originating in the ED to derive a hospital volume measure.

Statistical Analyses

We compared patient characteristics, hospital volume, and outcomes by whether or not an unplanned ICU transfer occurred. Unadjusted analyses were performed with analysis of variance (ANOVA) and chi‐square tests. We calculated crude rates of unplanned ICU transfer per 1,000 ED inpatient admissions by patient characteristics and by hospital, stratified by hospital volume.

We used a hierarchical multivariate logistic regression model to estimate adjusted odds ratios for unplanned ICU transfer as a function of both patient‐level variables (age, sex, COPS, LAPS, time of admission, admission to TCU vs ward, admitting diagnosis) and hospital‐level variables (volume) in the model. We planned to choose the reference group for admitting diagnosis as the one with an unadjusted odds ratio closest to the null (1.00). This model addresses correlations between patients with multiple hospitalizations and clustering by hospital, by fitting random intercepts for these clusters. All analyses were performed in Stata 12 (StataCorp, College Station, TX), and statistics are presented with 95% confidence intervals (CI). The Stata program gllamm (Generalized Linear Latent and Mixed Models) was used for hierarchical modeling.18

RESULTS

Of 178,315 ED non‐ICU hospitalizations meeting inclusion criteria, 4,252 (2.4%) were admitted to the ward and were transferred to the ICU within 24 hours of leaving the ED. There were 122,251 unique patients in our study population. Table 1 compares the characteristics of ED hospitalizations in which an unplanned transfer occurred to those that did not experience an unplanned transfer. Unplanned transfers were more likely to have a higher comorbidity burden, more deranged physiology, and more likely to arrive on the floor during the overnight shift.

Patient Characteristics and Outcomes by Need for Unplanned ICU Transfer
CharacteristicsUnplanned Transfer to ICU Within 24 h of Leaving ED?P Value*
YesNo
N = 4,252 (2.4%)N = 174,063 (97.6%)
  • Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; ED, emergency department; ICU, intensive care unit; IQR, interquartile range; NS, not statistically significant; SD, standard deviation.

  • P value calculated by analysis of variance (ANOVA) or chi‐square tests; P value >0.05, not statistically significant.

  • With respect to a patient's preexisting comorbidity burden, the unadjusted relationship of COPS and mortality is as follows: a COPS <50 is associated with a mortality risk of <1%, <100 with a mortality risk of <5%, and >145 with a mortality risk of 10% or more. See Escobar et al14 for additional details.

  • With respect to a patient's physiologic derangement, the unadjusted relationship of LAPS and mortality is as follows: a LAPS <7 is associated with a mortality risk of <1%, <30 with a mortality risk of <5%, and >60 with a mortality risk of 10% or more. See Escobar et al14 for additional details.

  • Includes aortic dissection, ruptured abdominal aortic aneurysm, all forms of shock except septic shock, and intracranial hemorrhage.

Age, median (IQR)69 (5680)70 (5681)<0.01
Male, %51.345.9<0.01
Comorbidity Points Score (COPS), median (IQR)100 (46158)89 (42144)<0.01
Laboratory Acute Physiology Score (LAPS), median (IQR)26 (1342)18 (633)<0.01
Nursing shift on arrival to floor, %
Day: 7 am3 pm (Reference)20.120.1NS
Evening: 3 pm11 pm47.650.2NS
Overnight: 11 pm7 am32.329.7<0.01
Weekend admission, %33.732.7NS
Admitted to monitored bed, %24.124.9NS
Emergency department annual volume, mean (SD)48,755 (15,379)50,570 (15,276)<0.01
Non‐ICU annual admission volume, mean (SD)5,562 (1,626)5,774 (1,568)<0.01
Admitting diagnosis, listed by descending frequency, %  NS
Pneumonia and respiratory infections16.311.8<0.01
Gastrointestinal bleeding12.813.6NS
Chest pain7.310.0<0.01
Miscellaneous conditions5.66.2NS
All other acute infections4.76.0<0.01
Seizures4.15.9<0.01
AMI3.93.3<0.05
COPD3.83.0<0.01
CHF3.53.7NS
Arrhythmias and pulmonary embolism3.53.3NS
Stroke3.43.5NS
Diabetic emergencies3.32.6<0.01
Metabolic, endocrine, electrolytes3.02.9NS
Sepsis3.01.2<0.01
Other neurology and toxicology3.02.9NS
Urinary tract infections2.93.2NS
Catastrophic conditions2.61.2<0.01
Rheumatology2.53.5<0.01
Hematology and oncology2.42.4NS
Acute renal failure1.91.1<0.01
Pancreatic and liver1.72.0NS
Trauma, fractures, and dislocations1.61.8NS
Bowel obstructions and diseases1.62.9<0.01
Other cardiac conditions1.51.3NS
Other renal conditions0.61.0<0.01
Inpatient length of stay, median days (IQR)4.7 (2.78.6)2.6 (1.54.4)<0.01
Died during hospitalization, %12.72.4<0.01

Unplanned ICU transfers were more frequent in lower volume hospitals (Table 1). Figure 1 displays the inverse relationship between hospital annual ED inpatient admission volume and unplanned ICU transfers rates. The lowest volume hospital had a crude rate twice as high as the 2 highest volume hospitals (39 vs 20, per 1,000 admissions).

Figure 1
Relationship between hospital volume and rate of unplanned ICU transfers within 24 hours. Abbreviations: ED, emergency department; ICU, intensive care unit. (Error bars represent 95% confidence intervals).

Pneumonia/respiratory infection was the most frequent admitting condition associated with unplanned transfer (16.3%) (Table 1). There was also wide variation in crude rates for unplanned ICU transfer by admitting condition (Figure 2). Patients admitted with sepsis had the highest rate (59 per 1,000 admissions), while patients admitted with renal conditions other than acute renal failure had the lowest rates (14.3 per 1,000 admissions).

Figure 2
Association between patient characteristics, hospital volume, and risk of unplanned ICU transfer within 24 hours in a hierarchical logistic regression model. Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; CI, confidence interval; COPD, chronic obstructive pulmonary disease; ED, emergency department; ICU, intensive care unit. (Error bars represent 95% confidence intervals).

We confirmed that almost all diagnoses found to account for a disproportionately high share of unplanned ICU transfers in Table 1 were indeed independently associated with this phenomenon after adjustment for patient and hospital differences (Figure 2). Pneumonia remained the most frequent condition associated with unplanned ICU transfer (odds ratio [OR] 1.50; 95% CI 1.201.86). Although less frequent, sepsis had the strongest association of any condition with unplanned transfer (OR 2.51; 95% CI 1.903.31). However, metabolic, endocrine, and electrolyte conditions were no longer associated with unplanned transfer after adjustment, while arrhythmias and pulmonary embolism were. Other conditions confirmed to be associated with increased risk of unplanned transfer included: myocardial infarction (MI), chronic obstructive pulmonary disease (COPD), stroke, diabetic emergencies, catastrophic conditions (includes aortic catastrophes, all forms of shock except septic shock, and intracranial hemorrhage), and acute renal failure. After taking into account the frequency of admitting diagnoses, respiratory conditions (COPD, pneumonia/acute respiratory infection) comprised nearly half (47%) of all conditions associated with increased risk of unplanned ICU transfer.

Other factors confirmed to be independently associated with unplanned ICU transfer included: male sex (OR 1.20; 95% CI 1.131.28), high comorbidity burden as measured by COPS >145 (OR 1.13; 95% CI 1.031.24), increasingly abnormal physiology compared to a LAPS <7, and arrival on ward during the overnight shift (OR 1.10; 95% CI 1.011.21). After adjustment, we did find that admission to the TCU rather than a medicalsurgical unit was associated with decreased risk of unplanned ICU transfer (OR 0.83; 95% CI 0.770.90). Age 85 was associated with decreased risk of unplanned ICU transfer relative to the youngest age group of 1834‐year‐old patients (OR 0.64; 95% CI 0.530.77).

ED admissions to higher volume hospitals were 6% less likely to experience an unplanned transfer for each additional 1,000 annual ED hospitalizations over a lower volume hospital (OR 0.94; 95% CI 0.910.98). In other words, a patient admitted to a hospital with 8,000 annual ED hospitalizations had 30% decreased odds of unplanned ICU transfer compared to a hospital with only 3,000 annual ED hospitalizations.

DISCUSSION

Patients admitted with respiratory conditions accounted for half of all admitting diagnoses associated with increased risk of unplanned transfer to the ICU within 24 hours of arrival to the ward. We found that 1 in 30 ED ward admissions for pneumonia, and 1 in 33 for COPD, were transferred to the ICU within 24 hours. These findings indicate that there is some room for improvement in early care of respiratory conditions, given the average unplanned transfer rate of 1 in 42, and previous research showing that patients with pneumonia and patients with COPD, who experience unplanned ICU transfer, have substantially worse mortality than those directly admitted to the ICU.1

Although less frequent than hospitalizations for respiratory conditions, patients admitted with sepsis were at the highest risk of unplanned ICU transfer (1 in 17 ED non‐ICU hospitalizations). We also found that MI and stroke ward admissions had a higher risk of unplanned ICU transfer. However, we previously found that unplanned ICU transfers for sepsis, MI, and stroke did not have worse mortality than direct ICU admits for these conditions.1 Therefore, quality improvement efforts to reduce excess mortality related to early decompensation in the hospital and unplanned ICU transfer would be most effective if targeted towards respiratory conditions such as pneumonia and COPD.

This is the only in‐depth study, to our knowledge, to explore the association between a set of mutually exclusive diagnostic categories and risk of unplanned ICU transfer within 24 hours, and it is the first study to identify risk factors for unplanned ICU transfer in a multi‐hospital cohort adjusted for patient‐ and hospital‐level characteristics. We also identified a novel hospital volumeoutcome relationship: Unplanned ICU transfers are up to twice as likely to occur in the smallest volume hospitals compared with highest volume hospitals. Hospital volume has long been proposed as a proxy for hospital resources; there are several studies showing a relationship between low‐volume hospitals and worse outcomes for a number of conditions.19, 20 Possible mechanisms may include decreased ICU capacity, decreased on‐call intensivists in the hospital after hours, and less experience with certain critical care conditions seen more frequently in high‐volume hospitals.21

Patients at risk of unplanned ICU transfer were also more likely to have physiologic derangement identified on laboratory testing, high comorbidity burden, and arrive on the ward between 11 PM and 7 AM. Given the strong correlation between comorbidity burden and physiologic derangement and mortality,14 it is not surprising that the COPS and LAPS were independent predictors of unplanned transfer. It is unclear, however, why arriving on the ward on the overnight shift is associated with higher risk. One possibility is that patients who arrive on the wards during 11 PM to 7 AM are also likely to have been in the ED during evening peak hours most associated with ED crowding.22 High levels of ED crowding have been associated with delays in care, worse quality care, lapses in patient safety, and even increased in‐hospital mortality.22, 23 Other possible reasons include decreased in‐hospital staffing and longer delays in critical diagnostic tests and interventions.2428

Admission to TCUs was associated with decreased risk of unplanned ICU transfer in the first 24 hours of hospitalization. This may be due to the continuous monitoring, decreased nursing‐to‐patient ratios, or the availability to provide some critical care interventions. In our study, age 85 was associated with lower likelihood of unplanned transfer. Unfortunately, we did not have access to data on advanced directives or patient preferences. Data on advanced directives would help to distinguish whether this phenomenon was related to end‐of‐life care goals versus other explanations.

Our study confirms some risk factors identified in previous studies. These include specific diagnoses such as pneumonia and COPD,12, 13, 29 heavy comorbidity burden,12, 13, 29 abnormal labs,29 and male sex.13 Pneumonia has consistently been shown to be a risk factor for unplanned ICU transfer. This may stem from the dynamic nature of this condition and its ability to rapidly progress, and the fact that some ICUs may not accept pneumonia patients unless they demonstrate a need for mechanical ventilation.30 Recently, a prediction rule has been developed to determine which patients with pneumonia are likely to have an unplanned ICU transfer.30 It is possible that with validation and application of this rule, unplanned transfer rates for pneumonia could be reduced. It is unclear whether males have unmeasured factors associated with increased risk of unplanned transfer or whether a true gender disparity exists.

Our findings should be interpreted within the context of this study's limitations. First, this study was not designed to distinguish the underlying cause of the unplanned transfer such as under‐recognition of illness severity in the ED, evolving clinical disease after leaving the ED, or delays in critical interventions on the ward. These are a focus of our ongoing research efforts. Second, while previous studies have demonstrated that our automated risk adjustment variables can accurately predict in‐hospital mortality (0.88 area under curve in external populations),17 additional data on vital signs and mental status could further improve risk adjustment. However, using automated data allowed us to study risk factors for unplanned transfer in a multi‐hospital cohort with a much larger population than has been previously studied. Serial data on vital signs and mental status both in the ED and during hospitalization could also be helpful in determining which unplanned transfers could be prevented with earlier recognition and intervention. Finally, all patient care occurred within an integrated healthcare delivery system. Thus, differences in case‐mix, hospital resources, ICU structure, and geographic location should be considered when applying our results to other healthcare systems.

This study raises several new areas for future research. With access to richer data becoming available in electronic medical records, prediction rules should be developed to enable better triage to appropriate levels of care for ED admissions. Future research should also analyze the comparative effectiveness of intermediate monitored units versus non‐monitored wards for preventing clinical deterioration by admitting diagnosis. Diagnoses that have been shown to have an increased risk of death after unplanned ICU transfer, such as pneumonia/respiratory infection and COPD,1 should be prioritized in this research. Better understanding is needed on the diagnosis‐specific differences and the differences in ED triage process and ICU structure that may explain why high‐volume hospitals have significantly lower rates of early unplanned ICU transfers compared with low‐volume hospitals. In particular, determining the effect of TCU and ICU capacities and census at the time of admission, and comparing patient risk characteristics across hospital‐volume strata would be very useful. Finally, more work is needed to determine whether the higher rate of unplanned transfers during overnight nursing shifts is related to decreased resource availability, preceding ED crowding, or other organizational causes.

In conclusion, patients admitted with respiratory conditions, sepsis, MI, high comorbidity, and abnormal labs are at modestly increased risk of unplanned ICU transfer within 24 hours of admission from the ED. Patients admitted with respiratory conditions (pneumonia/respiratory infections and COPD) accounted for half of the admitting diagnoses that are at increased risk for unplanned ICU transfer. These patients may benefit from better inpatient triage from the ED, earlier intervention, or closer monitoring. More research is needed to determine the specific aspects of care associated with admission to intermediate care units and high‐volume hospitals that reduce the risk of unplanned ICU transfer.

Acknowledgements

The authors thank John D. Greene, Juan Carlos La Guardia, and Benjamin Turk for their assistance with formatting of the dataset; Dr Alan S. Go, Acting Director of the Division of Research, for reviewing the manuscript; and Alina Schnake‐Mahl for formatting the manuscript.

References
  1. Liu V, Kipnis P, Rizk NW, et al. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2011;7(3):224230.
  2. Young MP, Gooder VJ, Bride K, et al. Inpatient transfers to the intensive care unit. J Gen Intern Med. 2003;18(2):7783.
  3. Escobar GJ, Greene JD, Gardner MN, et al. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6:7480.
  4. Chan PS, Khalid A, Longmore LS, et al. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300(21):25062513.
  5. Sharek PJ, Parast LM, Leong K, et al. Effect of a rapid response team on hospital‐wide mortality and code rates outside the ICU in a children's hospital. JAMA. 2007;298(19):22672274.
  6. Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):20912097.
  7. Winters BD, Pham JC, Hunt EA, et al. Rapid response systems: A systematic review. Crit Care Med. 2007;35(5):12381243.
  8. Ranji SR, Auerbach AD, Hurd CJ, et al. Effects of rapid response systems on clinical outcomes: systematic review and meta‐analysis. J Hosp Med. 2007;2(6):422432.
  9. Chan PS, Jain R, Nallmothu BK, et al. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):1826.
  10. McGaughey J, Alderdice F, Fowler R, et al. Outreach and early warning systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards. Cochrane Database Syst Rev. 2007;3:CD005529.
  11. Bapoje SR, Gaudiani JL, Narayanan V, et al. Unplanned transfers to a medical intensive care unit: Causes and relationship to preventable errors in care. J Hosp Med. 2011;6:6872.
  12. Tam V, Frost SA, Hillman KM, et al. Using administrative data to develop a nomogram for individualising risk of unplanned admission to intensive care. Resuscitation. 2008;79(2):241248.
  13. Frost SA, Alexandrou E, Bogdanovski T, et al. Unplanned admission to intensive care after emergency hospitalisation: risk factors and development of a nomogram for individualising risk. Resuscitation. 2009;80(2):224230.
  14. Escobar GJ, Greene JD, Scheirer P, et al. Risk‐adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232239.
  15. Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719724.
  16. Escobar GJ, Fireman BH, Palen TE, et al. Risk adjusting community‐acquired pneumonia hospital outcomes using automated databases. Am J Manag Care. 2008;14(3):158166.
  17. van Walraven C, Escobar GJ, Greene JD, et al. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2011;63(7):798803.
  18. Rabe‐Hesketh S, Skrondal A, Pickles A. Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects. J Econometrics. 2005;128(2):301323.
  19. Hannan EL. The relation between volume and outcome in health care. N Engl J Med. 1999;340(21):16771679.
  20. Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med. 2002;137(6):511520.
  21. Terwiesch C, Diwas K, Kahn JM. Working with capacity limitations: operations management in critical care. Crit Care. 2011;15(4):308.
  22. Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Intern Med. 2008;52(2):126136.
  23. Bernstein SL, Aronsky D, Duseja R, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med. 2009;16(1):110.
  24. Cavallazzi R, Marik PE, Hirani A, et al. Association between time of admission to the ICU and mortality. Chest. 2010;138(1):6875.
  25. Reeves MJ, Smith E, Fonarow G, et al. Off‐hour admission and in‐hospital stroke case fatality in the get with the guidelines‐stroke program. Stroke. 2009;40(2):569576.
  26. Magid DJ, Wang Y, Herrin J, et al. Relationship between time of day, day of week, timeliness of reperfusion, and in‐hospital mortality for patients with acute ST‐segment elevation myocardial infarction. JAMA. 2005;294(7):803812.
  27. Laupland KB, Shahpori R, Kirkpatrick AW, et al. Hospital mortality among adults admitted to and discharged from intensive care on weekends and evenings. J Crit Care. 2008;23(3):317324.
  28. Afessa B, Gajic O, Morales IJ, et al. Association between ICU admission during morning rounds and mortality. Chest. 2009;136(6):14891495.
  29. Kennedy M, Joyce N, Howell MD, et al. Identifying infected emergency department patients admitted to the hospital ward at risk of clinical deterioration and intensive care unit transfer. Acad Emerg Med. 2010;17(10):10801085.
  30. Renaud B, Labarère J, Coma E, et al. Risk stratification of early admission to the intensive care unit of patients with no major criteria of severe community‐acquired pneumonia: development of an international prediction rule. Crit Care. 2009;13(2):R54.
Article PDF
Issue
Journal of Hospital Medicine - 8(1)
Publications
Page Number
13-19
Sections
Article PDF
Article PDF

Emergency Department (ED) patients who are hospitalized and require unplanned transfer to the intensive care unit (ICU) within 24 hours of arrival on the ward have higher mortality than direct ICU admissions.1, 2 Previous research found that 5% of ED admissions experienced unplanned ICU transfer during their hospitalization, yet these patients account for 25% of in‐hospital deaths and have a longer length of stay than direct ICU admissions.1, 3 For these reasons, inpatient rapid‐response teams and early warning systems have been studied to reduce the mortality of patients who rapidly deteriorate on the hospital ward.410 However, there is little conclusive evidence that these interventions decrease mortality.710 It is possible that with better recognition and intervention in the ED, a portion of these unplanned ICU transfers and their subsequent adverse outcomes could be prevented.11

Previous research on risk factors for unplanned ICU transfers among ED admissions is limited. While 2 previous studies from non‐US hospitals used administrative data to identify some general populations at risk for unplanned ICU transfer,12, 13 these studies did not differentiate between transfers shortly after admission and those that occurred during a prolonged hospital staya critical distinction since the outcomes between these groups differs substantially.1 Another limitation of these studies is the absence of physiologic measures at ED presentation, which have been shown to be highly predictive of mortality.14

In this study, we describe risk factors for unplanned transfer to the ICU within 24 hours of arrival on the ward, among a large cohort of ED hospitalizations across 13 community hospitals. Focusing on admitting diagnoses most at risk, our goal was to inform efforts to improve the triage of ED admissions and determine which patients may benefit from additional interventions, such as improved resuscitation, closer monitoring, or risk stratification tools. We also hypothesized that higher volume hospitals would have lower rates of unplanned ICU transfers, as these hospitals are more likely have more patient care resources on the hospital ward and a higher threshold to transfer to the ICU.

METHODS

Setting and Patients

The setting for this study was Kaiser Permanente Northern California (KPNC), a large integrated healthcare delivery system serving approximately 3.3 million members.1, 3, 15, 16 We extracted data on all adult ED admissions (18 years old) to the hospital between 2007 and 2009. We excluded patients who went directly to the operating room or the ICU, as well as gynecological/pregnancy‐related admissions, as these patients have substantially different mortality risks.14 ED admissions to hospital wards could either go to medicalsurgical units or transitional care units (TCU), an intermediate level of care between the medicalsurgical units and the ICU. We chose to focus on hospitals with similar inpatient structures. Thus, 8 hospitals without TCUs were excluded, leaving 13 hospitals for analysis. The KPNC Institutional Review Board approved this study.

Main Outcome Measure

The main outcome measure was unplanned transfer to the ICU within 24 hours of arrival to the hospital ward, based upon bed history data. As in previous research, we make the assumptionwhich is supported by the high observed‐to‐expected mortality ratios found in these patientsthat these transfers to the ICU were due to clinical deterioration, and thus were unplanned, rather than a planned transfer to the ICU as is more common after an elective surgical procedure.13 The comparison population was patients admitted from the ED to the ward who never experienced a transfer to the ICU.

Patient and Hospital Characteristics

We extracted patient data on age, sex, admitting diagnosis, chronic illness burden, acute physiologic derangement in the ED, and hospital unit length of stay. Chronic illness was measured using the Comorbidity Point Score (COPS), and physiologic derangement was measured using the Laboratory Acute Physiology Score (LAPS) calculated from labs collected in the ED.1, 14, 17 The derivation of these variables from the electronic medical record has been previously described.14 The COPS was derived from International Classification of Diseases, Ninth Revision (ICD‐9) codes for all Kaiser Permanente Medical Care Program (KPMCP) inpatient and outpatient encounters prior to hospitalization. The LAPS is based on 14 possible lab tests that could be drawn in the ED or in the 72 hours prior to hospitalization. The admitting diagnosis is the ICD‐9 code assigned for the primary diagnosis determined by the admitting physician at the time when hospital admission orders are entered. We further collapsed a previously used categorization of 44 primary condition diagnoses, based on admission ICD‐9 codes,14 into 25 broad diagnostic categories based on pathophysiologic plausibility and mortality rates. We tabulated inpatient admissions originating in the ED to derive a hospital volume measure.

Statistical Analyses

We compared patient characteristics, hospital volume, and outcomes by whether or not an unplanned ICU transfer occurred. Unadjusted analyses were performed with analysis of variance (ANOVA) and chi‐square tests. We calculated crude rates of unplanned ICU transfer per 1,000 ED inpatient admissions by patient characteristics and by hospital, stratified by hospital volume.

We used a hierarchical multivariate logistic regression model to estimate adjusted odds ratios for unplanned ICU transfer as a function of both patient‐level variables (age, sex, COPS, LAPS, time of admission, admission to TCU vs ward, admitting diagnosis) and hospital‐level variables (volume) in the model. We planned to choose the reference group for admitting diagnosis as the one with an unadjusted odds ratio closest to the null (1.00). This model addresses correlations between patients with multiple hospitalizations and clustering by hospital, by fitting random intercepts for these clusters. All analyses were performed in Stata 12 (StataCorp, College Station, TX), and statistics are presented with 95% confidence intervals (CI). The Stata program gllamm (Generalized Linear Latent and Mixed Models) was used for hierarchical modeling.18

RESULTS

Of 178,315 ED non‐ICU hospitalizations meeting inclusion criteria, 4,252 (2.4%) were admitted to the ward and were transferred to the ICU within 24 hours of leaving the ED. There were 122,251 unique patients in our study population. Table 1 compares the characteristics of ED hospitalizations in which an unplanned transfer occurred to those that did not experience an unplanned transfer. Unplanned transfers were more likely to have a higher comorbidity burden, more deranged physiology, and more likely to arrive on the floor during the overnight shift.

Patient Characteristics and Outcomes by Need for Unplanned ICU Transfer
CharacteristicsUnplanned Transfer to ICU Within 24 h of Leaving ED?P Value*
YesNo
N = 4,252 (2.4%)N = 174,063 (97.6%)
  • Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; ED, emergency department; ICU, intensive care unit; IQR, interquartile range; NS, not statistically significant; SD, standard deviation.

  • P value calculated by analysis of variance (ANOVA) or chi‐square tests; P value >0.05, not statistically significant.

  • With respect to a patient's preexisting comorbidity burden, the unadjusted relationship of COPS and mortality is as follows: a COPS <50 is associated with a mortality risk of <1%, <100 with a mortality risk of <5%, and >145 with a mortality risk of 10% or more. See Escobar et al14 for additional details.

  • With respect to a patient's physiologic derangement, the unadjusted relationship of LAPS and mortality is as follows: a LAPS <7 is associated with a mortality risk of <1%, <30 with a mortality risk of <5%, and >60 with a mortality risk of 10% or more. See Escobar et al14 for additional details.

  • Includes aortic dissection, ruptured abdominal aortic aneurysm, all forms of shock except septic shock, and intracranial hemorrhage.

Age, median (IQR)69 (5680)70 (5681)<0.01
Male, %51.345.9<0.01
Comorbidity Points Score (COPS), median (IQR)100 (46158)89 (42144)<0.01
Laboratory Acute Physiology Score (LAPS), median (IQR)26 (1342)18 (633)<0.01
Nursing shift on arrival to floor, %
Day: 7 am3 pm (Reference)20.120.1NS
Evening: 3 pm11 pm47.650.2NS
Overnight: 11 pm7 am32.329.7<0.01
Weekend admission, %33.732.7NS
Admitted to monitored bed, %24.124.9NS
Emergency department annual volume, mean (SD)48,755 (15,379)50,570 (15,276)<0.01
Non‐ICU annual admission volume, mean (SD)5,562 (1,626)5,774 (1,568)<0.01
Admitting diagnosis, listed by descending frequency, %  NS
Pneumonia and respiratory infections16.311.8<0.01
Gastrointestinal bleeding12.813.6NS
Chest pain7.310.0<0.01
Miscellaneous conditions5.66.2NS
All other acute infections4.76.0<0.01
Seizures4.15.9<0.01
AMI3.93.3<0.05
COPD3.83.0<0.01
CHF3.53.7NS
Arrhythmias and pulmonary embolism3.53.3NS
Stroke3.43.5NS
Diabetic emergencies3.32.6<0.01
Metabolic, endocrine, electrolytes3.02.9NS
Sepsis3.01.2<0.01
Other neurology and toxicology3.02.9NS
Urinary tract infections2.93.2NS
Catastrophic conditions2.61.2<0.01
Rheumatology2.53.5<0.01
Hematology and oncology2.42.4NS
Acute renal failure1.91.1<0.01
Pancreatic and liver1.72.0NS
Trauma, fractures, and dislocations1.61.8NS
Bowel obstructions and diseases1.62.9<0.01
Other cardiac conditions1.51.3NS
Other renal conditions0.61.0<0.01
Inpatient length of stay, median days (IQR)4.7 (2.78.6)2.6 (1.54.4)<0.01
Died during hospitalization, %12.72.4<0.01

Unplanned ICU transfers were more frequent in lower volume hospitals (Table 1). Figure 1 displays the inverse relationship between hospital annual ED inpatient admission volume and unplanned ICU transfers rates. The lowest volume hospital had a crude rate twice as high as the 2 highest volume hospitals (39 vs 20, per 1,000 admissions).

Figure 1
Relationship between hospital volume and rate of unplanned ICU transfers within 24 hours. Abbreviations: ED, emergency department; ICU, intensive care unit. (Error bars represent 95% confidence intervals).

Pneumonia/respiratory infection was the most frequent admitting condition associated with unplanned transfer (16.3%) (Table 1). There was also wide variation in crude rates for unplanned ICU transfer by admitting condition (Figure 2). Patients admitted with sepsis had the highest rate (59 per 1,000 admissions), while patients admitted with renal conditions other than acute renal failure had the lowest rates (14.3 per 1,000 admissions).

Figure 2
Association between patient characteristics, hospital volume, and risk of unplanned ICU transfer within 24 hours in a hierarchical logistic regression model. Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; CI, confidence interval; COPD, chronic obstructive pulmonary disease; ED, emergency department; ICU, intensive care unit. (Error bars represent 95% confidence intervals).

We confirmed that almost all diagnoses found to account for a disproportionately high share of unplanned ICU transfers in Table 1 were indeed independently associated with this phenomenon after adjustment for patient and hospital differences (Figure 2). Pneumonia remained the most frequent condition associated with unplanned ICU transfer (odds ratio [OR] 1.50; 95% CI 1.201.86). Although less frequent, sepsis had the strongest association of any condition with unplanned transfer (OR 2.51; 95% CI 1.903.31). However, metabolic, endocrine, and electrolyte conditions were no longer associated with unplanned transfer after adjustment, while arrhythmias and pulmonary embolism were. Other conditions confirmed to be associated with increased risk of unplanned transfer included: myocardial infarction (MI), chronic obstructive pulmonary disease (COPD), stroke, diabetic emergencies, catastrophic conditions (includes aortic catastrophes, all forms of shock except septic shock, and intracranial hemorrhage), and acute renal failure. After taking into account the frequency of admitting diagnoses, respiratory conditions (COPD, pneumonia/acute respiratory infection) comprised nearly half (47%) of all conditions associated with increased risk of unplanned ICU transfer.

Other factors confirmed to be independently associated with unplanned ICU transfer included: male sex (OR 1.20; 95% CI 1.131.28), high comorbidity burden as measured by COPS >145 (OR 1.13; 95% CI 1.031.24), increasingly abnormal physiology compared to a LAPS <7, and arrival on ward during the overnight shift (OR 1.10; 95% CI 1.011.21). After adjustment, we did find that admission to the TCU rather than a medicalsurgical unit was associated with decreased risk of unplanned ICU transfer (OR 0.83; 95% CI 0.770.90). Age 85 was associated with decreased risk of unplanned ICU transfer relative to the youngest age group of 1834‐year‐old patients (OR 0.64; 95% CI 0.530.77).

ED admissions to higher volume hospitals were 6% less likely to experience an unplanned transfer for each additional 1,000 annual ED hospitalizations over a lower volume hospital (OR 0.94; 95% CI 0.910.98). In other words, a patient admitted to a hospital with 8,000 annual ED hospitalizations had 30% decreased odds of unplanned ICU transfer compared to a hospital with only 3,000 annual ED hospitalizations.

DISCUSSION

Patients admitted with respiratory conditions accounted for half of all admitting diagnoses associated with increased risk of unplanned transfer to the ICU within 24 hours of arrival to the ward. We found that 1 in 30 ED ward admissions for pneumonia, and 1 in 33 for COPD, were transferred to the ICU within 24 hours. These findings indicate that there is some room for improvement in early care of respiratory conditions, given the average unplanned transfer rate of 1 in 42, and previous research showing that patients with pneumonia and patients with COPD, who experience unplanned ICU transfer, have substantially worse mortality than those directly admitted to the ICU.1

Although less frequent than hospitalizations for respiratory conditions, patients admitted with sepsis were at the highest risk of unplanned ICU transfer (1 in 17 ED non‐ICU hospitalizations). We also found that MI and stroke ward admissions had a higher risk of unplanned ICU transfer. However, we previously found that unplanned ICU transfers for sepsis, MI, and stroke did not have worse mortality than direct ICU admits for these conditions.1 Therefore, quality improvement efforts to reduce excess mortality related to early decompensation in the hospital and unplanned ICU transfer would be most effective if targeted towards respiratory conditions such as pneumonia and COPD.

This is the only in‐depth study, to our knowledge, to explore the association between a set of mutually exclusive diagnostic categories and risk of unplanned ICU transfer within 24 hours, and it is the first study to identify risk factors for unplanned ICU transfer in a multi‐hospital cohort adjusted for patient‐ and hospital‐level characteristics. We also identified a novel hospital volumeoutcome relationship: Unplanned ICU transfers are up to twice as likely to occur in the smallest volume hospitals compared with highest volume hospitals. Hospital volume has long been proposed as a proxy for hospital resources; there are several studies showing a relationship between low‐volume hospitals and worse outcomes for a number of conditions.19, 20 Possible mechanisms may include decreased ICU capacity, decreased on‐call intensivists in the hospital after hours, and less experience with certain critical care conditions seen more frequently in high‐volume hospitals.21

Patients at risk of unplanned ICU transfer were also more likely to have physiologic derangement identified on laboratory testing, high comorbidity burden, and arrive on the ward between 11 PM and 7 AM. Given the strong correlation between comorbidity burden and physiologic derangement and mortality,14 it is not surprising that the COPS and LAPS were independent predictors of unplanned transfer. It is unclear, however, why arriving on the ward on the overnight shift is associated with higher risk. One possibility is that patients who arrive on the wards during 11 PM to 7 AM are also likely to have been in the ED during evening peak hours most associated with ED crowding.22 High levels of ED crowding have been associated with delays in care, worse quality care, lapses in patient safety, and even increased in‐hospital mortality.22, 23 Other possible reasons include decreased in‐hospital staffing and longer delays in critical diagnostic tests and interventions.2428

Admission to TCUs was associated with decreased risk of unplanned ICU transfer in the first 24 hours of hospitalization. This may be due to the continuous monitoring, decreased nursing‐to‐patient ratios, or the availability to provide some critical care interventions. In our study, age 85 was associated with lower likelihood of unplanned transfer. Unfortunately, we did not have access to data on advanced directives or patient preferences. Data on advanced directives would help to distinguish whether this phenomenon was related to end‐of‐life care goals versus other explanations.

Our study confirms some risk factors identified in previous studies. These include specific diagnoses such as pneumonia and COPD,12, 13, 29 heavy comorbidity burden,12, 13, 29 abnormal labs,29 and male sex.13 Pneumonia has consistently been shown to be a risk factor for unplanned ICU transfer. This may stem from the dynamic nature of this condition and its ability to rapidly progress, and the fact that some ICUs may not accept pneumonia patients unless they demonstrate a need for mechanical ventilation.30 Recently, a prediction rule has been developed to determine which patients with pneumonia are likely to have an unplanned ICU transfer.30 It is possible that with validation and application of this rule, unplanned transfer rates for pneumonia could be reduced. It is unclear whether males have unmeasured factors associated with increased risk of unplanned transfer or whether a true gender disparity exists.

Our findings should be interpreted within the context of this study's limitations. First, this study was not designed to distinguish the underlying cause of the unplanned transfer such as under‐recognition of illness severity in the ED, evolving clinical disease after leaving the ED, or delays in critical interventions on the ward. These are a focus of our ongoing research efforts. Second, while previous studies have demonstrated that our automated risk adjustment variables can accurately predict in‐hospital mortality (0.88 area under curve in external populations),17 additional data on vital signs and mental status could further improve risk adjustment. However, using automated data allowed us to study risk factors for unplanned transfer in a multi‐hospital cohort with a much larger population than has been previously studied. Serial data on vital signs and mental status both in the ED and during hospitalization could also be helpful in determining which unplanned transfers could be prevented with earlier recognition and intervention. Finally, all patient care occurred within an integrated healthcare delivery system. Thus, differences in case‐mix, hospital resources, ICU structure, and geographic location should be considered when applying our results to other healthcare systems.

This study raises several new areas for future research. With access to richer data becoming available in electronic medical records, prediction rules should be developed to enable better triage to appropriate levels of care for ED admissions. Future research should also analyze the comparative effectiveness of intermediate monitored units versus non‐monitored wards for preventing clinical deterioration by admitting diagnosis. Diagnoses that have been shown to have an increased risk of death after unplanned ICU transfer, such as pneumonia/respiratory infection and COPD,1 should be prioritized in this research. Better understanding is needed on the diagnosis‐specific differences and the differences in ED triage process and ICU structure that may explain why high‐volume hospitals have significantly lower rates of early unplanned ICU transfers compared with low‐volume hospitals. In particular, determining the effect of TCU and ICU capacities and census at the time of admission, and comparing patient risk characteristics across hospital‐volume strata would be very useful. Finally, more work is needed to determine whether the higher rate of unplanned transfers during overnight nursing shifts is related to decreased resource availability, preceding ED crowding, or other organizational causes.

In conclusion, patients admitted with respiratory conditions, sepsis, MI, high comorbidity, and abnormal labs are at modestly increased risk of unplanned ICU transfer within 24 hours of admission from the ED. Patients admitted with respiratory conditions (pneumonia/respiratory infections and COPD) accounted for half of the admitting diagnoses that are at increased risk for unplanned ICU transfer. These patients may benefit from better inpatient triage from the ED, earlier intervention, or closer monitoring. More research is needed to determine the specific aspects of care associated with admission to intermediate care units and high‐volume hospitals that reduce the risk of unplanned ICU transfer.

Acknowledgements

The authors thank John D. Greene, Juan Carlos La Guardia, and Benjamin Turk for their assistance with formatting of the dataset; Dr Alan S. Go, Acting Director of the Division of Research, for reviewing the manuscript; and Alina Schnake‐Mahl for formatting the manuscript.

Emergency Department (ED) patients who are hospitalized and require unplanned transfer to the intensive care unit (ICU) within 24 hours of arrival on the ward have higher mortality than direct ICU admissions.1, 2 Previous research found that 5% of ED admissions experienced unplanned ICU transfer during their hospitalization, yet these patients account for 25% of in‐hospital deaths and have a longer length of stay than direct ICU admissions.1, 3 For these reasons, inpatient rapid‐response teams and early warning systems have been studied to reduce the mortality of patients who rapidly deteriorate on the hospital ward.410 However, there is little conclusive evidence that these interventions decrease mortality.710 It is possible that with better recognition and intervention in the ED, a portion of these unplanned ICU transfers and their subsequent adverse outcomes could be prevented.11

Previous research on risk factors for unplanned ICU transfers among ED admissions is limited. While 2 previous studies from non‐US hospitals used administrative data to identify some general populations at risk for unplanned ICU transfer,12, 13 these studies did not differentiate between transfers shortly after admission and those that occurred during a prolonged hospital staya critical distinction since the outcomes between these groups differs substantially.1 Another limitation of these studies is the absence of physiologic measures at ED presentation, which have been shown to be highly predictive of mortality.14

In this study, we describe risk factors for unplanned transfer to the ICU within 24 hours of arrival on the ward, among a large cohort of ED hospitalizations across 13 community hospitals. Focusing on admitting diagnoses most at risk, our goal was to inform efforts to improve the triage of ED admissions and determine which patients may benefit from additional interventions, such as improved resuscitation, closer monitoring, or risk stratification tools. We also hypothesized that higher volume hospitals would have lower rates of unplanned ICU transfers, as these hospitals are more likely have more patient care resources on the hospital ward and a higher threshold to transfer to the ICU.

METHODS

Setting and Patients

The setting for this study was Kaiser Permanente Northern California (KPNC), a large integrated healthcare delivery system serving approximately 3.3 million members.1, 3, 15, 16 We extracted data on all adult ED admissions (18 years old) to the hospital between 2007 and 2009. We excluded patients who went directly to the operating room or the ICU, as well as gynecological/pregnancy‐related admissions, as these patients have substantially different mortality risks.14 ED admissions to hospital wards could either go to medicalsurgical units or transitional care units (TCU), an intermediate level of care between the medicalsurgical units and the ICU. We chose to focus on hospitals with similar inpatient structures. Thus, 8 hospitals without TCUs were excluded, leaving 13 hospitals for analysis. The KPNC Institutional Review Board approved this study.

Main Outcome Measure

The main outcome measure was unplanned transfer to the ICU within 24 hours of arrival to the hospital ward, based upon bed history data. As in previous research, we make the assumptionwhich is supported by the high observed‐to‐expected mortality ratios found in these patientsthat these transfers to the ICU were due to clinical deterioration, and thus were unplanned, rather than a planned transfer to the ICU as is more common after an elective surgical procedure.13 The comparison population was patients admitted from the ED to the ward who never experienced a transfer to the ICU.

Patient and Hospital Characteristics

We extracted patient data on age, sex, admitting diagnosis, chronic illness burden, acute physiologic derangement in the ED, and hospital unit length of stay. Chronic illness was measured using the Comorbidity Point Score (COPS), and physiologic derangement was measured using the Laboratory Acute Physiology Score (LAPS) calculated from labs collected in the ED.1, 14, 17 The derivation of these variables from the electronic medical record has been previously described.14 The COPS was derived from International Classification of Diseases, Ninth Revision (ICD‐9) codes for all Kaiser Permanente Medical Care Program (KPMCP) inpatient and outpatient encounters prior to hospitalization. The LAPS is based on 14 possible lab tests that could be drawn in the ED or in the 72 hours prior to hospitalization. The admitting diagnosis is the ICD‐9 code assigned for the primary diagnosis determined by the admitting physician at the time when hospital admission orders are entered. We further collapsed a previously used categorization of 44 primary condition diagnoses, based on admission ICD‐9 codes,14 into 25 broad diagnostic categories based on pathophysiologic plausibility and mortality rates. We tabulated inpatient admissions originating in the ED to derive a hospital volume measure.

Statistical Analyses

We compared patient characteristics, hospital volume, and outcomes by whether or not an unplanned ICU transfer occurred. Unadjusted analyses were performed with analysis of variance (ANOVA) and chi‐square tests. We calculated crude rates of unplanned ICU transfer per 1,000 ED inpatient admissions by patient characteristics and by hospital, stratified by hospital volume.

We used a hierarchical multivariate logistic regression model to estimate adjusted odds ratios for unplanned ICU transfer as a function of both patient‐level variables (age, sex, COPS, LAPS, time of admission, admission to TCU vs ward, admitting diagnosis) and hospital‐level variables (volume) in the model. We planned to choose the reference group for admitting diagnosis as the one with an unadjusted odds ratio closest to the null (1.00). This model addresses correlations between patients with multiple hospitalizations and clustering by hospital, by fitting random intercepts for these clusters. All analyses were performed in Stata 12 (StataCorp, College Station, TX), and statistics are presented with 95% confidence intervals (CI). The Stata program gllamm (Generalized Linear Latent and Mixed Models) was used for hierarchical modeling.18

RESULTS

Of 178,315 ED non‐ICU hospitalizations meeting inclusion criteria, 4,252 (2.4%) were admitted to the ward and were transferred to the ICU within 24 hours of leaving the ED. There were 122,251 unique patients in our study population. Table 1 compares the characteristics of ED hospitalizations in which an unplanned transfer occurred to those that did not experience an unplanned transfer. Unplanned transfers were more likely to have a higher comorbidity burden, more deranged physiology, and more likely to arrive on the floor during the overnight shift.

Patient Characteristics and Outcomes by Need for Unplanned ICU Transfer
CharacteristicsUnplanned Transfer to ICU Within 24 h of Leaving ED?P Value*
YesNo
N = 4,252 (2.4%)N = 174,063 (97.6%)
  • Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; ED, emergency department; ICU, intensive care unit; IQR, interquartile range; NS, not statistically significant; SD, standard deviation.

  • P value calculated by analysis of variance (ANOVA) or chi‐square tests; P value >0.05, not statistically significant.

  • With respect to a patient's preexisting comorbidity burden, the unadjusted relationship of COPS and mortality is as follows: a COPS <50 is associated with a mortality risk of <1%, <100 with a mortality risk of <5%, and >145 with a mortality risk of 10% or more. See Escobar et al14 for additional details.

  • With respect to a patient's physiologic derangement, the unadjusted relationship of LAPS and mortality is as follows: a LAPS <7 is associated with a mortality risk of <1%, <30 with a mortality risk of <5%, and >60 with a mortality risk of 10% or more. See Escobar et al14 for additional details.

  • Includes aortic dissection, ruptured abdominal aortic aneurysm, all forms of shock except septic shock, and intracranial hemorrhage.

Age, median (IQR)69 (5680)70 (5681)<0.01
Male, %51.345.9<0.01
Comorbidity Points Score (COPS), median (IQR)100 (46158)89 (42144)<0.01
Laboratory Acute Physiology Score (LAPS), median (IQR)26 (1342)18 (633)<0.01
Nursing shift on arrival to floor, %
Day: 7 am3 pm (Reference)20.120.1NS
Evening: 3 pm11 pm47.650.2NS
Overnight: 11 pm7 am32.329.7<0.01
Weekend admission, %33.732.7NS
Admitted to monitored bed, %24.124.9NS
Emergency department annual volume, mean (SD)48,755 (15,379)50,570 (15,276)<0.01
Non‐ICU annual admission volume, mean (SD)5,562 (1,626)5,774 (1,568)<0.01
Admitting diagnosis, listed by descending frequency, %  NS
Pneumonia and respiratory infections16.311.8<0.01
Gastrointestinal bleeding12.813.6NS
Chest pain7.310.0<0.01
Miscellaneous conditions5.66.2NS
All other acute infections4.76.0<0.01
Seizures4.15.9<0.01
AMI3.93.3<0.05
COPD3.83.0<0.01
CHF3.53.7NS
Arrhythmias and pulmonary embolism3.53.3NS
Stroke3.43.5NS
Diabetic emergencies3.32.6<0.01
Metabolic, endocrine, electrolytes3.02.9NS
Sepsis3.01.2<0.01
Other neurology and toxicology3.02.9NS
Urinary tract infections2.93.2NS
Catastrophic conditions2.61.2<0.01
Rheumatology2.53.5<0.01
Hematology and oncology2.42.4NS
Acute renal failure1.91.1<0.01
Pancreatic and liver1.72.0NS
Trauma, fractures, and dislocations1.61.8NS
Bowel obstructions and diseases1.62.9<0.01
Other cardiac conditions1.51.3NS
Other renal conditions0.61.0<0.01
Inpatient length of stay, median days (IQR)4.7 (2.78.6)2.6 (1.54.4)<0.01
Died during hospitalization, %12.72.4<0.01

Unplanned ICU transfers were more frequent in lower volume hospitals (Table 1). Figure 1 displays the inverse relationship between hospital annual ED inpatient admission volume and unplanned ICU transfers rates. The lowest volume hospital had a crude rate twice as high as the 2 highest volume hospitals (39 vs 20, per 1,000 admissions).

Figure 1
Relationship between hospital volume and rate of unplanned ICU transfers within 24 hours. Abbreviations: ED, emergency department; ICU, intensive care unit. (Error bars represent 95% confidence intervals).

Pneumonia/respiratory infection was the most frequent admitting condition associated with unplanned transfer (16.3%) (Table 1). There was also wide variation in crude rates for unplanned ICU transfer by admitting condition (Figure 2). Patients admitted with sepsis had the highest rate (59 per 1,000 admissions), while patients admitted with renal conditions other than acute renal failure had the lowest rates (14.3 per 1,000 admissions).

Figure 2
Association between patient characteristics, hospital volume, and risk of unplanned ICU transfer within 24 hours in a hierarchical logistic regression model. Abbreviations: AMI, acute myocardial infarction; CHF, congestive heart failure; CI, confidence interval; COPD, chronic obstructive pulmonary disease; ED, emergency department; ICU, intensive care unit. (Error bars represent 95% confidence intervals).

We confirmed that almost all diagnoses found to account for a disproportionately high share of unplanned ICU transfers in Table 1 were indeed independently associated with this phenomenon after adjustment for patient and hospital differences (Figure 2). Pneumonia remained the most frequent condition associated with unplanned ICU transfer (odds ratio [OR] 1.50; 95% CI 1.201.86). Although less frequent, sepsis had the strongest association of any condition with unplanned transfer (OR 2.51; 95% CI 1.903.31). However, metabolic, endocrine, and electrolyte conditions were no longer associated with unplanned transfer after adjustment, while arrhythmias and pulmonary embolism were. Other conditions confirmed to be associated with increased risk of unplanned transfer included: myocardial infarction (MI), chronic obstructive pulmonary disease (COPD), stroke, diabetic emergencies, catastrophic conditions (includes aortic catastrophes, all forms of shock except septic shock, and intracranial hemorrhage), and acute renal failure. After taking into account the frequency of admitting diagnoses, respiratory conditions (COPD, pneumonia/acute respiratory infection) comprised nearly half (47%) of all conditions associated with increased risk of unplanned ICU transfer.

Other factors confirmed to be independently associated with unplanned ICU transfer included: male sex (OR 1.20; 95% CI 1.131.28), high comorbidity burden as measured by COPS >145 (OR 1.13; 95% CI 1.031.24), increasingly abnormal physiology compared to a LAPS <7, and arrival on ward during the overnight shift (OR 1.10; 95% CI 1.011.21). After adjustment, we did find that admission to the TCU rather than a medicalsurgical unit was associated with decreased risk of unplanned ICU transfer (OR 0.83; 95% CI 0.770.90). Age 85 was associated with decreased risk of unplanned ICU transfer relative to the youngest age group of 1834‐year‐old patients (OR 0.64; 95% CI 0.530.77).

ED admissions to higher volume hospitals were 6% less likely to experience an unplanned transfer for each additional 1,000 annual ED hospitalizations over a lower volume hospital (OR 0.94; 95% CI 0.910.98). In other words, a patient admitted to a hospital with 8,000 annual ED hospitalizations had 30% decreased odds of unplanned ICU transfer compared to a hospital with only 3,000 annual ED hospitalizations.

DISCUSSION

Patients admitted with respiratory conditions accounted for half of all admitting diagnoses associated with increased risk of unplanned transfer to the ICU within 24 hours of arrival to the ward. We found that 1 in 30 ED ward admissions for pneumonia, and 1 in 33 for COPD, were transferred to the ICU within 24 hours. These findings indicate that there is some room for improvement in early care of respiratory conditions, given the average unplanned transfer rate of 1 in 42, and previous research showing that patients with pneumonia and patients with COPD, who experience unplanned ICU transfer, have substantially worse mortality than those directly admitted to the ICU.1

Although less frequent than hospitalizations for respiratory conditions, patients admitted with sepsis were at the highest risk of unplanned ICU transfer (1 in 17 ED non‐ICU hospitalizations). We also found that MI and stroke ward admissions had a higher risk of unplanned ICU transfer. However, we previously found that unplanned ICU transfers for sepsis, MI, and stroke did not have worse mortality than direct ICU admits for these conditions.1 Therefore, quality improvement efforts to reduce excess mortality related to early decompensation in the hospital and unplanned ICU transfer would be most effective if targeted towards respiratory conditions such as pneumonia and COPD.

This is the only in‐depth study, to our knowledge, to explore the association between a set of mutually exclusive diagnostic categories and risk of unplanned ICU transfer within 24 hours, and it is the first study to identify risk factors for unplanned ICU transfer in a multi‐hospital cohort adjusted for patient‐ and hospital‐level characteristics. We also identified a novel hospital volumeoutcome relationship: Unplanned ICU transfers are up to twice as likely to occur in the smallest volume hospitals compared with highest volume hospitals. Hospital volume has long been proposed as a proxy for hospital resources; there are several studies showing a relationship between low‐volume hospitals and worse outcomes for a number of conditions.19, 20 Possible mechanisms may include decreased ICU capacity, decreased on‐call intensivists in the hospital after hours, and less experience with certain critical care conditions seen more frequently in high‐volume hospitals.21

Patients at risk of unplanned ICU transfer were also more likely to have physiologic derangement identified on laboratory testing, high comorbidity burden, and arrive on the ward between 11 PM and 7 AM. Given the strong correlation between comorbidity burden and physiologic derangement and mortality,14 it is not surprising that the COPS and LAPS were independent predictors of unplanned transfer. It is unclear, however, why arriving on the ward on the overnight shift is associated with higher risk. One possibility is that patients who arrive on the wards during 11 PM to 7 AM are also likely to have been in the ED during evening peak hours most associated with ED crowding.22 High levels of ED crowding have been associated with delays in care, worse quality care, lapses in patient safety, and even increased in‐hospital mortality.22, 23 Other possible reasons include decreased in‐hospital staffing and longer delays in critical diagnostic tests and interventions.2428

Admission to TCUs was associated with decreased risk of unplanned ICU transfer in the first 24 hours of hospitalization. This may be due to the continuous monitoring, decreased nursing‐to‐patient ratios, or the availability to provide some critical care interventions. In our study, age 85 was associated with lower likelihood of unplanned transfer. Unfortunately, we did not have access to data on advanced directives or patient preferences. Data on advanced directives would help to distinguish whether this phenomenon was related to end‐of‐life care goals versus other explanations.

Our study confirms some risk factors identified in previous studies. These include specific diagnoses such as pneumonia and COPD,12, 13, 29 heavy comorbidity burden,12, 13, 29 abnormal labs,29 and male sex.13 Pneumonia has consistently been shown to be a risk factor for unplanned ICU transfer. This may stem from the dynamic nature of this condition and its ability to rapidly progress, and the fact that some ICUs may not accept pneumonia patients unless they demonstrate a need for mechanical ventilation.30 Recently, a prediction rule has been developed to determine which patients with pneumonia are likely to have an unplanned ICU transfer.30 It is possible that with validation and application of this rule, unplanned transfer rates for pneumonia could be reduced. It is unclear whether males have unmeasured factors associated with increased risk of unplanned transfer or whether a true gender disparity exists.

Our findings should be interpreted within the context of this study's limitations. First, this study was not designed to distinguish the underlying cause of the unplanned transfer such as under‐recognition of illness severity in the ED, evolving clinical disease after leaving the ED, or delays in critical interventions on the ward. These are a focus of our ongoing research efforts. Second, while previous studies have demonstrated that our automated risk adjustment variables can accurately predict in‐hospital mortality (0.88 area under curve in external populations),17 additional data on vital signs and mental status could further improve risk adjustment. However, using automated data allowed us to study risk factors for unplanned transfer in a multi‐hospital cohort with a much larger population than has been previously studied. Serial data on vital signs and mental status both in the ED and during hospitalization could also be helpful in determining which unplanned transfers could be prevented with earlier recognition and intervention. Finally, all patient care occurred within an integrated healthcare delivery system. Thus, differences in case‐mix, hospital resources, ICU structure, and geographic location should be considered when applying our results to other healthcare systems.

This study raises several new areas for future research. With access to richer data becoming available in electronic medical records, prediction rules should be developed to enable better triage to appropriate levels of care for ED admissions. Future research should also analyze the comparative effectiveness of intermediate monitored units versus non‐monitored wards for preventing clinical deterioration by admitting diagnosis. Diagnoses that have been shown to have an increased risk of death after unplanned ICU transfer, such as pneumonia/respiratory infection and COPD,1 should be prioritized in this research. Better understanding is needed on the diagnosis‐specific differences and the differences in ED triage process and ICU structure that may explain why high‐volume hospitals have significantly lower rates of early unplanned ICU transfers compared with low‐volume hospitals. In particular, determining the effect of TCU and ICU capacities and census at the time of admission, and comparing patient risk characteristics across hospital‐volume strata would be very useful. Finally, more work is needed to determine whether the higher rate of unplanned transfers during overnight nursing shifts is related to decreased resource availability, preceding ED crowding, or other organizational causes.

In conclusion, patients admitted with respiratory conditions, sepsis, MI, high comorbidity, and abnormal labs are at modestly increased risk of unplanned ICU transfer within 24 hours of admission from the ED. Patients admitted with respiratory conditions (pneumonia/respiratory infections and COPD) accounted for half of the admitting diagnoses that are at increased risk for unplanned ICU transfer. These patients may benefit from better inpatient triage from the ED, earlier intervention, or closer monitoring. More research is needed to determine the specific aspects of care associated with admission to intermediate care units and high‐volume hospitals that reduce the risk of unplanned ICU transfer.

Acknowledgements

The authors thank John D. Greene, Juan Carlos La Guardia, and Benjamin Turk for their assistance with formatting of the dataset; Dr Alan S. Go, Acting Director of the Division of Research, for reviewing the manuscript; and Alina Schnake‐Mahl for formatting the manuscript.

References
  1. Liu V, Kipnis P, Rizk NW, et al. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2011;7(3):224230.
  2. Young MP, Gooder VJ, Bride K, et al. Inpatient transfers to the intensive care unit. J Gen Intern Med. 2003;18(2):7783.
  3. Escobar GJ, Greene JD, Gardner MN, et al. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6:7480.
  4. Chan PS, Khalid A, Longmore LS, et al. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300(21):25062513.
  5. Sharek PJ, Parast LM, Leong K, et al. Effect of a rapid response team on hospital‐wide mortality and code rates outside the ICU in a children's hospital. JAMA. 2007;298(19):22672274.
  6. Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):20912097.
  7. Winters BD, Pham JC, Hunt EA, et al. Rapid response systems: A systematic review. Crit Care Med. 2007;35(5):12381243.
  8. Ranji SR, Auerbach AD, Hurd CJ, et al. Effects of rapid response systems on clinical outcomes: systematic review and meta‐analysis. J Hosp Med. 2007;2(6):422432.
  9. Chan PS, Jain R, Nallmothu BK, et al. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):1826.
  10. McGaughey J, Alderdice F, Fowler R, et al. Outreach and early warning systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards. Cochrane Database Syst Rev. 2007;3:CD005529.
  11. Bapoje SR, Gaudiani JL, Narayanan V, et al. Unplanned transfers to a medical intensive care unit: Causes and relationship to preventable errors in care. J Hosp Med. 2011;6:6872.
  12. Tam V, Frost SA, Hillman KM, et al. Using administrative data to develop a nomogram for individualising risk of unplanned admission to intensive care. Resuscitation. 2008;79(2):241248.
  13. Frost SA, Alexandrou E, Bogdanovski T, et al. Unplanned admission to intensive care after emergency hospitalisation: risk factors and development of a nomogram for individualising risk. Resuscitation. 2009;80(2):224230.
  14. Escobar GJ, Greene JD, Scheirer P, et al. Risk‐adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232239.
  15. Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719724.
  16. Escobar GJ, Fireman BH, Palen TE, et al. Risk adjusting community‐acquired pneumonia hospital outcomes using automated databases. Am J Manag Care. 2008;14(3):158166.
  17. van Walraven C, Escobar GJ, Greene JD, et al. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2011;63(7):798803.
  18. Rabe‐Hesketh S, Skrondal A, Pickles A. Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects. J Econometrics. 2005;128(2):301323.
  19. Hannan EL. The relation between volume and outcome in health care. N Engl J Med. 1999;340(21):16771679.
  20. Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med. 2002;137(6):511520.
  21. Terwiesch C, Diwas K, Kahn JM. Working with capacity limitations: operations management in critical care. Crit Care. 2011;15(4):308.
  22. Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Intern Med. 2008;52(2):126136.
  23. Bernstein SL, Aronsky D, Duseja R, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med. 2009;16(1):110.
  24. Cavallazzi R, Marik PE, Hirani A, et al. Association between time of admission to the ICU and mortality. Chest. 2010;138(1):6875.
  25. Reeves MJ, Smith E, Fonarow G, et al. Off‐hour admission and in‐hospital stroke case fatality in the get with the guidelines‐stroke program. Stroke. 2009;40(2):569576.
  26. Magid DJ, Wang Y, Herrin J, et al. Relationship between time of day, day of week, timeliness of reperfusion, and in‐hospital mortality for patients with acute ST‐segment elevation myocardial infarction. JAMA. 2005;294(7):803812.
  27. Laupland KB, Shahpori R, Kirkpatrick AW, et al. Hospital mortality among adults admitted to and discharged from intensive care on weekends and evenings. J Crit Care. 2008;23(3):317324.
  28. Afessa B, Gajic O, Morales IJ, et al. Association between ICU admission during morning rounds and mortality. Chest. 2009;136(6):14891495.
  29. Kennedy M, Joyce N, Howell MD, et al. Identifying infected emergency department patients admitted to the hospital ward at risk of clinical deterioration and intensive care unit transfer. Acad Emerg Med. 2010;17(10):10801085.
  30. Renaud B, Labarère J, Coma E, et al. Risk stratification of early admission to the intensive care unit of patients with no major criteria of severe community‐acquired pneumonia: development of an international prediction rule. Crit Care. 2009;13(2):R54.
References
  1. Liu V, Kipnis P, Rizk NW, et al. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2011;7(3):224230.
  2. Young MP, Gooder VJ, Bride K, et al. Inpatient transfers to the intensive care unit. J Gen Intern Med. 2003;18(2):7783.
  3. Escobar GJ, Greene JD, Gardner MN, et al. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2011;6:7480.
  4. Chan PS, Khalid A, Longmore LS, et al. Hospital‐wide code rates and mortality before and after implementation of a rapid response team. JAMA. 2008;300(21):25062513.
  5. Sharek PJ, Parast LM, Leong K, et al. Effect of a rapid response team on hospital‐wide mortality and code rates outside the ICU in a children's hospital. JAMA. 2007;298(19):22672274.
  6. Hillman K, Chen J, Cretikos M, et al. Introduction of the medical emergency team (MET) system: a cluster‐randomised controlled trial. Lancet. 2005;365(9477):20912097.
  7. Winters BD, Pham JC, Hunt EA, et al. Rapid response systems: A systematic review. Crit Care Med. 2007;35(5):12381243.
  8. Ranji SR, Auerbach AD, Hurd CJ, et al. Effects of rapid response systems on clinical outcomes: systematic review and meta‐analysis. J Hosp Med. 2007;2(6):422432.
  9. Chan PS, Jain R, Nallmothu BK, et al. Rapid response teams: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(1):1826.
  10. McGaughey J, Alderdice F, Fowler R, et al. Outreach and early warning systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards. Cochrane Database Syst Rev. 2007;3:CD005529.
  11. Bapoje SR, Gaudiani JL, Narayanan V, et al. Unplanned transfers to a medical intensive care unit: Causes and relationship to preventable errors in care. J Hosp Med. 2011;6:6872.
  12. Tam V, Frost SA, Hillman KM, et al. Using administrative data to develop a nomogram for individualising risk of unplanned admission to intensive care. Resuscitation. 2008;79(2):241248.
  13. Frost SA, Alexandrou E, Bogdanovski T, et al. Unplanned admission to intensive care after emergency hospitalisation: risk factors and development of a nomogram for individualising risk. Resuscitation. 2009;80(2):224230.
  14. Escobar GJ, Greene JD, Scheirer P, et al. Risk‐adjusting hospital inpatient mortality using automated inpatient, outpatient, and laboratory databases. Med Care. 2008;46(3):232239.
  15. Selby JV. Linking automated databases for research in managed care settings. Ann Intern Med. 1997;127(8 pt 2):719724.
  16. Escobar GJ, Fireman BH, Palen TE, et al. Risk adjusting community‐acquired pneumonia hospital outcomes using automated databases. Am J Manag Care. 2008;14(3):158166.
  17. van Walraven C, Escobar GJ, Greene JD, et al. The Kaiser Permanente inpatient risk adjustment methodology was valid in an external patient population. J Clin Epidemiol. 2011;63(7):798803.
  18. Rabe‐Hesketh S, Skrondal A, Pickles A. Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects. J Econometrics. 2005;128(2):301323.
  19. Hannan EL. The relation between volume and outcome in health care. N Engl J Med. 1999;340(21):16771679.
  20. Halm EA, Lee C, Chassin MR. Is volume related to outcome in health care? A systematic review and methodologic critique of the literature. Ann Intern Med. 2002;137(6):511520.
  21. Terwiesch C, Diwas K, Kahn JM. Working with capacity limitations: operations management in critical care. Crit Care. 2011;15(4):308.
  22. Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Intern Med. 2008;52(2):126136.
  23. Bernstein SL, Aronsky D, Duseja R, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med. 2009;16(1):110.
  24. Cavallazzi R, Marik PE, Hirani A, et al. Association between time of admission to the ICU and mortality. Chest. 2010;138(1):6875.
  25. Reeves MJ, Smith E, Fonarow G, et al. Off‐hour admission and in‐hospital stroke case fatality in the get with the guidelines‐stroke program. Stroke. 2009;40(2):569576.
  26. Magid DJ, Wang Y, Herrin J, et al. Relationship between time of day, day of week, timeliness of reperfusion, and in‐hospital mortality for patients with acute ST‐segment elevation myocardial infarction. JAMA. 2005;294(7):803812.
  27. Laupland KB, Shahpori R, Kirkpatrick AW, et al. Hospital mortality among adults admitted to and discharged from intensive care on weekends and evenings. J Crit Care. 2008;23(3):317324.
  28. Afessa B, Gajic O, Morales IJ, et al. Association between ICU admission during morning rounds and mortality. Chest. 2009;136(6):14891495.
  29. Kennedy M, Joyce N, Howell MD, et al. Identifying infected emergency department patients admitted to the hospital ward at risk of clinical deterioration and intensive care unit transfer. Acad Emerg Med. 2010;17(10):10801085.
  30. Renaud B, Labarère J, Coma E, et al. Risk stratification of early admission to the intensive care unit of patients with no major criteria of severe community‐acquired pneumonia: development of an international prediction rule. Crit Care. 2009;13(2):R54.
Issue
Journal of Hospital Medicine - 8(1)
Issue
Journal of Hospital Medicine - 8(1)
Page Number
13-19
Page Number
13-19
Publications
Publications
Article Type
Display Headline
Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system
Display Headline
Risk factors for unplanned transfer to intensive care within 24 hours of admission from the emergency department in an integrated healthcare system
Sections
Article Source

Copyright © 2012 Society of Hospital Medicine

Disallow All Ads
Correspondence Location
Division of Emergency Medicine, 300 Pasteur Drive, Alway Building, Room M121, Stanford, CA 94305
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Article PDF Media