High-flow nasal cannula (HFNC) oxygen therapy is effective in treating adults with acute hypoxemic respiratory failure, and to a lesser extent acute hypercapnic respiratory failure.^1-3 HFNC oxygen is capable of delivering oxygen with flows of 30-60 liters/minute, and can provide a high fraction of inspired oxygen, flush anatomic dead space, augment respiratory efforts, and provide mild continuous positive airway pressure effects. Several systematic reviews and meta-analyses have evaluated the effectiveness of HFNC oxygen and have shown modestly lower rates of intubation compared with conventional oxygen^4,5 and similar intubation rates compared with noninvasive positive pressure ventilation.^4-9 Although one randomized trial showed a lower risk of 90-day mortality for HFNC oxygen compared with either conventional oxygen or noninvasive positive pressure ventilation, several meta-analyses have shown no difference in intensive care unit (ICU) mortality.^4,6,8,10 The majority of studies have shown improvements in oxygenation, comfort, dyspnea scores, and breathing pattern with the initiation of HFNC oxygen.⁶

While the evidence to support the use of HFNC oxygen in patients with nonhypercapnic acute hypoxemic respiratory failure is growing, this evidence is based on patients enrolled in clinical trials who have no treatment limitations and consent to intubation if necessary. Indeed, several, if not all, randomized trials evaluating HFNC oxygen excluded patients who had do-not-intubate (DNI) or do-not-resuscitate (DNR) orders.^1,2,11 For patients with acute respiratory failure whose primary goal is not to extend life or utilize life support interventions such as invasive mechanical ventilation, HFNC oxygen may offer several benefits compared with other treatment options such as noninvasive positive pressure ventilation, conventional oxygen therapy, or palliative opioid therapy (Appendix Table 1). Determining which treatment options to use depends on the goals of care of the individual patient and the reasonable ability of a particular treatment to help the patient achieve those goals.

While a recent systematic review evaluated the existing evidence regarding the utility and outcomes of noninvasive positive pressure ventilation in adult patients with DNI orders,¹² a systematic review evaluating the evidence and rationale for HFNC oxygen in patients with DNI and/or DNR orders is lacking. Assessing such evidence is necessary to help clinicians and patients determine appropriate treatment choices and establish research priorities. Therefore, our primary objective was to determine what were the following outcomes: mortality, dyspnea, work of breathing, opioid doses, and quality of life in patients who received HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order.

We conducted a systematic review of studies that evaluated patients who used HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order. We reported the results using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements.¹³ This review was registered with the PROSPERO registry, CRD42017059914.

We included studies that enrolled patients who were (1) hospitalized, (2) >18 years old, (3) had an acute respiratory failure of any cause, (4) received HFNC oxygen, and (5) had a DNI or DNR or comfort measures only order. We included publications of all study designs (interventional, observational, and posthoc analyses) and all languages. We excluded studies that enrolled <5 patients. If necessary, we contacted the authors of the included studies for additional information.

Our search strategy included the following databases from inception to October 14, 2018: PubMed, MEDLINE, CINAHL, MICROMEDEX, EMBASE, Web of Science, and Scopus. The database-specific search strategy was developed using an experienced librarian (Appendix Table 2). In addition, we screened the reference lists of systematic reviews as well as the included studies to find additional relevant articles. Two authors (AM, MEW) independently assessed the inclusion criteria of the titles and abstracts that were identified in the search. In addition, these two authors abstracted relevant data of the included studies.

Using the search strategy, we identified 2,757 citations and included 301 of these in the full-text review (Figure). We included six studies, which enrolled 293 patients in the final systematic review. Table 1 summarizes the characteristics of the included investigations, all of which were observational studies.^15-20 The studies were conducted in the United States of America (n = 3), Europe (n = 2), and Asia (n = 1). Two studies were conducted in the general ICU populations and included patients with hypoxemic respiratory failure only. Four studies were conducted in cancer populations in the hospital wards or ICU and did not specify the type of respiratory failure (hypoxemic versus hypercapnic). Two studies included patients with DNI orders only.^15,20 One study included patients with DNR orders only (DNI orders were excluded).¹⁷ Three studies included patients with both DNR and DNI orders.^16,18,19 The numbers of enrolled patients with treatment limitations were generally low, with the two largest studies including 101 patients each on HFNC oxygen.^18,19

Risk of Bias

All included studies had a high risk of bias (Table 2). A high risk of bias was suggested because the investigations were single-center studies with unclear patient selection methods, did not explicitly report how decisions to limit treatments were made, and did not explicitly differentiate and separately analyze patients with “comfort measures only” goals of care.

Mortality

The hospital mortality rates of patients with DNI and/or DNR orders receiving HFNC were variable and ranged from 40% to 87%. In the two studies enrolling general ICU patient populations, the hospital mortality rates ranged from 40% to 60%. In the four studies enrolling patients with active malignancy, the hospital mortality rates ranged from 75% to 87%. No studies compared mortality rates with and without DNI and/or DNR orders.

Dyspnea, Work of Breathing, and Reduction in Opioid Doses

The impact of HFNC oxygen on symptom relief was reported in one retrospective observational study (published as a conference abstract only to date), which compared the effect of HFNC oxygen (n = 101) with conventional oxygen (n = 110).¹⁸ At first evaluation after hospital admission to a palliative care unit (after the patients had previously been started on either conventional oxygen or high-flow oxygen), patients in the HFNC oxygen group had worse (higher) dyspnea scores compared with patients who used conventional oxygen (Edmonton Symptom Assessment Scale score of 7.5 versus 5, P < .001). At follow-up, approximately 24 hours after admission to the hospital palliative care unit, there was no difference in the change of dyspnea between the HFNC oxygen group (dyspnea score change of 0) and the conventional oxygen group (dyspnea score change of −1, P = .18. In the same study, there was also no significant difference in the morphine dose requirement in each group, and exact doses were not reported.

Two studies reported improvement in oxygen saturation and respiratory rate after HFNC oxygen initiation (compared with before HFNC initiation).^16,20 Oxygen saturation increased from 89% to 95%, P < .01, in one study and 92% to 97%, P < .01, in a second study. The respiratory rate decreased from 31 to 25 breaths/minute in one study, and from 28 to 25 breaths/minute in a second study (both P < .01).

Quality of Life

No studies evaluated the quality of life of survivors.

Secondary Outcomes

Transition to Noninvasive Positive Pressure Ventilation

The proportion of patients who transitioned from HFNC oxygen to NPPV was relatively low in the two studies that reported this outcome, ranging from 0%²⁰ to 18%.¹⁶ In one observational study of a general ICU population, 9/50 (18%) of patients transitioned from HFNC oxygen to NPPV. There was no statistically significant difference in hospital mortality rates among those who progressed to NPPV (67%) versus those who did not progress to NPPV (58%), P = .72.

Tolerance of HFNC and Adverse Events

HFNC oxygen was generally well tolerated based on the assessment of three studies (Table 1). One study reported no adverse events,¹⁶ one study reported that HFNC oxygen had to be discontinued because of nasal discomfort in 1% of patients,¹⁹ and a second study reported that HFNC oxygen had to be discontinued because of agitation in 4% of patients.²⁰

Quality of Death in Nonsurvivors

No studies evaluated the quality of death in those patients who died.

In this systematic review of six studies, all with a high risk of bias, a significant proportion of patients with a DNI and/or DNR order who used HFNC oxygen survived to hospital discharge. Oxygen saturation and respiratory rate consistently improved in the three studies that reported these outcomes. Only one study (published as a conference abstract only to date),¹⁸ however, measured patient-important outcomes related to symptom management and found no significant difference in dyspnea or morphine dose requirements in patients on HFNC oxygen compared with patients on conventional oxygen. HFNC oxygen was generally well tolerated and only had to be stopped in <5% of patients due to intolerance. We found no studies that assessed the quality of life in survivors or the quality of death in nonsurvivors.

Based on the limited evidence in the included studies, HFNC may be a viable treatment option for patients with preset treatment limitations who have acute respiratory failure—with potential benefits of improved oxygenation, decreased respiratory rates, and hospital survival in a proportion of patients. Nevertheless, this systematic review highlights the vast paucity of data available to guide the use of HFNC oxygen in patients with treatment limitations and acute respiratory failure. Only a few studies, which were at high risk of bias, have been conducted on this topic to date. There is an inadequate evidence base to evaluate the comparative effectiveness of HFNC oxygen (versus NPPV versus conventional oxygen versus palliative opioids) in patients with DNI orders or comfort measures only orders.

Our review included two studies that evaluated the comparative effectiveness of HFNC oxygen in patients with DNI and/or DNR orders. The first retrospective observational study compared HFNC oxygen with conventional oxygen in patients with DNR and DNI orders and malignancy—and found no change in dyspnea—but did note an increase in mortality with HFNC oxygen (76% versus 51%).¹⁸ The second observational study compared HFNC oxygen with NPPV in patients with DNR orders with malignancy noted no difference in mortality.¹⁷ In patients with full-code orders, systematic reviews have shown that HFNC oxygen (compared with conventional oxygen) was associated with possible reductions in intubation rates, respiratory rates, and improvements in oxygenation—with no difference in mortality, dyspnea, patient comfort, or ICU/hospital length of stay. Compared with NPPV, HFNC oxygen was associated with similar rates of intubation and mortality.^4-6,21

Future studies in patients with acute respiratory failure and DNI and/or DNR orders should identify which treatment modality (HFNC oxygen compared with other modalities, such as NPPV, conventional oxygen, with or without palliative opioids) impacts outcomes, such as dyspnea reduction while maintaining an alert mental status, short- and long-term quality of life in survivors, and quality of death in nonsurvivors. Future studies should also identify the optimal treatment pathway to utilize when patients using HFNC oxygen fail this therapy (eg, transition to NPPV versus intensifying palliative opioids) as well as the optimal process to withdraw palliative HFNC oxygen.²² Identifying which patient populations may benefit from different treatment pathways should also be considered as different treatment strategies may be more beneficial in different patient populations (eg, based on cause and severity of acute respiratory failure). In addition, it should be noted that the primary goal of care might affect which outcomes are the most important to measure. While patients with comfort measures only, orders usually have a primary goal to prepare for a high-quality death, patients with DNI and/or DNR orders (but without comfort measures only orders) may have a primary goal to survive—but with the desire not to endure the high burden of intubation and mechanical ventilation if it became necessary. Finally, future studies should utilize high-quality study designs (eg, randomized controlled trials) that enable robust evaluation of comparative effectiveness of clinically relevant treatment strategies.

While several previous systematic reviews have evaluated the efficacy of HFNC in patients with acute respiratory failure without preset limitations on life support; to our knowledge, this is the first systematic review to assess outcomes in patients rigorously with preset treatment limitations. Our review is, however, limited by the high risk of bias of the studies that were included (single-center nature, retrospective observational study designs, small sample sizes, and lack of a description of how DNI and/or DNR statuses were determined) as well as the small number of studies available to be included.

This systematic review points to a significant evidence gap in our understanding of the role for HFNC oxygen (compared with other acceptable alternative treatment strategies) in adult patients with acute respiratory failure who have DNI and/or DNR orders. Further high-quality research is needed to explore these unanswered questions in an effort to best treat, guide, and engage in optimal end-of-life decision making among patients with acute respiratory failure.

High-flow nasal cannula (HFNC) oxygen therapy is effective in treating adults with acute hypoxemic respiratory failure, and to a lesser extent acute hypercapnic respiratory failure.^1-3 HFNC oxygen is capable of delivering oxygen with flows of 30-60 liters/minute, and can provide a high fraction of inspired oxygen, flush anatomic dead space, augment respiratory efforts, and provide mild continuous positive airway pressure effects. Several systematic reviews and meta-analyses have evaluated the effectiveness of HFNC oxygen and have shown modestly lower rates of intubation compared with conventional oxygen^4,5 and similar intubation rates compared with noninvasive positive pressure ventilation.^4-9 Although one randomized trial showed a lower risk of 90-day mortality for HFNC oxygen compared with either conventional oxygen or noninvasive positive pressure ventilation, several meta-analyses have shown no difference in intensive care unit (ICU) mortality.^4,6,8,10 The majority of studies have shown improvements in oxygenation, comfort, dyspnea scores, and breathing pattern with the initiation of HFNC oxygen.⁶

While the evidence to support the use of HFNC oxygen in patients with nonhypercapnic acute hypoxemic respiratory failure is growing, this evidence is based on patients enrolled in clinical trials who have no treatment limitations and consent to intubation if necessary. Indeed, several, if not all, randomized trials evaluating HFNC oxygen excluded patients who had do-not-intubate (DNI) or do-not-resuscitate (DNR) orders.^1,2,11 For patients with acute respiratory failure whose primary goal is not to extend life or utilize life support interventions such as invasive mechanical ventilation, HFNC oxygen may offer several benefits compared with other treatment options such as noninvasive positive pressure ventilation, conventional oxygen therapy, or palliative opioid therapy (Appendix Table 1). Determining which treatment options to use depends on the goals of care of the individual patient and the reasonable ability of a particular treatment to help the patient achieve those goals.

While a recent systematic review evaluated the existing evidence regarding the utility and outcomes of noninvasive positive pressure ventilation in adult patients with DNI orders,¹² a systematic review evaluating the evidence and rationale for HFNC oxygen in patients with DNI and/or DNR orders is lacking. Assessing such evidence is necessary to help clinicians and patients determine appropriate treatment choices and establish research priorities. Therefore, our primary objective was to determine what were the following outcomes: mortality, dyspnea, work of breathing, opioid doses, and quality of life in patients who received HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order.

We conducted a systematic review of studies that evaluated patients who used HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order. We reported the results using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements.¹³ This review was registered with the PROSPERO registry, CRD42017059914.

We included studies that enrolled patients who were (1) hospitalized, (2) >18 years old, (3) had an acute respiratory failure of any cause, (4) received HFNC oxygen, and (5) had a DNI or DNR or comfort measures only order. We included publications of all study designs (interventional, observational, and posthoc analyses) and all languages. We excluded studies that enrolled <5 patients. If necessary, we contacted the authors of the included studies for additional information.

Our search strategy included the following databases from inception to October 14, 2018: PubMed, MEDLINE, CINAHL, MICROMEDEX, EMBASE, Web of Science, and Scopus. The database-specific search strategy was developed using an experienced librarian (Appendix Table 2). In addition, we screened the reference lists of systematic reviews as well as the included studies to find additional relevant articles. Two authors (AM, MEW) independently assessed the inclusion criteria of the titles and abstracts that were identified in the search. In addition, these two authors abstracted relevant data of the included studies.

Using the search strategy, we identified 2,757 citations and included 301 of these in the full-text review (Figure). We included six studies, which enrolled 293 patients in the final systematic review. Table 1 summarizes the characteristics of the included investigations, all of which were observational studies.^15-20 The studies were conducted in the United States of America (n = 3), Europe (n = 2), and Asia (n = 1). Two studies were conducted in the general ICU populations and included patients with hypoxemic respiratory failure only. Four studies were conducted in cancer populations in the hospital wards or ICU and did not specify the type of respiratory failure (hypoxemic versus hypercapnic). Two studies included patients with DNI orders only.^15,20 One study included patients with DNR orders only (DNI orders were excluded).¹⁷ Three studies included patients with both DNR and DNI orders.^16,18,19 The numbers of enrolled patients with treatment limitations were generally low, with the two largest studies including 101 patients each on HFNC oxygen.^18,19

Risk of Bias

All included studies had a high risk of bias (Table 2). A high risk of bias was suggested because the investigations were single-center studies with unclear patient selection methods, did not explicitly report how decisions to limit treatments were made, and did not explicitly differentiate and separately analyze patients with “comfort measures only” goals of care.

Mortality

The hospital mortality rates of patients with DNI and/or DNR orders receiving HFNC were variable and ranged from 40% to 87%. In the two studies enrolling general ICU patient populations, the hospital mortality rates ranged from 40% to 60%. In the four studies enrolling patients with active malignancy, the hospital mortality rates ranged from 75% to 87%. No studies compared mortality rates with and without DNI and/or DNR orders.

Dyspnea, Work of Breathing, and Reduction in Opioid Doses

The impact of HFNC oxygen on symptom relief was reported in one retrospective observational study (published as a conference abstract only to date), which compared the effect of HFNC oxygen (n = 101) with conventional oxygen (n = 110).¹⁸ At first evaluation after hospital admission to a palliative care unit (after the patients had previously been started on either conventional oxygen or high-flow oxygen), patients in the HFNC oxygen group had worse (higher) dyspnea scores compared with patients who used conventional oxygen (Edmonton Symptom Assessment Scale score of 7.5 versus 5, P < .001). At follow-up, approximately 24 hours after admission to the hospital palliative care unit, there was no difference in the change of dyspnea between the HFNC oxygen group (dyspnea score change of 0) and the conventional oxygen group (dyspnea score change of −1, P = .18. In the same study, there was also no significant difference in the morphine dose requirement in each group, and exact doses were not reported.

Two studies reported improvement in oxygen saturation and respiratory rate after HFNC oxygen initiation (compared with before HFNC initiation).^16,20 Oxygen saturation increased from 89% to 95%, P < .01, in one study and 92% to 97%, P < .01, in a second study. The respiratory rate decreased from 31 to 25 breaths/minute in one study, and from 28 to 25 breaths/minute in a second study (both P < .01).

Quality of Life

No studies evaluated the quality of life of survivors.

Secondary Outcomes

Transition to Noninvasive Positive Pressure Ventilation

The proportion of patients who transitioned from HFNC oxygen to NPPV was relatively low in the two studies that reported this outcome, ranging from 0%²⁰ to 18%.¹⁶ In one observational study of a general ICU population, 9/50 (18%) of patients transitioned from HFNC oxygen to NPPV. There was no statistically significant difference in hospital mortality rates among those who progressed to NPPV (67%) versus those who did not progress to NPPV (58%), P = .72.

Tolerance of HFNC and Adverse Events

HFNC oxygen was generally well tolerated based on the assessment of three studies (Table 1). One study reported no adverse events,¹⁶ one study reported that HFNC oxygen had to be discontinued because of nasal discomfort in 1% of patients,¹⁹ and a second study reported that HFNC oxygen had to be discontinued because of agitation in 4% of patients.²⁰

Quality of Death in Nonsurvivors

No studies evaluated the quality of death in those patients who died.

In this systematic review of six studies, all with a high risk of bias, a significant proportion of patients with a DNI and/or DNR order who used HFNC oxygen survived to hospital discharge. Oxygen saturation and respiratory rate consistently improved in the three studies that reported these outcomes. Only one study (published as a conference abstract only to date),¹⁸ however, measured patient-important outcomes related to symptom management and found no significant difference in dyspnea or morphine dose requirements in patients on HFNC oxygen compared with patients on conventional oxygen. HFNC oxygen was generally well tolerated and only had to be stopped in <5% of patients due to intolerance. We found no studies that assessed the quality of life in survivors or the quality of death in nonsurvivors.

Based on the limited evidence in the included studies, HFNC may be a viable treatment option for patients with preset treatment limitations who have acute respiratory failure—with potential benefits of improved oxygenation, decreased respiratory rates, and hospital survival in a proportion of patients. Nevertheless, this systematic review highlights the vast paucity of data available to guide the use of HFNC oxygen in patients with treatment limitations and acute respiratory failure. Only a few studies, which were at high risk of bias, have been conducted on this topic to date. There is an inadequate evidence base to evaluate the comparative effectiveness of HFNC oxygen (versus NPPV versus conventional oxygen versus palliative opioids) in patients with DNI orders or comfort measures only orders.

Our review included two studies that evaluated the comparative effectiveness of HFNC oxygen in patients with DNI and/or DNR orders. The first retrospective observational study compared HFNC oxygen with conventional oxygen in patients with DNR and DNI orders and malignancy—and found no change in dyspnea—but did note an increase in mortality with HFNC oxygen (76% versus 51%).¹⁸ The second observational study compared HFNC oxygen with NPPV in patients with DNR orders with malignancy noted no difference in mortality.¹⁷ In patients with full-code orders, systematic reviews have shown that HFNC oxygen (compared with conventional oxygen) was associated with possible reductions in intubation rates, respiratory rates, and improvements in oxygenation—with no difference in mortality, dyspnea, patient comfort, or ICU/hospital length of stay. Compared with NPPV, HFNC oxygen was associated with similar rates of intubation and mortality.^4-6,21

Future studies in patients with acute respiratory failure and DNI and/or DNR orders should identify which treatment modality (HFNC oxygen compared with other modalities, such as NPPV, conventional oxygen, with or without palliative opioids) impacts outcomes, such as dyspnea reduction while maintaining an alert mental status, short- and long-term quality of life in survivors, and quality of death in nonsurvivors. Future studies should also identify the optimal treatment pathway to utilize when patients using HFNC oxygen fail this therapy (eg, transition to NPPV versus intensifying palliative opioids) as well as the optimal process to withdraw palliative HFNC oxygen.²² Identifying which patient populations may benefit from different treatment pathways should also be considered as different treatment strategies may be more beneficial in different patient populations (eg, based on cause and severity of acute respiratory failure). In addition, it should be noted that the primary goal of care might affect which outcomes are the most important to measure. While patients with comfort measures only, orders usually have a primary goal to prepare for a high-quality death, patients with DNI and/or DNR orders (but without comfort measures only orders) may have a primary goal to survive—but with the desire not to endure the high burden of intubation and mechanical ventilation if it became necessary. Finally, future studies should utilize high-quality study designs (eg, randomized controlled trials) that enable robust evaluation of comparative effectiveness of clinically relevant treatment strategies.

While several previous systematic reviews have evaluated the efficacy of HFNC in patients with acute respiratory failure without preset limitations on life support; to our knowledge, this is the first systematic review to assess outcomes in patients rigorously with preset treatment limitations. Our review is, however, limited by the high risk of bias of the studies that were included (single-center nature, retrospective observational study designs, small sample sizes, and lack of a description of how DNI and/or DNR statuses were determined) as well as the small number of studies available to be included.

This systematic review points to a significant evidence gap in our understanding of the role for HFNC oxygen (compared with other acceptable alternative treatment strategies) in adult patients with acute respiratory failure who have DNI and/or DNR orders. Further high-quality research is needed to explore these unanswered questions in an effort to best treat, guide, and engage in optimal end-of-life decision making among patients with acute respiratory failure.

High-flow nasal cannula (HFNC) oxygen therapy is effective in treating adults with acute hypoxemic respiratory failure, and to a lesser extent acute hypercapnic respiratory failure.^1-3 HFNC oxygen is capable of delivering oxygen with flows of 30-60 liters/minute, and can provide a high fraction of inspired oxygen, flush anatomic dead space, augment respiratory efforts, and provide mild continuous positive airway pressure effects. Several systematic reviews and meta-analyses have evaluated the effectiveness of HFNC oxygen and have shown modestly lower rates of intubation compared with conventional oxygen^4,5 and similar intubation rates compared with noninvasive positive pressure ventilation.^4-9 Although one randomized trial showed a lower risk of 90-day mortality for HFNC oxygen compared with either conventional oxygen or noninvasive positive pressure ventilation, several meta-analyses have shown no difference in intensive care unit (ICU) mortality.^4,6,8,10 The majority of studies have shown improvements in oxygenation, comfort, dyspnea scores, and breathing pattern with the initiation of HFNC oxygen.⁶

While the evidence to support the use of HFNC oxygen in patients with nonhypercapnic acute hypoxemic respiratory failure is growing, this evidence is based on patients enrolled in clinical trials who have no treatment limitations and consent to intubation if necessary. Indeed, several, if not all, randomized trials evaluating HFNC oxygen excluded patients who had do-not-intubate (DNI) or do-not-resuscitate (DNR) orders.^1,2,11 For patients with acute respiratory failure whose primary goal is not to extend life or utilize life support interventions such as invasive mechanical ventilation, HFNC oxygen may offer several benefits compared with other treatment options such as noninvasive positive pressure ventilation, conventional oxygen therapy, or palliative opioid therapy (Appendix Table 1). Determining which treatment options to use depends on the goals of care of the individual patient and the reasonable ability of a particular treatment to help the patient achieve those goals.

While a recent systematic review evaluated the existing evidence regarding the utility and outcomes of noninvasive positive pressure ventilation in adult patients with DNI orders,¹² a systematic review evaluating the evidence and rationale for HFNC oxygen in patients with DNI and/or DNR orders is lacking. Assessing such evidence is necessary to help clinicians and patients determine appropriate treatment choices and establish research priorities. Therefore, our primary objective was to determine what were the following outcomes: mortality, dyspnea, work of breathing, opioid doses, and quality of life in patients who received HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order.

We conducted a systematic review of studies that evaluated patients who used HFNC oxygen for acute respiratory failure and had a DNI and/or DNR order. We reported the results using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statements.¹³ This review was registered with the PROSPERO registry, CRD42017059914.

We included studies that enrolled patients who were (1) hospitalized, (2) >18 years old, (3) had an acute respiratory failure of any cause, (4) received HFNC oxygen, and (5) had a DNI or DNR or comfort measures only order. We included publications of all study designs (interventional, observational, and posthoc analyses) and all languages. We excluded studies that enrolled <5 patients. If necessary, we contacted the authors of the included studies for additional information.

Our search strategy included the following databases from inception to October 14, 2018: PubMed, MEDLINE, CINAHL, MICROMEDEX, EMBASE, Web of Science, and Scopus. The database-specific search strategy was developed using an experienced librarian (Appendix Table 2). In addition, we screened the reference lists of systematic reviews as well as the included studies to find additional relevant articles. Two authors (AM, MEW) independently assessed the inclusion criteria of the titles and abstracts that were identified in the search. In addition, these two authors abstracted relevant data of the included studies.

Using the search strategy, we identified 2,757 citations and included 301 of these in the full-text review (Figure). We included six studies, which enrolled 293 patients in the final systematic review. Table 1 summarizes the characteristics of the included investigations, all of which were observational studies.^15-20 The studies were conducted in the United States of America (n = 3), Europe (n = 2), and Asia (n = 1). Two studies were conducted in the general ICU populations and included patients with hypoxemic respiratory failure only. Four studies were conducted in cancer populations in the hospital wards or ICU and did not specify the type of respiratory failure (hypoxemic versus hypercapnic). Two studies included patients with DNI orders only.^15,20 One study included patients with DNR orders only (DNI orders were excluded).¹⁷ Three studies included patients with both DNR and DNI orders.^16,18,19 The numbers of enrolled patients with treatment limitations were generally low, with the two largest studies including 101 patients each on HFNC oxygen.^18,19

Risk of Bias

All included studies had a high risk of bias (Table 2). A high risk of bias was suggested because the investigations were single-center studies with unclear patient selection methods, did not explicitly report how decisions to limit treatments were made, and did not explicitly differentiate and separately analyze patients with “comfort measures only” goals of care.

Mortality

The hospital mortality rates of patients with DNI and/or DNR orders receiving HFNC were variable and ranged from 40% to 87%. In the two studies enrolling general ICU patient populations, the hospital mortality rates ranged from 40% to 60%. In the four studies enrolling patients with active malignancy, the hospital mortality rates ranged from 75% to 87%. No studies compared mortality rates with and without DNI and/or DNR orders.

Dyspnea, Work of Breathing, and Reduction in Opioid Doses

The impact of HFNC oxygen on symptom relief was reported in one retrospective observational study (published as a conference abstract only to date), which compared the effect of HFNC oxygen (n = 101) with conventional oxygen (n = 110).¹⁸ At first evaluation after hospital admission to a palliative care unit (after the patients had previously been started on either conventional oxygen or high-flow oxygen), patients in the HFNC oxygen group had worse (higher) dyspnea scores compared with patients who used conventional oxygen (Edmonton Symptom Assessment Scale score of 7.5 versus 5, P < .001). At follow-up, approximately 24 hours after admission to the hospital palliative care unit, there was no difference in the change of dyspnea between the HFNC oxygen group (dyspnea score change of 0) and the conventional oxygen group (dyspnea score change of −1, P = .18. In the same study, there was also no significant difference in the morphine dose requirement in each group, and exact doses were not reported.

Two studies reported improvement in oxygen saturation and respiratory rate after HFNC oxygen initiation (compared with before HFNC initiation).^16,20 Oxygen saturation increased from 89% to 95%, P < .01, in one study and 92% to 97%, P < .01, in a second study. The respiratory rate decreased from 31 to 25 breaths/minute in one study, and from 28 to 25 breaths/minute in a second study (both P < .01).

Quality of Life

No studies evaluated the quality of life of survivors.

Secondary Outcomes

Transition to Noninvasive Positive Pressure Ventilation

The proportion of patients who transitioned from HFNC oxygen to NPPV was relatively low in the two studies that reported this outcome, ranging from 0%²⁰ to 18%.¹⁶ In one observational study of a general ICU population, 9/50 (18%) of patients transitioned from HFNC oxygen to NPPV. There was no statistically significant difference in hospital mortality rates among those who progressed to NPPV (67%) versus those who did not progress to NPPV (58%), P = .72.

Tolerance of HFNC and Adverse Events

HFNC oxygen was generally well tolerated based on the assessment of three studies (Table 1). One study reported no adverse events,¹⁶ one study reported that HFNC oxygen had to be discontinued because of nasal discomfort in 1% of patients,¹⁹ and a second study reported that HFNC oxygen had to be discontinued because of agitation in 4% of patients.²⁰

Quality of Death in Nonsurvivors

No studies evaluated the quality of death in those patients who died.

In this systematic review of six studies, all with a high risk of bias, a significant proportion of patients with a DNI and/or DNR order who used HFNC oxygen survived to hospital discharge. Oxygen saturation and respiratory rate consistently improved in the three studies that reported these outcomes. Only one study (published as a conference abstract only to date),¹⁸ however, measured patient-important outcomes related to symptom management and found no significant difference in dyspnea or morphine dose requirements in patients on HFNC oxygen compared with patients on conventional oxygen. HFNC oxygen was generally well tolerated and only had to be stopped in <5% of patients due to intolerance. We found no studies that assessed the quality of life in survivors or the quality of death in nonsurvivors.

Based on the limited evidence in the included studies, HFNC may be a viable treatment option for patients with preset treatment limitations who have acute respiratory failure—with potential benefits of improved oxygenation, decreased respiratory rates, and hospital survival in a proportion of patients. Nevertheless, this systematic review highlights the vast paucity of data available to guide the use of HFNC oxygen in patients with treatment limitations and acute respiratory failure. Only a few studies, which were at high risk of bias, have been conducted on this topic to date. There is an inadequate evidence base to evaluate the comparative effectiveness of HFNC oxygen (versus NPPV versus conventional oxygen versus palliative opioids) in patients with DNI orders or comfort measures only orders.

Our review included two studies that evaluated the comparative effectiveness of HFNC oxygen in patients with DNI and/or DNR orders. The first retrospective observational study compared HFNC oxygen with conventional oxygen in patients with DNR and DNI orders and malignancy—and found no change in dyspnea—but did note an increase in mortality with HFNC oxygen (76% versus 51%).¹⁸ The second observational study compared HFNC oxygen with NPPV in patients with DNR orders with malignancy noted no difference in mortality.¹⁷ In patients with full-code orders, systematic reviews have shown that HFNC oxygen (compared with conventional oxygen) was associated with possible reductions in intubation rates, respiratory rates, and improvements in oxygenation—with no difference in mortality, dyspnea, patient comfort, or ICU/hospital length of stay. Compared with NPPV, HFNC oxygen was associated with similar rates of intubation and mortality.^4-6,21

Future studies in patients with acute respiratory failure and DNI and/or DNR orders should identify which treatment modality (HFNC oxygen compared with other modalities, such as NPPV, conventional oxygen, with or without palliative opioids) impacts outcomes, such as dyspnea reduction while maintaining an alert mental status, short- and long-term quality of life in survivors, and quality of death in nonsurvivors. Future studies should also identify the optimal treatment pathway to utilize when patients using HFNC oxygen fail this therapy (eg, transition to NPPV versus intensifying palliative opioids) as well as the optimal process to withdraw palliative HFNC oxygen.²² Identifying which patient populations may benefit from different treatment pathways should also be considered as different treatment strategies may be more beneficial in different patient populations (eg, based on cause and severity of acute respiratory failure). In addition, it should be noted that the primary goal of care might affect which outcomes are the most important to measure. While patients with comfort measures only, orders usually have a primary goal to prepare for a high-quality death, patients with DNI and/or DNR orders (but without comfort measures only orders) may have a primary goal to survive—but with the desire not to endure the high burden of intubation and mechanical ventilation if it became necessary. Finally, future studies should utilize high-quality study designs (eg, randomized controlled trials) that enable robust evaluation of comparative effectiveness of clinically relevant treatment strategies.

While several previous systematic reviews have evaluated the efficacy of HFNC in patients with acute respiratory failure without preset limitations on life support; to our knowledge, this is the first systematic review to assess outcomes in patients rigorously with preset treatment limitations. Our review is, however, limited by the high risk of bias of the studies that were included (single-center nature, retrospective observational study designs, small sample sizes, and lack of a description of how DNI and/or DNR statuses were determined) as well as the small number of studies available to be included.

This systematic review points to a significant evidence gap in our understanding of the role for HFNC oxygen (compared with other acceptable alternative treatment strategies) in adult patients with acute respiratory failure who have DNI and/or DNR orders. Further high-quality research is needed to explore these unanswered questions in an effort to best treat, guide, and engage in optimal end-of-life decision making among patients with acute respiratory failure.

The recent designation of Pediatric Hospital Medicine (PHM) as a board-certified subspecialty has provided the opportunity to define which skills are core to hospitalist practice. One skill that is novel to the field and gaining traction is point-of-care ultrasonography (POCUS). POCUS differs from traditional ultrasonography in that it is performed at the bedside by the primary clinician and aims to answer a focused clinical question (eg, does this patient have a skin abscess?) rather than to provide a comprehensive evaluation of the anatomy and physiology. The proposed advantages of POCUS include real-time image interpretation, cost savings, procedural guidance to minimize complications, and reduction of ionizing radiation. Although specialties such as Critical Care (CC) and Emergency Medicine (EM) have integrated POCUS into their practice and training, best practices in PHM have not been defined. This Progress Note is a summary of recent evidence to update past reviews and set the stage for future PHM POCUS research and education.

We met with an academic librarian in March 2019 and performed a search of PubMed using Medical Subject Headings (MESH) terms associated with POCUS as well as Pediatrics. We limited our search to studies published within the past five years. The search was originally focused to the field of PHM before expanding to a broader search since very few studies were found that focused on Hospital Medicine or general pediatric ward populations. This initial search generated 274 publications. We then performed a supplemental literature search using references from studies found in our initial search, as well as further ad hoc searching in Embase and Google Scholar.

After our literature search, we reviewed the PHM core competencies and identified the common clinical diagnoses and core skills for which there is POCUS literature published in the past five years. These included acute abdominal pain, bronchiolitis, pneumonia, skin and soft tissue infection, newborn care/delivery room management, bladder catheterization, fluid management, intravenous access, and lumbar puncture (LP). We chose to focus on one skill and two diagnoses that were generalizable to pediatric hospitalists across different settings and for which there was compelling evidence for POCUS use, such as pneumonia, skin abscess, and LP. We found few studies that included general pediatric ward patients, but we considered EM and CC studies to be relevant as several pediatric hospitalists practice in these clinical settings and with these patient populations.

POCUS can be useful for diagnosing pneumonia by direct visualization of lung consolidation or by identification of various sonographic artifacts that suggest pathology. For example, “B-lines” are vertical artifacts that extend from the pleura and suggest interstitial fluid or pneumonia when they are present in abnormally high numbers or density. POCUS can also be used to diagnose parapneumonic effusions by scanning dependent areas of the lung (eg, the diaphragm in children sitting upright) and looking for anechoic or hypoechoic areas.

Three recent meta-analyses found favorable operating characteristics when using POCUS for the diagnosis of pneumonia in children, with summary sensitivities of 93%-94% and specificities of 92%-96%.^1-3 However, these meta-analyses were limited by high heterogeneity due to the inclusion of multiple different care settings and the use of variable reference standards and sonographic criteria for diagnosing pneumonia. POCUS is superior to chest radiography for evaluating parapneumonic pleural effusions,⁴ allowing for rapid identification of loculations, fibrin strands, and proteinaceous material, and for serial bedside evaluation of effusion size and characteristics.

Additional advantages of POCUS include avoidance of ionizing radiation and the potential for cost and time savings. Two studies demonstrated reductions in radiography use and improved cost, although they were not conducted on hospitalized patients. One randomized controlled trial (RCT) conducted in a pediatric emergency department (ED) demonstrated a 38.8% reduction in chest radiography use without increasing the ED length of stay (EDLOS), antibiotic use, or unscheduled follow-up visits.⁵ A retrospective matched cohort study conducted in another pediatric ED reported that when compared with patients evaluated by chest radiography, those evaluated by POCUS had significantly shorter EDLOS (−60.9 min) and mean health systems savings ($187 per patient).⁶ We believe that POCUS has value in the evaluation and management of pneumonia and parapneumonic effusions, although further studies investigating patient outcomes and involving inpatient populations are required.

POCUS can augment the physical examination, helping to both avoid unnecessary incision and drainage (I+D) procedures and detect drainable fluid collections. Abscess is suggested when hypoechoic material without vascular flow is detected, and although other structures such as vessels, cysts, and lymph nodes can mimic skin abscesses, this is a relatively straightforward examination for clinicians to learn.

Two meta-analyses found that POCUS had high sensitivity for diagnosing skin abscesses in the ED.^7,8 A pediatric subgroup analysis conducted in a study by Barbic et al. found a sensitivity and a specificity of 94% (95% C: 88%-98%) and 83% (95% C: 47%-97%), respectively.⁷ Subramaniam et al. included six studies (four pediatric) with 800 patients (653 ≤ 18 years old) and found an overall pooled sensitivity of 97% (95% C: 94%-98%) and a specificity of 83% (95% C: 75%-88%).⁸ No subgroup analysis was performed, but the included pediatric studies reported sensitivities and specificities between 90%-98% and 68%-87%, respectively.

Although POCUS performs better than physical examination for the diagnosis of drainable abscesses, evidence regarding patient outcomes is mixed. A retrospective review from four pediatric EDs found that integration of POCUS lowered treatment failure rates, defined as any incision and drainage (I+D) or surgical manipulation after discharge from the initial ED visit (4.4% vs 15.6%, P < .005).⁹ A single-center retrospective cohort study found that POCUS reduced EDLOS by a median of 73 minutes (95% C: 52-94 min) when compared with radiology-performed studies.¹⁰ The aforementioned study conducted by Barbic et al. found that in pediatric studies, POCUS led to a change in management (eg, whether or not to attempt I+D) in 14%-27% of patients.⁷ However, a multicenter prospective observational cohort study involving seven pediatric EDs found that despite changing the management in 22.9% of cases, POCUS was not associated with any statistically significant differences in treatment failure rates, EDLOS, discharge rates, use of sedation, or use of alternative imaging.¹¹ These studies were limited by a lack of randomization or control group and emphasize the need for RCTs that measure patient outcomes. Future studies should investigate how POCUS can be used in inpatient settings both for initial diagnosis of drainable abscesses and for serial evaluation of evolving phlegmon or incompletely drained collections.

LP is commonly performed by pediatric hospitalists, although success can be influenced by numerous factors, including provider and staff expertise, patient anatomy, and body habitus. Requiring multiple attempts can increase patient discomfort and parental anxiety. Failure to obtain cerebrospinal fluid can delay diagnosis or leave providers in uncertain clinical situations that may commit patients to prolonged antibiotic courses. POCUS can be used to identify anatomic markers such as interspinous processes, anatomic midline, and depth of the ligamentum flavum.¹² It can also be used to identify epidural hematomas after failed LPs to avoid additional unsuccessful attempts.¹³ POCUS guidance for LP has been described using both static (preprocedural marking) and dynamic (scanning during the procedure) techniques, although most of the studies use the static approach. The Society for Hospital Medicine POCUS Task Force has recently released a position statement recommending that POCUS should be used for site selection before performing LP in adult patients when providers are adequately trained.¹² Although this position statement was for adult patients, recent evidence suggests that there is also benefit in Pediatrics.

Two recent meta-analyses have investigated POCUS use for pediatric LPs.^14,15 Olowoyeye et al. included four studies with a total of 277 patients and found that POCUS use was associated with a reduction in traumatic taps (risk ratio [RR] = 0.53, 95% C: 0.13-0.82) when compared with landmark approaches.¹⁴ However, there was no statistically significant reduction in LP failure, number of needle insertion attempts, or procedure length. A more recent meta-analysis performed a pediatric subgroup analysis of six studies including 452 patients and found a statistically significant reduction in traumatic taps (13.7% vs 31.8%, risk difference = −21.3%, 95% C: −38.2% to −4.3%) and number of needle insertion attempts (1.53 vs 2.07, mean difference = −0.47, 95% C: −0.73 to −0.21).¹⁵ The primary outcome of LP success trended toward favoring POCUS, but it was not statistically significant (88.4% vs 74.0%, OR = 2.55, 95% C: 0.99-6.52). We believe that recent evidence suggests that there is benefit in using POCUS when hospitalists attempt pediatric LPs, particularly when physical landmarks are difficult to identify or after failed attempts. However, adequate training with simulation and supervised practice should be undertaken before integrating this into clinical practice.

Evidence accumulated in the past five years has built on previous work suggesting that POCUS has a role in the diagnosis of pneumonia and skin abscess and in the performance of LPs. However, gaps in the literature remain when applying POCUS in PHM. Only a few studies to date were conducted in non-CC inpatient settings, and although several pediatric hospitalists work in EDs or care for critically ill children, our largest population comprises general pediatric ward patients. Studies have also used ultrasonographers with variable POCUS training and clinical experience, which makes comparing or combining studies challenging since POCUS is dependent on provider skills. Studies involving PHM providers and inpatient populations are needed. Additional studies evaluating the process and outcome measures are also needed to understand whether the theoretical advantages are consistently realized in real-world PHM practice. Finally, PHM-specific curricula should be designed in collaboration with various PHM stakeholders and with specialties who already have robust POCUS training pathways. There is opportunity within PHM for multi institutional research collaboration, identification of best practices, and development of PHM-specific training for fellowship and faculty development programs.

The recent designation of Pediatric Hospital Medicine (PHM) as a board-certified subspecialty has provided the opportunity to define which skills are core to hospitalist practice. One skill that is novel to the field and gaining traction is point-of-care ultrasonography (POCUS). POCUS differs from traditional ultrasonography in that it is performed at the bedside by the primary clinician and aims to answer a focused clinical question (eg, does this patient have a skin abscess?) rather than to provide a comprehensive evaluation of the anatomy and physiology. The proposed advantages of POCUS include real-time image interpretation, cost savings, procedural guidance to minimize complications, and reduction of ionizing radiation. Although specialties such as Critical Care (CC) and Emergency Medicine (EM) have integrated POCUS into their practice and training, best practices in PHM have not been defined. This Progress Note is a summary of recent evidence to update past reviews and set the stage for future PHM POCUS research and education.

We met with an academic librarian in March 2019 and performed a search of PubMed using Medical Subject Headings (MESH) terms associated with POCUS as well as Pediatrics. We limited our search to studies published within the past five years. The search was originally focused to the field of PHM before expanding to a broader search since very few studies were found that focused on Hospital Medicine or general pediatric ward populations. This initial search generated 274 publications. We then performed a supplemental literature search using references from studies found in our initial search, as well as further ad hoc searching in Embase and Google Scholar.

After our literature search, we reviewed the PHM core competencies and identified the common clinical diagnoses and core skills for which there is POCUS literature published in the past five years. These included acute abdominal pain, bronchiolitis, pneumonia, skin and soft tissue infection, newborn care/delivery room management, bladder catheterization, fluid management, intravenous access, and lumbar puncture (LP). We chose to focus on one skill and two diagnoses that were generalizable to pediatric hospitalists across different settings and for which there was compelling evidence for POCUS use, such as pneumonia, skin abscess, and LP. We found few studies that included general pediatric ward patients, but we considered EM and CC studies to be relevant as several pediatric hospitalists practice in these clinical settings and with these patient populations.

POCUS can be useful for diagnosing pneumonia by direct visualization of lung consolidation or by identification of various sonographic artifacts that suggest pathology. For example, “B-lines” are vertical artifacts that extend from the pleura and suggest interstitial fluid or pneumonia when they are present in abnormally high numbers or density. POCUS can also be used to diagnose parapneumonic effusions by scanning dependent areas of the lung (eg, the diaphragm in children sitting upright) and looking for anechoic or hypoechoic areas.

Three recent meta-analyses found favorable operating characteristics when using POCUS for the diagnosis of pneumonia in children, with summary sensitivities of 93%-94% and specificities of 92%-96%.^1-3 However, these meta-analyses were limited by high heterogeneity due to the inclusion of multiple different care settings and the use of variable reference standards and sonographic criteria for diagnosing pneumonia. POCUS is superior to chest radiography for evaluating parapneumonic pleural effusions,⁴ allowing for rapid identification of loculations, fibrin strands, and proteinaceous material, and for serial bedside evaluation of effusion size and characteristics.

Additional advantages of POCUS include avoidance of ionizing radiation and the potential for cost and time savings. Two studies demonstrated reductions in radiography use and improved cost, although they were not conducted on hospitalized patients. One randomized controlled trial (RCT) conducted in a pediatric emergency department (ED) demonstrated a 38.8% reduction in chest radiography use without increasing the ED length of stay (EDLOS), antibiotic use, or unscheduled follow-up visits.⁵ A retrospective matched cohort study conducted in another pediatric ED reported that when compared with patients evaluated by chest radiography, those evaluated by POCUS had significantly shorter EDLOS (−60.9 min) and mean health systems savings ($187 per patient).⁶ We believe that POCUS has value in the evaluation and management of pneumonia and parapneumonic effusions, although further studies investigating patient outcomes and involving inpatient populations are required.

POCUS can augment the physical examination, helping to both avoid unnecessary incision and drainage (I+D) procedures and detect drainable fluid collections. Abscess is suggested when hypoechoic material without vascular flow is detected, and although other structures such as vessels, cysts, and lymph nodes can mimic skin abscesses, this is a relatively straightforward examination for clinicians to learn.

Two meta-analyses found that POCUS had high sensitivity for diagnosing skin abscesses in the ED.^7,8 A pediatric subgroup analysis conducted in a study by Barbic et al. found a sensitivity and a specificity of 94% (95% C: 88%-98%) and 83% (95% C: 47%-97%), respectively.⁷ Subramaniam et al. included six studies (four pediatric) with 800 patients (653 ≤ 18 years old) and found an overall pooled sensitivity of 97% (95% C: 94%-98%) and a specificity of 83% (95% C: 75%-88%).⁸ No subgroup analysis was performed, but the included pediatric studies reported sensitivities and specificities between 90%-98% and 68%-87%, respectively.

Although POCUS performs better than physical examination for the diagnosis of drainable abscesses, evidence regarding patient outcomes is mixed. A retrospective review from four pediatric EDs found that integration of POCUS lowered treatment failure rates, defined as any incision and drainage (I+D) or surgical manipulation after discharge from the initial ED visit (4.4% vs 15.6%, P < .005).⁹ A single-center retrospective cohort study found that POCUS reduced EDLOS by a median of 73 minutes (95% C: 52-94 min) when compared with radiology-performed studies.¹⁰ The aforementioned study conducted by Barbic et al. found that in pediatric studies, POCUS led to a change in management (eg, whether or not to attempt I+D) in 14%-27% of patients.⁷ However, a multicenter prospective observational cohort study involving seven pediatric EDs found that despite changing the management in 22.9% of cases, POCUS was not associated with any statistically significant differences in treatment failure rates, EDLOS, discharge rates, use of sedation, or use of alternative imaging.¹¹ These studies were limited by a lack of randomization or control group and emphasize the need for RCTs that measure patient outcomes. Future studies should investigate how POCUS can be used in inpatient settings both for initial diagnosis of drainable abscesses and for serial evaluation of evolving phlegmon or incompletely drained collections.

LP is commonly performed by pediatric hospitalists, although success can be influenced by numerous factors, including provider and staff expertise, patient anatomy, and body habitus. Requiring multiple attempts can increase patient discomfort and parental anxiety. Failure to obtain cerebrospinal fluid can delay diagnosis or leave providers in uncertain clinical situations that may commit patients to prolonged antibiotic courses. POCUS can be used to identify anatomic markers such as interspinous processes, anatomic midline, and depth of the ligamentum flavum.¹² It can also be used to identify epidural hematomas after failed LPs to avoid additional unsuccessful attempts.¹³ POCUS guidance for LP has been described using both static (preprocedural marking) and dynamic (scanning during the procedure) techniques, although most of the studies use the static approach. The Society for Hospital Medicine POCUS Task Force has recently released a position statement recommending that POCUS should be used for site selection before performing LP in adult patients when providers are adequately trained.¹² Although this position statement was for adult patients, recent evidence suggests that there is also benefit in Pediatrics.

Two recent meta-analyses have investigated POCUS use for pediatric LPs.^14,15 Olowoyeye et al. included four studies with a total of 277 patients and found that POCUS use was associated with a reduction in traumatic taps (risk ratio [RR] = 0.53, 95% C: 0.13-0.82) when compared with landmark approaches.¹⁴ However, there was no statistically significant reduction in LP failure, number of needle insertion attempts, or procedure length. A more recent meta-analysis performed a pediatric subgroup analysis of six studies including 452 patients and found a statistically significant reduction in traumatic taps (13.7% vs 31.8%, risk difference = −21.3%, 95% C: −38.2% to −4.3%) and number of needle insertion attempts (1.53 vs 2.07, mean difference = −0.47, 95% C: −0.73 to −0.21).¹⁵ The primary outcome of LP success trended toward favoring POCUS, but it was not statistically significant (88.4% vs 74.0%, OR = 2.55, 95% C: 0.99-6.52). We believe that recent evidence suggests that there is benefit in using POCUS when hospitalists attempt pediatric LPs, particularly when physical landmarks are difficult to identify or after failed attempts. However, adequate training with simulation and supervised practice should be undertaken before integrating this into clinical practice.

Evidence accumulated in the past five years has built on previous work suggesting that POCUS has a role in the diagnosis of pneumonia and skin abscess and in the performance of LPs. However, gaps in the literature remain when applying POCUS in PHM. Only a few studies to date were conducted in non-CC inpatient settings, and although several pediatric hospitalists work in EDs or care for critically ill children, our largest population comprises general pediatric ward patients. Studies have also used ultrasonographers with variable POCUS training and clinical experience, which makes comparing or combining studies challenging since POCUS is dependent on provider skills. Studies involving PHM providers and inpatient populations are needed. Additional studies evaluating the process and outcome measures are also needed to understand whether the theoretical advantages are consistently realized in real-world PHM practice. Finally, PHM-specific curricula should be designed in collaboration with various PHM stakeholders and with specialties who already have robust POCUS training pathways. There is opportunity within PHM for multi institutional research collaboration, identification of best practices, and development of PHM-specific training for fellowship and faculty development programs.

The recent designation of Pediatric Hospital Medicine (PHM) as a board-certified subspecialty has provided the opportunity to define which skills are core to hospitalist practice. One skill that is novel to the field and gaining traction is point-of-care ultrasonography (POCUS). POCUS differs from traditional ultrasonography in that it is performed at the bedside by the primary clinician and aims to answer a focused clinical question (eg, does this patient have a skin abscess?) rather than to provide a comprehensive evaluation of the anatomy and physiology. The proposed advantages of POCUS include real-time image interpretation, cost savings, procedural guidance to minimize complications, and reduction of ionizing radiation. Although specialties such as Critical Care (CC) and Emergency Medicine (EM) have integrated POCUS into their practice and training, best practices in PHM have not been defined. This Progress Note is a summary of recent evidence to update past reviews and set the stage for future PHM POCUS research and education.

We met with an academic librarian in March 2019 and performed a search of PubMed using Medical Subject Headings (MESH) terms associated with POCUS as well as Pediatrics. We limited our search to studies published within the past five years. The search was originally focused to the field of PHM before expanding to a broader search since very few studies were found that focused on Hospital Medicine or general pediatric ward populations. This initial search generated 274 publications. We then performed a supplemental literature search using references from studies found in our initial search, as well as further ad hoc searching in Embase and Google Scholar.

After our literature search, we reviewed the PHM core competencies and identified the common clinical diagnoses and core skills for which there is POCUS literature published in the past five years. These included acute abdominal pain, bronchiolitis, pneumonia, skin and soft tissue infection, newborn care/delivery room management, bladder catheterization, fluid management, intravenous access, and lumbar puncture (LP). We chose to focus on one skill and two diagnoses that were generalizable to pediatric hospitalists across different settings and for which there was compelling evidence for POCUS use, such as pneumonia, skin abscess, and LP. We found few studies that included general pediatric ward patients, but we considered EM and CC studies to be relevant as several pediatric hospitalists practice in these clinical settings and with these patient populations.

POCUS can be useful for diagnosing pneumonia by direct visualization of lung consolidation or by identification of various sonographic artifacts that suggest pathology. For example, “B-lines” are vertical artifacts that extend from the pleura and suggest interstitial fluid or pneumonia when they are present in abnormally high numbers or density. POCUS can also be used to diagnose parapneumonic effusions by scanning dependent areas of the lung (eg, the diaphragm in children sitting upright) and looking for anechoic or hypoechoic areas.

Three recent meta-analyses found favorable operating characteristics when using POCUS for the diagnosis of pneumonia in children, with summary sensitivities of 93%-94% and specificities of 92%-96%.^1-3 However, these meta-analyses were limited by high heterogeneity due to the inclusion of multiple different care settings and the use of variable reference standards and sonographic criteria for diagnosing pneumonia. POCUS is superior to chest radiography for evaluating parapneumonic pleural effusions,⁴ allowing for rapid identification of loculations, fibrin strands, and proteinaceous material, and for serial bedside evaluation of effusion size and characteristics.

Additional advantages of POCUS include avoidance of ionizing radiation and the potential for cost and time savings. Two studies demonstrated reductions in radiography use and improved cost, although they were not conducted on hospitalized patients. One randomized controlled trial (RCT) conducted in a pediatric emergency department (ED) demonstrated a 38.8% reduction in chest radiography use without increasing the ED length of stay (EDLOS), antibiotic use, or unscheduled follow-up visits.⁵ A retrospective matched cohort study conducted in another pediatric ED reported that when compared with patients evaluated by chest radiography, those evaluated by POCUS had significantly shorter EDLOS (−60.9 min) and mean health systems savings ($187 per patient).⁶ We believe that POCUS has value in the evaluation and management of pneumonia and parapneumonic effusions, although further studies investigating patient outcomes and involving inpatient populations are required.

POCUS can augment the physical examination, helping to both avoid unnecessary incision and drainage (I+D) procedures and detect drainable fluid collections. Abscess is suggested when hypoechoic material without vascular flow is detected, and although other structures such as vessels, cysts, and lymph nodes can mimic skin abscesses, this is a relatively straightforward examination for clinicians to learn.

Two meta-analyses found that POCUS had high sensitivity for diagnosing skin abscesses in the ED.^7,8 A pediatric subgroup analysis conducted in a study by Barbic et al. found a sensitivity and a specificity of 94% (95% C: 88%-98%) and 83% (95% C: 47%-97%), respectively.⁷ Subramaniam et al. included six studies (four pediatric) with 800 patients (653 ≤ 18 years old) and found an overall pooled sensitivity of 97% (95% C: 94%-98%) and a specificity of 83% (95% C: 75%-88%).⁸ No subgroup analysis was performed, but the included pediatric studies reported sensitivities and specificities between 90%-98% and 68%-87%, respectively.

Although POCUS performs better than physical examination for the diagnosis of drainable abscesses, evidence regarding patient outcomes is mixed. A retrospective review from four pediatric EDs found that integration of POCUS lowered treatment failure rates, defined as any incision and drainage (I+D) or surgical manipulation after discharge from the initial ED visit (4.4% vs 15.6%, P < .005).⁹ A single-center retrospective cohort study found that POCUS reduced EDLOS by a median of 73 minutes (95% C: 52-94 min) when compared with radiology-performed studies.¹⁰ The aforementioned study conducted by Barbic et al. found that in pediatric studies, POCUS led to a change in management (eg, whether or not to attempt I+D) in 14%-27% of patients.⁷ However, a multicenter prospective observational cohort study involving seven pediatric EDs found that despite changing the management in 22.9% of cases, POCUS was not associated with any statistically significant differences in treatment failure rates, EDLOS, discharge rates, use of sedation, or use of alternative imaging.¹¹ These studies were limited by a lack of randomization or control group and emphasize the need for RCTs that measure patient outcomes. Future studies should investigate how POCUS can be used in inpatient settings both for initial diagnosis of drainable abscesses and for serial evaluation of evolving phlegmon or incompletely drained collections.

LP is commonly performed by pediatric hospitalists, although success can be influenced by numerous factors, including provider and staff expertise, patient anatomy, and body habitus. Requiring multiple attempts can increase patient discomfort and parental anxiety. Failure to obtain cerebrospinal fluid can delay diagnosis or leave providers in uncertain clinical situations that may commit patients to prolonged antibiotic courses. POCUS can be used to identify anatomic markers such as interspinous processes, anatomic midline, and depth of the ligamentum flavum.¹² It can also be used to identify epidural hematomas after failed LPs to avoid additional unsuccessful attempts.¹³ POCUS guidance for LP has been described using both static (preprocedural marking) and dynamic (scanning during the procedure) techniques, although most of the studies use the static approach. The Society for Hospital Medicine POCUS Task Force has recently released a position statement recommending that POCUS should be used for site selection before performing LP in adult patients when providers are adequately trained.¹² Although this position statement was for adult patients, recent evidence suggests that there is also benefit in Pediatrics.

Two recent meta-analyses have investigated POCUS use for pediatric LPs.^14,15 Olowoyeye et al. included four studies with a total of 277 patients and found that POCUS use was associated with a reduction in traumatic taps (risk ratio [RR] = 0.53, 95% C: 0.13-0.82) when compared with landmark approaches.¹⁴ However, there was no statistically significant reduction in LP failure, number of needle insertion attempts, or procedure length. A more recent meta-analysis performed a pediatric subgroup analysis of six studies including 452 patients and found a statistically significant reduction in traumatic taps (13.7% vs 31.8%, risk difference = −21.3%, 95% C: −38.2% to −4.3%) and number of needle insertion attempts (1.53 vs 2.07, mean difference = −0.47, 95% C: −0.73 to −0.21).¹⁵ The primary outcome of LP success trended toward favoring POCUS, but it was not statistically significant (88.4% vs 74.0%, OR = 2.55, 95% C: 0.99-6.52). We believe that recent evidence suggests that there is benefit in using POCUS when hospitalists attempt pediatric LPs, particularly when physical landmarks are difficult to identify or after failed attempts. However, adequate training with simulation and supervised practice should be undertaken before integrating this into clinical practice.

Evidence accumulated in the past five years has built on previous work suggesting that POCUS has a role in the diagnosis of pneumonia and skin abscess and in the performance of LPs. However, gaps in the literature remain when applying POCUS in PHM. Only a few studies to date were conducted in non-CC inpatient settings, and although several pediatric hospitalists work in EDs or care for critically ill children, our largest population comprises general pediatric ward patients. Studies have also used ultrasonographers with variable POCUS training and clinical experience, which makes comparing or combining studies challenging since POCUS is dependent on provider skills. Studies involving PHM providers and inpatient populations are needed. Additional studies evaluating the process and outcome measures are also needed to understand whether the theoretical advantages are consistently realized in real-world PHM practice. Finally, PHM-specific curricula should be designed in collaboration with various PHM stakeholders and with specialties who already have robust POCUS training pathways. There is opportunity within PHM for multi institutional research collaboration, identification of best practices, and development of PHM-specific training for fellowship and faculty development programs.

Research, in the field of Hospital Medicine, often leverages data collected for reasons other than research. For example, electronic medical record data or patient satisfaction survey results can be used to answer questions that are relevant to the practice of hospital medicine. In these types of datasets, data will inevitably be missing. Such missing data can compromise our ability to draw definitive conclusions from our research study. This review introduces the concept of missing data, describes patterns and mechanisms of missing data, and discusses common approaches for the handling of missing data, including sensitivity analyses for determining how robust the results are despite assumptions made about the missing data.

Missing data create a host of problems for researchers. First, missing data result in a loss of information and can diminish the power of the proposed study. Second, the irregular data complicate the analysis because many of the standard software procedures used have been developed for fully observed or “complete” data (ie, each subject has a value for all measures of interest). Finally, missing data may introduce bias due to the systematic difference between the observed and the unobserved data. For example, if men are less likely than women to complete all questions in a patient satisfaction survey when they are not satisfied, then hospital satisfaction analyses that rely on completed surveys would tend to provide biased estimates of the satisfaction males have with their care.

The ideal approach to mitigating problems caused by missing data is to anticipate and incorporate strategies to minimize missing data into the study design (ie, when planning data collection protocols for prospective studies). This plan should provide strategies for minimizing nonresponse and estimating the magnitude of anticipated missing data to ensure that the study achieves sufficient strength despite the missing data.

Strategies for minimizing nonresponse include (1) informing potential study participants, at initial contact, about the implications of missing data on the ability to answer the research question; (2) collecting several phone numbers, addresses, preferred method of contact and calling times, as well as an alternative contact, in case the primary study contact is unable to be reached; (3) specifying the number of call backs, as well as the time of contact; and (4) piloting data capture questions for phrasing, clarity, and sensitivity, in order to resolve problems before initiating the study. One approach that can be used to mitigate the impact of missing data in surveys is to contact a sample of the initial nonrespondents using a more intensive follow-up approach (eg, a nonresponse to a mailed survey is followed up by a telephone call in order to conduct the survey again over the phone), and this is referred to as “nonresponse two-phase sampling.” The additional data, captured in the second phase, not only reduces the nonresponse rate but can also provide important information on the missing data mechanism.^1,2 In longitudinal studies with dropouts, one can measure participants’ intent to drop out in order to evaluate how much the probability of dropping out depends on missing responses.³ One may also choose to determine the power and implications of sample size under different missing data assumptions.⁴

Different data sources are likely to have unique reasons for missing values due to the workflows involved in how the data are collected. In research involving the use of data from electronic medical records, missing data on specific diagnoses involving patients who are regularly engaged in care are often considered to be “not present” or “normal”, since clinical documentation workflows are largely governed by the concept of “documentation by exception” in which diagnoses are documented only when there is an exception to the expectation that these are not present. For example, “diabetes mellitus” is commonly documented, but “diabetes mellitus not present” is rarely documented in electronic medical records which are used for clinical care. Thus, lack of explicit documentation is likely to indicate that diabetes mellitus is, in fact, not present.

Certain variables may be missing simply because there is no quantifiable value­—ie, the data do not exist. Structural missingness refers to a value that does not exist for a logical reason (eg, “What is the gender of your first child?” for those who do not have a child). Censoring, which occurs during “time to event” analysis, refers to a situation where information about a subject stops before the event of interest happens, for example, when a subject in a study involving a 30-day outcome dies at day 14. The term “limit of detection” refers to the lowest or highest level at which two distinct values can reasonably be distinguished (eg, the lower limit of detection of a C-reactive protein assay may be 1 mg/dL, so lower values might simply be reported by the lab as <1 mg/dL).⁵ These types of missing data require specific methods that are not discussed in this review.

These examples illustrate that approaches to dealing with missing data vary depending on what data sources are used and how data are collected. Understanding the reasons missing data are present is a necessary step in formulating a robust analytic approach to handling missing data.

Missing Data Patterns

Evaluating missing data patterns provides information on the degree and complexity of the missing data problem and can aid in choosing an appropriate missing data handling method. This is because some analytic methods work well for a general pattern (nonmonotone) and other methods work for special patterns (eg, monotone, file matching). In longitudinal studies, missing data is commonly missing in a monotone pattern, where once one variable is missing then all subsequent variables are also missing for a particular subject. This occurs when a study participant is lost to follow-up. For example, a monotone missing data pattern may occur in a study that requires a series of follow-up visits for laboratory blood tests. If a patient drops out, it results in a monotone missing data pattern, as no data on blood test results are available once the patient drops out. If the patient just skips an intermediate visit but returns for the final blood test, this would show a nonmonotone missing data pattern. A file-matching pattern occurs when variables are never observed together. This pattern can occur when data from several studies are merged and some variables are not collected in all studies. For example, three studies are merged and all three collect blood pressure, but only one study collects age and only one study collects sex.

Missing Data Mechanisms

The missing data mechanism relates to the underlying reasons for missing values and the relationships between variables with and without missing data. In general, missing data can be either random or nonrandom with distinctions in randomness made by three types: (1) data missing completely at random (MCAR); (2) data missing at random (MAR); and (3) data missing not at random (MNAR).⁶ As with the missing data pattern, understanding the missing data mechanism can aid in selecting an appropriate approach to handling the missing data.

Data are MCAR if the missingness does not depend on any study variables, meaning that all subjects are equally likely to be missing certain data elements. When the data are MCAR, those with missing values can be viewed as a simple random sample from the complete (but never actually observed) data and can be dropped from analysis without causing bias in the results. If the values of some diagnostic tests were missing for some patients due to equipment malfunction or electricity outage, for example, then the missingness may be considered MCAR.

Data are MAR if the missingness depends on the observed characteristics but not the unobserved characteristics, meaning that the relationships observed in the data can be used to predict the occurrence of missing values. Because the “randomness” of MAR is conditional on observed characteristics, which distinguishes it from the “completely at random” type of MCAR, dropping or omitting those cases with missing values from the analysis may lead to biased results.⁷ In a study of quality of life (QOL) for patients with mild to moderate traumatic brain injury, if health-related QOL questions were not answered by some patients with high pain levels (even though the pain levels were recorded), the missingness of QOL may be considered as MAR. This is due to the fact that within subjects grouped by the observed characteristic of pain (that is, conditional on similar levels of pain) the missingness of QOL is the result of chance and does not depend on the values (observed or unobserved) of QOL. It follows then, that once grouped into a high (or low) pain stratum, if QOL is considered MAR, then, whether or not it is observed, is random.

Data are considered MNAR if their missingness depends on characteristics that are not observed and cannot be fully explained by the observed characteristics. Systematic differences between missing and nonmissing data exist for data that is MNAR. For example, if a survey of household income had an increased probability of missing incomes from the low-income families then the data would be considered as MNAR.

Randomness in the missing data mechanism may be ignored without affecting the inference in some circumstances.⁸ Both MCAR and MAR can be considered as “ignorable” in the sense that a proper method (eg, multiple imputation) may recover the missing information without modeling (ie, accounting for) the random process of the missing data mechanism (Table).⁹ In contrast, the MNAR mechanism requires a method that takes into account the missing data mechanism in order to make inferences about the complete (and partially unobserved) data; or in other words, a model for the missing data mechanism cannot be ignored. It is for this reason that the MNAR mechanism is often called “nonignorable”. Nonignorable missing data present a challenge to researchers because the mechanism underlying the missingness must be included in the analysis. Yet researchers rarely know what the missingness mechanism is, and the data needed to validate any putative mechanism is, in fact, missing. In cases when more than one variable is subject to missingness, researchers need to assess the missingness mechanism for each variable and tailor their approach to the specific missing data problems.⁹

There is no universally accepted standard to guide when statistical methods should be applied to account for missing data. The amount of missing data alone cannot fully assess the missing data problem; missing data patterns and mechanisms can have greater impact on research results than the proportion of missing data alone. A good statistical method for handling missing data should provide an unbiased estimate of the quantity that the investigators intend to estimate; make use of the partial information in the incomplete cases to improve efficiency (and in most cases also to reduce bias); and provide valid estimates of the standard errors, confidence intervals, and P values for statistical tests. There are generally four broadly defined classes of methods for handling missing data in clinical research: (1) the complete-case analysis, (2) single imputation methods, (3) the weighted estimating-equation approach, and (4) the model-based approach including maximum likelihood (ML) and multiple imputation (Table and Appendix).¹⁰

Since missing data mechanisms cannot be conclusively verified, it is good practice to conduct some sensitivity analyses to test the robustness of the primary results. For this purpose, pattern-mixture models provide a flexible framework for implementing sensitivity analyses to missing data assumptions and can be used to evaluate the possibility of the data being MNAR. In this framework, the missing data distribution is modeled and then incorporated into the outcome model of interest. Tipping-point analysis is a sensitivity analysis where the missing data is replaced with a range of values to determine how much the values must change for the results of the study to tip from significant to not significant. If the same general conclusions remain valid over a range of assumptions about the missing data values, then one can have greater confidence in the study conclusions.

In dealing with missing data from clinical research, clinicians and statisticians need to work together to minimize missingness at the data collection stage, document the reasons for missingness, use substantive knowledge, if possible, to assess the missing data mechanism, perform primary analysis based on a defensible missing data mechanism, and conduct a sensitivity analysis to assess whether the primary result is robust despite departure from the assumed missing data mechanism.

Acknowledgments

The following members of the Journal of Hospital Medicine Leadership team contributed to this review: Mel L. Anderson, MD; Peter Cram, MD, MBA; JoAnna K. Leyenaar, MD, PhD, MPH; Brian P. Lucas, MD, MS; Oanh Nguyen, MD, MAS; Samir S. Shah, MD, MSCE; Erin E. Shaughnessy, MD, MSHCM; and Heidi J. Sucharew, PhD.

Research, in the field of Hospital Medicine, often leverages data collected for reasons other than research. For example, electronic medical record data or patient satisfaction survey results can be used to answer questions that are relevant to the practice of hospital medicine. In these types of datasets, data will inevitably be missing. Such missing data can compromise our ability to draw definitive conclusions from our research study. This review introduces the concept of missing data, describes patterns and mechanisms of missing data, and discusses common approaches for the handling of missing data, including sensitivity analyses for determining how robust the results are despite assumptions made about the missing data.

Missing data create a host of problems for researchers. First, missing data result in a loss of information and can diminish the power of the proposed study. Second, the irregular data complicate the analysis because many of the standard software procedures used have been developed for fully observed or “complete” data (ie, each subject has a value for all measures of interest). Finally, missing data may introduce bias due to the systematic difference between the observed and the unobserved data. For example, if men are less likely than women to complete all questions in a patient satisfaction survey when they are not satisfied, then hospital satisfaction analyses that rely on completed surveys would tend to provide biased estimates of the satisfaction males have with their care.

The ideal approach to mitigating problems caused by missing data is to anticipate and incorporate strategies to minimize missing data into the study design (ie, when planning data collection protocols for prospective studies). This plan should provide strategies for minimizing nonresponse and estimating the magnitude of anticipated missing data to ensure that the study achieves sufficient strength despite the missing data.

Strategies for minimizing nonresponse include (1) informing potential study participants, at initial contact, about the implications of missing data on the ability to answer the research question; (2) collecting several phone numbers, addresses, preferred method of contact and calling times, as well as an alternative contact, in case the primary study contact is unable to be reached; (3) specifying the number of call backs, as well as the time of contact; and (4) piloting data capture questions for phrasing, clarity, and sensitivity, in order to resolve problems before initiating the study. One approach that can be used to mitigate the impact of missing data in surveys is to contact a sample of the initial nonrespondents using a more intensive follow-up approach (eg, a nonresponse to a mailed survey is followed up by a telephone call in order to conduct the survey again over the phone), and this is referred to as “nonresponse two-phase sampling.” The additional data, captured in the second phase, not only reduces the nonresponse rate but can also provide important information on the missing data mechanism.^1,2 In longitudinal studies with dropouts, one can measure participants’ intent to drop out in order to evaluate how much the probability of dropping out depends on missing responses.³ One may also choose to determine the power and implications of sample size under different missing data assumptions.⁴

Different data sources are likely to have unique reasons for missing values due to the workflows involved in how the data are collected. In research involving the use of data from electronic medical records, missing data on specific diagnoses involving patients who are regularly engaged in care are often considered to be “not present” or “normal”, since clinical documentation workflows are largely governed by the concept of “documentation by exception” in which diagnoses are documented only when there is an exception to the expectation that these are not present. For example, “diabetes mellitus” is commonly documented, but “diabetes mellitus not present” is rarely documented in electronic medical records which are used for clinical care. Thus, lack of explicit documentation is likely to indicate that diabetes mellitus is, in fact, not present.

Certain variables may be missing simply because there is no quantifiable value­—ie, the data do not exist. Structural missingness refers to a value that does not exist for a logical reason (eg, “What is the gender of your first child?” for those who do not have a child). Censoring, which occurs during “time to event” analysis, refers to a situation where information about a subject stops before the event of interest happens, for example, when a subject in a study involving a 30-day outcome dies at day 14. The term “limit of detection” refers to the lowest or highest level at which two distinct values can reasonably be distinguished (eg, the lower limit of detection of a C-reactive protein assay may be 1 mg/dL, so lower values might simply be reported by the lab as <1 mg/dL).⁵ These types of missing data require specific methods that are not discussed in this review.

These examples illustrate that approaches to dealing with missing data vary depending on what data sources are used and how data are collected. Understanding the reasons missing data are present is a necessary step in formulating a robust analytic approach to handling missing data.

Missing Data Patterns

Evaluating missing data patterns provides information on the degree and complexity of the missing data problem and can aid in choosing an appropriate missing data handling method. This is because some analytic methods work well for a general pattern (nonmonotone) and other methods work for special patterns (eg, monotone, file matching). In longitudinal studies, missing data is commonly missing in a monotone pattern, where once one variable is missing then all subsequent variables are also missing for a particular subject. This occurs when a study participant is lost to follow-up. For example, a monotone missing data pattern may occur in a study that requires a series of follow-up visits for laboratory blood tests. If a patient drops out, it results in a monotone missing data pattern, as no data on blood test results are available once the patient drops out. If the patient just skips an intermediate visit but returns for the final blood test, this would show a nonmonotone missing data pattern. A file-matching pattern occurs when variables are never observed together. This pattern can occur when data from several studies are merged and some variables are not collected in all studies. For example, three studies are merged and all three collect blood pressure, but only one study collects age and only one study collects sex.

Missing Data Mechanisms

The missing data mechanism relates to the underlying reasons for missing values and the relationships between variables with and without missing data. In general, missing data can be either random or nonrandom with distinctions in randomness made by three types: (1) data missing completely at random (MCAR); (2) data missing at random (MAR); and (3) data missing not at random (MNAR).⁶ As with the missing data pattern, understanding the missing data mechanism can aid in selecting an appropriate approach to handling the missing data.

Data are MCAR if the missingness does not depend on any study variables, meaning that all subjects are equally likely to be missing certain data elements. When the data are MCAR, those with missing values can be viewed as a simple random sample from the complete (but never actually observed) data and can be dropped from analysis without causing bias in the results. If the values of some diagnostic tests were missing for some patients due to equipment malfunction or electricity outage, for example, then the missingness may be considered MCAR.

Data are MAR if the missingness depends on the observed characteristics but not the unobserved characteristics, meaning that the relationships observed in the data can be used to predict the occurrence of missing values. Because the “randomness” of MAR is conditional on observed characteristics, which distinguishes it from the “completely at random” type of MCAR, dropping or omitting those cases with missing values from the analysis may lead to biased results.⁷ In a study of quality of life (QOL) for patients with mild to moderate traumatic brain injury, if health-related QOL questions were not answered by some patients with high pain levels (even though the pain levels were recorded), the missingness of QOL may be considered as MAR. This is due to the fact that within subjects grouped by the observed characteristic of pain (that is, conditional on similar levels of pain) the missingness of QOL is the result of chance and does not depend on the values (observed or unobserved) of QOL. It follows then, that once grouped into a high (or low) pain stratum, if QOL is considered MAR, then, whether or not it is observed, is random.

Data are considered MNAR if their missingness depends on characteristics that are not observed and cannot be fully explained by the observed characteristics. Systematic differences between missing and nonmissing data exist for data that is MNAR. For example, if a survey of household income had an increased probability of missing incomes from the low-income families then the data would be considered as MNAR.

Randomness in the missing data mechanism may be ignored without affecting the inference in some circumstances.⁸ Both MCAR and MAR can be considered as “ignorable” in the sense that a proper method (eg, multiple imputation) may recover the missing information without modeling (ie, accounting for) the random process of the missing data mechanism (Table).⁹ In contrast, the MNAR mechanism requires a method that takes into account the missing data mechanism in order to make inferences about the complete (and partially unobserved) data; or in other words, a model for the missing data mechanism cannot be ignored. It is for this reason that the MNAR mechanism is often called “nonignorable”. Nonignorable missing data present a challenge to researchers because the mechanism underlying the missingness must be included in the analysis. Yet researchers rarely know what the missingness mechanism is, and the data needed to validate any putative mechanism is, in fact, missing. In cases when more than one variable is subject to missingness, researchers need to assess the missingness mechanism for each variable and tailor their approach to the specific missing data problems.⁹

There is no universally accepted standard to guide when statistical methods should be applied to account for missing data. The amount of missing data alone cannot fully assess the missing data problem; missing data patterns and mechanisms can have greater impact on research results than the proportion of missing data alone. A good statistical method for handling missing data should provide an unbiased estimate of the quantity that the investigators intend to estimate; make use of the partial information in the incomplete cases to improve efficiency (and in most cases also to reduce bias); and provide valid estimates of the standard errors, confidence intervals, and P values for statistical tests. There are generally four broadly defined classes of methods for handling missing data in clinical research: (1) the complete-case analysis, (2) single imputation methods, (3) the weighted estimating-equation approach, and (4) the model-based approach including maximum likelihood (ML) and multiple imputation (Table and Appendix).¹⁰

Since missing data mechanisms cannot be conclusively verified, it is good practice to conduct some sensitivity analyses to test the robustness of the primary results. For this purpose, pattern-mixture models provide a flexible framework for implementing sensitivity analyses to missing data assumptions and can be used to evaluate the possibility of the data being MNAR. In this framework, the missing data distribution is modeled and then incorporated into the outcome model of interest. Tipping-point analysis is a sensitivity analysis where the missing data is replaced with a range of values to determine how much the values must change for the results of the study to tip from significant to not significant. If the same general conclusions remain valid over a range of assumptions about the missing data values, then one can have greater confidence in the study conclusions.

In dealing with missing data from clinical research, clinicians and statisticians need to work together to minimize missingness at the data collection stage, document the reasons for missingness, use substantive knowledge, if possible, to assess the missing data mechanism, perform primary analysis based on a defensible missing data mechanism, and conduct a sensitivity analysis to assess whether the primary result is robust despite departure from the assumed missing data mechanism.

Acknowledgments

The following members of the Journal of Hospital Medicine Leadership team contributed to this review: Mel L. Anderson, MD; Peter Cram, MD, MBA; JoAnna K. Leyenaar, MD, PhD, MPH; Brian P. Lucas, MD, MS; Oanh Nguyen, MD, MAS; Samir S. Shah, MD, MSCE; Erin E. Shaughnessy, MD, MSHCM; and Heidi J. Sucharew, PhD.

Research, in the field of Hospital Medicine, often leverages data collected for reasons other than research. For example, electronic medical record data or patient satisfaction survey results can be used to answer questions that are relevant to the practice of hospital medicine. In these types of datasets, data will inevitably be missing. Such missing data can compromise our ability to draw definitive conclusions from our research study. This review introduces the concept of missing data, describes patterns and mechanisms of missing data, and discusses common approaches for the handling of missing data, including sensitivity analyses for determining how robust the results are despite assumptions made about the missing data.

Missing data create a host of problems for researchers. First, missing data result in a loss of information and can diminish the power of the proposed study. Second, the irregular data complicate the analysis because many of the standard software procedures used have been developed for fully observed or “complete” data (ie, each subject has a value for all measures of interest). Finally, missing data may introduce bias due to the systematic difference between the observed and the unobserved data. For example, if men are less likely than women to complete all questions in a patient satisfaction survey when they are not satisfied, then hospital satisfaction analyses that rely on completed surveys would tend to provide biased estimates of the satisfaction males have with their care.

The ideal approach to mitigating problems caused by missing data is to anticipate and incorporate strategies to minimize missing data into the study design (ie, when planning data collection protocols for prospective studies). This plan should provide strategies for minimizing nonresponse and estimating the magnitude of anticipated missing data to ensure that the study achieves sufficient strength despite the missing data.

Strategies for minimizing nonresponse include (1) informing potential study participants, at initial contact, about the implications of missing data on the ability to answer the research question; (2) collecting several phone numbers, addresses, preferred method of contact and calling times, as well as an alternative contact, in case the primary study contact is unable to be reached; (3) specifying the number of call backs, as well as the time of contact; and (4) piloting data capture questions for phrasing, clarity, and sensitivity, in order to resolve problems before initiating the study. One approach that can be used to mitigate the impact of missing data in surveys is to contact a sample of the initial nonrespondents using a more intensive follow-up approach (eg, a nonresponse to a mailed survey is followed up by a telephone call in order to conduct the survey again over the phone), and this is referred to as “nonresponse two-phase sampling.” The additional data, captured in the second phase, not only reduces the nonresponse rate but can also provide important information on the missing data mechanism.^1,2 In longitudinal studies with dropouts, one can measure participants’ intent to drop out in order to evaluate how much the probability of dropping out depends on missing responses.³ One may also choose to determine the power and implications of sample size under different missing data assumptions.⁴

Different data sources are likely to have unique reasons for missing values due to the workflows involved in how the data are collected. In research involving the use of data from electronic medical records, missing data on specific diagnoses involving patients who are regularly engaged in care are often considered to be “not present” or “normal”, since clinical documentation workflows are largely governed by the concept of “documentation by exception” in which diagnoses are documented only when there is an exception to the expectation that these are not present. For example, “diabetes mellitus” is commonly documented, but “diabetes mellitus not present” is rarely documented in electronic medical records which are used for clinical care. Thus, lack of explicit documentation is likely to indicate that diabetes mellitus is, in fact, not present.

Certain variables may be missing simply because there is no quantifiable value­—ie, the data do not exist. Structural missingness refers to a value that does not exist for a logical reason (eg, “What is the gender of your first child?” for those who do not have a child). Censoring, which occurs during “time to event” analysis, refers to a situation where information about a subject stops before the event of interest happens, for example, when a subject in a study involving a 30-day outcome dies at day 14. The term “limit of detection” refers to the lowest or highest level at which two distinct values can reasonably be distinguished (eg, the lower limit of detection of a C-reactive protein assay may be 1 mg/dL, so lower values might simply be reported by the lab as <1 mg/dL).⁵ These types of missing data require specific methods that are not discussed in this review.

These examples illustrate that approaches to dealing with missing data vary depending on what data sources are used and how data are collected. Understanding the reasons missing data are present is a necessary step in formulating a robust analytic approach to handling missing data.

Missing Data Patterns

Evaluating missing data patterns provides information on the degree and complexity of the missing data problem and can aid in choosing an appropriate missing data handling method. This is because some analytic methods work well for a general pattern (nonmonotone) and other methods work for special patterns (eg, monotone, file matching). In longitudinal studies, missing data is commonly missing in a monotone pattern, where once one variable is missing then all subsequent variables are also missing for a particular subject. This occurs when a study participant is lost to follow-up. For example, a monotone missing data pattern may occur in a study that requires a series of follow-up visits for laboratory blood tests. If a patient drops out, it results in a monotone missing data pattern, as no data on blood test results are available once the patient drops out. If the patient just skips an intermediate visit but returns for the final blood test, this would show a nonmonotone missing data pattern. A file-matching pattern occurs when variables are never observed together. This pattern can occur when data from several studies are merged and some variables are not collected in all studies. For example, three studies are merged and all three collect blood pressure, but only one study collects age and only one study collects sex.

Missing Data Mechanisms

The missing data mechanism relates to the underlying reasons for missing values and the relationships between variables with and without missing data. In general, missing data can be either random or nonrandom with distinctions in randomness made by three types: (1) data missing completely at random (MCAR); (2) data missing at random (MAR); and (3) data missing not at random (MNAR).⁶ As with the missing data pattern, understanding the missing data mechanism can aid in selecting an appropriate approach to handling the missing data.

Data are MCAR if the missingness does not depend on any study variables, meaning that all subjects are equally likely to be missing certain data elements. When the data are MCAR, those with missing values can be viewed as a simple random sample from the complete (but never actually observed) data and can be dropped from analysis without causing bias in the results. If the values of some diagnostic tests were missing for some patients due to equipment malfunction or electricity outage, for example, then the missingness may be considered MCAR.

Data are MAR if the missingness depends on the observed characteristics but not the unobserved characteristics, meaning that the relationships observed in the data can be used to predict the occurrence of missing values. Because the “randomness” of MAR is conditional on observed characteristics, which distinguishes it from the “completely at random” type of MCAR, dropping or omitting those cases with missing values from the analysis may lead to biased results.⁷ In a study of quality of life (QOL) for patients with mild to moderate traumatic brain injury, if health-related QOL questions were not answered by some patients with high pain levels (even though the pain levels were recorded), the missingness of QOL may be considered as MAR. This is due to the fact that within subjects grouped by the observed characteristic of pain (that is, conditional on similar levels of pain) the missingness of QOL is the result of chance and does not depend on the values (observed or unobserved) of QOL. It follows then, that once grouped into a high (or low) pain stratum, if QOL is considered MAR, then, whether or not it is observed, is random.

Data are considered MNAR if their missingness depends on characteristics that are not observed and cannot be fully explained by the observed characteristics. Systematic differences between missing and nonmissing data exist for data that is MNAR. For example, if a survey of household income had an increased probability of missing incomes from the low-income families then the data would be considered as MNAR.

Randomness in the missing data mechanism may be ignored without affecting the inference in some circumstances.⁸ Both MCAR and MAR can be considered as “ignorable” in the sense that a proper method (eg, multiple imputation) may recover the missing information without modeling (ie, accounting for) the random process of the missing data mechanism (Table).⁹ In contrast, the MNAR mechanism requires a method that takes into account the missing data mechanism in order to make inferences about the complete (and partially unobserved) data; or in other words, a model for the missing data mechanism cannot be ignored. It is for this reason that the MNAR mechanism is often called “nonignorable”. Nonignorable missing data present a challenge to researchers because the mechanism underlying the missingness must be included in the analysis. Yet researchers rarely know what the missingness mechanism is, and the data needed to validate any putative mechanism is, in fact, missing. In cases when more than one variable is subject to missingness, researchers need to assess the missingness mechanism for each variable and tailor their approach to the specific missing data problems.⁹

There is no universally accepted standard to guide when statistical methods should be applied to account for missing data. The amount of missing data alone cannot fully assess the missing data problem; missing data patterns and mechanisms can have greater impact on research results than the proportion of missing data alone. A good statistical method for handling missing data should provide an unbiased estimate of the quantity that the investigators intend to estimate; make use of the partial information in the incomplete cases to improve efficiency (and in most cases also to reduce bias); and provide valid estimates of the standard errors, confidence intervals, and P values for statistical tests. There are generally four broadly defined classes of methods for handling missing data in clinical research: (1) the complete-case analysis, (2) single imputation methods, (3) the weighted estimating-equation approach, and (4) the model-based approach including maximum likelihood (ML) and multiple imputation (Table and Appendix).¹⁰

Since missing data mechanisms cannot be conclusively verified, it is good practice to conduct some sensitivity analyses to test the robustness of the primary results. For this purpose, pattern-mixture models provide a flexible framework for implementing sensitivity analyses to missing data assumptions and can be used to evaluate the possibility of the data being MNAR. In this framework, the missing data distribution is modeled and then incorporated into the outcome model of interest. Tipping-point analysis is a sensitivity analysis where the missing data is replaced with a range of values to determine how much the values must change for the results of the study to tip from significant to not significant. If the same general conclusions remain valid over a range of assumptions about the missing data values, then one can have greater confidence in the study conclusions.

In dealing with missing data from clinical research, clinicians and statisticians need to work together to minimize missingness at the data collection stage, document the reasons for missingness, use substantive knowledge, if possible, to assess the missing data mechanism, perform primary analysis based on a defensible missing data mechanism, and conduct a sensitivity analysis to assess whether the primary result is robust despite departure from the assumed missing data mechanism.

Acknowledgments

The following members of the Journal of Hospital Medicine Leadership team contributed to this review: Mel L. Anderson, MD; Peter Cram, MD, MBA; JoAnna K. Leyenaar, MD, PhD, MPH; Brian P. Lucas, MD, MS; Oanh Nguyen, MD, MAS; Samir S. Shah, MD, MSCE; Erin E. Shaughnessy, MD, MSHCM; and Heidi J. Sucharew, PhD.

Hilda and I shared childhood stories while we enjoyed one of her favorite Mexican dishes, grilled nopalitos (cactus). Hilda loved nopalitos, but she rarely ate them because they are high in potassium. Hilda had end-stage kidney disease (ESKD), and as an undocumented Mexican immigrant in Denver, CO, she relied on emergency-only hemodialysis. Instead of receiving standard hemodialysis three times per week as required, Hilda would arrive critically ill to the hospital after her nausea, vomiting, and shortness of breath became unbearable. After three cardiac arrests from high potassium levels, she fervently avoided foods high in it. This time, however, she was not worried about potassium. This was our last meal together. She would fly to Mexico a few days later to die.

Our hospital medicine team knew Hilda well. We had continuity because we had been admitting her to the intensive care unit or medicine floor one night each week to receive two hemodialysis sessions when she was critically ill. I immediately connected with Hilda because our lives were parallel in many ways. Hilda and I were both in our early 30s, English was our second language, we both grew up in poverty, and we now had children in elementary school. I, however, was documented. My United States citizenship allowed me the privilege of pursuing a medical degree and gaining access to quality healthcare. In contrast, Hilda had been forced to end her education prematurely, marry her mother’s friend for financial stability at the age of 14, and eventually flee to the US to escape poverty. She survived by cleaning homes until her kidneys failed. Initially, Hilda was my patient. Over time, she became a dear friend.

The first two years of emergency-only hemodialysis devastated Hilda. Too sick to work, she became homeless, staying with a nurse until we found a shelter for single mothers. Multiple cardiac arrests and resuscitations traumatized her young sons, who called 911 each time she collapsed and witnessed the resuscitations. Her boys did not understand the cycle of separation from their mother for her emergent, weekly dialysis hospital admissions and wondered if she would survive to the following week. After two years of emergency-only dialysis, Hilda’s deep love for her boys and concern about the possibility that her sudden death could leave them alone led her to pre-emptively decide to stop emergency-only dialysis. Had Hilda’s treatment costs been covered by emergency Medicaid, as undocumented immigrants with ESKD are in some other states, she may not have been forced into this terrible decision. Moving to a state where standard dialysis is covered was not an option for Hilda because she wanted her boys to stay in Colorado where they had family and friends. With no other options, she first sought a loving adoptive family in the US so that her boys could grow up and have the opportunity to pursue an education. After carefully finding the right adoptive parents, Hilda wanted to celebrate her life with the people she loved. To show her gratitude, she organized a large Mexican Christmas party and invited all of the healthcare providers and friends that had supported her. She generously gave everyone a small gift to remember her by from the few things she owned. I received the wooden rosary her father had left her. A short while later, Hilda flew home to Mexico and passed away on Mother’s Day in 2014.

Two years of caring for Hilda as an internal medicine hospitalist changed me. Grief gave way to anger, anger to determination. I found it morally distressing to continue to provide this type of care. Something had to change and there was little research in this area. One small study had demonstrated that emergency-only hemodialysis was nearly four-fold more expensive due to additional visits to the emergency department and admissions to the hospital, compared to standard outpatient hemodialysis.¹ After much soul-searching and advice seeking, I scaled down my clinical hospitalist shifts and gathered a team to do research. For four years, we worked on illuminating the suffering of undocumented immigrants with ESKD that rely on emergency-only hemodialysis. We conducted 20 individual face-to-face qualitative interviews with undocumented immigrants with ESKD and heard first-hand about the emotional and physical burdens and the existential anxiety associated with weekly threats to life.² We published a retrospective cohort study looking at differences in mortality and found that immigrants who relied on emergency-only hemodialysis had a 14-fold greater mortality rate than those on standard hemodialysis five years after initiating hemodialysis.³ In another retrospective study, we described the circumstances among undocumented immigrants with ESKD who died in the hospital after presenting with ESKD complications, and found that the majority presented with high potassium and a recorded rhythm disturbance.⁴ I discovered that as a hospitalist physician, I was not the only one distressed. We conducted 50 qualitative interviews to determine the perspectives of interdisciplinary clinicians on providing emergency dialysis and found that there are more clinicians experiencing moral distress. They described several important drivers of burnout,⁵ including emotional exhaustion from witnessing needless suffering and high mortality, as well as physical exhaustion from overextending themselves to bridge their patient’s care. Together, we discovered that the research told the larger narrative behind Hilda’s struggles. These publications caught the attention of the media and enabled us to speak to a wider audience of clinicians, health policy makers, and the general public.^6-10 They also became a catalyst to engaging and enlisting the good will and interest of a number of key stakeholders to look for solutions.

In the US, undocumented immigrants do not qualify for insurance through traditional Medicaid, Medicare, or the provisions from the Patient Protection and Affordable Care Act. Instead, emergency Medicaid provides reimbursements for care of undocumented immigrants. According to the 1986 Emergency Medicaid Treatment and Active Labor Act, federal Medicaid payments can only be made for the care of undocumented immigrants if care is necessary for the treatment of an emergency medical condition.¹¹ However, the Centers for Medicare and Medicaid (CMS) has outlined certain conditions that cannot qualify for matching federal funds under emergency Medicaid (ie, organ transplant and routine prenatal or postpartum care). Beyond these requirements, federal CMS and the Office of the Inspector General defer to states to define what constitutes a medical emergency. A few states include ESKD in the definition of “emergency medical condition,” thereby expanding access to standard hemodialysis to undocumented immigrants. We wanted Colorado to join that list.

On August 2018, after four years of research and months of dialog, everything changed: Colorado Medicaid announced that ESKD was now an “emergency medical condition.” As simple as that, undocumented immigrants would receive standard maintenance hemodialysis. Tears streamed down my face as I read a message from a policy specialist from the Colorado Medicaid: Your team “played a big role in bringing awareness to this issue, and your advocacy for these patients is impressive … thank you for fighting for such an important cause.” I reread her message, imagining what this would have meant to Hilda and her boys.

Our work to enhance care in this community is not over. To better understand the provision of dialysis care for undocumented immigrants in the United States, our team reviewed the Medicaid language for each of the 50 US states in addition to connecting with clinicians and organizations (eg, National Kidney Foundation and ESKD Networks). We found that only 12 states provide Medicaid reimbursement for standard dialysis and that a majority of the US states do not currently define need for dialysis as an emergency medical condition.¹² As our Colorado team works with stakeholders in other states interested in similarly redefining their state’s emergency Medicaid definition, our most important advice is that advocacy is a team-based effort. There may be resistance and some may argue that expanding access to care would be an economic burden on taxpayers; however, research demonstrates that undocumented immigrants contribute more to the US Medicare Trust Fund than they actually withdraw toward healthcare.¹³ Furthermore, a new study has demonstrated that a net savings of nearly $6,000 per person per month is realized when patients are transitioned from emergency-only hemodialysis to standard hemodialysis.¹⁴

Internal medicine hospitalists on the front-line of healthcare systems are regular witnesses to its horrible injustices. We rarely share our perspectives and do not expect change to follow. With Hilda, we saw how a powerful combination of research and coalition building could lift one patient’s tragic story to a level where it could produce change. Augmenting Hilda’s experience of tragically poor access to care with evidence-based research gave her story validity far beyond our immediate circle of friends and colleagues, making a singular tragedy, policy relevant. Each time we shared our research to community advocacy groups, health policy stakeholders, state legislators, nurses, and staff; we began with Hilda’s story, not just because it inspired us, but because its truth was undeniable. Our patients’ stories matter, and it is our responsibility to tell them.

Each time I prepare nopalitos for my family, I think of my last meal with Hilda. No matter how painful or difficult her struggle with ESKD, Hilda persisted. She protected her boys. They were her purpose. When she knew she could no longer give them the life she wanted for them, she found a family who would. Hilda’s sons now live with a loving adoptive family, are thriving in school, and her oldest is interested in becoming a physician. Nopal, or cactus, symbolizes such endurance—a plant with unique adaptations and strength that can flourish under extreme environmental stress. Like a cactus storing precious water, Hilda treasured her children, and her resolve to provide for them was unstoppable, right to the edge of death. When our team first took up Hilda’s cause, change seemed impossible. We discovered the opposite. As I clench the wooden rosary she left me that Christmas, I thank her for giving our team the courage to adapt and persist, for in doing so we found a path, first to research and then to broader partnerships and more meaningful policy changes.

Acknowledgments

The author would like to thank Hilda, her family, and the patients at Denver Health. She would also like to acknowledge Hilda’s family, Drs. Mark Earnest, John F. Steiner, Romana Hasnain-Wynia, Rudolph Rodriguez, Judy Regensteiner, and Michel Chonchol for reading and providing feedback on earlier drafts of this narrative.

Hilda and I shared childhood stories while we enjoyed one of her favorite Mexican dishes, grilled nopalitos (cactus). Hilda loved nopalitos, but she rarely ate them because they are high in potassium. Hilda had end-stage kidney disease (ESKD), and as an undocumented Mexican immigrant in Denver, CO, she relied on emergency-only hemodialysis. Instead of receiving standard hemodialysis three times per week as required, Hilda would arrive critically ill to the hospital after her nausea, vomiting, and shortness of breath became unbearable. After three cardiac arrests from high potassium levels, she fervently avoided foods high in it. This time, however, she was not worried about potassium. This was our last meal together. She would fly to Mexico a few days later to die.

Our hospital medicine team knew Hilda well. We had continuity because we had been admitting her to the intensive care unit or medicine floor one night each week to receive two hemodialysis sessions when she was critically ill. I immediately connected with Hilda because our lives were parallel in many ways. Hilda and I were both in our early 30s, English was our second language, we both grew up in poverty, and we now had children in elementary school. I, however, was documented. My United States citizenship allowed me the privilege of pursuing a medical degree and gaining access to quality healthcare. In contrast, Hilda had been forced to end her education prematurely, marry her mother’s friend for financial stability at the age of 14, and eventually flee to the US to escape poverty. She survived by cleaning homes until her kidneys failed. Initially, Hilda was my patient. Over time, she became a dear friend.

The first two years of emergency-only hemodialysis devastated Hilda. Too sick to work, she became homeless, staying with a nurse until we found a shelter for single mothers. Multiple cardiac arrests and resuscitations traumatized her young sons, who called 911 each time she collapsed and witnessed the resuscitations. Her boys did not understand the cycle of separation from their mother for her emergent, weekly dialysis hospital admissions and wondered if she would survive to the following week. After two years of emergency-only dialysis, Hilda’s deep love for her boys and concern about the possibility that her sudden death could leave them alone led her to pre-emptively decide to stop emergency-only dialysis. Had Hilda’s treatment costs been covered by emergency Medicaid, as undocumented immigrants with ESKD are in some other states, she may not have been forced into this terrible decision. Moving to a state where standard dialysis is covered was not an option for Hilda because she wanted her boys to stay in Colorado where they had family and friends. With no other options, she first sought a loving adoptive family in the US so that her boys could grow up and have the opportunity to pursue an education. After carefully finding the right adoptive parents, Hilda wanted to celebrate her life with the people she loved. To show her gratitude, she organized a large Mexican Christmas party and invited all of the healthcare providers and friends that had supported her. She generously gave everyone a small gift to remember her by from the few things she owned. I received the wooden rosary her father had left her. A short while later, Hilda flew home to Mexico and passed away on Mother’s Day in 2014.

Two years of caring for Hilda as an internal medicine hospitalist changed me. Grief gave way to anger, anger to determination. I found it morally distressing to continue to provide this type of care. Something had to change and there was little research in this area. One small study had demonstrated that emergency-only hemodialysis was nearly four-fold more expensive due to additional visits to the emergency department and admissions to the hospital, compared to standard outpatient hemodialysis.¹ After much soul-searching and advice seeking, I scaled down my clinical hospitalist shifts and gathered a team to do research. For four years, we worked on illuminating the suffering of undocumented immigrants with ESKD that rely on emergency-only hemodialysis. We conducted 20 individual face-to-face qualitative interviews with undocumented immigrants with ESKD and heard first-hand about the emotional and physical burdens and the existential anxiety associated with weekly threats to life.² We published a retrospective cohort study looking at differences in mortality and found that immigrants who relied on emergency-only hemodialysis had a 14-fold greater mortality rate than those on standard hemodialysis five years after initiating hemodialysis.³ In another retrospective study, we described the circumstances among undocumented immigrants with ESKD who died in the hospital after presenting with ESKD complications, and found that the majority presented with high potassium and a recorded rhythm disturbance.⁴ I discovered that as a hospitalist physician, I was not the only one distressed. We conducted 50 qualitative interviews to determine the perspectives of interdisciplinary clinicians on providing emergency dialysis and found that there are more clinicians experiencing moral distress. They described several important drivers of burnout,⁵ including emotional exhaustion from witnessing needless suffering and high mortality, as well as physical exhaustion from overextending themselves to bridge their patient’s care. Together, we discovered that the research told the larger narrative behind Hilda’s struggles. These publications caught the attention of the media and enabled us to speak to a wider audience of clinicians, health policy makers, and the general public.^6-10 They also became a catalyst to engaging and enlisting the good will and interest of a number of key stakeholders to look for solutions.

In the US, undocumented immigrants do not qualify for insurance through traditional Medicaid, Medicare, or the provisions from the Patient Protection and Affordable Care Act. Instead, emergency Medicaid provides reimbursements for care of undocumented immigrants. According to the 1986 Emergency Medicaid Treatment and Active Labor Act, federal Medicaid payments can only be made for the care of undocumented immigrants if care is necessary for the treatment of an emergency medical condition.¹¹ However, the Centers for Medicare and Medicaid (CMS) has outlined certain conditions that cannot qualify for matching federal funds under emergency Medicaid (ie, organ transplant and routine prenatal or postpartum care). Beyond these requirements, federal CMS and the Office of the Inspector General defer to states to define what constitutes a medical emergency. A few states include ESKD in the definition of “emergency medical condition,” thereby expanding access to standard hemodialysis to undocumented immigrants. We wanted Colorado to join that list.

On August 2018, after four years of research and months of dialog, everything changed: Colorado Medicaid announced that ESKD was now an “emergency medical condition.” As simple as that, undocumented immigrants would receive standard maintenance hemodialysis. Tears streamed down my face as I read a message from a policy specialist from the Colorado Medicaid: Your team “played a big role in bringing awareness to this issue, and your advocacy for these patients is impressive … thank you for fighting for such an important cause.” I reread her message, imagining what this would have meant to Hilda and her boys.

Our work to enhance care in this community is not over. To better understand the provision of dialysis care for undocumented immigrants in the United States, our team reviewed the Medicaid language for each of the 50 US states in addition to connecting with clinicians and organizations (eg, National Kidney Foundation and ESKD Networks). We found that only 12 states provide Medicaid reimbursement for standard dialysis and that a majority of the US states do not currently define need for dialysis as an emergency medical condition.¹² As our Colorado team works with stakeholders in other states interested in similarly redefining their state’s emergency Medicaid definition, our most important advice is that advocacy is a team-based effort. There may be resistance and some may argue that expanding access to care would be an economic burden on taxpayers; however, research demonstrates that undocumented immigrants contribute more to the US Medicare Trust Fund than they actually withdraw toward healthcare.¹³ Furthermore, a new study has demonstrated that a net savings of nearly $6,000 per person per month is realized when patients are transitioned from emergency-only hemodialysis to standard hemodialysis.¹⁴

Internal medicine hospitalists on the front-line of healthcare systems are regular witnesses to its horrible injustices. We rarely share our perspectives and do not expect change to follow. With Hilda, we saw how a powerful combination of research and coalition building could lift one patient’s tragic story to a level where it could produce change. Augmenting Hilda’s experience of tragically poor access to care with evidence-based research gave her story validity far beyond our immediate circle of friends and colleagues, making a singular tragedy, policy relevant. Each time we shared our research to community advocacy groups, health policy stakeholders, state legislators, nurses, and staff; we began with Hilda’s story, not just because it inspired us, but because its truth was undeniable. Our patients’ stories matter, and it is our responsibility to tell them.

Each time I prepare nopalitos for my family, I think of my last meal with Hilda. No matter how painful or difficult her struggle with ESKD, Hilda persisted. She protected her boys. They were her purpose. When she knew she could no longer give them the life she wanted for them, she found a family who would. Hilda’s sons now live with a loving adoptive family, are thriving in school, and her oldest is interested in becoming a physician. Nopal, or cactus, symbolizes such endurance—a plant with unique adaptations and strength that can flourish under extreme environmental stress. Like a cactus storing precious water, Hilda treasured her children, and her resolve to provide for them was unstoppable, right to the edge of death. When our team first took up Hilda’s cause, change seemed impossible. We discovered the opposite. As I clench the wooden rosary she left me that Christmas, I thank her for giving our team the courage to adapt and persist, for in doing so we found a path, first to research and then to broader partnerships and more meaningful policy changes.

Acknowledgments

The author would like to thank Hilda, her family, and the patients at Denver Health. She would also like to acknowledge Hilda’s family, Drs. Mark Earnest, John F. Steiner, Romana Hasnain-Wynia, Rudolph Rodriguez, Judy Regensteiner, and Michel Chonchol for reading and providing feedback on earlier drafts of this narrative.

Hilda and I shared childhood stories while we enjoyed one of her favorite Mexican dishes, grilled nopalitos (cactus). Hilda loved nopalitos, but she rarely ate them because they are high in potassium. Hilda had end-stage kidney disease (ESKD), and as an undocumented Mexican immigrant in Denver, CO, she relied on emergency-only hemodialysis. Instead of receiving standard hemodialysis three times per week as required, Hilda would arrive critically ill to the hospital after her nausea, vomiting, and shortness of breath became unbearable. After three cardiac arrests from high potassium levels, she fervently avoided foods high in it. This time, however, she was not worried about potassium. This was our last meal together. She would fly to Mexico a few days later to die.

Our hospital medicine team knew Hilda well. We had continuity because we had been admitting her to the intensive care unit or medicine floor one night each week to receive two hemodialysis sessions when she was critically ill. I immediately connected with Hilda because our lives were parallel in many ways. Hilda and I were both in our early 30s, English was our second language, we both grew up in poverty, and we now had children in elementary school. I, however, was documented. My United States citizenship allowed me the privilege of pursuing a medical degree and gaining access to quality healthcare. In contrast, Hilda had been forced to end her education prematurely, marry her mother’s friend for financial stability at the age of 14, and eventually flee to the US to escape poverty. She survived by cleaning homes until her kidneys failed. Initially, Hilda was my patient. Over time, she became a dear friend.

The first two years of emergency-only hemodialysis devastated Hilda. Too sick to work, she became homeless, staying with a nurse until we found a shelter for single mothers. Multiple cardiac arrests and resuscitations traumatized her young sons, who called 911 each time she collapsed and witnessed the resuscitations. Her boys did not understand the cycle of separation from their mother for her emergent, weekly dialysis hospital admissions and wondered if she would survive to the following week. After two years of emergency-only dialysis, Hilda’s deep love for her boys and concern about the possibility that her sudden death could leave them alone led her to pre-emptively decide to stop emergency-only dialysis. Had Hilda’s treatment costs been covered by emergency Medicaid, as undocumented immigrants with ESKD are in some other states, she may not have been forced into this terrible decision. Moving to a state where standard dialysis is covered was not an option for Hilda because she wanted her boys to stay in Colorado where they had family and friends. With no other options, she first sought a loving adoptive family in the US so that her boys could grow up and have the opportunity to pursue an education. After carefully finding the right adoptive parents, Hilda wanted to celebrate her life with the people she loved. To show her gratitude, she organized a large Mexican Christmas party and invited all of the healthcare providers and friends that had supported her. She generously gave everyone a small gift to remember her by from the few things she owned. I received the wooden rosary her father had left her. A short while later, Hilda flew home to Mexico and passed away on Mother’s Day in 2014.

Two years of caring for Hilda as an internal medicine hospitalist changed me. Grief gave way to anger, anger to determination. I found it morally distressing to continue to provide this type of care. Something had to change and there was little research in this area. One small study had demonstrated that emergency-only hemodialysis was nearly four-fold more expensive due to additional visits to the emergency department and admissions to the hospital, compared to standard outpatient hemodialysis.¹ After much soul-searching and advice seeking, I scaled down my clinical hospitalist shifts and gathered a team to do research. For four years, we worked on illuminating the suffering of undocumented immigrants with ESKD that rely on emergency-only hemodialysis. We conducted 20 individual face-to-face qualitative interviews with undocumented immigrants with ESKD and heard first-hand about the emotional and physical burdens and the existential anxiety associated with weekly threats to life.² We published a retrospective cohort study looking at differences in mortality and found that immigrants who relied on emergency-only hemodialysis had a 14-fold greater mortality rate than those on standard hemodialysis five years after initiating hemodialysis.³ In another retrospective study, we described the circumstances among undocumented immigrants with ESKD who died in the hospital after presenting with ESKD complications, and found that the majority presented with high potassium and a recorded rhythm disturbance.⁴ I discovered that as a hospitalist physician, I was not the only one distressed. We conducted 50 qualitative interviews to determine the perspectives of interdisciplinary clinicians on providing emergency dialysis and found that there are more clinicians experiencing moral distress. They described several important drivers of burnout,⁵ including emotional exhaustion from witnessing needless suffering and high mortality, as well as physical exhaustion from overextending themselves to bridge their patient’s care. Together, we discovered that the research told the larger narrative behind Hilda’s struggles. These publications caught the attention of the media and enabled us to speak to a wider audience of clinicians, health policy makers, and the general public.^6-10 They also became a catalyst to engaging and enlisting the good will and interest of a number of key stakeholders to look for solutions.

In the US, undocumented immigrants do not qualify for insurance through traditional Medicaid, Medicare, or the provisions from the Patient Protection and Affordable Care Act. Instead, emergency Medicaid provides reimbursements for care of undocumented immigrants. According to the 1986 Emergency Medicaid Treatment and Active Labor Act, federal Medicaid payments can only be made for the care of undocumented immigrants if care is necessary for the treatment of an emergency medical condition.¹¹ However, the Centers for Medicare and Medicaid (CMS) has outlined certain conditions that cannot qualify for matching federal funds under emergency Medicaid (ie, organ transplant and routine prenatal or postpartum care). Beyond these requirements, federal CMS and the Office of the Inspector General defer to states to define what constitutes a medical emergency. A few states include ESKD in the definition of “emergency medical condition,” thereby expanding access to standard hemodialysis to undocumented immigrants. We wanted Colorado to join that list.

On August 2018, after four years of research and months of dialog, everything changed: Colorado Medicaid announced that ESKD was now an “emergency medical condition.” As simple as that, undocumented immigrants would receive standard maintenance hemodialysis. Tears streamed down my face as I read a message from a policy specialist from the Colorado Medicaid: Your team “played a big role in bringing awareness to this issue, and your advocacy for these patients is impressive … thank you for fighting for such an important cause.” I reread her message, imagining what this would have meant to Hilda and her boys.

Our work to enhance care in this community is not over. To better understand the provision of dialysis care for undocumented immigrants in the United States, our team reviewed the Medicaid language for each of the 50 US states in addition to connecting with clinicians and organizations (eg, National Kidney Foundation and ESKD Networks). We found that only 12 states provide Medicaid reimbursement for standard dialysis and that a majority of the US states do not currently define need for dialysis as an emergency medical condition.¹² As our Colorado team works with stakeholders in other states interested in similarly redefining their state’s emergency Medicaid definition, our most important advice is that advocacy is a team-based effort. There may be resistance and some may argue that expanding access to care would be an economic burden on taxpayers; however, research demonstrates that undocumented immigrants contribute more to the US Medicare Trust Fund than they actually withdraw toward healthcare.¹³ Furthermore, a new study has demonstrated that a net savings of nearly $6,000 per person per month is realized when patients are transitioned from emergency-only hemodialysis to standard hemodialysis.¹⁴

Internal medicine hospitalists on the front-line of healthcare systems are regular witnesses to its horrible injustices. We rarely share our perspectives and do not expect change to follow. With Hilda, we saw how a powerful combination of research and coalition building could lift one patient’s tragic story to a level where it could produce change. Augmenting Hilda’s experience of tragically poor access to care with evidence-based research gave her story validity far beyond our immediate circle of friends and colleagues, making a singular tragedy, policy relevant. Each time we shared our research to community advocacy groups, health policy stakeholders, state legislators, nurses, and staff; we began with Hilda’s story, not just because it inspired us, but because its truth was undeniable. Our patients’ stories matter, and it is our responsibility to tell them.

Each time I prepare nopalitos for my family, I think of my last meal with Hilda. No matter how painful or difficult her struggle with ESKD, Hilda persisted. She protected her boys. They were her purpose. When she knew she could no longer give them the life she wanted for them, she found a family who would. Hilda’s sons now live with a loving adoptive family, are thriving in school, and her oldest is interested in becoming a physician. Nopal, or cactus, symbolizes such endurance—a plant with unique adaptations and strength that can flourish under extreme environmental stress. Like a cactus storing precious water, Hilda treasured her children, and her resolve to provide for them was unstoppable, right to the edge of death. When our team first took up Hilda’s cause, change seemed impossible. We discovered the opposite. As I clench the wooden rosary she left me that Christmas, I thank her for giving our team the courage to adapt and persist, for in doing so we found a path, first to research and then to broader partnerships and more meaningful policy changes.

Acknowledgments

The author would like to thank Hilda, her family, and the patients at Denver Health. She would also like to acknowledge Hilda’s family, Drs. Mark Earnest, John F. Steiner, Romana Hasnain-Wynia, Rudolph Rodriguez, Judy Regensteiner, and Michel Chonchol for reading and providing feedback on earlier drafts of this narrative.

Internists are experts in general medicine, skilled at mapping the few hundred ways the human body can go awry onto thousands of diagnoses, and managing the uncertainty inherent in that process. Generalists, almost by definition, consult specialists with their specialty-focused questions; but who does one call for a general consultation about diagnosis if a specific diagnosis remains elusive and the pathology does not fit cleanly into the purview of a consultant? Outside of sage advice from colleagues (usually senior), most medical centers lack a consultation service focused on diagnosis. There is no oracle to seek. In this perspective, we describe our institution’s answer to this problem: the creation of a service for difficult diagnosis based on Socratic principles, particularly the role of iterative hypothesis testing in the process of diagnosis.¹

In 2015, Northwestern Medicine began the Socrates Project, a physician-to-physician consultation service that assists doctors working to diagnose conditions that have so far eluded detection. Our service’s goal is to improve patient care by providing an opinion to the referring physician on diagnostic possibilities for a particular case and ideas to reduce—or at least manage—diagnostic uncertainty.

Most patients referred to the Socrates Project have already undergone an extensive evaluation at top medical centers by experienced clinicians. It would be hubris to assume that we will find a definitive diagnosis in every case; indeed, because of the types of cases referred to our group, it is rare that we find a “Eureka!” diagnosis. When a colleague consults our group, we under-promise in hopes of over-delivering. Instead, we convey to referring physicians that we will conduct a thorough case review and explain our thinking in hopes of uncovering an additional diagnostic avenue, even if that avenue does not ultimately lead to a definitive diagnosis. In addition, the Socrates Project often serves as a broker between consulting services that are deadlocked because of differing diagnostic opinions. We also assist with cases in which a functional disorder is suspected, yet the referring physician is hesitant to diagnose a patient with such a disorder out of concern about missing an important (and possibly obscure) diagnosis.

The Socrates Project receives approximately two consult requests per week, usually from general internists but also from specialists in nearly all disciplines. Around 80% of the referrals are for current inpatients. Our service model is similar to a tumor board, which exists as an interdisciplinary group operating in parallel to the clinical services, to provide consensus-based recommendations. As a result, we act as doctors for doctors, formalizing the curbside consultation. Our usual turnaround time is a week but can be faster for urgent cases. Currently, Socrates Project members, including the faculty leader, volunteer their time and effort at no cost, and there are no charges to patients when physicians consult our group. An overview of the Socrates Project’s personnel and process are outlined in the Figure.

Northwestern’s Chief Medical Residents (CMRs) serve as the fellows for the service, and one of them assumes primary responsibility for each new consultation request the service receives. After obtaining the patient’s case history from the referring provider, the CMR then undertakes a thorough review of the electronic health record and any other available records from other institutions. In the inpatient setting, the CMR performs a new history and physical; phone calls or video conferencing permit history taking for outpatients. In contrast with the standard consultant note, we do not redocument the history, physical, and lab and imaging findings but instead construct a detailed problem list that synthesizes relevant findings into a useful working document.

The service’s faculty leader (BDS) then reviews the problem list with the CMRs to help refine the problem list and begin producing a differential diagnosis during a weekly hour-long meeting. As evidence supports team-based diagnostic collaborations,² the problem list and preliminary differential diagnosis then becomes a shareable document that the CMR or team leader presents to ad hoc general internists, specialists, and the other CMRs. The presentation can be in person, by phone, or e-mail. These ad hoc members, approximately 20 in number and spanning from junior attending physicians to senior clinicians, have volunteered to help the Socrates Project by adding their thoughts on differential diagnoses that explain the problem list and how to move forward with further testing. The ad hoc members have self-identified as clinicians with an interest in medical diagnosis—including surgeons, neurologists, psychiatrists, radiologists, and pathologists—and range in expertise from general internists to subspecialists. Finally, we document our problem list, differential diagnosis, and recommendations in the medical record and discuss the case with the referring team. The service limits its scope of clinical recommendation to diagnosis and avoids commenting on management decisions outside of the use of therapies as empiric diagnostic tests. A sample note is provided as an online Appendix.

Despite our process, we are often left without a satisfying diagnosis. We then are then faced with three possibilities: (1) The diagnosis is identifiable, just not by the physicians involved in the case—we did not think of the diagnosis in our deliberations; (2) The diagnosis is a described condition but without an available test—autoimmune limbic encephalitis associated with an unassayable or unknown auto-antibody, or the acuity of a critically ill patient makes diagnostic testing unreliable or not feasible; (3) The diagnosis has not yet been described by medical science—we are seeing a case of HIV infection in 1971.

With the personnel and process outlined above, we hope to provide recommendations that are useful in guiding a diagnostic workup regardless of which of these three scenarios is applicable. Our flexibility with involving the appropriate specialists in the Socrates Project should minimize the number of patients with a knowable diagnosis that is unknown to us. In the second scenario, our recommendations may rest upon the incorporation of a treatment as a diagnostic test. In the limbic encephalitis example above, a trial of steroids with rapid improvement in the patient’s condition may increase diagnostic certainty. The third scenario is the most difficult to identify. Pattern recognition of similarly presenting patients, keeping ourselves updated on pertinent primary literature, and consideration of advanced diagnostic testing such as exome sequencing and other next-generation sequencing strategies are essential in hoping to characterize a specific clinical syndrome that has yet to be described.

For situations in which our recommendations do not yield a diagnosis, we recognize the role for protocols such as genomic or metagenomic sequencing that assess multiple diagnostic possibilities in parallel without an a priori hypothesis.^3,4 The utility of multi-omics testing in diagnostic workups has been detailed by the Undiagnosed Diseases Network (UDN), which has created a systematic approach to describing new syndromes with the aid of metabolomic and genomic profiling.⁵ It is important to note that even with the resources available to the UDN, the diagnosis rate is 35%, emphasizing that in the majority of diagnosis-refractory cases, a diagnosis will not be found. This low diagnosis rate underscores the need for continued inquiry and cataloging of cases and data for further review or synthesis as the body of medical knowledge continues to expand. For these reasons, we have a follow-up system in place, which involves the assigned CMR regularly reviewing the chart and reporting during our weekly meetings. We make phone calls to patients and providers for cases that appear to be lost to follow-up.

We recognize several important limitations to our care model that may represent barriers to establishing, maintaining, and evaluating a similar service at other institutions. For example, there are limitations and benefits of the CMR as point person for managing our consultations. While they are admittedly junior colleagues with limited experience, CMRs tend to be among the best-read and up-to-date clinicians in the hospital by virtue of their recent general-medicine training and identification as a top clinician and leader. Moreover, in their role with the Socrates Project, CMRs have more time to think, talk with patients, and review the medical record than other clinicians, who may be under pressure to see an increasing number of patients while billing at higher levels. Indeed, the Socrates Project CMRs have, on a number of occasions, been the team members who find the piece of data that no one else thought relevant.

Another factor that may limit establishment of a similar team at other institutions is our volunteer-based model. The Socrates Project members volunteer because they love clinical medicine and serve on the team without remuneration for professional effort. With the CMR role as a notable exception, pressure from achieving relative value unit targets, obtaining grant funding, and publishing primary research publications in their field may limit this care model, particularly when shifting from a clinical-only activity to one that also formally investigates the service’s process and outcomes.

Beyond our clinical objective, we hope that the Socrates Project will further the discovery and description of previously unrecognized disease processes. To that end, we are pursuing an institutional review board-approved protocol to perform a rigorous assessment of the Socrates Project’s process and outcomes, including a cataloging of case archetypes and the time to definitive diagnosis if a diagnosis is established. As we continue to collect data, increasing our referral network may also lead to refinement and improvement in diagnostic processes and outcomes. Over time, we expect that the diagnostic resources available to us will evolve. Utilizing collective intelligence has been shown to improve diagnostic accuracy,⁶ and emerging artificial intelligence technologies may improve diagnostic performance as well.^7,8 Most importantly, through this endeavor, we hope to serve less as an oracle and more as a humble Socratic consultant for clinicians working to reduce diagnostic uncertainty for their patients.

Acknowledgments

The authors wish to thank the Northwestern University Chief Medical Residents, 2015-present, for their tireless efforts in support of the Socrates Project.

Internists are experts in general medicine, skilled at mapping the few hundred ways the human body can go awry onto thousands of diagnoses, and managing the uncertainty inherent in that process. Generalists, almost by definition, consult specialists with their specialty-focused questions; but who does one call for a general consultation about diagnosis if a specific diagnosis remains elusive and the pathology does not fit cleanly into the purview of a consultant? Outside of sage advice from colleagues (usually senior), most medical centers lack a consultation service focused on diagnosis. There is no oracle to seek. In this perspective, we describe our institution’s answer to this problem: the creation of a service for difficult diagnosis based on Socratic principles, particularly the role of iterative hypothesis testing in the process of diagnosis.¹

In 2015, Northwestern Medicine began the Socrates Project, a physician-to-physician consultation service that assists doctors working to diagnose conditions that have so far eluded detection. Our service’s goal is to improve patient care by providing an opinion to the referring physician on diagnostic possibilities for a particular case and ideas to reduce—or at least manage—diagnostic uncertainty.

Most patients referred to the Socrates Project have already undergone an extensive evaluation at top medical centers by experienced clinicians. It would be hubris to assume that we will find a definitive diagnosis in every case; indeed, because of the types of cases referred to our group, it is rare that we find a “Eureka!” diagnosis. When a colleague consults our group, we under-promise in hopes of over-delivering. Instead, we convey to referring physicians that we will conduct a thorough case review and explain our thinking in hopes of uncovering an additional diagnostic avenue, even if that avenue does not ultimately lead to a definitive diagnosis. In addition, the Socrates Project often serves as a broker between consulting services that are deadlocked because of differing diagnostic opinions. We also assist with cases in which a functional disorder is suspected, yet the referring physician is hesitant to diagnose a patient with such a disorder out of concern about missing an important (and possibly obscure) diagnosis.

The Socrates Project receives approximately two consult requests per week, usually from general internists but also from specialists in nearly all disciplines. Around 80% of the referrals are for current inpatients. Our service model is similar to a tumor board, which exists as an interdisciplinary group operating in parallel to the clinical services, to provide consensus-based recommendations. As a result, we act as doctors for doctors, formalizing the curbside consultation. Our usual turnaround time is a week but can be faster for urgent cases. Currently, Socrates Project members, including the faculty leader, volunteer their time and effort at no cost, and there are no charges to patients when physicians consult our group. An overview of the Socrates Project’s personnel and process are outlined in the Figure.

Northwestern’s Chief Medical Residents (CMRs) serve as the fellows for the service, and one of them assumes primary responsibility for each new consultation request the service receives. After obtaining the patient’s case history from the referring provider, the CMR then undertakes a thorough review of the electronic health record and any other available records from other institutions. In the inpatient setting, the CMR performs a new history and physical; phone calls or video conferencing permit history taking for outpatients. In contrast with the standard consultant note, we do not redocument the history, physical, and lab and imaging findings but instead construct a detailed problem list that synthesizes relevant findings into a useful working document.

The service’s faculty leader (BDS) then reviews the problem list with the CMRs to help refine the problem list and begin producing a differential diagnosis during a weekly hour-long meeting. As evidence supports team-based diagnostic collaborations,² the problem list and preliminary differential diagnosis then becomes a shareable document that the CMR or team leader presents to ad hoc general internists, specialists, and the other CMRs. The presentation can be in person, by phone, or e-mail. These ad hoc members, approximately 20 in number and spanning from junior attending physicians to senior clinicians, have volunteered to help the Socrates Project by adding their thoughts on differential diagnoses that explain the problem list and how to move forward with further testing. The ad hoc members have self-identified as clinicians with an interest in medical diagnosis—including surgeons, neurologists, psychiatrists, radiologists, and pathologists—and range in expertise from general internists to subspecialists. Finally, we document our problem list, differential diagnosis, and recommendations in the medical record and discuss the case with the referring team. The service limits its scope of clinical recommendation to diagnosis and avoids commenting on management decisions outside of the use of therapies as empiric diagnostic tests. A sample note is provided as an online Appendix.

Despite our process, we are often left without a satisfying diagnosis. We then are then faced with three possibilities: (1) The diagnosis is identifiable, just not by the physicians involved in the case—we did not think of the diagnosis in our deliberations; (2) The diagnosis is a described condition but without an available test—autoimmune limbic encephalitis associated with an unassayable or unknown auto-antibody, or the acuity of a critically ill patient makes diagnostic testing unreliable or not feasible; (3) The diagnosis has not yet been described by medical science—we are seeing a case of HIV infection in 1971.

With the personnel and process outlined above, we hope to provide recommendations that are useful in guiding a diagnostic workup regardless of which of these three scenarios is applicable. Our flexibility with involving the appropriate specialists in the Socrates Project should minimize the number of patients with a knowable diagnosis that is unknown to us. In the second scenario, our recommendations may rest upon the incorporation of a treatment as a diagnostic test. In the limbic encephalitis example above, a trial of steroids with rapid improvement in the patient’s condition may increase diagnostic certainty. The third scenario is the most difficult to identify. Pattern recognition of similarly presenting patients, keeping ourselves updated on pertinent primary literature, and consideration of advanced diagnostic testing such as exome sequencing and other next-generation sequencing strategies are essential in hoping to characterize a specific clinical syndrome that has yet to be described.

For situations in which our recommendations do not yield a diagnosis, we recognize the role for protocols such as genomic or metagenomic sequencing that assess multiple diagnostic possibilities in parallel without an a priori hypothesis.^3,4 The utility of multi-omics testing in diagnostic workups has been detailed by the Undiagnosed Diseases Network (UDN), which has created a systematic approach to describing new syndromes with the aid of metabolomic and genomic profiling.⁵ It is important to note that even with the resources available to the UDN, the diagnosis rate is 35%, emphasizing that in the majority of diagnosis-refractory cases, a diagnosis will not be found. This low diagnosis rate underscores the need for continued inquiry and cataloging of cases and data for further review or synthesis as the body of medical knowledge continues to expand. For these reasons, we have a follow-up system in place, which involves the assigned CMR regularly reviewing the chart and reporting during our weekly meetings. We make phone calls to patients and providers for cases that appear to be lost to follow-up.

We recognize several important limitations to our care model that may represent barriers to establishing, maintaining, and evaluating a similar service at other institutions. For example, there are limitations and benefits of the CMR as point person for managing our consultations. While they are admittedly junior colleagues with limited experience, CMRs tend to be among the best-read and up-to-date clinicians in the hospital by virtue of their recent general-medicine training and identification as a top clinician and leader. Moreover, in their role with the Socrates Project, CMRs have more time to think, talk with patients, and review the medical record than other clinicians, who may be under pressure to see an increasing number of patients while billing at higher levels. Indeed, the Socrates Project CMRs have, on a number of occasions, been the team members who find the piece of data that no one else thought relevant.

Another factor that may limit establishment of a similar team at other institutions is our volunteer-based model. The Socrates Project members volunteer because they love clinical medicine and serve on the team without remuneration for professional effort. With the CMR role as a notable exception, pressure from achieving relative value unit targets, obtaining grant funding, and publishing primary research publications in their field may limit this care model, particularly when shifting from a clinical-only activity to one that also formally investigates the service’s process and outcomes.

Beyond our clinical objective, we hope that the Socrates Project will further the discovery and description of previously unrecognized disease processes. To that end, we are pursuing an institutional review board-approved protocol to perform a rigorous assessment of the Socrates Project’s process and outcomes, including a cataloging of case archetypes and the time to definitive diagnosis if a diagnosis is established. As we continue to collect data, increasing our referral network may also lead to refinement and improvement in diagnostic processes and outcomes. Over time, we expect that the diagnostic resources available to us will evolve. Utilizing collective intelligence has been shown to improve diagnostic accuracy,⁶ and emerging artificial intelligence technologies may improve diagnostic performance as well.^7,8 Most importantly, through this endeavor, we hope to serve less as an oracle and more as a humble Socratic consultant for clinicians working to reduce diagnostic uncertainty for their patients.

Acknowledgments

The authors wish to thank the Northwestern University Chief Medical Residents, 2015-present, for their tireless efforts in support of the Socrates Project.

Internists are experts in general medicine, skilled at mapping the few hundred ways the human body can go awry onto thousands of diagnoses, and managing the uncertainty inherent in that process. Generalists, almost by definition, consult specialists with their specialty-focused questions; but who does one call for a general consultation about diagnosis if a specific diagnosis remains elusive and the pathology does not fit cleanly into the purview of a consultant? Outside of sage advice from colleagues (usually senior), most medical centers lack a consultation service focused on diagnosis. There is no oracle to seek. In this perspective, we describe our institution’s answer to this problem: the creation of a service for difficult diagnosis based on Socratic principles, particularly the role of iterative hypothesis testing in the process of diagnosis.¹

In 2015, Northwestern Medicine began the Socrates Project, a physician-to-physician consultation service that assists doctors working to diagnose conditions that have so far eluded detection. Our service’s goal is to improve patient care by providing an opinion to the referring physician on diagnostic possibilities for a particular case and ideas to reduce—or at least manage—diagnostic uncertainty.

Most patients referred to the Socrates Project have already undergone an extensive evaluation at top medical centers by experienced clinicians. It would be hubris to assume that we will find a definitive diagnosis in every case; indeed, because of the types of cases referred to our group, it is rare that we find a “Eureka!” diagnosis. When a colleague consults our group, we under-promise in hopes of over-delivering. Instead, we convey to referring physicians that we will conduct a thorough case review and explain our thinking in hopes of uncovering an additional diagnostic avenue, even if that avenue does not ultimately lead to a definitive diagnosis. In addition, the Socrates Project often serves as a broker between consulting services that are deadlocked because of differing diagnostic opinions. We also assist with cases in which a functional disorder is suspected, yet the referring physician is hesitant to diagnose a patient with such a disorder out of concern about missing an important (and possibly obscure) diagnosis.

The Socrates Project receives approximately two consult requests per week, usually from general internists but also from specialists in nearly all disciplines. Around 80% of the referrals are for current inpatients. Our service model is similar to a tumor board, which exists as an interdisciplinary group operating in parallel to the clinical services, to provide consensus-based recommendations. As a result, we act as doctors for doctors, formalizing the curbside consultation. Our usual turnaround time is a week but can be faster for urgent cases. Currently, Socrates Project members, including the faculty leader, volunteer their time and effort at no cost, and there are no charges to patients when physicians consult our group. An overview of the Socrates Project’s personnel and process are outlined in the Figure.

Northwestern’s Chief Medical Residents (CMRs) serve as the fellows for the service, and one of them assumes primary responsibility for each new consultation request the service receives. After obtaining the patient’s case history from the referring provider, the CMR then undertakes a thorough review of the electronic health record and any other available records from other institutions. In the inpatient setting, the CMR performs a new history and physical; phone calls or video conferencing permit history taking for outpatients. In contrast with the standard consultant note, we do not redocument the history, physical, and lab and imaging findings but instead construct a detailed problem list that synthesizes relevant findings into a useful working document.

The service’s faculty leader (BDS) then reviews the problem list with the CMRs to help refine the problem list and begin producing a differential diagnosis during a weekly hour-long meeting. As evidence supports team-based diagnostic collaborations,² the problem list and preliminary differential diagnosis then becomes a shareable document that the CMR or team leader presents to ad hoc general internists, specialists, and the other CMRs. The presentation can be in person, by phone, or e-mail. These ad hoc members, approximately 20 in number and spanning from junior attending physicians to senior clinicians, have volunteered to help the Socrates Project by adding their thoughts on differential diagnoses that explain the problem list and how to move forward with further testing. The ad hoc members have self-identified as clinicians with an interest in medical diagnosis—including surgeons, neurologists, psychiatrists, radiologists, and pathologists—and range in expertise from general internists to subspecialists. Finally, we document our problem list, differential diagnosis, and recommendations in the medical record and discuss the case with the referring team. The service limits its scope of clinical recommendation to diagnosis and avoids commenting on management decisions outside of the use of therapies as empiric diagnostic tests. A sample note is provided as an online Appendix.

Despite our process, we are often left without a satisfying diagnosis. We then are then faced with three possibilities: (1) The diagnosis is identifiable, just not by the physicians involved in the case—we did not think of the diagnosis in our deliberations; (2) The diagnosis is a described condition but without an available test—autoimmune limbic encephalitis associated with an unassayable or unknown auto-antibody, or the acuity of a critically ill patient makes diagnostic testing unreliable or not feasible; (3) The diagnosis has not yet been described by medical science—we are seeing a case of HIV infection in 1971.

With the personnel and process outlined above, we hope to provide recommendations that are useful in guiding a diagnostic workup regardless of which of these three scenarios is applicable. Our flexibility with involving the appropriate specialists in the Socrates Project should minimize the number of patients with a knowable diagnosis that is unknown to us. In the second scenario, our recommendations may rest upon the incorporation of a treatment as a diagnostic test. In the limbic encephalitis example above, a trial of steroids with rapid improvement in the patient’s condition may increase diagnostic certainty. The third scenario is the most difficult to identify. Pattern recognition of similarly presenting patients, keeping ourselves updated on pertinent primary literature, and consideration of advanced diagnostic testing such as exome sequencing and other next-generation sequencing strategies are essential in hoping to characterize a specific clinical syndrome that has yet to be described.

For situations in which our recommendations do not yield a diagnosis, we recognize the role for protocols such as genomic or metagenomic sequencing that assess multiple diagnostic possibilities in parallel without an a priori hypothesis.^3,4 The utility of multi-omics testing in diagnostic workups has been detailed by the Undiagnosed Diseases Network (UDN), which has created a systematic approach to describing new syndromes with the aid of metabolomic and genomic profiling.⁵ It is important to note that even with the resources available to the UDN, the diagnosis rate is 35%, emphasizing that in the majority of diagnosis-refractory cases, a diagnosis will not be found. This low diagnosis rate underscores the need for continued inquiry and cataloging of cases and data for further review or synthesis as the body of medical knowledge continues to expand. For these reasons, we have a follow-up system in place, which involves the assigned CMR regularly reviewing the chart and reporting during our weekly meetings. We make phone calls to patients and providers for cases that appear to be lost to follow-up.

We recognize several important limitations to our care model that may represent barriers to establishing, maintaining, and evaluating a similar service at other institutions. For example, there are limitations and benefits of the CMR as point person for managing our consultations. While they are admittedly junior colleagues with limited experience, CMRs tend to be among the best-read and up-to-date clinicians in the hospital by virtue of their recent general-medicine training and identification as a top clinician and leader. Moreover, in their role with the Socrates Project, CMRs have more time to think, talk with patients, and review the medical record than other clinicians, who may be under pressure to see an increasing number of patients while billing at higher levels. Indeed, the Socrates Project CMRs have, on a number of occasions, been the team members who find the piece of data that no one else thought relevant.

Another factor that may limit establishment of a similar team at other institutions is our volunteer-based model. The Socrates Project members volunteer because they love clinical medicine and serve on the team without remuneration for professional effort. With the CMR role as a notable exception, pressure from achieving relative value unit targets, obtaining grant funding, and publishing primary research publications in their field may limit this care model, particularly when shifting from a clinical-only activity to one that also formally investigates the service’s process and outcomes.

Beyond our clinical objective, we hope that the Socrates Project will further the discovery and description of previously unrecognized disease processes. To that end, we are pursuing an institutional review board-approved protocol to perform a rigorous assessment of the Socrates Project’s process and outcomes, including a cataloging of case archetypes and the time to definitive diagnosis if a diagnosis is established. As we continue to collect data, increasing our referral network may also lead to refinement and improvement in diagnostic processes and outcomes. Over time, we expect that the diagnostic resources available to us will evolve. Utilizing collective intelligence has been shown to improve diagnostic accuracy,⁶ and emerging artificial intelligence technologies may improve diagnostic performance as well.^7,8 Most importantly, through this endeavor, we hope to serve less as an oracle and more as a humble Socratic consultant for clinicians working to reduce diagnostic uncertainty for their patients.

Acknowledgments

The authors wish to thank the Northwestern University Chief Medical Residents, 2015-present, for their tireless efforts in support of the Socrates Project.

Perioperative venous thromboembolism (VTE) is a major contributor to the morbidity and mortality of hospitalized patients. The historical incidence of postoperative VTE varied between 15%-80% depending on the type of procedure and monitoring strategies. Higher incidences of VTE occurred with major surgery (15%-40%), knee or hip arthroplasty (40%-60%), and trauma (60%-80%).¹ The use of VTE prophylaxis with subcutaneous heparin reduced DVTs by 70% and PEs by 50%.² A recent study from Olmstead County showed improved adherence to inpatient VTE prophylaxis from 2005 to 2010 but no difference in VTE incidence. However, 52% of VTE events were associated with hospitalization.³ As such, VTE continues to be a healthcare-associated adverse event, and surgery remains a significant risk factor for thrombosis.⁴

The Agency for Healthcare Research and Quality (AHRQ) released Patient Safety Indicators (PSI) in 2003 to provide a means of screening for adverse events.⁵ Over time, PSIs have been adopted as a measure of hospital performance and are utilized in several pay-for-performance programs. PSI-90 is a composite measure of several other PSIs and is a core metric in the Centers for Medicare and Medicaid Services (CMS) Hospital-Acquired Condition Reduction Program and the Hospital Value-Based Purchasing Program which impacts up to 2% of a hospital’s Medicare payments.^6,7 One component of PSI-90 is PSI-12, which captures perioperative VTE. PSI-12 events are identified using software that screens medical records based on International Classification of Diseases (ICD)-9/10 codes for thrombosis and procedure codes at time of discharge.⁸

Over the last several years, there has been concern regarding the validity of including PSI-12 in pay-for-performance metrics. Common areas for concern include PSI-12’s accuracy in detecting true postoperative VTE⁹ in addition to surveillance bias.^10,11 However, some note that PSI-12 is useful when applied with its original intent: as a screening tool for hospitals to identify specific areas to implement improvements.^9,12The aim of our study was to review all PSI-12 events at our institution to evaluate the accuracy of PSI-12 and identify areas for improvement to prevent VTE events in surgical patients. While several other studies have looked at the positive predictive values, accuracy, or surveillance bias of PSI-12, to the best of our knowledge, few, if any, previous studies have reported PSI-12 events in relation to their timing, type of prophylaxis used, and mitigating factors to identify areas for quality improvement.

PSI-12 events were identified between June 2015 and June 2017 using AHRQ software (version 5). Cases were also identified through Vizient and reviewed to ensure congruence between the methods. Patients’ electronic medical records were reviewed for patient demographics, type of VTE event, platelet count at VTE diagnosis, procedure type, and both the timing and type of VTE prophylaxis. Summary statistics were calculated.

We considered perioperative VTE pharmacologic prophylaxis appropriate if started within 24 hours of a low bleeding risk procedure or 72 hours of a high-bleeding risk procedure.¹³ Mechanical prophylaxis was considered appropriate if pharmacologic prophylaxis was not used because of procedure risk, thrombocytopenia, or active bleeding. The medication administration record was reviewed to determine if prophylaxis was ordered, given, and/or refused.

During the two-year period, 18,084 surgeries were performed, and 161 cases of VTE events were identified. A detailed chart review and correction of documentation led to the exclusion of seven cases (4%) because the VTE event occurred prior to admission (n = 5) or were incidental findings that did not meet the Uniform Hospital Discharge Data Set definition for reporting (n = 2). In total, 154 (0.9% of all surgeries) cases were considered PSI-12 events. Pulmonary embolism (PE) occurred in most cases (n = 97, 62.9%), followed by deep vein thrombosis (DVT) (n = 37, 24.0%). Twenty cases (12.9%) experienced concurrent PE and DVT. Within the PE group, 16 cases (14%) were subsegmental PE only. Eight patients (14% of DVT cases) had only a distal DVT. The mean age of patients was 56 years (+/− 16 years), and the majority (59%) were male. The clinical specialties with the most events included neurosurgery (21%), orthopedics (14%), general surgery (13%), and trauma (11%). Fourteen patients (9%) died during the hospitalization, and of these, six (43%) had either sudden death or death attributed to PE (Table 1).

Cases were also reviewed for the type of VTE prophylactic strategy administered at the time of the event. The top three prophylactic strategies were subcutaneous unfractionated heparin (61%), mechanical prophylaxis only (51%), and enoxaparin (31%). Nine cases of VTE occurred during therapeutic anticoagulation (6%; Table 1).

Utilizing AHRQ version 5 software for PSI-12, our institution identified 154 cases of perioperative VTE. Most of these cases were a pulmonary embolism, occurred within a week of admission, and were associated with surgical specialties that portend a higher risk of VTE. Very few of these cases were deficient in guideline-directed VTE prophylaxis, and several cases had associated factors such as trauma and thrombocytopenia that may have appropriately influenced the decision to use mechanical only prophylaxis. Sixteen percent of pharmacologic VTE prophylactic doses were refused or held for a procedure, a known but rarely quantified influence on rates of pharmacologic prophylaxis. The use of patient-level data and adjudication of the clinical decision that affected the administration of VTE prophylaxis is a major strength of this work. In all, our data raise several questions about the accuracy of PSI-12 in identifying preventable postoperative VTE, especially as it is utilized as a marker for pay-for-performance measures in addition to identifying further areas for research and improvement.

Our results align with previous studies that suggest that PSI-12 is an inaccurate measure of performance quality. A study by Bilimoria et al. in 2013 noted that surveillance biases associated with PSI-12, showing that hospitals with higher compliance to appropriate VTE prophylaxis paradoxically had worse outcomes.¹⁰ Furthermore, Blay et al. recently published a study evaluating PSI-90 scores for hospitals with and without the VTE measure included. Their results indicated that larger hospitals (teaching hospitals, level I trauma centers, etc) caring for sicker patients were noted to improve by 8%-25% when the VTE measure was removed.¹¹ Similarly, our data indicate that the PSI-12 may not be an accurate measure of quality performance, as only 6% of cases were noted to be deficient in appropriate guideline-directed prophylaxis. Even when accounting for the refusal of doses of pharmacologic prophylaxis, this figure only increased to 11%.

Procedures included in the current PSI-12 algorithm also vary in the risk they pose to developing VTE. For example, the current version of PSI-12 includes surgical Medicare Severity Diagnosis Related Groups (MS-DRGs) for procedures with a high risk of VTE such as orthopedic, abdominal, or thoracic procedures.^14,15, However, it is worth noting that procedures such as tracheostomy and ocular surgery are also included.¹⁴ The variation in risk for development of VTE is reflected in the Caprini score, a perioperative risk stratification tool. These latter procedures only contribute one point, whereas trauma would add five points each to the total score, and make the patient a high VTE risk from the procedure alone.¹⁶ With regard to PSI-12, in theory, scores could vary significantly between centers even if the quality of care is the same, based on the volume and risk of procedures performed.

Perioperative venous thromboembolism (VTE) is a major contributor to the morbidity and mortality of hospitalized patients. The historical incidence of postoperative VTE varied between 15%-80% depending on the type of procedure and monitoring strategies. Higher incidences of VTE occurred with major surgery (15%-40%), knee or hip arthroplasty (40%-60%), and trauma (60%-80%).¹ The use of VTE prophylaxis with subcutaneous heparin reduced DVTs by 70% and PEs by 50%.² A recent study from Olmstead County showed improved adherence to inpatient VTE prophylaxis from 2005 to 2010 but no difference in VTE incidence. However, 52% of VTE events were associated with hospitalization.³ As such, VTE continues to be a healthcare-associated adverse event, and surgery remains a significant risk factor for thrombosis.⁴

The Agency for Healthcare Research and Quality (AHRQ) released Patient Safety Indicators (PSI) in 2003 to provide a means of screening for adverse events.⁵ Over time, PSIs have been adopted as a measure of hospital performance and are utilized in several pay-for-performance programs. PSI-90 is a composite measure of several other PSIs and is a core metric in the Centers for Medicare and Medicaid Services (CMS) Hospital-Acquired Condition Reduction Program and the Hospital Value-Based Purchasing Program which impacts up to 2% of a hospital’s Medicare payments.^6,7 One component of PSI-90 is PSI-12, which captures perioperative VTE. PSI-12 events are identified using software that screens medical records based on International Classification of Diseases (ICD)-9/10 codes for thrombosis and procedure codes at time of discharge.⁸

Over the last several years, there has been concern regarding the validity of including PSI-12 in pay-for-performance metrics. Common areas for concern include PSI-12’s accuracy in detecting true postoperative VTE⁹ in addition to surveillance bias.^10,11 However, some note that PSI-12 is useful when applied with its original intent: as a screening tool for hospitals to identify specific areas to implement improvements.^9,12The aim of our study was to review all PSI-12 events at our institution to evaluate the accuracy of PSI-12 and identify areas for improvement to prevent VTE events in surgical patients. While several other studies have looked at the positive predictive values, accuracy, or surveillance bias of PSI-12, to the best of our knowledge, few, if any, previous studies have reported PSI-12 events in relation to their timing, type of prophylaxis used, and mitigating factors to identify areas for quality improvement.

PSI-12 events were identified between June 2015 and June 2017 using AHRQ software (version 5). Cases were also identified through Vizient and reviewed to ensure congruence between the methods. Patients’ electronic medical records were reviewed for patient demographics, type of VTE event, platelet count at VTE diagnosis, procedure type, and both the timing and type of VTE prophylaxis. Summary statistics were calculated.

We considered perioperative VTE pharmacologic prophylaxis appropriate if started within 24 hours of a low bleeding risk procedure or 72 hours of a high-bleeding risk procedure.¹³ Mechanical prophylaxis was considered appropriate if pharmacologic prophylaxis was not used because of procedure risk, thrombocytopenia, or active bleeding. The medication administration record was reviewed to determine if prophylaxis was ordered, given, and/or refused.

During the two-year period, 18,084 surgeries were performed, and 161 cases of VTE events were identified. A detailed chart review and correction of documentation led to the exclusion of seven cases (4%) because the VTE event occurred prior to admission (n = 5) or were incidental findings that did not meet the Uniform Hospital Discharge Data Set definition for reporting (n = 2). In total, 154 (0.9% of all surgeries) cases were considered PSI-12 events. Pulmonary embolism (PE) occurred in most cases (n = 97, 62.9%), followed by deep vein thrombosis (DVT) (n = 37, 24.0%). Twenty cases (12.9%) experienced concurrent PE and DVT. Within the PE group, 16 cases (14%) were subsegmental PE only. Eight patients (14% of DVT cases) had only a distal DVT. The mean age of patients was 56 years (+/− 16 years), and the majority (59%) were male. The clinical specialties with the most events included neurosurgery (21%), orthopedics (14%), general surgery (13%), and trauma (11%). Fourteen patients (9%) died during the hospitalization, and of these, six (43%) had either sudden death or death attributed to PE (Table 1).

Cases were also reviewed for the type of VTE prophylactic strategy administered at the time of the event. The top three prophylactic strategies were subcutaneous unfractionated heparin (61%), mechanical prophylaxis only (51%), and enoxaparin (31%). Nine cases of VTE occurred during therapeutic anticoagulation (6%; Table 1).

Utilizing AHRQ version 5 software for PSI-12, our institution identified 154 cases of perioperative VTE. Most of these cases were a pulmonary embolism, occurred within a week of admission, and were associated with surgical specialties that portend a higher risk of VTE. Very few of these cases were deficient in guideline-directed VTE prophylaxis, and several cases had associated factors such as trauma and thrombocytopenia that may have appropriately influenced the decision to use mechanical only prophylaxis. Sixteen percent of pharmacologic VTE prophylactic doses were refused or held for a procedure, a known but rarely quantified influence on rates of pharmacologic prophylaxis. The use of patient-level data and adjudication of the clinical decision that affected the administration of VTE prophylaxis is a major strength of this work. In all, our data raise several questions about the accuracy of PSI-12 in identifying preventable postoperative VTE, especially as it is utilized as a marker for pay-for-performance measures in addition to identifying further areas for research and improvement.

Our results align with previous studies that suggest that PSI-12 is an inaccurate measure of performance quality. A study by Bilimoria et al. in 2013 noted that surveillance biases associated with PSI-12, showing that hospitals with higher compliance to appropriate VTE prophylaxis paradoxically had worse outcomes.¹⁰ Furthermore, Blay et al. recently published a study evaluating PSI-90 scores for hospitals with and without the VTE measure included. Their results indicated that larger hospitals (teaching hospitals, level I trauma centers, etc) caring for sicker patients were noted to improve by 8%-25% when the VTE measure was removed.¹¹ Similarly, our data indicate that the PSI-12 may not be an accurate measure of quality performance, as only 6% of cases were noted to be deficient in appropriate guideline-directed prophylaxis. Even when accounting for the refusal of doses of pharmacologic prophylaxis, this figure only increased to 11%.

Procedures included in the current PSI-12 algorithm also vary in the risk they pose to developing VTE. For example, the current version of PSI-12 includes surgical Medicare Severity Diagnosis Related Groups (MS-DRGs) for procedures with a high risk of VTE such as orthopedic, abdominal, or thoracic procedures.^14,15, However, it is worth noting that procedures such as tracheostomy and ocular surgery are also included.¹⁴ The variation in risk for development of VTE is reflected in the Caprini score, a perioperative risk stratification tool. These latter procedures only contribute one point, whereas trauma would add five points each to the total score, and make the patient a high VTE risk from the procedure alone.¹⁶ With regard to PSI-12, in theory, scores could vary significantly between centers even if the quality of care is the same, based on the volume and risk of procedures performed.

Perioperative venous thromboembolism (VTE) is a major contributor to the morbidity and mortality of hospitalized patients. The historical incidence of postoperative VTE varied between 15%-80% depending on the type of procedure and monitoring strategies. Higher incidences of VTE occurred with major surgery (15%-40%), knee or hip arthroplasty (40%-60%), and trauma (60%-80%).¹ The use of VTE prophylaxis with subcutaneous heparin reduced DVTs by 70% and PEs by 50%.² A recent study from Olmstead County showed improved adherence to inpatient VTE prophylaxis from 2005 to 2010 but no difference in VTE incidence. However, 52% of VTE events were associated with hospitalization.³ As such, VTE continues to be a healthcare-associated adverse event, and surgery remains a significant risk factor for thrombosis.⁴

The Agency for Healthcare Research and Quality (AHRQ) released Patient Safety Indicators (PSI) in 2003 to provide a means of screening for adverse events.⁵ Over time, PSIs have been adopted as a measure of hospital performance and are utilized in several pay-for-performance programs. PSI-90 is a composite measure of several other PSIs and is a core metric in the Centers for Medicare and Medicaid Services (CMS) Hospital-Acquired Condition Reduction Program and the Hospital Value-Based Purchasing Program which impacts up to 2% of a hospital’s Medicare payments.^6,7 One component of PSI-90 is PSI-12, which captures perioperative VTE. PSI-12 events are identified using software that screens medical records based on International Classification of Diseases (ICD)-9/10 codes for thrombosis and procedure codes at time of discharge.⁸

Over the last several years, there has been concern regarding the validity of including PSI-12 in pay-for-performance metrics. Common areas for concern include PSI-12’s accuracy in detecting true postoperative VTE⁹ in addition to surveillance bias.^10,11 However, some note that PSI-12 is useful when applied with its original intent: as a screening tool for hospitals to identify specific areas to implement improvements.^9,12The aim of our study was to review all PSI-12 events at our institution to evaluate the accuracy of PSI-12 and identify areas for improvement to prevent VTE events in surgical patients. While several other studies have looked at the positive predictive values, accuracy, or surveillance bias of PSI-12, to the best of our knowledge, few, if any, previous studies have reported PSI-12 events in relation to their timing, type of prophylaxis used, and mitigating factors to identify areas for quality improvement.

PSI-12 events were identified between June 2015 and June 2017 using AHRQ software (version 5). Cases were also identified through Vizient and reviewed to ensure congruence between the methods. Patients’ electronic medical records were reviewed for patient demographics, type of VTE event, platelet count at VTE diagnosis, procedure type, and both the timing and type of VTE prophylaxis. Summary statistics were calculated.

We considered perioperative VTE pharmacologic prophylaxis appropriate if started within 24 hours of a low bleeding risk procedure or 72 hours of a high-bleeding risk procedure.¹³ Mechanical prophylaxis was considered appropriate if pharmacologic prophylaxis was not used because of procedure risk, thrombocytopenia, or active bleeding. The medication administration record was reviewed to determine if prophylaxis was ordered, given, and/or refused.

During the two-year period, 18,084 surgeries were performed, and 161 cases of VTE events were identified. A detailed chart review and correction of documentation led to the exclusion of seven cases (4%) because the VTE event occurred prior to admission (n = 5) or were incidental findings that did not meet the Uniform Hospital Discharge Data Set definition for reporting (n = 2). In total, 154 (0.9% of all surgeries) cases were considered PSI-12 events. Pulmonary embolism (PE) occurred in most cases (n = 97, 62.9%), followed by deep vein thrombosis (DVT) (n = 37, 24.0%). Twenty cases (12.9%) experienced concurrent PE and DVT. Within the PE group, 16 cases (14%) were subsegmental PE only. Eight patients (14% of DVT cases) had only a distal DVT. The mean age of patients was 56 years (+/− 16 years), and the majority (59%) were male. The clinical specialties with the most events included neurosurgery (21%), orthopedics (14%), general surgery (13%), and trauma (11%). Fourteen patients (9%) died during the hospitalization, and of these, six (43%) had either sudden death or death attributed to PE (Table 1).

Cases were also reviewed for the type of VTE prophylactic strategy administered at the time of the event. The top three prophylactic strategies were subcutaneous unfractionated heparin (61%), mechanical prophylaxis only (51%), and enoxaparin (31%). Nine cases of VTE occurred during therapeutic anticoagulation (6%; Table 1).

Utilizing AHRQ version 5 software for PSI-12, our institution identified 154 cases of perioperative VTE. Most of these cases were a pulmonary embolism, occurred within a week of admission, and were associated with surgical specialties that portend a higher risk of VTE. Very few of these cases were deficient in guideline-directed VTE prophylaxis, and several cases had associated factors such as trauma and thrombocytopenia that may have appropriately influenced the decision to use mechanical only prophylaxis. Sixteen percent of pharmacologic VTE prophylactic doses were refused or held for a procedure, a known but rarely quantified influence on rates of pharmacologic prophylaxis. The use of patient-level data and adjudication of the clinical decision that affected the administration of VTE prophylaxis is a major strength of this work. In all, our data raise several questions about the accuracy of PSI-12 in identifying preventable postoperative VTE, especially as it is utilized as a marker for pay-for-performance measures in addition to identifying further areas for research and improvement.

Our results align with previous studies that suggest that PSI-12 is an inaccurate measure of performance quality. A study by Bilimoria et al. in 2013 noted that surveillance biases associated with PSI-12, showing that hospitals with higher compliance to appropriate VTE prophylaxis paradoxically had worse outcomes.¹⁰ Furthermore, Blay et al. recently published a study evaluating PSI-90 scores for hospitals with and without the VTE measure included. Their results indicated that larger hospitals (teaching hospitals, level I trauma centers, etc) caring for sicker patients were noted to improve by 8%-25% when the VTE measure was removed.¹¹ Similarly, our data indicate that the PSI-12 may not be an accurate measure of quality performance, as only 6% of cases were noted to be deficient in appropriate guideline-directed prophylaxis. Even when accounting for the refusal of doses of pharmacologic prophylaxis, this figure only increased to 11%.

Procedures included in the current PSI-12 algorithm also vary in the risk they pose to developing VTE. For example, the current version of PSI-12 includes surgical Medicare Severity Diagnosis Related Groups (MS-DRGs) for procedures with a high risk of VTE such as orthopedic, abdominal, or thoracic procedures.^14,15, However, it is worth noting that procedures such as tracheostomy and ocular surgery are also included.¹⁴ The variation in risk for development of VTE is reflected in the Caprini score, a perioperative risk stratification tool. These latter procedures only contribute one point, whereas trauma would add five points each to the total score, and make the patient a high VTE risk from the procedure alone.¹⁶ With regard to PSI-12, in theory, scores could vary significantly between centers even if the quality of care is the same, based on the volume and risk of procedures performed.

Cervical lymphadenitis is a common superficial neck infection in childhood. While most children with cervical lymphadenitis recover with antibiotic therapy, a subset can develop an abscess that may require surgical drainage. Radiologic imaging, most commonly ultrasound or computed tomography (CT), is often performed to identify such an abscess.^1-3 However, no national standards exist to guide clinician decision making around imaging in this population. In the absence of evidence-based guidelines, variability in frequency, timing, and modality of imaging likely exists in children hospitalized with cervical lymphadenitis.

As demonstrated for several other common pediatric conditions,^4,5 variability in imaging practices may contribute to overutilization of resources in children with cervical lymphadenitis. In particular, routinely conducting imaging on presentation may constitute overuse, as children with cervical lymphadenitis who present with less than 72 hours of neck swelling rarely undergo surgical drainage within the first 24 hours of hospitalization.^1,6,7 Imaging performed on presentation is often repeated later during hospitalization, particularly if the patient has not improved with antibiotic therapy. The net result may be unnecessary, redundant radiologic studies. Furthermore, serious complications such as bacteremia, extension of infection into the retropharyngeal space, or involvement of the airway or vasculature rarely occur in children with cervical lymphadenitis.^6,8 In this context, deferring initial imaging in this population is unlikely to lead to adverse outcomes and may reduce radiation exposure.

The overall objectives of this study are to describe hospital-level variation in imaging practices for pediatric cervical lymphadenitis and to examine the association between early imaging and outcomes in this population.

Study Design and Data Source

We conducted a multicenter, cross-sectional study using the Pediatric Health Information Systems (PHIS) database, which contains administrative and billing data from 49 geographically diverse children’s hospitals across the United States (US) affiliated with the Children’s Hospital Association (Lenexa, Kansas). PHIS includes data on patient demographics, discharge diagnoses, and procedures using the International Classification of Diseases, 9th (ICD-9) and 10th Revision (ICD-10) diagnosis codes, as well as daily billed resource utilization for laboratory tests, imaging studies, and medications. Encrypted medical record numbers permit longitudinal identification of children across multiple visits to the same hospital. Use of de-identified PHIS data was deemed to be nonhuman subjects research; our approach to validation of ICD codes using local electronic medical record review was reviewed and approved by the Cincinnati Children’s Hospital Medical Center Institutional Review Board.

Study Population

Our study team developed an algorithm to identify children with cervical lymphadenitis and minimize misclassification using PHIS (Appendix A). All children with lymphadenitis-related ICD-9 and ICD-10 discharge diagnosis codes were eligible for inclusion. Codes were validated at a single center via electronic medical record review; clinician-documented discharge diagnosis of cervical lymphadenitis or the presence of fever and unilateral or asymmetrical neck swelling with overlying skin changes was used as the reference standard. We then excluded children who did not receive antibiotics, children who received radiologic imaging not involving the head or neck (which suggested noncervical lymphadenitis or other illness), and children who had discharge diagnosis codes for other specified conditions that are sometimes associated with enlarged cervical lymph nodes but warrant different evaluation or treatment (eg, Kawasaki disease, retropharyngeal abscess, and dental abscess; Appendix A). Our final algorithm yielded a positive predictive value of 87.5% (95% CI: 79.2%-93.4%) when ICD-9 codes were considered, and 95.1% (95% CI: 88.9%-98.4%) when ICD-10 codes were considered (Appendix A).

This algorithm was subsequently applied to the PHIS database. Children ages two months to 18 years hospitalized at participating PHIS institutions between July 2013 and December 2017 with a diagnosis of cervical lymphadenitis as per the algorithm (Appendix A) were eligible for inclusion. For children with multiple eligible admissions during the study period, we only included the first hospitalization. Children with complex chronic condition diagnosis codes⁹ were excluded as their clinical complexity could influence decisions around timing and modality of diagnostic imaging. In addition, we excluded children who did not have an emergency department (ED) visit associated with their hospitalization. This step was intended to exclude children who were transferred from another institution, as imaging performed at outside institutions prior to transfer is not available in PHIS. To avoid overinflating hospital-level variation in the setting of a small sample size, we also excluded all children admitted to the five hospitals with fewer than 50 cases of cervical lymphadenitis during the study period. Our final cohort consisted of 44 PHIS hospitals.

Measures of Interest

To examine hospital-level variation in imaging practices, we measured the proportion of children at each hospital who underwent any neck imaging study, CT or ultrasound imaging, early imaging, and multiple imaging studies within a single hospitalization. Neck imaging was defined as the presence of a billing code for ultrasound, CT, or magnetic resonance imaging (MRI) study of the neck (Appendix B). Early imaging was defined as neck imaging conducted on day 0 of hospitalization (ie, calendar day of admission and ending at midnight). Multiple imaging studies were defined as the receipt of more than one imaging study, regardless of timing or modality. We also measured the proportion of children by hospital who received surgical drainage, defined by the presence of procedure codes for incision and drainage of abscess of the neck (Appendix B).

In examining patient-level association between early imaging and clinical outcomes, our primary outcome of interest was the receipt of multiple imaging studies. Secondary outcomes included rates of surgical drainage, length of stay (in hospital days), and rates of lymphadenitis-related hospital readmission within 30 days of index discharge.

Covariates

Baseline demographic characteristics included age, gender, race/ethnicity, and insurance type. We measured ED visits associated with lymphadenitis-related diagnosis codes in the 30 days prior to admission as a proxy measure for illness duration prior to presentation. To approximate illness severity, we included the following covariates: rates of intensive care unit admission on presentation, rates of receipt of intravenous (IV) analgesia (Appendix B) on hospital days prior to surgical drainage, and rates of receipt of broad-spectrum antibiotics on day 0 or 1 of hospitalization. Broad-spectrum antibiotics (Appendix B) were defined by an independent three-person review of available antibiotic codes (SD, SSS, and JT); differences were resolved by group consensus.

Analysis

Categorical variables were described using frequencies and percentages, while continuous data were described using median and interquartile range. We described hospital-level variation in imaging practices by calculating and comparing the proportion of children at each hospital who underwent any neck imaging study, CT imaging, ultrasound imaging, early imaging, multiple imaging studies, and surgical drainage.

Patient-level demographics and clinical characteristics were compared across groups using chi-square test. To examine the association between early imaging and outcomes, we used generalized linear or logistic mixed effects models to control for patient demographic characteristics and clinical markers of illness duration and severity, with a random effect for hospital to account for clustering. Patient demographics in the model defined a priori included age, race/ethnicity, and insurance type; clinical characteristics included prior ED visit for lymphadenitis, initial intensive care unit (ICU) admission, use of IV analgesia, and use of broad-spectrum antibiotics on day 0 or 1 of hospitalization. To assess the potential for misclassification related to the availability of calendar day but not time of imaging in PHIS, we conducted a secondary analysis to examine the patient-level association between early imaging and outcomes using an alternative definition for early imaging (defined as imaging conducted on day 0 or day 1 of hospitalization).

We identified 19,785 PHIS hospitalizations with lymphadenitis-related discharge diagnosis codes between July 1, 2013 and December 31, 2017. Applying our algorithm and exclusion criteria, we assembled a cohort of 10,014 children hospitalized with cervical lymphadenitis (Figure 1). Two-thirds of the children in our cohort were <4 years old, 42% were non-Hispanic white, and 63% had a government payor (Table 1). Neck imaging (ultrasound, CT, or MRI) was conducted in 8,103 (81%) children. CT imaging was performed in 4,097 (41%) of children, and early imaging was conducted in 6,111 (61%) of children with cervical lymphadenitis.

We noted hospital-level variation in rates of any neck imaging (median: 82.1%, interquartile range [IQR]: 77.7%-85.5%, full range: 68.7%-93.1%), CT imaging (median: 42.3%, IQR: 26.7%-55.2%, full range: 12.0%-81.5%), early imaging (median: 64.4%, IQR: 59.8%-68.4%, full range: 13.8%-76.9%), and multiple imaging studies (median: 23.7%, IQR: 18.6%-28.9%, full range: 1.2%-40.7%; Figure 2). Rates of surgical drainage also varied by hospital (median: 35.1%, IQR: 31.3%-42.0%, full range: 17.1%-54.5%).

In this large multicenter study of children with cervical lymphadenitis, we found variation in imaging practices across 44 US children’s hospitals. Children with cervical lymphadenitis who underwent early imaging were more likely to receive multiple imaging studies during a single hospitalization than those who did not receive early imaging. At the patient level, early imaging was also associated with higher rates of surgical drainage, more frequent 30-day readmission, and longer lengths of stay.

To our knowledge, imaging practices in the population of children hospitalized with cervical lymphadenitis have not been previously characterized in the US; one study from Atlanta, Georgia, describes imaging practices in all children evaluated in the ED.¹ Single-center studies of children hospitalized with cervical lymphadenitis have been previously conducted in Canada⁶ and New Zealand,⁸ in which 42%-51% of children received imaging. In our study, most (81%) children hospitalized with lymphadenitis received some form of imaging, with 61% of all children receiving early imaging. Furthermore, 41% received CT imaging, as compared with 8%-10% of children in the aforementioned studies from Canada and New Zealand.^6,8 This finding is consistent with a pattern of imaging overuse in the US, which has amongst the highest utilization rates globally for advanced imaging such as CT and MRI.^10,11 Identifying opportunities to safely reduce routine imaging, particularly CT imaging, in this population could decrease unnecessary radiation exposure without compromising outcomes.

We also noted variability in imaging practices across PHIS hospitals. Some of this variability may be partially explained by differences in the patient population or illness severity across hospitals. However, given the absence of evidence-based best practices for children with cervical lymphadenitis, clinicians may rely on anecdotal experience or local practice culture to guide their decision making,¹² leading to variability in frequency, timing, and modality of imaging.

At the patient level, we found that children who received early imaging were more likely to receive multiple imaging studies. This finding supports our hypothesis that clinicians often order a second imaging study when the initial imaging study does not clearly demonstrate an abscess, and the child subsequently fails to demonstrate clear improvement after 24-48 hours of antibiotics.

Furthermore, early imaging was associated with overall increased utilization in our cohort, including increased likelihood of surgical drainage, 30-day readmission for lymphadenitis, as well as longer lengths of stay. Confounding may be one explanation for this finding. For instance, clinicians may pursue early imaging in children who present with longer duration of symptoms or more severe illness on presentation, as these factors may be associated with abscess formation.^1,6,7 These clinical covariates are not available in PHIS. Thus, we used prior ED visits for lymphadenitis to approximate illness duration, and initial admission to ICU, receipt of IV analgesia, and receipt of broad-spectrum antibiotics to approximate illness severity in an attempt to mitigate confounding. However, our proxy measures may not appropriately estimate illness duration and severity. For instance, children who had urgent care or outpatient visits for lymphadenitis would not be captured using the proxy of prior ED visit for lymphadenitis. Similarly, use of broad-spectrum antibiotics and IV analgesia may be influenced by provider or institutional preference rather than illness severity. Thus, residual confounding may exist despite adjusting for these measures.

On the other hand, it is also possible that a proportion of children with a small fluid collection on imaging may have improved with antibiotics alone. There is a growing body of evidence in children with other head and neck infections (eg, retropharyngeal abscess and orbital cellulitis with periosteal abscess)^13-15 that suggests that children with small abscesses often improve with antibiotic therapy alone. In children with cervical lymphadenitis who have small or developing abscesses identified via routine imaging on presentation, clinicians may be driven to pursue a surgical intervention with uncertain benefit. Deferring routine imaging in this population may provide an opportunity to improve the value of care in children with lymphadenitis without adversely affecting outcomes.

Upon closer examination of readmissions, children who received early imaging during index hospitalization were more likely to have a 30-day readmission when only evaluating the subset of patients who did not receive surgical drainage during the index admission. This suggests that readmissions are less likely attributable to surgical complications and more likely a reflection of the natural history of lymphadenitis in which a subset of patients eventually develop an abscess. Further supporting this, 61% of children who had a 30-day readmission for lymphadenitis underwent surgical drainage during readmission. Given that lymphadenitis is a slow-brewing infection in which serious complications are rare, patients who demonstrate gradual clinical improvement do not need to remain hospitalized and serially imaged to identify a possible abscess. Outpatient expectant management and readmission as needed for drainage may be an acceptable approach.

This study has several limitations given our use of an administrative database. Children with lymphadenitis may have been misclassified as these patients were identified using discharge diagnosis codes. To mitigate this potential misclassification, we conducted a structured validation process and found that the included codes had high positive predictive values (Appendix A). This validation process was conducted at a single hospital, and coding may vary across hospitals. To approximate sensitivity, we also sampled children without our included codes but with neck imaging and antibiotic use, and found that rates of cervical lymphadenitis were very low among children without our included diagnosis codes.

Furthermore, we were unable to measure the exact time of imaging study in PHIS; we used imaging conducted on hospital day 0 as a proxy measure for imaging conducted within the first 24 hours of presentation. With this definition, some children who had early imaging were likely misclassified as not having received early imaging. For example, a patient who arrived in the ED at 9 pm on day 0 of admission and had a neck ultrasound performed at 1 am would be classified as having had an imaging study on day 1 of hospitalization even though the imaging study was conducted within 4 hours of presentation. Using an alternative definition of early imaging as imaging conducted on hospital day 0 and day 1, we found a much higher adjusted OR for multiple imaging studies, with similar associations for secondary outcomes. As such, our definition of early imaging as day 0 likely biases the results toward the null; the true increase in likelihood of multiple imaging for those who receive early imaging is probably greater than our conservative estimation.

Additionally, there may be a subset of children who underwent imaging prior to presentation at the PHIS hospital ED for further workup and admission. Imaging conducted outside a PHIS hospital was not captured in this database. Similarly, children who had a readmission at a different hospital than their index admission would not be captured using PHIS. Finally, PHIS captures data from children’s hospitals; practices at these hospitals may not be generalizable to practices in the community hospital setting.

In conclusion, we found that imaging practices in children hospitalized with cervical lymphadenitis were widely variable across hospitals. Children receiving early imaging had more resource utilization and intervention when compared with children who did not receive early imaging. Our findings may represent a cascade effect, in which routinely conducted early imaging prompts clinicians to pursue more testing and interventions in this population. Future studies should obtain more detailed patient level covariates to further characterize clinical factors that may impact decisions around imaging and clinical outcomes for children with cervical lymphadenitis.

Acknowledgments

The authors would like to acknowledge the following investigators for their contributions to data interpretation and review of the final manuscript: Angela Choe MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Margaret Rush MD, Children’s National Medical Center, Washington, DC; Ryosuke Takei MD, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; Wallis Molchen DO, Texas Children’s Hospital, Houston, Texas; Stephanie Royer Moss MD, Cleveland Clinic, Cleveland, Ohio; Rebecca Dang, MD, Lucile Packard Children’s Hospital Stanford, Palo Alto, California; Joy Solano MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas; Nathaniel P. Goodrich MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Ngozi Eboh MD, Texas Tech University Health Sciences Center, Dallas, Texas; Ashley Jenkins MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Rebecca Steuart MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sonya Tang Girdwood MD, PhD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Alissa McInerney MD, Maria Fareri Children’s Hospital at Westchester Medical Center, Valhalla, New York; Sumeet Banker MD, MPH, New York Presbyterian Morgan Stanley Children’s Hospital, New York, New York; Corrie McDaniel DO, Seattle Children’s Hospital, Seattle, Washington; Christiane Lenzen MD, Rady Children’s Hospital, San Diego, California; Aleisha Nabower MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Waheeda Samady MD, Ann & Robert H. Lurie Children’s Hospital, Chicago, Illinois; Jennifer Chen MD, Rady Children’s Hospital, San Diego, California; Marquita Genies MD, MPH, John’s Hopkins Children’s Center, Baltimore, Maryland; Justin Lockwood MD, Children’s Hospital Colorado, Aurora, Colorado; David Synhorst MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas.

Cervical lymphadenitis is a common superficial neck infection in childhood. While most children with cervical lymphadenitis recover with antibiotic therapy, a subset can develop an abscess that may require surgical drainage. Radiologic imaging, most commonly ultrasound or computed tomography (CT), is often performed to identify such an abscess.^1-3 However, no national standards exist to guide clinician decision making around imaging in this population. In the absence of evidence-based guidelines, variability in frequency, timing, and modality of imaging likely exists in children hospitalized with cervical lymphadenitis.

As demonstrated for several other common pediatric conditions,^4,5 variability in imaging practices may contribute to overutilization of resources in children with cervical lymphadenitis. In particular, routinely conducting imaging on presentation may constitute overuse, as children with cervical lymphadenitis who present with less than 72 hours of neck swelling rarely undergo surgical drainage within the first 24 hours of hospitalization.^1,6,7 Imaging performed on presentation is often repeated later during hospitalization, particularly if the patient has not improved with antibiotic therapy. The net result may be unnecessary, redundant radiologic studies. Furthermore, serious complications such as bacteremia, extension of infection into the retropharyngeal space, or involvement of the airway or vasculature rarely occur in children with cervical lymphadenitis.^6,8 In this context, deferring initial imaging in this population is unlikely to lead to adverse outcomes and may reduce radiation exposure.

The overall objectives of this study are to describe hospital-level variation in imaging practices for pediatric cervical lymphadenitis and to examine the association between early imaging and outcomes in this population.

Study Design and Data Source

We conducted a multicenter, cross-sectional study using the Pediatric Health Information Systems (PHIS) database, which contains administrative and billing data from 49 geographically diverse children’s hospitals across the United States (US) affiliated with the Children’s Hospital Association (Lenexa, Kansas). PHIS includes data on patient demographics, discharge diagnoses, and procedures using the International Classification of Diseases, 9th (ICD-9) and 10th Revision (ICD-10) diagnosis codes, as well as daily billed resource utilization for laboratory tests, imaging studies, and medications. Encrypted medical record numbers permit longitudinal identification of children across multiple visits to the same hospital. Use of de-identified PHIS data was deemed to be nonhuman subjects research; our approach to validation of ICD codes using local electronic medical record review was reviewed and approved by the Cincinnati Children’s Hospital Medical Center Institutional Review Board.

Study Population

Our study team developed an algorithm to identify children with cervical lymphadenitis and minimize misclassification using PHIS (Appendix A). All children with lymphadenitis-related ICD-9 and ICD-10 discharge diagnosis codes were eligible for inclusion. Codes were validated at a single center via electronic medical record review; clinician-documented discharge diagnosis of cervical lymphadenitis or the presence of fever and unilateral or asymmetrical neck swelling with overlying skin changes was used as the reference standard. We then excluded children who did not receive antibiotics, children who received radiologic imaging not involving the head or neck (which suggested noncervical lymphadenitis or other illness), and children who had discharge diagnosis codes for other specified conditions that are sometimes associated with enlarged cervical lymph nodes but warrant different evaluation or treatment (eg, Kawasaki disease, retropharyngeal abscess, and dental abscess; Appendix A). Our final algorithm yielded a positive predictive value of 87.5% (95% CI: 79.2%-93.4%) when ICD-9 codes were considered, and 95.1% (95% CI: 88.9%-98.4%) when ICD-10 codes were considered (Appendix A).

This algorithm was subsequently applied to the PHIS database. Children ages two months to 18 years hospitalized at participating PHIS institutions between July 2013 and December 2017 with a diagnosis of cervical lymphadenitis as per the algorithm (Appendix A) were eligible for inclusion. For children with multiple eligible admissions during the study period, we only included the first hospitalization. Children with complex chronic condition diagnosis codes⁹ were excluded as their clinical complexity could influence decisions around timing and modality of diagnostic imaging. In addition, we excluded children who did not have an emergency department (ED) visit associated with their hospitalization. This step was intended to exclude children who were transferred from another institution, as imaging performed at outside institutions prior to transfer is not available in PHIS. To avoid overinflating hospital-level variation in the setting of a small sample size, we also excluded all children admitted to the five hospitals with fewer than 50 cases of cervical lymphadenitis during the study period. Our final cohort consisted of 44 PHIS hospitals.

Measures of Interest

To examine hospital-level variation in imaging practices, we measured the proportion of children at each hospital who underwent any neck imaging study, CT or ultrasound imaging, early imaging, and multiple imaging studies within a single hospitalization. Neck imaging was defined as the presence of a billing code for ultrasound, CT, or magnetic resonance imaging (MRI) study of the neck (Appendix B). Early imaging was defined as neck imaging conducted on day 0 of hospitalization (ie, calendar day of admission and ending at midnight). Multiple imaging studies were defined as the receipt of more than one imaging study, regardless of timing or modality. We also measured the proportion of children by hospital who received surgical drainage, defined by the presence of procedure codes for incision and drainage of abscess of the neck (Appendix B).

In examining patient-level association between early imaging and clinical outcomes, our primary outcome of interest was the receipt of multiple imaging studies. Secondary outcomes included rates of surgical drainage, length of stay (in hospital days), and rates of lymphadenitis-related hospital readmission within 30 days of index discharge.

Covariates

Baseline demographic characteristics included age, gender, race/ethnicity, and insurance type. We measured ED visits associated with lymphadenitis-related diagnosis codes in the 30 days prior to admission as a proxy measure for illness duration prior to presentation. To approximate illness severity, we included the following covariates: rates of intensive care unit admission on presentation, rates of receipt of intravenous (IV) analgesia (Appendix B) on hospital days prior to surgical drainage, and rates of receipt of broad-spectrum antibiotics on day 0 or 1 of hospitalization. Broad-spectrum antibiotics (Appendix B) were defined by an independent three-person review of available antibiotic codes (SD, SSS, and JT); differences were resolved by group consensus.

Analysis

Categorical variables were described using frequencies and percentages, while continuous data were described using median and interquartile range. We described hospital-level variation in imaging practices by calculating and comparing the proportion of children at each hospital who underwent any neck imaging study, CT imaging, ultrasound imaging, early imaging, multiple imaging studies, and surgical drainage.

Patient-level demographics and clinical characteristics were compared across groups using chi-square test. To examine the association between early imaging and outcomes, we used generalized linear or logistic mixed effects models to control for patient demographic characteristics and clinical markers of illness duration and severity, with a random effect for hospital to account for clustering. Patient demographics in the model defined a priori included age, race/ethnicity, and insurance type; clinical characteristics included prior ED visit for lymphadenitis, initial intensive care unit (ICU) admission, use of IV analgesia, and use of broad-spectrum antibiotics on day 0 or 1 of hospitalization. To assess the potential for misclassification related to the availability of calendar day but not time of imaging in PHIS, we conducted a secondary analysis to examine the patient-level association between early imaging and outcomes using an alternative definition for early imaging (defined as imaging conducted on day 0 or day 1 of hospitalization).

We identified 19,785 PHIS hospitalizations with lymphadenitis-related discharge diagnosis codes between July 1, 2013 and December 31, 2017. Applying our algorithm and exclusion criteria, we assembled a cohort of 10,014 children hospitalized with cervical lymphadenitis (Figure 1). Two-thirds of the children in our cohort were <4 years old, 42% were non-Hispanic white, and 63% had a government payor (Table 1). Neck imaging (ultrasound, CT, or MRI) was conducted in 8,103 (81%) children. CT imaging was performed in 4,097 (41%) of children, and early imaging was conducted in 6,111 (61%) of children with cervical lymphadenitis.

We noted hospital-level variation in rates of any neck imaging (median: 82.1%, interquartile range [IQR]: 77.7%-85.5%, full range: 68.7%-93.1%), CT imaging (median: 42.3%, IQR: 26.7%-55.2%, full range: 12.0%-81.5%), early imaging (median: 64.4%, IQR: 59.8%-68.4%, full range: 13.8%-76.9%), and multiple imaging studies (median: 23.7%, IQR: 18.6%-28.9%, full range: 1.2%-40.7%; Figure 2). Rates of surgical drainage also varied by hospital (median: 35.1%, IQR: 31.3%-42.0%, full range: 17.1%-54.5%).

In this large multicenter study of children with cervical lymphadenitis, we found variation in imaging practices across 44 US children’s hospitals. Children with cervical lymphadenitis who underwent early imaging were more likely to receive multiple imaging studies during a single hospitalization than those who did not receive early imaging. At the patient level, early imaging was also associated with higher rates of surgical drainage, more frequent 30-day readmission, and longer lengths of stay.

To our knowledge, imaging practices in the population of children hospitalized with cervical lymphadenitis have not been previously characterized in the US; one study from Atlanta, Georgia, describes imaging practices in all children evaluated in the ED.¹ Single-center studies of children hospitalized with cervical lymphadenitis have been previously conducted in Canada⁶ and New Zealand,⁸ in which 42%-51% of children received imaging. In our study, most (81%) children hospitalized with lymphadenitis received some form of imaging, with 61% of all children receiving early imaging. Furthermore, 41% received CT imaging, as compared with 8%-10% of children in the aforementioned studies from Canada and New Zealand.^6,8 This finding is consistent with a pattern of imaging overuse in the US, which has amongst the highest utilization rates globally for advanced imaging such as CT and MRI.^10,11 Identifying opportunities to safely reduce routine imaging, particularly CT imaging, in this population could decrease unnecessary radiation exposure without compromising outcomes.

We also noted variability in imaging practices across PHIS hospitals. Some of this variability may be partially explained by differences in the patient population or illness severity across hospitals. However, given the absence of evidence-based best practices for children with cervical lymphadenitis, clinicians may rely on anecdotal experience or local practice culture to guide their decision making,¹² leading to variability in frequency, timing, and modality of imaging.

At the patient level, we found that children who received early imaging were more likely to receive multiple imaging studies. This finding supports our hypothesis that clinicians often order a second imaging study when the initial imaging study does not clearly demonstrate an abscess, and the child subsequently fails to demonstrate clear improvement after 24-48 hours of antibiotics.

Furthermore, early imaging was associated with overall increased utilization in our cohort, including increased likelihood of surgical drainage, 30-day readmission for lymphadenitis, as well as longer lengths of stay. Confounding may be one explanation for this finding. For instance, clinicians may pursue early imaging in children who present with longer duration of symptoms or more severe illness on presentation, as these factors may be associated with abscess formation.^1,6,7 These clinical covariates are not available in PHIS. Thus, we used prior ED visits for lymphadenitis to approximate illness duration, and initial admission to ICU, receipt of IV analgesia, and receipt of broad-spectrum antibiotics to approximate illness severity in an attempt to mitigate confounding. However, our proxy measures may not appropriately estimate illness duration and severity. For instance, children who had urgent care or outpatient visits for lymphadenitis would not be captured using the proxy of prior ED visit for lymphadenitis. Similarly, use of broad-spectrum antibiotics and IV analgesia may be influenced by provider or institutional preference rather than illness severity. Thus, residual confounding may exist despite adjusting for these measures.

On the other hand, it is also possible that a proportion of children with a small fluid collection on imaging may have improved with antibiotics alone. There is a growing body of evidence in children with other head and neck infections (eg, retropharyngeal abscess and orbital cellulitis with periosteal abscess)^13-15 that suggests that children with small abscesses often improve with antibiotic therapy alone. In children with cervical lymphadenitis who have small or developing abscesses identified via routine imaging on presentation, clinicians may be driven to pursue a surgical intervention with uncertain benefit. Deferring routine imaging in this population may provide an opportunity to improve the value of care in children with lymphadenitis without adversely affecting outcomes.

Upon closer examination of readmissions, children who received early imaging during index hospitalization were more likely to have a 30-day readmission when only evaluating the subset of patients who did not receive surgical drainage during the index admission. This suggests that readmissions are less likely attributable to surgical complications and more likely a reflection of the natural history of lymphadenitis in which a subset of patients eventually develop an abscess. Further supporting this, 61% of children who had a 30-day readmission for lymphadenitis underwent surgical drainage during readmission. Given that lymphadenitis is a slow-brewing infection in which serious complications are rare, patients who demonstrate gradual clinical improvement do not need to remain hospitalized and serially imaged to identify a possible abscess. Outpatient expectant management and readmission as needed for drainage may be an acceptable approach.

This study has several limitations given our use of an administrative database. Children with lymphadenitis may have been misclassified as these patients were identified using discharge diagnosis codes. To mitigate this potential misclassification, we conducted a structured validation process and found that the included codes had high positive predictive values (Appendix A). This validation process was conducted at a single hospital, and coding may vary across hospitals. To approximate sensitivity, we also sampled children without our included codes but with neck imaging and antibiotic use, and found that rates of cervical lymphadenitis were very low among children without our included diagnosis codes.

Furthermore, we were unable to measure the exact time of imaging study in PHIS; we used imaging conducted on hospital day 0 as a proxy measure for imaging conducted within the first 24 hours of presentation. With this definition, some children who had early imaging were likely misclassified as not having received early imaging. For example, a patient who arrived in the ED at 9 pm on day 0 of admission and had a neck ultrasound performed at 1 am would be classified as having had an imaging study on day 1 of hospitalization even though the imaging study was conducted within 4 hours of presentation. Using an alternative definition of early imaging as imaging conducted on hospital day 0 and day 1, we found a much higher adjusted OR for multiple imaging studies, with similar associations for secondary outcomes. As such, our definition of early imaging as day 0 likely biases the results toward the null; the true increase in likelihood of multiple imaging for those who receive early imaging is probably greater than our conservative estimation.

Additionally, there may be a subset of children who underwent imaging prior to presentation at the PHIS hospital ED for further workup and admission. Imaging conducted outside a PHIS hospital was not captured in this database. Similarly, children who had a readmission at a different hospital than their index admission would not be captured using PHIS. Finally, PHIS captures data from children’s hospitals; practices at these hospitals may not be generalizable to practices in the community hospital setting.

In conclusion, we found that imaging practices in children hospitalized with cervical lymphadenitis were widely variable across hospitals. Children receiving early imaging had more resource utilization and intervention when compared with children who did not receive early imaging. Our findings may represent a cascade effect, in which routinely conducted early imaging prompts clinicians to pursue more testing and interventions in this population. Future studies should obtain more detailed patient level covariates to further characterize clinical factors that may impact decisions around imaging and clinical outcomes for children with cervical lymphadenitis.

Acknowledgments

The authors would like to acknowledge the following investigators for their contributions to data interpretation and review of the final manuscript: Angela Choe MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Margaret Rush MD, Children’s National Medical Center, Washington, DC; Ryosuke Takei MD, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; Wallis Molchen DO, Texas Children’s Hospital, Houston, Texas; Stephanie Royer Moss MD, Cleveland Clinic, Cleveland, Ohio; Rebecca Dang, MD, Lucile Packard Children’s Hospital Stanford, Palo Alto, California; Joy Solano MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas; Nathaniel P. Goodrich MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Ngozi Eboh MD, Texas Tech University Health Sciences Center, Dallas, Texas; Ashley Jenkins MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Rebecca Steuart MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sonya Tang Girdwood MD, PhD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Alissa McInerney MD, Maria Fareri Children’s Hospital at Westchester Medical Center, Valhalla, New York; Sumeet Banker MD, MPH, New York Presbyterian Morgan Stanley Children’s Hospital, New York, New York; Corrie McDaniel DO, Seattle Children’s Hospital, Seattle, Washington; Christiane Lenzen MD, Rady Children’s Hospital, San Diego, California; Aleisha Nabower MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Waheeda Samady MD, Ann & Robert H. Lurie Children’s Hospital, Chicago, Illinois; Jennifer Chen MD, Rady Children’s Hospital, San Diego, California; Marquita Genies MD, MPH, John’s Hopkins Children’s Center, Baltimore, Maryland; Justin Lockwood MD, Children’s Hospital Colorado, Aurora, Colorado; David Synhorst MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas.

Cervical lymphadenitis is a common superficial neck infection in childhood. While most children with cervical lymphadenitis recover with antibiotic therapy, a subset can develop an abscess that may require surgical drainage. Radiologic imaging, most commonly ultrasound or computed tomography (CT), is often performed to identify such an abscess.^1-3 However, no national standards exist to guide clinician decision making around imaging in this population. In the absence of evidence-based guidelines, variability in frequency, timing, and modality of imaging likely exists in children hospitalized with cervical lymphadenitis.

As demonstrated for several other common pediatric conditions,^4,5 variability in imaging practices may contribute to overutilization of resources in children with cervical lymphadenitis. In particular, routinely conducting imaging on presentation may constitute overuse, as children with cervical lymphadenitis who present with less than 72 hours of neck swelling rarely undergo surgical drainage within the first 24 hours of hospitalization.^1,6,7 Imaging performed on presentation is often repeated later during hospitalization, particularly if the patient has not improved with antibiotic therapy. The net result may be unnecessary, redundant radiologic studies. Furthermore, serious complications such as bacteremia, extension of infection into the retropharyngeal space, or involvement of the airway or vasculature rarely occur in children with cervical lymphadenitis.^6,8 In this context, deferring initial imaging in this population is unlikely to lead to adverse outcomes and may reduce radiation exposure.

The overall objectives of this study are to describe hospital-level variation in imaging practices for pediatric cervical lymphadenitis and to examine the association between early imaging and outcomes in this population.

Study Design and Data Source

We conducted a multicenter, cross-sectional study using the Pediatric Health Information Systems (PHIS) database, which contains administrative and billing data from 49 geographically diverse children’s hospitals across the United States (US) affiliated with the Children’s Hospital Association (Lenexa, Kansas). PHIS includes data on patient demographics, discharge diagnoses, and procedures using the International Classification of Diseases, 9th (ICD-9) and 10th Revision (ICD-10) diagnosis codes, as well as daily billed resource utilization for laboratory tests, imaging studies, and medications. Encrypted medical record numbers permit longitudinal identification of children across multiple visits to the same hospital. Use of de-identified PHIS data was deemed to be nonhuman subjects research; our approach to validation of ICD codes using local electronic medical record review was reviewed and approved by the Cincinnati Children’s Hospital Medical Center Institutional Review Board.

Study Population

Our study team developed an algorithm to identify children with cervical lymphadenitis and minimize misclassification using PHIS (Appendix A). All children with lymphadenitis-related ICD-9 and ICD-10 discharge diagnosis codes were eligible for inclusion. Codes were validated at a single center via electronic medical record review; clinician-documented discharge diagnosis of cervical lymphadenitis or the presence of fever and unilateral or asymmetrical neck swelling with overlying skin changes was used as the reference standard. We then excluded children who did not receive antibiotics, children who received radiologic imaging not involving the head or neck (which suggested noncervical lymphadenitis or other illness), and children who had discharge diagnosis codes for other specified conditions that are sometimes associated with enlarged cervical lymph nodes but warrant different evaluation or treatment (eg, Kawasaki disease, retropharyngeal abscess, and dental abscess; Appendix A). Our final algorithm yielded a positive predictive value of 87.5% (95% CI: 79.2%-93.4%) when ICD-9 codes were considered, and 95.1% (95% CI: 88.9%-98.4%) when ICD-10 codes were considered (Appendix A).

This algorithm was subsequently applied to the PHIS database. Children ages two months to 18 years hospitalized at participating PHIS institutions between July 2013 and December 2017 with a diagnosis of cervical lymphadenitis as per the algorithm (Appendix A) were eligible for inclusion. For children with multiple eligible admissions during the study period, we only included the first hospitalization. Children with complex chronic condition diagnosis codes⁹ were excluded as their clinical complexity could influence decisions around timing and modality of diagnostic imaging. In addition, we excluded children who did not have an emergency department (ED) visit associated with their hospitalization. This step was intended to exclude children who were transferred from another institution, as imaging performed at outside institutions prior to transfer is not available in PHIS. To avoid overinflating hospital-level variation in the setting of a small sample size, we also excluded all children admitted to the five hospitals with fewer than 50 cases of cervical lymphadenitis during the study period. Our final cohort consisted of 44 PHIS hospitals.

Measures of Interest

To examine hospital-level variation in imaging practices, we measured the proportion of children at each hospital who underwent any neck imaging study, CT or ultrasound imaging, early imaging, and multiple imaging studies within a single hospitalization. Neck imaging was defined as the presence of a billing code for ultrasound, CT, or magnetic resonance imaging (MRI) study of the neck (Appendix B). Early imaging was defined as neck imaging conducted on day 0 of hospitalization (ie, calendar day of admission and ending at midnight). Multiple imaging studies were defined as the receipt of more than one imaging study, regardless of timing or modality. We also measured the proportion of children by hospital who received surgical drainage, defined by the presence of procedure codes for incision and drainage of abscess of the neck (Appendix B).

In examining patient-level association between early imaging and clinical outcomes, our primary outcome of interest was the receipt of multiple imaging studies. Secondary outcomes included rates of surgical drainage, length of stay (in hospital days), and rates of lymphadenitis-related hospital readmission within 30 days of index discharge.

Covariates

Baseline demographic characteristics included age, gender, race/ethnicity, and insurance type. We measured ED visits associated with lymphadenitis-related diagnosis codes in the 30 days prior to admission as a proxy measure for illness duration prior to presentation. To approximate illness severity, we included the following covariates: rates of intensive care unit admission on presentation, rates of receipt of intravenous (IV) analgesia (Appendix B) on hospital days prior to surgical drainage, and rates of receipt of broad-spectrum antibiotics on day 0 or 1 of hospitalization. Broad-spectrum antibiotics (Appendix B) were defined by an independent three-person review of available antibiotic codes (SD, SSS, and JT); differences were resolved by group consensus.

Analysis

Categorical variables were described using frequencies and percentages, while continuous data were described using median and interquartile range. We described hospital-level variation in imaging practices by calculating and comparing the proportion of children at each hospital who underwent any neck imaging study, CT imaging, ultrasound imaging, early imaging, multiple imaging studies, and surgical drainage.

Patient-level demographics and clinical characteristics were compared across groups using chi-square test. To examine the association between early imaging and outcomes, we used generalized linear or logistic mixed effects models to control for patient demographic characteristics and clinical markers of illness duration and severity, with a random effect for hospital to account for clustering. Patient demographics in the model defined a priori included age, race/ethnicity, and insurance type; clinical characteristics included prior ED visit for lymphadenitis, initial intensive care unit (ICU) admission, use of IV analgesia, and use of broad-spectrum antibiotics on day 0 or 1 of hospitalization. To assess the potential for misclassification related to the availability of calendar day but not time of imaging in PHIS, we conducted a secondary analysis to examine the patient-level association between early imaging and outcomes using an alternative definition for early imaging (defined as imaging conducted on day 0 or day 1 of hospitalization).

We identified 19,785 PHIS hospitalizations with lymphadenitis-related discharge diagnosis codes between July 1, 2013 and December 31, 2017. Applying our algorithm and exclusion criteria, we assembled a cohort of 10,014 children hospitalized with cervical lymphadenitis (Figure 1). Two-thirds of the children in our cohort were <4 years old, 42% were non-Hispanic white, and 63% had a government payor (Table 1). Neck imaging (ultrasound, CT, or MRI) was conducted in 8,103 (81%) children. CT imaging was performed in 4,097 (41%) of children, and early imaging was conducted in 6,111 (61%) of children with cervical lymphadenitis.

We noted hospital-level variation in rates of any neck imaging (median: 82.1%, interquartile range [IQR]: 77.7%-85.5%, full range: 68.7%-93.1%), CT imaging (median: 42.3%, IQR: 26.7%-55.2%, full range: 12.0%-81.5%), early imaging (median: 64.4%, IQR: 59.8%-68.4%, full range: 13.8%-76.9%), and multiple imaging studies (median: 23.7%, IQR: 18.6%-28.9%, full range: 1.2%-40.7%; Figure 2). Rates of surgical drainage also varied by hospital (median: 35.1%, IQR: 31.3%-42.0%, full range: 17.1%-54.5%).

In this large multicenter study of children with cervical lymphadenitis, we found variation in imaging practices across 44 US children’s hospitals. Children with cervical lymphadenitis who underwent early imaging were more likely to receive multiple imaging studies during a single hospitalization than those who did not receive early imaging. At the patient level, early imaging was also associated with higher rates of surgical drainage, more frequent 30-day readmission, and longer lengths of stay.

To our knowledge, imaging practices in the population of children hospitalized with cervical lymphadenitis have not been previously characterized in the US; one study from Atlanta, Georgia, describes imaging practices in all children evaluated in the ED.¹ Single-center studies of children hospitalized with cervical lymphadenitis have been previously conducted in Canada⁶ and New Zealand,⁸ in which 42%-51% of children received imaging. In our study, most (81%) children hospitalized with lymphadenitis received some form of imaging, with 61% of all children receiving early imaging. Furthermore, 41% received CT imaging, as compared with 8%-10% of children in the aforementioned studies from Canada and New Zealand.^6,8 This finding is consistent with a pattern of imaging overuse in the US, which has amongst the highest utilization rates globally for advanced imaging such as CT and MRI.^10,11 Identifying opportunities to safely reduce routine imaging, particularly CT imaging, in this population could decrease unnecessary radiation exposure without compromising outcomes.

We also noted variability in imaging practices across PHIS hospitals. Some of this variability may be partially explained by differences in the patient population or illness severity across hospitals. However, given the absence of evidence-based best practices for children with cervical lymphadenitis, clinicians may rely on anecdotal experience or local practice culture to guide their decision making,¹² leading to variability in frequency, timing, and modality of imaging.

At the patient level, we found that children who received early imaging were more likely to receive multiple imaging studies. This finding supports our hypothesis that clinicians often order a second imaging study when the initial imaging study does not clearly demonstrate an abscess, and the child subsequently fails to demonstrate clear improvement after 24-48 hours of antibiotics.

Furthermore, early imaging was associated with overall increased utilization in our cohort, including increased likelihood of surgical drainage, 30-day readmission for lymphadenitis, as well as longer lengths of stay. Confounding may be one explanation for this finding. For instance, clinicians may pursue early imaging in children who present with longer duration of symptoms or more severe illness on presentation, as these factors may be associated with abscess formation.^1,6,7 These clinical covariates are not available in PHIS. Thus, we used prior ED visits for lymphadenitis to approximate illness duration, and initial admission to ICU, receipt of IV analgesia, and receipt of broad-spectrum antibiotics to approximate illness severity in an attempt to mitigate confounding. However, our proxy measures may not appropriately estimate illness duration and severity. For instance, children who had urgent care or outpatient visits for lymphadenitis would not be captured using the proxy of prior ED visit for lymphadenitis. Similarly, use of broad-spectrum antibiotics and IV analgesia may be influenced by provider or institutional preference rather than illness severity. Thus, residual confounding may exist despite adjusting for these measures.

On the other hand, it is also possible that a proportion of children with a small fluid collection on imaging may have improved with antibiotics alone. There is a growing body of evidence in children with other head and neck infections (eg, retropharyngeal abscess and orbital cellulitis with periosteal abscess)^13-15 that suggests that children with small abscesses often improve with antibiotic therapy alone. In children with cervical lymphadenitis who have small or developing abscesses identified via routine imaging on presentation, clinicians may be driven to pursue a surgical intervention with uncertain benefit. Deferring routine imaging in this population may provide an opportunity to improve the value of care in children with lymphadenitis without adversely affecting outcomes.

Upon closer examination of readmissions, children who received early imaging during index hospitalization were more likely to have a 30-day readmission when only evaluating the subset of patients who did not receive surgical drainage during the index admission. This suggests that readmissions are less likely attributable to surgical complications and more likely a reflection of the natural history of lymphadenitis in which a subset of patients eventually develop an abscess. Further supporting this, 61% of children who had a 30-day readmission for lymphadenitis underwent surgical drainage during readmission. Given that lymphadenitis is a slow-brewing infection in which serious complications are rare, patients who demonstrate gradual clinical improvement do not need to remain hospitalized and serially imaged to identify a possible abscess. Outpatient expectant management and readmission as needed for drainage may be an acceptable approach.

This study has several limitations given our use of an administrative database. Children with lymphadenitis may have been misclassified as these patients were identified using discharge diagnosis codes. To mitigate this potential misclassification, we conducted a structured validation process and found that the included codes had high positive predictive values (Appendix A). This validation process was conducted at a single hospital, and coding may vary across hospitals. To approximate sensitivity, we also sampled children without our included codes but with neck imaging and antibiotic use, and found that rates of cervical lymphadenitis were very low among children without our included diagnosis codes.

Furthermore, we were unable to measure the exact time of imaging study in PHIS; we used imaging conducted on hospital day 0 as a proxy measure for imaging conducted within the first 24 hours of presentation. With this definition, some children who had early imaging were likely misclassified as not having received early imaging. For example, a patient who arrived in the ED at 9 pm on day 0 of admission and had a neck ultrasound performed at 1 am would be classified as having had an imaging study on day 1 of hospitalization even though the imaging study was conducted within 4 hours of presentation. Using an alternative definition of early imaging as imaging conducted on hospital day 0 and day 1, we found a much higher adjusted OR for multiple imaging studies, with similar associations for secondary outcomes. As such, our definition of early imaging as day 0 likely biases the results toward the null; the true increase in likelihood of multiple imaging for those who receive early imaging is probably greater than our conservative estimation.

Additionally, there may be a subset of children who underwent imaging prior to presentation at the PHIS hospital ED for further workup and admission. Imaging conducted outside a PHIS hospital was not captured in this database. Similarly, children who had a readmission at a different hospital than their index admission would not be captured using PHIS. Finally, PHIS captures data from children’s hospitals; practices at these hospitals may not be generalizable to practices in the community hospital setting.

In conclusion, we found that imaging practices in children hospitalized with cervical lymphadenitis were widely variable across hospitals. Children receiving early imaging had more resource utilization and intervention when compared with children who did not receive early imaging. Our findings may represent a cascade effect, in which routinely conducted early imaging prompts clinicians to pursue more testing and interventions in this population. Future studies should obtain more detailed patient level covariates to further characterize clinical factors that may impact decisions around imaging and clinical outcomes for children with cervical lymphadenitis.

Acknowledgments

The authors would like to acknowledge the following investigators for their contributions to data interpretation and review of the final manuscript: Angela Choe MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Margaret Rush MD, Children’s National Medical Center, Washington, DC; Ryosuke Takei MD, Children’s Hospital of Philadelphia, Philadelphia, Pennsylvania; Wallis Molchen DO, Texas Children’s Hospital, Houston, Texas; Stephanie Royer Moss MD, Cleveland Clinic, Cleveland, Ohio; Rebecca Dang, MD, Lucile Packard Children’s Hospital Stanford, Palo Alto, California; Joy Solano MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas; Nathaniel P. Goodrich MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Ngozi Eboh MD, Texas Tech University Health Sciences Center, Dallas, Texas; Ashley Jenkins MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Rebecca Steuart MD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Sonya Tang Girdwood MD, PhD, Cincinnati Children’s Hospital Medical Center, Cincinnati, Ohio; Alissa McInerney MD, Maria Fareri Children’s Hospital at Westchester Medical Center, Valhalla, New York; Sumeet Banker MD, MPH, New York Presbyterian Morgan Stanley Children’s Hospital, New York, New York; Corrie McDaniel DO, Seattle Children’s Hospital, Seattle, Washington; Christiane Lenzen MD, Rady Children’s Hospital, San Diego, California; Aleisha Nabower MD, Children’s Hospital & Medical Center, Omaha, Nebraska; Waheeda Samady MD, Ann & Robert H. Lurie Children’s Hospital, Chicago, Illinois; Jennifer Chen MD, Rady Children’s Hospital, San Diego, California; Marquita Genies MD, MPH, John’s Hopkins Children’s Center, Baltimore, Maryland; Justin Lockwood MD, Children’s Hospital Colorado, Aurora, Colorado; David Synhorst MD, Children’s Mercy Hospital Kansas, Overland Park, Kansas.

Sepsis is defined as a life-threatening organ dysfunction that occurs in response to systemic infection.^1,2 It is frequently fatal, common in hospital medicine, and a leading contributor to critical illness, morbidity, and healthcare expenditures.^2-5 While sepsis care and outcomes have improved in the past decade,^6,7 inpatient mortality remains high.⁸

A number of studies have sought to determine whether race plays a role in sepsis mortality. While Black patients with sepsis have frequently been identified as having the highest rates of death,^9-14 similar observations have been made for most non-White races/ethnicities.^13-15 Studies have also demonstrated higher rates of hospital-acquired infections among Asian and Latino patients.¹⁶

There are several possible explanations for why racial minorities experience disparate outcomes in sepsis, including access to care, comorbidities, implicit biases, and biological or environmental factors,^17-20 as well as characteristics of hospitals most likely to care for racial minorities.^13,15,21 One explanation that has not been explored is that racial disparities in sepsis are mediated by language. Limited English proficiency (LEP) has previously been associated with increased rates of adverse hospital events,²² longer length of stay,²³ and greater likelihood of readmission.²⁴ LEP has also been shown to represent a significant barrier to accessing healthcare and preventive screening.²⁵ The role of LEP in sepsis mortality, however, has yet to be examined.

The diverse patient population at the University of California, San Francisco (UCSF) provides a unique opportunity to build upon existing literature by further exploring racial differences in sepsis, specifically by investigating the role of LEP. The objective of this study was to determine the association between LEP and inpatient mortality among adults hospitalized with sepsis.

Setting

The study was conducted at the University of California, San Francisco, California (UCSF), an 800-bed tertiary care, academic medical center. It was approved by the UCSF Institutional Review Board with waiver of informed consent. UCSF cares for a population of patients who are racially and linguistically diverse, with high proportions of patients of East Asian descent and with LEP. According to recent United States census estimates, more than half of San Francisco County residents identify as non-White (35% Asians, 15% Hispanic/Latino, 6% Black), and 44% report speaking a language other than English at home.²⁶

Study Population and Data Collection

The UCSF Medical Center uses the electronic health record (EHR) Epic (Epic 2017, Epic Systems Corporation, Verona, Wisconsin). We obtained computerized EHR data from Clarity, the relational database that stores Epic’s inpatient data in thousands of tables. We identified all patients ≥18 years of age presenting to the emergency department (ED) between June 1, 2012 and December 31, 2016 with suspected serious infection, defined as having blood cultures ordered within 72 hours of ED presentation (N = 25,441). Patients who did not receive at least two doses of intravenous (IV) antibiotics within 48 hours were excluded, as they were unlikely to have serious infections.

We defined sepsis based on Sepsis-3 consensus guidelines² as a change in sequential [sepsis-related] organ failure assessment (SOFA) score ≥2 within the first 48 hours of ED presentation. The SOFA score is comprised of six variables representing different organ systems, each rated 0-4 based on the degree of dysfunction.² Patient vital signs, laboratory data, vasopressor medication doses, and ventilator settings were used to determine the exact timestamp at which each patient attained a change in SOFA score ≥2. Missing values were considered to be normal. To adjust for baseline organ dysfunction, SOFA elements associated with elevated bilirubin and/or creatinine were excluded for patients with chronic liver/kidney disease based on Elixhauser comorbidities.²⁷ We chose to focus on the first 48 hours in an attempt to capture patients with the most severe illnesses and the highest probability of true sepsis.

All primary and secondary International Classification of Diseases (ICD)-9/10 diagnosis codes were extracted from Clarity coding tables at the time of hospital discharge. Diagnosis codes signifying bacterial infection were grouped into the following categories based on type/location: pneumonia; bacteremia; urinary tract infection; and skin and soft tissue infection. All remaining diagnostic codes indicating bacterial infections at other sites were categorized as “Other”. If no codes indicating infection were present, patients were categorized as “None coded”. Patients with discharge diagnosis codes of “sepsis” were also identified. Dates and times of antibiotic administrations were obtained from the medications table. Time to first antibiotic was defined as the time in minutes from ED presentation to initiation of the first IV antibacterial medication. This variable was transformed using a natural log transformation based on best fit for normal distribution.

We limited our analyses to 8,974 patients who were diagnosed with sepsis as defined above and had either (1) ≥4 qualifying antibiotic days (QADs) or (2) an ICD-9/10 discharge diagnosis code of “sepsis” (Figure). QADs were defined based on the recent publication by Rhee et al. as having received four or more consecutive days of antibiotics, with the first dose given IV within 48 hours of presentation.²⁸ Patients who died or were discharged to hospice prior to the 4th QAD were also included. These additional parameters were added to increase specificity of the study sample for patients with true sepsis. Patients admitted to all levels of care (acute care, transitional care unit [TCU], intensive care unit [ICU]) and under all hospital services were included. There were no missing data for mortality, race, or language. We chose to focus on patients with sepsis in this initial study as this is a common diagnosis in hospital medicine that is enriched for high mortality.

Primary Outcome

The primary outcome of the study was inpatient mortality, which was obtained from the hospital encounters table in Clarity.

Primary Predictors

The primary predictor of interest was LEP. The encounter numbers from the dataset were used to link to self-reported demographic data, including “preferred language” and need for interpreter services. A manual chart review of 60 patients speaking the top six languages was conducted to verify the accuracy of the data on language and interpreter use (KNK). Defining the gold standard for LEP as having any chart note indicating non-English language and/or that an interpreter was used, the “interpreter needed” variable in Epic was found to have a positive predictive value for LEP of 100%. Therefore, patients in the study cohort were defined as having LEP if they met both of the following criteria: (1) a self-reported “preferred language” other than English and (2) having the “interpreter needed” variable indicating “yes”.

Covariate Data Collection

Additional data were obtained from the demographics tables, including age, race, sex, and insurance status. Race and ethnicity were combined into a single five-category variable including White, Asian, Black, Latino, and Other. This approach has been suggested as the best way to operationalize these variables²⁹ and has been utilized by similar studies in the literature.^9,14,15 We considered the Asian race to include all people of East Asian, Southeast Asian, or South Asian descent, which is consistent with the United States Census Bureau definition.³⁰ Patients identifying as Native Hawaiians/Pacific Islanders, Native Americans/Alaskan Natives, as well as those with unspecified race or ethnicity, were categorized as Other. Insurance status was categorized as Commercial, Medicare, Medicaid, or Other.

We estimated illness severity in several ways. First, the total qualifying SOFA score was calculated for each patient, which was defined as the total score achieved at the time that SOFA criteria were first met (≥2, within 48 hours). Second, we dichotomized patients based on whether they had received mechanical ventilation at any point during hospitalization. Finally, we used admission location as a surrogate marker for severity at the time of initial hospitalization.

To estimate the burden of baseline comorbidities, we calculated the van Walraven score (VWS),³¹ a validated modification of the Elixhauser Comorbidity Index.²⁷ This score conveys an estimated risk of in hospital death based on ICD-9/10 diagnosis codes for preexisting conditions, which ranges from <1% for the minimum score of –19 to >99% for the maximum score of 89.

Statistical Analyses

All statistical analyses were performed using Stata software version 15 (StataCorp LLC, College Station, Texas). Baseline demographics and patient characteristics were stratified by LEP. These were compared using two-sample t-tests or chi-squared tests of significance. Wilcoxon rank-sum tests were used for non-normally distributed variables. Inpatient mortality was compared across all races stratified by LEP using chi-squared tests of significance.

We fit a series of multivariable logistic regression models to examine the association between race and inpatient mortality adjusting for LEP and other patient/clinical characteristics. We first examined the unadjusted association between mortality and race; then adjusted for LEP alone; and finally adjusted for all covariates of interest, including LEP, age, sex, insurance status, year, admission level of care, VWS, total qualifying SOFA score, need for mechanical ventilation, site of infection, and time to first IV antibiotic. A subgroup analysis was also performed using the fully adjusted model restricted to patients who were mechanically ventilated. This population was selected because the patients (1) have among the highest severity of illness and (2) share a common barrier to communication, regardless of English proficiency.

Several potential interactions between LEP with other covariates were explored, including age, race, ICU admission level of care, and need for mechanical ventilation. Lastly, a mediation analysis was performed based on Baron & Kenny’s four-step model³² in order to calculate the proportion of the association between race and mortality explained by the proposed mediator (LEP).

To evaluate for the likelihood of residual confounding, we calculated an E-value, which is defined as the minimum strength of association that an unmeasured confounder would need to have with both the predictor and outcome variables, above and beyond the measured covariates, in order to fully explain away an observed predictor-outcome association.^33,34

We identified 8,974 patients hospitalized with sepsis based on the above inclusion criteria. This represented a medically complex, racially and linguistically diverse population (Table 1). The cohort was comprised of 24% Asian, 12% Black, and 11% Latino patients. Among those categorized as Other race, Native Americans/Alaskan Natives and Native Hawaiians/Pacific Islanders accounted for 4% (n = 31) and 21% (n = 159), respectively. A fifth of all patients had LEP (n = 1,716), 62% of whom were Asian (n = 1,064). Patients with LEP tended to be older, female, and to have a greater number of comorbid conditions (Table 1). The total qualifying SOFA score was also higher among patients with LEP (median 5; interquartile range [IQR]: 4-8 vs 5; IQR: 3-7; P <.001), though there was no association between LEP and mechanical ventilation (P = .22). The prevalence of LEP differed significantly across races, with 50% LEP among Asians, 32% among Latinos, 5% among White patients (P < .001). Only eight Black patients had LEP. More than 40 unique languages were represented in the cohort, with English, Cantonese, Spanish, Russian, and Mandarin accounting for ~95% (Appendix Table 1). Among Latino patients, 63% spoke English and 36% spoke Spanish.

In-hospital mortality was significantly higher among patients who had LEP (n = 268/1,716, 16%) compared to non-LEP patients (n = 678/7,258, 9%), with 80% greater unadjusted odds of mortality (OR 1.80; 95% CI: 1.54-2.09; P < .001). Notably we also found that Asian race was associated with a 1.57 unadjusted odds of mortality compared to White race (95% CI: 1.34-1.85; P < .001). Age, VWS, total qualifying SOFA score, mechanical ventilation, and admission level of care all exhibited a positive dose-response association with mortality (Appendix Table 2). In unadjusted analyses, there was no evidence of interaction between LEP and age (P = .38), LEP and race (P = .45), LEP and ICU admission level of care (P = .31), or LEP and mechanical ventilation (P = .19). Asian patients had the highest overall mortality (14% total, 17% with LEP). LEP was associated with increased unadjusted mortality among White, Asian, and Other races compared to their non-LEP counterparts (Appendix Figure 1). There was no significant difference in mortality between Latino patients with and without LEP. The sample size for Black patients with LEP (n = 8) was too small to draw conclusions about mortality.

Following multivariable logistic regression modeling for the association between race and mortality, we found that the increased odds of death among Asian patients was partially attenuated after adjusting for LEP (odds ratio [OR] 1.23, 95% CI: 1.02-1.48; P = .03; Table 2). Meanwhile, LEP was associated with a 1.66 odds of mortality (95% CI: 1.38-1.99; P < .001) after adjustment for race. In the full multivariable model adjusting for demographics and clinical characteristics, illness severity, and comorbidities, LEP was associated with a 31% increase in the odds of mortality compared to non-LEP (95% CI: 1.06-1.63; P = .02). In this model, the association between Asian race and mortality was now fully attenuated, with a point estimate near 1.0 (OR 0.98; 95% CI: 0.79-1.22; P = .87). Markers of illness severity, including total qualifying SOFA score (OR 1.23; 95% CI: 1.20-1.27; P < .001) and need for mechanical ventilation (OR 1.88; 95% CI: 1.52-2.33; P < .001), were both associated with greater odds of death. Based on a four-step mediation analysis, LEP was found to be a partial mediator to the association between Asian race and mortality (76% proportion explained). The E-value for the association between LEP and mortality was 1.95, with an E-value for the corresponding confidence interval of 1.29.

At a single US academic medical center serving a diverse population, we found that LEP was associated with sepsis mortality across all races except Black and Latino, conveying a 31% increase in the odds of death after adjusting for illness severity, comorbidities, and baseline characteristics. The higher mortality among Asian patients was largely mediated by LEP (76% proportion explained). While previous studies have variably found Black, Asian, Latino, and other non-White races/ethnicities to be at an increased risk of death from sepsis,^9-15 LEP has not been previously evaluated as a mediator of sepsis mortality. We were uniquely suited to uncover such an association due to the racial and linguistic diversity of our patient population. LEP has previously been implicated in poor health outcomes among hospitalized patients in general.^22-24 Future studies will be necessary to determine whether similar associations between LEP and mortality are observed among broader patient populations outside of sepsis.

There are a number of possible explanations for how LEP could mediate the association between race and mortality. First, LEP is known to be associated with greater difficulties in accessing medical care,²⁵ which could result in poorer baseline control of chronic comorbid conditions, fewer opportunities for preventive screening, and greater reluctance to seek medical attention when ill, theoretically leading to more severe presentations and worse outcomes. Indeed, LEP patients in our cohort had both a shorter median time to receiving their first antibiotic, as well as a higher total qualifying SOFA score, both of which may suggest more severe initial presentations. LEP is also known to contribute to, or exacerbate, the impact of low health literacy, which is itself associated with poor health.³⁵ Second, implicit biases may also have been present, as they are known to be common among healthcare providers and have been shown to negatively impact patient care.³⁶

Finally,it is possible that the association is related to the language barrier itself, which impacts providers’ ability to take an appropriate clinical history, and can lead to clinical errors or delays in care.³⁷ The fact that the association between LEP and mortality was eliminated when the analysis was restricted to mechanically ventilated patients seems to support this, since differences in language proficiency become irrelevant in this subgroup. While we are unable to comment on causality based on this observational study, we included a directed acyclic graph (DAG) in the supplemental materials, which shows one proposed model for describing these associations (Appendix Figure 2).

Assuming that the language barrier itself does, at least in part, drive the observed association, LEP represents a potentially modifiable risk factor that could be a target for quality improvement interventions. There is evidence that the use of medical interpreters among patients with LEP leads to greater satisfaction, fewer errors, and improved clinical outcomes;³⁸ however, several recent studies have documented underutilization of professional interpreter services, even when readily available.^39,40 At our institution, phone and video interpreter services are available 24/7 for approximately 150 languages. Due to limitations inherent to the EHR, we were unable to ascertain the extent to which these services were used in the present study. Heavy clinical workloads, connectivity issues, and missing or faulty equipment represent theoretical barriers to utilization of these services.

There are some limitations to our study. First, by utilizing a large database of electronic data, the quality of our analyses was reliant on the accuracy of the EHR. Demographic data such as language may have been subject to misclassification due to self-reporting. We attempted to minimize this by also including the need for interpreter services within the definition of LEP, which was validated by manual chart review. Second, generalizability is limited in this single-center study conducted at an institution with unique demographics, wherein nearly two-thirds of the LEP patients were Asian, and the Chinese-speaking population outnumbered those who speak Spanish.

Finally, the most important limitation to our study is the potential for residual confounding. While we attempted to mitigate this by adjusting for as many clinically relevant covariates as possible, there may still be unmeasured confounders to the association between LEP and mortality, such as access to outpatient care, functional status, interpreter use, and other markers of illness severity like the number and type of supportive therapies received. Based on our E-value calculations, with an observed OR of 1.31 for the association between LEP and mortality, an unmeasured confounder with an OR of 1.95 would fully explain away this association, while an OR of 1.29 would shift the confidence interval to include the null. These values suggest at least some risk of residual confounding. The fact that our fully adjusted model included multiple covariates, including several markers of illness severity, does somewhat lessen the likelihood of a confounder achieving these values, since they represent the minimum strength of an unmeasured confounder above and beyond the measured covariates. Regardless, the finding that patients with LEP are more likely to die from sepsis remains an important one, recognizing the need for further studies including multicenter investigations.

In this study, we showed that LEP was associated with sepsis mortality across nearly all races in our cohort. While Asian race was associated with a higher unadjusted odds of death compared to White race, this was attenuated after adjusting for LEP. This may suggest that some of the racial disparities in sepsis identified in prior studies were in fact mediated by language proficiency. Further studies will be required to explore the causal nature of this novel association. If modifiable factors are identified, this could represent a potential target for future quality improvement initiatives aimed at improving sepsis outcomes.

Disclaimer

The contents are solely the responsibility of the authors and do not necessarily represent the official views of the University of California, San Francisco or the National Institutes of Health.

Sepsis is defined as a life-threatening organ dysfunction that occurs in response to systemic infection.^1,2 It is frequently fatal, common in hospital medicine, and a leading contributor to critical illness, morbidity, and healthcare expenditures.^2-5 While sepsis care and outcomes have improved in the past decade,^6,7 inpatient mortality remains high.⁸

A number of studies have sought to determine whether race plays a role in sepsis mortality. While Black patients with sepsis have frequently been identified as having the highest rates of death,^9-14 similar observations have been made for most non-White races/ethnicities.^13-15 Studies have also demonstrated higher rates of hospital-acquired infections among Asian and Latino patients.¹⁶

There are several possible explanations for why racial minorities experience disparate outcomes in sepsis, including access to care, comorbidities, implicit biases, and biological or environmental factors,^17-20 as well as characteristics of hospitals most likely to care for racial minorities.^13,15,21 One explanation that has not been explored is that racial disparities in sepsis are mediated by language. Limited English proficiency (LEP) has previously been associated with increased rates of adverse hospital events,²² longer length of stay,²³ and greater likelihood of readmission.²⁴ LEP has also been shown to represent a significant barrier to accessing healthcare and preventive screening.²⁵ The role of LEP in sepsis mortality, however, has yet to be examined.

The diverse patient population at the University of California, San Francisco (UCSF) provides a unique opportunity to build upon existing literature by further exploring racial differences in sepsis, specifically by investigating the role of LEP. The objective of this study was to determine the association between LEP and inpatient mortality among adults hospitalized with sepsis.

Setting

The study was conducted at the University of California, San Francisco, California (UCSF), an 800-bed tertiary care, academic medical center. It was approved by the UCSF Institutional Review Board with waiver of informed consent. UCSF cares for a population of patients who are racially and linguistically diverse, with high proportions of patients of East Asian descent and with LEP. According to recent United States census estimates, more than half of San Francisco County residents identify as non-White (35% Asians, 15% Hispanic/Latino, 6% Black), and 44% report speaking a language other than English at home.²⁶

Study Population and Data Collection

The UCSF Medical Center uses the electronic health record (EHR) Epic (Epic 2017, Epic Systems Corporation, Verona, Wisconsin). We obtained computerized EHR data from Clarity, the relational database that stores Epic’s inpatient data in thousands of tables. We identified all patients ≥18 years of age presenting to the emergency department (ED) between June 1, 2012 and December 31, 2016 with suspected serious infection, defined as having blood cultures ordered within 72 hours of ED presentation (N = 25,441). Patients who did not receive at least two doses of intravenous (IV) antibiotics within 48 hours were excluded, as they were unlikely to have serious infections.

We defined sepsis based on Sepsis-3 consensus guidelines² as a change in sequential [sepsis-related] organ failure assessment (SOFA) score ≥2 within the first 48 hours of ED presentation. The SOFA score is comprised of six variables representing different organ systems, each rated 0-4 based on the degree of dysfunction.² Patient vital signs, laboratory data, vasopressor medication doses, and ventilator settings were used to determine the exact timestamp at which each patient attained a change in SOFA score ≥2. Missing values were considered to be normal. To adjust for baseline organ dysfunction, SOFA elements associated with elevated bilirubin and/or creatinine were excluded for patients with chronic liver/kidney disease based on Elixhauser comorbidities.²⁷ We chose to focus on the first 48 hours in an attempt to capture patients with the most severe illnesses and the highest probability of true sepsis.

All primary and secondary International Classification of Diseases (ICD)-9/10 diagnosis codes were extracted from Clarity coding tables at the time of hospital discharge. Diagnosis codes signifying bacterial infection were grouped into the following categories based on type/location: pneumonia; bacteremia; urinary tract infection; and skin and soft tissue infection. All remaining diagnostic codes indicating bacterial infections at other sites were categorized as “Other”. If no codes indicating infection were present, patients were categorized as “None coded”. Patients with discharge diagnosis codes of “sepsis” were also identified. Dates and times of antibiotic administrations were obtained from the medications table. Time to first antibiotic was defined as the time in minutes from ED presentation to initiation of the first IV antibacterial medication. This variable was transformed using a natural log transformation based on best fit for normal distribution.

We limited our analyses to 8,974 patients who were diagnosed with sepsis as defined above and had either (1) ≥4 qualifying antibiotic days (QADs) or (2) an ICD-9/10 discharge diagnosis code of “sepsis” (Figure). QADs were defined based on the recent publication by Rhee et al. as having received four or more consecutive days of antibiotics, with the first dose given IV within 48 hours of presentation.²⁸ Patients who died or were discharged to hospice prior to the 4th QAD were also included. These additional parameters were added to increase specificity of the study sample for patients with true sepsis. Patients admitted to all levels of care (acute care, transitional care unit [TCU], intensive care unit [ICU]) and under all hospital services were included. There were no missing data for mortality, race, or language. We chose to focus on patients with sepsis in this initial study as this is a common diagnosis in hospital medicine that is enriched for high mortality.

Primary Outcome

The primary outcome of the study was inpatient mortality, which was obtained from the hospital encounters table in Clarity.

Primary Predictors

The primary predictor of interest was LEP. The encounter numbers from the dataset were used to link to self-reported demographic data, including “preferred language” and need for interpreter services. A manual chart review of 60 patients speaking the top six languages was conducted to verify the accuracy of the data on language and interpreter use (KNK). Defining the gold standard for LEP as having any chart note indicating non-English language and/or that an interpreter was used, the “interpreter needed” variable in Epic was found to have a positive predictive value for LEP of 100%. Therefore, patients in the study cohort were defined as having LEP if they met both of the following criteria: (1) a self-reported “preferred language” other than English and (2) having the “interpreter needed” variable indicating “yes”.

Covariate Data Collection

Additional data were obtained from the demographics tables, including age, race, sex, and insurance status. Race and ethnicity were combined into a single five-category variable including White, Asian, Black, Latino, and Other. This approach has been suggested as the best way to operationalize these variables²⁹ and has been utilized by similar studies in the literature.^9,14,15 We considered the Asian race to include all people of East Asian, Southeast Asian, or South Asian descent, which is consistent with the United States Census Bureau definition.³⁰ Patients identifying as Native Hawaiians/Pacific Islanders, Native Americans/Alaskan Natives, as well as those with unspecified race or ethnicity, were categorized as Other. Insurance status was categorized as Commercial, Medicare, Medicaid, or Other.

We estimated illness severity in several ways. First, the total qualifying SOFA score was calculated for each patient, which was defined as the total score achieved at the time that SOFA criteria were first met (≥2, within 48 hours). Second, we dichotomized patients based on whether they had received mechanical ventilation at any point during hospitalization. Finally, we used admission location as a surrogate marker for severity at the time of initial hospitalization.

To estimate the burden of baseline comorbidities, we calculated the van Walraven score (VWS),³¹ a validated modification of the Elixhauser Comorbidity Index.²⁷ This score conveys an estimated risk of in hospital death based on ICD-9/10 diagnosis codes for preexisting conditions, which ranges from <1% for the minimum score of –19 to >99% for the maximum score of 89.

Statistical Analyses

All statistical analyses were performed using Stata software version 15 (StataCorp LLC, College Station, Texas). Baseline demographics and patient characteristics were stratified by LEP. These were compared using two-sample t-tests or chi-squared tests of significance. Wilcoxon rank-sum tests were used for non-normally distributed variables. Inpatient mortality was compared across all races stratified by LEP using chi-squared tests of significance.

We fit a series of multivariable logistic regression models to examine the association between race and inpatient mortality adjusting for LEP and other patient/clinical characteristics. We first examined the unadjusted association between mortality and race; then adjusted for LEP alone; and finally adjusted for all covariates of interest, including LEP, age, sex, insurance status, year, admission level of care, VWS, total qualifying SOFA score, need for mechanical ventilation, site of infection, and time to first IV antibiotic. A subgroup analysis was also performed using the fully adjusted model restricted to patients who were mechanically ventilated. This population was selected because the patients (1) have among the highest severity of illness and (2) share a common barrier to communication, regardless of English proficiency.

Several potential interactions between LEP with other covariates were explored, including age, race, ICU admission level of care, and need for mechanical ventilation. Lastly, a mediation analysis was performed based on Baron & Kenny’s four-step model³² in order to calculate the proportion of the association between race and mortality explained by the proposed mediator (LEP).

To evaluate for the likelihood of residual confounding, we calculated an E-value, which is defined as the minimum strength of association that an unmeasured confounder would need to have with both the predictor and outcome variables, above and beyond the measured covariates, in order to fully explain away an observed predictor-outcome association.^33,34

We identified 8,974 patients hospitalized with sepsis based on the above inclusion criteria. This represented a medically complex, racially and linguistically diverse population (Table 1). The cohort was comprised of 24% Asian, 12% Black, and 11% Latino patients. Among those categorized as Other race, Native Americans/Alaskan Natives and Native Hawaiians/Pacific Islanders accounted for 4% (n = 31) and 21% (n = 159), respectively. A fifth of all patients had LEP (n = 1,716), 62% of whom were Asian (n = 1,064). Patients with LEP tended to be older, female, and to have a greater number of comorbid conditions (Table 1). The total qualifying SOFA score was also higher among patients with LEP (median 5; interquartile range [IQR]: 4-8 vs 5; IQR: 3-7; P <.001), though there was no association between LEP and mechanical ventilation (P = .22). The prevalence of LEP differed significantly across races, with 50% LEP among Asians, 32% among Latinos, 5% among White patients (P < .001). Only eight Black patients had LEP. More than 40 unique languages were represented in the cohort, with English, Cantonese, Spanish, Russian, and Mandarin accounting for ~95% (Appendix Table 1). Among Latino patients, 63% spoke English and 36% spoke Spanish.

In-hospital mortality was significantly higher among patients who had LEP (n = 268/1,716, 16%) compared to non-LEP patients (n = 678/7,258, 9%), with 80% greater unadjusted odds of mortality (OR 1.80; 95% CI: 1.54-2.09; P < .001). Notably we also found that Asian race was associated with a 1.57 unadjusted odds of mortality compared to White race (95% CI: 1.34-1.85; P < .001). Age, VWS, total qualifying SOFA score, mechanical ventilation, and admission level of care all exhibited a positive dose-response association with mortality (Appendix Table 2). In unadjusted analyses, there was no evidence of interaction between LEP and age (P = .38), LEP and race (P = .45), LEP and ICU admission level of care (P = .31), or LEP and mechanical ventilation (P = .19). Asian patients had the highest overall mortality (14% total, 17% with LEP). LEP was associated with increased unadjusted mortality among White, Asian, and Other races compared to their non-LEP counterparts (Appendix Figure 1). There was no significant difference in mortality between Latino patients with and without LEP. The sample size for Black patients with LEP (n = 8) was too small to draw conclusions about mortality.

Following multivariable logistic regression modeling for the association between race and mortality, we found that the increased odds of death among Asian patients was partially attenuated after adjusting for LEP (odds ratio [OR] 1.23, 95% CI: 1.02-1.48; P = .03; Table 2). Meanwhile, LEP was associated with a 1.66 odds of mortality (95% CI: 1.38-1.99; P < .001) after adjustment for race. In the full multivariable model adjusting for demographics and clinical characteristics, illness severity, and comorbidities, LEP was associated with a 31% increase in the odds of mortality compared to non-LEP (95% CI: 1.06-1.63; P = .02). In this model, the association between Asian race and mortality was now fully attenuated, with a point estimate near 1.0 (OR 0.98; 95% CI: 0.79-1.22; P = .87). Markers of illness severity, including total qualifying SOFA score (OR 1.23; 95% CI: 1.20-1.27; P < .001) and need for mechanical ventilation (OR 1.88; 95% CI: 1.52-2.33; P < .001), were both associated with greater odds of death. Based on a four-step mediation analysis, LEP was found to be a partial mediator to the association between Asian race and mortality (76% proportion explained). The E-value for the association between LEP and mortality was 1.95, with an E-value for the corresponding confidence interval of 1.29.

At a single US academic medical center serving a diverse population, we found that LEP was associated with sepsis mortality across all races except Black and Latino, conveying a 31% increase in the odds of death after adjusting for illness severity, comorbidities, and baseline characteristics. The higher mortality among Asian patients was largely mediated by LEP (76% proportion explained). While previous studies have variably found Black, Asian, Latino, and other non-White races/ethnicities to be at an increased risk of death from sepsis,^9-15 LEP has not been previously evaluated as a mediator of sepsis mortality. We were uniquely suited to uncover such an association due to the racial and linguistic diversity of our patient population. LEP has previously been implicated in poor health outcomes among hospitalized patients in general.^22-24 Future studies will be necessary to determine whether similar associations between LEP and mortality are observed among broader patient populations outside of sepsis.

There are a number of possible explanations for how LEP could mediate the association between race and mortality. First, LEP is known to be associated with greater difficulties in accessing medical care,²⁵ which could result in poorer baseline control of chronic comorbid conditions, fewer opportunities for preventive screening, and greater reluctance to seek medical attention when ill, theoretically leading to more severe presentations and worse outcomes. Indeed, LEP patients in our cohort had both a shorter median time to receiving their first antibiotic, as well as a higher total qualifying SOFA score, both of which may suggest more severe initial presentations. LEP is also known to contribute to, or exacerbate, the impact of low health literacy, which is itself associated with poor health.³⁵ Second, implicit biases may also have been present, as they are known to be common among healthcare providers and have been shown to negatively impact patient care.³⁶

Finally,it is possible that the association is related to the language barrier itself, which impacts providers’ ability to take an appropriate clinical history, and can lead to clinical errors or delays in care.³⁷ The fact that the association between LEP and mortality was eliminated when the analysis was restricted to mechanically ventilated patients seems to support this, since differences in language proficiency become irrelevant in this subgroup. While we are unable to comment on causality based on this observational study, we included a directed acyclic graph (DAG) in the supplemental materials, which shows one proposed model for describing these associations (Appendix Figure 2).

Assuming that the language barrier itself does, at least in part, drive the observed association, LEP represents a potentially modifiable risk factor that could be a target for quality improvement interventions. There is evidence that the use of medical interpreters among patients with LEP leads to greater satisfaction, fewer errors, and improved clinical outcomes;³⁸ however, several recent studies have documented underutilization of professional interpreter services, even when readily available.^39,40 At our institution, phone and video interpreter services are available 24/7 for approximately 150 languages. Due to limitations inherent to the EHR, we were unable to ascertain the extent to which these services were used in the present study. Heavy clinical workloads, connectivity issues, and missing or faulty equipment represent theoretical barriers to utilization of these services.

There are some limitations to our study. First, by utilizing a large database of electronic data, the quality of our analyses was reliant on the accuracy of the EHR. Demographic data such as language may have been subject to misclassification due to self-reporting. We attempted to minimize this by also including the need for interpreter services within the definition of LEP, which was validated by manual chart review. Second, generalizability is limited in this single-center study conducted at an institution with unique demographics, wherein nearly two-thirds of the LEP patients were Asian, and the Chinese-speaking population outnumbered those who speak Spanish.

Finally, the most important limitation to our study is the potential for residual confounding. While we attempted to mitigate this by adjusting for as many clinically relevant covariates as possible, there may still be unmeasured confounders to the association between LEP and mortality, such as access to outpatient care, functional status, interpreter use, and other markers of illness severity like the number and type of supportive therapies received. Based on our E-value calculations, with an observed OR of 1.31 for the association between LEP and mortality, an unmeasured confounder with an OR of 1.95 would fully explain away this association, while an OR of 1.29 would shift the confidence interval to include the null. These values suggest at least some risk of residual confounding. The fact that our fully adjusted model included multiple covariates, including several markers of illness severity, does somewhat lessen the likelihood of a confounder achieving these values, since they represent the minimum strength of an unmeasured confounder above and beyond the measured covariates. Regardless, the finding that patients with LEP are more likely to die from sepsis remains an important one, recognizing the need for further studies including multicenter investigations.

In this study, we showed that LEP was associated with sepsis mortality across nearly all races in our cohort. While Asian race was associated with a higher unadjusted odds of death compared to White race, this was attenuated after adjusting for LEP. This may suggest that some of the racial disparities in sepsis identified in prior studies were in fact mediated by language proficiency. Further studies will be required to explore the causal nature of this novel association. If modifiable factors are identified, this could represent a potential target for future quality improvement initiatives aimed at improving sepsis outcomes.

Disclaimer

The contents are solely the responsibility of the authors and do not necessarily represent the official views of the University of California, San Francisco or the National Institutes of Health.

Sepsis is defined as a life-threatening organ dysfunction that occurs in response to systemic infection.^1,2 It is frequently fatal, common in hospital medicine, and a leading contributor to critical illness, morbidity, and healthcare expenditures.^2-5 While sepsis care and outcomes have improved in the past decade,^6,7 inpatient mortality remains high.⁸

A number of studies have sought to determine whether race plays a role in sepsis mortality. While Black patients with sepsis have frequently been identified as having the highest rates of death,^9-14 similar observations have been made for most non-White races/ethnicities.^13-15 Studies have also demonstrated higher rates of hospital-acquired infections among Asian and Latino patients.¹⁶

There are several possible explanations for why racial minorities experience disparate outcomes in sepsis, including access to care, comorbidities, implicit biases, and biological or environmental factors,^17-20 as well as characteristics of hospitals most likely to care for racial minorities.^13,15,21 One explanation that has not been explored is that racial disparities in sepsis are mediated by language. Limited English proficiency (LEP) has previously been associated with increased rates of adverse hospital events,²² longer length of stay,²³ and greater likelihood of readmission.²⁴ LEP has also been shown to represent a significant barrier to accessing healthcare and preventive screening.²⁵ The role of LEP in sepsis mortality, however, has yet to be examined.

The diverse patient population at the University of California, San Francisco (UCSF) provides a unique opportunity to build upon existing literature by further exploring racial differences in sepsis, specifically by investigating the role of LEP. The objective of this study was to determine the association between LEP and inpatient mortality among adults hospitalized with sepsis.

Setting

The study was conducted at the University of California, San Francisco, California (UCSF), an 800-bed tertiary care, academic medical center. It was approved by the UCSF Institutional Review Board with waiver of informed consent. UCSF cares for a population of patients who are racially and linguistically diverse, with high proportions of patients of East Asian descent and with LEP. According to recent United States census estimates, more than half of San Francisco County residents identify as non-White (35% Asians, 15% Hispanic/Latino, 6% Black), and 44% report speaking a language other than English at home.²⁶

Study Population and Data Collection

The UCSF Medical Center uses the electronic health record (EHR) Epic (Epic 2017, Epic Systems Corporation, Verona, Wisconsin). We obtained computerized EHR data from Clarity, the relational database that stores Epic’s inpatient data in thousands of tables. We identified all patients ≥18 years of age presenting to the emergency department (ED) between June 1, 2012 and December 31, 2016 with suspected serious infection, defined as having blood cultures ordered within 72 hours of ED presentation (N = 25,441). Patients who did not receive at least two doses of intravenous (IV) antibiotics within 48 hours were excluded, as they were unlikely to have serious infections.

We defined sepsis based on Sepsis-3 consensus guidelines² as a change in sequential [sepsis-related] organ failure assessment (SOFA) score ≥2 within the first 48 hours of ED presentation. The SOFA score is comprised of six variables representing different organ systems, each rated 0-4 based on the degree of dysfunction.² Patient vital signs, laboratory data, vasopressor medication doses, and ventilator settings were used to determine the exact timestamp at which each patient attained a change in SOFA score ≥2. Missing values were considered to be normal. To adjust for baseline organ dysfunction, SOFA elements associated with elevated bilirubin and/or creatinine were excluded for patients with chronic liver/kidney disease based on Elixhauser comorbidities.²⁷ We chose to focus on the first 48 hours in an attempt to capture patients with the most severe illnesses and the highest probability of true sepsis.

All primary and secondary International Classification of Diseases (ICD)-9/10 diagnosis codes were extracted from Clarity coding tables at the time of hospital discharge. Diagnosis codes signifying bacterial infection were grouped into the following categories based on type/location: pneumonia; bacteremia; urinary tract infection; and skin and soft tissue infection. All remaining diagnostic codes indicating bacterial infections at other sites were categorized as “Other”. If no codes indicating infection were present, patients were categorized as “None coded”. Patients with discharge diagnosis codes of “sepsis” were also identified. Dates and times of antibiotic administrations were obtained from the medications table. Time to first antibiotic was defined as the time in minutes from ED presentation to initiation of the first IV antibacterial medication. This variable was transformed using a natural log transformation based on best fit for normal distribution.

We limited our analyses to 8,974 patients who were diagnosed with sepsis as defined above and had either (1) ≥4 qualifying antibiotic days (QADs) or (2) an ICD-9/10 discharge diagnosis code of “sepsis” (Figure). QADs were defined based on the recent publication by Rhee et al. as having received four or more consecutive days of antibiotics, with the first dose given IV within 48 hours of presentation.²⁸ Patients who died or were discharged to hospice prior to the 4th QAD were also included. These additional parameters were added to increase specificity of the study sample for patients with true sepsis. Patients admitted to all levels of care (acute care, transitional care unit [TCU], intensive care unit [ICU]) and under all hospital services were included. There were no missing data for mortality, race, or language. We chose to focus on patients with sepsis in this initial study as this is a common diagnosis in hospital medicine that is enriched for high mortality.

Primary Outcome

The primary outcome of the study was inpatient mortality, which was obtained from the hospital encounters table in Clarity.

Primary Predictors

The primary predictor of interest was LEP. The encounter numbers from the dataset were used to link to self-reported demographic data, including “preferred language” and need for interpreter services. A manual chart review of 60 patients speaking the top six languages was conducted to verify the accuracy of the data on language and interpreter use (KNK). Defining the gold standard for LEP as having any chart note indicating non-English language and/or that an interpreter was used, the “interpreter needed” variable in Epic was found to have a positive predictive value for LEP of 100%. Therefore, patients in the study cohort were defined as having LEP if they met both of the following criteria: (1) a self-reported “preferred language” other than English and (2) having the “interpreter needed” variable indicating “yes”.

Covariate Data Collection

Additional data were obtained from the demographics tables, including age, race, sex, and insurance status. Race and ethnicity were combined into a single five-category variable including White, Asian, Black, Latino, and Other. This approach has been suggested as the best way to operationalize these variables²⁹ and has been utilized by similar studies in the literature.^9,14,15 We considered the Asian race to include all people of East Asian, Southeast Asian, or South Asian descent, which is consistent with the United States Census Bureau definition.³⁰ Patients identifying as Native Hawaiians/Pacific Islanders, Native Americans/Alaskan Natives, as well as those with unspecified race or ethnicity, were categorized as Other. Insurance status was categorized as Commercial, Medicare, Medicaid, or Other.

We estimated illness severity in several ways. First, the total qualifying SOFA score was calculated for each patient, which was defined as the total score achieved at the time that SOFA criteria were first met (≥2, within 48 hours). Second, we dichotomized patients based on whether they had received mechanical ventilation at any point during hospitalization. Finally, we used admission location as a surrogate marker for severity at the time of initial hospitalization.

To estimate the burden of baseline comorbidities, we calculated the van Walraven score (VWS),³¹ a validated modification of the Elixhauser Comorbidity Index.²⁷ This score conveys an estimated risk of in hospital death based on ICD-9/10 diagnosis codes for preexisting conditions, which ranges from <1% for the minimum score of –19 to >99% for the maximum score of 89.

Statistical Analyses

All statistical analyses were performed using Stata software version 15 (StataCorp LLC, College Station, Texas). Baseline demographics and patient characteristics were stratified by LEP. These were compared using two-sample t-tests or chi-squared tests of significance. Wilcoxon rank-sum tests were used for non-normally distributed variables. Inpatient mortality was compared across all races stratified by LEP using chi-squared tests of significance.

We fit a series of multivariable logistic regression models to examine the association between race and inpatient mortality adjusting for LEP and other patient/clinical characteristics. We first examined the unadjusted association between mortality and race; then adjusted for LEP alone; and finally adjusted for all covariates of interest, including LEP, age, sex, insurance status, year, admission level of care, VWS, total qualifying SOFA score, need for mechanical ventilation, site of infection, and time to first IV antibiotic. A subgroup analysis was also performed using the fully adjusted model restricted to patients who were mechanically ventilated. This population was selected because the patients (1) have among the highest severity of illness and (2) share a common barrier to communication, regardless of English proficiency.

Several potential interactions between LEP with other covariates were explored, including age, race, ICU admission level of care, and need for mechanical ventilation. Lastly, a mediation analysis was performed based on Baron & Kenny’s four-step model³² in order to calculate the proportion of the association between race and mortality explained by the proposed mediator (LEP).

To evaluate for the likelihood of residual confounding, we calculated an E-value, which is defined as the minimum strength of association that an unmeasured confounder would need to have with both the predictor and outcome variables, above and beyond the measured covariates, in order to fully explain away an observed predictor-outcome association.^33,34

We identified 8,974 patients hospitalized with sepsis based on the above inclusion criteria. This represented a medically complex, racially and linguistically diverse population (Table 1). The cohort was comprised of 24% Asian, 12% Black, and 11% Latino patients. Among those categorized as Other race, Native Americans/Alaskan Natives and Native Hawaiians/Pacific Islanders accounted for 4% (n = 31) and 21% (n = 159), respectively. A fifth of all patients had LEP (n = 1,716), 62% of whom were Asian (n = 1,064). Patients with LEP tended to be older, female, and to have a greater number of comorbid conditions (Table 1). The total qualifying SOFA score was also higher among patients with LEP (median 5; interquartile range [IQR]: 4-8 vs 5; IQR: 3-7; P <.001), though there was no association between LEP and mechanical ventilation (P = .22). The prevalence of LEP differed significantly across races, with 50% LEP among Asians, 32% among Latinos, 5% among White patients (P < .001). Only eight Black patients had LEP. More than 40 unique languages were represented in the cohort, with English, Cantonese, Spanish, Russian, and Mandarin accounting for ~95% (Appendix Table 1). Among Latino patients, 63% spoke English and 36% spoke Spanish.

In-hospital mortality was significantly higher among patients who had LEP (n = 268/1,716, 16%) compared to non-LEP patients (n = 678/7,258, 9%), with 80% greater unadjusted odds of mortality (OR 1.80; 95% CI: 1.54-2.09; P < .001). Notably we also found that Asian race was associated with a 1.57 unadjusted odds of mortality compared to White race (95% CI: 1.34-1.85; P < .001). Age, VWS, total qualifying SOFA score, mechanical ventilation, and admission level of care all exhibited a positive dose-response association with mortality (Appendix Table 2). In unadjusted analyses, there was no evidence of interaction between LEP and age (P = .38), LEP and race (P = .45), LEP and ICU admission level of care (P = .31), or LEP and mechanical ventilation (P = .19). Asian patients had the highest overall mortality (14% total, 17% with LEP). LEP was associated with increased unadjusted mortality among White, Asian, and Other races compared to their non-LEP counterparts (Appendix Figure 1). There was no significant difference in mortality between Latino patients with and without LEP. The sample size for Black patients with LEP (n = 8) was too small to draw conclusions about mortality.

Following multivariable logistic regression modeling for the association between race and mortality, we found that the increased odds of death among Asian patients was partially attenuated after adjusting for LEP (odds ratio [OR] 1.23, 95% CI: 1.02-1.48; P = .03; Table 2). Meanwhile, LEP was associated with a 1.66 odds of mortality (95% CI: 1.38-1.99; P < .001) after adjustment for race. In the full multivariable model adjusting for demographics and clinical characteristics, illness severity, and comorbidities, LEP was associated with a 31% increase in the odds of mortality compared to non-LEP (95% CI: 1.06-1.63; P = .02). In this model, the association between Asian race and mortality was now fully attenuated, with a point estimate near 1.0 (OR 0.98; 95% CI: 0.79-1.22; P = .87). Markers of illness severity, including total qualifying SOFA score (OR 1.23; 95% CI: 1.20-1.27; P < .001) and need for mechanical ventilation (OR 1.88; 95% CI: 1.52-2.33; P < .001), were both associated with greater odds of death. Based on a four-step mediation analysis, LEP was found to be a partial mediator to the association between Asian race and mortality (76% proportion explained). The E-value for the association between LEP and mortality was 1.95, with an E-value for the corresponding confidence interval of 1.29.

At a single US academic medical center serving a diverse population, we found that LEP was associated with sepsis mortality across all races except Black and Latino, conveying a 31% increase in the odds of death after adjusting for illness severity, comorbidities, and baseline characteristics. The higher mortality among Asian patients was largely mediated by LEP (76% proportion explained). While previous studies have variably found Black, Asian, Latino, and other non-White races/ethnicities to be at an increased risk of death from sepsis,^9-15 LEP has not been previously evaluated as a mediator of sepsis mortality. We were uniquely suited to uncover such an association due to the racial and linguistic diversity of our patient population. LEP has previously been implicated in poor health outcomes among hospitalized patients in general.^22-24 Future studies will be necessary to determine whether similar associations between LEP and mortality are observed among broader patient populations outside of sepsis.

There are a number of possible explanations for how LEP could mediate the association between race and mortality. First, LEP is known to be associated with greater difficulties in accessing medical care,²⁵ which could result in poorer baseline control of chronic comorbid conditions, fewer opportunities for preventive screening, and greater reluctance to seek medical attention when ill, theoretically leading to more severe presentations and worse outcomes. Indeed, LEP patients in our cohort had both a shorter median time to receiving their first antibiotic, as well as a higher total qualifying SOFA score, both of which may suggest more severe initial presentations. LEP is also known to contribute to, or exacerbate, the impact of low health literacy, which is itself associated with poor health.³⁵ Second, implicit biases may also have been present, as they are known to be common among healthcare providers and have been shown to negatively impact patient care.³⁶

Finally,it is possible that the association is related to the language barrier itself, which impacts providers’ ability to take an appropriate clinical history, and can lead to clinical errors or delays in care.³⁷ The fact that the association between LEP and mortality was eliminated when the analysis was restricted to mechanically ventilated patients seems to support this, since differences in language proficiency become irrelevant in this subgroup. While we are unable to comment on causality based on this observational study, we included a directed acyclic graph (DAG) in the supplemental materials, which shows one proposed model for describing these associations (Appendix Figure 2).

Assuming that the language barrier itself does, at least in part, drive the observed association, LEP represents a potentially modifiable risk factor that could be a target for quality improvement interventions. There is evidence that the use of medical interpreters among patients with LEP leads to greater satisfaction, fewer errors, and improved clinical outcomes;³⁸ however, several recent studies have documented underutilization of professional interpreter services, even when readily available.^39,40 At our institution, phone and video interpreter services are available 24/7 for approximately 150 languages. Due to limitations inherent to the EHR, we were unable to ascertain the extent to which these services were used in the present study. Heavy clinical workloads, connectivity issues, and missing or faulty equipment represent theoretical barriers to utilization of these services.

There are some limitations to our study. First, by utilizing a large database of electronic data, the quality of our analyses was reliant on the accuracy of the EHR. Demographic data such as language may have been subject to misclassification due to self-reporting. We attempted to minimize this by also including the need for interpreter services within the definition of LEP, which was validated by manual chart review. Second, generalizability is limited in this single-center study conducted at an institution with unique demographics, wherein nearly two-thirds of the LEP patients were Asian, and the Chinese-speaking population outnumbered those who speak Spanish.

Finally, the most important limitation to our study is the potential for residual confounding. While we attempted to mitigate this by adjusting for as many clinically relevant covariates as possible, there may still be unmeasured confounders to the association between LEP and mortality, such as access to outpatient care, functional status, interpreter use, and other markers of illness severity like the number and type of supportive therapies received. Based on our E-value calculations, with an observed OR of 1.31 for the association between LEP and mortality, an unmeasured confounder with an OR of 1.95 would fully explain away this association, while an OR of 1.29 would shift the confidence interval to include the null. These values suggest at least some risk of residual confounding. The fact that our fully adjusted model included multiple covariates, including several markers of illness severity, does somewhat lessen the likelihood of a confounder achieving these values, since they represent the minimum strength of an unmeasured confounder above and beyond the measured covariates. Regardless, the finding that patients with LEP are more likely to die from sepsis remains an important one, recognizing the need for further studies including multicenter investigations.

In this study, we showed that LEP was associated with sepsis mortality across nearly all races in our cohort. While Asian race was associated with a higher unadjusted odds of death compared to White race, this was attenuated after adjusting for LEP. This may suggest that some of the racial disparities in sepsis identified in prior studies were in fact mediated by language proficiency. Further studies will be required to explore the causal nature of this novel association. If modifiable factors are identified, this could represent a potential target for future quality improvement initiatives aimed at improving sepsis outcomes.

Disclaimer

The contents are solely the responsibility of the authors and do not necessarily represent the official views of the University of California, San Francisco or the National Institutes of Health.

Emergency department (ED) crowding and boarding of patients awaiting admission to the hospital (ED boarding) are growing problems with important clinical care and public safety implications.^1-4 Increased ED boarding times have been associated with lower patient satisfaction, inadequate care of critically ill patients, adverse events, and increased mortality.^3,5-7 Furthermore, ED boarding can diminish the ED’s ability to evaluate new patients.^5,8,9 ED boarding is more severe in hospitals with high inpatient occupancy with resultant disproportionate burden on large urban institutions.^2,4,5,10

Earlier studies suggest, but have not consistently shown, an association between longer ED length of stay (LOS) and longer overall hospital LOS.⁵ This association implies that the additional time spent in the ED waiting for a bed does not meaningfully contribute to advancing the required inpatient care. Thus, this waiting time is “dead time” that is added to the overall hospital duration.

The complexity and the volume of medical patients boarding in the ED can challenge the resources of an already overtaxed ED staff. Potential solutions to mitigate ED boarding of medicine patients generally focus on reducing barriers to timely movement of patients from the ED to an inpatient unit.^1,3,11-13 Ultimately, these barriers are a function of inadequate hospital capacity (eg, hospital beds, staffing) and are difficult to overcome. Two primary strategies have been used to reduce these barriers. One strategy focuses on shifting inpatient discharge times earlier to better match inpatient bed supply with ED demand.^14-19 Another common strategy is utilizing inpatient attendings to triage and better match bed needs to bed availability.^20-22

A separate area of interest, and the focus of this study, is the deployment of inpatient teams to hasten delivery of inpatient care to patients waiting in the ED.^8,23 One institution implemented an “ED hospitalist” model.²³ Another created a hospital medicine team to provide inpatient medical care to ED boarder patients and to lend clinical input to bed management.⁸

At our large, urban academic medical center, the Department of Medicine in collaboration with the Department of Emergency Medicine created a full-time hospital medicine team dedicated to providing care in the ED for patients awaiting admission to a general medicine unit. We present our multiyear experience with this ED-based hospital medicine team. We hypothesized that this new team would expedite inpatient care delivery to medical boarder patients, thereby reducing the overall hospital LOS.

Study Setting and Design

This retrospective cross-sectional study, approved by the Institutional Review Board, was conducted at a 1,011-bed academic medical center in the northeast United States. The study period was July 1, 2016 through June 30, 2018, which was divided into Academic Year 16 (AY) (July 1, 2016 to June 30, 2017) and AY17 (July 1, 2017 to June 30, 2018).

The Hospital Medicine Unit (HMU) was a 60 full-time equivalent hospital medicine group consisting of 80 physicians and 25 advanced practice providers (APPs). During the study, the general medical services cared for an average of 260 patients per day on inpatient units with a wide variety of diagnoses and comorbidities. The ED had 48 monitored bed spaces for adult patients, as well as two dedicated ED observation units with 32 beds. The observation units are separate units within the hospital, staffed by ED clinicians, and were not included in this study. In 2016, the ED had a total of 110,741 patient visits and 13,908 patients were admitted to a medical service.

In 2010, the Department of Public Health for the state in which the medical center resides defined an ED boarder (EDB) patient as “a patient who remains in the ED two hours after the decision to admit.”²⁴ According to this definition, any patient waiting for an inpatient bed for more than two hours after a bed request was considered as an EDB. Operationally, further distinctions were made between patients who were “eligible” for care by an internal medicine team in the ED versus those who were actually “covered”. Before the intervention outlined in the current study, some care was provided by resident and hospitalist teams to eligible EDB patients from 2010 to 2015, although this was limited in scope. From July 1, 2015 to June 30, 2016, there was no coverage of medicine EDB patients.

Intervention

ED Boarder Service Staffing

On July 1, 2016, the HMU deployed a dedicated full-time team of clinicians to care for boarding patients, which was known as the EDB service. The service was created with the goal of seeing a maximum of 25 patients over 24 hours.

Inpatient medicine attending physicians (hospitalists) and APPs worked on the EDB service. During the day (7 am-7 pm), coverage was provided by three clinicians (generally an attending physician with two APPs). At times of increased census and demand, additional hospitalists were recruited to increase staffing on the service. During the night (7 pm-7 am), one physician was assigned to the EDB service. When the nighttime EDB census was high, other hospitalists providing care on inpatient units were expected to help care for boarding patients in the ED. Starting July 1, 2017, the dedicated nighttime staffing for the EDB service increased to two physicians during weeknights.

There was a dedicated nursing team for the EDB service. For AY16, there were two daytime EDB nurses and one night nurse, all with a coverage ratio of three to four patients per nurse. For AY17, there were four to five daytime nurses and two to three nighttime nurses with the same coverage ratio as that for AY16. EDB nurses received special training on caring for boarder patients and followed the usual inpatient nursing protocols and assessments. During each shift, an EDB charge nurse worked in conjunction with the hospitalist, bed management, and inpatient units to determine patients requiring coverage by the EDB team.

Patient Eligibility

Similar to the workflow before the intervention, the ED team was responsible for determining a patient’s need for admission to a medical service. Patients were eligible for EDB service coverage if they waited in the ED for more than two hours after the request for an inpatient bed was made. The EDB charge nurse was responsible for identifying all eligible boarder patients based on time elapsed since bed request. Patients were not eligible for the hospital medicine EDB service if they were in the ED observation units or were being admitted to the intensive care unit, cardiology service, oncology service, or any service outside of the Department of Medicine.

The EDB service did not automatically assume care of all eligible patients. Instead, eligible patients were accepted based on several factors including EDB clinician census, anticipated availability of an inpatient bed, and clinical appropriateness as deemed by the physician. If the EDB physician census was fewer than 10 patients and an eligible patient was not expected to move to an inpatient unit within the next hour, the patient was accepted by the EDB service. Patients who were not accepted by the EDB service remained under the care of the ED team until either the patient received an inpatient bed or space became available on the EDB service census. Eligible EDB patients who received an inpatient bed before being picked up by the EDB service were considered as noncovered EDB patients. Alternatively, an eligible patient may initially be declined from EDB service coverage due to, for example, a high census but later accepted when capacity allowed—this patient would be considered a covered EDB patient.

Handoff and Coordination

When an eligible patient was accepted onto the EDB service, clinical handoff between the ED and EDB teams occurred. The EDB physician wrote admission orders, including the inpatient admission order. Once on the EDB service, when space allowed, the patient was physically moved to a dedicated geographic space (8 beds) within the ED designed for the EDB service. When the dedicated EDB area was full, new patients would remain in their original patient bay and receive care from the EDB service. Multidisciplinary rounds with nursing, inpatient clinicians, and case management that normally occur every weekday on inpatient units were adapted to occur on the EDB service to discuss patient care needs. The duration of the patient’s stay in the ED, including the time on the EDB service, was dictated by bed availability rather than by clinical discretion of the EDB clinician. When an EDB patient was assigned a ready inpatient bed, the EDB clinician immediately passed off clinical care to the inpatient medical team. There was no change in the process of assigning patients to inpatient beds during the intervention period.

Study Population

This study included patients who were admitted to the general medical services through the ED during the defined period. We excluded medicine patients who did not pass through the ED (eg, direct admissions or outside transfer) as well as patients admitted to a specialty service (cardiology, oncology) or the intensive care unit. Patients admitted to a nonmedical service were also excluded.

Two hours following a bed request, an ED patient was designated as an eligible EDB patient. Operationally, and for the purposes of this study, patients were separated into three groups: (1) an eligible EDB patient for whom the EDB service assumed care for any portion of their ED stay was considered as a “covered ED boarder,” (2) an eligible EDB patient who did not have any coverage by the EDB service at any point during their ED stay was considered as a “noncovered boarder,” and (3) a patient who received an inpatient bed within two hours of bed request was considered as a “nonboarder”. Patients admitted to a specialty service, intensive care unit, or nonmedical services were not included in any of the abovementioned three groups.

We defined metrics to quantify the extent of EDB team coverage. First, the number of covered EDB patients was divided by all medicine boarders (covered + noncovered) to determine the percentage of medicine EDBs covered. Second, the total patient hours spent under the care of the EDB service was divided by the total boarding hours for all medicine boarders to determine the percentage of boarder hours covered.

Data Sources and Collection

The Electronic Health Record (EHR; Epic Systems Corporation, Verona, Wisconsin) captured whether patients were eligible EDBs. For covered EDB patients, the time when care was assumed by the EDB service was captured electronically. Patient demographics, admitting diagnoses, time stamps throughout the hospitalization, admission volumes, LOS, and discharge disposition were extracted from the EHR.

Primary and Secondary Outcome Measures

The primary outcome of this study was hospital LOS defined as the time from ED arrival to hospital departure (Figure). Secondary outcomes included ED LOS (time from ED arrival to ED departure) and the rate of 30-day ED readmission to the study institution.

Statistical Analysis

SAS version 9.4 (SAS Institute, Cary, North Carolina) was used for all statistical analyses. Continuous outcomes were compared using the Mann–Whitney test and dichotomized outcomes were compared using chi-square tests. We further analyzed the differences in the primary and secondary outcomes between covered and noncovered EDB groups using a multivariable regression analysis adjusting for age, gender, race, academic year, hour of the day, and day of the week at the time of becoming an EDB. We used quantile regression and linear regression with log-transformed continuous outcomes and logistic regression for the dichotomized outcome. A P value of .05 was used as a threshold for statistical significance.

Study Population and Demographics

There were a total of 16,668 patients admitted from the ED to the general medical services during the study period (Table 1). There were 8,776 (53%) patients in the covered EDB group, 5,866 (35%) patients in the noncovered EDB group, and 2,026 (12%) patients in the nonboarder group. There were more patients admitted during AY17 compared with AY16 (8,934 vs 7,734 patients, respectively, Appendix 1). Patient demographics, including age, gender, race, insurance coverage, admitting diagnoses, and discharge disposition, were similar among all three patient groups (Table 1). A majority of patients in the covered EDB and nonboarder groups presented to the ED in the afternoon, whereas noncovered EDB patients presented more in the morning (Table 1). Consistent with this pattern, inpatient bed requests for covered EDB and nonboarder patients were more frequent between 7 pm and 7 am, whereas bed requests for noncovered EDB patients were more frequent between 7 am and 7 pm. Median ED volume varied by hour with a peak in volume in the afternoon hours; however, the volume of eligible and covered EDB patients had a different peak in volume around noon that was consistent across the two years (Appendix 2). Overall, 59.9% of eligible patients (excluding nonboarders) were covered by the EDB service and 62.9% of the total boarding hours were covered by the EDB service.

Hospital Length of Stay

Nonboarders had the shortest median hospital LOS (4.76; interquartile range [IQR] 2.90-7.22 days). Covered EDB patients had a median hospital LOS that was 4.6 hours (0.19 day) shorter compared with noncovered EDB patients (4.92 [IQR 3.00-8.03] days vs 5.11 [IQR 3.16-8.34 days]; Table 2). The differences among the three groups were all significant in the univariate comparison (P < .001). Multivariable regression controlling for patient age, gender, race, academic year, and hour and day of the week at the time of becoming an EDB demonstrated that the difference in hospital LOS between covered and noncovered EDB patients remained significant (P < .001).

ED Length of Stay and 30-Day ED Readmission

Covered EDB patients had a longer median ED LOS compared with noncovered EDB patients and nonboarder patients (20.7 [IQR 15.8-24.9] hours vs 10.1 [IQR 7.9-13.8] hours vs 5.6 [IQR 4.2-7.5] hours, respectively, Table 2). These differences remained significant in the multivariable regression models (P < .001). Finally, the 30-day same-institution ED readmission rate was similar between covered and noncovered EDB patients.

We present two years of data describing a hospital medicine-led team designed to enhance the care of medical patients boarding in the ED. The period spent boarding in the ED is a vulnerable time for patients, and we created the EDB service with the goal of delivering inpatient medicine-led care to ED patients awaiting their inpatient bed.

When a bed request is made in an efficient ideal world, patients could be immediately transferred to an open inpatient bed to initiate care. In our study, patients who were not EDBs (ie, waited for less than two hours for their inpatient bed) had the most time-efficient care as they had the shortest ED and hospital LOS. However, nonboarders represented only 12% of patients and the majority of patients admitted to medicine were boarders. Patients covered by the EDB service had an overall hospital LOS that was 4.6 hours shorter compared with noncovered EDB patients despite having an ED LOS that was 15.1 hours longer. These LOS differences were observed without any difference to 30-day ED readmission rates.

Given that not all boarding patients were cared by the EDB service, the role of selection bias in our study warrants discussion. Similar to other studies, ED LOS for our patient cohort is heavily influenced by the availability of inpatient beds.^10-12 The EDB service handed off patients they were covering as soon as an inpatient bed became available. Although there was discretion from the EDB charge nurse and the EDB physician about which patient to accept, this was primarily focused on choosing patients who did not have a pending inpatient bed (eg, a patient who was assigned a bed but was awaiting room cleaning). Importantly, there was no change in the bed assignment process as a part of the intervention. Our intervention’s design did not allow for elucidation of causation; however, we believe that the longer ED LOS for covered EDB patients compared with noncovered EDB patients reflects the fact that the team chose patients with a higher expected ED LOS rather than that the patients had a longer LOS due to being cared by the service. Consistent with this, patients covered by the EDB service tended to have bed requests placed during the night shift compared with noncovered EDB patients; patients with bed requests at night are more likely to wait longer for their inpatient bed given that inpatient beds are generally freed up in the afternoon. We acknowledge that it is impossible to completely rule out the possibility that patient factors (eg, infectious precautions) influence inpatient bed wait time and could be another factor influencing the probability of EDB service coverage.

The current study adds to the expanding literature on EDB care models. Briones et al. demonstrated that an “ED hospitalist” led to increased care delivery as measured by an increased follow-up on laboratory results and medication orders.²³ However, their study was not structured to demonstrate LOS changes.²³ In another study, Chadaga et al. reported about their experience with a hospital medicine team providing care for EDB patients, similar to our study.⁸ Their hospital medicine team consisted of a hospitalist and APP deployed in the ED during the day, with night coverage provided by existing ED clinicians. They demonstrated less ED diversion, more ED discharges, and positive perceptions among the ED team.⁸ However, there was no impact on ED or hospital LOS, although their results may have been limited by the short duration of postintervention data and the lack of nighttime coverage.⁸ Finally, a modeling study demonstrated a reduction in ED LOS by adding ED clinicians only for patients being discharged from the ED and not for those being admitted, although there was no explicit adjustment for LOS accounting for initiation of inpatient care in the ED.¹⁵ Extending the current literature, our study suggests that a hospitalist team providing continuous coverage to a large portion of EDB patients could shorten the overall hospital LOS for boarding patients, but even this was not enough to reduce LOS to the same level as that of patients who did not board.

Practically, there were challenges to creating the EDB service described in our study. Additional clinical staff (physician, APP, and nursing) were hired for the team, requiring a financial commitment from the institution. The new team required space within the ED footprint incurring construction costs. Before the existence of the EDB service, other ancillary services (eg, physical therapy) were unaccustomed to seeing ED patients, and thus new workflows were created. Another challenge was that internal medicine clinicians were not used to caring for patients for short durations of time before passing off clinical care to another team. This required a different approach, focusing on acute issues rather than conducting an exhaustive evaluation. Finally, the EDB service workflow introduced an additional handoff, increasing discontinuity of care. These challenges are factors to consider for institutions considering a similar EDB team and should be weighed against other interventions to alleviate ED boarding or improve throughput such as expanding inpatient capacity.

Ideal metrics to track the coverage and performance of an EDB service such as the one described in this study are undefined. It was difficult to know whether the goal should be complete coverage given the increase in handoffs, particularly for patients with short boarding times. This EDB service covered 59.9% of boarding patients and 62.9% of total boarding hours. Factors that contributed to covering less than 100% included physician staffing that was insufficient to meet demand and discretion to not accept patients expected to quickly get an inpatient bed. Therefore, the percentage of patients and boarding hours covered are crude metrics and further investigation is needed to develop optimal metrics for an EDB team.

Future studies on care models for EDB patients are warranted. Recognizing that EDB teams require additional resources, studies to define which patients receive the most benefit from EDB coverage will be helpful. Moreover, the EDB team composition may need to adapt to different environments (eg, academic, urban, nonacademic, rural). Diving deeper to study whether specific patient populations benefit more than others from care by the EDB service, as measured by hospital LOS or other outcomes, would be important. Clinical outcomes, in addition to throughput metrics such as LOS, must be analyzed to understand whether factors such as increased handoffs outweigh any benefits in throughput.

There were several limitations to this study. First, it was performed at a single academic institution, potentially limiting its generalizability. However, although some workflows and team coverage structures may be institution-specific, the concept of a hospital medicine-led EDB team providing earlier inpatient care can be adapted locally and may probably achieve similar benefits. Our study population included only patients destined for general medical admission; thus, it is uncertain whether the gains demonstrated in our study would be realized for patients boarding for nonmedical services. In addition, considering the observational nature of this study, it is difficult to prove the causation that a hospitalist EDB service solely led to reductions in hospital LOS. Finally, we did not adjust for nor measure whether ED clinicians provided different care to patients whom they felt were destined for the EDB service.

In summary, nonboarder patients had the shortest overall LOS; however, among those patients who boarded, coverage by a hospitalist-led team was associated with a shorter LOS. Given the limited inpatient capacity, eliminating ED boarding is often not possible. We present a model to expedite inpatient care and allow ED clinicians to focus on newly arriving ED patients. Additional studies are required to better understand how to optimally care for patients boarding in the ED.

Emergency department (ED) crowding and boarding of patients awaiting admission to the hospital (ED boarding) are growing problems with important clinical care and public safety implications.^1-4 Increased ED boarding times have been associated with lower patient satisfaction, inadequate care of critically ill patients, adverse events, and increased mortality.^3,5-7 Furthermore, ED boarding can diminish the ED’s ability to evaluate new patients.^5,8,9 ED boarding is more severe in hospitals with high inpatient occupancy with resultant disproportionate burden on large urban institutions.^2,4,5,10

Earlier studies suggest, but have not consistently shown, an association between longer ED length of stay (LOS) and longer overall hospital LOS.⁵ This association implies that the additional time spent in the ED waiting for a bed does not meaningfully contribute to advancing the required inpatient care. Thus, this waiting time is “dead time” that is added to the overall hospital duration.

The complexity and the volume of medical patients boarding in the ED can challenge the resources of an already overtaxed ED staff. Potential solutions to mitigate ED boarding of medicine patients generally focus on reducing barriers to timely movement of patients from the ED to an inpatient unit.^1,3,11-13 Ultimately, these barriers are a function of inadequate hospital capacity (eg, hospital beds, staffing) and are difficult to overcome. Two primary strategies have been used to reduce these barriers. One strategy focuses on shifting inpatient discharge times earlier to better match inpatient bed supply with ED demand.^14-19 Another common strategy is utilizing inpatient attendings to triage and better match bed needs to bed availability.^20-22

A separate area of interest, and the focus of this study, is the deployment of inpatient teams to hasten delivery of inpatient care to patients waiting in the ED.^8,23 One institution implemented an “ED hospitalist” model.²³ Another created a hospital medicine team to provide inpatient medical care to ED boarder patients and to lend clinical input to bed management.⁸

At our large, urban academic medical center, the Department of Medicine in collaboration with the Department of Emergency Medicine created a full-time hospital medicine team dedicated to providing care in the ED for patients awaiting admission to a general medicine unit. We present our multiyear experience with this ED-based hospital medicine team. We hypothesized that this new team would expedite inpatient care delivery to medical boarder patients, thereby reducing the overall hospital LOS.

Study Setting and Design

This retrospective cross-sectional study, approved by the Institutional Review Board, was conducted at a 1,011-bed academic medical center in the northeast United States. The study period was July 1, 2016 through June 30, 2018, which was divided into Academic Year 16 (AY) (July 1, 2016 to June 30, 2017) and AY17 (July 1, 2017 to June 30, 2018).

The Hospital Medicine Unit (HMU) was a 60 full-time equivalent hospital medicine group consisting of 80 physicians and 25 advanced practice providers (APPs). During the study, the general medical services cared for an average of 260 patients per day on inpatient units with a wide variety of diagnoses and comorbidities. The ED had 48 monitored bed spaces for adult patients, as well as two dedicated ED observation units with 32 beds. The observation units are separate units within the hospital, staffed by ED clinicians, and were not included in this study. In 2016, the ED had a total of 110,741 patient visits and 13,908 patients were admitted to a medical service.

In 2010, the Department of Public Health for the state in which the medical center resides defined an ED boarder (EDB) patient as “a patient who remains in the ED two hours after the decision to admit.”²⁴ According to this definition, any patient waiting for an inpatient bed for more than two hours after a bed request was considered as an EDB. Operationally, further distinctions were made between patients who were “eligible” for care by an internal medicine team in the ED versus those who were actually “covered”. Before the intervention outlined in the current study, some care was provided by resident and hospitalist teams to eligible EDB patients from 2010 to 2015, although this was limited in scope. From July 1, 2015 to June 30, 2016, there was no coverage of medicine EDB patients.

Intervention

ED Boarder Service Staffing

On July 1, 2016, the HMU deployed a dedicated full-time team of clinicians to care for boarding patients, which was known as the EDB service. The service was created with the goal of seeing a maximum of 25 patients over 24 hours.

Inpatient medicine attending physicians (hospitalists) and APPs worked on the EDB service. During the day (7 am-7 pm), coverage was provided by three clinicians (generally an attending physician with two APPs). At times of increased census and demand, additional hospitalists were recruited to increase staffing on the service. During the night (7 pm-7 am), one physician was assigned to the EDB service. When the nighttime EDB census was high, other hospitalists providing care on inpatient units were expected to help care for boarding patients in the ED. Starting July 1, 2017, the dedicated nighttime staffing for the EDB service increased to two physicians during weeknights.

There was a dedicated nursing team for the EDB service. For AY16, there were two daytime EDB nurses and one night nurse, all with a coverage ratio of three to four patients per nurse. For AY17, there were four to five daytime nurses and two to three nighttime nurses with the same coverage ratio as that for AY16. EDB nurses received special training on caring for boarder patients and followed the usual inpatient nursing protocols and assessments. During each shift, an EDB charge nurse worked in conjunction with the hospitalist, bed management, and inpatient units to determine patients requiring coverage by the EDB team.

Patient Eligibility

Similar to the workflow before the intervention, the ED team was responsible for determining a patient’s need for admission to a medical service. Patients were eligible for EDB service coverage if they waited in the ED for more than two hours after the request for an inpatient bed was made. The EDB charge nurse was responsible for identifying all eligible boarder patients based on time elapsed since bed request. Patients were not eligible for the hospital medicine EDB service if they were in the ED observation units or were being admitted to the intensive care unit, cardiology service, oncology service, or any service outside of the Department of Medicine.

The EDB service did not automatically assume care of all eligible patients. Instead, eligible patients were accepted based on several factors including EDB clinician census, anticipated availability of an inpatient bed, and clinical appropriateness as deemed by the physician. If the EDB physician census was fewer than 10 patients and an eligible patient was not expected to move to an inpatient unit within the next hour, the patient was accepted by the EDB service. Patients who were not accepted by the EDB service remained under the care of the ED team until either the patient received an inpatient bed or space became available on the EDB service census. Eligible EDB patients who received an inpatient bed before being picked up by the EDB service were considered as noncovered EDB patients. Alternatively, an eligible patient may initially be declined from EDB service coverage due to, for example, a high census but later accepted when capacity allowed—this patient would be considered a covered EDB patient.

Handoff and Coordination

When an eligible patient was accepted onto the EDB service, clinical handoff between the ED and EDB teams occurred. The EDB physician wrote admission orders, including the inpatient admission order. Once on the EDB service, when space allowed, the patient was physically moved to a dedicated geographic space (8 beds) within the ED designed for the EDB service. When the dedicated EDB area was full, new patients would remain in their original patient bay and receive care from the EDB service. Multidisciplinary rounds with nursing, inpatient clinicians, and case management that normally occur every weekday on inpatient units were adapted to occur on the EDB service to discuss patient care needs. The duration of the patient’s stay in the ED, including the time on the EDB service, was dictated by bed availability rather than by clinical discretion of the EDB clinician. When an EDB patient was assigned a ready inpatient bed, the EDB clinician immediately passed off clinical care to the inpatient medical team. There was no change in the process of assigning patients to inpatient beds during the intervention period.

Study Population

This study included patients who were admitted to the general medical services through the ED during the defined period. We excluded medicine patients who did not pass through the ED (eg, direct admissions or outside transfer) as well as patients admitted to a specialty service (cardiology, oncology) or the intensive care unit. Patients admitted to a nonmedical service were also excluded.

Two hours following a bed request, an ED patient was designated as an eligible EDB patient. Operationally, and for the purposes of this study, patients were separated into three groups: (1) an eligible EDB patient for whom the EDB service assumed care for any portion of their ED stay was considered as a “covered ED boarder,” (2) an eligible EDB patient who did not have any coverage by the EDB service at any point during their ED stay was considered as a “noncovered boarder,” and (3) a patient who received an inpatient bed within two hours of bed request was considered as a “nonboarder”. Patients admitted to a specialty service, intensive care unit, or nonmedical services were not included in any of the abovementioned three groups.

We defined metrics to quantify the extent of EDB team coverage. First, the number of covered EDB patients was divided by all medicine boarders (covered + noncovered) to determine the percentage of medicine EDBs covered. Second, the total patient hours spent under the care of the EDB service was divided by the total boarding hours for all medicine boarders to determine the percentage of boarder hours covered.

Data Sources and Collection

The Electronic Health Record (EHR; Epic Systems Corporation, Verona, Wisconsin) captured whether patients were eligible EDBs. For covered EDB patients, the time when care was assumed by the EDB service was captured electronically. Patient demographics, admitting diagnoses, time stamps throughout the hospitalization, admission volumes, LOS, and discharge disposition were extracted from the EHR.

Primary and Secondary Outcome Measures

The primary outcome of this study was hospital LOS defined as the time from ED arrival to hospital departure (Figure). Secondary outcomes included ED LOS (time from ED arrival to ED departure) and the rate of 30-day ED readmission to the study institution.

Statistical Analysis

SAS version 9.4 (SAS Institute, Cary, North Carolina) was used for all statistical analyses. Continuous outcomes were compared using the Mann–Whitney test and dichotomized outcomes were compared using chi-square tests. We further analyzed the differences in the primary and secondary outcomes between covered and noncovered EDB groups using a multivariable regression analysis adjusting for age, gender, race, academic year, hour of the day, and day of the week at the time of becoming an EDB. We used quantile regression and linear regression with log-transformed continuous outcomes and logistic regression for the dichotomized outcome. A P value of .05 was used as a threshold for statistical significance.

Study Population and Demographics

There were a total of 16,668 patients admitted from the ED to the general medical services during the study period (Table 1). There were 8,776 (53%) patients in the covered EDB group, 5,866 (35%) patients in the noncovered EDB group, and 2,026 (12%) patients in the nonboarder group. There were more patients admitted during AY17 compared with AY16 (8,934 vs 7,734 patients, respectively, Appendix 1). Patient demographics, including age, gender, race, insurance coverage, admitting diagnoses, and discharge disposition, were similar among all three patient groups (Table 1). A majority of patients in the covered EDB and nonboarder groups presented to the ED in the afternoon, whereas noncovered EDB patients presented more in the morning (Table 1). Consistent with this pattern, inpatient bed requests for covered EDB and nonboarder patients were more frequent between 7 pm and 7 am, whereas bed requests for noncovered EDB patients were more frequent between 7 am and 7 pm. Median ED volume varied by hour with a peak in volume in the afternoon hours; however, the volume of eligible and covered EDB patients had a different peak in volume around noon that was consistent across the two years (Appendix 2). Overall, 59.9% of eligible patients (excluding nonboarders) were covered by the EDB service and 62.9% of the total boarding hours were covered by the EDB service.

Hospital Length of Stay

Nonboarders had the shortest median hospital LOS (4.76; interquartile range [IQR] 2.90-7.22 days). Covered EDB patients had a median hospital LOS that was 4.6 hours (0.19 day) shorter compared with noncovered EDB patients (4.92 [IQR 3.00-8.03] days vs 5.11 [IQR 3.16-8.34 days]; Table 2). The differences among the three groups were all significant in the univariate comparison (P < .001). Multivariable regression controlling for patient age, gender, race, academic year, and hour and day of the week at the time of becoming an EDB demonstrated that the difference in hospital LOS between covered and noncovered EDB patients remained significant (P < .001).

ED Length of Stay and 30-Day ED Readmission

Covered EDB patients had a longer median ED LOS compared with noncovered EDB patients and nonboarder patients (20.7 [IQR 15.8-24.9] hours vs 10.1 [IQR 7.9-13.8] hours vs 5.6 [IQR 4.2-7.5] hours, respectively, Table 2). These differences remained significant in the multivariable regression models (P < .001). Finally, the 30-day same-institution ED readmission rate was similar between covered and noncovered EDB patients.

We present two years of data describing a hospital medicine-led team designed to enhance the care of medical patients boarding in the ED. The period spent boarding in the ED is a vulnerable time for patients, and we created the EDB service with the goal of delivering inpatient medicine-led care to ED patients awaiting their inpatient bed.

When a bed request is made in an efficient ideal world, patients could be immediately transferred to an open inpatient bed to initiate care. In our study, patients who were not EDBs (ie, waited for less than two hours for their inpatient bed) had the most time-efficient care as they had the shortest ED and hospital LOS. However, nonboarders represented only 12% of patients and the majority of patients admitted to medicine were boarders. Patients covered by the EDB service had an overall hospital LOS that was 4.6 hours shorter compared with noncovered EDB patients despite having an ED LOS that was 15.1 hours longer. These LOS differences were observed without any difference to 30-day ED readmission rates.

Given that not all boarding patients were cared by the EDB service, the role of selection bias in our study warrants discussion. Similar to other studies, ED LOS for our patient cohort is heavily influenced by the availability of inpatient beds.^10-12 The EDB service handed off patients they were covering as soon as an inpatient bed became available. Although there was discretion from the EDB charge nurse and the EDB physician about which patient to accept, this was primarily focused on choosing patients who did not have a pending inpatient bed (eg, a patient who was assigned a bed but was awaiting room cleaning). Importantly, there was no change in the bed assignment process as a part of the intervention. Our intervention’s design did not allow for elucidation of causation; however, we believe that the longer ED LOS for covered EDB patients compared with noncovered EDB patients reflects the fact that the team chose patients with a higher expected ED LOS rather than that the patients had a longer LOS due to being cared by the service. Consistent with this, patients covered by the EDB service tended to have bed requests placed during the night shift compared with noncovered EDB patients; patients with bed requests at night are more likely to wait longer for their inpatient bed given that inpatient beds are generally freed up in the afternoon. We acknowledge that it is impossible to completely rule out the possibility that patient factors (eg, infectious precautions) influence inpatient bed wait time and could be another factor influencing the probability of EDB service coverage.

The current study adds to the expanding literature on EDB care models. Briones et al. demonstrated that an “ED hospitalist” led to increased care delivery as measured by an increased follow-up on laboratory results and medication orders.²³ However, their study was not structured to demonstrate LOS changes.²³ In another study, Chadaga et al. reported about their experience with a hospital medicine team providing care for EDB patients, similar to our study.⁸ Their hospital medicine team consisted of a hospitalist and APP deployed in the ED during the day, with night coverage provided by existing ED clinicians. They demonstrated less ED diversion, more ED discharges, and positive perceptions among the ED team.⁸ However, there was no impact on ED or hospital LOS, although their results may have been limited by the short duration of postintervention data and the lack of nighttime coverage.⁸ Finally, a modeling study demonstrated a reduction in ED LOS by adding ED clinicians only for patients being discharged from the ED and not for those being admitted, although there was no explicit adjustment for LOS accounting for initiation of inpatient care in the ED.¹⁵ Extending the current literature, our study suggests that a hospitalist team providing continuous coverage to a large portion of EDB patients could shorten the overall hospital LOS for boarding patients, but even this was not enough to reduce LOS to the same level as that of patients who did not board.

Practically, there were challenges to creating the EDB service described in our study. Additional clinical staff (physician, APP, and nursing) were hired for the team, requiring a financial commitment from the institution. The new team required space within the ED footprint incurring construction costs. Before the existence of the EDB service, other ancillary services (eg, physical therapy) were unaccustomed to seeing ED patients, and thus new workflows were created. Another challenge was that internal medicine clinicians were not used to caring for patients for short durations of time before passing off clinical care to another team. This required a different approach, focusing on acute issues rather than conducting an exhaustive evaluation. Finally, the EDB service workflow introduced an additional handoff, increasing discontinuity of care. These challenges are factors to consider for institutions considering a similar EDB team and should be weighed against other interventions to alleviate ED boarding or improve throughput such as expanding inpatient capacity.

Ideal metrics to track the coverage and performance of an EDB service such as the one described in this study are undefined. It was difficult to know whether the goal should be complete coverage given the increase in handoffs, particularly for patients with short boarding times. This EDB service covered 59.9% of boarding patients and 62.9% of total boarding hours. Factors that contributed to covering less than 100% included physician staffing that was insufficient to meet demand and discretion to not accept patients expected to quickly get an inpatient bed. Therefore, the percentage of patients and boarding hours covered are crude metrics and further investigation is needed to develop optimal metrics for an EDB team.

Future studies on care models for EDB patients are warranted. Recognizing that EDB teams require additional resources, studies to define which patients receive the most benefit from EDB coverage will be helpful. Moreover, the EDB team composition may need to adapt to different environments (eg, academic, urban, nonacademic, rural). Diving deeper to study whether specific patient populations benefit more than others from care by the EDB service, as measured by hospital LOS or other outcomes, would be important. Clinical outcomes, in addition to throughput metrics such as LOS, must be analyzed to understand whether factors such as increased handoffs outweigh any benefits in throughput.

There were several limitations to this study. First, it was performed at a single academic institution, potentially limiting its generalizability. However, although some workflows and team coverage structures may be institution-specific, the concept of a hospital medicine-led EDB team providing earlier inpatient care can be adapted locally and may probably achieve similar benefits. Our study population included only patients destined for general medical admission; thus, it is uncertain whether the gains demonstrated in our study would be realized for patients boarding for nonmedical services. In addition, considering the observational nature of this study, it is difficult to prove the causation that a hospitalist EDB service solely led to reductions in hospital LOS. Finally, we did not adjust for nor measure whether ED clinicians provided different care to patients whom they felt were destined for the EDB service.

In summary, nonboarder patients had the shortest overall LOS; however, among those patients who boarded, coverage by a hospitalist-led team was associated with a shorter LOS. Given the limited inpatient capacity, eliminating ED boarding is often not possible. We present a model to expedite inpatient care and allow ED clinicians to focus on newly arriving ED patients. Additional studies are required to better understand how to optimally care for patients boarding in the ED.

Emergency department (ED) crowding and boarding of patients awaiting admission to the hospital (ED boarding) are growing problems with important clinical care and public safety implications.^1-4 Increased ED boarding times have been associated with lower patient satisfaction, inadequate care of critically ill patients, adverse events, and increased mortality.^3,5-7 Furthermore, ED boarding can diminish the ED’s ability to evaluate new patients.^5,8,9 ED boarding is more severe in hospitals with high inpatient occupancy with resultant disproportionate burden on large urban institutions.^2,4,5,10

Earlier studies suggest, but have not consistently shown, an association between longer ED length of stay (LOS) and longer overall hospital LOS.⁵ This association implies that the additional time spent in the ED waiting for a bed does not meaningfully contribute to advancing the required inpatient care. Thus, this waiting time is “dead time” that is added to the overall hospital duration.

The complexity and the volume of medical patients boarding in the ED can challenge the resources of an already overtaxed ED staff. Potential solutions to mitigate ED boarding of medicine patients generally focus on reducing barriers to timely movement of patients from the ED to an inpatient unit.^1,3,11-13 Ultimately, these barriers are a function of inadequate hospital capacity (eg, hospital beds, staffing) and are difficult to overcome. Two primary strategies have been used to reduce these barriers. One strategy focuses on shifting inpatient discharge times earlier to better match inpatient bed supply with ED demand.^14-19 Another common strategy is utilizing inpatient attendings to triage and better match bed needs to bed availability.^20-22

A separate area of interest, and the focus of this study, is the deployment of inpatient teams to hasten delivery of inpatient care to patients waiting in the ED.^8,23 One institution implemented an “ED hospitalist” model.²³ Another created a hospital medicine team to provide inpatient medical care to ED boarder patients and to lend clinical input to bed management.⁸

At our large, urban academic medical center, the Department of Medicine in collaboration with the Department of Emergency Medicine created a full-time hospital medicine team dedicated to providing care in the ED for patients awaiting admission to a general medicine unit. We present our multiyear experience with this ED-based hospital medicine team. We hypothesized that this new team would expedite inpatient care delivery to medical boarder patients, thereby reducing the overall hospital LOS.

Study Setting and Design

This retrospective cross-sectional study, approved by the Institutional Review Board, was conducted at a 1,011-bed academic medical center in the northeast United States. The study period was July 1, 2016 through June 30, 2018, which was divided into Academic Year 16 (AY) (July 1, 2016 to June 30, 2017) and AY17 (July 1, 2017 to June 30, 2018).

The Hospital Medicine Unit (HMU) was a 60 full-time equivalent hospital medicine group consisting of 80 physicians and 25 advanced practice providers (APPs). During the study, the general medical services cared for an average of 260 patients per day on inpatient units with a wide variety of diagnoses and comorbidities. The ED had 48 monitored bed spaces for adult patients, as well as two dedicated ED observation units with 32 beds. The observation units are separate units within the hospital, staffed by ED clinicians, and were not included in this study. In 2016, the ED had a total of 110,741 patient visits and 13,908 patients were admitted to a medical service.

In 2010, the Department of Public Health for the state in which the medical center resides defined an ED boarder (EDB) patient as “a patient who remains in the ED two hours after the decision to admit.”²⁴ According to this definition, any patient waiting for an inpatient bed for more than two hours after a bed request was considered as an EDB. Operationally, further distinctions were made between patients who were “eligible” for care by an internal medicine team in the ED versus those who were actually “covered”. Before the intervention outlined in the current study, some care was provided by resident and hospitalist teams to eligible EDB patients from 2010 to 2015, although this was limited in scope. From July 1, 2015 to June 30, 2016, there was no coverage of medicine EDB patients.

Intervention

ED Boarder Service Staffing

On July 1, 2016, the HMU deployed a dedicated full-time team of clinicians to care for boarding patients, which was known as the EDB service. The service was created with the goal of seeing a maximum of 25 patients over 24 hours.

Inpatient medicine attending physicians (hospitalists) and APPs worked on the EDB service. During the day (7 am-7 pm), coverage was provided by three clinicians (generally an attending physician with two APPs). At times of increased census and demand, additional hospitalists were recruited to increase staffing on the service. During the night (7 pm-7 am), one physician was assigned to the EDB service. When the nighttime EDB census was high, other hospitalists providing care on inpatient units were expected to help care for boarding patients in the ED. Starting July 1, 2017, the dedicated nighttime staffing for the EDB service increased to two physicians during weeknights.

There was a dedicated nursing team for the EDB service. For AY16, there were two daytime EDB nurses and one night nurse, all with a coverage ratio of three to four patients per nurse. For AY17, there were four to five daytime nurses and two to three nighttime nurses with the same coverage ratio as that for AY16. EDB nurses received special training on caring for boarder patients and followed the usual inpatient nursing protocols and assessments. During each shift, an EDB charge nurse worked in conjunction with the hospitalist, bed management, and inpatient units to determine patients requiring coverage by the EDB team.

Patient Eligibility

Similar to the workflow before the intervention, the ED team was responsible for determining a patient’s need for admission to a medical service. Patients were eligible for EDB service coverage if they waited in the ED for more than two hours after the request for an inpatient bed was made. The EDB charge nurse was responsible for identifying all eligible boarder patients based on time elapsed since bed request. Patients were not eligible for the hospital medicine EDB service if they were in the ED observation units or were being admitted to the intensive care unit, cardiology service, oncology service, or any service outside of the Department of Medicine.

The EDB service did not automatically assume care of all eligible patients. Instead, eligible patients were accepted based on several factors including EDB clinician census, anticipated availability of an inpatient bed, and clinical appropriateness as deemed by the physician. If the EDB physician census was fewer than 10 patients and an eligible patient was not expected to move to an inpatient unit within the next hour, the patient was accepted by the EDB service. Patients who were not accepted by the EDB service remained under the care of the ED team until either the patient received an inpatient bed or space became available on the EDB service census. Eligible EDB patients who received an inpatient bed before being picked up by the EDB service were considered as noncovered EDB patients. Alternatively, an eligible patient may initially be declined from EDB service coverage due to, for example, a high census but later accepted when capacity allowed—this patient would be considered a covered EDB patient.

Handoff and Coordination

When an eligible patient was accepted onto the EDB service, clinical handoff between the ED and EDB teams occurred. The EDB physician wrote admission orders, including the inpatient admission order. Once on the EDB service, when space allowed, the patient was physically moved to a dedicated geographic space (8 beds) within the ED designed for the EDB service. When the dedicated EDB area was full, new patients would remain in their original patient bay and receive care from the EDB service. Multidisciplinary rounds with nursing, inpatient clinicians, and case management that normally occur every weekday on inpatient units were adapted to occur on the EDB service to discuss patient care needs. The duration of the patient’s stay in the ED, including the time on the EDB service, was dictated by bed availability rather than by clinical discretion of the EDB clinician. When an EDB patient was assigned a ready inpatient bed, the EDB clinician immediately passed off clinical care to the inpatient medical team. There was no change in the process of assigning patients to inpatient beds during the intervention period.

Study Population

This study included patients who were admitted to the general medical services through the ED during the defined period. We excluded medicine patients who did not pass through the ED (eg, direct admissions or outside transfer) as well as patients admitted to a specialty service (cardiology, oncology) or the intensive care unit. Patients admitted to a nonmedical service were also excluded.

Two hours following a bed request, an ED patient was designated as an eligible EDB patient. Operationally, and for the purposes of this study, patients were separated into three groups: (1) an eligible EDB patient for whom the EDB service assumed care for any portion of their ED stay was considered as a “covered ED boarder,” (2) an eligible EDB patient who did not have any coverage by the EDB service at any point during their ED stay was considered as a “noncovered boarder,” and (3) a patient who received an inpatient bed within two hours of bed request was considered as a “nonboarder”. Patients admitted to a specialty service, intensive care unit, or nonmedical services were not included in any of the abovementioned three groups.

We defined metrics to quantify the extent of EDB team coverage. First, the number of covered EDB patients was divided by all medicine boarders (covered + noncovered) to determine the percentage of medicine EDBs covered. Second, the total patient hours spent under the care of the EDB service was divided by the total boarding hours for all medicine boarders to determine the percentage of boarder hours covered.

Data Sources and Collection

The Electronic Health Record (EHR; Epic Systems Corporation, Verona, Wisconsin) captured whether patients were eligible EDBs. For covered EDB patients, the time when care was assumed by the EDB service was captured electronically. Patient demographics, admitting diagnoses, time stamps throughout the hospitalization, admission volumes, LOS, and discharge disposition were extracted from the EHR.

Primary and Secondary Outcome Measures

The primary outcome of this study was hospital LOS defined as the time from ED arrival to hospital departure (Figure). Secondary outcomes included ED LOS (time from ED arrival to ED departure) and the rate of 30-day ED readmission to the study institution.

Statistical Analysis

SAS version 9.4 (SAS Institute, Cary, North Carolina) was used for all statistical analyses. Continuous outcomes were compared using the Mann–Whitney test and dichotomized outcomes were compared using chi-square tests. We further analyzed the differences in the primary and secondary outcomes between covered and noncovered EDB groups using a multivariable regression analysis adjusting for age, gender, race, academic year, hour of the day, and day of the week at the time of becoming an EDB. We used quantile regression and linear regression with log-transformed continuous outcomes and logistic regression for the dichotomized outcome. A P value of .05 was used as a threshold for statistical significance.

Study Population and Demographics

There were a total of 16,668 patients admitted from the ED to the general medical services during the study period (Table 1). There were 8,776 (53%) patients in the covered EDB group, 5,866 (35%) patients in the noncovered EDB group, and 2,026 (12%) patients in the nonboarder group. There were more patients admitted during AY17 compared with AY16 (8,934 vs 7,734 patients, respectively, Appendix 1). Patient demographics, including age, gender, race, insurance coverage, admitting diagnoses, and discharge disposition, were similar among all three patient groups (Table 1). A majority of patients in the covered EDB and nonboarder groups presented to the ED in the afternoon, whereas noncovered EDB patients presented more in the morning (Table 1). Consistent with this pattern, inpatient bed requests for covered EDB and nonboarder patients were more frequent between 7 pm and 7 am, whereas bed requests for noncovered EDB patients were more frequent between 7 am and 7 pm. Median ED volume varied by hour with a peak in volume in the afternoon hours; however, the volume of eligible and covered EDB patients had a different peak in volume around noon that was consistent across the two years (Appendix 2). Overall, 59.9% of eligible patients (excluding nonboarders) were covered by the EDB service and 62.9% of the total boarding hours were covered by the EDB service.

Hospital Length of Stay

Nonboarders had the shortest median hospital LOS (4.76; interquartile range [IQR] 2.90-7.22 days). Covered EDB patients had a median hospital LOS that was 4.6 hours (0.19 day) shorter compared with noncovered EDB patients (4.92 [IQR 3.00-8.03] days vs 5.11 [IQR 3.16-8.34 days]; Table 2). The differences among the three groups were all significant in the univariate comparison (P < .001). Multivariable regression controlling for patient age, gender, race, academic year, and hour and day of the week at the time of becoming an EDB demonstrated that the difference in hospital LOS between covered and noncovered EDB patients remained significant (P < .001).

ED Length of Stay and 30-Day ED Readmission

Covered EDB patients had a longer median ED LOS compared with noncovered EDB patients and nonboarder patients (20.7 [IQR 15.8-24.9] hours vs 10.1 [IQR 7.9-13.8] hours vs 5.6 [IQR 4.2-7.5] hours, respectively, Table 2). These differences remained significant in the multivariable regression models (P < .001). Finally, the 30-day same-institution ED readmission rate was similar between covered and noncovered EDB patients.

We present two years of data describing a hospital medicine-led team designed to enhance the care of medical patients boarding in the ED. The period spent boarding in the ED is a vulnerable time for patients, and we created the EDB service with the goal of delivering inpatient medicine-led care to ED patients awaiting their inpatient bed.

When a bed request is made in an efficient ideal world, patients could be immediately transferred to an open inpatient bed to initiate care. In our study, patients who were not EDBs (ie, waited for less than two hours for their inpatient bed) had the most time-efficient care as they had the shortest ED and hospital LOS. However, nonboarders represented only 12% of patients and the majority of patients admitted to medicine were boarders. Patients covered by the EDB service had an overall hospital LOS that was 4.6 hours shorter compared with noncovered EDB patients despite having an ED LOS that was 15.1 hours longer. These LOS differences were observed without any difference to 30-day ED readmission rates.

Given that not all boarding patients were cared by the EDB service, the role of selection bias in our study warrants discussion. Similar to other studies, ED LOS for our patient cohort is heavily influenced by the availability of inpatient beds.^10-12 The EDB service handed off patients they were covering as soon as an inpatient bed became available. Although there was discretion from the EDB charge nurse and the EDB physician about which patient to accept, this was primarily focused on choosing patients who did not have a pending inpatient bed (eg, a patient who was assigned a bed but was awaiting room cleaning). Importantly, there was no change in the bed assignment process as a part of the intervention. Our intervention’s design did not allow for elucidation of causation; however, we believe that the longer ED LOS for covered EDB patients compared with noncovered EDB patients reflects the fact that the team chose patients with a higher expected ED LOS rather than that the patients had a longer LOS due to being cared by the service. Consistent with this, patients covered by the EDB service tended to have bed requests placed during the night shift compared with noncovered EDB patients; patients with bed requests at night are more likely to wait longer for their inpatient bed given that inpatient beds are generally freed up in the afternoon. We acknowledge that it is impossible to completely rule out the possibility that patient factors (eg, infectious precautions) influence inpatient bed wait time and could be another factor influencing the probability of EDB service coverage.

The current study adds to the expanding literature on EDB care models. Briones et al. demonstrated that an “ED hospitalist” led to increased care delivery as measured by an increased follow-up on laboratory results and medication orders.²³ However, their study was not structured to demonstrate LOS changes.²³ In another study, Chadaga et al. reported about their experience with a hospital medicine team providing care for EDB patients, similar to our study.⁸ Their hospital medicine team consisted of a hospitalist and APP deployed in the ED during the day, with night coverage provided by existing ED clinicians. They demonstrated less ED diversion, more ED discharges, and positive perceptions among the ED team.⁸ However, there was no impact on ED or hospital LOS, although their results may have been limited by the short duration of postintervention data and the lack of nighttime coverage.⁸ Finally, a modeling study demonstrated a reduction in ED LOS by adding ED clinicians only for patients being discharged from the ED and not for those being admitted, although there was no explicit adjustment for LOS accounting for initiation of inpatient care in the ED.¹⁵ Extending the current literature, our study suggests that a hospitalist team providing continuous coverage to a large portion of EDB patients could shorten the overall hospital LOS for boarding patients, but even this was not enough to reduce LOS to the same level as that of patients who did not board.

Practically, there were challenges to creating the EDB service described in our study. Additional clinical staff (physician, APP, and nursing) were hired for the team, requiring a financial commitment from the institution. The new team required space within the ED footprint incurring construction costs. Before the existence of the EDB service, other ancillary services (eg, physical therapy) were unaccustomed to seeing ED patients, and thus new workflows were created. Another challenge was that internal medicine clinicians were not used to caring for patients for short durations of time before passing off clinical care to another team. This required a different approach, focusing on acute issues rather than conducting an exhaustive evaluation. Finally, the EDB service workflow introduced an additional handoff, increasing discontinuity of care. These challenges are factors to consider for institutions considering a similar EDB team and should be weighed against other interventions to alleviate ED boarding or improve throughput such as expanding inpatient capacity.

Ideal metrics to track the coverage and performance of an EDB service such as the one described in this study are undefined. It was difficult to know whether the goal should be complete coverage given the increase in handoffs, particularly for patients with short boarding times. This EDB service covered 59.9% of boarding patients and 62.9% of total boarding hours. Factors that contributed to covering less than 100% included physician staffing that was insufficient to meet demand and discretion to not accept patients expected to quickly get an inpatient bed. Therefore, the percentage of patients and boarding hours covered are crude metrics and further investigation is needed to develop optimal metrics for an EDB team.

Future studies on care models for EDB patients are warranted. Recognizing that EDB teams require additional resources, studies to define which patients receive the most benefit from EDB coverage will be helpful. Moreover, the EDB team composition may need to adapt to different environments (eg, academic, urban, nonacademic, rural). Diving deeper to study whether specific patient populations benefit more than others from care by the EDB service, as measured by hospital LOS or other outcomes, would be important. Clinical outcomes, in addition to throughput metrics such as LOS, must be analyzed to understand whether factors such as increased handoffs outweigh any benefits in throughput.

There were several limitations to this study. First, it was performed at a single academic institution, potentially limiting its generalizability. However, although some workflows and team coverage structures may be institution-specific, the concept of a hospital medicine-led EDB team providing earlier inpatient care can be adapted locally and may probably achieve similar benefits. Our study population included only patients destined for general medical admission; thus, it is uncertain whether the gains demonstrated in our study would be realized for patients boarding for nonmedical services. In addition, considering the observational nature of this study, it is difficult to prove the causation that a hospitalist EDB service solely led to reductions in hospital LOS. Finally, we did not adjust for nor measure whether ED clinicians provided different care to patients whom they felt were destined for the EDB service.

In summary, nonboarder patients had the shortest overall LOS; however, among those patients who boarded, coverage by a hospitalist-led team was associated with a shorter LOS. Given the limited inpatient capacity, eliminating ED boarding is often not possible. We present a model to expedite inpatient care and allow ED clinicians to focus on newly arriving ED patients. Additional studies are required to better understand how to optimally care for patients boarding in the ED.

Neurologic impairment (NI) encompasses static and progressive diseases of the central and/or peripheral nervous systems that result in functional and intellectual impairments.¹ While a variety of neurologic diseases are responsible for NI (eg, hypoxic-ischemic encephalopathy, muscular dystrophy), consequences of these diseases extend beyond neurologic manifestations.¹ These children are at an increased risk for aspiration of oral and gastric contents given their common comorbidities of dysphagia, gastroesophageal reflux, impaired cough, and respiratory muscle weakness.² While aspiration may manifest as a self-resolving pneumonitis, the presence of oral or enteric bacteria in aspirated material may result in the development of bacterial pneumonia. Children with NI hospitalized with aspiration pneumonia have higher complication rates, longer and costlier hospitalizations, and higher readmission rates when compared with children with nonaspiration pneumonia.³

While pediatric aspiration pneumonia is commonly attributed to anaerobic bacteria, this is largely based on extrapolation from epidemiologic studies that were conducted in past decades.^4-8 A single randomized controlled trial found that penicillin and clindamycin, antimicrobials with similar antimicrobial activity against anaerobes, to be equally effective.⁹ However, the recent literature emphasizes the polymicrobial nature of aspiration pneumonia in adults, with the common isolation of Gram-negative enteric bacteria.¹⁰ Further, while Pseudomonas aeruginosa is often identified in respiratory cultures from children with NI and chronic respiratory insufficiency,^11,12 the significance of P. aeruginosa in lower airways remains unclear.

We designed this study to compare hospital outcomes associated with the most commonly prescribed empiric antimicrobial therapies for aspiration pneumonia in children with NI.

Study Design and Data Source

This multicenter, retrospective cohort study used the Pediatric Health Information System (PHIS) database. PHIS, an administrative database of 50 not-for-profit tertiary care pediatric hospitals, contains data regarding patient demographics, diagnoses and procedures, and daily billed resource utilization, including laboratory and imaging studies. Data quality and reliability are assured through the Children’s Hospital Association (CHA; Lenexa, Kansas) and participating hospitals. Due to incomplete data through the study period and data quality issues, six hospitals were excluded.

Inclusion Criteria

Children 1-18 years of age who were discharged between July 1, 2007 and June 30, 2015 were included if they had a NI diagnosis,¹ a principal diagnosis indicative of aspiration pneumonia (507.x),^3,13,14 and received antibiotics in the first two calendar days of admission. NI was determined using previously defined International Classification of Diseases, Ninth Revision-Clinical Modification (ICD-9-CM) diagnosis codes.¹ We only included children who received antibiotics in the first two calendar days of admission to minimize the likelihood of including children admitted for other reasons who acquired aspiration pneumonia after hospitalization. For children with multiple hospitalizations, one admission was randomly selected for inclusion to minimize weighting results toward repeat visits.

Exclusion Criteria

Children transferred from another hospital were excluded as records from their initial presentation, including treatment and outcomes, were not available. We also excluded children with tracheostomy^15,16 or chronic ventilator dependence,¹⁷ those with a diagnosis of human immunodeficiency virus or tuberculosis, and children who received chemotherapy during hospitalization given expected differences in etiology, treatment, and outcomes.¹⁸

Exposure

The primary exposure was antibiotic therapy received in the first two days of admission. Antibiotics were classified by their antimicrobial spectra of activity as defined by The Sanford Guide to Antimicrobial Therapy¹⁹ against the most commonly recognized pathogens of aspiration pneumonia: anaerobes, Gram-negatives, and P. aeruginosa (Appendix Table 1).^10,20 For example, penicillin G and clindamycin were among the antibiotics classified as providing anaerobic coverage alone, whereas ceftriaxone was classified as providing Gram-negative coverage alone and ampicillin-sulbactam or as combination therapy with clindamycin and ceftriaxone were classified as providing anaerobic and Gram-negative coverage. Piperacillin-tazobactam and meropenem were classified as providing anaerobic, Gram-negative, and P. aeruginosa coverage. We excluded antibiotics that do not provide coverage against anaerobes, Gram-negative, or P. aeruginosa (eg, ampicillin, azithromycin) or that provide coverage against Gram-negative and P. aeruginosa, but not anaerobes (eg, cefepime, tobramycin), as these therapies were prescribed for <5% of the cohort. We chose not to examine the coverage for Streptococcus pneumonia or Staphylococcus aureus as antibiotics included in this analysis covered these bacteria for 99.9% of our cohort.

Outcomes included acute respiratory failure during hospitalization, intensive care unit (ICU) transfer, and hospital length of stay (LOS). Acute respiratory failure during hospitalization was defined as the presence of Clinical Transaction Classification (CTC) or ICD-9 procedure code for noninvasive or invasive mechanical ventilation on day two or later of hospitalization, with or without the need for respiratory support on day 0 or day 1 (Appendix Table 2). Given the variability in hospital policies that may drive ICU admission criteria for complex patients, our outcome of ICU transfer was defined as the requirement for ICU level care on day two or later of hospitalization without ICU admission. Acute respiratory failure and ICU care occurring within the first two hospital days were not classified as outcomes because these early events likely reflect illness severity at presentation rather than outcomes attributable to treatment failure; these were included as markers of severity in the models.

Patient Demographics and Clinical Characteristics

Demographic and clinical characteristics that might influence antibiotic choice and/or hospital outcomes were assessed. Clinical characteristics included complex chronic conditions,^21-23 medical technology assistance,²⁴ performance of diagnostic testing, and markers of severe illness on presentation. Diagnostic testing included bacterial cultures (blood, respiratory, urine) and chest radiograph performance in the first two days of hospitalization. Results of diagnostic testing are not available in the PHIS. Illness severity on presentation included acute respiratory failure, pleural drainage, receipt of vasoactive agents, and transfusion of blood products in the first two days of hospitalization (Appendix Table 2).^17,25,26

Continuous data were described with median and interquartile ranges (IQR) due to nonnormal distribution. Categorical data were described with frequencies and percentages. Patient demographics, clinical characteristics, and hospital outcomes were stratified by empiric antimicrobial coverage and compared using chi-square and Kruskal–Wallis tests as appropriate.

Generalized linear mixed-effects models with random hospital intercepts were derived to assess the independent effect of antimicrobial spectra of activity on outcomes of acute respiratory failure, ICU transfer, and LOS while adjusting for important differences in demographic and clinical characteristics. LOS had a nonnormal distribution. Thus, we used an exponential distribution. Covariates were chosen a priori given the clinical and biological relevance to exposure and outcomes—age, presence of complex chronic condition diagnoses, the number of complex chronic conditions, technology dependence, the performance of diagnostic tests on presentation, and illness severity on presentation. ICU admission was included as a covariate in acute respiratory failure and LOS outcome models. The results of the model for acute respiratory failure and ICU transfer are presented as adjusted odds ratios (OR) with a 95% CI. LOS results are presented as adjusted rate ratios (RR) with 95% CI.

All analyses were performed with SAS 9.3 (SAS Institute, Cary, North Carolina). P values <.05 were considered statistically significant. Cincinnati Children’s Hospital Medical Center Institutional Review Board considered this deidentified dataset study as not human subjects research.

Study Cohort

At the 44 hospitals included, 4,812 children with NI hospitalized with the diagnosis of aspiration pneumonia met the eligibility criteria. However, 79 received antibiotics with the spectra of activity not examined, leaving 4,733 children in our final analysis (Appendix Figure). Demographic and clinical characteristics of the study cohort are shown in Table 1. Median age was five years (interquartile range [IQR]: 2-11 years). Most subjects were male (53.9%), non-Hispanic white (47.9%), and publicly insured (63.6%). There was a slight variation in the distribution of admissions across seasons (spring 31.6%, summer 19.2%, fall 21.3%, and winter 27.9%). One-third of children had four or more comorbid CCCs (complex chronic conditions; 34.2%). The three most common nonneurologic CCC diagnosis categories were gastrointestinal (63.1%), congenital and/or genetic defects (36.9%), and respiratory (8.9%). Assistance with medical technologies was also common (82%)—particularly gastrointestinal (63.1%) and neurologic/neuromuscular (9.8%) technologies. The vast majority of children (92.5%) had either a chest radiograph (90.5%), respiratory viral study (33.7%), or respiratory culture (10.0%) obtained on presentation. A minority required noninvasive or invasive respiratory support (25.4%), vasoactive agents (8.9%), blood products (1.2%), or pleural drainage (0.3%) in the first two hospital days.

Spectrum of Antimicrobial Coverage

Most children (57.9%) received anaerobic and Gram-negative coverage; 16.2% received anaerobic, Gram-negative and P. aeruginosa coverage; 15.3% received anaerobic coverage alone; and 10.6% received Gram-negative coverage alone. Empiric antimicrobial coverage varied substantially across hospitals: anaerobic coverage was prescribed for 0%-44% of patients; Gram-negative coverage was prescribed for 3%-26% of patients; anaerobic and Gram-negative coverage was prescribed for 25%-90% of patients; and anaerobic, Gram-negative, and P. aeruginosa coverage was prescribed for 0%-65% of patients (Figure 1).

Outcomes

Acute Respiratory Failure

One-quarter (25.4%) of patients had acute respiratory failure on presentation; 22.5% required respiratory support (continued from presentation or were new) on day two or later of hospitalization (Table 2). In the adjusted analysis, children receiving Gram-negative coverage alone had two-fold greater odds (OR 2.15, 95% CI: 1.41-3.27) and children receiving anaerobic and Gram-negative coverage had 1.6-fold greater odds (OR 1.65, 95% CI: 1.19-2.28), of respiratory failure during hospitalization compared with those receiving anaerobic coverage alone (Figure 2). Odds of respiratory failure during hospitalization did not significantly differ for children receiving anaerobic, Gram-negative, and P. aeruginosa coverage compared with those receiving anaerobic coverage alone.

ICU Transfer

Nearly thirty percent (29.0%) of children required ICU admission, with an additional 3.8% requiring ICU transfer following admission (Table 2). In the multivariable analysis, the odds of an ICU transfer were greater for children receiving Gram-negative coverage alone (OR 1.80, 95% CI: 1.03-3.14) compared with those receiving anaerobic coverage alone. There was no statistical difference in ICU transfer for those receiving anaerobic and Gram-negative coverage (with or without P. aeruginosa coverage) compared with those receiving anaerobic coverage alone (Figure 2).

Length of Stay

Median hospital LOS for the total cohort was five days (IQR: 3-9 days; Table 2). In the multivariable analysis, children receiving Gram-negative coverage alone had a longer LOS (RR 1.28; 95% CI: 1.16-1.41) compared with those receiving anaerobic coverage alone, whereas children receiving anaerobic, Gram-negative, and P. aeruginosa coverage had a shorter LOS (RR 0.83; 95% CI: 0.76-0.90) than those receiving anaerobic coverage alone (Figure 2). There was no statistical difference in the LOS between children receiving anaerobic and Gram-negative coverage and those receiving anaerobic coverage alone.

In this multicenter study of children with NI hospitalized with aspiration pneumonia, we found substantial variation in empiric antimicrobial coverage for children with aspiration pneumonia. When comparing outcomes across groups, children who received anaerobic and Gram-negative coverage had outcomes similar to children who received anaerobic therapy alone. However, children who did not receive anaerobic coverage (ie, Gram-negative coverage alone) had worse outcomes, most notably a greater than two-fold increase in the odds of experiencing acute respiratory failure during hospitalization when compared with children receiving anaerobic therapy. These findings support prior literature that has highlighted the importance of anaerobic therapy in the treatment of aspiration pneumonia. The benefit of antibiotics targeting Gram-negative organisms, in addition to anaerobes, remains uncertain.

The variability in empiric antimicrobial coverage likely reflects the paucity of available information on oral and/or enteric bacteria required to identify them as causative organisms in aspiration pneumonia. In part, this problem is due to the difficulty in obtaining adequate sputum for culture from pediatric patients.²⁷ While it may be more feasible to obtain tracheal aspirates for respiratory culture in children with a tracheostomy, interpretation of culture results remains challenging because the lower airways of children with tracheostomy are commonly colonized with bacterial pathogens.²⁸ Thus, physicians are often left to choose empiric antimicrobial coverage with inadequate supporting evidence.²⁹ Although the polymicrobial nature of aspiration pneumonia is well recognized in adult and pediatric literature,^10,30 it is less clear which organisms are of pathological significance and require treatment.

The treatment standard for aspiration pneumonia has long included anaerobic therapy.²⁹ The worse outcomes of children not receiving anaerobic therapy (ie, Gram-negative coverage alone) compared with children who received anaerobic therapy support the continued importance of anaerobic therapy in the treatment of aspiration pneumonia for hospitalized children with NI. The role of antibiotics covering Gram-negative organisms is less clear. Recent studies suggest the role of anaerobes is overemphasized in the etiology and treatment of aspiration pneumonia.^10,29,31-38 Multiple studies on aspiration pneumonia bacteriology in hospitalized adults have demonstrated a predominance of Gram-negative organisms (ranging from 37%-71% of isolates identified on respiratory culture) and a relative scarcity of anaerobes (ranging from 0%-16% of isolates).^31-37 A prospective study of 50 children hospitalized with clinical and radiographic evidence of pneumonia with known aspiration risk (eg, neuromuscular disease or dysphagia) found that ~80% of 163 bacterial isolates were Gram-negative.³⁸ However, this study included repeat cultures from the same children, and thus, may overestimate the prevalence of Gram-negative organisms. In our study, children who received both anaerobic and Gram-negative therapy had no differences in ICU transfer or LOS but did experience higher odds of acute respiratory failure. As these results may be due to unmeasured confounding, future studies should further explore the necessity of Gram-negative coverage in addition to anaerobic coverage in this population.

While these recent studies may seem to suggest that anaerobic coverage is not necessary for aspiration pneumonia, there are important limitations worth noting. First, these studies used a variety of sampling techniques. While organisms grown from samples obtained via bronchoalveolar lavage^31-34,36 are likely pathogenic, those grown from tracheal or oral samples obtained via percutaneous transtracheal aspiration,³⁴ a protected specimen brush,^34,36,37 or expectorated sputum^35,38 may not represent lower airway organisms. Second, anaerobic cultures were not obtained in all studies.^31,34,38 Anaerobic organisms are difficult to isolate using traditional clinical specimen collection techniques and aerobic culture media.¹⁸ Furthermore, anaerobes are not easily recovered from lung infections after the receipt of antibiotic therapy.³⁹ Details regarding pretreatment, which are largely lacking from these studies, are necessary to interpret the relative scarcity of anaerobes on respiratory culture. Finally, caution should be taken when extrapolating the results of studies focused on the etiology and treatment of aspiration pneumonia in elderly adults to children. Our results, particularly in the context of the limitation of these more recent studies, suggest that the role of anaerobes has been underestimated.

Recent studies examining populations of children with cerebral palsy and/or tracheostomy have emphasized the high rates of carriage and infection rates with Gram-negative and drug-resistant bacteria; in particular, P. aeruginosa accounts for 50%-72% of pathogenic bacteria.^11,12,38,40These studies note the generally poor outcomes of children with P. aeruginosa—including multiple and longer hospitalizations, frequent readmissions, and the increased severity of pneumonia, including the need for ICU admission, pleural effusions, the need for intubation, and mortality.^{11,12,38,40,41} In our study, nearly 35% of children who received anaerobic, Gram-negative, and P. aeruginosa coverage experienced acute respiratory failure during hospitalization compared with 20% of children who received other therapies. While these results might seem to suggest that broader spectrum therapy is harmful, they must be interpreted in the context of important population differences; children who received a combination of anaerobic, Gram-negative, and P. aeruginosa coverage had greater medical complexity and greater severity of illness on presentation. Such factors may provide the reason for the appropriate prescription of antipseudomonal antibiotics (eg, history of tracheostomy colonization or infection, long-term care facility resident).⁴² When we controlled for population differences, children who received antipseudomonal therapy had a significantly shorter LOS and no differences in outcomes of acute respiratory failure or ICU transfer compared with those receiving anaerobic therapy alone. This result suggests that worse outcomes were associated with antipseudomonal therapy on unadjusted analyses resulting from underlying medical complexity and illness severity rather than from colonization or infection with P. aeruginosa.

Our multicenter observational study has several limitations. We used diagnosis codes to identify patients with aspiration pneumonia. As validated clinical criteria for the diagnosis of aspiration pneumonia do not exist, clinicians may assign a diagnosis of and treatment for aspiration pneumonia by subjective suspicion based on a child’s severe NI or illness severity on presentation leading to selection bias. Although administrative data are not able to verify pneumonia type with absolute certainty, we previously demonstrated that the differences in the outcomes of children with aspiration and nonaspiration pneumonia diagnosis codes persist after accounting for the complexity that might influence the diagnosis.³It is also possible that the diagnosis of aspiration pneumonia was not made upon admission for a subset of patients leading to misclassification of exposure. Some children may have had aspiration pneumonia on admission but were not assigned that diagnosis or treated for presumed aspiration pneumonia until later in the hospital course as they demonstrated treatment failure or clinical worsening. It is also possible that some children had an aspiration event during hospitalization that developed into aspiration pneumonia. We attempted to adjust for medical complexity and illness severity through multivariable adjustment based on the diagnosis and procedure codes, as well as the laboratory testing performed. However, unmeasured or residual confounding may remain as administrative data are not equipped to distinguish detailed functional status (eg, ability to cough, chest wall strength) or illness severity (eg, respiratory distress) that might influence antibiotic selection and/or outcomes.

Frthermore, we were unable to account for laboratory, microbiology, or radiology test results, and other management practices (eg, frequency of airway clearance, previous antimicrobial therapy) that may influence outcomes. Future studies should certainly include an examination of the concordance of the antibiotics prescribed with causative organisms, as this undoubtedly affects patient outcomes. Other outcomes are important to examine (eg, time to return to respiratory baseline), but we were unable to do so, given the lack of clinical detail in our database. We randomly selected a single hospitalization for children with multiple admissions; alternative methods could have different results. Although children with NI predominately use children’s hospitals,¹ results may not be generalizable.

These findings support prior literature that has highlighted the important role anaerobic therapy plays in the treatment of aspiration pneumonia in children with NI. In light of the limitations of our study design, we believe that rigorous clinical trials comparing anaerobic with anaerobic and Gram-negative therapy are an important and necessary next step to determine the optimal treatment for aspiration pneumonia in this population.

Disclosures

The authors do not have any financial relationships relevant to this article to disclose.

Funding

Dr. Thomson was supported by the Agency for Healthcare Research and Quality (AHRQ) under award number K08HS025138. Dr. Ambroggio was supported by the National Institute for Allergy and Infectious Diseases (NIAID) under award number K01AI125413. The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ or NIAID.

Neurologic impairment (NI) encompasses static and progressive diseases of the central and/or peripheral nervous systems that result in functional and intellectual impairments.¹ While a variety of neurologic diseases are responsible for NI (eg, hypoxic-ischemic encephalopathy, muscular dystrophy), consequences of these diseases extend beyond neurologic manifestations.¹ These children are at an increased risk for aspiration of oral and gastric contents given their common comorbidities of dysphagia, gastroesophageal reflux, impaired cough, and respiratory muscle weakness.² While aspiration may manifest as a self-resolving pneumonitis, the presence of oral or enteric bacteria in aspirated material may result in the development of bacterial pneumonia. Children with NI hospitalized with aspiration pneumonia have higher complication rates, longer and costlier hospitalizations, and higher readmission rates when compared with children with nonaspiration pneumonia.³

While pediatric aspiration pneumonia is commonly attributed to anaerobic bacteria, this is largely based on extrapolation from epidemiologic studies that were conducted in past decades.^4-8 A single randomized controlled trial found that penicillin and clindamycin, antimicrobials with similar antimicrobial activity against anaerobes, to be equally effective.⁹ However, the recent literature emphasizes the polymicrobial nature of aspiration pneumonia in adults, with the common isolation of Gram-negative enteric bacteria.¹⁰ Further, while Pseudomonas aeruginosa is often identified in respiratory cultures from children with NI and chronic respiratory insufficiency,^11,12 the significance of P. aeruginosa in lower airways remains unclear.

We designed this study to compare hospital outcomes associated with the most commonly prescribed empiric antimicrobial therapies for aspiration pneumonia in children with NI.

Study Design and Data Source

This multicenter, retrospective cohort study used the Pediatric Health Information System (PHIS) database. PHIS, an administrative database of 50 not-for-profit tertiary care pediatric hospitals, contains data regarding patient demographics, diagnoses and procedures, and daily billed resource utilization, including laboratory and imaging studies. Data quality and reliability are assured through the Children’s Hospital Association (CHA; Lenexa, Kansas) and participating hospitals. Due to incomplete data through the study period and data quality issues, six hospitals were excluded.

Inclusion Criteria

Children 1-18 years of age who were discharged between July 1, 2007 and June 30, 2015 were included if they had a NI diagnosis,¹ a principal diagnosis indicative of aspiration pneumonia (507.x),^3,13,14 and received antibiotics in the first two calendar days of admission. NI was determined using previously defined International Classification of Diseases, Ninth Revision-Clinical Modification (ICD-9-CM) diagnosis codes.¹ We only included children who received antibiotics in the first two calendar days of admission to minimize the likelihood of including children admitted for other reasons who acquired aspiration pneumonia after hospitalization. For children with multiple hospitalizations, one admission was randomly selected for inclusion to minimize weighting results toward repeat visits.

Exclusion Criteria

Children transferred from another hospital were excluded as records from their initial presentation, including treatment and outcomes, were not available. We also excluded children with tracheostomy^15,16 or chronic ventilator dependence,¹⁷ those with a diagnosis of human immunodeficiency virus or tuberculosis, and children who received chemotherapy during hospitalization given expected differences in etiology, treatment, and outcomes.¹⁸

Exposure

The primary exposure was antibiotic therapy received in the first two days of admission. Antibiotics were classified by their antimicrobial spectra of activity as defined by The Sanford Guide to Antimicrobial Therapy¹⁹ against the most commonly recognized pathogens of aspiration pneumonia: anaerobes, Gram-negatives, and P. aeruginosa (Appendix Table 1).^10,20 For example, penicillin G and clindamycin were among the antibiotics classified as providing anaerobic coverage alone, whereas ceftriaxone was classified as providing Gram-negative coverage alone and ampicillin-sulbactam or as combination therapy with clindamycin and ceftriaxone were classified as providing anaerobic and Gram-negative coverage. Piperacillin-tazobactam and meropenem were classified as providing anaerobic, Gram-negative, and P. aeruginosa coverage. We excluded antibiotics that do not provide coverage against anaerobes, Gram-negative, or P. aeruginosa (eg, ampicillin, azithromycin) or that provide coverage against Gram-negative and P. aeruginosa, but not anaerobes (eg, cefepime, tobramycin), as these therapies were prescribed for <5% of the cohort. We chose not to examine the coverage for Streptococcus pneumonia or Staphylococcus aureus as antibiotics included in this analysis covered these bacteria for 99.9% of our cohort.

Outcomes included acute respiratory failure during hospitalization, intensive care unit (ICU) transfer, and hospital length of stay (LOS). Acute respiratory failure during hospitalization was defined as the presence of Clinical Transaction Classification (CTC) or ICD-9 procedure code for noninvasive or invasive mechanical ventilation on day two or later of hospitalization, with or without the need for respiratory support on day 0 or day 1 (Appendix Table 2). Given the variability in hospital policies that may drive ICU admission criteria for complex patients, our outcome of ICU transfer was defined as the requirement for ICU level care on day two or later of hospitalization without ICU admission. Acute respiratory failure and ICU care occurring within the first two hospital days were not classified as outcomes because these early events likely reflect illness severity at presentation rather than outcomes attributable to treatment failure; these were included as markers of severity in the models.

Patient Demographics and Clinical Characteristics

Demographic and clinical characteristics that might influence antibiotic choice and/or hospital outcomes were assessed. Clinical characteristics included complex chronic conditions,^21-23 medical technology assistance,²⁴ performance of diagnostic testing, and markers of severe illness on presentation. Diagnostic testing included bacterial cultures (blood, respiratory, urine) and chest radiograph performance in the first two days of hospitalization. Results of diagnostic testing are not available in the PHIS. Illness severity on presentation included acute respiratory failure, pleural drainage, receipt of vasoactive agents, and transfusion of blood products in the first two days of hospitalization (Appendix Table 2).^17,25,26

Continuous data were described with median and interquartile ranges (IQR) due to nonnormal distribution. Categorical data were described with frequencies and percentages. Patient demographics, clinical characteristics, and hospital outcomes were stratified by empiric antimicrobial coverage and compared using chi-square and Kruskal–Wallis tests as appropriate.

Generalized linear mixed-effects models with random hospital intercepts were derived to assess the independent effect of antimicrobial spectra of activity on outcomes of acute respiratory failure, ICU transfer, and LOS while adjusting for important differences in demographic and clinical characteristics. LOS had a nonnormal distribution. Thus, we used an exponential distribution. Covariates were chosen a priori given the clinical and biological relevance to exposure and outcomes—age, presence of complex chronic condition diagnoses, the number of complex chronic conditions, technology dependence, the performance of diagnostic tests on presentation, and illness severity on presentation. ICU admission was included as a covariate in acute respiratory failure and LOS outcome models. The results of the model for acute respiratory failure and ICU transfer are presented as adjusted odds ratios (OR) with a 95% CI. LOS results are presented as adjusted rate ratios (RR) with 95% CI.

All analyses were performed with SAS 9.3 (SAS Institute, Cary, North Carolina). P values <.05 were considered statistically significant. Cincinnati Children’s Hospital Medical Center Institutional Review Board considered this deidentified dataset study as not human subjects research.

Study Cohort

At the 44 hospitals included, 4,812 children with NI hospitalized with the diagnosis of aspiration pneumonia met the eligibility criteria. However, 79 received antibiotics with the spectra of activity not examined, leaving 4,733 children in our final analysis (Appendix Figure). Demographic and clinical characteristics of the study cohort are shown in Table 1. Median age was five years (interquartile range [IQR]: 2-11 years). Most subjects were male (53.9%), non-Hispanic white (47.9%), and publicly insured (63.6%). There was a slight variation in the distribution of admissions across seasons (spring 31.6%, summer 19.2%, fall 21.3%, and winter 27.9%). One-third of children had four or more comorbid CCCs (complex chronic conditions; 34.2%). The three most common nonneurologic CCC diagnosis categories were gastrointestinal (63.1%), congenital and/or genetic defects (36.9%), and respiratory (8.9%). Assistance with medical technologies was also common (82%)—particularly gastrointestinal (63.1%) and neurologic/neuromuscular (9.8%) technologies. The vast majority of children (92.5%) had either a chest radiograph (90.5%), respiratory viral study (33.7%), or respiratory culture (10.0%) obtained on presentation. A minority required noninvasive or invasive respiratory support (25.4%), vasoactive agents (8.9%), blood products (1.2%), or pleural drainage (0.3%) in the first two hospital days.

Spectrum of Antimicrobial Coverage

Most children (57.9%) received anaerobic and Gram-negative coverage; 16.2% received anaerobic, Gram-negative and P. aeruginosa coverage; 15.3% received anaerobic coverage alone; and 10.6% received Gram-negative coverage alone. Empiric antimicrobial coverage varied substantially across hospitals: anaerobic coverage was prescribed for 0%-44% of patients; Gram-negative coverage was prescribed for 3%-26% of patients; anaerobic and Gram-negative coverage was prescribed for 25%-90% of patients; and anaerobic, Gram-negative, and P. aeruginosa coverage was prescribed for 0%-65% of patients (Figure 1).

Outcomes

Acute Respiratory Failure

One-quarter (25.4%) of patients had acute respiratory failure on presentation; 22.5% required respiratory support (continued from presentation or were new) on day two or later of hospitalization (Table 2). In the adjusted analysis, children receiving Gram-negative coverage alone had two-fold greater odds (OR 2.15, 95% CI: 1.41-3.27) and children receiving anaerobic and Gram-negative coverage had 1.6-fold greater odds (OR 1.65, 95% CI: 1.19-2.28), of respiratory failure during hospitalization compared with those receiving anaerobic coverage alone (Figure 2). Odds of respiratory failure during hospitalization did not significantly differ for children receiving anaerobic, Gram-negative, and P. aeruginosa coverage compared with those receiving anaerobic coverage alone.

ICU Transfer

Nearly thirty percent (29.0%) of children required ICU admission, with an additional 3.8% requiring ICU transfer following admission (Table 2). In the multivariable analysis, the odds of an ICU transfer were greater for children receiving Gram-negative coverage alone (OR 1.80, 95% CI: 1.03-3.14) compared with those receiving anaerobic coverage alone. There was no statistical difference in ICU transfer for those receiving anaerobic and Gram-negative coverage (with or without P. aeruginosa coverage) compared with those receiving anaerobic coverage alone (Figure 2).

Length of Stay

Median hospital LOS for the total cohort was five days (IQR: 3-9 days; Table 2). In the multivariable analysis, children receiving Gram-negative coverage alone had a longer LOS (RR 1.28; 95% CI: 1.16-1.41) compared with those receiving anaerobic coverage alone, whereas children receiving anaerobic, Gram-negative, and P. aeruginosa coverage had a shorter LOS (RR 0.83; 95% CI: 0.76-0.90) than those receiving anaerobic coverage alone (Figure 2). There was no statistical difference in the LOS between children receiving anaerobic and Gram-negative coverage and those receiving anaerobic coverage alone.

In this multicenter study of children with NI hospitalized with aspiration pneumonia, we found substantial variation in empiric antimicrobial coverage for children with aspiration pneumonia. When comparing outcomes across groups, children who received anaerobic and Gram-negative coverage had outcomes similar to children who received anaerobic therapy alone. However, children who did not receive anaerobic coverage (ie, Gram-negative coverage alone) had worse outcomes, most notably a greater than two-fold increase in the odds of experiencing acute respiratory failure during hospitalization when compared with children receiving anaerobic therapy. These findings support prior literature that has highlighted the importance of anaerobic therapy in the treatment of aspiration pneumonia. The benefit of antibiotics targeting Gram-negative organisms, in addition to anaerobes, remains uncertain.

The variability in empiric antimicrobial coverage likely reflects the paucity of available information on oral and/or enteric bacteria required to identify them as causative organisms in aspiration pneumonia. In part, this problem is due to the difficulty in obtaining adequate sputum for culture from pediatric patients.²⁷ While it may be more feasible to obtain tracheal aspirates for respiratory culture in children with a tracheostomy, interpretation of culture results remains challenging because the lower airways of children with tracheostomy are commonly colonized with bacterial pathogens.²⁸ Thus, physicians are often left to choose empiric antimicrobial coverage with inadequate supporting evidence.²⁹ Although the polymicrobial nature of aspiration pneumonia is well recognized in adult and pediatric literature,^10,30 it is less clear which organisms are of pathological significance and require treatment.

The treatment standard for aspiration pneumonia has long included anaerobic therapy.²⁹ The worse outcomes of children not receiving anaerobic therapy (ie, Gram-negative coverage alone) compared with children who received anaerobic therapy support the continued importance of anaerobic therapy in the treatment of aspiration pneumonia for hospitalized children with NI. The role of antibiotics covering Gram-negative organisms is less clear. Recent studies suggest the role of anaerobes is overemphasized in the etiology and treatment of aspiration pneumonia.^10,29,31-38 Multiple studies on aspiration pneumonia bacteriology in hospitalized adults have demonstrated a predominance of Gram-negative organisms (ranging from 37%-71% of isolates identified on respiratory culture) and a relative scarcity of anaerobes (ranging from 0%-16% of isolates).^31-37 A prospective study of 50 children hospitalized with clinical and radiographic evidence of pneumonia with known aspiration risk (eg, neuromuscular disease or dysphagia) found that ~80% of 163 bacterial isolates were Gram-negative.³⁸ However, this study included repeat cultures from the same children, and thus, may overestimate the prevalence of Gram-negative organisms. In our study, children who received both anaerobic and Gram-negative therapy had no differences in ICU transfer or LOS but did experience higher odds of acute respiratory failure. As these results may be due to unmeasured confounding, future studies should further explore the necessity of Gram-negative coverage in addition to anaerobic coverage in this population.

While these recent studies may seem to suggest that anaerobic coverage is not necessary for aspiration pneumonia, there are important limitations worth noting. First, these studies used a variety of sampling techniques. While organisms grown from samples obtained via bronchoalveolar lavage^31-34,36 are likely pathogenic, those grown from tracheal or oral samples obtained via percutaneous transtracheal aspiration,³⁴ a protected specimen brush,^34,36,37 or expectorated sputum^35,38 may not represent lower airway organisms. Second, anaerobic cultures were not obtained in all studies.^31,34,38 Anaerobic organisms are difficult to isolate using traditional clinical specimen collection techniques and aerobic culture media.¹⁸ Furthermore, anaerobes are not easily recovered from lung infections after the receipt of antibiotic therapy.³⁹ Details regarding pretreatment, which are largely lacking from these studies, are necessary to interpret the relative scarcity of anaerobes on respiratory culture. Finally, caution should be taken when extrapolating the results of studies focused on the etiology and treatment of aspiration pneumonia in elderly adults to children. Our results, particularly in the context of the limitation of these more recent studies, suggest that the role of anaerobes has been underestimated.

Recent studies examining populations of children with cerebral palsy and/or tracheostomy have emphasized the high rates of carriage and infection rates with Gram-negative and drug-resistant bacteria; in particular, P. aeruginosa accounts for 50%-72% of pathogenic bacteria.^11,12,38,40These studies note the generally poor outcomes of children with P. aeruginosa—including multiple and longer hospitalizations, frequent readmissions, and the increased severity of pneumonia, including the need for ICU admission, pleural effusions, the need for intubation, and mortality.^{11,12,38,40,41} In our study, nearly 35% of children who received anaerobic, Gram-negative, and P. aeruginosa coverage experienced acute respiratory failure during hospitalization compared with 20% of children who received other therapies. While these results might seem to suggest that broader spectrum therapy is harmful, they must be interpreted in the context of important population differences; children who received a combination of anaerobic, Gram-negative, and P. aeruginosa coverage had greater medical complexity and greater severity of illness on presentation. Such factors may provide the reason for the appropriate prescription of antipseudomonal antibiotics (eg, history of tracheostomy colonization or infection, long-term care facility resident).⁴² When we controlled for population differences, children who received antipseudomonal therapy had a significantly shorter LOS and no differences in outcomes of acute respiratory failure or ICU transfer compared with those receiving anaerobic therapy alone. This result suggests that worse outcomes were associated with antipseudomonal therapy on unadjusted analyses resulting from underlying medical complexity and illness severity rather than from colonization or infection with P. aeruginosa.

Our multicenter observational study has several limitations. We used diagnosis codes to identify patients with aspiration pneumonia. As validated clinical criteria for the diagnosis of aspiration pneumonia do not exist, clinicians may assign a diagnosis of and treatment for aspiration pneumonia by subjective suspicion based on a child’s severe NI or illness severity on presentation leading to selection bias. Although administrative data are not able to verify pneumonia type with absolute certainty, we previously demonstrated that the differences in the outcomes of children with aspiration and nonaspiration pneumonia diagnosis codes persist after accounting for the complexity that might influence the diagnosis.³It is also possible that the diagnosis of aspiration pneumonia was not made upon admission for a subset of patients leading to misclassification of exposure. Some children may have had aspiration pneumonia on admission but were not assigned that diagnosis or treated for presumed aspiration pneumonia until later in the hospital course as they demonstrated treatment failure or clinical worsening. It is also possible that some children had an aspiration event during hospitalization that developed into aspiration pneumonia. We attempted to adjust for medical complexity and illness severity through multivariable adjustment based on the diagnosis and procedure codes, as well as the laboratory testing performed. However, unmeasured or residual confounding may remain as administrative data are not equipped to distinguish detailed functional status (eg, ability to cough, chest wall strength) or illness severity (eg, respiratory distress) that might influence antibiotic selection and/or outcomes.

Frthermore, we were unable to account for laboratory, microbiology, or radiology test results, and other management practices (eg, frequency of airway clearance, previous antimicrobial therapy) that may influence outcomes. Future studies should certainly include an examination of the concordance of the antibiotics prescribed with causative organisms, as this undoubtedly affects patient outcomes. Other outcomes are important to examine (eg, time to return to respiratory baseline), but we were unable to do so, given the lack of clinical detail in our database. We randomly selected a single hospitalization for children with multiple admissions; alternative methods could have different results. Although children with NI predominately use children’s hospitals,¹ results may not be generalizable.

These findings support prior literature that has highlighted the important role anaerobic therapy plays in the treatment of aspiration pneumonia in children with NI. In light of the limitations of our study design, we believe that rigorous clinical trials comparing anaerobic with anaerobic and Gram-negative therapy are an important and necessary next step to determine the optimal treatment for aspiration pneumonia in this population.

Disclosures

The authors do not have any financial relationships relevant to this article to disclose.

Funding

Dr. Thomson was supported by the Agency for Healthcare Research and Quality (AHRQ) under award number K08HS025138. Dr. Ambroggio was supported by the National Institute for Allergy and Infectious Diseases (NIAID) under award number K01AI125413. The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ or NIAID.

Neurologic impairment (NI) encompasses static and progressive diseases of the central and/or peripheral nervous systems that result in functional and intellectual impairments.¹ While a variety of neurologic diseases are responsible for NI (eg, hypoxic-ischemic encephalopathy, muscular dystrophy), consequences of these diseases extend beyond neurologic manifestations.¹ These children are at an increased risk for aspiration of oral and gastric contents given their common comorbidities of dysphagia, gastroesophageal reflux, impaired cough, and respiratory muscle weakness.² While aspiration may manifest as a self-resolving pneumonitis, the presence of oral or enteric bacteria in aspirated material may result in the development of bacterial pneumonia. Children with NI hospitalized with aspiration pneumonia have higher complication rates, longer and costlier hospitalizations, and higher readmission rates when compared with children with nonaspiration pneumonia.³

While pediatric aspiration pneumonia is commonly attributed to anaerobic bacteria, this is largely based on extrapolation from epidemiologic studies that were conducted in past decades.^4-8 A single randomized controlled trial found that penicillin and clindamycin, antimicrobials with similar antimicrobial activity against anaerobes, to be equally effective.⁹ However, the recent literature emphasizes the polymicrobial nature of aspiration pneumonia in adults, with the common isolation of Gram-negative enteric bacteria.¹⁰ Further, while Pseudomonas aeruginosa is often identified in respiratory cultures from children with NI and chronic respiratory insufficiency,^11,12 the significance of P. aeruginosa in lower airways remains unclear.

We designed this study to compare hospital outcomes associated with the most commonly prescribed empiric antimicrobial therapies for aspiration pneumonia in children with NI.

Study Design and Data Source

This multicenter, retrospective cohort study used the Pediatric Health Information System (PHIS) database. PHIS, an administrative database of 50 not-for-profit tertiary care pediatric hospitals, contains data regarding patient demographics, diagnoses and procedures, and daily billed resource utilization, including laboratory and imaging studies. Data quality and reliability are assured through the Children’s Hospital Association (CHA; Lenexa, Kansas) and participating hospitals. Due to incomplete data through the study period and data quality issues, six hospitals were excluded.

Inclusion Criteria

Children 1-18 years of age who were discharged between July 1, 2007 and June 30, 2015 were included if they had a NI diagnosis,¹ a principal diagnosis indicative of aspiration pneumonia (507.x),^3,13,14 and received antibiotics in the first two calendar days of admission. NI was determined using previously defined International Classification of Diseases, Ninth Revision-Clinical Modification (ICD-9-CM) diagnosis codes.¹ We only included children who received antibiotics in the first two calendar days of admission to minimize the likelihood of including children admitted for other reasons who acquired aspiration pneumonia after hospitalization. For children with multiple hospitalizations, one admission was randomly selected for inclusion to minimize weighting results toward repeat visits.

Exclusion Criteria

Children transferred from another hospital were excluded as records from their initial presentation, including treatment and outcomes, were not available. We also excluded children with tracheostomy^15,16 or chronic ventilator dependence,¹⁷ those with a diagnosis of human immunodeficiency virus or tuberculosis, and children who received chemotherapy during hospitalization given expected differences in etiology, treatment, and outcomes.¹⁸

Exposure

The primary exposure was antibiotic therapy received in the first two days of admission. Antibiotics were classified by their antimicrobial spectra of activity as defined by The Sanford Guide to Antimicrobial Therapy¹⁹ against the most commonly recognized pathogens of aspiration pneumonia: anaerobes, Gram-negatives, and P. aeruginosa (Appendix Table 1).^10,20 For example, penicillin G and clindamycin were among the antibiotics classified as providing anaerobic coverage alone, whereas ceftriaxone was classified as providing Gram-negative coverage alone and ampicillin-sulbactam or as combination therapy with clindamycin and ceftriaxone were classified as providing anaerobic and Gram-negative coverage. Piperacillin-tazobactam and meropenem were classified as providing anaerobic, Gram-negative, and P. aeruginosa coverage. We excluded antibiotics that do not provide coverage against anaerobes, Gram-negative, or P. aeruginosa (eg, ampicillin, azithromycin) or that provide coverage against Gram-negative and P. aeruginosa, but not anaerobes (eg, cefepime, tobramycin), as these therapies were prescribed for <5% of the cohort. We chose not to examine the coverage for Streptococcus pneumonia or Staphylococcus aureus as antibiotics included in this analysis covered these bacteria for 99.9% of our cohort.

Outcomes included acute respiratory failure during hospitalization, intensive care unit (ICU) transfer, and hospital length of stay (LOS). Acute respiratory failure during hospitalization was defined as the presence of Clinical Transaction Classification (CTC) or ICD-9 procedure code for noninvasive or invasive mechanical ventilation on day two or later of hospitalization, with or without the need for respiratory support on day 0 or day 1 (Appendix Table 2). Given the variability in hospital policies that may drive ICU admission criteria for complex patients, our outcome of ICU transfer was defined as the requirement for ICU level care on day two or later of hospitalization without ICU admission. Acute respiratory failure and ICU care occurring within the first two hospital days were not classified as outcomes because these early events likely reflect illness severity at presentation rather than outcomes attributable to treatment failure; these were included as markers of severity in the models.

Patient Demographics and Clinical Characteristics

Demographic and clinical characteristics that might influence antibiotic choice and/or hospital outcomes were assessed. Clinical characteristics included complex chronic conditions,^21-23 medical technology assistance,²⁴ performance of diagnostic testing, and markers of severe illness on presentation. Diagnostic testing included bacterial cultures (blood, respiratory, urine) and chest radiograph performance in the first two days of hospitalization. Results of diagnostic testing are not available in the PHIS. Illness severity on presentation included acute respiratory failure, pleural drainage, receipt of vasoactive agents, and transfusion of blood products in the first two days of hospitalization (Appendix Table 2).^17,25,26

Continuous data were described with median and interquartile ranges (IQR) due to nonnormal distribution. Categorical data were described with frequencies and percentages. Patient demographics, clinical characteristics, and hospital outcomes were stratified by empiric antimicrobial coverage and compared using chi-square and Kruskal–Wallis tests as appropriate.

Generalized linear mixed-effects models with random hospital intercepts were derived to assess the independent effect of antimicrobial spectra of activity on outcomes of acute respiratory failure, ICU transfer, and LOS while adjusting for important differences in demographic and clinical characteristics. LOS had a nonnormal distribution. Thus, we used an exponential distribution. Covariates were chosen a priori given the clinical and biological relevance to exposure and outcomes—age, presence of complex chronic condition diagnoses, the number of complex chronic conditions, technology dependence, the performance of diagnostic tests on presentation, and illness severity on presentation. ICU admission was included as a covariate in acute respiratory failure and LOS outcome models. The results of the model for acute respiratory failure and ICU transfer are presented as adjusted odds ratios (OR) with a 95% CI. LOS results are presented as adjusted rate ratios (RR) with 95% CI.

All analyses were performed with SAS 9.3 (SAS Institute, Cary, North Carolina). P values <.05 were considered statistically significant. Cincinnati Children’s Hospital Medical Center Institutional Review Board considered this deidentified dataset study as not human subjects research.

Study Cohort

At the 44 hospitals included, 4,812 children with NI hospitalized with the diagnosis of aspiration pneumonia met the eligibility criteria. However, 79 received antibiotics with the spectra of activity not examined, leaving 4,733 children in our final analysis (Appendix Figure). Demographic and clinical characteristics of the study cohort are shown in Table 1. Median age was five years (interquartile range [IQR]: 2-11 years). Most subjects were male (53.9%), non-Hispanic white (47.9%), and publicly insured (63.6%). There was a slight variation in the distribution of admissions across seasons (spring 31.6%, summer 19.2%, fall 21.3%, and winter 27.9%). One-third of children had four or more comorbid CCCs (complex chronic conditions; 34.2%). The three most common nonneurologic CCC diagnosis categories were gastrointestinal (63.1%), congenital and/or genetic defects (36.9%), and respiratory (8.9%). Assistance with medical technologies was also common (82%)—particularly gastrointestinal (63.1%) and neurologic/neuromuscular (9.8%) technologies. The vast majority of children (92.5%) had either a chest radiograph (90.5%), respiratory viral study (33.7%), or respiratory culture (10.0%) obtained on presentation. A minority required noninvasive or invasive respiratory support (25.4%), vasoactive agents (8.9%), blood products (1.2%), or pleural drainage (0.3%) in the first two hospital days.

Spectrum of Antimicrobial Coverage

Most children (57.9%) received anaerobic and Gram-negative coverage; 16.2% received anaerobic, Gram-negative and P. aeruginosa coverage; 15.3% received anaerobic coverage alone; and 10.6% received Gram-negative coverage alone. Empiric antimicrobial coverage varied substantially across hospitals: anaerobic coverage was prescribed for 0%-44% of patients; Gram-negative coverage was prescribed for 3%-26% of patients; anaerobic and Gram-negative coverage was prescribed for 25%-90% of patients; and anaerobic, Gram-negative, and P. aeruginosa coverage was prescribed for 0%-65% of patients (Figure 1).

Outcomes

Acute Respiratory Failure

One-quarter (25.4%) of patients had acute respiratory failure on presentation; 22.5% required respiratory support (continued from presentation or were new) on day two or later of hospitalization (Table 2). In the adjusted analysis, children receiving Gram-negative coverage alone had two-fold greater odds (OR 2.15, 95% CI: 1.41-3.27) and children receiving anaerobic and Gram-negative coverage had 1.6-fold greater odds (OR 1.65, 95% CI: 1.19-2.28), of respiratory failure during hospitalization compared with those receiving anaerobic coverage alone (Figure 2). Odds of respiratory failure during hospitalization did not significantly differ for children receiving anaerobic, Gram-negative, and P. aeruginosa coverage compared with those receiving anaerobic coverage alone.

ICU Transfer

Nearly thirty percent (29.0%) of children required ICU admission, with an additional 3.8% requiring ICU transfer following admission (Table 2). In the multivariable analysis, the odds of an ICU transfer were greater for children receiving Gram-negative coverage alone (OR 1.80, 95% CI: 1.03-3.14) compared with those receiving anaerobic coverage alone. There was no statistical difference in ICU transfer for those receiving anaerobic and Gram-negative coverage (with or without P. aeruginosa coverage) compared with those receiving anaerobic coverage alone (Figure 2).

Length of Stay

Median hospital LOS for the total cohort was five days (IQR: 3-9 days; Table 2). In the multivariable analysis, children receiving Gram-negative coverage alone had a longer LOS (RR 1.28; 95% CI: 1.16-1.41) compared with those receiving anaerobic coverage alone, whereas children receiving anaerobic, Gram-negative, and P. aeruginosa coverage had a shorter LOS (RR 0.83; 95% CI: 0.76-0.90) than those receiving anaerobic coverage alone (Figure 2). There was no statistical difference in the LOS between children receiving anaerobic and Gram-negative coverage and those receiving anaerobic coverage alone.

In this multicenter study of children with NI hospitalized with aspiration pneumonia, we found substantial variation in empiric antimicrobial coverage for children with aspiration pneumonia. When comparing outcomes across groups, children who received anaerobic and Gram-negative coverage had outcomes similar to children who received anaerobic therapy alone. However, children who did not receive anaerobic coverage (ie, Gram-negative coverage alone) had worse outcomes, most notably a greater than two-fold increase in the odds of experiencing acute respiratory failure during hospitalization when compared with children receiving anaerobic therapy. These findings support prior literature that has highlighted the importance of anaerobic therapy in the treatment of aspiration pneumonia. The benefit of antibiotics targeting Gram-negative organisms, in addition to anaerobes, remains uncertain.

The variability in empiric antimicrobial coverage likely reflects the paucity of available information on oral and/or enteric bacteria required to identify them as causative organisms in aspiration pneumonia. In part, this problem is due to the difficulty in obtaining adequate sputum for culture from pediatric patients.²⁷ While it may be more feasible to obtain tracheal aspirates for respiratory culture in children with a tracheostomy, interpretation of culture results remains challenging because the lower airways of children with tracheostomy are commonly colonized with bacterial pathogens.²⁸ Thus, physicians are often left to choose empiric antimicrobial coverage with inadequate supporting evidence.²⁹ Although the polymicrobial nature of aspiration pneumonia is well recognized in adult and pediatric literature,^10,30 it is less clear which organisms are of pathological significance and require treatment.

The treatment standard for aspiration pneumonia has long included anaerobic therapy.²⁹ The worse outcomes of children not receiving anaerobic therapy (ie, Gram-negative coverage alone) compared with children who received anaerobic therapy support the continued importance of anaerobic therapy in the treatment of aspiration pneumonia for hospitalized children with NI. The role of antibiotics covering Gram-negative organisms is less clear. Recent studies suggest the role of anaerobes is overemphasized in the etiology and treatment of aspiration pneumonia.^10,29,31-38 Multiple studies on aspiration pneumonia bacteriology in hospitalized adults have demonstrated a predominance of Gram-negative organisms (ranging from 37%-71% of isolates identified on respiratory culture) and a relative scarcity of anaerobes (ranging from 0%-16% of isolates).^31-37 A prospective study of 50 children hospitalized with clinical and radiographic evidence of pneumonia with known aspiration risk (eg, neuromuscular disease or dysphagia) found that ~80% of 163 bacterial isolates were Gram-negative.³⁸ However, this study included repeat cultures from the same children, and thus, may overestimate the prevalence of Gram-negative organisms. In our study, children who received both anaerobic and Gram-negative therapy had no differences in ICU transfer or LOS but did experience higher odds of acute respiratory failure. As these results may be due to unmeasured confounding, future studies should further explore the necessity of Gram-negative coverage in addition to anaerobic coverage in this population.

While these recent studies may seem to suggest that anaerobic coverage is not necessary for aspiration pneumonia, there are important limitations worth noting. First, these studies used a variety of sampling techniques. While organisms grown from samples obtained via bronchoalveolar lavage^31-34,36 are likely pathogenic, those grown from tracheal or oral samples obtained via percutaneous transtracheal aspiration,³⁴ a protected specimen brush,^34,36,37 or expectorated sputum^35,38 may not represent lower airway organisms. Second, anaerobic cultures were not obtained in all studies.^31,34,38 Anaerobic organisms are difficult to isolate using traditional clinical specimen collection techniques and aerobic culture media.¹⁸ Furthermore, anaerobes are not easily recovered from lung infections after the receipt of antibiotic therapy.³⁹ Details regarding pretreatment, which are largely lacking from these studies, are necessary to interpret the relative scarcity of anaerobes on respiratory culture. Finally, caution should be taken when extrapolating the results of studies focused on the etiology and treatment of aspiration pneumonia in elderly adults to children. Our results, particularly in the context of the limitation of these more recent studies, suggest that the role of anaerobes has been underestimated.

Recent studies examining populations of children with cerebral palsy and/or tracheostomy have emphasized the high rates of carriage and infection rates with Gram-negative and drug-resistant bacteria; in particular, P. aeruginosa accounts for 50%-72% of pathogenic bacteria.^11,12,38,40These studies note the generally poor outcomes of children with P. aeruginosa—including multiple and longer hospitalizations, frequent readmissions, and the increased severity of pneumonia, including the need for ICU admission, pleural effusions, the need for intubation, and mortality.^{11,12,38,40,41} In our study, nearly 35% of children who received anaerobic, Gram-negative, and P. aeruginosa coverage experienced acute respiratory failure during hospitalization compared with 20% of children who received other therapies. While these results might seem to suggest that broader spectrum therapy is harmful, they must be interpreted in the context of important population differences; children who received a combination of anaerobic, Gram-negative, and P. aeruginosa coverage had greater medical complexity and greater severity of illness on presentation. Such factors may provide the reason for the appropriate prescription of antipseudomonal antibiotics (eg, history of tracheostomy colonization or infection, long-term care facility resident).⁴² When we controlled for population differences, children who received antipseudomonal therapy had a significantly shorter LOS and no differences in outcomes of acute respiratory failure or ICU transfer compared with those receiving anaerobic therapy alone. This result suggests that worse outcomes were associated with antipseudomonal therapy on unadjusted analyses resulting from underlying medical complexity and illness severity rather than from colonization or infection with P. aeruginosa.

Our multicenter observational study has several limitations. We used diagnosis codes to identify patients with aspiration pneumonia. As validated clinical criteria for the diagnosis of aspiration pneumonia do not exist, clinicians may assign a diagnosis of and treatment for aspiration pneumonia by subjective suspicion based on a child’s severe NI or illness severity on presentation leading to selection bias. Although administrative data are not able to verify pneumonia type with absolute certainty, we previously demonstrated that the differences in the outcomes of children with aspiration and nonaspiration pneumonia diagnosis codes persist after accounting for the complexity that might influence the diagnosis.³It is also possible that the diagnosis of aspiration pneumonia was not made upon admission for a subset of patients leading to misclassification of exposure. Some children may have had aspiration pneumonia on admission but were not assigned that diagnosis or treated for presumed aspiration pneumonia until later in the hospital course as they demonstrated treatment failure or clinical worsening. It is also possible that some children had an aspiration event during hospitalization that developed into aspiration pneumonia. We attempted to adjust for medical complexity and illness severity through multivariable adjustment based on the diagnosis and procedure codes, as well as the laboratory testing performed. However, unmeasured or residual confounding may remain as administrative data are not equipped to distinguish detailed functional status (eg, ability to cough, chest wall strength) or illness severity (eg, respiratory distress) that might influence antibiotic selection and/or outcomes.

Frthermore, we were unable to account for laboratory, microbiology, or radiology test results, and other management practices (eg, frequency of airway clearance, previous antimicrobial therapy) that may influence outcomes. Future studies should certainly include an examination of the concordance of the antibiotics prescribed with causative organisms, as this undoubtedly affects patient outcomes. Other outcomes are important to examine (eg, time to return to respiratory baseline), but we were unable to do so, given the lack of clinical detail in our database. We randomly selected a single hospitalization for children with multiple admissions; alternative methods could have different results. Although children with NI predominately use children’s hospitals,¹ results may not be generalizable.

These findings support prior literature that has highlighted the important role anaerobic therapy plays in the treatment of aspiration pneumonia in children with NI. In light of the limitations of our study design, we believe that rigorous clinical trials comparing anaerobic with anaerobic and Gram-negative therapy are an important and necessary next step to determine the optimal treatment for aspiration pneumonia in this population.

Disclosures

The authors do not have any financial relationships relevant to this article to disclose.

Funding

Dr. Thomson was supported by the Agency for Healthcare Research and Quality (AHRQ) under award number K08HS025138. Dr. Ambroggio was supported by the National Institute for Allergy and Infectious Diseases (NIAID) under award number K01AI125413. The content is solely the responsibility of the authors and does not necessarily represent the official views of the AHRQ or NIAID.