In 2014, the United States spent $3 trillion on healthcare; hospitalization consumed 32% of these expenditures.¹ Today, Medicare patients account for over 50% of hospital days and over 30% of all hospital discharges in the United States.² Despite this staggering financial burden, hospitalization of older adults often results in poor patient outcomes.^3-6 The exponential growth of the hospitalist movement, from 350 hospitalists nationwide in 1995 to over 44,000 in 2014, has become the key strategy for providing care to hospitalized geriatric patients.^7-10 Most of these hospitalists have not received geriatric training.^11-15

There is growing evidence that a geriatric approach, emphasizing multidisciplinary management of the complex needs of older patients, leads to improved outcomes. Geriatric Evaluation and Management Units (GEMUs), such as Acute Care for Elderly (ACE) models, have demonstrated significant decreases in functional decline, institutionalization, and death in randomized controlled trials.^16,17 Multidisciplinary, nonunit based efforts, such as the mobile acute care of elderly (MACE), proactive consultation models (Sennour/Counsell), and the Hospital Elder Life Program (HELP), have demonstrated success in preventing adverse events and decreasing length of stay (LOS).^17-20

However, these models have not been systematically implemented due to challenges in generalizability and replicability in diverse settings. To address this concern, an alternative approach must be developed to widely “generalize” geriatric expertise throughout hospitals, regardless of their location, size, and resources. This initiative will require systematic integration of evidence-based decision support tools for the standardization of clinical management in hospitalized older adults.²¹

The 1998 Assessing Care of Vulnerable Elders (ACOVE) project developed a standardized tool to measure and evaluate the quality of care by using a comprehensive set of quality indicators (QIs) to improve the care of “vulnerable elders” (VEs) at a high risk for functional and cognitive decline and death.^22-24 The latest systematic review concludes that, although many studies have used ACOVE as an assessment tool of quality, there has been a dearth of studies investigating the ACOVE QIs as an intervention to improve patient care.²⁵

Our study investigated the role of ACOVE as an intervention by using the QIs as a standardized checklist in the acute care setting. We selected the 4 most commonly encountered QIs in the hospital setting, namely venous thrombosis prophylaxis (VTE), indwelling bladder catheter, mobilization, and delirium evaluation, in order to test the feasibility and impact of systematically implementing these ACOVE QIs as a therapeutic intervention for all hospitalized older adults.

This study (IRB #13-644B) was conducted using a prospective intervention with a nonequivalent control group design comprised of retrospective chart data from May 1, 2014, to June 30, 2015. Process and outcome variables were extracted from electronic medical records ([EMR], Sunrise Clinical Manager [SCM]) of 2,396 patients, with 530 patients in the intervention unit and 1,866 on the control units, at a large academic tertiary center operating in the greater New York metropolitan area. Our study investigated the role of ACOVE as an intervention to improve patient care by using selected QIs as a standardized checklist tool in the acute care setting. Of the original 30 hospital-specific QIs, our study focused on the care of older adults admitted to the medicine service.²⁶ We selected commonly encountered QIs, with the objective of testing the feasibility and impact of implementing the ACOVE QIs as an intervention to improve care of hospitalized older adults. This intervention consisted of applying the checklist tool, constructed with 4 selected ACOVE QIs and administered daily during interdisciplinary rounds, namely: 2 general “medical” indicators, VTE prophylaxis and indwelling bladder catheters, and 2 “geriatric”-focused indicators, mobilization and delirium evaluation.

Subject matter experts (hospitalists, geriatricians, researchers, administrators, and nurses) reviewed the ACOVE QIs and agreed upon the adaptation of the QIs from a quality measure assessment into a feasible and acceptable intervention checklist tool (Table 1). The checklist was reviewed during daily interdisciplinary rounds for all patients 75 years and older. While ACOVE defined vulnerable elders by using the Vulnerable Elder Screen (VES), we wanted to apply this intervention more broadly to all hospitalized older adults who are most at risk for poor outcomes.²⁷ Patients admitted to the intensive care unit, inpatient psychiatry, inpatient leukemia/lymphoma, and surgical services were excluded.

Daily interdisciplinary rounds are held on every one of the five 40-bed medical units; they last approximately 1 hour, and consist of a lead hospitalist, nurse manager, nurse practitioners, case managers, and the nursing staff. During interdisciplinary rounds, nurses present the case to the team members who then discuss the care plan. These 5 medical units did not differ in terms of patient characteristics or staffing patterns; the intervention unit was chosen simply for logistical reasons, in that the principal investigator (PI) had been assigned to this unit prior to study start-up.

Prior to the intervention, LS held an education session for staff on the intervention unit staff (who participated on interdisciplinary rounds) to explain the concept of the ACOVE QI initiative and describe the four QIs selected for the study. Three subsequent educational sessions were held during the first week of the intervention, with new incoming staff receiving a brief individual educational session. The staff demonstrated significant knowledge improvement after session completion (pre/post mean score 70.6% vs 90.0%; P < .0001).

The Clinical Information System for the Health System EMR, The Eclipsys SCM, has alerts with different levels of severity from “soft” (user must acknowledge a recommendation) to “hard” (requires an action in order to proceed).

To measure compliance of the quality indicators, we collected the following variables:

QI 1: VTE prophylaxis

Through SCM, we collected type of VTE prophylaxis ordered (pharmacologic and/or mechanical) as well as start and stop dates for all agents. International normalized ratio levels were checked for patients receiving warfarin. Days of compliance were calculated.

QI 2: Indwelling Bladder Catheters

SCM data were collected on catheter entry and discontinuation dates, the presence of an indication, and order renewal for bladder catheter at least every 3 days.

QI 3: Mobilization

Ambulation status prior to admission was extracted from nursing documentation completed on admission to the medical ward. Patients documented as bedfast were categorized as nonambulatory prior to admission. Nursing documentation of activity level and amount of feet ambulated per nursing shift were collected. In addition, hospital day of physical therapy (PT) order and hospital days with PT performed were charted. Compliance with QI 3 in patients documented as ambulatory prior to hospital admission was recorded as present if there was a PT order within 48 hours of admission.

QI 4: Delirium Evaluation

During daily rounds, the hospitalist (PI) questioned nurses about delirium evaluation, using the first feature of the Confusion Assessment Method (CAM) as well as the “single question in delirium,” namely, “Is there evidence of an acute change in mental status from the patient’s baseline?” and “Do you think [name of patient] has been more confused lately?”^28,29 Because EMR does not contain a specified field for delirium screening and documentation, and patients are not routinely included in rounds, documentation with QI 4 was recorded using the “key words” method as described in the work by Puelle et al.³⁰ To extract SCM key words, nursing documentation of the “cognitive/perceptual/neurological exam” section of the EMR on admission and on all subsequent documentation (once per shift) was retrieved to identify acute changes in mental status (eg, “altered mental status, delirium/delirious, alert and oriented X 3, confused/confusion, disoriented, lethargy/lethargic”).³⁰ In addition, nurses were asked to activate an SCM parameter, “Acute Confusion” SCM parameter, in the nursing documentation section, which includes potential risk factors for confusion.

In addition to QI compliance, we collected LOS, discharge disposition, and 30-day readmission data.

Generalized linear mixed models (GLMM) for binary clustered (ie, hierarchical) data were used to estimate compliance rates (ie, nurse adherence) for each group (intervention group or control group) in the postintervention period, along with their corresponding 95% confidence intervals. GLMM was used to account for the hierarchical structure of the data: nursing units within a hospital. In order to calculate the Charlson Comorbidity Index, we extracted past medical history from the EMR.³¹

Subjects (N = 2,396) were included in the comparison of the intervention group vs control group for each of the following 4 ACOVE QI compliance measures: DVT, mobilization, bladder catheter, and delirium.

Of the 2,396 patient admissions, 530 were in the intervention unit and 1,866 were in the control unit. In the intervention group, the average age was 84.65 years, 75.58% were white and 47.21% were married. There was no difference in patient demographics between groups (Table 2).

QI 1: VTE Prophylaxis

Compliance with VTE prophylaxis was met in 78.3% of the intervention subjects and 76.5% of the controls (P < .4371) (Table 3). Of note, the rate of VTE prophylaxis was 57% in the intervention vs 39% in the control group (P < .0056), in the 554 patients for whom compliance was not met. Mechanical prophylaxis was used in 35.6% of intervention subjects vs 30.6 in the control (P = .048). Patients who received no form of prophylaxis were 0.5% in the intervention and 3% in the control (P = .027).

QI 2: Indwelling Bladder Catheters

Out of 2,396 subjects, 406 had an indwelling bladder catheter (16.9%). Compliance with the catheter was met in 72.2% of the intervention group vs 54.4% in the control group (P = .1061). An indication for indwelling bladder catheters was documented in 100% of the subjects. The average number of catheter days was 5.16 in the intervention vs 5.88 in the control (P < .2284). There was statistical significance in catheter compliance in the longer stay (>15 days) subjects, decreasing to 23.32% in the control group while staying constant in the intervention group 71.5% (P = .0006).

QI 3: Mobilization

Of the 2,396 patients, 1,991 (83.1%) were reported as ambulatory prior to admission. In the intervention vs control group, 74 (14%) vs 297 (15.7%), respectively, were nonambulatory. Overall compliance with Q3 was 62.9% in the intervention vs 48.2% in the control (P < .0001). More specifically, the average time to PT order in the intervention group was 1.83 days vs 2.22 days in the control group (P < .0051) and the time to PT evaluation was 2.14 days vs 2.42 days, respectively (P < .0108). In the intervention group, 84 patients (15.8%) did not have a PT consult vs 511 (27%) in the control group (P < .0001). The average times per subject in which the nurses documented the approximate number of feet ambulated was 6.48 in the intervention group vs 0.11 in the control group.

QI 4: Delirium Evaluation

In terms of nursing documentation indicating the presence of an acute confusional state, the intervention group had 148 out of 530 nursing notes (27.9%) vs 405 out of 1,866 in the control group (21.7%; P = .0027). However, utilization of the “acute confusion” parameter with documentation of a risk factor did not differ between the groups (5.8% in the intervention group vs 5.6% in the control group, P < .94).

LOS, Discharge Disposition, and 30-Day Readmissions

LOS did not differ between intervention and control groups (6.37 days vs 6.27 days, respectively), with a median of 5 days (P = .877). Discharge disposition in the 2 groups included the following: home/home with services (71.32% vs 68.7%), skilled nursing facility/assisted living/long-term care (24.34 versus 25.83), inpatient hospice/home hospice (2.64 vs 2.25), and expired (1.13 vs 1.77; P < .3282). In addition, 30-day readmissions did not differ (21% vs 20%, respectively, P = .41).

Our goal was to explore an evidence-based, standardized approach to improve the care of hospitalized older adults. This approach leverages existing automated EMR alert functions with an additional level of decision support for VEs, integrated into daily multidisciplinary rounds. The use of a daily checklist-based tool offers a cost-effective and practical pathway to distribute the burden of compliance responsibility amongst team members.

As we anticipated and similar to study findings in hospitalized medicine, geriatric trauma, and primary care, compliance with general care QIs was better than geriatric-focused QIs.^27,32 Wenger et al³³ demonstrated significant improvements with screening for falls and incontinence; however, screening for cognitive impairment did not improve in the outpatient setting by imbedding ACOVE QIs into routine physician practice.

Increased compliance with VTE prophylaxis and indwelling bladder catheters may be explained by national financial incentives for widespread implementation of EMR alert systems. Conversely, mobilization, delirium assessment, and management in hospitalized older adults don’t benefit from similar incentives.

VTE Prophylaxis

The American College of Chest Physicians (ACCP) supports the use of VTE prophylaxis, especially in hospitalized older adults with decreased mobility.³⁴ While greater adoption of EMR has already increased adherence, our intervention resulted in an even higher rate of compliance with the use of pharmacologic VTE prophylaxis.³⁵ In the future, validated scores for risk of thrombosis and bleeding may be integrated into our QI-based checklist.

Indwelling Bladder Catheters

The potential harms of catheters have been described for over 50 years, yet remain frequently used.^36,37 Previous studies have shown success in decreasing catheter days with computer-based and multidisciplinary protocols.^36-39

Our health system’s EMR has built-in “soft” and “hard” alerts for indwelling bladder catheters, so we did not expect intervention-associated changes in compliance.

Mobilization

Hospitalization in older adults frequently results in functional decline.^4,5,40 In response, the mobilization QI recommends an ambulation plan within 48 hours for those patients who were ambulatory prior to admission; it does not specifically define the components of the plan.²⁶ There are several multicomponent interventions that have demonstrated improvement in functional decline, yet they require skilled providers.^41,42 Our intervention implemented specific ambulation plan components: daily ambulation and documentation reminders and early PT evaluation.

While functional status measures have existed for decades, most are primarily geared to assess community-residing individuals and not designed to measure changes in function during hospitalization.^43,44 Furthermore, performance-based hospital measures are difficult to integrate into the daily nursing workflow as they are time consuming.^45,46 In practice, nurses routinely use free text to document functional status in the hospital setting, rendering comparative analysis problematic. Yet, we demonstrated that nurses were more engaged in reporting mobilization (increased documentation of ambulation distance and a decrease in time to PT). Future research should focus on the development of a standardized tool, integrated into the EMR, to accurately measure function in the acute care setting.

Delirium Evaluation

Delirium evaluation remains one of the most difficult clinical challenges for healthcare providers in hospitalized individuals, and our study reiterated these concerns. Previous research has consistently demonstrated that the diagnosis of delirium is missed by up to 75% of clinicians.^47,48 Indeed, our study, which exclusively examined nursing documentation of the delirium evaluation QI, found that both groups showed strikingly low compliance rates. This may have been due to the fact that we only evaluated nursing documentation of suspected or definite diagnosis of delirium and a documented attempt to attribute the altered mental state to a potential etiology.³¹ By utilizing the concept of “key words,” as developed by Puelle et al.³⁰, we were able to demonstrate a statistically significant improvement in nursing delirium documentation in the intervention group. This result should be interpreted with caution, as this approach is not validated. Furthermore, our operational definition of delirium compliance (ie, nurse documentation of delirium, requiring the launching of a separate parameter) may have been simply too cumbersome to readily integrate into the daily workflow. Future research should study the efficacy of a sensitive EMR-integrated screening tool that facilitates recognition, by all team members, of acute changes in cognition.

Although a number of QI improved for the intervention group, acute care utilization measures such as LOS, discharge disposition, and 30-day readmissions did not differ between groups. It may well be that improving quality for this very frail, vulnerable population may simply not result in decreased utilization. Our ability to further decrease LOS and readmission rates may be limited due to restriction of range in this complex patient population (eg, median LOS value of 5 days).

Limitations

Although our study had a large sample size, data were only collected from a single-center and thus require further exploration in different settings to ensure generalizability. In addition, QI observance was based on the medical record, which was problematic for some indicators, notably delirium identification. While prior literature highlights the difficulty in identifying delirium, especially during clinical practice without specialized training, our compliance was strikingly low.⁴⁷ While validated measures such as CAM may have been included as part of the assessment, there is currently no EMR documentation of such measures and therefore, these data could not be obtained.

In summary, our study demonstrates the successful integration of the established ACOVE QIs as an intervention, rather than as an assessment method, for improving care of hospitalized older patients. By utilizing a checklist-based tool at the bedside allows the multidisciplinary team to implement evidence-based practices with the ultimate goal of standardizing care, not only for VEs, but potentially for other high-risk populations with multimorbidity.⁴⁹ This innovative approach provides a much-needed direction to healthcare providers in the ever increasing stressful conditions of today’s acute care environment and for the ultimate benefit and safety of our older patients.

Disclosure

The authors declare no conflicts of interest. This study was supported by New York State Empire Clinical Research Investigators Program (ECRIP). The sponsor had no role in the conception, study design, data collection, data analysis, interpretation of data, manuscript preparation, or the decision to submit the manuscript for publication.

In 2014, the United States spent $3 trillion on healthcare; hospitalization consumed 32% of these expenditures.¹ Today, Medicare patients account for over 50% of hospital days and over 30% of all hospital discharges in the United States.² Despite this staggering financial burden, hospitalization of older adults often results in poor patient outcomes.^3-6 The exponential growth of the hospitalist movement, from 350 hospitalists nationwide in 1995 to over 44,000 in 2014, has become the key strategy for providing care to hospitalized geriatric patients.^7-10 Most of these hospitalists have not received geriatric training.^11-15

There is growing evidence that a geriatric approach, emphasizing multidisciplinary management of the complex needs of older patients, leads to improved outcomes. Geriatric Evaluation and Management Units (GEMUs), such as Acute Care for Elderly (ACE) models, have demonstrated significant decreases in functional decline, institutionalization, and death in randomized controlled trials.^16,17 Multidisciplinary, nonunit based efforts, such as the mobile acute care of elderly (MACE), proactive consultation models (Sennour/Counsell), and the Hospital Elder Life Program (HELP), have demonstrated success in preventing adverse events and decreasing length of stay (LOS).^17-20

However, these models have not been systematically implemented due to challenges in generalizability and replicability in diverse settings. To address this concern, an alternative approach must be developed to widely “generalize” geriatric expertise throughout hospitals, regardless of their location, size, and resources. This initiative will require systematic integration of evidence-based decision support tools for the standardization of clinical management in hospitalized older adults.²¹

The 1998 Assessing Care of Vulnerable Elders (ACOVE) project developed a standardized tool to measure and evaluate the quality of care by using a comprehensive set of quality indicators (QIs) to improve the care of “vulnerable elders” (VEs) at a high risk for functional and cognitive decline and death.^22-24 The latest systematic review concludes that, although many studies have used ACOVE as an assessment tool of quality, there has been a dearth of studies investigating the ACOVE QIs as an intervention to improve patient care.²⁵

Our study investigated the role of ACOVE as an intervention by using the QIs as a standardized checklist in the acute care setting. We selected the 4 most commonly encountered QIs in the hospital setting, namely venous thrombosis prophylaxis (VTE), indwelling bladder catheter, mobilization, and delirium evaluation, in order to test the feasibility and impact of systematically implementing these ACOVE QIs as a therapeutic intervention for all hospitalized older adults.

This study (IRB #13-644B) was conducted using a prospective intervention with a nonequivalent control group design comprised of retrospective chart data from May 1, 2014, to June 30, 2015. Process and outcome variables were extracted from electronic medical records ([EMR], Sunrise Clinical Manager [SCM]) of 2,396 patients, with 530 patients in the intervention unit and 1,866 on the control units, at a large academic tertiary center operating in the greater New York metropolitan area. Our study investigated the role of ACOVE as an intervention to improve patient care by using selected QIs as a standardized checklist tool in the acute care setting. Of the original 30 hospital-specific QIs, our study focused on the care of older adults admitted to the medicine service.²⁶ We selected commonly encountered QIs, with the objective of testing the feasibility and impact of implementing the ACOVE QIs as an intervention to improve care of hospitalized older adults. This intervention consisted of applying the checklist tool, constructed with 4 selected ACOVE QIs and administered daily during interdisciplinary rounds, namely: 2 general “medical” indicators, VTE prophylaxis and indwelling bladder catheters, and 2 “geriatric”-focused indicators, mobilization and delirium evaluation.

Subject matter experts (hospitalists, geriatricians, researchers, administrators, and nurses) reviewed the ACOVE QIs and agreed upon the adaptation of the QIs from a quality measure assessment into a feasible and acceptable intervention checklist tool (Table 1). The checklist was reviewed during daily interdisciplinary rounds for all patients 75 years and older. While ACOVE defined vulnerable elders by using the Vulnerable Elder Screen (VES), we wanted to apply this intervention more broadly to all hospitalized older adults who are most at risk for poor outcomes.²⁷ Patients admitted to the intensive care unit, inpatient psychiatry, inpatient leukemia/lymphoma, and surgical services were excluded.

Daily interdisciplinary rounds are held on every one of the five 40-bed medical units; they last approximately 1 hour, and consist of a lead hospitalist, nurse manager, nurse practitioners, case managers, and the nursing staff. During interdisciplinary rounds, nurses present the case to the team members who then discuss the care plan. These 5 medical units did not differ in terms of patient characteristics or staffing patterns; the intervention unit was chosen simply for logistical reasons, in that the principal investigator (PI) had been assigned to this unit prior to study start-up.

Prior to the intervention, LS held an education session for staff on the intervention unit staff (who participated on interdisciplinary rounds) to explain the concept of the ACOVE QI initiative and describe the four QIs selected for the study. Three subsequent educational sessions were held during the first week of the intervention, with new incoming staff receiving a brief individual educational session. The staff demonstrated significant knowledge improvement after session completion (pre/post mean score 70.6% vs 90.0%; P < .0001).

The Clinical Information System for the Health System EMR, The Eclipsys SCM, has alerts with different levels of severity from “soft” (user must acknowledge a recommendation) to “hard” (requires an action in order to proceed).

To measure compliance of the quality indicators, we collected the following variables:

QI 1: VTE prophylaxis

Through SCM, we collected type of VTE prophylaxis ordered (pharmacologic and/or mechanical) as well as start and stop dates for all agents. International normalized ratio levels were checked for patients receiving warfarin. Days of compliance were calculated.

QI 2: Indwelling Bladder Catheters

SCM data were collected on catheter entry and discontinuation dates, the presence of an indication, and order renewal for bladder catheter at least every 3 days.

QI 3: Mobilization

Ambulation status prior to admission was extracted from nursing documentation completed on admission to the medical ward. Patients documented as bedfast were categorized as nonambulatory prior to admission. Nursing documentation of activity level and amount of feet ambulated per nursing shift were collected. In addition, hospital day of physical therapy (PT) order and hospital days with PT performed were charted. Compliance with QI 3 in patients documented as ambulatory prior to hospital admission was recorded as present if there was a PT order within 48 hours of admission.

QI 4: Delirium Evaluation

During daily rounds, the hospitalist (PI) questioned nurses about delirium evaluation, using the first feature of the Confusion Assessment Method (CAM) as well as the “single question in delirium,” namely, “Is there evidence of an acute change in mental status from the patient’s baseline?” and “Do you think [name of patient] has been more confused lately?”^28,29 Because EMR does not contain a specified field for delirium screening and documentation, and patients are not routinely included in rounds, documentation with QI 4 was recorded using the “key words” method as described in the work by Puelle et al.³⁰ To extract SCM key words, nursing documentation of the “cognitive/perceptual/neurological exam” section of the EMR on admission and on all subsequent documentation (once per shift) was retrieved to identify acute changes in mental status (eg, “altered mental status, delirium/delirious, alert and oriented X 3, confused/confusion, disoriented, lethargy/lethargic”).³⁰ In addition, nurses were asked to activate an SCM parameter, “Acute Confusion” SCM parameter, in the nursing documentation section, which includes potential risk factors for confusion.

In addition to QI compliance, we collected LOS, discharge disposition, and 30-day readmission data.

Generalized linear mixed models (GLMM) for binary clustered (ie, hierarchical) data were used to estimate compliance rates (ie, nurse adherence) for each group (intervention group or control group) in the postintervention period, along with their corresponding 95% confidence intervals. GLMM was used to account for the hierarchical structure of the data: nursing units within a hospital. In order to calculate the Charlson Comorbidity Index, we extracted past medical history from the EMR.³¹

Subjects (N = 2,396) were included in the comparison of the intervention group vs control group for each of the following 4 ACOVE QI compliance measures: DVT, mobilization, bladder catheter, and delirium.

Of the 2,396 patient admissions, 530 were in the intervention unit and 1,866 were in the control unit. In the intervention group, the average age was 84.65 years, 75.58% were white and 47.21% were married. There was no difference in patient demographics between groups (Table 2).

QI 1: VTE Prophylaxis

Compliance with VTE prophylaxis was met in 78.3% of the intervention subjects and 76.5% of the controls (P < .4371) (Table 3). Of note, the rate of VTE prophylaxis was 57% in the intervention vs 39% in the control group (P < .0056), in the 554 patients for whom compliance was not met. Mechanical prophylaxis was used in 35.6% of intervention subjects vs 30.6 in the control (P = .048). Patients who received no form of prophylaxis were 0.5% in the intervention and 3% in the control (P = .027).

QI 2: Indwelling Bladder Catheters

Out of 2,396 subjects, 406 had an indwelling bladder catheter (16.9%). Compliance with the catheter was met in 72.2% of the intervention group vs 54.4% in the control group (P = .1061). An indication for indwelling bladder catheters was documented in 100% of the subjects. The average number of catheter days was 5.16 in the intervention vs 5.88 in the control (P < .2284). There was statistical significance in catheter compliance in the longer stay (>15 days) subjects, decreasing to 23.32% in the control group while staying constant in the intervention group 71.5% (P = .0006).

QI 3: Mobilization

Of the 2,396 patients, 1,991 (83.1%) were reported as ambulatory prior to admission. In the intervention vs control group, 74 (14%) vs 297 (15.7%), respectively, were nonambulatory. Overall compliance with Q3 was 62.9% in the intervention vs 48.2% in the control (P < .0001). More specifically, the average time to PT order in the intervention group was 1.83 days vs 2.22 days in the control group (P < .0051) and the time to PT evaluation was 2.14 days vs 2.42 days, respectively (P < .0108). In the intervention group, 84 patients (15.8%) did not have a PT consult vs 511 (27%) in the control group (P < .0001). The average times per subject in which the nurses documented the approximate number of feet ambulated was 6.48 in the intervention group vs 0.11 in the control group.

QI 4: Delirium Evaluation

In terms of nursing documentation indicating the presence of an acute confusional state, the intervention group had 148 out of 530 nursing notes (27.9%) vs 405 out of 1,866 in the control group (21.7%; P = .0027). However, utilization of the “acute confusion” parameter with documentation of a risk factor did not differ between the groups (5.8% in the intervention group vs 5.6% in the control group, P < .94).

LOS, Discharge Disposition, and 30-Day Readmissions

LOS did not differ between intervention and control groups (6.37 days vs 6.27 days, respectively), with a median of 5 days (P = .877). Discharge disposition in the 2 groups included the following: home/home with services (71.32% vs 68.7%), skilled nursing facility/assisted living/long-term care (24.34 versus 25.83), inpatient hospice/home hospice (2.64 vs 2.25), and expired (1.13 vs 1.77; P < .3282). In addition, 30-day readmissions did not differ (21% vs 20%, respectively, P = .41).

Our goal was to explore an evidence-based, standardized approach to improve the care of hospitalized older adults. This approach leverages existing automated EMR alert functions with an additional level of decision support for VEs, integrated into daily multidisciplinary rounds. The use of a daily checklist-based tool offers a cost-effective and practical pathway to distribute the burden of compliance responsibility amongst team members.

As we anticipated and similar to study findings in hospitalized medicine, geriatric trauma, and primary care, compliance with general care QIs was better than geriatric-focused QIs.^27,32 Wenger et al³³ demonstrated significant improvements with screening for falls and incontinence; however, screening for cognitive impairment did not improve in the outpatient setting by imbedding ACOVE QIs into routine physician practice.

Increased compliance with VTE prophylaxis and indwelling bladder catheters may be explained by national financial incentives for widespread implementation of EMR alert systems. Conversely, mobilization, delirium assessment, and management in hospitalized older adults don’t benefit from similar incentives.

VTE Prophylaxis

The American College of Chest Physicians (ACCP) supports the use of VTE prophylaxis, especially in hospitalized older adults with decreased mobility.³⁴ While greater adoption of EMR has already increased adherence, our intervention resulted in an even higher rate of compliance with the use of pharmacologic VTE prophylaxis.³⁵ In the future, validated scores for risk of thrombosis and bleeding may be integrated into our QI-based checklist.

Indwelling Bladder Catheters

The potential harms of catheters have been described for over 50 years, yet remain frequently used.^36,37 Previous studies have shown success in decreasing catheter days with computer-based and multidisciplinary protocols.^36-39

Our health system’s EMR has built-in “soft” and “hard” alerts for indwelling bladder catheters, so we did not expect intervention-associated changes in compliance.

Mobilization

Hospitalization in older adults frequently results in functional decline.^4,5,40 In response, the mobilization QI recommends an ambulation plan within 48 hours for those patients who were ambulatory prior to admission; it does not specifically define the components of the plan.²⁶ There are several multicomponent interventions that have demonstrated improvement in functional decline, yet they require skilled providers.^41,42 Our intervention implemented specific ambulation plan components: daily ambulation and documentation reminders and early PT evaluation.

While functional status measures have existed for decades, most are primarily geared to assess community-residing individuals and not designed to measure changes in function during hospitalization.^43,44 Furthermore, performance-based hospital measures are difficult to integrate into the daily nursing workflow as they are time consuming.^45,46 In practice, nurses routinely use free text to document functional status in the hospital setting, rendering comparative analysis problematic. Yet, we demonstrated that nurses were more engaged in reporting mobilization (increased documentation of ambulation distance and a decrease in time to PT). Future research should focus on the development of a standardized tool, integrated into the EMR, to accurately measure function in the acute care setting.

Delirium Evaluation

Delirium evaluation remains one of the most difficult clinical challenges for healthcare providers in hospitalized individuals, and our study reiterated these concerns. Previous research has consistently demonstrated that the diagnosis of delirium is missed by up to 75% of clinicians.^47,48 Indeed, our study, which exclusively examined nursing documentation of the delirium evaluation QI, found that both groups showed strikingly low compliance rates. This may have been due to the fact that we only evaluated nursing documentation of suspected or definite diagnosis of delirium and a documented attempt to attribute the altered mental state to a potential etiology.³¹ By utilizing the concept of “key words,” as developed by Puelle et al.³⁰, we were able to demonstrate a statistically significant improvement in nursing delirium documentation in the intervention group. This result should be interpreted with caution, as this approach is not validated. Furthermore, our operational definition of delirium compliance (ie, nurse documentation of delirium, requiring the launching of a separate parameter) may have been simply too cumbersome to readily integrate into the daily workflow. Future research should study the efficacy of a sensitive EMR-integrated screening tool that facilitates recognition, by all team members, of acute changes in cognition.

Although a number of QI improved for the intervention group, acute care utilization measures such as LOS, discharge disposition, and 30-day readmissions did not differ between groups. It may well be that improving quality for this very frail, vulnerable population may simply not result in decreased utilization. Our ability to further decrease LOS and readmission rates may be limited due to restriction of range in this complex patient population (eg, median LOS value of 5 days).

Limitations

Although our study had a large sample size, data were only collected from a single-center and thus require further exploration in different settings to ensure generalizability. In addition, QI observance was based on the medical record, which was problematic for some indicators, notably delirium identification. While prior literature highlights the difficulty in identifying delirium, especially during clinical practice without specialized training, our compliance was strikingly low.⁴⁷ While validated measures such as CAM may have been included as part of the assessment, there is currently no EMR documentation of such measures and therefore, these data could not be obtained.

In summary, our study demonstrates the successful integration of the established ACOVE QIs as an intervention, rather than as an assessment method, for improving care of hospitalized older patients. By utilizing a checklist-based tool at the bedside allows the multidisciplinary team to implement evidence-based practices with the ultimate goal of standardizing care, not only for VEs, but potentially for other high-risk populations with multimorbidity.⁴⁹ This innovative approach provides a much-needed direction to healthcare providers in the ever increasing stressful conditions of today’s acute care environment and for the ultimate benefit and safety of our older patients.

Disclosure

The authors declare no conflicts of interest. This study was supported by New York State Empire Clinical Research Investigators Program (ECRIP). The sponsor had no role in the conception, study design, data collection, data analysis, interpretation of data, manuscript preparation, or the decision to submit the manuscript for publication.

In 2014, the United States spent $3 trillion on healthcare; hospitalization consumed 32% of these expenditures.¹ Today, Medicare patients account for over 50% of hospital days and over 30% of all hospital discharges in the United States.² Despite this staggering financial burden, hospitalization of older adults often results in poor patient outcomes.^3-6 The exponential growth of the hospitalist movement, from 350 hospitalists nationwide in 1995 to over 44,000 in 2014, has become the key strategy for providing care to hospitalized geriatric patients.^7-10 Most of these hospitalists have not received geriatric training.^11-15

There is growing evidence that a geriatric approach, emphasizing multidisciplinary management of the complex needs of older patients, leads to improved outcomes. Geriatric Evaluation and Management Units (GEMUs), such as Acute Care for Elderly (ACE) models, have demonstrated significant decreases in functional decline, institutionalization, and death in randomized controlled trials.^16,17 Multidisciplinary, nonunit based efforts, such as the mobile acute care of elderly (MACE), proactive consultation models (Sennour/Counsell), and the Hospital Elder Life Program (HELP), have demonstrated success in preventing adverse events and decreasing length of stay (LOS).^17-20

However, these models have not been systematically implemented due to challenges in generalizability and replicability in diverse settings. To address this concern, an alternative approach must be developed to widely “generalize” geriatric expertise throughout hospitals, regardless of their location, size, and resources. This initiative will require systematic integration of evidence-based decision support tools for the standardization of clinical management in hospitalized older adults.²¹

The 1998 Assessing Care of Vulnerable Elders (ACOVE) project developed a standardized tool to measure and evaluate the quality of care by using a comprehensive set of quality indicators (QIs) to improve the care of “vulnerable elders” (VEs) at a high risk for functional and cognitive decline and death.^22-24 The latest systematic review concludes that, although many studies have used ACOVE as an assessment tool of quality, there has been a dearth of studies investigating the ACOVE QIs as an intervention to improve patient care.²⁵

Our study investigated the role of ACOVE as an intervention by using the QIs as a standardized checklist in the acute care setting. We selected the 4 most commonly encountered QIs in the hospital setting, namely venous thrombosis prophylaxis (VTE), indwelling bladder catheter, mobilization, and delirium evaluation, in order to test the feasibility and impact of systematically implementing these ACOVE QIs as a therapeutic intervention for all hospitalized older adults.

This study (IRB #13-644B) was conducted using a prospective intervention with a nonequivalent control group design comprised of retrospective chart data from May 1, 2014, to June 30, 2015. Process and outcome variables were extracted from electronic medical records ([EMR], Sunrise Clinical Manager [SCM]) of 2,396 patients, with 530 patients in the intervention unit and 1,866 on the control units, at a large academic tertiary center operating in the greater New York metropolitan area. Our study investigated the role of ACOVE as an intervention to improve patient care by using selected QIs as a standardized checklist tool in the acute care setting. Of the original 30 hospital-specific QIs, our study focused on the care of older adults admitted to the medicine service.²⁶ We selected commonly encountered QIs, with the objective of testing the feasibility and impact of implementing the ACOVE QIs as an intervention to improve care of hospitalized older adults. This intervention consisted of applying the checklist tool, constructed with 4 selected ACOVE QIs and administered daily during interdisciplinary rounds, namely: 2 general “medical” indicators, VTE prophylaxis and indwelling bladder catheters, and 2 “geriatric”-focused indicators, mobilization and delirium evaluation.

Subject matter experts (hospitalists, geriatricians, researchers, administrators, and nurses) reviewed the ACOVE QIs and agreed upon the adaptation of the QIs from a quality measure assessment into a feasible and acceptable intervention checklist tool (Table 1). The checklist was reviewed during daily interdisciplinary rounds for all patients 75 years and older. While ACOVE defined vulnerable elders by using the Vulnerable Elder Screen (VES), we wanted to apply this intervention more broadly to all hospitalized older adults who are most at risk for poor outcomes.²⁷ Patients admitted to the intensive care unit, inpatient psychiatry, inpatient leukemia/lymphoma, and surgical services were excluded.

Daily interdisciplinary rounds are held on every one of the five 40-bed medical units; they last approximately 1 hour, and consist of a lead hospitalist, nurse manager, nurse practitioners, case managers, and the nursing staff. During interdisciplinary rounds, nurses present the case to the team members who then discuss the care plan. These 5 medical units did not differ in terms of patient characteristics or staffing patterns; the intervention unit was chosen simply for logistical reasons, in that the principal investigator (PI) had been assigned to this unit prior to study start-up.

Prior to the intervention, LS held an education session for staff on the intervention unit staff (who participated on interdisciplinary rounds) to explain the concept of the ACOVE QI initiative and describe the four QIs selected for the study. Three subsequent educational sessions were held during the first week of the intervention, with new incoming staff receiving a brief individual educational session. The staff demonstrated significant knowledge improvement after session completion (pre/post mean score 70.6% vs 90.0%; P < .0001).

The Clinical Information System for the Health System EMR, The Eclipsys SCM, has alerts with different levels of severity from “soft” (user must acknowledge a recommendation) to “hard” (requires an action in order to proceed).

To measure compliance of the quality indicators, we collected the following variables:

QI 1: VTE prophylaxis

Through SCM, we collected type of VTE prophylaxis ordered (pharmacologic and/or mechanical) as well as start and stop dates for all agents. International normalized ratio levels were checked for patients receiving warfarin. Days of compliance were calculated.

QI 2: Indwelling Bladder Catheters

SCM data were collected on catheter entry and discontinuation dates, the presence of an indication, and order renewal for bladder catheter at least every 3 days.

QI 3: Mobilization

Ambulation status prior to admission was extracted from nursing documentation completed on admission to the medical ward. Patients documented as bedfast were categorized as nonambulatory prior to admission. Nursing documentation of activity level and amount of feet ambulated per nursing shift were collected. In addition, hospital day of physical therapy (PT) order and hospital days with PT performed were charted. Compliance with QI 3 in patients documented as ambulatory prior to hospital admission was recorded as present if there was a PT order within 48 hours of admission.

QI 4: Delirium Evaluation

During daily rounds, the hospitalist (PI) questioned nurses about delirium evaluation, using the first feature of the Confusion Assessment Method (CAM) as well as the “single question in delirium,” namely, “Is there evidence of an acute change in mental status from the patient’s baseline?” and “Do you think [name of patient] has been more confused lately?”^28,29 Because EMR does not contain a specified field for delirium screening and documentation, and patients are not routinely included in rounds, documentation with QI 4 was recorded using the “key words” method as described in the work by Puelle et al.³⁰ To extract SCM key words, nursing documentation of the “cognitive/perceptual/neurological exam” section of the EMR on admission and on all subsequent documentation (once per shift) was retrieved to identify acute changes in mental status (eg, “altered mental status, delirium/delirious, alert and oriented X 3, confused/confusion, disoriented, lethargy/lethargic”).³⁰ In addition, nurses were asked to activate an SCM parameter, “Acute Confusion” SCM parameter, in the nursing documentation section, which includes potential risk factors for confusion.

In addition to QI compliance, we collected LOS, discharge disposition, and 30-day readmission data.

Generalized linear mixed models (GLMM) for binary clustered (ie, hierarchical) data were used to estimate compliance rates (ie, nurse adherence) for each group (intervention group or control group) in the postintervention period, along with their corresponding 95% confidence intervals. GLMM was used to account for the hierarchical structure of the data: nursing units within a hospital. In order to calculate the Charlson Comorbidity Index, we extracted past medical history from the EMR.³¹

Subjects (N = 2,396) were included in the comparison of the intervention group vs control group for each of the following 4 ACOVE QI compliance measures: DVT, mobilization, bladder catheter, and delirium.

Of the 2,396 patient admissions, 530 were in the intervention unit and 1,866 were in the control unit. In the intervention group, the average age was 84.65 years, 75.58% were white and 47.21% were married. There was no difference in patient demographics between groups (Table 2).

QI 1: VTE Prophylaxis

Compliance with VTE prophylaxis was met in 78.3% of the intervention subjects and 76.5% of the controls (P < .4371) (Table 3). Of note, the rate of VTE prophylaxis was 57% in the intervention vs 39% in the control group (P < .0056), in the 554 patients for whom compliance was not met. Mechanical prophylaxis was used in 35.6% of intervention subjects vs 30.6 in the control (P = .048). Patients who received no form of prophylaxis were 0.5% in the intervention and 3% in the control (P = .027).

QI 2: Indwelling Bladder Catheters

Out of 2,396 subjects, 406 had an indwelling bladder catheter (16.9%). Compliance with the catheter was met in 72.2% of the intervention group vs 54.4% in the control group (P = .1061). An indication for indwelling bladder catheters was documented in 100% of the subjects. The average number of catheter days was 5.16 in the intervention vs 5.88 in the control (P < .2284). There was statistical significance in catheter compliance in the longer stay (>15 days) subjects, decreasing to 23.32% in the control group while staying constant in the intervention group 71.5% (P = .0006).

QI 3: Mobilization

Of the 2,396 patients, 1,991 (83.1%) were reported as ambulatory prior to admission. In the intervention vs control group, 74 (14%) vs 297 (15.7%), respectively, were nonambulatory. Overall compliance with Q3 was 62.9% in the intervention vs 48.2% in the control (P < .0001). More specifically, the average time to PT order in the intervention group was 1.83 days vs 2.22 days in the control group (P < .0051) and the time to PT evaluation was 2.14 days vs 2.42 days, respectively (P < .0108). In the intervention group, 84 patients (15.8%) did not have a PT consult vs 511 (27%) in the control group (P < .0001). The average times per subject in which the nurses documented the approximate number of feet ambulated was 6.48 in the intervention group vs 0.11 in the control group.

QI 4: Delirium Evaluation

In terms of nursing documentation indicating the presence of an acute confusional state, the intervention group had 148 out of 530 nursing notes (27.9%) vs 405 out of 1,866 in the control group (21.7%; P = .0027). However, utilization of the “acute confusion” parameter with documentation of a risk factor did not differ between the groups (5.8% in the intervention group vs 5.6% in the control group, P < .94).

LOS, Discharge Disposition, and 30-Day Readmissions

LOS did not differ between intervention and control groups (6.37 days vs 6.27 days, respectively), with a median of 5 days (P = .877). Discharge disposition in the 2 groups included the following: home/home with services (71.32% vs 68.7%), skilled nursing facility/assisted living/long-term care (24.34 versus 25.83), inpatient hospice/home hospice (2.64 vs 2.25), and expired (1.13 vs 1.77; P < .3282). In addition, 30-day readmissions did not differ (21% vs 20%, respectively, P = .41).

Our goal was to explore an evidence-based, standardized approach to improve the care of hospitalized older adults. This approach leverages existing automated EMR alert functions with an additional level of decision support for VEs, integrated into daily multidisciplinary rounds. The use of a daily checklist-based tool offers a cost-effective and practical pathway to distribute the burden of compliance responsibility amongst team members.

As we anticipated and similar to study findings in hospitalized medicine, geriatric trauma, and primary care, compliance with general care QIs was better than geriatric-focused QIs.^27,32 Wenger et al³³ demonstrated significant improvements with screening for falls and incontinence; however, screening for cognitive impairment did not improve in the outpatient setting by imbedding ACOVE QIs into routine physician practice.

Increased compliance with VTE prophylaxis and indwelling bladder catheters may be explained by national financial incentives for widespread implementation of EMR alert systems. Conversely, mobilization, delirium assessment, and management in hospitalized older adults don’t benefit from similar incentives.

VTE Prophylaxis

The American College of Chest Physicians (ACCP) supports the use of VTE prophylaxis, especially in hospitalized older adults with decreased mobility.³⁴ While greater adoption of EMR has already increased adherence, our intervention resulted in an even higher rate of compliance with the use of pharmacologic VTE prophylaxis.³⁵ In the future, validated scores for risk of thrombosis and bleeding may be integrated into our QI-based checklist.

Indwelling Bladder Catheters

The potential harms of catheters have been described for over 50 years, yet remain frequently used.^36,37 Previous studies have shown success in decreasing catheter days with computer-based and multidisciplinary protocols.^36-39

Our health system’s EMR has built-in “soft” and “hard” alerts for indwelling bladder catheters, so we did not expect intervention-associated changes in compliance.

Mobilization

Hospitalization in older adults frequently results in functional decline.^4,5,40 In response, the mobilization QI recommends an ambulation plan within 48 hours for those patients who were ambulatory prior to admission; it does not specifically define the components of the plan.²⁶ There are several multicomponent interventions that have demonstrated improvement in functional decline, yet they require skilled providers.^41,42 Our intervention implemented specific ambulation plan components: daily ambulation and documentation reminders and early PT evaluation.

While functional status measures have existed for decades, most are primarily geared to assess community-residing individuals and not designed to measure changes in function during hospitalization.^43,44 Furthermore, performance-based hospital measures are difficult to integrate into the daily nursing workflow as they are time consuming.^45,46 In practice, nurses routinely use free text to document functional status in the hospital setting, rendering comparative analysis problematic. Yet, we demonstrated that nurses were more engaged in reporting mobilization (increased documentation of ambulation distance and a decrease in time to PT). Future research should focus on the development of a standardized tool, integrated into the EMR, to accurately measure function in the acute care setting.

Delirium Evaluation

Delirium evaluation remains one of the most difficult clinical challenges for healthcare providers in hospitalized individuals, and our study reiterated these concerns. Previous research has consistently demonstrated that the diagnosis of delirium is missed by up to 75% of clinicians.^47,48 Indeed, our study, which exclusively examined nursing documentation of the delirium evaluation QI, found that both groups showed strikingly low compliance rates. This may have been due to the fact that we only evaluated nursing documentation of suspected or definite diagnosis of delirium and a documented attempt to attribute the altered mental state to a potential etiology.³¹ By utilizing the concept of “key words,” as developed by Puelle et al.³⁰, we were able to demonstrate a statistically significant improvement in nursing delirium documentation in the intervention group. This result should be interpreted with caution, as this approach is not validated. Furthermore, our operational definition of delirium compliance (ie, nurse documentation of delirium, requiring the launching of a separate parameter) may have been simply too cumbersome to readily integrate into the daily workflow. Future research should study the efficacy of a sensitive EMR-integrated screening tool that facilitates recognition, by all team members, of acute changes in cognition.

Although a number of QI improved for the intervention group, acute care utilization measures such as LOS, discharge disposition, and 30-day readmissions did not differ between groups. It may well be that improving quality for this very frail, vulnerable population may simply not result in decreased utilization. Our ability to further decrease LOS and readmission rates may be limited due to restriction of range in this complex patient population (eg, median LOS value of 5 days).

Limitations

Although our study had a large sample size, data were only collected from a single-center and thus require further exploration in different settings to ensure generalizability. In addition, QI observance was based on the medical record, which was problematic for some indicators, notably delirium identification. While prior literature highlights the difficulty in identifying delirium, especially during clinical practice without specialized training, our compliance was strikingly low.⁴⁷ While validated measures such as CAM may have been included as part of the assessment, there is currently no EMR documentation of such measures and therefore, these data could not be obtained.

In summary, our study demonstrates the successful integration of the established ACOVE QIs as an intervention, rather than as an assessment method, for improving care of hospitalized older patients. By utilizing a checklist-based tool at the bedside allows the multidisciplinary team to implement evidence-based practices with the ultimate goal of standardizing care, not only for VEs, but potentially for other high-risk populations with multimorbidity.⁴⁹ This innovative approach provides a much-needed direction to healthcare providers in the ever increasing stressful conditions of today’s acute care environment and for the ultimate benefit and safety of our older patients.

Disclosure

The authors declare no conflicts of interest. This study was supported by New York State Empire Clinical Research Investigators Program (ECRIP). The sponsor had no role in the conception, study design, data collection, data analysis, interpretation of data, manuscript preparation, or the decision to submit the manuscript for publication.

Fever in hospitalized patients is a nonspecific finding with many potential causes. Blood cultures (BC) are commonly obtained prior to commencing parenteral antibiotics in febrile patients. However, as many as 35% to 50% of positive BCs represent a contamination with organisms inoculated from the skin into culture bottles at the time of sample collection.^1-3 Such results represent false-positive BCs that can lead to unnecessary investigations and treatment.

Recently, Coburn et al. reviewed the severity of chills (graded on an ordinal scale) as the most useful predictor of true bacteremia (positive likelihood ratio [LR], 4.7; 95% confidence interval [CI], 3.0–7.2),^4-6 and the lack of the systemic inflammatory response syndrome (SIRS) criteria as the best negative indicator of true bacteremia with a negative LR of 0.09 (95% CI, 0.03-0.3).^6,7 We have also previously reported normal food consumption as a negative indicator of true bacteremia, with a 98.3% negative predictive value.⁸ Henderson’s Basic Principles of Nursing Care emphasizes the importance of evaluating whether a patient can eat and drink adequately,⁹ and the evaluation of a patient’s food consumption is a routine nursing staff practice, which is treated as vital sign in Japan, in contrast to nursing practices in the United States.

However, these data were the result of a single-center retrospective study using the nursing staff’s assessment of food consumption, and they cannot be generalized to larger patient populations. Therefore, the aim of this prospective, multicenter study was to measure the accuracy of food consumption and shaking chills as predictive factors for true bacteremia.

Study Design

This was a prospective multicenter observational study (UMIN ID: R000013768) involving 3 hospitals in Tokyo, Japan, that enrolled consecutive patients who had BCs obtained. This study was approved by the ethical committee at Juntendo University Nerima Hospital and each of the participating centers, and the study was conducted in accordance with the Declaration of Helsinki 1971, as revised in 1983. We evaluated 2,792 consecutive hospitalized patients (mean age, 68.9 ± 17.1 years; 55.3% men) who had BCs obtained between April 2013 and August 2014, inclusive. The indication for BC acquisition was at the discretion of the treating physician. The study protocol and the indication for BCs are described in detail elsewhere.⁸ We excluded patients with anorexia-inducing conditions such as gastrointestinal disease, including gastrointestinal bleeding, enterocolitis, gastric ulceration, peritonitis, appendicitis, cholangitis, pancreatitis, diverticulitis, and ischemic colitis. We also excluded patients receiving chemotherapy for malignancy. In this study, true bacteremia was defined as identical organisms isolated from 2 sets of blood cultures (a set refers to one aerobic bottle and one anaerobic bottle). Moreover, even if only one set of blood cultures was acquired, when the identified pathogen could account for the clinical presentation, we also defined this as true bacteremia. Briefly, contaminants were defined as organisms common to skin flora, including Bacillus species, coagulase-negative Staphylococcus, Corynebacterium species, and Micrococcus species, without isolation of an identical organism with the same antibiotic susceptibilities from another potentially infected site in a patient with incompatible clinical features and no risk factors for infection with the isolated organism. Single BCs that were positive for organisms that were unlikely to explain the patient’s symptoms were also considered as contaminants. Patients with contaminated BCs were excluded from the analyses.

Structure of Reliability Study Procedures

Nurses in the 3 different hospitals performed daily independent food consumption ratings during each patient’s stay. Interrater reliability assessments were conducted in the morning or afternoon, and none of the raters had access to the other nurses’ scores at any time. The study nurses performed simultaneous ratings during these assessments (one interacted with and rated the patient while the other observed and rated the same patient).

Prediction Variables of True Bacteremia


1. Food consumption. Assessment of food consumption has been previously described in detail.⁸ Briefly, we characterized the patients’ oral intake based on the meal taken immediately prior to the BCs. For example, if a fever developed at 2 pm, lunch consumption was evaluated. If a fever developed at 2 am, dinner consumption was evaluated. We categorized the patients into 3 groups: low food consumption (<50% consumed), moderate food consumption (>50% to <80% consumed), and high food consumption (>80% consumed). To simplify our prediction rule, we subsequently divided food consumption into just 2 groups: high food consumption, referred to as the “normal food consumption group,” and the combination of low and moderate food consumption, referred to as the “poor food consumption group.”

2. Chills. As done previously, the physician evaluated the patient for a history of chills at the time of BCs and classified the patients into 1 of 4 grades^4,5: “no chills,” the absence of any chills; “mild chills,” feeling cold, equivalent to needing an outer jacket; “moderate chills,” feeling very cold, equivalent to needing a thick blanket; and “shaking chills,” feeling extremely cold with rigors and generalized bodily shaking, even under a thick blanket. To distinguish between those patients who had shaking chills and those who did not, we divided the patients into 2 groups: the “shaking chills group” and the combination of none, mild, and moderate chills, referred to as the “negative shaking chills group.”

3. Other predictive variables. We considered the following additional predictive variables: age, gender, axillary body temperature (BT), heart rate (HR), systolic blood pressure (SBP), respiratory rate (RR), white blood cell count (WBC), and serum C-reactive protein level (CRP). These predictive variables were obtained immediately prior to the BCs. We defined SIRS based on standard criteria (HR >90 beats/m, RR >20/m, BT <36°C or >38°C, and a WBC <4 × 10³ WBC/μL or >12 × 10³ WBC/μL). Patients were subcategorized by age into 2 groups (≤69 years and >70 years). CRP levels were dichotomized as >10.0 mg/dL or ≤10.0 mg/dL. We reviewed the patients’ charts to determine whether they had received antibiotics. In the case of walk-in patients, we interviewed the patients regarding whether they had visited a clinic; if they had, they were questioned as to whether any antibiotic therapy had been prescribed.

Statistical Analysis

Continuous variables are presented as the mean with the associated standard deviation (SD). All potential variables predictive of true bacteremia are shown in Table 1. The variables were dichotomized by clinically meaningful thresholds and used as potential risk-adjusted variables. We calculated the sensitivity and specificity and positive and negative predictive value for each criterion. Multiple logistic regression analysis was used to select components that were significantly associated with true bacteremia (the level of statistical significance determined with maximum likelihood methods was set at P < .05). To visualize and quantify other aspects in the prediction of true bacteremia, a recursive partitioning analysis (RPA) was used to make a decision tree model for true bacteremia. This nonparametric regression method produces a classification tree following a series of nonsequential top-down binary splits. The tree-building process starts by considering a set of predictive variables and selects the variable that produces 2 subsets of participants with the greatest purity. Two factors are considered when splitting a node into its daughter nodes: the goodness of the split and the amount of impurity in the daughter nodes. The splitting process is repeated until further partitioning is no longer possible and the terminal nodes have been reached. Details on this method are discussed in Monte Carlo Calibration of Distributions of Partition Statistics (www.jmp.com).

Probability was considered significant at a value of P < .05. All statistical tests were 2-tailed. Statistical analyses were conducted by a physician (KI) and an independent statistician (JM) with the use of the SPSS^® v.16.0 software package (SPSS Inc., Chicago, IL) and JMP^® version 8.0.2 (SAS Institute, Cary, NC).

Patients Characteristics

Two thousand seven hundred and ninety-two patients met the inclusion criteria for our study, from which 849 were excluded (see Figure 1 for flow diagram). Among the remaining 1,943 patients, there were 317 patients with positive BCs, of which 221 patients (69.7%) were considered to have true-positive BCs and 96 (30.3%) were considered to have contaminated BCs. After excluding these 96 patients, 221 patients with true bacteremia (true bacteremic group) were compared with 1,626 nonbacteremic patients (nonbacteremic group; Figure 1). The baseline characteristics of the subjects are shown in Table 1. The mean BT was 38.4 ± 1.2°C in the true bacteremic group and 37.9 ± 1.0°C in the nonbacteremic group. The mean serum CRP level was 11.6 ± 9.6 mg/dL in the true bacteremic group and 7.3 ± 6.9 mg/dL in the nonbacteremic group. In the true bacteremic group, there were 6 afebrile patients, and 27 patients without leukocytosis. The pathogens identified from the true-positive BCs were Escherichia coli (n = 59, 26.7%), including extended-spectrum beta-lactamase producing species, Staphylococcus aureus (n = 36, 16.3%), including methicillin-resistant Staphylococcus aureus, and Klebsiella pneumoniae (n = 22, 10.0%; Supplemental Table 1).

The underlying clinical diagnoses in the true bacteremic group included urinary tract infection (UTI), pneumonia, abscess, catheter-related bloodstream infection (CRBSI), cellulitis, osteomyelitis, infective endocarditis (IE), chorioamnionitis, iatrogenic infection at hemodialysis puncture sites, bacterial meningitis, septic arthritis, and infection of unknown cause (Supplemental Table 2).

Interrater Reliability Testing of Food Consumption

Patients were evaluated during their hospital stays. The interrater reliability of the evaluation of food consumption was very high across all participating hospitals (Supplemental Table 3). To assess the reliability of the evaluations of food consumption, patients (separate from this main study) were selected randomly and evaluated independently by 2 nurses in 3 different hospitals. The kappa scores of agreement between the nurses at the 3 different hospitals were 0.83 (95% CI, 0.63-0.88), 0.90 (95% CI, 0.80-0.99), and 0.80 (95% CI, 0.67-0.99), respectively. The interrater reliability of food consumption evaluation by the nurses was very high at all participating hospitals.

Food Consumption

The low, moderate, and high food consumption groups consisted of 964 (52.1%), 306 (16.6%), and 577 (31.2%) patients, respectively (Table 1). Of these, 174 (18.0%), 33 (10.8%), and 14 (2.4%) patients, respectively, had true bacteremia. The presence of poor food consumption had a sensitivity of 93.7% (95% CI, 89.4%-97.9%), specificity of 34.6% (95% CI, 33.0%-36.2%), and a positive LR of 1.43 (95% CI, 1.37-1.50) for predicting true bacteremia. Conversely, the absence of poor food consumption (ie, normal food consumption) had a negative LR of 0.18 (95% CI, 0.17-0.19).

Chills

The no, mild, moderate, and shaking chills groups consisted of 1,514 (82.0%), 148 (8.0%), 53 (2.9%), and 132 (7.1%) patients, respectively (Table 1). Of these, 136 (9.0%), 25 (16.9%), 8 (15.1%), and 52 (39.4%) patients, respectively, had true bacteremia. The presence of shaking chills had a sensitivity of 23.5% (95% CI, 22.5%-24.6%), a specificity of 95.1% (95% CI, 90.7%-99.4%), and a positive LR of 4.78 (95% CI, 4.56–5.00) for predicting true bacteremia. Conversely, the absence of shaking chills had a negative LR of 0.80 (95% CI, 0.77-0.84).

Prediction Model for True Bacteremia

The components identified as significantly related to true bacteremia by multiple logistic regression analysis are indicated in Table 2. The significant predictors of true bacteremia were shaking chills (odds ratio [OR], 5.6; 95% CI, 3.6-8.6; P < .01), SBP <90 mmHg (OR, 3.1; 95% CI, 1.6-5.7; P < 01), CRP levels >10.0 mg/dL (OR, 2.2; 95% CI, 1.6-3.1; P < .01), BT <36°C or >38°C (OR, 1.8; 95% CI, 1.3-2.6; P < .01), WBC <4 × 10³/μL or >12 × 10³/μL (OR, 1.6; 95% CI, 1.2-2.3; P = .003), HR >90 bpm (OR, 1.5; 95% CI, 1.1-2.1; P = .021), and female (OR, 1.4; 95% CI, 1.0-1.9; P = .036). An RPA to create an ideal prediction model for patients with true bacteremia or nonbacteremia is shown in Figure 2. The original group consisted of 1,847 patients, including 221 patients with true bacteremia. The pretest probability of true bacteremia was 2.4% (14/577) for those with normal food consumption (Group 1) and 2.4% (13/552) for those with both normal food consumption and the absence of shaking chills (Group 2). Conversely, the pretest probability of true bacteremia was 16.3% (207/1270) for those with poor food consumption and 47.7% (51/107) for those with both poor food consumption and shaking chills. The patients with true bacteremia with normal food consumption and without shaking chills consisted of 4 cases of CRBSI and UTI, 2 cases of osteomyelitis, 1 case of IE, 1 case of chorioamnionitis, and 1 case for which the focus was unknown (Supplemental Table 4).

In this observational study, we evaluated if a simple algorithm using food consumption and shaking chills was useful for assessing whether a patient had true bacteremia. A 2-item screening checklist (nursing assessment of food consumption and shaking chills) had excellent statistical properties as a brief screening instrument for true bacteremia.

We have prospectively validated that food consumption, as assessed by nurses, is a reliable predictor of true bacteremia.⁸ A previous single-center retrospective study showed similar findings, but these could not be generalized across all institutions because of the limited nature of the study. In this multicenter study, we used 2 statistical methods to reduce selection bias. First, we performed a kappa analysis across the hospitals to evaluate the interrater reliability of the evaluation of food consumption. Second, we used an RPA (Figure 2), also known as a decision tree model. RPA is a step-by-step process by which a decision tree is constructed by either splitting or not splitting each node on the tree into 2 daughter nodes.¹⁰ By using this method, we successfully generated an ideal approach to predict true bacteremia using food consumption and shaking chills. After adjusting for food consumption and shaking chills, groups 1 to 2 had sequentially decreasing diagnoses of true bacteremia, varying from 221 patients to only 13 patients.

Appetite is influenced by many factors that are integrated by the brain, most importantly within the hypothalamus. Signals that impinge on the hypothalamic center include neural afferents, hormones, cytokines, and metabolites.¹¹ These factors elicit “sickness behavior,” which includes a decrease in food-motivated behavior.¹² Furthermore, exposure to pathogenic bacteria increases serotonin, which has been shown to decrease metabolism in amphid neurons by transcriptional and post-transcriptional mechanisms.¹³ Therefore, nonbacteremic patients retain their appetites. Shaking chills are a well-known predictor of true bacteremia.^4,5 Several cytokines, including tumor necrosis factor-alpha and interleukins 6 and 10, may be related to shaking chills.¹⁴ Coburn et al. reviewed that shaking chills appear to be useful for identifying true bacteremia (positive LR, 4.7; 95% CI, 3.0-7.2),^5,6 similar to our study. In our study, the pretest probability of true bacteremia was the same whether shaking chills was included or not (ie, 2.4% for normal food consumption and 2.4% for normal food consumption plus absence of shaking chills). This would seem to imply that the assessment of shaking chills does not appear to add anything over food assessment alone when trying to rule out bacteremia. Rather, shaking chills seem more important for ruling in bacteremia rather than ruling it out. Moreover, the recent retrospective study revealed that age >60 years (OR = 2.75, 95% CI, 1.23-6.48, P = .015), female sex (OR = 2.21, 95% CI, 1.07- 4.67, P = .038), heart rate >90 bpm (OR = 5.18, 95% CI, 2.25-12.48, P < .001) and neutrophil percentage >80% (OR = 3.61, 95% CI, 1.71- 8.00, P = .001) were independent risk factors for true bacteremia.¹⁵ Conversely, the lack of the SIRS criteria was reported as the best negative indicator of true bacteremia with a negative LR of 0.09 (95% CI, 0.03-0.3).^6,7 However, the evaluation of SIRS criteria requires the acquisition of laboratory data. To our knowledge, no previous prospective studies have evaluated food consumption in terms of a risk prediction for true bacteremia. This extremely simple model can enable a physician to make a rapid bedside estimation of the risk of true bacteremia.

The strengths of this study include its relatively large sample size, multicenter design, uniformity of data collection across sites, and completeness of data collection from study participants. All of these factors allowed for a robust analysis.

However, there are several limitations of this study. First, the physicians or nurses asked the patients about the presence of shaking chills when they obtained the BCs. It may be difficult for patients, especially elderly patients, to provide this information promptly and accurately. Some patients did not call the nurse when they had shaking chills, and the chills were not witnessed by a healthcare provider. However, we used a more specific definition for shaking chills: a feeling of being extremely cold with rigors and generalized bodily shaking, even under a thick blanket. Second, this algorithm is not applicable to patients with immunosuppressed states because none of the hospitals involved in this study perform bone marrow or organ transplantation. Third, although we included patients with dementia in our cohort, we did not specifically evaluate performance of the algorithm in patients with this medical condition. It is possible that the algorithm would not perform well in this subset of patients owing to their unreliable appetite and food intake. Fourth, some medications may affect appetite, leading to reduced food consumption. Although we have not considered the details of medications in this study, we found that the pretest probability of true bacteremia was low for those patients with normal food consumption regardless of whether the medication affected their appetites or not. However, the question of whether medications truly affect patients’ appetites concurrently with bacteremia would need to be specifically addressed in a future study.

CONCLUSION

In conclusion, we have established a simple algorithm to identify patients with suspected true bacteremia who require the acquisition of blood cultures. This extremely simple model can enable physicians to make a rapid bedside estimation of the risk of true bacteremia.

Acknowledgment

The authors thank Drs. H. Honda and S. Saint, and Ms. A. Okada for their helpful discussions with regard to this study; Ms. M. Takigawa for the collection of data; and Ms. T. Oguri for providing infectious disease consultation on the pathogenicity of the identified organisms.

Disclosure

This work was supported by JSPS KAKENHI Grant Number 15K19294 (to TK) and 20590840 (to KI) from the Japan Society for the Promotion of Science. The authors report no potential conflicts of interest relevant to this article.

Fever in hospitalized patients is a nonspecific finding with many potential causes. Blood cultures (BC) are commonly obtained prior to commencing parenteral antibiotics in febrile patients. However, as many as 35% to 50% of positive BCs represent a contamination with organisms inoculated from the skin into culture bottles at the time of sample collection.^1-3 Such results represent false-positive BCs that can lead to unnecessary investigations and treatment.

Recently, Coburn et al. reviewed the severity of chills (graded on an ordinal scale) as the most useful predictor of true bacteremia (positive likelihood ratio [LR], 4.7; 95% confidence interval [CI], 3.0–7.2),^4-6 and the lack of the systemic inflammatory response syndrome (SIRS) criteria as the best negative indicator of true bacteremia with a negative LR of 0.09 (95% CI, 0.03-0.3).^6,7 We have also previously reported normal food consumption as a negative indicator of true bacteremia, with a 98.3% negative predictive value.⁸ Henderson’s Basic Principles of Nursing Care emphasizes the importance of evaluating whether a patient can eat and drink adequately,⁹ and the evaluation of a patient’s food consumption is a routine nursing staff practice, which is treated as vital sign in Japan, in contrast to nursing practices in the United States.

However, these data were the result of a single-center retrospective study using the nursing staff’s assessment of food consumption, and they cannot be generalized to larger patient populations. Therefore, the aim of this prospective, multicenter study was to measure the accuracy of food consumption and shaking chills as predictive factors for true bacteremia.

Study Design

This was a prospective multicenter observational study (UMIN ID: R000013768) involving 3 hospitals in Tokyo, Japan, that enrolled consecutive patients who had BCs obtained. This study was approved by the ethical committee at Juntendo University Nerima Hospital and each of the participating centers, and the study was conducted in accordance with the Declaration of Helsinki 1971, as revised in 1983. We evaluated 2,792 consecutive hospitalized patients (mean age, 68.9 ± 17.1 years; 55.3% men) who had BCs obtained between April 2013 and August 2014, inclusive. The indication for BC acquisition was at the discretion of the treating physician. The study protocol and the indication for BCs are described in detail elsewhere.⁸ We excluded patients with anorexia-inducing conditions such as gastrointestinal disease, including gastrointestinal bleeding, enterocolitis, gastric ulceration, peritonitis, appendicitis, cholangitis, pancreatitis, diverticulitis, and ischemic colitis. We also excluded patients receiving chemotherapy for malignancy. In this study, true bacteremia was defined as identical organisms isolated from 2 sets of blood cultures (a set refers to one aerobic bottle and one anaerobic bottle). Moreover, even if only one set of blood cultures was acquired, when the identified pathogen could account for the clinical presentation, we also defined this as true bacteremia. Briefly, contaminants were defined as organisms common to skin flora, including Bacillus species, coagulase-negative Staphylococcus, Corynebacterium species, and Micrococcus species, without isolation of an identical organism with the same antibiotic susceptibilities from another potentially infected site in a patient with incompatible clinical features and no risk factors for infection with the isolated organism. Single BCs that were positive for organisms that were unlikely to explain the patient’s symptoms were also considered as contaminants. Patients with contaminated BCs were excluded from the analyses.

Structure of Reliability Study Procedures

Nurses in the 3 different hospitals performed daily independent food consumption ratings during each patient’s stay. Interrater reliability assessments were conducted in the morning or afternoon, and none of the raters had access to the other nurses’ scores at any time. The study nurses performed simultaneous ratings during these assessments (one interacted with and rated the patient while the other observed and rated the same patient).

Prediction Variables of True Bacteremia


1. Food consumption. Assessment of food consumption has been previously described in detail.⁸ Briefly, we characterized the patients’ oral intake based on the meal taken immediately prior to the BCs. For example, if a fever developed at 2 pm, lunch consumption was evaluated. If a fever developed at 2 am, dinner consumption was evaluated. We categorized the patients into 3 groups: low food consumption (<50% consumed), moderate food consumption (>50% to <80% consumed), and high food consumption (>80% consumed). To simplify our prediction rule, we subsequently divided food consumption into just 2 groups: high food consumption, referred to as the “normal food consumption group,” and the combination of low and moderate food consumption, referred to as the “poor food consumption group.”

2. Chills. As done previously, the physician evaluated the patient for a history of chills at the time of BCs and classified the patients into 1 of 4 grades^4,5: “no chills,” the absence of any chills; “mild chills,” feeling cold, equivalent to needing an outer jacket; “moderate chills,” feeling very cold, equivalent to needing a thick blanket; and “shaking chills,” feeling extremely cold with rigors and generalized bodily shaking, even under a thick blanket. To distinguish between those patients who had shaking chills and those who did not, we divided the patients into 2 groups: the “shaking chills group” and the combination of none, mild, and moderate chills, referred to as the “negative shaking chills group.”

3. Other predictive variables. We considered the following additional predictive variables: age, gender, axillary body temperature (BT), heart rate (HR), systolic blood pressure (SBP), respiratory rate (RR), white blood cell count (WBC), and serum C-reactive protein level (CRP). These predictive variables were obtained immediately prior to the BCs. We defined SIRS based on standard criteria (HR >90 beats/m, RR >20/m, BT <36°C or >38°C, and a WBC <4 × 10³ WBC/μL or >12 × 10³ WBC/μL). Patients were subcategorized by age into 2 groups (≤69 years and >70 years). CRP levels were dichotomized as >10.0 mg/dL or ≤10.0 mg/dL. We reviewed the patients’ charts to determine whether they had received antibiotics. In the case of walk-in patients, we interviewed the patients regarding whether they had visited a clinic; if they had, they were questioned as to whether any antibiotic therapy had been prescribed.

Statistical Analysis

Continuous variables are presented as the mean with the associated standard deviation (SD). All potential variables predictive of true bacteremia are shown in Table 1. The variables were dichotomized by clinically meaningful thresholds and used as potential risk-adjusted variables. We calculated the sensitivity and specificity and positive and negative predictive value for each criterion. Multiple logistic regression analysis was used to select components that were significantly associated with true bacteremia (the level of statistical significance determined with maximum likelihood methods was set at P < .05). To visualize and quantify other aspects in the prediction of true bacteremia, a recursive partitioning analysis (RPA) was used to make a decision tree model for true bacteremia. This nonparametric regression method produces a classification tree following a series of nonsequential top-down binary splits. The tree-building process starts by considering a set of predictive variables and selects the variable that produces 2 subsets of participants with the greatest purity. Two factors are considered when splitting a node into its daughter nodes: the goodness of the split and the amount of impurity in the daughter nodes. The splitting process is repeated until further partitioning is no longer possible and the terminal nodes have been reached. Details on this method are discussed in Monte Carlo Calibration of Distributions of Partition Statistics (www.jmp.com).

Probability was considered significant at a value of P < .05. All statistical tests were 2-tailed. Statistical analyses were conducted by a physician (KI) and an independent statistician (JM) with the use of the SPSS^® v.16.0 software package (SPSS Inc., Chicago, IL) and JMP^® version 8.0.2 (SAS Institute, Cary, NC).

Patients Characteristics

Two thousand seven hundred and ninety-two patients met the inclusion criteria for our study, from which 849 were excluded (see Figure 1 for flow diagram). Among the remaining 1,943 patients, there were 317 patients with positive BCs, of which 221 patients (69.7%) were considered to have true-positive BCs and 96 (30.3%) were considered to have contaminated BCs. After excluding these 96 patients, 221 patients with true bacteremia (true bacteremic group) were compared with 1,626 nonbacteremic patients (nonbacteremic group; Figure 1). The baseline characteristics of the subjects are shown in Table 1. The mean BT was 38.4 ± 1.2°C in the true bacteremic group and 37.9 ± 1.0°C in the nonbacteremic group. The mean serum CRP level was 11.6 ± 9.6 mg/dL in the true bacteremic group and 7.3 ± 6.9 mg/dL in the nonbacteremic group. In the true bacteremic group, there were 6 afebrile patients, and 27 patients without leukocytosis. The pathogens identified from the true-positive BCs were Escherichia coli (n = 59, 26.7%), including extended-spectrum beta-lactamase producing species, Staphylococcus aureus (n = 36, 16.3%), including methicillin-resistant Staphylococcus aureus, and Klebsiella pneumoniae (n = 22, 10.0%; Supplemental Table 1).

The underlying clinical diagnoses in the true bacteremic group included urinary tract infection (UTI), pneumonia, abscess, catheter-related bloodstream infection (CRBSI), cellulitis, osteomyelitis, infective endocarditis (IE), chorioamnionitis, iatrogenic infection at hemodialysis puncture sites, bacterial meningitis, septic arthritis, and infection of unknown cause (Supplemental Table 2).

Interrater Reliability Testing of Food Consumption

Patients were evaluated during their hospital stays. The interrater reliability of the evaluation of food consumption was very high across all participating hospitals (Supplemental Table 3). To assess the reliability of the evaluations of food consumption, patients (separate from this main study) were selected randomly and evaluated independently by 2 nurses in 3 different hospitals. The kappa scores of agreement between the nurses at the 3 different hospitals were 0.83 (95% CI, 0.63-0.88), 0.90 (95% CI, 0.80-0.99), and 0.80 (95% CI, 0.67-0.99), respectively. The interrater reliability of food consumption evaluation by the nurses was very high at all participating hospitals.

Food Consumption

The low, moderate, and high food consumption groups consisted of 964 (52.1%), 306 (16.6%), and 577 (31.2%) patients, respectively (Table 1). Of these, 174 (18.0%), 33 (10.8%), and 14 (2.4%) patients, respectively, had true bacteremia. The presence of poor food consumption had a sensitivity of 93.7% (95% CI, 89.4%-97.9%), specificity of 34.6% (95% CI, 33.0%-36.2%), and a positive LR of 1.43 (95% CI, 1.37-1.50) for predicting true bacteremia. Conversely, the absence of poor food consumption (ie, normal food consumption) had a negative LR of 0.18 (95% CI, 0.17-0.19).

Chills

The no, mild, moderate, and shaking chills groups consisted of 1,514 (82.0%), 148 (8.0%), 53 (2.9%), and 132 (7.1%) patients, respectively (Table 1). Of these, 136 (9.0%), 25 (16.9%), 8 (15.1%), and 52 (39.4%) patients, respectively, had true bacteremia. The presence of shaking chills had a sensitivity of 23.5% (95% CI, 22.5%-24.6%), a specificity of 95.1% (95% CI, 90.7%-99.4%), and a positive LR of 4.78 (95% CI, 4.56–5.00) for predicting true bacteremia. Conversely, the absence of shaking chills had a negative LR of 0.80 (95% CI, 0.77-0.84).

Prediction Model for True Bacteremia

The components identified as significantly related to true bacteremia by multiple logistic regression analysis are indicated in Table 2. The significant predictors of true bacteremia were shaking chills (odds ratio [OR], 5.6; 95% CI, 3.6-8.6; P < .01), SBP <90 mmHg (OR, 3.1; 95% CI, 1.6-5.7; P < 01), CRP levels >10.0 mg/dL (OR, 2.2; 95% CI, 1.6-3.1; P < .01), BT <36°C or >38°C (OR, 1.8; 95% CI, 1.3-2.6; P < .01), WBC <4 × 10³/μL or >12 × 10³/μL (OR, 1.6; 95% CI, 1.2-2.3; P = .003), HR >90 bpm (OR, 1.5; 95% CI, 1.1-2.1; P = .021), and female (OR, 1.4; 95% CI, 1.0-1.9; P = .036). An RPA to create an ideal prediction model for patients with true bacteremia or nonbacteremia is shown in Figure 2. The original group consisted of 1,847 patients, including 221 patients with true bacteremia. The pretest probability of true bacteremia was 2.4% (14/577) for those with normal food consumption (Group 1) and 2.4% (13/552) for those with both normal food consumption and the absence of shaking chills (Group 2). Conversely, the pretest probability of true bacteremia was 16.3% (207/1270) for those with poor food consumption and 47.7% (51/107) for those with both poor food consumption and shaking chills. The patients with true bacteremia with normal food consumption and without shaking chills consisted of 4 cases of CRBSI and UTI, 2 cases of osteomyelitis, 1 case of IE, 1 case of chorioamnionitis, and 1 case for which the focus was unknown (Supplemental Table 4).

In this observational study, we evaluated if a simple algorithm using food consumption and shaking chills was useful for assessing whether a patient had true bacteremia. A 2-item screening checklist (nursing assessment of food consumption and shaking chills) had excellent statistical properties as a brief screening instrument for true bacteremia.

We have prospectively validated that food consumption, as assessed by nurses, is a reliable predictor of true bacteremia.⁸ A previous single-center retrospective study showed similar findings, but these could not be generalized across all institutions because of the limited nature of the study. In this multicenter study, we used 2 statistical methods to reduce selection bias. First, we performed a kappa analysis across the hospitals to evaluate the interrater reliability of the evaluation of food consumption. Second, we used an RPA (Figure 2), also known as a decision tree model. RPA is a step-by-step process by which a decision tree is constructed by either splitting or not splitting each node on the tree into 2 daughter nodes.¹⁰ By using this method, we successfully generated an ideal approach to predict true bacteremia using food consumption and shaking chills. After adjusting for food consumption and shaking chills, groups 1 to 2 had sequentially decreasing diagnoses of true bacteremia, varying from 221 patients to only 13 patients.

Appetite is influenced by many factors that are integrated by the brain, most importantly within the hypothalamus. Signals that impinge on the hypothalamic center include neural afferents, hormones, cytokines, and metabolites.¹¹ These factors elicit “sickness behavior,” which includes a decrease in food-motivated behavior.¹² Furthermore, exposure to pathogenic bacteria increases serotonin, which has been shown to decrease metabolism in amphid neurons by transcriptional and post-transcriptional mechanisms.¹³ Therefore, nonbacteremic patients retain their appetites. Shaking chills are a well-known predictor of true bacteremia.^4,5 Several cytokines, including tumor necrosis factor-alpha and interleukins 6 and 10, may be related to shaking chills.¹⁴ Coburn et al. reviewed that shaking chills appear to be useful for identifying true bacteremia (positive LR, 4.7; 95% CI, 3.0-7.2),^5,6 similar to our study. In our study, the pretest probability of true bacteremia was the same whether shaking chills was included or not (ie, 2.4% for normal food consumption and 2.4% for normal food consumption plus absence of shaking chills). This would seem to imply that the assessment of shaking chills does not appear to add anything over food assessment alone when trying to rule out bacteremia. Rather, shaking chills seem more important for ruling in bacteremia rather than ruling it out. Moreover, the recent retrospective study revealed that age >60 years (OR = 2.75, 95% CI, 1.23-6.48, P = .015), female sex (OR = 2.21, 95% CI, 1.07- 4.67, P = .038), heart rate >90 bpm (OR = 5.18, 95% CI, 2.25-12.48, P < .001) and neutrophil percentage >80% (OR = 3.61, 95% CI, 1.71- 8.00, P = .001) were independent risk factors for true bacteremia.¹⁵ Conversely, the lack of the SIRS criteria was reported as the best negative indicator of true bacteremia with a negative LR of 0.09 (95% CI, 0.03-0.3).^6,7 However, the evaluation of SIRS criteria requires the acquisition of laboratory data. To our knowledge, no previous prospective studies have evaluated food consumption in terms of a risk prediction for true bacteremia. This extremely simple model can enable a physician to make a rapid bedside estimation of the risk of true bacteremia.

The strengths of this study include its relatively large sample size, multicenter design, uniformity of data collection across sites, and completeness of data collection from study participants. All of these factors allowed for a robust analysis.

However, there are several limitations of this study. First, the physicians or nurses asked the patients about the presence of shaking chills when they obtained the BCs. It may be difficult for patients, especially elderly patients, to provide this information promptly and accurately. Some patients did not call the nurse when they had shaking chills, and the chills were not witnessed by a healthcare provider. However, we used a more specific definition for shaking chills: a feeling of being extremely cold with rigors and generalized bodily shaking, even under a thick blanket. Second, this algorithm is not applicable to patients with immunosuppressed states because none of the hospitals involved in this study perform bone marrow or organ transplantation. Third, although we included patients with dementia in our cohort, we did not specifically evaluate performance of the algorithm in patients with this medical condition. It is possible that the algorithm would not perform well in this subset of patients owing to their unreliable appetite and food intake. Fourth, some medications may affect appetite, leading to reduced food consumption. Although we have not considered the details of medications in this study, we found that the pretest probability of true bacteremia was low for those patients with normal food consumption regardless of whether the medication affected their appetites or not. However, the question of whether medications truly affect patients’ appetites concurrently with bacteremia would need to be specifically addressed in a future study.

CONCLUSION

In conclusion, we have established a simple algorithm to identify patients with suspected true bacteremia who require the acquisition of blood cultures. This extremely simple model can enable physicians to make a rapid bedside estimation of the risk of true bacteremia.

Acknowledgment

The authors thank Drs. H. Honda and S. Saint, and Ms. A. Okada for their helpful discussions with regard to this study; Ms. M. Takigawa for the collection of data; and Ms. T. Oguri for providing infectious disease consultation on the pathogenicity of the identified organisms.

Disclosure

This work was supported by JSPS KAKENHI Grant Number 15K19294 (to TK) and 20590840 (to KI) from the Japan Society for the Promotion of Science. The authors report no potential conflicts of interest relevant to this article.

Fever in hospitalized patients is a nonspecific finding with many potential causes. Blood cultures (BC) are commonly obtained prior to commencing parenteral antibiotics in febrile patients. However, as many as 35% to 50% of positive BCs represent a contamination with organisms inoculated from the skin into culture bottles at the time of sample collection.^1-3 Such results represent false-positive BCs that can lead to unnecessary investigations and treatment.

Recently, Coburn et al. reviewed the severity of chills (graded on an ordinal scale) as the most useful predictor of true bacteremia (positive likelihood ratio [LR], 4.7; 95% confidence interval [CI], 3.0–7.2),^4-6 and the lack of the systemic inflammatory response syndrome (SIRS) criteria as the best negative indicator of true bacteremia with a negative LR of 0.09 (95% CI, 0.03-0.3).^6,7 We have also previously reported normal food consumption as a negative indicator of true bacteremia, with a 98.3% negative predictive value.⁸ Henderson’s Basic Principles of Nursing Care emphasizes the importance of evaluating whether a patient can eat and drink adequately,⁹ and the evaluation of a patient’s food consumption is a routine nursing staff practice, which is treated as vital sign in Japan, in contrast to nursing practices in the United States.

However, these data were the result of a single-center retrospective study using the nursing staff’s assessment of food consumption, and they cannot be generalized to larger patient populations. Therefore, the aim of this prospective, multicenter study was to measure the accuracy of food consumption and shaking chills as predictive factors for true bacteremia.

Study Design

This was a prospective multicenter observational study (UMIN ID: R000013768) involving 3 hospitals in Tokyo, Japan, that enrolled consecutive patients who had BCs obtained. This study was approved by the ethical committee at Juntendo University Nerima Hospital and each of the participating centers, and the study was conducted in accordance with the Declaration of Helsinki 1971, as revised in 1983. We evaluated 2,792 consecutive hospitalized patients (mean age, 68.9 ± 17.1 years; 55.3% men) who had BCs obtained between April 2013 and August 2014, inclusive. The indication for BC acquisition was at the discretion of the treating physician. The study protocol and the indication for BCs are described in detail elsewhere.⁸ We excluded patients with anorexia-inducing conditions such as gastrointestinal disease, including gastrointestinal bleeding, enterocolitis, gastric ulceration, peritonitis, appendicitis, cholangitis, pancreatitis, diverticulitis, and ischemic colitis. We also excluded patients receiving chemotherapy for malignancy. In this study, true bacteremia was defined as identical organisms isolated from 2 sets of blood cultures (a set refers to one aerobic bottle and one anaerobic bottle). Moreover, even if only one set of blood cultures was acquired, when the identified pathogen could account for the clinical presentation, we also defined this as true bacteremia. Briefly, contaminants were defined as organisms common to skin flora, including Bacillus species, coagulase-negative Staphylococcus, Corynebacterium species, and Micrococcus species, without isolation of an identical organism with the same antibiotic susceptibilities from another potentially infected site in a patient with incompatible clinical features and no risk factors for infection with the isolated organism. Single BCs that were positive for organisms that were unlikely to explain the patient’s symptoms were also considered as contaminants. Patients with contaminated BCs were excluded from the analyses.

Structure of Reliability Study Procedures

Nurses in the 3 different hospitals performed daily independent food consumption ratings during each patient’s stay. Interrater reliability assessments were conducted in the morning or afternoon, and none of the raters had access to the other nurses’ scores at any time. The study nurses performed simultaneous ratings during these assessments (one interacted with and rated the patient while the other observed and rated the same patient).

Prediction Variables of True Bacteremia


1. Food consumption. Assessment of food consumption has been previously described in detail.⁸ Briefly, we characterized the patients’ oral intake based on the meal taken immediately prior to the BCs. For example, if a fever developed at 2 pm, lunch consumption was evaluated. If a fever developed at 2 am, dinner consumption was evaluated. We categorized the patients into 3 groups: low food consumption (<50% consumed), moderate food consumption (>50% to <80% consumed), and high food consumption (>80% consumed). To simplify our prediction rule, we subsequently divided food consumption into just 2 groups: high food consumption, referred to as the “normal food consumption group,” and the combination of low and moderate food consumption, referred to as the “poor food consumption group.”

2. Chills. As done previously, the physician evaluated the patient for a history of chills at the time of BCs and classified the patients into 1 of 4 grades^4,5: “no chills,” the absence of any chills; “mild chills,” feeling cold, equivalent to needing an outer jacket; “moderate chills,” feeling very cold, equivalent to needing a thick blanket; and “shaking chills,” feeling extremely cold with rigors and generalized bodily shaking, even under a thick blanket. To distinguish between those patients who had shaking chills and those who did not, we divided the patients into 2 groups: the “shaking chills group” and the combination of none, mild, and moderate chills, referred to as the “negative shaking chills group.”

3. Other predictive variables. We considered the following additional predictive variables: age, gender, axillary body temperature (BT), heart rate (HR), systolic blood pressure (SBP), respiratory rate (RR), white blood cell count (WBC), and serum C-reactive protein level (CRP). These predictive variables were obtained immediately prior to the BCs. We defined SIRS based on standard criteria (HR >90 beats/m, RR >20/m, BT <36°C or >38°C, and a WBC <4 × 10³ WBC/μL or >12 × 10³ WBC/μL). Patients were subcategorized by age into 2 groups (≤69 years and >70 years). CRP levels were dichotomized as >10.0 mg/dL or ≤10.0 mg/dL. We reviewed the patients’ charts to determine whether they had received antibiotics. In the case of walk-in patients, we interviewed the patients regarding whether they had visited a clinic; if they had, they were questioned as to whether any antibiotic therapy had been prescribed.

Statistical Analysis

Continuous variables are presented as the mean with the associated standard deviation (SD). All potential variables predictive of true bacteremia are shown in Table 1. The variables were dichotomized by clinically meaningful thresholds and used as potential risk-adjusted variables. We calculated the sensitivity and specificity and positive and negative predictive value for each criterion. Multiple logistic regression analysis was used to select components that were significantly associated with true bacteremia (the level of statistical significance determined with maximum likelihood methods was set at P < .05). To visualize and quantify other aspects in the prediction of true bacteremia, a recursive partitioning analysis (RPA) was used to make a decision tree model for true bacteremia. This nonparametric regression method produces a classification tree following a series of nonsequential top-down binary splits. The tree-building process starts by considering a set of predictive variables and selects the variable that produces 2 subsets of participants with the greatest purity. Two factors are considered when splitting a node into its daughter nodes: the goodness of the split and the amount of impurity in the daughter nodes. The splitting process is repeated until further partitioning is no longer possible and the terminal nodes have been reached. Details on this method are discussed in Monte Carlo Calibration of Distributions of Partition Statistics (www.jmp.com).

Probability was considered significant at a value of P < .05. All statistical tests were 2-tailed. Statistical analyses were conducted by a physician (KI) and an independent statistician (JM) with the use of the SPSS^® v.16.0 software package (SPSS Inc., Chicago, IL) and JMP^® version 8.0.2 (SAS Institute, Cary, NC).

Patients Characteristics

Two thousand seven hundred and ninety-two patients met the inclusion criteria for our study, from which 849 were excluded (see Figure 1 for flow diagram). Among the remaining 1,943 patients, there were 317 patients with positive BCs, of which 221 patients (69.7%) were considered to have true-positive BCs and 96 (30.3%) were considered to have contaminated BCs. After excluding these 96 patients, 221 patients with true bacteremia (true bacteremic group) were compared with 1,626 nonbacteremic patients (nonbacteremic group; Figure 1). The baseline characteristics of the subjects are shown in Table 1. The mean BT was 38.4 ± 1.2°C in the true bacteremic group and 37.9 ± 1.0°C in the nonbacteremic group. The mean serum CRP level was 11.6 ± 9.6 mg/dL in the true bacteremic group and 7.3 ± 6.9 mg/dL in the nonbacteremic group. In the true bacteremic group, there were 6 afebrile patients, and 27 patients without leukocytosis. The pathogens identified from the true-positive BCs were Escherichia coli (n = 59, 26.7%), including extended-spectrum beta-lactamase producing species, Staphylococcus aureus (n = 36, 16.3%), including methicillin-resistant Staphylococcus aureus, and Klebsiella pneumoniae (n = 22, 10.0%; Supplemental Table 1).

The underlying clinical diagnoses in the true bacteremic group included urinary tract infection (UTI), pneumonia, abscess, catheter-related bloodstream infection (CRBSI), cellulitis, osteomyelitis, infective endocarditis (IE), chorioamnionitis, iatrogenic infection at hemodialysis puncture sites, bacterial meningitis, septic arthritis, and infection of unknown cause (Supplemental Table 2).

Interrater Reliability Testing of Food Consumption

Patients were evaluated during their hospital stays. The interrater reliability of the evaluation of food consumption was very high across all participating hospitals (Supplemental Table 3). To assess the reliability of the evaluations of food consumption, patients (separate from this main study) were selected randomly and evaluated independently by 2 nurses in 3 different hospitals. The kappa scores of agreement between the nurses at the 3 different hospitals were 0.83 (95% CI, 0.63-0.88), 0.90 (95% CI, 0.80-0.99), and 0.80 (95% CI, 0.67-0.99), respectively. The interrater reliability of food consumption evaluation by the nurses was very high at all participating hospitals.

Food Consumption

The low, moderate, and high food consumption groups consisted of 964 (52.1%), 306 (16.6%), and 577 (31.2%) patients, respectively (Table 1). Of these, 174 (18.0%), 33 (10.8%), and 14 (2.4%) patients, respectively, had true bacteremia. The presence of poor food consumption had a sensitivity of 93.7% (95% CI, 89.4%-97.9%), specificity of 34.6% (95% CI, 33.0%-36.2%), and a positive LR of 1.43 (95% CI, 1.37-1.50) for predicting true bacteremia. Conversely, the absence of poor food consumption (ie, normal food consumption) had a negative LR of 0.18 (95% CI, 0.17-0.19).

Chills

The no, mild, moderate, and shaking chills groups consisted of 1,514 (82.0%), 148 (8.0%), 53 (2.9%), and 132 (7.1%) patients, respectively (Table 1). Of these, 136 (9.0%), 25 (16.9%), 8 (15.1%), and 52 (39.4%) patients, respectively, had true bacteremia. The presence of shaking chills had a sensitivity of 23.5% (95% CI, 22.5%-24.6%), a specificity of 95.1% (95% CI, 90.7%-99.4%), and a positive LR of 4.78 (95% CI, 4.56–5.00) for predicting true bacteremia. Conversely, the absence of shaking chills had a negative LR of 0.80 (95% CI, 0.77-0.84).

Prediction Model for True Bacteremia

The components identified as significantly related to true bacteremia by multiple logistic regression analysis are indicated in Table 2. The significant predictors of true bacteremia were shaking chills (odds ratio [OR], 5.6; 95% CI, 3.6-8.6; P < .01), SBP <90 mmHg (OR, 3.1; 95% CI, 1.6-5.7; P < 01), CRP levels >10.0 mg/dL (OR, 2.2; 95% CI, 1.6-3.1; P < .01), BT <36°C or >38°C (OR, 1.8; 95% CI, 1.3-2.6; P < .01), WBC <4 × 10³/μL or >12 × 10³/μL (OR, 1.6; 95% CI, 1.2-2.3; P = .003), HR >90 bpm (OR, 1.5; 95% CI, 1.1-2.1; P = .021), and female (OR, 1.4; 95% CI, 1.0-1.9; P = .036). An RPA to create an ideal prediction model for patients with true bacteremia or nonbacteremia is shown in Figure 2. The original group consisted of 1,847 patients, including 221 patients with true bacteremia. The pretest probability of true bacteremia was 2.4% (14/577) for those with normal food consumption (Group 1) and 2.4% (13/552) for those with both normal food consumption and the absence of shaking chills (Group 2). Conversely, the pretest probability of true bacteremia was 16.3% (207/1270) for those with poor food consumption and 47.7% (51/107) for those with both poor food consumption and shaking chills. The patients with true bacteremia with normal food consumption and without shaking chills consisted of 4 cases of CRBSI and UTI, 2 cases of osteomyelitis, 1 case of IE, 1 case of chorioamnionitis, and 1 case for which the focus was unknown (Supplemental Table 4).

In this observational study, we evaluated if a simple algorithm using food consumption and shaking chills was useful for assessing whether a patient had true bacteremia. A 2-item screening checklist (nursing assessment of food consumption and shaking chills) had excellent statistical properties as a brief screening instrument for true bacteremia.

We have prospectively validated that food consumption, as assessed by nurses, is a reliable predictor of true bacteremia.⁸ A previous single-center retrospective study showed similar findings, but these could not be generalized across all institutions because of the limited nature of the study. In this multicenter study, we used 2 statistical methods to reduce selection bias. First, we performed a kappa analysis across the hospitals to evaluate the interrater reliability of the evaluation of food consumption. Second, we used an RPA (Figure 2), also known as a decision tree model. RPA is a step-by-step process by which a decision tree is constructed by either splitting or not splitting each node on the tree into 2 daughter nodes.¹⁰ By using this method, we successfully generated an ideal approach to predict true bacteremia using food consumption and shaking chills. After adjusting for food consumption and shaking chills, groups 1 to 2 had sequentially decreasing diagnoses of true bacteremia, varying from 221 patients to only 13 patients.

Appetite is influenced by many factors that are integrated by the brain, most importantly within the hypothalamus. Signals that impinge on the hypothalamic center include neural afferents, hormones, cytokines, and metabolites.¹¹ These factors elicit “sickness behavior,” which includes a decrease in food-motivated behavior.¹² Furthermore, exposure to pathogenic bacteria increases serotonin, which has been shown to decrease metabolism in amphid neurons by transcriptional and post-transcriptional mechanisms.¹³ Therefore, nonbacteremic patients retain their appetites. Shaking chills are a well-known predictor of true bacteremia.^4,5 Several cytokines, including tumor necrosis factor-alpha and interleukins 6 and 10, may be related to shaking chills.¹⁴ Coburn et al. reviewed that shaking chills appear to be useful for identifying true bacteremia (positive LR, 4.7; 95% CI, 3.0-7.2),^5,6 similar to our study. In our study, the pretest probability of true bacteremia was the same whether shaking chills was included or not (ie, 2.4% for normal food consumption and 2.4% for normal food consumption plus absence of shaking chills). This would seem to imply that the assessment of shaking chills does not appear to add anything over food assessment alone when trying to rule out bacteremia. Rather, shaking chills seem more important for ruling in bacteremia rather than ruling it out. Moreover, the recent retrospective study revealed that age >60 years (OR = 2.75, 95% CI, 1.23-6.48, P = .015), female sex (OR = 2.21, 95% CI, 1.07- 4.67, P = .038), heart rate >90 bpm (OR = 5.18, 95% CI, 2.25-12.48, P < .001) and neutrophil percentage >80% (OR = 3.61, 95% CI, 1.71- 8.00, P = .001) were independent risk factors for true bacteremia.¹⁵ Conversely, the lack of the SIRS criteria was reported as the best negative indicator of true bacteremia with a negative LR of 0.09 (95% CI, 0.03-0.3).^6,7 However, the evaluation of SIRS criteria requires the acquisition of laboratory data. To our knowledge, no previous prospective studies have evaluated food consumption in terms of a risk prediction for true bacteremia. This extremely simple model can enable a physician to make a rapid bedside estimation of the risk of true bacteremia.

The strengths of this study include its relatively large sample size, multicenter design, uniformity of data collection across sites, and completeness of data collection from study participants. All of these factors allowed for a robust analysis.

However, there are several limitations of this study. First, the physicians or nurses asked the patients about the presence of shaking chills when they obtained the BCs. It may be difficult for patients, especially elderly patients, to provide this information promptly and accurately. Some patients did not call the nurse when they had shaking chills, and the chills were not witnessed by a healthcare provider. However, we used a more specific definition for shaking chills: a feeling of being extremely cold with rigors and generalized bodily shaking, even under a thick blanket. Second, this algorithm is not applicable to patients with immunosuppressed states because none of the hospitals involved in this study perform bone marrow or organ transplantation. Third, although we included patients with dementia in our cohort, we did not specifically evaluate performance of the algorithm in patients with this medical condition. It is possible that the algorithm would not perform well in this subset of patients owing to their unreliable appetite and food intake. Fourth, some medications may affect appetite, leading to reduced food consumption. Although we have not considered the details of medications in this study, we found that the pretest probability of true bacteremia was low for those patients with normal food consumption regardless of whether the medication affected their appetites or not. However, the question of whether medications truly affect patients’ appetites concurrently with bacteremia would need to be specifically addressed in a future study.

CONCLUSION

In conclusion, we have established a simple algorithm to identify patients with suspected true bacteremia who require the acquisition of blood cultures. This extremely simple model can enable physicians to make a rapid bedside estimation of the risk of true bacteremia.

Acknowledgment

The authors thank Drs. H. Honda and S. Saint, and Ms. A. Okada for their helpful discussions with regard to this study; Ms. M. Takigawa for the collection of data; and Ms. T. Oguri for providing infectious disease consultation on the pathogenicity of the identified organisms.

Disclosure

This work was supported by JSPS KAKENHI Grant Number 15K19294 (to TK) and 20590840 (to KI) from the Japan Society for the Promotion of Science. The authors report no potential conflicts of interest relevant to this article.

Implantable cardioverter-defibrillators (ICDs) offer lifesaving therapies to many patients and have been implanted in hundreds of thousands of patients.¹ The population of patients with ICDs is growing rapidly, and the national ICD Registry reports over 12,000 devices are implanted monthly.² This population includes patients with congenital heart disease, ischemic cardiomyopathy, and idiopathic arrhythmias. If these patients experience ventricular tachycardia or fibrillation, an ICD attempts to restore sinus rhythm and prevent death. While a shock from an ICD may be lifesaving, it can be a traumatic and startling experience for the patient and perhaps distressful for families to witness.^3,4

Although ICDs are intended to save lives, they do not slow the progress of the patient’s underlying cardiac and noncardiac comorbidities. All these patients will eventually die, whether from their cardiac disease or another condition. The literature includes many anecdotes about patients shocked multiple times by their defibrillator while actively dying.⁴ These situations could be prevented with preemptive ICD deactivation. (ICDs can function not only as cardioverters and defibrillators, as implied by their name, but also as pacemakers. “Deactivation” as used in this paper refers only to disabling the tachycardia therapies. No distinction was made between defibrillation with a shock and anti-tachycardia pacing.) Therefore, research on ICD deactivation has emphasized patients who are acutely terminally ill, while less emphasis has been placed on patients who are not actively dying.^4–8

Patients may, for a variety of reasons, request a do-not-resuscitate/do-not-intubate (DNR/DNI) order as their code status. However, it is not necessarily clear what a DNR/DNI order implies for ICD management. One survey of attending physicians found that 19% of respondents felt a DNR/DNI order was equivalent to requesting ICD deactivation.⁹ On the other hand, patients are split on whether they would want their device deactivated while in hospice or even at the very end of life.⁶ Heart Rhythm Society (HRS) guidelines favor a nuanced approach to ICD deactivation in DNR/DNI patients that emphasizes the individual patient’s comorbidities and goals.¹⁰ A patient’s individual circumstances might justify a choice to be DNR/DNI without deactivating the ICD. Decision-making in these circumstances requires a careful conversation between the patient and clinician. It is important to identify barriers that might prevent optimal shared decision-making.

Clinicians have been surveyed on ICD management at the end of life, but these studies have generally focused on attending physicians.^5,9,11 However, physician trainees (ie, residents and fellows) as well as advanced practice providers (ie, physician assistants and nurse practitioners) are responsible for much of the clinical care provided to hospitalized patients. In particular, they are often the clinicians to discuss code status with patients. Different specialties (eg, cardiology, general medicine, and geriatrics) manage different sets of patients, which might affect clinicians’ opinions on ICD management. We therefore designed a survey to assess clinician attitudes and beliefs regarding ICD deactivation, including in non-terminally ill patients, and to evaluate for differences according to training level and specialty.

Case-based and Likert-scale questions were considered for this survey, with the latter being chosen for ease of completion by respondents. An online survey tool (SurveyMonkey; San Mateo, CA) was used for data collection; no identifying data were collected. E-mail invitations to participate were sent to a combination of mailing lists and individual addresses for residents, fellows, advanced practice providers, and attending physicians in general internal medicine, cardiology, electrophysiology, and geriatrics. The survey remained open for 2.5 weeks. It was conducted 5 months into the academic year, thus trainees were well-established in their current roles. Two $25 gift cards were offered to respondents who entered their e-mail into a drawing; responses were not tied to e-mail addresses. Approval for the study was obtained from the University of Michigan Institutional Review Board.

The survey posed 12 questions assessing general attitudes about ICDs as well as individual beliefs and behaviors relating to ICD deactivation. Answers were on a Likert scale of 1 to 5 with 1 representing “strongly disagree” and 5 representing “strongly agree.” A score of 3 indicated “unsure or neutral.” The first 3 questions appeared together on the first page and were prefaced with “Please respond to the following statements about ICD shocks.” The next 9 were likewise grouped on the next page and were prefaced with “Please respond to the following statements about ICD deactivation.” All 12 questions are shown in Figures 1 and 2. Respondents could easily return to previous questions and change their answers. The survey ended with a third page showing 3 multiple choice demographic questions. The demographic questions were about clinical role (first-, second-, third-, or fourth-year resident, fellow, advanced practice provider, and attending), specialty, and number of ICD deactivations the respondent had been directly involved in (0, 1 to 5, 5 to 10, and more than 10). Specialty options were internal medicine resident, inpatient general medicine, outpatient general medicine, cardiology, electrophysiology, and geriatrics.

Likert scale answers of “agree” or “strongly agree” were grouped together as an affirmative response, while all other answers were grouped together as a nonaffirmative response. For analysis, residents were grouped together and their responses compared with attending physicians as a group. Additional analysis was done comparing attending physicians stratified by clinical specialty. Given the small number of responses from attending electrophysiologists, they were grouped with attending cardiologists for analysis. Due to the limited number of fellows and advanced practice providers who responded, further evaluation of these groups was not performed. Finally, the number of ICD deactivations respondents had been involved in was stratified by training level. All comparisons were performed using the two-tailed Pearson’s chi-squared test.

A total of 170 responses were collected from 508 individuals on the e-mail lists. Two responses were from registered nurses who were not part of the intended study sample and 7 responses were incomplete, having only answered the first 3 questions. These 9 responses were excluded from further analysis, yielding an overall response rate of 32%. The demographics of the remaining 161 respondents are shown in Table 1. Figure 1 shows overall responses to each question.

When comparing residents to attending physicians, there were no statistically significant between-differences except on questions 5 and 6. Specifically, residents were less comfortable than attending physicians discussing ICD deactivation and did so with less regularity (P < .001 and P = .018, respectively; Figure 2). Comfort levels improved markedly with experience: 29.2% of interns expressed comfort asking about ICD deactivation as compared with 60.7% of third- and-fourth year residents and 78.8% of attending physicians. Furthermore, comfort level seemed to parallel the regularity with which respondents asked about ICD deactivation: 4.2% of interns routinely asked about ICD deactivation as compared with 21.4% of third- and fourth-year residents and 34.8% of attending physicians.

Overall, our major findings were (1) residents, who provide much of the clinical care in a teaching hospital, are remarkably uncomfortable discussing ICD deactivation, (2) general internists and residents ask about ICD deactivation infrequently compared to geriatricians and cardiologists, and (3) about one fifth of our respondents believe ICD deactivation is automatically part of a DNR/DNI order.

Although the majority of respondents did not routinely address ICD deactivation in conjunction with code status, there was significant variability among subgroups. For example, 83.3% of geriatricians routinely discussed ICD deactivation as part of code status compared with only 4% of first-year residents and 10.5% of inpatient general internists. This finding is interesting because 90.7% of all respondents believed that discussions of code status should address preferences on ICD deactivation. This apparent discrepancy could be explained by the relatively small number of patients admitted to the hospital who have both an ICD and a request to be DNR/DNI. Residents and inpatient general internists see a very broad spectrum of patients; ICD deactivation is frequently irrelevant in the cases these physicians manage. The subset of patients seen in consultation by cardiologists and geriatricians, by contrast, is expected to include a larger proportion of patients with ICDs. Therefore, discussing ICD deactivation will be more relevant to their daily practice. Fear of alienating patients was not a reason for our findings, as our respondents did not express concern that recommending ICD deactivation would harm the patient-clinician relationship.

There are several possible reasons that residents, particularly interns, are uncomfortable discussing ICD deactivation. A lack of exposure to ICD deactivation is probably a key contributor. Over half of interns had never been involved in any ICD deactivations. Residents and hospitalists may also feel as if they are overstepping their boundaries to discuss deactivating ICD therapies. Their feelings may not be misplaced, as one survey of ICD patients found that over 75% thought responsibility for discussing ICD deactivation, at least at the end of life, rests with cardiologists or electrophysiologists.⁶

The HRS guidelines call for individualized decisions regarding ICD deactivation, even if a patient is DNR/DNI. However, our respondents frequently felt a standardized approach was indicated, with 21% believing that a DNR/DNI order included ICD deactivation. Additionally, 28% agreed that even non-terminally ill DNR/DNI patients should have their device deactivated. This is relevant because it is the role of clinicians to engage in shared decision-making with their patients. If the clinician holds the fixed belief that a DNR/DNI order, regardless of the precise clinical scenario, should include ICD deactivation, they may pressure a patient to have their device deactivated even if it could still benefit them.

In 2009, Kelley et al published results of a survey on ICD deactivation at the end of life.⁹ They contacted 4,876 attending physicians in cardiology, electrophysiology, geriatrics, and general medicine, receiving 558 responses. The survey included Likert-scale questions assessing attitudes and knowledge about ICD functionality. Demographic information was also collected, including how many patients in their practice had ICDs and how often they had previously discussed ICD deactivation.

There are some interesting comparisons between Kelley et al’s findings and ours, although we included trainees and the precise wording of questions was different. The specific questions used by Kelley et al to ask whether ICD shocks were painful or distressing and to ask if ICD deactivation is part of a DNR order were: “The shock from an ICD is very painful for most patients.” “The shock of an ICD at the end of life is distressing to a patient and their loved ones.” “A DNR order is equivalent to deactivation of an ICD.”

Only 47% of general internists in the Kelley et al survey thought that ICD shocks were painful, compared with 83% of electrophysiologists. In addition, 65% of general internists and 85% of electrophysiologists viewed shocks at the end of life to be distressing to patients and families. By contrast, our respondents were nearly unanimous in believing shocks to be painful and distressing. This discrepancy may be due to the growing prevalence of ICDs over the past several years as well as the growing body of literature on unnecessary shocks at the end of life. In line with our study, 19% of their respondents believed a DNR order was equivalent to ICD deactivation.⁹

Taken together, our findings indicate that additional education for clinicians of all levels could be helpful. Didactic lessons cannot replace experience, and it is important for residents to be exposed to discussions of ICD deactivation. However, lessons about ICD therapies and practical examples of how to broach the topic of deactivation could be beneficial, especially for interns whose responsibility includes discussions of code status. Within the context of an internal medicine residency, the fundamentals of ICD functionality could be covered during rotations on cardiology or palliative care services. Additionally, the recommendations of the HRS for device management can be covered in didactic sessions. Similar opportunities could be built into continuing medical education for practicing physicians and the training of advanced practice providers.

There are limitations to this survey, most notably the fact that it was restricted to a single academic medical center, the patient population and practices of which may not be generalizable to medical practice at large. Selection bias is also a distinct possibility given the 32% overall response rate; those who responded may feel more strongly about the survey topic. Our study subgroups may have interpreted questions differently because of their particular area of clinical practice. The small sample size also precluded an effective analysis of fellows and advanced practice practitioners due to lack of power. A major strength of this survey was the inclusion of a large number of residents upon whom the majority of inpatient contact rests. Future work could include expanding the survey to multiple medical centers, which would enhance generalizability and improve the ability to recruit sufficient fellows and advanced practice providers.

In summary, we conducted a single-center survey of residents, fellows, advanced practice providers, and attending physicians on their attitudes and beliefs about ICD deactivation in DNR/DNI patients. Residents are particularly uncomfortable discussing ICD deactivation with patients, which is an important finding because of their crucial role in providing patient care. Additionally, residents and hospitalists do not broach the topic of deactivation regularly, especially when compared to geriatricians and cardiologists. Despite HRS guidelines to the contrary, a fifth of our respondents believed that DNR/DNI orders include ICD deactivation. Overall, ICD deactivation in DNR/DNI patients is a topic that needs further attention in clinical education so that patients receive care that respects their individual wishes.

Disclosure

Nothing to report.

Implantable cardioverter-defibrillators (ICDs) offer lifesaving therapies to many patients and have been implanted in hundreds of thousands of patients.¹ The population of patients with ICDs is growing rapidly, and the national ICD Registry reports over 12,000 devices are implanted monthly.² This population includes patients with congenital heart disease, ischemic cardiomyopathy, and idiopathic arrhythmias. If these patients experience ventricular tachycardia or fibrillation, an ICD attempts to restore sinus rhythm and prevent death. While a shock from an ICD may be lifesaving, it can be a traumatic and startling experience for the patient and perhaps distressful for families to witness.^3,4

Although ICDs are intended to save lives, they do not slow the progress of the patient’s underlying cardiac and noncardiac comorbidities. All these patients will eventually die, whether from their cardiac disease or another condition. The literature includes many anecdotes about patients shocked multiple times by their defibrillator while actively dying.⁴ These situations could be prevented with preemptive ICD deactivation. (ICDs can function not only as cardioverters and defibrillators, as implied by their name, but also as pacemakers. “Deactivation” as used in this paper refers only to disabling the tachycardia therapies. No distinction was made between defibrillation with a shock and anti-tachycardia pacing.) Therefore, research on ICD deactivation has emphasized patients who are acutely terminally ill, while less emphasis has been placed on patients who are not actively dying.^4–8

Patients may, for a variety of reasons, request a do-not-resuscitate/do-not-intubate (DNR/DNI) order as their code status. However, it is not necessarily clear what a DNR/DNI order implies for ICD management. One survey of attending physicians found that 19% of respondents felt a DNR/DNI order was equivalent to requesting ICD deactivation.⁹ On the other hand, patients are split on whether they would want their device deactivated while in hospice or even at the very end of life.⁶ Heart Rhythm Society (HRS) guidelines favor a nuanced approach to ICD deactivation in DNR/DNI patients that emphasizes the individual patient’s comorbidities and goals.¹⁰ A patient’s individual circumstances might justify a choice to be DNR/DNI without deactivating the ICD. Decision-making in these circumstances requires a careful conversation between the patient and clinician. It is important to identify barriers that might prevent optimal shared decision-making.

Clinicians have been surveyed on ICD management at the end of life, but these studies have generally focused on attending physicians.^5,9,11 However, physician trainees (ie, residents and fellows) as well as advanced practice providers (ie, physician assistants and nurse practitioners) are responsible for much of the clinical care provided to hospitalized patients. In particular, they are often the clinicians to discuss code status with patients. Different specialties (eg, cardiology, general medicine, and geriatrics) manage different sets of patients, which might affect clinicians’ opinions on ICD management. We therefore designed a survey to assess clinician attitudes and beliefs regarding ICD deactivation, including in non-terminally ill patients, and to evaluate for differences according to training level and specialty.

Case-based and Likert-scale questions were considered for this survey, with the latter being chosen for ease of completion by respondents. An online survey tool (SurveyMonkey; San Mateo, CA) was used for data collection; no identifying data were collected. E-mail invitations to participate were sent to a combination of mailing lists and individual addresses for residents, fellows, advanced practice providers, and attending physicians in general internal medicine, cardiology, electrophysiology, and geriatrics. The survey remained open for 2.5 weeks. It was conducted 5 months into the academic year, thus trainees were well-established in their current roles. Two $25 gift cards were offered to respondents who entered their e-mail into a drawing; responses were not tied to e-mail addresses. Approval for the study was obtained from the University of Michigan Institutional Review Board.

The survey posed 12 questions assessing general attitudes about ICDs as well as individual beliefs and behaviors relating to ICD deactivation. Answers were on a Likert scale of 1 to 5 with 1 representing “strongly disagree” and 5 representing “strongly agree.” A score of 3 indicated “unsure or neutral.” The first 3 questions appeared together on the first page and were prefaced with “Please respond to the following statements about ICD shocks.” The next 9 were likewise grouped on the next page and were prefaced with “Please respond to the following statements about ICD deactivation.” All 12 questions are shown in Figures 1 and 2. Respondents could easily return to previous questions and change their answers. The survey ended with a third page showing 3 multiple choice demographic questions. The demographic questions were about clinical role (first-, second-, third-, or fourth-year resident, fellow, advanced practice provider, and attending), specialty, and number of ICD deactivations the respondent had been directly involved in (0, 1 to 5, 5 to 10, and more than 10). Specialty options were internal medicine resident, inpatient general medicine, outpatient general medicine, cardiology, electrophysiology, and geriatrics.

Likert scale answers of “agree” or “strongly agree” were grouped together as an affirmative response, while all other answers were grouped together as a nonaffirmative response. For analysis, residents were grouped together and their responses compared with attending physicians as a group. Additional analysis was done comparing attending physicians stratified by clinical specialty. Given the small number of responses from attending electrophysiologists, they were grouped with attending cardiologists for analysis. Due to the limited number of fellows and advanced practice providers who responded, further evaluation of these groups was not performed. Finally, the number of ICD deactivations respondents had been involved in was stratified by training level. All comparisons were performed using the two-tailed Pearson’s chi-squared test.

A total of 170 responses were collected from 508 individuals on the e-mail lists. Two responses were from registered nurses who were not part of the intended study sample and 7 responses were incomplete, having only answered the first 3 questions. These 9 responses were excluded from further analysis, yielding an overall response rate of 32%. The demographics of the remaining 161 respondents are shown in Table 1. Figure 1 shows overall responses to each question.

When comparing residents to attending physicians, there were no statistically significant between-differences except on questions 5 and 6. Specifically, residents were less comfortable than attending physicians discussing ICD deactivation and did so with less regularity (P < .001 and P = .018, respectively; Figure 2). Comfort levels improved markedly with experience: 29.2% of interns expressed comfort asking about ICD deactivation as compared with 60.7% of third- and-fourth year residents and 78.8% of attending physicians. Furthermore, comfort level seemed to parallel the regularity with which respondents asked about ICD deactivation: 4.2% of interns routinely asked about ICD deactivation as compared with 21.4% of third- and fourth-year residents and 34.8% of attending physicians.

Overall, our major findings were (1) residents, who provide much of the clinical care in a teaching hospital, are remarkably uncomfortable discussing ICD deactivation, (2) general internists and residents ask about ICD deactivation infrequently compared to geriatricians and cardiologists, and (3) about one fifth of our respondents believe ICD deactivation is automatically part of a DNR/DNI order.

Although the majority of respondents did not routinely address ICD deactivation in conjunction with code status, there was significant variability among subgroups. For example, 83.3% of geriatricians routinely discussed ICD deactivation as part of code status compared with only 4% of first-year residents and 10.5% of inpatient general internists. This finding is interesting because 90.7% of all respondents believed that discussions of code status should address preferences on ICD deactivation. This apparent discrepancy could be explained by the relatively small number of patients admitted to the hospital who have both an ICD and a request to be DNR/DNI. Residents and inpatient general internists see a very broad spectrum of patients; ICD deactivation is frequently irrelevant in the cases these physicians manage. The subset of patients seen in consultation by cardiologists and geriatricians, by contrast, is expected to include a larger proportion of patients with ICDs. Therefore, discussing ICD deactivation will be more relevant to their daily practice. Fear of alienating patients was not a reason for our findings, as our respondents did not express concern that recommending ICD deactivation would harm the patient-clinician relationship.

There are several possible reasons that residents, particularly interns, are uncomfortable discussing ICD deactivation. A lack of exposure to ICD deactivation is probably a key contributor. Over half of interns had never been involved in any ICD deactivations. Residents and hospitalists may also feel as if they are overstepping their boundaries to discuss deactivating ICD therapies. Their feelings may not be misplaced, as one survey of ICD patients found that over 75% thought responsibility for discussing ICD deactivation, at least at the end of life, rests with cardiologists or electrophysiologists.⁶

The HRS guidelines call for individualized decisions regarding ICD deactivation, even if a patient is DNR/DNI. However, our respondents frequently felt a standardized approach was indicated, with 21% believing that a DNR/DNI order included ICD deactivation. Additionally, 28% agreed that even non-terminally ill DNR/DNI patients should have their device deactivated. This is relevant because it is the role of clinicians to engage in shared decision-making with their patients. If the clinician holds the fixed belief that a DNR/DNI order, regardless of the precise clinical scenario, should include ICD deactivation, they may pressure a patient to have their device deactivated even if it could still benefit them.

In 2009, Kelley et al published results of a survey on ICD deactivation at the end of life.⁹ They contacted 4,876 attending physicians in cardiology, electrophysiology, geriatrics, and general medicine, receiving 558 responses. The survey included Likert-scale questions assessing attitudes and knowledge about ICD functionality. Demographic information was also collected, including how many patients in their practice had ICDs and how often they had previously discussed ICD deactivation.

There are some interesting comparisons between Kelley et al’s findings and ours, although we included trainees and the precise wording of questions was different. The specific questions used by Kelley et al to ask whether ICD shocks were painful or distressing and to ask if ICD deactivation is part of a DNR order were: “The shock from an ICD is very painful for most patients.” “The shock of an ICD at the end of life is distressing to a patient and their loved ones.” “A DNR order is equivalent to deactivation of an ICD.”

Only 47% of general internists in the Kelley et al survey thought that ICD shocks were painful, compared with 83% of electrophysiologists. In addition, 65% of general internists and 85% of electrophysiologists viewed shocks at the end of life to be distressing to patients and families. By contrast, our respondents were nearly unanimous in believing shocks to be painful and distressing. This discrepancy may be due to the growing prevalence of ICDs over the past several years as well as the growing body of literature on unnecessary shocks at the end of life. In line with our study, 19% of their respondents believed a DNR order was equivalent to ICD deactivation.⁹

Taken together, our findings indicate that additional education for clinicians of all levels could be helpful. Didactic lessons cannot replace experience, and it is important for residents to be exposed to discussions of ICD deactivation. However, lessons about ICD therapies and practical examples of how to broach the topic of deactivation could be beneficial, especially for interns whose responsibility includes discussions of code status. Within the context of an internal medicine residency, the fundamentals of ICD functionality could be covered during rotations on cardiology or palliative care services. Additionally, the recommendations of the HRS for device management can be covered in didactic sessions. Similar opportunities could be built into continuing medical education for practicing physicians and the training of advanced practice providers.

There are limitations to this survey, most notably the fact that it was restricted to a single academic medical center, the patient population and practices of which may not be generalizable to medical practice at large. Selection bias is also a distinct possibility given the 32% overall response rate; those who responded may feel more strongly about the survey topic. Our study subgroups may have interpreted questions differently because of their particular area of clinical practice. The small sample size also precluded an effective analysis of fellows and advanced practice practitioners due to lack of power. A major strength of this survey was the inclusion of a large number of residents upon whom the majority of inpatient contact rests. Future work could include expanding the survey to multiple medical centers, which would enhance generalizability and improve the ability to recruit sufficient fellows and advanced practice providers.

In summary, we conducted a single-center survey of residents, fellows, advanced practice providers, and attending physicians on their attitudes and beliefs about ICD deactivation in DNR/DNI patients. Residents are particularly uncomfortable discussing ICD deactivation with patients, which is an important finding because of their crucial role in providing patient care. Additionally, residents and hospitalists do not broach the topic of deactivation regularly, especially when compared to geriatricians and cardiologists. Despite HRS guidelines to the contrary, a fifth of our respondents believed that DNR/DNI orders include ICD deactivation. Overall, ICD deactivation in DNR/DNI patients is a topic that needs further attention in clinical education so that patients receive care that respects their individual wishes.

Disclosure

Nothing to report.

Implantable cardioverter-defibrillators (ICDs) offer lifesaving therapies to many patients and have been implanted in hundreds of thousands of patients.¹ The population of patients with ICDs is growing rapidly, and the national ICD Registry reports over 12,000 devices are implanted monthly.² This population includes patients with congenital heart disease, ischemic cardiomyopathy, and idiopathic arrhythmias. If these patients experience ventricular tachycardia or fibrillation, an ICD attempts to restore sinus rhythm and prevent death. While a shock from an ICD may be lifesaving, it can be a traumatic and startling experience for the patient and perhaps distressful for families to witness.^3,4

Although ICDs are intended to save lives, they do not slow the progress of the patient’s underlying cardiac and noncardiac comorbidities. All these patients will eventually die, whether from their cardiac disease or another condition. The literature includes many anecdotes about patients shocked multiple times by their defibrillator while actively dying.⁴ These situations could be prevented with preemptive ICD deactivation. (ICDs can function not only as cardioverters and defibrillators, as implied by their name, but also as pacemakers. “Deactivation” as used in this paper refers only to disabling the tachycardia therapies. No distinction was made between defibrillation with a shock and anti-tachycardia pacing.) Therefore, research on ICD deactivation has emphasized patients who are acutely terminally ill, while less emphasis has been placed on patients who are not actively dying.^4–8

Patients may, for a variety of reasons, request a do-not-resuscitate/do-not-intubate (DNR/DNI) order as their code status. However, it is not necessarily clear what a DNR/DNI order implies for ICD management. One survey of attending physicians found that 19% of respondents felt a DNR/DNI order was equivalent to requesting ICD deactivation.⁹ On the other hand, patients are split on whether they would want their device deactivated while in hospice or even at the very end of life.⁶ Heart Rhythm Society (HRS) guidelines favor a nuanced approach to ICD deactivation in DNR/DNI patients that emphasizes the individual patient’s comorbidities and goals.¹⁰ A patient’s individual circumstances might justify a choice to be DNR/DNI without deactivating the ICD. Decision-making in these circumstances requires a careful conversation between the patient and clinician. It is important to identify barriers that might prevent optimal shared decision-making.

Clinicians have been surveyed on ICD management at the end of life, but these studies have generally focused on attending physicians.^5,9,11 However, physician trainees (ie, residents and fellows) as well as advanced practice providers (ie, physician assistants and nurse practitioners) are responsible for much of the clinical care provided to hospitalized patients. In particular, they are often the clinicians to discuss code status with patients. Different specialties (eg, cardiology, general medicine, and geriatrics) manage different sets of patients, which might affect clinicians’ opinions on ICD management. We therefore designed a survey to assess clinician attitudes and beliefs regarding ICD deactivation, including in non-terminally ill patients, and to evaluate for differences according to training level and specialty.

Case-based and Likert-scale questions were considered for this survey, with the latter being chosen for ease of completion by respondents. An online survey tool (SurveyMonkey; San Mateo, CA) was used for data collection; no identifying data were collected. E-mail invitations to participate were sent to a combination of mailing lists and individual addresses for residents, fellows, advanced practice providers, and attending physicians in general internal medicine, cardiology, electrophysiology, and geriatrics. The survey remained open for 2.5 weeks. It was conducted 5 months into the academic year, thus trainees were well-established in their current roles. Two $25 gift cards were offered to respondents who entered their e-mail into a drawing; responses were not tied to e-mail addresses. Approval for the study was obtained from the University of Michigan Institutional Review Board.

The survey posed 12 questions assessing general attitudes about ICDs as well as individual beliefs and behaviors relating to ICD deactivation. Answers were on a Likert scale of 1 to 5 with 1 representing “strongly disagree” and 5 representing “strongly agree.” A score of 3 indicated “unsure or neutral.” The first 3 questions appeared together on the first page and were prefaced with “Please respond to the following statements about ICD shocks.” The next 9 were likewise grouped on the next page and were prefaced with “Please respond to the following statements about ICD deactivation.” All 12 questions are shown in Figures 1 and 2. Respondents could easily return to previous questions and change their answers. The survey ended with a third page showing 3 multiple choice demographic questions. The demographic questions were about clinical role (first-, second-, third-, or fourth-year resident, fellow, advanced practice provider, and attending), specialty, and number of ICD deactivations the respondent had been directly involved in (0, 1 to 5, 5 to 10, and more than 10). Specialty options were internal medicine resident, inpatient general medicine, outpatient general medicine, cardiology, electrophysiology, and geriatrics.

Likert scale answers of “agree” or “strongly agree” were grouped together as an affirmative response, while all other answers were grouped together as a nonaffirmative response. For analysis, residents were grouped together and their responses compared with attending physicians as a group. Additional analysis was done comparing attending physicians stratified by clinical specialty. Given the small number of responses from attending electrophysiologists, they were grouped with attending cardiologists for analysis. Due to the limited number of fellows and advanced practice providers who responded, further evaluation of these groups was not performed. Finally, the number of ICD deactivations respondents had been involved in was stratified by training level. All comparisons were performed using the two-tailed Pearson’s chi-squared test.

A total of 170 responses were collected from 508 individuals on the e-mail lists. Two responses were from registered nurses who were not part of the intended study sample and 7 responses were incomplete, having only answered the first 3 questions. These 9 responses were excluded from further analysis, yielding an overall response rate of 32%. The demographics of the remaining 161 respondents are shown in Table 1. Figure 1 shows overall responses to each question.

When comparing residents to attending physicians, there were no statistically significant between-differences except on questions 5 and 6. Specifically, residents were less comfortable than attending physicians discussing ICD deactivation and did so with less regularity (P < .001 and P = .018, respectively; Figure 2). Comfort levels improved markedly with experience: 29.2% of interns expressed comfort asking about ICD deactivation as compared with 60.7% of third- and-fourth year residents and 78.8% of attending physicians. Furthermore, comfort level seemed to parallel the regularity with which respondents asked about ICD deactivation: 4.2% of interns routinely asked about ICD deactivation as compared with 21.4% of third- and fourth-year residents and 34.8% of attending physicians.

Overall, our major findings were (1) residents, who provide much of the clinical care in a teaching hospital, are remarkably uncomfortable discussing ICD deactivation, (2) general internists and residents ask about ICD deactivation infrequently compared to geriatricians and cardiologists, and (3) about one fifth of our respondents believe ICD deactivation is automatically part of a DNR/DNI order.

Although the majority of respondents did not routinely address ICD deactivation in conjunction with code status, there was significant variability among subgroups. For example, 83.3% of geriatricians routinely discussed ICD deactivation as part of code status compared with only 4% of first-year residents and 10.5% of inpatient general internists. This finding is interesting because 90.7% of all respondents believed that discussions of code status should address preferences on ICD deactivation. This apparent discrepancy could be explained by the relatively small number of patients admitted to the hospital who have both an ICD and a request to be DNR/DNI. Residents and inpatient general internists see a very broad spectrum of patients; ICD deactivation is frequently irrelevant in the cases these physicians manage. The subset of patients seen in consultation by cardiologists and geriatricians, by contrast, is expected to include a larger proportion of patients with ICDs. Therefore, discussing ICD deactivation will be more relevant to their daily practice. Fear of alienating patients was not a reason for our findings, as our respondents did not express concern that recommending ICD deactivation would harm the patient-clinician relationship.

There are several possible reasons that residents, particularly interns, are uncomfortable discussing ICD deactivation. A lack of exposure to ICD deactivation is probably a key contributor. Over half of interns had never been involved in any ICD deactivations. Residents and hospitalists may also feel as if they are overstepping their boundaries to discuss deactivating ICD therapies. Their feelings may not be misplaced, as one survey of ICD patients found that over 75% thought responsibility for discussing ICD deactivation, at least at the end of life, rests with cardiologists or electrophysiologists.⁶

The HRS guidelines call for individualized decisions regarding ICD deactivation, even if a patient is DNR/DNI. However, our respondents frequently felt a standardized approach was indicated, with 21% believing that a DNR/DNI order included ICD deactivation. Additionally, 28% agreed that even non-terminally ill DNR/DNI patients should have their device deactivated. This is relevant because it is the role of clinicians to engage in shared decision-making with their patients. If the clinician holds the fixed belief that a DNR/DNI order, regardless of the precise clinical scenario, should include ICD deactivation, they may pressure a patient to have their device deactivated even if it could still benefit them.

In 2009, Kelley et al published results of a survey on ICD deactivation at the end of life.⁹ They contacted 4,876 attending physicians in cardiology, electrophysiology, geriatrics, and general medicine, receiving 558 responses. The survey included Likert-scale questions assessing attitudes and knowledge about ICD functionality. Demographic information was also collected, including how many patients in their practice had ICDs and how often they had previously discussed ICD deactivation.

There are some interesting comparisons between Kelley et al’s findings and ours, although we included trainees and the precise wording of questions was different. The specific questions used by Kelley et al to ask whether ICD shocks were painful or distressing and to ask if ICD deactivation is part of a DNR order were: “The shock from an ICD is very painful for most patients.” “The shock of an ICD at the end of life is distressing to a patient and their loved ones.” “A DNR order is equivalent to deactivation of an ICD.”

Only 47% of general internists in the Kelley et al survey thought that ICD shocks were painful, compared with 83% of electrophysiologists. In addition, 65% of general internists and 85% of electrophysiologists viewed shocks at the end of life to be distressing to patients and families. By contrast, our respondents were nearly unanimous in believing shocks to be painful and distressing. This discrepancy may be due to the growing prevalence of ICDs over the past several years as well as the growing body of literature on unnecessary shocks at the end of life. In line with our study, 19% of their respondents believed a DNR order was equivalent to ICD deactivation.⁹

Taken together, our findings indicate that additional education for clinicians of all levels could be helpful. Didactic lessons cannot replace experience, and it is important for residents to be exposed to discussions of ICD deactivation. However, lessons about ICD therapies and practical examples of how to broach the topic of deactivation could be beneficial, especially for interns whose responsibility includes discussions of code status. Within the context of an internal medicine residency, the fundamentals of ICD functionality could be covered during rotations on cardiology or palliative care services. Additionally, the recommendations of the HRS for device management can be covered in didactic sessions. Similar opportunities could be built into continuing medical education for practicing physicians and the training of advanced practice providers.

There are limitations to this survey, most notably the fact that it was restricted to a single academic medical center, the patient population and practices of which may not be generalizable to medical practice at large. Selection bias is also a distinct possibility given the 32% overall response rate; those who responded may feel more strongly about the survey topic. Our study subgroups may have interpreted questions differently because of their particular area of clinical practice. The small sample size also precluded an effective analysis of fellows and advanced practice practitioners due to lack of power. A major strength of this survey was the inclusion of a large number of residents upon whom the majority of inpatient contact rests. Future work could include expanding the survey to multiple medical centers, which would enhance generalizability and improve the ability to recruit sufficient fellows and advanced practice providers.

In summary, we conducted a single-center survey of residents, fellows, advanced practice providers, and attending physicians on their attitudes and beliefs about ICD deactivation in DNR/DNI patients. Residents are particularly uncomfortable discussing ICD deactivation with patients, which is an important finding because of their crucial role in providing patient care. Additionally, residents and hospitalists do not broach the topic of deactivation regularly, especially when compared to geriatricians and cardiologists. Despite HRS guidelines to the contrary, a fifth of our respondents believed that DNR/DNI orders include ICD deactivation. Overall, ICD deactivation in DNR/DNI patients is a topic that needs further attention in clinical education so that patients receive care that respects their individual wishes.

Disclosure

Nothing to report.

Nearly 70% of bariatric surgery patients received postoperative imaging, with more than one-third receiving CT imaging. This high level of screening resulted in symptom-related findings in only 23% of cases, and may be excessive, according to researchers who studied nearly 600 adults who underwent bariatric surgery.

As the volume of bariatric surgery has increased, so has the role of postoperative imaging, wrote Dana Haddad, MD, and her colleagues at Harlem Hospital Center, New York.

“However, there is a lack of well-defined postoperative imaging guidelines,” they said. “Detrimental aspects of postoperative imaging include the potential for false-positive findings leading to further and often unnecessary investigations, radiation exposure, and additional cost,” they added.

copyright picsfive/Fotolia.com

Nearly 70% of bariatric surgery patients received postoperative imaging, with more than one-third receiving CT imaging. This high level of screening resulted in symptom-related findings in only 23% of cases, and may be excessive, according to researchers who studied nearly 600 adults who underwent bariatric surgery.

As the volume of bariatric surgery has increased, so has the role of postoperative imaging, wrote Dana Haddad, MD, and her colleagues at Harlem Hospital Center, New York.

“However, there is a lack of well-defined postoperative imaging guidelines,” they said. “Detrimental aspects of postoperative imaging include the potential for false-positive findings leading to further and often unnecessary investigations, radiation exposure, and additional cost,” they added.

copyright picsfive/Fotolia.com

Nearly 70% of bariatric surgery patients received postoperative imaging, with more than one-third receiving CT imaging. This high level of screening resulted in symptom-related findings in only 23% of cases, and may be excessive, according to researchers who studied nearly 600 adults who underwent bariatric surgery.

As the volume of bariatric surgery has increased, so has the role of postoperative imaging, wrote Dana Haddad, MD, and her colleagues at Harlem Hospital Center, New York.

“However, there is a lack of well-defined postoperative imaging guidelines,” they said. “Detrimental aspects of postoperative imaging include the potential for false-positive findings leading to further and often unnecessary investigations, radiation exposure, and additional cost,” they added.

copyright picsfive/Fotolia.com

In recent years, the American Board of Internal Medicine (ABIM) Foundation’s Choosing Wisely™ campaign has advanced the dialogue on cost-consciousness by identifying potential examples of overuse in clinical practice.¹ Eliminating low-value care can decrease costs, improve quality, and potentially decrease patient harm.² In fact, there is growing consensus among health leaders and educators on the need for a physician workforce that is conscious of high-value care.^3,4 The Institute of Medicine has issued a call-to-action for graduate medical education (GME) to emphasize value-based care,⁵ and the Accreditation Council for Graduate Medical Education has outlined expectations that residents receive formal and experiential training on overuse as a part of its Clinical Learning Environment Review.⁶

However, recent reports highlight a lack of emphasis on value-based care in medical education.⁷ For example, few residency program directors believe that residents are prepared to incorporate value and cost into their medical decisions.⁸ In 2012, only 15% of medicine residencies reported having formal curricula addressing value, although many were developing one.⁸ Of the curricula reported, most were didactic in nature and did not include an assessment component.⁸

Experiential learning through simulation is one promising method to teach clinicians-in-training to practice value-based care. Simulation-based training promotes situational awareness (defined as being cognizant of one’s working environment), a concept that is crucial for recognizing both low-value and unsafe care.^9,10 Simulated training exercises are often included in GME orientation “boot-camps,” which have typically addressed safety.¹¹ The incorporation of value into existing GME boot-camp exercises could provide a promising model for the addition of value-based training to GME.

At the University of Chicago, we had successfully implemented the “Room of Horrors,” a simulation for entering interns to promote the detection of patient safety hazards.¹¹ Here, we describe a modification to this simulation to embed low-value hazards in addition to traditional patient safety hazards. The aim of this study is to assess the entering interns’ recognition of low-value care and their ability to recognize unsafe care in a simulation designed to promote situational awareness.

Setting and Participants

The simulation was conducted during GME orientation at a large, urban academic medical institution. One hundred and twenty-five entering postgraduate year one (PGY1) interns participated in the simulation, which was a required component of a multiday orientation “boot-camp” experience. All eligible interns participated in the simulation, representing 13 specialty programs and 60 medical schools. Interns entering into pathology were excluded because of infrequent patient contact. Participating interns were divided into 7 specialty groups for analysis in order to preserve the anonymity of interns in smaller residency programs (surgical subspecialties combined with general surgery, medicine-pediatrics grouped with internal medicine). The University of Chicago Institutional Review Board deemed this study exempt from review.

Program Description

A simulation of an inpatient hospital room, known as the “Room of Horrors,” was constructed in collaboration with the University of Chicago Simulation Center and adapted from a previous version of the exercise.¹¹ The simulation consisted of a mock door chart highlighting the patient had been admitted for diarrhea (Clostridium difficile positive) following a recent hospitalization for pneumonia. A clinical scenario was constructed by using a patient mannequin and an accompanying door chart that listed information on the patient’s hospital course, allergies, and medications. In addition to the 8 patient safety hazards utilized in the prior version, our team selected 4 low-value hazards to be included in the simulation.

The 8 safety hazards have been detailed in a prior study and were previously selected from Medicare’s Hospital-Acquired Conditions (HAC) Reduction Program and Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators.^11-13 Each of the hazards was represented either physically in the simulation room and/or was indicated on the patient’s chart. For example, the latex allergy hazard was represented by latex gloves at the bedside despite an allergy indicated on the patient’s chart and wristband. A complete list of the 8 safety hazards and their representations in the simulation is shown in Table 1.

The Choosing Wisely™ lists were reviewed to identify low-value hazards for addition to the simulation.¹⁴ Our team selected 3 low-value hazards from the Society of Hospital Medicine (SHM) list,¹⁵ including (1) arbitrary blood transfusion despite the patient’s stable hemoglobin level of 8.0 g/dL and absence of cardiac symptoms,¹⁶ (2) addition of a proton pump inhibitor (PPI) for stress ulcer prophylaxis in a patient without high risk for gastrointestinal (GI) complications who was not on a PPI prior to admission, and (3) placement of a urinary catheter without medical indication. We had originally selected continuous telemetry monitoring as a fourth hazard from the SHM list, but were unable to operationalize, as it was difficult to simulate continuous telemetry on a mannequin. Because many inpatients are older than 65 years, we reviewed the American Geriatrics Society list¹⁷ and selected our fourth low-value hazard: (4) unnecessary use of physical restraints to manage behavioral symptoms in a hospitalized patient with delirium. Several of these hazards were also quality and safety priorities at our institution, including the overuse of urinary catheters, physical restraints, and blood transfusions. All 4 low-value hazards were referenced in the patient’s door chart, and 3 were also physically represented in the room via presence of a hanging unit of blood, Foley catheter, and upper-arm restraints (Table 1). See Appendix for a photograph of the simulation setup.

Each intern was allowed 10 minutes inside the simulation room. During this time, they were instructed to read the 1-page door chart, inspect the simulation room, and write down as many potential low-value and safety hazards as they could identify on a free-response form (see Appendix). Upon exiting the room, they were allotted 5 additional minutes to complete their free-response answers and provide written feedback on the simulation. The simulation was conducted in 3 simulated hospital rooms over the course of 2 days, and the correct answers were provided via e-mail after all interns had completed the exercise.

To assess prior training and safety knowledge, interns were asked to complete a 3-question preassessment on a ScanTron^TM (Tustin, CA) form. The preassessment asked interns whether they had received training on hospital safety during medical school (yes, no, or unsure), if they were satisfied with the hospital safety training they received during medical school (strongly disagree to strongly agree on a Likert scale), and if they were confident in their ability to identify potential hazards in a hospital setting (strongly disagree to strongly agree). Interns were also given the opportunity to provide feedback on the simulation experience on the ScanTron^TM (Tustin, CA) form.

One month after participating in the simulation, interns were asked to complete an online follow-up survey on MedHub^TM (Ann Arbor, MI), which included 2 Likert-scale questions (strongly disagree to strongly agree) assessing the simulation’s impact on their experience mitigating hospital hazards during the first month of internship.

Data Analysis

Interns’ free-response answers were manually coded, and descriptive statistics were used to summarize the mean percent correct for each hazard. A paired t test was used to compare intern identification of low-value vs safety hazards. T tests were used to compare hazard identification for interns entering highly procedural-intensive specialties (ie, surgical specialties, emergency medicine, anesthesia, obstetrics/gynecology) and those entering less procedural-intensive specialties (ie, internal medicine, pediatrics, psychiatry), as well as among those graduating from “Top 30” medical schools (based on US News & World Report Medical School Rankings¹⁸) and our own institution. One-way analysis of variance (ANOVA) calculations were used to test for differences in hazard identification based on interns’ prior hospital safety training, with interns who rated their satisfaction with prior training or confidence in identifying hazards as a “4” or a “5” considered “satisfied” and “confident,” respectively. Responses to the MedHub^TM (Ann Arbor, MI) survey were dichotomized with “strongly agree” and “agree” considered positive responses. Statistical significance was defined at P < .05. All data analysis was conducted using Stata 14^TM software (College Station, TX).

Intern Characteristics

One hundred twenty-five entering PGY1 interns participated in the simulation, representing 60 medical schools and 7 different specialty groups (Table 2). Thirty-five percent (44/125) were graduates from “Top 30” medical schools, and 8.8% (11/125) graduated from our own institution. Seventy-four percent (89/121) had received prior hospital safety training during medical school, and 62.9% (56/89) were satisfied with their training. A majority of interns (64.2%, 79/123) felt confident in their ability to identify potential hazards in a hospital setting, although confidence was much higher among those with prior safety training (71.9%, 64/89) compared to those without prior training or who were unsure about their training (40.6%, 13/32; P = .09, t test).

Identification of Hazards

The mean percentage of hazards correctly identified by interns during the simulation was 50.4% (standard deviation [SD] 11.8%), with a normal distribution (Figure 1). Interns identified a significantly lower percentage of low-value hazards than safety hazards in the simulation (mean 19.2% [SD 18.6%] vs 66.0% [SD 16.0%], respectively; P < .001, paired t test). Interns also identified significantly more room-based errors than chart-based errors (mean 58.6% [SD 13.4%] vs 9.6% [SD 19.8%], respectively; P < .001, paired t test). The 3 most commonly identified hazards were unavailability of hand hygiene (120/125, 96.0%), presence of latex gloves despite the patient’s allergy (111/125, 88.8%), and fall risk due to the lowered bed rail (107/125, 85.6%). More than half of interns identified the incorrect name on the patient’s wristband and IV bag (91/125, 72.8%), a lack of isolation precautions (90/125, 72.0%), administration of penicillin despite the patient’s allergy (67/125, 53.6%), and unnecessary restraints (64/125, 51.2%). Less than half of interns identified the wrong medication being administered (50/125, 40.0%), unnecessary Foley catheter (25/125, 20.0%), and absence of venous thromboembolism (VTE) prophylaxis (24/125, 19.2%). Few interns identified the unnecessary blood transfusion (7/125, 5.6%), and no one identified the unnecessary stress ulcer prophylaxis (0/125, 0.0%; Figure 2).

Predictors of Hazard Identification

Interns who self-reported as confident in their ability to identify hazards were not any more likely to correctly identify hazards than those who were not confident (50.9% overall hazard identification vs 49.6%, respectively; P = .56, t test). Interns entering into less procedural-intensive specialties identified significantly more safety hazards than those entering highly procedural-intensive specialties (mean 69.1% [SD 16.9%] vs 61.8% [SD 13.7%], respectively; P = .01, t test). However, there was no statistically significant difference in their identification of low-value hazards (mean 19.8% [SD 18.3%] for less procedural-intensive vs 18.4% [SD 19.1%] for highly procedural-intensive; P = .68, t test). There was no statistically significant difference in hazard identification among graduates of “Top 30” medical schools or graduates of our own institution. Prior hospital safety training had no significant impact on interns’ ability to identify safety or low-value hazards. Overall, interns who were satisfied with their prior training identified a mean of 51.8% of hazards present (SD 11.8%), interns who were not satisfied with their prior training identified 51.5% (SD 12.7%), interns with no prior training identified 48.7% (SD 11.7%), and interns who were unsure about their prior training identified 47.4% (SD 11.5%) [F(3,117) = .79; P = .51, ANOVA]. There was also no significant association between prior training and the identification of any one of the 12 specific hazards (chi-square tests, all P values > .1).

Intern Feedback and Follow-Up Survey

Debriefing revealed that most interns passively assumed the patient’s chart was correct and did not think they should question the patient’s current care regimen. For example, many interns commented that they did not think to consider the patient’s blood transfusion as unnecessary, even though they were aware of the recommended hemoglobin cutoffs for stable patients.

Interns also provided formal feedback on the simulation through open-ended comments on their ScanTron^TM (Tustin, CA) form. For example, one intern wrote that they would “inherently approach every patient room ‘looking’ for safety issues, probably directly because of this exercise.” Another commented that the simulation was “more difficult than I expected, but very necessary to facilitate discussion and learning.” One intern wrote that “I wish I had done this earlier in my career.”

Ninety-six percent of participating interns (120/125) completed an online follow-up survey 1 month after beginning internship. In the survey, 68.9% (82/119) of interns indicated they were more aware of how to identify potential hazards facing hospitalized patients as a result of the simulation. Furthermore, 52.1% (62/119) of interns had taken action during internship to reduce a potential hazard that was present in the simulation.

While many GME orientations include simulation and safety training, this study is the first of its kind to incorporate low-value care from Choosing Wisely™ recommendations into simulated training. It is concerning that interns identified significantly fewer low-value hazards than safety hazards in the simulation. In some cases, no interns identified the low-value hazard. For example, while almost all interns identified the hand hygiene hazard, not one could identify the unnecessary stress ulcer prophylaxis. Furthermore, interns who self-reported as confident in their ability to identify hazards did not perform any better in the simulation. Interns entering less procedural-intensive specialties identified more safety hazards overall.

The simulation was well received by interns. Many commented that the experience was engaging, challenging, and effective in cultivating situational awareness towards low-value care. Our follow-up survey demonstrated the majority of interns reported taking action during their first month of internship to reduce a hazard included in the simulation. Most interns also reported a greater awareness of how to identify hospital hazards as a result of the simulation. These findings suggest that a brief simulation-based experience has the potential to create a lasting retention of situational awareness and behavior change.

It is worth exploring why interns identified significantly fewer low-value hazards than safety hazards in the simulation. One hypothesis is that interns were less attuned to low-value hazards, which may reflect a lacking emphasis on value-based care in undergraduate medical education (UME). It is especially concerning that so few interns identified the catheter-associated urinary tract infection (CAUTI) risk, as interns are primarily responsible for recognizing and removing an unnecessary catheter. Although the risks of low-value care should be apparent to most trainees, the process of recognizing and deliberately stopping or avoiding low-value care can be challenging for young clinicians.¹⁹ To promote value-based thinking among entering residents, UME programs should teach students to question the utility of the interventions their patients are receiving. One promising framework for doing so is the Subjective, Objective, Assessment, Plan- (SOAP)-V, in which a V for “Value” is added to the traditional SOAP note.²⁰ SOAP-V notes serve as a cognitive forcing function that requires students to pause and assess the value and cost-consciousness of their patients’ care.²⁰

The results from the “Room of Horrors” simulation can also guide health leaders and educators in identifying institutional areas of focus towards providing high-value and safe care. For example, at the University of Chicago we launched an initiative to improve the inappropriate use of urinary catheters after learning that few of our incoming interns recognized this during the simulation. Institutions could use this model to raise awareness of initiatives and redirect resources from areas that trainees perform well in (eg, hand hygiene) to areas that need improvement (eg, recognition of low-value care). Given the simulation’s low cost and minimal material requirements, it could be easily integrated into existing training programs with the support of an institution’s simulation center.

This study’s limitations include its conduction at single-institution, although the participants represented graduates of 60 different institutions. Furthermore, while the 12 hazards included in the simulation represent patient safety and value initiatives from a wide array of medical societies, they were not intended to be comprehensive and were not tailored to specific specialties. The simulation included only 4 low-value hazards, and future iterations of this exercise should aim to include an equal number of safety and low-value hazards. Furthermore, the evaluation of interns’ prior hospital safety training relied on self-reporting, and the specific context and content of each interns’ training was not examined. Finally, at this point we are unable to provide objective longitudinal data assessing the simulation’s impact on clinical practice and patient outcomes. Subsequent work will assess the sustained impact of the simulation by correlating with institutional data on measurable occurrences of low-value care.

In conclusion, interns identified significantly fewer low-value hazards than safety hazards in an inpatient simulation designed to promote situational awareness. Our results suggest that interns are on the lookout for errors of omission (eg, absence of hand hygiene, absence of isolation precautions) but are often blinded to errors of commission, such that when patients are started on therapies there is an assumption that the therapies are correct and necessary (eg, blood transfusions, stress ulcer prophylaxis). These findings suggest poor awareness of low-value care among incoming interns and highlight the need for additional training in both UME and GME to place a greater emphasis on preventing low-value care.

Disclosure

Dr. Arora is a member of the American Board of Medicine Board of Directors and has received grant funding from ABIM Foundation via Costs of Care for the Teaching Value Choosing Wisely™ Challenge. Dr. Farnan, Dr. Arora, and Ms. Hirsch receive grant funds from Accreditation Council of Graduate Medical Education as part of the Pursuing Excellence Initiative. Dr. Arora and Dr. Farnan also receive grant funds from the American Medical Association Accelerating Change in Medical Education initiative. Kathleen Wiest and Lukas Matern were funded through matching funds of the Pritzker Summer Research Program for NIA T35AG029795.

In recent years, the American Board of Internal Medicine (ABIM) Foundation’s Choosing Wisely™ campaign has advanced the dialogue on cost-consciousness by identifying potential examples of overuse in clinical practice.¹ Eliminating low-value care can decrease costs, improve quality, and potentially decrease patient harm.² In fact, there is growing consensus among health leaders and educators on the need for a physician workforce that is conscious of high-value care.^3,4 The Institute of Medicine has issued a call-to-action for graduate medical education (GME) to emphasize value-based care,⁵ and the Accreditation Council for Graduate Medical Education has outlined expectations that residents receive formal and experiential training on overuse as a part of its Clinical Learning Environment Review.⁶

However, recent reports highlight a lack of emphasis on value-based care in medical education.⁷ For example, few residency program directors believe that residents are prepared to incorporate value and cost into their medical decisions.⁸ In 2012, only 15% of medicine residencies reported having formal curricula addressing value, although many were developing one.⁸ Of the curricula reported, most were didactic in nature and did not include an assessment component.⁸

Experiential learning through simulation is one promising method to teach clinicians-in-training to practice value-based care. Simulation-based training promotes situational awareness (defined as being cognizant of one’s working environment), a concept that is crucial for recognizing both low-value and unsafe care.^9,10 Simulated training exercises are often included in GME orientation “boot-camps,” which have typically addressed safety.¹¹ The incorporation of value into existing GME boot-camp exercises could provide a promising model for the addition of value-based training to GME.

At the University of Chicago, we had successfully implemented the “Room of Horrors,” a simulation for entering interns to promote the detection of patient safety hazards.¹¹ Here, we describe a modification to this simulation to embed low-value hazards in addition to traditional patient safety hazards. The aim of this study is to assess the entering interns’ recognition of low-value care and their ability to recognize unsafe care in a simulation designed to promote situational awareness.

Setting and Participants

The simulation was conducted during GME orientation at a large, urban academic medical institution. One hundred and twenty-five entering postgraduate year one (PGY1) interns participated in the simulation, which was a required component of a multiday orientation “boot-camp” experience. All eligible interns participated in the simulation, representing 13 specialty programs and 60 medical schools. Interns entering into pathology were excluded because of infrequent patient contact. Participating interns were divided into 7 specialty groups for analysis in order to preserve the anonymity of interns in smaller residency programs (surgical subspecialties combined with general surgery, medicine-pediatrics grouped with internal medicine). The University of Chicago Institutional Review Board deemed this study exempt from review.

Program Description

A simulation of an inpatient hospital room, known as the “Room of Horrors,” was constructed in collaboration with the University of Chicago Simulation Center and adapted from a previous version of the exercise.¹¹ The simulation consisted of a mock door chart highlighting the patient had been admitted for diarrhea (Clostridium difficile positive) following a recent hospitalization for pneumonia. A clinical scenario was constructed by using a patient mannequin and an accompanying door chart that listed information on the patient’s hospital course, allergies, and medications. In addition to the 8 patient safety hazards utilized in the prior version, our team selected 4 low-value hazards to be included in the simulation.

The 8 safety hazards have been detailed in a prior study and were previously selected from Medicare’s Hospital-Acquired Conditions (HAC) Reduction Program and Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators.^11-13 Each of the hazards was represented either physically in the simulation room and/or was indicated on the patient’s chart. For example, the latex allergy hazard was represented by latex gloves at the bedside despite an allergy indicated on the patient’s chart and wristband. A complete list of the 8 safety hazards and their representations in the simulation is shown in Table 1.

The Choosing Wisely™ lists were reviewed to identify low-value hazards for addition to the simulation.¹⁴ Our team selected 3 low-value hazards from the Society of Hospital Medicine (SHM) list,¹⁵ including (1) arbitrary blood transfusion despite the patient’s stable hemoglobin level of 8.0 g/dL and absence of cardiac symptoms,¹⁶ (2) addition of a proton pump inhibitor (PPI) for stress ulcer prophylaxis in a patient without high risk for gastrointestinal (GI) complications who was not on a PPI prior to admission, and (3) placement of a urinary catheter without medical indication. We had originally selected continuous telemetry monitoring as a fourth hazard from the SHM list, but were unable to operationalize, as it was difficult to simulate continuous telemetry on a mannequin. Because many inpatients are older than 65 years, we reviewed the American Geriatrics Society list¹⁷ and selected our fourth low-value hazard: (4) unnecessary use of physical restraints to manage behavioral symptoms in a hospitalized patient with delirium. Several of these hazards were also quality and safety priorities at our institution, including the overuse of urinary catheters, physical restraints, and blood transfusions. All 4 low-value hazards were referenced in the patient’s door chart, and 3 were also physically represented in the room via presence of a hanging unit of blood, Foley catheter, and upper-arm restraints (Table 1). See Appendix for a photograph of the simulation setup.

Each intern was allowed 10 minutes inside the simulation room. During this time, they were instructed to read the 1-page door chart, inspect the simulation room, and write down as many potential low-value and safety hazards as they could identify on a free-response form (see Appendix). Upon exiting the room, they were allotted 5 additional minutes to complete their free-response answers and provide written feedback on the simulation. The simulation was conducted in 3 simulated hospital rooms over the course of 2 days, and the correct answers were provided via e-mail after all interns had completed the exercise.

To assess prior training and safety knowledge, interns were asked to complete a 3-question preassessment on a ScanTron^TM (Tustin, CA) form. The preassessment asked interns whether they had received training on hospital safety during medical school (yes, no, or unsure), if they were satisfied with the hospital safety training they received during medical school (strongly disagree to strongly agree on a Likert scale), and if they were confident in their ability to identify potential hazards in a hospital setting (strongly disagree to strongly agree). Interns were also given the opportunity to provide feedback on the simulation experience on the ScanTron^TM (Tustin, CA) form.

One month after participating in the simulation, interns were asked to complete an online follow-up survey on MedHub^TM (Ann Arbor, MI), which included 2 Likert-scale questions (strongly disagree to strongly agree) assessing the simulation’s impact on their experience mitigating hospital hazards during the first month of internship.

Data Analysis

Interns’ free-response answers were manually coded, and descriptive statistics were used to summarize the mean percent correct for each hazard. A paired t test was used to compare intern identification of low-value vs safety hazards. T tests were used to compare hazard identification for interns entering highly procedural-intensive specialties (ie, surgical specialties, emergency medicine, anesthesia, obstetrics/gynecology) and those entering less procedural-intensive specialties (ie, internal medicine, pediatrics, psychiatry), as well as among those graduating from “Top 30” medical schools (based on US News & World Report Medical School Rankings¹⁸) and our own institution. One-way analysis of variance (ANOVA) calculations were used to test for differences in hazard identification based on interns’ prior hospital safety training, with interns who rated their satisfaction with prior training or confidence in identifying hazards as a “4” or a “5” considered “satisfied” and “confident,” respectively. Responses to the MedHub^TM (Ann Arbor, MI) survey were dichotomized with “strongly agree” and “agree” considered positive responses. Statistical significance was defined at P < .05. All data analysis was conducted using Stata 14^TM software (College Station, TX).

Intern Characteristics

One hundred twenty-five entering PGY1 interns participated in the simulation, representing 60 medical schools and 7 different specialty groups (Table 2). Thirty-five percent (44/125) were graduates from “Top 30” medical schools, and 8.8% (11/125) graduated from our own institution. Seventy-four percent (89/121) had received prior hospital safety training during medical school, and 62.9% (56/89) were satisfied with their training. A majority of interns (64.2%, 79/123) felt confident in their ability to identify potential hazards in a hospital setting, although confidence was much higher among those with prior safety training (71.9%, 64/89) compared to those without prior training or who were unsure about their training (40.6%, 13/32; P = .09, t test).

Identification of Hazards

The mean percentage of hazards correctly identified by interns during the simulation was 50.4% (standard deviation [SD] 11.8%), with a normal distribution (Figure 1). Interns identified a significantly lower percentage of low-value hazards than safety hazards in the simulation (mean 19.2% [SD 18.6%] vs 66.0% [SD 16.0%], respectively; P < .001, paired t test). Interns also identified significantly more room-based errors than chart-based errors (mean 58.6% [SD 13.4%] vs 9.6% [SD 19.8%], respectively; P < .001, paired t test). The 3 most commonly identified hazards were unavailability of hand hygiene (120/125, 96.0%), presence of latex gloves despite the patient’s allergy (111/125, 88.8%), and fall risk due to the lowered bed rail (107/125, 85.6%). More than half of interns identified the incorrect name on the patient’s wristband and IV bag (91/125, 72.8%), a lack of isolation precautions (90/125, 72.0%), administration of penicillin despite the patient’s allergy (67/125, 53.6%), and unnecessary restraints (64/125, 51.2%). Less than half of interns identified the wrong medication being administered (50/125, 40.0%), unnecessary Foley catheter (25/125, 20.0%), and absence of venous thromboembolism (VTE) prophylaxis (24/125, 19.2%). Few interns identified the unnecessary blood transfusion (7/125, 5.6%), and no one identified the unnecessary stress ulcer prophylaxis (0/125, 0.0%; Figure 2).

Predictors of Hazard Identification

Interns who self-reported as confident in their ability to identify hazards were not any more likely to correctly identify hazards than those who were not confident (50.9% overall hazard identification vs 49.6%, respectively; P = .56, t test). Interns entering into less procedural-intensive specialties identified significantly more safety hazards than those entering highly procedural-intensive specialties (mean 69.1% [SD 16.9%] vs 61.8% [SD 13.7%], respectively; P = .01, t test). However, there was no statistically significant difference in their identification of low-value hazards (mean 19.8% [SD 18.3%] for less procedural-intensive vs 18.4% [SD 19.1%] for highly procedural-intensive; P = .68, t test). There was no statistically significant difference in hazard identification among graduates of “Top 30” medical schools or graduates of our own institution. Prior hospital safety training had no significant impact on interns’ ability to identify safety or low-value hazards. Overall, interns who were satisfied with their prior training identified a mean of 51.8% of hazards present (SD 11.8%), interns who were not satisfied with their prior training identified 51.5% (SD 12.7%), interns with no prior training identified 48.7% (SD 11.7%), and interns who were unsure about their prior training identified 47.4% (SD 11.5%) [F(3,117) = .79; P = .51, ANOVA]. There was also no significant association between prior training and the identification of any one of the 12 specific hazards (chi-square tests, all P values > .1).

Intern Feedback and Follow-Up Survey

Debriefing revealed that most interns passively assumed the patient’s chart was correct and did not think they should question the patient’s current care regimen. For example, many interns commented that they did not think to consider the patient’s blood transfusion as unnecessary, even though they were aware of the recommended hemoglobin cutoffs for stable patients.

Interns also provided formal feedback on the simulation through open-ended comments on their ScanTron^TM (Tustin, CA) form. For example, one intern wrote that they would “inherently approach every patient room ‘looking’ for safety issues, probably directly because of this exercise.” Another commented that the simulation was “more difficult than I expected, but very necessary to facilitate discussion and learning.” One intern wrote that “I wish I had done this earlier in my career.”

Ninety-six percent of participating interns (120/125) completed an online follow-up survey 1 month after beginning internship. In the survey, 68.9% (82/119) of interns indicated they were more aware of how to identify potential hazards facing hospitalized patients as a result of the simulation. Furthermore, 52.1% (62/119) of interns had taken action during internship to reduce a potential hazard that was present in the simulation.

While many GME orientations include simulation and safety training, this study is the first of its kind to incorporate low-value care from Choosing Wisely™ recommendations into simulated training. It is concerning that interns identified significantly fewer low-value hazards than safety hazards in the simulation. In some cases, no interns identified the low-value hazard. For example, while almost all interns identified the hand hygiene hazard, not one could identify the unnecessary stress ulcer prophylaxis. Furthermore, interns who self-reported as confident in their ability to identify hazards did not perform any better in the simulation. Interns entering less procedural-intensive specialties identified more safety hazards overall.

The simulation was well received by interns. Many commented that the experience was engaging, challenging, and effective in cultivating situational awareness towards low-value care. Our follow-up survey demonstrated the majority of interns reported taking action during their first month of internship to reduce a hazard included in the simulation. Most interns also reported a greater awareness of how to identify hospital hazards as a result of the simulation. These findings suggest that a brief simulation-based experience has the potential to create a lasting retention of situational awareness and behavior change.

It is worth exploring why interns identified significantly fewer low-value hazards than safety hazards in the simulation. One hypothesis is that interns were less attuned to low-value hazards, which may reflect a lacking emphasis on value-based care in undergraduate medical education (UME). It is especially concerning that so few interns identified the catheter-associated urinary tract infection (CAUTI) risk, as interns are primarily responsible for recognizing and removing an unnecessary catheter. Although the risks of low-value care should be apparent to most trainees, the process of recognizing and deliberately stopping or avoiding low-value care can be challenging for young clinicians.¹⁹ To promote value-based thinking among entering residents, UME programs should teach students to question the utility of the interventions their patients are receiving. One promising framework for doing so is the Subjective, Objective, Assessment, Plan- (SOAP)-V, in which a V for “Value” is added to the traditional SOAP note.²⁰ SOAP-V notes serve as a cognitive forcing function that requires students to pause and assess the value and cost-consciousness of their patients’ care.²⁰

The results from the “Room of Horrors” simulation can also guide health leaders and educators in identifying institutional areas of focus towards providing high-value and safe care. For example, at the University of Chicago we launched an initiative to improve the inappropriate use of urinary catheters after learning that few of our incoming interns recognized this during the simulation. Institutions could use this model to raise awareness of initiatives and redirect resources from areas that trainees perform well in (eg, hand hygiene) to areas that need improvement (eg, recognition of low-value care). Given the simulation’s low cost and minimal material requirements, it could be easily integrated into existing training programs with the support of an institution’s simulation center.

This study’s limitations include its conduction at single-institution, although the participants represented graduates of 60 different institutions. Furthermore, while the 12 hazards included in the simulation represent patient safety and value initiatives from a wide array of medical societies, they were not intended to be comprehensive and were not tailored to specific specialties. The simulation included only 4 low-value hazards, and future iterations of this exercise should aim to include an equal number of safety and low-value hazards. Furthermore, the evaluation of interns’ prior hospital safety training relied on self-reporting, and the specific context and content of each interns’ training was not examined. Finally, at this point we are unable to provide objective longitudinal data assessing the simulation’s impact on clinical practice and patient outcomes. Subsequent work will assess the sustained impact of the simulation by correlating with institutional data on measurable occurrences of low-value care.

In conclusion, interns identified significantly fewer low-value hazards than safety hazards in an inpatient simulation designed to promote situational awareness. Our results suggest that interns are on the lookout for errors of omission (eg, absence of hand hygiene, absence of isolation precautions) but are often blinded to errors of commission, such that when patients are started on therapies there is an assumption that the therapies are correct and necessary (eg, blood transfusions, stress ulcer prophylaxis). These findings suggest poor awareness of low-value care among incoming interns and highlight the need for additional training in both UME and GME to place a greater emphasis on preventing low-value care.

Disclosure

Dr. Arora is a member of the American Board of Medicine Board of Directors and has received grant funding from ABIM Foundation via Costs of Care for the Teaching Value Choosing Wisely™ Challenge. Dr. Farnan, Dr. Arora, and Ms. Hirsch receive grant funds from Accreditation Council of Graduate Medical Education as part of the Pursuing Excellence Initiative. Dr. Arora and Dr. Farnan also receive grant funds from the American Medical Association Accelerating Change in Medical Education initiative. Kathleen Wiest and Lukas Matern were funded through matching funds of the Pritzker Summer Research Program for NIA T35AG029795.

In recent years, the American Board of Internal Medicine (ABIM) Foundation’s Choosing Wisely™ campaign has advanced the dialogue on cost-consciousness by identifying potential examples of overuse in clinical practice.¹ Eliminating low-value care can decrease costs, improve quality, and potentially decrease patient harm.² In fact, there is growing consensus among health leaders and educators on the need for a physician workforce that is conscious of high-value care.^3,4 The Institute of Medicine has issued a call-to-action for graduate medical education (GME) to emphasize value-based care,⁵ and the Accreditation Council for Graduate Medical Education has outlined expectations that residents receive formal and experiential training on overuse as a part of its Clinical Learning Environment Review.⁶

However, recent reports highlight a lack of emphasis on value-based care in medical education.⁷ For example, few residency program directors believe that residents are prepared to incorporate value and cost into their medical decisions.⁸ In 2012, only 15% of medicine residencies reported having formal curricula addressing value, although many were developing one.⁸ Of the curricula reported, most were didactic in nature and did not include an assessment component.⁸

Experiential learning through simulation is one promising method to teach clinicians-in-training to practice value-based care. Simulation-based training promotes situational awareness (defined as being cognizant of one’s working environment), a concept that is crucial for recognizing both low-value and unsafe care.^9,10 Simulated training exercises are often included in GME orientation “boot-camps,” which have typically addressed safety.¹¹ The incorporation of value into existing GME boot-camp exercises could provide a promising model for the addition of value-based training to GME.

At the University of Chicago, we had successfully implemented the “Room of Horrors,” a simulation for entering interns to promote the detection of patient safety hazards.¹¹ Here, we describe a modification to this simulation to embed low-value hazards in addition to traditional patient safety hazards. The aim of this study is to assess the entering interns’ recognition of low-value care and their ability to recognize unsafe care in a simulation designed to promote situational awareness.

Setting and Participants

The simulation was conducted during GME orientation at a large, urban academic medical institution. One hundred and twenty-five entering postgraduate year one (PGY1) interns participated in the simulation, which was a required component of a multiday orientation “boot-camp” experience. All eligible interns participated in the simulation, representing 13 specialty programs and 60 medical schools. Interns entering into pathology were excluded because of infrequent patient contact. Participating interns were divided into 7 specialty groups for analysis in order to preserve the anonymity of interns in smaller residency programs (surgical subspecialties combined with general surgery, medicine-pediatrics grouped with internal medicine). The University of Chicago Institutional Review Board deemed this study exempt from review.

Program Description

A simulation of an inpatient hospital room, known as the “Room of Horrors,” was constructed in collaboration with the University of Chicago Simulation Center and adapted from a previous version of the exercise.¹¹ The simulation consisted of a mock door chart highlighting the patient had been admitted for diarrhea (Clostridium difficile positive) following a recent hospitalization for pneumonia. A clinical scenario was constructed by using a patient mannequin and an accompanying door chart that listed information on the patient’s hospital course, allergies, and medications. In addition to the 8 patient safety hazards utilized in the prior version, our team selected 4 low-value hazards to be included in the simulation.

The 8 safety hazards have been detailed in a prior study and were previously selected from Medicare’s Hospital-Acquired Conditions (HAC) Reduction Program and Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators.^11-13 Each of the hazards was represented either physically in the simulation room and/or was indicated on the patient’s chart. For example, the latex allergy hazard was represented by latex gloves at the bedside despite an allergy indicated on the patient’s chart and wristband. A complete list of the 8 safety hazards and their representations in the simulation is shown in Table 1.

The Choosing Wisely™ lists were reviewed to identify low-value hazards for addition to the simulation.¹⁴ Our team selected 3 low-value hazards from the Society of Hospital Medicine (SHM) list,¹⁵ including (1) arbitrary blood transfusion despite the patient’s stable hemoglobin level of 8.0 g/dL and absence of cardiac symptoms,¹⁶ (2) addition of a proton pump inhibitor (PPI) for stress ulcer prophylaxis in a patient without high risk for gastrointestinal (GI) complications who was not on a PPI prior to admission, and (3) placement of a urinary catheter without medical indication. We had originally selected continuous telemetry monitoring as a fourth hazard from the SHM list, but were unable to operationalize, as it was difficult to simulate continuous telemetry on a mannequin. Because many inpatients are older than 65 years, we reviewed the American Geriatrics Society list¹⁷ and selected our fourth low-value hazard: (4) unnecessary use of physical restraints to manage behavioral symptoms in a hospitalized patient with delirium. Several of these hazards were also quality and safety priorities at our institution, including the overuse of urinary catheters, physical restraints, and blood transfusions. All 4 low-value hazards were referenced in the patient’s door chart, and 3 were also physically represented in the room via presence of a hanging unit of blood, Foley catheter, and upper-arm restraints (Table 1). See Appendix for a photograph of the simulation setup.

Each intern was allowed 10 minutes inside the simulation room. During this time, they were instructed to read the 1-page door chart, inspect the simulation room, and write down as many potential low-value and safety hazards as they could identify on a free-response form (see Appendix). Upon exiting the room, they were allotted 5 additional minutes to complete their free-response answers and provide written feedback on the simulation. The simulation was conducted in 3 simulated hospital rooms over the course of 2 days, and the correct answers were provided via e-mail after all interns had completed the exercise.

To assess prior training and safety knowledge, interns were asked to complete a 3-question preassessment on a ScanTron^TM (Tustin, CA) form. The preassessment asked interns whether they had received training on hospital safety during medical school (yes, no, or unsure), if they were satisfied with the hospital safety training they received during medical school (strongly disagree to strongly agree on a Likert scale), and if they were confident in their ability to identify potential hazards in a hospital setting (strongly disagree to strongly agree). Interns were also given the opportunity to provide feedback on the simulation experience on the ScanTron^TM (Tustin, CA) form.

One month after participating in the simulation, interns were asked to complete an online follow-up survey on MedHub^TM (Ann Arbor, MI), which included 2 Likert-scale questions (strongly disagree to strongly agree) assessing the simulation’s impact on their experience mitigating hospital hazards during the first month of internship.

Data Analysis

Interns’ free-response answers were manually coded, and descriptive statistics were used to summarize the mean percent correct for each hazard. A paired t test was used to compare intern identification of low-value vs safety hazards. T tests were used to compare hazard identification for interns entering highly procedural-intensive specialties (ie, surgical specialties, emergency medicine, anesthesia, obstetrics/gynecology) and those entering less procedural-intensive specialties (ie, internal medicine, pediatrics, psychiatry), as well as among those graduating from “Top 30” medical schools (based on US News & World Report Medical School Rankings¹⁸) and our own institution. One-way analysis of variance (ANOVA) calculations were used to test for differences in hazard identification based on interns’ prior hospital safety training, with interns who rated their satisfaction with prior training or confidence in identifying hazards as a “4” or a “5” considered “satisfied” and “confident,” respectively. Responses to the MedHub^TM (Ann Arbor, MI) survey were dichotomized with “strongly agree” and “agree” considered positive responses. Statistical significance was defined at P < .05. All data analysis was conducted using Stata 14^TM software (College Station, TX).

Intern Characteristics

One hundred twenty-five entering PGY1 interns participated in the simulation, representing 60 medical schools and 7 different specialty groups (Table 2). Thirty-five percent (44/125) were graduates from “Top 30” medical schools, and 8.8% (11/125) graduated from our own institution. Seventy-four percent (89/121) had received prior hospital safety training during medical school, and 62.9% (56/89) were satisfied with their training. A majority of interns (64.2%, 79/123) felt confident in their ability to identify potential hazards in a hospital setting, although confidence was much higher among those with prior safety training (71.9%, 64/89) compared to those without prior training or who were unsure about their training (40.6%, 13/32; P = .09, t test).

Identification of Hazards

The mean percentage of hazards correctly identified by interns during the simulation was 50.4% (standard deviation [SD] 11.8%), with a normal distribution (Figure 1). Interns identified a significantly lower percentage of low-value hazards than safety hazards in the simulation (mean 19.2% [SD 18.6%] vs 66.0% [SD 16.0%], respectively; P < .001, paired t test). Interns also identified significantly more room-based errors than chart-based errors (mean 58.6% [SD 13.4%] vs 9.6% [SD 19.8%], respectively; P < .001, paired t test). The 3 most commonly identified hazards were unavailability of hand hygiene (120/125, 96.0%), presence of latex gloves despite the patient’s allergy (111/125, 88.8%), and fall risk due to the lowered bed rail (107/125, 85.6%). More than half of interns identified the incorrect name on the patient’s wristband and IV bag (91/125, 72.8%), a lack of isolation precautions (90/125, 72.0%), administration of penicillin despite the patient’s allergy (67/125, 53.6%), and unnecessary restraints (64/125, 51.2%). Less than half of interns identified the wrong medication being administered (50/125, 40.0%), unnecessary Foley catheter (25/125, 20.0%), and absence of venous thromboembolism (VTE) prophylaxis (24/125, 19.2%). Few interns identified the unnecessary blood transfusion (7/125, 5.6%), and no one identified the unnecessary stress ulcer prophylaxis (0/125, 0.0%; Figure 2).

Predictors of Hazard Identification

Interns who self-reported as confident in their ability to identify hazards were not any more likely to correctly identify hazards than those who were not confident (50.9% overall hazard identification vs 49.6%, respectively; P = .56, t test). Interns entering into less procedural-intensive specialties identified significantly more safety hazards than those entering highly procedural-intensive specialties (mean 69.1% [SD 16.9%] vs 61.8% [SD 13.7%], respectively; P = .01, t test). However, there was no statistically significant difference in their identification of low-value hazards (mean 19.8% [SD 18.3%] for less procedural-intensive vs 18.4% [SD 19.1%] for highly procedural-intensive; P = .68, t test). There was no statistically significant difference in hazard identification among graduates of “Top 30” medical schools or graduates of our own institution. Prior hospital safety training had no significant impact on interns’ ability to identify safety or low-value hazards. Overall, interns who were satisfied with their prior training identified a mean of 51.8% of hazards present (SD 11.8%), interns who were not satisfied with their prior training identified 51.5% (SD 12.7%), interns with no prior training identified 48.7% (SD 11.7%), and interns who were unsure about their prior training identified 47.4% (SD 11.5%) [F(3,117) = .79; P = .51, ANOVA]. There was also no significant association between prior training and the identification of any one of the 12 specific hazards (chi-square tests, all P values > .1).

Intern Feedback and Follow-Up Survey

Debriefing revealed that most interns passively assumed the patient’s chart was correct and did not think they should question the patient’s current care regimen. For example, many interns commented that they did not think to consider the patient’s blood transfusion as unnecessary, even though they were aware of the recommended hemoglobin cutoffs for stable patients.

Interns also provided formal feedback on the simulation through open-ended comments on their ScanTron^TM (Tustin, CA) form. For example, one intern wrote that they would “inherently approach every patient room ‘looking’ for safety issues, probably directly because of this exercise.” Another commented that the simulation was “more difficult than I expected, but very necessary to facilitate discussion and learning.” One intern wrote that “I wish I had done this earlier in my career.”

Ninety-six percent of participating interns (120/125) completed an online follow-up survey 1 month after beginning internship. In the survey, 68.9% (82/119) of interns indicated they were more aware of how to identify potential hazards facing hospitalized patients as a result of the simulation. Furthermore, 52.1% (62/119) of interns had taken action during internship to reduce a potential hazard that was present in the simulation.

While many GME orientations include simulation and safety training, this study is the first of its kind to incorporate low-value care from Choosing Wisely™ recommendations into simulated training. It is concerning that interns identified significantly fewer low-value hazards than safety hazards in the simulation. In some cases, no interns identified the low-value hazard. For example, while almost all interns identified the hand hygiene hazard, not one could identify the unnecessary stress ulcer prophylaxis. Furthermore, interns who self-reported as confident in their ability to identify hazards did not perform any better in the simulation. Interns entering less procedural-intensive specialties identified more safety hazards overall.

The simulation was well received by interns. Many commented that the experience was engaging, challenging, and effective in cultivating situational awareness towards low-value care. Our follow-up survey demonstrated the majority of interns reported taking action during their first month of internship to reduce a hazard included in the simulation. Most interns also reported a greater awareness of how to identify hospital hazards as a result of the simulation. These findings suggest that a brief simulation-based experience has the potential to create a lasting retention of situational awareness and behavior change.

It is worth exploring why interns identified significantly fewer low-value hazards than safety hazards in the simulation. One hypothesis is that interns were less attuned to low-value hazards, which may reflect a lacking emphasis on value-based care in undergraduate medical education (UME). It is especially concerning that so few interns identified the catheter-associated urinary tract infection (CAUTI) risk, as interns are primarily responsible for recognizing and removing an unnecessary catheter. Although the risks of low-value care should be apparent to most trainees, the process of recognizing and deliberately stopping or avoiding low-value care can be challenging for young clinicians.¹⁹ To promote value-based thinking among entering residents, UME programs should teach students to question the utility of the interventions their patients are receiving. One promising framework for doing so is the Subjective, Objective, Assessment, Plan- (SOAP)-V, in which a V for “Value” is added to the traditional SOAP note.²⁰ SOAP-V notes serve as a cognitive forcing function that requires students to pause and assess the value and cost-consciousness of their patients’ care.²⁰

The results from the “Room of Horrors” simulation can also guide health leaders and educators in identifying institutional areas of focus towards providing high-value and safe care. For example, at the University of Chicago we launched an initiative to improve the inappropriate use of urinary catheters after learning that few of our incoming interns recognized this during the simulation. Institutions could use this model to raise awareness of initiatives and redirect resources from areas that trainees perform well in (eg, hand hygiene) to areas that need improvement (eg, recognition of low-value care). Given the simulation’s low cost and minimal material requirements, it could be easily integrated into existing training programs with the support of an institution’s simulation center.

This study’s limitations include its conduction at single-institution, although the participants represented graduates of 60 different institutions. Furthermore, while the 12 hazards included in the simulation represent patient safety and value initiatives from a wide array of medical societies, they were not intended to be comprehensive and were not tailored to specific specialties. The simulation included only 4 low-value hazards, and future iterations of this exercise should aim to include an equal number of safety and low-value hazards. Furthermore, the evaluation of interns’ prior hospital safety training relied on self-reporting, and the specific context and content of each interns’ training was not examined. Finally, at this point we are unable to provide objective longitudinal data assessing the simulation’s impact on clinical practice and patient outcomes. Subsequent work will assess the sustained impact of the simulation by correlating with institutional data on measurable occurrences of low-value care.

In conclusion, interns identified significantly fewer low-value hazards than safety hazards in an inpatient simulation designed to promote situational awareness. Our results suggest that interns are on the lookout for errors of omission (eg, absence of hand hygiene, absence of isolation precautions) but are often blinded to errors of commission, such that when patients are started on therapies there is an assumption that the therapies are correct and necessary (eg, blood transfusions, stress ulcer prophylaxis). These findings suggest poor awareness of low-value care among incoming interns and highlight the need for additional training in both UME and GME to place a greater emphasis on preventing low-value care.

Disclosure

Dr. Arora is a member of the American Board of Medicine Board of Directors and has received grant funding from ABIM Foundation via Costs of Care for the Teaching Value Choosing Wisely™ Challenge. Dr. Farnan, Dr. Arora, and Ms. Hirsch receive grant funds from Accreditation Council of Graduate Medical Education as part of the Pursuing Excellence Initiative. Dr. Arora and Dr. Farnan also receive grant funds from the American Medical Association Accelerating Change in Medical Education initiative. Kathleen Wiest and Lukas Matern were funded through matching funds of the Pritzker Summer Research Program for NIA T35AG029795.

Inherited mutations in a single gene may contribute to a severe form of atopic dermatitis (AD), a study of eight patients showed.

Investigators from the National Institute of Allergy and Infectious Diseases (NIAID) and elsewhere identified eight individuals with severe AD from four unrelated families. All of the patients had a mutation in the CARD11 gene, which is part of the nuclear factor–kappa B (NF-kB) pathway.

Courtesy National Institute of Allergy and Infectious Diseases

Inherited mutations in a single gene may contribute to a severe form of atopic dermatitis (AD), a study of eight patients showed.

Investigators from the National Institute of Allergy and Infectious Diseases (NIAID) and elsewhere identified eight individuals with severe AD from four unrelated families. All of the patients had a mutation in the CARD11 gene, which is part of the nuclear factor–kappa B (NF-kB) pathway.

Courtesy National Institute of Allergy and Infectious Diseases

Inherited mutations in a single gene may contribute to a severe form of atopic dermatitis (AD), a study of eight patients showed.

Investigators from the National Institute of Allergy and Infectious Diseases (NIAID) and elsewhere identified eight individuals with severe AD from four unrelated families. All of the patients had a mutation in the CARD11 gene, which is part of the nuclear factor–kappa B (NF-kB) pathway.

Courtesy National Institute of Allergy and Infectious Diseases

SEATTLE – As patient satisfaction becomes increasingly important for reimbursements, it might be a good idea to switch to propofol for colonoscopies.

The reason is because patients prefer propofol over standard-of-care fentanyl/midazolam as their anesthetic for outpatient colonoscopies, according to a randomized, blinded trial at a single center. Importantly, clinical assessment also showed that propofol outperformed fentanyl/midazolam in terms of hypoxia, pain, nausea, and procedural difficulties.

Copyright Eraxion/Thinkstock

Alex Otto/Frontline Medical News

[email protected]

SEATTLE – As patient satisfaction becomes increasingly important for reimbursements, it might be a good idea to switch to propofol for colonoscopies.

The reason is because patients prefer propofol over standard-of-care fentanyl/midazolam as their anesthetic for outpatient colonoscopies, according to a randomized, blinded trial at a single center. Importantly, clinical assessment also showed that propofol outperformed fentanyl/midazolam in terms of hypoxia, pain, nausea, and procedural difficulties.

Copyright Eraxion/Thinkstock

Alex Otto/Frontline Medical News

[email protected]

SEATTLE – As patient satisfaction becomes increasingly important for reimbursements, it might be a good idea to switch to propofol for colonoscopies.

The reason is because patients prefer propofol over standard-of-care fentanyl/midazolam as their anesthetic for outpatient colonoscopies, according to a randomized, blinded trial at a single center. Importantly, clinical assessment also showed that propofol outperformed fentanyl/midazolam in terms of hypoxia, pain, nausea, and procedural difficulties.

Copyright Eraxion/Thinkstock

Alex Otto/Frontline Medical News

[email protected]

Seven months into the first year of the Quality Payment Program, the new value-based payment plan set up by the Medicare Access and CHIP Reauthorization Act (MACRA), and doctors’ knowledge of the program is still light.

“Physicians, especially those in small practices, need more help in preparing” for participation in QPP either through the Merit-Based Incentive Payment System (MIPS) or advanced Alternative Payment Models (APMs), according to a new report issued by the American Medical Association and consulting firm KPMG.

TheaDesign/Thinkstock

[email protected]
 

Seven months into the first year of the Quality Payment Program, the new value-based payment plan set up by the Medicare Access and CHIP Reauthorization Act (MACRA), and doctors’ knowledge of the program is still light.

“Physicians, especially those in small practices, need more help in preparing” for participation in QPP either through the Merit-Based Incentive Payment System (MIPS) or advanced Alternative Payment Models (APMs), according to a new report issued by the American Medical Association and consulting firm KPMG.

TheaDesign/Thinkstock

[email protected]
 

Seven months into the first year of the Quality Payment Program, the new value-based payment plan set up by the Medicare Access and CHIP Reauthorization Act (MACRA), and doctors’ knowledge of the program is still light.

“Physicians, especially those in small practices, need more help in preparing” for participation in QPP either through the Merit-Based Incentive Payment System (MIPS) or advanced Alternative Payment Models (APMs), according to a new report issued by the American Medical Association and consulting firm KPMG.

TheaDesign/Thinkstock

[email protected]
 

Take a brand-new research facility, then add a neighboring U.S. Army base with one of the largest veteran populations of any health care network and a world-class team of researchers—that’s a “recipe for success,” says Dr. Michael Russell, director of the VA Center of Excellence for Research on Returning War Veterans in Waco, Texas.

The 53,000-square-foot center is designed to conduct state-of-the-art research on mental health problems associated with PTSD and TBI, “signature wounds” of conflicts in Afghanistan and the Middle East. The flagship study is named Project MAVEREX. Researchers will examine whether the inability of the regions in injured brains to communicate with one another worsens behavior outcomes. Using “cutting-edge data analysis techniques,” they hope to characterize the effects of TBI on brain structure and function “with very high precision,” says Dr. Evan Gordon, a cognitive neuroscientist working on MAVEREX.

The Center of Excellence is on the campus of the historic Doris Miller VAMC. The facility has space for 75 staff members and faculty as well as 25 trainees. It features multiple examination rooms, observation rooms, electrocardiography, electroencephalography, a 3 Tesla MRI, a transcranial magnetic stimulation suite, and a custom-built laboratory wing.

Take a brand-new research facility, then add a neighboring U.S. Army base with one of the largest veteran populations of any health care network and a world-class team of researchers—that’s a “recipe for success,” says Dr. Michael Russell, director of the VA Center of Excellence for Research on Returning War Veterans in Waco, Texas.

The 53,000-square-foot center is designed to conduct state-of-the-art research on mental health problems associated with PTSD and TBI, “signature wounds” of conflicts in Afghanistan and the Middle East. The flagship study is named Project MAVEREX. Researchers will examine whether the inability of the regions in injured brains to communicate with one another worsens behavior outcomes. Using “cutting-edge data analysis techniques,” they hope to characterize the effects of TBI on brain structure and function “with very high precision,” says Dr. Evan Gordon, a cognitive neuroscientist working on MAVEREX.

The Center of Excellence is on the campus of the historic Doris Miller VAMC. The facility has space for 75 staff members and faculty as well as 25 trainees. It features multiple examination rooms, observation rooms, electrocardiography, electroencephalography, a 3 Tesla MRI, a transcranial magnetic stimulation suite, and a custom-built laboratory wing.

Take a brand-new research facility, then add a neighboring U.S. Army base with one of the largest veteran populations of any health care network and a world-class team of researchers—that’s a “recipe for success,” says Dr. Michael Russell, director of the VA Center of Excellence for Research on Returning War Veterans in Waco, Texas.

The 53,000-square-foot center is designed to conduct state-of-the-art research on mental health problems associated with PTSD and TBI, “signature wounds” of conflicts in Afghanistan and the Middle East. The flagship study is named Project MAVEREX. Researchers will examine whether the inability of the regions in injured brains to communicate with one another worsens behavior outcomes. Using “cutting-edge data analysis techniques,” they hope to characterize the effects of TBI on brain structure and function “with very high precision,” says Dr. Evan Gordon, a cognitive neuroscientist working on MAVEREX.

The Center of Excellence is on the campus of the historic Doris Miller VAMC. The facility has space for 75 staff members and faculty as well as 25 trainees. It features multiple examination rooms, observation rooms, electrocardiography, electroencephalography, a 3 Tesla MRI, a transcranial magnetic stimulation suite, and a custom-built laboratory wing.

Lymphedema is the swelling of tissue caused by the accumulation of interstitial fluid in any area of the body where lymphatic flow has been compromised.¹ Secondary lymphedema is an acquired abnormality in lymph drainage^1,2 and is the type commonly seen in cancer patients. Secondary lymphedema can be described as external or internal. Internal lymphedema, swelling of deep structures and tissues, is very difficult to quantify.

Lymphedema in patients with head and neck cancers
Lymphedema is a complicating morbidity frequently seen in head and neck cancer patients who have undergone treatment with surgery, radiation, and chemotherapy. However, although it is one of the most prevalent side effects of treatment, it is both under-recognized and under-treated.³

In head and neck cancer patients, internal swelling may develop in the soft tissues of the upper aero-digestive tract,⁴ affecting articulation and swallowing. Currently, there does not seem to be an effective practical and reliable tool with which to measure internal lymphedema. In addition, it is generally accepted that there is no effective way to treat internal lymphedema. By contrast, external lymphedema is more readily observed, but both subjective and objective assessments are difficult. External swelling may occur in the face, jaw, and neck. However, the subjective scales currently available are insufficient to capture very important characteristics of external lymphedema.⁵ The Edge Task Force on Head and Neck Cancer in 2015 was not able to recommend any outcome measures for objectively quantifying external edema.⁶ Furthermore, objective measurements of head and neck lymphedema can be expensive and time consuming.

Extent and risk

A combination of both internal and external swelling is seen in more than 50% of patients.⁷ Risk factors include “throat” tumors, multicancer treatment approaches, higher total radiation dose, a greater number of radiation procedures, and radiation at the surgical site.⁵ More than 500,000 survivors of head and neck cancer in the United States are at risk of lymphedema.⁵ Although recent advances in treatment have reduced the incidence of other morbidities, 50% of patients who are treated for head and neck cancer may still develop lymphedema.^1,8 The reported incidence in some centers may be much higher, with up to 75% of patients developing lymphedema following treatment.⁹

Measurement modalities for clinical evaluation

There is little current research into lymphedema of the head and neck, despite the high prevalence of the condition.⁸ According to Deng and colleagues, measurement of head and neck lymphedema is a challenge, which has an impact on clinical assessment, diagnosis, and treatment of this under-recognized, under-reported and under-addressed problem in head and neck cancer patients.¹⁰ In a review of the literature, Deng and colleagues identified three measurement modalities available for clinical evaluation: patient-reported outcomes, clinician-reported outcomes, and technology.¹⁰ One major factor, though, in detecting lymphedema, is physician awareness: physicians, health care professionals, and even some lymphedema therapists are not well educated about this problem.⁸

Treatment

The effectiveness of traditional lymphedema treatment is not well defined.⁸ Currently, complete decongestive therapy (CDT), is considered the standard of care for lymphedema. The National Lymphedema Network has stated that modifications of CDT, especially manual lymphatic drainage and modified compressive garments for external lymphedema, have been shown to be beneficial for the treatment of lymphedema in head and neck cancer patients.¹¹ Most findings in lymphedema research, mainly in breast cancer patients, have shown that early intervention is the best management and yields the best outcomes. As with other chronic conditions, early identification and timely, appropriate treatment of lymphedema is critical to improve clinical outcomes, to decrease symptom burden and functional impairment, and to improve overall quality of life in head and neck cancer patients.¹⁰

Improving recognition and treatment

Head and neck oncologic treatment is increasingly offered outside the network of specialist academic hospitals, at hospitals serving more localized communities where the neediest, sickest patient groups may be receiving less than optimal care.³ This challenges community hospitals to provide optimal treatment, similar to that being offered at nationally recognized institutions. In January 2012, we implemented a prehabilitation program in our community hospital cancer center to provide early intervention for our patients based on the understanding that proper and prompt treatment for patients with early signs of lymphedema should be a priority.¹² In this article, we outline how we implemented the program and the describe improvements we observed before and after the implementation of the program.

The prehabilitation program

The role of the nurse navigator

Before the introduction of the prehabilitation program, our pattern of practice was to refer patients to oncology rehabilitation for lymphedema management after they had completed their medical treatment with surgery, radiation, and chemotherapy. In 2012, that was changed to a prehabilitation model of care that was overseen by a head and neck nurse navigator. This focus on prehabilitation begins with patients being referred to oncology rehabilitation at the time of cancer diagnosis for baseline assessment of head and neck swelling. In addition, there is assessment of the many possible other side effects associated with head and neck cancer and its treatment, namely loss of range of motion of the neck, jaw (trismus), and/or shoulders, postural deficits, functional loss, pain, balance dysfunction with fall risk, weakness, and fatigue. Therapeutic interventions are initiated as needed and appropriate. This process also raises awareness of a condition that has been described as under-recognized and under-treated.³

The nurse navigator sits in on each radiation oncology consultation and aids in “navigating” patients through their treatment. The nurse ensures that each patient is referred to different ancillary services from the outset, such as seeing a dietician, social worker, physical/occupational therapist and certified lymphedema therapist, speech pathologist, and financial assistance advisor, if necessary (Table 1).

Assessment of lymphedema

Measurement of head and neck lymphedema is a challenge.¹⁰ In our program, the physical therapy assessment also includes the evaluation of several other morbidities associated with head and neck cancer and its treatment, such as range of motion, weakness, fatigue, radiation fibrosis, balance dysfunction, and risk of falling (Table 2).

Although objective criteria for external lymphedema have not been established, simple measurements such as using a tape measure to record neck circumference, allow a useful longitudinal assessment. Digital photography may be effective in the documentation and subjective evaluation of changes of external lymphedema.^10,17 However, there are some limitations with photography because although external photographs (including digital photography and three-dimensional imaging) can capture some features, such as changes in contours, symmetry, and changes in skin quality and color, they do not detect changes in skin and soft tissue texture and compliance (Table 3).¹⁰

Impact on clinical outcomes

We retrospectively reviewed the medical records of 230 head and neck cancer patients who had been treated at our center between June 2008 and June 2015. Complete clinical data were available for 190 patients. The following information was extracted from each patient’s chart: whether they developed lymphedema, tumor stage, had surgery, radiation dose, type of chemotherapy given, their smoking history, if they had had a neck dissection and the primary site of the tumor (Table 3).

Incidence in different time periods. Of the 190 patients with complete records 78 (41%) were found to have lymphedema. These were all patients undergoing treatment for head and neck cancer during June 2008-June 2015. The prehabilitation program was initiated with the hiring of a nurse navigator for head and neck cancer, starting in January 2012. It is interesting to note that the incidence of lymphedema was 27% before the program was started, but after nurse navigator joined the team, the incidence increased significantly to 48% (P = .0002), in line with published expectations. This increase in recorded incidence may be attributable to the greater awareness of lymphedema intentionally fostered by the prehabilitation program.

Smoking history. Patients’ lifetime smoking history was retrieved from their medical records, based on their verbal admission of tobacco use. Most of the patients (n = 110) self-reported a history of smoking. Of those with a history of smoking, 36 (33%) developed external lymphedema after treatment for head and neck cancer, and 74 (67%) did not. However, this difference was not statistically significant. Hence, although smoking is a risk factor for head and neck cancer, it was not associated with the development of external lymphedema in our cohort of patients.

Type of tumor

Most of the patients (n = 156, 82%) had squamous cell carcinomas (SCC). Of those, 45% developed external lymphedema and 55% did not. Therefore, having SCC did not predispose to lymphedema. The other cancers were mixed type, mainly adenocaricoma, but their numbers were too small to draw statistical conclusions.

Stage of the tumor

About two thirds of the patients (n = 121, 64%) had stage 3 or 4 cancer. However, treatment of more advanced cancers was not associated with lymphedema development.

Site of the tumor

The literature suggests that patients with a primary tumor in the throat are at increased risk for lymphedema.⁵ The American Cancer Society has defined cancers of the oropharynx (throat) as including the base of the tongue (back third of the tongue), the soft palate, the tonsils, and the side and back walls of the throat.¹⁸ In our head and neck cancer cohort, patients with primary tumors of the oropharnyx were, perhaps, more susceptible to lymphedema (P = .044, Table 3). By contrast, in our cohort of patients, those with nasopharyngeal, hypopharyngeal, and parotid gland tumors were significantly less likely to develop lymphedema (Ps = .017, .04, .012, respectively).

No surgery

Half of our patients (n = 95) were not treated with surgery. In the patients who did not have surgery, 25 (26%) developed lymphedema, whereas 70 (74%) did not. Hence, although the incidence of lymphedema was significantly lower in patients who did not have surgery (P = .015), lymphedema did develop in patients who did not have a surgical procedure.

Resection of primary tumor without neck dissection

Of the 64 patients who had surgery, but without neck dissection, 35 (55%) developed external lymphedema. Compared with the no-surgery patients, the doubling of the incidence (from 26% to 55%) was highly significant (P = .0004). These findings are compatible with the literature reports that surgery increases the incidence of lymphedema, which is not surprising because surgery and subsequent scarring is known to compromise the lymphatic system.

Resection of primary tumor with neck dissection

The incidence of external lymphedema was increased to 69% when patients were subjected to both surgery and neck dissection. Compared with the June 2008-June 2015 cohort, there was a significant increase in the incidence of lymphedema in the neck dissection group (P = .007). Neck dissection involves the removal of lymph nodes and disruption of the lymphatic vessels, so it is not surprising that there is a higher incidence of external lymphedema. In our practice, neck dissections increased in frequency every year from June 2008 until December 2011, when 8 patients underwent neck dissections, 6 (75%) of whom developed lymphedema. Since January 2012, when the prehabilitation program was implemented, the number of neck dissections have declined, with more patients receiving chemoradiation and surgery being reserved for surgery. Hamoir and colleagues have reported that neck dissection is no longer justified unless there is clinically residual disease in the neck.¹⁹

Radiation

Lymphedema occurred in patients regardless of the dose of radiation received. Although the incidence of lymphedema seemed to be higher in patients who received more than 60 cGy, that difference was not statistically significant (Table 3). We had expected a relationship between radiation damage and greater lymphedema, but that was not evident in our patients.

Chemotherapy

The majority of patients (n = 131, 69%) received chemotherapy. The exposure to chemotherapy was not correlated with the risk of external lymphedema in our cohort of patients, with 58 of the 131 treated patients (44%) developing lymphedema, compared with 73 (56%) of treated patients who did not (Table 3).

Complete decongestive therapy

All patients with documented lymphedema were evaluated for complete decongestive therapy (CDT). Contraindications to CDT included congestive heart failure, renal failure, acute infection, peripheral artery disease, upper-quadrant deep vein thrombosis, and carotid artery stenosis. Eligible patients were referred to a certified lymphedema therapist for CDT. As the program evolved, patients at risk for lymphedema were referred for CDT early on, usually at the time of diagnosis, to improve early identification and surveillance of lymphedema.

CDT included manual lymph drainage, compression bandaging (Figure), decongestive exercises, skin care, and education in swelling self-management. Manual lymph drainage is a specialized light pressure hands-on technique that reduces swelling by enhancing lymphatic reabsorption and flow. Compressive bandaging/garments increase venous and lymphatic drainage and soften fibrotic tissue. Continued use of compression depends on progress. In head and neck cancer patients, the need for lifelong compression is not evident when they are treated early and there is good patient compliance.⁸ Therapeutic exercise enhances lymphatic and venous circulation, and good skin care reduces the risk of infection.

Communication and education

The head and neck cancer nurse navigator attends the cancer center’s multidisciplinary head and neck tumor board, which has representation from otolaryngology, diagnostic radiology, pathology, radiation oncology, medical oncology, reconstructive surgery, oncology rehabilitation (physical/occupational therapist), dietary services, speech pathology, social services and clinical research. This regular contact allows for earlier awareness about which patients are at greater risk for developing lymphedema, thus enabling early intervention (and patient education) in a timely manner.

Education of the patient, before cancer therapy, of the risks of lymphedema is very important. Before the implementation of the prehabilitation program, some patients did not fully comprehend what a painful and debilitating consequence of cancer treatment lymphedema could be.

Discussion

We introduced a prehabilitation program to detect and treat lymphedema in head and neck cancer patients in January 2012 part way through following an observation cohort from June2008 through June2015. Central to this, in our center, was the appointment of a nurse navigator whose primary focus was on head and neck cancer patients. We placed a high priority on the early detection and treatment of lymphedema because do so has been associated with better outcomes in other centers.

One immediate consequence of the inception of our program was the identification of more patients with external lymphedema. Our detected incidence rose significantly (P = .0002), from 27% in the period June 2008-December 20112010, before the program, to 48% during the January 2012-June 2015 period, after the inception of the program. This later incidence rate is in line with published incidence rates in most centers. However, it is still somewhat short of the 75% suggested in one center,⁹ which suggests we are either we are underdetecting lymphedema or there are differences in definition criteria or sensitivity levels for defining lymphedema.

There are currently no specific objective measures of lymphedema, so there is bound to be some variation in diagnosis rates. In our program, we rely heavily on the patient-reported outcome measures, the NDI instrument, and digital photography to detect and monitor lymphedema, starting with the pretreatment baseline values that are established for each patient.

The use of digital photography in our community hospital setting, which includes taking photographs before and after treatment and at each visit, motivates and encourages patients and provides a tool for clinical lymphedema therapists to visually document benefits of treatment. Patients’ motivation and compliance with their established home program for head and neck lymphedema self-management are essential. The elements of the home program may include self-manual lymph drainage, home-modified compression bandaging and garment wear, therapeutic exercises, and skin care. Patients with lymphedema who adhered closely with their therapy program were more than 8 times more likely to improve compared with noncompliant patients.¹⁷

Some groups of patients have a greater risk of developing lymphedema than others,⁵ so the development of an algorithm to predict lymphedema seemed possible. However, in our cohort of patients, only neck dissection, with its disruption of the lymphatic system of the neck, was strongly associated with external lymphedema (Table 3). It is important to note that some patients who did not undergo surgery developed lymphedema. In our patients, high doses of radiation alone did not seem to predispose to lymphedema. That suggests that no group of head and neck cancer patients should be ignored, which is why we did routine screening of all patients before, during, and after treatment.

Our protocol falls short in the detection of internal lymphedema. For example, information on swallowing gathered by our speech pathologists (in a different department) has not, so far, been included in our assessment. This is one opportunity to improve on our approach, especially because speech difficulties may be associated with internal lymphedema. In addition, we are not equipped for the requisite internal examinations. Unfortunately, there are no practical and successful treatments for patients suffering from internal swelling. This represents a challenge for the medical community to better meet this need. Therefore, although we are missing some assessments of internal lymphedema, this is of little therapeutic consequence at this time.

The increase in the detected incidence of external lymphedema points to a practice gap that has been resolved by the appointment of a dedicated nurse navigator who attends oncology reviews to share knowledge and information. Another educational effort has been made with the patients themselves to increase compliance and improve continuous care at home.

There is always room for improvement, however, either by feedback acquired from other institutions and hospitals or through the future introduction of more objective assessment techniques.

Conclusions

The introduction of the prehabilitation program at our center has coincided with a significantly improved detection rate for external lymphedema in head and neck cancer patients. It may be because the program emphasizes education about lymphedema that awareness of the condition has increased throughout the center. It is now widely recognized that all patients are at risk of lymphedema regardless of whether they fall into an acknowledged high-risk group. Our experience shows that there is no significant difference between treatment modalities apart from neck dissection. In our population, the use of this procedure is decreasing. External lymphedema can develop even in patients who do not have surgery. Therefore, there is no sound way to predict which patients are most likely to suffer from the accumulation of fluid in their head and neck after treatment for head and neck cancer. Thus, an assessment as described here, during and after treatment for all patients, is warranted. Patients are now being seen earlier as a part of the prehabilitation program, which facilitates access to complete decongestive treatment at an earlier stage, improves patient outcomes, and increases patient satisfaction with their treatment. Our prehabilitation program could serve as a model for other community hospital centers in achieving outcomes that are as good as those in academic centers.

Acknowledgments

The authors thank Irene Kadota and Heather Peters, from the Department of Radiation Oncology, and Julianne Courtenay, from the Department of Physical Therapy at the Disney Family Cancer Center, Burbank, California, for providing the original clinical data for analysis.

Lymphedema is the swelling of tissue caused by the accumulation of interstitial fluid in any area of the body where lymphatic flow has been compromised.¹ Secondary lymphedema is an acquired abnormality in lymph drainage^1,2 and is the type commonly seen in cancer patients. Secondary lymphedema can be described as external or internal. Internal lymphedema, swelling of deep structures and tissues, is very difficult to quantify.

Lymphedema in patients with head and neck cancers
Lymphedema is a complicating morbidity frequently seen in head and neck cancer patients who have undergone treatment with surgery, radiation, and chemotherapy. However, although it is one of the most prevalent side effects of treatment, it is both under-recognized and under-treated.³

In head and neck cancer patients, internal swelling may develop in the soft tissues of the upper aero-digestive tract,⁴ affecting articulation and swallowing. Currently, there does not seem to be an effective practical and reliable tool with which to measure internal lymphedema. In addition, it is generally accepted that there is no effective way to treat internal lymphedema. By contrast, external lymphedema is more readily observed, but both subjective and objective assessments are difficult. External swelling may occur in the face, jaw, and neck. However, the subjective scales currently available are insufficient to capture very important characteristics of external lymphedema.⁵ The Edge Task Force on Head and Neck Cancer in 2015 was not able to recommend any outcome measures for objectively quantifying external edema.⁶ Furthermore, objective measurements of head and neck lymphedema can be expensive and time consuming.

Extent and risk

A combination of both internal and external swelling is seen in more than 50% of patients.⁷ Risk factors include “throat” tumors, multicancer treatment approaches, higher total radiation dose, a greater number of radiation procedures, and radiation at the surgical site.⁵ More than 500,000 survivors of head and neck cancer in the United States are at risk of lymphedema.⁵ Although recent advances in treatment have reduced the incidence of other morbidities, 50% of patients who are treated for head and neck cancer may still develop lymphedema.^1,8 The reported incidence in some centers may be much higher, with up to 75% of patients developing lymphedema following treatment.⁹

Measurement modalities for clinical evaluation

There is little current research into lymphedema of the head and neck, despite the high prevalence of the condition.⁸ According to Deng and colleagues, measurement of head and neck lymphedema is a challenge, which has an impact on clinical assessment, diagnosis, and treatment of this under-recognized, under-reported and under-addressed problem in head and neck cancer patients.¹⁰ In a review of the literature, Deng and colleagues identified three measurement modalities available for clinical evaluation: patient-reported outcomes, clinician-reported outcomes, and technology.¹⁰ One major factor, though, in detecting lymphedema, is physician awareness: physicians, health care professionals, and even some lymphedema therapists are not well educated about this problem.⁸

Treatment

The effectiveness of traditional lymphedema treatment is not well defined.⁸ Currently, complete decongestive therapy (CDT), is considered the standard of care for lymphedema. The National Lymphedema Network has stated that modifications of CDT, especially manual lymphatic drainage and modified compressive garments for external lymphedema, have been shown to be beneficial for the treatment of lymphedema in head and neck cancer patients.¹¹ Most findings in lymphedema research, mainly in breast cancer patients, have shown that early intervention is the best management and yields the best outcomes. As with other chronic conditions, early identification and timely, appropriate treatment of lymphedema is critical to improve clinical outcomes, to decrease symptom burden and functional impairment, and to improve overall quality of life in head and neck cancer patients.¹⁰

Improving recognition and treatment

Head and neck oncologic treatment is increasingly offered outside the network of specialist academic hospitals, at hospitals serving more localized communities where the neediest, sickest patient groups may be receiving less than optimal care.³ This challenges community hospitals to provide optimal treatment, similar to that being offered at nationally recognized institutions. In January 2012, we implemented a prehabilitation program in our community hospital cancer center to provide early intervention for our patients based on the understanding that proper and prompt treatment for patients with early signs of lymphedema should be a priority.¹² In this article, we outline how we implemented the program and the describe improvements we observed before and after the implementation of the program.

The prehabilitation program

The role of the nurse navigator

Before the introduction of the prehabilitation program, our pattern of practice was to refer patients to oncology rehabilitation for lymphedema management after they had completed their medical treatment with surgery, radiation, and chemotherapy. In 2012, that was changed to a prehabilitation model of care that was overseen by a head and neck nurse navigator. This focus on prehabilitation begins with patients being referred to oncology rehabilitation at the time of cancer diagnosis for baseline assessment of head and neck swelling. In addition, there is assessment of the many possible other side effects associated with head and neck cancer and its treatment, namely loss of range of motion of the neck, jaw (trismus), and/or shoulders, postural deficits, functional loss, pain, balance dysfunction with fall risk, weakness, and fatigue. Therapeutic interventions are initiated as needed and appropriate. This process also raises awareness of a condition that has been described as under-recognized and under-treated.³

The nurse navigator sits in on each radiation oncology consultation and aids in “navigating” patients through their treatment. The nurse ensures that each patient is referred to different ancillary services from the outset, such as seeing a dietician, social worker, physical/occupational therapist and certified lymphedema therapist, speech pathologist, and financial assistance advisor, if necessary (Table 1).

Assessment of lymphedema

Measurement of head and neck lymphedema is a challenge.¹⁰ In our program, the physical therapy assessment also includes the evaluation of several other morbidities associated with head and neck cancer and its treatment, such as range of motion, weakness, fatigue, radiation fibrosis, balance dysfunction, and risk of falling (Table 2).

Although objective criteria for external lymphedema have not been established, simple measurements such as using a tape measure to record neck circumference, allow a useful longitudinal assessment. Digital photography may be effective in the documentation and subjective evaluation of changes of external lymphedema.^10,17 However, there are some limitations with photography because although external photographs (including digital photography and three-dimensional imaging) can capture some features, such as changes in contours, symmetry, and changes in skin quality and color, they do not detect changes in skin and soft tissue texture and compliance (Table 3).¹⁰

Impact on clinical outcomes

We retrospectively reviewed the medical records of 230 head and neck cancer patients who had been treated at our center between June 2008 and June 2015. Complete clinical data were available for 190 patients. The following information was extracted from each patient’s chart: whether they developed lymphedema, tumor stage, had surgery, radiation dose, type of chemotherapy given, their smoking history, if they had had a neck dissection and the primary site of the tumor (Table 3).

Incidence in different time periods. Of the 190 patients with complete records 78 (41%) were found to have lymphedema. These were all patients undergoing treatment for head and neck cancer during June 2008-June 2015. The prehabilitation program was initiated with the hiring of a nurse navigator for head and neck cancer, starting in January 2012. It is interesting to note that the incidence of lymphedema was 27% before the program was started, but after nurse navigator joined the team, the incidence increased significantly to 48% (P = .0002), in line with published expectations. This increase in recorded incidence may be attributable to the greater awareness of lymphedema intentionally fostered by the prehabilitation program.

Smoking history. Patients’ lifetime smoking history was retrieved from their medical records, based on their verbal admission of tobacco use. Most of the patients (n = 110) self-reported a history of smoking. Of those with a history of smoking, 36 (33%) developed external lymphedema after treatment for head and neck cancer, and 74 (67%) did not. However, this difference was not statistically significant. Hence, although smoking is a risk factor for head and neck cancer, it was not associated with the development of external lymphedema in our cohort of patients.

Type of tumor

Most of the patients (n = 156, 82%) had squamous cell carcinomas (SCC). Of those, 45% developed external lymphedema and 55% did not. Therefore, having SCC did not predispose to lymphedema. The other cancers were mixed type, mainly adenocaricoma, but their numbers were too small to draw statistical conclusions.

Stage of the tumor

About two thirds of the patients (n = 121, 64%) had stage 3 or 4 cancer. However, treatment of more advanced cancers was not associated with lymphedema development.

Site of the tumor

The literature suggests that patients with a primary tumor in the throat are at increased risk for lymphedema.⁵ The American Cancer Society has defined cancers of the oropharynx (throat) as including the base of the tongue (back third of the tongue), the soft palate, the tonsils, and the side and back walls of the throat.¹⁸ In our head and neck cancer cohort, patients with primary tumors of the oropharnyx were, perhaps, more susceptible to lymphedema (P = .044, Table 3). By contrast, in our cohort of patients, those with nasopharyngeal, hypopharyngeal, and parotid gland tumors were significantly less likely to develop lymphedema (Ps = .017, .04, .012, respectively).

No surgery

Half of our patients (n = 95) were not treated with surgery. In the patients who did not have surgery, 25 (26%) developed lymphedema, whereas 70 (74%) did not. Hence, although the incidence of lymphedema was significantly lower in patients who did not have surgery (P = .015), lymphedema did develop in patients who did not have a surgical procedure.

Resection of primary tumor without neck dissection

Of the 64 patients who had surgery, but without neck dissection, 35 (55%) developed external lymphedema. Compared with the no-surgery patients, the doubling of the incidence (from 26% to 55%) was highly significant (P = .0004). These findings are compatible with the literature reports that surgery increases the incidence of lymphedema, which is not surprising because surgery and subsequent scarring is known to compromise the lymphatic system.

Resection of primary tumor with neck dissection

The incidence of external lymphedema was increased to 69% when patients were subjected to both surgery and neck dissection. Compared with the June 2008-June 2015 cohort, there was a significant increase in the incidence of lymphedema in the neck dissection group (P = .007). Neck dissection involves the removal of lymph nodes and disruption of the lymphatic vessels, so it is not surprising that there is a higher incidence of external lymphedema. In our practice, neck dissections increased in frequency every year from June 2008 until December 2011, when 8 patients underwent neck dissections, 6 (75%) of whom developed lymphedema. Since January 2012, when the prehabilitation program was implemented, the number of neck dissections have declined, with more patients receiving chemoradiation and surgery being reserved for surgery. Hamoir and colleagues have reported that neck dissection is no longer justified unless there is clinically residual disease in the neck.¹⁹

Radiation

Lymphedema occurred in patients regardless of the dose of radiation received. Although the incidence of lymphedema seemed to be higher in patients who received more than 60 cGy, that difference was not statistically significant (Table 3). We had expected a relationship between radiation damage and greater lymphedema, but that was not evident in our patients.

Chemotherapy

The majority of patients (n = 131, 69%) received chemotherapy. The exposure to chemotherapy was not correlated with the risk of external lymphedema in our cohort of patients, with 58 of the 131 treated patients (44%) developing lymphedema, compared with 73 (56%) of treated patients who did not (Table 3).

Complete decongestive therapy

All patients with documented lymphedema were evaluated for complete decongestive therapy (CDT). Contraindications to CDT included congestive heart failure, renal failure, acute infection, peripheral artery disease, upper-quadrant deep vein thrombosis, and carotid artery stenosis. Eligible patients were referred to a certified lymphedema therapist for CDT. As the program evolved, patients at risk for lymphedema were referred for CDT early on, usually at the time of diagnosis, to improve early identification and surveillance of lymphedema.

CDT included manual lymph drainage, compression bandaging (Figure), decongestive exercises, skin care, and education in swelling self-management. Manual lymph drainage is a specialized light pressure hands-on technique that reduces swelling by enhancing lymphatic reabsorption and flow. Compressive bandaging/garments increase venous and lymphatic drainage and soften fibrotic tissue. Continued use of compression depends on progress. In head and neck cancer patients, the need for lifelong compression is not evident when they are treated early and there is good patient compliance.⁸ Therapeutic exercise enhances lymphatic and venous circulation, and good skin care reduces the risk of infection.

Communication and education

The head and neck cancer nurse navigator attends the cancer center’s multidisciplinary head and neck tumor board, which has representation from otolaryngology, diagnostic radiology, pathology, radiation oncology, medical oncology, reconstructive surgery, oncology rehabilitation (physical/occupational therapist), dietary services, speech pathology, social services and clinical research. This regular contact allows for earlier awareness about which patients are at greater risk for developing lymphedema, thus enabling early intervention (and patient education) in a timely manner.

Education of the patient, before cancer therapy, of the risks of lymphedema is very important. Before the implementation of the prehabilitation program, some patients did not fully comprehend what a painful and debilitating consequence of cancer treatment lymphedema could be.

Discussion

We introduced a prehabilitation program to detect and treat lymphedema in head and neck cancer patients in January 2012 part way through following an observation cohort from June2008 through June2015. Central to this, in our center, was the appointment of a nurse navigator whose primary focus was on head and neck cancer patients. We placed a high priority on the early detection and treatment of lymphedema because do so has been associated with better outcomes in other centers.

One immediate consequence of the inception of our program was the identification of more patients with external lymphedema. Our detected incidence rose significantly (P = .0002), from 27% in the period June 2008-December 20112010, before the program, to 48% during the January 2012-June 2015 period, after the inception of the program. This later incidence rate is in line with published incidence rates in most centers. However, it is still somewhat short of the 75% suggested in one center,⁹ which suggests we are either we are underdetecting lymphedema or there are differences in definition criteria or sensitivity levels for defining lymphedema.

There are currently no specific objective measures of lymphedema, so there is bound to be some variation in diagnosis rates. In our program, we rely heavily on the patient-reported outcome measures, the NDI instrument, and digital photography to detect and monitor lymphedema, starting with the pretreatment baseline values that are established for each patient.

The use of digital photography in our community hospital setting, which includes taking photographs before and after treatment and at each visit, motivates and encourages patients and provides a tool for clinical lymphedema therapists to visually document benefits of treatment. Patients’ motivation and compliance with their established home program for head and neck lymphedema self-management are essential. The elements of the home program may include self-manual lymph drainage, home-modified compression bandaging and garment wear, therapeutic exercises, and skin care. Patients with lymphedema who adhered closely with their therapy program were more than 8 times more likely to improve compared with noncompliant patients.¹⁷

Some groups of patients have a greater risk of developing lymphedema than others,⁵ so the development of an algorithm to predict lymphedema seemed possible. However, in our cohort of patients, only neck dissection, with its disruption of the lymphatic system of the neck, was strongly associated with external lymphedema (Table 3). It is important to note that some patients who did not undergo surgery developed lymphedema. In our patients, high doses of radiation alone did not seem to predispose to lymphedema. That suggests that no group of head and neck cancer patients should be ignored, which is why we did routine screening of all patients before, during, and after treatment.

Our protocol falls short in the detection of internal lymphedema. For example, information on swallowing gathered by our speech pathologists (in a different department) has not, so far, been included in our assessment. This is one opportunity to improve on our approach, especially because speech difficulties may be associated with internal lymphedema. In addition, we are not equipped for the requisite internal examinations. Unfortunately, there are no practical and successful treatments for patients suffering from internal swelling. This represents a challenge for the medical community to better meet this need. Therefore, although we are missing some assessments of internal lymphedema, this is of little therapeutic consequence at this time.

The increase in the detected incidence of external lymphedema points to a practice gap that has been resolved by the appointment of a dedicated nurse navigator who attends oncology reviews to share knowledge and information. Another educational effort has been made with the patients themselves to increase compliance and improve continuous care at home.

There is always room for improvement, however, either by feedback acquired from other institutions and hospitals or through the future introduction of more objective assessment techniques.

Conclusions

The introduction of the prehabilitation program at our center has coincided with a significantly improved detection rate for external lymphedema in head and neck cancer patients. It may be because the program emphasizes education about lymphedema that awareness of the condition has increased throughout the center. It is now widely recognized that all patients are at risk of lymphedema regardless of whether they fall into an acknowledged high-risk group. Our experience shows that there is no significant difference between treatment modalities apart from neck dissection. In our population, the use of this procedure is decreasing. External lymphedema can develop even in patients who do not have surgery. Therefore, there is no sound way to predict which patients are most likely to suffer from the accumulation of fluid in their head and neck after treatment for head and neck cancer. Thus, an assessment as described here, during and after treatment for all patients, is warranted. Patients are now being seen earlier as a part of the prehabilitation program, which facilitates access to complete decongestive treatment at an earlier stage, improves patient outcomes, and increases patient satisfaction with their treatment. Our prehabilitation program could serve as a model for other community hospital centers in achieving outcomes that are as good as those in academic centers.

Acknowledgments

The authors thank Irene Kadota and Heather Peters, from the Department of Radiation Oncology, and Julianne Courtenay, from the Department of Physical Therapy at the Disney Family Cancer Center, Burbank, California, for providing the original clinical data for analysis.

Lymphedema is the swelling of tissue caused by the accumulation of interstitial fluid in any area of the body where lymphatic flow has been compromised.¹ Secondary lymphedema is an acquired abnormality in lymph drainage^1,2 and is the type commonly seen in cancer patients. Secondary lymphedema can be described as external or internal. Internal lymphedema, swelling of deep structures and tissues, is very difficult to quantify.

Lymphedema in patients with head and neck cancers
Lymphedema is a complicating morbidity frequently seen in head and neck cancer patients who have undergone treatment with surgery, radiation, and chemotherapy. However, although it is one of the most prevalent side effects of treatment, it is both under-recognized and under-treated.³

In head and neck cancer patients, internal swelling may develop in the soft tissues of the upper aero-digestive tract,⁴ affecting articulation and swallowing. Currently, there does not seem to be an effective practical and reliable tool with which to measure internal lymphedema. In addition, it is generally accepted that there is no effective way to treat internal lymphedema. By contrast, external lymphedema is more readily observed, but both subjective and objective assessments are difficult. External swelling may occur in the face, jaw, and neck. However, the subjective scales currently available are insufficient to capture very important characteristics of external lymphedema.⁵ The Edge Task Force on Head and Neck Cancer in 2015 was not able to recommend any outcome measures for objectively quantifying external edema.⁶ Furthermore, objective measurements of head and neck lymphedema can be expensive and time consuming.

Extent and risk

A combination of both internal and external swelling is seen in more than 50% of patients.⁷ Risk factors include “throat” tumors, multicancer treatment approaches, higher total radiation dose, a greater number of radiation procedures, and radiation at the surgical site.⁵ More than 500,000 survivors of head and neck cancer in the United States are at risk of lymphedema.⁵ Although recent advances in treatment have reduced the incidence of other morbidities, 50% of patients who are treated for head and neck cancer may still develop lymphedema.^1,8 The reported incidence in some centers may be much higher, with up to 75% of patients developing lymphedema following treatment.⁹

Measurement modalities for clinical evaluation

There is little current research into lymphedema of the head and neck, despite the high prevalence of the condition.⁸ According to Deng and colleagues, measurement of head and neck lymphedema is a challenge, which has an impact on clinical assessment, diagnosis, and treatment of this under-recognized, under-reported and under-addressed problem in head and neck cancer patients.¹⁰ In a review of the literature, Deng and colleagues identified three measurement modalities available for clinical evaluation: patient-reported outcomes, clinician-reported outcomes, and technology.¹⁰ One major factor, though, in detecting lymphedema, is physician awareness: physicians, health care professionals, and even some lymphedema therapists are not well educated about this problem.⁸

Treatment

The effectiveness of traditional lymphedema treatment is not well defined.⁸ Currently, complete decongestive therapy (CDT), is considered the standard of care for lymphedema. The National Lymphedema Network has stated that modifications of CDT, especially manual lymphatic drainage and modified compressive garments for external lymphedema, have been shown to be beneficial for the treatment of lymphedema in head and neck cancer patients.¹¹ Most findings in lymphedema research, mainly in breast cancer patients, have shown that early intervention is the best management and yields the best outcomes. As with other chronic conditions, early identification and timely, appropriate treatment of lymphedema is critical to improve clinical outcomes, to decrease symptom burden and functional impairment, and to improve overall quality of life in head and neck cancer patients.¹⁰

Improving recognition and treatment

Head and neck oncologic treatment is increasingly offered outside the network of specialist academic hospitals, at hospitals serving more localized communities where the neediest, sickest patient groups may be receiving less than optimal care.³ This challenges community hospitals to provide optimal treatment, similar to that being offered at nationally recognized institutions. In January 2012, we implemented a prehabilitation program in our community hospital cancer center to provide early intervention for our patients based on the understanding that proper and prompt treatment for patients with early signs of lymphedema should be a priority.¹² In this article, we outline how we implemented the program and the describe improvements we observed before and after the implementation of the program.

The prehabilitation program

The role of the nurse navigator

Before the introduction of the prehabilitation program, our pattern of practice was to refer patients to oncology rehabilitation for lymphedema management after they had completed their medical treatment with surgery, radiation, and chemotherapy. In 2012, that was changed to a prehabilitation model of care that was overseen by a head and neck nurse navigator. This focus on prehabilitation begins with patients being referred to oncology rehabilitation at the time of cancer diagnosis for baseline assessment of head and neck swelling. In addition, there is assessment of the many possible other side effects associated with head and neck cancer and its treatment, namely loss of range of motion of the neck, jaw (trismus), and/or shoulders, postural deficits, functional loss, pain, balance dysfunction with fall risk, weakness, and fatigue. Therapeutic interventions are initiated as needed and appropriate. This process also raises awareness of a condition that has been described as under-recognized and under-treated.³

The nurse navigator sits in on each radiation oncology consultation and aids in “navigating” patients through their treatment. The nurse ensures that each patient is referred to different ancillary services from the outset, such as seeing a dietician, social worker, physical/occupational therapist and certified lymphedema therapist, speech pathologist, and financial assistance advisor, if necessary (Table 1).

Assessment of lymphedema

Measurement of head and neck lymphedema is a challenge.¹⁰ In our program, the physical therapy assessment also includes the evaluation of several other morbidities associated with head and neck cancer and its treatment, such as range of motion, weakness, fatigue, radiation fibrosis, balance dysfunction, and risk of falling (Table 2).

Although objective criteria for external lymphedema have not been established, simple measurements such as using a tape measure to record neck circumference, allow a useful longitudinal assessment. Digital photography may be effective in the documentation and subjective evaluation of changes of external lymphedema.^10,17 However, there are some limitations with photography because although external photographs (including digital photography and three-dimensional imaging) can capture some features, such as changes in contours, symmetry, and changes in skin quality and color, they do not detect changes in skin and soft tissue texture and compliance (Table 3).¹⁰

Impact on clinical outcomes

We retrospectively reviewed the medical records of 230 head and neck cancer patients who had been treated at our center between June 2008 and June 2015. Complete clinical data were available for 190 patients. The following information was extracted from each patient’s chart: whether they developed lymphedema, tumor stage, had surgery, radiation dose, type of chemotherapy given, their smoking history, if they had had a neck dissection and the primary site of the tumor (Table 3).

Incidence in different time periods. Of the 190 patients with complete records 78 (41%) were found to have lymphedema. These were all patients undergoing treatment for head and neck cancer during June 2008-June 2015. The prehabilitation program was initiated with the hiring of a nurse navigator for head and neck cancer, starting in January 2012. It is interesting to note that the incidence of lymphedema was 27% before the program was started, but after nurse navigator joined the team, the incidence increased significantly to 48% (P = .0002), in line with published expectations. This increase in recorded incidence may be attributable to the greater awareness of lymphedema intentionally fostered by the prehabilitation program.

Smoking history. Patients’ lifetime smoking history was retrieved from their medical records, based on their verbal admission of tobacco use. Most of the patients (n = 110) self-reported a history of smoking. Of those with a history of smoking, 36 (33%) developed external lymphedema after treatment for head and neck cancer, and 74 (67%) did not. However, this difference was not statistically significant. Hence, although smoking is a risk factor for head and neck cancer, it was not associated with the development of external lymphedema in our cohort of patients.

Type of tumor

Most of the patients (n = 156, 82%) had squamous cell carcinomas (SCC). Of those, 45% developed external lymphedema and 55% did not. Therefore, having SCC did not predispose to lymphedema. The other cancers were mixed type, mainly adenocaricoma, but their numbers were too small to draw statistical conclusions.

Stage of the tumor

About two thirds of the patients (n = 121, 64%) had stage 3 or 4 cancer. However, treatment of more advanced cancers was not associated with lymphedema development.

Site of the tumor

The literature suggests that patients with a primary tumor in the throat are at increased risk for lymphedema.⁵ The American Cancer Society has defined cancers of the oropharynx (throat) as including the base of the tongue (back third of the tongue), the soft palate, the tonsils, and the side and back walls of the throat.¹⁸ In our head and neck cancer cohort, patients with primary tumors of the oropharnyx were, perhaps, more susceptible to lymphedema (P = .044, Table 3). By contrast, in our cohort of patients, those with nasopharyngeal, hypopharyngeal, and parotid gland tumors were significantly less likely to develop lymphedema (Ps = .017, .04, .012, respectively).

No surgery

Half of our patients (n = 95) were not treated with surgery. In the patients who did not have surgery, 25 (26%) developed lymphedema, whereas 70 (74%) did not. Hence, although the incidence of lymphedema was significantly lower in patients who did not have surgery (P = .015), lymphedema did develop in patients who did not have a surgical procedure.

Resection of primary tumor without neck dissection

Of the 64 patients who had surgery, but without neck dissection, 35 (55%) developed external lymphedema. Compared with the no-surgery patients, the doubling of the incidence (from 26% to 55%) was highly significant (P = .0004). These findings are compatible with the literature reports that surgery increases the incidence of lymphedema, which is not surprising because surgery and subsequent scarring is known to compromise the lymphatic system.

Resection of primary tumor with neck dissection

The incidence of external lymphedema was increased to 69% when patients were subjected to both surgery and neck dissection. Compared with the June 2008-June 2015 cohort, there was a significant increase in the incidence of lymphedema in the neck dissection group (P = .007). Neck dissection involves the removal of lymph nodes and disruption of the lymphatic vessels, so it is not surprising that there is a higher incidence of external lymphedema. In our practice, neck dissections increased in frequency every year from June 2008 until December 2011, when 8 patients underwent neck dissections, 6 (75%) of whom developed lymphedema. Since January 2012, when the prehabilitation program was implemented, the number of neck dissections have declined, with more patients receiving chemoradiation and surgery being reserved for surgery. Hamoir and colleagues have reported that neck dissection is no longer justified unless there is clinically residual disease in the neck.¹⁹

Radiation

Lymphedema occurred in patients regardless of the dose of radiation received. Although the incidence of lymphedema seemed to be higher in patients who received more than 60 cGy, that difference was not statistically significant (Table 3). We had expected a relationship between radiation damage and greater lymphedema, but that was not evident in our patients.

Chemotherapy

The majority of patients (n = 131, 69%) received chemotherapy. The exposure to chemotherapy was not correlated with the risk of external lymphedema in our cohort of patients, with 58 of the 131 treated patients (44%) developing lymphedema, compared with 73 (56%) of treated patients who did not (Table 3).

Complete decongestive therapy

All patients with documented lymphedema were evaluated for complete decongestive therapy (CDT). Contraindications to CDT included congestive heart failure, renal failure, acute infection, peripheral artery disease, upper-quadrant deep vein thrombosis, and carotid artery stenosis. Eligible patients were referred to a certified lymphedema therapist for CDT. As the program evolved, patients at risk for lymphedema were referred for CDT early on, usually at the time of diagnosis, to improve early identification and surveillance of lymphedema.

CDT included manual lymph drainage, compression bandaging (Figure), decongestive exercises, skin care, and education in swelling self-management. Manual lymph drainage is a specialized light pressure hands-on technique that reduces swelling by enhancing lymphatic reabsorption and flow. Compressive bandaging/garments increase venous and lymphatic drainage and soften fibrotic tissue. Continued use of compression depends on progress. In head and neck cancer patients, the need for lifelong compression is not evident when they are treated early and there is good patient compliance.⁸ Therapeutic exercise enhances lymphatic and venous circulation, and good skin care reduces the risk of infection.

Communication and education

The head and neck cancer nurse navigator attends the cancer center’s multidisciplinary head and neck tumor board, which has representation from otolaryngology, diagnostic radiology, pathology, radiation oncology, medical oncology, reconstructive surgery, oncology rehabilitation (physical/occupational therapist), dietary services, speech pathology, social services and clinical research. This regular contact allows for earlier awareness about which patients are at greater risk for developing lymphedema, thus enabling early intervention (and patient education) in a timely manner.

Education of the patient, before cancer therapy, of the risks of lymphedema is very important. Before the implementation of the prehabilitation program, some patients did not fully comprehend what a painful and debilitating consequence of cancer treatment lymphedema could be.

Discussion

We introduced a prehabilitation program to detect and treat lymphedema in head and neck cancer patients in January 2012 part way through following an observation cohort from June2008 through June2015. Central to this, in our center, was the appointment of a nurse navigator whose primary focus was on head and neck cancer patients. We placed a high priority on the early detection and treatment of lymphedema because do so has been associated with better outcomes in other centers.

One immediate consequence of the inception of our program was the identification of more patients with external lymphedema. Our detected incidence rose significantly (P = .0002), from 27% in the period June 2008-December 20112010, before the program, to 48% during the January 2012-June 2015 period, after the inception of the program. This later incidence rate is in line with published incidence rates in most centers. However, it is still somewhat short of the 75% suggested in one center,⁹ which suggests we are either we are underdetecting lymphedema or there are differences in definition criteria or sensitivity levels for defining lymphedema.

There are currently no specific objective measures of lymphedema, so there is bound to be some variation in diagnosis rates. In our program, we rely heavily on the patient-reported outcome measures, the NDI instrument, and digital photography to detect and monitor lymphedema, starting with the pretreatment baseline values that are established for each patient.

The use of digital photography in our community hospital setting, which includes taking photographs before and after treatment and at each visit, motivates and encourages patients and provides a tool for clinical lymphedema therapists to visually document benefits of treatment. Patients’ motivation and compliance with their established home program for head and neck lymphedema self-management are essential. The elements of the home program may include self-manual lymph drainage, home-modified compression bandaging and garment wear, therapeutic exercises, and skin care. Patients with lymphedema who adhered closely with their therapy program were more than 8 times more likely to improve compared with noncompliant patients.¹⁷

Some groups of patients have a greater risk of developing lymphedema than others,⁵ so the development of an algorithm to predict lymphedema seemed possible. However, in our cohort of patients, only neck dissection, with its disruption of the lymphatic system of the neck, was strongly associated with external lymphedema (Table 3). It is important to note that some patients who did not undergo surgery developed lymphedema. In our patients, high doses of radiation alone did not seem to predispose to lymphedema. That suggests that no group of head and neck cancer patients should be ignored, which is why we did routine screening of all patients before, during, and after treatment.

Our protocol falls short in the detection of internal lymphedema. For example, information on swallowing gathered by our speech pathologists (in a different department) has not, so far, been included in our assessment. This is one opportunity to improve on our approach, especially because speech difficulties may be associated with internal lymphedema. In addition, we are not equipped for the requisite internal examinations. Unfortunately, there are no practical and successful treatments for patients suffering from internal swelling. This represents a challenge for the medical community to better meet this need. Therefore, although we are missing some assessments of internal lymphedema, this is of little therapeutic consequence at this time.

The increase in the detected incidence of external lymphedema points to a practice gap that has been resolved by the appointment of a dedicated nurse navigator who attends oncology reviews to share knowledge and information. Another educational effort has been made with the patients themselves to increase compliance and improve continuous care at home.

There is always room for improvement, however, either by feedback acquired from other institutions and hospitals or through the future introduction of more objective assessment techniques.

Conclusions

The introduction of the prehabilitation program at our center has coincided with a significantly improved detection rate for external lymphedema in head and neck cancer patients. It may be because the program emphasizes education about lymphedema that awareness of the condition has increased throughout the center. It is now widely recognized that all patients are at risk of lymphedema regardless of whether they fall into an acknowledged high-risk group. Our experience shows that there is no significant difference between treatment modalities apart from neck dissection. In our population, the use of this procedure is decreasing. External lymphedema can develop even in patients who do not have surgery. Therefore, there is no sound way to predict which patients are most likely to suffer from the accumulation of fluid in their head and neck after treatment for head and neck cancer. Thus, an assessment as described here, during and after treatment for all patients, is warranted. Patients are now being seen earlier as a part of the prehabilitation program, which facilitates access to complete decongestive treatment at an earlier stage, improves patient outcomes, and increases patient satisfaction with their treatment. Our prehabilitation program could serve as a model for other community hospital centers in achieving outcomes that are as good as those in academic centers.

Acknowledgments

The authors thank Irene Kadota and Heather Peters, from the Department of Radiation Oncology, and Julianne Courtenay, from the Department of Physical Therapy at the Disney Family Cancer Center, Burbank, California, for providing the original clinical data for analysis.