Original Research

Psoriasis is a chronic, inflammatory, T-cell–mediated skin disease. Phototherapy, which consists of light used at various wavelengths, is a well-established treatment method for psoriasis vulgaris. Although successful results have been obtained with phototherapy in psoriasis, its mechanism of action is not fully understood. UV light has been shown to have an effect on T-lymphocyte function as well as various components of the natural and acquired immune response. It also has a suppressive effect on the immune system caused by many independent effects.¹ Phototherapy currently is available using broadband UVB (290–320 nm), narrowband UVB (NB-UVB)(311–313 nm), 308-nm excimer laser, UVA1 (340–400 nm), psoralen plus UVA, and photopheresis.² Narrowband UVB treatment with light sources that peak at 311 to 313 nm have been used with high efficacy and a low side-effect profile, becoming the standard phototherapy method for chronic plaque-type psoriasis.³

More than 90% of vitamin D synthesis is formed in the skin following UV exposure, and the wavelengths and the solar spectrum that stimulate vitamin D synthesis have been a focus of research.⁴ 7-Dehydrocholesterol (provitamin D₃) is first converted to previtamin D₃. Although the necessary UV wavelength for previtamin D₃ synthesis is 295 to 300 nm, it is known that production stops below 260 nm and above 315 nm.^4-6 Previtamin D₃ is unstable and is quickly converted to vitamin D₃ in the skinand then to the biologically active form of 1,25-dihydroxyvitamin D₃ (calcitriol) following hydroxylation in the liver and kidneys. Calcitriol shows its effect by binding to the special nuclear receptor for vitamin D.⁷ Many tissues including the keratinocytes, dendritic cells, melanocytes, and sebocytes in the skin have been shown to possess the enzymatic mechanism necessary for 1,25-dihydroxyvitamin D₃ production. Vitamin D also is known to have paracrine, autocrine, and intracrine effects on immunomodulation, cell proliferation, differentiation, and apoptosis, in addition to its role in calcium metabolism.^5-9 Topical vitamin D and its analogues are used effectively and safely in psoriasis treatment with these effects.¹⁰ A correlation between low serum vitamin D levels and chronic inflammation severity has been shown in psoriasis patients in some studies.^11,12

In this study, we sought to evaluate the effect of NB-UVB on vitamin D status and related metabolic markers in patients with psoriasis.

Methods

This prospective, single-center study included patients living in or around Eskisehir, Turkey, who were 18 years of age or older and had been diagnosed with chronic plaque psoriasis with a psoriasis area and severity index (PASI) score of 5 or higher. Permission was granted by the local ethics committee. Patients provided written informed consent prior to enrollment. Patients were excluded if they were younger than 18 years; were pregnant or breastfeeding; stayed in open environments for more than 2 hours per day during the summer months (May through September); used drugs affecting calcium metabolism in the last 8 weeks (eg, barbiturates, anticonvulsants, corticosteroids, vitamin D supplements, bisphosphonates); used systemic treatment for psoriasis in the last 8 weeks; used phototherapy or sunbathing in the last 8 weeks; used topical vitamin D analogues in the last 4 weeks; or had a history of psoriatic arthritis and other inflammatory disorders, renal disease, known calcium metabolism disorders, granulomatous disorders, thyroid disease, diabetes mellitus, skin cancer, or abnormal photosensitivity and known lack of response or hypersensitivity to phototherapy.

Clinical Evaluation and Laboratory Studies
The participants’ age, gender, Fitzpatrick skin type, disease duration, dairy intake and vitamin supplement levels, hours of sun exposure per week, detailed medical history, and medications were obtained and documented in the medical records.

Serum 25(OH)D levels were measured using high-performance liquid chromatography/mass spectrometry, serum calcium and phosphorus levels using colorimetric analysis, serum alkaline phosphatase (ALP) levels using the enzymatic colorimetric method, and serum parathyroid hormone (PTH) levels using electrochemiluminescence at baseline and after PASI 75 was achieved with treatment. Vitamin D levels were classified in 3 groups: (1) deficient (<20 ng/mL); (2) inadequate (20–30 ng/mL); and (3) adequate (>30 ng/mL). The PASI scores at baseline and posttreatment were calculated by the same dermatologist (S.S.).

Treatment Protocol and Patient Follow-up
Narrowband UVB treatment was started at 70% of the minimal erythema dose (MED). Phototherapy was administered 3 times weekly for 6 months or until PASI 75 response was achieved. An increase of 20% to 30% from the prior dose was made according to the participants’ clinical status at each treatment session, and the dose was stabilized once the maximum dose was achieved according to skin type—up to 2000 mJ/cm² for Fitzpatrick skin types I and II, 3000 mJ/cm² for skin types III and IV, and 5000 mJ/cm² for skin types V and VI. Participants were allowed to use low- and moderate-potency topical corticosteroids and moisturizers containing urea during the course of treatment. The study physician (S.S.) clinically evaluated participants every 4 weeks for 6 months or until PASI 75 was achieved, and the clinical improvement was calculated as the percentage decrease in PASI score.

Statistical Analysis
The Shapiro-Wilk normalcy test was used for the continuous variables in the study. Variables with a normal distribution were analyzed with the paired t test and 1-way analysis of variance test and presented as mean (SD). Variables without a normal distribution were analyzed with the Wilcoxon t test and the Kruskal-Wallis test and presented as the median and 25th and 75th quartiles. The serum 25(OH)D levels were evaluated according to the seasons with the Kruskal-Wallis test. Categorical variables were expressed as frequency and percentages. The Pearson and Spearman correlation analysis and regression analysis were used to show the relationship between the variables (ie, age, Fitzpatrick skin type, PASI score, maximum NB-UVB dose, and number of sessions). The statistical significance level was set at P≤.05. Statistical analyses were performed using SPSS software version 21.

 

Results

A total of 49 participants (30 [61.22%] males; 19 [38.78%] females) were included in the study. The mean age (SD) was 40.27 (14.62) years (range, 19–74 years). Three (6.12%) participants were Fitzpatrick skin type I, 15 (30.61%) were skin type II, and 31 (63.27%) were skin type III.

The baseline median PASI score for the 49 participants was 10.20 (7.85–13.65). Baseline serum 25(OH)D levels were noted to be deficient in 40 participants (81.63%) and inadequate in 9 participants (18.37%). The distribution of the serum 25(OH)D levels of the participants according to the season was evaluated with the Kruskal-Wallis test and no association was found between serum 25(OH)D levels and seasonal changes (P=.685). Comparison of 25(OH)D basal values with Fitzpatrick skin type revealed a statistically significant relationship between skin type and vitamin D level (P=.024). The basal serum 25(OH)D levels were significantly lower in Fitzpatrick skin type II versus skin type I (P=.039).

Thirty-two (65.31%) participants achieved PASI 75 by the end of treatment. The baseline median PASI score (25th-75th quartiles) for the 32 patients was 10.45 (8.20-13.83) and the posttreatment PASI score was 1.95 (1.20-3.55), a statistically significant decrease following treatment (P<.001)(Table 1). Mean (SD) baseline serum 25(OH)D levels were 14.14 (6.70) ng/mL and posttreatment levels were 46.42 (15.51) ng/mL in these participants, which demonstrated a statistically significant increase during NB-UVB treatment (P<.001). None of the participants reached the toxicity levels (>80 ng/mL) for serum 25(OH)D. There were no significant changes in serum calcium or phosphorus levels posttreatment (Table 1), but statistically significant decreases in serum ALP and PTH levels were noted (P=.001 and P=.019, respectively)(Table 1).

Participants who completed the study (n=32) received an average (SD) of 30.09 (7.53) sessions of NB-UVB treatment and the mean (SD) MED was 611.88 (240.14) mJ/cm². The mean (SD) maximum dose was 2090.09 (341.78) mJ/cm² (Table 2).

Posttreatment serum 25(OH)D levels were compared with the number of NB-UVB phototherapy sessions and the maximum dose values. We found that the posttreatment serum 25(OH)D levels correlated with the number of sessions (P=.031) but not with the maximum dose (P=.498).

Using regression analysis, we also evaluated the effect of the increase in vitamin D levels—posttreatment serum 25(OH)D level minus baseline serum 25(OH)D levels—on the decrease in PASI scores—baseline PASI score minus posttreatment PASI score—and found no effect of serum 25(OH)D level increase on PASI decrease (P=.530). There was no correlation between increased serum 25(OH)D levels and age, Fitzpatrick skin type, or baseline PASI score.

 

Comment

The most effective UV wavelength for vitamin D synthesis is 295 to 300 nm, and therefore broadband UVB is frequently studied when determining the relationship between phototherapy and serum vitamin D levels.⁴ The current study demonstrated a statistically significant increase in serum 25(OH)D levels following NB-UVB treatment in patients with moderate to severe chronic plaque psoriasis (P<.001). This result supports other studies reporting that NB-UVB treatment in psoriasis patients increases serum 25(OH)D levels.^13-18

The main factor in the effective UVB level for vitamin D synthesis is the angle at which solar radiation reaches the earth, which is affected by the longitude, latitude, and time of day.¹⁹ For this reason, we planned to perform our study at a single center. Patients who stayed in open areas for more than 2 hours per day during the summer months (May through September) were excluded from the study to decrease the effect of seasonal changes on vitamin D levels. We evaluated the seasonal variation of vitamin D levels and found no relationship between seasonal changes and serum 25(OH)D levels. Therefore, the potential effect of seasonal changes on the vitamin D levels of study participants was excluded from the study.

The response to UV radiation changes according to age and Fitzpatrick skin type because 7-dehydrocholesterol levels decrease with age and melanin prevents the access of UVB photons to 7-dehydrocholesterol.²⁰ The basal serum 25(OH)D levels were deficient in 81.63% of participants and inadequate in 18.37%. In this study, we also observed that the basal serum 25(OH)D levels were significantly lower in patients with Fitzpatrick skin type II than in Fitzpatrick skin type I (P=.039). The mean (SD) serum 25(OH)D level at baseline was 14.14 (6.70) ng/mL and posttreatment was 46.42 (15.51) ng/mL in the 32 patients who completed the study. Serum 25(OH)D levels showed a statistically significant increase after NB-UVB treatment (P<.001). The increased serum 25(OH)D levels after NB-UVB phototherapy were not associated with Fitzpatrick skin type, which was consistent with the results of Osmancevic et al.¹⁷ The adjusted NB-UVB doses according to the different skin types might be responsible for this result in our study.

Participant age did not have a significant effect on serum 25(OH)D levels, similar to other studies in the literature.^13,17 We believe that artificial UVB radiation at high doses can compensate for the 7-dehydrocholesterol that decreases in the skin with aging.

We observed no significant change in the serum calcium and phosphorus levels with NB-UVB treatment in our study. None of the participants had a metabolic disorder related to increased 25(OH)D levels. The serum ALP and PTH levels decreased significantly following treatment (P=.001 and P=.019, respectively), which may have been secondary to increased serum 25(OH)D levels.

Posttreatment serum 25(OH)D levels were compared with the number of NB-UVB phototherapy sessions and maximum dose values. The posttreatment serum 25(OH)D levels were found to be related to the number of sessions received, but this value was not correlated with the maximum dose received. The MED and maximum dose were determined according to the Fitzpatrick skin type of the participants. Therefore, increased serum 25(OH)D levels with an increased number of sessions was an expected result. Our observation is in accordance with the finding described by Ryan et al.¹⁴ On the other hand, an in vitro study conducted by Olds et al²¹ reported that the relationship between UV light and cholecalciferol synthesis was not linear.

We found that increased serum 25(OH)D levels after treatment were not correlated with the decrease in PASI score, similar to studies by Romaní et al¹⁸ and Ryan et al.¹⁴ These results suggest that the clinical improvement following NB-UVB treatment is independent of the increased serum 25(OH)D levels in psoriasis patients.

Conclusion

In conclusion, we found that the serum 25(OH)D levels that increase as a result of NB-UVB therapy for the treatment of chronic plaque psoriasis has no statistically significant relationship with the age, Fitzpatrick skin type, baseline PASI score, changes in PASI, or maximum dose, while a positive relationship is present between the serum 25(OH)D levels and the number of sessions of NB-UVB.

Psoriasis is a chronic, inflammatory, T-cell–mediated skin disease. Phototherapy, which consists of light used at various wavelengths, is a well-established treatment method for psoriasis vulgaris. Although successful results have been obtained with phototherapy in psoriasis, its mechanism of action is not fully understood. UV light has been shown to have an effect on T-lymphocyte function as well as various components of the natural and acquired immune response. It also has a suppressive effect on the immune system caused by many independent effects.¹ Phototherapy currently is available using broadband UVB (290–320 nm), narrowband UVB (NB-UVB)(311–313 nm), 308-nm excimer laser, UVA1 (340–400 nm), psoralen plus UVA, and photopheresis.² Narrowband UVB treatment with light sources that peak at 311 to 313 nm have been used with high efficacy and a low side-effect profile, becoming the standard phototherapy method for chronic plaque-type psoriasis.³

More than 90% of vitamin D synthesis is formed in the skin following UV exposure, and the wavelengths and the solar spectrum that stimulate vitamin D synthesis have been a focus of research.⁴ 7-Dehydrocholesterol (provitamin D₃) is first converted to previtamin D₃. Although the necessary UV wavelength for previtamin D₃ synthesis is 295 to 300 nm, it is known that production stops below 260 nm and above 315 nm.^4-6 Previtamin D₃ is unstable and is quickly converted to vitamin D₃ in the skinand then to the biologically active form of 1,25-dihydroxyvitamin D₃ (calcitriol) following hydroxylation in the liver and kidneys. Calcitriol shows its effect by binding to the special nuclear receptor for vitamin D.⁷ Many tissues including the keratinocytes, dendritic cells, melanocytes, and sebocytes in the skin have been shown to possess the enzymatic mechanism necessary for 1,25-dihydroxyvitamin D₃ production. Vitamin D also is known to have paracrine, autocrine, and intracrine effects on immunomodulation, cell proliferation, differentiation, and apoptosis, in addition to its role in calcium metabolism.^5-9 Topical vitamin D and its analogues are used effectively and safely in psoriasis treatment with these effects.¹⁰ A correlation between low serum vitamin D levels and chronic inflammation severity has been shown in psoriasis patients in some studies.^11,12

In this study, we sought to evaluate the effect of NB-UVB on vitamin D status and related metabolic markers in patients with psoriasis.

Methods

This prospective, single-center study included patients living in or around Eskisehir, Turkey, who were 18 years of age or older and had been diagnosed with chronic plaque psoriasis with a psoriasis area and severity index (PASI) score of 5 or higher. Permission was granted by the local ethics committee. Patients provided written informed consent prior to enrollment. Patients were excluded if they were younger than 18 years; were pregnant or breastfeeding; stayed in open environments for more than 2 hours per day during the summer months (May through September); used drugs affecting calcium metabolism in the last 8 weeks (eg, barbiturates, anticonvulsants, corticosteroids, vitamin D supplements, bisphosphonates); used systemic treatment for psoriasis in the last 8 weeks; used phototherapy or sunbathing in the last 8 weeks; used topical vitamin D analogues in the last 4 weeks; or had a history of psoriatic arthritis and other inflammatory disorders, renal disease, known calcium metabolism disorders, granulomatous disorders, thyroid disease, diabetes mellitus, skin cancer, or abnormal photosensitivity and known lack of response or hypersensitivity to phototherapy.

Clinical Evaluation and Laboratory Studies
The participants’ age, gender, Fitzpatrick skin type, disease duration, dairy intake and vitamin supplement levels, hours of sun exposure per week, detailed medical history, and medications were obtained and documented in the medical records.

Serum 25(OH)D levels were measured using high-performance liquid chromatography/mass spectrometry, serum calcium and phosphorus levels using colorimetric analysis, serum alkaline phosphatase (ALP) levels using the enzymatic colorimetric method, and serum parathyroid hormone (PTH) levels using electrochemiluminescence at baseline and after PASI 75 was achieved with treatment. Vitamin D levels were classified in 3 groups: (1) deficient (<20 ng/mL); (2) inadequate (20–30 ng/mL); and (3) adequate (>30 ng/mL). The PASI scores at baseline and posttreatment were calculated by the same dermatologist (S.S.).

Treatment Protocol and Patient Follow-up
Narrowband UVB treatment was started at 70% of the minimal erythema dose (MED). Phototherapy was administered 3 times weekly for 6 months or until PASI 75 response was achieved. An increase of 20% to 30% from the prior dose was made according to the participants’ clinical status at each treatment session, and the dose was stabilized once the maximum dose was achieved according to skin type—up to 2000 mJ/cm² for Fitzpatrick skin types I and II, 3000 mJ/cm² for skin types III and IV, and 5000 mJ/cm² for skin types V and VI. Participants were allowed to use low- and moderate-potency topical corticosteroids and moisturizers containing urea during the course of treatment. The study physician (S.S.) clinically evaluated participants every 4 weeks for 6 months or until PASI 75 was achieved, and the clinical improvement was calculated as the percentage decrease in PASI score.

Statistical Analysis
The Shapiro-Wilk normalcy test was used for the continuous variables in the study. Variables with a normal distribution were analyzed with the paired t test and 1-way analysis of variance test and presented as mean (SD). Variables without a normal distribution were analyzed with the Wilcoxon t test and the Kruskal-Wallis test and presented as the median and 25th and 75th quartiles. The serum 25(OH)D levels were evaluated according to the seasons with the Kruskal-Wallis test. Categorical variables were expressed as frequency and percentages. The Pearson and Spearman correlation analysis and regression analysis were used to show the relationship between the variables (ie, age, Fitzpatrick skin type, PASI score, maximum NB-UVB dose, and number of sessions). The statistical significance level was set at P≤.05. Statistical analyses were performed using SPSS software version 21.

 

Results

A total of 49 participants (30 [61.22%] males; 19 [38.78%] females) were included in the study. The mean age (SD) was 40.27 (14.62) years (range, 19–74 years). Three (6.12%) participants were Fitzpatrick skin type I, 15 (30.61%) were skin type II, and 31 (63.27%) were skin type III.

The baseline median PASI score for the 49 participants was 10.20 (7.85–13.65). Baseline serum 25(OH)D levels were noted to be deficient in 40 participants (81.63%) and inadequate in 9 participants (18.37%). The distribution of the serum 25(OH)D levels of the participants according to the season was evaluated with the Kruskal-Wallis test and no association was found between serum 25(OH)D levels and seasonal changes (P=.685). Comparison of 25(OH)D basal values with Fitzpatrick skin type revealed a statistically significant relationship between skin type and vitamin D level (P=.024). The basal serum 25(OH)D levels were significantly lower in Fitzpatrick skin type II versus skin type I (P=.039).

Thirty-two (65.31%) participants achieved PASI 75 by the end of treatment. The baseline median PASI score (25th-75th quartiles) for the 32 patients was 10.45 (8.20-13.83) and the posttreatment PASI score was 1.95 (1.20-3.55), a statistically significant decrease following treatment (P<.001)(Table 1). Mean (SD) baseline serum 25(OH)D levels were 14.14 (6.70) ng/mL and posttreatment levels were 46.42 (15.51) ng/mL in these participants, which demonstrated a statistically significant increase during NB-UVB treatment (P<.001). None of the participants reached the toxicity levels (>80 ng/mL) for serum 25(OH)D. There were no significant changes in serum calcium or phosphorus levels posttreatment (Table 1), but statistically significant decreases in serum ALP and PTH levels were noted (P=.001 and P=.019, respectively)(Table 1).

Participants who completed the study (n=32) received an average (SD) of 30.09 (7.53) sessions of NB-UVB treatment and the mean (SD) MED was 611.88 (240.14) mJ/cm². The mean (SD) maximum dose was 2090.09 (341.78) mJ/cm² (Table 2).

Posttreatment serum 25(OH)D levels were compared with the number of NB-UVB phototherapy sessions and the maximum dose values. We found that the posttreatment serum 25(OH)D levels correlated with the number of sessions (P=.031) but not with the maximum dose (P=.498).

Using regression analysis, we also evaluated the effect of the increase in vitamin D levels—posttreatment serum 25(OH)D level minus baseline serum 25(OH)D levels—on the decrease in PASI scores—baseline PASI score minus posttreatment PASI score—and found no effect of serum 25(OH)D level increase on PASI decrease (P=.530). There was no correlation between increased serum 25(OH)D levels and age, Fitzpatrick skin type, or baseline PASI score.

 

Comment

The most effective UV wavelength for vitamin D synthesis is 295 to 300 nm, and therefore broadband UVB is frequently studied when determining the relationship between phototherapy and serum vitamin D levels.⁴ The current study demonstrated a statistically significant increase in serum 25(OH)D levels following NB-UVB treatment in patients with moderate to severe chronic plaque psoriasis (P<.001). This result supports other studies reporting that NB-UVB treatment in psoriasis patients increases serum 25(OH)D levels.^13-18

The main factor in the effective UVB level for vitamin D synthesis is the angle at which solar radiation reaches the earth, which is affected by the longitude, latitude, and time of day.¹⁹ For this reason, we planned to perform our study at a single center. Patients who stayed in open areas for more than 2 hours per day during the summer months (May through September) were excluded from the study to decrease the effect of seasonal changes on vitamin D levels. We evaluated the seasonal variation of vitamin D levels and found no relationship between seasonal changes and serum 25(OH)D levels. Therefore, the potential effect of seasonal changes on the vitamin D levels of study participants was excluded from the study.

The response to UV radiation changes according to age and Fitzpatrick skin type because 7-dehydrocholesterol levels decrease with age and melanin prevents the access of UVB photons to 7-dehydrocholesterol.²⁰ The basal serum 25(OH)D levels were deficient in 81.63% of participants and inadequate in 18.37%. In this study, we also observed that the basal serum 25(OH)D levels were significantly lower in patients with Fitzpatrick skin type II than in Fitzpatrick skin type I (P=.039). The mean (SD) serum 25(OH)D level at baseline was 14.14 (6.70) ng/mL and posttreatment was 46.42 (15.51) ng/mL in the 32 patients who completed the study. Serum 25(OH)D levels showed a statistically significant increase after NB-UVB treatment (P<.001). The increased serum 25(OH)D levels after NB-UVB phototherapy were not associated with Fitzpatrick skin type, which was consistent with the results of Osmancevic et al.¹⁷ The adjusted NB-UVB doses according to the different skin types might be responsible for this result in our study.

Participant age did not have a significant effect on serum 25(OH)D levels, similar to other studies in the literature.^13,17 We believe that artificial UVB radiation at high doses can compensate for the 7-dehydrocholesterol that decreases in the skin with aging.

We observed no significant change in the serum calcium and phosphorus levels with NB-UVB treatment in our study. None of the participants had a metabolic disorder related to increased 25(OH)D levels. The serum ALP and PTH levels decreased significantly following treatment (P=.001 and P=.019, respectively), which may have been secondary to increased serum 25(OH)D levels.

Posttreatment serum 25(OH)D levels were compared with the number of NB-UVB phototherapy sessions and maximum dose values. The posttreatment serum 25(OH)D levels were found to be related to the number of sessions received, but this value was not correlated with the maximum dose received. The MED and maximum dose were determined according to the Fitzpatrick skin type of the participants. Therefore, increased serum 25(OH)D levels with an increased number of sessions was an expected result. Our observation is in accordance with the finding described by Ryan et al.¹⁴ On the other hand, an in vitro study conducted by Olds et al²¹ reported that the relationship between UV light and cholecalciferol synthesis was not linear.

We found that increased serum 25(OH)D levels after treatment were not correlated with the decrease in PASI score, similar to studies by Romaní et al¹⁸ and Ryan et al.¹⁴ These results suggest that the clinical improvement following NB-UVB treatment is independent of the increased serum 25(OH)D levels in psoriasis patients.

Conclusion

In conclusion, we found that the serum 25(OH)D levels that increase as a result of NB-UVB therapy for the treatment of chronic plaque psoriasis has no statistically significant relationship with the age, Fitzpatrick skin type, baseline PASI score, changes in PASI, or maximum dose, while a positive relationship is present between the serum 25(OH)D levels and the number of sessions of NB-UVB.

Psoriasis is a chronic, inflammatory, T-cell–mediated skin disease. Phototherapy, which consists of light used at various wavelengths, is a well-established treatment method for psoriasis vulgaris. Although successful results have been obtained with phototherapy in psoriasis, its mechanism of action is not fully understood. UV light has been shown to have an effect on T-lymphocyte function as well as various components of the natural and acquired immune response. It also has a suppressive effect on the immune system caused by many independent effects.¹ Phototherapy currently is available using broadband UVB (290–320 nm), narrowband UVB (NB-UVB)(311–313 nm), 308-nm excimer laser, UVA1 (340–400 nm), psoralen plus UVA, and photopheresis.² Narrowband UVB treatment with light sources that peak at 311 to 313 nm have been used with high efficacy and a low side-effect profile, becoming the standard phototherapy method for chronic plaque-type psoriasis.³

More than 90% of vitamin D synthesis is formed in the skin following UV exposure, and the wavelengths and the solar spectrum that stimulate vitamin D synthesis have been a focus of research.⁴ 7-Dehydrocholesterol (provitamin D₃) is first converted to previtamin D₃. Although the necessary UV wavelength for previtamin D₃ synthesis is 295 to 300 nm, it is known that production stops below 260 nm and above 315 nm.^4-6 Previtamin D₃ is unstable and is quickly converted to vitamin D₃ in the skinand then to the biologically active form of 1,25-dihydroxyvitamin D₃ (calcitriol) following hydroxylation in the liver and kidneys. Calcitriol shows its effect by binding to the special nuclear receptor for vitamin D.⁷ Many tissues including the keratinocytes, dendritic cells, melanocytes, and sebocytes in the skin have been shown to possess the enzymatic mechanism necessary for 1,25-dihydroxyvitamin D₃ production. Vitamin D also is known to have paracrine, autocrine, and intracrine effects on immunomodulation, cell proliferation, differentiation, and apoptosis, in addition to its role in calcium metabolism.^5-9 Topical vitamin D and its analogues are used effectively and safely in psoriasis treatment with these effects.¹⁰ A correlation between low serum vitamin D levels and chronic inflammation severity has been shown in psoriasis patients in some studies.^11,12

In this study, we sought to evaluate the effect of NB-UVB on vitamin D status and related metabolic markers in patients with psoriasis.

Methods

This prospective, single-center study included patients living in or around Eskisehir, Turkey, who were 18 years of age or older and had been diagnosed with chronic plaque psoriasis with a psoriasis area and severity index (PASI) score of 5 or higher. Permission was granted by the local ethics committee. Patients provided written informed consent prior to enrollment. Patients were excluded if they were younger than 18 years; were pregnant or breastfeeding; stayed in open environments for more than 2 hours per day during the summer months (May through September); used drugs affecting calcium metabolism in the last 8 weeks (eg, barbiturates, anticonvulsants, corticosteroids, vitamin D supplements, bisphosphonates); used systemic treatment for psoriasis in the last 8 weeks; used phototherapy or sunbathing in the last 8 weeks; used topical vitamin D analogues in the last 4 weeks; or had a history of psoriatic arthritis and other inflammatory disorders, renal disease, known calcium metabolism disorders, granulomatous disorders, thyroid disease, diabetes mellitus, skin cancer, or abnormal photosensitivity and known lack of response or hypersensitivity to phototherapy.

Clinical Evaluation and Laboratory Studies
The participants’ age, gender, Fitzpatrick skin type, disease duration, dairy intake and vitamin supplement levels, hours of sun exposure per week, detailed medical history, and medications were obtained and documented in the medical records.

Serum 25(OH)D levels were measured using high-performance liquid chromatography/mass spectrometry, serum calcium and phosphorus levels using colorimetric analysis, serum alkaline phosphatase (ALP) levels using the enzymatic colorimetric method, and serum parathyroid hormone (PTH) levels using electrochemiluminescence at baseline and after PASI 75 was achieved with treatment. Vitamin D levels were classified in 3 groups: (1) deficient (<20 ng/mL); (2) inadequate (20–30 ng/mL); and (3) adequate (>30 ng/mL). The PASI scores at baseline and posttreatment were calculated by the same dermatologist (S.S.).

Treatment Protocol and Patient Follow-up
Narrowband UVB treatment was started at 70% of the minimal erythema dose (MED). Phototherapy was administered 3 times weekly for 6 months or until PASI 75 response was achieved. An increase of 20% to 30% from the prior dose was made according to the participants’ clinical status at each treatment session, and the dose was stabilized once the maximum dose was achieved according to skin type—up to 2000 mJ/cm² for Fitzpatrick skin types I and II, 3000 mJ/cm² for skin types III and IV, and 5000 mJ/cm² for skin types V and VI. Participants were allowed to use low- and moderate-potency topical corticosteroids and moisturizers containing urea during the course of treatment. The study physician (S.S.) clinically evaluated participants every 4 weeks for 6 months or until PASI 75 was achieved, and the clinical improvement was calculated as the percentage decrease in PASI score.

Statistical Analysis
The Shapiro-Wilk normalcy test was used for the continuous variables in the study. Variables with a normal distribution were analyzed with the paired t test and 1-way analysis of variance test and presented as mean (SD). Variables without a normal distribution were analyzed with the Wilcoxon t test and the Kruskal-Wallis test and presented as the median and 25th and 75th quartiles. The serum 25(OH)D levels were evaluated according to the seasons with the Kruskal-Wallis test. Categorical variables were expressed as frequency and percentages. The Pearson and Spearman correlation analysis and regression analysis were used to show the relationship between the variables (ie, age, Fitzpatrick skin type, PASI score, maximum NB-UVB dose, and number of sessions). The statistical significance level was set at P≤.05. Statistical analyses were performed using SPSS software version 21.

 

Results

A total of 49 participants (30 [61.22%] males; 19 [38.78%] females) were included in the study. The mean age (SD) was 40.27 (14.62) years (range, 19–74 years). Three (6.12%) participants were Fitzpatrick skin type I, 15 (30.61%) were skin type II, and 31 (63.27%) were skin type III.

The baseline median PASI score for the 49 participants was 10.20 (7.85–13.65). Baseline serum 25(OH)D levels were noted to be deficient in 40 participants (81.63%) and inadequate in 9 participants (18.37%). The distribution of the serum 25(OH)D levels of the participants according to the season was evaluated with the Kruskal-Wallis test and no association was found between serum 25(OH)D levels and seasonal changes (P=.685). Comparison of 25(OH)D basal values with Fitzpatrick skin type revealed a statistically significant relationship between skin type and vitamin D level (P=.024). The basal serum 25(OH)D levels were significantly lower in Fitzpatrick skin type II versus skin type I (P=.039).

Thirty-two (65.31%) participants achieved PASI 75 by the end of treatment. The baseline median PASI score (25th-75th quartiles) for the 32 patients was 10.45 (8.20-13.83) and the posttreatment PASI score was 1.95 (1.20-3.55), a statistically significant decrease following treatment (P<.001)(Table 1). Mean (SD) baseline serum 25(OH)D levels were 14.14 (6.70) ng/mL and posttreatment levels were 46.42 (15.51) ng/mL in these participants, which demonstrated a statistically significant increase during NB-UVB treatment (P<.001). None of the participants reached the toxicity levels (>80 ng/mL) for serum 25(OH)D. There were no significant changes in serum calcium or phosphorus levels posttreatment (Table 1), but statistically significant decreases in serum ALP and PTH levels were noted (P=.001 and P=.019, respectively)(Table 1).

Participants who completed the study (n=32) received an average (SD) of 30.09 (7.53) sessions of NB-UVB treatment and the mean (SD) MED was 611.88 (240.14) mJ/cm². The mean (SD) maximum dose was 2090.09 (341.78) mJ/cm² (Table 2).

Posttreatment serum 25(OH)D levels were compared with the number of NB-UVB phototherapy sessions and the maximum dose values. We found that the posttreatment serum 25(OH)D levels correlated with the number of sessions (P=.031) but not with the maximum dose (P=.498).

Using regression analysis, we also evaluated the effect of the increase in vitamin D levels—posttreatment serum 25(OH)D level minus baseline serum 25(OH)D levels—on the decrease in PASI scores—baseline PASI score minus posttreatment PASI score—and found no effect of serum 25(OH)D level increase on PASI decrease (P=.530). There was no correlation between increased serum 25(OH)D levels and age, Fitzpatrick skin type, or baseline PASI score.

 

Comment

The most effective UV wavelength for vitamin D synthesis is 295 to 300 nm, and therefore broadband UVB is frequently studied when determining the relationship between phototherapy and serum vitamin D levels.⁴ The current study demonstrated a statistically significant increase in serum 25(OH)D levels following NB-UVB treatment in patients with moderate to severe chronic plaque psoriasis (P<.001). This result supports other studies reporting that NB-UVB treatment in psoriasis patients increases serum 25(OH)D levels.^13-18

The main factor in the effective UVB level for vitamin D synthesis is the angle at which solar radiation reaches the earth, which is affected by the longitude, latitude, and time of day.¹⁹ For this reason, we planned to perform our study at a single center. Patients who stayed in open areas for more than 2 hours per day during the summer months (May through September) were excluded from the study to decrease the effect of seasonal changes on vitamin D levels. We evaluated the seasonal variation of vitamin D levels and found no relationship between seasonal changes and serum 25(OH)D levels. Therefore, the potential effect of seasonal changes on the vitamin D levels of study participants was excluded from the study.

The response to UV radiation changes according to age and Fitzpatrick skin type because 7-dehydrocholesterol levels decrease with age and melanin prevents the access of UVB photons to 7-dehydrocholesterol.²⁰ The basal serum 25(OH)D levels were deficient in 81.63% of participants and inadequate in 18.37%. In this study, we also observed that the basal serum 25(OH)D levels were significantly lower in patients with Fitzpatrick skin type II than in Fitzpatrick skin type I (P=.039). The mean (SD) serum 25(OH)D level at baseline was 14.14 (6.70) ng/mL and posttreatment was 46.42 (15.51) ng/mL in the 32 patients who completed the study. Serum 25(OH)D levels showed a statistically significant increase after NB-UVB treatment (P<.001). The increased serum 25(OH)D levels after NB-UVB phototherapy were not associated with Fitzpatrick skin type, which was consistent with the results of Osmancevic et al.¹⁷ The adjusted NB-UVB doses according to the different skin types might be responsible for this result in our study.

Participant age did not have a significant effect on serum 25(OH)D levels, similar to other studies in the literature.^13,17 We believe that artificial UVB radiation at high doses can compensate for the 7-dehydrocholesterol that decreases in the skin with aging.

We observed no significant change in the serum calcium and phosphorus levels with NB-UVB treatment in our study. None of the participants had a metabolic disorder related to increased 25(OH)D levels. The serum ALP and PTH levels decreased significantly following treatment (P=.001 and P=.019, respectively), which may have been secondary to increased serum 25(OH)D levels.

Posttreatment serum 25(OH)D levels were compared with the number of NB-UVB phototherapy sessions and maximum dose values. The posttreatment serum 25(OH)D levels were found to be related to the number of sessions received, but this value was not correlated with the maximum dose received. The MED and maximum dose were determined according to the Fitzpatrick skin type of the participants. Therefore, increased serum 25(OH)D levels with an increased number of sessions was an expected result. Our observation is in accordance with the finding described by Ryan et al.¹⁴ On the other hand, an in vitro study conducted by Olds et al²¹ reported that the relationship between UV light and cholecalciferol synthesis was not linear.

We found that increased serum 25(OH)D levels after treatment were not correlated with the decrease in PASI score, similar to studies by Romaní et al¹⁸ and Ryan et al.¹⁴ These results suggest that the clinical improvement following NB-UVB treatment is independent of the increased serum 25(OH)D levels in psoriasis patients.

Conclusion

In conclusion, we found that the serum 25(OH)D levels that increase as a result of NB-UVB therapy for the treatment of chronic plaque psoriasis has no statistically significant relationship with the age, Fitzpatrick skin type, baseline PASI score, changes in PASI, or maximum dose, while a positive relationship is present between the serum 25(OH)D levels and the number of sessions of NB-UVB.

In-hospital mobility (walking and transferring) is an important modifiable factor for posthospital functional outcomes and mortality among older adults.^1-4 In fact, daily mobility assessment has been considered for a standard clinical evaluation of the hospitalized older adult.^5,6 This would provide a ready source for targeting patients at risk for mobility impairment and identifying strategies to prevent in-hospital mobility limitation and posthospital functional decline. Despite their potential importance, mobility assessment tools have not been readily adopted in the hospital setting.

There are various ways to assess mobility in hospital settings. Mobility tracking technology (radar and accelerometers) has demonstrated older adults have extremely low mobility during hospitalization. Although these objective methods provide an unbiased way to monitor physical activity level and track in-hospital mobility change,^6-8 and have provided important information about mobility in the hospital, they are largely impractical in real-world settings.

While mobility technology appears to be advancing, there is a potential to assess in-hospital mobility using commonly administered and inexpensive tools. Many hospitals ask staff to regularly rate physical function (Braden and Morse score) as part of their standard-of-care procedures. The rating scales used have the potential to provide valuable information about mobility variations without using special equipment or burdening patients. The Braden Scale for Predicting Pressure Sore Risk is a good example of a validated assessment instrument that is better than nurses’ judgment, which is often confounded by nursing experience.⁹ This scale, which has 6 subscales (Sensory Perception, Moisture, Activity, Mobility, Nutrition, Friction and Shear), has shown high sensitivity in detecting patient condition changes in the clinical setting.¹⁰ The scale typically is used holistically to evaluate pressure ulcer risk, but the Activity subscale, which assesses mobility, could serve as a useful tool for predicting posthospital recovery and identifying needs for posthospital mobility interventions.

We conducted a study to evaluate the prognostic value of using the Braden Activity subscale (BAS) to identify in-hospital incident mobility impairment and recovery for predicting mortality and discharge status among hospitalized older adults.

METHODS

The University of Florida Gainesville Health Science Center Institutional Review Board reviewed and approved the study protocol as exempt from human subjects’ research.

Design and Setting

The design followed a retrospective cohort study in which hospitalized patients were evaluated at admission (baseline) and assessed throughout their stay for incident mobility impairment and recovery. Data were collected in older adults (≥65 years old) hospitalized at UF Health Shands Hospital (University of Florida), an 852-bed level I trauma center in Gainesville, Florida.

 

Data Sources

Patient data from electronic medical records were warehoused in an integrated data repository (IDR) between January 1, 2009 and April 20, 2014. The IDR aggregates clinical and administrative system data, which can subsequently be used for research. The data were compiled in a de-identified longitudinal dataset that included demographics, Charlson Comorbidity Index,¹¹ hospital length of stay, BAS scores (at admission, during hospitalization, at discharge), discharge disposition (including in-hospital death), and mortality after hospitalization (from the national Social Security Death Index).

Patients

The study population consisted of 19,769 older adults (≥65 years old) hospitalized between January 1, 2009 and April 20, 2014.

Outcomes

The major outcomes were patients’ primary discharge disposition and posthospital mortality over 4.5-year follow-up. Discharge dispositions were divided into 9 categories: expired in hospital, other hospital admission, home, home care, hospice, rehabilitation, skilled nursing home, healthcare facility, or other, which included psychiatric facilities, court, or law enforcement.

Predictors

The BAS was used to identify incident mobility impairment and incident mobility recovery during hospitalization and subsequently was used to predict discharge disposition and mortality. The Braden scale,¹² which is commonly administered to predict pressure sores, has 6 subscales: Sensory Perception, Moisture, Activity, Mobility, Nutrition, and Friction and Shear. Each subscale has a score of 1 to 4, with higher scores representing higher activity levels. In particular, the BAS measures the mobility (walking and transferring) level of the hospitalized patient with a score of 1 (“patient is confined to bed”), 2 (“severely limited or nonexistent ability to walk; patient cannot bear his own weight and/or must be assisted into chair or wheelchair”), 3 (“patient walks occasionally during the day, but for very short distances, with or without assistance; he spends majority of each shift in bed or chair”), or 4 (“patient walks outside the room at least twice a day and inside the room at least once every 2 hours during waking hours”). The BAS is correlated with the total Braden scale¹⁰ and has shown excellent interrater reliability (interclass correlation coefficient, 0.96) among hospital staff.¹³ Analysis of the current dataset revealed excellent rater agreement across 3 working shifts (κ = 0.76 for first day of hospitalization in those hospitalized <3 days; κ = 0.70 for first day in those hospitalized ≥3 days).

UF Health Shands Hospital nursing staff administered the BAS at each shift change during a hospital stay (~3 times/d). Mobility scores were averaged across an entire day to reduce potential interrater variation. A daily average BAS score cutpoint was chosen to capture an absorbing mobility state. Average BAS score ≥3 was selected, as it indicates a patient is mobile most of the day, whereas average BAS score <3 indicates significant mobility impairment most of the day. The average daily score was calculated with a minimum of 3 determinations per day. Incident mobility impairment was defined as first transition from “being able to walk occasionally or twice a day outside or at least once every 2 hours during waking hours” to “severely limited or nonexistent ability to walk or confined to bed.” Numerically speaking, daily average BAS score transition from ≥3 at admission to <3 during hospitalization constituted a mobility impairment event. Incident mobility recovery was evaluated in those patient hospital observations that were “severely limited or nonexistent ability to walk or confined to bed” at admission. Incident mobility recovery was defined as first transition to “ability to walk occasionally or twice a day outside or at least once every 2 hours during waking hours.” A mobility recovery event was operationally defined as daily average BAS score transition from <3 at admission to daily average of ≥3 during hospitalization.

Data Analysis

Patient baseline characteristics are reported as counts, means, or medians. Chi-square statistics were used to test group differences for categorical variables, and analysis of variance was performed for continuous variables. Posthospital outcomes were evaluated descriptively and with time-to-event analyses. Kaplan-Meier curves and Wilcoxon P were also used to compare the survival probability for the mobility impairment and recovery groups. Although Cox proportional hazard regression is appropriate for these data, we found the proportionality assumption tenuous. As an alternative, logistic regression was used to model the probability of impairment/recovery outcomes. In addition, a survival time estimate that is robust to the proportionality assumption was derived according to Royston and Parmar^14,15 and Zhao et al.¹⁶ This approach reports the difference between 2 survival curves using the restricted mean—a measure of average survival using the area under the survival curve from time point zero to last observed follow-up time. All models were adjusted for age, sex, race, and hospital length of stay. Analyses were performed with R 3.1.1.¹⁷ All analyses were 2-tailed, and an α of 0.05 was considered statistically significant.

 

RESULTS

Table 1 lists the baseline characteristics of the hospitalized patients: 10,717 (54%) with normal mobility at admission and 9052 (46%) admitted with impaired mobility. Compared with patients admitted with normal mobility, those with impaired mobility at admission were older, mean (SD) 75.73 (7.84) years versus 73.73 (7.00) years; spent more days in the hospital, median 5 days versus 3 days; and had a higher Charlson Comorbidity Index, mean (SD) 2.59 (2.34) versus 2.22 (2.31). Patients with impaired mobility at admission had a significantly higher prevalence of myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular disease, dementia, and diabetes. However, cancer was significantly more prevalent among patients admitted with normal mobility compared with those admitted with impaired mobility.

Of the 10,717 patients with normal mobility at admission, 2218 (20.7%) had incident mobility impairment over a median follow-up of 3 days (interquartile range, 2-5 days). Of the 9052 patients admitted with impaired mobility, 4734 (52.3%) recovered from their impairment over a median follow-up of 5 days (interquartile range, 3-9 days).

The Kaplan-Meier curves in Figure 1 show survival probability between patients who did and did not develop incident mobility impairment during hospitalization, as well as between patients who did and did not recover incident mobility. Table 2 lists the odds ratios (ORs) and restricted mean survival times for patients who developed impairment and patients who recovered. The results are provided for the entire follow-up period and for before and after 6 months of follow-up. Older adults who became mobility impaired in the hospital had an odds of death higher than that of those who remained mobile (OR, 1.23; 95% confidence interval [CI], 1.08-1.39). This effect predominately occurred within the first 6 follow-up months (OR, 1.67; 95% CI, 1.40-1.96). Older adults who recovered from mobility impairment had an odds of death lower than that of those who did not recover mobility in the hospital (OR, 0.54; 95% CI, 0.49-0.59). This effect was slightly stronger within the first 6 months after hospitalization but remained significant after 6 months. Figure 2 shows the percentages of different discharge dispositions for mobility impairment and recovery. Older adults with mobility impairment were more likely to die in the hospital or to be discharged to hospice. Otherwise, patients who recovered their mobility during hospitalization were more likely to be discharged home and to home care.

DISCUSSION

In this study, we evaluated the predictive value of the BAS in assessing incident mobility impairment and recovery during hospitalization among older adults. Patients admitted with impaired mobility were older, spent more days in the hospital, and had more comorbidities than those admitted with normal mobility. Compared with older adults who did not develop incident mobility impairment during hospitalization, those who became mobility impaired had a higher posthospital mortality risk and a higher prevalence of in-hospital death and hospice discharge. In addition, compared with older adults who did not recover mobility in the hospital, those who recovered mobility had a lower posthospital mortality risk and a higher prevalence of home discharge. It is interesting that incident in the hospital appears to have a finite effect. The association was largely erased 6 months after discharge. This was also observed in patients who recovered their mobility in the hospital, but to a lesser extent. Overall, the results suggest that developing mobility impairment or recovering from mobility impairment in the hospital is an important predictor of discharge status and posthospital mortality.

The large number of patient observations and repeated evaluation of in-hospital mobility made this analysis possible. To our knowledge, this is the first large-scale study to evaluate the predictive value of the BAS in assessing mobility impairment and recovery during hospitalization among older adults. Such a test provides a simple and efficient assessment of in-hospital mobility changes that are sensitive to discharge locations and posthospital mortality risk.

Poor mobility in the hospital is associated with higher posthospital mortality. Kasotakis et al.¹⁸ evaluated the predictive value of a nursing staff–assessed clinical mobility score for surgical critically ill patients whose functional mobility was unimpaired on presentation. The Surgical Intensive Care Unit Optimal Mobility Score has been shown to be a reliable and valid tool for predicting mortality in a relatively young population (average age, 60 years). Using accelerometer technology with older adults, Ostir et al.⁷ found that each 100-step increase was associated with 2% and 3% lower risk of death over 2 years in the first and last 24 hours of hospitalization, respectively. The present mortality results show that mobility patterns in the hospital are crucially important for patients’ health the first 6 months after discharge. This finding suggests that developing mobility impairment in the hospital is a sign for significant and rapid health decline. It also suggests that interventions need to be started relatively early in order to reduce the risk of death. In contrast, patients who recover mobility in the hospital obtain a substantial mortality risk reduction. In-hospital interventions to enhance mobility recovery and prevent mobility impairment could have a large impact on posthospital adverse events, particularly for older patients, who are susceptible to disease complications.

Regarding discharge disposition, Sommerfeld and von Arbin¹⁹ found that the ability to rise from a chair (a component of mobility) during hospitalization was a strong predictor of early discharge home. Similarly, Vochteloo et al.²⁰ found that limited mobility as assessed with a questionnaire was associated with discharge to a location other than home among patients with hip fracture. We utilized existing information, collected at a relatively high resolution (3 times per day) that is often readily available without added patient burden. This is particularly important in the hospital setting, where added assessments in frail older adults and in those with multimorbid conditions is challenging. Although our approach is appealing, we should note that BAS scores were modified to reduce interrater variation and capture more absorbing mobility states over a hospitalized day, and that a similar approach would be required to replicate these results and provide clinical value to the BAS as a prognostic indicator of posthospital mortality.

Despite the strengths of this study, it had notable limitations. Pooling BAS scores could have modified the interpretation and clinical implications of the results. Although we had a large number of patient observations, this retrospective analysis may have had biases that were not completely considered. In addition, the results of this single-center study cannot be generalized across all hospital systems. The Braden activity sub score has demonstrated good validity and reliability for activity changes¹³, but this measure was not objectively ascertained as demonstrated by others using accelerometers^6-7. Moreover, the medical records used did not provide prehospital patient mobility status, limiting adjustments for prehospital mobility function. Despite these limitations, this study represents an important initial step in validating a simple and efficient clinical tool for identifying in-hospital mobility impairment and recovery and predicting posthospital adverse outcomes.

BAS assessment of incident mobility impairment and recovery in the hospital setting has prognostic value in predicting discharge disposition, in-hospital death, and posthospital mortality risk. That the majority of the effect appears to occur within the first 6 months after discharge suggests that interventions to improve mobility should be started during hospitalization or expeditiously after discharge. Overall, this study’s results showed that a simple and efficient mobility status assessment can become a valuable clinical and administrative tool for targeting and improving mobility in the hospital and after discharge in older adults.

 

Acknowledgments

This work was supported by the National Institutes of Health and the National Center for Advancing Translational Sciences (NIH/NCATS) Clinical and Translational Science Award to the University of Florida (UL1 TR000064) and by the University of Florida’s Claude D. Pepper Center (P30AG028740-R6, significant contributions from the Data and Applied Science Core and Biostatistical Core).

Disclosure

Nothing to report.

In-hospital mobility (walking and transferring) is an important modifiable factor for posthospital functional outcomes and mortality among older adults.^1-4 In fact, daily mobility assessment has been considered for a standard clinical evaluation of the hospitalized older adult.^5,6 This would provide a ready source for targeting patients at risk for mobility impairment and identifying strategies to prevent in-hospital mobility limitation and posthospital functional decline. Despite their potential importance, mobility assessment tools have not been readily adopted in the hospital setting.

There are various ways to assess mobility in hospital settings. Mobility tracking technology (radar and accelerometers) has demonstrated older adults have extremely low mobility during hospitalization. Although these objective methods provide an unbiased way to monitor physical activity level and track in-hospital mobility change,^6-8 and have provided important information about mobility in the hospital, they are largely impractical in real-world settings.

While mobility technology appears to be advancing, there is a potential to assess in-hospital mobility using commonly administered and inexpensive tools. Many hospitals ask staff to regularly rate physical function (Braden and Morse score) as part of their standard-of-care procedures. The rating scales used have the potential to provide valuable information about mobility variations without using special equipment or burdening patients. The Braden Scale for Predicting Pressure Sore Risk is a good example of a validated assessment instrument that is better than nurses’ judgment, which is often confounded by nursing experience.⁹ This scale, which has 6 subscales (Sensory Perception, Moisture, Activity, Mobility, Nutrition, Friction and Shear), has shown high sensitivity in detecting patient condition changes in the clinical setting.¹⁰ The scale typically is used holistically to evaluate pressure ulcer risk, but the Activity subscale, which assesses mobility, could serve as a useful tool for predicting posthospital recovery and identifying needs for posthospital mobility interventions.

We conducted a study to evaluate the prognostic value of using the Braden Activity subscale (BAS) to identify in-hospital incident mobility impairment and recovery for predicting mortality and discharge status among hospitalized older adults.

METHODS

The University of Florida Gainesville Health Science Center Institutional Review Board reviewed and approved the study protocol as exempt from human subjects’ research.

Design and Setting

The design followed a retrospective cohort study in which hospitalized patients were evaluated at admission (baseline) and assessed throughout their stay for incident mobility impairment and recovery. Data were collected in older adults (≥65 years old) hospitalized at UF Health Shands Hospital (University of Florida), an 852-bed level I trauma center in Gainesville, Florida.

 

Data Sources

Patient data from electronic medical records were warehoused in an integrated data repository (IDR) between January 1, 2009 and April 20, 2014. The IDR aggregates clinical and administrative system data, which can subsequently be used for research. The data were compiled in a de-identified longitudinal dataset that included demographics, Charlson Comorbidity Index,¹¹ hospital length of stay, BAS scores (at admission, during hospitalization, at discharge), discharge disposition (including in-hospital death), and mortality after hospitalization (from the national Social Security Death Index).

Patients

The study population consisted of 19,769 older adults (≥65 years old) hospitalized between January 1, 2009 and April 20, 2014.

Outcomes

The major outcomes were patients’ primary discharge disposition and posthospital mortality over 4.5-year follow-up. Discharge dispositions were divided into 9 categories: expired in hospital, other hospital admission, home, home care, hospice, rehabilitation, skilled nursing home, healthcare facility, or other, which included psychiatric facilities, court, or law enforcement.

Predictors

The BAS was used to identify incident mobility impairment and incident mobility recovery during hospitalization and subsequently was used to predict discharge disposition and mortality. The Braden scale,¹² which is commonly administered to predict pressure sores, has 6 subscales: Sensory Perception, Moisture, Activity, Mobility, Nutrition, and Friction and Shear. Each subscale has a score of 1 to 4, with higher scores representing higher activity levels. In particular, the BAS measures the mobility (walking and transferring) level of the hospitalized patient with a score of 1 (“patient is confined to bed”), 2 (“severely limited or nonexistent ability to walk; patient cannot bear his own weight and/or must be assisted into chair or wheelchair”), 3 (“patient walks occasionally during the day, but for very short distances, with or without assistance; he spends majority of each shift in bed or chair”), or 4 (“patient walks outside the room at least twice a day and inside the room at least once every 2 hours during waking hours”). The BAS is correlated with the total Braden scale¹⁰ and has shown excellent interrater reliability (interclass correlation coefficient, 0.96) among hospital staff.¹³ Analysis of the current dataset revealed excellent rater agreement across 3 working shifts (κ = 0.76 for first day of hospitalization in those hospitalized <3 days; κ = 0.70 for first day in those hospitalized ≥3 days).

UF Health Shands Hospital nursing staff administered the BAS at each shift change during a hospital stay (~3 times/d). Mobility scores were averaged across an entire day to reduce potential interrater variation. A daily average BAS score cutpoint was chosen to capture an absorbing mobility state. Average BAS score ≥3 was selected, as it indicates a patient is mobile most of the day, whereas average BAS score <3 indicates significant mobility impairment most of the day. The average daily score was calculated with a minimum of 3 determinations per day. Incident mobility impairment was defined as first transition from “being able to walk occasionally or twice a day outside or at least once every 2 hours during waking hours” to “severely limited or nonexistent ability to walk or confined to bed.” Numerically speaking, daily average BAS score transition from ≥3 at admission to <3 during hospitalization constituted a mobility impairment event. Incident mobility recovery was evaluated in those patient hospital observations that were “severely limited or nonexistent ability to walk or confined to bed” at admission. Incident mobility recovery was defined as first transition to “ability to walk occasionally or twice a day outside or at least once every 2 hours during waking hours.” A mobility recovery event was operationally defined as daily average BAS score transition from <3 at admission to daily average of ≥3 during hospitalization.

Data Analysis

Patient baseline characteristics are reported as counts, means, or medians. Chi-square statistics were used to test group differences for categorical variables, and analysis of variance was performed for continuous variables. Posthospital outcomes were evaluated descriptively and with time-to-event analyses. Kaplan-Meier curves and Wilcoxon P were also used to compare the survival probability for the mobility impairment and recovery groups. Although Cox proportional hazard regression is appropriate for these data, we found the proportionality assumption tenuous. As an alternative, logistic regression was used to model the probability of impairment/recovery outcomes. In addition, a survival time estimate that is robust to the proportionality assumption was derived according to Royston and Parmar^14,15 and Zhao et al.¹⁶ This approach reports the difference between 2 survival curves using the restricted mean—a measure of average survival using the area under the survival curve from time point zero to last observed follow-up time. All models were adjusted for age, sex, race, and hospital length of stay. Analyses were performed with R 3.1.1.¹⁷ All analyses were 2-tailed, and an α of 0.05 was considered statistically significant.

 

RESULTS

Table 1 lists the baseline characteristics of the hospitalized patients: 10,717 (54%) with normal mobility at admission and 9052 (46%) admitted with impaired mobility. Compared with patients admitted with normal mobility, those with impaired mobility at admission were older, mean (SD) 75.73 (7.84) years versus 73.73 (7.00) years; spent more days in the hospital, median 5 days versus 3 days; and had a higher Charlson Comorbidity Index, mean (SD) 2.59 (2.34) versus 2.22 (2.31). Patients with impaired mobility at admission had a significantly higher prevalence of myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular disease, dementia, and diabetes. However, cancer was significantly more prevalent among patients admitted with normal mobility compared with those admitted with impaired mobility.

Of the 10,717 patients with normal mobility at admission, 2218 (20.7%) had incident mobility impairment over a median follow-up of 3 days (interquartile range, 2-5 days). Of the 9052 patients admitted with impaired mobility, 4734 (52.3%) recovered from their impairment over a median follow-up of 5 days (interquartile range, 3-9 days).

The Kaplan-Meier curves in Figure 1 show survival probability between patients who did and did not develop incident mobility impairment during hospitalization, as well as between patients who did and did not recover incident mobility. Table 2 lists the odds ratios (ORs) and restricted mean survival times for patients who developed impairment and patients who recovered. The results are provided for the entire follow-up period and for before and after 6 months of follow-up. Older adults who became mobility impaired in the hospital had an odds of death higher than that of those who remained mobile (OR, 1.23; 95% confidence interval [CI], 1.08-1.39). This effect predominately occurred within the first 6 follow-up months (OR, 1.67; 95% CI, 1.40-1.96). Older adults who recovered from mobility impairment had an odds of death lower than that of those who did not recover mobility in the hospital (OR, 0.54; 95% CI, 0.49-0.59). This effect was slightly stronger within the first 6 months after hospitalization but remained significant after 6 months. Figure 2 shows the percentages of different discharge dispositions for mobility impairment and recovery. Older adults with mobility impairment were more likely to die in the hospital or to be discharged to hospice. Otherwise, patients who recovered their mobility during hospitalization were more likely to be discharged home and to home care.

DISCUSSION

In this study, we evaluated the predictive value of the BAS in assessing incident mobility impairment and recovery during hospitalization among older adults. Patients admitted with impaired mobility were older, spent more days in the hospital, and had more comorbidities than those admitted with normal mobility. Compared with older adults who did not develop incident mobility impairment during hospitalization, those who became mobility impaired had a higher posthospital mortality risk and a higher prevalence of in-hospital death and hospice discharge. In addition, compared with older adults who did not recover mobility in the hospital, those who recovered mobility had a lower posthospital mortality risk and a higher prevalence of home discharge. It is interesting that incident in the hospital appears to have a finite effect. The association was largely erased 6 months after discharge. This was also observed in patients who recovered their mobility in the hospital, but to a lesser extent. Overall, the results suggest that developing mobility impairment or recovering from mobility impairment in the hospital is an important predictor of discharge status and posthospital mortality.

The large number of patient observations and repeated evaluation of in-hospital mobility made this analysis possible. To our knowledge, this is the first large-scale study to evaluate the predictive value of the BAS in assessing mobility impairment and recovery during hospitalization among older adults. Such a test provides a simple and efficient assessment of in-hospital mobility changes that are sensitive to discharge locations and posthospital mortality risk.

Poor mobility in the hospital is associated with higher posthospital mortality. Kasotakis et al.¹⁸ evaluated the predictive value of a nursing staff–assessed clinical mobility score for surgical critically ill patients whose functional mobility was unimpaired on presentation. The Surgical Intensive Care Unit Optimal Mobility Score has been shown to be a reliable and valid tool for predicting mortality in a relatively young population (average age, 60 years). Using accelerometer technology with older adults, Ostir et al.⁷ found that each 100-step increase was associated with 2% and 3% lower risk of death over 2 years in the first and last 24 hours of hospitalization, respectively. The present mortality results show that mobility patterns in the hospital are crucially important for patients’ health the first 6 months after discharge. This finding suggests that developing mobility impairment in the hospital is a sign for significant and rapid health decline. It also suggests that interventions need to be started relatively early in order to reduce the risk of death. In contrast, patients who recover mobility in the hospital obtain a substantial mortality risk reduction. In-hospital interventions to enhance mobility recovery and prevent mobility impairment could have a large impact on posthospital adverse events, particularly for older patients, who are susceptible to disease complications.

Regarding discharge disposition, Sommerfeld and von Arbin¹⁹ found that the ability to rise from a chair (a component of mobility) during hospitalization was a strong predictor of early discharge home. Similarly, Vochteloo et al.²⁰ found that limited mobility as assessed with a questionnaire was associated with discharge to a location other than home among patients with hip fracture. We utilized existing information, collected at a relatively high resolution (3 times per day) that is often readily available without added patient burden. This is particularly important in the hospital setting, where added assessments in frail older adults and in those with multimorbid conditions is challenging. Although our approach is appealing, we should note that BAS scores were modified to reduce interrater variation and capture more absorbing mobility states over a hospitalized day, and that a similar approach would be required to replicate these results and provide clinical value to the BAS as a prognostic indicator of posthospital mortality.

Despite the strengths of this study, it had notable limitations. Pooling BAS scores could have modified the interpretation and clinical implications of the results. Although we had a large number of patient observations, this retrospective analysis may have had biases that were not completely considered. In addition, the results of this single-center study cannot be generalized across all hospital systems. The Braden activity sub score has demonstrated good validity and reliability for activity changes¹³, but this measure was not objectively ascertained as demonstrated by others using accelerometers^6-7. Moreover, the medical records used did not provide prehospital patient mobility status, limiting adjustments for prehospital mobility function. Despite these limitations, this study represents an important initial step in validating a simple and efficient clinical tool for identifying in-hospital mobility impairment and recovery and predicting posthospital adverse outcomes.

BAS assessment of incident mobility impairment and recovery in the hospital setting has prognostic value in predicting discharge disposition, in-hospital death, and posthospital mortality risk. That the majority of the effect appears to occur within the first 6 months after discharge suggests that interventions to improve mobility should be started during hospitalization or expeditiously after discharge. Overall, this study’s results showed that a simple and efficient mobility status assessment can become a valuable clinical and administrative tool for targeting and improving mobility in the hospital and after discharge in older adults.

 

Acknowledgments

This work was supported by the National Institutes of Health and the National Center for Advancing Translational Sciences (NIH/NCATS) Clinical and Translational Science Award to the University of Florida (UL1 TR000064) and by the University of Florida’s Claude D. Pepper Center (P30AG028740-R6, significant contributions from the Data and Applied Science Core and Biostatistical Core).

Disclosure

Nothing to report.

In-hospital mobility (walking and transferring) is an important modifiable factor for posthospital functional outcomes and mortality among older adults.^1-4 In fact, daily mobility assessment has been considered for a standard clinical evaluation of the hospitalized older adult.^5,6 This would provide a ready source for targeting patients at risk for mobility impairment and identifying strategies to prevent in-hospital mobility limitation and posthospital functional decline. Despite their potential importance, mobility assessment tools have not been readily adopted in the hospital setting.

There are various ways to assess mobility in hospital settings. Mobility tracking technology (radar and accelerometers) has demonstrated older adults have extremely low mobility during hospitalization. Although these objective methods provide an unbiased way to monitor physical activity level and track in-hospital mobility change,^6-8 and have provided important information about mobility in the hospital, they are largely impractical in real-world settings.

While mobility technology appears to be advancing, there is a potential to assess in-hospital mobility using commonly administered and inexpensive tools. Many hospitals ask staff to regularly rate physical function (Braden and Morse score) as part of their standard-of-care procedures. The rating scales used have the potential to provide valuable information about mobility variations without using special equipment or burdening patients. The Braden Scale for Predicting Pressure Sore Risk is a good example of a validated assessment instrument that is better than nurses’ judgment, which is often confounded by nursing experience.⁹ This scale, which has 6 subscales (Sensory Perception, Moisture, Activity, Mobility, Nutrition, Friction and Shear), has shown high sensitivity in detecting patient condition changes in the clinical setting.¹⁰ The scale typically is used holistically to evaluate pressure ulcer risk, but the Activity subscale, which assesses mobility, could serve as a useful tool for predicting posthospital recovery and identifying needs for posthospital mobility interventions.

We conducted a study to evaluate the prognostic value of using the Braden Activity subscale (BAS) to identify in-hospital incident mobility impairment and recovery for predicting mortality and discharge status among hospitalized older adults.

METHODS

The University of Florida Gainesville Health Science Center Institutional Review Board reviewed and approved the study protocol as exempt from human subjects’ research.

Design and Setting

The design followed a retrospective cohort study in which hospitalized patients were evaluated at admission (baseline) and assessed throughout their stay for incident mobility impairment and recovery. Data were collected in older adults (≥65 years old) hospitalized at UF Health Shands Hospital (University of Florida), an 852-bed level I trauma center in Gainesville, Florida.

 

Data Sources

Patient data from electronic medical records were warehoused in an integrated data repository (IDR) between January 1, 2009 and April 20, 2014. The IDR aggregates clinical and administrative system data, which can subsequently be used for research. The data were compiled in a de-identified longitudinal dataset that included demographics, Charlson Comorbidity Index,¹¹ hospital length of stay, BAS scores (at admission, during hospitalization, at discharge), discharge disposition (including in-hospital death), and mortality after hospitalization (from the national Social Security Death Index).

Patients

The study population consisted of 19,769 older adults (≥65 years old) hospitalized between January 1, 2009 and April 20, 2014.

Outcomes

The major outcomes were patients’ primary discharge disposition and posthospital mortality over 4.5-year follow-up. Discharge dispositions were divided into 9 categories: expired in hospital, other hospital admission, home, home care, hospice, rehabilitation, skilled nursing home, healthcare facility, or other, which included psychiatric facilities, court, or law enforcement.

Predictors

The BAS was used to identify incident mobility impairment and incident mobility recovery during hospitalization and subsequently was used to predict discharge disposition and mortality. The Braden scale,¹² which is commonly administered to predict pressure sores, has 6 subscales: Sensory Perception, Moisture, Activity, Mobility, Nutrition, and Friction and Shear. Each subscale has a score of 1 to 4, with higher scores representing higher activity levels. In particular, the BAS measures the mobility (walking and transferring) level of the hospitalized patient with a score of 1 (“patient is confined to bed”), 2 (“severely limited or nonexistent ability to walk; patient cannot bear his own weight and/or must be assisted into chair or wheelchair”), 3 (“patient walks occasionally during the day, but for very short distances, with or without assistance; he spends majority of each shift in bed or chair”), or 4 (“patient walks outside the room at least twice a day and inside the room at least once every 2 hours during waking hours”). The BAS is correlated with the total Braden scale¹⁰ and has shown excellent interrater reliability (interclass correlation coefficient, 0.96) among hospital staff.¹³ Analysis of the current dataset revealed excellent rater agreement across 3 working shifts (κ = 0.76 for first day of hospitalization in those hospitalized <3 days; κ = 0.70 for first day in those hospitalized ≥3 days).

UF Health Shands Hospital nursing staff administered the BAS at each shift change during a hospital stay (~3 times/d). Mobility scores were averaged across an entire day to reduce potential interrater variation. A daily average BAS score cutpoint was chosen to capture an absorbing mobility state. Average BAS score ≥3 was selected, as it indicates a patient is mobile most of the day, whereas average BAS score <3 indicates significant mobility impairment most of the day. The average daily score was calculated with a minimum of 3 determinations per day. Incident mobility impairment was defined as first transition from “being able to walk occasionally or twice a day outside or at least once every 2 hours during waking hours” to “severely limited or nonexistent ability to walk or confined to bed.” Numerically speaking, daily average BAS score transition from ≥3 at admission to <3 during hospitalization constituted a mobility impairment event. Incident mobility recovery was evaluated in those patient hospital observations that were “severely limited or nonexistent ability to walk or confined to bed” at admission. Incident mobility recovery was defined as first transition to “ability to walk occasionally or twice a day outside or at least once every 2 hours during waking hours.” A mobility recovery event was operationally defined as daily average BAS score transition from <3 at admission to daily average of ≥3 during hospitalization.

Data Analysis

Patient baseline characteristics are reported as counts, means, or medians. Chi-square statistics were used to test group differences for categorical variables, and analysis of variance was performed for continuous variables. Posthospital outcomes were evaluated descriptively and with time-to-event analyses. Kaplan-Meier curves and Wilcoxon P were also used to compare the survival probability for the mobility impairment and recovery groups. Although Cox proportional hazard regression is appropriate for these data, we found the proportionality assumption tenuous. As an alternative, logistic regression was used to model the probability of impairment/recovery outcomes. In addition, a survival time estimate that is robust to the proportionality assumption was derived according to Royston and Parmar^14,15 and Zhao et al.¹⁶ This approach reports the difference between 2 survival curves using the restricted mean—a measure of average survival using the area under the survival curve from time point zero to last observed follow-up time. All models were adjusted for age, sex, race, and hospital length of stay. Analyses were performed with R 3.1.1.¹⁷ All analyses were 2-tailed, and an α of 0.05 was considered statistically significant.

 

RESULTS

Table 1 lists the baseline characteristics of the hospitalized patients: 10,717 (54%) with normal mobility at admission and 9052 (46%) admitted with impaired mobility. Compared with patients admitted with normal mobility, those with impaired mobility at admission were older, mean (SD) 75.73 (7.84) years versus 73.73 (7.00) years; spent more days in the hospital, median 5 days versus 3 days; and had a higher Charlson Comorbidity Index, mean (SD) 2.59 (2.34) versus 2.22 (2.31). Patients with impaired mobility at admission had a significantly higher prevalence of myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular disease, dementia, and diabetes. However, cancer was significantly more prevalent among patients admitted with normal mobility compared with those admitted with impaired mobility.

Of the 10,717 patients with normal mobility at admission, 2218 (20.7%) had incident mobility impairment over a median follow-up of 3 days (interquartile range, 2-5 days). Of the 9052 patients admitted with impaired mobility, 4734 (52.3%) recovered from their impairment over a median follow-up of 5 days (interquartile range, 3-9 days).

The Kaplan-Meier curves in Figure 1 show survival probability between patients who did and did not develop incident mobility impairment during hospitalization, as well as between patients who did and did not recover incident mobility. Table 2 lists the odds ratios (ORs) and restricted mean survival times for patients who developed impairment and patients who recovered. The results are provided for the entire follow-up period and for before and after 6 months of follow-up. Older adults who became mobility impaired in the hospital had an odds of death higher than that of those who remained mobile (OR, 1.23; 95% confidence interval [CI], 1.08-1.39). This effect predominately occurred within the first 6 follow-up months (OR, 1.67; 95% CI, 1.40-1.96). Older adults who recovered from mobility impairment had an odds of death lower than that of those who did not recover mobility in the hospital (OR, 0.54; 95% CI, 0.49-0.59). This effect was slightly stronger within the first 6 months after hospitalization but remained significant after 6 months. Figure 2 shows the percentages of different discharge dispositions for mobility impairment and recovery. Older adults with mobility impairment were more likely to die in the hospital or to be discharged to hospice. Otherwise, patients who recovered their mobility during hospitalization were more likely to be discharged home and to home care.

DISCUSSION

In this study, we evaluated the predictive value of the BAS in assessing incident mobility impairment and recovery during hospitalization among older adults. Patients admitted with impaired mobility were older, spent more days in the hospital, and had more comorbidities than those admitted with normal mobility. Compared with older adults who did not develop incident mobility impairment during hospitalization, those who became mobility impaired had a higher posthospital mortality risk and a higher prevalence of in-hospital death and hospice discharge. In addition, compared with older adults who did not recover mobility in the hospital, those who recovered mobility had a lower posthospital mortality risk and a higher prevalence of home discharge. It is interesting that incident in the hospital appears to have a finite effect. The association was largely erased 6 months after discharge. This was also observed in patients who recovered their mobility in the hospital, but to a lesser extent. Overall, the results suggest that developing mobility impairment or recovering from mobility impairment in the hospital is an important predictor of discharge status and posthospital mortality.

The large number of patient observations and repeated evaluation of in-hospital mobility made this analysis possible. To our knowledge, this is the first large-scale study to evaluate the predictive value of the BAS in assessing mobility impairment and recovery during hospitalization among older adults. Such a test provides a simple and efficient assessment of in-hospital mobility changes that are sensitive to discharge locations and posthospital mortality risk.

Poor mobility in the hospital is associated with higher posthospital mortality. Kasotakis et al.¹⁸ evaluated the predictive value of a nursing staff–assessed clinical mobility score for surgical critically ill patients whose functional mobility was unimpaired on presentation. The Surgical Intensive Care Unit Optimal Mobility Score has been shown to be a reliable and valid tool for predicting mortality in a relatively young population (average age, 60 years). Using accelerometer technology with older adults, Ostir et al.⁷ found that each 100-step increase was associated with 2% and 3% lower risk of death over 2 years in the first and last 24 hours of hospitalization, respectively. The present mortality results show that mobility patterns in the hospital are crucially important for patients’ health the first 6 months after discharge. This finding suggests that developing mobility impairment in the hospital is a sign for significant and rapid health decline. It also suggests that interventions need to be started relatively early in order to reduce the risk of death. In contrast, patients who recover mobility in the hospital obtain a substantial mortality risk reduction. In-hospital interventions to enhance mobility recovery and prevent mobility impairment could have a large impact on posthospital adverse events, particularly for older patients, who are susceptible to disease complications.

Regarding discharge disposition, Sommerfeld and von Arbin¹⁹ found that the ability to rise from a chair (a component of mobility) during hospitalization was a strong predictor of early discharge home. Similarly, Vochteloo et al.²⁰ found that limited mobility as assessed with a questionnaire was associated with discharge to a location other than home among patients with hip fracture. We utilized existing information, collected at a relatively high resolution (3 times per day) that is often readily available without added patient burden. This is particularly important in the hospital setting, where added assessments in frail older adults and in those with multimorbid conditions is challenging. Although our approach is appealing, we should note that BAS scores were modified to reduce interrater variation and capture more absorbing mobility states over a hospitalized day, and that a similar approach would be required to replicate these results and provide clinical value to the BAS as a prognostic indicator of posthospital mortality.

Despite the strengths of this study, it had notable limitations. Pooling BAS scores could have modified the interpretation and clinical implications of the results. Although we had a large number of patient observations, this retrospective analysis may have had biases that were not completely considered. In addition, the results of this single-center study cannot be generalized across all hospital systems. The Braden activity sub score has demonstrated good validity and reliability for activity changes¹³, but this measure was not objectively ascertained as demonstrated by others using accelerometers^6-7. Moreover, the medical records used did not provide prehospital patient mobility status, limiting adjustments for prehospital mobility function. Despite these limitations, this study represents an important initial step in validating a simple and efficient clinical tool for identifying in-hospital mobility impairment and recovery and predicting posthospital adverse outcomes.

BAS assessment of incident mobility impairment and recovery in the hospital setting has prognostic value in predicting discharge disposition, in-hospital death, and posthospital mortality risk. That the majority of the effect appears to occur within the first 6 months after discharge suggests that interventions to improve mobility should be started during hospitalization or expeditiously after discharge. Overall, this study’s results showed that a simple and efficient mobility status assessment can become a valuable clinical and administrative tool for targeting and improving mobility in the hospital and after discharge in older adults.

 

Acknowledgments

This work was supported by the National Institutes of Health and the National Center for Advancing Translational Sciences (NIH/NCATS) Clinical and Translational Science Award to the University of Florida (UL1 TR000064) and by the University of Florida’s Claude D. Pepper Center (P30AG028740-R6, significant contributions from the Data and Applied Science Core and Biostatistical Core).

Disclosure

Nothing to report.

Physicians have historically had limited adoption of strategies to improve patient experience and often cite suboptimal data and lack of evidence-driven strategies. ^1,2 However, public reporting of hospital-level physician domain Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) experience scores, and more recent linking of payments to performance on patient experience metrics, have been associated with significant increases in physician domain scores for most of the hospitals. ³ Hospitals and healthcare organizations have deployed a broad range of strategies to engage physicians. These include emphasizing the relationship between patient experience and patient compliance, complaints, and malpractice lawsuits; appealing to physicians’ sense of competitiveness by publishing individual provider experience scores; educating physicians on HCAHPS and providing them with regularly updated data; and development of specific techniques for improving patient-physician interaction. ^4-8

Studies show that educational curricula on improving etiquette and communication skills for physicians lead to improvement in patient experience, and many such training programs are available to hospitals for a significant cost.^9-15 Other studies that have focused on providing timely and individual feedback to physicians using tools other than HCAHPS have shown improvement in experience in some instances. ^16,17 However, these strategies are resource intensive, require the presence of an independent observer in each patient room, and may not be practical in many settings. Further, long-term sustainability may be problematic.

Since the goal of any educational intervention targeting physicians is routinizing best practices, and since resource-intensive strategies of continuous assessment and feedback may not be practical, we sought to test the impact of periodic physician self-reporting of their etiquette-based behavior on their patient experience scores.

METHODS

Subjects

Hospitalists from 4 hospitals (2 community and 2 academic) that are part of the same healthcare system were the study subjects. Hospitalists who had at least 15 unique patients responding to the routinely administered Press Ganey experience survey during the baseline period were considered eligible. Eligible hospitalists were invited to enroll in the study if their site director confirmed that the provider was likely to stay with the group for the subsequent 12-month study period.

Randomization, Intervention and Control Group

Hospitalists were randomized to the study arm or control arm (1:1 randomization). Study arm participants received biweekly etiquette behavior (EB) surveys and were asked to report how frequently they performed 7 best-practice bedside etiquette behaviors during the previous 2-week period (Table 1). These behaviors were pre-defined by a consensus group of investigators as being amenable to self-report and commonly considered best practice as described in detail below. Control-arm participants received similarly worded survey on quality improvement behaviors (QIB) that would not be expected to impact patient experience (such as reviewing medications to ensure that antithrombotic prophylaxis was prescribed, Table 1).

 

Baseline and Study Periods

A 12-month period prior to the enrollment of each hospitalist was considered the baseline period for that individual. Hospitalist eligibility was assessed based on number of unique patients for each hospitalist who responded to the survey during this baseline period. Once enrolled, baseline provider-level patient experience scores were calculated based on the survey responses during this 12-month baseline period. Baseline etiquette behavior performance of the study was calculated from the first survey. After the initial survey, hospitalists received biweekly surveys (EB or QIB) for the 12-month study period for a total of 26 surveys (including the initial survey).

Survey Development, Nature of Survey, Survey Distribution Methods

The EB and QIB physician self-report surveys were developed through an iterative process by the study team. The EB survey included elements from an etiquette-based medicine checklist for hospitalized patients described by Kahn et al. ¹⁸ We conducted a review of literature to identify evidence-based practices.^19-22 Research team members contributed items on best practices in etiquette-based medicine from their experience. Specifically, behaviors were selected if they met the following 4 criteria: 1) performing the behavior did not lead to significant increase in workload and was relatively easy to incorporate in the work flow; 2) occurrence of the behavior would be easy to note for any outside observer or the providers themselves; 3) the practice was considered to be either an evidence-based or consensus-based best-practice; 4) there was consensus among study team members on including the item. The survey was tested for understandability by hospitalists who were not eligible for the study.

The EB survey contained 7 items related to behaviors that were expected to impact patient experience. The QIB survey contained 4 items related to behaviors that were expected to improve quality (Table 1). The initial survey also included questions about demographic characteristics of the participants.

Survey questionnaires were sent via email every 2 weeks for a period of 12 months. The survey questionnaire became available every other week, between Friday morning and Tuesday midnight, during the study period. Hospitalists received daily email reminders on each of these days with a link to the survey website if they did not complete the survey. They had the opportunity to report that they were not on service in the prior week and opt out of the survey for the specific 2-week period. The survey questions were available online as well as on a mobile device format.

Provider Level Patient Experience Scores

Provider-level patient experience scores were calculated from the physician domain Press Ganey survey items, which included the time that the physician spent with patients, the physician addressed questions/worries, the physician kept patients informed, the friendliness/courtesy of physician, and the skill of physician. Press Ganey responses were scored from 1 to 5 based on the Likert scale responses on the survey such that a response “very good” was scored 5 and a response “very poor” was scored 1. Additionally, physician domain HCAHPS item (doctors treat with courtesy/respect, doctors listen carefully, doctors explain in way patients understand) responses were utilized to calculate another set of HCAHPS provider level experience scores. The responses were scored as 1 for “always” response and “0” for any other response, consistent with CMS dichotomization of these results for public reporting. Weighted scores were calculated for individual hospitalists based on the proportion of days each hospitalist billed for the hospitalization so that experience scores of patients who were cared for by multiple providers were assigned to each provider in proportion to the percent of care delivered.²³ Separate composite physician scores were generated from the 5 Press Ganey and for the 3 HCAHPS physician items. Each item was weighted equally, with the maximum possible for Press Ganey composite score of 25 (sum of the maximum possible score of 5 on each of the 5 Press Ganey items) and the HCAHPS possible total was 3 (sum of the maximum possible score of 1 on each of the 3 HCAHPS items).

ANALYSIS AND STATISTICAL METHODS

We analyzed the data to assess for changes in frequency of self-reported behavior over the study period, changes in provider-level patient experience between baseline and study period, and the association between the these 2 outcomes. The self-reported etiquette-based behavior responses were scored as 1 for the lowest response (never) to 4 as the highest (always). With 7 questions, the maximum attainable score was 28. The maximum score was normalized to 100 for ease of interpretation (corresponding to percentage of time etiquette behaviors were employed, by self-report). Similarly, the maximum attainable self-reported QIB-related behavior score on the 4 questions was 16. This was also converted to 0-100 scale for ease of comparison.

 

Two additional sets of analyses were performed to evaluate changes in patient experience during the study period. First, the mean 12-month provider level patient experience composite score in the baseline period was compared with the 12-month composite score during the 12-month study period for the study group and the control group. These were assessed with and without adjusting for age, sex, race, and U.S. medical school graduate (USMG) status. In the second set of unadjusted and adjusted analyses, changes in biweekly composite scores during the study period were compared between the intervention and the control groups while accounting for correlation between observations from the same physician using mixed linear models. Linear mixed models were used to accommodate correlations among multiple observations made on the same physician by including random effects within each regression model. Furthermore, these models allowed us to account for unbalanced design in our data when not all physicians had an equal number of observations and data elements were collected asynchronously.²⁴ Analyses were performed in R version 3.2.2 (The R Project for Statistical Computing, Vienna, Austria); linear mixed models were performed using the ‘nlme’ package.²⁵

We hypothesized that self-reporting on biweekly surveys would result in increases in the frequency of the reported behavior in each arm. We also hypothesized that, because of biweekly reflection and self-reporting on etiquette-based bedside behavior, patient experience scores would increase in the study arm.

RESULTS

Of the 80 hospitalists approached to participate in the study, 64 elected to participate (80% participation rate). The mean response rate to the survey was 57.4% for the intervention arm and 85.7% for the control arm. Higher response rates were not associated with improved patient experience scores. Of the respondents, 43.1% were younger than 35 years of age, 51.5% practiced in academic settings, and 53.1% were female. There was no statistical difference between hospitalists’ baseline composite experience scores based on gender, age, academic hospitalist status, USMG status, and English as a second language status. Similarly, there were no differences in poststudy composite experience scores based on physician characteristics.

Physicians reported high rates of etiquette-based behavior at baseline (mean score, 83.9+/-3.3), and this showed moderate improvement over the study period (5.6 % [3.9%-7.3%, P < 0.0001]). Similarly, there was a moderate increase in frequency of self-reported behavior in the control arm (6.8% [3.5%-10.1%, P < 0.0001]). Hospitalists reported on 80.7% (77.6%-83.4%) of the biweekly surveys that they “almost always” wrapped up by asking, “Do you have any other questions or concerns” or something similar. In contrast, hospitalists reported on only 27.9% (24.7%-31.3%) of the biweekly survey that they “almost always” sat down in the patient room.

The composite physician domain Press Ganey experience scores were no different for the intervention arm and the control arm during the 12-month baseline period (21.8 vs. 21.7; P = 0.90) and the 12-month intervention period (21.6 vs. 21.5; P = 0.75). Baseline self-reported behaviors were not associated with baseline experience scores. Similarly, there were no differences between the arms on composite physician domain HCAHPS experience scores during baseline (2.1 vs. 2.3; P = 0.13) and intervention periods (2.2 vs. 2.1; P = 0.33).

The difference in difference analysis of the baseline and postintervention composite between the intervention arm and the control arm was not statistically significant for Press Ganey composite physician experience scores (-0.163 vs. -0.322; P = 0.71) or HCAHPS composite physician scores (-0.162 vs. -0.071; P = 0.06). The results did not change when controlled for survey response rate (percentage biweekly surveys completed by the hospitalist), age, gender, USMG status, English as a second language status, or percent clinical effort. The difference in difference analysis of the individual Press Ganey and HCAHPS physician domain items that were used to calculate the composite score was also not statistically significant (Table 2).

Changes in self-reported etiquette-based behavior were not associated with any changes in composite Press Ganey and HCAHPS experience score or individual items of the composite experience scores between baseline and intervention period. Similarly, biweekly self-reported etiquette behaviors were not associated with composite and individual item experience scores derived from responses of the patients discharged during the same 2-week reporting period. The intra-class correlation between observations from the same physician was only 0.02%, suggesting that most of the variation in scores was likely due to patient factors and did not result from differences between physicians.

DISCUSSION

This 12-month randomized multicenter study of hospitalists showed that repeated self-reporting of etiquette-based behavior results in modest reported increases in performance of these behaviors. However, there was no associated increase in provider level patient experience scores at the end of the study period when compared to baseline scores of the same physicians or when compared to the scores of the control group. The study demonstrated feasibility of self-reporting of behaviors by physicians with high participation when provided modest incentives.

 

Educational and feedback strategies used to improve patient experience are very resource intensive. Training sessions provided at some hospitals may take hours, and sustained effects are unproved. The presence of an independent observer in patient rooms to generate feedback for providers is not scalable and sustainable outside of a research study environment.^{9-11,15,17,26-29} We attempted to use physician repeated self-reporting to reinforce the important and easy to adopt components of etiquette-based behavior to develop a more easily sustainable strategy. This may have failed for several reasons.

When combining “always” and “usually” responses, the physicians in our study reported a high level of etiquette behavior at baseline. If physicians believe that they are performing well at baseline, they would not consider this to be an area in need of improvement. Bigger changes in behavior may have been possible had the physicians rated themselves less favorably at baseline. Inflated or high baseline self-assessment of performance might also have led to limited success of other types of educational interventions had they been employed.

Studies published since the rollout of our study have shown that physicians significantly overestimate how frequently they perform these etiquette behaviors.^30,31 It is likely that was the case in our study subjects. This may, at best, indicate that a much higher change in the level of self-reported performance would be needed to result in meaningful actual changes, or worse, may render self-reported etiquette behavior entirely unreliable. Interventions designed to improve etiquette-based behavior might need to provide feedback about performance.

A program that provides education on the importance of etiquette-based behaviors, obtains objective measures of performance of these behaviors, and offers individualized feedback may be more likely to increase the desired behaviors. This is a limitation of our study. However, we aimed to test a method that required limited resources. Additionally, our method for attributing HCAHPS scores to an individual physician, based on weighted scores that were calculated according to the proportion of days each hospitalist billed for the hospitalization, may be inaccurate. It is possible that each interaction does not contribute equally to the overall score. A team-based intervention and experience measurements could overcome this limitation.

CONCLUSION

This randomized trial demonstrated the feasibility of self-assessment of bedside etiquette behaviors by hospitalists but failed to demonstrate a meaningful impact on patient experience through self-report. These findings suggest that more intensive interventions, perhaps involving direct observation, peer-to-peer mentoring, or other techniques may be required to impact significantly physician etiquette behaviors.

Disclosure

Johns Hopkins Hospitalist Scholars Program provided funding support. Dr. Qayyum is a consultant for Sunovion. The other authors have nothing to report.

 

Physicians have historically had limited adoption of strategies to improve patient experience and often cite suboptimal data and lack of evidence-driven strategies. ^1,2 However, public reporting of hospital-level physician domain Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) experience scores, and more recent linking of payments to performance on patient experience metrics, have been associated with significant increases in physician domain scores for most of the hospitals. ³ Hospitals and healthcare organizations have deployed a broad range of strategies to engage physicians. These include emphasizing the relationship between patient experience and patient compliance, complaints, and malpractice lawsuits; appealing to physicians’ sense of competitiveness by publishing individual provider experience scores; educating physicians on HCAHPS and providing them with regularly updated data; and development of specific techniques for improving patient-physician interaction. ^4-8

Studies show that educational curricula on improving etiquette and communication skills for physicians lead to improvement in patient experience, and many such training programs are available to hospitals for a significant cost.^9-15 Other studies that have focused on providing timely and individual feedback to physicians using tools other than HCAHPS have shown improvement in experience in some instances. ^16,17 However, these strategies are resource intensive, require the presence of an independent observer in each patient room, and may not be practical in many settings. Further, long-term sustainability may be problematic.

Since the goal of any educational intervention targeting physicians is routinizing best practices, and since resource-intensive strategies of continuous assessment and feedback may not be practical, we sought to test the impact of periodic physician self-reporting of their etiquette-based behavior on their patient experience scores.

METHODS

Subjects

Hospitalists from 4 hospitals (2 community and 2 academic) that are part of the same healthcare system were the study subjects. Hospitalists who had at least 15 unique patients responding to the routinely administered Press Ganey experience survey during the baseline period were considered eligible. Eligible hospitalists were invited to enroll in the study if their site director confirmed that the provider was likely to stay with the group for the subsequent 12-month study period.

Randomization, Intervention and Control Group

Hospitalists were randomized to the study arm or control arm (1:1 randomization). Study arm participants received biweekly etiquette behavior (EB) surveys and were asked to report how frequently they performed 7 best-practice bedside etiquette behaviors during the previous 2-week period (Table 1). These behaviors were pre-defined by a consensus group of investigators as being amenable to self-report and commonly considered best practice as described in detail below. Control-arm participants received similarly worded survey on quality improvement behaviors (QIB) that would not be expected to impact patient experience (such as reviewing medications to ensure that antithrombotic prophylaxis was prescribed, Table 1).

 

Baseline and Study Periods

A 12-month period prior to the enrollment of each hospitalist was considered the baseline period for that individual. Hospitalist eligibility was assessed based on number of unique patients for each hospitalist who responded to the survey during this baseline period. Once enrolled, baseline provider-level patient experience scores were calculated based on the survey responses during this 12-month baseline period. Baseline etiquette behavior performance of the study was calculated from the first survey. After the initial survey, hospitalists received biweekly surveys (EB or QIB) for the 12-month study period for a total of 26 surveys (including the initial survey).

Survey Development, Nature of Survey, Survey Distribution Methods

The EB and QIB physician self-report surveys were developed through an iterative process by the study team. The EB survey included elements from an etiquette-based medicine checklist for hospitalized patients described by Kahn et al. ¹⁸ We conducted a review of literature to identify evidence-based practices.^19-22 Research team members contributed items on best practices in etiquette-based medicine from their experience. Specifically, behaviors were selected if they met the following 4 criteria: 1) performing the behavior did not lead to significant increase in workload and was relatively easy to incorporate in the work flow; 2) occurrence of the behavior would be easy to note for any outside observer or the providers themselves; 3) the practice was considered to be either an evidence-based or consensus-based best-practice; 4) there was consensus among study team members on including the item. The survey was tested for understandability by hospitalists who were not eligible for the study.

The EB survey contained 7 items related to behaviors that were expected to impact patient experience. The QIB survey contained 4 items related to behaviors that were expected to improve quality (Table 1). The initial survey also included questions about demographic characteristics of the participants.

Survey questionnaires were sent via email every 2 weeks for a period of 12 months. The survey questionnaire became available every other week, between Friday morning and Tuesday midnight, during the study period. Hospitalists received daily email reminders on each of these days with a link to the survey website if they did not complete the survey. They had the opportunity to report that they were not on service in the prior week and opt out of the survey for the specific 2-week period. The survey questions were available online as well as on a mobile device format.

Provider Level Patient Experience Scores

Provider-level patient experience scores were calculated from the physician domain Press Ganey survey items, which included the time that the physician spent with patients, the physician addressed questions/worries, the physician kept patients informed, the friendliness/courtesy of physician, and the skill of physician. Press Ganey responses were scored from 1 to 5 based on the Likert scale responses on the survey such that a response “very good” was scored 5 and a response “very poor” was scored 1. Additionally, physician domain HCAHPS item (doctors treat with courtesy/respect, doctors listen carefully, doctors explain in way patients understand) responses were utilized to calculate another set of HCAHPS provider level experience scores. The responses were scored as 1 for “always” response and “0” for any other response, consistent with CMS dichotomization of these results for public reporting. Weighted scores were calculated for individual hospitalists based on the proportion of days each hospitalist billed for the hospitalization so that experience scores of patients who were cared for by multiple providers were assigned to each provider in proportion to the percent of care delivered.²³ Separate composite physician scores were generated from the 5 Press Ganey and for the 3 HCAHPS physician items. Each item was weighted equally, with the maximum possible for Press Ganey composite score of 25 (sum of the maximum possible score of 5 on each of the 5 Press Ganey items) and the HCAHPS possible total was 3 (sum of the maximum possible score of 1 on each of the 3 HCAHPS items).

ANALYSIS AND STATISTICAL METHODS

We analyzed the data to assess for changes in frequency of self-reported behavior over the study period, changes in provider-level patient experience between baseline and study period, and the association between the these 2 outcomes. The self-reported etiquette-based behavior responses were scored as 1 for the lowest response (never) to 4 as the highest (always). With 7 questions, the maximum attainable score was 28. The maximum score was normalized to 100 for ease of interpretation (corresponding to percentage of time etiquette behaviors were employed, by self-report). Similarly, the maximum attainable self-reported QIB-related behavior score on the 4 questions was 16. This was also converted to 0-100 scale for ease of comparison.

 

Two additional sets of analyses were performed to evaluate changes in patient experience during the study period. First, the mean 12-month provider level patient experience composite score in the baseline period was compared with the 12-month composite score during the 12-month study period for the study group and the control group. These were assessed with and without adjusting for age, sex, race, and U.S. medical school graduate (USMG) status. In the second set of unadjusted and adjusted analyses, changes in biweekly composite scores during the study period were compared between the intervention and the control groups while accounting for correlation between observations from the same physician using mixed linear models. Linear mixed models were used to accommodate correlations among multiple observations made on the same physician by including random effects within each regression model. Furthermore, these models allowed us to account for unbalanced design in our data when not all physicians had an equal number of observations and data elements were collected asynchronously.²⁴ Analyses were performed in R version 3.2.2 (The R Project for Statistical Computing, Vienna, Austria); linear mixed models were performed using the ‘nlme’ package.²⁵

We hypothesized that self-reporting on biweekly surveys would result in increases in the frequency of the reported behavior in each arm. We also hypothesized that, because of biweekly reflection and self-reporting on etiquette-based bedside behavior, patient experience scores would increase in the study arm.

RESULTS

Of the 80 hospitalists approached to participate in the study, 64 elected to participate (80% participation rate). The mean response rate to the survey was 57.4% for the intervention arm and 85.7% for the control arm. Higher response rates were not associated with improved patient experience scores. Of the respondents, 43.1% were younger than 35 years of age, 51.5% practiced in academic settings, and 53.1% were female. There was no statistical difference between hospitalists’ baseline composite experience scores based on gender, age, academic hospitalist status, USMG status, and English as a second language status. Similarly, there were no differences in poststudy composite experience scores based on physician characteristics.

Physicians reported high rates of etiquette-based behavior at baseline (mean score, 83.9+/-3.3), and this showed moderate improvement over the study period (5.6 % [3.9%-7.3%, P < 0.0001]). Similarly, there was a moderate increase in frequency of self-reported behavior in the control arm (6.8% [3.5%-10.1%, P < 0.0001]). Hospitalists reported on 80.7% (77.6%-83.4%) of the biweekly surveys that they “almost always” wrapped up by asking, “Do you have any other questions or concerns” or something similar. In contrast, hospitalists reported on only 27.9% (24.7%-31.3%) of the biweekly survey that they “almost always” sat down in the patient room.

The composite physician domain Press Ganey experience scores were no different for the intervention arm and the control arm during the 12-month baseline period (21.8 vs. 21.7; P = 0.90) and the 12-month intervention period (21.6 vs. 21.5; P = 0.75). Baseline self-reported behaviors were not associated with baseline experience scores. Similarly, there were no differences between the arms on composite physician domain HCAHPS experience scores during baseline (2.1 vs. 2.3; P = 0.13) and intervention periods (2.2 vs. 2.1; P = 0.33).

The difference in difference analysis of the baseline and postintervention composite between the intervention arm and the control arm was not statistically significant for Press Ganey composite physician experience scores (-0.163 vs. -0.322; P = 0.71) or HCAHPS composite physician scores (-0.162 vs. -0.071; P = 0.06). The results did not change when controlled for survey response rate (percentage biweekly surveys completed by the hospitalist), age, gender, USMG status, English as a second language status, or percent clinical effort. The difference in difference analysis of the individual Press Ganey and HCAHPS physician domain items that were used to calculate the composite score was also not statistically significant (Table 2).

Changes in self-reported etiquette-based behavior were not associated with any changes in composite Press Ganey and HCAHPS experience score or individual items of the composite experience scores between baseline and intervention period. Similarly, biweekly self-reported etiquette behaviors were not associated with composite and individual item experience scores derived from responses of the patients discharged during the same 2-week reporting period. The intra-class correlation between observations from the same physician was only 0.02%, suggesting that most of the variation in scores was likely due to patient factors and did not result from differences between physicians.

DISCUSSION

This 12-month randomized multicenter study of hospitalists showed that repeated self-reporting of etiquette-based behavior results in modest reported increases in performance of these behaviors. However, there was no associated increase in provider level patient experience scores at the end of the study period when compared to baseline scores of the same physicians or when compared to the scores of the control group. The study demonstrated feasibility of self-reporting of behaviors by physicians with high participation when provided modest incentives.

 

Educational and feedback strategies used to improve patient experience are very resource intensive. Training sessions provided at some hospitals may take hours, and sustained effects are unproved. The presence of an independent observer in patient rooms to generate feedback for providers is not scalable and sustainable outside of a research study environment.^{9-11,15,17,26-29} We attempted to use physician repeated self-reporting to reinforce the important and easy to adopt components of etiquette-based behavior to develop a more easily sustainable strategy. This may have failed for several reasons.

When combining “always” and “usually” responses, the physicians in our study reported a high level of etiquette behavior at baseline. If physicians believe that they are performing well at baseline, they would not consider this to be an area in need of improvement. Bigger changes in behavior may have been possible had the physicians rated themselves less favorably at baseline. Inflated or high baseline self-assessment of performance might also have led to limited success of other types of educational interventions had they been employed.

Studies published since the rollout of our study have shown that physicians significantly overestimate how frequently they perform these etiquette behaviors.^30,31 It is likely that was the case in our study subjects. This may, at best, indicate that a much higher change in the level of self-reported performance would be needed to result in meaningful actual changes, or worse, may render self-reported etiquette behavior entirely unreliable. Interventions designed to improve etiquette-based behavior might need to provide feedback about performance.

A program that provides education on the importance of etiquette-based behaviors, obtains objective measures of performance of these behaviors, and offers individualized feedback may be more likely to increase the desired behaviors. This is a limitation of our study. However, we aimed to test a method that required limited resources. Additionally, our method for attributing HCAHPS scores to an individual physician, based on weighted scores that were calculated according to the proportion of days each hospitalist billed for the hospitalization, may be inaccurate. It is possible that each interaction does not contribute equally to the overall score. A team-based intervention and experience measurements could overcome this limitation.

CONCLUSION

This randomized trial demonstrated the feasibility of self-assessment of bedside etiquette behaviors by hospitalists but failed to demonstrate a meaningful impact on patient experience through self-report. These findings suggest that more intensive interventions, perhaps involving direct observation, peer-to-peer mentoring, or other techniques may be required to impact significantly physician etiquette behaviors.

Disclosure

Johns Hopkins Hospitalist Scholars Program provided funding support. Dr. Qayyum is a consultant for Sunovion. The other authors have nothing to report.

 

Physicians have historically had limited adoption of strategies to improve patient experience and often cite suboptimal data and lack of evidence-driven strategies. ^1,2 However, public reporting of hospital-level physician domain Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) experience scores, and more recent linking of payments to performance on patient experience metrics, have been associated with significant increases in physician domain scores for most of the hospitals. ³ Hospitals and healthcare organizations have deployed a broad range of strategies to engage physicians. These include emphasizing the relationship between patient experience and patient compliance, complaints, and malpractice lawsuits; appealing to physicians’ sense of competitiveness by publishing individual provider experience scores; educating physicians on HCAHPS and providing them with regularly updated data; and development of specific techniques for improving patient-physician interaction. ^4-8

Studies show that educational curricula on improving etiquette and communication skills for physicians lead to improvement in patient experience, and many such training programs are available to hospitals for a significant cost.^9-15 Other studies that have focused on providing timely and individual feedback to physicians using tools other than HCAHPS have shown improvement in experience in some instances. ^16,17 However, these strategies are resource intensive, require the presence of an independent observer in each patient room, and may not be practical in many settings. Further, long-term sustainability may be problematic.

Since the goal of any educational intervention targeting physicians is routinizing best practices, and since resource-intensive strategies of continuous assessment and feedback may not be practical, we sought to test the impact of periodic physician self-reporting of their etiquette-based behavior on their patient experience scores.

METHODS

Subjects

Hospitalists from 4 hospitals (2 community and 2 academic) that are part of the same healthcare system were the study subjects. Hospitalists who had at least 15 unique patients responding to the routinely administered Press Ganey experience survey during the baseline period were considered eligible. Eligible hospitalists were invited to enroll in the study if their site director confirmed that the provider was likely to stay with the group for the subsequent 12-month study period.

Randomization, Intervention and Control Group

Hospitalists were randomized to the study arm or control arm (1:1 randomization). Study arm participants received biweekly etiquette behavior (EB) surveys and were asked to report how frequently they performed 7 best-practice bedside etiquette behaviors during the previous 2-week period (Table 1). These behaviors were pre-defined by a consensus group of investigators as being amenable to self-report and commonly considered best practice as described in detail below. Control-arm participants received similarly worded survey on quality improvement behaviors (QIB) that would not be expected to impact patient experience (such as reviewing medications to ensure that antithrombotic prophylaxis was prescribed, Table 1).

 

Baseline and Study Periods

A 12-month period prior to the enrollment of each hospitalist was considered the baseline period for that individual. Hospitalist eligibility was assessed based on number of unique patients for each hospitalist who responded to the survey during this baseline period. Once enrolled, baseline provider-level patient experience scores were calculated based on the survey responses during this 12-month baseline period. Baseline etiquette behavior performance of the study was calculated from the first survey. After the initial survey, hospitalists received biweekly surveys (EB or QIB) for the 12-month study period for a total of 26 surveys (including the initial survey).

Survey Development, Nature of Survey, Survey Distribution Methods

The EB and QIB physician self-report surveys were developed through an iterative process by the study team. The EB survey included elements from an etiquette-based medicine checklist for hospitalized patients described by Kahn et al. ¹⁸ We conducted a review of literature to identify evidence-based practices.^19-22 Research team members contributed items on best practices in etiquette-based medicine from their experience. Specifically, behaviors were selected if they met the following 4 criteria: 1) performing the behavior did not lead to significant increase in workload and was relatively easy to incorporate in the work flow; 2) occurrence of the behavior would be easy to note for any outside observer or the providers themselves; 3) the practice was considered to be either an evidence-based or consensus-based best-practice; 4) there was consensus among study team members on including the item. The survey was tested for understandability by hospitalists who were not eligible for the study.

The EB survey contained 7 items related to behaviors that were expected to impact patient experience. The QIB survey contained 4 items related to behaviors that were expected to improve quality (Table 1). The initial survey also included questions about demographic characteristics of the participants.

Survey questionnaires were sent via email every 2 weeks for a period of 12 months. The survey questionnaire became available every other week, between Friday morning and Tuesday midnight, during the study period. Hospitalists received daily email reminders on each of these days with a link to the survey website if they did not complete the survey. They had the opportunity to report that they were not on service in the prior week and opt out of the survey for the specific 2-week period. The survey questions were available online as well as on a mobile device format.

Provider Level Patient Experience Scores

Provider-level patient experience scores were calculated from the physician domain Press Ganey survey items, which included the time that the physician spent with patients, the physician addressed questions/worries, the physician kept patients informed, the friendliness/courtesy of physician, and the skill of physician. Press Ganey responses were scored from 1 to 5 based on the Likert scale responses on the survey such that a response “very good” was scored 5 and a response “very poor” was scored 1. Additionally, physician domain HCAHPS item (doctors treat with courtesy/respect, doctors listen carefully, doctors explain in way patients understand) responses were utilized to calculate another set of HCAHPS provider level experience scores. The responses were scored as 1 for “always” response and “0” for any other response, consistent with CMS dichotomization of these results for public reporting. Weighted scores were calculated for individual hospitalists based on the proportion of days each hospitalist billed for the hospitalization so that experience scores of patients who were cared for by multiple providers were assigned to each provider in proportion to the percent of care delivered.²³ Separate composite physician scores were generated from the 5 Press Ganey and for the 3 HCAHPS physician items. Each item was weighted equally, with the maximum possible for Press Ganey composite score of 25 (sum of the maximum possible score of 5 on each of the 5 Press Ganey items) and the HCAHPS possible total was 3 (sum of the maximum possible score of 1 on each of the 3 HCAHPS items).

ANALYSIS AND STATISTICAL METHODS

We analyzed the data to assess for changes in frequency of self-reported behavior over the study period, changes in provider-level patient experience between baseline and study period, and the association between the these 2 outcomes. The self-reported etiquette-based behavior responses were scored as 1 for the lowest response (never) to 4 as the highest (always). With 7 questions, the maximum attainable score was 28. The maximum score was normalized to 100 for ease of interpretation (corresponding to percentage of time etiquette behaviors were employed, by self-report). Similarly, the maximum attainable self-reported QIB-related behavior score on the 4 questions was 16. This was also converted to 0-100 scale for ease of comparison.

 

Two additional sets of analyses were performed to evaluate changes in patient experience during the study period. First, the mean 12-month provider level patient experience composite score in the baseline period was compared with the 12-month composite score during the 12-month study period for the study group and the control group. These were assessed with and without adjusting for age, sex, race, and U.S. medical school graduate (USMG) status. In the second set of unadjusted and adjusted analyses, changes in biweekly composite scores during the study period were compared between the intervention and the control groups while accounting for correlation between observations from the same physician using mixed linear models. Linear mixed models were used to accommodate correlations among multiple observations made on the same physician by including random effects within each regression model. Furthermore, these models allowed us to account for unbalanced design in our data when not all physicians had an equal number of observations and data elements were collected asynchronously.²⁴ Analyses were performed in R version 3.2.2 (The R Project for Statistical Computing, Vienna, Austria); linear mixed models were performed using the ‘nlme’ package.²⁵

We hypothesized that self-reporting on biweekly surveys would result in increases in the frequency of the reported behavior in each arm. We also hypothesized that, because of biweekly reflection and self-reporting on etiquette-based bedside behavior, patient experience scores would increase in the study arm.

RESULTS

Of the 80 hospitalists approached to participate in the study, 64 elected to participate (80% participation rate). The mean response rate to the survey was 57.4% for the intervention arm and 85.7% for the control arm. Higher response rates were not associated with improved patient experience scores. Of the respondents, 43.1% were younger than 35 years of age, 51.5% practiced in academic settings, and 53.1% were female. There was no statistical difference between hospitalists’ baseline composite experience scores based on gender, age, academic hospitalist status, USMG status, and English as a second language status. Similarly, there were no differences in poststudy composite experience scores based on physician characteristics.

Physicians reported high rates of etiquette-based behavior at baseline (mean score, 83.9+/-3.3), and this showed moderate improvement over the study period (5.6 % [3.9%-7.3%, P < 0.0001]). Similarly, there was a moderate increase in frequency of self-reported behavior in the control arm (6.8% [3.5%-10.1%, P < 0.0001]). Hospitalists reported on 80.7% (77.6%-83.4%) of the biweekly surveys that they “almost always” wrapped up by asking, “Do you have any other questions or concerns” or something similar. In contrast, hospitalists reported on only 27.9% (24.7%-31.3%) of the biweekly survey that they “almost always” sat down in the patient room.

The composite physician domain Press Ganey experience scores were no different for the intervention arm and the control arm during the 12-month baseline period (21.8 vs. 21.7; P = 0.90) and the 12-month intervention period (21.6 vs. 21.5; P = 0.75). Baseline self-reported behaviors were not associated with baseline experience scores. Similarly, there were no differences between the arms on composite physician domain HCAHPS experience scores during baseline (2.1 vs. 2.3; P = 0.13) and intervention periods (2.2 vs. 2.1; P = 0.33).

The difference in difference analysis of the baseline and postintervention composite between the intervention arm and the control arm was not statistically significant for Press Ganey composite physician experience scores (-0.163 vs. -0.322; P = 0.71) or HCAHPS composite physician scores (-0.162 vs. -0.071; P = 0.06). The results did not change when controlled for survey response rate (percentage biweekly surveys completed by the hospitalist), age, gender, USMG status, English as a second language status, or percent clinical effort. The difference in difference analysis of the individual Press Ganey and HCAHPS physician domain items that were used to calculate the composite score was also not statistically significant (Table 2).

Changes in self-reported etiquette-based behavior were not associated with any changes in composite Press Ganey and HCAHPS experience score or individual items of the composite experience scores between baseline and intervention period. Similarly, biweekly self-reported etiquette behaviors were not associated with composite and individual item experience scores derived from responses of the patients discharged during the same 2-week reporting period. The intra-class correlation between observations from the same physician was only 0.02%, suggesting that most of the variation in scores was likely due to patient factors and did not result from differences between physicians.

DISCUSSION

This 12-month randomized multicenter study of hospitalists showed that repeated self-reporting of etiquette-based behavior results in modest reported increases in performance of these behaviors. However, there was no associated increase in provider level patient experience scores at the end of the study period when compared to baseline scores of the same physicians or when compared to the scores of the control group. The study demonstrated feasibility of self-reporting of behaviors by physicians with high participation when provided modest incentives.

 

Educational and feedback strategies used to improve patient experience are very resource intensive. Training sessions provided at some hospitals may take hours, and sustained effects are unproved. The presence of an independent observer in patient rooms to generate feedback for providers is not scalable and sustainable outside of a research study environment.^{9-11,15,17,26-29} We attempted to use physician repeated self-reporting to reinforce the important and easy to adopt components of etiquette-based behavior to develop a more easily sustainable strategy. This may have failed for several reasons.

When combining “always” and “usually” responses, the physicians in our study reported a high level of etiquette behavior at baseline. If physicians believe that they are performing well at baseline, they would not consider this to be an area in need of improvement. Bigger changes in behavior may have been possible had the physicians rated themselves less favorably at baseline. Inflated or high baseline self-assessment of performance might also have led to limited success of other types of educational interventions had they been employed.

Studies published since the rollout of our study have shown that physicians significantly overestimate how frequently they perform these etiquette behaviors.^30,31 It is likely that was the case in our study subjects. This may, at best, indicate that a much higher change in the level of self-reported performance would be needed to result in meaningful actual changes, or worse, may render self-reported etiquette behavior entirely unreliable. Interventions designed to improve etiquette-based behavior might need to provide feedback about performance.

A program that provides education on the importance of etiquette-based behaviors, obtains objective measures of performance of these behaviors, and offers individualized feedback may be more likely to increase the desired behaviors. This is a limitation of our study. However, we aimed to test a method that required limited resources. Additionally, our method for attributing HCAHPS scores to an individual physician, based on weighted scores that were calculated according to the proportion of days each hospitalist billed for the hospitalization, may be inaccurate. It is possible that each interaction does not contribute equally to the overall score. A team-based intervention and experience measurements could overcome this limitation.

CONCLUSION

This randomized trial demonstrated the feasibility of self-assessment of bedside etiquette behaviors by hospitalists but failed to demonstrate a meaningful impact on patient experience through self-report. These findings suggest that more intensive interventions, perhaps involving direct observation, peer-to-peer mentoring, or other techniques may be required to impact significantly physician etiquette behaviors.

Disclosure

Johns Hopkins Hospitalist Scholars Program provided funding support. Dr. Qayyum is a consultant for Sunovion. The other authors have nothing to report.

 

Of the major chronic conditions that affect adult patients in the United States, cancer accounts for the highest levels of per capita spending.¹ Cost growth for cancer treatment has been substantial and persistent, from $72 billion in 2004 to $125 billion in 2010, and is projected to increase to $173 billion by 2020.² Thirty-five percent of US direct medical cancer costs are attributable to inpatient hospital stays.³ Policy responses that can provide financially sustainable, high-quality models of care for patients with advanced cancer and other serious illness are urgently sought.^4-7

Patterns and levels of resource utilization in providing healthcare to patients with serious illness reflect not only treatment choices but a complex set of relationships among demographic, clinical, and system factors.^8-10 Patient-level factors previously identified as potentially significant drivers of resource utilization among cancer populations specifically include age,¹¹ sex,¹² primary diagnosis,¹³ and comorbidities.¹¹ Among end-of-life populations, significant associations have been found between cost and ethnicity,¹⁴ socioeconomic status,¹⁵ advance directive status,¹⁶ insurance status,¹⁶ and functional status.¹⁷

Evidence on factors strongly associated with cost of hospital admission for patients with advanced cancer can therefore inform provision and planning of healthcare. For example, when a specific diagnosis or clinical condition is found to be associated with high cost, then improving coordination and provision of care for this patient group may reduce avoidable utilization. Determining associations between sociodemographics and hospital care cost can help in identifying possible disparities in care, such as those that might occur when care differs by race, class, or insurance status.

We conducted the Palliative Care for Cancer (PC4C) study, a prospective multisite cohort study of the palliative care consultation team intervention for hospitalized adults with advanced cancer.^18,19 In our primary analysis, we controlled for receipt of palliative care and analyzed a rich patient-reported dataset to examine associations between hospital care cost, and sociodemographic factors, clinical variables, and prior healthcare utilization. The results provide evidence regarding the factors most associated with the cost of hospital-based cancer care.

METHODS

Design, Setting, Participants, Data Sources

The PC4C study has been described in detail by authors who estimated the impact of specialist palliative care consultation teams on hospitalization cost.^19-21 We prospectively collected sociodemographic, clinical, prior utilization, and cost data for adult patients with a primary diagnosis of advanced cancer admitted to 4 large US hospitals between 2007 and 2011.

 

All 4 of these high-volume tertiary-care medical centers were selected for their high patient volume (to facilitate sample size) and research capacity (to facilitate proficient recruitment and data collection). Before the study was initiated, it was approved by the institutional review board of each facility. In addition, approval was sought from each attending physician at each hospital site; patients whose physician did not grant approval were not considered for enrollment. More than 95% of physicians gave their approval.

Patients were at least 18 years old and had a primary diagnosis of metastatic solid tumor; central nervous system malignancy; locally advanced head, neck, or pancreas cancer; metastatic melanoma; or transplant-ineligible lymphoma or multiple myeloma. Patients were excluded if they did not speak English, had a diagnosis of dementia, were unresponsive or nonverbal, had been admitted for routine chemotherapy, died or were discharged within 48 hours of admission, or had had a previous palliative care consultation.

Eligible patients were identified through daily review of admissions records and administrative databases. For each potential study patient identified, that patient’s bedside nurse inquired about willingness to participate in the study. Then, for each willing patient, a trained clinical interviewer approached to explain the study and obtain informed consent. With the patient’s consent, family members were also approached and enrolled with written informed consent.

Quantitative Variables

Independent variables. In the dataset, we identified 17 patient-level variables we hypothesized could be significantly associated with hospitalization cost. These variables covered 4 domains:

• Demographics: age, sex, race.
• Socioeconomics/systems: education level, insurance status, presence of advance directive (living will or healthcare proxy).
• Clinical care: primary cancer diagnosis, admitting diagnosis, comorbidities (Elixhauser index²²), symptom burden and severity (Condensed Memorial Symptom Assessment Scale [CMSAS]²³), and activities of daily living²⁴ or presence of a hospital-acquired condition or complication.²⁵
• Prior utilization: visiting homecare nurse and home health aide within 2 weeks before admission, and analgesic use in morphine sulfate equivalents within week before admission.

Data were collected through a combination of medical record review (age, sex, diagnoses, comorbidities, complications), patient interview (race, education, advance directive, CMSAS, activities of daily living, prior utilization), and hospital administrative databases (insurance). For use in regression, variables were divided into categories when appropriate. Table 1 lists these predictors and their prevalence in the analytic sample.

Dependent variable. The outcome of interest in this analysis was total direct cost of hospital stay. Direct costs are those attributable to the care of a specific patient, as distinct from indirect costs, the shared overhead costs of running a hospital.²⁶ Cost data were extracted from hospital accounting databases and therefore reflect actual costs, the US dollar cost to the hospitals of care provided, also known as direct measurement.²⁷ Costs were standardized for geographical region using the Medicare Wage Index²⁸ and year using the Consumer Price Index²⁹ and are presented here in US dollars for 2011, the final year of data collection.

Statistical Methods

Primary analyses. We regressed total direct hospital costs against all predictors listed in Table 1. To control for receipt of palliative care, we used additional independent variables—a fixed-effects variable for each of 3 hospitals (the fourth hospital was used as the reference case) and a binary treatment variable (whether or not the patient was seen by a palliative care consultation team within 2 days of hospital admission).^19,20

Associations between cost and patient-level covariates were derived with use of a generalized linear model with a γ distribution and a log link,³⁰ selected after comparative evaluation of performance for multiple linear and nonlinear modeling options.³¹

For each patient-level covariate, we estimated average marginal effects. For continuous variables, we estimated the marginal increase in cost associated with a 1-unit increase in the variable. For binary variables, we estimated the average incremental effect, the increase in cost associated with a move from the reference group, holding all other covariates to their original values. All analyses were performed with Stata Version 12.³²

Secondary analyses. Primary analyses showed that number of patient comorbidities (Elixhauser index) was strongly associated with complications and comorbidity count. Prior analyses with these data have shown that palliative care had a larger cost-saving effect for patients with a larger number of comorbidities.²⁰ Additional analyses were therefore performed to examine associations between complications, utilization, and palliative care. First, we cross-tabulated the sample by complications status (none; minor or major) and receipt of timely palliative care, and we present their summary utilization data. Second, we estimated the effect for each complications stratum (none; minor or major) of receiving timely palliative care on cost. These estimates are calculated consistent with prior work with these data: We used propensity scores to balance patients who received the treatment (palliative care) with patients who did not (usual care only),^33,34 and we used a generalized linear model with a γ distribution and a log link to regress the direct hospital care cost on the binary treatment variable and all predictors listed in Table 1.^19-21

 

RESULTS

Participants

We have previously detailed that in our study there were 1023 patients eligible for cost analysis,¹⁹ of whom three were missing data in a field in Table 1 and excluded from this paper. The final analytic sample (N = 1020) is presented according to baseline covariates in Table 1 and according to summary utilization measures in Table 2.

Main Results

The results of the primary analysis, estimating the association between patient-level factors and cost of hospitalization, are presented in Table 3.

These results show the evidence of an association with cost is strongest for 3 clinical factors: a major complication (+$8267; 95% confidence interval [CI], $4509-$12,025), a minor but not a major complication (+$5289; CI, $3480-$7097), and number of comorbidities (+$852; CI, $550-$1153). In addition, there is evidence of associations between lower cost and admitting diagnosis of electrolyte disorders (–$4759; CI, –$7928 to –$1590) and older age (–$53; CI, –$99 to –$6). There is no significant association between primary diagnosis, symptom burden or other clinical factors, sociodemographic factors or healthcare utilization prior to admission and direct hospitalization costs.

Results of the secondary analyses of associations between complications, utilization, and palliative care are listed in Table 4. Patients are stratified by complication (none; major | minor) and their direct cost of hospital care and hospital length of stay (LOS) presented by treatment group (palliative care; usual care only). The data show that within each strata patients who received palliative care had lower costs and LOS than those who received usual care only. Estimated effects of palliative care on utilization is found to be statistically significant in all four quadrants, with a larger cost-effect in the complications stratum than the non-complications stratum.

Sensitivity Analysis

Fifty-one patients died during admission. After removing these cases, because of concerns about possible unobserved heterogeneity,³⁵ we checked our primary (Table 3) and secondary (Table 4) results. Patients discharged alive had results substantively similar to those of the entire sample.

DISCUSSION

Results from our primary analysis (Table 3) suggest that complications and number of comorbidities are the key drivers of hospitalization cost for adults with advanced cancer. Hospitalization for electrolyte disorders and age are both negatively associated with cost.

The association found between higher cost and hospital-acquired complications (HACs) is consistent with other studies’ finding that HACs often result in higher cost, longer LOS, and increased inhospital mortality.³⁶ Since those studies were reported, policy attention has been increasingly focused on HACs.³⁷ Our findings are notable in that, though prior evidence has also suggested high hospital cost is multifactorial, driven by a diversity of demographic, socioeconomic, and clinical factors, this rich patient-reported dataset suggests that, compared with other variables, HACs are emphatically the largest driver of cost. Moreover, cancer patients typically are a vulnerable population, more prone to complications and thus also to potentially avoidable treatments and higher cost. Our prior work suggested earlier palliative care consultation can reduce cost, in part by shortening LOS and reducing the opportunity for HACs to develop^19,20; our secondary analysis (Table 4) suggested a palliative care team’s involvement in HAC treatment can significantly reduce cost of care as well. These associations possibly derive from changed treatment choices and shorter LOS. Further work is needed to better elucidate the role of palliative care in the prevention of HACs in seriously ill patients.

That the number of comorbidities was found to be a key driver of hospitalization cost is consistent with recent findings that high spending on seriously ill patients is associated with having multiple chronic conditions rather than any specific primary diagnosis.^38,39 It is important to note that, unlike impending complications, serious chronic conditions generally are known at admission and can be addressed prospectively through provision and policy. A prior analysis with these data found that palliative care consultation was more cost-effective for patients with a larger number of comorbidities.²⁰ Our 2 studies together suggest that, notwithstanding the preferable alternative of avoiding hospitalization entirely, palliative care and other skilled coordination of care services ought to be prioritized for inpatients with multiple serious illnesses and the highest medical complexity. This patient group has both the highest costs and the greatest amenability to skilled transdisciplinary intervention, possibly because multiple chronic conditions affect patients interactively, complicating identification of appropriate polypharmacy responses and prioritization of treatments.

Our findings also may help direct appropriate use of palliative care services. The recently published American Society of Clinical Oncology palliative care guidelines note that all patients with advanced cancer (eg, those enrolled in our study) should receive dedicated palliative care services, early in the disease course, concurrent with active treatment.⁴⁰ Workforce estimates suggest that the current and future numbers of palliative care practitioners will be unable to meet the ASCO recommendations alone never mind patients with other serious illnesses (eg, advanced heart failure, COPD, CKD).⁴¹ As such, specialized palliative care services will need to be targeted to the patient populations that can benefit most from these services. Whereas cost should not be the principle driver specialized palliative care provision, it will likely be an important component due to both the necessity of allocating scarce resources in the most effective way and the evidence that in care of the seriously-ill lower costs are often a proxy for improved patient experience.

These findings also have implications for research: Different conditions and presumably different combinations of conditions have very different implications for hospital care costs for a cohort of adults with advanced cancer. Given the increasing number of co-occurring conditions among seriously ill patients, and the increasing costs of cancer care and of treating multimorbidity cases, it is essential to further our understanding of the relationship between comorbidities and costs in order to plan and finance care for advanced cancer patients.

 

Limitations and Generalizability

In this observational study, reported associations may be attributable to unobserved confounding that our analyses failed to control.

Our results reflect associations in a prospective multisite study of advanced cancer patients hospitalized in the United States. It is not clear how generalizable our findings are to patients without cancer, to patients in nonhospital settings, and to patients in other health systems and countries. Analyzing cost from the hospital perspective does not take into account that the most impactful way to reduce cost is to avoid hospitalization entirely.

Results of our secondary analysis will not necessarily be robust to patient groups, as specific weights likely will vary by sample. The idea that costs vary by condition, however, is important nevertheless. Elixhauser total was derived with use of the enhanced ICD-9-CM (International Classification of Diseases, Ninth Revision, Clinical Modification) algorithm from Quan et al.⁴² and does not include subsequent Elixhauser Comorbidity Software updates recommended by the Healthcare Cost and Utilization Project (HCUP; Agency for Healthcare Research and Quality).⁴³ The Elixhauser index is recommended over Charlson and other comorbidity indices by both HCUP⁴⁵ and a recent systematic review.⁴⁴

One possible unobserved factor is prior chemotherapy, which is associated with increased hospitalization risk. Related factors that are somewhat controlled for in the study include cancer stage (advanced cancer was an eligibility criterion) and receipt of analgesics within the week before admission (patients admitted for routine chemotherapy were excluded from analyses at the outset).

CONCLUSION

Other studies have identified a wide range of sociodemographic, clinical, and health system factors associated with healthcare utilization. Our results suggest that, for cost of hospital admission among adults with advanced cancer, the most important drivers of utilization are complications and comorbidities. Hospital costs for patients with advanced cancer constitute a major part of US healthcare spending, and these results suggest the need to prioritize high-quality, cost-effective care for patients with multiple serious illnesses.

Acknowledgments

The authors thank Robert Arnold, Phil Santa Emma, Mary Beth Happ, Tim Smith, and David Weissman for contributing to the Palliative Care for Cancer (PC4C) project.

Disclosure

The study was funded by grant R01 CA116227 from the National Cancer Institute and the National Institute of Nursing Research. The study sponsors had no role in design or conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the US Department of Veterans Affairs or the US government. All authors are independent of the study sponsors. Dr. May was supported by a HRB/NCI Health Economics Fellowship during this work. Dr. Garrido is supported by a Veterans Affairs HSR&D career development award (CDA 11-201/CDP 12-255). Dr. Kelley’s time was funded by the National Institute on Aging (1K23AG040774-01A1) and the American Federation for Aging. Dr. Smith is funded by the NCI Core Grant P 30 006973, 1-R01 CA177562-01A1, 1-R01 NR014050 01, and the Harry J. Duffey Family Endowment for Palliative Care. Dr. Morrison was the recipient of a Midcareer Investigator Award in Patient-Oriented Research (5K24AG022345) during the course of this work. This work was supported by the NIA, Claude D. Pepper Older Americans Independence Center at the Icahn School of Medicine at Mount Sinai [5P30AG028741], and the National Palliative Care Research Center.

Of the major chronic conditions that affect adult patients in the United States, cancer accounts for the highest levels of per capita spending.¹ Cost growth for cancer treatment has been substantial and persistent, from $72 billion in 2004 to $125 billion in 2010, and is projected to increase to $173 billion by 2020.² Thirty-five percent of US direct medical cancer costs are attributable to inpatient hospital stays.³ Policy responses that can provide financially sustainable, high-quality models of care for patients with advanced cancer and other serious illness are urgently sought.^4-7

Patterns and levels of resource utilization in providing healthcare to patients with serious illness reflect not only treatment choices but a complex set of relationships among demographic, clinical, and system factors.^8-10 Patient-level factors previously identified as potentially significant drivers of resource utilization among cancer populations specifically include age,¹¹ sex,¹² primary diagnosis,¹³ and comorbidities.¹¹ Among end-of-life populations, significant associations have been found between cost and ethnicity,¹⁴ socioeconomic status,¹⁵ advance directive status,¹⁶ insurance status,¹⁶ and functional status.¹⁷

Evidence on factors strongly associated with cost of hospital admission for patients with advanced cancer can therefore inform provision and planning of healthcare. For example, when a specific diagnosis or clinical condition is found to be associated with high cost, then improving coordination and provision of care for this patient group may reduce avoidable utilization. Determining associations between sociodemographics and hospital care cost can help in identifying possible disparities in care, such as those that might occur when care differs by race, class, or insurance status.

We conducted the Palliative Care for Cancer (PC4C) study, a prospective multisite cohort study of the palliative care consultation team intervention for hospitalized adults with advanced cancer.^18,19 In our primary analysis, we controlled for receipt of palliative care and analyzed a rich patient-reported dataset to examine associations between hospital care cost, and sociodemographic factors, clinical variables, and prior healthcare utilization. The results provide evidence regarding the factors most associated with the cost of hospital-based cancer care.

METHODS

Design, Setting, Participants, Data Sources

The PC4C study has been described in detail by authors who estimated the impact of specialist palliative care consultation teams on hospitalization cost.^19-21 We prospectively collected sociodemographic, clinical, prior utilization, and cost data for adult patients with a primary diagnosis of advanced cancer admitted to 4 large US hospitals between 2007 and 2011.

 

All 4 of these high-volume tertiary-care medical centers were selected for their high patient volume (to facilitate sample size) and research capacity (to facilitate proficient recruitment and data collection). Before the study was initiated, it was approved by the institutional review board of each facility. In addition, approval was sought from each attending physician at each hospital site; patients whose physician did not grant approval were not considered for enrollment. More than 95% of physicians gave their approval.

Patients were at least 18 years old and had a primary diagnosis of metastatic solid tumor; central nervous system malignancy; locally advanced head, neck, or pancreas cancer; metastatic melanoma; or transplant-ineligible lymphoma or multiple myeloma. Patients were excluded if they did not speak English, had a diagnosis of dementia, were unresponsive or nonverbal, had been admitted for routine chemotherapy, died or were discharged within 48 hours of admission, or had had a previous palliative care consultation.

Eligible patients were identified through daily review of admissions records and administrative databases. For each potential study patient identified, that patient’s bedside nurse inquired about willingness to participate in the study. Then, for each willing patient, a trained clinical interviewer approached to explain the study and obtain informed consent. With the patient’s consent, family members were also approached and enrolled with written informed consent.

Quantitative Variables

Independent variables. In the dataset, we identified 17 patient-level variables we hypothesized could be significantly associated with hospitalization cost. These variables covered 4 domains:

• Demographics: age, sex, race.
• Socioeconomics/systems: education level, insurance status, presence of advance directive (living will or healthcare proxy).
• Clinical care: primary cancer diagnosis, admitting diagnosis, comorbidities (Elixhauser index²²), symptom burden and severity (Condensed Memorial Symptom Assessment Scale [CMSAS]²³), and activities of daily living²⁴ or presence of a hospital-acquired condition or complication.²⁵
• Prior utilization: visiting homecare nurse and home health aide within 2 weeks before admission, and analgesic use in morphine sulfate equivalents within week before admission.

Data were collected through a combination of medical record review (age, sex, diagnoses, comorbidities, complications), patient interview (race, education, advance directive, CMSAS, activities of daily living, prior utilization), and hospital administrative databases (insurance). For use in regression, variables were divided into categories when appropriate. Table 1 lists these predictors and their prevalence in the analytic sample.

Dependent variable. The outcome of interest in this analysis was total direct cost of hospital stay. Direct costs are those attributable to the care of a specific patient, as distinct from indirect costs, the shared overhead costs of running a hospital.²⁶ Cost data were extracted from hospital accounting databases and therefore reflect actual costs, the US dollar cost to the hospitals of care provided, also known as direct measurement.²⁷ Costs were standardized for geographical region using the Medicare Wage Index²⁸ and year using the Consumer Price Index²⁹ and are presented here in US dollars for 2011, the final year of data collection.

Statistical Methods

Primary analyses. We regressed total direct hospital costs against all predictors listed in Table 1. To control for receipt of palliative care, we used additional independent variables—a fixed-effects variable for each of 3 hospitals (the fourth hospital was used as the reference case) and a binary treatment variable (whether or not the patient was seen by a palliative care consultation team within 2 days of hospital admission).^19,20

Associations between cost and patient-level covariates were derived with use of a generalized linear model with a γ distribution and a log link,³⁰ selected after comparative evaluation of performance for multiple linear and nonlinear modeling options.³¹

For each patient-level covariate, we estimated average marginal effects. For continuous variables, we estimated the marginal increase in cost associated with a 1-unit increase in the variable. For binary variables, we estimated the average incremental effect, the increase in cost associated with a move from the reference group, holding all other covariates to their original values. All analyses were performed with Stata Version 12.³²

Secondary analyses. Primary analyses showed that number of patient comorbidities (Elixhauser index) was strongly associated with complications and comorbidity count. Prior analyses with these data have shown that palliative care had a larger cost-saving effect for patients with a larger number of comorbidities.²⁰ Additional analyses were therefore performed to examine associations between complications, utilization, and palliative care. First, we cross-tabulated the sample by complications status (none; minor or major) and receipt of timely palliative care, and we present their summary utilization data. Second, we estimated the effect for each complications stratum (none; minor or major) of receiving timely palliative care on cost. These estimates are calculated consistent with prior work with these data: We used propensity scores to balance patients who received the treatment (palliative care) with patients who did not (usual care only),^33,34 and we used a generalized linear model with a γ distribution and a log link to regress the direct hospital care cost on the binary treatment variable and all predictors listed in Table 1.^19-21

 

RESULTS

Participants

We have previously detailed that in our study there were 1023 patients eligible for cost analysis,¹⁹ of whom three were missing data in a field in Table 1 and excluded from this paper. The final analytic sample (N = 1020) is presented according to baseline covariates in Table 1 and according to summary utilization measures in Table 2.

Main Results

The results of the primary analysis, estimating the association between patient-level factors and cost of hospitalization, are presented in Table 3.

These results show the evidence of an association with cost is strongest for 3 clinical factors: a major complication (+$8267; 95% confidence interval [CI], $4509-$12,025), a minor but not a major complication (+$5289; CI, $3480-$7097), and number of comorbidities (+$852; CI, $550-$1153). In addition, there is evidence of associations between lower cost and admitting diagnosis of electrolyte disorders (–$4759; CI, –$7928 to –$1590) and older age (–$53; CI, –$99 to –$6). There is no significant association between primary diagnosis, symptom burden or other clinical factors, sociodemographic factors or healthcare utilization prior to admission and direct hospitalization costs.

Results of the secondary analyses of associations between complications, utilization, and palliative care are listed in Table 4. Patients are stratified by complication (none; major | minor) and their direct cost of hospital care and hospital length of stay (LOS) presented by treatment group (palliative care; usual care only). The data show that within each strata patients who received palliative care had lower costs and LOS than those who received usual care only. Estimated effects of palliative care on utilization is found to be statistically significant in all four quadrants, with a larger cost-effect in the complications stratum than the non-complications stratum.

Sensitivity Analysis

Fifty-one patients died during admission. After removing these cases, because of concerns about possible unobserved heterogeneity,³⁵ we checked our primary (Table 3) and secondary (Table 4) results. Patients discharged alive had results substantively similar to those of the entire sample.

DISCUSSION

Results from our primary analysis (Table 3) suggest that complications and number of comorbidities are the key drivers of hospitalization cost for adults with advanced cancer. Hospitalization for electrolyte disorders and age are both negatively associated with cost.

The association found between higher cost and hospital-acquired complications (HACs) is consistent with other studies’ finding that HACs often result in higher cost, longer LOS, and increased inhospital mortality.³⁶ Since those studies were reported, policy attention has been increasingly focused on HACs.³⁷ Our findings are notable in that, though prior evidence has also suggested high hospital cost is multifactorial, driven by a diversity of demographic, socioeconomic, and clinical factors, this rich patient-reported dataset suggests that, compared with other variables, HACs are emphatically the largest driver of cost. Moreover, cancer patients typically are a vulnerable population, more prone to complications and thus also to potentially avoidable treatments and higher cost. Our prior work suggested earlier palliative care consultation can reduce cost, in part by shortening LOS and reducing the opportunity for HACs to develop^19,20; our secondary analysis (Table 4) suggested a palliative care team’s involvement in HAC treatment can significantly reduce cost of care as well. These associations possibly derive from changed treatment choices and shorter LOS. Further work is needed to better elucidate the role of palliative care in the prevention of HACs in seriously ill patients.

That the number of comorbidities was found to be a key driver of hospitalization cost is consistent with recent findings that high spending on seriously ill patients is associated with having multiple chronic conditions rather than any specific primary diagnosis.^38,39 It is important to note that, unlike impending complications, serious chronic conditions generally are known at admission and can be addressed prospectively through provision and policy. A prior analysis with these data found that palliative care consultation was more cost-effective for patients with a larger number of comorbidities.²⁰ Our 2 studies together suggest that, notwithstanding the preferable alternative of avoiding hospitalization entirely, palliative care and other skilled coordination of care services ought to be prioritized for inpatients with multiple serious illnesses and the highest medical complexity. This patient group has both the highest costs and the greatest amenability to skilled transdisciplinary intervention, possibly because multiple chronic conditions affect patients interactively, complicating identification of appropriate polypharmacy responses and prioritization of treatments.

Our findings also may help direct appropriate use of palliative care services. The recently published American Society of Clinical Oncology palliative care guidelines note that all patients with advanced cancer (eg, those enrolled in our study) should receive dedicated palliative care services, early in the disease course, concurrent with active treatment.⁴⁰ Workforce estimates suggest that the current and future numbers of palliative care practitioners will be unable to meet the ASCO recommendations alone never mind patients with other serious illnesses (eg, advanced heart failure, COPD, CKD).⁴¹ As such, specialized palliative care services will need to be targeted to the patient populations that can benefit most from these services. Whereas cost should not be the principle driver specialized palliative care provision, it will likely be an important component due to both the necessity of allocating scarce resources in the most effective way and the evidence that in care of the seriously-ill lower costs are often a proxy for improved patient experience.

These findings also have implications for research: Different conditions and presumably different combinations of conditions have very different implications for hospital care costs for a cohort of adults with advanced cancer. Given the increasing number of co-occurring conditions among seriously ill patients, and the increasing costs of cancer care and of treating multimorbidity cases, it is essential to further our understanding of the relationship between comorbidities and costs in order to plan and finance care for advanced cancer patients.

 

Limitations and Generalizability

In this observational study, reported associations may be attributable to unobserved confounding that our analyses failed to control.

Our results reflect associations in a prospective multisite study of advanced cancer patients hospitalized in the United States. It is not clear how generalizable our findings are to patients without cancer, to patients in nonhospital settings, and to patients in other health systems and countries. Analyzing cost from the hospital perspective does not take into account that the most impactful way to reduce cost is to avoid hospitalization entirely.

Results of our secondary analysis will not necessarily be robust to patient groups, as specific weights likely will vary by sample. The idea that costs vary by condition, however, is important nevertheless. Elixhauser total was derived with use of the enhanced ICD-9-CM (International Classification of Diseases, Ninth Revision, Clinical Modification) algorithm from Quan et al.⁴² and does not include subsequent Elixhauser Comorbidity Software updates recommended by the Healthcare Cost and Utilization Project (HCUP; Agency for Healthcare Research and Quality).⁴³ The Elixhauser index is recommended over Charlson and other comorbidity indices by both HCUP⁴⁵ and a recent systematic review.⁴⁴

One possible unobserved factor is prior chemotherapy, which is associated with increased hospitalization risk. Related factors that are somewhat controlled for in the study include cancer stage (advanced cancer was an eligibility criterion) and receipt of analgesics within the week before admission (patients admitted for routine chemotherapy were excluded from analyses at the outset).

CONCLUSION

Other studies have identified a wide range of sociodemographic, clinical, and health system factors associated with healthcare utilization. Our results suggest that, for cost of hospital admission among adults with advanced cancer, the most important drivers of utilization are complications and comorbidities. Hospital costs for patients with advanced cancer constitute a major part of US healthcare spending, and these results suggest the need to prioritize high-quality, cost-effective care for patients with multiple serious illnesses.

Acknowledgments

The authors thank Robert Arnold, Phil Santa Emma, Mary Beth Happ, Tim Smith, and David Weissman for contributing to the Palliative Care for Cancer (PC4C) project.

Disclosure

The study was funded by grant R01 CA116227 from the National Cancer Institute and the National Institute of Nursing Research. The study sponsors had no role in design or conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the US Department of Veterans Affairs or the US government. All authors are independent of the study sponsors. Dr. May was supported by a HRB/NCI Health Economics Fellowship during this work. Dr. Garrido is supported by a Veterans Affairs HSR&D career development award (CDA 11-201/CDP 12-255). Dr. Kelley’s time was funded by the National Institute on Aging (1K23AG040774-01A1) and the American Federation for Aging. Dr. Smith is funded by the NCI Core Grant P 30 006973, 1-R01 CA177562-01A1, 1-R01 NR014050 01, and the Harry J. Duffey Family Endowment for Palliative Care. Dr. Morrison was the recipient of a Midcareer Investigator Award in Patient-Oriented Research (5K24AG022345) during the course of this work. This work was supported by the NIA, Claude D. Pepper Older Americans Independence Center at the Icahn School of Medicine at Mount Sinai [5P30AG028741], and the National Palliative Care Research Center.

Of the major chronic conditions that affect adult patients in the United States, cancer accounts for the highest levels of per capita spending.¹ Cost growth for cancer treatment has been substantial and persistent, from $72 billion in 2004 to $125 billion in 2010, and is projected to increase to $173 billion by 2020.² Thirty-five percent of US direct medical cancer costs are attributable to inpatient hospital stays.³ Policy responses that can provide financially sustainable, high-quality models of care for patients with advanced cancer and other serious illness are urgently sought.^4-7

Patterns and levels of resource utilization in providing healthcare to patients with serious illness reflect not only treatment choices but a complex set of relationships among demographic, clinical, and system factors.^8-10 Patient-level factors previously identified as potentially significant drivers of resource utilization among cancer populations specifically include age,¹¹ sex,¹² primary diagnosis,¹³ and comorbidities.¹¹ Among end-of-life populations, significant associations have been found between cost and ethnicity,¹⁴ socioeconomic status,¹⁵ advance directive status,¹⁶ insurance status,¹⁶ and functional status.¹⁷

Evidence on factors strongly associated with cost of hospital admission for patients with advanced cancer can therefore inform provision and planning of healthcare. For example, when a specific diagnosis or clinical condition is found to be associated with high cost, then improving coordination and provision of care for this patient group may reduce avoidable utilization. Determining associations between sociodemographics and hospital care cost can help in identifying possible disparities in care, such as those that might occur when care differs by race, class, or insurance status.

We conducted the Palliative Care for Cancer (PC4C) study, a prospective multisite cohort study of the palliative care consultation team intervention for hospitalized adults with advanced cancer.^18,19 In our primary analysis, we controlled for receipt of palliative care and analyzed a rich patient-reported dataset to examine associations between hospital care cost, and sociodemographic factors, clinical variables, and prior healthcare utilization. The results provide evidence regarding the factors most associated with the cost of hospital-based cancer care.

METHODS

Design, Setting, Participants, Data Sources

The PC4C study has been described in detail by authors who estimated the impact of specialist palliative care consultation teams on hospitalization cost.^19-21 We prospectively collected sociodemographic, clinical, prior utilization, and cost data for adult patients with a primary diagnosis of advanced cancer admitted to 4 large US hospitals between 2007 and 2011.

 

All 4 of these high-volume tertiary-care medical centers were selected for their high patient volume (to facilitate sample size) and research capacity (to facilitate proficient recruitment and data collection). Before the study was initiated, it was approved by the institutional review board of each facility. In addition, approval was sought from each attending physician at each hospital site; patients whose physician did not grant approval were not considered for enrollment. More than 95% of physicians gave their approval.

Patients were at least 18 years old and had a primary diagnosis of metastatic solid tumor; central nervous system malignancy; locally advanced head, neck, or pancreas cancer; metastatic melanoma; or transplant-ineligible lymphoma or multiple myeloma. Patients were excluded if they did not speak English, had a diagnosis of dementia, were unresponsive or nonverbal, had been admitted for routine chemotherapy, died or were discharged within 48 hours of admission, or had had a previous palliative care consultation.

Eligible patients were identified through daily review of admissions records and administrative databases. For each potential study patient identified, that patient’s bedside nurse inquired about willingness to participate in the study. Then, for each willing patient, a trained clinical interviewer approached to explain the study and obtain informed consent. With the patient’s consent, family members were also approached and enrolled with written informed consent.

Quantitative Variables

Independent variables. In the dataset, we identified 17 patient-level variables we hypothesized could be significantly associated with hospitalization cost. These variables covered 4 domains:

• Demographics: age, sex, race.
• Socioeconomics/systems: education level, insurance status, presence of advance directive (living will or healthcare proxy).
• Clinical care: primary cancer diagnosis, admitting diagnosis, comorbidities (Elixhauser index²²), symptom burden and severity (Condensed Memorial Symptom Assessment Scale [CMSAS]²³), and activities of daily living²⁴ or presence of a hospital-acquired condition or complication.²⁵
• Prior utilization: visiting homecare nurse and home health aide within 2 weeks before admission, and analgesic use in morphine sulfate equivalents within week before admission.

Data were collected through a combination of medical record review (age, sex, diagnoses, comorbidities, complications), patient interview (race, education, advance directive, CMSAS, activities of daily living, prior utilization), and hospital administrative databases (insurance). For use in regression, variables were divided into categories when appropriate. Table 1 lists these predictors and their prevalence in the analytic sample.

Dependent variable. The outcome of interest in this analysis was total direct cost of hospital stay. Direct costs are those attributable to the care of a specific patient, as distinct from indirect costs, the shared overhead costs of running a hospital.²⁶ Cost data were extracted from hospital accounting databases and therefore reflect actual costs, the US dollar cost to the hospitals of care provided, also known as direct measurement.²⁷ Costs were standardized for geographical region using the Medicare Wage Index²⁸ and year using the Consumer Price Index²⁹ and are presented here in US dollars for 2011, the final year of data collection.

Statistical Methods

Primary analyses. We regressed total direct hospital costs against all predictors listed in Table 1. To control for receipt of palliative care, we used additional independent variables—a fixed-effects variable for each of 3 hospitals (the fourth hospital was used as the reference case) and a binary treatment variable (whether or not the patient was seen by a palliative care consultation team within 2 days of hospital admission).^19,20

Associations between cost and patient-level covariates were derived with use of a generalized linear model with a γ distribution and a log link,³⁰ selected after comparative evaluation of performance for multiple linear and nonlinear modeling options.³¹

For each patient-level covariate, we estimated average marginal effects. For continuous variables, we estimated the marginal increase in cost associated with a 1-unit increase in the variable. For binary variables, we estimated the average incremental effect, the increase in cost associated with a move from the reference group, holding all other covariates to their original values. All analyses were performed with Stata Version 12.³²

Secondary analyses. Primary analyses showed that number of patient comorbidities (Elixhauser index) was strongly associated with complications and comorbidity count. Prior analyses with these data have shown that palliative care had a larger cost-saving effect for patients with a larger number of comorbidities.²⁰ Additional analyses were therefore performed to examine associations between complications, utilization, and palliative care. First, we cross-tabulated the sample by complications status (none; minor or major) and receipt of timely palliative care, and we present their summary utilization data. Second, we estimated the effect for each complications stratum (none; minor or major) of receiving timely palliative care on cost. These estimates are calculated consistent with prior work with these data: We used propensity scores to balance patients who received the treatment (palliative care) with patients who did not (usual care only),^33,34 and we used a generalized linear model with a γ distribution and a log link to regress the direct hospital care cost on the binary treatment variable and all predictors listed in Table 1.^19-21

 

RESULTS

Participants

We have previously detailed that in our study there were 1023 patients eligible for cost analysis,¹⁹ of whom three were missing data in a field in Table 1 and excluded from this paper. The final analytic sample (N = 1020) is presented according to baseline covariates in Table 1 and according to summary utilization measures in Table 2.

Main Results

The results of the primary analysis, estimating the association between patient-level factors and cost of hospitalization, are presented in Table 3.

These results show the evidence of an association with cost is strongest for 3 clinical factors: a major complication (+$8267; 95% confidence interval [CI], $4509-$12,025), a minor but not a major complication (+$5289; CI, $3480-$7097), and number of comorbidities (+$852; CI, $550-$1153). In addition, there is evidence of associations between lower cost and admitting diagnosis of electrolyte disorders (–$4759; CI, –$7928 to –$1590) and older age (–$53; CI, –$99 to –$6). There is no significant association between primary diagnosis, symptom burden or other clinical factors, sociodemographic factors or healthcare utilization prior to admission and direct hospitalization costs.

Results of the secondary analyses of associations between complications, utilization, and palliative care are listed in Table 4. Patients are stratified by complication (none; major | minor) and their direct cost of hospital care and hospital length of stay (LOS) presented by treatment group (palliative care; usual care only). The data show that within each strata patients who received palliative care had lower costs and LOS than those who received usual care only. Estimated effects of palliative care on utilization is found to be statistically significant in all four quadrants, with a larger cost-effect in the complications stratum than the non-complications stratum.

Sensitivity Analysis

Fifty-one patients died during admission. After removing these cases, because of concerns about possible unobserved heterogeneity,³⁵ we checked our primary (Table 3) and secondary (Table 4) results. Patients discharged alive had results substantively similar to those of the entire sample.

DISCUSSION

Results from our primary analysis (Table 3) suggest that complications and number of comorbidities are the key drivers of hospitalization cost for adults with advanced cancer. Hospitalization for electrolyte disorders and age are both negatively associated with cost.

The association found between higher cost and hospital-acquired complications (HACs) is consistent with other studies’ finding that HACs often result in higher cost, longer LOS, and increased inhospital mortality.³⁶ Since those studies were reported, policy attention has been increasingly focused on HACs.³⁷ Our findings are notable in that, though prior evidence has also suggested high hospital cost is multifactorial, driven by a diversity of demographic, socioeconomic, and clinical factors, this rich patient-reported dataset suggests that, compared with other variables, HACs are emphatically the largest driver of cost. Moreover, cancer patients typically are a vulnerable population, more prone to complications and thus also to potentially avoidable treatments and higher cost. Our prior work suggested earlier palliative care consultation can reduce cost, in part by shortening LOS and reducing the opportunity for HACs to develop^19,20; our secondary analysis (Table 4) suggested a palliative care team’s involvement in HAC treatment can significantly reduce cost of care as well. These associations possibly derive from changed treatment choices and shorter LOS. Further work is needed to better elucidate the role of palliative care in the prevention of HACs in seriously ill patients.

That the number of comorbidities was found to be a key driver of hospitalization cost is consistent with recent findings that high spending on seriously ill patients is associated with having multiple chronic conditions rather than any specific primary diagnosis.^38,39 It is important to note that, unlike impending complications, serious chronic conditions generally are known at admission and can be addressed prospectively through provision and policy. A prior analysis with these data found that palliative care consultation was more cost-effective for patients with a larger number of comorbidities.²⁰ Our 2 studies together suggest that, notwithstanding the preferable alternative of avoiding hospitalization entirely, palliative care and other skilled coordination of care services ought to be prioritized for inpatients with multiple serious illnesses and the highest medical complexity. This patient group has both the highest costs and the greatest amenability to skilled transdisciplinary intervention, possibly because multiple chronic conditions affect patients interactively, complicating identification of appropriate polypharmacy responses and prioritization of treatments.

Our findings also may help direct appropriate use of palliative care services. The recently published American Society of Clinical Oncology palliative care guidelines note that all patients with advanced cancer (eg, those enrolled in our study) should receive dedicated palliative care services, early in the disease course, concurrent with active treatment.⁴⁰ Workforce estimates suggest that the current and future numbers of palliative care practitioners will be unable to meet the ASCO recommendations alone never mind patients with other serious illnesses (eg, advanced heart failure, COPD, CKD).⁴¹ As such, specialized palliative care services will need to be targeted to the patient populations that can benefit most from these services. Whereas cost should not be the principle driver specialized palliative care provision, it will likely be an important component due to both the necessity of allocating scarce resources in the most effective way and the evidence that in care of the seriously-ill lower costs are often a proxy for improved patient experience.

These findings also have implications for research: Different conditions and presumably different combinations of conditions have very different implications for hospital care costs for a cohort of adults with advanced cancer. Given the increasing number of co-occurring conditions among seriously ill patients, and the increasing costs of cancer care and of treating multimorbidity cases, it is essential to further our understanding of the relationship between comorbidities and costs in order to plan and finance care for advanced cancer patients.

 

Limitations and Generalizability

In this observational study, reported associations may be attributable to unobserved confounding that our analyses failed to control.

Our results reflect associations in a prospective multisite study of advanced cancer patients hospitalized in the United States. It is not clear how generalizable our findings are to patients without cancer, to patients in nonhospital settings, and to patients in other health systems and countries. Analyzing cost from the hospital perspective does not take into account that the most impactful way to reduce cost is to avoid hospitalization entirely.

Results of our secondary analysis will not necessarily be robust to patient groups, as specific weights likely will vary by sample. The idea that costs vary by condition, however, is important nevertheless. Elixhauser total was derived with use of the enhanced ICD-9-CM (International Classification of Diseases, Ninth Revision, Clinical Modification) algorithm from Quan et al.⁴² and does not include subsequent Elixhauser Comorbidity Software updates recommended by the Healthcare Cost and Utilization Project (HCUP; Agency for Healthcare Research and Quality).⁴³ The Elixhauser index is recommended over Charlson and other comorbidity indices by both HCUP⁴⁵ and a recent systematic review.⁴⁴

One possible unobserved factor is prior chemotherapy, which is associated with increased hospitalization risk. Related factors that are somewhat controlled for in the study include cancer stage (advanced cancer was an eligibility criterion) and receipt of analgesics within the week before admission (patients admitted for routine chemotherapy were excluded from analyses at the outset).

CONCLUSION

Other studies have identified a wide range of sociodemographic, clinical, and health system factors associated with healthcare utilization. Our results suggest that, for cost of hospital admission among adults with advanced cancer, the most important drivers of utilization are complications and comorbidities. Hospital costs for patients with advanced cancer constitute a major part of US healthcare spending, and these results suggest the need to prioritize high-quality, cost-effective care for patients with multiple serious illnesses.

Acknowledgments

The authors thank Robert Arnold, Phil Santa Emma, Mary Beth Happ, Tim Smith, and David Weissman for contributing to the Palliative Care for Cancer (PC4C) project.

Disclosure

The study was funded by grant R01 CA116227 from the National Cancer Institute and the National Institute of Nursing Research. The study sponsors had no role in design or conduct of the study; collection, management, analysis, or interpretation of the data; or preparation, review, or approval of the manuscript. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the US Department of Veterans Affairs or the US government. All authors are independent of the study sponsors. Dr. May was supported by a HRB/NCI Health Economics Fellowship during this work. Dr. Garrido is supported by a Veterans Affairs HSR&D career development award (CDA 11-201/CDP 12-255). Dr. Kelley’s time was funded by the National Institute on Aging (1K23AG040774-01A1) and the American Federation for Aging. Dr. Smith is funded by the NCI Core Grant P 30 006973, 1-R01 CA177562-01A1, 1-R01 NR014050 01, and the Harry J. Duffey Family Endowment for Palliative Care. Dr. Morrison was the recipient of a Midcareer Investigator Award in Patient-Oriented Research (5K24AG022345) during the course of this work. This work was supported by the NIA, Claude D. Pepper Older Americans Independence Center at the Icahn School of Medicine at Mount Sinai [5P30AG028741], and the National Palliative Care Research Center.

Infective endocarditis (IE) affected an estimated 46,800 Americans in 2011, and its incidence is increasing due to greater numbers of invasive procedures and prevalence of IE risk factors.^1-3 Despite recent advances in the treatment of IE, morbidity and mortality remain high: in-hospital mortality in IE patients is 15% to 20%, and the 1-year mortality rate is approximately 40%.^2,4,5

Poor IE outcomes may be the result of difficulty in diagnosing IE and identifying its optimal treatments. The American Heart Association (AHA), the American College of Cardiology (ACC), and the European Society of Cardiology (ESC) have published guidelines to address these challenges. Recent guidelines recommend a multidisciplinary approach that includes cardiology, cardiac surgery, and infectious disease (ID) specialty involvement in decision-making.^5,6

In the absence of published quality measures for IE management, guidelines can be used to evaluate the quality of care of IE. Studies have showed poor concordance with guideline recommendations but did not examine agreement with more recently published guidelines.^7,8 Furthermore, few studies have examined the management, outcomes, and quality of care received by IE patients. Therefore, we aimed to describe a modern cohort of patients with IE admitted to a tertiary medical center over a 4-year period. In particular, we aimed to assess quality of care received by this cohort, as measured by concordance with AHA and ACC guidelines to identify gaps in care and spur quality improvement (QI) efforts.

METHODS

Design and Study Population

We conducted a retrospective cohort study of adult IE patients admitted to Baystate Medical Center (BMC), a 716-bed tertiary academic center that covers a population of 800,000 people throughout western New England. We used the International Classification of Diseases (ICD)–Ninth Revision, to identify IE patients discharged with a principal or secondary diagnosis of IE between 2007 and 2011 (codes 421.0, 421.1, 421.9, 424.9, 424.90, and 424.91). Three co-authors confirmed the diagnosis by conducting a review of the electronic health records.

We included only patients who met modified Duke criteria for definite or possible IE.⁵ Definite IE defines patients with pathological criteria (microorganisms demonstrated by culture or histologic examination or a histologic examination showing active endocarditis); or patients with 2 major criteria (positive blood culture and evidence of endocardial involvement by echocardiogram), 1 major criterion and 3 minor criteria (minor criteria: predisposing heart conditions or intravenous drug (IVD) use, fever, vascular phenomena, immunologic phenomena, and microbiologic evidence that do not meet the major criteria) or 5 minor criteria. Possible IE defines patients with 1 major and 1 minor criterion or 3 minor criteria.⁵

 

Data Collection

We used billing and clinical databases to collect demographics, comorbidities, antibiotic treatment, 6-month readmission and 1-year mortality. Comorbid conditions were classified into Elixhauser comorbidities using software provided by the Healthcare Costs and Utilization Project of the Agency for Healthcare Research and Quality.^9,10

We obtained all other data through electronic health record abstraction. These included microbiology, type of endocarditis (native valve endocarditis [NVE] or prosthetic valve endocarditis [PVE]), echocardiographic location of the vegetation, and complications involving the valve (eg, valve perforation, ruptured chorda, perivalvular abscess, or valvular insufficiency).

Using 2006 AHA/ACC guidelines,¹¹ we identified quality metrics, including the presence of at least 2 sets of blood cultures prior to start of antibiotics and use of transthoracic echocardiogram (TTE) and transesophageal echocardiogram (TEE). Guidelines recommend using TTE as first-line to detect valvular vegetations and assess IE complications. TEE is recommended if TTE is nondiagnostic and also as first-line to diagnose PVE. We assessed the rate of consultation with ID, cardiology, and cardiac surgery specialties. While these consultations were not explicitly emphasized in the 2006 AHA/ACC guidelines, there is a class I recommendation in 2014 AHA/ACC guidelines⁵ to manage IE patients with consultation of all these specialties.

We reported the number of patients with intracardiac leads (pacemaker or defibrillator) who had documentation of intracardiac lead removal. Complete removal of intracardiac leads is indicated in IE patients with infection of leads or device (class I) and suggested for IE caused by Staphylococcus aureus or fungi (even without evidence of device or lead infection), and for patients undergoing valve surgery (class IIa).⁵ We entered abstracted data elements into a RedCap database, hosted by Tufts Clinical and Translational Science Institute.¹²

Outcomes

Outcomes included embolic events, strokes, need for cardiac surgery, LOS, inhospital mortality, 6-month readmission, and 1-year mortality. We identified embolic events using documentation of clinical or imaging evidence of an embolic event to the cerebral, coronary, peripheral arterial, renal, splenic, or pulmonary vasculature. We used record extraction to identify incidence of valve surgery. Nearly all patients who require surgery at BMC have it done onsite. We compared outcomes among patients who received less than 3 vs. 3 consultations provided by ID, cardiology, and cardiac surgery specialties. We also compared outcomes among patients who received 0, 1, 2, or 3 consultations to look for a trend.

Statistical Analysis

We divided the cohort into patients with NVE and PVE because there are differences in pathophysiology, treatment, and outcomes of these groups. We calculated descriptive statistics, including means/standard deviation (SD) and n (%). We conducted univariable analyses using Fisher exact (categorical), unpaired t tests (Gaussian), or Kruskal-Wallis equality-of-populations rank test (non-Gaussian). Common language effect sizes were also calculated to quantify group differences without respect to sample size.^13,14 Analyses were performed using Stata 14.1. (StataCorp LLC, College Station, Texas). The BMC Institutional Review Board approved the protocol.

RESULTS

We identified a total of 317 hospitalizations at BMC meeting criteria for IE. Of these, 147 hospitalizations were readmissions or did not meet the clinical criteria of definite or possible IE. Thus, we included a total of 170 patients in the final analysis. Definite IE was present in 135 (79.4%) and possible IE in 35 (20.6%) patients.

Patient Characteristics

Of 170 patients, 127 (74.7%) had NVE and 43 (25.3%) had PVE. Mean ± SD age was 60.0 ± 17.9 years, 66.5% (n = 113) of patients were male, and 79.4% (n = 135) were white (Table 1). Hypertension and chronic kidney disease were the most common comorbidities. The median Gagne score¹⁵ was 4, corresponding to a 1-year mortality risk of 15%. Predisposing factors for IE included previous history of IE (n = 14 or 8.2%), IVD use (n = 23 or 13.5%), and presence of long-term venous catheters (n = 19 or 11.2%). Intracardiac leads were present in 17.1% (n = 29) of patients. Bicuspid aortic valve was reported in 6.5% (n = 11) of patients with NVE. Patients with PVE were older (+11.5 years, 95% confidence interval [CI] 5.5, 17.5) and more likely to have intracardiac leads (44.2% vs. 7.9%; P < 0.001; Table 1).

Microbiology and Antibiotics

Staphylococcus aureus was isolated in 40.0% of patients (methicillin-sensitive: 21.2%, n = 36; methicillin-resistant: 18.8%, n = 32) and vancomycin (88.2%, n = 150) was the most common initial antibiotic used. Nearly half (44.7%, n = 76) of patients received gentamicin as part of their initial antibiotic regimen. Appendix 1 provides information on final blood culture results, prosthetic versus native valve IE, and antimicrobial agents that each patient received. PVE patients were more likely to receive gentamicin as their initial antibiotic regimen than NVE (58.1% vs. 40.2%; P = 0.051; Table 1).

 

Echocardiography and Affected Valves

As per study inclusion criteria, all patients received echocardiography (either TTE, TEE, or both). Overall, the most common infected valve was mitral (41.3%), n = 59), followed by aortic valve (28.7%), n = 41). Patients in whom the location of infected valve could not be determined (15.9%, n = 27) had echocardiographic features of intracardiac device infection or intracardiac mass (Table 1).

Quality of Care

Nearly all (n = 165, 97.1%) of patients had at least 2 sets of blood cultures drawn, most on the first day of admission (71.2%). The vast majority of patients (n = 152, 89.4%) also received their first dose of antibiotics on the day of admission. Ten (5.9%) patients did not receive any consults, and 160 (94.1%) received at least 1 consultation. An ID consultation was obtained for most (147, 86.5%) patients; cardiac surgery consultation was obtained for about half of patients (92, 54.1%), and cardiology consultation was also obtained for nearly half of patients (80, 47.1%). One-third (53, 31.2%) did not receive a cardiology or cardiac surgery consult, two-thirds (117, 68.8%) received either a cardiology or a cardiac surgery consult, and one-third (55, 32.4%) received both.

Of the 29 patients who had an intracardiac lead, 6 patients had documentation of the device removal during the index hospitalization (5 or 50.0% of patients with NVE and 1 or 5.3% of patients with PVE; P = 0.02; Table 2).

Outcomes

Evidence of any embolic events was seen in 27.7% (n = 47) of patients, including stroke in 17.1% (n = 29). Median LOS for all patients was 13.5 days, and 6-month readmission among patients who survived their index admission was 51.0% (n = 74/145; 95% CI, 45.9%-62.7%). Inhospital mortality was 14.7% (n = 25; 95% CI: 10.1%-20.9%) and 12-month mortality was 22.4% (n = 38; 95% CI, 16.7%-29.3%). Inhospital mortality was more frequent among patients with PVE than NVE (20.9% vs. 12.6%; P = 0.21), although this difference was not statistically significant. Complications were more common in NVE than PVE (any embolic event: 32.3% vs. 14.0%, P = 0.03; stroke, 20.5% vs. 7.0%, P = 0.06; Table 3).

Although there was a trend toward reduction in 6-month readmission and 12-month mortality with incremental increase in the number of specialties consulted (ID, cardiology and cardiac surgery), the difference was not statistically significant (Figure). In addition, comparing outcomes of embolic events, stroke, 6-month readmission, and 12-month mortality between those who received 3 consults (28.8%, n = 49) to those with fewer than 3 (71.2%, n = 121) did not show statistically significant differences.

Of 92 patients who received a cardiac surgery consult, 73 had NVE and 19 had PVE. Of these, 47 underwent valve surgery, 39 (of 73) with NVE (53.42%) and 8 (of 19) with PVE (42.1%). Most of the NVE patients (73.2%) had more than 1 indication for surgery. The most common indications for surgery among NVE patients were significant valvular dysfunction resulting in heart failure (65.9%), followed by mobile vegetation (56.1%) and recurrent embolic events (26.8%). The most common indication for surgery in PVE was persistent bacteremia or recurrent embolic events (75.0%).

DISCUSSION

In this study, we described the management, quality of care, and outcomes of IE patients in a tertiary medical center. We found that the majority of hospitalized patients with IE were older white men with comorbidities and IE risk factors. The complication rate was high (27.7% with embolic events) and the inhospital mortality rate was in the lower range reported by prior studies [14.7% vs. 15%-20%].⁵ Nearly one-third of patients (n = 47, 27.7%) received valve surgery. Quality of care received was generally good, with most patients receiving early blood cultures, echocardiograms, early antibiotics, and timely ID consultation. We identified important gaps in care, including a failure to consult cardiac surgery in nearly half of patients and failure to consult cardiology in more than half of patients.

Our findings support work suggesting that IE is no longer primarily a chronic or subacute disease of younger patients with IVD use, positive human immunodeficiency virus status, or bicuspid aortic valves.^1,4,16,17 The International Collaboration on Endocarditis-Prospective Cohort Study,⁴ a multinational prospective cohort study (2000-2005) of 2781 adults with IE, reported a higher prevalence of patients with diabetes or on hemodialysis, IVD users, and patients with long-term venous catheter and intracardiac leads than we found. Yet both studies suggest that the demographics of IE are changing. This may partially explain why IE mortality has not improved in recent years:^2,3 patients with older age and higher comorbidity burden may not be considered good surgical candidates.

This study is among the first to contribute information on concordance with IE guidelines in a cohort of U.S. patients. Our findings suggest that most patients received timely blood culture, same-day administration of empiric antibiotics, and ID consultation, which is similar to European studies.^7,18 Guideline concordance could be improved in some areas. Overall documentation of the management plan regarding the intracardiac leads could be improved. Only 6 of 29 patients with intracardiac leads had documentation of their removal during the index hospitalization.

The 2014 AHA/ACC guidelines⁵ and the ESC guidelines⁶ emphasized the importance of multidisciplinary management of IE. As part of the Heart Valve Team at BMC, cardiologists provide expertise in diagnosis, imaging and clinical management of IE, and cardiac surgeons provide consultation on whether to pursue surgery and optimal timing of surgery. Early discussion with surgical team is considered mandatory in all complicated cases of IE.^6,18 Infectious disease consultation has been shown to improve the rate of IE diagnosis, reduce the 6-month relapse rate,¹⁹ and improve outcomes in patients with S aureus bacteremia.²⁰ In our study 86.5% of patients had documentation of an ID consultation; cardiac surgery consultation was obtained in 54.1% and cardiology consultation in 47.1% of patients.

We observed a trend towards lower rates of 6-month readmission and 12-month mortality among patients who received all 3 consults (Figure 1), despite the fact that rates of embolic events and stroke were higher in patients with 3 consults compared to those with fewer than 3. Obviously, the lack of confounder adjustment and lack of power limits our ability to make inferences about this association, but it generates hypotheses for future work. Because subjects in our study were cared for prior to 2014, multidisciplinary management of IE with involvement of cardiology, cardiac surgery, and ID physicians was observed in only one-third of patients. However, 117 (68.8%) patients received either cardiology or cardiac surgery consults. It is possible that some physicians considered involving both cardiology and cardiac surgery consultants as unnecessary and, therefore, did not consult both specialties. We will focus future QI efforts in our institution on educating physicians about the benefits of multidisciplinary care and the importance of fully implementing the 2014 AHA/ACC guidelines.

Our findings around quality of care should be placed in the context of 2 studies by González de Molina et al⁸ and Delahaye et al⁷ These studies described considerable discordance between guideline recommendations and real-world IE care. However, these studies were performed more than a decade ago and were conducted before current recommendations to consult cardiology and cardiac surgery were published.

In the 2014 AHA/ACC guidelines, surgery prior to completion of antibiotics is indicated in patients with valve dysfunction resulting in heart failure; left-sided IE caused by highly resistant organisms (including fungus or S aureus); IE complicated by heart block, aortic abscess, or penetrating lesions; and presence of persistent infection (bacteremia or fever lasting longer than 5 to 7 days) after onset of appropriate antimicrobial therapy. In addition, there is a Class IIa indication of early surgery in patients with recurrent emboli and persistent vegetation despite appropriate antibiotic therapy and a Class IIb indication of early surgery in patients with NVE with mobile vegetation greater than 10 mm in length. Surgery is recommended for patients with PVE and relapsing infection.

It is recommended that IE patients be cared for in centers with immediate access to cardiac surgery because the urgent need for surgical intervention can arise rapidly.⁵ We found that nearly one-third of included patients underwent surgery. Although we did not collect data on indications for surgery in patients who did not receive surgery, we observed that 50% had a surgery consult, suggesting the presence of 1 or more surgical indications. Of these, half underwent valve surgery. Most of the NVE patients who underwent surgery had more than 1 indication for surgery. Our surgical rate is similar to a study from Italy³ and overall in the lower range of reported surgical rate (25%-50%) from other studies.²¹ The low rate of surgery at our center may be related to the fact that the use of surgery for IE has been hotly debated in the literature,²¹ and may also be due to the low rate of cardiac surgery consultation.

Our study has several limitations. We identified eligible patients using a discharge ICD-9 coding of IE and then confirmed the presence of Duke criteria using record review. Using discharge diagnosis codes for endocarditis has been validated, and our additional manual chart review to confirm Duke criteria likely improved the specificity significantly. However, by excluding patients who did not have documented evidence of Duke criteria, we may have missed some cases, lowering sensitivity. The performance on selected quality metrics may also have been affected by our inclusion criteria. Because we included only patients who met Duke criteria, we tended to include patients who had received blood cultures and echocardiograms, which are part of the criteria. Thus, we cannot comment on use of diagnostic testing or specialty consultation in patients with suspected IE. This was a single-center study and may not represent patients or current practices seen in other institutions. We did not collect data on some of the predisposing factors to NVE (for example, baseline rheumatic heart disease or preexisting valvular heart disease) because it is estimated that less than 5% of IE in the U.S. is superimposed on rheumatic heart disease.⁴ We likely underestimated 12-month mortality rate because we did not cross-reference our findings again the National Death Index; however, this should not affect the comparison of this outcome between groups.

 

CONCLUSION

Our study confirms reports that IE epidemiology has changed significantly in recent years. It also suggests that concordance with guideline recommendations is good for some aspects of care (eg, echocardiogram, blood cultures), but can be improved in other areas, particularly in use of specialty consultation during the hospitalization. Future QI efforts should emphasize the role of the heart valve team or endocarditis team that consists of an internist, ID physician, cardiologist, cardiac surgeon, and nursing. Finally, efforts should be made to develop strategies for community hospitals that do not have access to all of these specialists (eg, early transfer, telehealth).

Disclosure

Nothing to report.

Infective endocarditis (IE) affected an estimated 46,800 Americans in 2011, and its incidence is increasing due to greater numbers of invasive procedures and prevalence of IE risk factors.^1-3 Despite recent advances in the treatment of IE, morbidity and mortality remain high: in-hospital mortality in IE patients is 15% to 20%, and the 1-year mortality rate is approximately 40%.^2,4,5

Poor IE outcomes may be the result of difficulty in diagnosing IE and identifying its optimal treatments. The American Heart Association (AHA), the American College of Cardiology (ACC), and the European Society of Cardiology (ESC) have published guidelines to address these challenges. Recent guidelines recommend a multidisciplinary approach that includes cardiology, cardiac surgery, and infectious disease (ID) specialty involvement in decision-making.^5,6

In the absence of published quality measures for IE management, guidelines can be used to evaluate the quality of care of IE. Studies have showed poor concordance with guideline recommendations but did not examine agreement with more recently published guidelines.^7,8 Furthermore, few studies have examined the management, outcomes, and quality of care received by IE patients. Therefore, we aimed to describe a modern cohort of patients with IE admitted to a tertiary medical center over a 4-year period. In particular, we aimed to assess quality of care received by this cohort, as measured by concordance with AHA and ACC guidelines to identify gaps in care and spur quality improvement (QI) efforts.

METHODS

Design and Study Population

We conducted a retrospective cohort study of adult IE patients admitted to Baystate Medical Center (BMC), a 716-bed tertiary academic center that covers a population of 800,000 people throughout western New England. We used the International Classification of Diseases (ICD)–Ninth Revision, to identify IE patients discharged with a principal or secondary diagnosis of IE between 2007 and 2011 (codes 421.0, 421.1, 421.9, 424.9, 424.90, and 424.91). Three co-authors confirmed the diagnosis by conducting a review of the electronic health records.

We included only patients who met modified Duke criteria for definite or possible IE.⁵ Definite IE defines patients with pathological criteria (microorganisms demonstrated by culture or histologic examination or a histologic examination showing active endocarditis); or patients with 2 major criteria (positive blood culture and evidence of endocardial involvement by echocardiogram), 1 major criterion and 3 minor criteria (minor criteria: predisposing heart conditions or intravenous drug (IVD) use, fever, vascular phenomena, immunologic phenomena, and microbiologic evidence that do not meet the major criteria) or 5 minor criteria. Possible IE defines patients with 1 major and 1 minor criterion or 3 minor criteria.⁵

 

Data Collection

We used billing and clinical databases to collect demographics, comorbidities, antibiotic treatment, 6-month readmission and 1-year mortality. Comorbid conditions were classified into Elixhauser comorbidities using software provided by the Healthcare Costs and Utilization Project of the Agency for Healthcare Research and Quality.^9,10

We obtained all other data through electronic health record abstraction. These included microbiology, type of endocarditis (native valve endocarditis [NVE] or prosthetic valve endocarditis [PVE]), echocardiographic location of the vegetation, and complications involving the valve (eg, valve perforation, ruptured chorda, perivalvular abscess, or valvular insufficiency).

Using 2006 AHA/ACC guidelines,¹¹ we identified quality metrics, including the presence of at least 2 sets of blood cultures prior to start of antibiotics and use of transthoracic echocardiogram (TTE) and transesophageal echocardiogram (TEE). Guidelines recommend using TTE as first-line to detect valvular vegetations and assess IE complications. TEE is recommended if TTE is nondiagnostic and also as first-line to diagnose PVE. We assessed the rate of consultation with ID, cardiology, and cardiac surgery specialties. While these consultations were not explicitly emphasized in the 2006 AHA/ACC guidelines, there is a class I recommendation in 2014 AHA/ACC guidelines⁵ to manage IE patients with consultation of all these specialties.

We reported the number of patients with intracardiac leads (pacemaker or defibrillator) who had documentation of intracardiac lead removal. Complete removal of intracardiac leads is indicated in IE patients with infection of leads or device (class I) and suggested for IE caused by Staphylococcus aureus or fungi (even without evidence of device or lead infection), and for patients undergoing valve surgery (class IIa).⁵ We entered abstracted data elements into a RedCap database, hosted by Tufts Clinical and Translational Science Institute.¹²

Outcomes

Outcomes included embolic events, strokes, need for cardiac surgery, LOS, inhospital mortality, 6-month readmission, and 1-year mortality. We identified embolic events using documentation of clinical or imaging evidence of an embolic event to the cerebral, coronary, peripheral arterial, renal, splenic, or pulmonary vasculature. We used record extraction to identify incidence of valve surgery. Nearly all patients who require surgery at BMC have it done onsite. We compared outcomes among patients who received less than 3 vs. 3 consultations provided by ID, cardiology, and cardiac surgery specialties. We also compared outcomes among patients who received 0, 1, 2, or 3 consultations to look for a trend.

Statistical Analysis

We divided the cohort into patients with NVE and PVE because there are differences in pathophysiology, treatment, and outcomes of these groups. We calculated descriptive statistics, including means/standard deviation (SD) and n (%). We conducted univariable analyses using Fisher exact (categorical), unpaired t tests (Gaussian), or Kruskal-Wallis equality-of-populations rank test (non-Gaussian). Common language effect sizes were also calculated to quantify group differences without respect to sample size.^13,14 Analyses were performed using Stata 14.1. (StataCorp LLC, College Station, Texas). The BMC Institutional Review Board approved the protocol.

RESULTS

We identified a total of 317 hospitalizations at BMC meeting criteria for IE. Of these, 147 hospitalizations were readmissions or did not meet the clinical criteria of definite or possible IE. Thus, we included a total of 170 patients in the final analysis. Definite IE was present in 135 (79.4%) and possible IE in 35 (20.6%) patients.

Patient Characteristics

Of 170 patients, 127 (74.7%) had NVE and 43 (25.3%) had PVE. Mean ± SD age was 60.0 ± 17.9 years, 66.5% (n = 113) of patients were male, and 79.4% (n = 135) were white (Table 1). Hypertension and chronic kidney disease were the most common comorbidities. The median Gagne score¹⁵ was 4, corresponding to a 1-year mortality risk of 15%. Predisposing factors for IE included previous history of IE (n = 14 or 8.2%), IVD use (n = 23 or 13.5%), and presence of long-term venous catheters (n = 19 or 11.2%). Intracardiac leads were present in 17.1% (n = 29) of patients. Bicuspid aortic valve was reported in 6.5% (n = 11) of patients with NVE. Patients with PVE were older (+11.5 years, 95% confidence interval [CI] 5.5, 17.5) and more likely to have intracardiac leads (44.2% vs. 7.9%; P < 0.001; Table 1).

Microbiology and Antibiotics

Staphylococcus aureus was isolated in 40.0% of patients (methicillin-sensitive: 21.2%, n = 36; methicillin-resistant: 18.8%, n = 32) and vancomycin (88.2%, n = 150) was the most common initial antibiotic used. Nearly half (44.7%, n = 76) of patients received gentamicin as part of their initial antibiotic regimen. Appendix 1 provides information on final blood culture results, prosthetic versus native valve IE, and antimicrobial agents that each patient received. PVE patients were more likely to receive gentamicin as their initial antibiotic regimen than NVE (58.1% vs. 40.2%; P = 0.051; Table 1).

 

Echocardiography and Affected Valves

As per study inclusion criteria, all patients received echocardiography (either TTE, TEE, or both). Overall, the most common infected valve was mitral (41.3%), n = 59), followed by aortic valve (28.7%), n = 41). Patients in whom the location of infected valve could not be determined (15.9%, n = 27) had echocardiographic features of intracardiac device infection or intracardiac mass (Table 1).

Quality of Care

Nearly all (n = 165, 97.1%) of patients had at least 2 sets of blood cultures drawn, most on the first day of admission (71.2%). The vast majority of patients (n = 152, 89.4%) also received their first dose of antibiotics on the day of admission. Ten (5.9%) patients did not receive any consults, and 160 (94.1%) received at least 1 consultation. An ID consultation was obtained for most (147, 86.5%) patients; cardiac surgery consultation was obtained for about half of patients (92, 54.1%), and cardiology consultation was also obtained for nearly half of patients (80, 47.1%). One-third (53, 31.2%) did not receive a cardiology or cardiac surgery consult, two-thirds (117, 68.8%) received either a cardiology or a cardiac surgery consult, and one-third (55, 32.4%) received both.

Of the 29 patients who had an intracardiac lead, 6 patients had documentation of the device removal during the index hospitalization (5 or 50.0% of patients with NVE and 1 or 5.3% of patients with PVE; P = 0.02; Table 2).

Outcomes

Evidence of any embolic events was seen in 27.7% (n = 47) of patients, including stroke in 17.1% (n = 29). Median LOS for all patients was 13.5 days, and 6-month readmission among patients who survived their index admission was 51.0% (n = 74/145; 95% CI, 45.9%-62.7%). Inhospital mortality was 14.7% (n = 25; 95% CI: 10.1%-20.9%) and 12-month mortality was 22.4% (n = 38; 95% CI, 16.7%-29.3%). Inhospital mortality was more frequent among patients with PVE than NVE (20.9% vs. 12.6%; P = 0.21), although this difference was not statistically significant. Complications were more common in NVE than PVE (any embolic event: 32.3% vs. 14.0%, P = 0.03; stroke, 20.5% vs. 7.0%, P = 0.06; Table 3).

Although there was a trend toward reduction in 6-month readmission and 12-month mortality with incremental increase in the number of specialties consulted (ID, cardiology and cardiac surgery), the difference was not statistically significant (Figure). In addition, comparing outcomes of embolic events, stroke, 6-month readmission, and 12-month mortality between those who received 3 consults (28.8%, n = 49) to those with fewer than 3 (71.2%, n = 121) did not show statistically significant differences.

Of 92 patients who received a cardiac surgery consult, 73 had NVE and 19 had PVE. Of these, 47 underwent valve surgery, 39 (of 73) with NVE (53.42%) and 8 (of 19) with PVE (42.1%). Most of the NVE patients (73.2%) had more than 1 indication for surgery. The most common indications for surgery among NVE patients were significant valvular dysfunction resulting in heart failure (65.9%), followed by mobile vegetation (56.1%) and recurrent embolic events (26.8%). The most common indication for surgery in PVE was persistent bacteremia or recurrent embolic events (75.0%).

DISCUSSION

In this study, we described the management, quality of care, and outcomes of IE patients in a tertiary medical center. We found that the majority of hospitalized patients with IE were older white men with comorbidities and IE risk factors. The complication rate was high (27.7% with embolic events) and the inhospital mortality rate was in the lower range reported by prior studies [14.7% vs. 15%-20%].⁵ Nearly one-third of patients (n = 47, 27.7%) received valve surgery. Quality of care received was generally good, with most patients receiving early blood cultures, echocardiograms, early antibiotics, and timely ID consultation. We identified important gaps in care, including a failure to consult cardiac surgery in nearly half of patients and failure to consult cardiology in more than half of patients.

Our findings support work suggesting that IE is no longer primarily a chronic or subacute disease of younger patients with IVD use, positive human immunodeficiency virus status, or bicuspid aortic valves.^1,4,16,17 The International Collaboration on Endocarditis-Prospective Cohort Study,⁴ a multinational prospective cohort study (2000-2005) of 2781 adults with IE, reported a higher prevalence of patients with diabetes or on hemodialysis, IVD users, and patients with long-term venous catheter and intracardiac leads than we found. Yet both studies suggest that the demographics of IE are changing. This may partially explain why IE mortality has not improved in recent years:^2,3 patients with older age and higher comorbidity burden may not be considered good surgical candidates.

This study is among the first to contribute information on concordance with IE guidelines in a cohort of U.S. patients. Our findings suggest that most patients received timely blood culture, same-day administration of empiric antibiotics, and ID consultation, which is similar to European studies.^7,18 Guideline concordance could be improved in some areas. Overall documentation of the management plan regarding the intracardiac leads could be improved. Only 6 of 29 patients with intracardiac leads had documentation of their removal during the index hospitalization.

The 2014 AHA/ACC guidelines⁵ and the ESC guidelines⁶ emphasized the importance of multidisciplinary management of IE. As part of the Heart Valve Team at BMC, cardiologists provide expertise in diagnosis, imaging and clinical management of IE, and cardiac surgeons provide consultation on whether to pursue surgery and optimal timing of surgery. Early discussion with surgical team is considered mandatory in all complicated cases of IE.^6,18 Infectious disease consultation has been shown to improve the rate of IE diagnosis, reduce the 6-month relapse rate,¹⁹ and improve outcomes in patients with S aureus bacteremia.²⁰ In our study 86.5% of patients had documentation of an ID consultation; cardiac surgery consultation was obtained in 54.1% and cardiology consultation in 47.1% of patients.

We observed a trend towards lower rates of 6-month readmission and 12-month mortality among patients who received all 3 consults (Figure 1), despite the fact that rates of embolic events and stroke were higher in patients with 3 consults compared to those with fewer than 3. Obviously, the lack of confounder adjustment and lack of power limits our ability to make inferences about this association, but it generates hypotheses for future work. Because subjects in our study were cared for prior to 2014, multidisciplinary management of IE with involvement of cardiology, cardiac surgery, and ID physicians was observed in only one-third of patients. However, 117 (68.8%) patients received either cardiology or cardiac surgery consults. It is possible that some physicians considered involving both cardiology and cardiac surgery consultants as unnecessary and, therefore, did not consult both specialties. We will focus future QI efforts in our institution on educating physicians about the benefits of multidisciplinary care and the importance of fully implementing the 2014 AHA/ACC guidelines.

Our findings around quality of care should be placed in the context of 2 studies by González de Molina et al⁸ and Delahaye et al⁷ These studies described considerable discordance between guideline recommendations and real-world IE care. However, these studies were performed more than a decade ago and were conducted before current recommendations to consult cardiology and cardiac surgery were published.

In the 2014 AHA/ACC guidelines, surgery prior to completion of antibiotics is indicated in patients with valve dysfunction resulting in heart failure; left-sided IE caused by highly resistant organisms (including fungus or S aureus); IE complicated by heart block, aortic abscess, or penetrating lesions; and presence of persistent infection (bacteremia or fever lasting longer than 5 to 7 days) after onset of appropriate antimicrobial therapy. In addition, there is a Class IIa indication of early surgery in patients with recurrent emboli and persistent vegetation despite appropriate antibiotic therapy and a Class IIb indication of early surgery in patients with NVE with mobile vegetation greater than 10 mm in length. Surgery is recommended for patients with PVE and relapsing infection.

It is recommended that IE patients be cared for in centers with immediate access to cardiac surgery because the urgent need for surgical intervention can arise rapidly.⁵ We found that nearly one-third of included patients underwent surgery. Although we did not collect data on indications for surgery in patients who did not receive surgery, we observed that 50% had a surgery consult, suggesting the presence of 1 or more surgical indications. Of these, half underwent valve surgery. Most of the NVE patients who underwent surgery had more than 1 indication for surgery. Our surgical rate is similar to a study from Italy³ and overall in the lower range of reported surgical rate (25%-50%) from other studies.²¹ The low rate of surgery at our center may be related to the fact that the use of surgery for IE has been hotly debated in the literature,²¹ and may also be due to the low rate of cardiac surgery consultation.

Our study has several limitations. We identified eligible patients using a discharge ICD-9 coding of IE and then confirmed the presence of Duke criteria using record review. Using discharge diagnosis codes for endocarditis has been validated, and our additional manual chart review to confirm Duke criteria likely improved the specificity significantly. However, by excluding patients who did not have documented evidence of Duke criteria, we may have missed some cases, lowering sensitivity. The performance on selected quality metrics may also have been affected by our inclusion criteria. Because we included only patients who met Duke criteria, we tended to include patients who had received blood cultures and echocardiograms, which are part of the criteria. Thus, we cannot comment on use of diagnostic testing or specialty consultation in patients with suspected IE. This was a single-center study and may not represent patients or current practices seen in other institutions. We did not collect data on some of the predisposing factors to NVE (for example, baseline rheumatic heart disease or preexisting valvular heart disease) because it is estimated that less than 5% of IE in the U.S. is superimposed on rheumatic heart disease.⁴ We likely underestimated 12-month mortality rate because we did not cross-reference our findings again the National Death Index; however, this should not affect the comparison of this outcome between groups.

 

CONCLUSION

Our study confirms reports that IE epidemiology has changed significantly in recent years. It also suggests that concordance with guideline recommendations is good for some aspects of care (eg, echocardiogram, blood cultures), but can be improved in other areas, particularly in use of specialty consultation during the hospitalization. Future QI efforts should emphasize the role of the heart valve team or endocarditis team that consists of an internist, ID physician, cardiologist, cardiac surgeon, and nursing. Finally, efforts should be made to develop strategies for community hospitals that do not have access to all of these specialists (eg, early transfer, telehealth).

Disclosure

Nothing to report.

Infective endocarditis (IE) affected an estimated 46,800 Americans in 2011, and its incidence is increasing due to greater numbers of invasive procedures and prevalence of IE risk factors.^1-3 Despite recent advances in the treatment of IE, morbidity and mortality remain high: in-hospital mortality in IE patients is 15% to 20%, and the 1-year mortality rate is approximately 40%.^2,4,5

Poor IE outcomes may be the result of difficulty in diagnosing IE and identifying its optimal treatments. The American Heart Association (AHA), the American College of Cardiology (ACC), and the European Society of Cardiology (ESC) have published guidelines to address these challenges. Recent guidelines recommend a multidisciplinary approach that includes cardiology, cardiac surgery, and infectious disease (ID) specialty involvement in decision-making.^5,6

In the absence of published quality measures for IE management, guidelines can be used to evaluate the quality of care of IE. Studies have showed poor concordance with guideline recommendations but did not examine agreement with more recently published guidelines.^7,8 Furthermore, few studies have examined the management, outcomes, and quality of care received by IE patients. Therefore, we aimed to describe a modern cohort of patients with IE admitted to a tertiary medical center over a 4-year period. In particular, we aimed to assess quality of care received by this cohort, as measured by concordance with AHA and ACC guidelines to identify gaps in care and spur quality improvement (QI) efforts.

METHODS

Design and Study Population

We conducted a retrospective cohort study of adult IE patients admitted to Baystate Medical Center (BMC), a 716-bed tertiary academic center that covers a population of 800,000 people throughout western New England. We used the International Classification of Diseases (ICD)–Ninth Revision, to identify IE patients discharged with a principal or secondary diagnosis of IE between 2007 and 2011 (codes 421.0, 421.1, 421.9, 424.9, 424.90, and 424.91). Three co-authors confirmed the diagnosis by conducting a review of the electronic health records.

We included only patients who met modified Duke criteria for definite or possible IE.⁵ Definite IE defines patients with pathological criteria (microorganisms demonstrated by culture or histologic examination or a histologic examination showing active endocarditis); or patients with 2 major criteria (positive blood culture and evidence of endocardial involvement by echocardiogram), 1 major criterion and 3 minor criteria (minor criteria: predisposing heart conditions or intravenous drug (IVD) use, fever, vascular phenomena, immunologic phenomena, and microbiologic evidence that do not meet the major criteria) or 5 minor criteria. Possible IE defines patients with 1 major and 1 minor criterion or 3 minor criteria.⁵

 

Data Collection

We used billing and clinical databases to collect demographics, comorbidities, antibiotic treatment, 6-month readmission and 1-year mortality. Comorbid conditions were classified into Elixhauser comorbidities using software provided by the Healthcare Costs and Utilization Project of the Agency for Healthcare Research and Quality.^9,10

We obtained all other data through electronic health record abstraction. These included microbiology, type of endocarditis (native valve endocarditis [NVE] or prosthetic valve endocarditis [PVE]), echocardiographic location of the vegetation, and complications involving the valve (eg, valve perforation, ruptured chorda, perivalvular abscess, or valvular insufficiency).

Using 2006 AHA/ACC guidelines,¹¹ we identified quality metrics, including the presence of at least 2 sets of blood cultures prior to start of antibiotics and use of transthoracic echocardiogram (TTE) and transesophageal echocardiogram (TEE). Guidelines recommend using TTE as first-line to detect valvular vegetations and assess IE complications. TEE is recommended if TTE is nondiagnostic and also as first-line to diagnose PVE. We assessed the rate of consultation with ID, cardiology, and cardiac surgery specialties. While these consultations were not explicitly emphasized in the 2006 AHA/ACC guidelines, there is a class I recommendation in 2014 AHA/ACC guidelines⁵ to manage IE patients with consultation of all these specialties.

We reported the number of patients with intracardiac leads (pacemaker or defibrillator) who had documentation of intracardiac lead removal. Complete removal of intracardiac leads is indicated in IE patients with infection of leads or device (class I) and suggested for IE caused by Staphylococcus aureus or fungi (even without evidence of device or lead infection), and for patients undergoing valve surgery (class IIa).⁵ We entered abstracted data elements into a RedCap database, hosted by Tufts Clinical and Translational Science Institute.¹²

Outcomes

Outcomes included embolic events, strokes, need for cardiac surgery, LOS, inhospital mortality, 6-month readmission, and 1-year mortality. We identified embolic events using documentation of clinical or imaging evidence of an embolic event to the cerebral, coronary, peripheral arterial, renal, splenic, or pulmonary vasculature. We used record extraction to identify incidence of valve surgery. Nearly all patients who require surgery at BMC have it done onsite. We compared outcomes among patients who received less than 3 vs. 3 consultations provided by ID, cardiology, and cardiac surgery specialties. We also compared outcomes among patients who received 0, 1, 2, or 3 consultations to look for a trend.

Statistical Analysis

We divided the cohort into patients with NVE and PVE because there are differences in pathophysiology, treatment, and outcomes of these groups. We calculated descriptive statistics, including means/standard deviation (SD) and n (%). We conducted univariable analyses using Fisher exact (categorical), unpaired t tests (Gaussian), or Kruskal-Wallis equality-of-populations rank test (non-Gaussian). Common language effect sizes were also calculated to quantify group differences without respect to sample size.^13,14 Analyses were performed using Stata 14.1. (StataCorp LLC, College Station, Texas). The BMC Institutional Review Board approved the protocol.

RESULTS

We identified a total of 317 hospitalizations at BMC meeting criteria for IE. Of these, 147 hospitalizations were readmissions or did not meet the clinical criteria of definite or possible IE. Thus, we included a total of 170 patients in the final analysis. Definite IE was present in 135 (79.4%) and possible IE in 35 (20.6%) patients.

Patient Characteristics

Of 170 patients, 127 (74.7%) had NVE and 43 (25.3%) had PVE. Mean ± SD age was 60.0 ± 17.9 years, 66.5% (n = 113) of patients were male, and 79.4% (n = 135) were white (Table 1). Hypertension and chronic kidney disease were the most common comorbidities. The median Gagne score¹⁵ was 4, corresponding to a 1-year mortality risk of 15%. Predisposing factors for IE included previous history of IE (n = 14 or 8.2%), IVD use (n = 23 or 13.5%), and presence of long-term venous catheters (n = 19 or 11.2%). Intracardiac leads were present in 17.1% (n = 29) of patients. Bicuspid aortic valve was reported in 6.5% (n = 11) of patients with NVE. Patients with PVE were older (+11.5 years, 95% confidence interval [CI] 5.5, 17.5) and more likely to have intracardiac leads (44.2% vs. 7.9%; P < 0.001; Table 1).

Microbiology and Antibiotics

Staphylococcus aureus was isolated in 40.0% of patients (methicillin-sensitive: 21.2%, n = 36; methicillin-resistant: 18.8%, n = 32) and vancomycin (88.2%, n = 150) was the most common initial antibiotic used. Nearly half (44.7%, n = 76) of patients received gentamicin as part of their initial antibiotic regimen. Appendix 1 provides information on final blood culture results, prosthetic versus native valve IE, and antimicrobial agents that each patient received. PVE patients were more likely to receive gentamicin as their initial antibiotic regimen than NVE (58.1% vs. 40.2%; P = 0.051; Table 1).

 

Echocardiography and Affected Valves

As per study inclusion criteria, all patients received echocardiography (either TTE, TEE, or both). Overall, the most common infected valve was mitral (41.3%), n = 59), followed by aortic valve (28.7%), n = 41). Patients in whom the location of infected valve could not be determined (15.9%, n = 27) had echocardiographic features of intracardiac device infection or intracardiac mass (Table 1).

Quality of Care

Nearly all (n = 165, 97.1%) of patients had at least 2 sets of blood cultures drawn, most on the first day of admission (71.2%). The vast majority of patients (n = 152, 89.4%) also received their first dose of antibiotics on the day of admission. Ten (5.9%) patients did not receive any consults, and 160 (94.1%) received at least 1 consultation. An ID consultation was obtained for most (147, 86.5%) patients; cardiac surgery consultation was obtained for about half of patients (92, 54.1%), and cardiology consultation was also obtained for nearly half of patients (80, 47.1%). One-third (53, 31.2%) did not receive a cardiology or cardiac surgery consult, two-thirds (117, 68.8%) received either a cardiology or a cardiac surgery consult, and one-third (55, 32.4%) received both.

Of the 29 patients who had an intracardiac lead, 6 patients had documentation of the device removal during the index hospitalization (5 or 50.0% of patients with NVE and 1 or 5.3% of patients with PVE; P = 0.02; Table 2).

Outcomes

Evidence of any embolic events was seen in 27.7% (n = 47) of patients, including stroke in 17.1% (n = 29). Median LOS for all patients was 13.5 days, and 6-month readmission among patients who survived their index admission was 51.0% (n = 74/145; 95% CI, 45.9%-62.7%). Inhospital mortality was 14.7% (n = 25; 95% CI: 10.1%-20.9%) and 12-month mortality was 22.4% (n = 38; 95% CI, 16.7%-29.3%). Inhospital mortality was more frequent among patients with PVE than NVE (20.9% vs. 12.6%; P = 0.21), although this difference was not statistically significant. Complications were more common in NVE than PVE (any embolic event: 32.3% vs. 14.0%, P = 0.03; stroke, 20.5% vs. 7.0%, P = 0.06; Table 3).

Although there was a trend toward reduction in 6-month readmission and 12-month mortality with incremental increase in the number of specialties consulted (ID, cardiology and cardiac surgery), the difference was not statistically significant (Figure). In addition, comparing outcomes of embolic events, stroke, 6-month readmission, and 12-month mortality between those who received 3 consults (28.8%, n = 49) to those with fewer than 3 (71.2%, n = 121) did not show statistically significant differences.

Of 92 patients who received a cardiac surgery consult, 73 had NVE and 19 had PVE. Of these, 47 underwent valve surgery, 39 (of 73) with NVE (53.42%) and 8 (of 19) with PVE (42.1%). Most of the NVE patients (73.2%) had more than 1 indication for surgery. The most common indications for surgery among NVE patients were significant valvular dysfunction resulting in heart failure (65.9%), followed by mobile vegetation (56.1%) and recurrent embolic events (26.8%). The most common indication for surgery in PVE was persistent bacteremia or recurrent embolic events (75.0%).

DISCUSSION

In this study, we described the management, quality of care, and outcomes of IE patients in a tertiary medical center. We found that the majority of hospitalized patients with IE were older white men with comorbidities and IE risk factors. The complication rate was high (27.7% with embolic events) and the inhospital mortality rate was in the lower range reported by prior studies [14.7% vs. 15%-20%].⁵ Nearly one-third of patients (n = 47, 27.7%) received valve surgery. Quality of care received was generally good, with most patients receiving early blood cultures, echocardiograms, early antibiotics, and timely ID consultation. We identified important gaps in care, including a failure to consult cardiac surgery in nearly half of patients and failure to consult cardiology in more than half of patients.

Our findings support work suggesting that IE is no longer primarily a chronic or subacute disease of younger patients with IVD use, positive human immunodeficiency virus status, or bicuspid aortic valves.^1,4,16,17 The International Collaboration on Endocarditis-Prospective Cohort Study,⁴ a multinational prospective cohort study (2000-2005) of 2781 adults with IE, reported a higher prevalence of patients with diabetes or on hemodialysis, IVD users, and patients with long-term venous catheter and intracardiac leads than we found. Yet both studies suggest that the demographics of IE are changing. This may partially explain why IE mortality has not improved in recent years:^2,3 patients with older age and higher comorbidity burden may not be considered good surgical candidates.

This study is among the first to contribute information on concordance with IE guidelines in a cohort of U.S. patients. Our findings suggest that most patients received timely blood culture, same-day administration of empiric antibiotics, and ID consultation, which is similar to European studies.^7,18 Guideline concordance could be improved in some areas. Overall documentation of the management plan regarding the intracardiac leads could be improved. Only 6 of 29 patients with intracardiac leads had documentation of their removal during the index hospitalization.

The 2014 AHA/ACC guidelines⁵ and the ESC guidelines⁶ emphasized the importance of multidisciplinary management of IE. As part of the Heart Valve Team at BMC, cardiologists provide expertise in diagnosis, imaging and clinical management of IE, and cardiac surgeons provide consultation on whether to pursue surgery and optimal timing of surgery. Early discussion with surgical team is considered mandatory in all complicated cases of IE.^6,18 Infectious disease consultation has been shown to improve the rate of IE diagnosis, reduce the 6-month relapse rate,¹⁹ and improve outcomes in patients with S aureus bacteremia.²⁰ In our study 86.5% of patients had documentation of an ID consultation; cardiac surgery consultation was obtained in 54.1% and cardiology consultation in 47.1% of patients.

We observed a trend towards lower rates of 6-month readmission and 12-month mortality among patients who received all 3 consults (Figure 1), despite the fact that rates of embolic events and stroke were higher in patients with 3 consults compared to those with fewer than 3. Obviously, the lack of confounder adjustment and lack of power limits our ability to make inferences about this association, but it generates hypotheses for future work. Because subjects in our study were cared for prior to 2014, multidisciplinary management of IE with involvement of cardiology, cardiac surgery, and ID physicians was observed in only one-third of patients. However, 117 (68.8%) patients received either cardiology or cardiac surgery consults. It is possible that some physicians considered involving both cardiology and cardiac surgery consultants as unnecessary and, therefore, did not consult both specialties. We will focus future QI efforts in our institution on educating physicians about the benefits of multidisciplinary care and the importance of fully implementing the 2014 AHA/ACC guidelines.

Our findings around quality of care should be placed in the context of 2 studies by González de Molina et al⁸ and Delahaye et al⁷ These studies described considerable discordance between guideline recommendations and real-world IE care. However, these studies were performed more than a decade ago and were conducted before current recommendations to consult cardiology and cardiac surgery were published.

In the 2014 AHA/ACC guidelines, surgery prior to completion of antibiotics is indicated in patients with valve dysfunction resulting in heart failure; left-sided IE caused by highly resistant organisms (including fungus or S aureus); IE complicated by heart block, aortic abscess, or penetrating lesions; and presence of persistent infection (bacteremia or fever lasting longer than 5 to 7 days) after onset of appropriate antimicrobial therapy. In addition, there is a Class IIa indication of early surgery in patients with recurrent emboli and persistent vegetation despite appropriate antibiotic therapy and a Class IIb indication of early surgery in patients with NVE with mobile vegetation greater than 10 mm in length. Surgery is recommended for patients with PVE and relapsing infection.

It is recommended that IE patients be cared for in centers with immediate access to cardiac surgery because the urgent need for surgical intervention can arise rapidly.⁵ We found that nearly one-third of included patients underwent surgery. Although we did not collect data on indications for surgery in patients who did not receive surgery, we observed that 50% had a surgery consult, suggesting the presence of 1 or more surgical indications. Of these, half underwent valve surgery. Most of the NVE patients who underwent surgery had more than 1 indication for surgery. Our surgical rate is similar to a study from Italy³ and overall in the lower range of reported surgical rate (25%-50%) from other studies.²¹ The low rate of surgery at our center may be related to the fact that the use of surgery for IE has been hotly debated in the literature,²¹ and may also be due to the low rate of cardiac surgery consultation.

Our study has several limitations. We identified eligible patients using a discharge ICD-9 coding of IE and then confirmed the presence of Duke criteria using record review. Using discharge diagnosis codes for endocarditis has been validated, and our additional manual chart review to confirm Duke criteria likely improved the specificity significantly. However, by excluding patients who did not have documented evidence of Duke criteria, we may have missed some cases, lowering sensitivity. The performance on selected quality metrics may also have been affected by our inclusion criteria. Because we included only patients who met Duke criteria, we tended to include patients who had received blood cultures and echocardiograms, which are part of the criteria. Thus, we cannot comment on use of diagnostic testing or specialty consultation in patients with suspected IE. This was a single-center study and may not represent patients or current practices seen in other institutions. We did not collect data on some of the predisposing factors to NVE (for example, baseline rheumatic heart disease or preexisting valvular heart disease) because it is estimated that less than 5% of IE in the U.S. is superimposed on rheumatic heart disease.⁴ We likely underestimated 12-month mortality rate because we did not cross-reference our findings again the National Death Index; however, this should not affect the comparison of this outcome between groups.

 

CONCLUSION

Our study confirms reports that IE epidemiology has changed significantly in recent years. It also suggests that concordance with guideline recommendations is good for some aspects of care (eg, echocardiogram, blood cultures), but can be improved in other areas, particularly in use of specialty consultation during the hospitalization. Future QI efforts should emphasize the role of the heart valve team or endocarditis team that consists of an internist, ID physician, cardiologist, cardiac surgeon, and nursing. Finally, efforts should be made to develop strategies for community hospitals that do not have access to all of these specialists (eg, early transfer, telehealth).

Disclosure

Nothing to report.

Communication is the foundation of medical care.¹ Effective communication can improve health outcomes, safety, adherence, satisfaction, trust, and enable genuine informed consent and decision-making.^2-9 Furthermore, high-quality communication increases provider engagement and workplace satisfaction, while reducing stress and malpractice risk.^10-15

Direct measurement of communication in the healthcare setting can be challenging. The “Four Habits Model,” which is derived from a synthesis of empiric studies^8,16-20 and theoretical models^21-24 of communication, offers 1 framework for assessing healthcare communication. The conceptual model underlying the 4 habits has been validated in studies of physician and patient satisfaction.^1,4,25-27 The 4 habits are: investing in the beginning, eliciting the patient’s perspective, demonstrating empathy, and investing in the end. Each habit is divided into several identifiable tasks or skill sets, which can be reliably measured using validated tools and checklists.²⁸ One such instrument, the Four Habits Coding Scheme (4HCS), has been evaluated against other tools and demonstrated overall satisfactory inter-rater reliability and validity.^29,30

The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey, developed under the direction of the Centers for Medicare and Medicaid Services (CMS) and the Agency for Healthcare Research and Quality, is an established national standard for measuring patient perceptions of care. HCAHPS retrospectively measures global perceptions of communication, support and empathy from physicians and staff, processes of care, and the overall patient experience. HCAHPS scores were first collected nationally in 2006 and have been publicly reported since 2008.³¹ With the introduction of value-based purchasing in 2012, health system revenues are now tied to HCAHPS survey performance.³² As a result, hospitals are financially motivated to improve HCAHPS scores but lack evidence-based methods for doing so. Some healthcare organizations have invested in communication training programs based on the available literature and best practices.^2,33-35 However, it is not known how, if at all, HCAHPS scores relate to physicians’ real-time observed communication skills.

To examine the relationship between physician communication, as reported by global HCAHPS scores, and the quality of physician communication skills in specific encounters, we observed hospitalist physicians during inpatient bedside rounds and measured their communication skills using the 4HCS.

 

METHODS

Study Design

The study utilized a cross sectional design; physicians who consented were observed on rounds during 3 separate encounters, and we compared hospitalists’ 4HCS scores to their HCAHPS scores to assess the correlation. The study was approved by the Institutional Review Board of the Cleveland Clinic.

Population

The study was conducted at the main campus of the Cleveland Clinic. All physicians specializing in hospital medicine who had received 10 or more completed HCAHPS survey responses while rounding on a medicine service in the past year were invited to participate in the study. Participation was voluntary; night hospitalists were excluded. A research nurse was trained in the Four Habits Model²⁸ and in the use of the 4HCS coding scheme by the principal investigator. The nurse observed each physician and ascertained the presence of communication behaviors using the 4HCS tool. Physicians were observed between August 2013 and August 2014. Multiple observations per physician could occur on the same day, but only 1 observation per patient was used for analysis. Observations consisted of a physician’s first encounter with a hospitalized patient, with the patient’s consent. Observations were conducted during encounters with English-speaking and cognitively intact patients only. Resident physicians were permitted to stay and conduct rounds per their normal routine. Patient information was not collected as part of the study.

Measures

HCAHPS. For each physician, we extracted all HCAHPS scores that were collected from our hospital’s Press Ganey database. The HCAHPS survey contains 22 core questions divided into 7 themes or domains, 1 of which is doctor communication. The survey uses frequency-based questions with possible answers fixed on a 4-point scale (4=always, 3=usually, 2=sometimes, 1=never). Our primary outcome was the doctor communication domain, which comprises 3 questions: 1) During this hospital stay, how often did the doctors treat you with respect? 2) During this hospital stay, how often did the doctors listen to you? and 3) During this hospital stay, how often did the doctors explain things in a language you can understand? Because CMS counts only the percentage of responses that are graded “always,” so-called “top box” scoring, we used the same measure.

The HCAHPS scores are always attributed to the physician at the time of discharge even if he may not have been responsible for the care of the patient during the entire hospital course. To mitigate contamination from patients seen by multiple providers, we cross-matched length of stay (LOS) data with billing data to determine the proportion of days a patient was seen by a single provider during the entire length of stay. We stratified patients seen by the attending providers to less than 50%, 50% to less than 100%, and at 100% of the LOS. However, we were unable to identify which patients were seen by other consultants or by residents due to limitations in data gathering and the nature of the database.

The Four Habits. The Four Habits are: invest in the beginning, elicit the patient’s perspective, demonstrate empathy, and invest in the end (Figure 1). Specific behaviors for Habits 1 to 4 are outlined in the Appendix, but we will briefly describe the themes as follows. Habit 1, invest in the beginning, describes the ability of the physician to set a welcoming environment for the patient, establish rapport, and collaborate on an agenda for the visit. Habit 2, elicit the patient’s perspective, describes the ability of the physician to explore the patients’ worries, ideas, expectations, and the impact of the illness on their lifestyle. Habit 3, demonstrate empathy, describes the physician’s openness to the patient’s emotions as well as the ability to explore, validate; express curiosity, and openly accept these feelings. Habit 4, invest in the end, is a measure of the physician’s ability to counsel patients in a language built around their original concerns or worries, as well as the ability to check the patients’ understanding of the plan.^2,29-30

4HCS. The 4HCS tool (Appendix) measures discreet behaviors and phrases based on each of the Four Habits (Figure 1). With a scoring range from a low of 4 to a high of 20, the rater at bedside assigns a range of points on a scale of 1 to 5 for each habit. It is an instrument based on a teaching model used widely throughout Kaiser Permanente to improve clinicians’ communication skills. The 4HCS was first tested for interrater reliability and validity against the Roter Interaction Analysis System using 100 videotaped primary care physician encounters.²⁹ It was further evaluated in a randomized control trial. Videotapes from 497 hospital encounters involving 71 doctors from a variety of clinical specialties were rated by 4 trained raters using the coding scheme. The total score Pearson’s R and intraclass correlation coefficient (ICC) exceeded 0.70 for all pairs of raters, and the interrater reliability was satisfactory for the 4HCS as applied to heterogeneous material.³⁰

 

STATISTICAL ANALYSIS

Physician characteristics were summarized with standard descriptive statistics. Pearson correlation coefficients were computed between HCAHPS and 4HCS scores. All analyses were performed with RStudio (Boston, MA). The Pearson correlation between the averaged HCAHPS and 4HCS scores was also computed. A correlation with a P value less than 0.05 was considered statistically significant. With 28 physicians, the study had a power of 88% to detect a moderate correlation (greater than 0.50) with a 2-sided alpha of 0.05. We also computed the correlations based on the subgroups of data with patients seen by providers for less than 50%, 50% to less than 100%, and 100% of LOS. All analyses were conducted in SAS 9.2 (SAS Institute Inc., Cary, NC).³⁶

RESULTS

There were 31 physicians who met our inclusion criteria. Of 29 volunteers, 28 were observed during 3 separate inpatient encounters and made up the final sample. A total of 1003 HCAHPS survey responses were available for these physicians. Participants were predominantly female (60.7%), with an average age of 39 years. They were in practice for an average of 4 years (12 were in practice more than 5 years), and 9 were observed on a teaching rotation.

The means of the overall 4HCS scores per observation were 17.39 ± 2.33 for the first, 17.00 ± 2.37 for the second, and 17.43 ± 2.36 for third bedside observation. The mean 4HCS scores per observation, broken down by habit, appear in Table 1. The ICC among the repeated scores within the same physician was 0.81. The median number of HCAHPS survey returns was 32 (range = [8, 85], with mean = 35.8, interquartile range = [16, 54]). The median overall HCAHPS doctor communication score was 89.6 (range = 80.9-93.7). Participants scored the highest in the respect subdomain and the lowest in the explain subdomain. Median HCAHPS scores and ranges appear in Table 2.

Because there were no significant associations between 4HCS scores or HCAHPS scores and physician age, sex, years in practice, or teaching site, correlations were not adjusted. Figure 2A and 2B show the association between mean 4HCS scores and HCAHPS scores by physician. There was no significant correlation between overall 4HCS and HCAHPS doctor communication scores (Pearson correlation coefficient 0.098; 95% confidence interval [CI], -0.285, 0.455). The individual habits also were not correlated with overall HCAHPS scores or with their corresponding HCAHPS domain (Table 3).

For 325 patients, 1 hospitalist was present for the entire LOS. In sensitivity analysis limiting observations to these patients (Figure 2C, Figure 2D, Table 3), we found a moderate correlation between habit 3 and the HCAHPS respect score (Pearson correlation coefficient 0.515; 95% CI, 0.176, 0.745; P = 0.005), and a weaker correlation between habit 3 and the HCAHPS overall doctor communication score (0.442; 95% CI, 0.082, 0.7; P = 0.019). There were no other significant correlations between specific habits and HCAHPS scores.

DISCUSSION

In this observational study of hospitalist physicians at a large tertiary care center, we found that communication skills, as measured by the 4HCS, varied substantially among physicians but were highly correlated within patients of the same physician. However, there was virtually no correlation between the attending physician of record’s 4HCS scores and their HCAHPS communication scores. When we limited our analysis to patients who saw only 1 hospitalist throughout their stay, there were moderate correlations between demonstration of empathy and both the HCAHPS respect score and overall doctor communication score. There were no trends across the strata of hospitalist involvement. It is important to note that the addition of even 1 different hospitalist to the LOS removes any association. Habits 1 and 2 are close to significance in the 100% subgroup, with a weak correlation. Interestingly, Habit 4, which focuses on creating a plan with the patient, showed no correlation at all with patients reporting that doctors explained things in language they could understand.

Development and testing of the HCAHPS survey began in 2002, commissioned by CMS and the Agency for Healthcare Research and Quality for the purpose of measuring patient experience in the hospital. The HCAHPS survey was endorsed by the National Quality Forum in 2005, with final approval of the national implementation granted by the Office of Management and Budget later that year. The CMS began implementation of the HCAHPS survey in 2006, with the first required public reporting of all hospitals taking place in March 2008.^37-41 Based on CMS’ value-based purchasing initiative, hospitals with low HCAHPS scores have faced substantial penalties since 2012. Under these circumstances, it is important that the HCAHPS measures what it purports to measure. Because HCAHPS was designed to compare hospitals, testing was limited to assessment of internal reliability, hospital-level reliability, and construct validity. External validation with known measures of physician communication was not performed.⁴¹ Our study appears to be the first to compare HCAHPS scores to directly observed measures of physician communication skills. The lack of association between the 2 should sound a cautionary note to hospitals who seek to tie individual compensation to HCAHPS scores to improve them. In particular, the survey asks for a rating for all the patient’s doctors, not just the primary hospitalist. We found that, for hospital stays with just 1 hospitalist, the HCAHPS score reflected observed expression of empathy, although the correlation was only moderate, and HCAHPS were not correlated with other communication skills. Of all communication skills, empathy may be most important. Almost the entire body of research on physician communication cites empathy as a central skill. Empathy improves patient outcomes^{1-9,13-14,16-18,42} such as adherence to treatment, loyalty, and perception of care; and provider outcomes^10-12,15 such as reduced burnout and a decreased likelihood of malpractice litigation.

It is less clear why other communication skills did not correlate with HCAHPS, but several differences in the measures themselves and how they were obtained might be responsible. It is possible that HCAHPS measures something broader than physician communication. In addition, the 4HCS was developed and normed on outpatient encounters as is true for virtually all doctor-patient coding schemes.⁴³ Little is known about inpatient communication best practices. The timing of HCAHPS may also degrade the relationship between observed and reported communication. The HCAHPS questionnaires, collected after discharge, are retrospective reconstructions that are subject to recall bias and recency effects.^44,45 In contrast, our observations took place in real time and were specific to the face-to-face interactions that take place when physicians engage patients at the bedside. Third, the response rate for HCAHPS surveys is only 30%, leading to potential sample bias.⁴⁶ Respondents represent discharged patients who are willing and able to answer surveys, and may not be representative of all hospitalized patients. Finally, as with all global questions, the meaning any individual patient assigns to terms like “respect” may vary.

Our study has several limitations. The HCAHPS and 4HCS scores were not obtained from the same sample of patients. It is possible that the patients who were observed were not representative of the patients who completed the HCAHPS surveys. In addition, the only type of encounter observed was the initial visit between the hospitalist and the patient, and did not include communication during follow-up visits or on the day of discharge. However, there was a strong ICC among the 4HCS scores, implying that the 4HCS measures an inherent physician skill, which should be consistent across patients and encounters. Coding bias of the habits by a single observer could not be excluded. High intra-class correlation could be due in part to observer preferences for particular communication styles. Our sample included only 28 physicians. Although our study was powered to rule out a moderate correlation between 4HCS scores and HCAHPS scores (Pearson correlation coefficient greater than 0.5), we cannot exclude weaker correlations. Most correlations that we observed were so small that they would not be clinically meaningful, even in a much larger sample.

 

CONCLUSIONS

Our findings that HCAHPS scores did not correlate with the communication skills of the attending of record have some important implications. In an environment of value-based purchasing, most hospital systems are interested in identifying modifiable provider behaviors that optimize efficiency and payment structures. This study shows that directly measured communication skills do not correlate with HCAHPS scores as generally reported, indicating that HCAHPS may be measuring a broader domain than only physician communication skills. Better attribution based on the proportion of care provided by an individual physician could make the scores more useful for individual comparisons, but most institutions do not report their data in this way. Given this limitation, hospitals should refrain from comparing and incentivizing individual physicians based on their HCAHPS scores, because this measure was not designed for this purpose and does not appear to reflect an individual’s skills. This is important in the current environment in which hospitals face substantial penalties for underperformance but lack specific tools to improve their scores. Furthermore, there is concern that this type of measurement creates perverse incentives that may adversely alter clinical practice with the aim of improving scores.⁴⁶

Training clinicians in communication and teaming skills is one potential means of increasing overall scores.¹⁵ Improving doctor-patient and team relationships is also the right thing to do. It is increasingly being demanded by patients and has always been a deep source of satisfaction for physicians.^15,47 Moreover, there is an increasingly robust literature that relates face-to-face communication to biomedical and psychosocial outcomes of care.⁴⁸ Identifying individual physicians who need help with communication skills is a worthwhile goal. Unfortunately, the HCAHPS survey does not appear to be the appropriate tool for this purpose.

Disclosure

The Cleveland Clinic Foundation, Division of Clinical Research, Research Programs Committees provided funding support. No funding source had any role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the article for publication. The authors have no conflicts of interest for this study.

 

Communication is the foundation of medical care.¹ Effective communication can improve health outcomes, safety, adherence, satisfaction, trust, and enable genuine informed consent and decision-making.^2-9 Furthermore, high-quality communication increases provider engagement and workplace satisfaction, while reducing stress and malpractice risk.^10-15

Direct measurement of communication in the healthcare setting can be challenging. The “Four Habits Model,” which is derived from a synthesis of empiric studies^8,16-20 and theoretical models^21-24 of communication, offers 1 framework for assessing healthcare communication. The conceptual model underlying the 4 habits has been validated in studies of physician and patient satisfaction.^1,4,25-27 The 4 habits are: investing in the beginning, eliciting the patient’s perspective, demonstrating empathy, and investing in the end. Each habit is divided into several identifiable tasks or skill sets, which can be reliably measured using validated tools and checklists.²⁸ One such instrument, the Four Habits Coding Scheme (4HCS), has been evaluated against other tools and demonstrated overall satisfactory inter-rater reliability and validity.^29,30

The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey, developed under the direction of the Centers for Medicare and Medicaid Services (CMS) and the Agency for Healthcare Research and Quality, is an established national standard for measuring patient perceptions of care. HCAHPS retrospectively measures global perceptions of communication, support and empathy from physicians and staff, processes of care, and the overall patient experience. HCAHPS scores were first collected nationally in 2006 and have been publicly reported since 2008.³¹ With the introduction of value-based purchasing in 2012, health system revenues are now tied to HCAHPS survey performance.³² As a result, hospitals are financially motivated to improve HCAHPS scores but lack evidence-based methods for doing so. Some healthcare organizations have invested in communication training programs based on the available literature and best practices.^2,33-35 However, it is not known how, if at all, HCAHPS scores relate to physicians’ real-time observed communication skills.

To examine the relationship between physician communication, as reported by global HCAHPS scores, and the quality of physician communication skills in specific encounters, we observed hospitalist physicians during inpatient bedside rounds and measured their communication skills using the 4HCS.

 

METHODS

Study Design

The study utilized a cross sectional design; physicians who consented were observed on rounds during 3 separate encounters, and we compared hospitalists’ 4HCS scores to their HCAHPS scores to assess the correlation. The study was approved by the Institutional Review Board of the Cleveland Clinic.

Population

The study was conducted at the main campus of the Cleveland Clinic. All physicians specializing in hospital medicine who had received 10 or more completed HCAHPS survey responses while rounding on a medicine service in the past year were invited to participate in the study. Participation was voluntary; night hospitalists were excluded. A research nurse was trained in the Four Habits Model²⁸ and in the use of the 4HCS coding scheme by the principal investigator. The nurse observed each physician and ascertained the presence of communication behaviors using the 4HCS tool. Physicians were observed between August 2013 and August 2014. Multiple observations per physician could occur on the same day, but only 1 observation per patient was used for analysis. Observations consisted of a physician’s first encounter with a hospitalized patient, with the patient’s consent. Observations were conducted during encounters with English-speaking and cognitively intact patients only. Resident physicians were permitted to stay and conduct rounds per their normal routine. Patient information was not collected as part of the study.

Measures

HCAHPS. For each physician, we extracted all HCAHPS scores that were collected from our hospital’s Press Ganey database. The HCAHPS survey contains 22 core questions divided into 7 themes or domains, 1 of which is doctor communication. The survey uses frequency-based questions with possible answers fixed on a 4-point scale (4=always, 3=usually, 2=sometimes, 1=never). Our primary outcome was the doctor communication domain, which comprises 3 questions: 1) During this hospital stay, how often did the doctors treat you with respect? 2) During this hospital stay, how often did the doctors listen to you? and 3) During this hospital stay, how often did the doctors explain things in a language you can understand? Because CMS counts only the percentage of responses that are graded “always,” so-called “top box” scoring, we used the same measure.

The HCAHPS scores are always attributed to the physician at the time of discharge even if he may not have been responsible for the care of the patient during the entire hospital course. To mitigate contamination from patients seen by multiple providers, we cross-matched length of stay (LOS) data with billing data to determine the proportion of days a patient was seen by a single provider during the entire length of stay. We stratified patients seen by the attending providers to less than 50%, 50% to less than 100%, and at 100% of the LOS. However, we were unable to identify which patients were seen by other consultants or by residents due to limitations in data gathering and the nature of the database.

The Four Habits. The Four Habits are: invest in the beginning, elicit the patient’s perspective, demonstrate empathy, and invest in the end (Figure 1). Specific behaviors for Habits 1 to 4 are outlined in the Appendix, but we will briefly describe the themes as follows. Habit 1, invest in the beginning, describes the ability of the physician to set a welcoming environment for the patient, establish rapport, and collaborate on an agenda for the visit. Habit 2, elicit the patient’s perspective, describes the ability of the physician to explore the patients’ worries, ideas, expectations, and the impact of the illness on their lifestyle. Habit 3, demonstrate empathy, describes the physician’s openness to the patient’s emotions as well as the ability to explore, validate; express curiosity, and openly accept these feelings. Habit 4, invest in the end, is a measure of the physician’s ability to counsel patients in a language built around their original concerns or worries, as well as the ability to check the patients’ understanding of the plan.^2,29-30

4HCS. The 4HCS tool (Appendix) measures discreet behaviors and phrases based on each of the Four Habits (Figure 1). With a scoring range from a low of 4 to a high of 20, the rater at bedside assigns a range of points on a scale of 1 to 5 for each habit. It is an instrument based on a teaching model used widely throughout Kaiser Permanente to improve clinicians’ communication skills. The 4HCS was first tested for interrater reliability and validity against the Roter Interaction Analysis System using 100 videotaped primary care physician encounters.²⁹ It was further evaluated in a randomized control trial. Videotapes from 497 hospital encounters involving 71 doctors from a variety of clinical specialties were rated by 4 trained raters using the coding scheme. The total score Pearson’s R and intraclass correlation coefficient (ICC) exceeded 0.70 for all pairs of raters, and the interrater reliability was satisfactory for the 4HCS as applied to heterogeneous material.³⁰

 

STATISTICAL ANALYSIS

Physician characteristics were summarized with standard descriptive statistics. Pearson correlation coefficients were computed between HCAHPS and 4HCS scores. All analyses were performed with RStudio (Boston, MA). The Pearson correlation between the averaged HCAHPS and 4HCS scores was also computed. A correlation with a P value less than 0.05 was considered statistically significant. With 28 physicians, the study had a power of 88% to detect a moderate correlation (greater than 0.50) with a 2-sided alpha of 0.05. We also computed the correlations based on the subgroups of data with patients seen by providers for less than 50%, 50% to less than 100%, and 100% of LOS. All analyses were conducted in SAS 9.2 (SAS Institute Inc., Cary, NC).³⁶

RESULTS

There were 31 physicians who met our inclusion criteria. Of 29 volunteers, 28 were observed during 3 separate inpatient encounters and made up the final sample. A total of 1003 HCAHPS survey responses were available for these physicians. Participants were predominantly female (60.7%), with an average age of 39 years. They were in practice for an average of 4 years (12 were in practice more than 5 years), and 9 were observed on a teaching rotation.

The means of the overall 4HCS scores per observation were 17.39 ± 2.33 for the first, 17.00 ± 2.37 for the second, and 17.43 ± 2.36 for third bedside observation. The mean 4HCS scores per observation, broken down by habit, appear in Table 1. The ICC among the repeated scores within the same physician was 0.81. The median number of HCAHPS survey returns was 32 (range = [8, 85], with mean = 35.8, interquartile range = [16, 54]). The median overall HCAHPS doctor communication score was 89.6 (range = 80.9-93.7). Participants scored the highest in the respect subdomain and the lowest in the explain subdomain. Median HCAHPS scores and ranges appear in Table 2.

Because there were no significant associations between 4HCS scores or HCAHPS scores and physician age, sex, years in practice, or teaching site, correlations were not adjusted. Figure 2A and 2B show the association between mean 4HCS scores and HCAHPS scores by physician. There was no significant correlation between overall 4HCS and HCAHPS doctor communication scores (Pearson correlation coefficient 0.098; 95% confidence interval [CI], -0.285, 0.455). The individual habits also were not correlated with overall HCAHPS scores or with their corresponding HCAHPS domain (Table 3).

For 325 patients, 1 hospitalist was present for the entire LOS. In sensitivity analysis limiting observations to these patients (Figure 2C, Figure 2D, Table 3), we found a moderate correlation between habit 3 and the HCAHPS respect score (Pearson correlation coefficient 0.515; 95% CI, 0.176, 0.745; P = 0.005), and a weaker correlation between habit 3 and the HCAHPS overall doctor communication score (0.442; 95% CI, 0.082, 0.7; P = 0.019). There were no other significant correlations between specific habits and HCAHPS scores.

DISCUSSION

In this observational study of hospitalist physicians at a large tertiary care center, we found that communication skills, as measured by the 4HCS, varied substantially among physicians but were highly correlated within patients of the same physician. However, there was virtually no correlation between the attending physician of record’s 4HCS scores and their HCAHPS communication scores. When we limited our analysis to patients who saw only 1 hospitalist throughout their stay, there were moderate correlations between demonstration of empathy and both the HCAHPS respect score and overall doctor communication score. There were no trends across the strata of hospitalist involvement. It is important to note that the addition of even 1 different hospitalist to the LOS removes any association. Habits 1 and 2 are close to significance in the 100% subgroup, with a weak correlation. Interestingly, Habit 4, which focuses on creating a plan with the patient, showed no correlation at all with patients reporting that doctors explained things in language they could understand.

Development and testing of the HCAHPS survey began in 2002, commissioned by CMS and the Agency for Healthcare Research and Quality for the purpose of measuring patient experience in the hospital. The HCAHPS survey was endorsed by the National Quality Forum in 2005, with final approval of the national implementation granted by the Office of Management and Budget later that year. The CMS began implementation of the HCAHPS survey in 2006, with the first required public reporting of all hospitals taking place in March 2008.^37-41 Based on CMS’ value-based purchasing initiative, hospitals with low HCAHPS scores have faced substantial penalties since 2012. Under these circumstances, it is important that the HCAHPS measures what it purports to measure. Because HCAHPS was designed to compare hospitals, testing was limited to assessment of internal reliability, hospital-level reliability, and construct validity. External validation with known measures of physician communication was not performed.⁴¹ Our study appears to be the first to compare HCAHPS scores to directly observed measures of physician communication skills. The lack of association between the 2 should sound a cautionary note to hospitals who seek to tie individual compensation to HCAHPS scores to improve them. In particular, the survey asks for a rating for all the patient’s doctors, not just the primary hospitalist. We found that, for hospital stays with just 1 hospitalist, the HCAHPS score reflected observed expression of empathy, although the correlation was only moderate, and HCAHPS were not correlated with other communication skills. Of all communication skills, empathy may be most important. Almost the entire body of research on physician communication cites empathy as a central skill. Empathy improves patient outcomes^{1-9,13-14,16-18,42} such as adherence to treatment, loyalty, and perception of care; and provider outcomes^10-12,15 such as reduced burnout and a decreased likelihood of malpractice litigation.

It is less clear why other communication skills did not correlate with HCAHPS, but several differences in the measures themselves and how they were obtained might be responsible. It is possible that HCAHPS measures something broader than physician communication. In addition, the 4HCS was developed and normed on outpatient encounters as is true for virtually all doctor-patient coding schemes.⁴³ Little is known about inpatient communication best practices. The timing of HCAHPS may also degrade the relationship between observed and reported communication. The HCAHPS questionnaires, collected after discharge, are retrospective reconstructions that are subject to recall bias and recency effects.^44,45 In contrast, our observations took place in real time and were specific to the face-to-face interactions that take place when physicians engage patients at the bedside. Third, the response rate for HCAHPS surveys is only 30%, leading to potential sample bias.⁴⁶ Respondents represent discharged patients who are willing and able to answer surveys, and may not be representative of all hospitalized patients. Finally, as with all global questions, the meaning any individual patient assigns to terms like “respect” may vary.

Our study has several limitations. The HCAHPS and 4HCS scores were not obtained from the same sample of patients. It is possible that the patients who were observed were not representative of the patients who completed the HCAHPS surveys. In addition, the only type of encounter observed was the initial visit between the hospitalist and the patient, and did not include communication during follow-up visits or on the day of discharge. However, there was a strong ICC among the 4HCS scores, implying that the 4HCS measures an inherent physician skill, which should be consistent across patients and encounters. Coding bias of the habits by a single observer could not be excluded. High intra-class correlation could be due in part to observer preferences for particular communication styles. Our sample included only 28 physicians. Although our study was powered to rule out a moderate correlation between 4HCS scores and HCAHPS scores (Pearson correlation coefficient greater than 0.5), we cannot exclude weaker correlations. Most correlations that we observed were so small that they would not be clinically meaningful, even in a much larger sample.

 

CONCLUSIONS

Our findings that HCAHPS scores did not correlate with the communication skills of the attending of record have some important implications. In an environment of value-based purchasing, most hospital systems are interested in identifying modifiable provider behaviors that optimize efficiency and payment structures. This study shows that directly measured communication skills do not correlate with HCAHPS scores as generally reported, indicating that HCAHPS may be measuring a broader domain than only physician communication skills. Better attribution based on the proportion of care provided by an individual physician could make the scores more useful for individual comparisons, but most institutions do not report their data in this way. Given this limitation, hospitals should refrain from comparing and incentivizing individual physicians based on their HCAHPS scores, because this measure was not designed for this purpose and does not appear to reflect an individual’s skills. This is important in the current environment in which hospitals face substantial penalties for underperformance but lack specific tools to improve their scores. Furthermore, there is concern that this type of measurement creates perverse incentives that may adversely alter clinical practice with the aim of improving scores.⁴⁶

Training clinicians in communication and teaming skills is one potential means of increasing overall scores.¹⁵ Improving doctor-patient and team relationships is also the right thing to do. It is increasingly being demanded by patients and has always been a deep source of satisfaction for physicians.^15,47 Moreover, there is an increasingly robust literature that relates face-to-face communication to biomedical and psychosocial outcomes of care.⁴⁸ Identifying individual physicians who need help with communication skills is a worthwhile goal. Unfortunately, the HCAHPS survey does not appear to be the appropriate tool for this purpose.

Disclosure

The Cleveland Clinic Foundation, Division of Clinical Research, Research Programs Committees provided funding support. No funding source had any role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the article for publication. The authors have no conflicts of interest for this study.

 

Communication is the foundation of medical care.¹ Effective communication can improve health outcomes, safety, adherence, satisfaction, trust, and enable genuine informed consent and decision-making.^2-9 Furthermore, high-quality communication increases provider engagement and workplace satisfaction, while reducing stress and malpractice risk.^10-15

Direct measurement of communication in the healthcare setting can be challenging. The “Four Habits Model,” which is derived from a synthesis of empiric studies^8,16-20 and theoretical models^21-24 of communication, offers 1 framework for assessing healthcare communication. The conceptual model underlying the 4 habits has been validated in studies of physician and patient satisfaction.^1,4,25-27 The 4 habits are: investing in the beginning, eliciting the patient’s perspective, demonstrating empathy, and investing in the end. Each habit is divided into several identifiable tasks or skill sets, which can be reliably measured using validated tools and checklists.²⁸ One such instrument, the Four Habits Coding Scheme (4HCS), has been evaluated against other tools and demonstrated overall satisfactory inter-rater reliability and validity.^29,30

The Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey, developed under the direction of the Centers for Medicare and Medicaid Services (CMS) and the Agency for Healthcare Research and Quality, is an established national standard for measuring patient perceptions of care. HCAHPS retrospectively measures global perceptions of communication, support and empathy from physicians and staff, processes of care, and the overall patient experience. HCAHPS scores were first collected nationally in 2006 and have been publicly reported since 2008.³¹ With the introduction of value-based purchasing in 2012, health system revenues are now tied to HCAHPS survey performance.³² As a result, hospitals are financially motivated to improve HCAHPS scores but lack evidence-based methods for doing so. Some healthcare organizations have invested in communication training programs based on the available literature and best practices.^2,33-35 However, it is not known how, if at all, HCAHPS scores relate to physicians’ real-time observed communication skills.

To examine the relationship between physician communication, as reported by global HCAHPS scores, and the quality of physician communication skills in specific encounters, we observed hospitalist physicians during inpatient bedside rounds and measured their communication skills using the 4HCS.

 

METHODS

Study Design

The study utilized a cross sectional design; physicians who consented were observed on rounds during 3 separate encounters, and we compared hospitalists’ 4HCS scores to their HCAHPS scores to assess the correlation. The study was approved by the Institutional Review Board of the Cleveland Clinic.

Population

The study was conducted at the main campus of the Cleveland Clinic. All physicians specializing in hospital medicine who had received 10 or more completed HCAHPS survey responses while rounding on a medicine service in the past year were invited to participate in the study. Participation was voluntary; night hospitalists were excluded. A research nurse was trained in the Four Habits Model²⁸ and in the use of the 4HCS coding scheme by the principal investigator. The nurse observed each physician and ascertained the presence of communication behaviors using the 4HCS tool. Physicians were observed between August 2013 and August 2014. Multiple observations per physician could occur on the same day, but only 1 observation per patient was used for analysis. Observations consisted of a physician’s first encounter with a hospitalized patient, with the patient’s consent. Observations were conducted during encounters with English-speaking and cognitively intact patients only. Resident physicians were permitted to stay and conduct rounds per their normal routine. Patient information was not collected as part of the study.

Measures

HCAHPS. For each physician, we extracted all HCAHPS scores that were collected from our hospital’s Press Ganey database. The HCAHPS survey contains 22 core questions divided into 7 themes or domains, 1 of which is doctor communication. The survey uses frequency-based questions with possible answers fixed on a 4-point scale (4=always, 3=usually, 2=sometimes, 1=never). Our primary outcome was the doctor communication domain, which comprises 3 questions: 1) During this hospital stay, how often did the doctors treat you with respect? 2) During this hospital stay, how often did the doctors listen to you? and 3) During this hospital stay, how often did the doctors explain things in a language you can understand? Because CMS counts only the percentage of responses that are graded “always,” so-called “top box” scoring, we used the same measure.

The HCAHPS scores are always attributed to the physician at the time of discharge even if he may not have been responsible for the care of the patient during the entire hospital course. To mitigate contamination from patients seen by multiple providers, we cross-matched length of stay (LOS) data with billing data to determine the proportion of days a patient was seen by a single provider during the entire length of stay. We stratified patients seen by the attending providers to less than 50%, 50% to less than 100%, and at 100% of the LOS. However, we were unable to identify which patients were seen by other consultants or by residents due to limitations in data gathering and the nature of the database.

The Four Habits. The Four Habits are: invest in the beginning, elicit the patient’s perspective, demonstrate empathy, and invest in the end (Figure 1). Specific behaviors for Habits 1 to 4 are outlined in the Appendix, but we will briefly describe the themes as follows. Habit 1, invest in the beginning, describes the ability of the physician to set a welcoming environment for the patient, establish rapport, and collaborate on an agenda for the visit. Habit 2, elicit the patient’s perspective, describes the ability of the physician to explore the patients’ worries, ideas, expectations, and the impact of the illness on their lifestyle. Habit 3, demonstrate empathy, describes the physician’s openness to the patient’s emotions as well as the ability to explore, validate; express curiosity, and openly accept these feelings. Habit 4, invest in the end, is a measure of the physician’s ability to counsel patients in a language built around their original concerns or worries, as well as the ability to check the patients’ understanding of the plan.^2,29-30

4HCS. The 4HCS tool (Appendix) measures discreet behaviors and phrases based on each of the Four Habits (Figure 1). With a scoring range from a low of 4 to a high of 20, the rater at bedside assigns a range of points on a scale of 1 to 5 for each habit. It is an instrument based on a teaching model used widely throughout Kaiser Permanente to improve clinicians’ communication skills. The 4HCS was first tested for interrater reliability and validity against the Roter Interaction Analysis System using 100 videotaped primary care physician encounters.²⁹ It was further evaluated in a randomized control trial. Videotapes from 497 hospital encounters involving 71 doctors from a variety of clinical specialties were rated by 4 trained raters using the coding scheme. The total score Pearson’s R and intraclass correlation coefficient (ICC) exceeded 0.70 for all pairs of raters, and the interrater reliability was satisfactory for the 4HCS as applied to heterogeneous material.³⁰

 

STATISTICAL ANALYSIS

Physician characteristics were summarized with standard descriptive statistics. Pearson correlation coefficients were computed between HCAHPS and 4HCS scores. All analyses were performed with RStudio (Boston, MA). The Pearson correlation between the averaged HCAHPS and 4HCS scores was also computed. A correlation with a P value less than 0.05 was considered statistically significant. With 28 physicians, the study had a power of 88% to detect a moderate correlation (greater than 0.50) with a 2-sided alpha of 0.05. We also computed the correlations based on the subgroups of data with patients seen by providers for less than 50%, 50% to less than 100%, and 100% of LOS. All analyses were conducted in SAS 9.2 (SAS Institute Inc., Cary, NC).³⁶

RESULTS

There were 31 physicians who met our inclusion criteria. Of 29 volunteers, 28 were observed during 3 separate inpatient encounters and made up the final sample. A total of 1003 HCAHPS survey responses were available for these physicians. Participants were predominantly female (60.7%), with an average age of 39 years. They were in practice for an average of 4 years (12 were in practice more than 5 years), and 9 were observed on a teaching rotation.

The means of the overall 4HCS scores per observation were 17.39 ± 2.33 for the first, 17.00 ± 2.37 for the second, and 17.43 ± 2.36 for third bedside observation. The mean 4HCS scores per observation, broken down by habit, appear in Table 1. The ICC among the repeated scores within the same physician was 0.81. The median number of HCAHPS survey returns was 32 (range = [8, 85], with mean = 35.8, interquartile range = [16, 54]). The median overall HCAHPS doctor communication score was 89.6 (range = 80.9-93.7). Participants scored the highest in the respect subdomain and the lowest in the explain subdomain. Median HCAHPS scores and ranges appear in Table 2.

Because there were no significant associations between 4HCS scores or HCAHPS scores and physician age, sex, years in practice, or teaching site, correlations were not adjusted. Figure 2A and 2B show the association between mean 4HCS scores and HCAHPS scores by physician. There was no significant correlation between overall 4HCS and HCAHPS doctor communication scores (Pearson correlation coefficient 0.098; 95% confidence interval [CI], -0.285, 0.455). The individual habits also were not correlated with overall HCAHPS scores or with their corresponding HCAHPS domain (Table 3).

For 325 patients, 1 hospitalist was present for the entire LOS. In sensitivity analysis limiting observations to these patients (Figure 2C, Figure 2D, Table 3), we found a moderate correlation between habit 3 and the HCAHPS respect score (Pearson correlation coefficient 0.515; 95% CI, 0.176, 0.745; P = 0.005), and a weaker correlation between habit 3 and the HCAHPS overall doctor communication score (0.442; 95% CI, 0.082, 0.7; P = 0.019). There were no other significant correlations between specific habits and HCAHPS scores.

DISCUSSION

In this observational study of hospitalist physicians at a large tertiary care center, we found that communication skills, as measured by the 4HCS, varied substantially among physicians but were highly correlated within patients of the same physician. However, there was virtually no correlation between the attending physician of record’s 4HCS scores and their HCAHPS communication scores. When we limited our analysis to patients who saw only 1 hospitalist throughout their stay, there were moderate correlations between demonstration of empathy and both the HCAHPS respect score and overall doctor communication score. There were no trends across the strata of hospitalist involvement. It is important to note that the addition of even 1 different hospitalist to the LOS removes any association. Habits 1 and 2 are close to significance in the 100% subgroup, with a weak correlation. Interestingly, Habit 4, which focuses on creating a plan with the patient, showed no correlation at all with patients reporting that doctors explained things in language they could understand.

Development and testing of the HCAHPS survey began in 2002, commissioned by CMS and the Agency for Healthcare Research and Quality for the purpose of measuring patient experience in the hospital. The HCAHPS survey was endorsed by the National Quality Forum in 2005, with final approval of the national implementation granted by the Office of Management and Budget later that year. The CMS began implementation of the HCAHPS survey in 2006, with the first required public reporting of all hospitals taking place in March 2008.^37-41 Based on CMS’ value-based purchasing initiative, hospitals with low HCAHPS scores have faced substantial penalties since 2012. Under these circumstances, it is important that the HCAHPS measures what it purports to measure. Because HCAHPS was designed to compare hospitals, testing was limited to assessment of internal reliability, hospital-level reliability, and construct validity. External validation with known measures of physician communication was not performed.⁴¹ Our study appears to be the first to compare HCAHPS scores to directly observed measures of physician communication skills. The lack of association between the 2 should sound a cautionary note to hospitals who seek to tie individual compensation to HCAHPS scores to improve them. In particular, the survey asks for a rating for all the patient’s doctors, not just the primary hospitalist. We found that, for hospital stays with just 1 hospitalist, the HCAHPS score reflected observed expression of empathy, although the correlation was only moderate, and HCAHPS were not correlated with other communication skills. Of all communication skills, empathy may be most important. Almost the entire body of research on physician communication cites empathy as a central skill. Empathy improves patient outcomes^{1-9,13-14,16-18,42} such as adherence to treatment, loyalty, and perception of care; and provider outcomes^10-12,15 such as reduced burnout and a decreased likelihood of malpractice litigation.

It is less clear why other communication skills did not correlate with HCAHPS, but several differences in the measures themselves and how they were obtained might be responsible. It is possible that HCAHPS measures something broader than physician communication. In addition, the 4HCS was developed and normed on outpatient encounters as is true for virtually all doctor-patient coding schemes.⁴³ Little is known about inpatient communication best practices. The timing of HCAHPS may also degrade the relationship between observed and reported communication. The HCAHPS questionnaires, collected after discharge, are retrospective reconstructions that are subject to recall bias and recency effects.^44,45 In contrast, our observations took place in real time and were specific to the face-to-face interactions that take place when physicians engage patients at the bedside. Third, the response rate for HCAHPS surveys is only 30%, leading to potential sample bias.⁴⁶ Respondents represent discharged patients who are willing and able to answer surveys, and may not be representative of all hospitalized patients. Finally, as with all global questions, the meaning any individual patient assigns to terms like “respect” may vary.

Our study has several limitations. The HCAHPS and 4HCS scores were not obtained from the same sample of patients. It is possible that the patients who were observed were not representative of the patients who completed the HCAHPS surveys. In addition, the only type of encounter observed was the initial visit between the hospitalist and the patient, and did not include communication during follow-up visits or on the day of discharge. However, there was a strong ICC among the 4HCS scores, implying that the 4HCS measures an inherent physician skill, which should be consistent across patients and encounters. Coding bias of the habits by a single observer could not be excluded. High intra-class correlation could be due in part to observer preferences for particular communication styles. Our sample included only 28 physicians. Although our study was powered to rule out a moderate correlation between 4HCS scores and HCAHPS scores (Pearson correlation coefficient greater than 0.5), we cannot exclude weaker correlations. Most correlations that we observed were so small that they would not be clinically meaningful, even in a much larger sample.

 

CONCLUSIONS

Our findings that HCAHPS scores did not correlate with the communication skills of the attending of record have some important implications. In an environment of value-based purchasing, most hospital systems are interested in identifying modifiable provider behaviors that optimize efficiency and payment structures. This study shows that directly measured communication skills do not correlate with HCAHPS scores as generally reported, indicating that HCAHPS may be measuring a broader domain than only physician communication skills. Better attribution based on the proportion of care provided by an individual physician could make the scores more useful for individual comparisons, but most institutions do not report their data in this way. Given this limitation, hospitals should refrain from comparing and incentivizing individual physicians based on their HCAHPS scores, because this measure was not designed for this purpose and does not appear to reflect an individual’s skills. This is important in the current environment in which hospitals face substantial penalties for underperformance but lack specific tools to improve their scores. Furthermore, there is concern that this type of measurement creates perverse incentives that may adversely alter clinical practice with the aim of improving scores.⁴⁶

Training clinicians in communication and teaming skills is one potential means of increasing overall scores.¹⁵ Improving doctor-patient and team relationships is also the right thing to do. It is increasingly being demanded by patients and has always been a deep source of satisfaction for physicians.^15,47 Moreover, there is an increasingly robust literature that relates face-to-face communication to biomedical and psychosocial outcomes of care.⁴⁸ Identifying individual physicians who need help with communication skills is a worthwhile goal. Unfortunately, the HCAHPS survey does not appear to be the appropriate tool for this purpose.

Disclosure

The Cleveland Clinic Foundation, Division of Clinical Research, Research Programs Committees provided funding support. No funding source had any role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the article for publication. The authors have no conflicts of interest for this study.

 

Chronic opioid and benzodiazepine use is common and increasing.^1-5 Outpatient use of these medications has been associated with hospital readmission and death,^6-12 with concurrent use associated with particularly increased risk.^13,14 Less is known about outcomes for hospitalized patients receiving these medications.

More than half of hospital inpatients in the United States receive opioids,¹⁵ many of which are new prescriptions rather than continuation of chronic therapy.^16,17 Less is known about inpatient benzodiazepine administration, but the prevalence may exceed 10% among elderly populations.¹⁸ Hospitalized patients often have comorbidities or physiological disturbances that might increase their risk related to use of these medications. Opioids can cause central and obstructive sleep apneas,^19-21 and benzodiazepines contribute to respiratory depression and airway relaxation.²² Benzodiazepines also impair psychomotor function and recall,²³ which could mediate the recognized risk for delirium and falls in the hospital.^24,25 These findings suggest pathways by which these medications might contribute to clinical deterioration.

Most studies in hospitalized patients have been limited to specific populations^15,26-28 and have not explicitly controlled for severity of illness over time. It remains unclear whether associations identified within particular groups of patients hold true for the broader population of general ward inpatients. Therefore, we aimed to determine the independent association between opioid and benzodiazepine administration and clinical deterioration in ward patients.

MATERIALS AND METHODS

Setting and Study Population

We performed an observational cohort study at a 500-bed urban academic hospital. Data were obtained from all adults hospitalized on the wards between November 1, 2008, and January 21, 2016. The study protocol was approved by the University of Chicago Institutional Review Board (IRB#15-0195).

Data Collection

The study utilized de-identified data from the electronic health record (EHR; Epic Systems Corporation, Verona, Wisconsin) and administrative databases collected by the University of Chicago Clinical Research Data Warehouse. Patient age, sex, race, body mass index (BMI), and ward admission source (ie, emergency department (ED), transferred from the intensive care unit (ICU), or directly admitted to the wards) were collected. International Classification of Diseases, 9^th Revision, Clinical Modification (ICD-9-CM) codes were used to identify Elixhauser Comorbidity Index categories.^29,30 Because patients with similar diagnoses (eg, active cancer) are cohorted within particular areas in our hospital, we obtained the ward unit for all patients. Patients who underwent surgery were identified using the hospital’s admission-transfer-discharge database.

 

To determine severity of illness, routinely collected vital signs and laboratory values were utilized to calculate the electronic cardiac arrest risk triage (eCART) score, an accurate risk score we previously developed and validated for predicting adverse events among ward patients.³¹ If any vital sign or laboratory value was missing, the next available measurement was carried forward. If any value remained missing after this change, the median value for that location (ie, wards, ICU, or ED) was imputed.^32,33 Additionally, patient-reported pain scores at the time of opioid administration were extracted from nursing flowsheets. If no pain score was present at the time of opioid administration, the patient’s previous score was carried forward.

We excluded patients with sickle-cell disease or seizure history and admissions with diagnoses of alcohol withdrawal from the analysis, because these diagnoses were expected to be associated with different medication administration practices compared to other inpatients. We also excluded patients with a tracheostomy because we expected their respiratory monitoring to differ from the other patients in our cohort. Finally, because ward deaths resulting from a comfort care scenario often involve opioids and/or benzodiazepines, ward segments involving comfort care deaths (defined as death without attempted resuscitation) were excluded from the analysis (Supplemental Figure 1). Patients with sickle-cell disease were identified using ICD-9 codes, and encounters during which a seizure may have occurred were identified using a combination of ICD-9 codes and receipt of anti-epileptic medication (Supplemental Table 1). Patients at risk for alcohol withdrawal were identified by the presence of any Clinical Institute Withdrawal Assessment for Alcohol score within nursing flowsheets, and patients with tracheostomies were identified using documentation of ventilator support within their first 12 hours on the wards. In addition to these exclusion criteria, patients with obstructive sleep apnea (OSA) were identified by the following ICD-9 codes: 278.03, 327.23, 780.51, 780.53, and 780.57.

Medications

Ward administrations of opioids and benzodiazepines—dose, route, and administration time—were collected from the EHR. We excluded all administrations in nonward locations such as the ED, ICU, operating room, or procedure suite. Additionally, because patients emergently intubated may receive sedative and analgesic medications to facilitate intubation, and because patients experiencing cardiac arrest are frequently intubated periresuscitation, we a priori excluded all administrations within 15 minutes of a ward cardiac arrest or an intubation.

For consistent comparisons, opioid doses were converted to oral morphine equivalents³⁴ and adjusted by a factor of 15 to reflect the smallest routinely available oral morphine tablet in our hospital (Supplemental Table 2). Benzodiazepine doses were converted to oral lorazepam equivalents (Supplemental Table 2).³⁴ Thus, the independent variables were oral morphine or lorazepam equivalents administered within each 6-hour window. We a priori presumed opioid doses greater than the 99^th percentile (1200 mg) or benzodiazepine doses greater than 10 mg oral lorazepam equivalents within a 6-hour window to be erroneous entries, and replaced these outlier values with the median value for each medication category.

Outcomes

The primary outcome was the composite of ICU transfer or cardiac arrest (loss of pulse with attempted resuscitation) on the wards, with individual outcomes investigated secondarily. An ICU transfer (patient movement from a ward directly to the ICU) was identified using the hospital’s admission-transfer-discharge database. Cardiac arrests were identified using a prospectively validated quality improvement database.³⁵

Because deaths on the wards resulted either from cardiac arrest or from a comfort care scenario, mortality was not studied as an outcome.

Statistical Analysis

Patient characteristics were compared using Student t tests, Wilcoxon rank sum tests, and chi-squared statistics, as appropriate. Unadjusted and adjusted models were created using discrete-time survival analysis,^36-39 which involved dividing time into discrete 6-hour intervals and employing the predictor variables chronologically closest to the beginning of each time window to forecast whether the outcome occurred within each interval. Predictor variables in the adjusted model included patient characteristics (age, sex, BMI, and Elixhauser Agency for Healthcare Research and Quality-Web comorbidities³⁰ [a priori excluding comorbidities recorded for fewer than 1000 admissions from the model]), ward unit, surgical status, prior ICU admission during the hospitalization, cumulative opioid or benzodiazepine dose during the previous 24 hours, and severity of illness (measured by eCART score). The adjusted model for opioids also included the patient’s pain score. Age, eCART score, and pain score were entered linearly while race, BMI (underweight, less than 18.5 kg/m²; normal, 18.5-24.9 kg/m²; overweight, 25.0-29.9 kg/m²; obese, 30-39.9 kg/m²; and severely obese, 40 mg/m² or greater), and ward unit were modeled as categorical variables.

Since repeat hospitalization could confound the results of our study, we performed a sensitivity analysis including only 1 randomly selected hospital admission per patient. We also performed a sensitivity analysis including receipt of both opioids and benzodiazepines, and an interaction term within each ward segment, as well as an analysis in which zolpidem—the most commonly administered nonbenzodiazepine hypnotic medication in our hospital—was included along with both opioids and benzodiazepines. Finally, we performed a sensitivity analysis replacing missing pain scores with imputed values ranging from 0 to the median ward pain score.

We also performed subgroup analyses of adjusted models across age quartiles and for each BMI category, as well as for surgical status, OSA status, gender, time of medication administration, and route of administration (intravenous vs. oral). We also performed an analysis across pain score severity⁴⁰ to determine whether these medications produce differential effects at various levels of pain.

All tests of significance used a 2-sided P value less than 0.05. Statistical analyses were completed using Stata version 14.1 (StataCorp, LLC, College Station, Texas).

 

RESULTS

Patient Characteristics

A total of 144,895 admissions, from 75,369 patients, had ward vital signs or laboratory values documented during the study period. Ward segments from 634 admissions were excluded due to comfort care status, which resulted in exclusion of 479 complete patient admissions. Additionally, 139 patients with tracheostomies were excluded. Furthermore, 2934 patient admissions with a sickle-cell diagnosis were excluded, of which 95% (n = 2791) received an opioid and 11% (n = 310) received a benzodiazepine. Another 14,029 admissions associated with seizures, 6134 admissions involving alcohol withdrawal, and 1332 with both were excluded, of which 66% (n = 14,174) received an opioid and 35% (n = 7504) received a benzodiazepine. After exclusions, 120,518 admissions were included in the final analysis, with 67% (n = 80,463) associated with at least 1 administration of an opioid and 21% (n = 25,279) associated with at least 1 benzodiazepine administration.

In total, there were 672,851 intervals when an opioid was administered during the study, with a median dose of 12 mg oral morphine equivalents (interquartile range, 8-30). Of these, 21,634 doses were replaced due to outlier status outside the 99^th percentile. Patients receiving opioids were younger (median age 56 vs 61 years), less likely to be African American (48% vs 59%), more likely to have undergone surgery (18% vs 6%), and less likely to have most noncancer medical comorbidities than those who never received an opioid (all P < 0.001) (Table 1).

Additionally, there were a total of 98,286 6-hour intervals in which a benzodiazepine was administered in the study, with a median dose of 1 mg oral lorazepam (interquartile range, 0.5-1). A total of 790 doses of benzodiazepines (less than 1%) were replaced due to outlier status. Patients who received benzodiazepines were more likely to be male (49% vs. 41%), less likely to be African-American, less likely to be obese or morbidly obese (33% vs. 39%), and more likely to have medical comorbidities compared to patients who never received a benzodiazepine (all P < 0.001) (Table 1).

The eCART scores were similar between all patient groups. The frequency of missing variables differed by data type, with vital signs rarely missing (all less than 1.1% except AVPU [10%]), followed by hematology labs (8%-9%), electrolytes and renal function results (12%-15%), and hepatic function tests (40%-45%). In addition to imputed data for missing vital signs and laboratory values, our model omitted human immunodeficiency virus/acquired immune deficiency syndrome and peptic ulcer disease from the adjusted models on the basis of fewer than 1000 admissions with these diagnoses listed.

Patient Outcomes

The incidence of the composite outcome was higher in admissions with at least 1 opioid medication than those without an opioid (7% vs. 4%, P < 0.001), and in admissions with at least 1 dose of benzodiazepines compared to those without a benzodiazepine (11% vs. 4%, P < 0.001) (Table 2).

Within 6-hour segments, increasing doses of opioids were associated with an initial decrease in the frequency of the composite outcome followed by a dose-related increase in the frequency of the composite outcome with morphine equivalents greater than 45 mg. By contrast, the frequency of the composite outcome increased with additional benzodiazepine equivalents (Figure).

In the adjusted model, opioid administration was associated with increased risk for the composite outcome (Table 3) in a dose-dependent fashion, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of ICU transfer or cardiac arrest within the subsequent 6-hour time interval (odds ratio [OR], 1.019; 95% confidence interval [CI], 1.013-1.026; P < 0.001).

Similarly, benzodiazepine administration was also associated with increased adjusted risk for the composite outcome within 6 hours in a dose-dependent manner. Each 1 mg oral lorazepam equivalent was associated with a 29% increase in the odds of ward cardiac arrest or ICU transfer (OR, 1.29; 95% CI, 1.16-1.44; P < 0.001) (Table 3).

Sensitivity Analyses

A sensitivity analysis including 1 randomly selected hospitalization per patient involved 67,097 admissions and found results similar to the primary analysis, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of the composite outcome (OR, 1.019; 95% CI, 1.011-1.028; P < 0.001) and each 1 mg oral lorazepam equivalent associated with a 41% increase in the odds of the composite outcome (OR, 1.41; 95% CI, 1.21-1.65; P < 0.001). Inclusion of both opioids and benzodiazepines in the adjusted model again yielded results similar to the main analysis for both opioids (OR, 1.020; 95% CI, 1.013-1.026; P < 0.001) and benzodiazepines (OR, 1.35; 95% CI, 1.18-1.54; P < 0.001), without a significant interaction detected (P = 0.09). These results were unchanged with the addition of zolpidem to the model as an additional potential confounder, and zolpidem did not increase the risk of the study outcomes (P = 0.2).

 

A final sensitivity analysis for the opioid model involved replacing missing pain scores with imputed values ranging from 0 to the median ward score, which was 5. The results of these analyses did not differ from the primary model and were consistent regardless of imputation value (OR, 1.018; 95% CI, 1.012-1.023; P < 0.001).

Subgroup Analyses

Analyses of opioid administration by subgroup (sex, age quartiles, BMI categories, OSA diagnosis, surgical status, daytime/nighttime medication administration, IV/PO administration, and pain severity) yielded similar results to the overall analysis (Supplemental Figure 2). Subgroup analysis of patients receiving benzodiazepines revealed similarly increased adjusted odds of the composite outcome across strata of gender, BMI, surgical status, and medication administration time (Supplemental Figure 3). Notably, patients older than 70 years who received a benzodiazepine were at 64% increased odds of the composite outcome (OR, 1.64; 95% CI, 1.30-2.08), compared to 2% to 38% increased risk for patients under 70 years. Finally, IV doses of benzodiazepines were associated with 48% increased odds for deterioration (OR, 1.48; 95% CI, 1.18-1.84; P = 0.001), compared to a nonsignificant 14% increase in the odds for PO doses (OR, 1.14; 95% CI, 0.99-1.31; P = 0.066).

DISCUSSION

In a large, single-center, observational study of ward inpatients, we found that opioid use was associated with a small but significant increased risk for clinical deterioration on the wards, with every 15 mg oral morphine equivalent increasing the odds of ICU transfer or cardiac arrest in the next 6 hours by 1.9%. Benzodiazepines were associated with a much higher risk: each equivalent of 1 mg of oral lorazepam increased the odds of ICU transfer or cardiac arrest by almost 30%. These results have important implications for care at the bedside of hospitalized ward patients and suggest the need for closer monitoring after receipt of these medications, particularly benzodiazepines.

 

Previous work has described negative effects of opioid medications among select inpatient populations. In surgical patients, opioids have been associated with hospital readmission, increased length of stay, and hospital mortality.^26,28 More recently, Herzig et al.¹⁵ found more adverse events in nonsurgical ward patients within the hospitals prescribing opioids the most frequently. These studies may have been limited by the populations studied and the inability to control for confounders such as severity of illness and pain score. Our study expands these findings to a more generalizable population and shows that even after adjustment for potential confounders, such as severity of illness, pain score, and medication dose, opioids are associated with increased short-term risk of clinical deterioration.

By contrast, few studies have characterized the risks associated with benzodiazepine use among ward inpatients. Recently, Overdyk et al.²⁷ found that inpatient use of opioids and sedatives was associated with increased risk for cardiac arrest and hospital death. However, this study included ICU patients, which may confound the results, as ICU patients often receive high doses of opioids or benzodiazepines to facilitate mechanical ventilation or other invasive procedures, while also having a particularly high risk of adverse outcomes like cardiac arrest and inhospital death.

Several mechanisms may explain the magnitude of effect seen with regard to benzodiazepines. First, benzodiazepines may directly produce clinical deterioration by decreased respiratory drive, diminished airway tone, or hemodynamic decompensation. It is possible that the broad spectrum of cardiorespiratory side effects of benzodiazepines—and potential unpredictability of these effects—increases the difficulty of observation and management for patients receiving them. This difficulty may be compounded with intravenous administration of benzodiazepines, which was associated with a higher risk for deterioration than oral doses in our cohort. Alternatively, benzodiazepines may contribute to clinical decompensation by masking signs of deterioration such as encephalopathy or vital sign instability like tachycardia or tachypnea that may be mistaken as anxiety. Notably, while our hospital has a nursing-driven protocol for monitoring patients receiving opioids (in which pain is serially assessed, leading to additional bedside observation), we do not have protocols for ward patients receiving benzodiazepines. Finally, although we found that orders for opioids and benzodiazepines were more common in white patients than African American patients, this finding may be due to differences in the types or number of medical comorbidities experienced by these patients.

Our study has several strengths, including the large number of admissions we included. Additionally, we included a broad range of medical and surgical ward admissions, which should increase the generalizability of our results. Further, our rates of ICU transfer are in line with data reported from other groups,^41,42 which again may add to the generalizability of our findings. We also addressed many potential confounders by including patient characteristics, individual ward units, and (for opioids) pain score in our model, and by controlling for severity of illness with the eCART score, an accurate predictor of ICU transfer and ward cardiac arrest within our population.^32,37 Finally, our robust methodology allowed us to include acute and cumulative medication doses, as well as time, in the model. By performing a discrete-time survival analysis, we were able to evaluate receipt of opioids and benzodiazepines—as well as risk for clinical deterioration—longitudinally, lending strength to our results.

Limitations of our study include its single-center cohort, which may reduce generalizability to other populations. Additionally, because we could not validate the accuracy of—or adherence to—outpatient medication lists, we were unable to identify chronic opioid or benzodiazepine users by these lists. However, patients chronically taking opioids or benzodiazepines would likely receive doses each hospital day; by including 24-hour cumulative doses in our model, we attempted to adjust for some portion of their chronic use. Also, because evaluation of delirium was not objectively recorded in our dataset, we were unable to evaluate the relationship between receipt of these medications and development of delirium, which is an important outcome for hospitalized patients. Finally, neither the diagnoses for which these medications were prescribed, nor the reason for ICU transfer, were present in our dataset, which leaves open the possibility of unmeasured confounding.

 

CONCLUSION

After adjustment for important confounders including severity of illness, medication dose, and time, opioids were associated with a slight increase in clinical deterioration on the wards, while benzodiazepines were associated with a much larger risk for deterioration. This finding raises concern about the safety of benzodiazepine use among ward patients and suggests that increased monitoring of patients receiving these medications may be warranted.

Acknowledgment

The authors thank Nicole Twu for administrative support.

Disclosure

Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Dr. Churpek has received honoraria from Chest for invited speaking engagements. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, Massachusetts), research support from the American Heart Association (Dallas, Texas) and Laerdal Medical (Stavanger, Norway), and research support from Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, Illinois), which is developing products for risk stratification of hospitalized patients. Dr. Mokhlesi is supported by National Institutes of Health grant R01HL119161. Dr. Mokhlesi has served as a consultant to Philips/Respironics and has received research support from Philips/Respironics. Preliminary versions of these data were presented as a poster presentation at the 2016 meeting of the American Thoracic Society, May 17, 2016; San Francisco, California.

 

Chronic opioid and benzodiazepine use is common and increasing.^1-5 Outpatient use of these medications has been associated with hospital readmission and death,^6-12 with concurrent use associated with particularly increased risk.^13,14 Less is known about outcomes for hospitalized patients receiving these medications.

More than half of hospital inpatients in the United States receive opioids,¹⁵ many of which are new prescriptions rather than continuation of chronic therapy.^16,17 Less is known about inpatient benzodiazepine administration, but the prevalence may exceed 10% among elderly populations.¹⁸ Hospitalized patients often have comorbidities or physiological disturbances that might increase their risk related to use of these medications. Opioids can cause central and obstructive sleep apneas,^19-21 and benzodiazepines contribute to respiratory depression and airway relaxation.²² Benzodiazepines also impair psychomotor function and recall,²³ which could mediate the recognized risk for delirium and falls in the hospital.^24,25 These findings suggest pathways by which these medications might contribute to clinical deterioration.

Most studies in hospitalized patients have been limited to specific populations^15,26-28 and have not explicitly controlled for severity of illness over time. It remains unclear whether associations identified within particular groups of patients hold true for the broader population of general ward inpatients. Therefore, we aimed to determine the independent association between opioid and benzodiazepine administration and clinical deterioration in ward patients.

MATERIALS AND METHODS

Setting and Study Population

We performed an observational cohort study at a 500-bed urban academic hospital. Data were obtained from all adults hospitalized on the wards between November 1, 2008, and January 21, 2016. The study protocol was approved by the University of Chicago Institutional Review Board (IRB#15-0195).

Data Collection

The study utilized de-identified data from the electronic health record (EHR; Epic Systems Corporation, Verona, Wisconsin) and administrative databases collected by the University of Chicago Clinical Research Data Warehouse. Patient age, sex, race, body mass index (BMI), and ward admission source (ie, emergency department (ED), transferred from the intensive care unit (ICU), or directly admitted to the wards) were collected. International Classification of Diseases, 9^th Revision, Clinical Modification (ICD-9-CM) codes were used to identify Elixhauser Comorbidity Index categories.^29,30 Because patients with similar diagnoses (eg, active cancer) are cohorted within particular areas in our hospital, we obtained the ward unit for all patients. Patients who underwent surgery were identified using the hospital’s admission-transfer-discharge database.

 

To determine severity of illness, routinely collected vital signs and laboratory values were utilized to calculate the electronic cardiac arrest risk triage (eCART) score, an accurate risk score we previously developed and validated for predicting adverse events among ward patients.³¹ If any vital sign or laboratory value was missing, the next available measurement was carried forward. If any value remained missing after this change, the median value for that location (ie, wards, ICU, or ED) was imputed.^32,33 Additionally, patient-reported pain scores at the time of opioid administration were extracted from nursing flowsheets. If no pain score was present at the time of opioid administration, the patient’s previous score was carried forward.

We excluded patients with sickle-cell disease or seizure history and admissions with diagnoses of alcohol withdrawal from the analysis, because these diagnoses were expected to be associated with different medication administration practices compared to other inpatients. We also excluded patients with a tracheostomy because we expected their respiratory monitoring to differ from the other patients in our cohort. Finally, because ward deaths resulting from a comfort care scenario often involve opioids and/or benzodiazepines, ward segments involving comfort care deaths (defined as death without attempted resuscitation) were excluded from the analysis (Supplemental Figure 1). Patients with sickle-cell disease were identified using ICD-9 codes, and encounters during which a seizure may have occurred were identified using a combination of ICD-9 codes and receipt of anti-epileptic medication (Supplemental Table 1). Patients at risk for alcohol withdrawal were identified by the presence of any Clinical Institute Withdrawal Assessment for Alcohol score within nursing flowsheets, and patients with tracheostomies were identified using documentation of ventilator support within their first 12 hours on the wards. In addition to these exclusion criteria, patients with obstructive sleep apnea (OSA) were identified by the following ICD-9 codes: 278.03, 327.23, 780.51, 780.53, and 780.57.

Medications

Ward administrations of opioids and benzodiazepines—dose, route, and administration time—were collected from the EHR. We excluded all administrations in nonward locations such as the ED, ICU, operating room, or procedure suite. Additionally, because patients emergently intubated may receive sedative and analgesic medications to facilitate intubation, and because patients experiencing cardiac arrest are frequently intubated periresuscitation, we a priori excluded all administrations within 15 minutes of a ward cardiac arrest or an intubation.

For consistent comparisons, opioid doses were converted to oral morphine equivalents³⁴ and adjusted by a factor of 15 to reflect the smallest routinely available oral morphine tablet in our hospital (Supplemental Table 2). Benzodiazepine doses were converted to oral lorazepam equivalents (Supplemental Table 2).³⁴ Thus, the independent variables were oral morphine or lorazepam equivalents administered within each 6-hour window. We a priori presumed opioid doses greater than the 99^th percentile (1200 mg) or benzodiazepine doses greater than 10 mg oral lorazepam equivalents within a 6-hour window to be erroneous entries, and replaced these outlier values with the median value for each medication category.

Outcomes

The primary outcome was the composite of ICU transfer or cardiac arrest (loss of pulse with attempted resuscitation) on the wards, with individual outcomes investigated secondarily. An ICU transfer (patient movement from a ward directly to the ICU) was identified using the hospital’s admission-transfer-discharge database. Cardiac arrests were identified using a prospectively validated quality improvement database.³⁵

Because deaths on the wards resulted either from cardiac arrest or from a comfort care scenario, mortality was not studied as an outcome.

Statistical Analysis

Patient characteristics were compared using Student t tests, Wilcoxon rank sum tests, and chi-squared statistics, as appropriate. Unadjusted and adjusted models were created using discrete-time survival analysis,^36-39 which involved dividing time into discrete 6-hour intervals and employing the predictor variables chronologically closest to the beginning of each time window to forecast whether the outcome occurred within each interval. Predictor variables in the adjusted model included patient characteristics (age, sex, BMI, and Elixhauser Agency for Healthcare Research and Quality-Web comorbidities³⁰ [a priori excluding comorbidities recorded for fewer than 1000 admissions from the model]), ward unit, surgical status, prior ICU admission during the hospitalization, cumulative opioid or benzodiazepine dose during the previous 24 hours, and severity of illness (measured by eCART score). The adjusted model for opioids also included the patient’s pain score. Age, eCART score, and pain score were entered linearly while race, BMI (underweight, less than 18.5 kg/m²; normal, 18.5-24.9 kg/m²; overweight, 25.0-29.9 kg/m²; obese, 30-39.9 kg/m²; and severely obese, 40 mg/m² or greater), and ward unit were modeled as categorical variables.

Since repeat hospitalization could confound the results of our study, we performed a sensitivity analysis including only 1 randomly selected hospital admission per patient. We also performed a sensitivity analysis including receipt of both opioids and benzodiazepines, and an interaction term within each ward segment, as well as an analysis in which zolpidem—the most commonly administered nonbenzodiazepine hypnotic medication in our hospital—was included along with both opioids and benzodiazepines. Finally, we performed a sensitivity analysis replacing missing pain scores with imputed values ranging from 0 to the median ward pain score.

We also performed subgroup analyses of adjusted models across age quartiles and for each BMI category, as well as for surgical status, OSA status, gender, time of medication administration, and route of administration (intravenous vs. oral). We also performed an analysis across pain score severity⁴⁰ to determine whether these medications produce differential effects at various levels of pain.

All tests of significance used a 2-sided P value less than 0.05. Statistical analyses were completed using Stata version 14.1 (StataCorp, LLC, College Station, Texas).

 

RESULTS

Patient Characteristics

A total of 144,895 admissions, from 75,369 patients, had ward vital signs or laboratory values documented during the study period. Ward segments from 634 admissions were excluded due to comfort care status, which resulted in exclusion of 479 complete patient admissions. Additionally, 139 patients with tracheostomies were excluded. Furthermore, 2934 patient admissions with a sickle-cell diagnosis were excluded, of which 95% (n = 2791) received an opioid and 11% (n = 310) received a benzodiazepine. Another 14,029 admissions associated with seizures, 6134 admissions involving alcohol withdrawal, and 1332 with both were excluded, of which 66% (n = 14,174) received an opioid and 35% (n = 7504) received a benzodiazepine. After exclusions, 120,518 admissions were included in the final analysis, with 67% (n = 80,463) associated with at least 1 administration of an opioid and 21% (n = 25,279) associated with at least 1 benzodiazepine administration.

In total, there were 672,851 intervals when an opioid was administered during the study, with a median dose of 12 mg oral morphine equivalents (interquartile range, 8-30). Of these, 21,634 doses were replaced due to outlier status outside the 99^th percentile. Patients receiving opioids were younger (median age 56 vs 61 years), less likely to be African American (48% vs 59%), more likely to have undergone surgery (18% vs 6%), and less likely to have most noncancer medical comorbidities than those who never received an opioid (all P < 0.001) (Table 1).

Additionally, there were a total of 98,286 6-hour intervals in which a benzodiazepine was administered in the study, with a median dose of 1 mg oral lorazepam (interquartile range, 0.5-1). A total of 790 doses of benzodiazepines (less than 1%) were replaced due to outlier status. Patients who received benzodiazepines were more likely to be male (49% vs. 41%), less likely to be African-American, less likely to be obese or morbidly obese (33% vs. 39%), and more likely to have medical comorbidities compared to patients who never received a benzodiazepine (all P < 0.001) (Table 1).

The eCART scores were similar between all patient groups. The frequency of missing variables differed by data type, with vital signs rarely missing (all less than 1.1% except AVPU [10%]), followed by hematology labs (8%-9%), electrolytes and renal function results (12%-15%), and hepatic function tests (40%-45%). In addition to imputed data for missing vital signs and laboratory values, our model omitted human immunodeficiency virus/acquired immune deficiency syndrome and peptic ulcer disease from the adjusted models on the basis of fewer than 1000 admissions with these diagnoses listed.

Patient Outcomes

The incidence of the composite outcome was higher in admissions with at least 1 opioid medication than those without an opioid (7% vs. 4%, P < 0.001), and in admissions with at least 1 dose of benzodiazepines compared to those without a benzodiazepine (11% vs. 4%, P < 0.001) (Table 2).

Within 6-hour segments, increasing doses of opioids were associated with an initial decrease in the frequency of the composite outcome followed by a dose-related increase in the frequency of the composite outcome with morphine equivalents greater than 45 mg. By contrast, the frequency of the composite outcome increased with additional benzodiazepine equivalents (Figure).

In the adjusted model, opioid administration was associated with increased risk for the composite outcome (Table 3) in a dose-dependent fashion, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of ICU transfer or cardiac arrest within the subsequent 6-hour time interval (odds ratio [OR], 1.019; 95% confidence interval [CI], 1.013-1.026; P < 0.001).

Similarly, benzodiazepine administration was also associated with increased adjusted risk for the composite outcome within 6 hours in a dose-dependent manner. Each 1 mg oral lorazepam equivalent was associated with a 29% increase in the odds of ward cardiac arrest or ICU transfer (OR, 1.29; 95% CI, 1.16-1.44; P < 0.001) (Table 3).

Sensitivity Analyses

A sensitivity analysis including 1 randomly selected hospitalization per patient involved 67,097 admissions and found results similar to the primary analysis, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of the composite outcome (OR, 1.019; 95% CI, 1.011-1.028; P < 0.001) and each 1 mg oral lorazepam equivalent associated with a 41% increase in the odds of the composite outcome (OR, 1.41; 95% CI, 1.21-1.65; P < 0.001). Inclusion of both opioids and benzodiazepines in the adjusted model again yielded results similar to the main analysis for both opioids (OR, 1.020; 95% CI, 1.013-1.026; P < 0.001) and benzodiazepines (OR, 1.35; 95% CI, 1.18-1.54; P < 0.001), without a significant interaction detected (P = 0.09). These results were unchanged with the addition of zolpidem to the model as an additional potential confounder, and zolpidem did not increase the risk of the study outcomes (P = 0.2).

 

A final sensitivity analysis for the opioid model involved replacing missing pain scores with imputed values ranging from 0 to the median ward score, which was 5. The results of these analyses did not differ from the primary model and were consistent regardless of imputation value (OR, 1.018; 95% CI, 1.012-1.023; P < 0.001).

Subgroup Analyses

Analyses of opioid administration by subgroup (sex, age quartiles, BMI categories, OSA diagnosis, surgical status, daytime/nighttime medication administration, IV/PO administration, and pain severity) yielded similar results to the overall analysis (Supplemental Figure 2). Subgroup analysis of patients receiving benzodiazepines revealed similarly increased adjusted odds of the composite outcome across strata of gender, BMI, surgical status, and medication administration time (Supplemental Figure 3). Notably, patients older than 70 years who received a benzodiazepine were at 64% increased odds of the composite outcome (OR, 1.64; 95% CI, 1.30-2.08), compared to 2% to 38% increased risk for patients under 70 years. Finally, IV doses of benzodiazepines were associated with 48% increased odds for deterioration (OR, 1.48; 95% CI, 1.18-1.84; P = 0.001), compared to a nonsignificant 14% increase in the odds for PO doses (OR, 1.14; 95% CI, 0.99-1.31; P = 0.066).

DISCUSSION

In a large, single-center, observational study of ward inpatients, we found that opioid use was associated with a small but significant increased risk for clinical deterioration on the wards, with every 15 mg oral morphine equivalent increasing the odds of ICU transfer or cardiac arrest in the next 6 hours by 1.9%. Benzodiazepines were associated with a much higher risk: each equivalent of 1 mg of oral lorazepam increased the odds of ICU transfer or cardiac arrest by almost 30%. These results have important implications for care at the bedside of hospitalized ward patients and suggest the need for closer monitoring after receipt of these medications, particularly benzodiazepines.

 

Previous work has described negative effects of opioid medications among select inpatient populations. In surgical patients, opioids have been associated with hospital readmission, increased length of stay, and hospital mortality.^26,28 More recently, Herzig et al.¹⁵ found more adverse events in nonsurgical ward patients within the hospitals prescribing opioids the most frequently. These studies may have been limited by the populations studied and the inability to control for confounders such as severity of illness and pain score. Our study expands these findings to a more generalizable population and shows that even after adjustment for potential confounders, such as severity of illness, pain score, and medication dose, opioids are associated with increased short-term risk of clinical deterioration.

By contrast, few studies have characterized the risks associated with benzodiazepine use among ward inpatients. Recently, Overdyk et al.²⁷ found that inpatient use of opioids and sedatives was associated with increased risk for cardiac arrest and hospital death. However, this study included ICU patients, which may confound the results, as ICU patients often receive high doses of opioids or benzodiazepines to facilitate mechanical ventilation or other invasive procedures, while also having a particularly high risk of adverse outcomes like cardiac arrest and inhospital death.

Several mechanisms may explain the magnitude of effect seen with regard to benzodiazepines. First, benzodiazepines may directly produce clinical deterioration by decreased respiratory drive, diminished airway tone, or hemodynamic decompensation. It is possible that the broad spectrum of cardiorespiratory side effects of benzodiazepines—and potential unpredictability of these effects—increases the difficulty of observation and management for patients receiving them. This difficulty may be compounded with intravenous administration of benzodiazepines, which was associated with a higher risk for deterioration than oral doses in our cohort. Alternatively, benzodiazepines may contribute to clinical decompensation by masking signs of deterioration such as encephalopathy or vital sign instability like tachycardia or tachypnea that may be mistaken as anxiety. Notably, while our hospital has a nursing-driven protocol for monitoring patients receiving opioids (in which pain is serially assessed, leading to additional bedside observation), we do not have protocols for ward patients receiving benzodiazepines. Finally, although we found that orders for opioids and benzodiazepines were more common in white patients than African American patients, this finding may be due to differences in the types or number of medical comorbidities experienced by these patients.

Our study has several strengths, including the large number of admissions we included. Additionally, we included a broad range of medical and surgical ward admissions, which should increase the generalizability of our results. Further, our rates of ICU transfer are in line with data reported from other groups,^41,42 which again may add to the generalizability of our findings. We also addressed many potential confounders by including patient characteristics, individual ward units, and (for opioids) pain score in our model, and by controlling for severity of illness with the eCART score, an accurate predictor of ICU transfer and ward cardiac arrest within our population.^32,37 Finally, our robust methodology allowed us to include acute and cumulative medication doses, as well as time, in the model. By performing a discrete-time survival analysis, we were able to evaluate receipt of opioids and benzodiazepines—as well as risk for clinical deterioration—longitudinally, lending strength to our results.

Limitations of our study include its single-center cohort, which may reduce generalizability to other populations. Additionally, because we could not validate the accuracy of—or adherence to—outpatient medication lists, we were unable to identify chronic opioid or benzodiazepine users by these lists. However, patients chronically taking opioids or benzodiazepines would likely receive doses each hospital day; by including 24-hour cumulative doses in our model, we attempted to adjust for some portion of their chronic use. Also, because evaluation of delirium was not objectively recorded in our dataset, we were unable to evaluate the relationship between receipt of these medications and development of delirium, which is an important outcome for hospitalized patients. Finally, neither the diagnoses for which these medications were prescribed, nor the reason for ICU transfer, were present in our dataset, which leaves open the possibility of unmeasured confounding.

 

CONCLUSION

After adjustment for important confounders including severity of illness, medication dose, and time, opioids were associated with a slight increase in clinical deterioration on the wards, while benzodiazepines were associated with a much larger risk for deterioration. This finding raises concern about the safety of benzodiazepine use among ward patients and suggests that increased monitoring of patients receiving these medications may be warranted.

Acknowledgment

The authors thank Nicole Twu for administrative support.

Disclosure

Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Dr. Churpek has received honoraria from Chest for invited speaking engagements. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, Massachusetts), research support from the American Heart Association (Dallas, Texas) and Laerdal Medical (Stavanger, Norway), and research support from Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, Illinois), which is developing products for risk stratification of hospitalized patients. Dr. Mokhlesi is supported by National Institutes of Health grant R01HL119161. Dr. Mokhlesi has served as a consultant to Philips/Respironics and has received research support from Philips/Respironics. Preliminary versions of these data were presented as a poster presentation at the 2016 meeting of the American Thoracic Society, May 17, 2016; San Francisco, California.

 

Chronic opioid and benzodiazepine use is common and increasing.^1-5 Outpatient use of these medications has been associated with hospital readmission and death,^6-12 with concurrent use associated with particularly increased risk.^13,14 Less is known about outcomes for hospitalized patients receiving these medications.

More than half of hospital inpatients in the United States receive opioids,¹⁵ many of which are new prescriptions rather than continuation of chronic therapy.^16,17 Less is known about inpatient benzodiazepine administration, but the prevalence may exceed 10% among elderly populations.¹⁸ Hospitalized patients often have comorbidities or physiological disturbances that might increase their risk related to use of these medications. Opioids can cause central and obstructive sleep apneas,^19-21 and benzodiazepines contribute to respiratory depression and airway relaxation.²² Benzodiazepines also impair psychomotor function and recall,²³ which could mediate the recognized risk for delirium and falls in the hospital.^24,25 These findings suggest pathways by which these medications might contribute to clinical deterioration.

Most studies in hospitalized patients have been limited to specific populations^15,26-28 and have not explicitly controlled for severity of illness over time. It remains unclear whether associations identified within particular groups of patients hold true for the broader population of general ward inpatients. Therefore, we aimed to determine the independent association between opioid and benzodiazepine administration and clinical deterioration in ward patients.

MATERIALS AND METHODS

Setting and Study Population

We performed an observational cohort study at a 500-bed urban academic hospital. Data were obtained from all adults hospitalized on the wards between November 1, 2008, and January 21, 2016. The study protocol was approved by the University of Chicago Institutional Review Board (IRB#15-0195).

Data Collection

The study utilized de-identified data from the electronic health record (EHR; Epic Systems Corporation, Verona, Wisconsin) and administrative databases collected by the University of Chicago Clinical Research Data Warehouse. Patient age, sex, race, body mass index (BMI), and ward admission source (ie, emergency department (ED), transferred from the intensive care unit (ICU), or directly admitted to the wards) were collected. International Classification of Diseases, 9^th Revision, Clinical Modification (ICD-9-CM) codes were used to identify Elixhauser Comorbidity Index categories.^29,30 Because patients with similar diagnoses (eg, active cancer) are cohorted within particular areas in our hospital, we obtained the ward unit for all patients. Patients who underwent surgery were identified using the hospital’s admission-transfer-discharge database.

 

To determine severity of illness, routinely collected vital signs and laboratory values were utilized to calculate the electronic cardiac arrest risk triage (eCART) score, an accurate risk score we previously developed and validated for predicting adverse events among ward patients.³¹ If any vital sign or laboratory value was missing, the next available measurement was carried forward. If any value remained missing after this change, the median value for that location (ie, wards, ICU, or ED) was imputed.^32,33 Additionally, patient-reported pain scores at the time of opioid administration were extracted from nursing flowsheets. If no pain score was present at the time of opioid administration, the patient’s previous score was carried forward.

We excluded patients with sickle-cell disease or seizure history and admissions with diagnoses of alcohol withdrawal from the analysis, because these diagnoses were expected to be associated with different medication administration practices compared to other inpatients. We also excluded patients with a tracheostomy because we expected their respiratory monitoring to differ from the other patients in our cohort. Finally, because ward deaths resulting from a comfort care scenario often involve opioids and/or benzodiazepines, ward segments involving comfort care deaths (defined as death without attempted resuscitation) were excluded from the analysis (Supplemental Figure 1). Patients with sickle-cell disease were identified using ICD-9 codes, and encounters during which a seizure may have occurred were identified using a combination of ICD-9 codes and receipt of anti-epileptic medication (Supplemental Table 1). Patients at risk for alcohol withdrawal were identified by the presence of any Clinical Institute Withdrawal Assessment for Alcohol score within nursing flowsheets, and patients with tracheostomies were identified using documentation of ventilator support within their first 12 hours on the wards. In addition to these exclusion criteria, patients with obstructive sleep apnea (OSA) were identified by the following ICD-9 codes: 278.03, 327.23, 780.51, 780.53, and 780.57.

Medications

Ward administrations of opioids and benzodiazepines—dose, route, and administration time—were collected from the EHR. We excluded all administrations in nonward locations such as the ED, ICU, operating room, or procedure suite. Additionally, because patients emergently intubated may receive sedative and analgesic medications to facilitate intubation, and because patients experiencing cardiac arrest are frequently intubated periresuscitation, we a priori excluded all administrations within 15 minutes of a ward cardiac arrest or an intubation.

For consistent comparisons, opioid doses were converted to oral morphine equivalents³⁴ and adjusted by a factor of 15 to reflect the smallest routinely available oral morphine tablet in our hospital (Supplemental Table 2). Benzodiazepine doses were converted to oral lorazepam equivalents (Supplemental Table 2).³⁴ Thus, the independent variables were oral morphine or lorazepam equivalents administered within each 6-hour window. We a priori presumed opioid doses greater than the 99^th percentile (1200 mg) or benzodiazepine doses greater than 10 mg oral lorazepam equivalents within a 6-hour window to be erroneous entries, and replaced these outlier values with the median value for each medication category.

Outcomes

The primary outcome was the composite of ICU transfer or cardiac arrest (loss of pulse with attempted resuscitation) on the wards, with individual outcomes investigated secondarily. An ICU transfer (patient movement from a ward directly to the ICU) was identified using the hospital’s admission-transfer-discharge database. Cardiac arrests were identified using a prospectively validated quality improvement database.³⁵

Because deaths on the wards resulted either from cardiac arrest or from a comfort care scenario, mortality was not studied as an outcome.

Statistical Analysis

Patient characteristics were compared using Student t tests, Wilcoxon rank sum tests, and chi-squared statistics, as appropriate. Unadjusted and adjusted models were created using discrete-time survival analysis,^36-39 which involved dividing time into discrete 6-hour intervals and employing the predictor variables chronologically closest to the beginning of each time window to forecast whether the outcome occurred within each interval. Predictor variables in the adjusted model included patient characteristics (age, sex, BMI, and Elixhauser Agency for Healthcare Research and Quality-Web comorbidities³⁰ [a priori excluding comorbidities recorded for fewer than 1000 admissions from the model]), ward unit, surgical status, prior ICU admission during the hospitalization, cumulative opioid or benzodiazepine dose during the previous 24 hours, and severity of illness (measured by eCART score). The adjusted model for opioids also included the patient’s pain score. Age, eCART score, and pain score were entered linearly while race, BMI (underweight, less than 18.5 kg/m²; normal, 18.5-24.9 kg/m²; overweight, 25.0-29.9 kg/m²; obese, 30-39.9 kg/m²; and severely obese, 40 mg/m² or greater), and ward unit were modeled as categorical variables.

Since repeat hospitalization could confound the results of our study, we performed a sensitivity analysis including only 1 randomly selected hospital admission per patient. We also performed a sensitivity analysis including receipt of both opioids and benzodiazepines, and an interaction term within each ward segment, as well as an analysis in which zolpidem—the most commonly administered nonbenzodiazepine hypnotic medication in our hospital—was included along with both opioids and benzodiazepines. Finally, we performed a sensitivity analysis replacing missing pain scores with imputed values ranging from 0 to the median ward pain score.

We also performed subgroup analyses of adjusted models across age quartiles and for each BMI category, as well as for surgical status, OSA status, gender, time of medication administration, and route of administration (intravenous vs. oral). We also performed an analysis across pain score severity⁴⁰ to determine whether these medications produce differential effects at various levels of pain.

All tests of significance used a 2-sided P value less than 0.05. Statistical analyses were completed using Stata version 14.1 (StataCorp, LLC, College Station, Texas).

 

RESULTS

Patient Characteristics

A total of 144,895 admissions, from 75,369 patients, had ward vital signs or laboratory values documented during the study period. Ward segments from 634 admissions were excluded due to comfort care status, which resulted in exclusion of 479 complete patient admissions. Additionally, 139 patients with tracheostomies were excluded. Furthermore, 2934 patient admissions with a sickle-cell diagnosis were excluded, of which 95% (n = 2791) received an opioid and 11% (n = 310) received a benzodiazepine. Another 14,029 admissions associated with seizures, 6134 admissions involving alcohol withdrawal, and 1332 with both were excluded, of which 66% (n = 14,174) received an opioid and 35% (n = 7504) received a benzodiazepine. After exclusions, 120,518 admissions were included in the final analysis, with 67% (n = 80,463) associated with at least 1 administration of an opioid and 21% (n = 25,279) associated with at least 1 benzodiazepine administration.

In total, there were 672,851 intervals when an opioid was administered during the study, with a median dose of 12 mg oral morphine equivalents (interquartile range, 8-30). Of these, 21,634 doses were replaced due to outlier status outside the 99^th percentile. Patients receiving opioids were younger (median age 56 vs 61 years), less likely to be African American (48% vs 59%), more likely to have undergone surgery (18% vs 6%), and less likely to have most noncancer medical comorbidities than those who never received an opioid (all P < 0.001) (Table 1).

Additionally, there were a total of 98,286 6-hour intervals in which a benzodiazepine was administered in the study, with a median dose of 1 mg oral lorazepam (interquartile range, 0.5-1). A total of 790 doses of benzodiazepines (less than 1%) were replaced due to outlier status. Patients who received benzodiazepines were more likely to be male (49% vs. 41%), less likely to be African-American, less likely to be obese or morbidly obese (33% vs. 39%), and more likely to have medical comorbidities compared to patients who never received a benzodiazepine (all P < 0.001) (Table 1).

The eCART scores were similar between all patient groups. The frequency of missing variables differed by data type, with vital signs rarely missing (all less than 1.1% except AVPU [10%]), followed by hematology labs (8%-9%), electrolytes and renal function results (12%-15%), and hepatic function tests (40%-45%). In addition to imputed data for missing vital signs and laboratory values, our model omitted human immunodeficiency virus/acquired immune deficiency syndrome and peptic ulcer disease from the adjusted models on the basis of fewer than 1000 admissions with these diagnoses listed.

Patient Outcomes

The incidence of the composite outcome was higher in admissions with at least 1 opioid medication than those without an opioid (7% vs. 4%, P < 0.001), and in admissions with at least 1 dose of benzodiazepines compared to those without a benzodiazepine (11% vs. 4%, P < 0.001) (Table 2).

Within 6-hour segments, increasing doses of opioids were associated with an initial decrease in the frequency of the composite outcome followed by a dose-related increase in the frequency of the composite outcome with morphine equivalents greater than 45 mg. By contrast, the frequency of the composite outcome increased with additional benzodiazepine equivalents (Figure).

In the adjusted model, opioid administration was associated with increased risk for the composite outcome (Table 3) in a dose-dependent fashion, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of ICU transfer or cardiac arrest within the subsequent 6-hour time interval (odds ratio [OR], 1.019; 95% confidence interval [CI], 1.013-1.026; P < 0.001).

Similarly, benzodiazepine administration was also associated with increased adjusted risk for the composite outcome within 6 hours in a dose-dependent manner. Each 1 mg oral lorazepam equivalent was associated with a 29% increase in the odds of ward cardiac arrest or ICU transfer (OR, 1.29; 95% CI, 1.16-1.44; P < 0.001) (Table 3).

Sensitivity Analyses

A sensitivity analysis including 1 randomly selected hospitalization per patient involved 67,097 admissions and found results similar to the primary analysis, with each 15 mg oral morphine equivalent associated with a 1.9% increase in the odds of the composite outcome (OR, 1.019; 95% CI, 1.011-1.028; P < 0.001) and each 1 mg oral lorazepam equivalent associated with a 41% increase in the odds of the composite outcome (OR, 1.41; 95% CI, 1.21-1.65; P < 0.001). Inclusion of both opioids and benzodiazepines in the adjusted model again yielded results similar to the main analysis for both opioids (OR, 1.020; 95% CI, 1.013-1.026; P < 0.001) and benzodiazepines (OR, 1.35; 95% CI, 1.18-1.54; P < 0.001), without a significant interaction detected (P = 0.09). These results were unchanged with the addition of zolpidem to the model as an additional potential confounder, and zolpidem did not increase the risk of the study outcomes (P = 0.2).

 

A final sensitivity analysis for the opioid model involved replacing missing pain scores with imputed values ranging from 0 to the median ward score, which was 5. The results of these analyses did not differ from the primary model and were consistent regardless of imputation value (OR, 1.018; 95% CI, 1.012-1.023; P < 0.001).

Subgroup Analyses

Analyses of opioid administration by subgroup (sex, age quartiles, BMI categories, OSA diagnosis, surgical status, daytime/nighttime medication administration, IV/PO administration, and pain severity) yielded similar results to the overall analysis (Supplemental Figure 2). Subgroup analysis of patients receiving benzodiazepines revealed similarly increased adjusted odds of the composite outcome across strata of gender, BMI, surgical status, and medication administration time (Supplemental Figure 3). Notably, patients older than 70 years who received a benzodiazepine were at 64% increased odds of the composite outcome (OR, 1.64; 95% CI, 1.30-2.08), compared to 2% to 38% increased risk for patients under 70 years. Finally, IV doses of benzodiazepines were associated with 48% increased odds for deterioration (OR, 1.48; 95% CI, 1.18-1.84; P = 0.001), compared to a nonsignificant 14% increase in the odds for PO doses (OR, 1.14; 95% CI, 0.99-1.31; P = 0.066).

DISCUSSION

In a large, single-center, observational study of ward inpatients, we found that opioid use was associated with a small but significant increased risk for clinical deterioration on the wards, with every 15 mg oral morphine equivalent increasing the odds of ICU transfer or cardiac arrest in the next 6 hours by 1.9%. Benzodiazepines were associated with a much higher risk: each equivalent of 1 mg of oral lorazepam increased the odds of ICU transfer or cardiac arrest by almost 30%. These results have important implications for care at the bedside of hospitalized ward patients and suggest the need for closer monitoring after receipt of these medications, particularly benzodiazepines.

 

Previous work has described negative effects of opioid medications among select inpatient populations. In surgical patients, opioids have been associated with hospital readmission, increased length of stay, and hospital mortality.^26,28 More recently, Herzig et al.¹⁵ found more adverse events in nonsurgical ward patients within the hospitals prescribing opioids the most frequently. These studies may have been limited by the populations studied and the inability to control for confounders such as severity of illness and pain score. Our study expands these findings to a more generalizable population and shows that even after adjustment for potential confounders, such as severity of illness, pain score, and medication dose, opioids are associated with increased short-term risk of clinical deterioration.

By contrast, few studies have characterized the risks associated with benzodiazepine use among ward inpatients. Recently, Overdyk et al.²⁷ found that inpatient use of opioids and sedatives was associated with increased risk for cardiac arrest and hospital death. However, this study included ICU patients, which may confound the results, as ICU patients often receive high doses of opioids or benzodiazepines to facilitate mechanical ventilation or other invasive procedures, while also having a particularly high risk of adverse outcomes like cardiac arrest and inhospital death.

Several mechanisms may explain the magnitude of effect seen with regard to benzodiazepines. First, benzodiazepines may directly produce clinical deterioration by decreased respiratory drive, diminished airway tone, or hemodynamic decompensation. It is possible that the broad spectrum of cardiorespiratory side effects of benzodiazepines—and potential unpredictability of these effects—increases the difficulty of observation and management for patients receiving them. This difficulty may be compounded with intravenous administration of benzodiazepines, which was associated with a higher risk for deterioration than oral doses in our cohort. Alternatively, benzodiazepines may contribute to clinical decompensation by masking signs of deterioration such as encephalopathy or vital sign instability like tachycardia or tachypnea that may be mistaken as anxiety. Notably, while our hospital has a nursing-driven protocol for monitoring patients receiving opioids (in which pain is serially assessed, leading to additional bedside observation), we do not have protocols for ward patients receiving benzodiazepines. Finally, although we found that orders for opioids and benzodiazepines were more common in white patients than African American patients, this finding may be due to differences in the types or number of medical comorbidities experienced by these patients.

Our study has several strengths, including the large number of admissions we included. Additionally, we included a broad range of medical and surgical ward admissions, which should increase the generalizability of our results. Further, our rates of ICU transfer are in line with data reported from other groups,^41,42 which again may add to the generalizability of our findings. We also addressed many potential confounders by including patient characteristics, individual ward units, and (for opioids) pain score in our model, and by controlling for severity of illness with the eCART score, an accurate predictor of ICU transfer and ward cardiac arrest within our population.^32,37 Finally, our robust methodology allowed us to include acute and cumulative medication doses, as well as time, in the model. By performing a discrete-time survival analysis, we were able to evaluate receipt of opioids and benzodiazepines—as well as risk for clinical deterioration—longitudinally, lending strength to our results.

Limitations of our study include its single-center cohort, which may reduce generalizability to other populations. Additionally, because we could not validate the accuracy of—or adherence to—outpatient medication lists, we were unable to identify chronic opioid or benzodiazepine users by these lists. However, patients chronically taking opioids or benzodiazepines would likely receive doses each hospital day; by including 24-hour cumulative doses in our model, we attempted to adjust for some portion of their chronic use. Also, because evaluation of delirium was not objectively recorded in our dataset, we were unable to evaluate the relationship between receipt of these medications and development of delirium, which is an important outcome for hospitalized patients. Finally, neither the diagnoses for which these medications were prescribed, nor the reason for ICU transfer, were present in our dataset, which leaves open the possibility of unmeasured confounding.

 

CONCLUSION

After adjustment for important confounders including severity of illness, medication dose, and time, opioids were associated with a slight increase in clinical deterioration on the wards, while benzodiazepines were associated with a much larger risk for deterioration. This finding raises concern about the safety of benzodiazepine use among ward patients and suggests that increased monitoring of patients receiving these medications may be warranted.

Acknowledgment

The authors thank Nicole Twu for administrative support.

Disclosure

Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. Dr. Churpek is supported by a career development award from the National Heart, Lung, and Blood Institute (K08 HL121080). Dr. Churpek has received honoraria from Chest for invited speaking engagements. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, Massachusetts), research support from the American Heart Association (Dallas, Texas) and Laerdal Medical (Stavanger, Norway), and research support from Early Sense (Tel Aviv, Israel). She has ownership interest in Quant HC (Chicago, Illinois), which is developing products for risk stratification of hospitalized patients. Dr. Mokhlesi is supported by National Institutes of Health grant R01HL119161. Dr. Mokhlesi has served as a consultant to Philips/Respironics and has received research support from Philips/Respironics. Preliminary versions of these data were presented as a poster presentation at the 2016 meeting of the American Thoracic Society, May 17, 2016; San Francisco, California.

 

In the U.S. about 2 million men have osteoporosis.¹ About 1 in 5 men will experience an osteoporotic-related fracture in his lifetime.² In addition, men with hip fracture have a higher mortality rate compared with that of women with hip fracture.³ The National Osteoporosis Foundation guidelines and the Endocrine Society guidelines recommend that all men aged ≥ 70 years have bone mineral density (BMD) testing. Depending on risk factors, osteoporosis screening may be appropriate for men aged ≥ 50 years. A BMD with a T-score of -2.5 or lower is classified as osteoporosis.²

In addition to osteoporosis, osteopenia also negatively impacts men. Osteopenia is defined as a BMD with a T-score of -1 to -2.5.² According to the National Health and Nutrition Examination Survey (NHANES), about 30% of men aged ≥ 50 years have osteopenia.⁴ FRAX is a fracture risk assessment tool that is used to predict the 10-year risk of fracture in untreated patients with osteopenia. The FRAX tool has been validated with the use of BMD testing only at the femoral neck; it has not been validated in other parts of the body. Treatment is indicated if the 10-year fracture risk is > 20% for major osteoporotic fractures and > 3% for hip fractures, based on the FRAX calculation.²

The following risk factors are used in the FRAX calculation: age; sex; weight (kilograms); height (centimeters); previous fracture (yes or no); parental history of hip fracture (yes or no); current smoker (yes or no); oral glucocorticoid exposure currently or for > 3 months in the past (yes or no); rheumatoid arthritis (yes or no); secondary osteoporosis or a disorder strongly associated with osteoporosis, including type 1 diabetes mellitus, osteogenesis imperfecta in adults, untreated long-standing hyperthyroidism, hypogonadism, premature menopause, chronic malnutrition, malabsorption, or chronic liver disease (yes or no); 3 or more units of alcohol daily (yes or no); and BMD.⁵

A dual-energy X-ray absorptiometry (DXA) examination is needed to determine BMD. However, a DXA examination is not always feasible for patients who have limited access, transportation challenges, require the use of assistive devices, and may be unaware of the importance of BMD testing.

The FRAX calculation can be obtained with or without BMD. Gadam and colleagues compared FRAX calculations with and without BMD to predict the 10-year risk of fracture.⁶ Their study found that 84% of patients had an identical fracture risk prediction whether or not BMD was included. The only risk factor evaluated that was significantly different between those with different treatment predictions and those with identical treatment predictions was age. However, the majority of patients included were female (96%).

No studies existed that compared fracture prediction risk with and without BMD in a male-only population. The purpose of this study was to determine whether FRAX without BMD was as effective as FRAX with BMD to predict the risk of osteoporotic fractures and provide an identical treatment recommendation in male veteran patients at the Lexington VAMC in Kentucky.

Methods

A retrospective chart review was conducted at the Lexington VAMC. Approval was obtained from the Lexington VAMC Institutional Review Board and Research and Development Committee. Patients were identified using the computerized patient record system (CPRS). Included patients were male, ≥ 50 years, had a documented DXA in CPRS from January 2006 to September 2015, and had a previous fracture determined by ICD-9 codes. Patients were excluded if they were diagnosed with osteoporosis or were ever treated for osteoporosis before a DXA scan.

Data collection included patient’s age, gender, race, glucocorticoid use for at least 3 months within 1 year prior to DXA, body weight within 3 months prior to DXA, height within 1 year prior to DXA, family history of fracture, previous fall or fracture, diagnosis of rheumatoid arthritis, smoking status at the time of DXA, alcohol intake of at least 3 drinks per day at the time of DXA, and vitamin D level within 1 year prior to DXA. In order to find a clinically significant difference (P < .05) with a power of 80%, a sample size of 64 patients was needed.

Each patient’s FRAX predictions were calculated with and without BMD. Patients were then separated into 2 groups: those who had an identical treatment recommendation when calculating FRAX with and without BMD, and those who had a different treatment recommendation when calculating FRAX with and without BMD. Binary variables for each group were compared using the Fisher exact test, and numeric variables were compared using a simple Student’s t test.

Results

After screening 1,510 patients, only 119 patients met the criteria and were included in the study (Figure). All patients included were male. Mean age was 71.2 years and 113 (95.0%) were white (Table 1).

 

Of the 119 patients included in the study, 98 patients (82.4%) had the same treatment recommendation when the FRAX score was calculated with and without BMD. The remaining 21 patients (17.6%) had different treatment recommendations when FRAX scores were calculated with BMD compared with FRAX scores calculated without BMD. Treatment was recommended based on risk prediction for 43 of the 98 patients who had identical treatment recommendations. Of the 21 patients who had different treatment recommendations, treatment was recommended based on risk prediction for 14 patients when FRAX scores were calculated with BMD. Treatment was recommended for the other 7 patients when FRAX scores were calculated without BMD.

Of the numeric variables evaluated, mean age, femoral neck BMD, and T-score were all significantly different between the 2 groups (Table 2). Patients with an identical treatment recommendation were a mean age of 67.9 years (SD: 10.2 y), and patients with different treatment recommendations were a mean age of 62.2 years (SD: 8.9 y) (P = .011). Patients with an identical treatment recommendation had a mean BMD of 0.9 (SD: 0.2), and patients with different treatment recommendations had a mean BMD of 0.8 (SD: 0.1) (P = .021). Patients with an identical treatment recommendation had a mean T-score of -1.7 (SD: 1.2), and patients with different treatment recommendations had a mean T-score of -2.3 (SD: 1.1) (P = .031). Mean weight, height, and vitamin D level were not statistically significantly different between the 2 groups.

Of the binary variables evaluated, only glucocorticoid use was significantly different between the 2 groups. Of the patients with an identical treatment recommendation, 4 (4.1%) received a glucocorticoid.

Of the patients with different treatment recommendations, 4 (19%) received a glucocorticoid (P = .032). Vitamin D supplementation, previous fracture, smoking status, rheumatoid arthritis, and alcohol use were not statistically significantly different between the 2 groups (Table 3).

Discussion

The purpose of this retrospective study was to determine whether using FRAX without BMD was as effective as using FRAX with BMD in predicting the risk of osteoporotic fractures and in providing identical treatment recommendations in male veteran patients. The results of this study revealed that FRAX calculations without BMD provided identical treatment recommendations as FRAX calculations with BMD for 82.4% of male veteran patients. These findings were similar to the findings of another study by Gadam and colleagues, in which 84% of patients had identical treatment recommendations when calculating FRAX scores with and without BMD.⁶ In contrast, a prospective cohort study by Ettinger and colleagues found that the addition of BMD to the FRAX calculation enhanced the performance of the FRAX tool by correctly identifying more patients who experienced a fracture within the following 10 years.⁸

Several of the risk factors evaluated in the present study were indicative of an identical treatment recommendation. Age was one of the risk factors that differed significantly between the 2 groups. The mean age of patients with an identical treatment recommendation was 67.9 years, and the mean age of patients with different treatment recommendations was 62.2 years (P = .011). These findings opposed the findings in the Gadam and colleagues’ study.⁶ The results of that study revealed that younger age rather than older age was more indicative of an identical treatment recommendation. The study by Gadam and colleagues included both male and female patients; however, the majority of patients included in the Gadam study were female (96%).⁶ Because the present study included only male patients, a comparison of the results was difficult because of the different patient populations.

A higher T-score (P = .031) and a higher BMD (P = .021) were the other 2 risk factors associated with an identical treatment recommendation with and without BMD. The Gadam and colleagues study did not find these to be significant risk factors for identifying an identical treatment recommendation.⁶

The FRAX calculation without BMD identified all the patients meeting treatment criteria based on the FRAX calculation with BMD except for 14 of the 119 patients (11.8%). Therefore, > 88% of patients who met treatment criteria based on FRAX calculated with BMD also met treatment criteria based on FRAX without BMD.

The FRAX calculation has several advantages, including risk stratification in men and identifying those with other conditions that may predispose them to a fracture.⁷ Therefore, before obtaining a DXA scan, it would be reasonable to calculate a FRAX score without BMD to identify patients who are at high risk for fracture but who may not receive treatment because they are not considered to need a DXA scan or a DXA scan is not feasible.

 

Limitations

Currently, FRAX is validated only using femoral neck BMD. This study was a retrospective chart review only; no information was obtained from communicating with the patient, including the patient’s past medical history and family history. Also, this study had a small sample size: Of the 1,510 patients screened, only 119 met inclusion criteria. None of the 119 patients evaluated had a family history of fracture documented in their CPRS. Therefore, several of the patient’s 10-year fracture risk scores may be underestimated if one or both of their parents experienced a fracture. Last, the majority of patients included in this study were white, so the results of this study cannot necessarily be generalized to other races.

Conclusion

The majority of male patients had an identical treatment recommendation when a FRAX score was calculated with and without BMD. Older age, higher BMD, and higher T-score were all indicative of an identical treatment recommendation. Larger studies are necessary in order to validate the FRAX tool without the use of femoral neck BMD. However, the FRAX tool alone can be beneficial to identify male patients who should have a DXA scan performed to obtain a BMD. If a male patient’s FRAX score suggests risk for osteoporotic fracture, then a DXA scan should be completed to obtain a BMD if feasible.

Additionally, when obtaining a BMD is not feasible to predict fracture risk, the FRAX tool alone may be useful a majority of the time to accurately determine treatment recommendations in male patients aged > 65 years. The results of this study lead the authors to believe that FRAX without BMD in male patients aged > 65 years will appropriately identify more patients for treatment. ˜

Acknowledgments
This material is the result of work supported with resources and the use of facilities at the Lexington VA Medical Center.

In the U.S. about 2 million men have osteoporosis.¹ About 1 in 5 men will experience an osteoporotic-related fracture in his lifetime.² In addition, men with hip fracture have a higher mortality rate compared with that of women with hip fracture.³ The National Osteoporosis Foundation guidelines and the Endocrine Society guidelines recommend that all men aged ≥ 70 years have bone mineral density (BMD) testing. Depending on risk factors, osteoporosis screening may be appropriate for men aged ≥ 50 years. A BMD with a T-score of -2.5 or lower is classified as osteoporosis.²

In addition to osteoporosis, osteopenia also negatively impacts men. Osteopenia is defined as a BMD with a T-score of -1 to -2.5.² According to the National Health and Nutrition Examination Survey (NHANES), about 30% of men aged ≥ 50 years have osteopenia.⁴ FRAX is a fracture risk assessment tool that is used to predict the 10-year risk of fracture in untreated patients with osteopenia. The FRAX tool has been validated with the use of BMD testing only at the femoral neck; it has not been validated in other parts of the body. Treatment is indicated if the 10-year fracture risk is > 20% for major osteoporotic fractures and > 3% for hip fractures, based on the FRAX calculation.²

The following risk factors are used in the FRAX calculation: age; sex; weight (kilograms); height (centimeters); previous fracture (yes or no); parental history of hip fracture (yes or no); current smoker (yes or no); oral glucocorticoid exposure currently or for > 3 months in the past (yes or no); rheumatoid arthritis (yes or no); secondary osteoporosis or a disorder strongly associated with osteoporosis, including type 1 diabetes mellitus, osteogenesis imperfecta in adults, untreated long-standing hyperthyroidism, hypogonadism, premature menopause, chronic malnutrition, malabsorption, or chronic liver disease (yes or no); 3 or more units of alcohol daily (yes or no); and BMD.⁵

A dual-energy X-ray absorptiometry (DXA) examination is needed to determine BMD. However, a DXA examination is not always feasible for patients who have limited access, transportation challenges, require the use of assistive devices, and may be unaware of the importance of BMD testing.

The FRAX calculation can be obtained with or without BMD. Gadam and colleagues compared FRAX calculations with and without BMD to predict the 10-year risk of fracture.⁶ Their study found that 84% of patients had an identical fracture risk prediction whether or not BMD was included. The only risk factor evaluated that was significantly different between those with different treatment predictions and those with identical treatment predictions was age. However, the majority of patients included were female (96%).

No studies existed that compared fracture prediction risk with and without BMD in a male-only population. The purpose of this study was to determine whether FRAX without BMD was as effective as FRAX with BMD to predict the risk of osteoporotic fractures and provide an identical treatment recommendation in male veteran patients at the Lexington VAMC in Kentucky.

Methods

A retrospective chart review was conducted at the Lexington VAMC. Approval was obtained from the Lexington VAMC Institutional Review Board and Research and Development Committee. Patients were identified using the computerized patient record system (CPRS). Included patients were male, ≥ 50 years, had a documented DXA in CPRS from January 2006 to September 2015, and had a previous fracture determined by ICD-9 codes. Patients were excluded if they were diagnosed with osteoporosis or were ever treated for osteoporosis before a DXA scan.

Data collection included patient’s age, gender, race, glucocorticoid use for at least 3 months within 1 year prior to DXA, body weight within 3 months prior to DXA, height within 1 year prior to DXA, family history of fracture, previous fall or fracture, diagnosis of rheumatoid arthritis, smoking status at the time of DXA, alcohol intake of at least 3 drinks per day at the time of DXA, and vitamin D level within 1 year prior to DXA. In order to find a clinically significant difference (P < .05) with a power of 80%, a sample size of 64 patients was needed.

Each patient’s FRAX predictions were calculated with and without BMD. Patients were then separated into 2 groups: those who had an identical treatment recommendation when calculating FRAX with and without BMD, and those who had a different treatment recommendation when calculating FRAX with and without BMD. Binary variables for each group were compared using the Fisher exact test, and numeric variables were compared using a simple Student’s t test.

Results

After screening 1,510 patients, only 119 patients met the criteria and were included in the study (Figure). All patients included were male. Mean age was 71.2 years and 113 (95.0%) were white (Table 1).

 

Of the 119 patients included in the study, 98 patients (82.4%) had the same treatment recommendation when the FRAX score was calculated with and without BMD. The remaining 21 patients (17.6%) had different treatment recommendations when FRAX scores were calculated with BMD compared with FRAX scores calculated without BMD. Treatment was recommended based on risk prediction for 43 of the 98 patients who had identical treatment recommendations. Of the 21 patients who had different treatment recommendations, treatment was recommended based on risk prediction for 14 patients when FRAX scores were calculated with BMD. Treatment was recommended for the other 7 patients when FRAX scores were calculated without BMD.

Of the numeric variables evaluated, mean age, femoral neck BMD, and T-score were all significantly different between the 2 groups (Table 2). Patients with an identical treatment recommendation were a mean age of 67.9 years (SD: 10.2 y), and patients with different treatment recommendations were a mean age of 62.2 years (SD: 8.9 y) (P = .011). Patients with an identical treatment recommendation had a mean BMD of 0.9 (SD: 0.2), and patients with different treatment recommendations had a mean BMD of 0.8 (SD: 0.1) (P = .021). Patients with an identical treatment recommendation had a mean T-score of -1.7 (SD: 1.2), and patients with different treatment recommendations had a mean T-score of -2.3 (SD: 1.1) (P = .031). Mean weight, height, and vitamin D level were not statistically significantly different between the 2 groups.

Of the binary variables evaluated, only glucocorticoid use was significantly different between the 2 groups. Of the patients with an identical treatment recommendation, 4 (4.1%) received a glucocorticoid.

Of the patients with different treatment recommendations, 4 (19%) received a glucocorticoid (P = .032). Vitamin D supplementation, previous fracture, smoking status, rheumatoid arthritis, and alcohol use were not statistically significantly different between the 2 groups (Table 3).

Discussion

The purpose of this retrospective study was to determine whether using FRAX without BMD was as effective as using FRAX with BMD in predicting the risk of osteoporotic fractures and in providing identical treatment recommendations in male veteran patients. The results of this study revealed that FRAX calculations without BMD provided identical treatment recommendations as FRAX calculations with BMD for 82.4% of male veteran patients. These findings were similar to the findings of another study by Gadam and colleagues, in which 84% of patients had identical treatment recommendations when calculating FRAX scores with and without BMD.⁶ In contrast, a prospective cohort study by Ettinger and colleagues found that the addition of BMD to the FRAX calculation enhanced the performance of the FRAX tool by correctly identifying more patients who experienced a fracture within the following 10 years.⁸

Several of the risk factors evaluated in the present study were indicative of an identical treatment recommendation. Age was one of the risk factors that differed significantly between the 2 groups. The mean age of patients with an identical treatment recommendation was 67.9 years, and the mean age of patients with different treatment recommendations was 62.2 years (P = .011). These findings opposed the findings in the Gadam and colleagues’ study.⁶ The results of that study revealed that younger age rather than older age was more indicative of an identical treatment recommendation. The study by Gadam and colleagues included both male and female patients; however, the majority of patients included in the Gadam study were female (96%).⁶ Because the present study included only male patients, a comparison of the results was difficult because of the different patient populations.

A higher T-score (P = .031) and a higher BMD (P = .021) were the other 2 risk factors associated with an identical treatment recommendation with and without BMD. The Gadam and colleagues study did not find these to be significant risk factors for identifying an identical treatment recommendation.⁶

The FRAX calculation without BMD identified all the patients meeting treatment criteria based on the FRAX calculation with BMD except for 14 of the 119 patients (11.8%). Therefore, > 88% of patients who met treatment criteria based on FRAX calculated with BMD also met treatment criteria based on FRAX without BMD.

The FRAX calculation has several advantages, including risk stratification in men and identifying those with other conditions that may predispose them to a fracture.⁷ Therefore, before obtaining a DXA scan, it would be reasonable to calculate a FRAX score without BMD to identify patients who are at high risk for fracture but who may not receive treatment because they are not considered to need a DXA scan or a DXA scan is not feasible.

 

Limitations

Currently, FRAX is validated only using femoral neck BMD. This study was a retrospective chart review only; no information was obtained from communicating with the patient, including the patient’s past medical history and family history. Also, this study had a small sample size: Of the 1,510 patients screened, only 119 met inclusion criteria. None of the 119 patients evaluated had a family history of fracture documented in their CPRS. Therefore, several of the patient’s 10-year fracture risk scores may be underestimated if one or both of their parents experienced a fracture. Last, the majority of patients included in this study were white, so the results of this study cannot necessarily be generalized to other races.

Conclusion

The majority of male patients had an identical treatment recommendation when a FRAX score was calculated with and without BMD. Older age, higher BMD, and higher T-score were all indicative of an identical treatment recommendation. Larger studies are necessary in order to validate the FRAX tool without the use of femoral neck BMD. However, the FRAX tool alone can be beneficial to identify male patients who should have a DXA scan performed to obtain a BMD. If a male patient’s FRAX score suggests risk for osteoporotic fracture, then a DXA scan should be completed to obtain a BMD if feasible.

Additionally, when obtaining a BMD is not feasible to predict fracture risk, the FRAX tool alone may be useful a majority of the time to accurately determine treatment recommendations in male patients aged > 65 years. The results of this study lead the authors to believe that FRAX without BMD in male patients aged > 65 years will appropriately identify more patients for treatment. ˜

Acknowledgments
This material is the result of work supported with resources and the use of facilities at the Lexington VA Medical Center.

In the U.S. about 2 million men have osteoporosis.¹ About 1 in 5 men will experience an osteoporotic-related fracture in his lifetime.² In addition, men with hip fracture have a higher mortality rate compared with that of women with hip fracture.³ The National Osteoporosis Foundation guidelines and the Endocrine Society guidelines recommend that all men aged ≥ 70 years have bone mineral density (BMD) testing. Depending on risk factors, osteoporosis screening may be appropriate for men aged ≥ 50 years. A BMD with a T-score of -2.5 or lower is classified as osteoporosis.²

In addition to osteoporosis, osteopenia also negatively impacts men. Osteopenia is defined as a BMD with a T-score of -1 to -2.5.² According to the National Health and Nutrition Examination Survey (NHANES), about 30% of men aged ≥ 50 years have osteopenia.⁴ FRAX is a fracture risk assessment tool that is used to predict the 10-year risk of fracture in untreated patients with osteopenia. The FRAX tool has been validated with the use of BMD testing only at the femoral neck; it has not been validated in other parts of the body. Treatment is indicated if the 10-year fracture risk is > 20% for major osteoporotic fractures and > 3% for hip fractures, based on the FRAX calculation.²

The following risk factors are used in the FRAX calculation: age; sex; weight (kilograms); height (centimeters); previous fracture (yes or no); parental history of hip fracture (yes or no); current smoker (yes or no); oral glucocorticoid exposure currently or for > 3 months in the past (yes or no); rheumatoid arthritis (yes or no); secondary osteoporosis or a disorder strongly associated with osteoporosis, including type 1 diabetes mellitus, osteogenesis imperfecta in adults, untreated long-standing hyperthyroidism, hypogonadism, premature menopause, chronic malnutrition, malabsorption, or chronic liver disease (yes or no); 3 or more units of alcohol daily (yes or no); and BMD.⁵

A dual-energy X-ray absorptiometry (DXA) examination is needed to determine BMD. However, a DXA examination is not always feasible for patients who have limited access, transportation challenges, require the use of assistive devices, and may be unaware of the importance of BMD testing.

The FRAX calculation can be obtained with or without BMD. Gadam and colleagues compared FRAX calculations with and without BMD to predict the 10-year risk of fracture.⁶ Their study found that 84% of patients had an identical fracture risk prediction whether or not BMD was included. The only risk factor evaluated that was significantly different between those with different treatment predictions and those with identical treatment predictions was age. However, the majority of patients included were female (96%).

No studies existed that compared fracture prediction risk with and without BMD in a male-only population. The purpose of this study was to determine whether FRAX without BMD was as effective as FRAX with BMD to predict the risk of osteoporotic fractures and provide an identical treatment recommendation in male veteran patients at the Lexington VAMC in Kentucky.

Methods

A retrospective chart review was conducted at the Lexington VAMC. Approval was obtained from the Lexington VAMC Institutional Review Board and Research and Development Committee. Patients were identified using the computerized patient record system (CPRS). Included patients were male, ≥ 50 years, had a documented DXA in CPRS from January 2006 to September 2015, and had a previous fracture determined by ICD-9 codes. Patients were excluded if they were diagnosed with osteoporosis or were ever treated for osteoporosis before a DXA scan.

Data collection included patient’s age, gender, race, glucocorticoid use for at least 3 months within 1 year prior to DXA, body weight within 3 months prior to DXA, height within 1 year prior to DXA, family history of fracture, previous fall or fracture, diagnosis of rheumatoid arthritis, smoking status at the time of DXA, alcohol intake of at least 3 drinks per day at the time of DXA, and vitamin D level within 1 year prior to DXA. In order to find a clinically significant difference (P < .05) with a power of 80%, a sample size of 64 patients was needed.

Each patient’s FRAX predictions were calculated with and without BMD. Patients were then separated into 2 groups: those who had an identical treatment recommendation when calculating FRAX with and without BMD, and those who had a different treatment recommendation when calculating FRAX with and without BMD. Binary variables for each group were compared using the Fisher exact test, and numeric variables were compared using a simple Student’s t test.

Results

After screening 1,510 patients, only 119 patients met the criteria and were included in the study (Figure). All patients included were male. Mean age was 71.2 years and 113 (95.0%) were white (Table 1).

 

Of the 119 patients included in the study, 98 patients (82.4%) had the same treatment recommendation when the FRAX score was calculated with and without BMD. The remaining 21 patients (17.6%) had different treatment recommendations when FRAX scores were calculated with BMD compared with FRAX scores calculated without BMD. Treatment was recommended based on risk prediction for 43 of the 98 patients who had identical treatment recommendations. Of the 21 patients who had different treatment recommendations, treatment was recommended based on risk prediction for 14 patients when FRAX scores were calculated with BMD. Treatment was recommended for the other 7 patients when FRAX scores were calculated without BMD.

Of the numeric variables evaluated, mean age, femoral neck BMD, and T-score were all significantly different between the 2 groups (Table 2). Patients with an identical treatment recommendation were a mean age of 67.9 years (SD: 10.2 y), and patients with different treatment recommendations were a mean age of 62.2 years (SD: 8.9 y) (P = .011). Patients with an identical treatment recommendation had a mean BMD of 0.9 (SD: 0.2), and patients with different treatment recommendations had a mean BMD of 0.8 (SD: 0.1) (P = .021). Patients with an identical treatment recommendation had a mean T-score of -1.7 (SD: 1.2), and patients with different treatment recommendations had a mean T-score of -2.3 (SD: 1.1) (P = .031). Mean weight, height, and vitamin D level were not statistically significantly different between the 2 groups.

Of the binary variables evaluated, only glucocorticoid use was significantly different between the 2 groups. Of the patients with an identical treatment recommendation, 4 (4.1%) received a glucocorticoid.

Of the patients with different treatment recommendations, 4 (19%) received a glucocorticoid (P = .032). Vitamin D supplementation, previous fracture, smoking status, rheumatoid arthritis, and alcohol use were not statistically significantly different between the 2 groups (Table 3).

Discussion

The purpose of this retrospective study was to determine whether using FRAX without BMD was as effective as using FRAX with BMD in predicting the risk of osteoporotic fractures and in providing identical treatment recommendations in male veteran patients. The results of this study revealed that FRAX calculations without BMD provided identical treatment recommendations as FRAX calculations with BMD for 82.4% of male veteran patients. These findings were similar to the findings of another study by Gadam and colleagues, in which 84% of patients had identical treatment recommendations when calculating FRAX scores with and without BMD.⁶ In contrast, a prospective cohort study by Ettinger and colleagues found that the addition of BMD to the FRAX calculation enhanced the performance of the FRAX tool by correctly identifying more patients who experienced a fracture within the following 10 years.⁸

Several of the risk factors evaluated in the present study were indicative of an identical treatment recommendation. Age was one of the risk factors that differed significantly between the 2 groups. The mean age of patients with an identical treatment recommendation was 67.9 years, and the mean age of patients with different treatment recommendations was 62.2 years (P = .011). These findings opposed the findings in the Gadam and colleagues’ study.⁶ The results of that study revealed that younger age rather than older age was more indicative of an identical treatment recommendation. The study by Gadam and colleagues included both male and female patients; however, the majority of patients included in the Gadam study were female (96%).⁶ Because the present study included only male patients, a comparison of the results was difficult because of the different patient populations.

A higher T-score (P = .031) and a higher BMD (P = .021) were the other 2 risk factors associated with an identical treatment recommendation with and without BMD. The Gadam and colleagues study did not find these to be significant risk factors for identifying an identical treatment recommendation.⁶

The FRAX calculation without BMD identified all the patients meeting treatment criteria based on the FRAX calculation with BMD except for 14 of the 119 patients (11.8%). Therefore, > 88% of patients who met treatment criteria based on FRAX calculated with BMD also met treatment criteria based on FRAX without BMD.

The FRAX calculation has several advantages, including risk stratification in men and identifying those with other conditions that may predispose them to a fracture.⁷ Therefore, before obtaining a DXA scan, it would be reasonable to calculate a FRAX score without BMD to identify patients who are at high risk for fracture but who may not receive treatment because they are not considered to need a DXA scan or a DXA scan is not feasible.

 

Limitations

Currently, FRAX is validated only using femoral neck BMD. This study was a retrospective chart review only; no information was obtained from communicating with the patient, including the patient’s past medical history and family history. Also, this study had a small sample size: Of the 1,510 patients screened, only 119 met inclusion criteria. None of the 119 patients evaluated had a family history of fracture documented in their CPRS. Therefore, several of the patient’s 10-year fracture risk scores may be underestimated if one or both of their parents experienced a fracture. Last, the majority of patients included in this study were white, so the results of this study cannot necessarily be generalized to other races.

Conclusion

The majority of male patients had an identical treatment recommendation when a FRAX score was calculated with and without BMD. Older age, higher BMD, and higher T-score were all indicative of an identical treatment recommendation. Larger studies are necessary in order to validate the FRAX tool without the use of femoral neck BMD. However, the FRAX tool alone can be beneficial to identify male patients who should have a DXA scan performed to obtain a BMD. If a male patient’s FRAX score suggests risk for osteoporotic fracture, then a DXA scan should be completed to obtain a BMD if feasible.

Additionally, when obtaining a BMD is not feasible to predict fracture risk, the FRAX tool alone may be useful a majority of the time to accurately determine treatment recommendations in male patients aged > 65 years. The results of this study lead the authors to believe that FRAX without BMD in male patients aged > 65 years will appropriately identify more patients for treatment. ˜

Acknowledgments
This material is the result of work supported with resources and the use of facilities at the Lexington VA Medical Center.

Take-Home Points

• The Flesch-Kincaid Readability Scale is a useful tool in evaluating the readability of PEMs.
• Only 1 article analyzed in our study was below a sixth-grade readability level.
• Coauthorship of PEMs with other subspecialty groups had no effect on readability.
• Poor health literacy has been associated with poor health outcomes.
• Efforts must be undertaken to make PEMs more readable across medical subspecialties.

Patients increasingly turn to the Internet to self-educate about orthopedic conditions.^1,2 Accordingly, the Internet has become a valuable tool in maintaining effective physician-patient communication.^3-5 Given the Internet’s importance as a medium for conveying patient information, it is important that orthopedic patient education materials (PEMs) on the Internet provide high-quality information that is easily read by the target patient population. Unfortunately, studies have found that many of the Internet’s orthopedic PEMs have been neither of high quality^6-8 nor presented such that they are easy for patients to read and comprehend.^1,9-12

Readability, which is the reading comprehension level (school grade level) a person must have to understand written materials, is determined by systematic formulae¹²; readability levels correlate with the ability to comprehend written information.² Studies have consistently found that orthopedic PEMs are written at readability levels too high for the average patient to understand.^1,9,13 The readability of PEMs in orthopedics as a whole⁹ and within the orthopedic subspecialties of arthroplasty,¹ foot and ankle surgery,² sports medicine,¹² and spine surgery¹³ has been evaluated, but so far there has been no evaluation of PEMs in orthopedic trauma (OT).

We conducted a study to assess the readability of OT-PEMs available online from the American Academy of Orthopaedic Surgeons (AAOS) in conjunction with the Orthopaedic Trauma Association (OTA) and other orthopedic subspecialty societies. We hypothesized the readability levels of these OT-PEMs would be above the level (sixth to eighth grade) recommended by several healthcare organizations, including the Centers for Disease Control and Prevention.^9,11,14 We also assessed the effect that orthopedic subspecialty coauthorship has on PEM readability.

Methods

In July 2014, we searched the AAOS online patient education library (Broken Bones & Injuries section, http://orthoinfo.aaos.org/menus/injury.cfm) and the AAOS OrthoPortal website (Trauma section, http://pubsearch.aaos.org/search?q=trauma&client=OrthoInfo&site=PATIENT&output=xml_no_dtd&proxystylesheet=OrthoInfo&filter=0) for all relevant OT-PEMs. Although OTA does not publish its own PEMs on its website, it coauthored several of the articles in the AAOS patient education library. Other subspecialty organizations, including the American Orthopaedic Society for Sports Medicine (AOSSM), the American Society for Surgery of the Hand (ASSH), the Pediatric Orthopaedic Society of North America (POSNA), the American Shoulder and Elbow Surgeons (ASES), the American Association of Hip and Knee Surgeons (AAHKS), and the American Orthopaedic Foot and Ankle Society (AOFAS), coauthored several of these online OT-PEMs as well.

Using the technique described by Badarudeen and Sabharwal,¹⁰ we saved all articles to be included in the study as separate Microsoft Word 2011 files. We saved them in plain-text format to remove any HTML tags and any other hidden formatting that might affect readability results. Then we edited them to remove elements that might affect readability result accuracy—deleted article topic–unrelated information (eg, copyright notice, disclaimers, author information) and all numerals, decimal points, bullets, abbreviations, paragraph breaks, colons, semicolons, and dashes.¹⁰Mr. Mohan used the Flesch-Kincaid (FK) Readability Scale to calculate grade level for each article. Microsoft Word 2011 was used as described in other investigations of orthopedic PEM readability^2,10,12,13: Its readability function is enabled by going to the Tools tab and then to the Spelling & Grammar tool, where the “Show readability statistics” option is selected.¹⁰ Readability scores are calculated with the Spelling & Grammar tool; the readability score is displayed after completion of the spelling-and-grammar check. The formula used to calculate FK grade level is¹⁵: (0.39 × average number of words per sentence) + (11.8 × average number of syllables per word) – 15.59.

Statistical Analysis

Descriptive statistics, including means and 95% confidence intervals (CIs), were calculated for the FK grade levels. Student t tests were used to compare average FK grade levels of articles written exclusively by AAOS with those of articles coauthored by AAOS and other orthopedic subspecialty societies. A 2-sample unequal-variance t test was used, and significance was set at P < .05. Total number of articles written at or below the sixth- and eighth-grade levels, the reading levels recommended for PEMs, were tabulated.^1,9-12 Intraobserver and interobserver reliabilities were calculated with intraclass correlation coefficients (ICCs): Mr. Mohan, who calculated the FK scores earlier, now 1 week later calculated the readability levels of 15 randomly selected articles^10,11; in addition, Mr. Mohan and Dr. Yi independently calculated the readability levels of 30 randomly selected articles.^10,11 The same method described earlier—edit plain-text files, then use Microsoft Word to obtain FK scores—was again used. ICCs of 0 to 0.24 correspond to poor correlation; 0.25 to 0.49, low correlation; 0.5 to 0.69, fair correlation; 0.7 to 0.89, good correlation; and 0.9 to 1.0, excellent correlation.^10,11 All statistical analyses were performed with Microsoft Excel 2011 and VassarStats (http://vassarstats.net/tu.html).

 

Results

Of the 115 AAOS website articles included in the study and reviewed, 18 were coauthored by OTA, 10 by AOSSM, 14 by POSNA, 2 by ASSH, 2 by ASES, 1 by AAHKS, 3 by AOFAS, 1 by AOSSM and ASES, and 1 by AOFAS and AOSSM.

Mean FK grade level was 9.1 (range, 6.2-12; 95% CI, 8.9-9.3) for all articles reviewed and 9.1 (range, 6.2-12; 95% CI, 8.8-9.4) for articles exclusively written by AAOS. For coauthored articles, mean FK grade level was 9.3 (range, 7.6-11.3; 95% CI, 8.8-9.8) for AAOS-OTA; 8.9 (range, 7.4-10.4; 95% CI, 8.4-9.6) for AAOS-AOSSM; 9.4 (range, 7-11.8; 95% CI, 8.9-10.1) for AAOS-POSNA; 7.8 (range, 7.8-9.1; 95% CI, 7.2-9.8) for AAOS-ASSH; 9 (range, 8.2-9.6; 95% CI, 7.6-10.2) for AAOS-ASES; 9 (range, 7.9-9; 95% CI, 7.9-9.3) for AAOS-AOFAS; 8.1 for the 1 AAOS-AAHKS article; 8.5 for the 1 AAOS-AOSSM-ASES article; and 8 for the 1 AAOS-AOFAS-AOSSM article (Figure).

Nineteen articles (16.5%) were found to be at or below the eighth-grade reading level, which is the average reading level of a US adult,¹⁰ and only 1 article was at or below the sixth-grade level, the level widely recommended for PEMs.¹¹ In addition, there was no statistically significant difference between articles coauthored by the various orthopedic subspecialties and those written exclusively by AAOS.

For FK readability calculations, interobserver reliability (ICC, 0.9982) and intraobserver reliability (ICC, 1) were both excellent.

Discussion

Although increasing numbers of patients are using information from the Internet to inform their healthcare decisions,¹² studies have shown that online PEMs are written at a readability level above that of the average patient.^1,9,13 In the present study, we also found that OT-PEMs from AAOS are written at a level considerably higher than the recommended sixth-grade reading level,¹⁶ potentially impairing patient comprehension and leading to poorer health outcomes.¹⁷

The pervasiveness of too-high PEM readability levels has been found across orthopedic subspecialties.^2,9,12,13 Following this trend, the OT articles we reviewed had a ninth-grade reading level on average, and only 1 of 115 articles was below the recommended sixth-grade level.¹⁰ The issue of too-high PEM readability levels is thus a problem both in OT and in orthopedics in general. Accordingly, efforts to address this problem are warranted, especially as orthopedic PEM readability has not substantially improved over the past several years.¹⁸In this study, we also tried to identify any readability differences between articles coauthored by orthopedic societies and articles that were not coauthored by orthopedic societies. We hypothesized that multidisciplinary authorship could improve PEM readability; for example, orthopedic societies could collaborate with other medical specialties (eg, family medicine) that have produced appropriately readable PEMs. One study found that the majority of PEMs from the American Academy of Family Physicians (AAFP) were written below the sixth-grade reading level because of strict organizational regulation of the production of such materials.¹⁹ By noting and adopting successful PEM development methods used by groups such as AAFP,^19,20 we might be able to improve OT-PEM readability. However, this was not the case in our study, though our observations may have been limited by the small sample of reviewable articles.

One factor contributing to the poor readability of orthopedic PEMs is that orthopedics terminology is complex and includes words that are often difficult to translate into simpler terms without losing their meaning.¹⁰ When PEMs are written at a level that is too complex, patients cannot fully comprehend them, which may lead to poor health literacy. This problem may be even more harmful when considering the poor literacy levels of patients at baseline. Kadakia and colleagues¹⁶ found that OT patients had poor health literacy; for example, fewer than half knew which bone they fractured. As health literacy is associated with poorer health outcomes and reduced use of healthcare services,²¹ optimizing patients’ health literacy is of crucial importance to both their education and their outcomes.

Our study should be viewed in light of some important limitations. As OTA does not publish its own PEMs, we assessed only OT-related articles that were available on the AAOS website and were exclusively written by AAOS, or coauthored by AAOS and by OTA and/or another orthopedic subspecialty organization. As these articles represent only a subset of the full spectrum of OT-PEMs available on the Internet, our results may not be generalizable to the entire scope of such materials. However, as AAOS and OTA represent the most authoritative OT organizations, we think these PEMs would be among those most likely to be recommended to patients by their surgeons. In addition, although we used a well-established tool for examining readability—the FK readability scale^10-13—this tool has its own inherent limitations, as FK readability grade level is calculated purely on the basis of words per sentence and total syllables per word, and does not take into account other article elements, such as images, which also provide information.^1,10 Nevertheless, the FK scale is an inexpensive, easily accessed readability tool that provides a reproducible readability value that is easily comparable to results from earlier studies.¹⁰ The final limitation is that we excluded from the study AAOS website articles written in a language other than English. Such articles, however, are important, as a large portion of the patient population speaks English as a second language. Indeed, the readability of Spanish PEMs has been investigated—albeit using a readability measure other than the FK scale—and may be a topic pertinent to orthopedic PEMs.²²Most of the literature on the readability of orthopedic PEMs has found their reading levels too high for the average patient to comprehend.^1,9-12 The trend continues with our study findings regarding OT-PEMs available online from AAOS. Although the literature on the inadequacies of orthopedic PEMs is vast,^1,9-12 more work is needed to improve the quality, accuracy, and readability of these materials. There has been some success in improving PEM readability and producing appropriately readable materials within the medical profession,^19,23 so we know that appropriately readable orthopedic PEMs are feasible.

Am J Orthop. 2017;46(3):E190-E194. Copyright Frontline Medical Communications Inc. 2017. All rights reserved.

 

Take-Home Points

• The Flesch-Kincaid Readability Scale is a useful tool in evaluating the readability of PEMs.
• Only 1 article analyzed in our study was below a sixth-grade readability level.
• Coauthorship of PEMs with other subspecialty groups had no effect on readability.
• Poor health literacy has been associated with poor health outcomes.
• Efforts must be undertaken to make PEMs more readable across medical subspecialties.

Patients increasingly turn to the Internet to self-educate about orthopedic conditions.^1,2 Accordingly, the Internet has become a valuable tool in maintaining effective physician-patient communication.^3-5 Given the Internet’s importance as a medium for conveying patient information, it is important that orthopedic patient education materials (PEMs) on the Internet provide high-quality information that is easily read by the target patient population. Unfortunately, studies have found that many of the Internet’s orthopedic PEMs have been neither of high quality^6-8 nor presented such that they are easy for patients to read and comprehend.^1,9-12

Readability, which is the reading comprehension level (school grade level) a person must have to understand written materials, is determined by systematic formulae¹²; readability levels correlate with the ability to comprehend written information.² Studies have consistently found that orthopedic PEMs are written at readability levels too high for the average patient to understand.^1,9,13 The readability of PEMs in orthopedics as a whole⁹ and within the orthopedic subspecialties of arthroplasty,¹ foot and ankle surgery,² sports medicine,¹² and spine surgery¹³ has been evaluated, but so far there has been no evaluation of PEMs in orthopedic trauma (OT).

We conducted a study to assess the readability of OT-PEMs available online from the American Academy of Orthopaedic Surgeons (AAOS) in conjunction with the Orthopaedic Trauma Association (OTA) and other orthopedic subspecialty societies. We hypothesized the readability levels of these OT-PEMs would be above the level (sixth to eighth grade) recommended by several healthcare organizations, including the Centers for Disease Control and Prevention.^9,11,14 We also assessed the effect that orthopedic subspecialty coauthorship has on PEM readability.

Methods

In July 2014, we searched the AAOS online patient education library (Broken Bones & Injuries section, http://orthoinfo.aaos.org/menus/injury.cfm) and the AAOS OrthoPortal website (Trauma section, http://pubsearch.aaos.org/search?q=trauma&client=OrthoInfo&site=PATIENT&output=xml_no_dtd&proxystylesheet=OrthoInfo&filter=0) for all relevant OT-PEMs. Although OTA does not publish its own PEMs on its website, it coauthored several of the articles in the AAOS patient education library. Other subspecialty organizations, including the American Orthopaedic Society for Sports Medicine (AOSSM), the American Society for Surgery of the Hand (ASSH), the Pediatric Orthopaedic Society of North America (POSNA), the American Shoulder and Elbow Surgeons (ASES), the American Association of Hip and Knee Surgeons (AAHKS), and the American Orthopaedic Foot and Ankle Society (AOFAS), coauthored several of these online OT-PEMs as well.

Using the technique described by Badarudeen and Sabharwal,¹⁰ we saved all articles to be included in the study as separate Microsoft Word 2011 files. We saved them in plain-text format to remove any HTML tags and any other hidden formatting that might affect readability results. Then we edited them to remove elements that might affect readability result accuracy—deleted article topic–unrelated information (eg, copyright notice, disclaimers, author information) and all numerals, decimal points, bullets, abbreviations, paragraph breaks, colons, semicolons, and dashes.¹⁰Mr. Mohan used the Flesch-Kincaid (FK) Readability Scale to calculate grade level for each article. Microsoft Word 2011 was used as described in other investigations of orthopedic PEM readability^2,10,12,13: Its readability function is enabled by going to the Tools tab and then to the Spelling & Grammar tool, where the “Show readability statistics” option is selected.¹⁰ Readability scores are calculated with the Spelling & Grammar tool; the readability score is displayed after completion of the spelling-and-grammar check. The formula used to calculate FK grade level is¹⁵: (0.39 × average number of words per sentence) + (11.8 × average number of syllables per word) – 15.59.

Statistical Analysis

Descriptive statistics, including means and 95% confidence intervals (CIs), were calculated for the FK grade levels. Student t tests were used to compare average FK grade levels of articles written exclusively by AAOS with those of articles coauthored by AAOS and other orthopedic subspecialty societies. A 2-sample unequal-variance t test was used, and significance was set at P < .05. Total number of articles written at or below the sixth- and eighth-grade levels, the reading levels recommended for PEMs, were tabulated.^1,9-12 Intraobserver and interobserver reliabilities were calculated with intraclass correlation coefficients (ICCs): Mr. Mohan, who calculated the FK scores earlier, now 1 week later calculated the readability levels of 15 randomly selected articles^10,11; in addition, Mr. Mohan and Dr. Yi independently calculated the readability levels of 30 randomly selected articles.^10,11 The same method described earlier—edit plain-text files, then use Microsoft Word to obtain FK scores—was again used. ICCs of 0 to 0.24 correspond to poor correlation; 0.25 to 0.49, low correlation; 0.5 to 0.69, fair correlation; 0.7 to 0.89, good correlation; and 0.9 to 1.0, excellent correlation.^10,11 All statistical analyses were performed with Microsoft Excel 2011 and VassarStats (http://vassarstats.net/tu.html).

 

Results

Of the 115 AAOS website articles included in the study and reviewed, 18 were coauthored by OTA, 10 by AOSSM, 14 by POSNA, 2 by ASSH, 2 by ASES, 1 by AAHKS, 3 by AOFAS, 1 by AOSSM and ASES, and 1 by AOFAS and AOSSM.

Mean FK grade level was 9.1 (range, 6.2-12; 95% CI, 8.9-9.3) for all articles reviewed and 9.1 (range, 6.2-12; 95% CI, 8.8-9.4) for articles exclusively written by AAOS. For coauthored articles, mean FK grade level was 9.3 (range, 7.6-11.3; 95% CI, 8.8-9.8) for AAOS-OTA; 8.9 (range, 7.4-10.4; 95% CI, 8.4-9.6) for AAOS-AOSSM; 9.4 (range, 7-11.8; 95% CI, 8.9-10.1) for AAOS-POSNA; 7.8 (range, 7.8-9.1; 95% CI, 7.2-9.8) for AAOS-ASSH; 9 (range, 8.2-9.6; 95% CI, 7.6-10.2) for AAOS-ASES; 9 (range, 7.9-9; 95% CI, 7.9-9.3) for AAOS-AOFAS; 8.1 for the 1 AAOS-AAHKS article; 8.5 for the 1 AAOS-AOSSM-ASES article; and 8 for the 1 AAOS-AOFAS-AOSSM article (Figure).

Nineteen articles (16.5%) were found to be at or below the eighth-grade reading level, which is the average reading level of a US adult,¹⁰ and only 1 article was at or below the sixth-grade level, the level widely recommended for PEMs.¹¹ In addition, there was no statistically significant difference between articles coauthored by the various orthopedic subspecialties and those written exclusively by AAOS.

For FK readability calculations, interobserver reliability (ICC, 0.9982) and intraobserver reliability (ICC, 1) were both excellent.

Discussion

Although increasing numbers of patients are using information from the Internet to inform their healthcare decisions,¹² studies have shown that online PEMs are written at a readability level above that of the average patient.^1,9,13 In the present study, we also found that OT-PEMs from AAOS are written at a level considerably higher than the recommended sixth-grade reading level,¹⁶ potentially impairing patient comprehension and leading to poorer health outcomes.¹⁷

The pervasiveness of too-high PEM readability levels has been found across orthopedic subspecialties.^2,9,12,13 Following this trend, the OT articles we reviewed had a ninth-grade reading level on average, and only 1 of 115 articles was below the recommended sixth-grade level.¹⁰ The issue of too-high PEM readability levels is thus a problem both in OT and in orthopedics in general. Accordingly, efforts to address this problem are warranted, especially as orthopedic PEM readability has not substantially improved over the past several years.¹⁸In this study, we also tried to identify any readability differences between articles coauthored by orthopedic societies and articles that were not coauthored by orthopedic societies. We hypothesized that multidisciplinary authorship could improve PEM readability; for example, orthopedic societies could collaborate with other medical specialties (eg, family medicine) that have produced appropriately readable PEMs. One study found that the majority of PEMs from the American Academy of Family Physicians (AAFP) were written below the sixth-grade reading level because of strict organizational regulation of the production of such materials.¹⁹ By noting and adopting successful PEM development methods used by groups such as AAFP,^19,20 we might be able to improve OT-PEM readability. However, this was not the case in our study, though our observations may have been limited by the small sample of reviewable articles.

One factor contributing to the poor readability of orthopedic PEMs is that orthopedics terminology is complex and includes words that are often difficult to translate into simpler terms without losing their meaning.¹⁰ When PEMs are written at a level that is too complex, patients cannot fully comprehend them, which may lead to poor health literacy. This problem may be even more harmful when considering the poor literacy levels of patients at baseline. Kadakia and colleagues¹⁶ found that OT patients had poor health literacy; for example, fewer than half knew which bone they fractured. As health literacy is associated with poorer health outcomes and reduced use of healthcare services,²¹ optimizing patients’ health literacy is of crucial importance to both their education and their outcomes.

Our study should be viewed in light of some important limitations. As OTA does not publish its own PEMs, we assessed only OT-related articles that were available on the AAOS website and were exclusively written by AAOS, or coauthored by AAOS and by OTA and/or another orthopedic subspecialty organization. As these articles represent only a subset of the full spectrum of OT-PEMs available on the Internet, our results may not be generalizable to the entire scope of such materials. However, as AAOS and OTA represent the most authoritative OT organizations, we think these PEMs would be among those most likely to be recommended to patients by their surgeons. In addition, although we used a well-established tool for examining readability—the FK readability scale^10-13—this tool has its own inherent limitations, as FK readability grade level is calculated purely on the basis of words per sentence and total syllables per word, and does not take into account other article elements, such as images, which also provide information.^1,10 Nevertheless, the FK scale is an inexpensive, easily accessed readability tool that provides a reproducible readability value that is easily comparable to results from earlier studies.¹⁰ The final limitation is that we excluded from the study AAOS website articles written in a language other than English. Such articles, however, are important, as a large portion of the patient population speaks English as a second language. Indeed, the readability of Spanish PEMs has been investigated—albeit using a readability measure other than the FK scale—and may be a topic pertinent to orthopedic PEMs.²²Most of the literature on the readability of orthopedic PEMs has found their reading levels too high for the average patient to comprehend.^1,9-12 The trend continues with our study findings regarding OT-PEMs available online from AAOS. Although the literature on the inadequacies of orthopedic PEMs is vast,^1,9-12 more work is needed to improve the quality, accuracy, and readability of these materials. There has been some success in improving PEM readability and producing appropriately readable materials within the medical profession,^19,23 so we know that appropriately readable orthopedic PEMs are feasible.

Am J Orthop. 2017;46(3):E190-E194. Copyright Frontline Medical Communications Inc. 2017. All rights reserved.

 

Take-Home Points

• The Flesch-Kincaid Readability Scale is a useful tool in evaluating the readability of PEMs.
• Only 1 article analyzed in our study was below a sixth-grade readability level.
• Coauthorship of PEMs with other subspecialty groups had no effect on readability.
• Poor health literacy has been associated with poor health outcomes.
• Efforts must be undertaken to make PEMs more readable across medical subspecialties.

Patients increasingly turn to the Internet to self-educate about orthopedic conditions.^1,2 Accordingly, the Internet has become a valuable tool in maintaining effective physician-patient communication.^3-5 Given the Internet’s importance as a medium for conveying patient information, it is important that orthopedic patient education materials (PEMs) on the Internet provide high-quality information that is easily read by the target patient population. Unfortunately, studies have found that many of the Internet’s orthopedic PEMs have been neither of high quality^6-8 nor presented such that they are easy for patients to read and comprehend.^1,9-12

Readability, which is the reading comprehension level (school grade level) a person must have to understand written materials, is determined by systematic formulae¹²; readability levels correlate with the ability to comprehend written information.² Studies have consistently found that orthopedic PEMs are written at readability levels too high for the average patient to understand.^1,9,13 The readability of PEMs in orthopedics as a whole⁹ and within the orthopedic subspecialties of arthroplasty,¹ foot and ankle surgery,² sports medicine,¹² and spine surgery¹³ has been evaluated, but so far there has been no evaluation of PEMs in orthopedic trauma (OT).

We conducted a study to assess the readability of OT-PEMs available online from the American Academy of Orthopaedic Surgeons (AAOS) in conjunction with the Orthopaedic Trauma Association (OTA) and other orthopedic subspecialty societies. We hypothesized the readability levels of these OT-PEMs would be above the level (sixth to eighth grade) recommended by several healthcare organizations, including the Centers for Disease Control and Prevention.^9,11,14 We also assessed the effect that orthopedic subspecialty coauthorship has on PEM readability.

Methods

In July 2014, we searched the AAOS online patient education library (Broken Bones & Injuries section, http://orthoinfo.aaos.org/menus/injury.cfm) and the AAOS OrthoPortal website (Trauma section, http://pubsearch.aaos.org/search?q=trauma&client=OrthoInfo&site=PATIENT&output=xml_no_dtd&proxystylesheet=OrthoInfo&filter=0) for all relevant OT-PEMs. Although OTA does not publish its own PEMs on its website, it coauthored several of the articles in the AAOS patient education library. Other subspecialty organizations, including the American Orthopaedic Society for Sports Medicine (AOSSM), the American Society for Surgery of the Hand (ASSH), the Pediatric Orthopaedic Society of North America (POSNA), the American Shoulder and Elbow Surgeons (ASES), the American Association of Hip and Knee Surgeons (AAHKS), and the American Orthopaedic Foot and Ankle Society (AOFAS), coauthored several of these online OT-PEMs as well.

Using the technique described by Badarudeen and Sabharwal,¹⁰ we saved all articles to be included in the study as separate Microsoft Word 2011 files. We saved them in plain-text format to remove any HTML tags and any other hidden formatting that might affect readability results. Then we edited them to remove elements that might affect readability result accuracy—deleted article topic–unrelated information (eg, copyright notice, disclaimers, author information) and all numerals, decimal points, bullets, abbreviations, paragraph breaks, colons, semicolons, and dashes.¹⁰Mr. Mohan used the Flesch-Kincaid (FK) Readability Scale to calculate grade level for each article. Microsoft Word 2011 was used as described in other investigations of orthopedic PEM readability^2,10,12,13: Its readability function is enabled by going to the Tools tab and then to the Spelling & Grammar tool, where the “Show readability statistics” option is selected.¹⁰ Readability scores are calculated with the Spelling & Grammar tool; the readability score is displayed after completion of the spelling-and-grammar check. The formula used to calculate FK grade level is¹⁵: (0.39 × average number of words per sentence) + (11.8 × average number of syllables per word) – 15.59.

Statistical Analysis

Descriptive statistics, including means and 95% confidence intervals (CIs), were calculated for the FK grade levels. Student t tests were used to compare average FK grade levels of articles written exclusively by AAOS with those of articles coauthored by AAOS and other orthopedic subspecialty societies. A 2-sample unequal-variance t test was used, and significance was set at P < .05. Total number of articles written at or below the sixth- and eighth-grade levels, the reading levels recommended for PEMs, were tabulated.^1,9-12 Intraobserver and interobserver reliabilities were calculated with intraclass correlation coefficients (ICCs): Mr. Mohan, who calculated the FK scores earlier, now 1 week later calculated the readability levels of 15 randomly selected articles^10,11; in addition, Mr. Mohan and Dr. Yi independently calculated the readability levels of 30 randomly selected articles.^10,11 The same method described earlier—edit plain-text files, then use Microsoft Word to obtain FK scores—was again used. ICCs of 0 to 0.24 correspond to poor correlation; 0.25 to 0.49, low correlation; 0.5 to 0.69, fair correlation; 0.7 to 0.89, good correlation; and 0.9 to 1.0, excellent correlation.^10,11 All statistical analyses were performed with Microsoft Excel 2011 and VassarStats (http://vassarstats.net/tu.html).

 

Results

Of the 115 AAOS website articles included in the study and reviewed, 18 were coauthored by OTA, 10 by AOSSM, 14 by POSNA, 2 by ASSH, 2 by ASES, 1 by AAHKS, 3 by AOFAS, 1 by AOSSM and ASES, and 1 by AOFAS and AOSSM.

Mean FK grade level was 9.1 (range, 6.2-12; 95% CI, 8.9-9.3) for all articles reviewed and 9.1 (range, 6.2-12; 95% CI, 8.8-9.4) for articles exclusively written by AAOS. For coauthored articles, mean FK grade level was 9.3 (range, 7.6-11.3; 95% CI, 8.8-9.8) for AAOS-OTA; 8.9 (range, 7.4-10.4; 95% CI, 8.4-9.6) for AAOS-AOSSM; 9.4 (range, 7-11.8; 95% CI, 8.9-10.1) for AAOS-POSNA; 7.8 (range, 7.8-9.1; 95% CI, 7.2-9.8) for AAOS-ASSH; 9 (range, 8.2-9.6; 95% CI, 7.6-10.2) for AAOS-ASES; 9 (range, 7.9-9; 95% CI, 7.9-9.3) for AAOS-AOFAS; 8.1 for the 1 AAOS-AAHKS article; 8.5 for the 1 AAOS-AOSSM-ASES article; and 8 for the 1 AAOS-AOFAS-AOSSM article (Figure).

Nineteen articles (16.5%) were found to be at or below the eighth-grade reading level, which is the average reading level of a US adult,¹⁰ and only 1 article was at or below the sixth-grade level, the level widely recommended for PEMs.¹¹ In addition, there was no statistically significant difference between articles coauthored by the various orthopedic subspecialties and those written exclusively by AAOS.

For FK readability calculations, interobserver reliability (ICC, 0.9982) and intraobserver reliability (ICC, 1) were both excellent.

Discussion

Although increasing numbers of patients are using information from the Internet to inform their healthcare decisions,¹² studies have shown that online PEMs are written at a readability level above that of the average patient.^1,9,13 In the present study, we also found that OT-PEMs from AAOS are written at a level considerably higher than the recommended sixth-grade reading level,¹⁶ potentially impairing patient comprehension and leading to poorer health outcomes.¹⁷

The pervasiveness of too-high PEM readability levels has been found across orthopedic subspecialties.^2,9,12,13 Following this trend, the OT articles we reviewed had a ninth-grade reading level on average, and only 1 of 115 articles was below the recommended sixth-grade level.¹⁰ The issue of too-high PEM readability levels is thus a problem both in OT and in orthopedics in general. Accordingly, efforts to address this problem are warranted, especially as orthopedic PEM readability has not substantially improved over the past several years.¹⁸In this study, we also tried to identify any readability differences between articles coauthored by orthopedic societies and articles that were not coauthored by orthopedic societies. We hypothesized that multidisciplinary authorship could improve PEM readability; for example, orthopedic societies could collaborate with other medical specialties (eg, family medicine) that have produced appropriately readable PEMs. One study found that the majority of PEMs from the American Academy of Family Physicians (AAFP) were written below the sixth-grade reading level because of strict organizational regulation of the production of such materials.¹⁹ By noting and adopting successful PEM development methods used by groups such as AAFP,^19,20 we might be able to improve OT-PEM readability. However, this was not the case in our study, though our observations may have been limited by the small sample of reviewable articles.

One factor contributing to the poor readability of orthopedic PEMs is that orthopedics terminology is complex and includes words that are often difficult to translate into simpler terms without losing their meaning.¹⁰ When PEMs are written at a level that is too complex, patients cannot fully comprehend them, which may lead to poor health literacy. This problem may be even more harmful when considering the poor literacy levels of patients at baseline. Kadakia and colleagues¹⁶ found that OT patients had poor health literacy; for example, fewer than half knew which bone they fractured. As health literacy is associated with poorer health outcomes and reduced use of healthcare services,²¹ optimizing patients’ health literacy is of crucial importance to both their education and their outcomes.

Our study should be viewed in light of some important limitations. As OTA does not publish its own PEMs, we assessed only OT-related articles that were available on the AAOS website and were exclusively written by AAOS, or coauthored by AAOS and by OTA and/or another orthopedic subspecialty organization. As these articles represent only a subset of the full spectrum of OT-PEMs available on the Internet, our results may not be generalizable to the entire scope of such materials. However, as AAOS and OTA represent the most authoritative OT organizations, we think these PEMs would be among those most likely to be recommended to patients by their surgeons. In addition, although we used a well-established tool for examining readability—the FK readability scale^10-13—this tool has its own inherent limitations, as FK readability grade level is calculated purely on the basis of words per sentence and total syllables per word, and does not take into account other article elements, such as images, which also provide information.^1,10 Nevertheless, the FK scale is an inexpensive, easily accessed readability tool that provides a reproducible readability value that is easily comparable to results from earlier studies.¹⁰ The final limitation is that we excluded from the study AAOS website articles written in a language other than English. Such articles, however, are important, as a large portion of the patient population speaks English as a second language. Indeed, the readability of Spanish PEMs has been investigated—albeit using a readability measure other than the FK scale—and may be a topic pertinent to orthopedic PEMs.²²Most of the literature on the readability of orthopedic PEMs has found their reading levels too high for the average patient to comprehend.^1,9-12 The trend continues with our study findings regarding OT-PEMs available online from AAOS. Although the literature on the inadequacies of orthopedic PEMs is vast,^1,9-12 more work is needed to improve the quality, accuracy, and readability of these materials. There has been some success in improving PEM readability and producing appropriately readable materials within the medical profession,^19,23 so we know that appropriately readable orthopedic PEMs are feasible.

Am J Orthop. 2017;46(3):E190-E194. Copyright Frontline Medical Communications Inc. 2017. All rights reserved.

 

Interhospital transfer (IHT) is defined as the transfer of hospitalized patients between acute care hospitals. Although cited reasons for transfer include providing patients access to unique specialty services,¹ patterns and practices of IHT remain largely unstudied. Interhospital transfer is known to be common in certain patient populations, including selected patients presenting to the intensive care unit² and those with acute myocardial infarction (AMI),^3-5 but no recent studies have looked at frequency of IHT among a broader group of hospitalized patients nationally. Little is known about which patients are selected for transfer and why.⁶ Limited evidence suggests poor concordance between cited reason for transfer among patients, transferring physicians, and receiving physicians,⁷ indicating ambiguity in this care process.

Interhospital transfer exposes patients to the potential risks associated with discontinuity of care. Communication is particularly vulnerable to error during times of transition.^8-10 Patients transferred between acute care hospitals are especially vulnerable, given the severity of illness in this patient population,¹¹ and the absence of other factors to fill in gaps in communication, such as common electronic health records. Limited existing literature suggests transferred patients use more resources ^12-13 and experience worse outcomes compared to nontransferred patients,¹¹ although these data involved limited patient populations, and adjustment for illness severity and other factors was variably addressed.^14-16

To improve the quality and safety of IHT, therefore, it is necessary to understand which patients benefit from IHT and identify best practices in the IHT process.¹⁷ A fundamental first step is to study patterns and practices of IHT, in particular with an eye towards identifying unwarranted variation.¹⁸ This is important to understand the prevalence of the issue, provide possible evidence of lack of standardization, and natural experiments with which to identify best practices.

To address this, we conducted a foundational study examining a national sample of Medicare patients to determine the nationwide frequency of IHT among elderly patients, patient and hospital-level predictors of transfer, and hospital variability in IHT practices.

METHODS

We performed a cross-sectional analysis using 2 nationally representative datasets: (1) Center for Medicare and Medicaid Services (CMS) 2013 100% Master Beneficiary Summary and Inpatient claims files, which contains data on all fee-for-service program Medicare enrollees’ demographic information, date of death, and hospitalization claims, including ICD-9 codes for diagnoses, diagnosis-related group (DRG), and dates of service; merged with (2) 2013 American Hospital Association (AHA) data,¹⁹ which contains hospital-level characteristics for all acute care hospitals in the U.S. Our study protocol was approved by the Partners Healthcare Human Subjects Review Committee.

 

Beneficiaries were eligible for inclusion if they were 65 years or older, continuously enrolled in Medicare A and B, with an acute care hospitalization claim in 2013, excluding Medicare managed care and end-stage renal disease (ESRD) beneficiaries. We additionally excluded beneficiaries hospitalized at federal or nonacute care hospitals, or critical access hospitals given their mission to stabilize and transfer patients to referral hospitals.²⁰

Transferred patients were defined as: (1) beneficiaries with a “transfer out” claim and a corresponding “transfer in” claim at a different hospital; as well as (2) beneficiaries with a “transfer out” claim and a corresponding date of admission to another hospital within 1 day following the date of claim; and (3) beneficiaries with a “transfer in” claim and a corresponding date of discharge from another hospital within 1 day preceding the date of claim. Beneficiaries transferred to the same hospital, or cared for at hospitals with “outlier” transfer in rates equal to 100% or transfer out rates greater than 35%, were excluded from analysis given the suggestion of nonstandard claims practices. Beneficiaries with greater than 1 transfer within the same hospitalization were additionally excluded.

Patient Characteristics

Patient characteristics were obtained from the CMS data files and included: demographics (age, sex, race); DRG-weight, categorized into quartiles; primary diagnosis for the index hospitalization using ICD-9 codes; patient comorbidity using ICD-9 codes compiled into a CMS-Hierarchical Condition Category (HCC) risk score;²¹ presence of Medicaid co-insurance; number of hospitalizations in the past 12 months, categorized into 0, 1, 2-3, and 4 or more; season, defined as calendar quarters; and median income per household by census tract. These characteristics were chosen a priori given expert opinion in combination with prior research demonstrating association with IHT.^11,22

Hospital Characteristics

Hospital characteristics were obtained from AHA data files and included hospitals’ size, categorized into small, medium, and large (less than 100, 100 to 399, 400 or more beds); geographic location; ownership; teaching status; setting (urban vs. rural); case mix index (CMI) for all patients cared for at the hospital; and presence of selected specialty services, including certified trauma center, medical intensive care unit, cardiac intensive care unit, cardiac surgery services, adult interventional cardiac catheterization, adult cardiac electrophysiology, and composite score of presence of 55 other specialty services (complete list in Appendix A). All characteristics were chosen a priori given expert opinion or relationship of characteristics with IHT, and prior research utilizing AHA data.^23-24

Analysis

Descriptive statistics were used to evaluate the frequency of IHT, characteristics of transferred patients, and number of days to transfer. Patient and hospital characteristics of transferred vs. nontransferred patients were compared using chi-square analyses.

To analyze the effects of each patient and hospital characteristic on the odds of transfer, we used logistic regression models incorporating all patient and hospital characteristics, accounting for fixed effects for diagnosis, and utilizing generalized estimating equations (the GENMOD procedure in SAS statistical software, v 9.4; SAS Institute Inc., Cary, North Carolina) to account for the clustering of patients within hospitals.²⁵ Indicator variables were created for missing covariate data and included in analyses when missing data accounted for greater than 10% of the total cohort.

To measure the variability in transfer rates between hospitals, we used a sequence of random effects logistic regression models. We first ran a model with no covariates, representing the unadjusted differences in transfer rates between hospitals. We then added patient characteristics to see if the unadjusted differences in IHT rates were explained by differences in patient characteristics between hospitals. Lastly, we added hospital characteristics to determine if these explained the remaining differences in transfer rates. Each of the 3 models provided a measure of between-hospital variability, reflecting the degree to which IHT rates differed between hospitals. Additionally, we used the intercept from the unadjusted model and the measure of between-hospital variability from each model to calculate the 95% confidence intervals, illustrating the range of IHT rates spanning 95% of all hospitals. We used those same numbers to calculate the 25^th and 75^th percentiles, illustrating the range of IHT rates for the middle half of hospitals.

RESULTS

Among 28 million eligible beneficiaries, 6.6 million had an acute care hospitalization to nonfederal, noncritical access hospitals, and 107,741 met our defined criteria for IHT. An additional 3790 beneficiaries were excluded for being transferred to the same facility, 416 beneficiaries (115 transferred, 301 nontransferred) were excluded as they were cared for at 1 of the 11 hospitals with “outlier” transfer in/out rates, and 2329 were excluded because they had more than 1 transfer during hospitalization. Thus, the final cohort consisted of 101,507 transferred (1.5%) and 6,625,474 nontransferred beneficiaries (Figure 1). Of the 101,507 transferred beneficiaries, 2799 (2.8%) were included more than once (ie, experienced more than 1 IHT on separate hospitalizations throughout the study period; the vast majority of these had 2 separate hospitalizations resulting in IHT). Characteristics of transferred and nontransferred beneficiaries are shown (Table 1).

 

Among transferred patients, the top 5 primary diagnoses at time of transfer included AMI (12.2%), congestive heart failure (CHF) (7.2%), sepsis (6.6%), arrhythmia (6.6%), and pneumonia (3.4%). Comorbid conditions most commonly present in transferred patients included CHF (52.6%), renal failure (51.8%), arrhythmia (49.8%), and chronic obstructive pulmonary disease (COPD; 37.0%). The most common day of transfer was day after admission (hospital day 2, 24.7%), with 75% of transferred patients transferred before hospital day 6 (Appendix B).

After adjusting for all other patient and hospital characteristics and clustering by hospital, the following variables were associated with greater odds of transfer: older age, male sex, nonblack race, non-Medicaid co-insurance, higher comorbidity (HCC score), lower DRG-weight, and fewer hospitalizations in the prior 12 months. Beneficiaries also had greater odds of transfer if initially hospitalized at smaller hospitals, nonteaching hospitals, public hospitals, at hospitals in the Northeast, those with fewer specialty services, and those with a low CMI (Table 2).

In examining the between-hospital variability in IHT, our unadjusted model estimated an average transfer rate of 1.79%, and showed a variance estimate of 1.33 (P=0.009), demonstrating that 95% of hospitals have transfer rates between 0.83% and 3.80%. The variance estimate increased by 19% to 1.58 (P=0.009) when adjusting for patient characteristics. After adjusting for hospital characteristics, variance decreased by 83% to 0.28 (P=0.01), showing 95% of hospitals have transfer rates between 1.26% and 2.54% (Figure 2).

DISCUSSION

In this nationally representative study of 6.6 million Medicare beneficiaries, we found that 1.5% of patients were transferred between acute care facilities and were most often transferred prior to hospital day 6. Older age, male sex, nonblack race, higher medical comorbidity, lower DRG weight, and fewer recent hospitalizations were associated with greater odds of transfer. Initial hospitalization at smaller, nonteaching, public hospitals, with fewer specialty services were associated with greater odds of transfer, while higher CMI was associated with a lower odds of transfer. The most common comorbid conditions among transferred patients included CHF, renal failure, arrhythmia, and COPD; particularly notable was the very high prevalence of these conditions among transferred as compared with nontransferred patients. Importantly, we found significant variation in IHT by region and a large variation in transfer practices by hospital, with significant variability in transfer rates even after accounting for known patient and hospital characteristics.

Among our examined population, we found that a sizable number of patients undergo IHT—more than 100,000 per year. Primary diagnoses at time of transfer consist of common inpatient conditions, including AMI, CHF, sepsis, arrhythmia, and pneumonia. Limited prior data support our findings, with up to 50% of AMI patients reportedly undergoing IHT,^3-5 and severe sepsis and respiratory illness reported as common diagnoses at transfer.¹¹ Although knowledge of these primary diagnoses does not directly confer an understanding of reason for transfer, one can speculate based on our findings. For example, research demonstrates the majority of AMI patients who undergo IHT had further intervention, including stress testing, cardiac catheterization, and/or coronary artery bypass graft surgery.^5,26 Thus, it is reasonable to presume that many of the beneficiaries

transferred with AMI were transferred to receive this more specialized cardiac care. We further found the majority of patients are transferred prior to hospital day 6 with the highest prevalence on day 2, supporting the hypothesis that these patients may be transferred for receipt of specialty services for their admission diagnosis. However, we cannot prove this presumption, and for other conditions, such as pneumonia, the plan after IHT is less obvious. There are numerous possible reasons for transfer,¹ including patient preference and prior affiliation with receiving hospital. Further research is required to more fully define these reasons in greater detail.

We additionally found that certain patient characteristics were associated with greater odds of transfer. Research suggests that transferred patients are “sicker” than nontransferred patients.^1,11 Although our findings in part confirm these data, we paradoxically found that higher DRG-weight and 4 or more hospitalizations in the past year were actually associated with lower odds of transfer. In addition, the oldest patients in our cohort (85 years or older) were actually less likely to be transferred than their slightly younger counterparts (75 to 84 years). These variables may reflect extreme illness or frailty,²⁷ and providers consciously (or subconsciously) may factor this in to their decision to transfer, considering a threshold past which transfer would confer more risk than benefit (eg, a patient may be “too sick” for transfer). Indeed, in a secondary analysis without hospital characteristics or comorbidities, and with fixed effects by hospital, we found the highest rates of IHT in patients in the middle 2 quartiles of DRG-weight, supporting this threshold hypothesis. It is also possible that patients with numerous hospitalizations may be less likely to be transferred because of familiarity and a strong sense of responsibility to continue to care for those patients (although we cannot confirm that those prior hospitalizations were all with the same index hospital).

It is also notable that odds of transfer differed by race, with black patients 17% less likely to undergo transfer compared to whites, similar to findings in other IHT studies.¹¹ This finding, in combination with our demonstration that Medicaid patients also have lower odds of transfer, warrants further investigation to ensure the process of IHT does not bias against these populations, as with other well-documented health disparities.^28-30

The hospital predictors of transfer were largely expected. However, interestingly, when we controlled for all other patient and hospital characteristics, regional variation persisted, with highest odds of transfer with hospitalization in the Northeast, indicating variability by region not explained by other factors, and findings supported by other limited data.³¹ This variability was further elucidated in our examination of change in variance estimates accounting for patient, then hospital, characteristics. Although we expected and found marked variability in hospital transfer rates in our null model (without accounting for any patient or hospital characteristics), we interestingly found that variability increased upon adjusting for patient characteristics. This result is presumably due to the fact that patients who are more likely to be transferred (ie, “sick” patients) are more often already at hospitals less likely to transfer patients, supported by our findings that hospital CMI is inversely associated with odds of transfer (in other words, hospitals that care for a less sick patient population are more likely to transfer their patients, and hospitals that care for a sicker patient population [higher CMI] are less likely to transfer). Adjusting solely for patient characteristics effectively equalizes these patients across hospitals, which would lead to even increased variability in transfer rates. Conversely, when we then adjusted for hospital characteristics, variability in hospital transfer rates decreased by 83% (in other words, hospital characteristics, rather than patient characteristics, explained much of the variability in transfer rates), although significant unexplained variability remained. We should note that although the observed reduction in variability was explained by the patient and hospital characteristics included in the model, these characteristics do not necessarily justify the variability they accounted for; although patients’ race or hospitals’ location may explain some of the observed variability, this does not reasonably justify it.

This observed variability in transfer practices is not surprising given the absence of standardization and clear guidelines to direct clinical IHT practice.¹⁷ Selection of patients that may benefit from transfer is often ambiguous and subjective.⁶ The Emergency Medical Treatment and Active Labor Act laws dictate that hospitals transfer patients requiring a more specialized service, or when “medical benefits ... outweigh the increased risks to the individual...,” although in practice this provides little guidance to practitioners.¹ Thus, clearer guidelines may be necessary to achieve less variable practices.

Our study is subject to several limitations. First, although nationally representative, the Medicare population is not reflective of all hospitalized patients nationwide. Additionally, we excluded patients transferred from the emergency room. Thus, the total number of patients who undergo IHT nationally is expected to be much higher than reflected in our analysis. We also excluded patients who were transferred more than once during a given hospitalization. This enabled us to focus on the initial transfer decision but does not allow us to look at patients who are transferred to a referral center and then transferred back. Second, given the criteria we used to define transfer, it is possible that we included nontransferred patients within our transferred cohort if they were discharged from one hospital and admitted to a different hospital within 1 day. However, on quality assurance analyses where we limited our cohort to only those beneficiaries with corresponding “transfer in” and “transfer out” claims (87% of the total cohort), we found no marked differences in our results. Additionally, although we assume that patient transfer status was coded correctly within the Medicare dataset, we could not confirm by individually examining each patient we defined as “transferred.” However, on additional quality assurance analyses where we examined randomly selected excluded patients with greater than 1 transfer during hospitalization, we found differing provider numbers with each transfer, suggesting validity of the coding. Third, because there are likely many unmeasured patient confounders, we cannot be sure how much of the between-hospital variation is due to incomplete adjustment for patient characteristics. However, since adjusting for patient characteristics actually increased variability in hospital transfer rates, it is unlikely that residual patient confounders fully explain our observed results. Despite this, other variables that are not available within the CMS or AHA datasets may further elucidate hospital transfer practices, including variables reflective of the transfer process (eg, time of day of patient transfer, time delay between initiation of transfer and patient arrival at accepting hospital, accepting service on transfer, etc.); other markers of illness severity (eg, clinical service at the time of index admission, acute physiology score, utilization of critical care services on arrival at receiving hospital); and other hospital system variables (ie, membership in an accountable care organization and/or regional care network, the density of nearby tertiary referral centers (indicating possible supply-induced demand), other variables reflective of the “transfer culture” (such as the transfer rate at the hospital or region where the attending physician trained, etc.). Lastly, though our examination provides important foundational information regarding IHT nationally, this study did not examine patient outcomes in transferred and nontransferred patients, which may help to determine which patients benefit (or do not benefit) from transfer and why. Further investigation is needed to study these outcomes.

 

CONCLUSION

In this national study of IHT, we found that a sizable number of patients admitted to the hospital undergo transfer to another acute care facility. Patients are transferred with common medical conditions, including those requiring specialized care such as AMI, and a high rate of comorbid clinical conditions, and certain patient and hospital characteristics are associated with greater odds of transfer. Although many of the observed associations between characteristics and odds of transfer were expected based on limited existing literature, we found several unexpected findings, eg, suggesting the possibility of a threshold beyond which sicker patients are not transferred. Additionally, we found that black and Medicaid patients had lower odds of transfer, which warrants further investigation for potential health care disparity. Importantly, we found much variability in the practice of IHT, as evidenced by the inexplicable differences in transfer by hospital region, and by residual unexplained variability in hospital transfer rates after accounting for patient and hospital characteristics, which may be due to lack of standard guidelines to direct IHT practices. In conclusion, this study of hospitalized Medicare patients provides important foundational information regarding rates and predictors of IHT nationally, as well as unexplained variability that exists within this complex care transition. Further investigation will be essential to understand reasons for, processes related to, and outcomes of transferred patients, to help guide standardization in best practices in care.

Disclosure

Nothing to report.