Original Research

Unplanned hospital readmissions are frequent, costly, and potentially avoidable.[1, 2] Due to major federal financial readmissions penalties targeting excessive 30‐day readmissions, there is increasing attention to implementing hospital‐initiated interventions to reduce readmissions.[3, 4] However, universal enrollment of all hospitalized patients into such programs may be too resource intensive for many hospitals.[5] To optimize efficiency and effectiveness, interventions should be targeted to individuals most likely to benefit.[6, 7] However, existing readmission risk‐prediction models have achieved only modest discrimination, have largely used administrative claims data not available until months after discharge, or are limited to only a subset of patients with Medicare or a specific clinical condition.[8, 9, 10, 11, 12, 13, 14] These limitations have precluded accurate identification of high‐risk individuals in an all‐payer general medical inpatient population to provide actionable information for intervention prior to discharge.

Approaches using electronic health record (EHR) data could allow early identification of high‐risk patients during the index hospitalization to enable initiation of interventions prior to discharge. To date, such strategies have relied largely on EHR data from the day of admission.[15, 16] However, given that variation in 30‐day readmission rates are thought to reflect the quality of in‐hospital care, incorporating EHR data from the entire hospital stay to reflect hospital care processes and clinical trajectory may more accurately identify at‐risk patients.[17, 18, 19, 20] Improved accuracy in risk prediction would help better target intervention efforts in the immediate postdischarge period, an interval characterized by heightened vulnerability for adverse events.[21]

To help hospitals target transitional care interventions more effectively to high‐risk individuals prior to discharge, we derived and validated a readmissions risk‐prediction model incorporating EHR data from the entire course of the index hospitalization, which we termed the full‐stay EHR model. We also compared the full‐stay EHR model performance to our group's previously derived prediction model based on EHR data on the day of admission, termed the first‐day EHR model, as well as to 2 other validated readmission models similarly intended to yield near real‐time risk predictions prior to or shortly after hospital discharge.[9, 10, 15]

METHODS

RESULTS

Overall, 32,922 index hospitalizations were included in our study cohort; 12.7% resulted in a 30‐day readmission (see Supporting Figure 1 in the online version of this article). Individuals had a mean age of 62 years and had diverse race/ethnicity and primary insurance status; half were female (Table 1). The study sample was randomly split into a derivation cohort (50%, n = 16,492) and validation cohort (50%, n = 16,430). Individuals in the derivation cohort with a 30‐day readmission had markedly different socioeconomic and clinical characteristics compared to those not readmitted (Table 1).

Derivation and Validation of the Full‐Stay EHR Model for 30‐Day Readmission

Our final model included 24 independent variables, including demographic characteristics, utilization history, clinical factors from the first day of admission, and clinical factors from the remainder of the hospital stay (Table 2). The strongest independent predictor of readmission was hospital‐acquired Clostridium difficile infection (adjusted odds ratio [AOR]: 2.03, 95% confidence interval [CI] 1.18‐3.48); other hospital‐acquired complications including pressure ulcers and venous thromboembolism were also significant predictors. Though having Medicaid was associated with increased odds of readmission (AOR: 1.55, 95% CI: 1.31‐1.83), other zip codelevel measures of socioeconomic disadvantage were not predictive and were not included in the final model. Being discharged to hospice was associated with markedly lower odds of readmission (AOR: 0.23, 95% CI: 0.13‐0.40).

Table 2.Final Full‐Stay EHR Model Predicting 30‐Day Readmissions (Derivation Cohort, N = 16,492)

	Odds Ratio (95% CI)
	Univariate	Multivariate*
NOTE: Abbreviations: CI, confidence interval; ED, emergency department. *Values shown reflect adjusted odds ratios and 95% CI for each factor after adjustment for all other factors listed in the table.
Demographic characteristics
Age, per 10 years	1.08 (1.051.11)	1.07 (1.041.10)
Medicaid	1.97 (1.702.29)	1.55 (1.311.83)
Widow	1.44 (1.281.63)	1.27 (1.111.45)
Utilization history
Prior ED visit, per visit	1.08 (1.061.10)	1.04 (1.021.06)
Prior hospitalization, per hospitalization	1.30 (1.271.34)	1.16 (1.121.20)
Hospital and clinical factors from first day of hospitalization
Nonelective admission	1.75 (1.512.03)	1.42 (1.221.65)
Charlson Comorbidity Index, per point	1.19 (1.171.21)	1.06 (1.041.09)
Laboratory abnormalities within 24 hours of admission
Albumin 2 g/dL	2.57 (1.823.62)	1.52 (1.052.21)
Albumin 23 g/dL	1.68 (1.501.88)	1.20 (1.061.36)
Aspartate aminotransferase >40 U/L	1.37 (1.221.55)	1.21 (1.061.38)
Creatine phosphokinase 60 g/L	1.48 (1.301.69)	1.28 (1.111.46)
Mean corpuscular volume >100 fL/red cell	1.68 (1.382.04)	1.32 (1.071.62)
Platelets 90 103/L	2.20 (1.772.72)	1.56 (1.231.97)
Platelets >350 103/L	1.34 (1.171.54)	1.24 (1.081.44)
Prothrombin time >35 seconds	2.58 (1.743.82)	1.92 (1.272.90)
Hospital and clinical factors from remainder of hospital stay
Length of stay, per day	1.08 (1.071.09)	1.06 (1.041.07)
Hospital complications
Clostridium difficile infection	3.61 (2.195.95)	2.03 (1.183.48)
Pressure ulcer	2.43 (1.733.41)	1.64 (1.152.34)
Venous thromboembolism	2.01 (1.362.96)	1.55 (1.032.32)
Laboratory abnormalities at discharge
Blood urea nitrogen >20 mg/dL	1.86 (1.702.04)	1.37 (1.241.52)
Sodium 135 mEq/L	1.70 (1.521.91)	1.34 (1.181.51)
Hematocrit 27	1.61 (1.401.85)	1.22 (1.051.41)
Vital sign instability at discharge, per instability	1.29 (1.201.40)	1.25 (1.151.36)
Discharged to hospice	0.51 (0.300.89)	0.23 (0.130.40)

In our validation cohort, the full‐stay EHR model had fair discrimination, with a C statistic of 0.69 (95% CI: 0.68‐0.70) (Table 3). The full‐stay EHR model was well calibrated across all quintiles of risk, with slight overestimation of predicted risk in the lowest and highest quintiles (Figure 1a) (see Supporting Table 5 in the online version of this article). It also effectively stratified individuals across a broad range of predicted readmission risk from 4.1% in the lowest decile to 36.5% in the highest decile (Table 3).

Table 3.Comparison of the Discrimination and Reclassification of Different Readmission Models*

Model Name	C‐Statistic (95% CI)	IDI, % (95% CI)	NRI (95% CI)	Average Predicted Risk, %
				Lowest Decile	Highest Decile
NOTE: Abbreviations; CI, confidence interval; EHR, electronic health record; IDI, Integrated Discrimination Improvement; NRI, Net Reclassification Index. *All measures were assessed using the validation cohort (N = 16,430), except for estimating the C‐statistic for the derivation cohort. P value 0.001 for all pairwise comparisons of C‐statistic between full‐stay model and first‐day, LACE, and HOSPITAL models, respectively. The LACE model includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year. The HOSPITAL model includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay.
Full‐stay EHR model
Derivation cohort	0.72 (0.70 to 0.73)			4.1	36.5
Validation cohort	0.69 (0.68 to 0.70)	[Reference]	[Reference]	4.1	36.5
First‐day EHR model	0.67 (0.66 to 0.68)	1.2 (1.4 to 1.0)	0.020 (0.038 to 0.002)	5.8	31.9
LACE model	0.65 (0.64 to 0.66)	2.6 (2.9 to 2.3)	0.046 (0.067 to 0.024)	6.1	27.5
HOSPITAL model	0.64 (0.62 to 0.65)	3.2 (3.5 to 2.9)	0.058 (0.080 to 0.035)	6.7	26.6

Figure 1

DISCUSSION

In this study, we used clinically detailed EHR data from the entire hospitalization on 32,922 individuals treated in 6 diverse hospitals to develop an all‐payer, multicondition readmission risk‐prediction model. To our knowledge, this is the first 30‐day hospital readmission risk‐prediction model to use a comprehensive set of factors from EHR data from the entire hospital stay. Prior EHR‐based models have focused exclusively on data available on or prior to the first day of admission, which account for clinical severity on admission but do not account for factors uncovered during the inpatient stay that influence the chance of a postdischarge adverse outcome.[15, 30] We specifically assessed the prognostic impact of a comprehensive set of factors from the entire index hospitalization, including hospital‐acquired complications, clinical trajectory, and stability on discharge in predicting hospital readmissions. Our full‐stay EHR model had statistically better discrimination, calibration, and diagnostic accuracy than our existing all‐cause first‐day EHR model[15] and 2 previously published readmissions models that included more limited information from hospitalization (such as length of stay).[9, 10] However, although the more complicated full‐stay EHR model was statistically better than previously published models, we were surprised that the predictive performance was only modestly improved despite the inclusion of many additional clinically relevant prognostic factors.

Taken together, our study has several important implications. First, the added complexity and resource intensity of implementing a full‐stay EHR model yields only modestly improved readmission risk prediction. Thus, hospitals and healthcare systems interested in targeting their highest‐risk individuals for interventions to reduce 30‐day readmission should consider doing so within the first day of hospital admission. Our group's previously derived and validated first‐day EHR model, which used data only from the first day of admission, qualitatively performed nearly as well as the full‐stay EHR model.[15] Additionally, a recent study using only preadmission EHR data to predict 30‐day readmissions also achieved similar discrimination and diagnostic accuracy as our full‐stay model.[30]

Second, the field of readmissions risk‐prediction modeling may be reaching the maximum achievable model performance using data that are currently available in the EHR. Our limited ability to accurately predict all‐cause 30‐day readmission risk may reflect the influence of currently unmeasured patient, system, and community factors on readmissions.[31, 32, 33] Due to the constraints of data collected in the EHR, we were unable to include several patient‐level clinical characteristics associated with hospital readmission, including self‐perceived health status, functional impairment, and cognition.[33, 34, 35, 36] However, given their modest effect sizes (ORs ranging from 1.062.10), adequately measuring and including these risk factors in our model may not meaningfully improve model performance and diagnostic accuracy. Further, many social and behavioral patient‐level factors are also not consistently available in EHR data. Though we explored the role of several neighborhood‐level socioeconomic measuresincluding prevalence of poverty, median income, education, and unemploymentwe found that none were significantly associated with 30‐day readmissions. These particular measures may have been inadequate to characterize individual‐level social and behavioral factors, as several previous studies have demonstrated that patient‐level factors such as social support, substance abuse, and medication and visit adherence can influence readmission risk in heart failure and pneumonia.[11, 16, 22, 25] This underscores the need for more standardized routine collection of data across functional, social, and behavioral domains in clinical settings, as recently championed by the Institute of Medicine.[11, 37] Integrating data from outside the EHR on postdischarge health behaviors, self‐management, follow‐up care, recovery, and home environment may be another important but untapped strategy for further improving prediction of readmissions.[25, 38]

Third, a multicondition readmission risk‐prediction model may be a less effective strategy than more customized disease‐specific models for selected conditions associated with high 30‐day readmission rates. Our group's previously derived and internally validated models for heart failure and human immunodeficiency virus had superior discrimination compared to our full‐stay EHR model (C statistic of 0.72 for each).[11, 13] However, given differences in the included population and time periods studied, a head‐to‐head comparison of these different strategies is needed to assess differences in model performance and utility.

Our study had several strengths. To our knowledge, this is the first study to rigorously measure the additive influence of in‐hospital complications, clinical trajectory, and stability on discharge on the risk of 30‐day hospital readmission. Additionally, our study included a large, diverse study population that included all payers, all ages of adults, a mix of community, academic, and safety net hospitals, and individuals from a broad array of racial/ethnic and socioeconomic backgrounds.

Our results should be interpreted in light of several limitations. First, though we sought to represent a diverse group of hospitals, all study sites were located within north Texas and generalizability to other regions is uncertain. Second, our ascertainment of prior hospitalizations and readmissions was more inclusive than what could be typically accomplished in real time using only EHR data from a single clinical site. We performed a sensitivity analysis using only prior utilization data available within the EHR from the index hospital with no meaningful difference in our findings (data not shown). Additionally, a recent study found that 30‐day readmissions occur at the index hospital for over 75% of events, suggesting that 30‐day readmissions are fairly comprehensively captured even with only single‐site data.[39] Third, we were not able to include data on outpatient visits before or after the index hospitalization, which may influence the risk of readmission.[1, 40]

In conclusion, incorporating clinically granular EHR data from the entire course of hospitalization modestly improves prediction of 30‐day readmissions compared to models that only include information from the first 24 hours of hospital admission or models that use far fewer variables. However, given the limited improvement in prediction, our findings suggest that from the practical perspective of implementing real‐time models to identify those at highest risk for readmission, it may not be worth the added complexity of waiting until the end of a hospitalization to leverage additional data on hospital complications, and the trajectory of laboratory and vital sign values currently available in the EHR. Further improvement in prediction of readmissions will likely require accounting for psychosocial, functional, behavioral, and postdischarge factors not currently present in the inpatient EHR.

Disclosures: This study was presented at the Society of Hospital Medicine 2015 Annual Meeting in National Harbor, Maryland, and the Society of General Internal Medicine 2015 Annual Meeting in Toronto, Canada. This work was supported by the Agency for Healthcare Research and Qualityfunded UT Southwestern Center for Patient‐Centered Outcomes Research (1R24HS022418‐01) and the Commonwealth Foundation (#20100323). Drs. Nguyen and Makam received funding from the UT Southwestern KL2 Scholars Program (NIH/NCATS KL2 TR001103). Dr. Halm was also supported in part by NIH/NCATS U54 RFA‐TR‐12‐006. The study sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no conflicts of interest to disclose.

Unplanned hospital readmissions are frequent, costly, and potentially avoidable.[1, 2] Due to major federal financial readmissions penalties targeting excessive 30‐day readmissions, there is increasing attention to implementing hospital‐initiated interventions to reduce readmissions.[3, 4] However, universal enrollment of all hospitalized patients into such programs may be too resource intensive for many hospitals.[5] To optimize efficiency and effectiveness, interventions should be targeted to individuals most likely to benefit.[6, 7] However, existing readmission risk‐prediction models have achieved only modest discrimination, have largely used administrative claims data not available until months after discharge, or are limited to only a subset of patients with Medicare or a specific clinical condition.[8, 9, 10, 11, 12, 13, 14] These limitations have precluded accurate identification of high‐risk individuals in an all‐payer general medical inpatient population to provide actionable information for intervention prior to discharge.

Approaches using electronic health record (EHR) data could allow early identification of high‐risk patients during the index hospitalization to enable initiation of interventions prior to discharge. To date, such strategies have relied largely on EHR data from the day of admission.[15, 16] However, given that variation in 30‐day readmission rates are thought to reflect the quality of in‐hospital care, incorporating EHR data from the entire hospital stay to reflect hospital care processes and clinical trajectory may more accurately identify at‐risk patients.[17, 18, 19, 20] Improved accuracy in risk prediction would help better target intervention efforts in the immediate postdischarge period, an interval characterized by heightened vulnerability for adverse events.[21]

To help hospitals target transitional care interventions more effectively to high‐risk individuals prior to discharge, we derived and validated a readmissions risk‐prediction model incorporating EHR data from the entire course of the index hospitalization, which we termed the full‐stay EHR model. We also compared the full‐stay EHR model performance to our group's previously derived prediction model based on EHR data on the day of admission, termed the first‐day EHR model, as well as to 2 other validated readmission models similarly intended to yield near real‐time risk predictions prior to or shortly after hospital discharge.[9, 10, 15]

METHODS

RESULTS

Overall, 32,922 index hospitalizations were included in our study cohort; 12.7% resulted in a 30‐day readmission (see Supporting Figure 1 in the online version of this article). Individuals had a mean age of 62 years and had diverse race/ethnicity and primary insurance status; half were female (Table 1). The study sample was randomly split into a derivation cohort (50%, n = 16,492) and validation cohort (50%, n = 16,430). Individuals in the derivation cohort with a 30‐day readmission had markedly different socioeconomic and clinical characteristics compared to those not readmitted (Table 1).

Derivation and Validation of the Full‐Stay EHR Model for 30‐Day Readmission

Our final model included 24 independent variables, including demographic characteristics, utilization history, clinical factors from the first day of admission, and clinical factors from the remainder of the hospital stay (Table 2). The strongest independent predictor of readmission was hospital‐acquired Clostridium difficile infection (adjusted odds ratio [AOR]: 2.03, 95% confidence interval [CI] 1.18‐3.48); other hospital‐acquired complications including pressure ulcers and venous thromboembolism were also significant predictors. Though having Medicaid was associated with increased odds of readmission (AOR: 1.55, 95% CI: 1.31‐1.83), other zip codelevel measures of socioeconomic disadvantage were not predictive and were not included in the final model. Being discharged to hospice was associated with markedly lower odds of readmission (AOR: 0.23, 95% CI: 0.13‐0.40).

Table 2.Final Full‐Stay EHR Model Predicting 30‐Day Readmissions (Derivation Cohort, N = 16,492)

	Odds Ratio (95% CI)
	Univariate	Multivariate*
NOTE: Abbreviations: CI, confidence interval; ED, emergency department. *Values shown reflect adjusted odds ratios and 95% CI for each factor after adjustment for all other factors listed in the table.
Demographic characteristics
Age, per 10 years	1.08 (1.051.11)	1.07 (1.041.10)
Medicaid	1.97 (1.702.29)	1.55 (1.311.83)
Widow	1.44 (1.281.63)	1.27 (1.111.45)
Utilization history
Prior ED visit, per visit	1.08 (1.061.10)	1.04 (1.021.06)
Prior hospitalization, per hospitalization	1.30 (1.271.34)	1.16 (1.121.20)
Hospital and clinical factors from first day of hospitalization
Nonelective admission	1.75 (1.512.03)	1.42 (1.221.65)
Charlson Comorbidity Index, per point	1.19 (1.171.21)	1.06 (1.041.09)
Laboratory abnormalities within 24 hours of admission
Albumin 2 g/dL	2.57 (1.823.62)	1.52 (1.052.21)
Albumin 23 g/dL	1.68 (1.501.88)	1.20 (1.061.36)
Aspartate aminotransferase >40 U/L	1.37 (1.221.55)	1.21 (1.061.38)
Creatine phosphokinase 60 g/L	1.48 (1.301.69)	1.28 (1.111.46)
Mean corpuscular volume >100 fL/red cell	1.68 (1.382.04)	1.32 (1.071.62)
Platelets 90 103/L	2.20 (1.772.72)	1.56 (1.231.97)
Platelets >350 103/L	1.34 (1.171.54)	1.24 (1.081.44)
Prothrombin time >35 seconds	2.58 (1.743.82)	1.92 (1.272.90)
Hospital and clinical factors from remainder of hospital stay
Length of stay, per day	1.08 (1.071.09)	1.06 (1.041.07)
Hospital complications
Clostridium difficile infection	3.61 (2.195.95)	2.03 (1.183.48)
Pressure ulcer	2.43 (1.733.41)	1.64 (1.152.34)
Venous thromboembolism	2.01 (1.362.96)	1.55 (1.032.32)
Laboratory abnormalities at discharge
Blood urea nitrogen >20 mg/dL	1.86 (1.702.04)	1.37 (1.241.52)
Sodium 135 mEq/L	1.70 (1.521.91)	1.34 (1.181.51)
Hematocrit 27	1.61 (1.401.85)	1.22 (1.051.41)
Vital sign instability at discharge, per instability	1.29 (1.201.40)	1.25 (1.151.36)
Discharged to hospice	0.51 (0.300.89)	0.23 (0.130.40)

In our validation cohort, the full‐stay EHR model had fair discrimination, with a C statistic of 0.69 (95% CI: 0.68‐0.70) (Table 3). The full‐stay EHR model was well calibrated across all quintiles of risk, with slight overestimation of predicted risk in the lowest and highest quintiles (Figure 1a) (see Supporting Table 5 in the online version of this article). It also effectively stratified individuals across a broad range of predicted readmission risk from 4.1% in the lowest decile to 36.5% in the highest decile (Table 3).

Table 3.Comparison of the Discrimination and Reclassification of Different Readmission Models*

Model Name	C‐Statistic (95% CI)	IDI, % (95% CI)	NRI (95% CI)	Average Predicted Risk, %
				Lowest Decile	Highest Decile
NOTE: Abbreviations; CI, confidence interval; EHR, electronic health record; IDI, Integrated Discrimination Improvement; NRI, Net Reclassification Index. *All measures were assessed using the validation cohort (N = 16,430), except for estimating the C‐statistic for the derivation cohort. P value 0.001 for all pairwise comparisons of C‐statistic between full‐stay model and first‐day, LACE, and HOSPITAL models, respectively. The LACE model includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year. The HOSPITAL model includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay.
Full‐stay EHR model
Derivation cohort	0.72 (0.70 to 0.73)			4.1	36.5
Validation cohort	0.69 (0.68 to 0.70)	[Reference]	[Reference]	4.1	36.5
First‐day EHR model	0.67 (0.66 to 0.68)	1.2 (1.4 to 1.0)	0.020 (0.038 to 0.002)	5.8	31.9
LACE model	0.65 (0.64 to 0.66)	2.6 (2.9 to 2.3)	0.046 (0.067 to 0.024)	6.1	27.5
HOSPITAL model	0.64 (0.62 to 0.65)	3.2 (3.5 to 2.9)	0.058 (0.080 to 0.035)	6.7	26.6

Figure 1

DISCUSSION

In this study, we used clinically detailed EHR data from the entire hospitalization on 32,922 individuals treated in 6 diverse hospitals to develop an all‐payer, multicondition readmission risk‐prediction model. To our knowledge, this is the first 30‐day hospital readmission risk‐prediction model to use a comprehensive set of factors from EHR data from the entire hospital stay. Prior EHR‐based models have focused exclusively on data available on or prior to the first day of admission, which account for clinical severity on admission but do not account for factors uncovered during the inpatient stay that influence the chance of a postdischarge adverse outcome.[15, 30] We specifically assessed the prognostic impact of a comprehensive set of factors from the entire index hospitalization, including hospital‐acquired complications, clinical trajectory, and stability on discharge in predicting hospital readmissions. Our full‐stay EHR model had statistically better discrimination, calibration, and diagnostic accuracy than our existing all‐cause first‐day EHR model[15] and 2 previously published readmissions models that included more limited information from hospitalization (such as length of stay).[9, 10] However, although the more complicated full‐stay EHR model was statistically better than previously published models, we were surprised that the predictive performance was only modestly improved despite the inclusion of many additional clinically relevant prognostic factors.

Taken together, our study has several important implications. First, the added complexity and resource intensity of implementing a full‐stay EHR model yields only modestly improved readmission risk prediction. Thus, hospitals and healthcare systems interested in targeting their highest‐risk individuals for interventions to reduce 30‐day readmission should consider doing so within the first day of hospital admission. Our group's previously derived and validated first‐day EHR model, which used data only from the first day of admission, qualitatively performed nearly as well as the full‐stay EHR model.[15] Additionally, a recent study using only preadmission EHR data to predict 30‐day readmissions also achieved similar discrimination and diagnostic accuracy as our full‐stay model.[30]

Second, the field of readmissions risk‐prediction modeling may be reaching the maximum achievable model performance using data that are currently available in the EHR. Our limited ability to accurately predict all‐cause 30‐day readmission risk may reflect the influence of currently unmeasured patient, system, and community factors on readmissions.[31, 32, 33] Due to the constraints of data collected in the EHR, we were unable to include several patient‐level clinical characteristics associated with hospital readmission, including self‐perceived health status, functional impairment, and cognition.[33, 34, 35, 36] However, given their modest effect sizes (ORs ranging from 1.062.10), adequately measuring and including these risk factors in our model may not meaningfully improve model performance and diagnostic accuracy. Further, many social and behavioral patient‐level factors are also not consistently available in EHR data. Though we explored the role of several neighborhood‐level socioeconomic measuresincluding prevalence of poverty, median income, education, and unemploymentwe found that none were significantly associated with 30‐day readmissions. These particular measures may have been inadequate to characterize individual‐level social and behavioral factors, as several previous studies have demonstrated that patient‐level factors such as social support, substance abuse, and medication and visit adherence can influence readmission risk in heart failure and pneumonia.[11, 16, 22, 25] This underscores the need for more standardized routine collection of data across functional, social, and behavioral domains in clinical settings, as recently championed by the Institute of Medicine.[11, 37] Integrating data from outside the EHR on postdischarge health behaviors, self‐management, follow‐up care, recovery, and home environment may be another important but untapped strategy for further improving prediction of readmissions.[25, 38]

Third, a multicondition readmission risk‐prediction model may be a less effective strategy than more customized disease‐specific models for selected conditions associated with high 30‐day readmission rates. Our group's previously derived and internally validated models for heart failure and human immunodeficiency virus had superior discrimination compared to our full‐stay EHR model (C statistic of 0.72 for each).[11, 13] However, given differences in the included population and time periods studied, a head‐to‐head comparison of these different strategies is needed to assess differences in model performance and utility.

Our study had several strengths. To our knowledge, this is the first study to rigorously measure the additive influence of in‐hospital complications, clinical trajectory, and stability on discharge on the risk of 30‐day hospital readmission. Additionally, our study included a large, diverse study population that included all payers, all ages of adults, a mix of community, academic, and safety net hospitals, and individuals from a broad array of racial/ethnic and socioeconomic backgrounds.

Our results should be interpreted in light of several limitations. First, though we sought to represent a diverse group of hospitals, all study sites were located within north Texas and generalizability to other regions is uncertain. Second, our ascertainment of prior hospitalizations and readmissions was more inclusive than what could be typically accomplished in real time using only EHR data from a single clinical site. We performed a sensitivity analysis using only prior utilization data available within the EHR from the index hospital with no meaningful difference in our findings (data not shown). Additionally, a recent study found that 30‐day readmissions occur at the index hospital for over 75% of events, suggesting that 30‐day readmissions are fairly comprehensively captured even with only single‐site data.[39] Third, we were not able to include data on outpatient visits before or after the index hospitalization, which may influence the risk of readmission.[1, 40]

In conclusion, incorporating clinically granular EHR data from the entire course of hospitalization modestly improves prediction of 30‐day readmissions compared to models that only include information from the first 24 hours of hospital admission or models that use far fewer variables. However, given the limited improvement in prediction, our findings suggest that from the practical perspective of implementing real‐time models to identify those at highest risk for readmission, it may not be worth the added complexity of waiting until the end of a hospitalization to leverage additional data on hospital complications, and the trajectory of laboratory and vital sign values currently available in the EHR. Further improvement in prediction of readmissions will likely require accounting for psychosocial, functional, behavioral, and postdischarge factors not currently present in the inpatient EHR.

Disclosures: This study was presented at the Society of Hospital Medicine 2015 Annual Meeting in National Harbor, Maryland, and the Society of General Internal Medicine 2015 Annual Meeting in Toronto, Canada. This work was supported by the Agency for Healthcare Research and Qualityfunded UT Southwestern Center for Patient‐Centered Outcomes Research (1R24HS022418‐01) and the Commonwealth Foundation (#20100323). Drs. Nguyen and Makam received funding from the UT Southwestern KL2 Scholars Program (NIH/NCATS KL2 TR001103). Dr. Halm was also supported in part by NIH/NCATS U54 RFA‐TR‐12‐006. The study sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no conflicts of interest to disclose.

Unplanned hospital readmissions are frequent, costly, and potentially avoidable.[1, 2] Due to major federal financial readmissions penalties targeting excessive 30‐day readmissions, there is increasing attention to implementing hospital‐initiated interventions to reduce readmissions.[3, 4] However, universal enrollment of all hospitalized patients into such programs may be too resource intensive for many hospitals.[5] To optimize efficiency and effectiveness, interventions should be targeted to individuals most likely to benefit.[6, 7] However, existing readmission risk‐prediction models have achieved only modest discrimination, have largely used administrative claims data not available until months after discharge, or are limited to only a subset of patients with Medicare or a specific clinical condition.[8, 9, 10, 11, 12, 13, 14] These limitations have precluded accurate identification of high‐risk individuals in an all‐payer general medical inpatient population to provide actionable information for intervention prior to discharge.

Approaches using electronic health record (EHR) data could allow early identification of high‐risk patients during the index hospitalization to enable initiation of interventions prior to discharge. To date, such strategies have relied largely on EHR data from the day of admission.[15, 16] However, given that variation in 30‐day readmission rates are thought to reflect the quality of in‐hospital care, incorporating EHR data from the entire hospital stay to reflect hospital care processes and clinical trajectory may more accurately identify at‐risk patients.[17, 18, 19, 20] Improved accuracy in risk prediction would help better target intervention efforts in the immediate postdischarge period, an interval characterized by heightened vulnerability for adverse events.[21]

To help hospitals target transitional care interventions more effectively to high‐risk individuals prior to discharge, we derived and validated a readmissions risk‐prediction model incorporating EHR data from the entire course of the index hospitalization, which we termed the full‐stay EHR model. We also compared the full‐stay EHR model performance to our group's previously derived prediction model based on EHR data on the day of admission, termed the first‐day EHR model, as well as to 2 other validated readmission models similarly intended to yield near real‐time risk predictions prior to or shortly after hospital discharge.[9, 10, 15]

METHODS

RESULTS

Overall, 32,922 index hospitalizations were included in our study cohort; 12.7% resulted in a 30‐day readmission (see Supporting Figure 1 in the online version of this article). Individuals had a mean age of 62 years and had diverse race/ethnicity and primary insurance status; half were female (Table 1). The study sample was randomly split into a derivation cohort (50%, n = 16,492) and validation cohort (50%, n = 16,430). Individuals in the derivation cohort with a 30‐day readmission had markedly different socioeconomic and clinical characteristics compared to those not readmitted (Table 1).

Derivation and Validation of the Full‐Stay EHR Model for 30‐Day Readmission

Our final model included 24 independent variables, including demographic characteristics, utilization history, clinical factors from the first day of admission, and clinical factors from the remainder of the hospital stay (Table 2). The strongest independent predictor of readmission was hospital‐acquired Clostridium difficile infection (adjusted odds ratio [AOR]: 2.03, 95% confidence interval [CI] 1.18‐3.48); other hospital‐acquired complications including pressure ulcers and venous thromboembolism were also significant predictors. Though having Medicaid was associated with increased odds of readmission (AOR: 1.55, 95% CI: 1.31‐1.83), other zip codelevel measures of socioeconomic disadvantage were not predictive and were not included in the final model. Being discharged to hospice was associated with markedly lower odds of readmission (AOR: 0.23, 95% CI: 0.13‐0.40).

Table 2.Final Full‐Stay EHR Model Predicting 30‐Day Readmissions (Derivation Cohort, N = 16,492)

	Odds Ratio (95% CI)
	Univariate	Multivariate*
NOTE: Abbreviations: CI, confidence interval; ED, emergency department. *Values shown reflect adjusted odds ratios and 95% CI for each factor after adjustment for all other factors listed in the table.
Demographic characteristics
Age, per 10 years	1.08 (1.051.11)	1.07 (1.041.10)
Medicaid	1.97 (1.702.29)	1.55 (1.311.83)
Widow	1.44 (1.281.63)	1.27 (1.111.45)
Utilization history
Prior ED visit, per visit	1.08 (1.061.10)	1.04 (1.021.06)
Prior hospitalization, per hospitalization	1.30 (1.271.34)	1.16 (1.121.20)
Hospital and clinical factors from first day of hospitalization
Nonelective admission	1.75 (1.512.03)	1.42 (1.221.65)
Charlson Comorbidity Index, per point	1.19 (1.171.21)	1.06 (1.041.09)
Laboratory abnormalities within 24 hours of admission
Albumin 2 g/dL	2.57 (1.823.62)	1.52 (1.052.21)
Albumin 23 g/dL	1.68 (1.501.88)	1.20 (1.061.36)
Aspartate aminotransferase >40 U/L	1.37 (1.221.55)	1.21 (1.061.38)
Creatine phosphokinase 60 g/L	1.48 (1.301.69)	1.28 (1.111.46)
Mean corpuscular volume >100 fL/red cell	1.68 (1.382.04)	1.32 (1.071.62)
Platelets 90 103/L	2.20 (1.772.72)	1.56 (1.231.97)
Platelets >350 103/L	1.34 (1.171.54)	1.24 (1.081.44)
Prothrombin time >35 seconds	2.58 (1.743.82)	1.92 (1.272.90)
Hospital and clinical factors from remainder of hospital stay
Length of stay, per day	1.08 (1.071.09)	1.06 (1.041.07)
Hospital complications
Clostridium difficile infection	3.61 (2.195.95)	2.03 (1.183.48)
Pressure ulcer	2.43 (1.733.41)	1.64 (1.152.34)
Venous thromboembolism	2.01 (1.362.96)	1.55 (1.032.32)
Laboratory abnormalities at discharge
Blood urea nitrogen >20 mg/dL	1.86 (1.702.04)	1.37 (1.241.52)
Sodium 135 mEq/L	1.70 (1.521.91)	1.34 (1.181.51)
Hematocrit 27	1.61 (1.401.85)	1.22 (1.051.41)
Vital sign instability at discharge, per instability	1.29 (1.201.40)	1.25 (1.151.36)
Discharged to hospice	0.51 (0.300.89)	0.23 (0.130.40)

In our validation cohort, the full‐stay EHR model had fair discrimination, with a C statistic of 0.69 (95% CI: 0.68‐0.70) (Table 3). The full‐stay EHR model was well calibrated across all quintiles of risk, with slight overestimation of predicted risk in the lowest and highest quintiles (Figure 1a) (see Supporting Table 5 in the online version of this article). It also effectively stratified individuals across a broad range of predicted readmission risk from 4.1% in the lowest decile to 36.5% in the highest decile (Table 3).

Table 3.Comparison of the Discrimination and Reclassification of Different Readmission Models*

Model Name	C‐Statistic (95% CI)	IDI, % (95% CI)	NRI (95% CI)	Average Predicted Risk, %
				Lowest Decile	Highest Decile
NOTE: Abbreviations; CI, confidence interval; EHR, electronic health record; IDI, Integrated Discrimination Improvement; NRI, Net Reclassification Index. *All measures were assessed using the validation cohort (N = 16,430), except for estimating the C‐statistic for the derivation cohort. P value 0.001 for all pairwise comparisons of C‐statistic between full‐stay model and first‐day, LACE, and HOSPITAL models, respectively. The LACE model includes Length of stay, Acute (nonelective) admission status, Charlson Comorbidity Index, and Emergency department visits in the past year. The HOSPITAL model includes Hemoglobin at discharge, discharge from Oncology service, Sodium level at discharge, Procedure during index hospitalization, Index hospitalization Type (nonelective), number of Admissions in the past year, and Length of stay.
Full‐stay EHR model
Derivation cohort	0.72 (0.70 to 0.73)			4.1	36.5
Validation cohort	0.69 (0.68 to 0.70)	[Reference]	[Reference]	4.1	36.5
First‐day EHR model	0.67 (0.66 to 0.68)	1.2 (1.4 to 1.0)	0.020 (0.038 to 0.002)	5.8	31.9
LACE model	0.65 (0.64 to 0.66)	2.6 (2.9 to 2.3)	0.046 (0.067 to 0.024)	6.1	27.5
HOSPITAL model	0.64 (0.62 to 0.65)	3.2 (3.5 to 2.9)	0.058 (0.080 to 0.035)	6.7	26.6

Figure 1

DISCUSSION

In this study, we used clinically detailed EHR data from the entire hospitalization on 32,922 individuals treated in 6 diverse hospitals to develop an all‐payer, multicondition readmission risk‐prediction model. To our knowledge, this is the first 30‐day hospital readmission risk‐prediction model to use a comprehensive set of factors from EHR data from the entire hospital stay. Prior EHR‐based models have focused exclusively on data available on or prior to the first day of admission, which account for clinical severity on admission but do not account for factors uncovered during the inpatient stay that influence the chance of a postdischarge adverse outcome.[15, 30] We specifically assessed the prognostic impact of a comprehensive set of factors from the entire index hospitalization, including hospital‐acquired complications, clinical trajectory, and stability on discharge in predicting hospital readmissions. Our full‐stay EHR model had statistically better discrimination, calibration, and diagnostic accuracy than our existing all‐cause first‐day EHR model[15] and 2 previously published readmissions models that included more limited information from hospitalization (such as length of stay).[9, 10] However, although the more complicated full‐stay EHR model was statistically better than previously published models, we were surprised that the predictive performance was only modestly improved despite the inclusion of many additional clinically relevant prognostic factors.

Taken together, our study has several important implications. First, the added complexity and resource intensity of implementing a full‐stay EHR model yields only modestly improved readmission risk prediction. Thus, hospitals and healthcare systems interested in targeting their highest‐risk individuals for interventions to reduce 30‐day readmission should consider doing so within the first day of hospital admission. Our group's previously derived and validated first‐day EHR model, which used data only from the first day of admission, qualitatively performed nearly as well as the full‐stay EHR model.[15] Additionally, a recent study using only preadmission EHR data to predict 30‐day readmissions also achieved similar discrimination and diagnostic accuracy as our full‐stay model.[30]

Second, the field of readmissions risk‐prediction modeling may be reaching the maximum achievable model performance using data that are currently available in the EHR. Our limited ability to accurately predict all‐cause 30‐day readmission risk may reflect the influence of currently unmeasured patient, system, and community factors on readmissions.[31, 32, 33] Due to the constraints of data collected in the EHR, we were unable to include several patient‐level clinical characteristics associated with hospital readmission, including self‐perceived health status, functional impairment, and cognition.[33, 34, 35, 36] However, given their modest effect sizes (ORs ranging from 1.062.10), adequately measuring and including these risk factors in our model may not meaningfully improve model performance and diagnostic accuracy. Further, many social and behavioral patient‐level factors are also not consistently available in EHR data. Though we explored the role of several neighborhood‐level socioeconomic measuresincluding prevalence of poverty, median income, education, and unemploymentwe found that none were significantly associated with 30‐day readmissions. These particular measures may have been inadequate to characterize individual‐level social and behavioral factors, as several previous studies have demonstrated that patient‐level factors such as social support, substance abuse, and medication and visit adherence can influence readmission risk in heart failure and pneumonia.[11, 16, 22, 25] This underscores the need for more standardized routine collection of data across functional, social, and behavioral domains in clinical settings, as recently championed by the Institute of Medicine.[11, 37] Integrating data from outside the EHR on postdischarge health behaviors, self‐management, follow‐up care, recovery, and home environment may be another important but untapped strategy for further improving prediction of readmissions.[25, 38]

Third, a multicondition readmission risk‐prediction model may be a less effective strategy than more customized disease‐specific models for selected conditions associated with high 30‐day readmission rates. Our group's previously derived and internally validated models for heart failure and human immunodeficiency virus had superior discrimination compared to our full‐stay EHR model (C statistic of 0.72 for each).[11, 13] However, given differences in the included population and time periods studied, a head‐to‐head comparison of these different strategies is needed to assess differences in model performance and utility.

Our study had several strengths. To our knowledge, this is the first study to rigorously measure the additive influence of in‐hospital complications, clinical trajectory, and stability on discharge on the risk of 30‐day hospital readmission. Additionally, our study included a large, diverse study population that included all payers, all ages of adults, a mix of community, academic, and safety net hospitals, and individuals from a broad array of racial/ethnic and socioeconomic backgrounds.

Our results should be interpreted in light of several limitations. First, though we sought to represent a diverse group of hospitals, all study sites were located within north Texas and generalizability to other regions is uncertain. Second, our ascertainment of prior hospitalizations and readmissions was more inclusive than what could be typically accomplished in real time using only EHR data from a single clinical site. We performed a sensitivity analysis using only prior utilization data available within the EHR from the index hospital with no meaningful difference in our findings (data not shown). Additionally, a recent study found that 30‐day readmissions occur at the index hospital for over 75% of events, suggesting that 30‐day readmissions are fairly comprehensively captured even with only single‐site data.[39] Third, we were not able to include data on outpatient visits before or after the index hospitalization, which may influence the risk of readmission.[1, 40]

In conclusion, incorporating clinically granular EHR data from the entire course of hospitalization modestly improves prediction of 30‐day readmissions compared to models that only include information from the first 24 hours of hospital admission or models that use far fewer variables. However, given the limited improvement in prediction, our findings suggest that from the practical perspective of implementing real‐time models to identify those at highest risk for readmission, it may not be worth the added complexity of waiting until the end of a hospitalization to leverage additional data on hospital complications, and the trajectory of laboratory and vital sign values currently available in the EHR. Further improvement in prediction of readmissions will likely require accounting for psychosocial, functional, behavioral, and postdischarge factors not currently present in the inpatient EHR.

Disclosures: This study was presented at the Society of Hospital Medicine 2015 Annual Meeting in National Harbor, Maryland, and the Society of General Internal Medicine 2015 Annual Meeting in Toronto, Canada. This work was supported by the Agency for Healthcare Research and Qualityfunded UT Southwestern Center for Patient‐Centered Outcomes Research (1R24HS022418‐01) and the Commonwealth Foundation (#20100323). Drs. Nguyen and Makam received funding from the UT Southwestern KL2 Scholars Program (NIH/NCATS KL2 TR001103). Dr. Halm was also supported in part by NIH/NCATS U54 RFA‐TR‐12‐006. The study sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript. The authors have no conflicts of interest to disclose.

Hospital care accounts for the single largest category of national healthcare expenditures, totaling $936.9 billion in 2013.[1] With ongoing scrutiny of US healthcare spending, hospitals are under increasing pressure to justify high costs and robust profits.[2] However, the dominant fee‐for‐service reimbursement model creates incentives for hospitals to prioritize high volume over high‐quality care to maximize profits.[3] Because hospitals may be reluctant to implement improvements if better quality is not accompanied by better payment or improved financial margins, an approach to stimulate quality improvement among hospitals has been to leverage consumer pressure through required public reporting of selected outcome metrics.[4, 5] Public reporting of outcomes is thought to influence hospital reputation; in turn, reputation affects patient perceptions and influences demand for hospital services, potentially enabling reputable hospitals to command higher prices for services to enhance hospital revenue.[6, 7]

Though improving outcomes is thought to reduce overall healthcare costs, it is unclear whether improving outcomes results in a hospital's financial return on investment.[4, 5, 8] Quality improvement can require substantial upfront investment, requiring that hospitals already have robust financial health to engage in such initiatives.[9, 10] Consequently, instead of stimulating broad efforts in quality improvement, public reporting may exacerbate existing disparities in hospital quality and finances, by rewarding already financially healthy hospitals, and by inadvertently penalizing hospitals without the means to invest in quality improvement.[11, 12, 13, 14, 15] Alternately, because fee‐for‐service remains the dominant reimbursement model for hospitals, loss of revenue through reducing readmissions may outweigh any financial gains from improved public reputation and result in worse overall financial performance, though robust evidence for this concern is lacking.[16, 17]

A small number of existing studies suggest a limited correlation between improved hospital financial performance and improved quality, patient safety, and lower readmission rates.[18, 19, 20] However, these studies had several limitations. They were conducted prior to public reporting of selected outcome metrics by the Centers for Medicare and Medicaid Services (CMS)[18, 19, 20]; used data from the Medicare Cost Report, which is not uniformly audited and thus prone to measurement error[19, 20]; used only relative measures of hospital financial performance (eg, operating margin), which do not capture the absolute amount of revenue potentially available for investment in quality improvement[18, 19]; or compared only hospitals at the extremes of financial performance, potentially exaggerating the magnitude of the relationship between hospital financial performance and quality outcomes.[19]

To address this gap in the literature, we sought to assess whether hospitals with robust financial performance have lower 30‐day risk‐standardized mortality and hospital readmission rates for acute myocardial infarction (AMI), congestive heart failure (CHF), and pneumonia (PNA). Given the concern that hospitals with the lowest mortality and readmission rates may experience a decrease in financial performance due to the lower volume of hospitalizations, we also assessed whether hospitals with the lowest readmission and mortality rates had a differential change in financial performance over time compared to hospitals with the highest rates.

METHODS

RESULTS

Among the 279 included hospitals (see Supporting Figure 1 in the online version of this article), 278 also had financial data available for 2012. In 2008, the median net revenue from operations was $1.6 million (interquartile range [IQR], $2.4 to $10.3 million), the median operating margin was 1.5% (IQR, 4.6% to 6%), and the median total margin was 2.5% (IQR, 2.2% to 7.5% (Table 1). The number of hospitals reporting each outcome, and median outcome rates, are shown in Table 2.

Relationship Between Financial Performance and Publicly Reported Outcomes

Acute Myocardial Infarction

We did not observe a consistent relationship between hospital financial performance and AMI mortality and readmission rates. In our overall adjusted analyses, net revenue from operations was not associated with mortality, but was significantly associated with a decrease in AMI readmissions among hospitals with net revenue from operations between approximately $5 million to $145 million (nonlinear relationship, F statistic = 4.8, P 0.001 (Table 2, Figure 1A). However, after excluding 23 extreme outlying hospitals by net revenue from operations (10 underperformers with net revenue $49.4 million and 13 overperformers with net revenue >$52.1 million), this relationship was no longer observed. Using operating margin instead of net revenue from operations as the measure of hospital financial performance, we observed a 0.2% increase in AMI mortality (95% confidence interval [CI]: 0.06%‐0.35%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article) for each 10% increase in operating margin, which persisted with the exclusion of 5 outlying hospitals by operating margin (all 5 were underperformers, with operating margins 38.6%). However, using total margin as the measure of financial performance, there was no significant relationship with either mortality or readmissions (see Supporting Table 2 and Supporting Figure 3 in the online version of this article).

Figure 1

Congestive Heart Failure

In our primary analyses, we did not observe a significant relationship between net revenue from operations and CHF mortality and readmission rates. However, after excluding 23 extreme outliers, increasing net revenue from operations was associated with a modest increase in CHF mortality among hospitals, with net revenue between approximately $35 million and $20 million (nonlinear relationship, F statistic = 3.6, P = 0.008 (Table 2, Figure 1B). Using alternate measures of financial performance, we observed a consistent relationship between increasing hospital financial performance and higher 30‐day CHF mortality rate. Using operating margin, we observed a slight increase in the mortality rate for CHF (0.26% increase in CHF RSMR for every 10% increase in operating margin) (95% CI: 0.07%‐0.45%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article), which persisted after the exclusion of 5 extreme outliers. Using total margin, we observed a significant but modest association between improved hospital financial performance and increased mortality rate for CHF (nonlinear relationship, F statistic = 2.9, P = 0.03) (see Supporting Table 2 and Supporting Figure 3 in the online version of this article), which persisted after the exclusion of 3 extreme outliers (0.32% increase in CHF RSMR for every 10% increase in total margin) (95% CI: 0.03%‐0.62%).

Pneumonia

Hospital financial performance (using net revenue, operating margin, or total margin) was not associated with 30‐day PNA mortality or readmission rates.

DISCUSSION

Using audited financial data from California hospitals in 2008 and 2012, and CMS data on publicly reported outcomes from 2008 to 2011, we found no consistent relationship between hospital financial performance and publicly reported outcomes for AMI and PNA. However, better hospital financial performance was associated with a modest increase in 30‐day risk‐standardized CHF mortality rates, which was consistent across all 3 measures of hospital financial performance. Reassuringly, there was no difference in the change in net revenue from operations between 2008 and 2012 between hospitals in the highest and lowest tertiles of readmission and mortality rates for AMI, CHF, and PNA. In other words, hospitals with the lowest rates of 30‐day readmissions and mortality for AMI, CHF, and PNA did not experience a loss in net revenue from operations over time, compared to hospitals with the highest readmission and mortality rates.

Our study differs in several important ways from Ly et al., the only other study to our knowledge that investigated the relationship between hospital financial performance and outcomes for patients with AMI, CHF, and PNA.[19] First, outcomes in the Ly et al. study were ascertained in 2007, which preceded public reporting of outcomes. Second, the primary comparison was between hospitals in the bottom versus top decile of operating margin. Although Ly and colleagues also found no association between hospital financial performance and mortality rates for these 3 conditions, they found a significant absolute decrease of approximately 3% in readmission rates among hospitals in the top decile of operating margin versus those in bottom decile. However, readmission rates were comparable among the remaining 80% of hospitals, suggesting that these findings primarily reflected the influence of a few outlier hospitals. Third, the use of nonuniformly audited hospital financial data may have resulted in misclassification of financial performance. Our findings also differ from 2 previous studies that identified a modest association between improved hospital financial performance and decreased adverse patient safety events.[18, 20] However, publicly reported outcomes may not be fully representative of hospital quality and patient safety.[28, 29]

The limited association between hospital financial performance and publicly reported outcomes for AMI and PNA is noteworthy for several reasons. First, publicly reporting outcomes alone without concomitant changes to reimbursement may be inadequate to create strong financial incentives for hospital investment in quality improvement initiatives. Hospitals participating in both public reporting of outcomes and pay‐for‐performance have been shown to achieve greater improvements in outcomes than hospitals engaged only in public reporting.[30] Our time interval for ascertainment of outcomes preceded CMS implementation of the Hospital Readmissions Reduction Program (HRRP) in October 2012, which withholds up to 3% of Medicare hospital reimbursements for higher than expected mortality and readmission rates for AMI, CHF, and PNA. Once outcomes data become available for a 3‐year post‐HRRP implementation period, the impact of this combined approach can be assessed. Second, because adherence to many evidence‐based process measures for these conditions (ie, aspirin use in AMI) is already high, there may be a ceiling effect present that obviates the need for further hospital financial investment to optimize delivery of best practices.[31, 32] Third, hospitals themselves may contribute little to variation in mortality and readmission risk. Of the total variation in mortality and readmission rates among Texas Medicare beneficiaries, only about 1% is attributable to hospitals, whereas 42% to 56% of the variation is explained by differences in patient characteristics.[33, 34] Fourth, there is either low‐quality or insufficient evidence that transitional care interventions specifically targeted to patients with AMI or PNA result in better outcomes.[35] Thus, greater financial investment in hospital‐initiated and postdischarge transitional care interventions for these specific conditions may result in less than the desired effect. Lastly, many hospitalizations for these conditions are emergency hospitalizations that occur after patients present to the emergency department with unexpected and potentially life‐threatening symptoms. Thus, patients may not be able to incorporate the reputation or performance metrics of a hospital in their decisions for where they are hospitalized for AMI, CHF, or PNA despite the public reporting of outcomes.

Given the strong evidence that transitional care interventions reduce readmissions and mortality among patients hospitalized with CHF, we were surprised to find that improved hospital financial performance was associated with an increased risk‐adjusted CHF mortality rate.[36] This association held true for all 3 different measures of hospital financial performance, suggesting that this unexpected finding is unlikely to be the result of statistical chance, though potential reasons for this association remain unclear. One possibility is that the CMS model for CHF mortality may not adequately risk adjust for severity of illness.[37, 38] Thus, robust financial performance may be a marker for hospitals with more advanced heart failure services that care for more patients with severe illness.

Our findings should be interpreted in the context of certain limitations. Our study only included an analysis of outcomes for AMI, CHF, and PNA among older fee‐for‐service Medicare beneficiaries aggregated at the hospital level in California between 2008 and 2012, so generalizability to other populations, conditions, states, and time periods is uncertain. The observational design precludes a robust causal inference between financial performance and outcomes. For readmissions, rates were publicly reported for only the last 2 years of the 3‐year reporting period; thus, our findings may underestimate the association between hospital financial performance and publicly reported readmission rates.

CONCLUSION

There is no consistent relationship between hospital financial performance and subsequent publicly reported outcomes for AMI and PNA. However, for unclear reasons, hospitals with better financial performance had modestly higher CHF mortality rates. Given this limited association, public reporting of outcomes may have had less than the intended impact in motivating hospitals to invest in quality improvement. Additional financial incentives in addition to public reporting, such as readmissions penalties, may help motivate hospitals with robust financial performance to further improve outcomes. This would be a key area for future investigation once outcomes data are available for the 3‐year period following CMS implementation of readmissions penalties in 2012. Reassuringly, there was no association between low 30‐day mortality and readmissions rates and subsequent poor financial performance, suggesting that improved outcomes do not necessarily lead to loss of revenue.

Hospital care accounts for the single largest category of national healthcare expenditures, totaling $936.9 billion in 2013.[1] With ongoing scrutiny of US healthcare spending, hospitals are under increasing pressure to justify high costs and robust profits.[2] However, the dominant fee‐for‐service reimbursement model creates incentives for hospitals to prioritize high volume over high‐quality care to maximize profits.[3] Because hospitals may be reluctant to implement improvements if better quality is not accompanied by better payment or improved financial margins, an approach to stimulate quality improvement among hospitals has been to leverage consumer pressure through required public reporting of selected outcome metrics.[4, 5] Public reporting of outcomes is thought to influence hospital reputation; in turn, reputation affects patient perceptions and influences demand for hospital services, potentially enabling reputable hospitals to command higher prices for services to enhance hospital revenue.[6, 7]

Though improving outcomes is thought to reduce overall healthcare costs, it is unclear whether improving outcomes results in a hospital's financial return on investment.[4, 5, 8] Quality improvement can require substantial upfront investment, requiring that hospitals already have robust financial health to engage in such initiatives.[9, 10] Consequently, instead of stimulating broad efforts in quality improvement, public reporting may exacerbate existing disparities in hospital quality and finances, by rewarding already financially healthy hospitals, and by inadvertently penalizing hospitals without the means to invest in quality improvement.[11, 12, 13, 14, 15] Alternately, because fee‐for‐service remains the dominant reimbursement model for hospitals, loss of revenue through reducing readmissions may outweigh any financial gains from improved public reputation and result in worse overall financial performance, though robust evidence for this concern is lacking.[16, 17]

A small number of existing studies suggest a limited correlation between improved hospital financial performance and improved quality, patient safety, and lower readmission rates.[18, 19, 20] However, these studies had several limitations. They were conducted prior to public reporting of selected outcome metrics by the Centers for Medicare and Medicaid Services (CMS)[18, 19, 20]; used data from the Medicare Cost Report, which is not uniformly audited and thus prone to measurement error[19, 20]; used only relative measures of hospital financial performance (eg, operating margin), which do not capture the absolute amount of revenue potentially available for investment in quality improvement[18, 19]; or compared only hospitals at the extremes of financial performance, potentially exaggerating the magnitude of the relationship between hospital financial performance and quality outcomes.[19]

To address this gap in the literature, we sought to assess whether hospitals with robust financial performance have lower 30‐day risk‐standardized mortality and hospital readmission rates for acute myocardial infarction (AMI), congestive heart failure (CHF), and pneumonia (PNA). Given the concern that hospitals with the lowest mortality and readmission rates may experience a decrease in financial performance due to the lower volume of hospitalizations, we also assessed whether hospitals with the lowest readmission and mortality rates had a differential change in financial performance over time compared to hospitals with the highest rates.

METHODS

RESULTS

Among the 279 included hospitals (see Supporting Figure 1 in the online version of this article), 278 also had financial data available for 2012. In 2008, the median net revenue from operations was $1.6 million (interquartile range [IQR], $2.4 to $10.3 million), the median operating margin was 1.5% (IQR, 4.6% to 6%), and the median total margin was 2.5% (IQR, 2.2% to 7.5% (Table 1). The number of hospitals reporting each outcome, and median outcome rates, are shown in Table 2.

Relationship Between Financial Performance and Publicly Reported Outcomes

Acute Myocardial Infarction

We did not observe a consistent relationship between hospital financial performance and AMI mortality and readmission rates. In our overall adjusted analyses, net revenue from operations was not associated with mortality, but was significantly associated with a decrease in AMI readmissions among hospitals with net revenue from operations between approximately $5 million to $145 million (nonlinear relationship, F statistic = 4.8, P 0.001 (Table 2, Figure 1A). However, after excluding 23 extreme outlying hospitals by net revenue from operations (10 underperformers with net revenue $49.4 million and 13 overperformers with net revenue >$52.1 million), this relationship was no longer observed. Using operating margin instead of net revenue from operations as the measure of hospital financial performance, we observed a 0.2% increase in AMI mortality (95% confidence interval [CI]: 0.06%‐0.35%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article) for each 10% increase in operating margin, which persisted with the exclusion of 5 outlying hospitals by operating margin (all 5 were underperformers, with operating margins 38.6%). However, using total margin as the measure of financial performance, there was no significant relationship with either mortality or readmissions (see Supporting Table 2 and Supporting Figure 3 in the online version of this article).

Figure 1

Congestive Heart Failure

In our primary analyses, we did not observe a significant relationship between net revenue from operations and CHF mortality and readmission rates. However, after excluding 23 extreme outliers, increasing net revenue from operations was associated with a modest increase in CHF mortality among hospitals, with net revenue between approximately $35 million and $20 million (nonlinear relationship, F statistic = 3.6, P = 0.008 (Table 2, Figure 1B). Using alternate measures of financial performance, we observed a consistent relationship between increasing hospital financial performance and higher 30‐day CHF mortality rate. Using operating margin, we observed a slight increase in the mortality rate for CHF (0.26% increase in CHF RSMR for every 10% increase in operating margin) (95% CI: 0.07%‐0.45%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article), which persisted after the exclusion of 5 extreme outliers. Using total margin, we observed a significant but modest association between improved hospital financial performance and increased mortality rate for CHF (nonlinear relationship, F statistic = 2.9, P = 0.03) (see Supporting Table 2 and Supporting Figure 3 in the online version of this article), which persisted after the exclusion of 3 extreme outliers (0.32% increase in CHF RSMR for every 10% increase in total margin) (95% CI: 0.03%‐0.62%).

Pneumonia

Hospital financial performance (using net revenue, operating margin, or total margin) was not associated with 30‐day PNA mortality or readmission rates.

DISCUSSION

Using audited financial data from California hospitals in 2008 and 2012, and CMS data on publicly reported outcomes from 2008 to 2011, we found no consistent relationship between hospital financial performance and publicly reported outcomes for AMI and PNA. However, better hospital financial performance was associated with a modest increase in 30‐day risk‐standardized CHF mortality rates, which was consistent across all 3 measures of hospital financial performance. Reassuringly, there was no difference in the change in net revenue from operations between 2008 and 2012 between hospitals in the highest and lowest tertiles of readmission and mortality rates for AMI, CHF, and PNA. In other words, hospitals with the lowest rates of 30‐day readmissions and mortality for AMI, CHF, and PNA did not experience a loss in net revenue from operations over time, compared to hospitals with the highest readmission and mortality rates.

Our study differs in several important ways from Ly et al., the only other study to our knowledge that investigated the relationship between hospital financial performance and outcomes for patients with AMI, CHF, and PNA.[19] First, outcomes in the Ly et al. study were ascertained in 2007, which preceded public reporting of outcomes. Second, the primary comparison was between hospitals in the bottom versus top decile of operating margin. Although Ly and colleagues also found no association between hospital financial performance and mortality rates for these 3 conditions, they found a significant absolute decrease of approximately 3% in readmission rates among hospitals in the top decile of operating margin versus those in bottom decile. However, readmission rates were comparable among the remaining 80% of hospitals, suggesting that these findings primarily reflected the influence of a few outlier hospitals. Third, the use of nonuniformly audited hospital financial data may have resulted in misclassification of financial performance. Our findings also differ from 2 previous studies that identified a modest association between improved hospital financial performance and decreased adverse patient safety events.[18, 20] However, publicly reported outcomes may not be fully representative of hospital quality and patient safety.[28, 29]

The limited association between hospital financial performance and publicly reported outcomes for AMI and PNA is noteworthy for several reasons. First, publicly reporting outcomes alone without concomitant changes to reimbursement may be inadequate to create strong financial incentives for hospital investment in quality improvement initiatives. Hospitals participating in both public reporting of outcomes and pay‐for‐performance have been shown to achieve greater improvements in outcomes than hospitals engaged only in public reporting.[30] Our time interval for ascertainment of outcomes preceded CMS implementation of the Hospital Readmissions Reduction Program (HRRP) in October 2012, which withholds up to 3% of Medicare hospital reimbursements for higher than expected mortality and readmission rates for AMI, CHF, and PNA. Once outcomes data become available for a 3‐year post‐HRRP implementation period, the impact of this combined approach can be assessed. Second, because adherence to many evidence‐based process measures for these conditions (ie, aspirin use in AMI) is already high, there may be a ceiling effect present that obviates the need for further hospital financial investment to optimize delivery of best practices.[31, 32] Third, hospitals themselves may contribute little to variation in mortality and readmission risk. Of the total variation in mortality and readmission rates among Texas Medicare beneficiaries, only about 1% is attributable to hospitals, whereas 42% to 56% of the variation is explained by differences in patient characteristics.[33, 34] Fourth, there is either low‐quality or insufficient evidence that transitional care interventions specifically targeted to patients with AMI or PNA result in better outcomes.[35] Thus, greater financial investment in hospital‐initiated and postdischarge transitional care interventions for these specific conditions may result in less than the desired effect. Lastly, many hospitalizations for these conditions are emergency hospitalizations that occur after patients present to the emergency department with unexpected and potentially life‐threatening symptoms. Thus, patients may not be able to incorporate the reputation or performance metrics of a hospital in their decisions for where they are hospitalized for AMI, CHF, or PNA despite the public reporting of outcomes.

Given the strong evidence that transitional care interventions reduce readmissions and mortality among patients hospitalized with CHF, we were surprised to find that improved hospital financial performance was associated with an increased risk‐adjusted CHF mortality rate.[36] This association held true for all 3 different measures of hospital financial performance, suggesting that this unexpected finding is unlikely to be the result of statistical chance, though potential reasons for this association remain unclear. One possibility is that the CMS model for CHF mortality may not adequately risk adjust for severity of illness.[37, 38] Thus, robust financial performance may be a marker for hospitals with more advanced heart failure services that care for more patients with severe illness.

Our findings should be interpreted in the context of certain limitations. Our study only included an analysis of outcomes for AMI, CHF, and PNA among older fee‐for‐service Medicare beneficiaries aggregated at the hospital level in California between 2008 and 2012, so generalizability to other populations, conditions, states, and time periods is uncertain. The observational design precludes a robust causal inference between financial performance and outcomes. For readmissions, rates were publicly reported for only the last 2 years of the 3‐year reporting period; thus, our findings may underestimate the association between hospital financial performance and publicly reported readmission rates.

CONCLUSION

There is no consistent relationship between hospital financial performance and subsequent publicly reported outcomes for AMI and PNA. However, for unclear reasons, hospitals with better financial performance had modestly higher CHF mortality rates. Given this limited association, public reporting of outcomes may have had less than the intended impact in motivating hospitals to invest in quality improvement. Additional financial incentives in addition to public reporting, such as readmissions penalties, may help motivate hospitals with robust financial performance to further improve outcomes. This would be a key area for future investigation once outcomes data are available for the 3‐year period following CMS implementation of readmissions penalties in 2012. Reassuringly, there was no association between low 30‐day mortality and readmissions rates and subsequent poor financial performance, suggesting that improved outcomes do not necessarily lead to loss of revenue.

Hospital care accounts for the single largest category of national healthcare expenditures, totaling $936.9 billion in 2013.[1] With ongoing scrutiny of US healthcare spending, hospitals are under increasing pressure to justify high costs and robust profits.[2] However, the dominant fee‐for‐service reimbursement model creates incentives for hospitals to prioritize high volume over high‐quality care to maximize profits.[3] Because hospitals may be reluctant to implement improvements if better quality is not accompanied by better payment or improved financial margins, an approach to stimulate quality improvement among hospitals has been to leverage consumer pressure through required public reporting of selected outcome metrics.[4, 5] Public reporting of outcomes is thought to influence hospital reputation; in turn, reputation affects patient perceptions and influences demand for hospital services, potentially enabling reputable hospitals to command higher prices for services to enhance hospital revenue.[6, 7]

Though improving outcomes is thought to reduce overall healthcare costs, it is unclear whether improving outcomes results in a hospital's financial return on investment.[4, 5, 8] Quality improvement can require substantial upfront investment, requiring that hospitals already have robust financial health to engage in such initiatives.[9, 10] Consequently, instead of stimulating broad efforts in quality improvement, public reporting may exacerbate existing disparities in hospital quality and finances, by rewarding already financially healthy hospitals, and by inadvertently penalizing hospitals without the means to invest in quality improvement.[11, 12, 13, 14, 15] Alternately, because fee‐for‐service remains the dominant reimbursement model for hospitals, loss of revenue through reducing readmissions may outweigh any financial gains from improved public reputation and result in worse overall financial performance, though robust evidence for this concern is lacking.[16, 17]

A small number of existing studies suggest a limited correlation between improved hospital financial performance and improved quality, patient safety, and lower readmission rates.[18, 19, 20] However, these studies had several limitations. They were conducted prior to public reporting of selected outcome metrics by the Centers for Medicare and Medicaid Services (CMS)[18, 19, 20]; used data from the Medicare Cost Report, which is not uniformly audited and thus prone to measurement error[19, 20]; used only relative measures of hospital financial performance (eg, operating margin), which do not capture the absolute amount of revenue potentially available for investment in quality improvement[18, 19]; or compared only hospitals at the extremes of financial performance, potentially exaggerating the magnitude of the relationship between hospital financial performance and quality outcomes.[19]

To address this gap in the literature, we sought to assess whether hospitals with robust financial performance have lower 30‐day risk‐standardized mortality and hospital readmission rates for acute myocardial infarction (AMI), congestive heart failure (CHF), and pneumonia (PNA). Given the concern that hospitals with the lowest mortality and readmission rates may experience a decrease in financial performance due to the lower volume of hospitalizations, we also assessed whether hospitals with the lowest readmission and mortality rates had a differential change in financial performance over time compared to hospitals with the highest rates.

METHODS

RESULTS

Among the 279 included hospitals (see Supporting Figure 1 in the online version of this article), 278 also had financial data available for 2012. In 2008, the median net revenue from operations was $1.6 million (interquartile range [IQR], $2.4 to $10.3 million), the median operating margin was 1.5% (IQR, 4.6% to 6%), and the median total margin was 2.5% (IQR, 2.2% to 7.5% (Table 1). The number of hospitals reporting each outcome, and median outcome rates, are shown in Table 2.

Relationship Between Financial Performance and Publicly Reported Outcomes

Acute Myocardial Infarction

We did not observe a consistent relationship between hospital financial performance and AMI mortality and readmission rates. In our overall adjusted analyses, net revenue from operations was not associated with mortality, but was significantly associated with a decrease in AMI readmissions among hospitals with net revenue from operations between approximately $5 million to $145 million (nonlinear relationship, F statistic = 4.8, P 0.001 (Table 2, Figure 1A). However, after excluding 23 extreme outlying hospitals by net revenue from operations (10 underperformers with net revenue $49.4 million and 13 overperformers with net revenue >$52.1 million), this relationship was no longer observed. Using operating margin instead of net revenue from operations as the measure of hospital financial performance, we observed a 0.2% increase in AMI mortality (95% confidence interval [CI]: 0.06%‐0.35%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article) for each 10% increase in operating margin, which persisted with the exclusion of 5 outlying hospitals by operating margin (all 5 were underperformers, with operating margins 38.6%). However, using total margin as the measure of financial performance, there was no significant relationship with either mortality or readmissions (see Supporting Table 2 and Supporting Figure 3 in the online version of this article).

Figure 1

Congestive Heart Failure

In our primary analyses, we did not observe a significant relationship between net revenue from operations and CHF mortality and readmission rates. However, after excluding 23 extreme outliers, increasing net revenue from operations was associated with a modest increase in CHF mortality among hospitals, with net revenue between approximately $35 million and $20 million (nonlinear relationship, F statistic = 3.6, P = 0.008 (Table 2, Figure 1B). Using alternate measures of financial performance, we observed a consistent relationship between increasing hospital financial performance and higher 30‐day CHF mortality rate. Using operating margin, we observed a slight increase in the mortality rate for CHF (0.26% increase in CHF RSMR for every 10% increase in operating margin) (95% CI: 0.07%‐0.45%) (see Supporting Table 1 and Supporting Figure 2 in the online version of this article), which persisted after the exclusion of 5 extreme outliers. Using total margin, we observed a significant but modest association between improved hospital financial performance and increased mortality rate for CHF (nonlinear relationship, F statistic = 2.9, P = 0.03) (see Supporting Table 2 and Supporting Figure 3 in the online version of this article), which persisted after the exclusion of 3 extreme outliers (0.32% increase in CHF RSMR for every 10% increase in total margin) (95% CI: 0.03%‐0.62%).

Pneumonia

Hospital financial performance (using net revenue, operating margin, or total margin) was not associated with 30‐day PNA mortality or readmission rates.

DISCUSSION

Using audited financial data from California hospitals in 2008 and 2012, and CMS data on publicly reported outcomes from 2008 to 2011, we found no consistent relationship between hospital financial performance and publicly reported outcomes for AMI and PNA. However, better hospital financial performance was associated with a modest increase in 30‐day risk‐standardized CHF mortality rates, which was consistent across all 3 measures of hospital financial performance. Reassuringly, there was no difference in the change in net revenue from operations between 2008 and 2012 between hospitals in the highest and lowest tertiles of readmission and mortality rates for AMI, CHF, and PNA. In other words, hospitals with the lowest rates of 30‐day readmissions and mortality for AMI, CHF, and PNA did not experience a loss in net revenue from operations over time, compared to hospitals with the highest readmission and mortality rates.

Our study differs in several important ways from Ly et al., the only other study to our knowledge that investigated the relationship between hospital financial performance and outcomes for patients with AMI, CHF, and PNA.[19] First, outcomes in the Ly et al. study were ascertained in 2007, which preceded public reporting of outcomes. Second, the primary comparison was between hospitals in the bottom versus top decile of operating margin. Although Ly and colleagues also found no association between hospital financial performance and mortality rates for these 3 conditions, they found a significant absolute decrease of approximately 3% in readmission rates among hospitals in the top decile of operating margin versus those in bottom decile. However, readmission rates were comparable among the remaining 80% of hospitals, suggesting that these findings primarily reflected the influence of a few outlier hospitals. Third, the use of nonuniformly audited hospital financial data may have resulted in misclassification of financial performance. Our findings also differ from 2 previous studies that identified a modest association between improved hospital financial performance and decreased adverse patient safety events.[18, 20] However, publicly reported outcomes may not be fully representative of hospital quality and patient safety.[28, 29]

The limited association between hospital financial performance and publicly reported outcomes for AMI and PNA is noteworthy for several reasons. First, publicly reporting outcomes alone without concomitant changes to reimbursement may be inadequate to create strong financial incentives for hospital investment in quality improvement initiatives. Hospitals participating in both public reporting of outcomes and pay‐for‐performance have been shown to achieve greater improvements in outcomes than hospitals engaged only in public reporting.[30] Our time interval for ascertainment of outcomes preceded CMS implementation of the Hospital Readmissions Reduction Program (HRRP) in October 2012, which withholds up to 3% of Medicare hospital reimbursements for higher than expected mortality and readmission rates for AMI, CHF, and PNA. Once outcomes data become available for a 3‐year post‐HRRP implementation period, the impact of this combined approach can be assessed. Second, because adherence to many evidence‐based process measures for these conditions (ie, aspirin use in AMI) is already high, there may be a ceiling effect present that obviates the need for further hospital financial investment to optimize delivery of best practices.[31, 32] Third, hospitals themselves may contribute little to variation in mortality and readmission risk. Of the total variation in mortality and readmission rates among Texas Medicare beneficiaries, only about 1% is attributable to hospitals, whereas 42% to 56% of the variation is explained by differences in patient characteristics.[33, 34] Fourth, there is either low‐quality or insufficient evidence that transitional care interventions specifically targeted to patients with AMI or PNA result in better outcomes.[35] Thus, greater financial investment in hospital‐initiated and postdischarge transitional care interventions for these specific conditions may result in less than the desired effect. Lastly, many hospitalizations for these conditions are emergency hospitalizations that occur after patients present to the emergency department with unexpected and potentially life‐threatening symptoms. Thus, patients may not be able to incorporate the reputation or performance metrics of a hospital in their decisions for where they are hospitalized for AMI, CHF, or PNA despite the public reporting of outcomes.

Given the strong evidence that transitional care interventions reduce readmissions and mortality among patients hospitalized with CHF, we were surprised to find that improved hospital financial performance was associated with an increased risk‐adjusted CHF mortality rate.[36] This association held true for all 3 different measures of hospital financial performance, suggesting that this unexpected finding is unlikely to be the result of statistical chance, though potential reasons for this association remain unclear. One possibility is that the CMS model for CHF mortality may not adequately risk adjust for severity of illness.[37, 38] Thus, robust financial performance may be a marker for hospitals with more advanced heart failure services that care for more patients with severe illness.

Our findings should be interpreted in the context of certain limitations. Our study only included an analysis of outcomes for AMI, CHF, and PNA among older fee‐for‐service Medicare beneficiaries aggregated at the hospital level in California between 2008 and 2012, so generalizability to other populations, conditions, states, and time periods is uncertain. The observational design precludes a robust causal inference between financial performance and outcomes. For readmissions, rates were publicly reported for only the last 2 years of the 3‐year reporting period; thus, our findings may underestimate the association between hospital financial performance and publicly reported readmission rates.

CONCLUSION

There is no consistent relationship between hospital financial performance and subsequent publicly reported outcomes for AMI and PNA. However, for unclear reasons, hospitals with better financial performance had modestly higher CHF mortality rates. Given this limited association, public reporting of outcomes may have had less than the intended impact in motivating hospitals to invest in quality improvement. Additional financial incentives in addition to public reporting, such as readmissions penalties, may help motivate hospitals with robust financial performance to further improve outcomes. This would be a key area for future investigation once outcomes data are available for the 3‐year period following CMS implementation of readmissions penalties in 2012. Reassuringly, there was no association between low 30‐day mortality and readmissions rates and subsequent poor financial performance, suggesting that improved outcomes do not necessarily lead to loss of revenue.

Infections acquired in the hospital are termed healthcare‐associated infections (HAIs) and include central lineassociated blood stream infections (CLABSIs). Among HAIs, CLABSIs cause the highest number of preventable deaths.[1] Central venous catheters (CVCs) or central lines are commonly used in the hospital.[2] Each year their use is linked to 250,000 cases of CLABSIs in the United States.[3] Some CLABSIs may be prevented by the prompt removal of the line.[4] However, CVCs are often retained after their clinical indication has lapsed and are then referred to as idle lines.[5, 6] In this work, we propose and theoretically test a guideline to facilitate the safe removal of an idle line by observing the agreement and disagreement between actual practice and the proposed guideline.

METHODS

RESULTS

A total of 126 lines were observed in 126 patients. Eighty‐three (65.9%) of the lines were PICCs. The remaining 43 (34.1%) were CVLs. The indications for line placement were distributed between the need for central access, total parenteral nutrition, or antibiotics (Table 2).

Out of the 126 patients, 50 (39.7%) were discharged from the hospital, died, were transferred to a higher level of care, or transitioned to palliative or hospice care with the line in place. In the remaining 76 patients, the audit captured 635 days, out of which a line was in place for 522 (82.2%) days. Of these 522 days, the line's presence was justified by our criteria for 351 (67.2%) days. The most common reason for a line to be justified on any given day was the need for antibiotics followed by the presence of SIRS criteria (Table 3). The remaining 171 (32.7%) days were idle.

A comparison of the actual removal of the 76 central lines in practice relative to the proposed guideline of removing it the day following the first idle day is displayed in Figure 1. The central line was removed prior to our proposed guideline in 11 (14.5%) patients, and waiting for an idle day in these patients would have added 46 line days. In almost half the patients (n = 36, 47.4%), the line was removed in agreement with the proposed guideline. None of the patients in whom the line was removed prior to or in accordance with our proposed guideline required a line reinsertion. Line removal was delayed in 29 (38.2%) patients when compared to our proposed guideline. In these patients, following the guideline would have created 122 line‐free days. Most (n = 102, 83.6%) of these potential line‐free days were idle. Twenty (16.4%) were justified, of which half (n = 10) were justified by meeting SIRS criteria.

Figure 1

DISCUSSION

Approximately 1 in every 25 inpatients in the United States has at least 1 HAI on any given day.[11] The case fatality rate from a CLABSI may be as high as 12%, and up to 70% of these infections may be preventable.[1, 12] Interventions successful in decreasing CLABSIs have focused on patients in ICUs.[13] However, CVCs are increasingly prevalent outside the ICU, with over 4.5 million line days in non‐ICU beds reported to the National Healthcare Safety Network in 2012 compared to 2.5 million in 2010.[2, 14] However, adherence rates to infection control practices may be lower on the wards than in the ICUs.[6, 15] Consequently, although the number of CLABSIs has declined over the last decade, most are now occurring outside the ICU.[16] These trends underscore the need to develop strategies aimed at CLABSI prevention on the floors.

Analogous to the life cycle of a urinary catheter described by Meddings et al.,[17] strategies to prevent CLABSIs and other CVC‐related complications may be designed around the life cycle of a CVC. The life cycle starts with insertion and moves on to the maintenance, removal, and possible reinsertion of the line. The process thus starts with the decision to place the line. Over the last decade, this decision making has changed in part due to PICCs. This shift is reflected in PICC prevalence rates: in 2001, 11% of audited central lines were PICCs compared to 56% in 2007.[5, 6] In our audit, 66% of the CVCs were PICCs. This increase in the use of PICCs may be attributable to the ease and safety of their placement coupled with the increased availability of vascular access placement teams.[18] The risk of overuse that may result from such expediency may be countered by adhering to guidelines such as the Michigan Appropriateness Guide for Intravenous Catheters, which provides both clinically detailed guidance and an impetus for reflective decision making around intravenous access.[19]

The placement of CVCs for prolonged parenteral antibiotics may be a particular subset that bears further exploration. Similar to previous reports, we found that a large number of the CVCs were both inserted for and justified by the need for IV antibiotics.[5] Guidelines delineated by the Infectious Diseases Society of America regarding outpatient parenteral antibiotics weigh both the duration of therapy and the antimicrobial's potential for causing phlebitis when recommending the type of intravascular access.[20] Many courses may therefore be completed through peripheral or midline catheters. Developing strong partnerships between infectious disease specialists, hospitalists, and the facilities or home‐care services treating these patients may curtail the use of CVCs for antimicrobial administration.

The main focus of our work is on facilitating the safe removal of CVCs. The risk of CLABSIs increases each day a CVC is in place, and guidelines to prevent CLABSIs include recommendations to promptly remove nonessential catheters.[4, 21] There is also an emerging understanding that the risk of a PICC‐related CLABSI approaches that from a traditional central line in hospitalized patients, and PICCs confer an increased risk of venous thromboembolism.[18, 22] Although nearly half of surveyed hospitalists recently reported leaving PICCs in place until discharge day, our data suggest that this practice may be driven by the trajectory of a patient's recovery as much as by knowledge gaps related to the use of PICCs.[23] In nearly half the instances, clinical practice already mirrors our proposed guideline, with line removal coinciding with both the timing proposed by our guideline and discharge day. However, there is room for improvement, as line removal may have been expedited in the 29 patients in whom the line was retained after the first idle day. Maintaining an awareness of its presence and weighing its risks and benefits daily may facilitate the removal of a CVC. Based on the recent findings that up to a quarter of clinicians are unaware that their patients have a central line, the mere reminder of the presence of a line using such criteria may expedite its removal by triggering a purposeful reassessment of its ongoing need.[24] Premature CVC removal requiring line reinsertion is an unintended consequence that may emerge from the earlier removal of lines. In our sample, none of the patients who had lines removed either prior to or in accordance with our proposed guideline required a line reinsertion. In addition to line reinsertion, delays in laboratory testing and reporting due to the unavailability of access, increased patient discomfort, or increased workload on the bedside nurse or vascular access team must also be considered when implementing strategies aimed at decreasing line days.

We envisage using these criteria to both empower practitioners with knowledge and foster shared accountability between all team members by using a uniform tool. This can occur through partnerships between infection control, clinical nurse specialists, bedside nursing, and physicians. The electronic medical record could be leveraged to scan the record for the criteria and create a notification when the line becomes idle. In alignment with the Michigan Appropriateness Guide for Intravenous Catheters guidelines, we do not support the removal of lines by nursing staff without physician notification.[19] Such principles have been successfully harnessed in strategies to prevent both catheter‐associated urinary tract infections and CLABSIs in ICUs.[13, 25] In light of the complexity surrounding the decision making for CVCs, our criteria were focused on the wards and erred on the side of clinical caution. This clinical conservatism is apparent in the patients in whom lines were removed prior to what our guideline would propose, yet none of the patients required a line reinsertion. As concerns about recrudescent clinical instability may drive decision making around line removal, such conservatism may be warranted initially. However, the fidelity of these criteria in the clinical setting will need prospective validation. In particular, the inclusion of SIRS criteria may have led to an overestimation of justified days. Further studies may be needed to refine the criteria and find a clinical hierarchy that balances the risks and benefits of retaining a central line.

Our work has certain limitations. It is a single center's experience, and our findings may not therefore be generalizable. Except for when the indication for the line was for difficult access, we did not attempt to verify the presence of a peripheral IV. This, in combination with the inclusion of SIRS criteria, likely leads to an underestimation of idle days. In the interest of focusing on patients in whom the decision making around a line would be the least controversial, we did not continue to follow patients who were transferred to a higher level of care. It is possible, however, that these transfers were precipitated by line‐associated complications such as sepsis and would be important to track. We did not measure the agreement between data collectors, although definitions and methodologies were standardized and reviewed prior to data collection. As this was an observational assessment of a proposed guideline, we cannot predict how the recommendations generated by it will be received by clinicians. Although this may prove to be a barrier in adoption, we hope that the conversation it initiates leads to change.

Hospitalists are positioned to potentially influence the entire life cycle of a central line on the floor. Strategies can be enacted at each stage to help decrease the potential of harm from these devices to our patients. Creating and testing criteria and guidelines such as we propose represents just 1 such strategy in a multidisciplinary effort to provide the best possible care we can.

Infections acquired in the hospital are termed healthcare‐associated infections (HAIs) and include central lineassociated blood stream infections (CLABSIs). Among HAIs, CLABSIs cause the highest number of preventable deaths.[1] Central venous catheters (CVCs) or central lines are commonly used in the hospital.[2] Each year their use is linked to 250,000 cases of CLABSIs in the United States.[3] Some CLABSIs may be prevented by the prompt removal of the line.[4] However, CVCs are often retained after their clinical indication has lapsed and are then referred to as idle lines.[5, 6] In this work, we propose and theoretically test a guideline to facilitate the safe removal of an idle line by observing the agreement and disagreement between actual practice and the proposed guideline.

METHODS

RESULTS

A total of 126 lines were observed in 126 patients. Eighty‐three (65.9%) of the lines were PICCs. The remaining 43 (34.1%) were CVLs. The indications for line placement were distributed between the need for central access, total parenteral nutrition, or antibiotics (Table 2).

Out of the 126 patients, 50 (39.7%) were discharged from the hospital, died, were transferred to a higher level of care, or transitioned to palliative or hospice care with the line in place. In the remaining 76 patients, the audit captured 635 days, out of which a line was in place for 522 (82.2%) days. Of these 522 days, the line's presence was justified by our criteria for 351 (67.2%) days. The most common reason for a line to be justified on any given day was the need for antibiotics followed by the presence of SIRS criteria (Table 3). The remaining 171 (32.7%) days were idle.

A comparison of the actual removal of the 76 central lines in practice relative to the proposed guideline of removing it the day following the first idle day is displayed in Figure 1. The central line was removed prior to our proposed guideline in 11 (14.5%) patients, and waiting for an idle day in these patients would have added 46 line days. In almost half the patients (n = 36, 47.4%), the line was removed in agreement with the proposed guideline. None of the patients in whom the line was removed prior to or in accordance with our proposed guideline required a line reinsertion. Line removal was delayed in 29 (38.2%) patients when compared to our proposed guideline. In these patients, following the guideline would have created 122 line‐free days. Most (n = 102, 83.6%) of these potential line‐free days were idle. Twenty (16.4%) were justified, of which half (n = 10) were justified by meeting SIRS criteria.

Figure 1

DISCUSSION

Approximately 1 in every 25 inpatients in the United States has at least 1 HAI on any given day.[11] The case fatality rate from a CLABSI may be as high as 12%, and up to 70% of these infections may be preventable.[1, 12] Interventions successful in decreasing CLABSIs have focused on patients in ICUs.[13] However, CVCs are increasingly prevalent outside the ICU, with over 4.5 million line days in non‐ICU beds reported to the National Healthcare Safety Network in 2012 compared to 2.5 million in 2010.[2, 14] However, adherence rates to infection control practices may be lower on the wards than in the ICUs.[6, 15] Consequently, although the number of CLABSIs has declined over the last decade, most are now occurring outside the ICU.[16] These trends underscore the need to develop strategies aimed at CLABSI prevention on the floors.

Analogous to the life cycle of a urinary catheter described by Meddings et al.,[17] strategies to prevent CLABSIs and other CVC‐related complications may be designed around the life cycle of a CVC. The life cycle starts with insertion and moves on to the maintenance, removal, and possible reinsertion of the line. The process thus starts with the decision to place the line. Over the last decade, this decision making has changed in part due to PICCs. This shift is reflected in PICC prevalence rates: in 2001, 11% of audited central lines were PICCs compared to 56% in 2007.[5, 6] In our audit, 66% of the CVCs were PICCs. This increase in the use of PICCs may be attributable to the ease and safety of their placement coupled with the increased availability of vascular access placement teams.[18] The risk of overuse that may result from such expediency may be countered by adhering to guidelines such as the Michigan Appropriateness Guide for Intravenous Catheters, which provides both clinically detailed guidance and an impetus for reflective decision making around intravenous access.[19]

The placement of CVCs for prolonged parenteral antibiotics may be a particular subset that bears further exploration. Similar to previous reports, we found that a large number of the CVCs were both inserted for and justified by the need for IV antibiotics.[5] Guidelines delineated by the Infectious Diseases Society of America regarding outpatient parenteral antibiotics weigh both the duration of therapy and the antimicrobial's potential for causing phlebitis when recommending the type of intravascular access.[20] Many courses may therefore be completed through peripheral or midline catheters. Developing strong partnerships between infectious disease specialists, hospitalists, and the facilities or home‐care services treating these patients may curtail the use of CVCs for antimicrobial administration.

The main focus of our work is on facilitating the safe removal of CVCs. The risk of CLABSIs increases each day a CVC is in place, and guidelines to prevent CLABSIs include recommendations to promptly remove nonessential catheters.[4, 21] There is also an emerging understanding that the risk of a PICC‐related CLABSI approaches that from a traditional central line in hospitalized patients, and PICCs confer an increased risk of venous thromboembolism.[18, 22] Although nearly half of surveyed hospitalists recently reported leaving PICCs in place until discharge day, our data suggest that this practice may be driven by the trajectory of a patient's recovery as much as by knowledge gaps related to the use of PICCs.[23] In nearly half the instances, clinical practice already mirrors our proposed guideline, with line removal coinciding with both the timing proposed by our guideline and discharge day. However, there is room for improvement, as line removal may have been expedited in the 29 patients in whom the line was retained after the first idle day. Maintaining an awareness of its presence and weighing its risks and benefits daily may facilitate the removal of a CVC. Based on the recent findings that up to a quarter of clinicians are unaware that their patients have a central line, the mere reminder of the presence of a line using such criteria may expedite its removal by triggering a purposeful reassessment of its ongoing need.[24] Premature CVC removal requiring line reinsertion is an unintended consequence that may emerge from the earlier removal of lines. In our sample, none of the patients who had lines removed either prior to or in accordance with our proposed guideline required a line reinsertion. In addition to line reinsertion, delays in laboratory testing and reporting due to the unavailability of access, increased patient discomfort, or increased workload on the bedside nurse or vascular access team must also be considered when implementing strategies aimed at decreasing line days.

We envisage using these criteria to both empower practitioners with knowledge and foster shared accountability between all team members by using a uniform tool. This can occur through partnerships between infection control, clinical nurse specialists, bedside nursing, and physicians. The electronic medical record could be leveraged to scan the record for the criteria and create a notification when the line becomes idle. In alignment with the Michigan Appropriateness Guide for Intravenous Catheters guidelines, we do not support the removal of lines by nursing staff without physician notification.[19] Such principles have been successfully harnessed in strategies to prevent both catheter‐associated urinary tract infections and CLABSIs in ICUs.[13, 25] In light of the complexity surrounding the decision making for CVCs, our criteria were focused on the wards and erred on the side of clinical caution. This clinical conservatism is apparent in the patients in whom lines were removed prior to what our guideline would propose, yet none of the patients required a line reinsertion. As concerns about recrudescent clinical instability may drive decision making around line removal, such conservatism may be warranted initially. However, the fidelity of these criteria in the clinical setting will need prospective validation. In particular, the inclusion of SIRS criteria may have led to an overestimation of justified days. Further studies may be needed to refine the criteria and find a clinical hierarchy that balances the risks and benefits of retaining a central line.

Our work has certain limitations. It is a single center's experience, and our findings may not therefore be generalizable. Except for when the indication for the line was for difficult access, we did not attempt to verify the presence of a peripheral IV. This, in combination with the inclusion of SIRS criteria, likely leads to an underestimation of idle days. In the interest of focusing on patients in whom the decision making around a line would be the least controversial, we did not continue to follow patients who were transferred to a higher level of care. It is possible, however, that these transfers were precipitated by line‐associated complications such as sepsis and would be important to track. We did not measure the agreement between data collectors, although definitions and methodologies were standardized and reviewed prior to data collection. As this was an observational assessment of a proposed guideline, we cannot predict how the recommendations generated by it will be received by clinicians. Although this may prove to be a barrier in adoption, we hope that the conversation it initiates leads to change.

Hospitalists are positioned to potentially influence the entire life cycle of a central line on the floor. Strategies can be enacted at each stage to help decrease the potential of harm from these devices to our patients. Creating and testing criteria and guidelines such as we propose represents just 1 such strategy in a multidisciplinary effort to provide the best possible care we can.

Infections acquired in the hospital are termed healthcare‐associated infections (HAIs) and include central lineassociated blood stream infections (CLABSIs). Among HAIs, CLABSIs cause the highest number of preventable deaths.[1] Central venous catheters (CVCs) or central lines are commonly used in the hospital.[2] Each year their use is linked to 250,000 cases of CLABSIs in the United States.[3] Some CLABSIs may be prevented by the prompt removal of the line.[4] However, CVCs are often retained after their clinical indication has lapsed and are then referred to as idle lines.[5, 6] In this work, we propose and theoretically test a guideline to facilitate the safe removal of an idle line by observing the agreement and disagreement between actual practice and the proposed guideline.

METHODS