Brian Leas, MS, MA

Hospital readmissions cost Medicare $15 to $17 billion per year.[1, 2] In 2010, the Hospital Readmission Reduction Program (HRRP), created by the Patient Protection and Affordable Care Act, authorized the Centers for Medicare and Medicaid Services (CMS) to penalize hospitals with higher‐than‐expected readmission rates for certain index conditions.[3] Other payers may follow suit, so hospitals and health systems nationwide are devoting significant resources to reducing readmissions.[4, 5, 6]

Implicit in these efforts are the assumptions that a significant proportion of readmissions are preventable, and that preventable readmissions can be identified. Unfortunately, estimates of preventability vary widely.[7, 8] In this article, we examine how preventable readmissions have been defined, measured, and calculated, and explore the associated implications for readmission reduction efforts.

THE MEDICARE READMISSION METRIC

The medical literature reveals substantial heterogeneity in how readmissions are assessed. Time periods range from 14 days to 4 years, and readmissions may be counted differently depending on whether they are to the same hospital or to any hospital, whether they are for the same (or a related) condition or for any condition, whether a patient is allowed to count only once during the follow‐up period, how mortality is treated, and whether observation stays are considered.[9]

Despite a lack of consensus in the literature, the approach adopted by CMS is endorsed by the National Quality Forum (NQF)[10] and has become the de facto standard for calculating readmission rates. CMS derives risk‐standardized readmission rates for acute myocardial infarction (AMI), heart failure (HF), and pneumonia (PN), using administrative claims data for each Medicare fee‐for‐service beneficiary 65 years or older.[11, 12, 13, 14] CMS counts the first readmission (but not subsequent ones) for any cause within 30 days of the index discharge, including readmissions to other facilities. Certain planned readmissions for revascularization are excluded, as are patients who left against medical advice, transferred to another acute‐care hospital, or died during the index admission. Admissions to psychiatric, rehabilitation, cancer specialty, and children's hospitals[12] are also excluded, as well as patients classified as observation status for either hospital stay.[15] Only administrative data are used in readmission calculations (ie, there are no chart reviews or interviews with healthcare personnel or patients). Details are published online and updated at least annually.[15]

EFFECTS AND LIMITATIONS OF THE HRRP AND THE CMS READMISSION METRIC

Penalizing hospitals for higher‐than‐expected readmission rates based on the CMS metric has been successful in the sense that hospitals now feel more accountable for patient outcomes after discharge; they are implementing transitional care programs, improving communication, and building relationships with community programs.[4, 5, 16] Early data suggest a small decline in readmission rates of Medicare beneficiaries nationally.[17] Previously, such readmission rates were constant.[18]

Nevertheless, significant concerns with the current approach have surfaced.[19, 20, 21] First, why choose 30 days? This time horizon was believed to be long enough to identify readmissions attributable to an index admission and short enough to reflect hospital‐delivered care and transitions to the outpatient setting, and it allows for collaboration between hospitals and their communities to reduce readmissions.[3] However, some have argued that this time horizon has little scientific basis,[22] and that hospitals are unfairly held accountable for a timeframe when outcomes may largely be influenced by the quality of outpatient care or the development of new problems.[23, 24] Approximately one‐third of 30‐day readmissions occur within the first 7 days, and more than half (55.7%) occur within the first 14 days[22, 25]; such time frames may be more appropriate for hospital accountability.[26]

Second, spurred by the focus of CMS penalties, efforts to reduce readmissions have largely concerned patients admitted for HF, AMI, or PN, although these 3 medical conditions account for only 10% of Medicare hospitalizations.[18] Programs focused on a narrow patient population may not benefit other patients with high readmission rates, such as those with gastrointestinal or psychiatric problems,[2] or lead to improvements in the underlying processes of care that could benefit patients in additional ways. Indeed, research suggests that low readmission rates may not be related to other measures of hospital quality.[27, 28]

Third, public reporting and hospital penalties are based on 3‐year historical performance, in part to accumulate a large enough sample size for each diagnosis. Hospitals that seek real‐time performance monitoring are limited to tracking surrogate outcomes, such as readmissions back to their own facility.[29, 30] Moreover, because of the long performance time frame, hospitals that achieve rapid improvement may endure penalties precisely when they are attempting to sustain their achievements.

Fourth, the CMS approach utilizes a complex risk‐standardization methodology, which has only modest ability to predict readmissions and allow hospital comparisons.[9] There is no adjustment for community characteristics, even though practice patterns are significantly associated with readmission rates,[9, 31] and more than half of the variation in readmission rates across hospitals can be explained by characteristics of the community such as access to care.[32] Moreover, patient factors, such as race and socioeconomic status, are currently not included in an attempt to hold hospitals to similar standards regardless of their patient population. This is hotly contested, however, and critics note this policy penalizes hospitals for factors outside of their control, such as patients' ability to afford medications.[33] Indeed, the June 2013 Medicare Payment Advisory Committee (MedPAC) report to Congress recommended evaluating hospital performance against facilities with a like percentage of low‐income patients as a way to take into account socioeconomic status.[34]

Fifth, observation stays are excluded, so patients who remain in observation status during their index or subsequent hospitalization cannot be counted as a readmission. Prevalence of observation care has increased, raising concerns that inpatient admissions are being shifted to observation status, producing an artificial decline in readmissions.[35] Fortunately, recent population‐level data provide some reassuring evidence to the contrary.[36]

Finally, and perhaps most significantly, the current readmission metric does not consider preventability. Recent reviews have demonstrated that estimates of preventability vary widely in individual studies, ranging from 5% to 79%, depending on study methodology and setting.[7, 8] Across these studies, on average, only 23% of 30‐day readmissions appear to be avoidable.[8] Another way to consider the preventability of hospital readmissions is by noting that the most effective multimodal care‐transition interventions reduce readmission rates by only about 30%, and most interventions are much less effective.[26] The likely fact that only 23% to 30% of readmissions are preventable has profound implications for the anticipated results of hospital readmission reduction efforts. Interventions that are 75% effective in reducing preventable readmissions should be expected to produce only an 18% to 22% reduction in overall readmission rates.[37]

FOCUSING ON PREVENTABLE READMISSIONS

A greater focus on identifying and targeting preventable readmissions would offer a number of advantages over the present approach. First, it is more meaningful to compare hospitals based on their percentage of discharges resulting in a preventable readmission, than on the basis of highly complex risk standardization procedures for selected conditions. Second, a focus on preventable readmissions more clearly identifies and permits hospitals to target opportunities for improvement. Third, if the focus were on preventable readmissions for a large number of conditions, the necessary sample size could be obtained over a shorter period of time. Overall, such a preventable readmissions metric could serve as a more agile and undiluted performance indicator, as opposed to the present 3‐year rolling average rate of all‐cause readmissions for certain conditions, the majority of which are probably not preventable.

DEFINING PREVENTABILITY

Defining a preventable readmission is critically important. However, neither a consensus definition nor a validated standard for assessing preventable hospital readmissions exists. Different conceptual frameworks and terms (eg, avoidable, potentially preventable, or urgent readmission) complicate the issue.[38, 39, 40]

Although the CMS measure does not address preventability, it is helpful to consider whether other readmission metrics incorporate this concept. The United Health Group's (UHG, formerly Pacificare) All‐Cause Readmission Index, University HealthSystem Consortium's 30‐Day Readmission Rate (all cause), and 3M Health Information Systems' (3M) Potentially Preventable Readmissions (PPR) are 3 commonly used measures.

Of these, only the 3M PPR metric includes the concept of preventability. 3M created a proprietary matrix of 98,000 readmission‐index admission All Patient Refined Diagnosis Related Group pairs based on the review of several physicians and the logical assumption that a readmission for a clinically related diagnosis is potentially preventable.[24, 41] Readmission and index admissions are considered clinically related if any of the following occur: (1) medical readmission for continuation or recurrence of an initial, or closely related, condition; (2) medical readmission for acute decompensation of a chronic condition that was not the reason for the index admission but was plausibly related to care during or immediately afterward (eg, readmission for diabetes in a patient whose index admission was AMI); (3) medical readmission for acute complication plausibly related to care during index admission; (4) readmission for surgical procedure for continuation or recurrence of initial problem (eg, readmission for appendectomy following admission for abdominal pain and fever); or (5) readmission for surgical procedure to address complication resulting from care during index admission.[24, 41] The readmission time frame is not standardized and may be set by the user. Though conceptually appealing in some ways, CMS and the NQF have expressed concern about this specific approach because of the uncertain reliability of the relatedness of the admission‐readmission diagnosis dyads.[3]

In the research literature, only a few studies have examined the 3M PPR or other preventability assessments that rely on the relatedness of diagnostic codes.[8] Using the 3M PPR, a study showed that 78% of readmissions were classified as potentially preventable,[42] which explains why the 3M PPR and all‐cause readmission metric may correlate highly.[43] Others have demonstrated that ratings of hospital performance on readmission rates vary by a moderate to large amount, depending on whether the 3M PPR, CMS, or UHG methodology is used.[43, 44] An algorithm called SQLape[45, 46] is used in Switzerland to benchmark hospitals and defines potentially avoidable readmissions as being related to index diagnoses or complications of those conditions. It has recently been tested in the United States in a single‐center study,[47] and a multihospital study is underway.

Aside from these algorithms using related diagnosis codes, most ratings of preventability have relied on subjective assessments made primarily through a review of hospital records, and approximately one‐third also included data from clinic visits or interviews with the treating medical team or patients/families.[8] Unfortunately, these reports provide insufficient detail on how to apply their preventability criteria to subsequent readmission reviews. Studies did, however, provide categories of preventability into which readmissions could be organized (see Supporting Information, Appendix Table 1, in the online version of this article for details from a subset of studies cited in van Walraven's reviews that illustrate this point).

Assessment of preventability by clinician review can be challenging. In general, such assessments have considered readmissions resulting from factors within the hospital's control to be avoidable (eg, providing appropriate discharge instructions, reconciling medications, arranging timely postdischarge follow‐up appointments), whereas readmissions resulting from factors not within the hospital's control are unavoidable (eg, patient socioeconomic status, social support, disease progression). However, readmissions resulting from patient behaviors or social reasons could potentially be classified as avoidable or unavoidable depending on the circumstances. For example, if a patient decides not to take a prescribed antibiotic and is readmitted with worsening infection, this could be classified as an unavoidable readmission from the hospital's perspective. Alternatively, if the physician prescribing the antibiotic was inattentive to the cost of the medication and the patient would have taken a less expensive medication had it been prescribed, this could be classified as an avoidable readmission. Differing interpretations of contextual factors may partially account for the variability in clinical assessments of preventability.

Indeed, despite the lack of consensus around a standard method of defining preventability, hospitals and health systems are moving forward to address the issue and reduce readmissions. A recent survey by America's Essential Hospitals (previously the National Association of Public Hospitals and Health Systems), indicated that: (1) reducing readmissions was a high priority for the majority (86%) of members, (2) most had established interdisciplinary teams to address the issue, and (3) over half had a formal process for determining which readmissions were potentially preventable. Of the survey respondents, just over one‐third rely on staff review of individual patient charts or patient and family interviews, and slightly less than one‐third rely on other mechanisms such as external consultants, criteria developed by other entities, or the Institute for Clinical Systems Improvement methodology.[48] Approximately one‐fifth make use of 3M's PPR product, and slightly fewer use the list of the Agency for Healthcare Research and Quality's ambulatory care sensitive conditions (ACSCs). These are medical conditions for which it is believed that good outpatient care could prevent the need for hospitalization (eg, asthma, congestive heart failure, diabetes) or for which early intervention minimizes complications.[49] Hospitalization rates for ACSCs may represent a good measure of excess hospitalization, with a focus on the quality of outpatient care.

RECOMMENDATIONS

We recommend that reporting of hospital readmission rates be based on preventable or potentially preventable readmissions. Although we acknowledge the challenges in doing so, the advantages are notable. At minimum, a preventable readmission rate would more accurately reflect the true gap in care and therefore hospitals' real opportunity for improvement, without being obscured by readmissions that are not preventable.

Because readmission rates are used for public reporting and financial penalties for hospitals, we favor a measure of preventability that reflects the readmissions that the hospital or hospital system has the ability to prevent. This would not penalize hospitals for factors that are under the control of others, namely patients and caregivers, community supports, or society at large. We further recommend that this measure apply to a broader composite of unplanned care, inclusive of both inpatient and observation stays, which have little distinction in patients' eyes, and both represent potentially unnecessary utilization of acute‐care resources.[50] Such a measure would require development, validation, and appropriate vetting before it is implemented.

The first step is for researchers and policy makers to agree on how a measure of preventable or potentially preventable readmissions could be defined. A common element of preventability assessment is to identify the degree to which the reasons for readmission are related to the diagnoses of the index hospitalization. To be reliable and scalable, this measure will need to be based on algorithms that relate the index and readmission diagnoses, most likely using claims data. Choosing common medical and surgical conditions and developing a consensus‐based list of related readmission diagnoses is an important first step. It would also be important to include some less common conditions, because they may reflect very different aspects of hospital care.

An approach based on a list of related diagnoses would represent potentially preventable rehospitalizations. Generally, clinical review is required to determine actual preventability, taking into account patient factors such as a high level of illness or functional impairment that leads to clinical decompensation in spite of excellent management.[51, 52] Clinical review, like a root cause analysis, also provides greater insight into hospital processes that may warrant improvement. Therefore, even if an administrative measure of potentially preventable readmissions is implemented, hospitals may wish to continue performing detailed clinical review of some readmissions for quality improvement purposes. When clinical review becomes more standardized,[53] a combined approach that uses administrative data plus clinical verification and arbitration may be feasible, as with hospital‐acquired infections.

Similar work to develop related sets of admission and readmission diagnoses has already been undertaken in development of the 3M PPR and SQLape measures.[41, 46] However, the 3M PPR is a proprietary system that has low specificity and a high false‐positive rate for identifying preventable readmissions when compared to clinical review.[42] Moreover, neither measure has yet achieved the consensus required for widespread adoption in the United States. What is needed is a nonproprietary listing of related admission and readmission diagnoses, developed with the engagement of relevant stakeholders, that goes through a period of public comment and vetting by a body such as the NQF.

Until a validated measure of potentially preventable readmission can be developed, how could the current approach evolve toward preventability? The most feasible, rapidly implementable change would be to alter the readmission time horizon from 30 days to 7 or 15 days. A 30‐day period holds hospitals accountable for complications of outpatient care or new problems that may develop weeks after discharge. Even though this may foster shared accountability and collaboration among hospitals and outpatient or community settings, research has demonstrated that early readmissions (eg, within 715 days of discharge) are more likely preventable.[54] Second, consideration of the socioeconomic status of hospital patients, as recommended by MedPAC,[34] would improve on the current model by comparing hospitals to like facilities when determining penalties for excess readmission rates. Finally, adjustment for community factors, such as practice patterns and access to care, would enable readmission metrics to better reflect factors under the hospital's control.[32]

CONCLUSION

Holding hospitals accountable for the quality of acute and transitional care is an important policy initiative that has accelerated many improvements in discharge planning and care coordination. Optimally, the policies, public reporting, and penalties should target preventable readmissions, which may represent as little as one‐quarter of all readmissions. By summarizing some of the issues in defining preventability, we hope to foster continued refinement of quality metrics used in this arena.

Hospital readmissions cost Medicare $15 to $17 billion per year.[1, 2] In 2010, the Hospital Readmission Reduction Program (HRRP), created by the Patient Protection and Affordable Care Act, authorized the Centers for Medicare and Medicaid Services (CMS) to penalize hospitals with higher‐than‐expected readmission rates for certain index conditions.[3] Other payers may follow suit, so hospitals and health systems nationwide are devoting significant resources to reducing readmissions.[4, 5, 6]

Implicit in these efforts are the assumptions that a significant proportion of readmissions are preventable, and that preventable readmissions can be identified. Unfortunately, estimates of preventability vary widely.[7, 8] In this article, we examine how preventable readmissions have been defined, measured, and calculated, and explore the associated implications for readmission reduction efforts.

THE MEDICARE READMISSION METRIC

The medical literature reveals substantial heterogeneity in how readmissions are assessed. Time periods range from 14 days to 4 years, and readmissions may be counted differently depending on whether they are to the same hospital or to any hospital, whether they are for the same (or a related) condition or for any condition, whether a patient is allowed to count only once during the follow‐up period, how mortality is treated, and whether observation stays are considered.[9]

Despite a lack of consensus in the literature, the approach adopted by CMS is endorsed by the National Quality Forum (NQF)[10] and has become the de facto standard for calculating readmission rates. CMS derives risk‐standardized readmission rates for acute myocardial infarction (AMI), heart failure (HF), and pneumonia (PN), using administrative claims data for each Medicare fee‐for‐service beneficiary 65 years or older.[11, 12, 13, 14] CMS counts the first readmission (but not subsequent ones) for any cause within 30 days of the index discharge, including readmissions to other facilities. Certain planned readmissions for revascularization are excluded, as are patients who left against medical advice, transferred to another acute‐care hospital, or died during the index admission. Admissions to psychiatric, rehabilitation, cancer specialty, and children's hospitals[12] are also excluded, as well as patients classified as observation status for either hospital stay.[15] Only administrative data are used in readmission calculations (ie, there are no chart reviews or interviews with healthcare personnel or patients). Details are published online and updated at least annually.[15]

EFFECTS AND LIMITATIONS OF THE HRRP AND THE CMS READMISSION METRIC

Penalizing hospitals for higher‐than‐expected readmission rates based on the CMS metric has been successful in the sense that hospitals now feel more accountable for patient outcomes after discharge; they are implementing transitional care programs, improving communication, and building relationships with community programs.[4, 5, 16] Early data suggest a small decline in readmission rates of Medicare beneficiaries nationally.[17] Previously, such readmission rates were constant.[18]

Nevertheless, significant concerns with the current approach have surfaced.[19, 20, 21] First, why choose 30 days? This time horizon was believed to be long enough to identify readmissions attributable to an index admission and short enough to reflect hospital‐delivered care and transitions to the outpatient setting, and it allows for collaboration between hospitals and their communities to reduce readmissions.[3] However, some have argued that this time horizon has little scientific basis,[22] and that hospitals are unfairly held accountable for a timeframe when outcomes may largely be influenced by the quality of outpatient care or the development of new problems.[23, 24] Approximately one‐third of 30‐day readmissions occur within the first 7 days, and more than half (55.7%) occur within the first 14 days[22, 25]; such time frames may be more appropriate for hospital accountability.[26]

Second, spurred by the focus of CMS penalties, efforts to reduce readmissions have largely concerned patients admitted for HF, AMI, or PN, although these 3 medical conditions account for only 10% of Medicare hospitalizations.[18] Programs focused on a narrow patient population may not benefit other patients with high readmission rates, such as those with gastrointestinal or psychiatric problems,[2] or lead to improvements in the underlying processes of care that could benefit patients in additional ways. Indeed, research suggests that low readmission rates may not be related to other measures of hospital quality.[27, 28]

Third, public reporting and hospital penalties are based on 3‐year historical performance, in part to accumulate a large enough sample size for each diagnosis. Hospitals that seek real‐time performance monitoring are limited to tracking surrogate outcomes, such as readmissions back to their own facility.[29, 30] Moreover, because of the long performance time frame, hospitals that achieve rapid improvement may endure penalties precisely when they are attempting to sustain their achievements.

Fourth, the CMS approach utilizes a complex risk‐standardization methodology, which has only modest ability to predict readmissions and allow hospital comparisons.[9] There is no adjustment for community characteristics, even though practice patterns are significantly associated with readmission rates,[9, 31] and more than half of the variation in readmission rates across hospitals can be explained by characteristics of the community such as access to care.[32] Moreover, patient factors, such as race and socioeconomic status, are currently not included in an attempt to hold hospitals to similar standards regardless of their patient population. This is hotly contested, however, and critics note this policy penalizes hospitals for factors outside of their control, such as patients' ability to afford medications.[33] Indeed, the June 2013 Medicare Payment Advisory Committee (MedPAC) report to Congress recommended evaluating hospital performance against facilities with a like percentage of low‐income patients as a way to take into account socioeconomic status.[34]

Fifth, observation stays are excluded, so patients who remain in observation status during their index or subsequent hospitalization cannot be counted as a readmission. Prevalence of observation care has increased, raising concerns that inpatient admissions are being shifted to observation status, producing an artificial decline in readmissions.[35] Fortunately, recent population‐level data provide some reassuring evidence to the contrary.[36]

Finally, and perhaps most significantly, the current readmission metric does not consider preventability. Recent reviews have demonstrated that estimates of preventability vary widely in individual studies, ranging from 5% to 79%, depending on study methodology and setting.[7, 8] Across these studies, on average, only 23% of 30‐day readmissions appear to be avoidable.[8] Another way to consider the preventability of hospital readmissions is by noting that the most effective multimodal care‐transition interventions reduce readmission rates by only about 30%, and most interventions are much less effective.[26] The likely fact that only 23% to 30% of readmissions are preventable has profound implications for the anticipated results of hospital readmission reduction efforts. Interventions that are 75% effective in reducing preventable readmissions should be expected to produce only an 18% to 22% reduction in overall readmission rates.[37]

FOCUSING ON PREVENTABLE READMISSIONS

A greater focus on identifying and targeting preventable readmissions would offer a number of advantages over the present approach. First, it is more meaningful to compare hospitals based on their percentage of discharges resulting in a preventable readmission, than on the basis of highly complex risk standardization procedures for selected conditions. Second, a focus on preventable readmissions more clearly identifies and permits hospitals to target opportunities for improvement. Third, if the focus were on preventable readmissions for a large number of conditions, the necessary sample size could be obtained over a shorter period of time. Overall, such a preventable readmissions metric could serve as a more agile and undiluted performance indicator, as opposed to the present 3‐year rolling average rate of all‐cause readmissions for certain conditions, the majority of which are probably not preventable.

DEFINING PREVENTABILITY

Defining a preventable readmission is critically important. However, neither a consensus definition nor a validated standard for assessing preventable hospital readmissions exists. Different conceptual frameworks and terms (eg, avoidable, potentially preventable, or urgent readmission) complicate the issue.[38, 39, 40]

Although the CMS measure does not address preventability, it is helpful to consider whether other readmission metrics incorporate this concept. The United Health Group's (UHG, formerly Pacificare) All‐Cause Readmission Index, University HealthSystem Consortium's 30‐Day Readmission Rate (all cause), and 3M Health Information Systems' (3M) Potentially Preventable Readmissions (PPR) are 3 commonly used measures.

Of these, only the 3M PPR metric includes the concept of preventability. 3M created a proprietary matrix of 98,000 readmission‐index admission All Patient Refined Diagnosis Related Group pairs based on the review of several physicians and the logical assumption that a readmission for a clinically related diagnosis is potentially preventable.[24, 41] Readmission and index admissions are considered clinically related if any of the following occur: (1) medical readmission for continuation or recurrence of an initial, or closely related, condition; (2) medical readmission for acute decompensation of a chronic condition that was not the reason for the index admission but was plausibly related to care during or immediately afterward (eg, readmission for diabetes in a patient whose index admission was AMI); (3) medical readmission for acute complication plausibly related to care during index admission; (4) readmission for surgical procedure for continuation or recurrence of initial problem (eg, readmission for appendectomy following admission for abdominal pain and fever); or (5) readmission for surgical procedure to address complication resulting from care during index admission.[24, 41] The readmission time frame is not standardized and may be set by the user. Though conceptually appealing in some ways, CMS and the NQF have expressed concern about this specific approach because of the uncertain reliability of the relatedness of the admission‐readmission diagnosis dyads.[3]

In the research literature, only a few studies have examined the 3M PPR or other preventability assessments that rely on the relatedness of diagnostic codes.[8] Using the 3M PPR, a study showed that 78% of readmissions were classified as potentially preventable,[42] which explains why the 3M PPR and all‐cause readmission metric may correlate highly.[43] Others have demonstrated that ratings of hospital performance on readmission rates vary by a moderate to large amount, depending on whether the 3M PPR, CMS, or UHG methodology is used.[43, 44] An algorithm called SQLape[45, 46] is used in Switzerland to benchmark hospitals and defines potentially avoidable readmissions as being related to index diagnoses or complications of those conditions. It has recently been tested in the United States in a single‐center study,[47] and a multihospital study is underway.

Aside from these algorithms using related diagnosis codes, most ratings of preventability have relied on subjective assessments made primarily through a review of hospital records, and approximately one‐third also included data from clinic visits or interviews with the treating medical team or patients/families.[8] Unfortunately, these reports provide insufficient detail on how to apply their preventability criteria to subsequent readmission reviews. Studies did, however, provide categories of preventability into which readmissions could be organized (see Supporting Information, Appendix Table 1, in the online version of this article for details from a subset of studies cited in van Walraven's reviews that illustrate this point).

Assessment of preventability by clinician review can be challenging. In general, such assessments have considered readmissions resulting from factors within the hospital's control to be avoidable (eg, providing appropriate discharge instructions, reconciling medications, arranging timely postdischarge follow‐up appointments), whereas readmissions resulting from factors not within the hospital's control are unavoidable (eg, patient socioeconomic status, social support, disease progression). However, readmissions resulting from patient behaviors or social reasons could potentially be classified as avoidable or unavoidable depending on the circumstances. For example, if a patient decides not to take a prescribed antibiotic and is readmitted with worsening infection, this could be classified as an unavoidable readmission from the hospital's perspective. Alternatively, if the physician prescribing the antibiotic was inattentive to the cost of the medication and the patient would have taken a less expensive medication had it been prescribed, this could be classified as an avoidable readmission. Differing interpretations of contextual factors may partially account for the variability in clinical assessments of preventability.

Indeed, despite the lack of consensus around a standard method of defining preventability, hospitals and health systems are moving forward to address the issue and reduce readmissions. A recent survey by America's Essential Hospitals (previously the National Association of Public Hospitals and Health Systems), indicated that: (1) reducing readmissions was a high priority for the majority (86%) of members, (2) most had established interdisciplinary teams to address the issue, and (3) over half had a formal process for determining which readmissions were potentially preventable. Of the survey respondents, just over one‐third rely on staff review of individual patient charts or patient and family interviews, and slightly less than one‐third rely on other mechanisms such as external consultants, criteria developed by other entities, or the Institute for Clinical Systems Improvement methodology.[48] Approximately one‐fifth make use of 3M's PPR product, and slightly fewer use the list of the Agency for Healthcare Research and Quality's ambulatory care sensitive conditions (ACSCs). These are medical conditions for which it is believed that good outpatient care could prevent the need for hospitalization (eg, asthma, congestive heart failure, diabetes) or for which early intervention minimizes complications.[49] Hospitalization rates for ACSCs may represent a good measure of excess hospitalization, with a focus on the quality of outpatient care.

RECOMMENDATIONS

We recommend that reporting of hospital readmission rates be based on preventable or potentially preventable readmissions. Although we acknowledge the challenges in doing so, the advantages are notable. At minimum, a preventable readmission rate would more accurately reflect the true gap in care and therefore hospitals' real opportunity for improvement, without being obscured by readmissions that are not preventable.

Because readmission rates are used for public reporting and financial penalties for hospitals, we favor a measure of preventability that reflects the readmissions that the hospital or hospital system has the ability to prevent. This would not penalize hospitals for factors that are under the control of others, namely patients and caregivers, community supports, or society at large. We further recommend that this measure apply to a broader composite of unplanned care, inclusive of both inpatient and observation stays, which have little distinction in patients' eyes, and both represent potentially unnecessary utilization of acute‐care resources.[50] Such a measure would require development, validation, and appropriate vetting before it is implemented.

The first step is for researchers and policy makers to agree on how a measure of preventable or potentially preventable readmissions could be defined. A common element of preventability assessment is to identify the degree to which the reasons for readmission are related to the diagnoses of the index hospitalization. To be reliable and scalable, this measure will need to be based on algorithms that relate the index and readmission diagnoses, most likely using claims data. Choosing common medical and surgical conditions and developing a consensus‐based list of related readmission diagnoses is an important first step. It would also be important to include some less common conditions, because they may reflect very different aspects of hospital care.

An approach based on a list of related diagnoses would represent potentially preventable rehospitalizations. Generally, clinical review is required to determine actual preventability, taking into account patient factors such as a high level of illness or functional impairment that leads to clinical decompensation in spite of excellent management.[51, 52] Clinical review, like a root cause analysis, also provides greater insight into hospital processes that may warrant improvement. Therefore, even if an administrative measure of potentially preventable readmissions is implemented, hospitals may wish to continue performing detailed clinical review of some readmissions for quality improvement purposes. When clinical review becomes more standardized,[53] a combined approach that uses administrative data plus clinical verification and arbitration may be feasible, as with hospital‐acquired infections.

Similar work to develop related sets of admission and readmission diagnoses has already been undertaken in development of the 3M PPR and SQLape measures.[41, 46] However, the 3M PPR is a proprietary system that has low specificity and a high false‐positive rate for identifying preventable readmissions when compared to clinical review.[42] Moreover, neither measure has yet achieved the consensus required for widespread adoption in the United States. What is needed is a nonproprietary listing of related admission and readmission diagnoses, developed with the engagement of relevant stakeholders, that goes through a period of public comment and vetting by a body such as the NQF.

Until a validated measure of potentially preventable readmission can be developed, how could the current approach evolve toward preventability? The most feasible, rapidly implementable change would be to alter the readmission time horizon from 30 days to 7 or 15 days. A 30‐day period holds hospitals accountable for complications of outpatient care or new problems that may develop weeks after discharge. Even though this may foster shared accountability and collaboration among hospitals and outpatient or community settings, research has demonstrated that early readmissions (eg, within 715 days of discharge) are more likely preventable.[54] Second, consideration of the socioeconomic status of hospital patients, as recommended by MedPAC,[34] would improve on the current model by comparing hospitals to like facilities when determining penalties for excess readmission rates. Finally, adjustment for community factors, such as practice patterns and access to care, would enable readmission metrics to better reflect factors under the hospital's control.[32]

CONCLUSION

Holding hospitals accountable for the quality of acute and transitional care is an important policy initiative that has accelerated many improvements in discharge planning and care coordination. Optimally, the policies, public reporting, and penalties should target preventable readmissions, which may represent as little as one‐quarter of all readmissions. By summarizing some of the issues in defining preventability, we hope to foster continued refinement of quality metrics used in this arena.

Hospital readmissions cost Medicare $15 to $17 billion per year.[1, 2] In 2010, the Hospital Readmission Reduction Program (HRRP), created by the Patient Protection and Affordable Care Act, authorized the Centers for Medicare and Medicaid Services (CMS) to penalize hospitals with higher‐than‐expected readmission rates for certain index conditions.[3] Other payers may follow suit, so hospitals and health systems nationwide are devoting significant resources to reducing readmissions.[4, 5, 6]

Implicit in these efforts are the assumptions that a significant proportion of readmissions are preventable, and that preventable readmissions can be identified. Unfortunately, estimates of preventability vary widely.[7, 8] In this article, we examine how preventable readmissions have been defined, measured, and calculated, and explore the associated implications for readmission reduction efforts.

THE MEDICARE READMISSION METRIC

The medical literature reveals substantial heterogeneity in how readmissions are assessed. Time periods range from 14 days to 4 years, and readmissions may be counted differently depending on whether they are to the same hospital or to any hospital, whether they are for the same (or a related) condition or for any condition, whether a patient is allowed to count only once during the follow‐up period, how mortality is treated, and whether observation stays are considered.[9]

Despite a lack of consensus in the literature, the approach adopted by CMS is endorsed by the National Quality Forum (NQF)[10] and has become the de facto standard for calculating readmission rates. CMS derives risk‐standardized readmission rates for acute myocardial infarction (AMI), heart failure (HF), and pneumonia (PN), using administrative claims data for each Medicare fee‐for‐service beneficiary 65 years or older.[11, 12, 13, 14] CMS counts the first readmission (but not subsequent ones) for any cause within 30 days of the index discharge, including readmissions to other facilities. Certain planned readmissions for revascularization are excluded, as are patients who left against medical advice, transferred to another acute‐care hospital, or died during the index admission. Admissions to psychiatric, rehabilitation, cancer specialty, and children's hospitals[12] are also excluded, as well as patients classified as observation status for either hospital stay.[15] Only administrative data are used in readmission calculations (ie, there are no chart reviews or interviews with healthcare personnel or patients). Details are published online and updated at least annually.[15]

EFFECTS AND LIMITATIONS OF THE HRRP AND THE CMS READMISSION METRIC

Penalizing hospitals for higher‐than‐expected readmission rates based on the CMS metric has been successful in the sense that hospitals now feel more accountable for patient outcomes after discharge; they are implementing transitional care programs, improving communication, and building relationships with community programs.[4, 5, 16] Early data suggest a small decline in readmission rates of Medicare beneficiaries nationally.[17] Previously, such readmission rates were constant.[18]

Nevertheless, significant concerns with the current approach have surfaced.[19, 20, 21] First, why choose 30 days? This time horizon was believed to be long enough to identify readmissions attributable to an index admission and short enough to reflect hospital‐delivered care and transitions to the outpatient setting, and it allows for collaboration between hospitals and their communities to reduce readmissions.[3] However, some have argued that this time horizon has little scientific basis,[22] and that hospitals are unfairly held accountable for a timeframe when outcomes may largely be influenced by the quality of outpatient care or the development of new problems.[23, 24] Approximately one‐third of 30‐day readmissions occur within the first 7 days, and more than half (55.7%) occur within the first 14 days[22, 25]; such time frames may be more appropriate for hospital accountability.[26]

Second, spurred by the focus of CMS penalties, efforts to reduce readmissions have largely concerned patients admitted for HF, AMI, or PN, although these 3 medical conditions account for only 10% of Medicare hospitalizations.[18] Programs focused on a narrow patient population may not benefit other patients with high readmission rates, such as those with gastrointestinal or psychiatric problems,[2] or lead to improvements in the underlying processes of care that could benefit patients in additional ways. Indeed, research suggests that low readmission rates may not be related to other measures of hospital quality.[27, 28]

Third, public reporting and hospital penalties are based on 3‐year historical performance, in part to accumulate a large enough sample size for each diagnosis. Hospitals that seek real‐time performance monitoring are limited to tracking surrogate outcomes, such as readmissions back to their own facility.[29, 30] Moreover, because of the long performance time frame, hospitals that achieve rapid improvement may endure penalties precisely when they are attempting to sustain their achievements.

Fourth, the CMS approach utilizes a complex risk‐standardization methodology, which has only modest ability to predict readmissions and allow hospital comparisons.[9] There is no adjustment for community characteristics, even though practice patterns are significantly associated with readmission rates,[9, 31] and more than half of the variation in readmission rates across hospitals can be explained by characteristics of the community such as access to care.[32] Moreover, patient factors, such as race and socioeconomic status, are currently not included in an attempt to hold hospitals to similar standards regardless of their patient population. This is hotly contested, however, and critics note this policy penalizes hospitals for factors outside of their control, such as patients' ability to afford medications.[33] Indeed, the June 2013 Medicare Payment Advisory Committee (MedPAC) report to Congress recommended evaluating hospital performance against facilities with a like percentage of low‐income patients as a way to take into account socioeconomic status.[34]

Fifth, observation stays are excluded, so patients who remain in observation status during their index or subsequent hospitalization cannot be counted as a readmission. Prevalence of observation care has increased, raising concerns that inpatient admissions are being shifted to observation status, producing an artificial decline in readmissions.[35] Fortunately, recent population‐level data provide some reassuring evidence to the contrary.[36]

Finally, and perhaps most significantly, the current readmission metric does not consider preventability. Recent reviews have demonstrated that estimates of preventability vary widely in individual studies, ranging from 5% to 79%, depending on study methodology and setting.[7, 8] Across these studies, on average, only 23% of 30‐day readmissions appear to be avoidable.[8] Another way to consider the preventability of hospital readmissions is by noting that the most effective multimodal care‐transition interventions reduce readmission rates by only about 30%, and most interventions are much less effective.[26] The likely fact that only 23% to 30% of readmissions are preventable has profound implications for the anticipated results of hospital readmission reduction efforts. Interventions that are 75% effective in reducing preventable readmissions should be expected to produce only an 18% to 22% reduction in overall readmission rates.[37]

FOCUSING ON PREVENTABLE READMISSIONS

A greater focus on identifying and targeting preventable readmissions would offer a number of advantages over the present approach. First, it is more meaningful to compare hospitals based on their percentage of discharges resulting in a preventable readmission, than on the basis of highly complex risk standardization procedures for selected conditions. Second, a focus on preventable readmissions more clearly identifies and permits hospitals to target opportunities for improvement. Third, if the focus were on preventable readmissions for a large number of conditions, the necessary sample size could be obtained over a shorter period of time. Overall, such a preventable readmissions metric could serve as a more agile and undiluted performance indicator, as opposed to the present 3‐year rolling average rate of all‐cause readmissions for certain conditions, the majority of which are probably not preventable.

DEFINING PREVENTABILITY

Defining a preventable readmission is critically important. However, neither a consensus definition nor a validated standard for assessing preventable hospital readmissions exists. Different conceptual frameworks and terms (eg, avoidable, potentially preventable, or urgent readmission) complicate the issue.[38, 39, 40]

Although the CMS measure does not address preventability, it is helpful to consider whether other readmission metrics incorporate this concept. The United Health Group's (UHG, formerly Pacificare) All‐Cause Readmission Index, University HealthSystem Consortium's 30‐Day Readmission Rate (all cause), and 3M Health Information Systems' (3M) Potentially Preventable Readmissions (PPR) are 3 commonly used measures.

Of these, only the 3M PPR metric includes the concept of preventability. 3M created a proprietary matrix of 98,000 readmission‐index admission All Patient Refined Diagnosis Related Group pairs based on the review of several physicians and the logical assumption that a readmission for a clinically related diagnosis is potentially preventable.[24, 41] Readmission and index admissions are considered clinically related if any of the following occur: (1) medical readmission for continuation or recurrence of an initial, or closely related, condition; (2) medical readmission for acute decompensation of a chronic condition that was not the reason for the index admission but was plausibly related to care during or immediately afterward (eg, readmission for diabetes in a patient whose index admission was AMI); (3) medical readmission for acute complication plausibly related to care during index admission; (4) readmission for surgical procedure for continuation or recurrence of initial problem (eg, readmission for appendectomy following admission for abdominal pain and fever); or (5) readmission for surgical procedure to address complication resulting from care during index admission.[24, 41] The readmission time frame is not standardized and may be set by the user. Though conceptually appealing in some ways, CMS and the NQF have expressed concern about this specific approach because of the uncertain reliability of the relatedness of the admission‐readmission diagnosis dyads.[3]

In the research literature, only a few studies have examined the 3M PPR or other preventability assessments that rely on the relatedness of diagnostic codes.[8] Using the 3M PPR, a study showed that 78% of readmissions were classified as potentially preventable,[42] which explains why the 3M PPR and all‐cause readmission metric may correlate highly.[43] Others have demonstrated that ratings of hospital performance on readmission rates vary by a moderate to large amount, depending on whether the 3M PPR, CMS, or UHG methodology is used.[43, 44] An algorithm called SQLape[45, 46] is used in Switzerland to benchmark hospitals and defines potentially avoidable readmissions as being related to index diagnoses or complications of those conditions. It has recently been tested in the United States in a single‐center study,[47] and a multihospital study is underway.

Aside from these algorithms using related diagnosis codes, most ratings of preventability have relied on subjective assessments made primarily through a review of hospital records, and approximately one‐third also included data from clinic visits or interviews with the treating medical team or patients/families.[8] Unfortunately, these reports provide insufficient detail on how to apply their preventability criteria to subsequent readmission reviews. Studies did, however, provide categories of preventability into which readmissions could be organized (see Supporting Information, Appendix Table 1, in the online version of this article for details from a subset of studies cited in van Walraven's reviews that illustrate this point).

Assessment of preventability by clinician review can be challenging. In general, such assessments have considered readmissions resulting from factors within the hospital's control to be avoidable (eg, providing appropriate discharge instructions, reconciling medications, arranging timely postdischarge follow‐up appointments), whereas readmissions resulting from factors not within the hospital's control are unavoidable (eg, patient socioeconomic status, social support, disease progression). However, readmissions resulting from patient behaviors or social reasons could potentially be classified as avoidable or unavoidable depending on the circumstances. For example, if a patient decides not to take a prescribed antibiotic and is readmitted with worsening infection, this could be classified as an unavoidable readmission from the hospital's perspective. Alternatively, if the physician prescribing the antibiotic was inattentive to the cost of the medication and the patient would have taken a less expensive medication had it been prescribed, this could be classified as an avoidable readmission. Differing interpretations of contextual factors may partially account for the variability in clinical assessments of preventability.

Indeed, despite the lack of consensus around a standard method of defining preventability, hospitals and health systems are moving forward to address the issue and reduce readmissions. A recent survey by America's Essential Hospitals (previously the National Association of Public Hospitals and Health Systems), indicated that: (1) reducing readmissions was a high priority for the majority (86%) of members, (2) most had established interdisciplinary teams to address the issue, and (3) over half had a formal process for determining which readmissions were potentially preventable. Of the survey respondents, just over one‐third rely on staff review of individual patient charts or patient and family interviews, and slightly less than one‐third rely on other mechanisms such as external consultants, criteria developed by other entities, or the Institute for Clinical Systems Improvement methodology.[48] Approximately one‐fifth make use of 3M's PPR product, and slightly fewer use the list of the Agency for Healthcare Research and Quality's ambulatory care sensitive conditions (ACSCs). These are medical conditions for which it is believed that good outpatient care could prevent the need for hospitalization (eg, asthma, congestive heart failure, diabetes) or for which early intervention minimizes complications.[49] Hospitalization rates for ACSCs may represent a good measure of excess hospitalization, with a focus on the quality of outpatient care.

RECOMMENDATIONS

We recommend that reporting of hospital readmission rates be based on preventable or potentially preventable readmissions. Although we acknowledge the challenges in doing so, the advantages are notable. At minimum, a preventable readmission rate would more accurately reflect the true gap in care and therefore hospitals' real opportunity for improvement, without being obscured by readmissions that are not preventable.

Because readmission rates are used for public reporting and financial penalties for hospitals, we favor a measure of preventability that reflects the readmissions that the hospital or hospital system has the ability to prevent. This would not penalize hospitals for factors that are under the control of others, namely patients and caregivers, community supports, or society at large. We further recommend that this measure apply to a broader composite of unplanned care, inclusive of both inpatient and observation stays, which have little distinction in patients' eyes, and both represent potentially unnecessary utilization of acute‐care resources.[50] Such a measure would require development, validation, and appropriate vetting before it is implemented.

The first step is for researchers and policy makers to agree on how a measure of preventable or potentially preventable readmissions could be defined. A common element of preventability assessment is to identify the degree to which the reasons for readmission are related to the diagnoses of the index hospitalization. To be reliable and scalable, this measure will need to be based on algorithms that relate the index and readmission diagnoses, most likely using claims data. Choosing common medical and surgical conditions and developing a consensus‐based list of related readmission diagnoses is an important first step. It would also be important to include some less common conditions, because they may reflect very different aspects of hospital care.

An approach based on a list of related diagnoses would represent potentially preventable rehospitalizations. Generally, clinical review is required to determine actual preventability, taking into account patient factors such as a high level of illness or functional impairment that leads to clinical decompensation in spite of excellent management.[51, 52] Clinical review, like a root cause analysis, also provides greater insight into hospital processes that may warrant improvement. Therefore, even if an administrative measure of potentially preventable readmissions is implemented, hospitals may wish to continue performing detailed clinical review of some readmissions for quality improvement purposes. When clinical review becomes more standardized,[53] a combined approach that uses administrative data plus clinical verification and arbitration may be feasible, as with hospital‐acquired infections.

Similar work to develop related sets of admission and readmission diagnoses has already been undertaken in development of the 3M PPR and SQLape measures.[41, 46] However, the 3M PPR is a proprietary system that has low specificity and a high false‐positive rate for identifying preventable readmissions when compared to clinical review.[42] Moreover, neither measure has yet achieved the consensus required for widespread adoption in the United States. What is needed is a nonproprietary listing of related admission and readmission diagnoses, developed with the engagement of relevant stakeholders, that goes through a period of public comment and vetting by a body such as the NQF.

Until a validated measure of potentially preventable readmission can be developed, how could the current approach evolve toward preventability? The most feasible, rapidly implementable change would be to alter the readmission time horizon from 30 days to 7 or 15 days. A 30‐day period holds hospitals accountable for complications of outpatient care or new problems that may develop weeks after discharge. Even though this may foster shared accountability and collaboration among hospitals and outpatient or community settings, research has demonstrated that early readmissions (eg, within 715 days of discharge) are more likely preventable.[54] Second, consideration of the socioeconomic status of hospital patients, as recommended by MedPAC,[34] would improve on the current model by comparing hospitals to like facilities when determining penalties for excess readmission rates. Finally, adjustment for community factors, such as practice patterns and access to care, would enable readmission metrics to better reflect factors under the hospital's control.[32]

CONCLUSION

Holding hospitals accountable for the quality of acute and transitional care is an important policy initiative that has accelerated many improvements in discharge planning and care coordination. Optimally, the policies, public reporting, and penalties should target preventable readmissions, which may represent as little as one‐quarter of all readmissions. By summarizing some of the issues in defining preventability, we hope to foster continued refinement of quality metrics used in this arena.

Unplanned hospital readmissions are common, costly, and potentially avoidable. Approximately 20% of Medicare patients are readmitted within 30 days of discharge.[1] Readmission rates are estimated to be similarly high in other population subgroups,[2, 3, 4] with approximately 80% of patients[1, 5, 6] readmitted to the original discharging hospital. A recent systematic review suggested that 27% of readmissions may be preventable.[7]

Hospital readmissions have increasingly been viewed as a correctable marker of poor quality care and have been adopted by a number of organizations as quality indicators.[8, 9, 10] As a result, hospitals have important internal and external motivations to address readmissions. Identification of patients at high risk for readmissions may be an important first step toward preventing them. In particular, readmission risk assessment could be used to help providers target the delivery of resource‐intensive transitional care interventions[11, 12, 13, 14] to patients with the greatest needs. Such an approach is appealing because it allows hospitals to focus scarce resources where the impact may be greatest and provides a starting point for organizations struggling to develop robust models of transitional care delivery.

Electronic health records (EHRs) may prove to be an important component of strategies designed to risk stratify patients at the point of care. Algorithms integrated into the EHR that automatically generate risk predictions have the potential to (1) improve provider time efficiency by automating the prediction process, (2) improve consistency of data collection and risk score calculation, (3) increase adoption through improved usability, and (4) provide clinically important information in real‐time to all healthcare team members caring for a hospitalized patient.

We thus sought to derive a predictive model for 30‐day readmissions using data reliably present in our EHR at the time of admission, and integrate this predictive model into our hospital's EHR to create an automated prediction tool that identifies on admission patients at high risk for readmission within 30 days of discharge. In addition, we prospectively validated this model using the 12‐month period after implementation and examined the impact on readmissions.

METHODS

Implementation

An automated readmission risk flag was then integrated into the EHR. Patients classified as being at high risk for readmission with the automated prediction model were flagged in the EHR on admission (Figure 1A). The flag can be double‐clicked to display a separate screen with information relevant to discharge planning including inpatient and emergency department (ED) visits in the prior 12 months, as well as information about the primary team, length of stay, and admitting problem associated with those admissions (Figure 1B). The prediction model was integrated into our EHR using Arden Syntax for Medical Logic Modules.[17] The readmission risk screen involved presenting the provider with a new screen and was thus developed in Microsoft .NET using C# and Windows Forms (Microsoft Corp., Redmond, WA).

Figure 1

The flag was visible on the patient lists of all providers who utilized the EHR. This included but was not limited to nurses, social workers, unit pharmacists, and physicians. At the time of implementation, educational events regarding the readmission risk flag were provided in forums targeting administrators, pharmacists, social workers, and housestaff. Information about the flag and recommendations for use were distributed through emails and broadcast screensaver messages disseminated throughout the inpatient units of the health system. Providers were asked to pay special attention to discharge planning for patients triggering the readmission risk flag, including medication reconciliation by pharmacists for these patients prior to discharge, and arrangement of available home services by social workers.

The risk flag was 1 of 4 classes of interventions developed and endorsed by the health system in its efforts to reduce readmissions. Besides risk stratification, the other classes were: interdisciplinary rounding, patient education, and discharge communication. None of the interventions alone were expected to decrease readmissions, but as all 4 classes of interventions were implemented and performed routinely, the expectation was that they would work in concert to reduce readmissions.

RESULTS

Readmission Rates

The mean 30‐day all‐cause readmission rate for the 24‐month period prior to the intervention was 14.4%, whereas the mean for the 12‐month period after the implementation was 15.1%. Thirty‐day all‐cause and 7‐day unplanned monthly readmission rates do not appear to have been impacted by the intervention (Figure 2). There was no evidence for either an immediate or sustained effect (Table 2).

Figure 2

Table 2.Interrupted Time Series of Readmission Rates

Hospital	Preimplementation Period			Immediate Effect		Postimplementation Period			P Value Change in Trenda
	Monthly % Change in Readmission Rates		P Value	Immediate % Change	P Value	Monthly % Change in Readmission Rates		P Value
NOTE: Regression coefficients represent the absolute change in the monthly readmission rate (percentage) per unit time (month). Models are adjusted for autocorrelation using the Cochrane‐Orcutt estimator. a P value compares the pre‐ and postimplementation trends in readmission rates.
30‐Day All‐Cause Readmission Rates
Hosp A	0.023	Stable	0.153	0.480	0.991	0.100	Increasing	0.044	0.134
Hosp B	0.061	Increasing	0.002	0.492	0.125	0.060	Stable	0.296	0.048
Hosp C	0.026	Stable	0.413	0.447	0.585	0.046	Stable	0.629	0.476
Health System	0.032	Increasing	0.014	0.344	0.302	0.026	Stable	0.499	0.881
7‐Day Unplanned Readmission Rates
Hosp A	0.004	Stable	0.642	0.271	0.417	0.005	Stable	0.891	0.967
Hosp B	0.012	Stable	0.201	0.298	0.489	0.038	Stable	0.429	0.602
Hosp C	0.008	Stable	0.213	0.353	0.204	0.004	Stable	0.895	0.899
Health System	0.005	Stable	0.358	0.003	0.990	0.010	Stable	0.712	0.583

DISCUSSION

In this proof‐of‐concept study, we demonstrated the feasibility of an automated readmission risk prediction model integrated into a health system's EHR for a mixed population of hospitalized medical and surgical patients. To our knowledge, this is the first study in a general population of hospitalized patients to examine the impact of providing readmission risk assessment on readmission rates. We used a simple prediction model potentially generalizable to EHRs and healthcare populations beyond our own.

Existing risk prediction models for hospital readmission have important limitations and are difficult to implement in clinical practice.[22] Prediction models for hospital readmission are often dependent on retrospective claims data, developed for specific patient populations, and not designed for use early in the course of hospitalization when transitional care interventions can be initiated.[22] In addition, the time required to gather the necessary data and calculate the risk score remains a barrier to the adoption of prediction models in practice. By automating the process of readmission risk prediction, we were able to help integrate risk assessment into the healthcare process across many providers in a large multihospital healthcare organization. This has allowed us to consistently share risk assessment in real time with all members of the inpatient team, facilitating a team‐based approach to discharge planning.[23]

Two prior studies have developed readmission risk prediction models designed to be implemented into the EHR. Amarasingham et al.[24] developed and implemented[25] a heart failure‐specific prediction model based on the 18‐item Tabak mortality score.[26] Bradley et al.[27] studied in a broader population of medicine and surgery patients the predictive ability of a 26‐item score that utilized vital sign, cardiac rhythm, and nursing assessment data. Although EHRs are developing rapidly, currently the majority of EHRs do not support the use of many of the variables used in these models. In addition, both models were complex, raising concerns about generalizability to other healthcare settings and populations.

A distinctive characteristic of our model is its simplicity. We were cognizant of the realities of running a prediction model in a high‐volume production environment and the diminishing returns of adding more variables. We thus favored simplicity at all stages of model development, with the associated belief that complexity could be added with future iterations once feasibility had been established. Finally, we were aware that we were constructing a medical decision support tool rather than a simple classifier.[26] As such, the optimal model was not purely driven by discriminative ability, but also by our subjective assessment of the optimal trade‐off between sensitivity and specificity (the test‐treatment threshold) for such a model.[26] To facilitate model assessment, we thus categorized the potential predictor variables and evaluated the test characteristics of each combination of categorized variables. Although the C statistic of a model using continuous variables will be higher than a model using categorical values, model performance at the chosen trade‐off point is unlikely to be different.

Although the overall predictive ability of our model was fair, we found that it was associated with clinically meaningful differences in readmission rates between those triggering and not triggering the flag. The 30‐day all‐cause readmission rate in the 12‐month prospective sample was 15.1%, yet among those flagged as being at high risk for readmission the readmission rate was 30.4%. Given resource constraints and the need to selectively apply potentially costly care transition interventions, this may in practice translate into a meaningful discriminative ability.

Readmission rates did not change significantly during the study period. A number of plausible reasons for this exist, including: (1) the current model may not exhibit sufficient predictive ability to classify those at high risk or impact the behavior of providers appropriately, (2) those patients classified as high risk of readmission may not be at high risk of readmissions that are preventable, (3) information provided by the model may not yet routinely be used such that it can affect care, or (4) providing readmission risk assessment alone is not sufficient to influence readmission rates, and the other interventions or organizational changes necessary to impact care of those defined as high risk have not yet been implemented or are not yet being performed routinely. If the primary reasons for our results are those outlined in numbers 3 or 4, then readmission rates should improve over time as the risk flag becomes more routinely used, and those interventions necessary to impact readmission rates of those defined as high risk are implemented and performed.

CONCLUSION

An automated prediction model was effectively integrated into an existing EHR and was able to identify patients on admission who are at risk for readmission within 30 days of discharge. Future work will aim to further examine the impact of the flag on readmission rates, further refine the prediction model, and gather data on how providers and care teams use the information provided by the flag.

Unplanned hospital readmissions are common, costly, and potentially avoidable. Approximately 20% of Medicare patients are readmitted within 30 days of discharge.[1] Readmission rates are estimated to be similarly high in other population subgroups,[2, 3, 4] with approximately 80% of patients[1, 5, 6] readmitted to the original discharging hospital. A recent systematic review suggested that 27% of readmissions may be preventable.[7]

Hospital readmissions have increasingly been viewed as a correctable marker of poor quality care and have been adopted by a number of organizations as quality indicators.[8, 9, 10] As a result, hospitals have important internal and external motivations to address readmissions. Identification of patients at high risk for readmissions may be an important first step toward preventing them. In particular, readmission risk assessment could be used to help providers target the delivery of resource‐intensive transitional care interventions[11, 12, 13, 14] to patients with the greatest needs. Such an approach is appealing because it allows hospitals to focus scarce resources where the impact may be greatest and provides a starting point for organizations struggling to develop robust models of transitional care delivery.

Electronic health records (EHRs) may prove to be an important component of strategies designed to risk stratify patients at the point of care. Algorithms integrated into the EHR that automatically generate risk predictions have the potential to (1) improve provider time efficiency by automating the prediction process, (2) improve consistency of data collection and risk score calculation, (3) increase adoption through improved usability, and (4) provide clinically important information in real‐time to all healthcare team members caring for a hospitalized patient.

We thus sought to derive a predictive model for 30‐day readmissions using data reliably present in our EHR at the time of admission, and integrate this predictive model into our hospital's EHR to create an automated prediction tool that identifies on admission patients at high risk for readmission within 30 days of discharge. In addition, we prospectively validated this model using the 12‐month period after implementation and examined the impact on readmissions.

METHODS

Implementation

An automated readmission risk flag was then integrated into the EHR. Patients classified as being at high risk for readmission with the automated prediction model were flagged in the EHR on admission (Figure 1A). The flag can be double‐clicked to display a separate screen with information relevant to discharge planning including inpatient and emergency department (ED) visits in the prior 12 months, as well as information about the primary team, length of stay, and admitting problem associated with those admissions (Figure 1B). The prediction model was integrated into our EHR using Arden Syntax for Medical Logic Modules.[17] The readmission risk screen involved presenting the provider with a new screen and was thus developed in Microsoft .NET using C# and Windows Forms (Microsoft Corp., Redmond, WA).

Figure 1

The flag was visible on the patient lists of all providers who utilized the EHR. This included but was not limited to nurses, social workers, unit pharmacists, and physicians. At the time of implementation, educational events regarding the readmission risk flag were provided in forums targeting administrators, pharmacists, social workers, and housestaff. Information about the flag and recommendations for use were distributed through emails and broadcast screensaver messages disseminated throughout the inpatient units of the health system. Providers were asked to pay special attention to discharge planning for patients triggering the readmission risk flag, including medication reconciliation by pharmacists for these patients prior to discharge, and arrangement of available home services by social workers.

The risk flag was 1 of 4 classes of interventions developed and endorsed by the health system in its efforts to reduce readmissions. Besides risk stratification, the other classes were: interdisciplinary rounding, patient education, and discharge communication. None of the interventions alone were expected to decrease readmissions, but as all 4 classes of interventions were implemented and performed routinely, the expectation was that they would work in concert to reduce readmissions.

RESULTS

Readmission Rates

The mean 30‐day all‐cause readmission rate for the 24‐month period prior to the intervention was 14.4%, whereas the mean for the 12‐month period after the implementation was 15.1%. Thirty‐day all‐cause and 7‐day unplanned monthly readmission rates do not appear to have been impacted by the intervention (Figure 2). There was no evidence for either an immediate or sustained effect (Table 2).

Figure 2

Table 2.Interrupted Time Series of Readmission Rates

Hospital	Preimplementation Period			Immediate Effect		Postimplementation Period			P Value Change in Trenda
	Monthly % Change in Readmission Rates		P Value	Immediate % Change	P Value	Monthly % Change in Readmission Rates		P Value
NOTE: Regression coefficients represent the absolute change in the monthly readmission rate (percentage) per unit time (month). Models are adjusted for autocorrelation using the Cochrane‐Orcutt estimator. a P value compares the pre‐ and postimplementation trends in readmission rates.
30‐Day All‐Cause Readmission Rates
Hosp A	0.023	Stable	0.153	0.480	0.991	0.100	Increasing	0.044	0.134
Hosp B	0.061	Increasing	0.002	0.492	0.125	0.060	Stable	0.296	0.048
Hosp C	0.026	Stable	0.413	0.447	0.585	0.046	Stable	0.629	0.476
Health System	0.032	Increasing	0.014	0.344	0.302	0.026	Stable	0.499	0.881
7‐Day Unplanned Readmission Rates
Hosp A	0.004	Stable	0.642	0.271	0.417	0.005	Stable	0.891	0.967
Hosp B	0.012	Stable	0.201	0.298	0.489	0.038	Stable	0.429	0.602
Hosp C	0.008	Stable	0.213	0.353	0.204	0.004	Stable	0.895	0.899
Health System	0.005	Stable	0.358	0.003	0.990	0.010	Stable	0.712	0.583

DISCUSSION

In this proof‐of‐concept study, we demonstrated the feasibility of an automated readmission risk prediction model integrated into a health system's EHR for a mixed population of hospitalized medical and surgical patients. To our knowledge, this is the first study in a general population of hospitalized patients to examine the impact of providing readmission risk assessment on readmission rates. We used a simple prediction model potentially generalizable to EHRs and healthcare populations beyond our own.

Existing risk prediction models for hospital readmission have important limitations and are difficult to implement in clinical practice.[22] Prediction models for hospital readmission are often dependent on retrospective claims data, developed for specific patient populations, and not designed for use early in the course of hospitalization when transitional care interventions can be initiated.[22] In addition, the time required to gather the necessary data and calculate the risk score remains a barrier to the adoption of prediction models in practice. By automating the process of readmission risk prediction, we were able to help integrate risk assessment into the healthcare process across many providers in a large multihospital healthcare organization. This has allowed us to consistently share risk assessment in real time with all members of the inpatient team, facilitating a team‐based approach to discharge planning.[23]

Two prior studies have developed readmission risk prediction models designed to be implemented into the EHR. Amarasingham et al.[24] developed and implemented[25] a heart failure‐specific prediction model based on the 18‐item Tabak mortality score.[26] Bradley et al.[27] studied in a broader population of medicine and surgery patients the predictive ability of a 26‐item score that utilized vital sign, cardiac rhythm, and nursing assessment data. Although EHRs are developing rapidly, currently the majority of EHRs do not support the use of many of the variables used in these models. In addition, both models were complex, raising concerns about generalizability to other healthcare settings and populations.

A distinctive characteristic of our model is its simplicity. We were cognizant of the realities of running a prediction model in a high‐volume production environment and the diminishing returns of adding more variables. We thus favored simplicity at all stages of model development, with the associated belief that complexity could be added with future iterations once feasibility had been established. Finally, we were aware that we were constructing a medical decision support tool rather than a simple classifier.[26] As such, the optimal model was not purely driven by discriminative ability, but also by our subjective assessment of the optimal trade‐off between sensitivity and specificity (the test‐treatment threshold) for such a model.[26] To facilitate model assessment, we thus categorized the potential predictor variables and evaluated the test characteristics of each combination of categorized variables. Although the C statistic of a model using continuous variables will be higher than a model using categorical values, model performance at the chosen trade‐off point is unlikely to be different.

Although the overall predictive ability of our model was fair, we found that it was associated with clinically meaningful differences in readmission rates between those triggering and not triggering the flag. The 30‐day all‐cause readmission rate in the 12‐month prospective sample was 15.1%, yet among those flagged as being at high risk for readmission the readmission rate was 30.4%. Given resource constraints and the need to selectively apply potentially costly care transition interventions, this may in practice translate into a meaningful discriminative ability.

Readmission rates did not change significantly during the study period. A number of plausible reasons for this exist, including: (1) the current model may not exhibit sufficient predictive ability to classify those at high risk or impact the behavior of providers appropriately, (2) those patients classified as high risk of readmission may not be at high risk of readmissions that are preventable, (3) information provided by the model may not yet routinely be used such that it can affect care, or (4) providing readmission risk assessment alone is not sufficient to influence readmission rates, and the other interventions or organizational changes necessary to impact care of those defined as high risk have not yet been implemented or are not yet being performed routinely. If the primary reasons for our results are those outlined in numbers 3 or 4, then readmission rates should improve over time as the risk flag becomes more routinely used, and those interventions necessary to impact readmission rates of those defined as high risk are implemented and performed.

CONCLUSION

An automated prediction model was effectively integrated into an existing EHR and was able to identify patients on admission who are at risk for readmission within 30 days of discharge. Future work will aim to further examine the impact of the flag on readmission rates, further refine the prediction model, and gather data on how providers and care teams use the information provided by the flag.

Unplanned hospital readmissions are common, costly, and potentially avoidable. Approximately 20% of Medicare patients are readmitted within 30 days of discharge.[1] Readmission rates are estimated to be similarly high in other population subgroups,[2, 3, 4] with approximately 80% of patients[1, 5, 6] readmitted to the original discharging hospital. A recent systematic review suggested that 27% of readmissions may be preventable.[7]

Hospital readmissions have increasingly been viewed as a correctable marker of poor quality care and have been adopted by a number of organizations as quality indicators.[8, 9, 10] As a result, hospitals have important internal and external motivations to address readmissions. Identification of patients at high risk for readmissions may be an important first step toward preventing them. In particular, readmission risk assessment could be used to help providers target the delivery of resource‐intensive transitional care interventions[11, 12, 13, 14] to patients with the greatest needs. Such an approach is appealing because it allows hospitals to focus scarce resources where the impact may be greatest and provides a starting point for organizations struggling to develop robust models of transitional care delivery.

Electronic health records (EHRs) may prove to be an important component of strategies designed to risk stratify patients at the point of care. Algorithms integrated into the EHR that automatically generate risk predictions have the potential to (1) improve provider time efficiency by automating the prediction process, (2) improve consistency of data collection and risk score calculation, (3) increase adoption through improved usability, and (4) provide clinically important information in real‐time to all healthcare team members caring for a hospitalized patient.

We thus sought to derive a predictive model for 30‐day readmissions using data reliably present in our EHR at the time of admission, and integrate this predictive model into our hospital's EHR to create an automated prediction tool that identifies on admission patients at high risk for readmission within 30 days of discharge. In addition, we prospectively validated this model using the 12‐month period after implementation and examined the impact on readmissions.

METHODS

Implementation

An automated readmission risk flag was then integrated into the EHR. Patients classified as being at high risk for readmission with the automated prediction model were flagged in the EHR on admission (Figure 1A). The flag can be double‐clicked to display a separate screen with information relevant to discharge planning including inpatient and emergency department (ED) visits in the prior 12 months, as well as information about the primary team, length of stay, and admitting problem associated with those admissions (Figure 1B). The prediction model was integrated into our EHR using Arden Syntax for Medical Logic Modules.[17] The readmission risk screen involved presenting the provider with a new screen and was thus developed in Microsoft .NET using C# and Windows Forms (Microsoft Corp., Redmond, WA).

Figure 1

The flag was visible on the patient lists of all providers who utilized the EHR. This included but was not limited to nurses, social workers, unit pharmacists, and physicians. At the time of implementation, educational events regarding the readmission risk flag were provided in forums targeting administrators, pharmacists, social workers, and housestaff. Information about the flag and recommendations for use were distributed through emails and broadcast screensaver messages disseminated throughout the inpatient units of the health system. Providers were asked to pay special attention to discharge planning for patients triggering the readmission risk flag, including medication reconciliation by pharmacists for these patients prior to discharge, and arrangement of available home services by social workers.

The risk flag was 1 of 4 classes of interventions developed and endorsed by the health system in its efforts to reduce readmissions. Besides risk stratification, the other classes were: interdisciplinary rounding, patient education, and discharge communication. None of the interventions alone were expected to decrease readmissions, but as all 4 classes of interventions were implemented and performed routinely, the expectation was that they would work in concert to reduce readmissions.

RESULTS

Readmission Rates

The mean 30‐day all‐cause readmission rate for the 24‐month period prior to the intervention was 14.4%, whereas the mean for the 12‐month period after the implementation was 15.1%. Thirty‐day all‐cause and 7‐day unplanned monthly readmission rates do not appear to have been impacted by the intervention (Figure 2). There was no evidence for either an immediate or sustained effect (Table 2).

Figure 2

Table 2.Interrupted Time Series of Readmission Rates

Hospital	Preimplementation Period			Immediate Effect		Postimplementation Period			P Value Change in Trenda
	Monthly % Change in Readmission Rates		P Value	Immediate % Change	P Value	Monthly % Change in Readmission Rates		P Value
NOTE: Regression coefficients represent the absolute change in the monthly readmission rate (percentage) per unit time (month). Models are adjusted for autocorrelation using the Cochrane‐Orcutt estimator. a P value compares the pre‐ and postimplementation trends in readmission rates.
30‐Day All‐Cause Readmission Rates
Hosp A	0.023	Stable	0.153	0.480	0.991	0.100	Increasing	0.044	0.134
Hosp B	0.061	Increasing	0.002	0.492	0.125	0.060	Stable	0.296	0.048
Hosp C	0.026	Stable	0.413	0.447	0.585	0.046	Stable	0.629	0.476
Health System	0.032	Increasing	0.014	0.344	0.302	0.026	Stable	0.499	0.881
7‐Day Unplanned Readmission Rates
Hosp A	0.004	Stable	0.642	0.271	0.417	0.005	Stable	0.891	0.967
Hosp B	0.012	Stable	0.201	0.298	0.489	0.038	Stable	0.429	0.602
Hosp C	0.008	Stable	0.213	0.353	0.204	0.004	Stable	0.895	0.899
Health System	0.005	Stable	0.358	0.003	0.990	0.010	Stable	0.712	0.583

DISCUSSION

In this proof‐of‐concept study, we demonstrated the feasibility of an automated readmission risk prediction model integrated into a health system's EHR for a mixed population of hospitalized medical and surgical patients. To our knowledge, this is the first study in a general population of hospitalized patients to examine the impact of providing readmission risk assessment on readmission rates. We used a simple prediction model potentially generalizable to EHRs and healthcare populations beyond our own.

Existing risk prediction models for hospital readmission have important limitations and are difficult to implement in clinical practice.[22] Prediction models for hospital readmission are often dependent on retrospective claims data, developed for specific patient populations, and not designed for use early in the course of hospitalization when transitional care interventions can be initiated.[22] In addition, the time required to gather the necessary data and calculate the risk score remains a barrier to the adoption of prediction models in practice. By automating the process of readmission risk prediction, we were able to help integrate risk assessment into the healthcare process across many providers in a large multihospital healthcare organization. This has allowed us to consistently share risk assessment in real time with all members of the inpatient team, facilitating a team‐based approach to discharge planning.[23]

Two prior studies have developed readmission risk prediction models designed to be implemented into the EHR. Amarasingham et al.[24] developed and implemented[25] a heart failure‐specific prediction model based on the 18‐item Tabak mortality score.[26] Bradley et al.[27] studied in a broader population of medicine and surgery patients the predictive ability of a 26‐item score that utilized vital sign, cardiac rhythm, and nursing assessment data. Although EHRs are developing rapidly, currently the majority of EHRs do not support the use of many of the variables used in these models. In addition, both models were complex, raising concerns about generalizability to other healthcare settings and populations.

A distinctive characteristic of our model is its simplicity. We were cognizant of the realities of running a prediction model in a high‐volume production environment and the diminishing returns of adding more variables. We thus favored simplicity at all stages of model development, with the associated belief that complexity could be added with future iterations once feasibility had been established. Finally, we were aware that we were constructing a medical decision support tool rather than a simple classifier.[26] As such, the optimal model was not purely driven by discriminative ability, but also by our subjective assessment of the optimal trade‐off between sensitivity and specificity (the test‐treatment threshold) for such a model.[26] To facilitate model assessment, we thus categorized the potential predictor variables and evaluated the test characteristics of each combination of categorized variables. Although the C statistic of a model using continuous variables will be higher than a model using categorical values, model performance at the chosen trade‐off point is unlikely to be different.

Although the overall predictive ability of our model was fair, we found that it was associated with clinically meaningful differences in readmission rates between those triggering and not triggering the flag. The 30‐day all‐cause readmission rate in the 12‐month prospective sample was 15.1%, yet among those flagged as being at high risk for readmission the readmission rate was 30.4%. Given resource constraints and the need to selectively apply potentially costly care transition interventions, this may in practice translate into a meaningful discriminative ability.

Readmission rates did not change significantly during the study period. A number of plausible reasons for this exist, including: (1) the current model may not exhibit sufficient predictive ability to classify those at high risk or impact the behavior of providers appropriately, (2) those patients classified as high risk of readmission may not be at high risk of readmissions that are preventable, (3) information provided by the model may not yet routinely be used such that it can affect care, or (4) providing readmission risk assessment alone is not sufficient to influence readmission rates, and the other interventions or organizational changes necessary to impact care of those defined as high risk have not yet been implemented or are not yet being performed routinely. If the primary reasons for our results are those outlined in numbers 3 or 4, then readmission rates should improve over time as the risk flag becomes more routinely used, and those interventions necessary to impact readmission rates of those defined as high risk are implemented and performed.

CONCLUSION

An automated prediction model was effectively integrated into an existing EHR and was able to identify patients on admission who are at risk for readmission within 30 days of discharge. Future work will aim to further examine the impact of the flag on readmission rates, further refine the prediction model, and gather data on how providers and care teams use the information provided by the flag.