Amy Deutschendorf, MS, RN

Centers for Medicare and Medicaid Services (CMS)–promulgated quality metrics continue to attract critics. Physicians decry that many metrics are outside their control, while patient groups are frustrated that metrics lack meaning for beneficiaries. The Hospital Readmissions Reduction Program (HRRP) reduces payments for “excess” 30-day risk-standardized readmissions for six conditions and procedures, and may be less effective in reducing readmissions than previously reported due to intentional and increasing use of hospital observation stays.¹

In this issue, Sheehy et al² report that nearly one in five rehospitalizations were unrecognized because either the index hospitalization or the rehospitalization was an observation stay, highlighting yet another challenge with the HRRP. Limitations of their study include the use of a single year of claims data and the exclusion of Medicare Advantage claims data, as one might expect lower readmission rates in this capitated program. Opportunities for improving the HRRP could consist of updating the HRRP metric to include observation stays and, for surgical hospitalizations, extended-stay surgical recovery, wherein patients may be observed for up to 2 days following a procedure. Unfortunately, despite the HRRP missing nearly one in five readmissions, CMS would likely need additional statutory authority from Congress in order to reinterpret the definition of readmission³ to include observation stays.

Challenges with the HRRP metrics raise broader concerns about the program. For decades, administrators viewed readmissions as a utilization metric, only to have the Affordable Care Act re-designate and define all-cause readmissions as a quality metric. Yet hospitals and health systems control only some factors driving readmission. Readmissions occur for a variety of reasons, including not only poor quality of initial hospital care and inadequate care coordination, but also factors that are beyond the hospital’s purview, such as lack of access to ambulatory services, multiple and severe chronic conditions that progress or remain unresponsive to intervention,⁴ and demographic and social factors such as housing instability, health literacy, or residence in a food desert. These non-hospital factors reside within the domain of other market participants or local, state, and federal government agencies.

Challenges to the utility, validity, and appropriateness of HRRP metrics should remind policymakers of the dangers of over-legislating the details of healthcare policy and the statutory inflexibility that can ensue. Clinical care evolves, and artificial constructs—including payment categories such as observation status—may age poorly over time, exemplified best by the challenges of accessing post-acute care due to the 3-day rule.⁵ Introduced as a statutory requirement in 1967, when the average length of stay was 13.8 days and observation care did not exist as a payment category, the 3-day rule requires Medicare beneficiaries to spend 3 days admitted to the hospital in order to qualify for coverage of post-acute care, creating care gaps for observation stay patients.

Observation care itself is an artificial construct of CMS payment policy. In the Medicare program, observation care falls under Part B, exposing patients to both greater financial responsibility and billing complexity through the engagement of their supplemental insurance, even though those receiving observation care experience the same care as if hospitalized— routine monitoring, nursing care, blood draws, imaging, and diagnostic tests. While CMS requires notification of observation status and explanation of the difference in patient financial responsibility, in clinical practice, patient understanding is limited. Policymakers can support both Medicare beneficiaries and hospitals by reexamining observation care as a payment category.

Sheehy and colleagues’ work simultaneously challenges the face validity of the HRRP and the reasonableness of categorizing some inpatient stays as outpatient care in the hospital—issues that policymakers can and should address.

Centers for Medicare and Medicaid Services (CMS)–promulgated quality metrics continue to attract critics. Physicians decry that many metrics are outside their control, while patient groups are frustrated that metrics lack meaning for beneficiaries. The Hospital Readmissions Reduction Program (HRRP) reduces payments for “excess” 30-day risk-standardized readmissions for six conditions and procedures, and may be less effective in reducing readmissions than previously reported due to intentional and increasing use of hospital observation stays.¹

In this issue, Sheehy et al² report that nearly one in five rehospitalizations were unrecognized because either the index hospitalization or the rehospitalization was an observation stay, highlighting yet another challenge with the HRRP. Limitations of their study include the use of a single year of claims data and the exclusion of Medicare Advantage claims data, as one might expect lower readmission rates in this capitated program. Opportunities for improving the HRRP could consist of updating the HRRP metric to include observation stays and, for surgical hospitalizations, extended-stay surgical recovery, wherein patients may be observed for up to 2 days following a procedure. Unfortunately, despite the HRRP missing nearly one in five readmissions, CMS would likely need additional statutory authority from Congress in order to reinterpret the definition of readmission³ to include observation stays.

Challenges with the HRRP metrics raise broader concerns about the program. For decades, administrators viewed readmissions as a utilization metric, only to have the Affordable Care Act re-designate and define all-cause readmissions as a quality metric. Yet hospitals and health systems control only some factors driving readmission. Readmissions occur for a variety of reasons, including not only poor quality of initial hospital care and inadequate care coordination, but also factors that are beyond the hospital’s purview, such as lack of access to ambulatory services, multiple and severe chronic conditions that progress or remain unresponsive to intervention,⁴ and demographic and social factors such as housing instability, health literacy, or residence in a food desert. These non-hospital factors reside within the domain of other market participants or local, state, and federal government agencies.

Challenges to the utility, validity, and appropriateness of HRRP metrics should remind policymakers of the dangers of over-legislating the details of healthcare policy and the statutory inflexibility that can ensue. Clinical care evolves, and artificial constructs—including payment categories such as observation status—may age poorly over time, exemplified best by the challenges of accessing post-acute care due to the 3-day rule.⁵ Introduced as a statutory requirement in 1967, when the average length of stay was 13.8 days and observation care did not exist as a payment category, the 3-day rule requires Medicare beneficiaries to spend 3 days admitted to the hospital in order to qualify for coverage of post-acute care, creating care gaps for observation stay patients.

Observation care itself is an artificial construct of CMS payment policy. In the Medicare program, observation care falls under Part B, exposing patients to both greater financial responsibility and billing complexity through the engagement of their supplemental insurance, even though those receiving observation care experience the same care as if hospitalized— routine monitoring, nursing care, blood draws, imaging, and diagnostic tests. While CMS requires notification of observation status and explanation of the difference in patient financial responsibility, in clinical practice, patient understanding is limited. Policymakers can support both Medicare beneficiaries and hospitals by reexamining observation care as a payment category.

Sheehy and colleagues’ work simultaneously challenges the face validity of the HRRP and the reasonableness of categorizing some inpatient stays as outpatient care in the hospital—issues that policymakers can and should address.

Centers for Medicare and Medicaid Services (CMS)–promulgated quality metrics continue to attract critics. Physicians decry that many metrics are outside their control, while patient groups are frustrated that metrics lack meaning for beneficiaries. The Hospital Readmissions Reduction Program (HRRP) reduces payments for “excess” 30-day risk-standardized readmissions for six conditions and procedures, and may be less effective in reducing readmissions than previously reported due to intentional and increasing use of hospital observation stays.¹

In this issue, Sheehy et al² report that nearly one in five rehospitalizations were unrecognized because either the index hospitalization or the rehospitalization was an observation stay, highlighting yet another challenge with the HRRP. Limitations of their study include the use of a single year of claims data and the exclusion of Medicare Advantage claims data, as one might expect lower readmission rates in this capitated program. Opportunities for improving the HRRP could consist of updating the HRRP metric to include observation stays and, for surgical hospitalizations, extended-stay surgical recovery, wherein patients may be observed for up to 2 days following a procedure. Unfortunately, despite the HRRP missing nearly one in five readmissions, CMS would likely need additional statutory authority from Congress in order to reinterpret the definition of readmission³ to include observation stays.

Challenges with the HRRP metrics raise broader concerns about the program. For decades, administrators viewed readmissions as a utilization metric, only to have the Affordable Care Act re-designate and define all-cause readmissions as a quality metric. Yet hospitals and health systems control only some factors driving readmission. Readmissions occur for a variety of reasons, including not only poor quality of initial hospital care and inadequate care coordination, but also factors that are beyond the hospital’s purview, such as lack of access to ambulatory services, multiple and severe chronic conditions that progress or remain unresponsive to intervention,⁴ and demographic and social factors such as housing instability, health literacy, or residence in a food desert. These non-hospital factors reside within the domain of other market participants or local, state, and federal government agencies.

Challenges to the utility, validity, and appropriateness of HRRP metrics should remind policymakers of the dangers of over-legislating the details of healthcare policy and the statutory inflexibility that can ensue. Clinical care evolves, and artificial constructs—including payment categories such as observation status—may age poorly over time, exemplified best by the challenges of accessing post-acute care due to the 3-day rule.⁵ Introduced as a statutory requirement in 1967, when the average length of stay was 13.8 days and observation care did not exist as a payment category, the 3-day rule requires Medicare beneficiaries to spend 3 days admitted to the hospital in order to qualify for coverage of post-acute care, creating care gaps for observation stay patients.

Observation care itself is an artificial construct of CMS payment policy. In the Medicare program, observation care falls under Part B, exposing patients to both greater financial responsibility and billing complexity through the engagement of their supplemental insurance, even though those receiving observation care experience the same care as if hospitalized— routine monitoring, nursing care, blood draws, imaging, and diagnostic tests. While CMS requires notification of observation status and explanation of the difference in patient financial responsibility, in clinical practice, patient understanding is limited. Policymakers can support both Medicare beneficiaries and hospitals by reexamining observation care as a payment category.

Sheehy and colleagues’ work simultaneously challenges the face validity of the HRRP and the reasonableness of categorizing some inpatient stays as outpatient care in the hospital—issues that policymakers can and should address.

Still in its infancy, the Hospital Compare overall hospital quality star rating program introduced by the Centers for Medicare & Medicaid Services (CMS) has generated intense industry debate. Individual health systems are microcosms of the challenges of ratings and measurement design. Sibley Memorial Hospital, a member of Johns Hopkins Medicine, is a well-run, 288-bed, community hospital located in a wealthy section of northwest District of Columbia with a five-star rating. In contrast, its academic partner, the Johns Hopkins Hospital, a 1,162-bed hospital with a century-long history of innovation situated in an impoverished Baltimore, Maryland, neighborhood, received a three-star rating.

Hospital ratings are the product of an industry in transition: As care delivery has shifted from an individual provider-driven industry to an increasingly scaled systems enterprise, policymakers implemented regulatory standards targeting quality measurement. Subsequent to the National Academy of Medicine’s 1999 report To Err is Human, policy efforts brought public reporting of quality ratings to multiple market segments, including dialysis facilities (2001), nursing homes (2003), Medicare Advantage plans (2007), and physicians (2015). The hospital industry was no exception, and in 2016—with much controversy¹—CMS launched the hospital star ratings program.

CMS Star Ratings for hospitals are based on seven measure groups: mortality, safety, readmission, patient experience, effectiveness, timeliness, and efficient use of medical imaging. Both industry and researchers have decried the challenges of star ratings, noting that hospitals with a narrower scope of services are more likely to receive higher ratings.² Measure groupings may be further flawed as shown by recent work demonstrating that larger, safety net, or academic hospitals, as well as hospitals offering transplant services, have higher readmission rates,³ which may be caused by differences in patient complexity. Other research has demonstrated that overall quality ratings inappropriately pool all hospitals together, when it may be fairer to initially categorize hospitals and then score them.⁴

It is within this maelstrom of debate that, in this month’s issue of the Journal of Hospital Medicine, Shi and colleagues explore the relationship between hospital star ratings and the socioeconomic features of the surrounding communities.⁵ Conducting their analysis by linking multiple reputable government and industry sources, Shi and colleagues found that counties with higher education attainment and a lower proportion of dual Medicare-Medicaid–eligible populations had higher hospital star ratings. Furthermore, a county’s minority population percentage negatively correlated with hospital ratings. Validating the experience of many rural hospital executives—who frequently experience financial challenges—Shi and colleagues noted that rural hospitals were less likely to receive five-star ratings.

Do these findings reflect a true disparity and lack of access to high-quality hospitals, or are they artifactual—secondary to a flawed construct of hospital quality measurement? Many lower-ranking hospitals are urban academic centers frequently providing services not offered at their five-star community counterparts, such as neurosurgery, comprehensive cancer care, and organ transplants, while simultaneously serving as safety net hospitals, research institutions, trauma centers, and national referral centers.

Sociodemographics factor significantly in self-care management for hospital aftercare. Health literacy, access to primary and behavioral healthcare, and transportation all affect star indicators. Recent work⁶ demonstrated that comprehensive investments in transitional care strategies and the social determinants of health were ineffective at reducing readmissions, which suggests that high readmission rates for hospitals in impoverished areas are not only common, but also may not accurately reflect hospital quality and local investment.

Patient experience is also complicating, with research demonstrating that patient perceptions vary significantly by education, age, primary language, ethnicity, and overall health. For example, one-third of average-ranked hospitals would have rankings vary by at least 18 percentile points when evaluated by Spanish-speaking patients. Star ratings fail to capture and communicate this granularity.⁷

More concerning is that star ratings inherently assume that hospital performance is being compared across the same tasks, regardless of patient characteristics, local resources, or the scope of services provided, the latter of which may vary between hospitals. For example, communication may differ in both complexity and time intensity: Explaining an antibiotic to the uncomplicated patient with pneumonia differs from prescribing an antibiotic to a patient who is legally blind from optic neuritis, walks with a cane because of multiple sclerosis, and has 24 other prescription medications. Similar challenges exist for differences in local neighborhood resources and for facilities with differing service scope.

Although one strategy to handle these “disparities” in star ratings might be to risk-adjust for social determinants of health, patients may be better served by first rethinking how star ratings are constructed. Clustering hospitals by scope of services provided and geographic region prior to determining star ratings would provide consumers with meaningful information by helping patients compare and make choices among either local or regional hospitals; national quality rankings are unhelpful for patients.

Arguably one of the most complex and person-dependent service enterprises, care delivery presents unique challenges for evaluation of customer experience and medical quality. Hospital star ratings are no exception: We must rethink their construction so they can be more meaningful for both patients and physicians.

The authors would like to acknowledge Daniel J Brotman, MD, for his editorial advice and input.

Dr Miller reported consulting for the Federal Trade Commission and serving as a member of the Centers for Medicare & Medicaid Services Medicare Evidence Development Coverage Advisory Committee. Drs Siddiqui and Deutschendorf have nothing to disclose.

Still in its infancy, the Hospital Compare overall hospital quality star rating program introduced by the Centers for Medicare & Medicaid Services (CMS) has generated intense industry debate. Individual health systems are microcosms of the challenges of ratings and measurement design. Sibley Memorial Hospital, a member of Johns Hopkins Medicine, is a well-run, 288-bed, community hospital located in a wealthy section of northwest District of Columbia with a five-star rating. In contrast, its academic partner, the Johns Hopkins Hospital, a 1,162-bed hospital with a century-long history of innovation situated in an impoverished Baltimore, Maryland, neighborhood, received a three-star rating.

Hospital ratings are the product of an industry in transition: As care delivery has shifted from an individual provider-driven industry to an increasingly scaled systems enterprise, policymakers implemented regulatory standards targeting quality measurement. Subsequent to the National Academy of Medicine’s 1999 report To Err is Human, policy efforts brought public reporting of quality ratings to multiple market segments, including dialysis facilities (2001), nursing homes (2003), Medicare Advantage plans (2007), and physicians (2015). The hospital industry was no exception, and in 2016—with much controversy¹—CMS launched the hospital star ratings program.

CMS Star Ratings for hospitals are based on seven measure groups: mortality, safety, readmission, patient experience, effectiveness, timeliness, and efficient use of medical imaging. Both industry and researchers have decried the challenges of star ratings, noting that hospitals with a narrower scope of services are more likely to receive higher ratings.² Measure groupings may be further flawed as shown by recent work demonstrating that larger, safety net, or academic hospitals, as well as hospitals offering transplant services, have higher readmission rates,³ which may be caused by differences in patient complexity. Other research has demonstrated that overall quality ratings inappropriately pool all hospitals together, when it may be fairer to initially categorize hospitals and then score them.⁴

It is within this maelstrom of debate that, in this month’s issue of the Journal of Hospital Medicine, Shi and colleagues explore the relationship between hospital star ratings and the socioeconomic features of the surrounding communities.⁵ Conducting their analysis by linking multiple reputable government and industry sources, Shi and colleagues found that counties with higher education attainment and a lower proportion of dual Medicare-Medicaid–eligible populations had higher hospital star ratings. Furthermore, a county’s minority population percentage negatively correlated with hospital ratings. Validating the experience of many rural hospital executives—who frequently experience financial challenges—Shi and colleagues noted that rural hospitals were less likely to receive five-star ratings.

Do these findings reflect a true disparity and lack of access to high-quality hospitals, or are they artifactual—secondary to a flawed construct of hospital quality measurement? Many lower-ranking hospitals are urban academic centers frequently providing services not offered at their five-star community counterparts, such as neurosurgery, comprehensive cancer care, and organ transplants, while simultaneously serving as safety net hospitals, research institutions, trauma centers, and national referral centers.

Sociodemographics factor significantly in self-care management for hospital aftercare. Health literacy, access to primary and behavioral healthcare, and transportation all affect star indicators. Recent work⁶ demonstrated that comprehensive investments in transitional care strategies and the social determinants of health were ineffective at reducing readmissions, which suggests that high readmission rates for hospitals in impoverished areas are not only common, but also may not accurately reflect hospital quality and local investment.

Patient experience is also complicating, with research demonstrating that patient perceptions vary significantly by education, age, primary language, ethnicity, and overall health. For example, one-third of average-ranked hospitals would have rankings vary by at least 18 percentile points when evaluated by Spanish-speaking patients. Star ratings fail to capture and communicate this granularity.⁷

More concerning is that star ratings inherently assume that hospital performance is being compared across the same tasks, regardless of patient characteristics, local resources, or the scope of services provided, the latter of which may vary between hospitals. For example, communication may differ in both complexity and time intensity: Explaining an antibiotic to the uncomplicated patient with pneumonia differs from prescribing an antibiotic to a patient who is legally blind from optic neuritis, walks with a cane because of multiple sclerosis, and has 24 other prescription medications. Similar challenges exist for differences in local neighborhood resources and for facilities with differing service scope.

Although one strategy to handle these “disparities” in star ratings might be to risk-adjust for social determinants of health, patients may be better served by first rethinking how star ratings are constructed. Clustering hospitals by scope of services provided and geographic region prior to determining star ratings would provide consumers with meaningful information by helping patients compare and make choices among either local or regional hospitals; national quality rankings are unhelpful for patients.

Arguably one of the most complex and person-dependent service enterprises, care delivery presents unique challenges for evaluation of customer experience and medical quality. Hospital star ratings are no exception: We must rethink their construction so they can be more meaningful for both patients and physicians.

The authors would like to acknowledge Daniel J Brotman, MD, for his editorial advice and input.

Dr Miller reported consulting for the Federal Trade Commission and serving as a member of the Centers for Medicare & Medicaid Services Medicare Evidence Development Coverage Advisory Committee. Drs Siddiqui and Deutschendorf have nothing to disclose.

Still in its infancy, the Hospital Compare overall hospital quality star rating program introduced by the Centers for Medicare & Medicaid Services (CMS) has generated intense industry debate. Individual health systems are microcosms of the challenges of ratings and measurement design. Sibley Memorial Hospital, a member of Johns Hopkins Medicine, is a well-run, 288-bed, community hospital located in a wealthy section of northwest District of Columbia with a five-star rating. In contrast, its academic partner, the Johns Hopkins Hospital, a 1,162-bed hospital with a century-long history of innovation situated in an impoverished Baltimore, Maryland, neighborhood, received a three-star rating.

Hospital ratings are the product of an industry in transition: As care delivery has shifted from an individual provider-driven industry to an increasingly scaled systems enterprise, policymakers implemented regulatory standards targeting quality measurement. Subsequent to the National Academy of Medicine’s 1999 report To Err is Human, policy efforts brought public reporting of quality ratings to multiple market segments, including dialysis facilities (2001), nursing homes (2003), Medicare Advantage plans (2007), and physicians (2015). The hospital industry was no exception, and in 2016—with much controversy¹—CMS launched the hospital star ratings program.

CMS Star Ratings for hospitals are based on seven measure groups: mortality, safety, readmission, patient experience, effectiveness, timeliness, and efficient use of medical imaging. Both industry and researchers have decried the challenges of star ratings, noting that hospitals with a narrower scope of services are more likely to receive higher ratings.² Measure groupings may be further flawed as shown by recent work demonstrating that larger, safety net, or academic hospitals, as well as hospitals offering transplant services, have higher readmission rates,³ which may be caused by differences in patient complexity. Other research has demonstrated that overall quality ratings inappropriately pool all hospitals together, when it may be fairer to initially categorize hospitals and then score them.⁴

It is within this maelstrom of debate that, in this month’s issue of the Journal of Hospital Medicine, Shi and colleagues explore the relationship between hospital star ratings and the socioeconomic features of the surrounding communities.⁵ Conducting their analysis by linking multiple reputable government and industry sources, Shi and colleagues found that counties with higher education attainment and a lower proportion of dual Medicare-Medicaid–eligible populations had higher hospital star ratings. Furthermore, a county’s minority population percentage negatively correlated with hospital ratings. Validating the experience of many rural hospital executives—who frequently experience financial challenges—Shi and colleagues noted that rural hospitals were less likely to receive five-star ratings.

Do these findings reflect a true disparity and lack of access to high-quality hospitals, or are they artifactual—secondary to a flawed construct of hospital quality measurement? Many lower-ranking hospitals are urban academic centers frequently providing services not offered at their five-star community counterparts, such as neurosurgery, comprehensive cancer care, and organ transplants, while simultaneously serving as safety net hospitals, research institutions, trauma centers, and national referral centers.

Sociodemographics factor significantly in self-care management for hospital aftercare. Health literacy, access to primary and behavioral healthcare, and transportation all affect star indicators. Recent work⁶ demonstrated that comprehensive investments in transitional care strategies and the social determinants of health were ineffective at reducing readmissions, which suggests that high readmission rates for hospitals in impoverished areas are not only common, but also may not accurately reflect hospital quality and local investment.

Patient experience is also complicating, with research demonstrating that patient perceptions vary significantly by education, age, primary language, ethnicity, and overall health. For example, one-third of average-ranked hospitals would have rankings vary by at least 18 percentile points when evaluated by Spanish-speaking patients. Star ratings fail to capture and communicate this granularity.⁷

More concerning is that star ratings inherently assume that hospital performance is being compared across the same tasks, regardless of patient characteristics, local resources, or the scope of services provided, the latter of which may vary between hospitals. For example, communication may differ in both complexity and time intensity: Explaining an antibiotic to the uncomplicated patient with pneumonia differs from prescribing an antibiotic to a patient who is legally blind from optic neuritis, walks with a cane because of multiple sclerosis, and has 24 other prescription medications. Similar challenges exist for differences in local neighborhood resources and for facilities with differing service scope.

Although one strategy to handle these “disparities” in star ratings might be to risk-adjust for social determinants of health, patients may be better served by first rethinking how star ratings are constructed. Clustering hospitals by scope of services provided and geographic region prior to determining star ratings would provide consumers with meaningful information by helping patients compare and make choices among either local or regional hospitals; national quality rankings are unhelpful for patients.

Arguably one of the most complex and person-dependent service enterprises, care delivery presents unique challenges for evaluation of customer experience and medical quality. Hospital star ratings are no exception: We must rethink their construction so they can be more meaningful for both patients and physicians.

The authors would like to acknowledge Daniel J Brotman, MD, for his editorial advice and input.

Dr Miller reported consulting for the Federal Trade Commission and serving as a member of the Centers for Medicare & Medicaid Services Medicare Evidence Development Coverage Advisory Committee. Drs Siddiqui and Deutschendorf have nothing to disclose.

Recent years have seen a proliferation of programs designed to prevent readmissions, including patient education initiatives, financial assistance programs, postdischarge services, and clinical personnel assigned to help patients navigate their posthospitalization clinical care. Although some strategies do not require direct patient participation (such as timely and effective handoffs between inpatient and outpatient care teams), many rely upon a commitment by the patient to participate in the postdischarge care plan. At our hospital, we have found that only about 2/3 of patients who are offered transitional interventions (such as postdischarge phone calls by nurses or home nursing through a “transition guide” program) receive the intended interventions, and those who do not receive them are more likely to be readmitted.¹ While limited patient uptake may relate, in part, to factors that are difficult to overcome, such as inadequate housing or phone service, we have also encountered patients whose values, beliefs, or preferences about their care do not align with those of the care team. The purposes of this exploratory study were to (1) assess patient attitudes surrounding readmission, (2) ascertain whether these attitudes are associated with actual readmission, and (3) determine whether patients can estimate their own risk of readmission.

From January 2014 to September 2016, we circulated surveys to patients on internal medicine nursing units who were being discharged home within 24 hours. Blank surveys were distributed to nursing units by the researchers. Unit clerks and support staff were educated on the purpose of the project and asked to distribute surveys to patients who were identified by unit case managers or nurses as slated for discharge. Staff members were not asked to help with or supervise survey completion. Surveys were generally filled out by patients, but we allowed family members to assist patients if needed, and to indicate so with a checkbox. There were no exclusion criteria. Because surveys were distributed by clinical staff, the received surveys can be considered a convenience sample. Patients were asked 5 questions with 4- or 5-point Likert scale responses:

(1) “How likely is it that you will be admitted to the hospital (have to stay in the hospital overnight) again within the next 30 days after you leave the hospital this time?” [answers ranging from “Very Unlikely (<5% chance)” to “Very Likely (>50% chance)”];

(2) “How would you feel about being rehospitalized in the next month?” [answers ranging from “Very sad, frustrated, or disappointed” to “Very happy or relieved”];

(3) “How much do you think that you personally can control whether or not you will be rehospitalized (based on what you do to take care of your body, take your medicines, and follow-up with your healthcare team)?” [answers ranging from “I have no control over whether I will be rehospitalized” to “I have complete control over whether I will be rehospitalized”];

(4) “Which of the options below best describes how you plan to follow the medical instructions after you leave the hospital?” [answers ranging from “I do NOT plan to do very much of what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I plan to do EVERYTHING I am being asked to do by the doctors, nurses, therapists and other members of the care team”]; and

(5) “Pick the item below that best describes YOUR OWN VIEW of the care team’s recommendations:” [answers ranging from “I DO NOT AGREE AT ALL that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I FULLY AGREE that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team”].

Responses were linked, based on discharge date and medical record number, to administrative data, including age, sex, race, payer, and clinical data. Subsequent hospitalizations to our hospital were ascertained from administrative data. We estimated expected risk of readmission using the all payer refined diagnosis related group coupled with the associated severity-of-illness (SOI) score, as we have reported previously.^2-5 We restricted our analysis to patients who answered the question related to the likelihood of readmission. Logistic regression models were constructed using actual 30-day readmission as the dependent variable to determine whether patients could predict their own readmissions and whether patient attitudes and beliefs about their care were predictive of subsequent readmission. Patient survey responses were entered as continuous independent variables (ranging from 1-4 or 1-5, as appropriate). Multivariable logistic regression was used to determine whether patients could predict their readmissions independent of demographic variables and expected readmission rate (modeled continuously); we repeated this model after dichotomizing the patient’s estimate of the likelihood of readmission as either “unlikely” or “likely.” Patients with missing survey responses were excluded from individual models without imputation. The study was approved by the Johns Hopkins institutional review board.

Responses were obtained from 895 patients. Their median age was 56 years [interquartile range, 43-67], 51.4% were female, and 41.7% were white. Mean SOI was 2.53 (on a 1-4 scale), and median length-of-stay was representative for our medical service at 5.2 days (range, 1-66 days). Family members reported filling out the survey in 57 cases. The primary payer was Medicare in 40.7%, Medicaid in 24.9%, and other in 34.4%. A total of 138 patients (15.4%) were readmitted within 30 days. The Table shows survey responses and associated readmission rates. None of the attitudes related to readmission were predictive of actual readmission. However, patients were able to predict their own readmissions (P = .002 for linear trend). After adjustment for expected readmission rate, race, sex, age, and payer, the trend remained significant (P = .005). Other significant predictors of readmissions in this model included expected readmission rate (P = .002), age (P = .02), and payer (P = .002). After dichotomizing the patient estimate of readmission rate as “unlikely” (N = 581) or “likely” (N = 314), the unadjusted odds ratio associating a patient-estimated risk of readmission as “likely” with actual readmission was 1.8 (95% confidence interval, 1.2-2.5). The adjusted odds ratio (including the variables above) was 1.6 (1.1-2.4).

Our findings demonstrate that patients are able to quantify their own readmission risk. This was true even after adjustment for expected readmission rate, age, sex, race, and payer. However, we did not identify any patient attitudes, beliefs, or preferences related to readmission or discharge instructions that were associated with subsequent rehospitalization. Reassuringly, more than 80% of patients who responded to the survey indicated that they would be sad, frustrated, or disappointed should readmission occur. This suggests that most patients are invested in preventing rehospitalization. Also reassuring was that patients indicated that they agreed with the discharge care plan and intended to follow their discharge instructions.

The major limitation of this study is that it was a convenience sample. Surveys were distributed inconsistently by nursing unit staff, preventing us from calculating a response rate. Further, it is possible, if not likely, that those patients with higher levels of engagement were more likely to take the time to respond, enriching our sample with activated patients. Although we allowed family members to fill out surveys on behalf of patients, this was done in fewer than 10% of instances; as such, our data may have limited applicability to patients who are physically or cognitively unable to participate in the discharge process. Finally, in this study, we did not capture readmissions to other facilities.

We conclude that patients are able to predict their own readmissions, even after accounting for other potential predictors of readmission. However, we found no evidence to support the possibility that low levels of engagement, limited trust in the healthcare team, or nonchalance about being readmitted are associated with subsequent rehospitalization. Whether asking patients about their perceived risk of readmission might help target readmission prevention programs deserves further study.

Dr. Daniel J. Brotman had full access to the data in the study and takes responsibility for the integrity of the study data and the accuracy of the data analysis. The authors also thank the following individuals for their contributions: Drafting the manuscript (Brotman); revising the manuscript for important intellectual content (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); acquiring the data (Brotman, Shihab, Tieu, Cheng, Bertram, Deutschendorf); interpreting the data (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); and analyzing the data (Brotman). The authors thank nursing leadership and nursing unit staff for their assistance in distributing surveys.

Funding support: Johns Hopkins Hospitalist Scholars Program

Disclosures: The authors have declared no conflicts of interest.

Recent years have seen a proliferation of programs designed to prevent readmissions, including patient education initiatives, financial assistance programs, postdischarge services, and clinical personnel assigned to help patients navigate their posthospitalization clinical care. Although some strategies do not require direct patient participation (such as timely and effective handoffs between inpatient and outpatient care teams), many rely upon a commitment by the patient to participate in the postdischarge care plan. At our hospital, we have found that only about 2/3 of patients who are offered transitional interventions (such as postdischarge phone calls by nurses or home nursing through a “transition guide” program) receive the intended interventions, and those who do not receive them are more likely to be readmitted.¹ While limited patient uptake may relate, in part, to factors that are difficult to overcome, such as inadequate housing or phone service, we have also encountered patients whose values, beliefs, or preferences about their care do not align with those of the care team. The purposes of this exploratory study were to (1) assess patient attitudes surrounding readmission, (2) ascertain whether these attitudes are associated with actual readmission, and (3) determine whether patients can estimate their own risk of readmission.

From January 2014 to September 2016, we circulated surveys to patients on internal medicine nursing units who were being discharged home within 24 hours. Blank surveys were distributed to nursing units by the researchers. Unit clerks and support staff were educated on the purpose of the project and asked to distribute surveys to patients who were identified by unit case managers or nurses as slated for discharge. Staff members were not asked to help with or supervise survey completion. Surveys were generally filled out by patients, but we allowed family members to assist patients if needed, and to indicate so with a checkbox. There were no exclusion criteria. Because surveys were distributed by clinical staff, the received surveys can be considered a convenience sample. Patients were asked 5 questions with 4- or 5-point Likert scale responses:

(1) “How likely is it that you will be admitted to the hospital (have to stay in the hospital overnight) again within the next 30 days after you leave the hospital this time?” [answers ranging from “Very Unlikely (<5% chance)” to “Very Likely (>50% chance)”];

(2) “How would you feel about being rehospitalized in the next month?” [answers ranging from “Very sad, frustrated, or disappointed” to “Very happy or relieved”];

(3) “How much do you think that you personally can control whether or not you will be rehospitalized (based on what you do to take care of your body, take your medicines, and follow-up with your healthcare team)?” [answers ranging from “I have no control over whether I will be rehospitalized” to “I have complete control over whether I will be rehospitalized”];

(4) “Which of the options below best describes how you plan to follow the medical instructions after you leave the hospital?” [answers ranging from “I do NOT plan to do very much of what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I plan to do EVERYTHING I am being asked to do by the doctors, nurses, therapists and other members of the care team”]; and

(5) “Pick the item below that best describes YOUR OWN VIEW of the care team’s recommendations:” [answers ranging from “I DO NOT AGREE AT ALL that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I FULLY AGREE that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team”].

Responses were linked, based on discharge date and medical record number, to administrative data, including age, sex, race, payer, and clinical data. Subsequent hospitalizations to our hospital were ascertained from administrative data. We estimated expected risk of readmission using the all payer refined diagnosis related group coupled with the associated severity-of-illness (SOI) score, as we have reported previously.^2-5 We restricted our analysis to patients who answered the question related to the likelihood of readmission. Logistic regression models were constructed using actual 30-day readmission as the dependent variable to determine whether patients could predict their own readmissions and whether patient attitudes and beliefs about their care were predictive of subsequent readmission. Patient survey responses were entered as continuous independent variables (ranging from 1-4 or 1-5, as appropriate). Multivariable logistic regression was used to determine whether patients could predict their readmissions independent of demographic variables and expected readmission rate (modeled continuously); we repeated this model after dichotomizing the patient’s estimate of the likelihood of readmission as either “unlikely” or “likely.” Patients with missing survey responses were excluded from individual models without imputation. The study was approved by the Johns Hopkins institutional review board.

Responses were obtained from 895 patients. Their median age was 56 years [interquartile range, 43-67], 51.4% were female, and 41.7% were white. Mean SOI was 2.53 (on a 1-4 scale), and median length-of-stay was representative for our medical service at 5.2 days (range, 1-66 days). Family members reported filling out the survey in 57 cases. The primary payer was Medicare in 40.7%, Medicaid in 24.9%, and other in 34.4%. A total of 138 patients (15.4%) were readmitted within 30 days. The Table shows survey responses and associated readmission rates. None of the attitudes related to readmission were predictive of actual readmission. However, patients were able to predict their own readmissions (P = .002 for linear trend). After adjustment for expected readmission rate, race, sex, age, and payer, the trend remained significant (P = .005). Other significant predictors of readmissions in this model included expected readmission rate (P = .002), age (P = .02), and payer (P = .002). After dichotomizing the patient estimate of readmission rate as “unlikely” (N = 581) or “likely” (N = 314), the unadjusted odds ratio associating a patient-estimated risk of readmission as “likely” with actual readmission was 1.8 (95% confidence interval, 1.2-2.5). The adjusted odds ratio (including the variables above) was 1.6 (1.1-2.4).

Our findings demonstrate that patients are able to quantify their own readmission risk. This was true even after adjustment for expected readmission rate, age, sex, race, and payer. However, we did not identify any patient attitudes, beliefs, or preferences related to readmission or discharge instructions that were associated with subsequent rehospitalization. Reassuringly, more than 80% of patients who responded to the survey indicated that they would be sad, frustrated, or disappointed should readmission occur. This suggests that most patients are invested in preventing rehospitalization. Also reassuring was that patients indicated that they agreed with the discharge care plan and intended to follow their discharge instructions.

The major limitation of this study is that it was a convenience sample. Surveys were distributed inconsistently by nursing unit staff, preventing us from calculating a response rate. Further, it is possible, if not likely, that those patients with higher levels of engagement were more likely to take the time to respond, enriching our sample with activated patients. Although we allowed family members to fill out surveys on behalf of patients, this was done in fewer than 10% of instances; as such, our data may have limited applicability to patients who are physically or cognitively unable to participate in the discharge process. Finally, in this study, we did not capture readmissions to other facilities.

We conclude that patients are able to predict their own readmissions, even after accounting for other potential predictors of readmission. However, we found no evidence to support the possibility that low levels of engagement, limited trust in the healthcare team, or nonchalance about being readmitted are associated with subsequent rehospitalization. Whether asking patients about their perceived risk of readmission might help target readmission prevention programs deserves further study.

Dr. Daniel J. Brotman had full access to the data in the study and takes responsibility for the integrity of the study data and the accuracy of the data analysis. The authors also thank the following individuals for their contributions: Drafting the manuscript (Brotman); revising the manuscript for important intellectual content (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); acquiring the data (Brotman, Shihab, Tieu, Cheng, Bertram, Deutschendorf); interpreting the data (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); and analyzing the data (Brotman). The authors thank nursing leadership and nursing unit staff for their assistance in distributing surveys.

Funding support: Johns Hopkins Hospitalist Scholars Program

Disclosures: The authors have declared no conflicts of interest.

Recent years have seen a proliferation of programs designed to prevent readmissions, including patient education initiatives, financial assistance programs, postdischarge services, and clinical personnel assigned to help patients navigate their posthospitalization clinical care. Although some strategies do not require direct patient participation (such as timely and effective handoffs between inpatient and outpatient care teams), many rely upon a commitment by the patient to participate in the postdischarge care plan. At our hospital, we have found that only about 2/3 of patients who are offered transitional interventions (such as postdischarge phone calls by nurses or home nursing through a “transition guide” program) receive the intended interventions, and those who do not receive them are more likely to be readmitted.¹ While limited patient uptake may relate, in part, to factors that are difficult to overcome, such as inadequate housing or phone service, we have also encountered patients whose values, beliefs, or preferences about their care do not align with those of the care team. The purposes of this exploratory study were to (1) assess patient attitudes surrounding readmission, (2) ascertain whether these attitudes are associated with actual readmission, and (3) determine whether patients can estimate their own risk of readmission.

From January 2014 to September 2016, we circulated surveys to patients on internal medicine nursing units who were being discharged home within 24 hours. Blank surveys were distributed to nursing units by the researchers. Unit clerks and support staff were educated on the purpose of the project and asked to distribute surveys to patients who were identified by unit case managers or nurses as slated for discharge. Staff members were not asked to help with or supervise survey completion. Surveys were generally filled out by patients, but we allowed family members to assist patients if needed, and to indicate so with a checkbox. There were no exclusion criteria. Because surveys were distributed by clinical staff, the received surveys can be considered a convenience sample. Patients were asked 5 questions with 4- or 5-point Likert scale responses:

(1) “How likely is it that you will be admitted to the hospital (have to stay in the hospital overnight) again within the next 30 days after you leave the hospital this time?” [answers ranging from “Very Unlikely (<5% chance)” to “Very Likely (>50% chance)”];

(2) “How would you feel about being rehospitalized in the next month?” [answers ranging from “Very sad, frustrated, or disappointed” to “Very happy or relieved”];

(3) “How much do you think that you personally can control whether or not you will be rehospitalized (based on what you do to take care of your body, take your medicines, and follow-up with your healthcare team)?” [answers ranging from “I have no control over whether I will be rehospitalized” to “I have complete control over whether I will be rehospitalized”];

(4) “Which of the options below best describes how you plan to follow the medical instructions after you leave the hospital?” [answers ranging from “I do NOT plan to do very much of what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I plan to do EVERYTHING I am being asked to do by the doctors, nurses, therapists and other members of the care team”]; and

(5) “Pick the item below that best describes YOUR OWN VIEW of the care team’s recommendations:” [answers ranging from “I DO NOT AGREE AT ALL that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team” to “I FULLY AGREE that the best way to be healthy is to do exactly what I am being asked to do by the doctors, nurses, therapists, and other members of the care team”].

Responses were linked, based on discharge date and medical record number, to administrative data, including age, sex, race, payer, and clinical data. Subsequent hospitalizations to our hospital were ascertained from administrative data. We estimated expected risk of readmission using the all payer refined diagnosis related group coupled with the associated severity-of-illness (SOI) score, as we have reported previously.^2-5 We restricted our analysis to patients who answered the question related to the likelihood of readmission. Logistic regression models were constructed using actual 30-day readmission as the dependent variable to determine whether patients could predict their own readmissions and whether patient attitudes and beliefs about their care were predictive of subsequent readmission. Patient survey responses were entered as continuous independent variables (ranging from 1-4 or 1-5, as appropriate). Multivariable logistic regression was used to determine whether patients could predict their readmissions independent of demographic variables and expected readmission rate (modeled continuously); we repeated this model after dichotomizing the patient’s estimate of the likelihood of readmission as either “unlikely” or “likely.” Patients with missing survey responses were excluded from individual models without imputation. The study was approved by the Johns Hopkins institutional review board.

Responses were obtained from 895 patients. Their median age was 56 years [interquartile range, 43-67], 51.4% were female, and 41.7% were white. Mean SOI was 2.53 (on a 1-4 scale), and median length-of-stay was representative for our medical service at 5.2 days (range, 1-66 days). Family members reported filling out the survey in 57 cases. The primary payer was Medicare in 40.7%, Medicaid in 24.9%, and other in 34.4%. A total of 138 patients (15.4%) were readmitted within 30 days. The Table shows survey responses and associated readmission rates. None of the attitudes related to readmission were predictive of actual readmission. However, patients were able to predict their own readmissions (P = .002 for linear trend). After adjustment for expected readmission rate, race, sex, age, and payer, the trend remained significant (P = .005). Other significant predictors of readmissions in this model included expected readmission rate (P = .002), age (P = .02), and payer (P = .002). After dichotomizing the patient estimate of readmission rate as “unlikely” (N = 581) or “likely” (N = 314), the unadjusted odds ratio associating a patient-estimated risk of readmission as “likely” with actual readmission was 1.8 (95% confidence interval, 1.2-2.5). The adjusted odds ratio (including the variables above) was 1.6 (1.1-2.4).

Our findings demonstrate that patients are able to quantify their own readmission risk. This was true even after adjustment for expected readmission rate, age, sex, race, and payer. However, we did not identify any patient attitudes, beliefs, or preferences related to readmission or discharge instructions that were associated with subsequent rehospitalization. Reassuringly, more than 80% of patients who responded to the survey indicated that they would be sad, frustrated, or disappointed should readmission occur. This suggests that most patients are invested in preventing rehospitalization. Also reassuring was that patients indicated that they agreed with the discharge care plan and intended to follow their discharge instructions.

The major limitation of this study is that it was a convenience sample. Surveys were distributed inconsistently by nursing unit staff, preventing us from calculating a response rate. Further, it is possible, if not likely, that those patients with higher levels of engagement were more likely to take the time to respond, enriching our sample with activated patients. Although we allowed family members to fill out surveys on behalf of patients, this was done in fewer than 10% of instances; as such, our data may have limited applicability to patients who are physically or cognitively unable to participate in the discharge process. Finally, in this study, we did not capture readmissions to other facilities.

We conclude that patients are able to predict their own readmissions, even after accounting for other potential predictors of readmission. However, we found no evidence to support the possibility that low levels of engagement, limited trust in the healthcare team, or nonchalance about being readmitted are associated with subsequent rehospitalization. Whether asking patients about their perceived risk of readmission might help target readmission prevention programs deserves further study.

Dr. Daniel J. Brotman had full access to the data in the study and takes responsibility for the integrity of the study data and the accuracy of the data analysis. The authors also thank the following individuals for their contributions: Drafting the manuscript (Brotman); revising the manuscript for important intellectual content (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); acquiring the data (Brotman, Shihab, Tieu, Cheng, Bertram, Deutschendorf); interpreting the data (Brotman, Shihab, Tieu, Cheng, Bertram, Hoyer, Deutschendorf); and analyzing the data (Brotman). The authors thank nursing leadership and nursing unit staff for their assistance in distributing surveys.

Funding support: Johns Hopkins Hospitalist Scholars Program

Disclosures: The authors have declared no conflicts of interest.

Hospital readmission rates are a consequential and contentious measure of hospital quality. Readmissions within 30 days of hospital discharge are part of the Centers for Medicare & Medicaid Services (CMS) Value-Based Purchasing Program and are publicly reported. Hospital-wide readmissions and condition-specific readmissions are heavily weighted by US News & World Report in its hospital rankings and in the new CMS Five-Star Quality Rating System.¹ However, clinicians and researchers question the construct validity of current readmission measures.^2,3

The focus on readmissions began in 2009 when Jencks et al.⁴ reported that 20% of Medicare patients were readmitted within 30 days after hospital discharge. Policy makers embraced readmission reduction, assuming that a hospital readmission so soon after discharge reflected poor quality of hospital care and that, with focused efforts, hospitals could reduce readmissions and save CMS money. In 2010, the Affordable Care Act introduced an initiative to reduce readmissions and, in 2012, the Hospital Readmission Reduction Program was implemented, financially penalizing hospitals with higher-than-expected readmission rates for patients hospitalized with principal diagnoses of heart failure, myocardial infarction, and pneumonia.⁵ Readmission measures have since proliferated and now include pay-for-performance metrics for hospitalizations for chronic obstructive pulmonary disease (COPD), coronary artery bypass grafting, and total hip or knee arthroplasty. Measures are also reported for stroke patients and for “hospital-wide readmissions,” a catch-all measure intended to capture readmission rates across most diagnoses, with various exclusions intended to prevent counting planned readmissions (eg, hospitalization for cholecystectomy following a hospitalization for cholecystitis). These measures use claims data to construct hierarchical regression models at the patient and hospital levels, assuming that variation among readmission rates are due to hospital quality effects. The goal of this approach is to level the playing field to avoid penalizing hospitals for caring for sicker patients who are at higher risk for readmission for reasons unrelated to hospital care. Yet hospital readmissions are influenced by a complex set of variables that go well beyond hospital care, some of which may be better captured by existing models than others. Below we review several potential biases in the hospital readmission measures and offer policy recommendations to improve the accuracy of these measures.

Variation in a quality measure is influenced by the quality of the underlying data, the mix of patients served, bias in the performance measure, and the degree of systemic or random error.⁶ Hospital readmission rates are subject to multiple sources of variation, and true differences in the quality of care are often a much smaller source of this variation. A recent analysis of patient readmissions following general surgery found that the majority were unrelated to suboptimal medical care.⁷ Consider 3 scenarios in which a patient with COPD is readmitted 22 days after discharge. In hospital 1, the patient was discharged without a prescription for a steroid inhaler. In hospital 2, the patient was discharged on a steroid inhaler, filled the prescription, and elected not to use it. In hospital 3, the patient was discharged on a steroid inhaler and was provided medical assistance to fill the prescription but still could not afford the $15 copay. In all 3 scenarios, the hospital would be equally culpable under the current readmission measures, suffering financial and reputational penalties.

Yet the hospitals in these scenarios are not equally culpable. Variation in the mix of patients and bias in the measure impacted performance. Hospital 1 should clearly be held accountable for the readmission. In the cases of hospitals 2 and 3, the situations are more nuanced. More education about COPD, financial investment by the hospital to cover a copay, or a different transitional care approach may have increased the likelihood of patient compliance, but, ultimately, hospitals 2 and 3 were impacted by personal health behaviors and access to public health services and financial assistance, and the readmissions were less within their control.⁸

To be valid, hospital readmission measures would need to ensure that all hospitals are similar in patient characteristics and in the need for an availability of public health services. Yet these factors vary among hospitals and cannot be accounted for by models that rely exclusively on patient-level variables, such as the nature and severity of illness. As a result, the existing readmission measures are biased against certain types of hospitals. Hospitals that treat a greater proportion of patients who are socioeconomically disadvantaged; who lack access to primary care, medical assistance, or public health programs; and who have substance abuse and mental health issues will have higher readmission rates. Hospitals that care for patients who fail initial treatments and require referral for complex care will also have higher readmission rates. These types of patients are not randomly distributed throughout our healthcare system. They are clustered at rural hospitals in underserved areas, certain urban health systems, safety net hospitals, and academic health centers. It is not surprising that readmission penalties have most severely impacted large academic hospitals that care for disadvantaged populations.² These penalties may have unintended consequences, reducing a hospital’s willingness to care for disadvantaged populations.

While these biases may unfairly harm hospitals caring for disadvantaged patients, the readmission measures may also indirectly harm patients. Low hospital readmission rates are not associated with reduced mortality and, in some instances, track with higher mortality.^9-11 This may result from measurement factors (patients who die cannot be readmitted), from neighborhood socioeconomic status (SES) factors that may impact readmissions more,¹² or from actual patient harm (some patients need acute care following discharge and may have worse outcomes if that care is delayed).¹¹ Doctors have long recognized this potential risk; empiric evidence now supports them. While mortality measures may also be impacted by sociodemographic variables,¹³ whether to adjust for SES should be defined by the purpose of the measure. If the measure is meant to evaluate hospital quality (or utilization in the case of readmissions), adjusting for SES is appropriate because it is unrealistic to expect a health system to reduce income inequality and provide safe housing. Failure to adjust for SES, which has a large impact on outcomes, may mask a quality of care issue. Conversely, if the purpose of a measure is for a community to improve population health, then it should not be adjusted for SES because the community could adjust for income inequality.

Despite the complex ethical challenges created by the efforts to reduce readmissions, there has been virtually no public dialogue with patients, physicians, and policy makers regarding how to balance the trade-offs between reducing readmission and maintaining safety. Patients would likely value increased survival more than reduced readmissions, yet the current CMS Five-Star Rating System for hospital quality weighs readmissions equally with mortality in its hospital rankings, potentially misinforming patients. For example, many well-known academic medical centers score well (4 or 5 stars) on mortality and poorly (1 or 2 stars) on readmissions, resulting in a low or average overall score, calling into question face validity and confounding consumers struggling to make decisions about where to seek care. The Medicare Payment Advisory Commission’s Report to the Congress¹⁴ highlights the multiple significant systematic and random errors with the hospital readmission data.

Revisiting the Hospital Readmission Measures

Given significant bias in the hospital readmission measures and the ethical challenges imposed by reducing readmissions, potentially at the expense of survival, we believe CMS needs to take action to remedy the problem. First, CMS should drop hospital readmissions as a quality measure from its hospital rankings. Other hospital-rating groups and insurers should do the same. When included in payment schemes, readmissions should not be construed as a quality measure but as a utilization measure, like length of stay.

Second, the Department of Health & Human Services (HHS) should invest in maturing the hospital readmission measures to ensure construct, content, and criterion validity and reliability. No doubt the risk adjustment is complex and may be inherently limited using Medicare claims data. In the case of SES adjustment, for example, limited numbers of SES measures can be constructed from current data sources.^8,13 There are other approaches to address this recommendation. For example, HHS could define a preventable readmission as one linked to some process or outcome of hospital care, such as whether the patient was discharged on an inhaler. The National Quality Forum used this approach to define a preventable venous thromboembolic event as one occurring when a patient did not receive appropriate prophylaxis. In this way, only hospital 1 in the 3 scenarios for the patient with COPD would be penalized. However, we recognize that it is not always simple to define specific process measures (eg, prescribing an inhaler) that link to readmission outcomes and that there may be other important yet hard-to-measure interventions (eg, patient and family education) that are important components of patient-centered care and readmission prevention. This is why readmissions are so challenging as a quality measure. If experts cannot define clinician behaviors that have a strong theory of change or are causally related to reduced readmissions, it is hard to call readmissions a modifiable quality measure. Another potential strategy to level the playing field would be to compare readmission rates across peer institutions only. For instance, tertiary-care safety net hospitals would be compared to one another and rural community hospitals would be compared to one another.¹⁴ Lastly, new data sources could be added to account for the social, community-level, public health, and personal health factors that heavily influence a patient’s risk for readmission, in addition to hospital-level factors. Appropriate methods will be needed to develop statistical models for risk adjustment; however, this is a complex topic and beyond the scope of the current paper.

Third, HHS could continue to use the current readmission measures as population health measures while supporting multistakeholder teams to better understand how people and their communities, public health agencies, insurers, and healthcare providers can collaborate to help patients thrive and avoid readmissions by addressing true defects in care and care coordination.

While it is understandable why policy makers chose to focus on hospital readmissions, and while we recognize that concerns about the measures were unknown when they were created, emerging evidence demonstrates that the current readmission measures (particularly when used as a quality metric) lack construct validity, contain significant bias and systematic errors, and create ethical tension by rewarding hospitals both financially and reputationally for turning away sick and socially disadvantaged patients who may, consequently, have adverse outcomes. Current readmission measures need to be reconsidered.

Acknowledgments

The authors thank Christine G. Holzmueller, BLA, with the Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, for her assistance in editing the manuscript and preparing it for journal submission.

Disclosure

Dr. Pronovost errs on the side of full disclosure and reports receiving grant or contract support from the Agency for Healthcare Research and Quality, the Gordon and Betty Moore Foundation (research related to patient safety and quality of care), the National Institutes of Health (acute lung injury research), and the American Medical Association Inc. (improve blood pressure control); honoraria from various healthcare organizations for speaking on patient safety and quality (the Leigh Bureau manages engagements); book royalties from the Penguin Group for his book Safe Patients, Smart Hospitals; and was receiving stock and fees to serve as a director for Cantel Medical up until 24 months ago. Dr. Pronovost is a founder of Patient Doctor Technologies, a startup company that seeks to enhance the partnership between patients and clinicians with an application called Doctella. Dr. Brotman, Dr. Hoyer, and Ms. Deutschendorf report no relevant conflicts of interest.

Hospital readmission rates are a consequential and contentious measure of hospital quality. Readmissions within 30 days of hospital discharge are part of the Centers for Medicare & Medicaid Services (CMS) Value-Based Purchasing Program and are publicly reported. Hospital-wide readmissions and condition-specific readmissions are heavily weighted by US News & World Report in its hospital rankings and in the new CMS Five-Star Quality Rating System.¹ However, clinicians and researchers question the construct validity of current readmission measures.^2,3

The focus on readmissions began in 2009 when Jencks et al.⁴ reported that 20% of Medicare patients were readmitted within 30 days after hospital discharge. Policy makers embraced readmission reduction, assuming that a hospital readmission so soon after discharge reflected poor quality of hospital care and that, with focused efforts, hospitals could reduce readmissions and save CMS money. In 2010, the Affordable Care Act introduced an initiative to reduce readmissions and, in 2012, the Hospital Readmission Reduction Program was implemented, financially penalizing hospitals with higher-than-expected readmission rates for patients hospitalized with principal diagnoses of heart failure, myocardial infarction, and pneumonia.⁵ Readmission measures have since proliferated and now include pay-for-performance metrics for hospitalizations for chronic obstructive pulmonary disease (COPD), coronary artery bypass grafting, and total hip or knee arthroplasty. Measures are also reported for stroke patients and for “hospital-wide readmissions,” a catch-all measure intended to capture readmission rates across most diagnoses, with various exclusions intended to prevent counting planned readmissions (eg, hospitalization for cholecystectomy following a hospitalization for cholecystitis). These measures use claims data to construct hierarchical regression models at the patient and hospital levels, assuming that variation among readmission rates are due to hospital quality effects. The goal of this approach is to level the playing field to avoid penalizing hospitals for caring for sicker patients who are at higher risk for readmission for reasons unrelated to hospital care. Yet hospital readmissions are influenced by a complex set of variables that go well beyond hospital care, some of which may be better captured by existing models than others. Below we review several potential biases in the hospital readmission measures and offer policy recommendations to improve the accuracy of these measures.

Variation in a quality measure is influenced by the quality of the underlying data, the mix of patients served, bias in the performance measure, and the degree of systemic or random error.⁶ Hospital readmission rates are subject to multiple sources of variation, and true differences in the quality of care are often a much smaller source of this variation. A recent analysis of patient readmissions following general surgery found that the majority were unrelated to suboptimal medical care.⁷ Consider 3 scenarios in which a patient with COPD is readmitted 22 days after discharge. In hospital 1, the patient was discharged without a prescription for a steroid inhaler. In hospital 2, the patient was discharged on a steroid inhaler, filled the prescription, and elected not to use it. In hospital 3, the patient was discharged on a steroid inhaler and was provided medical assistance to fill the prescription but still could not afford the $15 copay. In all 3 scenarios, the hospital would be equally culpable under the current readmission measures, suffering financial and reputational penalties.

Yet the hospitals in these scenarios are not equally culpable. Variation in the mix of patients and bias in the measure impacted performance. Hospital 1 should clearly be held accountable for the readmission. In the cases of hospitals 2 and 3, the situations are more nuanced. More education about COPD, financial investment by the hospital to cover a copay, or a different transitional care approach may have increased the likelihood of patient compliance, but, ultimately, hospitals 2 and 3 were impacted by personal health behaviors and access to public health services and financial assistance, and the readmissions were less within their control.⁸

To be valid, hospital readmission measures would need to ensure that all hospitals are similar in patient characteristics and in the need for an availability of public health services. Yet these factors vary among hospitals and cannot be accounted for by models that rely exclusively on patient-level variables, such as the nature and severity of illness. As a result, the existing readmission measures are biased against certain types of hospitals. Hospitals that treat a greater proportion of patients who are socioeconomically disadvantaged; who lack access to primary care, medical assistance, or public health programs; and who have substance abuse and mental health issues will have higher readmission rates. Hospitals that care for patients who fail initial treatments and require referral for complex care will also have higher readmission rates. These types of patients are not randomly distributed throughout our healthcare system. They are clustered at rural hospitals in underserved areas, certain urban health systems, safety net hospitals, and academic health centers. It is not surprising that readmission penalties have most severely impacted large academic hospitals that care for disadvantaged populations.² These penalties may have unintended consequences, reducing a hospital’s willingness to care for disadvantaged populations.

While these biases may unfairly harm hospitals caring for disadvantaged patients, the readmission measures may also indirectly harm patients. Low hospital readmission rates are not associated with reduced mortality and, in some instances, track with higher mortality.^9-11 This may result from measurement factors (patients who die cannot be readmitted), from neighborhood socioeconomic status (SES) factors that may impact readmissions more,¹² or from actual patient harm (some patients need acute care following discharge and may have worse outcomes if that care is delayed).¹¹ Doctors have long recognized this potential risk; empiric evidence now supports them. While mortality measures may also be impacted by sociodemographic variables,¹³ whether to adjust for SES should be defined by the purpose of the measure. If the measure is meant to evaluate hospital quality (or utilization in the case of readmissions), adjusting for SES is appropriate because it is unrealistic to expect a health system to reduce income inequality and provide safe housing. Failure to adjust for SES, which has a large impact on outcomes, may mask a quality of care issue. Conversely, if the purpose of a measure is for a community to improve population health, then it should not be adjusted for SES because the community could adjust for income inequality.

Despite the complex ethical challenges created by the efforts to reduce readmissions, there has been virtually no public dialogue with patients, physicians, and policy makers regarding how to balance the trade-offs between reducing readmission and maintaining safety. Patients would likely value increased survival more than reduced readmissions, yet the current CMS Five-Star Rating System for hospital quality weighs readmissions equally with mortality in its hospital rankings, potentially misinforming patients. For example, many well-known academic medical centers score well (4 or 5 stars) on mortality and poorly (1 or 2 stars) on readmissions, resulting in a low or average overall score, calling into question face validity and confounding consumers struggling to make decisions about where to seek care. The Medicare Payment Advisory Commission’s Report to the Congress¹⁴ highlights the multiple significant systematic and random errors with the hospital readmission data.

Revisiting the Hospital Readmission Measures

Given significant bias in the hospital readmission measures and the ethical challenges imposed by reducing readmissions, potentially at the expense of survival, we believe CMS needs to take action to remedy the problem. First, CMS should drop hospital readmissions as a quality measure from its hospital rankings. Other hospital-rating groups and insurers should do the same. When included in payment schemes, readmissions should not be construed as a quality measure but as a utilization measure, like length of stay.

Second, the Department of Health & Human Services (HHS) should invest in maturing the hospital readmission measures to ensure construct, content, and criterion validity and reliability. No doubt the risk adjustment is complex and may be inherently limited using Medicare claims data. In the case of SES adjustment, for example, limited numbers of SES measures can be constructed from current data sources.^8,13 There are other approaches to address this recommendation. For example, HHS could define a preventable readmission as one linked to some process or outcome of hospital care, such as whether the patient was discharged on an inhaler. The National Quality Forum used this approach to define a preventable venous thromboembolic event as one occurring when a patient did not receive appropriate prophylaxis. In this way, only hospital 1 in the 3 scenarios for the patient with COPD would be penalized. However, we recognize that it is not always simple to define specific process measures (eg, prescribing an inhaler) that link to readmission outcomes and that there may be other important yet hard-to-measure interventions (eg, patient and family education) that are important components of patient-centered care and readmission prevention. This is why readmissions are so challenging as a quality measure. If experts cannot define clinician behaviors that have a strong theory of change or are causally related to reduced readmissions, it is hard to call readmissions a modifiable quality measure. Another potential strategy to level the playing field would be to compare readmission rates across peer institutions only. For instance, tertiary-care safety net hospitals would be compared to one another and rural community hospitals would be compared to one another.¹⁴ Lastly, new data sources could be added to account for the social, community-level, public health, and personal health factors that heavily influence a patient’s risk for readmission, in addition to hospital-level factors. Appropriate methods will be needed to develop statistical models for risk adjustment; however, this is a complex topic and beyond the scope of the current paper.

Third, HHS could continue to use the current readmission measures as population health measures while supporting multistakeholder teams to better understand how people and their communities, public health agencies, insurers, and healthcare providers can collaborate to help patients thrive and avoid readmissions by addressing true defects in care and care coordination.

While it is understandable why policy makers chose to focus on hospital readmissions, and while we recognize that concerns about the measures were unknown when they were created, emerging evidence demonstrates that the current readmission measures (particularly when used as a quality metric) lack construct validity, contain significant bias and systematic errors, and create ethical tension by rewarding hospitals both financially and reputationally for turning away sick and socially disadvantaged patients who may, consequently, have adverse outcomes. Current readmission measures need to be reconsidered.

Acknowledgments

The authors thank Christine G. Holzmueller, BLA, with the Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, for her assistance in editing the manuscript and preparing it for journal submission.

Disclosure

Dr. Pronovost errs on the side of full disclosure and reports receiving grant or contract support from the Agency for Healthcare Research and Quality, the Gordon and Betty Moore Foundation (research related to patient safety and quality of care), the National Institutes of Health (acute lung injury research), and the American Medical Association Inc. (improve blood pressure control); honoraria from various healthcare organizations for speaking on patient safety and quality (the Leigh Bureau manages engagements); book royalties from the Penguin Group for his book Safe Patients, Smart Hospitals; and was receiving stock and fees to serve as a director for Cantel Medical up until 24 months ago. Dr. Pronovost is a founder of Patient Doctor Technologies, a startup company that seeks to enhance the partnership between patients and clinicians with an application called Doctella. Dr. Brotman, Dr. Hoyer, and Ms. Deutschendorf report no relevant conflicts of interest.

Hospital readmission rates are a consequential and contentious measure of hospital quality. Readmissions within 30 days of hospital discharge are part of the Centers for Medicare & Medicaid Services (CMS) Value-Based Purchasing Program and are publicly reported. Hospital-wide readmissions and condition-specific readmissions are heavily weighted by US News & World Report in its hospital rankings and in the new CMS Five-Star Quality Rating System.¹ However, clinicians and researchers question the construct validity of current readmission measures.^2,3

The focus on readmissions began in 2009 when Jencks et al.⁴ reported that 20% of Medicare patients were readmitted within 30 days after hospital discharge. Policy makers embraced readmission reduction, assuming that a hospital readmission so soon after discharge reflected poor quality of hospital care and that, with focused efforts, hospitals could reduce readmissions and save CMS money. In 2010, the Affordable Care Act introduced an initiative to reduce readmissions and, in 2012, the Hospital Readmission Reduction Program was implemented, financially penalizing hospitals with higher-than-expected readmission rates for patients hospitalized with principal diagnoses of heart failure, myocardial infarction, and pneumonia.⁵ Readmission measures have since proliferated and now include pay-for-performance metrics for hospitalizations for chronic obstructive pulmonary disease (COPD), coronary artery bypass grafting, and total hip or knee arthroplasty. Measures are also reported for stroke patients and for “hospital-wide readmissions,” a catch-all measure intended to capture readmission rates across most diagnoses, with various exclusions intended to prevent counting planned readmissions (eg, hospitalization for cholecystectomy following a hospitalization for cholecystitis). These measures use claims data to construct hierarchical regression models at the patient and hospital levels, assuming that variation among readmission rates are due to hospital quality effects. The goal of this approach is to level the playing field to avoid penalizing hospitals for caring for sicker patients who are at higher risk for readmission for reasons unrelated to hospital care. Yet hospital readmissions are influenced by a complex set of variables that go well beyond hospital care, some of which may be better captured by existing models than others. Below we review several potential biases in the hospital readmission measures and offer policy recommendations to improve the accuracy of these measures.

Variation in a quality measure is influenced by the quality of the underlying data, the mix of patients served, bias in the performance measure, and the degree of systemic or random error.⁶ Hospital readmission rates are subject to multiple sources of variation, and true differences in the quality of care are often a much smaller source of this variation. A recent analysis of patient readmissions following general surgery found that the majority were unrelated to suboptimal medical care.⁷ Consider 3 scenarios in which a patient with COPD is readmitted 22 days after discharge. In hospital 1, the patient was discharged without a prescription for a steroid inhaler. In hospital 2, the patient was discharged on a steroid inhaler, filled the prescription, and elected not to use it. In hospital 3, the patient was discharged on a steroid inhaler and was provided medical assistance to fill the prescription but still could not afford the $15 copay. In all 3 scenarios, the hospital would be equally culpable under the current readmission measures, suffering financial and reputational penalties.

Yet the hospitals in these scenarios are not equally culpable. Variation in the mix of patients and bias in the measure impacted performance. Hospital 1 should clearly be held accountable for the readmission. In the cases of hospitals 2 and 3, the situations are more nuanced. More education about COPD, financial investment by the hospital to cover a copay, or a different transitional care approach may have increased the likelihood of patient compliance, but, ultimately, hospitals 2 and 3 were impacted by personal health behaviors and access to public health services and financial assistance, and the readmissions were less within their control.⁸

To be valid, hospital readmission measures would need to ensure that all hospitals are similar in patient characteristics and in the need for an availability of public health services. Yet these factors vary among hospitals and cannot be accounted for by models that rely exclusively on patient-level variables, such as the nature and severity of illness. As a result, the existing readmission measures are biased against certain types of hospitals. Hospitals that treat a greater proportion of patients who are socioeconomically disadvantaged; who lack access to primary care, medical assistance, or public health programs; and who have substance abuse and mental health issues will have higher readmission rates. Hospitals that care for patients who fail initial treatments and require referral for complex care will also have higher readmission rates. These types of patients are not randomly distributed throughout our healthcare system. They are clustered at rural hospitals in underserved areas, certain urban health systems, safety net hospitals, and academic health centers. It is not surprising that readmission penalties have most severely impacted large academic hospitals that care for disadvantaged populations.² These penalties may have unintended consequences, reducing a hospital’s willingness to care for disadvantaged populations.

While these biases may unfairly harm hospitals caring for disadvantaged patients, the readmission measures may also indirectly harm patients. Low hospital readmission rates are not associated with reduced mortality and, in some instances, track with higher mortality.^9-11 This may result from measurement factors (patients who die cannot be readmitted), from neighborhood socioeconomic status (SES) factors that may impact readmissions more,¹² or from actual patient harm (some patients need acute care following discharge and may have worse outcomes if that care is delayed).¹¹ Doctors have long recognized this potential risk; empiric evidence now supports them. While mortality measures may also be impacted by sociodemographic variables,¹³ whether to adjust for SES should be defined by the purpose of the measure. If the measure is meant to evaluate hospital quality (or utilization in the case of readmissions), adjusting for SES is appropriate because it is unrealistic to expect a health system to reduce income inequality and provide safe housing. Failure to adjust for SES, which has a large impact on outcomes, may mask a quality of care issue. Conversely, if the purpose of a measure is for a community to improve population health, then it should not be adjusted for SES because the community could adjust for income inequality.

Despite the complex ethical challenges created by the efforts to reduce readmissions, there has been virtually no public dialogue with patients, physicians, and policy makers regarding how to balance the trade-offs between reducing readmission and maintaining safety. Patients would likely value increased survival more than reduced readmissions, yet the current CMS Five-Star Rating System for hospital quality weighs readmissions equally with mortality in its hospital rankings, potentially misinforming patients. For example, many well-known academic medical centers score well (4 or 5 stars) on mortality and poorly (1 or 2 stars) on readmissions, resulting in a low or average overall score, calling into question face validity and confounding consumers struggling to make decisions about where to seek care. The Medicare Payment Advisory Commission’s Report to the Congress¹⁴ highlights the multiple significant systematic and random errors with the hospital readmission data.

Revisiting the Hospital Readmission Measures

Given significant bias in the hospital readmission measures and the ethical challenges imposed by reducing readmissions, potentially at the expense of survival, we believe CMS needs to take action to remedy the problem. First, CMS should drop hospital readmissions as a quality measure from its hospital rankings. Other hospital-rating groups and insurers should do the same. When included in payment schemes, readmissions should not be construed as a quality measure but as a utilization measure, like length of stay.

Second, the Department of Health & Human Services (HHS) should invest in maturing the hospital readmission measures to ensure construct, content, and criterion validity and reliability. No doubt the risk adjustment is complex and may be inherently limited using Medicare claims data. In the case of SES adjustment, for example, limited numbers of SES measures can be constructed from current data sources.^8,13 There are other approaches to address this recommendation. For example, HHS could define a preventable readmission as one linked to some process or outcome of hospital care, such as whether the patient was discharged on an inhaler. The National Quality Forum used this approach to define a preventable venous thromboembolic event as one occurring when a patient did not receive appropriate prophylaxis. In this way, only hospital 1 in the 3 scenarios for the patient with COPD would be penalized. However, we recognize that it is not always simple to define specific process measures (eg, prescribing an inhaler) that link to readmission outcomes and that there may be other important yet hard-to-measure interventions (eg, patient and family education) that are important components of patient-centered care and readmission prevention. This is why readmissions are so challenging as a quality measure. If experts cannot define clinician behaviors that have a strong theory of change or are causally related to reduced readmissions, it is hard to call readmissions a modifiable quality measure. Another potential strategy to level the playing field would be to compare readmission rates across peer institutions only. For instance, tertiary-care safety net hospitals would be compared to one another and rural community hospitals would be compared to one another.¹⁴ Lastly, new data sources could be added to account for the social, community-level, public health, and personal health factors that heavily influence a patient’s risk for readmission, in addition to hospital-level factors. Appropriate methods will be needed to develop statistical models for risk adjustment; however, this is a complex topic and beyond the scope of the current paper.

Third, HHS could continue to use the current readmission measures as population health measures while supporting multistakeholder teams to better understand how people and their communities, public health agencies, insurers, and healthcare providers can collaborate to help patients thrive and avoid readmissions by addressing true defects in care and care coordination.

While it is understandable why policy makers chose to focus on hospital readmissions, and while we recognize that concerns about the measures were unknown when they were created, emerging evidence demonstrates that the current readmission measures (particularly when used as a quality metric) lack construct validity, contain significant bias and systematic errors, and create ethical tension by rewarding hospitals both financially and reputationally for turning away sick and socially disadvantaged patients who may, consequently, have adverse outcomes. Current readmission measures need to be reconsidered.

Acknowledgments

The authors thank Christine G. Holzmueller, BLA, with the Armstrong Institute for Patient Safety and Quality, Johns Hopkins Medicine, for her assistance in editing the manuscript and preparing it for journal submission.

Disclosure

Dr. Pronovost errs on the side of full disclosure and reports receiving grant or contract support from the Agency for Healthcare Research and Quality, the Gordon and Betty Moore Foundation (research related to patient safety and quality of care), the National Institutes of Health (acute lung injury research), and the American Medical Association Inc. (improve blood pressure control); honoraria from various healthcare organizations for speaking on patient safety and quality (the Leigh Bureau manages engagements); book royalties from the Penguin Group for his book Safe Patients, Smart Hospitals; and was receiving stock and fees to serve as a director for Cantel Medical up until 24 months ago. Dr. Pronovost is a founder of Patient Doctor Technologies, a startup company that seeks to enhance the partnership between patients and clinicians with an application called Doctella. Dr. Brotman, Dr. Hoyer, and Ms. Deutschendorf report no relevant conflicts of interest.