Hospital Rankings for Stroke Outcomes Must Consider Stroke Severity

Rankings of hospital performance in treating acute ischemic stroke must take into consideration the severity of each case, or the rankings will be extremely inaccurate, researchers say in the July 18 issue of JAMA.

Unfortunately, the rankings that are currently used by accreditation organizations, the Centers for Medicare and Medicaid Services (CMS), and other payers do not incorporate stroke severity. A study that corrected for this oversight found that close to half of the U.S. hospitals ranked in the top or bottom 5%, according to stroke patients’ 30-day mortality, should be reclassified into the middle range of the rankings.

For the 782 hospitals included in this study, the median change in rank position was 79 places when stroke severity was incorporated into the statistical model, said Dr. Gregg C. Fonarow of the Ahmanson-UCLA Cardiomyopathy Center, Los Angeles, and his associates.

When stroke severity is not considered, the rankings systematically favor hospitals that care for patients with less severe illness, regardless of whether the patient care at these hospitals produces better or worse patient outcomes.

If reliance on inaccurate rankings persists, hospitals that want to improve their ranking may consider turning away patients with more severe strokes or transferring them to other hospitals after emergency department assessment, to avoid being classified as low performance, Dr. Fonarow and his associates said.

The researchers used data from the Get With the Guidelines–Stroke Registry and from CMS inpatient claims files to create a risk-adjustment model that incorporated stroke severity, as measured by the NIHSS (National Institutes of Health Stroke Scale), into hospitals’ 30-day mortality profiling. The NIHSS is a 15-item scale that assesses the effect of acute stroke on level of consciousness, language, neglect, visual-field loss, extraocular movement, motor strength, ataxia, dysarthria, and sensory loss.

All types of hospitals in all regions of the United States were represented. The study population included 127,950 patients aged 65 years and older who had acute ischemic stroke and were treated at 782 hospitals participating in the Get With the Guidelines program from April 2003 to December 2009.

The median patient age was 80 years; 57% were women and 86% were white. Patients frequently had serious comorbidities, including hypertension (83%), diabetes (29%), coronary artery disease or prior myocardial infarction (34%), and a history of atrial fibrillation or flutter (27%).

There were 18,186 deaths within 30 days of admission.

The statistical model that incorporated stroke severity into its assessment of hospital performance "demonstrated substantially more accurate classification of hospital 30-day mortality" than did the model currently in use. When hospitals were ranked according to this more accurate profile, their rank position changed by a median of 79 places.

Overall, 206 of the 782 hospitals (26%) ended up in a different performance category once the NIHSS score was incorporated into the model.

Of the 39 hospitals that had been categorized as top performers using the standard model, only 23 remained top performers using the more accurate model. And another 16 hospitals that hadn’t made the grade with the standard model were reclassified as top performers with the more accurate model.

"There was even greater disagreement about the bottom-performing hospitals," Dr. Fonarow and his colleagues said (JAMA 2012;308:257-64).

Of the 40 worst-performing hospitals according to the standard model, nearly half (19) were reclassified as having a middling performance with the more accurate model.

"These findings highlight the importance of including a valid specific measure of stroke severity in hospital risk models for mortality after acute ischemic stroke. ... Furthermore, this study suggests that inclusion of admission stroke severity may be essential for optimal ranking of hospitals with respect to 30-day mortality," they said.

This study was supported by the American Heart Association, American Stroke Association, and Janssen Pharmaceutical Companies of Johnson & Johnson. Dr. Fonarow is an employee of the University of California, which holds a patent on retriever devices for stroke. His associates reported ties to numerous industry sources.

Stroke Severity Is a Key Consideration

In an editorial accompanying Dr. Fonarow’s report, Dr. Tobias Kurth and Dr. Mitchell S.V. Elkind said that the study "clearly highlights the importance of incorporating information on stroke severity when conducting health outcomes research in stroke. Excluding this information will lead to incorrect ranking of hospital performance by failing to consider that hospitals care for different patient populations" (JAMA 2012;308:292-4).

In this study, when considering only the hospitals ranked in the best 20% and worst 20% of the total, close to one-third would have been reclassified if the more accurate statistical model incorporating stroke severity had been used, they noted.

Dr. Kurth is in neuroepidemiology at the University of Bordeaux (France). Dr. Elkind is in the department of neurology in the school of medicine and the department of epidemiology in the school of public health at Columbia University, New York. Dr. Kurth reported ties to Allergan, Merck, and MAP Pharmaceuticals, and Dr. Elkind reported ties to diaDexus, Britol-Myers Squibb/Sanofi Pharmaceuticals Partnership, Novartis, Organon, and GlaxoSmithKline.

Rankings of hospital performance in treating acute ischemic stroke must take into consideration the severity of each case, or the rankings will be extremely inaccurate, researchers say in the July 18 issue of JAMA.

Unfortunately, the rankings that are currently used by accreditation organizations, the Centers for Medicare and Medicaid Services (CMS), and other payers do not incorporate stroke severity. A study that corrected for this oversight found that close to half of the U.S. hospitals ranked in the top or bottom 5%, according to stroke patients’ 30-day mortality, should be reclassified into the middle range of the rankings.

For the 782 hospitals included in this study, the median change in rank position was 79 places when stroke severity was incorporated into the statistical model, said Dr. Gregg C. Fonarow of the Ahmanson-UCLA Cardiomyopathy Center, Los Angeles, and his associates.

When stroke severity is not considered, the rankings systematically favor hospitals that care for patients with less severe illness, regardless of whether the patient care at these hospitals produces better or worse patient outcomes.

If reliance on inaccurate rankings persists, hospitals that want to improve their ranking may consider turning away patients with more severe strokes or transferring them to other hospitals after emergency department assessment, to avoid being classified as low performance, Dr. Fonarow and his associates said.

The researchers used data from the Get With the Guidelines–Stroke Registry and from CMS inpatient claims files to create a risk-adjustment model that incorporated stroke severity, as measured by the NIHSS (National Institutes of Health Stroke Scale), into hospitals’ 30-day mortality profiling. The NIHSS is a 15-item scale that assesses the effect of acute stroke on level of consciousness, language, neglect, visual-field loss, extraocular movement, motor strength, ataxia, dysarthria, and sensory loss.

All types of hospitals in all regions of the United States were represented. The study population included 127,950 patients aged 65 years and older who had acute ischemic stroke and were treated at 782 hospitals participating in the Get With the Guidelines program from April 2003 to December 2009.

The median patient age was 80 years; 57% were women and 86% were white. Patients frequently had serious comorbidities, including hypertension (83%), diabetes (29%), coronary artery disease or prior myocardial infarction (34%), and a history of atrial fibrillation or flutter (27%).

There were 18,186 deaths within 30 days of admission.

The statistical model that incorporated stroke severity into its assessment of hospital performance "demonstrated substantially more accurate classification of hospital 30-day mortality" than did the model currently in use. When hospitals were ranked according to this more accurate profile, their rank position changed by a median of 79 places.

Overall, 206 of the 782 hospitals (26%) ended up in a different performance category once the NIHSS score was incorporated into the model.

Of the 39 hospitals that had been categorized as top performers using the standard model, only 23 remained top performers using the more accurate model. And another 16 hospitals that hadn’t made the grade with the standard model were reclassified as top performers with the more accurate model.

"There was even greater disagreement about the bottom-performing hospitals," Dr. Fonarow and his colleagues said (JAMA 2012;308:257-64).

Of the 40 worst-performing hospitals according to the standard model, nearly half (19) were reclassified as having a middling performance with the more accurate model.

"These findings highlight the importance of including a valid specific measure of stroke severity in hospital risk models for mortality after acute ischemic stroke. ... Furthermore, this study suggests that inclusion of admission stroke severity may be essential for optimal ranking of hospitals with respect to 30-day mortality," they said.

This study was supported by the American Heart Association, American Stroke Association, and Janssen Pharmaceutical Companies of Johnson & Johnson. Dr. Fonarow is an employee of the University of California, which holds a patent on retriever devices for stroke. His associates reported ties to numerous industry sources.

Stroke Severity Is a Key Consideration

In an editorial accompanying Dr. Fonarow’s report, Dr. Tobias Kurth and Dr. Mitchell S.V. Elkind said that the study "clearly highlights the importance of incorporating information on stroke severity when conducting health outcomes research in stroke. Excluding this information will lead to incorrect ranking of hospital performance by failing to consider that hospitals care for different patient populations" (JAMA 2012;308:292-4).

In this study, when considering only the hospitals ranked in the best 20% and worst 20% of the total, close to one-third would have been reclassified if the more accurate statistical model incorporating stroke severity had been used, they noted.

Dr. Kurth is in neuroepidemiology at the University of Bordeaux (France). Dr. Elkind is in the department of neurology in the school of medicine and the department of epidemiology in the school of public health at Columbia University, New York. Dr. Kurth reported ties to Allergan, Merck, and MAP Pharmaceuticals, and Dr. Elkind reported ties to diaDexus, Britol-Myers Squibb/Sanofi Pharmaceuticals Partnership, Novartis, Organon, and GlaxoSmithKline.

Rankings of hospital performance in treating acute ischemic stroke must take into consideration the severity of each case, or the rankings will be extremely inaccurate, researchers say in the July 18 issue of JAMA.

Unfortunately, the rankings that are currently used by accreditation organizations, the Centers for Medicare and Medicaid Services (CMS), and other payers do not incorporate stroke severity. A study that corrected for this oversight found that close to half of the U.S. hospitals ranked in the top or bottom 5%, according to stroke patients’ 30-day mortality, should be reclassified into the middle range of the rankings.

For the 782 hospitals included in this study, the median change in rank position was 79 places when stroke severity was incorporated into the statistical model, said Dr. Gregg C. Fonarow of the Ahmanson-UCLA Cardiomyopathy Center, Los Angeles, and his associates.

When stroke severity is not considered, the rankings systematically favor hospitals that care for patients with less severe illness, regardless of whether the patient care at these hospitals produces better or worse patient outcomes.

If reliance on inaccurate rankings persists, hospitals that want to improve their ranking may consider turning away patients with more severe strokes or transferring them to other hospitals after emergency department assessment, to avoid being classified as low performance, Dr. Fonarow and his associates said.

The researchers used data from the Get With the Guidelines–Stroke Registry and from CMS inpatient claims files to create a risk-adjustment model that incorporated stroke severity, as measured by the NIHSS (National Institutes of Health Stroke Scale), into hospitals’ 30-day mortality profiling. The NIHSS is a 15-item scale that assesses the effect of acute stroke on level of consciousness, language, neglect, visual-field loss, extraocular movement, motor strength, ataxia, dysarthria, and sensory loss.

All types of hospitals in all regions of the United States were represented. The study population included 127,950 patients aged 65 years and older who had acute ischemic stroke and were treated at 782 hospitals participating in the Get With the Guidelines program from April 2003 to December 2009.

The median patient age was 80 years; 57% were women and 86% were white. Patients frequently had serious comorbidities, including hypertension (83%), diabetes (29%), coronary artery disease or prior myocardial infarction (34%), and a history of atrial fibrillation or flutter (27%).

There were 18,186 deaths within 30 days of admission.

The statistical model that incorporated stroke severity into its assessment of hospital performance "demonstrated substantially more accurate classification of hospital 30-day mortality" than did the model currently in use. When hospitals were ranked according to this more accurate profile, their rank position changed by a median of 79 places.

Overall, 206 of the 782 hospitals (26%) ended up in a different performance category once the NIHSS score was incorporated into the model.

Of the 39 hospitals that had been categorized as top performers using the standard model, only 23 remained top performers using the more accurate model. And another 16 hospitals that hadn’t made the grade with the standard model were reclassified as top performers with the more accurate model.

"There was even greater disagreement about the bottom-performing hospitals," Dr. Fonarow and his colleagues said (JAMA 2012;308:257-64).

Of the 40 worst-performing hospitals according to the standard model, nearly half (19) were reclassified as having a middling performance with the more accurate model.

"These findings highlight the importance of including a valid specific measure of stroke severity in hospital risk models for mortality after acute ischemic stroke. ... Furthermore, this study suggests that inclusion of admission stroke severity may be essential for optimal ranking of hospitals with respect to 30-day mortality," they said.

This study was supported by the American Heart Association, American Stroke Association, and Janssen Pharmaceutical Companies of Johnson & Johnson. Dr. Fonarow is an employee of the University of California, which holds a patent on retriever devices for stroke. His associates reported ties to numerous industry sources.

Stroke Severity Is a Key Consideration

In an editorial accompanying Dr. Fonarow’s report, Dr. Tobias Kurth and Dr. Mitchell S.V. Elkind said that the study "clearly highlights the importance of incorporating information on stroke severity when conducting health outcomes research in stroke. Excluding this information will lead to incorrect ranking of hospital performance by failing to consider that hospitals care for different patient populations" (JAMA 2012;308:292-4).

In this study, when considering only the hospitals ranked in the best 20% and worst 20% of the total, close to one-third would have been reclassified if the more accurate statistical model incorporating stroke severity had been used, they noted.

Dr. Kurth is in neuroepidemiology at the University of Bordeaux (France). Dr. Elkind is in the department of neurology in the school of medicine and the department of epidemiology in the school of public health at Columbia University, New York. Dr. Kurth reported ties to Allergan, Merck, and MAP Pharmaceuticals, and Dr. Elkind reported ties to diaDexus, Britol-Myers Squibb/Sanofi Pharmaceuticals Partnership, Novartis, Organon, and GlaxoSmithKline.