User login
Planned Readmission Algorithm
The Centers for Medicare & Medicaid Services (CMS) publicly reports all‐cause risk‐standardized readmission rates after acute‐care hospitalization for acute myocardial infarction, pneumonia, heart failure, total hip and knee arthroplasty, chronic obstructive pulmonary disease, stroke, and for patients hospital‐wide.[1, 2, 3, 4, 5] Ideally, these measures should capture unplanned readmissions that arise from acute clinical events requiring urgent rehospitalization. Planned readmissions, which are scheduled admissions usually involving nonurgent procedures, may not be a signal of quality of care. Including planned readmissions in readmission quality measures could create a disincentive to provide appropriate care to patients who are scheduled for elective or necessary procedures unrelated to the quality of the prior admission. Accordingly, under contract to the CMS, we were asked to develop an algorithm to identify planned readmissions. A version of this algorithm is now incorporated into all publicly reported readmission measures.
Given the widespread use of the planned readmission algorithm in public reporting and its implications for hospital quality measurement and evaluation, the objective of this study was to describe the development process, and to validate and refine the algorithm by reviewing charts of readmitted patients.
METHODS
Algorithm Development
To create a planned readmission algorithm, we first defined planned. We determined that readmissions for obstetrical delivery, maintenance chemotherapy, major organ transplant, and rehabilitation should always be considered planned in the sense that they are desired and/or inevitable, even if not specifically planned on a certain date. Apart from these specific types of readmissions, we defined planned readmissions as nonacute readmissions for scheduled procedures, because the vast majority of planned admissions are related to procedures. We also defined readmissions for acute illness or for complications of care as unplanned for the purposes of a quality measure. Even if such readmissions included a potentially planned procedure, because complications of care represent an important dimension of quality that should not be excluded from outcome measurement, these admissions should not be removed from the measure outcome. This definition of planned readmissions does not imply that all unplanned readmissions are unexpected or avoidable. However, it has proven very difficult to reliably define avoidable readmissions, even by expert review of charts, and we did not attempt to do so here.[6, 7]
In the second stage, we operationalized this definition into an algorithm. We used the Agency for Healthcare Research and Quality's Clinical Classification Software (CCS) codes to group thousands of individual procedure and diagnosis International Classification of Disease, Ninth Revision, Clinical Modification (ICD‐9‐CM) codes into clinically coherent, mutually exclusive procedure CCS categories and mutually exclusive diagnosis CCS categories, respectively. Clinicians on the investigative team reviewed the procedure categories to identify those that are commonly planned and that would require inpatient admission. We also reviewed the diagnosis categories to identify acute diagnoses unlikely to accompany elective procedures. We then created a flow diagram through which every readmission could be run to determine whether it was planned or unplanned based on our categorizations of procedures and diagnoses (Figure 1, and Supporting Information, Appendix A, in the online version of this article). This version of the algorithm (v1.0) was submitted to the National Quality Forum (NQF) as part of the hospital‐wide readmission measure. The measure (NQR #1789) received endorsement in April 2012.
In the third stage of development, we posted the algorithm for 2 public comment periods and recruited 27 outside experts to review and refine the algorithm following a standardized, structured process (see Supporting Information, Appendix B, in the online version of this article). Because the measures publicly report and hold hospitals accountable for unplanned readmission rates, we felt it most important that the algorithm include as few planned readmissions in the reported, unplanned outcome as possible (ie, have high negative predictive value). Therefore, in equivocal situations in which experts felt procedure categories were equally often planned or unplanned, we added those procedures to the potentially planned list. We also solicited feedback from hospitals on algorithm performance during a confidential test run of the hospital‐wide readmission measure in the fall of 2012. Based on all of this feedback, we made a number of changes to the algorithm, which was then identified as v2.1. Version 2.1 of the algorithm was submitted to the NQF as part of the endorsement process for the acute myocardial infarction and heart failure readmission measures and was endorsed by the NQF in January 2013. The algorithm (v2.1) is now applied, adapted if necessary, to all publicly reported readmission measures.[8]
Algorithm Validation: Study Cohort
We recruited 2 hospital systems to participate in a chart validation study of the accuracy of the planned readmission algorithm (v2.1). Within these 2 health systems, we selected 7 hospitals with varying bed size, teaching status, and safety‐net status. Each included 1 large academic teaching hospital that serves as a regional referral center. For each hospital's index admissions, we applied the inclusion and exclusion criteria from the hospital‐wide readmission measure. Index admissions were included for patients age 65 years or older; enrolled in Medicare fee‐for‐service (FFS); discharged from a nonfederal, short‐stay, acute‐care hospital or critical access hospital; without an in‐hospital death; not transferred to another acute‐care facility; and enrolled in Part A Medicare for 1 year prior to discharge. We excluded index admissions for patients without at least 30 days postdischarge enrollment in FFS Medicare, discharged against medical advice, admitted for medical treatment of cancer or primary psychiatric disease, admitted to a Prospective Payment System‐exempt cancer hospital, or who died during the index hospitalization. In addition, for this study, we included only index admissions that were followed by a readmission to a hospital within the participating health system between July 1, 2011 and June 30, 2012. Institutional review board approval was obtained from each of the participating health systems, which granted waivers of signed informed consent and Health Insurance Portability and Accountability Act waivers.
Algorithm Validation: Sample Size Calculation
We determined a priori that the minimum acceptable positive predictive value, or proportion of all readmissions the algorithm labels planned that are truly planned, would be 60%, and the minimum acceptable negative predictive value, or proportion of all readmissions the algorithm labels as unplanned that are truly unplanned, would be 80%. We calculated the sample size required to be confident of these values 10% and determined we would need a total of 291 planned charts and 162 unplanned charts. We inflated these numbers by 20% to account for missing or unobtainable charts for a total of 550 charts. To achieve this sample size, we included all eligible readmissions from all participating hospitals that were categorized as planned. At the 5 smaller hospitals, we randomly selected an equal number of unplanned readmissions occurring at any hospital in its healthcare system. At the 2 largest hospitals, we randomly selected 50 unplanned readmissions occurring at any hospital in its healthcare system.
Algorithm Validation: Data Abstraction
We developed an abstraction tool, tested and refined it using sample charts, and built the final the tool into a secure, password‐protected Microsoft Access 2007 (Microsoft Corp., Redmond, WA) database (see Supporting Information, Appendix C, in the online version of this article). Experienced chart abstractors with RN or MD degrees from each hospital site participated in a 1‐hour training session to become familiar with reviewing medical charts, defining planned/unplanned readmissions, and the data abstraction process. For each readmission, we asked abstractors to review as needed: emergency department triage and physician notes, admission history and physical, operative report, discharge summary, and/or discharge summary from a prior admission. The abstractors verified the accuracy of the administrative billing data, including procedures and principal diagnosis. In addition, they abstracted the source of admission and dates of all major procedures. Then the abstractors provided their opinion and supporting rationale as to whether a readmission was planned or unplanned. They were not asked to determine whether the readmission was preventable. To determine the inter‐rater reliability of data abstraction, an independent abstractor at each health system recoded a random sample of 10% of the charts.
Statistical Analysis
To ensure that we had obtained a representative sample of charts, we identified the 10 most commonly planned procedures among cases identified as planned by the algorithm in the validation cohort and then compared this with planned cases nationally. To confirm the reliability of the abstraction process, we used the kappa statistic to determine the inter‐rater reliability of the determination of planned or unplanned status. Additionally, the full study team, including 5 practicing clinicians, reviewed the details of every chart abstraction in which the algorithm was found to have misclassified the readmission as planned or unplanned. In 11 cases we determined that the abstractor had misunderstood the definition of planned readmission (ie, not all direct admissions are necessarily planned) and we reclassified the chart review assignment accordingly.
We calculated sensitivity, specificity, positive predictive value, and negative predictive value of the algorithm for the validation cohort as a whole, weighted to account for the prevalence of planned readmissions as defined by the algorithm in the national data (7.8%). Weighting is necessary because we did not obtain a pure random sample, but rather selected a stratified sample that oversampled algorithm‐identified planned readmissions.[9] We also calculated these rates separately for large hospitals (>600 beds) and for small hospitals (600 beds).
Finally, we examined performance of the algorithm for individual procedures and diagnoses to determine whether any procedures or diagnoses should be added or removed from the algorithm. First, we reviewed the diagnoses, procedures, and brief narratives provided by the abstractors for all cases in which the algorithm misclassified the readmission as either planned or unplanned. Second, we calculated the positive predictive value for each procedure that had been flagged as planned by the algorithm, and reviewed all readmissions (correctly and incorrectly classified) in which procedures with low positive predictive value took place. We also calculated the frequency with which the procedure was the only qualifying procedure resulting in an accurate or inaccurate classification. Third, to identify changes that should be made to the lists of acute and nonacute diagnoses, we reviewed the principal diagnosis for all readmissions misclassified by the algorithm as either planned or unplanned, and examined the specific ICD‐9‐CM codes within each CCS group that were most commonly associated with misclassifications.
After determining the changes that should be made to the algorithm based on these analyses, we recalculated the sensitivity, specificity, positive predictive value, and negative predictive value of the proposed revised algorithm (v3.0). All analyses used SAS version 9.3 (SAS Institute, Cary, NC).
RESULTS
Study Cohort
Characteristics of participating hospitals are shown in Table 1. Hospitals represented in this sample ranged in size, teaching status, and safety net status, although all were nonprofit. We selected 663 readmissions for review, 363 planned and 300 unplanned. Overall we were able to select 80% of hospitals planned cases for review; the remainder occurred at hospitals outside the participating hospital system. Abstractors were able to locate and review 634 (96%) of the eligible charts (range, 86%100% per hospital). The kappa statistic for inter‐rater reliability was 0.83.
Description | Hospitals, N | Readmissions Selected for Review, N* | Readmissions Reviewed, N (% of Eligible) | Unplanned Readmissions Reviewed, N | Planned Readmissions Reviewed, N | % of Hospital's Planned Readmissions Reviewed* | |
---|---|---|---|---|---|---|---|
| |||||||
All hospitals | 7 | 663 | 634 (95.6) | 283 | 351 | 77.3 | |
No. of beds | >600 | 2 | 346 | 339 (98.0) | 116 | 223 | 84.5 |
>300600 | 2 | 190 | 173 (91.1) | 85 | 88 | 87.1 | |
<300 | 3 | 127 | 122 (96.0) | 82 | 40 | 44.9 | |
Ownership | Government | 0 | |||||
For profit | 0 | ||||||
Not for profit | 7 | 663 | 634 (95.6) | 283 | 351 | 77.3 | |
Teaching status | Teaching | 2 | 346 | 339 (98.0) | 116 | 223 | 84.5 |
Nonteaching | 5 | 317 | 295 (93.1) | 167 | 128 | 67.4 | |
Safety net status | Safety net | 2 | 346 | 339 (98.0) | 116 | 223 | 84.5 |
Nonsafety net | 5 | 317 | 295 (93.1) | 167 | 128 | 67.4 | |
Region | New England | 3 | 409 | 392 (95.8) | 155 | 237 | 85.9 |
South Central | 4 | 254 | 242 (95.3) | 128 | 114 | 64.0 |
The study sample included 57/67 (85%) of the procedure or condition categories on the potentially planned list. The most common procedure CCS categories among planned readmissions (v2.1) in the validation cohort were very similar to those in the national dataset (see Supporting Information, Appendix D, in the online version of this article). Of the top 20 most commonly planned procedure CCS categories in the validation set, all but 2, therapeutic radiology for cancer treatment (CCS 211) and peripheral vascular bypass (CCS 55), were among the top 20 most commonly planned procedure CCS categories in the national data.
Test Characteristics of Algorithm
The weighted test characteristics of the current algorithm (v2.1) are shown in Table 2. Overall, the algorithm correctly identified 266 readmissions as unplanned and 181 readmissions as planned, and misidentified 170 readmissions as planned and 15 as unplanned. Once weighted to account for the stratified sampling design, the overall prevalence of true planned readmissions was 8.9% of readmissions. The weighted sensitivity was 45.1% overall and was higher in large teaching centers than in smaller community hospitals. The weighted specificity was 95.9%. The positive predictive value was 51.6%, and the negative predictive value was 94.7%.
Cohort | Sensitivity | Specificity | Positive Predictive Value | Negative Predictive Value |
---|---|---|---|---|
Algorithm v2.1 | ||||
Full cohort | 45.1% | 95.9% | 51.6% | 94.7% |
Large hospitals | 50.9% | 96.1% | 53.8% | 95.6% |
Small hospitals | 40.2% | 95.5% | 47.7% | 94.0% |
Revised algorithm v3.0 | ||||
Full cohort | 49.8% | 96.5% | 58.7% | 94.5% |
Large hospitals | 57.1% | 96.8% | 63.0% | 95.9% |
Small hospitals | 42.6% | 95.9% | 52.6% | 93.9% |
Accuracy of Individual Diagnoses and Procedures
The positive predictive value of the algorithm for individual procedure categories varied widely, from 0% to 100% among procedures with at least 10 cases (Table 3). The procedure for which the algorithm was least accurate was CCS 211, therapeutic radiology for cancer treatment (0% positive predictive value). By contrast, maintenance chemotherapy (90%) and other therapeutic procedures, hemic and lymphatic system (100%) were most accurate. Common procedures with less than 50% positive predictive value (ie, that the algorithm commonly misclassified as planned) were diagnostic cardiac catheterization (25%); debridement of wound, infection, or burn (25%); amputation of lower extremity (29%); insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator (33%); and other hernia repair (43%). Of these, diagnostic cardiac catheterization and cardiac devices are the first and second most common procedures nationally, respectively.
Readmission Procedure CCS Code | Total Categorized as Planned by Algorithm, N | Verified as Planned by Chart Review, N | Positive Predictive Value |
---|---|---|---|
| |||
47 Diagnostic cardiac catheterization; coronary arteriography | 44 | 11 | 25% |
224 Cancer chemotherapy | 40 | 22 | 55% |
157 Amputation of lower extremity | 31 | 9 | 29% |
49 Other operating room heart procedures | 27 | 16 | 59% |
48 Insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator | 24 | 8 | 33% |
43 Heart valve procedures | 20 | 16 | 80% |
Maintenance chemotherapy (diagnosis CCS 45) | 20 | 18 | 90% |
78 Colorectal resection | 18 | 9 | 50% |
169 Debridement of wound, infection or burn | 16 | 4 | 25% |
84 Cholecystectomy and common duct exploration | 16 | 5 | 31% |
99 Other OR gastrointestinal therapeutic procedures | 16 | 8 | 50% |
158 Spinal fusion | 15 | 11 | 73% |
142 Partial excision bone | 14 | 10 | 71% |
86 Other hernia repair | 14 | 6 | 42% |
44 Coronary artery bypass graft | 13 | 10 | 77% |
67 Other therapeutic procedures, hemic and lymphatic system | 13 | 13 | 100% |
211 Therapeutic radiology for cancer treatment | 12 | 0 | 0% |
45 Percutaneous transluminal coronary angioplasty | 11 | 7 | 64% |
Total | 497 | 272 | 54.7% |
The readmissions with least abstractor agreement were those involving CCS 157 (amputation of lower extremity) and CCS 169 (debridement of wound, infection or burn). Readmissions for these procedures were nearly always performed as a consequence of acute worsening of chronic conditions such as osteomyelitis or ulceration. Abstractors were divided over whether these readmissions were appropriate to call planned.
Changes to the Algorithm
We determined that the accuracy of the algorithm would be improved by removing 2 procedure categories from the planned procedure list (therapeutic radiation [CCS 211] and cancer chemotherapy [CCS 224]), adding 1 diagnosis category to the acute diagnosis list (hypertension with complications [CCS 99]), and splitting 2 diagnosis condition categories into acute and nonacute ICD‐9‐CM codes (pancreatic disorders [CCS 149] and biliary tract disease [CCS 152]). Detailed rationales for each modification to the planned readmission algorithm are described in Table 4. We felt further examination of diagnostic cardiac catheterization and cardiac devices was warranted given their high frequency, despite low positive predictive value. We also elected not to alter the categorization of amputation or debridement because it was not easy to determine whether these admissions were planned or unplanned even with chart review. We plan further analyses of these procedure categories.
Action | Diagnosis or Procedure Category | Algorithm | Chart | N | Rationale for Change |
---|---|---|---|---|---|
| |||||
Remove from planned procedure list | Therapeutic radiation (CCS 211) | Accurate | The algorithm was inaccurate in every case. All therapeutic radiology during readmissions was performed because of acute illness (pain crisis, neurologic crisis) or because scheduled treatment occurred during an unplanned readmission. In national data, this ranks as the 25th most common planned procedure identified by the algorithm v2.1. | ||
Planned | Planned | 0 | |||
Unplanned | Unplanned | 0 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 12 | |||
Cancer chemotherapy (CCS 224) | Accurate | Of the 22 correctly identified as planned, 18 (82%) would already have been categorized as planned because of a principal diagnosis of maintenance chemotherapy. Therefore, removing CCS 224 from the planned procedure list would only miss a small fraction of planned readmissions but would avoid a large number of misclassifications. In national data, this ranks as the 8th most common planned procedure identified by the algorithm v2.1. | |||
Planned | Planned | 22 | |||
Unplanned | Unplanned | 0 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 18 | |||
Add to planned procedure list | None | The abstractors felt a planned readmission was missed by the algorithm in 15 cases. A handful of these cases were missed because the planned procedure was not on the current planned procedure list; however, those procedures (eg, abdominal paracentesis, colonoscopy, endoscopy) were nearly always unplanned overall and should therefore not be added as procedures that potentially qualify as an admission as planned. | |||
Remove from acute diagnosis list | None | The abstractors felt a planned readmission was missed by the algorithm in 15 cases. The relevant disqualifying acute diagnoses were much more often associated with unplanned readmissions in our dataset. | |||
Add to acute diagnosis list | Hypertension with complications (CCS 99) | Accurate | This CCS was associated with only 1 planned readmission (for elective nephrectomy, a very rare procedure). Every other time this CCS appeared in the dataset, it was associated with an unplanned readmission (12/13, 92%); 10 of those, however, were misclassified by the algorithm as planned because they were not excluded by diagnosis (91% error rate). Consequently, adding this CCS to the acute diagnosis list is likely to miss only a very small fraction of planned readmissions, while making the overall algorithm much more accurate. | ||
Planned | Planned | 1 | |||
Unplanned | Unplanned | 2 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 10 | |||
Split diagnosis condition category into component ICD‐9 codes | Pancreatic disorders (CCS 152) | Accurate | ICD‐9 code 577.0 (acute pancreatitis) is the only acute code in this CCS. Acute pancreatitis was present in 2 cases that were misclassified as planned. Clinically, there is no situation in which a planned procedure would reasonably be performed in the setting of acute pancreatitis. Moving ICD‐9 code 577.0 to the acute list and leaving the rest of the ICD‐9 codes in CCS 152 on the nonacute list will enable the algorithm to continue to identify planned procedures for chronic pancreatitis. | ||
Planned | Planned | 0 | |||
Unplanned | Unplanned | 1 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 2 | |||
Biliary tract disease (CCS 149) | Accurate | This CCS is a mix of acute and chronic diagnoses. Of 14 charts classified as planned with CCS 149 in the principal diagnosis field, 12 were misclassified (of which 10 were associated with cholecystectomy). Separating out the acute and nonacute diagnoses will increase the accuracy of the algorithm while still ensuring that planned cholecystectomies and other procedures can be identified. Of the ICD‐9 codes in CCS 149, the following will be added to the acute diagnosis list: 574.0, 574.3, 574.6, 574.8, 575.0, 575.12, 576.1. | |||
Planned | Planned | 2 | |||
Unplanned | Unplanned | 3 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 12 | |||
Consider for change after additional study | Diagnostic cardiac catheterization (CCS 47) | Accurate | The algorithm misclassified as planned 25/38 (66%) unplanned readmissions in which diagnostic catheterizations were the only qualifying planned procedure. It also correctly identified 3/3 (100%) planned readmissions in which diagnostic cardiac catheterizations were the only qualifying planned procedure. This is the highest volume procedure in national data. | ||
Planned | Planned | 3* | |||
Unplanned | Unplanned | 13* | |||
Inaccurate | |||||
Unplanned | Planned | 0* | |||
Planned | Unplanned | 25* | |||
Insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator (CCS 48) | Accurate | The algorithm misclassified as planned 4/5 (80%) unplanned readmissions in which cardiac devices were the only qualifying procedure. However, it also correctly identified 7/8 (87.5%) planned readmissions in which cardiac devices were the only qualifying planned procedure. CCS 48 is the second most common planned procedure category nationally. | |||
Planned | Planned | 7 | |||
Unplanned | Unplanned | 1 | |||
Inaccurate | |||||
Unplanned | Planned | 1 | |||
Planned | Unplanned | 4 |
The revised algorithm (v3.0) had a weighted sensitivity of 49.8%, weighted specificity of 96.5%, positive predictive value of 58.7%, and negative predictive value of 94.5% (Table 2). In aggregate, these changes would increase the reported unplanned readmission rate from 16.0% to 16.1% in the hospital‐wide readmission measure, using 2011 to 2012 data, and would decrease the fraction of all readmissions considered planned from 7.8% to 7.2%.
DISCUSSION
We developed an algorithm based on administrative data that in its currently implemented form is very accurate at identifying unplanned readmissions, ensuring that readmissions included in publicly reported readmission measures are likely to be truly unplanned. However, nearly half of readmissions the algorithm classifies as planned are actually unplanned. That is, the algorithm is overcautious in excluding unplanned readmissions that could have counted as outcomes, particularly among admissions that include diagnostic cardiac catheterization or placement of cardiac devices (pacemakers, defibrillators). However, these errors only occur within the 7.8% of readmissions that are classified as planned and therefore do not affect overall readmission rates dramatically. A perfect algorithm would reclassify approximately half of these planned readmissions as unplanned, increasing the overall readmission rate by 0.6 percentage points.
On the other hand, the algorithm also only identifies approximately half of true planned readmissions as planned. Because the true prevalence of planned readmissions is low (approximately 9% of readmissions based on weighted chart review prevalence, or an absolute rate of 1.4%), this low sensitivity has a small effect on algorithm performance. Removing all true planned readmissions from the measure outcome would decrease the overall readmission rate by 0.8 percentage points, similar to the expected 0.6 percentage point increase that would result from better identifying unplanned readmissions; thus, a perfect algorithm would likely decrease the reported unplanned readmission rate by a net 0.2%. Overall, the existing algorithm appears to come close to the true prevalence of planned readmissions, despite inaccuracy on an individual‐case basis. The algorithm performed best at large hospitals, which are at greatest risk of being statistical outliers and of accruing penalties under the Hospital Readmissions Reduction Program.[10]
We identified several changes that marginally improved the performance of the algorithm by reducing the number of unplanned readmissions that are incorrectly removed from the measure, while avoiding the inappropriate inclusion of planned readmissions in the outcome. This revised algorithm, v3.0, was applied to public reporting of readmission rates at the end of 2014. Overall, implementing these changes increases the reported readmission rate very slightly. We also identified other procedures associated with high inaccuracy rates, removal of which would have larger impact on reporting rates, and which therefore merit further evaluation.
There are other potential methods of identifying planned readmissions. For instance, as of October 1, 2013, new administrative billing codes were created to allow hospitals to indicate that a patient was discharged with a planned acute‐care hospital inpatient readmission, without limitation as to when it will take place.[11] This code must be used at the time of the index admission to indicate that a future planned admission is expected, and was specified only to be used for neonates and patients with acute myocardial infarction. This approach, however, would omit planned readmissions that are not known to the initial discharging team, potentially missing planned readmissions. Conversely, some patients discharged with a plan for readmission may be unexpectedly readmitted for an unplanned reason. Given that the new codes were not available at the time we conducted the validation study, we were not able to determine how often the billing codes accurately identified planned readmissions. This would be an important area to consider for future study.
An alternative approach would be to create indicator codes to be applied at the time of readmission that would indicate whether that admission was planned or unplanned. Such a code would have the advantage of allowing each planned readmission to be flagged by the admitting clinicians at the time of admission rather than by an algorithm that inherently cannot be perfect. However, identifying planned readmissions at the time of readmission would also create opportunity for gaming and inconsistent application of definitions between hospitals; additional checks would need to be put in place to guard against these possibilities.
Our study has some limitations. We relied on the opinion of chart abstractors to determine whether a readmission was planned or unplanned; in a few cases, such as smoldering wounds that ultimately require surgical intervention, that determination is debatable. Abstractions were done at local institutions to minimize risks to patient privacy, and therefore we could not centrally verify determinations of planned status except by reviewing source of admission, dates of procedures, and narrative comments reported by the abstractors. Finally, we did not have sufficient volume of planned procedures to determine accuracy of the algorithm for less common procedure categories or individual procedures within categories.
In summary, we developed an algorithm to identify planned readmissions from administrative data that had high specificity and moderate sensitivity, and refined it based on chart validation. This algorithm is in use in public reporting of readmission measures to maximize the probability that the reported readmission rates represent truly unplanned readmissions.[12]
Disclosures: Financial supportThis work was performed under contract HHSM‐500‐2008‐0025I/HHSM‐500‐T0001, Modification No. 000008, titled Measure Instrument Development and Support, funded by the Centers for Medicare and Medicaid Services (CMS), an agency of the US Department of Health and Human Services. Drs. Horwitz and Ross are supported by the National Institute on Aging (K08 AG038336 and K08 AG032886, respectively) and by the American Federation for Aging Research through the Paul B. Beeson Career Development Award Program. Dr. Krumholz is supported by grant U01 HL105270‐05 (Center for Cardiovascular Outcomes Research at Yale University) from the National Heart, Lung, and Blood Institute. No funding source had any role in the study design; in the collection, analysis, and interpretation of data; or in the writing of the article. The CMS reviewed and approved the use of its data for this work and approved submission of the manuscript. All authors have completed the Unified Competing Interest form at
- Development, validation, and results of a measure of 30‐day readmission following hospitalization for pneumonia. J Hosp Med. 2011;6(3):142–150. , , , et al.
- An administrative claims measure suitable for profiling hospital performance based on 30‐day all‐cause readmission rates among patients with acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2011;4(2):243–252. , , , et al.
- An administrative claims measure suitable for profiling hospital performance on the basis of 30‐day all‐cause readmission rates among patients with heart failure. Circ Cardiovasc Qual Outcomes. 2008;1:29–37. , , , et al.
- http://www.qualitynet.org/dcs/ContentServer?c=Page161(supp10 l):S66–S75. , , , et al. Hospital‐level 30‐day all‐cause risk‐standardized readmission rate following elective primary total hip arthroplasty (THA) and/or total knee arthroplasty (TKA). Available at:
- A meta‐analysis of hospital 30‐day avoidable readmission rates. J Eval Clin Pract. 2011;18(6):1211–1218. , ,
- Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ. 2011;183(7):E391–E402. , , , ,
- 3(4):477–492. , , , et al. Centers for Medicare
- Characteristics of hospitals receiving penalties under the Hospital Readmissions Reduction Program. JAMA. 2013;309(4):342–343. ,
- Centers for Medicare and Medicaid Services. Inpatient Prospective Payment System/Long‐Term Care Hospital (IPPS/LTCH) final rule. Fed Regist. 2013;78:50533–50534.
- Massachusetts health reforms: uninsurance remains low, self‐reported health status improves as state prepares to tackle costs. Health Aff (Millwood). 2012;31(2):444–451. , ,
The Centers for Medicare & Medicaid Services (CMS) publicly reports all‐cause risk‐standardized readmission rates after acute‐care hospitalization for acute myocardial infarction, pneumonia, heart failure, total hip and knee arthroplasty, chronic obstructive pulmonary disease, stroke, and for patients hospital‐wide.[1, 2, 3, 4, 5] Ideally, these measures should capture unplanned readmissions that arise from acute clinical events requiring urgent rehospitalization. Planned readmissions, which are scheduled admissions usually involving nonurgent procedures, may not be a signal of quality of care. Including planned readmissions in readmission quality measures could create a disincentive to provide appropriate care to patients who are scheduled for elective or necessary procedures unrelated to the quality of the prior admission. Accordingly, under contract to the CMS, we were asked to develop an algorithm to identify planned readmissions. A version of this algorithm is now incorporated into all publicly reported readmission measures.
Given the widespread use of the planned readmission algorithm in public reporting and its implications for hospital quality measurement and evaluation, the objective of this study was to describe the development process, and to validate and refine the algorithm by reviewing charts of readmitted patients.
METHODS
Algorithm Development
To create a planned readmission algorithm, we first defined planned. We determined that readmissions for obstetrical delivery, maintenance chemotherapy, major organ transplant, and rehabilitation should always be considered planned in the sense that they are desired and/or inevitable, even if not specifically planned on a certain date. Apart from these specific types of readmissions, we defined planned readmissions as nonacute readmissions for scheduled procedures, because the vast majority of planned admissions are related to procedures. We also defined readmissions for acute illness or for complications of care as unplanned for the purposes of a quality measure. Even if such readmissions included a potentially planned procedure, because complications of care represent an important dimension of quality that should not be excluded from outcome measurement, these admissions should not be removed from the measure outcome. This definition of planned readmissions does not imply that all unplanned readmissions are unexpected or avoidable. However, it has proven very difficult to reliably define avoidable readmissions, even by expert review of charts, and we did not attempt to do so here.[6, 7]
In the second stage, we operationalized this definition into an algorithm. We used the Agency for Healthcare Research and Quality's Clinical Classification Software (CCS) codes to group thousands of individual procedure and diagnosis International Classification of Disease, Ninth Revision, Clinical Modification (ICD‐9‐CM) codes into clinically coherent, mutually exclusive procedure CCS categories and mutually exclusive diagnosis CCS categories, respectively. Clinicians on the investigative team reviewed the procedure categories to identify those that are commonly planned and that would require inpatient admission. We also reviewed the diagnosis categories to identify acute diagnoses unlikely to accompany elective procedures. We then created a flow diagram through which every readmission could be run to determine whether it was planned or unplanned based on our categorizations of procedures and diagnoses (Figure 1, and Supporting Information, Appendix A, in the online version of this article). This version of the algorithm (v1.0) was submitted to the National Quality Forum (NQF) as part of the hospital‐wide readmission measure. The measure (NQR #1789) received endorsement in April 2012.
In the third stage of development, we posted the algorithm for 2 public comment periods and recruited 27 outside experts to review and refine the algorithm following a standardized, structured process (see Supporting Information, Appendix B, in the online version of this article). Because the measures publicly report and hold hospitals accountable for unplanned readmission rates, we felt it most important that the algorithm include as few planned readmissions in the reported, unplanned outcome as possible (ie, have high negative predictive value). Therefore, in equivocal situations in which experts felt procedure categories were equally often planned or unplanned, we added those procedures to the potentially planned list. We also solicited feedback from hospitals on algorithm performance during a confidential test run of the hospital‐wide readmission measure in the fall of 2012. Based on all of this feedback, we made a number of changes to the algorithm, which was then identified as v2.1. Version 2.1 of the algorithm was submitted to the NQF as part of the endorsement process for the acute myocardial infarction and heart failure readmission measures and was endorsed by the NQF in January 2013. The algorithm (v2.1) is now applied, adapted if necessary, to all publicly reported readmission measures.[8]
Algorithm Validation: Study Cohort
We recruited 2 hospital systems to participate in a chart validation study of the accuracy of the planned readmission algorithm (v2.1). Within these 2 health systems, we selected 7 hospitals with varying bed size, teaching status, and safety‐net status. Each included 1 large academic teaching hospital that serves as a regional referral center. For each hospital's index admissions, we applied the inclusion and exclusion criteria from the hospital‐wide readmission measure. Index admissions were included for patients age 65 years or older; enrolled in Medicare fee‐for‐service (FFS); discharged from a nonfederal, short‐stay, acute‐care hospital or critical access hospital; without an in‐hospital death; not transferred to another acute‐care facility; and enrolled in Part A Medicare for 1 year prior to discharge. We excluded index admissions for patients without at least 30 days postdischarge enrollment in FFS Medicare, discharged against medical advice, admitted for medical treatment of cancer or primary psychiatric disease, admitted to a Prospective Payment System‐exempt cancer hospital, or who died during the index hospitalization. In addition, for this study, we included only index admissions that were followed by a readmission to a hospital within the participating health system between July 1, 2011 and June 30, 2012. Institutional review board approval was obtained from each of the participating health systems, which granted waivers of signed informed consent and Health Insurance Portability and Accountability Act waivers.
Algorithm Validation: Sample Size Calculation
We determined a priori that the minimum acceptable positive predictive value, or proportion of all readmissions the algorithm labels planned that are truly planned, would be 60%, and the minimum acceptable negative predictive value, or proportion of all readmissions the algorithm labels as unplanned that are truly unplanned, would be 80%. We calculated the sample size required to be confident of these values 10% and determined we would need a total of 291 planned charts and 162 unplanned charts. We inflated these numbers by 20% to account for missing or unobtainable charts for a total of 550 charts. To achieve this sample size, we included all eligible readmissions from all participating hospitals that were categorized as planned. At the 5 smaller hospitals, we randomly selected an equal number of unplanned readmissions occurring at any hospital in its healthcare system. At the 2 largest hospitals, we randomly selected 50 unplanned readmissions occurring at any hospital in its healthcare system.
Algorithm Validation: Data Abstraction
We developed an abstraction tool, tested and refined it using sample charts, and built the final the tool into a secure, password‐protected Microsoft Access 2007 (Microsoft Corp., Redmond, WA) database (see Supporting Information, Appendix C, in the online version of this article). Experienced chart abstractors with RN or MD degrees from each hospital site participated in a 1‐hour training session to become familiar with reviewing medical charts, defining planned/unplanned readmissions, and the data abstraction process. For each readmission, we asked abstractors to review as needed: emergency department triage and physician notes, admission history and physical, operative report, discharge summary, and/or discharge summary from a prior admission. The abstractors verified the accuracy of the administrative billing data, including procedures and principal diagnosis. In addition, they abstracted the source of admission and dates of all major procedures. Then the abstractors provided their opinion and supporting rationale as to whether a readmission was planned or unplanned. They were not asked to determine whether the readmission was preventable. To determine the inter‐rater reliability of data abstraction, an independent abstractor at each health system recoded a random sample of 10% of the charts.
Statistical Analysis
To ensure that we had obtained a representative sample of charts, we identified the 10 most commonly planned procedures among cases identified as planned by the algorithm in the validation cohort and then compared this with planned cases nationally. To confirm the reliability of the abstraction process, we used the kappa statistic to determine the inter‐rater reliability of the determination of planned or unplanned status. Additionally, the full study team, including 5 practicing clinicians, reviewed the details of every chart abstraction in which the algorithm was found to have misclassified the readmission as planned or unplanned. In 11 cases we determined that the abstractor had misunderstood the definition of planned readmission (ie, not all direct admissions are necessarily planned) and we reclassified the chart review assignment accordingly.
We calculated sensitivity, specificity, positive predictive value, and negative predictive value of the algorithm for the validation cohort as a whole, weighted to account for the prevalence of planned readmissions as defined by the algorithm in the national data (7.8%). Weighting is necessary because we did not obtain a pure random sample, but rather selected a stratified sample that oversampled algorithm‐identified planned readmissions.[9] We also calculated these rates separately for large hospitals (>600 beds) and for small hospitals (600 beds).
Finally, we examined performance of the algorithm for individual procedures and diagnoses to determine whether any procedures or diagnoses should be added or removed from the algorithm. First, we reviewed the diagnoses, procedures, and brief narratives provided by the abstractors for all cases in which the algorithm misclassified the readmission as either planned or unplanned. Second, we calculated the positive predictive value for each procedure that had been flagged as planned by the algorithm, and reviewed all readmissions (correctly and incorrectly classified) in which procedures with low positive predictive value took place. We also calculated the frequency with which the procedure was the only qualifying procedure resulting in an accurate or inaccurate classification. Third, to identify changes that should be made to the lists of acute and nonacute diagnoses, we reviewed the principal diagnosis for all readmissions misclassified by the algorithm as either planned or unplanned, and examined the specific ICD‐9‐CM codes within each CCS group that were most commonly associated with misclassifications.
After determining the changes that should be made to the algorithm based on these analyses, we recalculated the sensitivity, specificity, positive predictive value, and negative predictive value of the proposed revised algorithm (v3.0). All analyses used SAS version 9.3 (SAS Institute, Cary, NC).
RESULTS
Study Cohort
Characteristics of participating hospitals are shown in Table 1. Hospitals represented in this sample ranged in size, teaching status, and safety net status, although all were nonprofit. We selected 663 readmissions for review, 363 planned and 300 unplanned. Overall we were able to select 80% of hospitals planned cases for review; the remainder occurred at hospitals outside the participating hospital system. Abstractors were able to locate and review 634 (96%) of the eligible charts (range, 86%100% per hospital). The kappa statistic for inter‐rater reliability was 0.83.
Description | Hospitals, N | Readmissions Selected for Review, N* | Readmissions Reviewed, N (% of Eligible) | Unplanned Readmissions Reviewed, N | Planned Readmissions Reviewed, N | % of Hospital's Planned Readmissions Reviewed* | |
---|---|---|---|---|---|---|---|
| |||||||
All hospitals | 7 | 663 | 634 (95.6) | 283 | 351 | 77.3 | |
No. of beds | >600 | 2 | 346 | 339 (98.0) | 116 | 223 | 84.5 |
>300600 | 2 | 190 | 173 (91.1) | 85 | 88 | 87.1 | |
<300 | 3 | 127 | 122 (96.0) | 82 | 40 | 44.9 | |
Ownership | Government | 0 | |||||
For profit | 0 | ||||||
Not for profit | 7 | 663 | 634 (95.6) | 283 | 351 | 77.3 | |
Teaching status | Teaching | 2 | 346 | 339 (98.0) | 116 | 223 | 84.5 |
Nonteaching | 5 | 317 | 295 (93.1) | 167 | 128 | 67.4 | |
Safety net status | Safety net | 2 | 346 | 339 (98.0) | 116 | 223 | 84.5 |
Nonsafety net | 5 | 317 | 295 (93.1) | 167 | 128 | 67.4 | |
Region | New England | 3 | 409 | 392 (95.8) | 155 | 237 | 85.9 |
South Central | 4 | 254 | 242 (95.3) | 128 | 114 | 64.0 |
The study sample included 57/67 (85%) of the procedure or condition categories on the potentially planned list. The most common procedure CCS categories among planned readmissions (v2.1) in the validation cohort were very similar to those in the national dataset (see Supporting Information, Appendix D, in the online version of this article). Of the top 20 most commonly planned procedure CCS categories in the validation set, all but 2, therapeutic radiology for cancer treatment (CCS 211) and peripheral vascular bypass (CCS 55), were among the top 20 most commonly planned procedure CCS categories in the national data.
Test Characteristics of Algorithm
The weighted test characteristics of the current algorithm (v2.1) are shown in Table 2. Overall, the algorithm correctly identified 266 readmissions as unplanned and 181 readmissions as planned, and misidentified 170 readmissions as planned and 15 as unplanned. Once weighted to account for the stratified sampling design, the overall prevalence of true planned readmissions was 8.9% of readmissions. The weighted sensitivity was 45.1% overall and was higher in large teaching centers than in smaller community hospitals. The weighted specificity was 95.9%. The positive predictive value was 51.6%, and the negative predictive value was 94.7%.
Cohort | Sensitivity | Specificity | Positive Predictive Value | Negative Predictive Value |
---|---|---|---|---|
Algorithm v2.1 | ||||
Full cohort | 45.1% | 95.9% | 51.6% | 94.7% |
Large hospitals | 50.9% | 96.1% | 53.8% | 95.6% |
Small hospitals | 40.2% | 95.5% | 47.7% | 94.0% |
Revised algorithm v3.0 | ||||
Full cohort | 49.8% | 96.5% | 58.7% | 94.5% |
Large hospitals | 57.1% | 96.8% | 63.0% | 95.9% |
Small hospitals | 42.6% | 95.9% | 52.6% | 93.9% |
Accuracy of Individual Diagnoses and Procedures
The positive predictive value of the algorithm for individual procedure categories varied widely, from 0% to 100% among procedures with at least 10 cases (Table 3). The procedure for which the algorithm was least accurate was CCS 211, therapeutic radiology for cancer treatment (0% positive predictive value). By contrast, maintenance chemotherapy (90%) and other therapeutic procedures, hemic and lymphatic system (100%) were most accurate. Common procedures with less than 50% positive predictive value (ie, that the algorithm commonly misclassified as planned) were diagnostic cardiac catheterization (25%); debridement of wound, infection, or burn (25%); amputation of lower extremity (29%); insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator (33%); and other hernia repair (43%). Of these, diagnostic cardiac catheterization and cardiac devices are the first and second most common procedures nationally, respectively.
Readmission Procedure CCS Code | Total Categorized as Planned by Algorithm, N | Verified as Planned by Chart Review, N | Positive Predictive Value |
---|---|---|---|
| |||
47 Diagnostic cardiac catheterization; coronary arteriography | 44 | 11 | 25% |
224 Cancer chemotherapy | 40 | 22 | 55% |
157 Amputation of lower extremity | 31 | 9 | 29% |
49 Other operating room heart procedures | 27 | 16 | 59% |
48 Insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator | 24 | 8 | 33% |
43 Heart valve procedures | 20 | 16 | 80% |
Maintenance chemotherapy (diagnosis CCS 45) | 20 | 18 | 90% |
78 Colorectal resection | 18 | 9 | 50% |
169 Debridement of wound, infection or burn | 16 | 4 | 25% |
84 Cholecystectomy and common duct exploration | 16 | 5 | 31% |
99 Other OR gastrointestinal therapeutic procedures | 16 | 8 | 50% |
158 Spinal fusion | 15 | 11 | 73% |
142 Partial excision bone | 14 | 10 | 71% |
86 Other hernia repair | 14 | 6 | 42% |
44 Coronary artery bypass graft | 13 | 10 | 77% |
67 Other therapeutic procedures, hemic and lymphatic system | 13 | 13 | 100% |
211 Therapeutic radiology for cancer treatment | 12 | 0 | 0% |
45 Percutaneous transluminal coronary angioplasty | 11 | 7 | 64% |
Total | 497 | 272 | 54.7% |
The readmissions with least abstractor agreement were those involving CCS 157 (amputation of lower extremity) and CCS 169 (debridement of wound, infection or burn). Readmissions for these procedures were nearly always performed as a consequence of acute worsening of chronic conditions such as osteomyelitis or ulceration. Abstractors were divided over whether these readmissions were appropriate to call planned.
Changes to the Algorithm
We determined that the accuracy of the algorithm would be improved by removing 2 procedure categories from the planned procedure list (therapeutic radiation [CCS 211] and cancer chemotherapy [CCS 224]), adding 1 diagnosis category to the acute diagnosis list (hypertension with complications [CCS 99]), and splitting 2 diagnosis condition categories into acute and nonacute ICD‐9‐CM codes (pancreatic disorders [CCS 149] and biliary tract disease [CCS 152]). Detailed rationales for each modification to the planned readmission algorithm are described in Table 4. We felt further examination of diagnostic cardiac catheterization and cardiac devices was warranted given their high frequency, despite low positive predictive value. We also elected not to alter the categorization of amputation or debridement because it was not easy to determine whether these admissions were planned or unplanned even with chart review. We plan further analyses of these procedure categories.
Action | Diagnosis or Procedure Category | Algorithm | Chart | N | Rationale for Change |
---|---|---|---|---|---|
| |||||
Remove from planned procedure list | Therapeutic radiation (CCS 211) | Accurate | The algorithm was inaccurate in every case. All therapeutic radiology during readmissions was performed because of acute illness (pain crisis, neurologic crisis) or because scheduled treatment occurred during an unplanned readmission. In national data, this ranks as the 25th most common planned procedure identified by the algorithm v2.1. | ||
Planned | Planned | 0 | |||
Unplanned | Unplanned | 0 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 12 | |||
Cancer chemotherapy (CCS 224) | Accurate | Of the 22 correctly identified as planned, 18 (82%) would already have been categorized as planned because of a principal diagnosis of maintenance chemotherapy. Therefore, removing CCS 224 from the planned procedure list would only miss a small fraction of planned readmissions but would avoid a large number of misclassifications. In national data, this ranks as the 8th most common planned procedure identified by the algorithm v2.1. | |||
Planned | Planned | 22 | |||
Unplanned | Unplanned | 0 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 18 | |||
Add to planned procedure list | None | The abstractors felt a planned readmission was missed by the algorithm in 15 cases. A handful of these cases were missed because the planned procedure was not on the current planned procedure list; however, those procedures (eg, abdominal paracentesis, colonoscopy, endoscopy) were nearly always unplanned overall and should therefore not be added as procedures that potentially qualify as an admission as planned. | |||
Remove from acute diagnosis list | None | The abstractors felt a planned readmission was missed by the algorithm in 15 cases. The relevant disqualifying acute diagnoses were much more often associated with unplanned readmissions in our dataset. | |||
Add to acute diagnosis list | Hypertension with complications (CCS 99) | Accurate | This CCS was associated with only 1 planned readmission (for elective nephrectomy, a very rare procedure). Every other time this CCS appeared in the dataset, it was associated with an unplanned readmission (12/13, 92%); 10 of those, however, were misclassified by the algorithm as planned because they were not excluded by diagnosis (91% error rate). Consequently, adding this CCS to the acute diagnosis list is likely to miss only a very small fraction of planned readmissions, while making the overall algorithm much more accurate. | ||
Planned | Planned | 1 | |||
Unplanned | Unplanned | 2 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 10 | |||
Split diagnosis condition category into component ICD‐9 codes | Pancreatic disorders (CCS 152) | Accurate | ICD‐9 code 577.0 (acute pancreatitis) is the only acute code in this CCS. Acute pancreatitis was present in 2 cases that were misclassified as planned. Clinically, there is no situation in which a planned procedure would reasonably be performed in the setting of acute pancreatitis. Moving ICD‐9 code 577.0 to the acute list and leaving the rest of the ICD‐9 codes in CCS 152 on the nonacute list will enable the algorithm to continue to identify planned procedures for chronic pancreatitis. | ||
Planned | Planned | 0 | |||
Unplanned | Unplanned | 1 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 2 | |||
Biliary tract disease (CCS 149) | Accurate | This CCS is a mix of acute and chronic diagnoses. Of 14 charts classified as planned with CCS 149 in the principal diagnosis field, 12 were misclassified (of which 10 were associated with cholecystectomy). Separating out the acute and nonacute diagnoses will increase the accuracy of the algorithm while still ensuring that planned cholecystectomies and other procedures can be identified. Of the ICD‐9 codes in CCS 149, the following will be added to the acute diagnosis list: 574.0, 574.3, 574.6, 574.8, 575.0, 575.12, 576.1. | |||
Planned | Planned | 2 | |||
Unplanned | Unplanned | 3 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 12 | |||
Consider for change after additional study | Diagnostic cardiac catheterization (CCS 47) | Accurate | The algorithm misclassified as planned 25/38 (66%) unplanned readmissions in which diagnostic catheterizations were the only qualifying planned procedure. It also correctly identified 3/3 (100%) planned readmissions in which diagnostic cardiac catheterizations were the only qualifying planned procedure. This is the highest volume procedure in national data. | ||
Planned | Planned | 3* | |||
Unplanned | Unplanned | 13* | |||
Inaccurate | |||||
Unplanned | Planned | 0* | |||
Planned | Unplanned | 25* | |||
Insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator (CCS 48) | Accurate | The algorithm misclassified as planned 4/5 (80%) unplanned readmissions in which cardiac devices were the only qualifying procedure. However, it also correctly identified 7/8 (87.5%) planned readmissions in which cardiac devices were the only qualifying planned procedure. CCS 48 is the second most common planned procedure category nationally. | |||
Planned | Planned | 7 | |||
Unplanned | Unplanned | 1 | |||
Inaccurate | |||||
Unplanned | Planned | 1 | |||
Planned | Unplanned | 4 |
The revised algorithm (v3.0) had a weighted sensitivity of 49.8%, weighted specificity of 96.5%, positive predictive value of 58.7%, and negative predictive value of 94.5% (Table 2). In aggregate, these changes would increase the reported unplanned readmission rate from 16.0% to 16.1% in the hospital‐wide readmission measure, using 2011 to 2012 data, and would decrease the fraction of all readmissions considered planned from 7.8% to 7.2%.
DISCUSSION
We developed an algorithm based on administrative data that in its currently implemented form is very accurate at identifying unplanned readmissions, ensuring that readmissions included in publicly reported readmission measures are likely to be truly unplanned. However, nearly half of readmissions the algorithm classifies as planned are actually unplanned. That is, the algorithm is overcautious in excluding unplanned readmissions that could have counted as outcomes, particularly among admissions that include diagnostic cardiac catheterization or placement of cardiac devices (pacemakers, defibrillators). However, these errors only occur within the 7.8% of readmissions that are classified as planned and therefore do not affect overall readmission rates dramatically. A perfect algorithm would reclassify approximately half of these planned readmissions as unplanned, increasing the overall readmission rate by 0.6 percentage points.
On the other hand, the algorithm also only identifies approximately half of true planned readmissions as planned. Because the true prevalence of planned readmissions is low (approximately 9% of readmissions based on weighted chart review prevalence, or an absolute rate of 1.4%), this low sensitivity has a small effect on algorithm performance. Removing all true planned readmissions from the measure outcome would decrease the overall readmission rate by 0.8 percentage points, similar to the expected 0.6 percentage point increase that would result from better identifying unplanned readmissions; thus, a perfect algorithm would likely decrease the reported unplanned readmission rate by a net 0.2%. Overall, the existing algorithm appears to come close to the true prevalence of planned readmissions, despite inaccuracy on an individual‐case basis. The algorithm performed best at large hospitals, which are at greatest risk of being statistical outliers and of accruing penalties under the Hospital Readmissions Reduction Program.[10]
We identified several changes that marginally improved the performance of the algorithm by reducing the number of unplanned readmissions that are incorrectly removed from the measure, while avoiding the inappropriate inclusion of planned readmissions in the outcome. This revised algorithm, v3.0, was applied to public reporting of readmission rates at the end of 2014. Overall, implementing these changes increases the reported readmission rate very slightly. We also identified other procedures associated with high inaccuracy rates, removal of which would have larger impact on reporting rates, and which therefore merit further evaluation.
There are other potential methods of identifying planned readmissions. For instance, as of October 1, 2013, new administrative billing codes were created to allow hospitals to indicate that a patient was discharged with a planned acute‐care hospital inpatient readmission, without limitation as to when it will take place.[11] This code must be used at the time of the index admission to indicate that a future planned admission is expected, and was specified only to be used for neonates and patients with acute myocardial infarction. This approach, however, would omit planned readmissions that are not known to the initial discharging team, potentially missing planned readmissions. Conversely, some patients discharged with a plan for readmission may be unexpectedly readmitted for an unplanned reason. Given that the new codes were not available at the time we conducted the validation study, we were not able to determine how often the billing codes accurately identified planned readmissions. This would be an important area to consider for future study.
An alternative approach would be to create indicator codes to be applied at the time of readmission that would indicate whether that admission was planned or unplanned. Such a code would have the advantage of allowing each planned readmission to be flagged by the admitting clinicians at the time of admission rather than by an algorithm that inherently cannot be perfect. However, identifying planned readmissions at the time of readmission would also create opportunity for gaming and inconsistent application of definitions between hospitals; additional checks would need to be put in place to guard against these possibilities.
Our study has some limitations. We relied on the opinion of chart abstractors to determine whether a readmission was planned or unplanned; in a few cases, such as smoldering wounds that ultimately require surgical intervention, that determination is debatable. Abstractions were done at local institutions to minimize risks to patient privacy, and therefore we could not centrally verify determinations of planned status except by reviewing source of admission, dates of procedures, and narrative comments reported by the abstractors. Finally, we did not have sufficient volume of planned procedures to determine accuracy of the algorithm for less common procedure categories or individual procedures within categories.
In summary, we developed an algorithm to identify planned readmissions from administrative data that had high specificity and moderate sensitivity, and refined it based on chart validation. This algorithm is in use in public reporting of readmission measures to maximize the probability that the reported readmission rates represent truly unplanned readmissions.[12]
Disclosures: Financial supportThis work was performed under contract HHSM‐500‐2008‐0025I/HHSM‐500‐T0001, Modification No. 000008, titled Measure Instrument Development and Support, funded by the Centers for Medicare and Medicaid Services (CMS), an agency of the US Department of Health and Human Services. Drs. Horwitz and Ross are supported by the National Institute on Aging (K08 AG038336 and K08 AG032886, respectively) and by the American Federation for Aging Research through the Paul B. Beeson Career Development Award Program. Dr. Krumholz is supported by grant U01 HL105270‐05 (Center for Cardiovascular Outcomes Research at Yale University) from the National Heart, Lung, and Blood Institute. No funding source had any role in the study design; in the collection, analysis, and interpretation of data; or in the writing of the article. The CMS reviewed and approved the use of its data for this work and approved submission of the manuscript. All authors have completed the Unified Competing Interest form at
The Centers for Medicare & Medicaid Services (CMS) publicly reports all‐cause risk‐standardized readmission rates after acute‐care hospitalization for acute myocardial infarction, pneumonia, heart failure, total hip and knee arthroplasty, chronic obstructive pulmonary disease, stroke, and for patients hospital‐wide.[1, 2, 3, 4, 5] Ideally, these measures should capture unplanned readmissions that arise from acute clinical events requiring urgent rehospitalization. Planned readmissions, which are scheduled admissions usually involving nonurgent procedures, may not be a signal of quality of care. Including planned readmissions in readmission quality measures could create a disincentive to provide appropriate care to patients who are scheduled for elective or necessary procedures unrelated to the quality of the prior admission. Accordingly, under contract to the CMS, we were asked to develop an algorithm to identify planned readmissions. A version of this algorithm is now incorporated into all publicly reported readmission measures.
Given the widespread use of the planned readmission algorithm in public reporting and its implications for hospital quality measurement and evaluation, the objective of this study was to describe the development process, and to validate and refine the algorithm by reviewing charts of readmitted patients.
METHODS
Algorithm Development
To create a planned readmission algorithm, we first defined planned. We determined that readmissions for obstetrical delivery, maintenance chemotherapy, major organ transplant, and rehabilitation should always be considered planned in the sense that they are desired and/or inevitable, even if not specifically planned on a certain date. Apart from these specific types of readmissions, we defined planned readmissions as nonacute readmissions for scheduled procedures, because the vast majority of planned admissions are related to procedures. We also defined readmissions for acute illness or for complications of care as unplanned for the purposes of a quality measure. Even if such readmissions included a potentially planned procedure, because complications of care represent an important dimension of quality that should not be excluded from outcome measurement, these admissions should not be removed from the measure outcome. This definition of planned readmissions does not imply that all unplanned readmissions are unexpected or avoidable. However, it has proven very difficult to reliably define avoidable readmissions, even by expert review of charts, and we did not attempt to do so here.[6, 7]
In the second stage, we operationalized this definition into an algorithm. We used the Agency for Healthcare Research and Quality's Clinical Classification Software (CCS) codes to group thousands of individual procedure and diagnosis International Classification of Disease, Ninth Revision, Clinical Modification (ICD‐9‐CM) codes into clinically coherent, mutually exclusive procedure CCS categories and mutually exclusive diagnosis CCS categories, respectively. Clinicians on the investigative team reviewed the procedure categories to identify those that are commonly planned and that would require inpatient admission. We also reviewed the diagnosis categories to identify acute diagnoses unlikely to accompany elective procedures. We then created a flow diagram through which every readmission could be run to determine whether it was planned or unplanned based on our categorizations of procedures and diagnoses (Figure 1, and Supporting Information, Appendix A, in the online version of this article). This version of the algorithm (v1.0) was submitted to the National Quality Forum (NQF) as part of the hospital‐wide readmission measure. The measure (NQR #1789) received endorsement in April 2012.
In the third stage of development, we posted the algorithm for 2 public comment periods and recruited 27 outside experts to review and refine the algorithm following a standardized, structured process (see Supporting Information, Appendix B, in the online version of this article). Because the measures publicly report and hold hospitals accountable for unplanned readmission rates, we felt it most important that the algorithm include as few planned readmissions in the reported, unplanned outcome as possible (ie, have high negative predictive value). Therefore, in equivocal situations in which experts felt procedure categories were equally often planned or unplanned, we added those procedures to the potentially planned list. We also solicited feedback from hospitals on algorithm performance during a confidential test run of the hospital‐wide readmission measure in the fall of 2012. Based on all of this feedback, we made a number of changes to the algorithm, which was then identified as v2.1. Version 2.1 of the algorithm was submitted to the NQF as part of the endorsement process for the acute myocardial infarction and heart failure readmission measures and was endorsed by the NQF in January 2013. The algorithm (v2.1) is now applied, adapted if necessary, to all publicly reported readmission measures.[8]
Algorithm Validation: Study Cohort
We recruited 2 hospital systems to participate in a chart validation study of the accuracy of the planned readmission algorithm (v2.1). Within these 2 health systems, we selected 7 hospitals with varying bed size, teaching status, and safety‐net status. Each included 1 large academic teaching hospital that serves as a regional referral center. For each hospital's index admissions, we applied the inclusion and exclusion criteria from the hospital‐wide readmission measure. Index admissions were included for patients age 65 years or older; enrolled in Medicare fee‐for‐service (FFS); discharged from a nonfederal, short‐stay, acute‐care hospital or critical access hospital; without an in‐hospital death; not transferred to another acute‐care facility; and enrolled in Part A Medicare for 1 year prior to discharge. We excluded index admissions for patients without at least 30 days postdischarge enrollment in FFS Medicare, discharged against medical advice, admitted for medical treatment of cancer or primary psychiatric disease, admitted to a Prospective Payment System‐exempt cancer hospital, or who died during the index hospitalization. In addition, for this study, we included only index admissions that were followed by a readmission to a hospital within the participating health system between July 1, 2011 and June 30, 2012. Institutional review board approval was obtained from each of the participating health systems, which granted waivers of signed informed consent and Health Insurance Portability and Accountability Act waivers.
Algorithm Validation: Sample Size Calculation
We determined a priori that the minimum acceptable positive predictive value, or proportion of all readmissions the algorithm labels planned that are truly planned, would be 60%, and the minimum acceptable negative predictive value, or proportion of all readmissions the algorithm labels as unplanned that are truly unplanned, would be 80%. We calculated the sample size required to be confident of these values 10% and determined we would need a total of 291 planned charts and 162 unplanned charts. We inflated these numbers by 20% to account for missing or unobtainable charts for a total of 550 charts. To achieve this sample size, we included all eligible readmissions from all participating hospitals that were categorized as planned. At the 5 smaller hospitals, we randomly selected an equal number of unplanned readmissions occurring at any hospital in its healthcare system. At the 2 largest hospitals, we randomly selected 50 unplanned readmissions occurring at any hospital in its healthcare system.
Algorithm Validation: Data Abstraction
We developed an abstraction tool, tested and refined it using sample charts, and built the final the tool into a secure, password‐protected Microsoft Access 2007 (Microsoft Corp., Redmond, WA) database (see Supporting Information, Appendix C, in the online version of this article). Experienced chart abstractors with RN or MD degrees from each hospital site participated in a 1‐hour training session to become familiar with reviewing medical charts, defining planned/unplanned readmissions, and the data abstraction process. For each readmission, we asked abstractors to review as needed: emergency department triage and physician notes, admission history and physical, operative report, discharge summary, and/or discharge summary from a prior admission. The abstractors verified the accuracy of the administrative billing data, including procedures and principal diagnosis. In addition, they abstracted the source of admission and dates of all major procedures. Then the abstractors provided their opinion and supporting rationale as to whether a readmission was planned or unplanned. They were not asked to determine whether the readmission was preventable. To determine the inter‐rater reliability of data abstraction, an independent abstractor at each health system recoded a random sample of 10% of the charts.
Statistical Analysis
To ensure that we had obtained a representative sample of charts, we identified the 10 most commonly planned procedures among cases identified as planned by the algorithm in the validation cohort and then compared this with planned cases nationally. To confirm the reliability of the abstraction process, we used the kappa statistic to determine the inter‐rater reliability of the determination of planned or unplanned status. Additionally, the full study team, including 5 practicing clinicians, reviewed the details of every chart abstraction in which the algorithm was found to have misclassified the readmission as planned or unplanned. In 11 cases we determined that the abstractor had misunderstood the definition of planned readmission (ie, not all direct admissions are necessarily planned) and we reclassified the chart review assignment accordingly.
We calculated sensitivity, specificity, positive predictive value, and negative predictive value of the algorithm for the validation cohort as a whole, weighted to account for the prevalence of planned readmissions as defined by the algorithm in the national data (7.8%). Weighting is necessary because we did not obtain a pure random sample, but rather selected a stratified sample that oversampled algorithm‐identified planned readmissions.[9] We also calculated these rates separately for large hospitals (>600 beds) and for small hospitals (600 beds).
Finally, we examined performance of the algorithm for individual procedures and diagnoses to determine whether any procedures or diagnoses should be added or removed from the algorithm. First, we reviewed the diagnoses, procedures, and brief narratives provided by the abstractors for all cases in which the algorithm misclassified the readmission as either planned or unplanned. Second, we calculated the positive predictive value for each procedure that had been flagged as planned by the algorithm, and reviewed all readmissions (correctly and incorrectly classified) in which procedures with low positive predictive value took place. We also calculated the frequency with which the procedure was the only qualifying procedure resulting in an accurate or inaccurate classification. Third, to identify changes that should be made to the lists of acute and nonacute diagnoses, we reviewed the principal diagnosis for all readmissions misclassified by the algorithm as either planned or unplanned, and examined the specific ICD‐9‐CM codes within each CCS group that were most commonly associated with misclassifications.
After determining the changes that should be made to the algorithm based on these analyses, we recalculated the sensitivity, specificity, positive predictive value, and negative predictive value of the proposed revised algorithm (v3.0). All analyses used SAS version 9.3 (SAS Institute, Cary, NC).
RESULTS
Study Cohort
Characteristics of participating hospitals are shown in Table 1. Hospitals represented in this sample ranged in size, teaching status, and safety net status, although all were nonprofit. We selected 663 readmissions for review, 363 planned and 300 unplanned. Overall we were able to select 80% of hospitals planned cases for review; the remainder occurred at hospitals outside the participating hospital system. Abstractors were able to locate and review 634 (96%) of the eligible charts (range, 86%100% per hospital). The kappa statistic for inter‐rater reliability was 0.83.
Description | Hospitals, N | Readmissions Selected for Review, N* | Readmissions Reviewed, N (% of Eligible) | Unplanned Readmissions Reviewed, N | Planned Readmissions Reviewed, N | % of Hospital's Planned Readmissions Reviewed* | |
---|---|---|---|---|---|---|---|
| |||||||
All hospitals | 7 | 663 | 634 (95.6) | 283 | 351 | 77.3 | |
No. of beds | >600 | 2 | 346 | 339 (98.0) | 116 | 223 | 84.5 |
>300600 | 2 | 190 | 173 (91.1) | 85 | 88 | 87.1 | |
<300 | 3 | 127 | 122 (96.0) | 82 | 40 | 44.9 | |
Ownership | Government | 0 | |||||
For profit | 0 | ||||||
Not for profit | 7 | 663 | 634 (95.6) | 283 | 351 | 77.3 | |
Teaching status | Teaching | 2 | 346 | 339 (98.0) | 116 | 223 | 84.5 |
Nonteaching | 5 | 317 | 295 (93.1) | 167 | 128 | 67.4 | |
Safety net status | Safety net | 2 | 346 | 339 (98.0) | 116 | 223 | 84.5 |
Nonsafety net | 5 | 317 | 295 (93.1) | 167 | 128 | 67.4 | |
Region | New England | 3 | 409 | 392 (95.8) | 155 | 237 | 85.9 |
South Central | 4 | 254 | 242 (95.3) | 128 | 114 | 64.0 |
The study sample included 57/67 (85%) of the procedure or condition categories on the potentially planned list. The most common procedure CCS categories among planned readmissions (v2.1) in the validation cohort were very similar to those in the national dataset (see Supporting Information, Appendix D, in the online version of this article). Of the top 20 most commonly planned procedure CCS categories in the validation set, all but 2, therapeutic radiology for cancer treatment (CCS 211) and peripheral vascular bypass (CCS 55), were among the top 20 most commonly planned procedure CCS categories in the national data.
Test Characteristics of Algorithm
The weighted test characteristics of the current algorithm (v2.1) are shown in Table 2. Overall, the algorithm correctly identified 266 readmissions as unplanned and 181 readmissions as planned, and misidentified 170 readmissions as planned and 15 as unplanned. Once weighted to account for the stratified sampling design, the overall prevalence of true planned readmissions was 8.9% of readmissions. The weighted sensitivity was 45.1% overall and was higher in large teaching centers than in smaller community hospitals. The weighted specificity was 95.9%. The positive predictive value was 51.6%, and the negative predictive value was 94.7%.
Cohort | Sensitivity | Specificity | Positive Predictive Value | Negative Predictive Value |
---|---|---|---|---|
Algorithm v2.1 | ||||
Full cohort | 45.1% | 95.9% | 51.6% | 94.7% |
Large hospitals | 50.9% | 96.1% | 53.8% | 95.6% |
Small hospitals | 40.2% | 95.5% | 47.7% | 94.0% |
Revised algorithm v3.0 | ||||
Full cohort | 49.8% | 96.5% | 58.7% | 94.5% |
Large hospitals | 57.1% | 96.8% | 63.0% | 95.9% |
Small hospitals | 42.6% | 95.9% | 52.6% | 93.9% |
Accuracy of Individual Diagnoses and Procedures
The positive predictive value of the algorithm for individual procedure categories varied widely, from 0% to 100% among procedures with at least 10 cases (Table 3). The procedure for which the algorithm was least accurate was CCS 211, therapeutic radiology for cancer treatment (0% positive predictive value). By contrast, maintenance chemotherapy (90%) and other therapeutic procedures, hemic and lymphatic system (100%) were most accurate. Common procedures with less than 50% positive predictive value (ie, that the algorithm commonly misclassified as planned) were diagnostic cardiac catheterization (25%); debridement of wound, infection, or burn (25%); amputation of lower extremity (29%); insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator (33%); and other hernia repair (43%). Of these, diagnostic cardiac catheterization and cardiac devices are the first and second most common procedures nationally, respectively.
Readmission Procedure CCS Code | Total Categorized as Planned by Algorithm, N | Verified as Planned by Chart Review, N | Positive Predictive Value |
---|---|---|---|
| |||
47 Diagnostic cardiac catheterization; coronary arteriography | 44 | 11 | 25% |
224 Cancer chemotherapy | 40 | 22 | 55% |
157 Amputation of lower extremity | 31 | 9 | 29% |
49 Other operating room heart procedures | 27 | 16 | 59% |
48 Insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator | 24 | 8 | 33% |
43 Heart valve procedures | 20 | 16 | 80% |
Maintenance chemotherapy (diagnosis CCS 45) | 20 | 18 | 90% |
78 Colorectal resection | 18 | 9 | 50% |
169 Debridement of wound, infection or burn | 16 | 4 | 25% |
84 Cholecystectomy and common duct exploration | 16 | 5 | 31% |
99 Other OR gastrointestinal therapeutic procedures | 16 | 8 | 50% |
158 Spinal fusion | 15 | 11 | 73% |
142 Partial excision bone | 14 | 10 | 71% |
86 Other hernia repair | 14 | 6 | 42% |
44 Coronary artery bypass graft | 13 | 10 | 77% |
67 Other therapeutic procedures, hemic and lymphatic system | 13 | 13 | 100% |
211 Therapeutic radiology for cancer treatment | 12 | 0 | 0% |
45 Percutaneous transluminal coronary angioplasty | 11 | 7 | 64% |
Total | 497 | 272 | 54.7% |
The readmissions with least abstractor agreement were those involving CCS 157 (amputation of lower extremity) and CCS 169 (debridement of wound, infection or burn). Readmissions for these procedures were nearly always performed as a consequence of acute worsening of chronic conditions such as osteomyelitis or ulceration. Abstractors were divided over whether these readmissions were appropriate to call planned.
Changes to the Algorithm
We determined that the accuracy of the algorithm would be improved by removing 2 procedure categories from the planned procedure list (therapeutic radiation [CCS 211] and cancer chemotherapy [CCS 224]), adding 1 diagnosis category to the acute diagnosis list (hypertension with complications [CCS 99]), and splitting 2 diagnosis condition categories into acute and nonacute ICD‐9‐CM codes (pancreatic disorders [CCS 149] and biliary tract disease [CCS 152]). Detailed rationales for each modification to the planned readmission algorithm are described in Table 4. We felt further examination of diagnostic cardiac catheterization and cardiac devices was warranted given their high frequency, despite low positive predictive value. We also elected not to alter the categorization of amputation or debridement because it was not easy to determine whether these admissions were planned or unplanned even with chart review. We plan further analyses of these procedure categories.
Action | Diagnosis or Procedure Category | Algorithm | Chart | N | Rationale for Change |
---|---|---|---|---|---|
| |||||
Remove from planned procedure list | Therapeutic radiation (CCS 211) | Accurate | The algorithm was inaccurate in every case. All therapeutic radiology during readmissions was performed because of acute illness (pain crisis, neurologic crisis) or because scheduled treatment occurred during an unplanned readmission. In national data, this ranks as the 25th most common planned procedure identified by the algorithm v2.1. | ||
Planned | Planned | 0 | |||
Unplanned | Unplanned | 0 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 12 | |||
Cancer chemotherapy (CCS 224) | Accurate | Of the 22 correctly identified as planned, 18 (82%) would already have been categorized as planned because of a principal diagnosis of maintenance chemotherapy. Therefore, removing CCS 224 from the planned procedure list would only miss a small fraction of planned readmissions but would avoid a large number of misclassifications. In national data, this ranks as the 8th most common planned procedure identified by the algorithm v2.1. | |||
Planned | Planned | 22 | |||
Unplanned | Unplanned | 0 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 18 | |||
Add to planned procedure list | None | The abstractors felt a planned readmission was missed by the algorithm in 15 cases. A handful of these cases were missed because the planned procedure was not on the current planned procedure list; however, those procedures (eg, abdominal paracentesis, colonoscopy, endoscopy) were nearly always unplanned overall and should therefore not be added as procedures that potentially qualify as an admission as planned. | |||
Remove from acute diagnosis list | None | The abstractors felt a planned readmission was missed by the algorithm in 15 cases. The relevant disqualifying acute diagnoses were much more often associated with unplanned readmissions in our dataset. | |||
Add to acute diagnosis list | Hypertension with complications (CCS 99) | Accurate | This CCS was associated with only 1 planned readmission (for elective nephrectomy, a very rare procedure). Every other time this CCS appeared in the dataset, it was associated with an unplanned readmission (12/13, 92%); 10 of those, however, were misclassified by the algorithm as planned because they were not excluded by diagnosis (91% error rate). Consequently, adding this CCS to the acute diagnosis list is likely to miss only a very small fraction of planned readmissions, while making the overall algorithm much more accurate. | ||
Planned | Planned | 1 | |||
Unplanned | Unplanned | 2 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 10 | |||
Split diagnosis condition category into component ICD‐9 codes | Pancreatic disorders (CCS 152) | Accurate | ICD‐9 code 577.0 (acute pancreatitis) is the only acute code in this CCS. Acute pancreatitis was present in 2 cases that were misclassified as planned. Clinically, there is no situation in which a planned procedure would reasonably be performed in the setting of acute pancreatitis. Moving ICD‐9 code 577.0 to the acute list and leaving the rest of the ICD‐9 codes in CCS 152 on the nonacute list will enable the algorithm to continue to identify planned procedures for chronic pancreatitis. | ||
Planned | Planned | 0 | |||
Unplanned | Unplanned | 1 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 2 | |||
Biliary tract disease (CCS 149) | Accurate | This CCS is a mix of acute and chronic diagnoses. Of 14 charts classified as planned with CCS 149 in the principal diagnosis field, 12 were misclassified (of which 10 were associated with cholecystectomy). Separating out the acute and nonacute diagnoses will increase the accuracy of the algorithm while still ensuring that planned cholecystectomies and other procedures can be identified. Of the ICD‐9 codes in CCS 149, the following will be added to the acute diagnosis list: 574.0, 574.3, 574.6, 574.8, 575.0, 575.12, 576.1. | |||
Planned | Planned | 2 | |||
Unplanned | Unplanned | 3 | |||
Inaccurate | |||||
Unplanned | Planned | 0 | |||
Planned | Unplanned | 12 | |||
Consider for change after additional study | Diagnostic cardiac catheterization (CCS 47) | Accurate | The algorithm misclassified as planned 25/38 (66%) unplanned readmissions in which diagnostic catheterizations were the only qualifying planned procedure. It also correctly identified 3/3 (100%) planned readmissions in which diagnostic cardiac catheterizations were the only qualifying planned procedure. This is the highest volume procedure in national data. | ||
Planned | Planned | 3* | |||
Unplanned | Unplanned | 13* | |||
Inaccurate | |||||
Unplanned | Planned | 0* | |||
Planned | Unplanned | 25* | |||
Insertion, revision, replacement, removal of cardiac pacemaker or cardioverter/defibrillator (CCS 48) | Accurate | The algorithm misclassified as planned 4/5 (80%) unplanned readmissions in which cardiac devices were the only qualifying procedure. However, it also correctly identified 7/8 (87.5%) planned readmissions in which cardiac devices were the only qualifying planned procedure. CCS 48 is the second most common planned procedure category nationally. | |||
Planned | Planned | 7 | |||
Unplanned | Unplanned | 1 | |||
Inaccurate | |||||
Unplanned | Planned | 1 | |||
Planned | Unplanned | 4 |
The revised algorithm (v3.0) had a weighted sensitivity of 49.8%, weighted specificity of 96.5%, positive predictive value of 58.7%, and negative predictive value of 94.5% (Table 2). In aggregate, these changes would increase the reported unplanned readmission rate from 16.0% to 16.1% in the hospital‐wide readmission measure, using 2011 to 2012 data, and would decrease the fraction of all readmissions considered planned from 7.8% to 7.2%.
DISCUSSION
We developed an algorithm based on administrative data that in its currently implemented form is very accurate at identifying unplanned readmissions, ensuring that readmissions included in publicly reported readmission measures are likely to be truly unplanned. However, nearly half of readmissions the algorithm classifies as planned are actually unplanned. That is, the algorithm is overcautious in excluding unplanned readmissions that could have counted as outcomes, particularly among admissions that include diagnostic cardiac catheterization or placement of cardiac devices (pacemakers, defibrillators). However, these errors only occur within the 7.8% of readmissions that are classified as planned and therefore do not affect overall readmission rates dramatically. A perfect algorithm would reclassify approximately half of these planned readmissions as unplanned, increasing the overall readmission rate by 0.6 percentage points.
On the other hand, the algorithm also only identifies approximately half of true planned readmissions as planned. Because the true prevalence of planned readmissions is low (approximately 9% of readmissions based on weighted chart review prevalence, or an absolute rate of 1.4%), this low sensitivity has a small effect on algorithm performance. Removing all true planned readmissions from the measure outcome would decrease the overall readmission rate by 0.8 percentage points, similar to the expected 0.6 percentage point increase that would result from better identifying unplanned readmissions; thus, a perfect algorithm would likely decrease the reported unplanned readmission rate by a net 0.2%. Overall, the existing algorithm appears to come close to the true prevalence of planned readmissions, despite inaccuracy on an individual‐case basis. The algorithm performed best at large hospitals, which are at greatest risk of being statistical outliers and of accruing penalties under the Hospital Readmissions Reduction Program.[10]
We identified several changes that marginally improved the performance of the algorithm by reducing the number of unplanned readmissions that are incorrectly removed from the measure, while avoiding the inappropriate inclusion of planned readmissions in the outcome. This revised algorithm, v3.0, was applied to public reporting of readmission rates at the end of 2014. Overall, implementing these changes increases the reported readmission rate very slightly. We also identified other procedures associated with high inaccuracy rates, removal of which would have larger impact on reporting rates, and which therefore merit further evaluation.
There are other potential methods of identifying planned readmissions. For instance, as of October 1, 2013, new administrative billing codes were created to allow hospitals to indicate that a patient was discharged with a planned acute‐care hospital inpatient readmission, without limitation as to when it will take place.[11] This code must be used at the time of the index admission to indicate that a future planned admission is expected, and was specified only to be used for neonates and patients with acute myocardial infarction. This approach, however, would omit planned readmissions that are not known to the initial discharging team, potentially missing planned readmissions. Conversely, some patients discharged with a plan for readmission may be unexpectedly readmitted for an unplanned reason. Given that the new codes were not available at the time we conducted the validation study, we were not able to determine how often the billing codes accurately identified planned readmissions. This would be an important area to consider for future study.
An alternative approach would be to create indicator codes to be applied at the time of readmission that would indicate whether that admission was planned or unplanned. Such a code would have the advantage of allowing each planned readmission to be flagged by the admitting clinicians at the time of admission rather than by an algorithm that inherently cannot be perfect. However, identifying planned readmissions at the time of readmission would also create opportunity for gaming and inconsistent application of definitions between hospitals; additional checks would need to be put in place to guard against these possibilities.
Our study has some limitations. We relied on the opinion of chart abstractors to determine whether a readmission was planned or unplanned; in a few cases, such as smoldering wounds that ultimately require surgical intervention, that determination is debatable. Abstractions were done at local institutions to minimize risks to patient privacy, and therefore we could not centrally verify determinations of planned status except by reviewing source of admission, dates of procedures, and narrative comments reported by the abstractors. Finally, we did not have sufficient volume of planned procedures to determine accuracy of the algorithm for less common procedure categories or individual procedures within categories.
In summary, we developed an algorithm to identify planned readmissions from administrative data that had high specificity and moderate sensitivity, and refined it based on chart validation. This algorithm is in use in public reporting of readmission measures to maximize the probability that the reported readmission rates represent truly unplanned readmissions.[12]
Disclosures: Financial supportThis work was performed under contract HHSM‐500‐2008‐0025I/HHSM‐500‐T0001, Modification No. 000008, titled Measure Instrument Development and Support, funded by the Centers for Medicare and Medicaid Services (CMS), an agency of the US Department of Health and Human Services. Drs. Horwitz and Ross are supported by the National Institute on Aging (K08 AG038336 and K08 AG032886, respectively) and by the American Federation for Aging Research through the Paul B. Beeson Career Development Award Program. Dr. Krumholz is supported by grant U01 HL105270‐05 (Center for Cardiovascular Outcomes Research at Yale University) from the National Heart, Lung, and Blood Institute. No funding source had any role in the study design; in the collection, analysis, and interpretation of data; or in the writing of the article. The CMS reviewed and approved the use of its data for this work and approved submission of the manuscript. All authors have completed the Unified Competing Interest form at
- Development, validation, and results of a measure of 30‐day readmission following hospitalization for pneumonia. J Hosp Med. 2011;6(3):142–150. , , , et al.
- An administrative claims measure suitable for profiling hospital performance based on 30‐day all‐cause readmission rates among patients with acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2011;4(2):243–252. , , , et al.
- An administrative claims measure suitable for profiling hospital performance on the basis of 30‐day all‐cause readmission rates among patients with heart failure. Circ Cardiovasc Qual Outcomes. 2008;1:29–37. , , , et al.
- http://www.qualitynet.org/dcs/ContentServer?c=Page161(supp10 l):S66–S75. , , , et al. Hospital‐level 30‐day all‐cause risk‐standardized readmission rate following elective primary total hip arthroplasty (THA) and/or total knee arthroplasty (TKA). Available at:
- A meta‐analysis of hospital 30‐day avoidable readmission rates. J Eval Clin Pract. 2011;18(6):1211–1218. , ,
- Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ. 2011;183(7):E391–E402. , , , ,
- 3(4):477–492. , , , et al. Centers for Medicare
- Characteristics of hospitals receiving penalties under the Hospital Readmissions Reduction Program. JAMA. 2013;309(4):342–343. ,
- Centers for Medicare and Medicaid Services. Inpatient Prospective Payment System/Long‐Term Care Hospital (IPPS/LTCH) final rule. Fed Regist. 2013;78:50533–50534.
- Massachusetts health reforms: uninsurance remains low, self‐reported health status improves as state prepares to tackle costs. Health Aff (Millwood). 2012;31(2):444–451. , ,
- Development, validation, and results of a measure of 30‐day readmission following hospitalization for pneumonia. J Hosp Med. 2011;6(3):142–150. , , , et al.
- An administrative claims measure suitable for profiling hospital performance based on 30‐day all‐cause readmission rates among patients with acute myocardial infarction. Circ Cardiovasc Qual Outcomes. 2011;4(2):243–252. , , , et al.
- An administrative claims measure suitable for profiling hospital performance on the basis of 30‐day all‐cause readmission rates among patients with heart failure. Circ Cardiovasc Qual Outcomes. 2008;1:29–37. , , , et al.
- http://www.qualitynet.org/dcs/ContentServer?c=Page161(supp10 l):S66–S75. , , , et al. Hospital‐level 30‐day all‐cause risk‐standardized readmission rate following elective primary total hip arthroplasty (THA) and/or total knee arthroplasty (TKA). Available at:
- A meta‐analysis of hospital 30‐day avoidable readmission rates. J Eval Clin Pract. 2011;18(6):1211–1218. , ,
- Proportion of hospital readmissions deemed avoidable: a systematic review. CMAJ. 2011;183(7):E391–E402. , , , ,
- 3(4):477–492. , , , et al. Centers for Medicare
- Characteristics of hospitals receiving penalties under the Hospital Readmissions Reduction Program. JAMA. 2013;309(4):342–343. ,
- Centers for Medicare and Medicaid Services. Inpatient Prospective Payment System/Long‐Term Care Hospital (IPPS/LTCH) final rule. Fed Regist. 2013;78:50533–50534.
- Massachusetts health reforms: uninsurance remains low, self‐reported health status improves as state prepares to tackle costs. Health Aff (Millwood). 2012;31(2):444–451. , ,
© 2015 Society of Hospital Medicine
Impact of Elderly Inpatient Care Bundle
Elderly patients (aged 65 years and older) consume a disproportionate amount of acute health care resources, composing up to 20% of emergency department (ED) visits,1, 2 having a 2‐fold to 5‐fold increase in likelihood of hospital admission,1 and frequently incurring lengths of hospital stay (LOS) approximately 15% higher than the national averages.3 In addition, they are at increased risk for hospital readmission in the 90‐day interval following hospital discharge.1, 4, 5 Specific risk factors for readmission include age above 80 years, discharge within the previous 30 days, the presence of 3 or more comorbid diagnoses, use of 5 or more prescription medications, difficulty with at least 1 activity of daily living (ADL), and lack of discharge education.6 These risk factors can translate into adverse drug events,79 exacerbations of chronic diseases,10 or functional decline4, 5 that can trigger ED visits or hospital readmission.
Hospital‐based care coordinationdefined as a multidisciplinary interaction between inpatients and providers that focuses on education, communication, and discharge planning with the primary aim of improving outcomeshas demonstrated inconsistent results as a mechanism to reduce LOS, postdischarge ED visits, or hospital readmission rates. While disease‐specific care coordination programs for congestive heart failure and chronic obstructive pulmonary disease have been effective in reducing rehospitalization rates,1015 the benefits of comprehensive care coordination for elderly general medical inpatients with a broader range of diagnoses are less clear. In a group of 750 elderly patients with 1 of 11 common inpatient diagnoses (such as stroke or hip fracture) likely to ultimately require a high level of home support, Coleman et al.16 found that a structured transitional care program centered on a personal coach decreased rehospitalization rates at 30 and 90 days. Preen et al.17 found improved patient involvement and perceived quality of life with care coordination focused on discharge planning, but no impact on LOS. Likewise, a recent meta‐analysis18 failed to demonstrate statistically significant differences in mortality, LOS, or readmission rates in hospitalized patients who received intensive care coordination versus usual care; however, variation in the components of the care coordination intervention and reported outcomes restricted the ability to pool data in this study.
Care coordination programs demonstrating efficacy in reducing health care utilization in elderly medical patients have generally included an outpatient transitional component with out‐of‐hospital postacute care visits by health care personnel such as a nurse, pharmacist, or physician.1923 These offsite interventions generate additional expenses and resource demands that may not be practical for smaller hospitals to implement.24, 25 In contrast, hospital‐based care coordination programs have clear ownership and thus may be more practical to disseminate. Individual elements of hospital‐based care coordination such as pharmacist counseling, discharge education, and telephone follow‐up have been shown to reduce ED visitation and readmission rates in high‐risk elderly patients. Less information is available regarding the impact of these interventions delivered in an aggregate bundle by hospital staff in the absence of bridging transitional visits.2629
The objective of this pilot study was to determine whether a supplemental elderly care bundle, targeted to high‐risk inpatients by hospital staff as an enhancement to existing care coordination, would affect postdischarge readmission and ED visit rates. The intervention was designed to capitalize on existing resources, and focused specifically on elderly inpatients who were hospitalized with diagnoses commonly encountered in a general medical unit and predisposed to recidivism.
Patients and Methods
Patient Selection and Enrollment
The screening population consisted of elderly patients admitted to 1 of 2 hospital‐medicine groups (MedProvider Inpatient Care Unit or Texas Primary Care) at the 900‐bed Baylor University Medical Center (BUMC, Dallas, TX) between March and June 2007 with a diagnosis likely to fall within 1 of 20 frequent Medicare medical diagnosis reimbursement groups (DRGs) at BUMC, as listed in Table 1. Study personnel performed daily chart review to establish eligibility criteria, which included age 70 years, use of 5 medications regularly, 3 chronic comorbid conditions, requirement for assistance with 1 ADL, and preadmission residence at home or assisted living with a reasonable expectation of disposition back to that domicile. These criteria were based on factors found in the literature to be associated with extended LOS and postdischarge readmission/ED visit events.5, 6, 3032 Potential enrollees needed to be conversant in English (a multilingual staff was not feasible due to limited resources for this pilot project) and have reliable phone contact, or have a proxy caregiver who could speak English and be reached by phone. Predefined exclusion criteria were admission primarily for a surgical procedure, terminal diagnosis with life expectancy 6 months, residence in a long‐term care facility (long‐term acute care [LTAC], skilled nursing facility [SNF], or nursing home) prior to hospitalization with anticipated discharge back to that facility, and patient/family refusal to participate. Additionally, with an average LOS between 5 and 6 days in BUMC's Medicare population for the DRGs of interest, it was felt that the effects of the care bundle intervention would be obscured unless initiated early in the hospitalization. Thus, patients who could not be enrolled within 72 hours following admission were subsequently excluded. The Baylor Health Care System (BHCS) Institutional Review Board (IRB) approved this study, and written informed consent was obtained from all patients or their surrogates.
DRG | DRG Name |
---|---|
| |
127 | Heart failure and shock |
14 | Intracranial hemorrhage or cerebral infarction |
89 | Simple pneumonia/pleurisy |
416 | Septicemia |
316 | Renal failure |
182 | Esophagitis/gastroenterological/miscellaneous digestive disorders with complications |
174 | Gastrointestinal hemorrhage with complications |
88 | Chronic obstructive pulmonary disease |
320 | Kidney/urinary tract infection with complications |
144 | Other circulatory diagnoses with complications |
138 | Arrhythmia/conduction disorders with complications |
277 | Cellulitis with complications |
124 | Circulatory disorders except acute myocardial infarction with cardiac catheterization and complex diagnosis |
430 | Psychoses |
188 | Other digestive diagnoses with complications |
395 | Red blood cell disorders |
79 | Respiratory infections and inflammations with complications |
524 | Transient ischemia |
143 | Chest pain |
141 | Syncope and collapse with complications |
Patients meeting eligibility criteria were approached within 72 hours of admission for study participation. After consent and enrollment, patients were randomized to intervention or usual care arms in permuted blocks of 8 via a random number generator and sealed opaque envelopes. Nursing and care coordination staff providing usual care to patients (independent of the research team) were blinded to the treatment group status of enrollees; trial design precluded blinding of study personnel and patients.
Delivery of the Supplemental Care Bundle
Starting no later than 24 hours after enrollment and continuing up to 1 week following hospital discharge, intervention group patients received a targeted care bundle provided by 1 of 3 care coordinators (CCs) and 1 of 4 clinical pharmacists (CPs) working with the study team. The care bundle was designed as an intensive patient‐centered educational program that would augment BUMC's existing care coordination processes (delivered to all patients regardless of study participation); specific elements are displayed in Figure 1. Study CCs saw patients daily throughout their hospital stay, and instructed patients on specific health conditions, with an emphasis on optimizing home self‐care and contingency plans if problems arose. CP visits focused on medication reconciliation and education regarding any new agents started during the hospitalization. The personal health record (PHR) provided a tool to engage patients in self‐care, and as discussed by Coleman et al.,7, 16, 33 promoted information transfer from the hospital to outpatient settings. During the postdischarge phone call, CCs followed a basic script to confirm receipt of medical equipment, medications, home health arrangements, and scheduling of follow‐up appointments. They also used this contact as an opportunity to reinforce patient education on managing their conditions. CPs reviewed medication use (type, schedule, dose), and spoke with patients about any symptoms they may have experienced as medication side effects. If indicated based on their phone discussions, both CCs and CPs could recommend an action plan to the patient.
The study CCs and CPs were existing hospital staff and performed their research activities in addition to their usual duties. Study CCs were highly experienced (averaging 8 years of inpatient floor nursing plus 10 years as CCs) and all had advanced nursing certifications (ACM, BSN, or MSN). The CPs were upper‐level pharmacy residents completing their inpatient clinical rotations. Additional training for both study CCs and CPs was limited to a series of 3 meetings (each 45 minutes in duration) regarding the intent and delivery of the supplemental care bundle, including use of study forms.
At the time of the trial, the particular CCs and CPs chosen to deliver the supplemental care bundle had work assignments ensuring that crossover between intervention and usual care groups would not occur. For example, 1 of the study CCs normally covered a surgical floor such that her normal scope of responsibilities would not influence the medical patients in the study (their baseline care coordination was provided by nonstudy personnel). Medication reconciliation and medication education is generally performed by floor nursing staff rather than CPs at BUMC.
Data Collection and Outcomes Measurement
Following enrollment, demographic information and a basic medical history were documented by research staff. Inaccuracies in medication lists discovered by pharmacists during the medication reconciliation process were entered directly into the universal medication list on the hospital chart. CPs also kept a log of the medication education given to patients (and recommendations for changes to patients' regimens given to physicians) throughout their hospital stay. Study CCs recorded their assessments of patient needs and associated responses. Furthermore, the research team CC prepared an enhanced discharge form that was given to intervention patients in addition to BUMC's standard form. Data on LOS, illness severity (APR‐DRGs), and unplanned hospital readmission or ED visitation at 30 and 60 days postdischarge were collected via BUMC's electronic reporting systems. All patient follow‐up was completed as of September 1, 2007.
Statistical Analyses
Resource and time constraints necessitated a sample size that would allow implementation of the intervention despite a limited number of study CCs and pharmacists. To accommodate these conditions while still generating pilot data, an a priori decision was made to enroll up to 80 patients. Continuous data variables were normally distributed. Differences between groups for continuous variables were assessed with the Student t‐test; differences in proportions between groups were compared with Fisher's exact tests. Time to readmission events between the groups were evaluated in a post hoc manner using the log‐rank test. Data were analyzed using Prism version 5 for Windows (GraphPad Software, Inc., San Diego, CA) and SPSS version 15 for Windows (SPSS Inc., Chicago, IL). P values < 0.05 were considered statistically significant.
Results
The final sample size for this pilot was small, with 41 total patients (21 controls, 20 interventions). The main reason for enrollment failure of patients meeting study criteria was an inability to obtain informed consent. Sixty patients declined participation after being approached, and another 56 patients were unable to give their informed consent due to impairments (poor cognition, medication induced sedation, severity of illness) with lack of an available proxy to give written consent during the 72‐hour postadmission recruitment window. There were no statistically significant differences in the baseline characteristics of the intervention and control groups (Table 2). A similar proportion of patients (23% in the intervention, 15% in controls; P = 0.70) had preexisting diagnoses of dementia or depression. However, on APR‐DRG measures relating to acuity of illness and mortality risk, patients in the intervention group trended toward higher severity (Table 2). Likewise, although it was not a statistically significant difference, 13 of 20 patients in the intervention group were taking medications from 2 drug classes commonly implicated in adverse drug events (warfarin, insulin, diuretics, sedating agents) as part of their discharge medication regimen compared to 10 of 21 patients in the control group.
Control (n = 21) | Intervention (n = 20) | P Value | |
---|---|---|---|
Age in years (mean SD) | 79.8 5.6 | 77.2 5.3 | 0.14 |
Males, n (%) | 8 (38) | 3 (15) | 0.10 |
Females, n (%) | 13 (62) | 17 (85) | 0.10 |
Race, n (%) | |||
African‐American | 3 (14) | 5 (25) | 0.45 |
Asian | 0 (0) | 1 (5) | 0.49 |
Caucasian | 17 (81) | 14 (70) | 0.48 |
Hispanic | 1 (5) | 0 | 1.0 |
Preadmission living status, n (%) | |||
Alone | 6 (29) | 4 (20) | 0.72 |
With spouse or other family | 11 (52) | 15 (75) | 0.20 |
Assisted living | 4 (19) | 1 (5) | 0.34 |
Inpatient medications (mean SD) | 11 3 | 12 5 | 0.18 |
Charlson score (mean, SD) | 3.2 1.3 | 3.7 1.1 | 0.21 |
% with APR DRG severity rating 3 | 57.5 | 83.3 | 0.12 |
% with APR DRG mortality rating 3 | 20.0 | 55.6 | 0.07 |
Primary admission diagnoses (n cases, in order of frequency) | 3 pneumonia | 3 pneumonia | |
3 CHF | 3 syncope | ||
2 syncope | 2 CHF | ||
2 COPD | 2 COPD | ||
2 cellulitis | 2 cellulitis | ||
2 GI disorder (nonbleed) | 2 GI disorder (nonbleed) | ||
2 GI bleed | 1 GI bleed | ||
2 UTI | 1 atrial fibrillation | ||
1 atrial fibrillation | 1 encephalopathy | ||
1 stroke | 1 TIA | ||
1 renal failure | 1 renal failure | ||
1 volume depletion |
Study outcomes are displayed in Table 3. Mean LOS is reported as a descriptive finding; there was insufficient power to compare this outcome statistically between groups. The majority of patients were discharged to home. A similar proportion of patients in the intervention (20%) and control groups (22%) who had lived at home immediately prior to admission were discharged from the hospital to skilled care facilities (P = 0.87). The number of readmissions/ED visits (taken as a composite measure of unplanned healthcare utilization) within 30 days of discharge was lower in the intervention group; by 60 days, there was no longer a statistically significant difference in readmission/ED visit rates between groups. For those patients who had a readmission or ED visit following hospital discharge, the intervention group had a longer time interval to first event compared to controls (36.2 versus 15.7 days, P = 0.05). Of the patients discharged to skilled care, 1 in the intervention group (at 53 days) and 1 in the control group (at 16 days) had a readmission/ED visit event. Figure 2 shows time‐to‐first readmission or ED visit event curves at 30 and 60 days for both intervention and control groups. For patients who had a readmission/ED visit event, LOS for this episode was 2.2 2.1 days in controls and 3.7 2.1 days in the intervention group (insufficient power for statistical comparison). The study's small sample size prevented development of a meaningful regression model.
Outcome Measure | Control (n = 21) | Intervention (n = 20) | P Value |
---|---|---|---|
| |||
Length of stay for index hospitalization (days)* | 4.7 3.7 | 6.2 4.1 | |
0‐30 day postdischarge readmissions/ED visits | 8 (38%) | 2 (10%) | 0.03 |
31‐60 day postdischarge readmissions/ED visits | 1 (5%) | 4 (20%) | 0.18 |
Total postdischarge readmissions/ED visits at 60 days | 9 | 6 | 0.52 |
Resource utilization and the specifics of patient‐study personnel interaction associated with the intervention were tracked. Research assistants spent an average of 50 minutes daily screening charts for potential candidates. For the 20 patients who received the supplemental elderly care bundle, study CCs averaged 20 to 25 minutes per patient daily of additional time counseling patients and families, identifying and attending to discharge barriers, filling out documentation, and faxing the supplemental study discharge form to the patient's primary care physician. Any residual home care needs or issues unresolved at discharge were addressed with the patient in the 5 to 7 day follow‐up phone call. Similarly, study CPs expended approximately 20 minutes daily per patient providing medication education, reconciliation, and optimization of drug therapy. Study pharmacists recommended a change to the medication regimens of 10 patients in the intervention group; physicians acted upon these recommendations for 7 of the patients. The changes included dosage adjustment, discontinuation of medications due to possible drug interaction or duplication of drugs with the same pharmacologic effect, and addition of medications as indicated by patient condition or to reconcile with patients' at‐home medication regimens. Patients contacted via phone by the study pharmacist within 1 week after discharge were able to describe proper use of new medications started in the hospital and confirm that they obtained or had the means to obtain the prescribed drugs.
Discussion
This pilot study examined the effects of a supplemental care bundle involving patient education and discharge planning delivered by hospital‐based CCs and CPs on the rate of readmission/ED visitation in 41 elderly (70 years of age) patients. The study was not adequately powered to detect an impact of the intervention on index LOS. The care bundle did lead to significantly fewer readmissions or ED visits 30 days postdischarge and appeared to increase the time interval to first unplanned readmission or ED visit compared to usual care. This effect was no longer present at 60 days postdischarge. Resource allocations and scope of duties for CCs and CPs (an average of 20 minutes per patient per day) related to delivering the intervention were realistic for broader implementation in the hospitalized elderly population at high risk for readmission or ED visitation following discharge.
Length of stay for the initial hospitalization associated with the care bundle was an original outcome of interest to the study team. However, with the final enrollment of 41 patients and a power of 0.8, the between group difference would have needed to be 2.6 days to be statistically significant. It is likely that any change in LOS related to the care bundle would be much smaller, particularly since 2 key determinants of LOS, severity of illness and physician behavior, were beyond this patient education‐oriented intervention's scope of influence.3437 Furthermore, the diverse range of eligible diagnoses limited the study CCs' ability to reduce variability through use of clinical care pathways. One approach in leveraging an elderly care bundle to reduce LOS may be to focus on a specific disease that has well‐established inpatient benchmarks and treatment algorithms. For example, in patients with community‐acquired pneumonia, the use of care coordination in combination with standardized order sets decreased LOS without compromising safety, mainly by shortening the time from clinical stability to discharge.38
On separation of the readmission/ED visit outcome into 30 and 60 day postdischarge time frames, the intervention group had a lower rate of unplanned acute health care use within 30 days postdischarge; the difference between groups had dissipated by 60 days postdischarge. This convergence suggests that a hospital‐based intervention's influence is strongest closer to the time of the initial hospital stay, and wanes as more time has elapsed. Indeed, interventions that have successfully maintained lower readmission rates beyond 60 and 90 days postdischarge in a high‐risk elderly population (such as the program advocated by Coleman et al.16) have included a transitional care provider engaging patients during the hospitalization and performing subsequent visits to the home or nursing facility.33 An optimal intervention would capitalize on the hospital‐based staff's ability to improve short‐term readmission/ED visit rates while linking patients to longer‐term transitional care to extend these outcomes. Electronic health records could potentially facilitate these care transitions, beginning with an automated screening process for identification of high‐risk inpatients and continuing through postdischarge follow‐up. How to develop these resources in settings where outpatient practices are independent or only loosely affiliated with hospitals is an area for continued investigation.
In a group of elderly patients with multiple comorbidities and complex pharmacotherapy regimens, the study bundle component targeting medication management appears to be a high‐yield intervention to reduce unplanned health care utilization following hospital discharge. These patients are more susceptible to nonadherence and drug‐related adverse events, which may contribute to hospital readmission or ED visitation.7, 9, 39 Consistent with findings at other sites,28, 40 a heightened level of CP involvement in the care of high‐risk elderly patients may have helped reduce these undesirable outcomes. Of the 9 readmission/ED visit events in the control group, 3 were attributable to medication related complications (2 from sedatives, 1 from a diuretic). None of the readmission/ED visit events in intervention group patients stemmed from medication effects.
Correspondingly, the research CCs' provision of daily condition‐specific education, additional time to more thoroughly investigate discharge needs, engagement of patients' families as active partners in self‐care, and the use of a structured discharge form along with follow‐up phone calls may have better prepared patients to manage their health problems once released from the hospital.26, 28, 29 For example, 1 patient in the control group was readmitted less than 24 hours after initial discharge due to inability to perform self‐care at home. Given the study power issues described previously, data on LOS for the second hospitalization for patients who had a readmission event are difficult to interpret, but could suggest the occurrence of some shorter, preventable readmissions in the control group. Conversely, the readmission/ED visit events in intervention patients appeared to be associated with a specific medical condition (eg, failure of diabetic cellulitis to respond to appropriate outpatient treatment) rather than problems that would have been corrected with an educational/self‐management program such as this targeted care bundle.
This pilot study had several limitations. The main issue was a small patient sample size that was primarily due to an inability to obtain informed consent. Design of the study as a randomized controlled trial and plans to disseminate study findings beyond BHCS necessitated IRB approval rather than delivery of the supplemental care bundle as a quality improvement (QI) project. Placing QI initiatives under research regulations can lead to project delays, higher costs, and patient frustrations with the process.41, 42 This tension was evident during study screening and enrollment, as many patients who otherwise met criteria and would potentially benefit from the intervention were hesitant to participate in a research study or refused to sign a multipage consent document. The difficulties of enrolling elderly patients in clinical trials have been well‐described.43, 44 Further research involving a minimal‐risk, educational intervention such as this elderly care‐bundle would likely better fit under the category of expedited IRB review with waiver or modification of the informed consent process.45
Incomplete blinding could have potentially affected our results. At the study site, the team members delivering the care bundle were a regular part of the hospital staff (as opposed to external researchers), and it is not unusual for a CC or a pharmacist to enter a patient's room (eg, to confirm a drug allergy history). In view of this, the impact of imperfect blinding on 30‐day outcomes would likely be minimal. Furthermore, a floor staff perception that a specific patient was being taken care of by the study team resulting in a lower than usual level of care, would tend to bias the result of the intervention toward the null effect.
vThe study cohort did not have enough subjects to perform analyses (ie, modeling or examination of subgroups) beyond basic comparative findings. Issues such as preadmission living situation and the presence of depression or cognitive impairment (Mini‐Mental Status Exams were not performed on these patients) may potentially influence postdischarge recidivism; their effects can not be reliably ascertained from these data. Additionally, to prevent study personnel from engaging patients who would soon be going home, it was felt that the benefits from the care bundle would be recognized only if the intervention could be initiated within 72 hours of admission and delivered in full, a requirement that further reduced the enrollment pool. The intent of this pilot work was to guide future investigations surrounding hospital‐to‐home transitional care. The next phase of research in this area will need an enhanced sample size with more extensive baseline data collection so that potential confounding factors or outcomes in specific populations can be explored.
Another problem restricting applicability of study findings was the use of only 3 different CCs and 3 pharmacists on the research team to deliver the components of patient education, discharge planning, and medication counseling in the elderly care bundle. Personnel for the trial were chosen for their experience and interest in the area of care transitions. To distinguish the benefit of the elderly care bundle in general versus the expertise of these particular CCs and study pharmacists, a larger‐scale, multisite trial would be necessary. Lastly, due to resource constraints, patients who resided in long‐term care (either LTAC, SNFs, or nursing homes) prior to admission with anticipated return to those sites were not eligible for the study. Similar to the patients whose comorbidities or acute severity of illness prevented informed consent, this segment of the elderly population may have derived even more benefit from receipt of the elderly care bundle.10, 15, 46 Despite exclusion of this group (which would be expected to lessen the impact of the intervention), a difference in readmission/ED visits rates at 30 days following discharge was observed.
Conclusions
This pilot randomized clinical trial (RCT) evaluated the effects of a supplemental, aggregate care bundle centered on patient education, discharge planning, and medication counseling and reconciliation compared to usual care in a group of elderly patients at high risk of readmission or ED visitation following an index hospitalization. The intervention was designed to be reproducible and make use of existing hospital resources. Probably through facilitation of patient self‐care and home management, the elderly care bundle reduced the composite outcome of readmission/ED visits at 30 days postdischarge. By 60 days, this effect had waned, demonstrating the short‐term benefit of a hospital‐based educational intervention and stressing the need to incorporate additional outpatient transitional care support to sustain favorable outcomes. The study was not powered to detect small differences (which would be more likely than a change of multiple days) in length of index hospital stay related to the care bundle. There were important study limitations (primarily associated with small sample size), and this work should be viewed as hypothesis‐generating. Future trials should assess the impact of a standardized targeted care bundle delivered across multiple healthcare systems on a larger cohort of high‐risk elderly patients, including analysis of financial and personnel allocations relative to the benefits of the intervention.
Acknowledgements
The authors thank study pharmacists Kristen Hesch (PharmD), Renee Danysh (PharmD), Rema Thyagarajan (PharmD), and Betina Thomas (PharmD) for providing patients with medication education and conducting medication reconciliation. They also thank Jeanne Bradbury (RN, ACM), Diana Davis (RN, BSN), and Gail McVea (RN, MSN) for their involvement as care coordinators; Veronica Odom (RN) for her contributions as a research nurse; and Marilyn Callies (RN, MBA) for her role as project advisor.
- Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions.Ann Emerg Med.2002;39(3):238–247. , .
- Emergency department utilization by the elderly: analysis of the National Hospital Ambulatory Medical Care Survey.Acad Emerg Med.1996;3(7):694–699. , , , , , .
- 2005 National Hospital Discharge Survey.Adv Data.2007(385):1–19. , .
- Short‐term outcomes of elderly patients discharged from an emergency department.J Am Geriatr Soc.1989;37(10):937–943. , , , , .
- The discharge of elderly patients from an accident and emergency department: functional changes and risk of readmission.Age Ageing.1990;19(6):415–418. , , , , .
- Factors associated with unplanned hospital readmission among patients 65 years of age and older in a Medicare managed care plan.Am J Med.1999;107(1):13–17. , , , , , .
- Posthospital medication discrepancies: prevalence and contributing factors.Arch Intern Med.2005;165(16):1842–1847. , , , .
- From the emergency department to home.J Clin Nurs.2005;14(6):776–785. , .
- Adverse drug events in elderly patients receiving home health services following hospital discharge.Ann Pharmacother.1999;33(11):1147–1153. , , .
- A randomized trial of the efficacy of multidisciplinary care in heart failure outpatients at high risk of hospital readmission.J Am Coll Cardiol.2002;39(3):471–480. , , , et al.
- A systematic review of randomized trials of disease management programs in heart failure.Am J Med.2001;110(5):378–384. , , , .
- Comprehensive discharge planning with postdischarge support for older patients with congestive heart failure: a meta‐analysis.JAMA.2004;291(11):1358–1367. , , , , , .
- Case management may reduce length of hospital stay in patients with recurrent admissions for chronic obstructive pulmonary disease.Respirology.2001;6(1):37–42. , , , .
- A chronic disease management programme can reduce days in hospital for patients with chronic obstructive pulmonary disease.Intern Med J.2004;34(11):608–614. , , , , , .
- Disease management programmes for older people with heart failure: crucial characteristics which improve post‐discharge outcomes.Eur Heart J.2006;27(5):596–612. , , .
- The care transitions intervention: results of a randomized controlled trial.Arch Intern Med.2006;166(17):1822–1828. , , , .
- Effects of a multidisciplinary, post‐discharge continuance of care intervention on quality of life, discharge satisfaction, and hospital length of stay: a randomized controlled trial.Int J Qual Health Care.2005;17(1):43–51. , , , et al.
- Discharge planning from hospital to home.Cochrane Database Syst Rev.2004(1):CD000313. , , , .
- A systematic review of interventions to improve outcomes for elders discharged from the emergency department.Acad Emerg Med.2005;12(10):978–986. , .
- Transitional care for older adults: a cost‐effective model.LDI Issue Brief.2004;9(6):1–4. .
- Comprehensive discharge planning and home follow‐up of hospitalized elders: a randomized clinical trial.JAMA.1999;281(7):613–620. , , , et al.
- A multidisciplinary intervention to prevent the readmission of elderly patients with congestive heart failure.N Engl J Med.1995;333(18):1190–1195. , , , , , .
- Home‐based intervention in congestive heart failure: long‐term implications on readmission and survival.Circulation.2002;105(24):2861–2866. , .
- Effectiveness of team‐managed home‐based primary care: a randomized multicenter trial.JAMA.2000;284(22):2877–2885. , , , et al.
- Home‐based medication review in older people: is it cost effective?Pharmacoeconomics.2007;25(2):171–180. , , , .
- The value of inpatient pharmaceutical counselling to elderly patients prior to discharge.Br J Clin Pharmacol.2002;54(6):657–664. , , , , .
- Does the addition of a pharmacist transition coordinator improve evidence‐based medication management and health outcomes in older adults moving from the hospital to a long‐term care facility? Results of a randomized, controlled trial.Am J Geriatr Pharmacother.2004;2(4):257–264. , , , , .
- Role of pharmacist counseling in preventing adverse drug events after hospitalization.Arch Intern Med.2006;166(5):565–571. , , , et al.
- An investigation of hospital generated pharmaceutical care when patients are discharged home from hospital.Br J Clin Pharmacol.1997;44(2):163–165. , , , , , .
- The community assessment risk screen (CARS): identifying elderly persons at risk for hospitalization or emergency department visit.Am J Manag Care.2000;6(8):925–933. , , .
- “Risk” factors affecting readmission of the elderly into the health care system.Med Care.1986;24(5):429–437. , , .
- Re‐admission to intensive care: identification of risk factors.Physiother Res Int.2005;10(3):154–163. , , .
- Preparing patients and caregivers to participate in care delivered across settings: the Care Transitions Intervention.J Am Geriatr Soc.2004;52(11):1817–1825. , , , , , .
- Impact of the rapid diagnosis of influenza on physician decision‐making and patient management in the pediatric emergency department: results of a randomized, prospective, controlled trial.Pediatrics.2003;112(2):363–367. , , , , .
- A survey on hospitalised community‐acquired pneumonia in Italy.Monaldi Arch Chest Dis.2006;65(2):82–88. , , , , .
- Determinants of the length of stay in intensive care and in hospital after coronary artery surgery.Br Heart J.1995;73(1):92–98. , , , , .
- Variation in duration of hospital stay between hospitals and between doctors within hospitals.Soc Sci Med.1993;37(6):833–839. , , .
- The impact of standardized order sets and intensive clinical case management on outcomes in community‐acquired pneumonia.Arch Intern Med.2007;167(15):1664–1669. , , , et al.
- Compliance with medication orders among the elderly after hospital discharge.Hosp Formul.1992;27(7):720–724. , , .
- Clinical pharmacists and inpatient medical care: a systematic review.Arch Intern Med.2006;166(9):955–964. , , , .
- Quality improvement and ethical oversight.Ann Intern Med.2007;146(9):680–681. .
- The ethics of using quality improvement methods in health care.Ann Intern Med.2007;146(9):666–673. , , , et al.
- Enrollment of elderly patients in clinical trials for cancer drug registration: a 7‐year experience by the US Food and Drug Administration.J Clin Oncol.2004;22(22):4626–4631. , , .
- Striving to recruit: the difficulties of conducting clinical research on elderly care home residents.J R Soc Med.2007;100(6):258–261. , , , .
- Quality‐improvement research and informed consent.N Engl J Med.2008;358(8):765–767. , .
- An evaluation of the impact of the ventilator care bundle.Nurs Crit Care.2005;10(5):242–246. , , , .
Elderly patients (aged 65 years and older) consume a disproportionate amount of acute health care resources, composing up to 20% of emergency department (ED) visits,1, 2 having a 2‐fold to 5‐fold increase in likelihood of hospital admission,1 and frequently incurring lengths of hospital stay (LOS) approximately 15% higher than the national averages.3 In addition, they are at increased risk for hospital readmission in the 90‐day interval following hospital discharge.1, 4, 5 Specific risk factors for readmission include age above 80 years, discharge within the previous 30 days, the presence of 3 or more comorbid diagnoses, use of 5 or more prescription medications, difficulty with at least 1 activity of daily living (ADL), and lack of discharge education.6 These risk factors can translate into adverse drug events,79 exacerbations of chronic diseases,10 or functional decline4, 5 that can trigger ED visits or hospital readmission.
Hospital‐based care coordinationdefined as a multidisciplinary interaction between inpatients and providers that focuses on education, communication, and discharge planning with the primary aim of improving outcomeshas demonstrated inconsistent results as a mechanism to reduce LOS, postdischarge ED visits, or hospital readmission rates. While disease‐specific care coordination programs for congestive heart failure and chronic obstructive pulmonary disease have been effective in reducing rehospitalization rates,1015 the benefits of comprehensive care coordination for elderly general medical inpatients with a broader range of diagnoses are less clear. In a group of 750 elderly patients with 1 of 11 common inpatient diagnoses (such as stroke or hip fracture) likely to ultimately require a high level of home support, Coleman et al.16 found that a structured transitional care program centered on a personal coach decreased rehospitalization rates at 30 and 90 days. Preen et al.17 found improved patient involvement and perceived quality of life with care coordination focused on discharge planning, but no impact on LOS. Likewise, a recent meta‐analysis18 failed to demonstrate statistically significant differences in mortality, LOS, or readmission rates in hospitalized patients who received intensive care coordination versus usual care; however, variation in the components of the care coordination intervention and reported outcomes restricted the ability to pool data in this study.
Care coordination programs demonstrating efficacy in reducing health care utilization in elderly medical patients have generally included an outpatient transitional component with out‐of‐hospital postacute care visits by health care personnel such as a nurse, pharmacist, or physician.1923 These offsite interventions generate additional expenses and resource demands that may not be practical for smaller hospitals to implement.24, 25 In contrast, hospital‐based care coordination programs have clear ownership and thus may be more practical to disseminate. Individual elements of hospital‐based care coordination such as pharmacist counseling, discharge education, and telephone follow‐up have been shown to reduce ED visitation and readmission rates in high‐risk elderly patients. Less information is available regarding the impact of these interventions delivered in an aggregate bundle by hospital staff in the absence of bridging transitional visits.2629
The objective of this pilot study was to determine whether a supplemental elderly care bundle, targeted to high‐risk inpatients by hospital staff as an enhancement to existing care coordination, would affect postdischarge readmission and ED visit rates. The intervention was designed to capitalize on existing resources, and focused specifically on elderly inpatients who were hospitalized with diagnoses commonly encountered in a general medical unit and predisposed to recidivism.
Patients and Methods
Patient Selection and Enrollment
The screening population consisted of elderly patients admitted to 1 of 2 hospital‐medicine groups (MedProvider Inpatient Care Unit or Texas Primary Care) at the 900‐bed Baylor University Medical Center (BUMC, Dallas, TX) between March and June 2007 with a diagnosis likely to fall within 1 of 20 frequent Medicare medical diagnosis reimbursement groups (DRGs) at BUMC, as listed in Table 1. Study personnel performed daily chart review to establish eligibility criteria, which included age 70 years, use of 5 medications regularly, 3 chronic comorbid conditions, requirement for assistance with 1 ADL, and preadmission residence at home or assisted living with a reasonable expectation of disposition back to that domicile. These criteria were based on factors found in the literature to be associated with extended LOS and postdischarge readmission/ED visit events.5, 6, 3032 Potential enrollees needed to be conversant in English (a multilingual staff was not feasible due to limited resources for this pilot project) and have reliable phone contact, or have a proxy caregiver who could speak English and be reached by phone. Predefined exclusion criteria were admission primarily for a surgical procedure, terminal diagnosis with life expectancy 6 months, residence in a long‐term care facility (long‐term acute care [LTAC], skilled nursing facility [SNF], or nursing home) prior to hospitalization with anticipated discharge back to that facility, and patient/family refusal to participate. Additionally, with an average LOS between 5 and 6 days in BUMC's Medicare population for the DRGs of interest, it was felt that the effects of the care bundle intervention would be obscured unless initiated early in the hospitalization. Thus, patients who could not be enrolled within 72 hours following admission were subsequently excluded. The Baylor Health Care System (BHCS) Institutional Review Board (IRB) approved this study, and written informed consent was obtained from all patients or their surrogates.
DRG | DRG Name |
---|---|
| |
127 | Heart failure and shock |
14 | Intracranial hemorrhage or cerebral infarction |
89 | Simple pneumonia/pleurisy |
416 | Septicemia |
316 | Renal failure |
182 | Esophagitis/gastroenterological/miscellaneous digestive disorders with complications |
174 | Gastrointestinal hemorrhage with complications |
88 | Chronic obstructive pulmonary disease |
320 | Kidney/urinary tract infection with complications |
144 | Other circulatory diagnoses with complications |
138 | Arrhythmia/conduction disorders with complications |
277 | Cellulitis with complications |
124 | Circulatory disorders except acute myocardial infarction with cardiac catheterization and complex diagnosis |
430 | Psychoses |
188 | Other digestive diagnoses with complications |
395 | Red blood cell disorders |
79 | Respiratory infections and inflammations with complications |
524 | Transient ischemia |
143 | Chest pain |
141 | Syncope and collapse with complications |
Patients meeting eligibility criteria were approached within 72 hours of admission for study participation. After consent and enrollment, patients were randomized to intervention or usual care arms in permuted blocks of 8 via a random number generator and sealed opaque envelopes. Nursing and care coordination staff providing usual care to patients (independent of the research team) were blinded to the treatment group status of enrollees; trial design precluded blinding of study personnel and patients.
Delivery of the Supplemental Care Bundle
Starting no later than 24 hours after enrollment and continuing up to 1 week following hospital discharge, intervention group patients received a targeted care bundle provided by 1 of 3 care coordinators (CCs) and 1 of 4 clinical pharmacists (CPs) working with the study team. The care bundle was designed as an intensive patient‐centered educational program that would augment BUMC's existing care coordination processes (delivered to all patients regardless of study participation); specific elements are displayed in Figure 1. Study CCs saw patients daily throughout their hospital stay, and instructed patients on specific health conditions, with an emphasis on optimizing home self‐care and contingency plans if problems arose. CP visits focused on medication reconciliation and education regarding any new agents started during the hospitalization. The personal health record (PHR) provided a tool to engage patients in self‐care, and as discussed by Coleman et al.,7, 16, 33 promoted information transfer from the hospital to outpatient settings. During the postdischarge phone call, CCs followed a basic script to confirm receipt of medical equipment, medications, home health arrangements, and scheduling of follow‐up appointments. They also used this contact as an opportunity to reinforce patient education on managing their conditions. CPs reviewed medication use (type, schedule, dose), and spoke with patients about any symptoms they may have experienced as medication side effects. If indicated based on their phone discussions, both CCs and CPs could recommend an action plan to the patient.
The study CCs and CPs were existing hospital staff and performed their research activities in addition to their usual duties. Study CCs were highly experienced (averaging 8 years of inpatient floor nursing plus 10 years as CCs) and all had advanced nursing certifications (ACM, BSN, or MSN). The CPs were upper‐level pharmacy residents completing their inpatient clinical rotations. Additional training for both study CCs and CPs was limited to a series of 3 meetings (each 45 minutes in duration) regarding the intent and delivery of the supplemental care bundle, including use of study forms.
At the time of the trial, the particular CCs and CPs chosen to deliver the supplemental care bundle had work assignments ensuring that crossover between intervention and usual care groups would not occur. For example, 1 of the study CCs normally covered a surgical floor such that her normal scope of responsibilities would not influence the medical patients in the study (their baseline care coordination was provided by nonstudy personnel). Medication reconciliation and medication education is generally performed by floor nursing staff rather than CPs at BUMC.
Data Collection and Outcomes Measurement
Following enrollment, demographic information and a basic medical history were documented by research staff. Inaccuracies in medication lists discovered by pharmacists during the medication reconciliation process were entered directly into the universal medication list on the hospital chart. CPs also kept a log of the medication education given to patients (and recommendations for changes to patients' regimens given to physicians) throughout their hospital stay. Study CCs recorded their assessments of patient needs and associated responses. Furthermore, the research team CC prepared an enhanced discharge form that was given to intervention patients in addition to BUMC's standard form. Data on LOS, illness severity (APR‐DRGs), and unplanned hospital readmission or ED visitation at 30 and 60 days postdischarge were collected via BUMC's electronic reporting systems. All patient follow‐up was completed as of September 1, 2007.
Statistical Analyses
Resource and time constraints necessitated a sample size that would allow implementation of the intervention despite a limited number of study CCs and pharmacists. To accommodate these conditions while still generating pilot data, an a priori decision was made to enroll up to 80 patients. Continuous data variables were normally distributed. Differences between groups for continuous variables were assessed with the Student t‐test; differences in proportions between groups were compared with Fisher's exact tests. Time to readmission events between the groups were evaluated in a post hoc manner using the log‐rank test. Data were analyzed using Prism version 5 for Windows (GraphPad Software, Inc., San Diego, CA) and SPSS version 15 for Windows (SPSS Inc., Chicago, IL). P values < 0.05 were considered statistically significant.
Results
The final sample size for this pilot was small, with 41 total patients (21 controls, 20 interventions). The main reason for enrollment failure of patients meeting study criteria was an inability to obtain informed consent. Sixty patients declined participation after being approached, and another 56 patients were unable to give their informed consent due to impairments (poor cognition, medication induced sedation, severity of illness) with lack of an available proxy to give written consent during the 72‐hour postadmission recruitment window. There were no statistically significant differences in the baseline characteristics of the intervention and control groups (Table 2). A similar proportion of patients (23% in the intervention, 15% in controls; P = 0.70) had preexisting diagnoses of dementia or depression. However, on APR‐DRG measures relating to acuity of illness and mortality risk, patients in the intervention group trended toward higher severity (Table 2). Likewise, although it was not a statistically significant difference, 13 of 20 patients in the intervention group were taking medications from 2 drug classes commonly implicated in adverse drug events (warfarin, insulin, diuretics, sedating agents) as part of their discharge medication regimen compared to 10 of 21 patients in the control group.
Control (n = 21) | Intervention (n = 20) | P Value | |
---|---|---|---|
Age in years (mean SD) | 79.8 5.6 | 77.2 5.3 | 0.14 |
Males, n (%) | 8 (38) | 3 (15) | 0.10 |
Females, n (%) | 13 (62) | 17 (85) | 0.10 |
Race, n (%) | |||
African‐American | 3 (14) | 5 (25) | 0.45 |
Asian | 0 (0) | 1 (5) | 0.49 |
Caucasian | 17 (81) | 14 (70) | 0.48 |
Hispanic | 1 (5) | 0 | 1.0 |
Preadmission living status, n (%) | |||
Alone | 6 (29) | 4 (20) | 0.72 |
With spouse or other family | 11 (52) | 15 (75) | 0.20 |
Assisted living | 4 (19) | 1 (5) | 0.34 |
Inpatient medications (mean SD) | 11 3 | 12 5 | 0.18 |
Charlson score (mean, SD) | 3.2 1.3 | 3.7 1.1 | 0.21 |
% with APR DRG severity rating 3 | 57.5 | 83.3 | 0.12 |
% with APR DRG mortality rating 3 | 20.0 | 55.6 | 0.07 |
Primary admission diagnoses (n cases, in order of frequency) | 3 pneumonia | 3 pneumonia | |
3 CHF | 3 syncope | ||
2 syncope | 2 CHF | ||
2 COPD | 2 COPD | ||
2 cellulitis | 2 cellulitis | ||
2 GI disorder (nonbleed) | 2 GI disorder (nonbleed) | ||
2 GI bleed | 1 GI bleed | ||
2 UTI | 1 atrial fibrillation | ||
1 atrial fibrillation | 1 encephalopathy | ||
1 stroke | 1 TIA | ||
1 renal failure | 1 renal failure | ||
1 volume depletion |
Study outcomes are displayed in Table 3. Mean LOS is reported as a descriptive finding; there was insufficient power to compare this outcome statistically between groups. The majority of patients were discharged to home. A similar proportion of patients in the intervention (20%) and control groups (22%) who had lived at home immediately prior to admission were discharged from the hospital to skilled care facilities (P = 0.87). The number of readmissions/ED visits (taken as a composite measure of unplanned healthcare utilization) within 30 days of discharge was lower in the intervention group; by 60 days, there was no longer a statistically significant difference in readmission/ED visit rates between groups. For those patients who had a readmission or ED visit following hospital discharge, the intervention group had a longer time interval to first event compared to controls (36.2 versus 15.7 days, P = 0.05). Of the patients discharged to skilled care, 1 in the intervention group (at 53 days) and 1 in the control group (at 16 days) had a readmission/ED visit event. Figure 2 shows time‐to‐first readmission or ED visit event curves at 30 and 60 days for both intervention and control groups. For patients who had a readmission/ED visit event, LOS for this episode was 2.2 2.1 days in controls and 3.7 2.1 days in the intervention group (insufficient power for statistical comparison). The study's small sample size prevented development of a meaningful regression model.
Outcome Measure | Control (n = 21) | Intervention (n = 20) | P Value |
---|---|---|---|
| |||
Length of stay for index hospitalization (days)* | 4.7 3.7 | 6.2 4.1 | |
0‐30 day postdischarge readmissions/ED visits | 8 (38%) | 2 (10%) | 0.03 |
31‐60 day postdischarge readmissions/ED visits | 1 (5%) | 4 (20%) | 0.18 |
Total postdischarge readmissions/ED visits at 60 days | 9 | 6 | 0.52 |
Resource utilization and the specifics of patient‐study personnel interaction associated with the intervention were tracked. Research assistants spent an average of 50 minutes daily screening charts for potential candidates. For the 20 patients who received the supplemental elderly care bundle, study CCs averaged 20 to 25 minutes per patient daily of additional time counseling patients and families, identifying and attending to discharge barriers, filling out documentation, and faxing the supplemental study discharge form to the patient's primary care physician. Any residual home care needs or issues unresolved at discharge were addressed with the patient in the 5 to 7 day follow‐up phone call. Similarly, study CPs expended approximately 20 minutes daily per patient providing medication education, reconciliation, and optimization of drug therapy. Study pharmacists recommended a change to the medication regimens of 10 patients in the intervention group; physicians acted upon these recommendations for 7 of the patients. The changes included dosage adjustment, discontinuation of medications due to possible drug interaction or duplication of drugs with the same pharmacologic effect, and addition of medications as indicated by patient condition or to reconcile with patients' at‐home medication regimens. Patients contacted via phone by the study pharmacist within 1 week after discharge were able to describe proper use of new medications started in the hospital and confirm that they obtained or had the means to obtain the prescribed drugs.
Discussion
This pilot study examined the effects of a supplemental care bundle involving patient education and discharge planning delivered by hospital‐based CCs and CPs on the rate of readmission/ED visitation in 41 elderly (70 years of age) patients. The study was not adequately powered to detect an impact of the intervention on index LOS. The care bundle did lead to significantly fewer readmissions or ED visits 30 days postdischarge and appeared to increase the time interval to first unplanned readmission or ED visit compared to usual care. This effect was no longer present at 60 days postdischarge. Resource allocations and scope of duties for CCs and CPs (an average of 20 minutes per patient per day) related to delivering the intervention were realistic for broader implementation in the hospitalized elderly population at high risk for readmission or ED visitation following discharge.
Length of stay for the initial hospitalization associated with the care bundle was an original outcome of interest to the study team. However, with the final enrollment of 41 patients and a power of 0.8, the between group difference would have needed to be 2.6 days to be statistically significant. It is likely that any change in LOS related to the care bundle would be much smaller, particularly since 2 key determinants of LOS, severity of illness and physician behavior, were beyond this patient education‐oriented intervention's scope of influence.3437 Furthermore, the diverse range of eligible diagnoses limited the study CCs' ability to reduce variability through use of clinical care pathways. One approach in leveraging an elderly care bundle to reduce LOS may be to focus on a specific disease that has well‐established inpatient benchmarks and treatment algorithms. For example, in patients with community‐acquired pneumonia, the use of care coordination in combination with standardized order sets decreased LOS without compromising safety, mainly by shortening the time from clinical stability to discharge.38
On separation of the readmission/ED visit outcome into 30 and 60 day postdischarge time frames, the intervention group had a lower rate of unplanned acute health care use within 30 days postdischarge; the difference between groups had dissipated by 60 days postdischarge. This convergence suggests that a hospital‐based intervention's influence is strongest closer to the time of the initial hospital stay, and wanes as more time has elapsed. Indeed, interventions that have successfully maintained lower readmission rates beyond 60 and 90 days postdischarge in a high‐risk elderly population (such as the program advocated by Coleman et al.16) have included a transitional care provider engaging patients during the hospitalization and performing subsequent visits to the home or nursing facility.33 An optimal intervention would capitalize on the hospital‐based staff's ability to improve short‐term readmission/ED visit rates while linking patients to longer‐term transitional care to extend these outcomes. Electronic health records could potentially facilitate these care transitions, beginning with an automated screening process for identification of high‐risk inpatients and continuing through postdischarge follow‐up. How to develop these resources in settings where outpatient practices are independent or only loosely affiliated with hospitals is an area for continued investigation.
In a group of elderly patients with multiple comorbidities and complex pharmacotherapy regimens, the study bundle component targeting medication management appears to be a high‐yield intervention to reduce unplanned health care utilization following hospital discharge. These patients are more susceptible to nonadherence and drug‐related adverse events, which may contribute to hospital readmission or ED visitation.7, 9, 39 Consistent with findings at other sites,28, 40 a heightened level of CP involvement in the care of high‐risk elderly patients may have helped reduce these undesirable outcomes. Of the 9 readmission/ED visit events in the control group, 3 were attributable to medication related complications (2 from sedatives, 1 from a diuretic). None of the readmission/ED visit events in intervention group patients stemmed from medication effects.
Correspondingly, the research CCs' provision of daily condition‐specific education, additional time to more thoroughly investigate discharge needs, engagement of patients' families as active partners in self‐care, and the use of a structured discharge form along with follow‐up phone calls may have better prepared patients to manage their health problems once released from the hospital.26, 28, 29 For example, 1 patient in the control group was readmitted less than 24 hours after initial discharge due to inability to perform self‐care at home. Given the study power issues described previously, data on LOS for the second hospitalization for patients who had a readmission event are difficult to interpret, but could suggest the occurrence of some shorter, preventable readmissions in the control group. Conversely, the readmission/ED visit events in intervention patients appeared to be associated with a specific medical condition (eg, failure of diabetic cellulitis to respond to appropriate outpatient treatment) rather than problems that would have been corrected with an educational/self‐management program such as this targeted care bundle.
This pilot study had several limitations. The main issue was a small patient sample size that was primarily due to an inability to obtain informed consent. Design of the study as a randomized controlled trial and plans to disseminate study findings beyond BHCS necessitated IRB approval rather than delivery of the supplemental care bundle as a quality improvement (QI) project. Placing QI initiatives under research regulations can lead to project delays, higher costs, and patient frustrations with the process.41, 42 This tension was evident during study screening and enrollment, as many patients who otherwise met criteria and would potentially benefit from the intervention were hesitant to participate in a research study or refused to sign a multipage consent document. The difficulties of enrolling elderly patients in clinical trials have been well‐described.43, 44 Further research involving a minimal‐risk, educational intervention such as this elderly care‐bundle would likely better fit under the category of expedited IRB review with waiver or modification of the informed consent process.45
Incomplete blinding could have potentially affected our results. At the study site, the team members delivering the care bundle were a regular part of the hospital staff (as opposed to external researchers), and it is not unusual for a CC or a pharmacist to enter a patient's room (eg, to confirm a drug allergy history). In view of this, the impact of imperfect blinding on 30‐day outcomes would likely be minimal. Furthermore, a floor staff perception that a specific patient was being taken care of by the study team resulting in a lower than usual level of care, would tend to bias the result of the intervention toward the null effect.
vThe study cohort did not have enough subjects to perform analyses (ie, modeling or examination of subgroups) beyond basic comparative findings. Issues such as preadmission living situation and the presence of depression or cognitive impairment (Mini‐Mental Status Exams were not performed on these patients) may potentially influence postdischarge recidivism; their effects can not be reliably ascertained from these data. Additionally, to prevent study personnel from engaging patients who would soon be going home, it was felt that the benefits from the care bundle would be recognized only if the intervention could be initiated within 72 hours of admission and delivered in full, a requirement that further reduced the enrollment pool. The intent of this pilot work was to guide future investigations surrounding hospital‐to‐home transitional care. The next phase of research in this area will need an enhanced sample size with more extensive baseline data collection so that potential confounding factors or outcomes in specific populations can be explored.
Another problem restricting applicability of study findings was the use of only 3 different CCs and 3 pharmacists on the research team to deliver the components of patient education, discharge planning, and medication counseling in the elderly care bundle. Personnel for the trial were chosen for their experience and interest in the area of care transitions. To distinguish the benefit of the elderly care bundle in general versus the expertise of these particular CCs and study pharmacists, a larger‐scale, multisite trial would be necessary. Lastly, due to resource constraints, patients who resided in long‐term care (either LTAC, SNFs, or nursing homes) prior to admission with anticipated return to those sites were not eligible for the study. Similar to the patients whose comorbidities or acute severity of illness prevented informed consent, this segment of the elderly population may have derived even more benefit from receipt of the elderly care bundle.10, 15, 46 Despite exclusion of this group (which would be expected to lessen the impact of the intervention), a difference in readmission/ED visits rates at 30 days following discharge was observed.
Conclusions
This pilot randomized clinical trial (RCT) evaluated the effects of a supplemental, aggregate care bundle centered on patient education, discharge planning, and medication counseling and reconciliation compared to usual care in a group of elderly patients at high risk of readmission or ED visitation following an index hospitalization. The intervention was designed to be reproducible and make use of existing hospital resources. Probably through facilitation of patient self‐care and home management, the elderly care bundle reduced the composite outcome of readmission/ED visits at 30 days postdischarge. By 60 days, this effect had waned, demonstrating the short‐term benefit of a hospital‐based educational intervention and stressing the need to incorporate additional outpatient transitional care support to sustain favorable outcomes. The study was not powered to detect small differences (which would be more likely than a change of multiple days) in length of index hospital stay related to the care bundle. There were important study limitations (primarily associated with small sample size), and this work should be viewed as hypothesis‐generating. Future trials should assess the impact of a standardized targeted care bundle delivered across multiple healthcare systems on a larger cohort of high‐risk elderly patients, including analysis of financial and personnel allocations relative to the benefits of the intervention.
Acknowledgements
The authors thank study pharmacists Kristen Hesch (PharmD), Renee Danysh (PharmD), Rema Thyagarajan (PharmD), and Betina Thomas (PharmD) for providing patients with medication education and conducting medication reconciliation. They also thank Jeanne Bradbury (RN, ACM), Diana Davis (RN, BSN), and Gail McVea (RN, MSN) for their involvement as care coordinators; Veronica Odom (RN) for her contributions as a research nurse; and Marilyn Callies (RN, MBA) for her role as project advisor.
Elderly patients (aged 65 years and older) consume a disproportionate amount of acute health care resources, composing up to 20% of emergency department (ED) visits,1, 2 having a 2‐fold to 5‐fold increase in likelihood of hospital admission,1 and frequently incurring lengths of hospital stay (LOS) approximately 15% higher than the national averages.3 In addition, they are at increased risk for hospital readmission in the 90‐day interval following hospital discharge.1, 4, 5 Specific risk factors for readmission include age above 80 years, discharge within the previous 30 days, the presence of 3 or more comorbid diagnoses, use of 5 or more prescription medications, difficulty with at least 1 activity of daily living (ADL), and lack of discharge education.6 These risk factors can translate into adverse drug events,79 exacerbations of chronic diseases,10 or functional decline4, 5 that can trigger ED visits or hospital readmission.
Hospital‐based care coordinationdefined as a multidisciplinary interaction between inpatients and providers that focuses on education, communication, and discharge planning with the primary aim of improving outcomeshas demonstrated inconsistent results as a mechanism to reduce LOS, postdischarge ED visits, or hospital readmission rates. While disease‐specific care coordination programs for congestive heart failure and chronic obstructive pulmonary disease have been effective in reducing rehospitalization rates,1015 the benefits of comprehensive care coordination for elderly general medical inpatients with a broader range of diagnoses are less clear. In a group of 750 elderly patients with 1 of 11 common inpatient diagnoses (such as stroke or hip fracture) likely to ultimately require a high level of home support, Coleman et al.16 found that a structured transitional care program centered on a personal coach decreased rehospitalization rates at 30 and 90 days. Preen et al.17 found improved patient involvement and perceived quality of life with care coordination focused on discharge planning, but no impact on LOS. Likewise, a recent meta‐analysis18 failed to demonstrate statistically significant differences in mortality, LOS, or readmission rates in hospitalized patients who received intensive care coordination versus usual care; however, variation in the components of the care coordination intervention and reported outcomes restricted the ability to pool data in this study.
Care coordination programs demonstrating efficacy in reducing health care utilization in elderly medical patients have generally included an outpatient transitional component with out‐of‐hospital postacute care visits by health care personnel such as a nurse, pharmacist, or physician.1923 These offsite interventions generate additional expenses and resource demands that may not be practical for smaller hospitals to implement.24, 25 In contrast, hospital‐based care coordination programs have clear ownership and thus may be more practical to disseminate. Individual elements of hospital‐based care coordination such as pharmacist counseling, discharge education, and telephone follow‐up have been shown to reduce ED visitation and readmission rates in high‐risk elderly patients. Less information is available regarding the impact of these interventions delivered in an aggregate bundle by hospital staff in the absence of bridging transitional visits.2629
The objective of this pilot study was to determine whether a supplemental elderly care bundle, targeted to high‐risk inpatients by hospital staff as an enhancement to existing care coordination, would affect postdischarge readmission and ED visit rates. The intervention was designed to capitalize on existing resources, and focused specifically on elderly inpatients who were hospitalized with diagnoses commonly encountered in a general medical unit and predisposed to recidivism.
Patients and Methods
Patient Selection and Enrollment
The screening population consisted of elderly patients admitted to 1 of 2 hospital‐medicine groups (MedProvider Inpatient Care Unit or Texas Primary Care) at the 900‐bed Baylor University Medical Center (BUMC, Dallas, TX) between March and June 2007 with a diagnosis likely to fall within 1 of 20 frequent Medicare medical diagnosis reimbursement groups (DRGs) at BUMC, as listed in Table 1. Study personnel performed daily chart review to establish eligibility criteria, which included age 70 years, use of 5 medications regularly, 3 chronic comorbid conditions, requirement for assistance with 1 ADL, and preadmission residence at home or assisted living with a reasonable expectation of disposition back to that domicile. These criteria were based on factors found in the literature to be associated with extended LOS and postdischarge readmission/ED visit events.5, 6, 3032 Potential enrollees needed to be conversant in English (a multilingual staff was not feasible due to limited resources for this pilot project) and have reliable phone contact, or have a proxy caregiver who could speak English and be reached by phone. Predefined exclusion criteria were admission primarily for a surgical procedure, terminal diagnosis with life expectancy 6 months, residence in a long‐term care facility (long‐term acute care [LTAC], skilled nursing facility [SNF], or nursing home) prior to hospitalization with anticipated discharge back to that facility, and patient/family refusal to participate. Additionally, with an average LOS between 5 and 6 days in BUMC's Medicare population for the DRGs of interest, it was felt that the effects of the care bundle intervention would be obscured unless initiated early in the hospitalization. Thus, patients who could not be enrolled within 72 hours following admission were subsequently excluded. The Baylor Health Care System (BHCS) Institutional Review Board (IRB) approved this study, and written informed consent was obtained from all patients or their surrogates.
DRG | DRG Name |
---|---|
| |
127 | Heart failure and shock |
14 | Intracranial hemorrhage or cerebral infarction |
89 | Simple pneumonia/pleurisy |
416 | Septicemia |
316 | Renal failure |
182 | Esophagitis/gastroenterological/miscellaneous digestive disorders with complications |
174 | Gastrointestinal hemorrhage with complications |
88 | Chronic obstructive pulmonary disease |
320 | Kidney/urinary tract infection with complications |
144 | Other circulatory diagnoses with complications |
138 | Arrhythmia/conduction disorders with complications |
277 | Cellulitis with complications |
124 | Circulatory disorders except acute myocardial infarction with cardiac catheterization and complex diagnosis |
430 | Psychoses |
188 | Other digestive diagnoses with complications |
395 | Red blood cell disorders |
79 | Respiratory infections and inflammations with complications |
524 | Transient ischemia |
143 | Chest pain |
141 | Syncope and collapse with complications |
Patients meeting eligibility criteria were approached within 72 hours of admission for study participation. After consent and enrollment, patients were randomized to intervention or usual care arms in permuted blocks of 8 via a random number generator and sealed opaque envelopes. Nursing and care coordination staff providing usual care to patients (independent of the research team) were blinded to the treatment group status of enrollees; trial design precluded blinding of study personnel and patients.
Delivery of the Supplemental Care Bundle
Starting no later than 24 hours after enrollment and continuing up to 1 week following hospital discharge, intervention group patients received a targeted care bundle provided by 1 of 3 care coordinators (CCs) and 1 of 4 clinical pharmacists (CPs) working with the study team. The care bundle was designed as an intensive patient‐centered educational program that would augment BUMC's existing care coordination processes (delivered to all patients regardless of study participation); specific elements are displayed in Figure 1. Study CCs saw patients daily throughout their hospital stay, and instructed patients on specific health conditions, with an emphasis on optimizing home self‐care and contingency plans if problems arose. CP visits focused on medication reconciliation and education regarding any new agents started during the hospitalization. The personal health record (PHR) provided a tool to engage patients in self‐care, and as discussed by Coleman et al.,7, 16, 33 promoted information transfer from the hospital to outpatient settings. During the postdischarge phone call, CCs followed a basic script to confirm receipt of medical equipment, medications, home health arrangements, and scheduling of follow‐up appointments. They also used this contact as an opportunity to reinforce patient education on managing their conditions. CPs reviewed medication use (type, schedule, dose), and spoke with patients about any symptoms they may have experienced as medication side effects. If indicated based on their phone discussions, both CCs and CPs could recommend an action plan to the patient.
The study CCs and CPs were existing hospital staff and performed their research activities in addition to their usual duties. Study CCs were highly experienced (averaging 8 years of inpatient floor nursing plus 10 years as CCs) and all had advanced nursing certifications (ACM, BSN, or MSN). The CPs were upper‐level pharmacy residents completing their inpatient clinical rotations. Additional training for both study CCs and CPs was limited to a series of 3 meetings (each 45 minutes in duration) regarding the intent and delivery of the supplemental care bundle, including use of study forms.
At the time of the trial, the particular CCs and CPs chosen to deliver the supplemental care bundle had work assignments ensuring that crossover between intervention and usual care groups would not occur. For example, 1 of the study CCs normally covered a surgical floor such that her normal scope of responsibilities would not influence the medical patients in the study (their baseline care coordination was provided by nonstudy personnel). Medication reconciliation and medication education is generally performed by floor nursing staff rather than CPs at BUMC.
Data Collection and Outcomes Measurement
Following enrollment, demographic information and a basic medical history were documented by research staff. Inaccuracies in medication lists discovered by pharmacists during the medication reconciliation process were entered directly into the universal medication list on the hospital chart. CPs also kept a log of the medication education given to patients (and recommendations for changes to patients' regimens given to physicians) throughout their hospital stay. Study CCs recorded their assessments of patient needs and associated responses. Furthermore, the research team CC prepared an enhanced discharge form that was given to intervention patients in addition to BUMC's standard form. Data on LOS, illness severity (APR‐DRGs), and unplanned hospital readmission or ED visitation at 30 and 60 days postdischarge were collected via BUMC's electronic reporting systems. All patient follow‐up was completed as of September 1, 2007.
Statistical Analyses
Resource and time constraints necessitated a sample size that would allow implementation of the intervention despite a limited number of study CCs and pharmacists. To accommodate these conditions while still generating pilot data, an a priori decision was made to enroll up to 80 patients. Continuous data variables were normally distributed. Differences between groups for continuous variables were assessed with the Student t‐test; differences in proportions between groups were compared with Fisher's exact tests. Time to readmission events between the groups were evaluated in a post hoc manner using the log‐rank test. Data were analyzed using Prism version 5 for Windows (GraphPad Software, Inc., San Diego, CA) and SPSS version 15 for Windows (SPSS Inc., Chicago, IL). P values < 0.05 were considered statistically significant.
Results
The final sample size for this pilot was small, with 41 total patients (21 controls, 20 interventions). The main reason for enrollment failure of patients meeting study criteria was an inability to obtain informed consent. Sixty patients declined participation after being approached, and another 56 patients were unable to give their informed consent due to impairments (poor cognition, medication induced sedation, severity of illness) with lack of an available proxy to give written consent during the 72‐hour postadmission recruitment window. There were no statistically significant differences in the baseline characteristics of the intervention and control groups (Table 2). A similar proportion of patients (23% in the intervention, 15% in controls; P = 0.70) had preexisting diagnoses of dementia or depression. However, on APR‐DRG measures relating to acuity of illness and mortality risk, patients in the intervention group trended toward higher severity (Table 2). Likewise, although it was not a statistically significant difference, 13 of 20 patients in the intervention group were taking medications from 2 drug classes commonly implicated in adverse drug events (warfarin, insulin, diuretics, sedating agents) as part of their discharge medication regimen compared to 10 of 21 patients in the control group.
Control (n = 21) | Intervention (n = 20) | P Value | |
---|---|---|---|
Age in years (mean SD) | 79.8 5.6 | 77.2 5.3 | 0.14 |
Males, n (%) | 8 (38) | 3 (15) | 0.10 |
Females, n (%) | 13 (62) | 17 (85) | 0.10 |
Race, n (%) | |||
African‐American | 3 (14) | 5 (25) | 0.45 |
Asian | 0 (0) | 1 (5) | 0.49 |
Caucasian | 17 (81) | 14 (70) | 0.48 |
Hispanic | 1 (5) | 0 | 1.0 |
Preadmission living status, n (%) | |||
Alone | 6 (29) | 4 (20) | 0.72 |
With spouse or other family | 11 (52) | 15 (75) | 0.20 |
Assisted living | 4 (19) | 1 (5) | 0.34 |
Inpatient medications (mean SD) | 11 3 | 12 5 | 0.18 |
Charlson score (mean, SD) | 3.2 1.3 | 3.7 1.1 | 0.21 |
% with APR DRG severity rating 3 | 57.5 | 83.3 | 0.12 |
% with APR DRG mortality rating 3 | 20.0 | 55.6 | 0.07 |
Primary admission diagnoses (n cases, in order of frequency) | 3 pneumonia | 3 pneumonia | |
3 CHF | 3 syncope | ||
2 syncope | 2 CHF | ||
2 COPD | 2 COPD | ||
2 cellulitis | 2 cellulitis | ||
2 GI disorder (nonbleed) | 2 GI disorder (nonbleed) | ||
2 GI bleed | 1 GI bleed | ||
2 UTI | 1 atrial fibrillation | ||
1 atrial fibrillation | 1 encephalopathy | ||
1 stroke | 1 TIA | ||
1 renal failure | 1 renal failure | ||
1 volume depletion |
Study outcomes are displayed in Table 3. Mean LOS is reported as a descriptive finding; there was insufficient power to compare this outcome statistically between groups. The majority of patients were discharged to home. A similar proportion of patients in the intervention (20%) and control groups (22%) who had lived at home immediately prior to admission were discharged from the hospital to skilled care facilities (P = 0.87). The number of readmissions/ED visits (taken as a composite measure of unplanned healthcare utilization) within 30 days of discharge was lower in the intervention group; by 60 days, there was no longer a statistically significant difference in readmission/ED visit rates between groups. For those patients who had a readmission or ED visit following hospital discharge, the intervention group had a longer time interval to first event compared to controls (36.2 versus 15.7 days, P = 0.05). Of the patients discharged to skilled care, 1 in the intervention group (at 53 days) and 1 in the control group (at 16 days) had a readmission/ED visit event. Figure 2 shows time‐to‐first readmission or ED visit event curves at 30 and 60 days for both intervention and control groups. For patients who had a readmission/ED visit event, LOS for this episode was 2.2 2.1 days in controls and 3.7 2.1 days in the intervention group (insufficient power for statistical comparison). The study's small sample size prevented development of a meaningful regression model.
Outcome Measure | Control (n = 21) | Intervention (n = 20) | P Value |
---|---|---|---|
| |||
Length of stay for index hospitalization (days)* | 4.7 3.7 | 6.2 4.1 | |
0‐30 day postdischarge readmissions/ED visits | 8 (38%) | 2 (10%) | 0.03 |
31‐60 day postdischarge readmissions/ED visits | 1 (5%) | 4 (20%) | 0.18 |
Total postdischarge readmissions/ED visits at 60 days | 9 | 6 | 0.52 |
Resource utilization and the specifics of patient‐study personnel interaction associated with the intervention were tracked. Research assistants spent an average of 50 minutes daily screening charts for potential candidates. For the 20 patients who received the supplemental elderly care bundle, study CCs averaged 20 to 25 minutes per patient daily of additional time counseling patients and families, identifying and attending to discharge barriers, filling out documentation, and faxing the supplemental study discharge form to the patient's primary care physician. Any residual home care needs or issues unresolved at discharge were addressed with the patient in the 5 to 7 day follow‐up phone call. Similarly, study CPs expended approximately 20 minutes daily per patient providing medication education, reconciliation, and optimization of drug therapy. Study pharmacists recommended a change to the medication regimens of 10 patients in the intervention group; physicians acted upon these recommendations for 7 of the patients. The changes included dosage adjustment, discontinuation of medications due to possible drug interaction or duplication of drugs with the same pharmacologic effect, and addition of medications as indicated by patient condition or to reconcile with patients' at‐home medication regimens. Patients contacted via phone by the study pharmacist within 1 week after discharge were able to describe proper use of new medications started in the hospital and confirm that they obtained or had the means to obtain the prescribed drugs.
Discussion
This pilot study examined the effects of a supplemental care bundle involving patient education and discharge planning delivered by hospital‐based CCs and CPs on the rate of readmission/ED visitation in 41 elderly (70 years of age) patients. The study was not adequately powered to detect an impact of the intervention on index LOS. The care bundle did lead to significantly fewer readmissions or ED visits 30 days postdischarge and appeared to increase the time interval to first unplanned readmission or ED visit compared to usual care. This effect was no longer present at 60 days postdischarge. Resource allocations and scope of duties for CCs and CPs (an average of 20 minutes per patient per day) related to delivering the intervention were realistic for broader implementation in the hospitalized elderly population at high risk for readmission or ED visitation following discharge.
Length of stay for the initial hospitalization associated with the care bundle was an original outcome of interest to the study team. However, with the final enrollment of 41 patients and a power of 0.8, the between group difference would have needed to be 2.6 days to be statistically significant. It is likely that any change in LOS related to the care bundle would be much smaller, particularly since 2 key determinants of LOS, severity of illness and physician behavior, were beyond this patient education‐oriented intervention's scope of influence.3437 Furthermore, the diverse range of eligible diagnoses limited the study CCs' ability to reduce variability through use of clinical care pathways. One approach in leveraging an elderly care bundle to reduce LOS may be to focus on a specific disease that has well‐established inpatient benchmarks and treatment algorithms. For example, in patients with community‐acquired pneumonia, the use of care coordination in combination with standardized order sets decreased LOS without compromising safety, mainly by shortening the time from clinical stability to discharge.38
On separation of the readmission/ED visit outcome into 30 and 60 day postdischarge time frames, the intervention group had a lower rate of unplanned acute health care use within 30 days postdischarge; the difference between groups had dissipated by 60 days postdischarge. This convergence suggests that a hospital‐based intervention's influence is strongest closer to the time of the initial hospital stay, and wanes as more time has elapsed. Indeed, interventions that have successfully maintained lower readmission rates beyond 60 and 90 days postdischarge in a high‐risk elderly population (such as the program advocated by Coleman et al.16) have included a transitional care provider engaging patients during the hospitalization and performing subsequent visits to the home or nursing facility.33 An optimal intervention would capitalize on the hospital‐based staff's ability to improve short‐term readmission/ED visit rates while linking patients to longer‐term transitional care to extend these outcomes. Electronic health records could potentially facilitate these care transitions, beginning with an automated screening process for identification of high‐risk inpatients and continuing through postdischarge follow‐up. How to develop these resources in settings where outpatient practices are independent or only loosely affiliated with hospitals is an area for continued investigation.
In a group of elderly patients with multiple comorbidities and complex pharmacotherapy regimens, the study bundle component targeting medication management appears to be a high‐yield intervention to reduce unplanned health care utilization following hospital discharge. These patients are more susceptible to nonadherence and drug‐related adverse events, which may contribute to hospital readmission or ED visitation.7, 9, 39 Consistent with findings at other sites,28, 40 a heightened level of CP involvement in the care of high‐risk elderly patients may have helped reduce these undesirable outcomes. Of the 9 readmission/ED visit events in the control group, 3 were attributable to medication related complications (2 from sedatives, 1 from a diuretic). None of the readmission/ED visit events in intervention group patients stemmed from medication effects.
Correspondingly, the research CCs' provision of daily condition‐specific education, additional time to more thoroughly investigate discharge needs, engagement of patients' families as active partners in self‐care, and the use of a structured discharge form along with follow‐up phone calls may have better prepared patients to manage their health problems once released from the hospital.26, 28, 29 For example, 1 patient in the control group was readmitted less than 24 hours after initial discharge due to inability to perform self‐care at home. Given the study power issues described previously, data on LOS for the second hospitalization for patients who had a readmission event are difficult to interpret, but could suggest the occurrence of some shorter, preventable readmissions in the control group. Conversely, the readmission/ED visit events in intervention patients appeared to be associated with a specific medical condition (eg, failure of diabetic cellulitis to respond to appropriate outpatient treatment) rather than problems that would have been corrected with an educational/self‐management program such as this targeted care bundle.
This pilot study had several limitations. The main issue was a small patient sample size that was primarily due to an inability to obtain informed consent. Design of the study as a randomized controlled trial and plans to disseminate study findings beyond BHCS necessitated IRB approval rather than delivery of the supplemental care bundle as a quality improvement (QI) project. Placing QI initiatives under research regulations can lead to project delays, higher costs, and patient frustrations with the process.41, 42 This tension was evident during study screening and enrollment, as many patients who otherwise met criteria and would potentially benefit from the intervention were hesitant to participate in a research study or refused to sign a multipage consent document. The difficulties of enrolling elderly patients in clinical trials have been well‐described.43, 44 Further research involving a minimal‐risk, educational intervention such as this elderly care‐bundle would likely better fit under the category of expedited IRB review with waiver or modification of the informed consent process.45
Incomplete blinding could have potentially affected our results. At the study site, the team members delivering the care bundle were a regular part of the hospital staff (as opposed to external researchers), and it is not unusual for a CC or a pharmacist to enter a patient's room (eg, to confirm a drug allergy history). In view of this, the impact of imperfect blinding on 30‐day outcomes would likely be minimal. Furthermore, a floor staff perception that a specific patient was being taken care of by the study team resulting in a lower than usual level of care, would tend to bias the result of the intervention toward the null effect.
vThe study cohort did not have enough subjects to perform analyses (ie, modeling or examination of subgroups) beyond basic comparative findings. Issues such as preadmission living situation and the presence of depression or cognitive impairment (Mini‐Mental Status Exams were not performed on these patients) may potentially influence postdischarge recidivism; their effects can not be reliably ascertained from these data. Additionally, to prevent study personnel from engaging patients who would soon be going home, it was felt that the benefits from the care bundle would be recognized only if the intervention could be initiated within 72 hours of admission and delivered in full, a requirement that further reduced the enrollment pool. The intent of this pilot work was to guide future investigations surrounding hospital‐to‐home transitional care. The next phase of research in this area will need an enhanced sample size with more extensive baseline data collection so that potential confounding factors or outcomes in specific populations can be explored.
Another problem restricting applicability of study findings was the use of only 3 different CCs and 3 pharmacists on the research team to deliver the components of patient education, discharge planning, and medication counseling in the elderly care bundle. Personnel for the trial were chosen for their experience and interest in the area of care transitions. To distinguish the benefit of the elderly care bundle in general versus the expertise of these particular CCs and study pharmacists, a larger‐scale, multisite trial would be necessary. Lastly, due to resource constraints, patients who resided in long‐term care (either LTAC, SNFs, or nursing homes) prior to admission with anticipated return to those sites were not eligible for the study. Similar to the patients whose comorbidities or acute severity of illness prevented informed consent, this segment of the elderly population may have derived even more benefit from receipt of the elderly care bundle.10, 15, 46 Despite exclusion of this group (which would be expected to lessen the impact of the intervention), a difference in readmission/ED visits rates at 30 days following discharge was observed.
Conclusions
This pilot randomized clinical trial (RCT) evaluated the effects of a supplemental, aggregate care bundle centered on patient education, discharge planning, and medication counseling and reconciliation compared to usual care in a group of elderly patients at high risk of readmission or ED visitation following an index hospitalization. The intervention was designed to be reproducible and make use of existing hospital resources. Probably through facilitation of patient self‐care and home management, the elderly care bundle reduced the composite outcome of readmission/ED visits at 30 days postdischarge. By 60 days, this effect had waned, demonstrating the short‐term benefit of a hospital‐based educational intervention and stressing the need to incorporate additional outpatient transitional care support to sustain favorable outcomes. The study was not powered to detect small differences (which would be more likely than a change of multiple days) in length of index hospital stay related to the care bundle. There were important study limitations (primarily associated with small sample size), and this work should be viewed as hypothesis‐generating. Future trials should assess the impact of a standardized targeted care bundle delivered across multiple healthcare systems on a larger cohort of high‐risk elderly patients, including analysis of financial and personnel allocations relative to the benefits of the intervention.
Acknowledgements
The authors thank study pharmacists Kristen Hesch (PharmD), Renee Danysh (PharmD), Rema Thyagarajan (PharmD), and Betina Thomas (PharmD) for providing patients with medication education and conducting medication reconciliation. They also thank Jeanne Bradbury (RN, ACM), Diana Davis (RN, BSN), and Gail McVea (RN, MSN) for their involvement as care coordinators; Veronica Odom (RN) for her contributions as a research nurse; and Marilyn Callies (RN, MBA) for her role as project advisor.
- Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions.Ann Emerg Med.2002;39(3):238–247. , .
- Emergency department utilization by the elderly: analysis of the National Hospital Ambulatory Medical Care Survey.Acad Emerg Med.1996;3(7):694–699. , , , , , .
- 2005 National Hospital Discharge Survey.Adv Data.2007(385):1–19. , .
- Short‐term outcomes of elderly patients discharged from an emergency department.J Am Geriatr Soc.1989;37(10):937–943. , , , , .
- The discharge of elderly patients from an accident and emergency department: functional changes and risk of readmission.Age Ageing.1990;19(6):415–418. , , , , .
- Factors associated with unplanned hospital readmission among patients 65 years of age and older in a Medicare managed care plan.Am J Med.1999;107(1):13–17. , , , , , .
- Posthospital medication discrepancies: prevalence and contributing factors.Arch Intern Med.2005;165(16):1842–1847. , , , .
- From the emergency department to home.J Clin Nurs.2005;14(6):776–785. , .
- Adverse drug events in elderly patients receiving home health services following hospital discharge.Ann Pharmacother.1999;33(11):1147–1153. , , .
- A randomized trial of the efficacy of multidisciplinary care in heart failure outpatients at high risk of hospital readmission.J Am Coll Cardiol.2002;39(3):471–480. , , , et al.
- A systematic review of randomized trials of disease management programs in heart failure.Am J Med.2001;110(5):378–384. , , , .
- Comprehensive discharge planning with postdischarge support for older patients with congestive heart failure: a meta‐analysis.JAMA.2004;291(11):1358–1367. , , , , , .
- Case management may reduce length of hospital stay in patients with recurrent admissions for chronic obstructive pulmonary disease.Respirology.2001;6(1):37–42. , , , .
- A chronic disease management programme can reduce days in hospital for patients with chronic obstructive pulmonary disease.Intern Med J.2004;34(11):608–614. , , , , , .
- Disease management programmes for older people with heart failure: crucial characteristics which improve post‐discharge outcomes.Eur Heart J.2006;27(5):596–612. , , .
- The care transitions intervention: results of a randomized controlled trial.Arch Intern Med.2006;166(17):1822–1828. , , , .
- Effects of a multidisciplinary, post‐discharge continuance of care intervention on quality of life, discharge satisfaction, and hospital length of stay: a randomized controlled trial.Int J Qual Health Care.2005;17(1):43–51. , , , et al.
- Discharge planning from hospital to home.Cochrane Database Syst Rev.2004(1):CD000313. , , , .
- A systematic review of interventions to improve outcomes for elders discharged from the emergency department.Acad Emerg Med.2005;12(10):978–986. , .
- Transitional care for older adults: a cost‐effective model.LDI Issue Brief.2004;9(6):1–4. .
- Comprehensive discharge planning and home follow‐up of hospitalized elders: a randomized clinical trial.JAMA.1999;281(7):613–620. , , , et al.
- A multidisciplinary intervention to prevent the readmission of elderly patients with congestive heart failure.N Engl J Med.1995;333(18):1190–1195. , , , , , .
- Home‐based intervention in congestive heart failure: long‐term implications on readmission and survival.Circulation.2002;105(24):2861–2866. , .
- Effectiveness of team‐managed home‐based primary care: a randomized multicenter trial.JAMA.2000;284(22):2877–2885. , , , et al.
- Home‐based medication review in older people: is it cost effective?Pharmacoeconomics.2007;25(2):171–180. , , , .
- The value of inpatient pharmaceutical counselling to elderly patients prior to discharge.Br J Clin Pharmacol.2002;54(6):657–664. , , , , .
- Does the addition of a pharmacist transition coordinator improve evidence‐based medication management and health outcomes in older adults moving from the hospital to a long‐term care facility? Results of a randomized, controlled trial.Am J Geriatr Pharmacother.2004;2(4):257–264. , , , , .
- Role of pharmacist counseling in preventing adverse drug events after hospitalization.Arch Intern Med.2006;166(5):565–571. , , , et al.
- An investigation of hospital generated pharmaceutical care when patients are discharged home from hospital.Br J Clin Pharmacol.1997;44(2):163–165. , , , , , .
- The community assessment risk screen (CARS): identifying elderly persons at risk for hospitalization or emergency department visit.Am J Manag Care.2000;6(8):925–933. , , .
- “Risk” factors affecting readmission of the elderly into the health care system.Med Care.1986;24(5):429–437. , , .
- Re‐admission to intensive care: identification of risk factors.Physiother Res Int.2005;10(3):154–163. , , .
- Preparing patients and caregivers to participate in care delivered across settings: the Care Transitions Intervention.J Am Geriatr Soc.2004;52(11):1817–1825. , , , , , .
- Impact of the rapid diagnosis of influenza on physician decision‐making and patient management in the pediatric emergency department: results of a randomized, prospective, controlled trial.Pediatrics.2003;112(2):363–367. , , , , .
- A survey on hospitalised community‐acquired pneumonia in Italy.Monaldi Arch Chest Dis.2006;65(2):82–88. , , , , .
- Determinants of the length of stay in intensive care and in hospital after coronary artery surgery.Br Heart J.1995;73(1):92–98. , , , , .
- Variation in duration of hospital stay between hospitals and between doctors within hospitals.Soc Sci Med.1993;37(6):833–839. , , .
- The impact of standardized order sets and intensive clinical case management on outcomes in community‐acquired pneumonia.Arch Intern Med.2007;167(15):1664–1669. , , , et al.
- Compliance with medication orders among the elderly after hospital discharge.Hosp Formul.1992;27(7):720–724. , , .
- Clinical pharmacists and inpatient medical care: a systematic review.Arch Intern Med.2006;166(9):955–964. , , , .
- Quality improvement and ethical oversight.Ann Intern Med.2007;146(9):680–681. .
- The ethics of using quality improvement methods in health care.Ann Intern Med.2007;146(9):666–673. , , , et al.
- Enrollment of elderly patients in clinical trials for cancer drug registration: a 7‐year experience by the US Food and Drug Administration.J Clin Oncol.2004;22(22):4626–4631. , , .
- Striving to recruit: the difficulties of conducting clinical research on elderly care home residents.J R Soc Med.2007;100(6):258–261. , , , .
- Quality‐improvement research and informed consent.N Engl J Med.2008;358(8):765–767. , .
- An evaluation of the impact of the ventilator care bundle.Nurs Crit Care.2005;10(5):242–246. , , , .
- Older adults in the emergency department: a systematic review of patterns of use, adverse outcomes, and effectiveness of interventions.Ann Emerg Med.2002;39(3):238–247. , .
- Emergency department utilization by the elderly: analysis of the National Hospital Ambulatory Medical Care Survey.Acad Emerg Med.1996;3(7):694–699. , , , , , .
- 2005 National Hospital Discharge Survey.Adv Data.2007(385):1–19. , .
- Short‐term outcomes of elderly patients discharged from an emergency department.J Am Geriatr Soc.1989;37(10):937–943. , , , , .
- The discharge of elderly patients from an accident and emergency department: functional changes and risk of readmission.Age Ageing.1990;19(6):415–418. , , , , .
- Factors associated with unplanned hospital readmission among patients 65 years of age and older in a Medicare managed care plan.Am J Med.1999;107(1):13–17. , , , , , .
- Posthospital medication discrepancies: prevalence and contributing factors.Arch Intern Med.2005;165(16):1842–1847. , , , .
- From the emergency department to home.J Clin Nurs.2005;14(6):776–785. , .
- Adverse drug events in elderly patients receiving home health services following hospital discharge.Ann Pharmacother.1999;33(11):1147–1153. , , .
- A randomized trial of the efficacy of multidisciplinary care in heart failure outpatients at high risk of hospital readmission.J Am Coll Cardiol.2002;39(3):471–480. , , , et al.
- A systematic review of randomized trials of disease management programs in heart failure.Am J Med.2001;110(5):378–384. , , , .
- Comprehensive discharge planning with postdischarge support for older patients with congestive heart failure: a meta‐analysis.JAMA.2004;291(11):1358–1367. , , , , , .
- Case management may reduce length of hospital stay in patients with recurrent admissions for chronic obstructive pulmonary disease.Respirology.2001;6(1):37–42. , , , .
- A chronic disease management programme can reduce days in hospital for patients with chronic obstructive pulmonary disease.Intern Med J.2004;34(11):608–614. , , , , , .
- Disease management programmes for older people with heart failure: crucial characteristics which improve post‐discharge outcomes.Eur Heart J.2006;27(5):596–612. , , .
- The care transitions intervention: results of a randomized controlled trial.Arch Intern Med.2006;166(17):1822–1828. , , , .
- Effects of a multidisciplinary, post‐discharge continuance of care intervention on quality of life, discharge satisfaction, and hospital length of stay: a randomized controlled trial.Int J Qual Health Care.2005;17(1):43–51. , , , et al.
- Discharge planning from hospital to home.Cochrane Database Syst Rev.2004(1):CD000313. , , , .
- A systematic review of interventions to improve outcomes for elders discharged from the emergency department.Acad Emerg Med.2005;12(10):978–986. , .
- Transitional care for older adults: a cost‐effective model.LDI Issue Brief.2004;9(6):1–4. .
- Comprehensive discharge planning and home follow‐up of hospitalized elders: a randomized clinical trial.JAMA.1999;281(7):613–620. , , , et al.
- A multidisciplinary intervention to prevent the readmission of elderly patients with congestive heart failure.N Engl J Med.1995;333(18):1190–1195. , , , , , .
- Home‐based intervention in congestive heart failure: long‐term implications on readmission and survival.Circulation.2002;105(24):2861–2866. , .
- Effectiveness of team‐managed home‐based primary care: a randomized multicenter trial.JAMA.2000;284(22):2877–2885. , , , et al.
- Home‐based medication review in older people: is it cost effective?Pharmacoeconomics.2007;25(2):171–180. , , , .
- The value of inpatient pharmaceutical counselling to elderly patients prior to discharge.Br J Clin Pharmacol.2002;54(6):657–664. , , , , .
- Does the addition of a pharmacist transition coordinator improve evidence‐based medication management and health outcomes in older adults moving from the hospital to a long‐term care facility? Results of a randomized, controlled trial.Am J Geriatr Pharmacother.2004;2(4):257–264. , , , , .
- Role of pharmacist counseling in preventing adverse drug events after hospitalization.Arch Intern Med.2006;166(5):565–571. , , , et al.
- An investigation of hospital generated pharmaceutical care when patients are discharged home from hospital.Br J Clin Pharmacol.1997;44(2):163–165. , , , , , .
- The community assessment risk screen (CARS): identifying elderly persons at risk for hospitalization or emergency department visit.Am J Manag Care.2000;6(8):925–933. , , .
- “Risk” factors affecting readmission of the elderly into the health care system.Med Care.1986;24(5):429–437. , , .
- Re‐admission to intensive care: identification of risk factors.Physiother Res Int.2005;10(3):154–163. , , .
- Preparing patients and caregivers to participate in care delivered across settings: the Care Transitions Intervention.J Am Geriatr Soc.2004;52(11):1817–1825. , , , , , .
- Impact of the rapid diagnosis of influenza on physician decision‐making and patient management in the pediatric emergency department: results of a randomized, prospective, controlled trial.Pediatrics.2003;112(2):363–367. , , , , .
- A survey on hospitalised community‐acquired pneumonia in Italy.Monaldi Arch Chest Dis.2006;65(2):82–88. , , , , .
- Determinants of the length of stay in intensive care and in hospital after coronary artery surgery.Br Heart J.1995;73(1):92–98. , , , , .
- Variation in duration of hospital stay between hospitals and between doctors within hospitals.Soc Sci Med.1993;37(6):833–839. , , .
- The impact of standardized order sets and intensive clinical case management on outcomes in community‐acquired pneumonia.Arch Intern Med.2007;167(15):1664–1669. , , , et al.
- Compliance with medication orders among the elderly after hospital discharge.Hosp Formul.1992;27(7):720–724. , , .
- Clinical pharmacists and inpatient medical care: a systematic review.Arch Intern Med.2006;166(9):955–964. , , , .
- Quality improvement and ethical oversight.Ann Intern Med.2007;146(9):680–681. .
- The ethics of using quality improvement methods in health care.Ann Intern Med.2007;146(9):666–673. , , , et al.
- Enrollment of elderly patients in clinical trials for cancer drug registration: a 7‐year experience by the US Food and Drug Administration.J Clin Oncol.2004;22(22):4626–4631. , , .
- Striving to recruit: the difficulties of conducting clinical research on elderly care home residents.J R Soc Med.2007;100(6):258–261. , , , .
- Quality‐improvement research and informed consent.N Engl J Med.2008;358(8):765–767. , .
- An evaluation of the impact of the ventilator care bundle.Nurs Crit Care.2005;10(5):242–246. , , , .
Copyright © 2009 Society of Hospital Medicine