User login
An Initiative to Improve 30-Day Readmission Rates Using a Transitions-of-Care Clinic Among a Mixed Urban and Rural Veteran Population
Hospital readmissions are a significant problem in the United States, affecting 15% to 30% of discharges and incurring costs of more than $17 billion annually.1 Timely posthospitalization follow-up visits are critical to ensure the effective transfer of patients to the outpatient setting; such visits reduce readmission rates as well as hospital length of stay and overall health care resource utilization.2-4 Patients who receive inadequate follow-up care (ie, within 4 weeks of discharge) are significantly more likely to be readmitted than those who receive close follow-up care.5
Due to the large clinical and financial consequences associated with hospital readmission, a variety of interventions have been studied, including home visits, telemonitoring, medication management, telephone calls, and postdischarge clinics.6,7 While studies have not shown postdischarge clinics to be universally efficacious in reducing readmission rates, there is increasing evidence of reduced readmission rates in clinics that target high-risk patients (eg, patients with congestive heart failure [CHF]) rather than the total population.2 A study by Hernandez et al that evaluated the relationship between early physician follow-up and 30-day readmissions showed a significantly lower readmission rate among hospitals with higher follow-up rates.8 Similarly, patients with CHF in a large, integrated health system who were seen within 7 days of discharge had an odds ratio (OR) of 0.81 (95% CI, 0.70-0.94) for 30-day readmissions.9
Transitions-of-care clinics (TOCC), designed to provide early postdischarge follow-up to high-risk patients, have been shown to reduce 30-day readmission rates,3,4,10,11 especially in clinics that have same-physician follow-up visits rather than follow-up visits with a community primary care physician (PCP).12 The most pronounced impact of postdischarge follow-up is seen in high-risk patients with high complexity or high severity of disease; however, complex rural patients are less likely to have access to specialty care.13 As a result, since rural residents must travel farther for specialty care, they are seen less frequently than their urban counterparts.14,15
Prior to our TOCC initiative, the Iowa City VA (ICVA) ranked in the fifth quintile of the Veterans Health Administration (VHA)
To meet these challenges, we implemented a TOCC to deliver timely postdischarge care focusing on high-risk and high-complexity patients. To address access-to-care issues of patients living in rural areas within the ICVA, we included virtual follow-up visits as a key component of our intervention.16,17 The aim of this project was to decrease 30-day readmission rates of ICVA patients by 20% within 12 months of implementation.
METHODS
Setting/Study Population
The ICVA serves 184,000 veterans stretched over 50 counties in eastern Iowa, western Illinois, and northern Missouri, with more than 60% of these patients residing in rural areas. Patients were initially eligible for the TOCC if they had an admission diagnosis of CHF and a CAN score > 85 at the time of discharge. The CAN score, developed by the VA to assess the risk of hospital readmission in individual patients, factors in several variables, including demographics, coexisting conditions, vital signs, utilization of services, pharmacy visits, and laboratory results. Patients in the top 5% (95-99) have a readmission rate of 20% at 90 days. Since the CAN is a proprietary tool, it may not be published in full; however, this assessment tool is commonly used and frequently cited in VA research.18-22 The CAN score is expressed as a percentile ranging from 0 (lowest risk) to 99 (highest risk). Patient eligibility was expanded during subsequent Plan-Do-Study-Act (PDSA) cycles, as outlined below. Patient eligibility was expanded during subsequent PDSA cycles (also outlined below). A review by a local institutional review board was obtained, and the study was classified as exempt due to the use of deidentified data. Standards for Quality Improvement Reporting Excellence 2.0 guidelines were used to construct the manuscript.
Magnitude Assessment
The numbers of discharges, readmissions within 30 days, emergency department (ED) visits by all discharged veterans, and veterans discharged with a CHF hospital diagnosis were recorded from February 2017 to February 2018, which were the 12 months immediately preceding the pilot implementation.
Intervention
The primary intervention was referral to the newly formed ICVA TOCC. The multidisciplinary TOCC team consisted of hospitalists, pharmacists, schedulers, and discharge planners/care managers. Patients were identified by the hospitalist team during admission; prior to hospital discharge, these patients were referred to TOCC discharge planners to schedule appropriate follow-up appointments. Virtual follow-up visits were conducted using a patient’s home technology; in cases where a patient lacked adequate technology capabilities (eg, no computer or internet access), the ICVA provided a tablet device with cellular internet capability for temporary use. Specific clinical activities included medication reconciliation by a pharmacist, follow-up of pending laboratory studies, imaging studies, pathology results, medical diagnosis education, counseling regarding dietary restrictions, and contingency planning outside of an ED visit in the event of a change in clinical status. In addition, the TOCC aimed to facilitate a smooth transition of care back to the PCP by arranging follow-up appointments, providing visit summaries, and scheduling consults with specialty care, as appropriate.
Measures
The primary objective measure was the 30-day readmission rate in the ICVA hospital. Secondary measures included the number of VHA ED visits within 30 days of discharge. The main process measures were the number of hospital discharges per month, the number of TOCC referrals, the number of TOCC appointments made, the number of virtual and in-person visits, and the percentage of appointment “no-shows.”
Implementation
The TOCC was piloted from April 2018 to October 2018. During the pilot phase, TOCC enrollment was limited to virtual appointments and to patients with an admission diagnosis of CHF and a CAN score of > 85. The TOCC had staff on-site 2 days a week; this included pharmacists to reconcile medications and hospitalists to address follow-up care needs.
The TOCC clinic was temporarily closed at the end of October 2018 to analyze pilot results. Based on stakeholder feedback, changes made as part of the second PDSA cycle included expanding eligibility criteria to any hospital admission diagnosis and to patients with a CAN score < 85 if the hospitalist team felt the patient was likely to benefit from TOCC follow-up. In addition, on-site clinic staffing was expanded from 2 to 5 days per week to improve access, and the option for an in-person visit was added based on concerns some veterans expressed regarding the use of the technology at home. Finally, a formal resident program was added, and the order set for referrals was simplified. The TOCC was restarted in February 2019, and TOCC metrics were reviewed monthly. By July 2019, we identified issues with TOCC referrals and appointment creation that required additional modifications to the intervention.
A third PDSA cycle was initiated in July 2019 and included major changes, notably the formation of a designated TOCC committee. The committee appointed a dedicated TOCC scheduler whose role was to reduce confusion regarding scheduling, to update the discharge instructions/orders template to lower incidences of “double-booking” that occurred with PCP and TOCC appointments, to modify discharge educational instruction regarding virtual visits and tablet use, to adjust the TOCC-PCP handoff, and to formalize interactions between discharge coordinators and residents to review possible referrals every morning (Appendix Figure 1).
Statistical Analysis
Run charts were constructed by plotting monthly primary outcome values and monthly process metrics (Figure, Appendix Figure 2, Appendix Figure 3). Chi-square tests were used to compare 30-day readmission rates before and after the intervention.
Mean (SD) or counts and percentages were used to describe the distribution of continuous and categorical variables, respectively. Kruskal-Wallis test, t test, or chi-square tests were used, as appropriate, across categories. Generalized linear models with a logistic link function were used to test for differences between patients who kept their appointment at the TOCC and those who did not keep their TOCC appointment (both unadjusted and adjusted for all of the covariates previously mentioned). In addition, generalized linear models were also used to compare outcomes between TOCC patients seen virtually vs those seen in-person (both unadjusted and adjusted for all the covariates previously mentioned). All statistical tests were considered significant at a two-sided P < .05. All analyses were performed using SAS software version 9.4 (SAS Institute Inc).
RESULTS
Magnitude Assessment
During the preimplementation period (February 2017-February 2018), there were 3014 patient discharges from ICVA and 343 readmissions, resulting in a readmission rate of 11.4%. Among patients with a hospital-admission diagnosis of cardiorespiratory disease, which included patients with CHF, there were 381 discharges and 46 readmissions, resulting in a readmission rate of 12.1%.
Primary Outcome
During the pilot phase, which was conducted from April 2018 to October 2018, 142 patients who met inclusion criteria (CHF diagnosis and a CAN score > 85) were discharged from ICVA, and 56 referrals to the TOCC were placed. The readmission rate among the cardiorespiratory cohort of veterans was 9.5%.
During the expansion of the intervention from February 2019 to February 2020, there were 2844 discharges from the ICVA and 291 readmissions, resulting in a readmission rate of 10.2%. However, there was a further decrease in the readmission rate after the third PDSA cycle was initiated in July 2019 (Appendix Figure 1). The readmission rate was 9.2% in the final 6 months of the intervention period, and 7.9% in the final 3 months.
When comparing the 6 months following the third PDSA cycle to the magnitude assessment period, there was a relative readmission reduction of 19.3% (P = .04), and an absolute reduction of 2.2%. If the final 3 months of the intervention period are included, there was an absolute reduction of 3.5% and a relative reduction of 30.7% (P = .01). Notably, before the pilot phase, ICVA was in the fifth quintile for HWR among VA hospitals but improved to the second quintile by the end of the expansion phase.
Process Outcomes
Process metrics for TOCC referrals, the number of patients seen, and the number of virtual and in-person visits over time are shown in Appendix Figure 3. Rates of TOCC referrals and the number of TOCC visits were lower than anticipated during the first 5 months of the intervention. However, TOCC referrals increased significantly after we implemented the previously described changes as part of the third PDSA cycle. As a result, total, virtual, and in-person visits also significantly increased from July 2019 to February 2020. The proportion of patients choosing virtual vs in-person visits fluctuated over time, but virtual visits were generally chosen more often than in-person visits.
Statistical Modeling
Baseline Data
Cohort characteristics are shown in Table 1. The cohort, which reflected the ICVA population, was predominantly male (96%) and White (93%), with a mean age of 67 years. The population was approximately half urban and half rural in composition, and the most common reason for hospital admission was cardiac. Other than a small but statistically significant difference in CAN scores, there were no significant differences between patients who kept their TOCC appointment and those who did not. There were also no differences in baseline characteristics between patients who chose virtual follow-up and patients who chose in-person follow-up, including the proportion of urban and rural patients.
Outcomes
Patients who kept their TOCC appointments had a 30-day readmission rate of 9.6%, which was significantly lower than the 30-day readmission rate of 27% in the group that did not keep their TOCC appointment (P < .001). Similarly, the percentage of patients treated in the ED was 15% in the TOCC group compared to 31.2% in the group that canceled their appointment (P < .001) (Table 1). In the multivariable analysis, patients who were seen in the TOCC group had an OR for 30-day readmission of 0.35 (95% CI, 0.19-0.62, P < .001), and an OR for ED visits of 0.39 (95% CI, 0.23-0.65; P < .001) (Table 2). There was no statistically significant difference in 6-month mortality between the two groups. In the virtual group compared to the in-person group, there were no statistically significant differences in outcomes between the two groups in the unadjusted or adjusted analysis (Table 2).
DISCUSSION
In the expansion phase, eligibility was expanded to include any hospital indication but continued to focus on high-risk patients. Existing literature suggests that providing postdischarge care to all patients, including low- or medium-risk patients, may not be as impactful as enrolling high-risk patients only. For instance, a postdischarge clinic offered to all patients at a VA system in Colorado did not reduce readmission rates compared to PCP follow-up.23 In contrast, a study of more than 10,000 high-risk urban patients demonstrated that postdischarge care resulted in a 9.3% reduction in readmission risk.24 Our data are consistent with the previously published studies, as the average CAN score of patients seen in TOCC was 90, suggesting a high risk of readmission. In the final 12 months of the intervention, 15% of discharged patients were seen at the TOCC clinic, suggesting that targeted intervention within the small subset of high-risk patients was sufficient to achieve our primary aim. Of note, among patients who did not meet the inclusion criteria for TOCC referral (ie, patients not considered high risk [CAN score ≤ 85]), the rate of readmissions was 8.6%.
Most of the available research on the efficacy of postdischarge clinics was conducted in urban environments. Our ICVA population sees a large proportion of rural veterans, who account for just over 50% of the discharge population. In a study of more than 2 million Medicare patients discharged from US hospitals, the 30-day readmission rates and adjusted mortality rates were higher among patients in rural counties, and post–acute care seemed to have a greater impact in rural rather than urban settings.25 Previous studies have demonstrated that virtual visits have the potential to improve readmission rates, especially in patients with CHF26 and in patients at the highest risk for readmission.27 In our study, the pilot phase offered only virtual visits, but we subsequently added an in-person option based on veteran feedback. Interestingly, over the next 12 months, virtual visits were more popular with both urban and rural veterans, and there were no differences in the number of rural patients in the in-person vs the virtual group. These findings suggest factors other than rurality influenced the decision to choose virtual follow-up visits over in-person visits. Future studies should seek to determine the extent to which factors such as age, race, educational level, and socioeconomic circumstances impact veterans’ follow-up decisions. Not only were outcomes among patients who chose virtual visits the same as those of patients who chose in-person visits, but both of these groups had better outcomes compared to the non-TOCC group (Table 2). This finding demonstrating the efficacy of virtual visits among rural and urban patients has taken on increased significance due to the COVID-19 pandemic, as virtual visits offer a safer option, one that minimizes physical contact.
Our quality improvement analysis included a statistical comparison of patients seen vs those not seen at the TOCC. Patients who were referred to the TOCC but chose not to keep their appointment were similar to those seen in TOCC in terms of age, CAN score, rurality, and hospital diagnosis, but readmission rates were substantially higher in this group even after adjustments for covariates (Table 2). Evaluating causality in interventions aimed to reduce hospital readmission rates is complicated.28 Our findings add greater plausibility to the utility of TOCC in accounting for at least a portion of the reported decrease in ICVA 30-day readmissions.
Our study has several strengths, including an observation period longer than 2 years, a large population of discharged veterans within an integrated healthcare system, and a large proportion of patients living in rural areas. Another strength of our study is the innovative nature of the intervention, which features a multidisciplinary team and the option of virtual or in-person visits. Nevertheless, this study also has several important limitations. As a single-center study, our findings may not be generalizable to other institutions, especially those outside the VHA system. Similarly, our study population reflected that of the ICVA, which may limit generalizability to a more diverse population. While we attempted to account in our statistical modeling for baseline differences between referred patients seen by the TOCC and those referred but not seen, we cannot exclude residual confounding between the groups. Specifically, the comparison of patients who did and did not choose TOCC follow-up introduces the possibility of selection bias. Future randomized/controlled studies will need to evaluate whether TOCC is more effective than the standard of care to reduce readmissions. Finally, since the analysis period following the final PDSA cycle was compressed due to the onset of the COVID-19 pandemic in the United States, no data are available regarding the sustained impacts of changes made during this cycle.
CONCLUSION
A multidisciplinary TOCC within the ICVA, featuring both virtual and in-person visits, reduced 30-day readmission rates by 19.3%; this approach to care was especially effective in patients with CHF. Virtual visits were the follow-up mode of choice for both urban and rural veterans, and there was no difference in outcomes between these two follow-up options. Future studies will focus on additional quality metrics, including cost-effectiveness and patient satisfaction.
1. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418-1428. https://doi.org/10.1056/nejmsa0803563
2. Doctoroff L. Postdischarge clinics and hospitalists: a review of the evidence and existing models. J Hosp Med. 2017;12(6):467-471. https://doi.org/10.12788/jhm.2750
3. Koehler BE, Richter KM, Youngblood L, et al. Reduction of 30-day postdischarge hospital readmission or emergency department (ED) visit rates in high-risk elderly medical patients through delivery of a targeted care bundle. J Hosp Med. 2009;4(4):211-218. https://doi.org/10.1002/jhm.427
4. Abrashkin KA, Cho HJ, Torgalkar S, Markoff B. Improving transitions of care from hospital to home: what works? Mt Sinai J Med. 2012;79(5):535-544. https://doi.org/10.1002/msj.21332
5. Misky GJ, Wald HL, Coleman EA. Post-hospitalization transitions: examining the effects of timing of primary care provider follow-up. J Hosp Med. 2010;5(7):392-397. https://doi.org/10.1002/jhm.666
6. Greysen SR, Harrison JD, Kripalani S, et al. Understanding patient-centred readmission factors: a multi-site, mixed-methods study. BMJ Qual Saf. 2017;26(1):33-41. https://doi.org/10.1136/bmjqs-2015-004570
7. Hansen LO, Young RS, Hinami K, Leung A, Williams MV. Interventions to reduce 30-day rehospitalization: a systematic review. Ann Intern Med. 2011;155(8):520-528. https://doi.org/10.7326/0003-4819-155-8-201110180-00008
8. Hernandez AF, Greiner MA, Fonarow GC, et al. Relationship between early physician follow-up and 30-day readmission among Medicare beneficiaries hospitalized for heart failure. JAMA. 2010;303(17):1716-1722. https://doi.org/10.1001/jama.2010.533
9. Lee KK, Yang J, Hernandez AF, Steimle AE, Go AS. Post-discharge follow-up characteristics associated with 30-day readmission after heart failure hospitalization. Med Care. 2016;54(4):365-372. https://doi.org/10.1097/mlr.0000000000000492
10. Balaban RB, Williams MV. Improving care transitions: hospitalists partnering with primary care. J Hosp Med. 2010;5(7):375-377. https://doi.org/10.1002/jhm.824
11. Rodrigues CR, Harrington AR, Murdock N, et al. Effect of pharmacy-supported transition-of-care interventions on 30-day readmissions: a systematic review and meta-analysis. Ann Pharmacother. 2017;51(10):866-889. https://doi.org/10.1177/1060028017712725
12. van Walraven C, Taljaard M, Etchells E, et al. The independent association of provider and information continuity on outcomes after hospital discharge: implications for hospitalists. J Hosp Med. 2010;5(7):398-405. https://doi.org/10.1002/jhm.716
13. Gruca TS, Pyo TH, Nelson GC. Providing cardiology care in rural areas through vsiting consultant clinics. J Am Heart Assoc. 2016;5(7):e002909. https://doi.org/10.1161/jaha.115.002909
14. Chan L, Hart LG, Goodman DC. Geographic access to health care for rural Medicare beneficiaries. J Rural Health. 2006;22(2):140-146. https://doi.org/10.1111/j.1748-0361.2006.00022.x
15. Burke RE, Jones CD, Coleman EA, Falvey JR, Stevens-Lapsley JE, Ginde AA. Use of post-acute care after hospital discharge in urban and rural hospitals. Am J Accountable Care. 2017;5(1):16-22.
16. Jetty A, Moore MA, Coffman M, Petterson S, Bazemore A. Rural family physicians are twice as likely to use telehealth as urban family physicians. Telemed J E Health. 2018;24(4):268-276. https://doi.org/10.1089/tmj.2017.0161
17. Harrison PL, Hara PA, Pope JE, Young MC, Rula EY. The impact of postdischarge telephonic follow-up on hospital readmissions. Popul Health Manag. 2011;14(1):27-32. https://doi.org/10.1089/pop.2009.0076
18. Wang L, Porter B, Maynard C, et al. Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration. Med Care. 2013;51(4):368-373. https://doi.org/10.1097/mlr.0b013e31827da95a
19. Spece LJ, Donovan LM, Griffith MF, et al. Initiating low-value inhaled corticosteroids in an inception cohort with chronic obstructive pulmonary disease. Ann Am Thorac Soc. 2020;17(5):589-595. https://doi.org/10.1513/annalsats.201911-854oc
20. Osborne TF, Suarez P, Edwards D, Hernandez-Boussard T, Curtin C. Patient electronic health records score for preoperative risk assessment before total knee arthroplasty. JB JS Open Access. 2020;5(2):e0061. https://doi.org/10.2106/jbjs.oa.19.00061
21. Levy C, Ersek M, Scott W, et al. Life-sustaining treatment decisions initiative: early implementation results of a national Veterans Affairs program to honor veterans’ care preferences. J Gen Intern Med. 2020;35(6):1803-1812. https://doi.org/10.1007/s11606-020-05697-2
22. Ibrahim SA. High-risk patients and utilization of primary care in the US Veterans Affairs health system. JAMA Netw Open. 2020;3(6):e209518. https://doi.org/10.1001/jamanetworkopen.2020.9518
23. Burke RE, Whitfield E, Prochazka AV. Effect of a hospitalist-run postdischarge clinic on outcomes. J Hosp Med. 2014;9(1):7-12. https://doi.org/10.1002/jhm.2099
24. Jenq GY, Doyle MM, Belton BM, Herrin J, Horwitz LI. Quasi-experimental evaluation of the effectiveness of a large-scale readmission reduction program. JAMA Intern Med. 2016;176(5):681-690. https://doi.org/10.1001/jamainternmed.2016.0833
25. Kosar CM, Loomer L, Ferdows NB, Trivedi AN, Panagiotou OA, Rahman M. Assessment of rural-urban differences in postacute care utilization and outcomes among older US adults. JAMA Netw Open. 2020;3(1):e1918738. https://doi.org/10.1001/jamanetworkopen.2019.18738
26. Pandor A, Thokala P, Gomersall T, et al. Home telemonitoring or structured telephone support programmes after recent discharge in patients with heart failure: systematic review and economic evaluation. Health Technol Assess. 2013;17(32):1-207, v-vi. https://doi.org/10.3310/hta17320
27. Low LL, Tan SY, Ng MJM, et al. Applying the integrated practice unit concept to a modified virtual ward model of care for patients at highest risk of readmission: a randomized controlled trial. PloS One. 2017;12(1):e0168757. https://doi.org/10.1371/journal.pone.0168757
28. McIlvennan CK, Eapen ZJ, Allen LA. Hospital readmissions reduction program. Circulation. 2015;131(20):1796-1803. https://doi.org/10.1161/circulationaha.114.010270
Hospital readmissions are a significant problem in the United States, affecting 15% to 30% of discharges and incurring costs of more than $17 billion annually.1 Timely posthospitalization follow-up visits are critical to ensure the effective transfer of patients to the outpatient setting; such visits reduce readmission rates as well as hospital length of stay and overall health care resource utilization.2-4 Patients who receive inadequate follow-up care (ie, within 4 weeks of discharge) are significantly more likely to be readmitted than those who receive close follow-up care.5
Due to the large clinical and financial consequences associated with hospital readmission, a variety of interventions have been studied, including home visits, telemonitoring, medication management, telephone calls, and postdischarge clinics.6,7 While studies have not shown postdischarge clinics to be universally efficacious in reducing readmission rates, there is increasing evidence of reduced readmission rates in clinics that target high-risk patients (eg, patients with congestive heart failure [CHF]) rather than the total population.2 A study by Hernandez et al that evaluated the relationship between early physician follow-up and 30-day readmissions showed a significantly lower readmission rate among hospitals with higher follow-up rates.8 Similarly, patients with CHF in a large, integrated health system who were seen within 7 days of discharge had an odds ratio (OR) of 0.81 (95% CI, 0.70-0.94) for 30-day readmissions.9
Transitions-of-care clinics (TOCC), designed to provide early postdischarge follow-up to high-risk patients, have been shown to reduce 30-day readmission rates,3,4,10,11 especially in clinics that have same-physician follow-up visits rather than follow-up visits with a community primary care physician (PCP).12 The most pronounced impact of postdischarge follow-up is seen in high-risk patients with high complexity or high severity of disease; however, complex rural patients are less likely to have access to specialty care.13 As a result, since rural residents must travel farther for specialty care, they are seen less frequently than their urban counterparts.14,15
Prior to our TOCC initiative, the Iowa City VA (ICVA) ranked in the fifth quintile of the Veterans Health Administration (VHA)
To meet these challenges, we implemented a TOCC to deliver timely postdischarge care focusing on high-risk and high-complexity patients. To address access-to-care issues of patients living in rural areas within the ICVA, we included virtual follow-up visits as a key component of our intervention.16,17 The aim of this project was to decrease 30-day readmission rates of ICVA patients by 20% within 12 months of implementation.
METHODS
Setting/Study Population
The ICVA serves 184,000 veterans stretched over 50 counties in eastern Iowa, western Illinois, and northern Missouri, with more than 60% of these patients residing in rural areas. Patients were initially eligible for the TOCC if they had an admission diagnosis of CHF and a CAN score > 85 at the time of discharge. The CAN score, developed by the VA to assess the risk of hospital readmission in individual patients, factors in several variables, including demographics, coexisting conditions, vital signs, utilization of services, pharmacy visits, and laboratory results. Patients in the top 5% (95-99) have a readmission rate of 20% at 90 days. Since the CAN is a proprietary tool, it may not be published in full; however, this assessment tool is commonly used and frequently cited in VA research.18-22 The CAN score is expressed as a percentile ranging from 0 (lowest risk) to 99 (highest risk). Patient eligibility was expanded during subsequent Plan-Do-Study-Act (PDSA) cycles, as outlined below. Patient eligibility was expanded during subsequent PDSA cycles (also outlined below). A review by a local institutional review board was obtained, and the study was classified as exempt due to the use of deidentified data. Standards for Quality Improvement Reporting Excellence 2.0 guidelines were used to construct the manuscript.
Magnitude Assessment
The numbers of discharges, readmissions within 30 days, emergency department (ED) visits by all discharged veterans, and veterans discharged with a CHF hospital diagnosis were recorded from February 2017 to February 2018, which were the 12 months immediately preceding the pilot implementation.
Intervention
The primary intervention was referral to the newly formed ICVA TOCC. The multidisciplinary TOCC team consisted of hospitalists, pharmacists, schedulers, and discharge planners/care managers. Patients were identified by the hospitalist team during admission; prior to hospital discharge, these patients were referred to TOCC discharge planners to schedule appropriate follow-up appointments. Virtual follow-up visits were conducted using a patient’s home technology; in cases where a patient lacked adequate technology capabilities (eg, no computer or internet access), the ICVA provided a tablet device with cellular internet capability for temporary use. Specific clinical activities included medication reconciliation by a pharmacist, follow-up of pending laboratory studies, imaging studies, pathology results, medical diagnosis education, counseling regarding dietary restrictions, and contingency planning outside of an ED visit in the event of a change in clinical status. In addition, the TOCC aimed to facilitate a smooth transition of care back to the PCP by arranging follow-up appointments, providing visit summaries, and scheduling consults with specialty care, as appropriate.
Measures
The primary objective measure was the 30-day readmission rate in the ICVA hospital. Secondary measures included the number of VHA ED visits within 30 days of discharge. The main process measures were the number of hospital discharges per month, the number of TOCC referrals, the number of TOCC appointments made, the number of virtual and in-person visits, and the percentage of appointment “no-shows.”
Implementation
The TOCC was piloted from April 2018 to October 2018. During the pilot phase, TOCC enrollment was limited to virtual appointments and to patients with an admission diagnosis of CHF and a CAN score of > 85. The TOCC had staff on-site 2 days a week; this included pharmacists to reconcile medications and hospitalists to address follow-up care needs.
The TOCC clinic was temporarily closed at the end of October 2018 to analyze pilot results. Based on stakeholder feedback, changes made as part of the second PDSA cycle included expanding eligibility criteria to any hospital admission diagnosis and to patients with a CAN score < 85 if the hospitalist team felt the patient was likely to benefit from TOCC follow-up. In addition, on-site clinic staffing was expanded from 2 to 5 days per week to improve access, and the option for an in-person visit was added based on concerns some veterans expressed regarding the use of the technology at home. Finally, a formal resident program was added, and the order set for referrals was simplified. The TOCC was restarted in February 2019, and TOCC metrics were reviewed monthly. By July 2019, we identified issues with TOCC referrals and appointment creation that required additional modifications to the intervention.
A third PDSA cycle was initiated in July 2019 and included major changes, notably the formation of a designated TOCC committee. The committee appointed a dedicated TOCC scheduler whose role was to reduce confusion regarding scheduling, to update the discharge instructions/orders template to lower incidences of “double-booking” that occurred with PCP and TOCC appointments, to modify discharge educational instruction regarding virtual visits and tablet use, to adjust the TOCC-PCP handoff, and to formalize interactions between discharge coordinators and residents to review possible referrals every morning (Appendix Figure 1).
Statistical Analysis
Run charts were constructed by plotting monthly primary outcome values and monthly process metrics (Figure, Appendix Figure 2, Appendix Figure 3). Chi-square tests were used to compare 30-day readmission rates before and after the intervention.
Mean (SD) or counts and percentages were used to describe the distribution of continuous and categorical variables, respectively. Kruskal-Wallis test, t test, or chi-square tests were used, as appropriate, across categories. Generalized linear models with a logistic link function were used to test for differences between patients who kept their appointment at the TOCC and those who did not keep their TOCC appointment (both unadjusted and adjusted for all of the covariates previously mentioned). In addition, generalized linear models were also used to compare outcomes between TOCC patients seen virtually vs those seen in-person (both unadjusted and adjusted for all the covariates previously mentioned). All statistical tests were considered significant at a two-sided P < .05. All analyses were performed using SAS software version 9.4 (SAS Institute Inc).
RESULTS
Magnitude Assessment
During the preimplementation period (February 2017-February 2018), there were 3014 patient discharges from ICVA and 343 readmissions, resulting in a readmission rate of 11.4%. Among patients with a hospital-admission diagnosis of cardiorespiratory disease, which included patients with CHF, there were 381 discharges and 46 readmissions, resulting in a readmission rate of 12.1%.
Primary Outcome
During the pilot phase, which was conducted from April 2018 to October 2018, 142 patients who met inclusion criteria (CHF diagnosis and a CAN score > 85) were discharged from ICVA, and 56 referrals to the TOCC were placed. The readmission rate among the cardiorespiratory cohort of veterans was 9.5%.
During the expansion of the intervention from February 2019 to February 2020, there were 2844 discharges from the ICVA and 291 readmissions, resulting in a readmission rate of 10.2%. However, there was a further decrease in the readmission rate after the third PDSA cycle was initiated in July 2019 (Appendix Figure 1). The readmission rate was 9.2% in the final 6 months of the intervention period, and 7.9% in the final 3 months.
When comparing the 6 months following the third PDSA cycle to the magnitude assessment period, there was a relative readmission reduction of 19.3% (P = .04), and an absolute reduction of 2.2%. If the final 3 months of the intervention period are included, there was an absolute reduction of 3.5% and a relative reduction of 30.7% (P = .01). Notably, before the pilot phase, ICVA was in the fifth quintile for HWR among VA hospitals but improved to the second quintile by the end of the expansion phase.
Process Outcomes
Process metrics for TOCC referrals, the number of patients seen, and the number of virtual and in-person visits over time are shown in Appendix Figure 3. Rates of TOCC referrals and the number of TOCC visits were lower than anticipated during the first 5 months of the intervention. However, TOCC referrals increased significantly after we implemented the previously described changes as part of the third PDSA cycle. As a result, total, virtual, and in-person visits also significantly increased from July 2019 to February 2020. The proportion of patients choosing virtual vs in-person visits fluctuated over time, but virtual visits were generally chosen more often than in-person visits.
Statistical Modeling
Baseline Data
Cohort characteristics are shown in Table 1. The cohort, which reflected the ICVA population, was predominantly male (96%) and White (93%), with a mean age of 67 years. The population was approximately half urban and half rural in composition, and the most common reason for hospital admission was cardiac. Other than a small but statistically significant difference in CAN scores, there were no significant differences between patients who kept their TOCC appointment and those who did not. There were also no differences in baseline characteristics between patients who chose virtual follow-up and patients who chose in-person follow-up, including the proportion of urban and rural patients.
Outcomes
Patients who kept their TOCC appointments had a 30-day readmission rate of 9.6%, which was significantly lower than the 30-day readmission rate of 27% in the group that did not keep their TOCC appointment (P < .001). Similarly, the percentage of patients treated in the ED was 15% in the TOCC group compared to 31.2% in the group that canceled their appointment (P < .001) (Table 1). In the multivariable analysis, patients who were seen in the TOCC group had an OR for 30-day readmission of 0.35 (95% CI, 0.19-0.62, P < .001), and an OR for ED visits of 0.39 (95% CI, 0.23-0.65; P < .001) (Table 2). There was no statistically significant difference in 6-month mortality between the two groups. In the virtual group compared to the in-person group, there were no statistically significant differences in outcomes between the two groups in the unadjusted or adjusted analysis (Table 2).
DISCUSSION
In the expansion phase, eligibility was expanded to include any hospital indication but continued to focus on high-risk patients. Existing literature suggests that providing postdischarge care to all patients, including low- or medium-risk patients, may not be as impactful as enrolling high-risk patients only. For instance, a postdischarge clinic offered to all patients at a VA system in Colorado did not reduce readmission rates compared to PCP follow-up.23 In contrast, a study of more than 10,000 high-risk urban patients demonstrated that postdischarge care resulted in a 9.3% reduction in readmission risk.24 Our data are consistent with the previously published studies, as the average CAN score of patients seen in TOCC was 90, suggesting a high risk of readmission. In the final 12 months of the intervention, 15% of discharged patients were seen at the TOCC clinic, suggesting that targeted intervention within the small subset of high-risk patients was sufficient to achieve our primary aim. Of note, among patients who did not meet the inclusion criteria for TOCC referral (ie, patients not considered high risk [CAN score ≤ 85]), the rate of readmissions was 8.6%.
Most of the available research on the efficacy of postdischarge clinics was conducted in urban environments. Our ICVA population sees a large proportion of rural veterans, who account for just over 50% of the discharge population. In a study of more than 2 million Medicare patients discharged from US hospitals, the 30-day readmission rates and adjusted mortality rates were higher among patients in rural counties, and post–acute care seemed to have a greater impact in rural rather than urban settings.25 Previous studies have demonstrated that virtual visits have the potential to improve readmission rates, especially in patients with CHF26 and in patients at the highest risk for readmission.27 In our study, the pilot phase offered only virtual visits, but we subsequently added an in-person option based on veteran feedback. Interestingly, over the next 12 months, virtual visits were more popular with both urban and rural veterans, and there were no differences in the number of rural patients in the in-person vs the virtual group. These findings suggest factors other than rurality influenced the decision to choose virtual follow-up visits over in-person visits. Future studies should seek to determine the extent to which factors such as age, race, educational level, and socioeconomic circumstances impact veterans’ follow-up decisions. Not only were outcomes among patients who chose virtual visits the same as those of patients who chose in-person visits, but both of these groups had better outcomes compared to the non-TOCC group (Table 2). This finding demonstrating the efficacy of virtual visits among rural and urban patients has taken on increased significance due to the COVID-19 pandemic, as virtual visits offer a safer option, one that minimizes physical contact.
Our quality improvement analysis included a statistical comparison of patients seen vs those not seen at the TOCC. Patients who were referred to the TOCC but chose not to keep their appointment were similar to those seen in TOCC in terms of age, CAN score, rurality, and hospital diagnosis, but readmission rates were substantially higher in this group even after adjustments for covariates (Table 2). Evaluating causality in interventions aimed to reduce hospital readmission rates is complicated.28 Our findings add greater plausibility to the utility of TOCC in accounting for at least a portion of the reported decrease in ICVA 30-day readmissions.
Our study has several strengths, including an observation period longer than 2 years, a large population of discharged veterans within an integrated healthcare system, and a large proportion of patients living in rural areas. Another strength of our study is the innovative nature of the intervention, which features a multidisciplinary team and the option of virtual or in-person visits. Nevertheless, this study also has several important limitations. As a single-center study, our findings may not be generalizable to other institutions, especially those outside the VHA system. Similarly, our study population reflected that of the ICVA, which may limit generalizability to a more diverse population. While we attempted to account in our statistical modeling for baseline differences between referred patients seen by the TOCC and those referred but not seen, we cannot exclude residual confounding between the groups. Specifically, the comparison of patients who did and did not choose TOCC follow-up introduces the possibility of selection bias. Future randomized/controlled studies will need to evaluate whether TOCC is more effective than the standard of care to reduce readmissions. Finally, since the analysis period following the final PDSA cycle was compressed due to the onset of the COVID-19 pandemic in the United States, no data are available regarding the sustained impacts of changes made during this cycle.
CONCLUSION
A multidisciplinary TOCC within the ICVA, featuring both virtual and in-person visits, reduced 30-day readmission rates by 19.3%; this approach to care was especially effective in patients with CHF. Virtual visits were the follow-up mode of choice for both urban and rural veterans, and there was no difference in outcomes between these two follow-up options. Future studies will focus on additional quality metrics, including cost-effectiveness and patient satisfaction.
Hospital readmissions are a significant problem in the United States, affecting 15% to 30% of discharges and incurring costs of more than $17 billion annually.1 Timely posthospitalization follow-up visits are critical to ensure the effective transfer of patients to the outpatient setting; such visits reduce readmission rates as well as hospital length of stay and overall health care resource utilization.2-4 Patients who receive inadequate follow-up care (ie, within 4 weeks of discharge) are significantly more likely to be readmitted than those who receive close follow-up care.5
Due to the large clinical and financial consequences associated with hospital readmission, a variety of interventions have been studied, including home visits, telemonitoring, medication management, telephone calls, and postdischarge clinics.6,7 While studies have not shown postdischarge clinics to be universally efficacious in reducing readmission rates, there is increasing evidence of reduced readmission rates in clinics that target high-risk patients (eg, patients with congestive heart failure [CHF]) rather than the total population.2 A study by Hernandez et al that evaluated the relationship between early physician follow-up and 30-day readmissions showed a significantly lower readmission rate among hospitals with higher follow-up rates.8 Similarly, patients with CHF in a large, integrated health system who were seen within 7 days of discharge had an odds ratio (OR) of 0.81 (95% CI, 0.70-0.94) for 30-day readmissions.9
Transitions-of-care clinics (TOCC), designed to provide early postdischarge follow-up to high-risk patients, have been shown to reduce 30-day readmission rates,3,4,10,11 especially in clinics that have same-physician follow-up visits rather than follow-up visits with a community primary care physician (PCP).12 The most pronounced impact of postdischarge follow-up is seen in high-risk patients with high complexity or high severity of disease; however, complex rural patients are less likely to have access to specialty care.13 As a result, since rural residents must travel farther for specialty care, they are seen less frequently than their urban counterparts.14,15
Prior to our TOCC initiative, the Iowa City VA (ICVA) ranked in the fifth quintile of the Veterans Health Administration (VHA)
To meet these challenges, we implemented a TOCC to deliver timely postdischarge care focusing on high-risk and high-complexity patients. To address access-to-care issues of patients living in rural areas within the ICVA, we included virtual follow-up visits as a key component of our intervention.16,17 The aim of this project was to decrease 30-day readmission rates of ICVA patients by 20% within 12 months of implementation.
METHODS
Setting/Study Population
The ICVA serves 184,000 veterans stretched over 50 counties in eastern Iowa, western Illinois, and northern Missouri, with more than 60% of these patients residing in rural areas. Patients were initially eligible for the TOCC if they had an admission diagnosis of CHF and a CAN score > 85 at the time of discharge. The CAN score, developed by the VA to assess the risk of hospital readmission in individual patients, factors in several variables, including demographics, coexisting conditions, vital signs, utilization of services, pharmacy visits, and laboratory results. Patients in the top 5% (95-99) have a readmission rate of 20% at 90 days. Since the CAN is a proprietary tool, it may not be published in full; however, this assessment tool is commonly used and frequently cited in VA research.18-22 The CAN score is expressed as a percentile ranging from 0 (lowest risk) to 99 (highest risk). Patient eligibility was expanded during subsequent Plan-Do-Study-Act (PDSA) cycles, as outlined below. Patient eligibility was expanded during subsequent PDSA cycles (also outlined below). A review by a local institutional review board was obtained, and the study was classified as exempt due to the use of deidentified data. Standards for Quality Improvement Reporting Excellence 2.0 guidelines were used to construct the manuscript.
Magnitude Assessment
The numbers of discharges, readmissions within 30 days, emergency department (ED) visits by all discharged veterans, and veterans discharged with a CHF hospital diagnosis were recorded from February 2017 to February 2018, which were the 12 months immediately preceding the pilot implementation.
Intervention
The primary intervention was referral to the newly formed ICVA TOCC. The multidisciplinary TOCC team consisted of hospitalists, pharmacists, schedulers, and discharge planners/care managers. Patients were identified by the hospitalist team during admission; prior to hospital discharge, these patients were referred to TOCC discharge planners to schedule appropriate follow-up appointments. Virtual follow-up visits were conducted using a patient’s home technology; in cases where a patient lacked adequate technology capabilities (eg, no computer or internet access), the ICVA provided a tablet device with cellular internet capability for temporary use. Specific clinical activities included medication reconciliation by a pharmacist, follow-up of pending laboratory studies, imaging studies, pathology results, medical diagnosis education, counseling regarding dietary restrictions, and contingency planning outside of an ED visit in the event of a change in clinical status. In addition, the TOCC aimed to facilitate a smooth transition of care back to the PCP by arranging follow-up appointments, providing visit summaries, and scheduling consults with specialty care, as appropriate.
Measures
The primary objective measure was the 30-day readmission rate in the ICVA hospital. Secondary measures included the number of VHA ED visits within 30 days of discharge. The main process measures were the number of hospital discharges per month, the number of TOCC referrals, the number of TOCC appointments made, the number of virtual and in-person visits, and the percentage of appointment “no-shows.”
Implementation
The TOCC was piloted from April 2018 to October 2018. During the pilot phase, TOCC enrollment was limited to virtual appointments and to patients with an admission diagnosis of CHF and a CAN score of > 85. The TOCC had staff on-site 2 days a week; this included pharmacists to reconcile medications and hospitalists to address follow-up care needs.
The TOCC clinic was temporarily closed at the end of October 2018 to analyze pilot results. Based on stakeholder feedback, changes made as part of the second PDSA cycle included expanding eligibility criteria to any hospital admission diagnosis and to patients with a CAN score < 85 if the hospitalist team felt the patient was likely to benefit from TOCC follow-up. In addition, on-site clinic staffing was expanded from 2 to 5 days per week to improve access, and the option for an in-person visit was added based on concerns some veterans expressed regarding the use of the technology at home. Finally, a formal resident program was added, and the order set for referrals was simplified. The TOCC was restarted in February 2019, and TOCC metrics were reviewed monthly. By July 2019, we identified issues with TOCC referrals and appointment creation that required additional modifications to the intervention.
A third PDSA cycle was initiated in July 2019 and included major changes, notably the formation of a designated TOCC committee. The committee appointed a dedicated TOCC scheduler whose role was to reduce confusion regarding scheduling, to update the discharge instructions/orders template to lower incidences of “double-booking” that occurred with PCP and TOCC appointments, to modify discharge educational instruction regarding virtual visits and tablet use, to adjust the TOCC-PCP handoff, and to formalize interactions between discharge coordinators and residents to review possible referrals every morning (Appendix Figure 1).
Statistical Analysis
Run charts were constructed by plotting monthly primary outcome values and monthly process metrics (Figure, Appendix Figure 2, Appendix Figure 3). Chi-square tests were used to compare 30-day readmission rates before and after the intervention.
Mean (SD) or counts and percentages were used to describe the distribution of continuous and categorical variables, respectively. Kruskal-Wallis test, t test, or chi-square tests were used, as appropriate, across categories. Generalized linear models with a logistic link function were used to test for differences between patients who kept their appointment at the TOCC and those who did not keep their TOCC appointment (both unadjusted and adjusted for all of the covariates previously mentioned). In addition, generalized linear models were also used to compare outcomes between TOCC patients seen virtually vs those seen in-person (both unadjusted and adjusted for all the covariates previously mentioned). All statistical tests were considered significant at a two-sided P < .05. All analyses were performed using SAS software version 9.4 (SAS Institute Inc).
RESULTS
Magnitude Assessment
During the preimplementation period (February 2017-February 2018), there were 3014 patient discharges from ICVA and 343 readmissions, resulting in a readmission rate of 11.4%. Among patients with a hospital-admission diagnosis of cardiorespiratory disease, which included patients with CHF, there were 381 discharges and 46 readmissions, resulting in a readmission rate of 12.1%.
Primary Outcome
During the pilot phase, which was conducted from April 2018 to October 2018, 142 patients who met inclusion criteria (CHF diagnosis and a CAN score > 85) were discharged from ICVA, and 56 referrals to the TOCC were placed. The readmission rate among the cardiorespiratory cohort of veterans was 9.5%.
During the expansion of the intervention from February 2019 to February 2020, there were 2844 discharges from the ICVA and 291 readmissions, resulting in a readmission rate of 10.2%. However, there was a further decrease in the readmission rate after the third PDSA cycle was initiated in July 2019 (Appendix Figure 1). The readmission rate was 9.2% in the final 6 months of the intervention period, and 7.9% in the final 3 months.
When comparing the 6 months following the third PDSA cycle to the magnitude assessment period, there was a relative readmission reduction of 19.3% (P = .04), and an absolute reduction of 2.2%. If the final 3 months of the intervention period are included, there was an absolute reduction of 3.5% and a relative reduction of 30.7% (P = .01). Notably, before the pilot phase, ICVA was in the fifth quintile for HWR among VA hospitals but improved to the second quintile by the end of the expansion phase.
Process Outcomes
Process metrics for TOCC referrals, the number of patients seen, and the number of virtual and in-person visits over time are shown in Appendix Figure 3. Rates of TOCC referrals and the number of TOCC visits were lower than anticipated during the first 5 months of the intervention. However, TOCC referrals increased significantly after we implemented the previously described changes as part of the third PDSA cycle. As a result, total, virtual, and in-person visits also significantly increased from July 2019 to February 2020. The proportion of patients choosing virtual vs in-person visits fluctuated over time, but virtual visits were generally chosen more often than in-person visits.
Statistical Modeling
Baseline Data
Cohort characteristics are shown in Table 1. The cohort, which reflected the ICVA population, was predominantly male (96%) and White (93%), with a mean age of 67 years. The population was approximately half urban and half rural in composition, and the most common reason for hospital admission was cardiac. Other than a small but statistically significant difference in CAN scores, there were no significant differences between patients who kept their TOCC appointment and those who did not. There were also no differences in baseline characteristics between patients who chose virtual follow-up and patients who chose in-person follow-up, including the proportion of urban and rural patients.
Outcomes
Patients who kept their TOCC appointments had a 30-day readmission rate of 9.6%, which was significantly lower than the 30-day readmission rate of 27% in the group that did not keep their TOCC appointment (P < .001). Similarly, the percentage of patients treated in the ED was 15% in the TOCC group compared to 31.2% in the group that canceled their appointment (P < .001) (Table 1). In the multivariable analysis, patients who were seen in the TOCC group had an OR for 30-day readmission of 0.35 (95% CI, 0.19-0.62, P < .001), and an OR for ED visits of 0.39 (95% CI, 0.23-0.65; P < .001) (Table 2). There was no statistically significant difference in 6-month mortality between the two groups. In the virtual group compared to the in-person group, there were no statistically significant differences in outcomes between the two groups in the unadjusted or adjusted analysis (Table 2).
DISCUSSION
In the expansion phase, eligibility was expanded to include any hospital indication but continued to focus on high-risk patients. Existing literature suggests that providing postdischarge care to all patients, including low- or medium-risk patients, may not be as impactful as enrolling high-risk patients only. For instance, a postdischarge clinic offered to all patients at a VA system in Colorado did not reduce readmission rates compared to PCP follow-up.23 In contrast, a study of more than 10,000 high-risk urban patients demonstrated that postdischarge care resulted in a 9.3% reduction in readmission risk.24 Our data are consistent with the previously published studies, as the average CAN score of patients seen in TOCC was 90, suggesting a high risk of readmission. In the final 12 months of the intervention, 15% of discharged patients were seen at the TOCC clinic, suggesting that targeted intervention within the small subset of high-risk patients was sufficient to achieve our primary aim. Of note, among patients who did not meet the inclusion criteria for TOCC referral (ie, patients not considered high risk [CAN score ≤ 85]), the rate of readmissions was 8.6%.
Most of the available research on the efficacy of postdischarge clinics was conducted in urban environments. Our ICVA population sees a large proportion of rural veterans, who account for just over 50% of the discharge population. In a study of more than 2 million Medicare patients discharged from US hospitals, the 30-day readmission rates and adjusted mortality rates were higher among patients in rural counties, and post–acute care seemed to have a greater impact in rural rather than urban settings.25 Previous studies have demonstrated that virtual visits have the potential to improve readmission rates, especially in patients with CHF26 and in patients at the highest risk for readmission.27 In our study, the pilot phase offered only virtual visits, but we subsequently added an in-person option based on veteran feedback. Interestingly, over the next 12 months, virtual visits were more popular with both urban and rural veterans, and there were no differences in the number of rural patients in the in-person vs the virtual group. These findings suggest factors other than rurality influenced the decision to choose virtual follow-up visits over in-person visits. Future studies should seek to determine the extent to which factors such as age, race, educational level, and socioeconomic circumstances impact veterans’ follow-up decisions. Not only were outcomes among patients who chose virtual visits the same as those of patients who chose in-person visits, but both of these groups had better outcomes compared to the non-TOCC group (Table 2). This finding demonstrating the efficacy of virtual visits among rural and urban patients has taken on increased significance due to the COVID-19 pandemic, as virtual visits offer a safer option, one that minimizes physical contact.
Our quality improvement analysis included a statistical comparison of patients seen vs those not seen at the TOCC. Patients who were referred to the TOCC but chose not to keep their appointment were similar to those seen in TOCC in terms of age, CAN score, rurality, and hospital diagnosis, but readmission rates were substantially higher in this group even after adjustments for covariates (Table 2). Evaluating causality in interventions aimed to reduce hospital readmission rates is complicated.28 Our findings add greater plausibility to the utility of TOCC in accounting for at least a portion of the reported decrease in ICVA 30-day readmissions.
Our study has several strengths, including an observation period longer than 2 years, a large population of discharged veterans within an integrated healthcare system, and a large proportion of patients living in rural areas. Another strength of our study is the innovative nature of the intervention, which features a multidisciplinary team and the option of virtual or in-person visits. Nevertheless, this study also has several important limitations. As a single-center study, our findings may not be generalizable to other institutions, especially those outside the VHA system. Similarly, our study population reflected that of the ICVA, which may limit generalizability to a more diverse population. While we attempted to account in our statistical modeling for baseline differences between referred patients seen by the TOCC and those referred but not seen, we cannot exclude residual confounding between the groups. Specifically, the comparison of patients who did and did not choose TOCC follow-up introduces the possibility of selection bias. Future randomized/controlled studies will need to evaluate whether TOCC is more effective than the standard of care to reduce readmissions. Finally, since the analysis period following the final PDSA cycle was compressed due to the onset of the COVID-19 pandemic in the United States, no data are available regarding the sustained impacts of changes made during this cycle.
CONCLUSION
A multidisciplinary TOCC within the ICVA, featuring both virtual and in-person visits, reduced 30-day readmission rates by 19.3%; this approach to care was especially effective in patients with CHF. Virtual visits were the follow-up mode of choice for both urban and rural veterans, and there was no difference in outcomes between these two follow-up options. Future studies will focus on additional quality metrics, including cost-effectiveness and patient satisfaction.
1. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418-1428. https://doi.org/10.1056/nejmsa0803563
2. Doctoroff L. Postdischarge clinics and hospitalists: a review of the evidence and existing models. J Hosp Med. 2017;12(6):467-471. https://doi.org/10.12788/jhm.2750
3. Koehler BE, Richter KM, Youngblood L, et al. Reduction of 30-day postdischarge hospital readmission or emergency department (ED) visit rates in high-risk elderly medical patients through delivery of a targeted care bundle. J Hosp Med. 2009;4(4):211-218. https://doi.org/10.1002/jhm.427
4. Abrashkin KA, Cho HJ, Torgalkar S, Markoff B. Improving transitions of care from hospital to home: what works? Mt Sinai J Med. 2012;79(5):535-544. https://doi.org/10.1002/msj.21332
5. Misky GJ, Wald HL, Coleman EA. Post-hospitalization transitions: examining the effects of timing of primary care provider follow-up. J Hosp Med. 2010;5(7):392-397. https://doi.org/10.1002/jhm.666
6. Greysen SR, Harrison JD, Kripalani S, et al. Understanding patient-centred readmission factors: a multi-site, mixed-methods study. BMJ Qual Saf. 2017;26(1):33-41. https://doi.org/10.1136/bmjqs-2015-004570
7. Hansen LO, Young RS, Hinami K, Leung A, Williams MV. Interventions to reduce 30-day rehospitalization: a systematic review. Ann Intern Med. 2011;155(8):520-528. https://doi.org/10.7326/0003-4819-155-8-201110180-00008
8. Hernandez AF, Greiner MA, Fonarow GC, et al. Relationship between early physician follow-up and 30-day readmission among Medicare beneficiaries hospitalized for heart failure. JAMA. 2010;303(17):1716-1722. https://doi.org/10.1001/jama.2010.533
9. Lee KK, Yang J, Hernandez AF, Steimle AE, Go AS. Post-discharge follow-up characteristics associated with 30-day readmission after heart failure hospitalization. Med Care. 2016;54(4):365-372. https://doi.org/10.1097/mlr.0000000000000492
10. Balaban RB, Williams MV. Improving care transitions: hospitalists partnering with primary care. J Hosp Med. 2010;5(7):375-377. https://doi.org/10.1002/jhm.824
11. Rodrigues CR, Harrington AR, Murdock N, et al. Effect of pharmacy-supported transition-of-care interventions on 30-day readmissions: a systematic review and meta-analysis. Ann Pharmacother. 2017;51(10):866-889. https://doi.org/10.1177/1060028017712725
12. van Walraven C, Taljaard M, Etchells E, et al. The independent association of provider and information continuity on outcomes after hospital discharge: implications for hospitalists. J Hosp Med. 2010;5(7):398-405. https://doi.org/10.1002/jhm.716
13. Gruca TS, Pyo TH, Nelson GC. Providing cardiology care in rural areas through vsiting consultant clinics. J Am Heart Assoc. 2016;5(7):e002909. https://doi.org/10.1161/jaha.115.002909
14. Chan L, Hart LG, Goodman DC. Geographic access to health care for rural Medicare beneficiaries. J Rural Health. 2006;22(2):140-146. https://doi.org/10.1111/j.1748-0361.2006.00022.x
15. Burke RE, Jones CD, Coleman EA, Falvey JR, Stevens-Lapsley JE, Ginde AA. Use of post-acute care after hospital discharge in urban and rural hospitals. Am J Accountable Care. 2017;5(1):16-22.
16. Jetty A, Moore MA, Coffman M, Petterson S, Bazemore A. Rural family physicians are twice as likely to use telehealth as urban family physicians. Telemed J E Health. 2018;24(4):268-276. https://doi.org/10.1089/tmj.2017.0161
17. Harrison PL, Hara PA, Pope JE, Young MC, Rula EY. The impact of postdischarge telephonic follow-up on hospital readmissions. Popul Health Manag. 2011;14(1):27-32. https://doi.org/10.1089/pop.2009.0076
18. Wang L, Porter B, Maynard C, et al. Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration. Med Care. 2013;51(4):368-373. https://doi.org/10.1097/mlr.0b013e31827da95a
19. Spece LJ, Donovan LM, Griffith MF, et al. Initiating low-value inhaled corticosteroids in an inception cohort with chronic obstructive pulmonary disease. Ann Am Thorac Soc. 2020;17(5):589-595. https://doi.org/10.1513/annalsats.201911-854oc
20. Osborne TF, Suarez P, Edwards D, Hernandez-Boussard T, Curtin C. Patient electronic health records score for preoperative risk assessment before total knee arthroplasty. JB JS Open Access. 2020;5(2):e0061. https://doi.org/10.2106/jbjs.oa.19.00061
21. Levy C, Ersek M, Scott W, et al. Life-sustaining treatment decisions initiative: early implementation results of a national Veterans Affairs program to honor veterans’ care preferences. J Gen Intern Med. 2020;35(6):1803-1812. https://doi.org/10.1007/s11606-020-05697-2
22. Ibrahim SA. High-risk patients and utilization of primary care in the US Veterans Affairs health system. JAMA Netw Open. 2020;3(6):e209518. https://doi.org/10.1001/jamanetworkopen.2020.9518
23. Burke RE, Whitfield E, Prochazka AV. Effect of a hospitalist-run postdischarge clinic on outcomes. J Hosp Med. 2014;9(1):7-12. https://doi.org/10.1002/jhm.2099
24. Jenq GY, Doyle MM, Belton BM, Herrin J, Horwitz LI. Quasi-experimental evaluation of the effectiveness of a large-scale readmission reduction program. JAMA Intern Med. 2016;176(5):681-690. https://doi.org/10.1001/jamainternmed.2016.0833
25. Kosar CM, Loomer L, Ferdows NB, Trivedi AN, Panagiotou OA, Rahman M. Assessment of rural-urban differences in postacute care utilization and outcomes among older US adults. JAMA Netw Open. 2020;3(1):e1918738. https://doi.org/10.1001/jamanetworkopen.2019.18738
26. Pandor A, Thokala P, Gomersall T, et al. Home telemonitoring or structured telephone support programmes after recent discharge in patients with heart failure: systematic review and economic evaluation. Health Technol Assess. 2013;17(32):1-207, v-vi. https://doi.org/10.3310/hta17320
27. Low LL, Tan SY, Ng MJM, et al. Applying the integrated practice unit concept to a modified virtual ward model of care for patients at highest risk of readmission: a randomized controlled trial. PloS One. 2017;12(1):e0168757. https://doi.org/10.1371/journal.pone.0168757
28. McIlvennan CK, Eapen ZJ, Allen LA. Hospital readmissions reduction program. Circulation. 2015;131(20):1796-1803. https://doi.org/10.1161/circulationaha.114.010270
1. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418-1428. https://doi.org/10.1056/nejmsa0803563
2. Doctoroff L. Postdischarge clinics and hospitalists: a review of the evidence and existing models. J Hosp Med. 2017;12(6):467-471. https://doi.org/10.12788/jhm.2750
3. Koehler BE, Richter KM, Youngblood L, et al. Reduction of 30-day postdischarge hospital readmission or emergency department (ED) visit rates in high-risk elderly medical patients through delivery of a targeted care bundle. J Hosp Med. 2009;4(4):211-218. https://doi.org/10.1002/jhm.427
4. Abrashkin KA, Cho HJ, Torgalkar S, Markoff B. Improving transitions of care from hospital to home: what works? Mt Sinai J Med. 2012;79(5):535-544. https://doi.org/10.1002/msj.21332
5. Misky GJ, Wald HL, Coleman EA. Post-hospitalization transitions: examining the effects of timing of primary care provider follow-up. J Hosp Med. 2010;5(7):392-397. https://doi.org/10.1002/jhm.666
6. Greysen SR, Harrison JD, Kripalani S, et al. Understanding patient-centred readmission factors: a multi-site, mixed-methods study. BMJ Qual Saf. 2017;26(1):33-41. https://doi.org/10.1136/bmjqs-2015-004570
7. Hansen LO, Young RS, Hinami K, Leung A, Williams MV. Interventions to reduce 30-day rehospitalization: a systematic review. Ann Intern Med. 2011;155(8):520-528. https://doi.org/10.7326/0003-4819-155-8-201110180-00008
8. Hernandez AF, Greiner MA, Fonarow GC, et al. Relationship between early physician follow-up and 30-day readmission among Medicare beneficiaries hospitalized for heart failure. JAMA. 2010;303(17):1716-1722. https://doi.org/10.1001/jama.2010.533
9. Lee KK, Yang J, Hernandez AF, Steimle AE, Go AS. Post-discharge follow-up characteristics associated with 30-day readmission after heart failure hospitalization. Med Care. 2016;54(4):365-372. https://doi.org/10.1097/mlr.0000000000000492
10. Balaban RB, Williams MV. Improving care transitions: hospitalists partnering with primary care. J Hosp Med. 2010;5(7):375-377. https://doi.org/10.1002/jhm.824
11. Rodrigues CR, Harrington AR, Murdock N, et al. Effect of pharmacy-supported transition-of-care interventions on 30-day readmissions: a systematic review and meta-analysis. Ann Pharmacother. 2017;51(10):866-889. https://doi.org/10.1177/1060028017712725
12. van Walraven C, Taljaard M, Etchells E, et al. The independent association of provider and information continuity on outcomes after hospital discharge: implications for hospitalists. J Hosp Med. 2010;5(7):398-405. https://doi.org/10.1002/jhm.716
13. Gruca TS, Pyo TH, Nelson GC. Providing cardiology care in rural areas through vsiting consultant clinics. J Am Heart Assoc. 2016;5(7):e002909. https://doi.org/10.1161/jaha.115.002909
14. Chan L, Hart LG, Goodman DC. Geographic access to health care for rural Medicare beneficiaries. J Rural Health. 2006;22(2):140-146. https://doi.org/10.1111/j.1748-0361.2006.00022.x
15. Burke RE, Jones CD, Coleman EA, Falvey JR, Stevens-Lapsley JE, Ginde AA. Use of post-acute care after hospital discharge in urban and rural hospitals. Am J Accountable Care. 2017;5(1):16-22.
16. Jetty A, Moore MA, Coffman M, Petterson S, Bazemore A. Rural family physicians are twice as likely to use telehealth as urban family physicians. Telemed J E Health. 2018;24(4):268-276. https://doi.org/10.1089/tmj.2017.0161
17. Harrison PL, Hara PA, Pope JE, Young MC, Rula EY. The impact of postdischarge telephonic follow-up on hospital readmissions. Popul Health Manag. 2011;14(1):27-32. https://doi.org/10.1089/pop.2009.0076
18. Wang L, Porter B, Maynard C, et al. Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration. Med Care. 2013;51(4):368-373. https://doi.org/10.1097/mlr.0b013e31827da95a
19. Spece LJ, Donovan LM, Griffith MF, et al. Initiating low-value inhaled corticosteroids in an inception cohort with chronic obstructive pulmonary disease. Ann Am Thorac Soc. 2020;17(5):589-595. https://doi.org/10.1513/annalsats.201911-854oc
20. Osborne TF, Suarez P, Edwards D, Hernandez-Boussard T, Curtin C. Patient electronic health records score for preoperative risk assessment before total knee arthroplasty. JB JS Open Access. 2020;5(2):e0061. https://doi.org/10.2106/jbjs.oa.19.00061
21. Levy C, Ersek M, Scott W, et al. Life-sustaining treatment decisions initiative: early implementation results of a national Veterans Affairs program to honor veterans’ care preferences. J Gen Intern Med. 2020;35(6):1803-1812. https://doi.org/10.1007/s11606-020-05697-2
22. Ibrahim SA. High-risk patients and utilization of primary care in the US Veterans Affairs health system. JAMA Netw Open. 2020;3(6):e209518. https://doi.org/10.1001/jamanetworkopen.2020.9518
23. Burke RE, Whitfield E, Prochazka AV. Effect of a hospitalist-run postdischarge clinic on outcomes. J Hosp Med. 2014;9(1):7-12. https://doi.org/10.1002/jhm.2099
24. Jenq GY, Doyle MM, Belton BM, Herrin J, Horwitz LI. Quasi-experimental evaluation of the effectiveness of a large-scale readmission reduction program. JAMA Intern Med. 2016;176(5):681-690. https://doi.org/10.1001/jamainternmed.2016.0833
25. Kosar CM, Loomer L, Ferdows NB, Trivedi AN, Panagiotou OA, Rahman M. Assessment of rural-urban differences in postacute care utilization and outcomes among older US adults. JAMA Netw Open. 2020;3(1):e1918738. https://doi.org/10.1001/jamanetworkopen.2019.18738
26. Pandor A, Thokala P, Gomersall T, et al. Home telemonitoring or structured telephone support programmes after recent discharge in patients with heart failure: systematic review and economic evaluation. Health Technol Assess. 2013;17(32):1-207, v-vi. https://doi.org/10.3310/hta17320
27. Low LL, Tan SY, Ng MJM, et al. Applying the integrated practice unit concept to a modified virtual ward model of care for patients at highest risk of readmission: a randomized controlled trial. PloS One. 2017;12(1):e0168757. https://doi.org/10.1371/journal.pone.0168757
28. McIlvennan CK, Eapen ZJ, Allen LA. Hospital readmissions reduction program. Circulation. 2015;131(20):1796-1803. https://doi.org/10.1161/circulationaha.114.010270
© 2021 Society of Hospital Medicine
A Qualitative Study of Treating Dual-Use Patients Across Health Care Systems
The VHA assigns all enrolled veterans to a primary care provider (PCP). However, almost 80% of veterans enrolled in VHA have another form of health care coverage, including Medicare, Medicaid, private insurance, and TRICARE for Life program.1 Consequently, veterans may choose to use more than 1 health care system to manage their health care needs.
Studies based on merged VHA and Medicare claims data have demonstrated substantial dual use by VHA enrollees with Medicare. Petersen and colleagues reported that about 80% of VHA enrollees with Medicare chose to use services in both systems and that greater distance to VHA facilities and lower priority level for VHA care predicted lower VHA reliance.2 Among those aged < 65 years who had Medicare due to disability, 58% weredual users. These dual users relied more on private sector care for many health conditions, with the notable exception of substance abuse and mental health disorders, for which reliance on VHA care was greater.2 Another study found that over half of VHA enrollees assigned to a PCP at a community-based outpatient clinic (CBOC) received some or all of their care outside VHA and that reliance on VHA outpatient care declined over the 4-year study period.3
Related: Mutual Alignment Trumps Merger for Joint VA/DoD Health Care Programs
This use of multiple health care providers (HCPs), facilities, and modalities is often described as dual use or comanagement. Dual use in the case of veterans refers to use of both VHA and non-VHA health care, whereas comanagement implies an expectation of shared decision making and open communication between VHA and non-VHA providers. In addition to VHA PCPs, rural veterans frequently receive care from local, non-VHA HCPs in the community where they live. As health care in the U.S. evolves and patients have increasing choices through the Affordable Care Act (ACA), the challenge of comanagement for patients receiving care in multiple systems is likely to increase both within and outside VHA.
This study was part of a qualitative rural health needs assessment designed to ascertain the issues facing rural veterans and their providers in the upper Midwest.4 The objective was to examine VHA primary care clinic staff perspectives on dual users, perceived barriers that inhibit comanagement, and factors that contribute to the need for dual use in rural areas.
Methods
A qualitative study design with in-person interviews was used to elicit the perspective of VHA clinic staff on the current and ideal states of comanagement. Clinics were selected using a stratified purposeful sample of 15 urban and rural primary care clinics at VHA CBOCs and VAMCs in 8 Midwestern states (Illinois, Iowa, Minnesota, Nebraska, North Dakota, South Dakota, Wisconsin, and Wyoming). The stratification criteria included (1) urban and rural; (2) geographic coverage of VISN 23; and (3) VHA-managed and contract clinics, resulting in a purposeful sample of 2 urban VAMC clinics, 3 urban CBOCs, 7 rural VHA-managed CBOCs, and 3 rural contract CBOCs. The distance from the CBOC to the closest VAMC ranged from 32 to 242 miles.
Related: VA Relaxes Rules for Choice Program
Interview guides were developed and tested by the research team for comprehension, length, and timing prior to data collection and iteratively revised as analysis evolved and new topics emerged. Clinic staff were asked about their perceptions of rural veteran use of VHA care; barriers and facilitators to accessing care; and their personal experience working within VHA. Several questions focused on dual use and why rural veterans use multiple health care systems, their perspectives of dual use, their expectations of patients’ role(s) in health care coordination, and the perceived barriers that inhibit comanagement. Interviewers used comanagement and dual use interchangeably to discuss patients with multiple care providers, allowing interviewees to use their preferred terminology; assigned meanings were probed for clarification but not corrected by interviewers.
Between June and October 2009, teams of 2 to 3 researchers visited 15 clinics for 1 to 2 business days each. Researchers conducted interviews with a convenience sample of clinical staff. Consent forms and an explanation of the study were distributed, and those electing to participate voluntarily came to a designated room to complete an interview. All interviews were audio recorded for accuracy.
Interview recordings were transcribed verbatim and reviewed for accuracy. Prior to coding, transcripts were imported into a qualitative data management software program. A codebook, including a priori research hypotheses and de novo themes, was developed based on a systematic review of a randomly selected subset of interview transcripts.5 Four coders were responsible for coding all transcripts and validating coding through tests of agreement at predetermined intervals.
Regular meetings were conducted with coders and the lead qualitative investigator to discuss disagreements, clarify code definitions, or add new codes as needed. As codes were added, previous transcripts were coded/recoded for content related to the new codes. An audit trail was maintained, and iterative mediation of codes continued throughout the process. The final codebook contained 42 thematic codes, which reached saturation or data redundancy.6 Detailed analysis of the codes dual use, distance, and care coordination were used to inform this study.
Results
Among the 15 sites, 64 in-depth individual interviews were conducted, ranging from 5 to 53 minutes (average 26 minutes). Clinic staff demographic characteristics are depicted in the Table. Analysis of data captured in the codes dual use, distance, and care coordination resulted in notable concentration in 4 thematic areas: (1) clinic staff perceptions of the influence of access, convenience, and distance on dual use for rural patients; (2) communication and patient’s role in comanagement; (3) rules and regulations related to comanagement from the VHA perspective; and (4) barriers to comanagement and recommendations for education.
Influence of Access, Convenience, and Distance
Access to health care was central to the discussion of dual use and comanagement by clinic staff. Convenience was identified as the primary reason for rural patients’ use of non-VHA services, as many rural patients must travel outside their local community to access VHA care. Thus, dual use was most often noted for services typically available in patients’ local communities, especially management of chronic conditions.
The CBOCs provide important services for primary care and management of chronic conditions but are not available in all communities and may have limited hours/days that do not fit with patients’ schedules. The CBOCs are often unable to provide needed services, including but not limited to emergency care, diagnostic tests, physical and occupational therapy, and other specialty care services. As one VHA provider put it, “The biggest factor for [dual use] is availability, access, convenience.… It’s a lot more convenient to go to the hospital down the street than it is to go 120 miles to [the VAMC], or for some guys who live 30, 40 minutes the other side of here it becomes 150, 160-mile one-way trip.”
Related to access, distance and transportation barriers were identified by clinic staff as obstacles to care for rural patients. Despite efforts to offset the expense of travel through reimbursement to qualified veterans and coordinated van transport with Veterans Service Organizations, travel costs—both time and money—were seen as significant barriers to accessing VHA care, as was an inability to travel for those who are ill or frail and elderly. “We send people … in the van and for the most part that works, but eventually it gets expensive, or you’ve got somebody with chronic pain that can’t tolerate the van ride for 2 hours,” one interviewee
reported.
According to clinic staff, dual-use patients also rely on non-VHA providers in particular for urgent or emergency care, while relying on VHA primary care for reduced-cost medications, diagnostic testing, chronic disease management, or annual exams. When asked why rural patients may choose to see more than 1 provider, VHA providers responded. “[It’s] more convenient to have a local doctor just in case something went wrong and they need to see a doctor right away. So distance to this clinic would be the number one reason.” Another reported, “If it’s once or twice a year routine appointments they’ll come here, but… they’d rather go to a walk-in clinic nearby than spend so much [money] on gas.”
Communication and Patients’ Role
Communication between VHA and non-VHA providers is a necessary element of comanagement. Although phone calls or faxing patient medical records are available options, clinic staff reported it was more common to encounter patients hand carrying their records between providers. For dual-use patients, clinic staff indicated it was often unclear who was responsible for relaying information between providers. There is often ambiguity about who will (and should) fulfill this role and not enough time to adequately address or clarify how this is done. Some clinic staff believed that acting as the main conduits of information placed an undue burden on the patients, particularly asking them to be able to accurately relay medical information about tests or prescriptions that they may not fully understand. Others said that it was primarily the patients’ responsibility to give relevant information about their care to all their providers, because of VHA regulations and patient privacy laws. “[The] patient should tell the primary doctor to send them [medical records] because we can’t get the medical records without the patient’s permission,” said one provider.
Another provider utilized the nursing staff to call patients after their appointments to remind them to give their medical records to their non-VHA provider. The data suggest that responsibility for maintaining communication between providers ultimately falls on the patient. From the perspective of a nurse practitioner, “We just keep trying to educate the community…. I’ve been told that if the patient wants that privilege of using the VA for a pharmacy for an outside provider that we’re glad to do that. But it is their responsibility to communicate with their [non-VHA] physician. I think we just need to keep educating the patients.”
Rules and Regulations
VHA policies governing prescriptions, hospitalizations at outside facilities, and release of patient information regulate, and in some cases hinder, information flow between VHA and non-VHA providers. Many patients use VHA to obtain medications for lower out-of-pocket costs. This contributes to the number of dual-use patients in VHA and results in several challenges for VHA providers trying to manage patients’ prescriptions. For example, patients will ask to fill a prescription at a VHA pharmacy from their non-VHA providers; however, VHA pharmacies can only fill prescriptions from VHA providers.
Many VHA providers are willing to rewrite these prescriptions, but they may need to see the patient before adding or changing the prescription and require documentation to address contraindications, adverse reactions and/or therapeutic failure, and associated risks before making the authorization. VHA providers noted that because the VHA formulary does not contain all medications, non-VHA providers are often unfamiliar with the VHA National Formulary specifics and will write prescriptions for nonformulary medications, which require a nonformulary request from a VHA provider.
Clinic staff also mentioned difficulty in obtaining records from non-VHA providers. This can be particularly problematic if the patient lives a distance away from a VHA facility and does not have the necessary authorization to share records on file.
Barriers and Education Recommendations
Clinic staff identified coordination of care for dual-use patients as a barrier to providing care. Specifically, providers identified coordination as complicated by communication difficulties, inefficient medical record exchange, short staffing in VHA clinics, duplication of diagnostic services, and non-VHA providers’ lack of understanding regarding the services that VHA provides. Specific to rural clinics, comanagement was reportedly hindered by limitations in technology (eg, consistent Internet access), access to routine diagnostic services, and lack of relationships with non-VHA providers. Providers most frequently reported that the critical piece missing in comanagement is a relationship—and implied communication—between VHA clinics and non-VHA community clinics. The concept of a relationship between providers is evoked as a critical element to comanaging dual-use patients; however, clinic staff had a difficult time articulating what that relationship would actually look like if put into practice.
Related: Patients Benefit From ICU Telemedicine
In spite of the numerous barriers identified by clinic staff, the recommendation for education to improve comanagement was consistent across study sites and clinic staff roles. Education was proposed for patients and non-VHA providers as the best intervention. In response to a question about ideas and recommendations to improve comanagement, clinic staff drew on varied experiences. To illustrate this theme, a provider gave this example of dual-use patients seeking prescription medication from VHA and its impact on comanagement: “I would [recommend] an outreach program to community resources and [non-VHA] providers. To let them know more about how the VA works and the resources that are available, and how specifically to coordinate care through the VA, would be a significant benefit.… If the [non-VHA] providers knew how to—who to—talk to, what information the VA needs, for example, for medication changes, it would help the patients make it work…without having to overburden the patients with having to physically hand carry their blood test results, or their notes, discharge summaries, procedure notes.”
Along with providing outreach and education on working with the VHA, clinic staff addressed the need to educate patients more effectively, because they are seen as central to the information exchange. There is motivation on the part of patients to learn the system. “Just making sure that the patients realize that they need to tell their local providers to send us the records and make sure that there is an exchange going on consistently,” explained a case manager. “If the patient wants to get those medications that are costly, then they figure out pretty quick what they have to have, what they need to send to us.” The need for education is an ongoing process; who is responsible for this continues to be a point of debate.
Discussion
In order to better understand comanagement of dual-use patients, this study focused on the experiences and perceptions of staff at VHA primary care clinics in the upper Midwest. The data indicated that:
- VHA clinical staff perceive the primary reason patients choose to seek non-VHA care is because of access, convenience, and
distance - In order for comanagement to occur, communication and information exchange—currently facilitated largely by patients—needs to improve
- Education of patients and their non-VHA providers is recommended, to increase understanding of rules and regulations tied to exchange of patient information across health care systems
- Education may facilitate communication, develop relationships, and overcome barriers to information exchange
Distance to health care and perceived convenience were clearly seen by clinic staff as the driving factors behind their patients’ dual use. In the authors’ prior work, interviews with veterans and their VA providers supported this assertion as well; however, it was also found that distance must not be understood in isolation of other contingencies, such as urgency of need.4
Clinic staff identified institutional and individual barriers that lead to miscommunication and confusion on the part of patients and reported misunderstandings with non-VHA providers, including 3 potential barriers to comanagement. These included (1) inconsistent communication and flow of information between VHA and non-VHA providers; (2) uncertainty about who will (and should) be responsible for information flow between providers; and (3) VHA and federal regulations over patient privacy. Throughout the interviews, access to less expensive prescription medications in VHA was considered an additional driver of dual use. According to clinic staff interviewed, education of patients and non-VHA providers could facilitate efficient and safe comanagement for dual-use patients.7
This study suggests both advantages and disadvantages for patients choosing to use multiple health care systems from the perspective of the clinic staff. The primary advantage is better overall health care access, especially for rural patients and those with longer travel times to VHA facilities. The primary disadvantage of dual use is discontinuity of care between multiple care sites. Specifically, this study identified concerns regarding poor communication between providers and transfer of patient medical records. An underlying theme was a concern for quality of care and patient safety, which are recognized by others in the literature as potential consequences of inadequate comanagement.8-12
If there is one aspect of co-management for dual-use patients to target, this study’s findings point to developing strategies to improve communication between providers caring for dual-use patients and, more specifically, cultivating relationships that are currently underdeveloped. This will necessitate a clearer articulation of what constitutes a relationship between comanaging providers and is a direction for further research that would have applicability beyond VHA to any comanagement of patients using multiple health care systems.
There are 3 simultaneous, yet unrelated, factors that may contribute to increasing dual use. First is the rise in VHA eligible veterans from Operation Enduring Freedom, Operation Iraqi Freedom, and Operation New Dawn.13,14 All returning veterans who meet minimal requirements are eligible for 5 years of VHA health care. A large proportion of these individuals are in the Reserve and National Guard, most of whom have nonmilitary jobs that may provide employer-based health insurance. Thus, these veterans have a greater opportunity for dual use. Second, with the aging cohort of Vietnam-era veterans, a greater proportion is becoming Medicare eligible. Third, with the recent passing of the ACA, more patients, including veterans, may choose to purchase insurance through ACA health exchanges. Taken individually or collectively, these factors will likely have effects reaching beyond VHA, especially when veterans receiving care in non-VHA health care systems engage in dual use.3,13,15,16
Limitations
This study has a number of limitations. First, it was limited to VHA facilities located in the upper Midwest, which may limit generalizability to other parts of the country. The convenience sample of clinic staff at VHA clinics may not represent the full range of perspectives among HCPs generally. This study did not interview clinic staff in non-VHA clinics, although this has been the focus of other studies.17,18 Although dual use also applies to specialty care and related access issues in rural areas, this was not a focus of this study. Last, the data were collected in 2009, prior to the implementation of the patient-aligned care team (PACT) model and prior to the recently revealed issues regarding patient wait times for VHA care. Thus, perceptions may have changed, and additional study is needed.
Conclusions
The results of this study support prior assumptions of barriers to care, but also introduce previously unreported challenges. Dual use is perceived to have both positive and negative impacts, but for the positives to outweigh the negatives, thoughtful comanagement is critical. This may be particularly so in rural areas where dual use is encouraged as a way to overcome distance and increase convenience in accessing care.
As demonstrated by recent events, there are still VHA health care access issues for veterans. Recently, VA leadership and the U.S. Congress proposed that veterans have greater access to community providers as well as VHA in order to overcome delays in care.19 As this option is explored and put into practice, it is more important than ever to consider the need for care coordination and management of dual-use patients, to ensure good communication and care that is timely, safe, and high quality.
Few models exist in which 2 PCPs coordinate across health care systems, and greater understanding of this dual use is needed. This information is important in designing interventions to improve care coordination across systems to ensure continuity of care, patient safety, and patient satisfaction. Although some work has been done to examine the perspectives of non-VA PCPs, little is known about VHA provider perspectives on rural veteran dual use.17,18 This study explores VHA provider perspectives and identifies areas where interventions to improve care coordination across systems might be targeted.
Next steps for intervention studies would be to improve communication and develop educational tools to aid in the coordination of care between VHA and non-VHA providers. A recent example of this is the Co-Management Toolkit developed by the Veterans Rural Health Resource Center-Central Region, which provides information on VHA policies and targets non-VHA providers.20 Although VHA perceptions of comanageing dual-use patients were the target, a similar study of non-VHA providers is important to understand this complex and multifaceted dynamic. Additional work is needed to measure the impact of dual use on clinical outcomes, patient safety and quality, and efficient use of resources, as these are understudied. As dual use continues and potentially increases with the ACA and changing health care in the U.S., it is important to understand the management of patients using multiple health care systems. This is salient as primary care adopts the PACT model and to inform interventions to improve quality and safety while eliminating duplicative health care and costs.
Acknowledgements
The research reported here was supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Rural Health, Veterans Rural Health Resource Center-Central Region (VRHRC-CR) and the VA Health Services Research and Development (HSR&D) Service, the Comprehensive Access and Delivery Research and Evaluation (CADRE) Center at the Iowa City VA Health Care System, and Center to Improve Veteran Involvement in Care (CIVIC) at VA Portland Health Care System. Dr. Reisinger was supported by a Research Career Development Award from the Health Services Research and Development Service, Department of Veterans Affairs (CD1 08-013-1).
We would like to thank all health care providers who graciously agreed to participate in this study and VRHRC-CR staff, in particular Monica Paez for assistance on this manuscript.
Author disclosures
The authors report no actual or potential conflicts of interest with regard to this article.
Disclaimer
The opinions expressed herein are those of the authors and do not necessarily reflect those of Federal Practitioner, Frontline Medical Communications Inc., the U.S. Government, or any of its agencies. This article may discuss unlabeled or investigational use of certain drugs. Please review complete prescribing information for specific drugs or drug combinations—including indications, contraindications, warnings, and adverse effects—before administering pharmacologic therapy to patients.
1. Department of Veterans Affairs Office of Rural Health, VHA. Veterans Rural Health: Perspectives and Opportunities. Rockville, MD: Booz Allen Hamilton; 2008. http://www.ruralhealth.va.gov/docs/PAO-final-report-0208.pdf. Accessed July 6, 2015.
2. Petersen LA, Byrne MM, Daw CN, Hasche J, Reis B, Pietz K. Relationship between clinical conditions and use of Veterans Affairs health care among Medicare‐enrolled veterans. Health Serv Res. 2010;45(3):762-791.
3. Liu CF, Chapko M, Bryson CL, et al. Use of outpatient care in Veterans Health Administration and Medicare among veterans receiving primary care in community-based and hospital outpatient clinics. Health Serv Res. 2010;45(5 pt 1):1268-1286.
4. Buzza C, Ono SS, Turvey C, et al. Distance is relative: unpacking a principal barrier in rural healthcare. J Gen Intern Med. 2011;26(suppl 2):648-654.
5. Bernard HR, Ryan GW. Analyzing Qualitative Data: Systematic Approaches. Los Angeles, CA: SAGE; 2010.
6. Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18(1):59-82.
7. Kramer BJ, Vivrette RL, Satter DE, Jouldjian S, McDonald LR. Dual use of Veterans Health Administration and Indian Health Service: healthcare provider and patient perspectives. J Gen Intern Med. 2009;24(6):758-764.
8. Ajmera M, Wilkins TL, Sambamoorthi U. Dual Medicare and Veteran Health Administration use and ambulatory care sensitive hospitalizations. J Gen Intern Med. 2011;26(suppl 2):669-675.
9. Jia H, Zheng Y, Reker DM, et al. Multiple system utilization and mortality for veterans with stroke. Stroke. 2007;38(2):355-360.
10. Trivedi AN, Grebla RC, Jiang L, Yoon J, Mor V, Kizer KW. Duplicate federal payments for dual enrollees in Medicare Advantage plans and the Veterans Affairs health care system. JAMA. 2012;308(1):67-72.
11. Kaboli PJ, Shivapour DM, Henderson MS, Ishani A, Charlton ME. The impact of primary care dual-management on quality of care. J Prim Care Community Health. 2012;3(1):11-16.
12. Wolinsky FD, Miller TR, An H, Brezinski PR, Vaughn TE, Rosenthal GE. Dual use of Medicare and the Veterans Health Administration: are there adverse health outcomes? BMC Health Serv Res. 2006;6:131.
13. Liu CF, Bryson CL, Burgess JF Jr, Sharp N, Perkins M, Maciejewski ML. Use of outpatient care in VA and Medicare among disability-eligible and age-eligible veteran patients. BMC Health Serv Res. 2012;12:51.
14. Miller EA, Intrator O. Veterans use of non-VHA services: implications for policy and planning. Soc Work Public Health. 2012;27(4):379-391.
15. Bachman SS, Gonyea JG. Improving health care delivery to aging adults with disabilities: social work with dual eligibles in a climate of health care reform. J Gerontol Soc Work. 2012;55(2):191-207.
16. Kizer KW. Veterans and the Affordable Care Act. JAMA. 2012;307(8):789-790.
17. Lampman MA, Mueller KJ. Experiences of rural non-VA providers in treating dual care veterans and the development of electronic health information exchange networks between the two systems. J Rural Soc Sci. 2011;26(3):201-219.
18. Gaglioti A, Cozad A, Wittrock S, et al. Non-VA primary care providers’ perspectives on comanagement for rural veterans. Mil Med. 2014;179(11):1236-1243.
19. U.S. Department of Veterans Affairs. Acting Secretary Gibson outlines problems, actions taken, and budget resources needed to ensure access to care. U.S. Department of Veterans Affairs Website. http://www.va.gov/opa/pressrel/pressrelease.cfm?id=2586. Published July 16, 2014. Accessed July 6, 2015.
20. Office of Rural Health Central Region. Co-managed care toolkit. U.S. Department of Veterans Affairs Website. http://www.ruralhealth.va.gov/resource-centers/central/comanagement-toolkit.asp. Updated June 3, 2015. Accessed July 6, 2015.
The VHA assigns all enrolled veterans to a primary care provider (PCP). However, almost 80% of veterans enrolled in VHA have another form of health care coverage, including Medicare, Medicaid, private insurance, and TRICARE for Life program.1 Consequently, veterans may choose to use more than 1 health care system to manage their health care needs.
Studies based on merged VHA and Medicare claims data have demonstrated substantial dual use by VHA enrollees with Medicare. Petersen and colleagues reported that about 80% of VHA enrollees with Medicare chose to use services in both systems and that greater distance to VHA facilities and lower priority level for VHA care predicted lower VHA reliance.2 Among those aged < 65 years who had Medicare due to disability, 58% weredual users. These dual users relied more on private sector care for many health conditions, with the notable exception of substance abuse and mental health disorders, for which reliance on VHA care was greater.2 Another study found that over half of VHA enrollees assigned to a PCP at a community-based outpatient clinic (CBOC) received some or all of their care outside VHA and that reliance on VHA outpatient care declined over the 4-year study period.3
Related: Mutual Alignment Trumps Merger for Joint VA/DoD Health Care Programs
This use of multiple health care providers (HCPs), facilities, and modalities is often described as dual use or comanagement. Dual use in the case of veterans refers to use of both VHA and non-VHA health care, whereas comanagement implies an expectation of shared decision making and open communication between VHA and non-VHA providers. In addition to VHA PCPs, rural veterans frequently receive care from local, non-VHA HCPs in the community where they live. As health care in the U.S. evolves and patients have increasing choices through the Affordable Care Act (ACA), the challenge of comanagement for patients receiving care in multiple systems is likely to increase both within and outside VHA.
This study was part of a qualitative rural health needs assessment designed to ascertain the issues facing rural veterans and their providers in the upper Midwest.4 The objective was to examine VHA primary care clinic staff perspectives on dual users, perceived barriers that inhibit comanagement, and factors that contribute to the need for dual use in rural areas.
Methods
A qualitative study design with in-person interviews was used to elicit the perspective of VHA clinic staff on the current and ideal states of comanagement. Clinics were selected using a stratified purposeful sample of 15 urban and rural primary care clinics at VHA CBOCs and VAMCs in 8 Midwestern states (Illinois, Iowa, Minnesota, Nebraska, North Dakota, South Dakota, Wisconsin, and Wyoming). The stratification criteria included (1) urban and rural; (2) geographic coverage of VISN 23; and (3) VHA-managed and contract clinics, resulting in a purposeful sample of 2 urban VAMC clinics, 3 urban CBOCs, 7 rural VHA-managed CBOCs, and 3 rural contract CBOCs. The distance from the CBOC to the closest VAMC ranged from 32 to 242 miles.
Related: VA Relaxes Rules for Choice Program
Interview guides were developed and tested by the research team for comprehension, length, and timing prior to data collection and iteratively revised as analysis evolved and new topics emerged. Clinic staff were asked about their perceptions of rural veteran use of VHA care; barriers and facilitators to accessing care; and their personal experience working within VHA. Several questions focused on dual use and why rural veterans use multiple health care systems, their perspectives of dual use, their expectations of patients’ role(s) in health care coordination, and the perceived barriers that inhibit comanagement. Interviewers used comanagement and dual use interchangeably to discuss patients with multiple care providers, allowing interviewees to use their preferred terminology; assigned meanings were probed for clarification but not corrected by interviewers.
Between June and October 2009, teams of 2 to 3 researchers visited 15 clinics for 1 to 2 business days each. Researchers conducted interviews with a convenience sample of clinical staff. Consent forms and an explanation of the study were distributed, and those electing to participate voluntarily came to a designated room to complete an interview. All interviews were audio recorded for accuracy.
Interview recordings were transcribed verbatim and reviewed for accuracy. Prior to coding, transcripts were imported into a qualitative data management software program. A codebook, including a priori research hypotheses and de novo themes, was developed based on a systematic review of a randomly selected subset of interview transcripts.5 Four coders were responsible for coding all transcripts and validating coding through tests of agreement at predetermined intervals.
Regular meetings were conducted with coders and the lead qualitative investigator to discuss disagreements, clarify code definitions, or add new codes as needed. As codes were added, previous transcripts were coded/recoded for content related to the new codes. An audit trail was maintained, and iterative mediation of codes continued throughout the process. The final codebook contained 42 thematic codes, which reached saturation or data redundancy.6 Detailed analysis of the codes dual use, distance, and care coordination were used to inform this study.
Results
Among the 15 sites, 64 in-depth individual interviews were conducted, ranging from 5 to 53 minutes (average 26 minutes). Clinic staff demographic characteristics are depicted in the Table. Analysis of data captured in the codes dual use, distance, and care coordination resulted in notable concentration in 4 thematic areas: (1) clinic staff perceptions of the influence of access, convenience, and distance on dual use for rural patients; (2) communication and patient’s role in comanagement; (3) rules and regulations related to comanagement from the VHA perspective; and (4) barriers to comanagement and recommendations for education.
Influence of Access, Convenience, and Distance
Access to health care was central to the discussion of dual use and comanagement by clinic staff. Convenience was identified as the primary reason for rural patients’ use of non-VHA services, as many rural patients must travel outside their local community to access VHA care. Thus, dual use was most often noted for services typically available in patients’ local communities, especially management of chronic conditions.
The CBOCs provide important services for primary care and management of chronic conditions but are not available in all communities and may have limited hours/days that do not fit with patients’ schedules. The CBOCs are often unable to provide needed services, including but not limited to emergency care, diagnostic tests, physical and occupational therapy, and other specialty care services. As one VHA provider put it, “The biggest factor for [dual use] is availability, access, convenience.… It’s a lot more convenient to go to the hospital down the street than it is to go 120 miles to [the VAMC], or for some guys who live 30, 40 minutes the other side of here it becomes 150, 160-mile one-way trip.”
Related to access, distance and transportation barriers were identified by clinic staff as obstacles to care for rural patients. Despite efforts to offset the expense of travel through reimbursement to qualified veterans and coordinated van transport with Veterans Service Organizations, travel costs—both time and money—were seen as significant barriers to accessing VHA care, as was an inability to travel for those who are ill or frail and elderly. “We send people … in the van and for the most part that works, but eventually it gets expensive, or you’ve got somebody with chronic pain that can’t tolerate the van ride for 2 hours,” one interviewee
reported.
According to clinic staff, dual-use patients also rely on non-VHA providers in particular for urgent or emergency care, while relying on VHA primary care for reduced-cost medications, diagnostic testing, chronic disease management, or annual exams. When asked why rural patients may choose to see more than 1 provider, VHA providers responded. “[It’s] more convenient to have a local doctor just in case something went wrong and they need to see a doctor right away. So distance to this clinic would be the number one reason.” Another reported, “If it’s once or twice a year routine appointments they’ll come here, but… they’d rather go to a walk-in clinic nearby than spend so much [money] on gas.”
Communication and Patients’ Role
Communication between VHA and non-VHA providers is a necessary element of comanagement. Although phone calls or faxing patient medical records are available options, clinic staff reported it was more common to encounter patients hand carrying their records between providers. For dual-use patients, clinic staff indicated it was often unclear who was responsible for relaying information between providers. There is often ambiguity about who will (and should) fulfill this role and not enough time to adequately address or clarify how this is done. Some clinic staff believed that acting as the main conduits of information placed an undue burden on the patients, particularly asking them to be able to accurately relay medical information about tests or prescriptions that they may not fully understand. Others said that it was primarily the patients’ responsibility to give relevant information about their care to all their providers, because of VHA regulations and patient privacy laws. “[The] patient should tell the primary doctor to send them [medical records] because we can’t get the medical records without the patient’s permission,” said one provider.
Another provider utilized the nursing staff to call patients after their appointments to remind them to give their medical records to their non-VHA provider. The data suggest that responsibility for maintaining communication between providers ultimately falls on the patient. From the perspective of a nurse practitioner, “We just keep trying to educate the community…. I’ve been told that if the patient wants that privilege of using the VA for a pharmacy for an outside provider that we’re glad to do that. But it is their responsibility to communicate with their [non-VHA] physician. I think we just need to keep educating the patients.”
Rules and Regulations
VHA policies governing prescriptions, hospitalizations at outside facilities, and release of patient information regulate, and in some cases hinder, information flow between VHA and non-VHA providers. Many patients use VHA to obtain medications for lower out-of-pocket costs. This contributes to the number of dual-use patients in VHA and results in several challenges for VHA providers trying to manage patients’ prescriptions. For example, patients will ask to fill a prescription at a VHA pharmacy from their non-VHA providers; however, VHA pharmacies can only fill prescriptions from VHA providers.
Many VHA providers are willing to rewrite these prescriptions, but they may need to see the patient before adding or changing the prescription and require documentation to address contraindications, adverse reactions and/or therapeutic failure, and associated risks before making the authorization. VHA providers noted that because the VHA formulary does not contain all medications, non-VHA providers are often unfamiliar with the VHA National Formulary specifics and will write prescriptions for nonformulary medications, which require a nonformulary request from a VHA provider.
Clinic staff also mentioned difficulty in obtaining records from non-VHA providers. This can be particularly problematic if the patient lives a distance away from a VHA facility and does not have the necessary authorization to share records on file.
Barriers and Education Recommendations
Clinic staff identified coordination of care for dual-use patients as a barrier to providing care. Specifically, providers identified coordination as complicated by communication difficulties, inefficient medical record exchange, short staffing in VHA clinics, duplication of diagnostic services, and non-VHA providers’ lack of understanding regarding the services that VHA provides. Specific to rural clinics, comanagement was reportedly hindered by limitations in technology (eg, consistent Internet access), access to routine diagnostic services, and lack of relationships with non-VHA providers. Providers most frequently reported that the critical piece missing in comanagement is a relationship—and implied communication—between VHA clinics and non-VHA community clinics. The concept of a relationship between providers is evoked as a critical element to comanaging dual-use patients; however, clinic staff had a difficult time articulating what that relationship would actually look like if put into practice.
Related: Patients Benefit From ICU Telemedicine
In spite of the numerous barriers identified by clinic staff, the recommendation for education to improve comanagement was consistent across study sites and clinic staff roles. Education was proposed for patients and non-VHA providers as the best intervention. In response to a question about ideas and recommendations to improve comanagement, clinic staff drew on varied experiences. To illustrate this theme, a provider gave this example of dual-use patients seeking prescription medication from VHA and its impact on comanagement: “I would [recommend] an outreach program to community resources and [non-VHA] providers. To let them know more about how the VA works and the resources that are available, and how specifically to coordinate care through the VA, would be a significant benefit.… If the [non-VHA] providers knew how to—who to—talk to, what information the VA needs, for example, for medication changes, it would help the patients make it work…without having to overburden the patients with having to physically hand carry their blood test results, or their notes, discharge summaries, procedure notes.”
Along with providing outreach and education on working with the VHA, clinic staff addressed the need to educate patients more effectively, because they are seen as central to the information exchange. There is motivation on the part of patients to learn the system. “Just making sure that the patients realize that they need to tell their local providers to send us the records and make sure that there is an exchange going on consistently,” explained a case manager. “If the patient wants to get those medications that are costly, then they figure out pretty quick what they have to have, what they need to send to us.” The need for education is an ongoing process; who is responsible for this continues to be a point of debate.
Discussion
In order to better understand comanagement of dual-use patients, this study focused on the experiences and perceptions of staff at VHA primary care clinics in the upper Midwest. The data indicated that:
- VHA clinical staff perceive the primary reason patients choose to seek non-VHA care is because of access, convenience, and
distance - In order for comanagement to occur, communication and information exchange—currently facilitated largely by patients—needs to improve
- Education of patients and their non-VHA providers is recommended, to increase understanding of rules and regulations tied to exchange of patient information across health care systems
- Education may facilitate communication, develop relationships, and overcome barriers to information exchange
Distance to health care and perceived convenience were clearly seen by clinic staff as the driving factors behind their patients’ dual use. In the authors’ prior work, interviews with veterans and their VA providers supported this assertion as well; however, it was also found that distance must not be understood in isolation of other contingencies, such as urgency of need.4
Clinic staff identified institutional and individual barriers that lead to miscommunication and confusion on the part of patients and reported misunderstandings with non-VHA providers, including 3 potential barriers to comanagement. These included (1) inconsistent communication and flow of information between VHA and non-VHA providers; (2) uncertainty about who will (and should) be responsible for information flow between providers; and (3) VHA and federal regulations over patient privacy. Throughout the interviews, access to less expensive prescription medications in VHA was considered an additional driver of dual use. According to clinic staff interviewed, education of patients and non-VHA providers could facilitate efficient and safe comanagement for dual-use patients.7
This study suggests both advantages and disadvantages for patients choosing to use multiple health care systems from the perspective of the clinic staff. The primary advantage is better overall health care access, especially for rural patients and those with longer travel times to VHA facilities. The primary disadvantage of dual use is discontinuity of care between multiple care sites. Specifically, this study identified concerns regarding poor communication between providers and transfer of patient medical records. An underlying theme was a concern for quality of care and patient safety, which are recognized by others in the literature as potential consequences of inadequate comanagement.8-12
If there is one aspect of co-management for dual-use patients to target, this study’s findings point to developing strategies to improve communication between providers caring for dual-use patients and, more specifically, cultivating relationships that are currently underdeveloped. This will necessitate a clearer articulation of what constitutes a relationship between comanaging providers and is a direction for further research that would have applicability beyond VHA to any comanagement of patients using multiple health care systems.
There are 3 simultaneous, yet unrelated, factors that may contribute to increasing dual use. First is the rise in VHA eligible veterans from Operation Enduring Freedom, Operation Iraqi Freedom, and Operation New Dawn.13,14 All returning veterans who meet minimal requirements are eligible for 5 years of VHA health care. A large proportion of these individuals are in the Reserve and National Guard, most of whom have nonmilitary jobs that may provide employer-based health insurance. Thus, these veterans have a greater opportunity for dual use. Second, with the aging cohort of Vietnam-era veterans, a greater proportion is becoming Medicare eligible. Third, with the recent passing of the ACA, more patients, including veterans, may choose to purchase insurance through ACA health exchanges. Taken individually or collectively, these factors will likely have effects reaching beyond VHA, especially when veterans receiving care in non-VHA health care systems engage in dual use.3,13,15,16
Limitations
This study has a number of limitations. First, it was limited to VHA facilities located in the upper Midwest, which may limit generalizability to other parts of the country. The convenience sample of clinic staff at VHA clinics may not represent the full range of perspectives among HCPs generally. This study did not interview clinic staff in non-VHA clinics, although this has been the focus of other studies.17,18 Although dual use also applies to specialty care and related access issues in rural areas, this was not a focus of this study. Last, the data were collected in 2009, prior to the implementation of the patient-aligned care team (PACT) model and prior to the recently revealed issues regarding patient wait times for VHA care. Thus, perceptions may have changed, and additional study is needed.
Conclusions
The results of this study support prior assumptions of barriers to care, but also introduce previously unreported challenges. Dual use is perceived to have both positive and negative impacts, but for the positives to outweigh the negatives, thoughtful comanagement is critical. This may be particularly so in rural areas where dual use is encouraged as a way to overcome distance and increase convenience in accessing care.
As demonstrated by recent events, there are still VHA health care access issues for veterans. Recently, VA leadership and the U.S. Congress proposed that veterans have greater access to community providers as well as VHA in order to overcome delays in care.19 As this option is explored and put into practice, it is more important than ever to consider the need for care coordination and management of dual-use patients, to ensure good communication and care that is timely, safe, and high quality.
Few models exist in which 2 PCPs coordinate across health care systems, and greater understanding of this dual use is needed. This information is important in designing interventions to improve care coordination across systems to ensure continuity of care, patient safety, and patient satisfaction. Although some work has been done to examine the perspectives of non-VA PCPs, little is known about VHA provider perspectives on rural veteran dual use.17,18 This study explores VHA provider perspectives and identifies areas where interventions to improve care coordination across systems might be targeted.
Next steps for intervention studies would be to improve communication and develop educational tools to aid in the coordination of care between VHA and non-VHA providers. A recent example of this is the Co-Management Toolkit developed by the Veterans Rural Health Resource Center-Central Region, which provides information on VHA policies and targets non-VHA providers.20 Although VHA perceptions of comanageing dual-use patients were the target, a similar study of non-VHA providers is important to understand this complex and multifaceted dynamic. Additional work is needed to measure the impact of dual use on clinical outcomes, patient safety and quality, and efficient use of resources, as these are understudied. As dual use continues and potentially increases with the ACA and changing health care in the U.S., it is important to understand the management of patients using multiple health care systems. This is salient as primary care adopts the PACT model and to inform interventions to improve quality and safety while eliminating duplicative health care and costs.
Acknowledgements
The research reported here was supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Rural Health, Veterans Rural Health Resource Center-Central Region (VRHRC-CR) and the VA Health Services Research and Development (HSR&D) Service, the Comprehensive Access and Delivery Research and Evaluation (CADRE) Center at the Iowa City VA Health Care System, and Center to Improve Veteran Involvement in Care (CIVIC) at VA Portland Health Care System. Dr. Reisinger was supported by a Research Career Development Award from the Health Services Research and Development Service, Department of Veterans Affairs (CD1 08-013-1).
We would like to thank all health care providers who graciously agreed to participate in this study and VRHRC-CR staff, in particular Monica Paez for assistance on this manuscript.
Author disclosures
The authors report no actual or potential conflicts of interest with regard to this article.
Disclaimer
The opinions expressed herein are those of the authors and do not necessarily reflect those of Federal Practitioner, Frontline Medical Communications Inc., the U.S. Government, or any of its agencies. This article may discuss unlabeled or investigational use of certain drugs. Please review complete prescribing information for specific drugs or drug combinations—including indications, contraindications, warnings, and adverse effects—before administering pharmacologic therapy to patients.
The VHA assigns all enrolled veterans to a primary care provider (PCP). However, almost 80% of veterans enrolled in VHA have another form of health care coverage, including Medicare, Medicaid, private insurance, and TRICARE for Life program.1 Consequently, veterans may choose to use more than 1 health care system to manage their health care needs.
Studies based on merged VHA and Medicare claims data have demonstrated substantial dual use by VHA enrollees with Medicare. Petersen and colleagues reported that about 80% of VHA enrollees with Medicare chose to use services in both systems and that greater distance to VHA facilities and lower priority level for VHA care predicted lower VHA reliance.2 Among those aged < 65 years who had Medicare due to disability, 58% weredual users. These dual users relied more on private sector care for many health conditions, with the notable exception of substance abuse and mental health disorders, for which reliance on VHA care was greater.2 Another study found that over half of VHA enrollees assigned to a PCP at a community-based outpatient clinic (CBOC) received some or all of their care outside VHA and that reliance on VHA outpatient care declined over the 4-year study period.3
Related: Mutual Alignment Trumps Merger for Joint VA/DoD Health Care Programs
This use of multiple health care providers (HCPs), facilities, and modalities is often described as dual use or comanagement. Dual use in the case of veterans refers to use of both VHA and non-VHA health care, whereas comanagement implies an expectation of shared decision making and open communication between VHA and non-VHA providers. In addition to VHA PCPs, rural veterans frequently receive care from local, non-VHA HCPs in the community where they live. As health care in the U.S. evolves and patients have increasing choices through the Affordable Care Act (ACA), the challenge of comanagement for patients receiving care in multiple systems is likely to increase both within and outside VHA.
This study was part of a qualitative rural health needs assessment designed to ascertain the issues facing rural veterans and their providers in the upper Midwest.4 The objective was to examine VHA primary care clinic staff perspectives on dual users, perceived barriers that inhibit comanagement, and factors that contribute to the need for dual use in rural areas.
Methods
A qualitative study design with in-person interviews was used to elicit the perspective of VHA clinic staff on the current and ideal states of comanagement. Clinics were selected using a stratified purposeful sample of 15 urban and rural primary care clinics at VHA CBOCs and VAMCs in 8 Midwestern states (Illinois, Iowa, Minnesota, Nebraska, North Dakota, South Dakota, Wisconsin, and Wyoming). The stratification criteria included (1) urban and rural; (2) geographic coverage of VISN 23; and (3) VHA-managed and contract clinics, resulting in a purposeful sample of 2 urban VAMC clinics, 3 urban CBOCs, 7 rural VHA-managed CBOCs, and 3 rural contract CBOCs. The distance from the CBOC to the closest VAMC ranged from 32 to 242 miles.
Related: VA Relaxes Rules for Choice Program
Interview guides were developed and tested by the research team for comprehension, length, and timing prior to data collection and iteratively revised as analysis evolved and new topics emerged. Clinic staff were asked about their perceptions of rural veteran use of VHA care; barriers and facilitators to accessing care; and their personal experience working within VHA. Several questions focused on dual use and why rural veterans use multiple health care systems, their perspectives of dual use, their expectations of patients’ role(s) in health care coordination, and the perceived barriers that inhibit comanagement. Interviewers used comanagement and dual use interchangeably to discuss patients with multiple care providers, allowing interviewees to use their preferred terminology; assigned meanings were probed for clarification but not corrected by interviewers.
Between June and October 2009, teams of 2 to 3 researchers visited 15 clinics for 1 to 2 business days each. Researchers conducted interviews with a convenience sample of clinical staff. Consent forms and an explanation of the study were distributed, and those electing to participate voluntarily came to a designated room to complete an interview. All interviews were audio recorded for accuracy.
Interview recordings were transcribed verbatim and reviewed for accuracy. Prior to coding, transcripts were imported into a qualitative data management software program. A codebook, including a priori research hypotheses and de novo themes, was developed based on a systematic review of a randomly selected subset of interview transcripts.5 Four coders were responsible for coding all transcripts and validating coding through tests of agreement at predetermined intervals.
Regular meetings were conducted with coders and the lead qualitative investigator to discuss disagreements, clarify code definitions, or add new codes as needed. As codes were added, previous transcripts were coded/recoded for content related to the new codes. An audit trail was maintained, and iterative mediation of codes continued throughout the process. The final codebook contained 42 thematic codes, which reached saturation or data redundancy.6 Detailed analysis of the codes dual use, distance, and care coordination were used to inform this study.
Results
Among the 15 sites, 64 in-depth individual interviews were conducted, ranging from 5 to 53 minutes (average 26 minutes). Clinic staff demographic characteristics are depicted in the Table. Analysis of data captured in the codes dual use, distance, and care coordination resulted in notable concentration in 4 thematic areas: (1) clinic staff perceptions of the influence of access, convenience, and distance on dual use for rural patients; (2) communication and patient’s role in comanagement; (3) rules and regulations related to comanagement from the VHA perspective; and (4) barriers to comanagement and recommendations for education.
Influence of Access, Convenience, and Distance
Access to health care was central to the discussion of dual use and comanagement by clinic staff. Convenience was identified as the primary reason for rural patients’ use of non-VHA services, as many rural patients must travel outside their local community to access VHA care. Thus, dual use was most often noted for services typically available in patients’ local communities, especially management of chronic conditions.
The CBOCs provide important services for primary care and management of chronic conditions but are not available in all communities and may have limited hours/days that do not fit with patients’ schedules. The CBOCs are often unable to provide needed services, including but not limited to emergency care, diagnostic tests, physical and occupational therapy, and other specialty care services. As one VHA provider put it, “The biggest factor for [dual use] is availability, access, convenience.… It’s a lot more convenient to go to the hospital down the street than it is to go 120 miles to [the VAMC], or for some guys who live 30, 40 minutes the other side of here it becomes 150, 160-mile one-way trip.”
Related to access, distance and transportation barriers were identified by clinic staff as obstacles to care for rural patients. Despite efforts to offset the expense of travel through reimbursement to qualified veterans and coordinated van transport with Veterans Service Organizations, travel costs—both time and money—were seen as significant barriers to accessing VHA care, as was an inability to travel for those who are ill or frail and elderly. “We send people … in the van and for the most part that works, but eventually it gets expensive, or you’ve got somebody with chronic pain that can’t tolerate the van ride for 2 hours,” one interviewee
reported.
According to clinic staff, dual-use patients also rely on non-VHA providers in particular for urgent or emergency care, while relying on VHA primary care for reduced-cost medications, diagnostic testing, chronic disease management, or annual exams. When asked why rural patients may choose to see more than 1 provider, VHA providers responded. “[It’s] more convenient to have a local doctor just in case something went wrong and they need to see a doctor right away. So distance to this clinic would be the number one reason.” Another reported, “If it’s once or twice a year routine appointments they’ll come here, but… they’d rather go to a walk-in clinic nearby than spend so much [money] on gas.”
Communication and Patients’ Role
Communication between VHA and non-VHA providers is a necessary element of comanagement. Although phone calls or faxing patient medical records are available options, clinic staff reported it was more common to encounter patients hand carrying their records between providers. For dual-use patients, clinic staff indicated it was often unclear who was responsible for relaying information between providers. There is often ambiguity about who will (and should) fulfill this role and not enough time to adequately address or clarify how this is done. Some clinic staff believed that acting as the main conduits of information placed an undue burden on the patients, particularly asking them to be able to accurately relay medical information about tests or prescriptions that they may not fully understand. Others said that it was primarily the patients’ responsibility to give relevant information about their care to all their providers, because of VHA regulations and patient privacy laws. “[The] patient should tell the primary doctor to send them [medical records] because we can’t get the medical records without the patient’s permission,” said one provider.
Another provider utilized the nursing staff to call patients after their appointments to remind them to give their medical records to their non-VHA provider. The data suggest that responsibility for maintaining communication between providers ultimately falls on the patient. From the perspective of a nurse practitioner, “We just keep trying to educate the community…. I’ve been told that if the patient wants that privilege of using the VA for a pharmacy for an outside provider that we’re glad to do that. But it is their responsibility to communicate with their [non-VHA] physician. I think we just need to keep educating the patients.”
Rules and Regulations
VHA policies governing prescriptions, hospitalizations at outside facilities, and release of patient information regulate, and in some cases hinder, information flow between VHA and non-VHA providers. Many patients use VHA to obtain medications for lower out-of-pocket costs. This contributes to the number of dual-use patients in VHA and results in several challenges for VHA providers trying to manage patients’ prescriptions. For example, patients will ask to fill a prescription at a VHA pharmacy from their non-VHA providers; however, VHA pharmacies can only fill prescriptions from VHA providers.
Many VHA providers are willing to rewrite these prescriptions, but they may need to see the patient before adding or changing the prescription and require documentation to address contraindications, adverse reactions and/or therapeutic failure, and associated risks before making the authorization. VHA providers noted that because the VHA formulary does not contain all medications, non-VHA providers are often unfamiliar with the VHA National Formulary specifics and will write prescriptions for nonformulary medications, which require a nonformulary request from a VHA provider.
Clinic staff also mentioned difficulty in obtaining records from non-VHA providers. This can be particularly problematic if the patient lives a distance away from a VHA facility and does not have the necessary authorization to share records on file.
Barriers and Education Recommendations
Clinic staff identified coordination of care for dual-use patients as a barrier to providing care. Specifically, providers identified coordination as complicated by communication difficulties, inefficient medical record exchange, short staffing in VHA clinics, duplication of diagnostic services, and non-VHA providers’ lack of understanding regarding the services that VHA provides. Specific to rural clinics, comanagement was reportedly hindered by limitations in technology (eg, consistent Internet access), access to routine diagnostic services, and lack of relationships with non-VHA providers. Providers most frequently reported that the critical piece missing in comanagement is a relationship—and implied communication—between VHA clinics and non-VHA community clinics. The concept of a relationship between providers is evoked as a critical element to comanaging dual-use patients; however, clinic staff had a difficult time articulating what that relationship would actually look like if put into practice.
Related: Patients Benefit From ICU Telemedicine
In spite of the numerous barriers identified by clinic staff, the recommendation for education to improve comanagement was consistent across study sites and clinic staff roles. Education was proposed for patients and non-VHA providers as the best intervention. In response to a question about ideas and recommendations to improve comanagement, clinic staff drew on varied experiences. To illustrate this theme, a provider gave this example of dual-use patients seeking prescription medication from VHA and its impact on comanagement: “I would [recommend] an outreach program to community resources and [non-VHA] providers. To let them know more about how the VA works and the resources that are available, and how specifically to coordinate care through the VA, would be a significant benefit.… If the [non-VHA] providers knew how to—who to—talk to, what information the VA needs, for example, for medication changes, it would help the patients make it work…without having to overburden the patients with having to physically hand carry their blood test results, or their notes, discharge summaries, procedure notes.”
Along with providing outreach and education on working with the VHA, clinic staff addressed the need to educate patients more effectively, because they are seen as central to the information exchange. There is motivation on the part of patients to learn the system. “Just making sure that the patients realize that they need to tell their local providers to send us the records and make sure that there is an exchange going on consistently,” explained a case manager. “If the patient wants to get those medications that are costly, then they figure out pretty quick what they have to have, what they need to send to us.” The need for education is an ongoing process; who is responsible for this continues to be a point of debate.
Discussion
In order to better understand comanagement of dual-use patients, this study focused on the experiences and perceptions of staff at VHA primary care clinics in the upper Midwest. The data indicated that:
- VHA clinical staff perceive the primary reason patients choose to seek non-VHA care is because of access, convenience, and
distance - In order for comanagement to occur, communication and information exchange—currently facilitated largely by patients—needs to improve
- Education of patients and their non-VHA providers is recommended, to increase understanding of rules and regulations tied to exchange of patient information across health care systems
- Education may facilitate communication, develop relationships, and overcome barriers to information exchange
Distance to health care and perceived convenience were clearly seen by clinic staff as the driving factors behind their patients’ dual use. In the authors’ prior work, interviews with veterans and their VA providers supported this assertion as well; however, it was also found that distance must not be understood in isolation of other contingencies, such as urgency of need.4
Clinic staff identified institutional and individual barriers that lead to miscommunication and confusion on the part of patients and reported misunderstandings with non-VHA providers, including 3 potential barriers to comanagement. These included (1) inconsistent communication and flow of information between VHA and non-VHA providers; (2) uncertainty about who will (and should) be responsible for information flow between providers; and (3) VHA and federal regulations over patient privacy. Throughout the interviews, access to less expensive prescription medications in VHA was considered an additional driver of dual use. According to clinic staff interviewed, education of patients and non-VHA providers could facilitate efficient and safe comanagement for dual-use patients.7
This study suggests both advantages and disadvantages for patients choosing to use multiple health care systems from the perspective of the clinic staff. The primary advantage is better overall health care access, especially for rural patients and those with longer travel times to VHA facilities. The primary disadvantage of dual use is discontinuity of care between multiple care sites. Specifically, this study identified concerns regarding poor communication between providers and transfer of patient medical records. An underlying theme was a concern for quality of care and patient safety, which are recognized by others in the literature as potential consequences of inadequate comanagement.8-12
If there is one aspect of co-management for dual-use patients to target, this study’s findings point to developing strategies to improve communication between providers caring for dual-use patients and, more specifically, cultivating relationships that are currently underdeveloped. This will necessitate a clearer articulation of what constitutes a relationship between comanaging providers and is a direction for further research that would have applicability beyond VHA to any comanagement of patients using multiple health care systems.
There are 3 simultaneous, yet unrelated, factors that may contribute to increasing dual use. First is the rise in VHA eligible veterans from Operation Enduring Freedom, Operation Iraqi Freedom, and Operation New Dawn.13,14 All returning veterans who meet minimal requirements are eligible for 5 years of VHA health care. A large proportion of these individuals are in the Reserve and National Guard, most of whom have nonmilitary jobs that may provide employer-based health insurance. Thus, these veterans have a greater opportunity for dual use. Second, with the aging cohort of Vietnam-era veterans, a greater proportion is becoming Medicare eligible. Third, with the recent passing of the ACA, more patients, including veterans, may choose to purchase insurance through ACA health exchanges. Taken individually or collectively, these factors will likely have effects reaching beyond VHA, especially when veterans receiving care in non-VHA health care systems engage in dual use.3,13,15,16
Limitations
This study has a number of limitations. First, it was limited to VHA facilities located in the upper Midwest, which may limit generalizability to other parts of the country. The convenience sample of clinic staff at VHA clinics may not represent the full range of perspectives among HCPs generally. This study did not interview clinic staff in non-VHA clinics, although this has been the focus of other studies.17,18 Although dual use also applies to specialty care and related access issues in rural areas, this was not a focus of this study. Last, the data were collected in 2009, prior to the implementation of the patient-aligned care team (PACT) model and prior to the recently revealed issues regarding patient wait times for VHA care. Thus, perceptions may have changed, and additional study is needed.
Conclusions
The results of this study support prior assumptions of barriers to care, but also introduce previously unreported challenges. Dual use is perceived to have both positive and negative impacts, but for the positives to outweigh the negatives, thoughtful comanagement is critical. This may be particularly so in rural areas where dual use is encouraged as a way to overcome distance and increase convenience in accessing care.
As demonstrated by recent events, there are still VHA health care access issues for veterans. Recently, VA leadership and the U.S. Congress proposed that veterans have greater access to community providers as well as VHA in order to overcome delays in care.19 As this option is explored and put into practice, it is more important than ever to consider the need for care coordination and management of dual-use patients, to ensure good communication and care that is timely, safe, and high quality.
Few models exist in which 2 PCPs coordinate across health care systems, and greater understanding of this dual use is needed. This information is important in designing interventions to improve care coordination across systems to ensure continuity of care, patient safety, and patient satisfaction. Although some work has been done to examine the perspectives of non-VA PCPs, little is known about VHA provider perspectives on rural veteran dual use.17,18 This study explores VHA provider perspectives and identifies areas where interventions to improve care coordination across systems might be targeted.
Next steps for intervention studies would be to improve communication and develop educational tools to aid in the coordination of care between VHA and non-VHA providers. A recent example of this is the Co-Management Toolkit developed by the Veterans Rural Health Resource Center-Central Region, which provides information on VHA policies and targets non-VHA providers.20 Although VHA perceptions of comanageing dual-use patients were the target, a similar study of non-VHA providers is important to understand this complex and multifaceted dynamic. Additional work is needed to measure the impact of dual use on clinical outcomes, patient safety and quality, and efficient use of resources, as these are understudied. As dual use continues and potentially increases with the ACA and changing health care in the U.S., it is important to understand the management of patients using multiple health care systems. This is salient as primary care adopts the PACT model and to inform interventions to improve quality and safety while eliminating duplicative health care and costs.
Acknowledgements
The research reported here was supported by the Department of Veterans Affairs, Veterans Health Administration, Office of Rural Health, Veterans Rural Health Resource Center-Central Region (VRHRC-CR) and the VA Health Services Research and Development (HSR&D) Service, the Comprehensive Access and Delivery Research and Evaluation (CADRE) Center at the Iowa City VA Health Care System, and Center to Improve Veteran Involvement in Care (CIVIC) at VA Portland Health Care System. Dr. Reisinger was supported by a Research Career Development Award from the Health Services Research and Development Service, Department of Veterans Affairs (CD1 08-013-1).
We would like to thank all health care providers who graciously agreed to participate in this study and VRHRC-CR staff, in particular Monica Paez for assistance on this manuscript.
Author disclosures
The authors report no actual or potential conflicts of interest with regard to this article.
Disclaimer
The opinions expressed herein are those of the authors and do not necessarily reflect those of Federal Practitioner, Frontline Medical Communications Inc., the U.S. Government, or any of its agencies. This article may discuss unlabeled or investigational use of certain drugs. Please review complete prescribing information for specific drugs or drug combinations—including indications, contraindications, warnings, and adverse effects—before administering pharmacologic therapy to patients.
1. Department of Veterans Affairs Office of Rural Health, VHA. Veterans Rural Health: Perspectives and Opportunities. Rockville, MD: Booz Allen Hamilton; 2008. http://www.ruralhealth.va.gov/docs/PAO-final-report-0208.pdf. Accessed July 6, 2015.
2. Petersen LA, Byrne MM, Daw CN, Hasche J, Reis B, Pietz K. Relationship between clinical conditions and use of Veterans Affairs health care among Medicare‐enrolled veterans. Health Serv Res. 2010;45(3):762-791.
3. Liu CF, Chapko M, Bryson CL, et al. Use of outpatient care in Veterans Health Administration and Medicare among veterans receiving primary care in community-based and hospital outpatient clinics. Health Serv Res. 2010;45(5 pt 1):1268-1286.
4. Buzza C, Ono SS, Turvey C, et al. Distance is relative: unpacking a principal barrier in rural healthcare. J Gen Intern Med. 2011;26(suppl 2):648-654.
5. Bernard HR, Ryan GW. Analyzing Qualitative Data: Systematic Approaches. Los Angeles, CA: SAGE; 2010.
6. Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18(1):59-82.
7. Kramer BJ, Vivrette RL, Satter DE, Jouldjian S, McDonald LR. Dual use of Veterans Health Administration and Indian Health Service: healthcare provider and patient perspectives. J Gen Intern Med. 2009;24(6):758-764.
8. Ajmera M, Wilkins TL, Sambamoorthi U. Dual Medicare and Veteran Health Administration use and ambulatory care sensitive hospitalizations. J Gen Intern Med. 2011;26(suppl 2):669-675.
9. Jia H, Zheng Y, Reker DM, et al. Multiple system utilization and mortality for veterans with stroke. Stroke. 2007;38(2):355-360.
10. Trivedi AN, Grebla RC, Jiang L, Yoon J, Mor V, Kizer KW. Duplicate federal payments for dual enrollees in Medicare Advantage plans and the Veterans Affairs health care system. JAMA. 2012;308(1):67-72.
11. Kaboli PJ, Shivapour DM, Henderson MS, Ishani A, Charlton ME. The impact of primary care dual-management on quality of care. J Prim Care Community Health. 2012;3(1):11-16.
12. Wolinsky FD, Miller TR, An H, Brezinski PR, Vaughn TE, Rosenthal GE. Dual use of Medicare and the Veterans Health Administration: are there adverse health outcomes? BMC Health Serv Res. 2006;6:131.
13. Liu CF, Bryson CL, Burgess JF Jr, Sharp N, Perkins M, Maciejewski ML. Use of outpatient care in VA and Medicare among disability-eligible and age-eligible veteran patients. BMC Health Serv Res. 2012;12:51.
14. Miller EA, Intrator O. Veterans use of non-VHA services: implications for policy and planning. Soc Work Public Health. 2012;27(4):379-391.
15. Bachman SS, Gonyea JG. Improving health care delivery to aging adults with disabilities: social work with dual eligibles in a climate of health care reform. J Gerontol Soc Work. 2012;55(2):191-207.
16. Kizer KW. Veterans and the Affordable Care Act. JAMA. 2012;307(8):789-790.
17. Lampman MA, Mueller KJ. Experiences of rural non-VA providers in treating dual care veterans and the development of electronic health information exchange networks between the two systems. J Rural Soc Sci. 2011;26(3):201-219.
18. Gaglioti A, Cozad A, Wittrock S, et al. Non-VA primary care providers’ perspectives on comanagement for rural veterans. Mil Med. 2014;179(11):1236-1243.
19. U.S. Department of Veterans Affairs. Acting Secretary Gibson outlines problems, actions taken, and budget resources needed to ensure access to care. U.S. Department of Veterans Affairs Website. http://www.va.gov/opa/pressrel/pressrelease.cfm?id=2586. Published July 16, 2014. Accessed July 6, 2015.
20. Office of Rural Health Central Region. Co-managed care toolkit. U.S. Department of Veterans Affairs Website. http://www.ruralhealth.va.gov/resource-centers/central/comanagement-toolkit.asp. Updated June 3, 2015. Accessed July 6, 2015.
1. Department of Veterans Affairs Office of Rural Health, VHA. Veterans Rural Health: Perspectives and Opportunities. Rockville, MD: Booz Allen Hamilton; 2008. http://www.ruralhealth.va.gov/docs/PAO-final-report-0208.pdf. Accessed July 6, 2015.
2. Petersen LA, Byrne MM, Daw CN, Hasche J, Reis B, Pietz K. Relationship between clinical conditions and use of Veterans Affairs health care among Medicare‐enrolled veterans. Health Serv Res. 2010;45(3):762-791.
3. Liu CF, Chapko M, Bryson CL, et al. Use of outpatient care in Veterans Health Administration and Medicare among veterans receiving primary care in community-based and hospital outpatient clinics. Health Serv Res. 2010;45(5 pt 1):1268-1286.
4. Buzza C, Ono SS, Turvey C, et al. Distance is relative: unpacking a principal barrier in rural healthcare. J Gen Intern Med. 2011;26(suppl 2):648-654.
5. Bernard HR, Ryan GW. Analyzing Qualitative Data: Systematic Approaches. Los Angeles, CA: SAGE; 2010.
6. Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18(1):59-82.
7. Kramer BJ, Vivrette RL, Satter DE, Jouldjian S, McDonald LR. Dual use of Veterans Health Administration and Indian Health Service: healthcare provider and patient perspectives. J Gen Intern Med. 2009;24(6):758-764.
8. Ajmera M, Wilkins TL, Sambamoorthi U. Dual Medicare and Veteran Health Administration use and ambulatory care sensitive hospitalizations. J Gen Intern Med. 2011;26(suppl 2):669-675.
9. Jia H, Zheng Y, Reker DM, et al. Multiple system utilization and mortality for veterans with stroke. Stroke. 2007;38(2):355-360.
10. Trivedi AN, Grebla RC, Jiang L, Yoon J, Mor V, Kizer KW. Duplicate federal payments for dual enrollees in Medicare Advantage plans and the Veterans Affairs health care system. JAMA. 2012;308(1):67-72.
11. Kaboli PJ, Shivapour DM, Henderson MS, Ishani A, Charlton ME. The impact of primary care dual-management on quality of care. J Prim Care Community Health. 2012;3(1):11-16.
12. Wolinsky FD, Miller TR, An H, Brezinski PR, Vaughn TE, Rosenthal GE. Dual use of Medicare and the Veterans Health Administration: are there adverse health outcomes? BMC Health Serv Res. 2006;6:131.
13. Liu CF, Bryson CL, Burgess JF Jr, Sharp N, Perkins M, Maciejewski ML. Use of outpatient care in VA and Medicare among disability-eligible and age-eligible veteran patients. BMC Health Serv Res. 2012;12:51.
14. Miller EA, Intrator O. Veterans use of non-VHA services: implications for policy and planning. Soc Work Public Health. 2012;27(4):379-391.
15. Bachman SS, Gonyea JG. Improving health care delivery to aging adults with disabilities: social work with dual eligibles in a climate of health care reform. J Gerontol Soc Work. 2012;55(2):191-207.
16. Kizer KW. Veterans and the Affordable Care Act. JAMA. 2012;307(8):789-790.
17. Lampman MA, Mueller KJ. Experiences of rural non-VA providers in treating dual care veterans and the development of electronic health information exchange networks between the two systems. J Rural Soc Sci. 2011;26(3):201-219.
18. Gaglioti A, Cozad A, Wittrock S, et al. Non-VA primary care providers’ perspectives on comanagement for rural veterans. Mil Med. 2014;179(11):1236-1243.
19. U.S. Department of Veterans Affairs. Acting Secretary Gibson outlines problems, actions taken, and budget resources needed to ensure access to care. U.S. Department of Veterans Affairs Website. http://www.va.gov/opa/pressrel/pressrelease.cfm?id=2586. Published July 16, 2014. Accessed July 6, 2015.
20. Office of Rural Health Central Region. Co-managed care toolkit. U.S. Department of Veterans Affairs Website. http://www.ruralhealth.va.gov/resource-centers/central/comanagement-toolkit.asp. Updated June 3, 2015. Accessed July 6, 2015.
Housestaff Teams and Patient Outcomes
Since the Institute of Medicine Report To Err is Human, increased attention has been paid to improving the care of hospitalized patients.[1] Strategies include utilization of guidelines and pathways, and the application of quality improvement techniques to improve or standardize processes. Despite improvements in focused areas such as prevention of hospital‐acquired infections, evidence suggests that outcomes for hospitalized patients remain suboptimal.[2] Rates of errors and hospital‐related complications such as falls, decubitus ulcers, and infections remain high,[3, 4, 5] and not all patients receive what is known to be appropriate care.[6]
Many attempts to improve inpatient care have used process‐improvement approaches, focusing on impacting individuals' behaviors, or on breaking down processes into component parts. Examples include central line bundles or checklists.[7, 8] These approaches attempt to ensure that providers do things in a standardized way, but are implicitly based on the reductionist assumption that we can break processes down into predictable parts to improve the system. An alternative way to understand clinical systems is based on interdependencies between individuals in the system, or the ways in which parts of the system interact with each other, which may be unpredictable over time.[1, 9] Whereas these interdependencies include care processes, they also encompass the providers who care for patients. Providers working together vary in terms of the kinds of relationships they have with each other. Those relationships are crucial to system function because they are the foundation for the interactions that lead to effective patient care.
The application of several frameworks or approaches for considering healthcare systems in terms of relationships highlights the importance of this way of understanding system function. The include complexity science,[1, 7] relational coordination (which is grounded in complexity science),[10] high reliability,[11] and the Big Five for teamwork.[12]
Research indicates that interactions among healthcare providers can have important influences on outcomes.[13, 14, 15, 16, 17] Additionally, the initial implementation of checklists to prevent central‐line associated infections appeared to change provider relationships in a way that significantly influenced their success.[18] For example, positive primary care clinic member relationships as assessed by the Lanham framework have been associated with better chronic care model implementation, learning, and patient experience of care.[19, 20] This framework, which we apply here, identifies 7 relationship characteristics: (1) trust; (2) diversity; (3) respect; (4) mindfulness, or being open to new ideas from others; (5) heedfulness, or an understanding of how one's roles influence those of others; (6) use of rich in‐person or verbal communication, particularly for potentially ambiguous information open to multiple interpretations; and (7) having a mixture of social and task relatedness among teams, or relatedness outside of only work‐related tasks.[19] Relationships within surgical teams that are characterized by psychological safety and diversity are associated with successful uptake of new techniques and decreased mortality.[13, 14] Relationships are important because the ability of patients and providers to learn and make sense of their patients' illnesses is grounded in relationships.
We sought to better understand and characterize inpatient physician teams' relationships, and assess the association between team relationships as evaluated by Lanham's framework and outcomes for hospitalized patients. Data on relationships among inpatient medical teams are few, despite the fact that these teams provide a great proportion of inpatient care. Additionally, the care of hospitalized medical patients is complex and uncertain, often involving multiple providers, making provider relationships potentially even more important to outcomes than in other settings.
METHODS
Overview
We conducted an observational, convergent mixed‐methods study of inpatient medicine teams.[21, 22, 23] We focused on inpatient physician teams, defining them as the functional work group responsible for medical decision making in academic medical centers. Physician teams in this context have been studied in terms of social hierarchy, authority, and delegation.[24, 25, 26] Focusing on the relationships within these groups could provide insights into strategies to mitigate potential negative effects of hierarchy. We recognize that other providers are closely involved in the care of hospitalized patients, and although we did not have standard interactions between physicians, nurses, case managers, and other providers that we could consistently observe, we did include interactions with these other providers in our observations and assessments of team relationships. Because this work is among the first in inpatient medical teams, we chose to study a small number of teams in great depth, allowing us to make rich assessments of team relationships.
We chose patient outcomes of length of stay (LOS), unnecessary LOS (ULOS), and complication rates, adjusted for patient characteristics and team workload. LOS is an important metric of inpatient care delivery. We feel ULOS is an aspect of LOS that is dependent on the physician team, as it reflects their preparation of the patient for discharge. Finally, we chose complication rates because hospital‐acquired conditions and complications are important contributors to inpatient morbidity, and because recent surgical literature has identified complication rates as a contributor to mortality that could be related to providers' collective ability to recognize complications and act quickly.
This study was approved by the institutional review board at the University of Texas Health Science Center at San Antonio (UTHSCSA), the Research and Development Committee for the South Texas Veterans Health Care System (STVHCS), and the Research Committee at University Health System (UHS). All physicians consented to participate in the study. We obtained a waiver of consent for inclusion of patient data.
Setting and Study Participants
This study was conducted at the 2 UTHSCSA primary teaching affiliates. The Audie L. Murphy Veterans Affairs Hospital is the 220‐bed acute‐care hospital of the STVHCS. University Hospital is the 614‐bed, level‐I trauma, acute‐care facility for UHS, the county system for Bexar County, which includes the San Antonio, Texas major metropolitan area.
The inpatient internal medicine physician team was our unit of study. Inpatient medicine teams consisted of 1 faculty attending physician, 1 postgraduate year (PGY)‐2 or PGY‐3 resident, and 2 PGY‐1 members. In addition, typically 2 to 3 third‐year medical students were part of the team, and a subintern was sometimes present. Doctor of Pharmacy faculty and students were also occasionally part of the team. Social workers and case managers often joined team rounds for portions of the time, and nurses sometimes joined bedside rounds on specific patients. These teams admit all medicine patients with the exception of those with acute coronary syndromes, new onset congestive heart failure, or arrhythmias. Patients are randomly assigned to teams based on time of admission and call schedules.
Between these 2 hospitals, there are 10 inpatient medicine teams caring for patients, with a pool of over 40 potential faculty attendings. Our goal was to observe teams that would be most likely to vary in terms of their relationship characteristics and patient outcomes through observing teams with a range of individual members. We used a purposeful sampling approach to obtain a diverse sample, sampling based on physician attributes and time of year.[16, 17] Three characteristics were most important: attending physician years of experience, attending involvement in educational and administrative leadership, and the presence of struggling resident members, as defined by being on probation or having been discussed in the residency Clinical Competency Committee. We did not set explicit thresholds in terms of attending experience, but instead sought to ensure a range. The attendings we observed were more likely to be involved in education and administrative leadership activities, but were otherwise similar to those we did not observe in terms of years of experience. We included struggling residents to observe individuals with a range of skill sets, and not just high‐performing individuals. We obtained attending information based on our knowledge of the attending faculty pool, and from the internal medicine residency program. We sampled across the year to ensure a diversity of trainee experience, but did not observe teams in either July or August, as these months were early in the academic year. Interns spend approximately 5 months per year on inpatient services, whereas residents spend 2 to 3 months per year. Thus, interns but not residents observed later in the year might have spent significantly more time on an inpatient service. However, in all instances, none of the team members observed had worked together previously.
Data Collection
Data were collected over nine 1‐month periods from September 2008 through June 2011. Teams were observed daily for 2‐ to 4‐week periods during morning rounds, the time when the team discusses each patient and makes clinical decisions. Data collection started on the first day of the month, the first day that all team members worked together, and continued for approximately 27 days, the last day before the resident rotated to a different service. By comprehensively and systematically observing these teams' daily rounds, we obtained rich, in‐depth data with multiple data points, enabling us to assess specific team behaviors and interactions.
During the third and fourth months, we collected data on teams in which the attending changed partway through. We did this to understand the impact of individual attending change on team relationships. Because the team relationships differed with each attending, we analyzed them separately. Thus, we observed 7 teams for approximately 4‐week periods and 4 teams for approximately 2‐week periods.
Observers arrived in the team room prior to rounds to begin observations, staying until after rounds were completed. Detailed free‐text field notes were taken regarding team activities and behaviors, including how the teams made patient care decisions. Field notes included: length of rounds, which team members spoke during each patient discussion, who contributed to management discussions, how information from consultants was incorporated, how communication with others outside of the team occurred, how team members spoke with each other including the types of words used, and team member willingness to perform tasks outside of their usually defined role, among others. Field notes were collected in an open‐ended format to allow for inductive observations. Observers also recorded clinical data daily regarding each patient, including admission and discharge dates, and presenting complaint.
The observation team consisted of the principle investigator (PI) (hospitalist) and 2 research assistants (a graduate‐level medical anthropologist and social psychologist), all of whom were trained by a qualitative research expert to systematically collect data related to topics of interest. Observers were instructed to record what the teams were doing and talking about at all times, noting any behaviors that they felt reflected how team members related to each other and came to decisions about their patients, or that were characteristic of the team. To ensure consistency, the PI and 1 research assistant conducted observations jointly at the start of data collection for each team, checking concordance of observations daily using a percent agreement until general agreement on field note content and patient information reached 90%. Two individuals observed 24 days of data collection, representing 252 patient discussions (13% of observed discussions).
An age‐adjusted Charlson‐Deyo comorbidity score was calculated for each patient admitted to each team, using data from rounds and from each hospital's electronic health records (EHR).[27] We collected data regarding mental health conditions for each patient (substance use, mood disorder, cognitive disorder, or a combination) because these comorbidities could impact LOS or ULOS. Discharge diagnoses were based on the discharge summary in the EHR. We also collected data daily regarding team census and numbers of admissions to and discharges from each team to assess workload.
Three patient outcomes were measured: LOS, ULOS, and complications. LOS was defined as the total number of days the patient was in the hospital. ULOS was defined as the number of days a patient remained in the hospital after the day the team determined the patient was medically ready for discharge (assessed by either discussion on rounds or EHR documentation). ULOS may occur when postdischarge needs have been adequately assessed, or because of delays in care, which may be related to provider communication during the hospitalization. Complications were defined on a per‐patient, per‐day basis in 2 ways: the development of a new problem in the hospital, for example acute kidney injury, a hospital‐acquired infection, or delirium, or by the team noting a clinical deterioration after at least 24 hours of clinical stability, such as the patient requiring transfer to a higher level of care. Complications were determined based on discussions during rounds, with EHR verification if needed.
Analysis Phase I: Assessment of Relationship Characteristics
After the completion of data collection, field notes were reviewed by a research team member not involved in the original study design or primary data collection (senior medical student). We took this approach to guard against biasing the reviewer's view of team behaviors, both in terms of not having conducted observations of the teams and being blinded to patient outcomes.
The reviewer completed a series of 3 readings of all field notes. The first reading provided a summary of the content of the data and the individual teams. Behavioral patterns of each team were used to create an initial team profile. The field notes and profiles were reviewed by the PI and a coauthor not involved in data collection to ensure that the profiles adequately reflected the field notes. No significant changes to the profiles were made based on this review. The profiles were discussed at a meeting with members of the larger research team, including the PI, research assistants, and coinvestigators (with backgrounds in medicine, anthropology, and information and organization management). Behavior characteristics that could be used to distinguish teams were identified in the profiles using a grounded theory approach.
The second review of field notes was conducted to test the applicability of the characteristics identified in the first review. To systematically record the appearance of the behaviors, we created a matrix with a row for each behavior and columns for each team to note whether they exhibited each behavior. If the behavior was exhibited, specific examples were cataloged in the matrix. This matrix was reviewed and refined by the research team. During the final field note review meeting, the research team compared the summary matrix for each team, with the specific behaviors noted during the first reading of the field notes to ensure that all behaviors were recorded.
After cataloging behaviors, the research team assigned each behavior to 1 of the 7 Lanham relationship characteristics. We wanted to assess our observations against a relationship framework to ensure that we were able to systematically assess all aspects of relationships. The Lanham framework was initially developed based on a systematic review of the organizational and educational literatures, making it relevant to the complex environment of an academic medical inpatient team and allowing us to assess relationships at a fine‐grained, richly detailed level. This assignment was done by the author team as a group. Any questions were discussed and different interpretations resolved through consensus. The Lanham framework has 7 characteristics.[19] Based on the presence of behaviors associated with each relationship characteristic, we assigned a point to each team for each relationship characteristic observed. We considered a behavior type to be present if we observed it on at least 3 occasions on separate days. Though we used a threshold of at least 3 occurrences, most teams that did not receive a point for a particular characteristic did not have any instances in which we observed the characteristic. This was particularly true for trust and mindfulness, and least so for social/task relatedness. By summing these points, we calculated a total relationship score for each team, with potential scores ranging from 0 (for teams exhibiting no behaviors reflecting a particular relationship characteristic) to 7.
Analysis Phase II: Factor Analysis
To formally determine which relationship characteristics were most highly related, data were submitted to a principal components factor analysis using oblique rotation. Item separation was determined by visual inspection of the scree plot and eigenvalues over 1.
Analysis Phase III: Assessing the Association between Physician Team Relationship Characteristics and Patient Outcomes
We examined the association between team relationships and patient outcomes using team relationship scores. For the LOS/ULOS analysis, we only included patients whose entire hospitalization occurred under the care of the team we observed. Patients who were on the team at the start of the month, were transferred from another service, or who remained hospitalized after the end of the team's time together were excluded. The longest possible LOS for patients whose entire hospitalization occurred on teams that were observed for half a month was 12 days. To facilitate accurate comparison between teams, we only included patients whose LOS was 12 days.
Complication rates were defined on a per‐patient per‐day basis to normalize for different team volumes and days of observation. For this analysis, we included patients who remained on the team after data collection completion, patients transferred to another team, or patients transferred from another team. However, we only counted complications that occurred at least 24 hours following transfer to minimize the likelihood that the complication was related to the care of other physicians.
Preliminary analysis involved inspection and assessment of the distribution of all variables followed, by a general linear modeling approach to assess the association between patient and workload covariates and outcomes.[28, 29] Because we anticipated that outcome variables would be markedly skewed, we also planned to assess the association between relationship characteristics with outcomes using the Kruskal‐Wallis rank sum test to compare groups with Dunn's test[30] for pairwise comparisons if overall significance occurred.[31] There are no known acceptable methods for covariate adjustments using the Kruskal‐Wallis method. All models were run using SAS software (SAS Institute Inc., Cary, NC).[32]
RESULTS
The research team observed 1941 discussions of 576 individual patients. Observations were conducted over 352 hours and 54 minutes, resulting in 741 pages of notes (see Supporting Table 1 in the online version of this article for data regarding individual team members). Teams observed over half‐months are referred to with a and b designations.
| Relationship Characteristic | Definition | Thirteen Types of Behaviors Observed in Field Notes | Observed Examples |
|---|---|---|---|
| Trust | Willingness to be vulnerable to others | Use of we instead of you or I by the attending | Where are we going with this guy? |
| Attending admitting I don't know | Let's go talk to him, I can't figure this out | ||
| Asking questions to help team members to think through problems | Will the echo change our management? How will it help us? | ||
| Diversity | Including different perspectives and different thinking | Team member participation in conversations about patients that are not theirs | One intern is presenting, another intern asks a question, and the resident joins the discussion |
| Inclusion of perspectives of those outside the team (nursing and family members) | Taking a break to call the nurse, having a family meeting | ||
| Respect | Valuing the opinions of others, honest and tactful interactions | Use of positive reinforcement by the attending | Being encouraging of the medical student's differential, saying excellent |
| How the team talks with patients | Asking if the patient has any concerns, what they can do to make them comfortable | ||
| Heedfulness | Awareness of how each person's roles impact the rest of the team | Team members performing tasks not expected of their role | One intern helping another with changing orders to transfer a patient |
| Summarizing plans and strategizing | Attending recaps the plan for the day, asks what they can do | ||
| Mindfulness | Openness to new ideas/free discussion about what is and is not working | Entire team engaged in discussion | Attending asks the medical student, intern, and resident what they think is going on |
| Social relatedness | Having socially related interactions | Social conversation among team members | Intern talks about their day off |
| Jokes by the attending | Showers and a bowel movement is the key to making people happy | ||
| Appropriate use of rich communication | Use of in‐person communication for sensitive or difficult issues | Using verbal communication with consultants or family | Intern is on the phone with the pharm D because there is a problem with the medication |
Creation of team profiles yielded 13 common behavior characteristics that were inductively identified and that could potentially distinguish teams, including consideration of perspectives outside of the team and team members performing tasks normally outside of their roles. Table 1 provides examples of and summarizes observed behaviors using examples from the field notes, mapping these behavior characteristics onto the Lanham relationship characteristics. The distribution of relationship characteristics and scores for each team are shown in Table 2.
| Relationship Characteristic | Team | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3a | 3b | 4a | 4b | 5 | 6 | 7 | 8 | 9 | |
| Trust | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 1 |
| Diversity | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
| Respect | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 1 |
| Heedfulness | 0 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 1 | 1 |
| Mindfulness | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 1 | 1 |
| Social/task relatedness | 0 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 1 |
| Rich/lean communication | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 0 |
| Relationship score (no. of characteristics observed) | 0 | 5 | 7 | 2 | 2 | 3 | 5 | 0 | 7 | 7 | 6 |
Correlation between relationship characteristics ranged from 0.32 to 0.95 (see Supporting Table 2 in the online version of this article). Mindfulness and trust are more highly correlated with each other than with other variables, as are diversity and respect. We performed a principal components factor analysis. Based on scree plot inspection and eigenvalues >1, we kept 3 factors that explained 85% of the total variance (see Supporting Table 3 in the online version of this article).
| No. of Relationship Characteristics | |||
|---|---|---|---|
| 02 | 35 | 67 | |
| |||
| LOS, d, n=293 | |||
| Median | 4 | 5 | 3 |
| IQR | 5 | 4 | 3 |
| Mean | 4.7 (2.72) | 4.7 (2.52) | 4.1 (2.51), P=0.12a |
| ULOS, d, n=293 | |||
| Median | 0 | 0 | 0 |
| IQR | 0 | 0 | 0 |
| Mean | 0.37 (0.99) | 0.33 (0.96) | 0.13 (0.56), P=0.09a |
| Complications (per patient per day), n=398 | |||
| Median | 0 | 0 | 0 |
| IQR | 1 | 1 | 0 |
| Mean | 0.58 (1.06) | 0.45 (0.77) | 0.18 (0.59), P=0.001 compared to teams with 02 or 35 characteristics |
Our analyses of LOS and ULOS included 298 of the 576 patients. Two hundred sixty‐seven patients were excluded because their entire LOS did not occur while under the care of the observed teams. Eleven patients were removed from the analysis because their LOS was >12 days. The analysis of complications included 398 patients. In our preliminary general linear modeling approach, only patient workload was significantly associated with outcomes using a cutoff of P=0.05. Charlson‐Deyo score and mental health comorbidities were not associated with outcomes.
The results of the Kruskal‐Wallis test show the patient average ranking on each of the outcome variables by 3 groups (Table 3). Overall, teams with higher relationship scores had lower rank scores on all outcomes measures. However, the only statistically significant comparisons were for complications. Teams having 6 to 7 characteristics had a significantly lower complication rate ranking than teams with 0 to 2 and 3 to 5 (P=0.001). We did not find consistent differences between individual teams or groups of teams with relationship scores from 0 to 2, 3 to 5, and 6 to 7 with regard to Charlson score, mental health issues, or workload. The only significant differences were between Charlson‐Deyo scores for patients admitted to teams with low relationship scores of 0 to 2 versus high relationship scores of 6 to 7 (6.7 vs 5.1); scores for teams with relationship scores of 3 to 5 were not significantly different from the low or high groups.
Table 4 shows the Kruskal‐Wallace rank test results for each group of relationship characteristics identified in the factor analysis based on whether teams displayed all or none of the characteristics in the factor. There were no differences in these groupings for LOS. Teams that exhibited both mindfulness and trust had lower ranks on ULOS than teams that did not have either. Similarly, teams with heedfulness, social‐task relatedness, and more rich communication demonstrated lower ULOS rankings than teams who did not have all 3 characteristics.
| Mind/Trust | Diversity/Respect | Heed/Relate/Communicate | ||||
|---|---|---|---|---|---|---|
| Patient Outcome | None | Both | None | Both | None | All 3 |
| ||||||
| LOS, d, n=293 | ||||||
| Median | 4 | 4 | 4 | 4 | 4 | 4 |
| IQR | 5 | 3 | 4.5 | 3 | 4 | 4 |
| Mean | 4.7 (2.6) | 4.2 (2.5) | 4.7 (2.6) | 4.3 (2.5) | 4.4 (2.6) | 4.4 (2.6) |
| P value | 0.06a | 0.23a | 0.85a | |||
| ULOS, d, n=293 | ||||||
| Median | 0 | 0 | 0 | 0 | 0 | 0 |
| IQR | 0 | 0 | 0 | 0 | 0 | 0 |
| Mean | 0.39 (1.01) | 0.15 (0.62) | 0.33 (0.92) | 0.18 (0.71) | 0.32 (0.93) | 0.18 (0.69) |
| P value | 0.009 | 0.06 | 0.03 | |||
| Complications (per patient), n=389 | ||||||
| Median | 0 | 0 | 0 | 0 | 0 | 0 |
| IQR | 1 | 0 | 1 | 0 | 1 | 0 |
| Mean | 0.58 (1.01) | 0.19 (0.58) | 0.47 (0.81) | 0.29 (0.82) | 0.26 (0.92) | 0.28 (0.70) |
| P value | <0.0001 | 0.001 | 0.02 | |||
DISCUSSION
Relationships are critical to team function because they are the basis for the social interactions that are central to patient care. These interactions include how providers recognize and make sense of what is happening with patients, and how they learn to care for patients more effectively. Additionally, the high task interdependencies among inpatient providers require effective relationships for optimal care. In our study, inpatient medicine physician teams' relationships varied, and these differences were associated with ULOS and complications. Relationship characteristics are not mutually exclusive, and as our factor analysis demonstrates, are intercorrelated. Trust and mindfulness appear to be particularly important. Trust may foster psychological safety that in turn promotes the willingness of individuals to contribute their thoughts and ideas.[13] In low‐trust teams, providers may fear a negative impact for bringing forward a concern based on limited data. Mindful teams may be more likely to notice nuanced changes, or are more likely to talk when things just do not appear to be going in the right direction with the patient. In the case of acutely ill medical patients, trust and mindfulness may lead to an increased likelihood that clinical changes are recognized and discussed quickly. For example, on a team characterized by trust and mindfulness, the entire team was typically involved in care discussions, and the interns and students frequently asked a lot of questions, even regarding the care of patients they were not directly following. We observed that these questions and discussions often led the team to realize that they needed to make a change in management decisions (eg, discontinuing Bactrim, lowering insulin doses, adjusting antihypertensives, premedicating for intravenous contrast) that they had not caught in the assessment and plan portion of the patient care discussion. In another example, a medical student asked a tentative question after a patient needed to go quickly to the bathroom while they were examining her, leading the team to ask more questions that led to a more rapid evaluation of a potential urinary tract infection. This finding is consistent with the description of failure to rescue among surgical patients, in which mortality has been associated with the failure to recognize complications rapidly and act effectively.[33]
Our findings are limited in several ways. First, these data are from a single academic institution. Although we sought diversity among our teams and collected data across 2 hospitals, there may be local contextual factors that influenced our results. Second, our data demonstrate an association, but not causality. Our findings should be tested in studies that assess causality and potential mechanisms through which relationships influence outcomes. Third, the individuals observing the teams had some knowledge of patient outcomes through hearing patient discussions. However, by involving individuals who did not participate in observations and were blinded to outcomes in assessing team relationships, we addressed this potential bias. Fourth, our observations were largely focused on physician teams, not directly including other providers. Our difficulty in observing regular interactions between physicians and other providers underscores the need to increase contact among those caring for hospitalized patients, such as occurs through multidisciplinary rounds. We did include team communication with other disciplines in our assessment of the relationship characteristics of diversity and rich communication. Finally, our analysis was limited by our sample size. We observed a relatively small number of teams. Although we benefitted from seeing the change in team relationships that occurred with attending changes halfway through some of our data collection months, this did limit the number of patients we could include in our analyses. Though we did not observe obvious differences in relationships between the teams observed across the 2 hospitals, the small number of teams and hospitals precluded our ability to perform multilevel modeling analyses, which would have allowed us to assess or account for the influence of team or organizational factors. However, this small sample size did allow for a richer assessment of team behaviors.
Although preliminary, our findings are an important step in understanding the function of inpatient medical teams not only in terms of processes of care, but also in terms of relationships. Patient care is a social activity, requiring effective communication to develop working diagnoses, recognize changes in patients' clinical courses, and formulate effective treatment plans during and after hospitalization. Future work could follow several directions. One would be to assess the causal mechanisms through which relationships influence patient outcomes. These may include sensemaking, learning, and improved coordination. Positive relationships may facilitate interaction of tacit and explicit information, facilitating the creation of understandings that foster more effective patient care.[34] The dynamic nature of relationships and how patient outcomes in turn feed back into relationships could be an area of exploration. This line of research could build on the idea of teaming.[35] Understanding relationships across multidisciplinary teams or with patients and families would be another direction. Finally, our results could point to potential interventions to improve patient outcomes through improving relationships. Better understanding of the nature of effective relationships among providers should enable us to develop more effective strategies to improve the care of hospitalized patients. In the larger context of payment reforms that require greater coordination and communication among and across providers, a greater understanding of how relationships influence patient outcomes will be important.
Acknowledgements
The authors thank the physicians involved in this study and Ms. Shannon Provost for her involvement in discussions of this work.
Disclosures: The research reported herein was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (CDA 07‐022). Investigator salary support was provided through this funding, and through the South Texas Veterans Health Care System. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs. Dr. McDaniel receives support from the IC[2] Institute of the University of Texas at Austin. Dr. Luci Leykum had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. The authors report no conflicts of interest.
- . Redesigning health care with insights from the science of complex adaptive systems. In: Crossing the Quality Chasm: A New Heath System for the 21st Century. Washington, DC: National Academy of Sciences; 2000:309–322.
- , , , , , . Temporal trends in rates of patient harm resulting from medical care. N Engl J Med. 2010;323(22):2124–2135.
- , , , et al. Circumstances of patient falls and injuries in 9 hospitals in a mid‐western healthcare system. Infect Control Hosp Epidemiol. 2007;28(5):544–550.
- , . Point prevalence of wounds in a sample of acute hospitals in Canada. Int Wound J. 2009;6(4):287–293.
- , , , , , . Non‐adherence to guidelines: an avoidable cause of failure of empirical antimicrobial therapy in the presence of difficult‐to‐treat bacteria. Intensive Care Med. 2010;36(1):75–82.
- , , , , . Quality of care in U.S. hospitals as reflected by standardized measures, 2002–2004. N Engl J Med. 2005;353(3):255–264.
- Centers for Disease Control and Prevention. National Center for Emerging and Zoonotic Infectious Diseases. Division of Healthcare Quality Promotion. Checklist for prevention of central line associated blood stream infections. Available at: http://www.cdc.gov/HAI/pdfs/bsi/checklist‐for‐CLABSI.pdf. Accessed August 3, 2014.
- Safer Healthcare Partners, LLC. Checklists: a critical patient safety tool. Available at: http://www.saferhealthcare.com/high‐reliability‐topics/checklists. Accessed July 31, 2014.
- . Making Things Work: Solving Complex Problems in a Complex World. Boston, MA: Knowledge Press; 2004:117–160.
- . High Performance Healthcare: Using The Power of Relationships to Achieve Quality, Efficiency, and Resilience. 1st ed. New York, NY: McGraw‐Hill; 2009.
- , . Design of high reliability organizations in health care. Qual Saf Health Care. 2006;15(suppl 1):i4–i9.
- , , , . Does team training work? Principles for health care. Acad Emerg Med. 2008;15(11):1002–1009.
- . Speaking up in the operating room: how team leaders promote learning in interdisciplinary action teams. J Manag Stud. 2003;40(6):1419–1452.
- , , , et al. Association between implementation of a medical team training program and surgical mortality. JAMA. 2010;304(15):1693–1700.
- , , , . Group cognition, membership change, and performance: Investigating the benefits and detriments of collective knowledge. Organ Behav Hum Decis Process. 2007;103(2):159–178.
- , , , , , . Reciprocal learning and chronic care model implementation in primary care: results from a new scale of learning in primary care settings. BMC Health Serv Res. 2011;11:44.
- , , , , . The importance of relational coordination and reciprocal learning for chronic illness care within primary care teams. Health Care Manage Rev. 2012;38(1):20–28.
- , , , , . Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q. 2011;89(2):167–205.
- , , , et al. How improving practice relationships among clinicians and nonclinicians can improve quality in primary care. Jt Comm J Qual Patient Saf. 2009;35(9):457–466.
- , , , et al. Relationship quality and patient‐assessed quality of care in VA primary care clinics: development and validation of the work relationships scale. Ann Fam Med. 2013;11(6):543–549.
- , . Designing and Conducting Mixed Methods Research. 2nd ed. Thousand Oaks, CA: Sage; 2011.
- . Qualitative Evaluation Methods. Thousand Oaks, CA: Sage; 2002.
- , , . Qualitative methods in research on health care quality. Qual Saf Health Care. 2002;11:148–152.
- . Managing the negatives of experience in physician teams. Health Care Manage Rev. 2010;35(1):65–76.
- , , , , . Rethinking resident supervision to improve safety: from hierarchical to interprofessional models. J Hosp Med. 2011;6(8):445 b452.
- , , , . Dynamic delegation: shared, hierarchical, and deindividualized leadership in extreme action teams. Adm Sci Q. 2006;51(4):590–621.
- , , . Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases. J Clin Epidemiol. 1992;45(6):613–619.
- . Exploratory Data Analysis. Reading, MA: Addison‐Wesley; 1977.
- . Biostatistical Analysis. 4th ed. Upper Saddle River, NJ: Pearson Prentice‐Hall; 2010.
- . Multiple contrasts using rank sums. Technometrics. 1964;6:241–252.
- , . A SAS macro implementation of a multiple comparison post hoc test for a Kruskal–Wallis analysis. Comput Methods Programs Biomed. 2011;102:75–80.
- SAS/STAT Software [computer program]. Version 9.1. Cary, NC: SAS Institute Inc.; 2003.
- , , . Complications, failure to rescue, and mortality with major inpatient surgery in Medicare patients. Ann Surg. 2009;250(6):1029–1034.
- . A dynamic theory of organizational knowledge creation. Org Sci. 1994;5(1):14–37.
- . Teaming: How Organizations Learn, Innovate, and Compete in the Knowledge Economy. 1st ed. Boston, MA: Harvard Business School; 2012.
Since the Institute of Medicine Report To Err is Human, increased attention has been paid to improving the care of hospitalized patients.[1] Strategies include utilization of guidelines and pathways, and the application of quality improvement techniques to improve or standardize processes. Despite improvements in focused areas such as prevention of hospital‐acquired infections, evidence suggests that outcomes for hospitalized patients remain suboptimal.[2] Rates of errors and hospital‐related complications such as falls, decubitus ulcers, and infections remain high,[3, 4, 5] and not all patients receive what is known to be appropriate care.[6]
Many attempts to improve inpatient care have used process‐improvement approaches, focusing on impacting individuals' behaviors, or on breaking down processes into component parts. Examples include central line bundles or checklists.[7, 8] These approaches attempt to ensure that providers do things in a standardized way, but are implicitly based on the reductionist assumption that we can break processes down into predictable parts to improve the system. An alternative way to understand clinical systems is based on interdependencies between individuals in the system, or the ways in which parts of the system interact with each other, which may be unpredictable over time.[1, 9] Whereas these interdependencies include care processes, they also encompass the providers who care for patients. Providers working together vary in terms of the kinds of relationships they have with each other. Those relationships are crucial to system function because they are the foundation for the interactions that lead to effective patient care.
The application of several frameworks or approaches for considering healthcare systems in terms of relationships highlights the importance of this way of understanding system function. The include complexity science,[1, 7] relational coordination (which is grounded in complexity science),[10] high reliability,[11] and the Big Five for teamwork.[12]
Research indicates that interactions among healthcare providers can have important influences on outcomes.[13, 14, 15, 16, 17] Additionally, the initial implementation of checklists to prevent central‐line associated infections appeared to change provider relationships in a way that significantly influenced their success.[18] For example, positive primary care clinic member relationships as assessed by the Lanham framework have been associated with better chronic care model implementation, learning, and patient experience of care.[19, 20] This framework, which we apply here, identifies 7 relationship characteristics: (1) trust; (2) diversity; (3) respect; (4) mindfulness, or being open to new ideas from others; (5) heedfulness, or an understanding of how one's roles influence those of others; (6) use of rich in‐person or verbal communication, particularly for potentially ambiguous information open to multiple interpretations; and (7) having a mixture of social and task relatedness among teams, or relatedness outside of only work‐related tasks.[19] Relationships within surgical teams that are characterized by psychological safety and diversity are associated with successful uptake of new techniques and decreased mortality.[13, 14] Relationships are important because the ability of patients and providers to learn and make sense of their patients' illnesses is grounded in relationships.
We sought to better understand and characterize inpatient physician teams' relationships, and assess the association between team relationships as evaluated by Lanham's framework and outcomes for hospitalized patients. Data on relationships among inpatient medical teams are few, despite the fact that these teams provide a great proportion of inpatient care. Additionally, the care of hospitalized medical patients is complex and uncertain, often involving multiple providers, making provider relationships potentially even more important to outcomes than in other settings.
METHODS
Overview
We conducted an observational, convergent mixed‐methods study of inpatient medicine teams.[21, 22, 23] We focused on inpatient physician teams, defining them as the functional work group responsible for medical decision making in academic medical centers. Physician teams in this context have been studied in terms of social hierarchy, authority, and delegation.[24, 25, 26] Focusing on the relationships within these groups could provide insights into strategies to mitigate potential negative effects of hierarchy. We recognize that other providers are closely involved in the care of hospitalized patients, and although we did not have standard interactions between physicians, nurses, case managers, and other providers that we could consistently observe, we did include interactions with these other providers in our observations and assessments of team relationships. Because this work is among the first in inpatient medical teams, we chose to study a small number of teams in great depth, allowing us to make rich assessments of team relationships.
We chose patient outcomes of length of stay (LOS), unnecessary LOS (ULOS), and complication rates, adjusted for patient characteristics and team workload. LOS is an important metric of inpatient care delivery. We feel ULOS is an aspect of LOS that is dependent on the physician team, as it reflects their preparation of the patient for discharge. Finally, we chose complication rates because hospital‐acquired conditions and complications are important contributors to inpatient morbidity, and because recent surgical literature has identified complication rates as a contributor to mortality that could be related to providers' collective ability to recognize complications and act quickly.
This study was approved by the institutional review board at the University of Texas Health Science Center at San Antonio (UTHSCSA), the Research and Development Committee for the South Texas Veterans Health Care System (STVHCS), and the Research Committee at University Health System (UHS). All physicians consented to participate in the study. We obtained a waiver of consent for inclusion of patient data.
Setting and Study Participants
This study was conducted at the 2 UTHSCSA primary teaching affiliates. The Audie L. Murphy Veterans Affairs Hospital is the 220‐bed acute‐care hospital of the STVHCS. University Hospital is the 614‐bed, level‐I trauma, acute‐care facility for UHS, the county system for Bexar County, which includes the San Antonio, Texas major metropolitan area.
The inpatient internal medicine physician team was our unit of study. Inpatient medicine teams consisted of 1 faculty attending physician, 1 postgraduate year (PGY)‐2 or PGY‐3 resident, and 2 PGY‐1 members. In addition, typically 2 to 3 third‐year medical students were part of the team, and a subintern was sometimes present. Doctor of Pharmacy faculty and students were also occasionally part of the team. Social workers and case managers often joined team rounds for portions of the time, and nurses sometimes joined bedside rounds on specific patients. These teams admit all medicine patients with the exception of those with acute coronary syndromes, new onset congestive heart failure, or arrhythmias. Patients are randomly assigned to teams based on time of admission and call schedules.
Between these 2 hospitals, there are 10 inpatient medicine teams caring for patients, with a pool of over 40 potential faculty attendings. Our goal was to observe teams that would be most likely to vary in terms of their relationship characteristics and patient outcomes through observing teams with a range of individual members. We used a purposeful sampling approach to obtain a diverse sample, sampling based on physician attributes and time of year.[16, 17] Three characteristics were most important: attending physician years of experience, attending involvement in educational and administrative leadership, and the presence of struggling resident members, as defined by being on probation or having been discussed in the residency Clinical Competency Committee. We did not set explicit thresholds in terms of attending experience, but instead sought to ensure a range. The attendings we observed were more likely to be involved in education and administrative leadership activities, but were otherwise similar to those we did not observe in terms of years of experience. We included struggling residents to observe individuals with a range of skill sets, and not just high‐performing individuals. We obtained attending information based on our knowledge of the attending faculty pool, and from the internal medicine residency program. We sampled across the year to ensure a diversity of trainee experience, but did not observe teams in either July or August, as these months were early in the academic year. Interns spend approximately 5 months per year on inpatient services, whereas residents spend 2 to 3 months per year. Thus, interns but not residents observed later in the year might have spent significantly more time on an inpatient service. However, in all instances, none of the team members observed had worked together previously.
Data Collection
Data were collected over nine 1‐month periods from September 2008 through June 2011. Teams were observed daily for 2‐ to 4‐week periods during morning rounds, the time when the team discusses each patient and makes clinical decisions. Data collection started on the first day of the month, the first day that all team members worked together, and continued for approximately 27 days, the last day before the resident rotated to a different service. By comprehensively and systematically observing these teams' daily rounds, we obtained rich, in‐depth data with multiple data points, enabling us to assess specific team behaviors and interactions.
During the third and fourth months, we collected data on teams in which the attending changed partway through. We did this to understand the impact of individual attending change on team relationships. Because the team relationships differed with each attending, we analyzed them separately. Thus, we observed 7 teams for approximately 4‐week periods and 4 teams for approximately 2‐week periods.
Observers arrived in the team room prior to rounds to begin observations, staying until after rounds were completed. Detailed free‐text field notes were taken regarding team activities and behaviors, including how the teams made patient care decisions. Field notes included: length of rounds, which team members spoke during each patient discussion, who contributed to management discussions, how information from consultants was incorporated, how communication with others outside of the team occurred, how team members spoke with each other including the types of words used, and team member willingness to perform tasks outside of their usually defined role, among others. Field notes were collected in an open‐ended format to allow for inductive observations. Observers also recorded clinical data daily regarding each patient, including admission and discharge dates, and presenting complaint.
The observation team consisted of the principle investigator (PI) (hospitalist) and 2 research assistants (a graduate‐level medical anthropologist and social psychologist), all of whom were trained by a qualitative research expert to systematically collect data related to topics of interest. Observers were instructed to record what the teams were doing and talking about at all times, noting any behaviors that they felt reflected how team members related to each other and came to decisions about their patients, or that were characteristic of the team. To ensure consistency, the PI and 1 research assistant conducted observations jointly at the start of data collection for each team, checking concordance of observations daily using a percent agreement until general agreement on field note content and patient information reached 90%. Two individuals observed 24 days of data collection, representing 252 patient discussions (13% of observed discussions).
An age‐adjusted Charlson‐Deyo comorbidity score was calculated for each patient admitted to each team, using data from rounds and from each hospital's electronic health records (EHR).[27] We collected data regarding mental health conditions for each patient (substance use, mood disorder, cognitive disorder, or a combination) because these comorbidities could impact LOS or ULOS. Discharge diagnoses were based on the discharge summary in the EHR. We also collected data daily regarding team census and numbers of admissions to and discharges from each team to assess workload.
Three patient outcomes were measured: LOS, ULOS, and complications. LOS was defined as the total number of days the patient was in the hospital. ULOS was defined as the number of days a patient remained in the hospital after the day the team determined the patient was medically ready for discharge (assessed by either discussion on rounds or EHR documentation). ULOS may occur when postdischarge needs have been adequately assessed, or because of delays in care, which may be related to provider communication during the hospitalization. Complications were defined on a per‐patient, per‐day basis in 2 ways: the development of a new problem in the hospital, for example acute kidney injury, a hospital‐acquired infection, or delirium, or by the team noting a clinical deterioration after at least 24 hours of clinical stability, such as the patient requiring transfer to a higher level of care. Complications were determined based on discussions during rounds, with EHR verification if needed.
Analysis Phase I: Assessment of Relationship Characteristics
After the completion of data collection, field notes were reviewed by a research team member not involved in the original study design or primary data collection (senior medical student). We took this approach to guard against biasing the reviewer's view of team behaviors, both in terms of not having conducted observations of the teams and being blinded to patient outcomes.
The reviewer completed a series of 3 readings of all field notes. The first reading provided a summary of the content of the data and the individual teams. Behavioral patterns of each team were used to create an initial team profile. The field notes and profiles were reviewed by the PI and a coauthor not involved in data collection to ensure that the profiles adequately reflected the field notes. No significant changes to the profiles were made based on this review. The profiles were discussed at a meeting with members of the larger research team, including the PI, research assistants, and coinvestigators (with backgrounds in medicine, anthropology, and information and organization management). Behavior characteristics that could be used to distinguish teams were identified in the profiles using a grounded theory approach.
The second review of field notes was conducted to test the applicability of the characteristics identified in the first review. To systematically record the appearance of the behaviors, we created a matrix with a row for each behavior and columns for each team to note whether they exhibited each behavior. If the behavior was exhibited, specific examples were cataloged in the matrix. This matrix was reviewed and refined by the research team. During the final field note review meeting, the research team compared the summary matrix for each team, with the specific behaviors noted during the first reading of the field notes to ensure that all behaviors were recorded.
After cataloging behaviors, the research team assigned each behavior to 1 of the 7 Lanham relationship characteristics. We wanted to assess our observations against a relationship framework to ensure that we were able to systematically assess all aspects of relationships. The Lanham framework was initially developed based on a systematic review of the organizational and educational literatures, making it relevant to the complex environment of an academic medical inpatient team and allowing us to assess relationships at a fine‐grained, richly detailed level. This assignment was done by the author team as a group. Any questions were discussed and different interpretations resolved through consensus. The Lanham framework has 7 characteristics.[19] Based on the presence of behaviors associated with each relationship characteristic, we assigned a point to each team for each relationship characteristic observed. We considered a behavior type to be present if we observed it on at least 3 occasions on separate days. Though we used a threshold of at least 3 occurrences, most teams that did not receive a point for a particular characteristic did not have any instances in which we observed the characteristic. This was particularly true for trust and mindfulness, and least so for social/task relatedness. By summing these points, we calculated a total relationship score for each team, with potential scores ranging from 0 (for teams exhibiting no behaviors reflecting a particular relationship characteristic) to 7.
Analysis Phase II: Factor Analysis
To formally determine which relationship characteristics were most highly related, data were submitted to a principal components factor analysis using oblique rotation. Item separation was determined by visual inspection of the scree plot and eigenvalues over 1.
Analysis Phase III: Assessing the Association between Physician Team Relationship Characteristics and Patient Outcomes
We examined the association between team relationships and patient outcomes using team relationship scores. For the LOS/ULOS analysis, we only included patients whose entire hospitalization occurred under the care of the team we observed. Patients who were on the team at the start of the month, were transferred from another service, or who remained hospitalized after the end of the team's time together were excluded. The longest possible LOS for patients whose entire hospitalization occurred on teams that were observed for half a month was 12 days. To facilitate accurate comparison between teams, we only included patients whose LOS was 12 days.
Complication rates were defined on a per‐patient per‐day basis to normalize for different team volumes and days of observation. For this analysis, we included patients who remained on the team after data collection completion, patients transferred to another team, or patients transferred from another team. However, we only counted complications that occurred at least 24 hours following transfer to minimize the likelihood that the complication was related to the care of other physicians.
Preliminary analysis involved inspection and assessment of the distribution of all variables followed, by a general linear modeling approach to assess the association between patient and workload covariates and outcomes.[28, 29] Because we anticipated that outcome variables would be markedly skewed, we also planned to assess the association between relationship characteristics with outcomes using the Kruskal‐Wallis rank sum test to compare groups with Dunn's test[30] for pairwise comparisons if overall significance occurred.[31] There are no known acceptable methods for covariate adjustments using the Kruskal‐Wallis method. All models were run using SAS software (SAS Institute Inc., Cary, NC).[32]
RESULTS
The research team observed 1941 discussions of 576 individual patients. Observations were conducted over 352 hours and 54 minutes, resulting in 741 pages of notes (see Supporting Table 1 in the online version of this article for data regarding individual team members). Teams observed over half‐months are referred to with a and b designations.
| Relationship Characteristic | Definition | Thirteen Types of Behaviors Observed in Field Notes | Observed Examples |
|---|---|---|---|
| Trust | Willingness to be vulnerable to others | Use of we instead of you or I by the attending | Where are we going with this guy? |
| Attending admitting I don't know | Let's go talk to him, I can't figure this out | ||
| Asking questions to help team members to think through problems | Will the echo change our management? How will it help us? | ||
| Diversity | Including different perspectives and different thinking | Team member participation in conversations about patients that are not theirs | One intern is presenting, another intern asks a question, and the resident joins the discussion |
| Inclusion of perspectives of those outside the team (nursing and family members) | Taking a break to call the nurse, having a family meeting | ||
| Respect | Valuing the opinions of others, honest and tactful interactions | Use of positive reinforcement by the attending | Being encouraging of the medical student's differential, saying excellent |
| How the team talks with patients | Asking if the patient has any concerns, what they can do to make them comfortable | ||
| Heedfulness | Awareness of how each person's roles impact the rest of the team | Team members performing tasks not expected of their role | One intern helping another with changing orders to transfer a patient |
| Summarizing plans and strategizing | Attending recaps the plan for the day, asks what they can do | ||
| Mindfulness | Openness to new ideas/free discussion about what is and is not working | Entire team engaged in discussion | Attending asks the medical student, intern, and resident what they think is going on |
| Social relatedness | Having socially related interactions | Social conversation among team members | Intern talks about their day off |
| Jokes by the attending | Showers and a bowel movement is the key to making people happy | ||
| Appropriate use of rich communication | Use of in‐person communication for sensitive or difficult issues | Using verbal communication with consultants or family | Intern is on the phone with the pharm D because there is a problem with the medication |
Creation of team profiles yielded 13 common behavior characteristics that were inductively identified and that could potentially distinguish teams, including consideration of perspectives outside of the team and team members performing tasks normally outside of their roles. Table 1 provides examples of and summarizes observed behaviors using examples from the field notes, mapping these behavior characteristics onto the Lanham relationship characteristics. The distribution of relationship characteristics and scores for each team are shown in Table 2.
| Relationship Characteristic | Team | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3a | 3b | 4a | 4b | 5 | 6 | 7 | 8 | 9 | |
| Trust | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 1 |
| Diversity | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
| Respect | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 1 |
| Heedfulness | 0 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 1 | 1 |
| Mindfulness | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 1 | 1 |
| Social/task relatedness | 0 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 1 |
| Rich/lean communication | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 0 |
| Relationship score (no. of characteristics observed) | 0 | 5 | 7 | 2 | 2 | 3 | 5 | 0 | 7 | 7 | 6 |
Correlation between relationship characteristics ranged from 0.32 to 0.95 (see Supporting Table 2 in the online version of this article). Mindfulness and trust are more highly correlated with each other than with other variables, as are diversity and respect. We performed a principal components factor analysis. Based on scree plot inspection and eigenvalues >1, we kept 3 factors that explained 85% of the total variance (see Supporting Table 3 in the online version of this article).
| No. of Relationship Characteristics | |||
|---|---|---|---|
| 02 | 35 | 67 | |
| |||
| LOS, d, n=293 | |||
| Median | 4 | 5 | 3 |
| IQR | 5 | 4 | 3 |
| Mean | 4.7 (2.72) | 4.7 (2.52) | 4.1 (2.51), P=0.12a |
| ULOS, d, n=293 | |||
| Median | 0 | 0 | 0 |
| IQR | 0 | 0 | 0 |
| Mean | 0.37 (0.99) | 0.33 (0.96) | 0.13 (0.56), P=0.09a |
| Complications (per patient per day), n=398 | |||
| Median | 0 | 0 | 0 |
| IQR | 1 | 1 | 0 |
| Mean | 0.58 (1.06) | 0.45 (0.77) | 0.18 (0.59), P=0.001 compared to teams with 02 or 35 characteristics |
Our analyses of LOS and ULOS included 298 of the 576 patients. Two hundred sixty‐seven patients were excluded because their entire LOS did not occur while under the care of the observed teams. Eleven patients were removed from the analysis because their LOS was >12 days. The analysis of complications included 398 patients. In our preliminary general linear modeling approach, only patient workload was significantly associated with outcomes using a cutoff of P=0.05. Charlson‐Deyo score and mental health comorbidities were not associated with outcomes.
The results of the Kruskal‐Wallis test show the patient average ranking on each of the outcome variables by 3 groups (Table 3). Overall, teams with higher relationship scores had lower rank scores on all outcomes measures. However, the only statistically significant comparisons were for complications. Teams having 6 to 7 characteristics had a significantly lower complication rate ranking than teams with 0 to 2 and 3 to 5 (P=0.001). We did not find consistent differences between individual teams or groups of teams with relationship scores from 0 to 2, 3 to 5, and 6 to 7 with regard to Charlson score, mental health issues, or workload. The only significant differences were between Charlson‐Deyo scores for patients admitted to teams with low relationship scores of 0 to 2 versus high relationship scores of 6 to 7 (6.7 vs 5.1); scores for teams with relationship scores of 3 to 5 were not significantly different from the low or high groups.
Table 4 shows the Kruskal‐Wallace rank test results for each group of relationship characteristics identified in the factor analysis based on whether teams displayed all or none of the characteristics in the factor. There were no differences in these groupings for LOS. Teams that exhibited both mindfulness and trust had lower ranks on ULOS than teams that did not have either. Similarly, teams with heedfulness, social‐task relatedness, and more rich communication demonstrated lower ULOS rankings than teams who did not have all 3 characteristics.
| Mind/Trust | Diversity/Respect | Heed/Relate/Communicate | ||||
|---|---|---|---|---|---|---|
| Patient Outcome | None | Both | None | Both | None | All 3 |
| ||||||
| LOS, d, n=293 | ||||||
| Median | 4 | 4 | 4 | 4 | 4 | 4 |
| IQR | 5 | 3 | 4.5 | 3 | 4 | 4 |
| Mean | 4.7 (2.6) | 4.2 (2.5) | 4.7 (2.6) | 4.3 (2.5) | 4.4 (2.6) | 4.4 (2.6) |
| P value | 0.06a | 0.23a | 0.85a | |||
| ULOS, d, n=293 | ||||||
| Median | 0 | 0 | 0 | 0 | 0 | 0 |
| IQR | 0 | 0 | 0 | 0 | 0 | 0 |
| Mean | 0.39 (1.01) | 0.15 (0.62) | 0.33 (0.92) | 0.18 (0.71) | 0.32 (0.93) | 0.18 (0.69) |
| P value | 0.009 | 0.06 | 0.03 | |||
| Complications (per patient), n=389 | ||||||
| Median | 0 | 0 | 0 | 0 | 0 | 0 |
| IQR | 1 | 0 | 1 | 0 | 1 | 0 |
| Mean | 0.58 (1.01) | 0.19 (0.58) | 0.47 (0.81) | 0.29 (0.82) | 0.26 (0.92) | 0.28 (0.70) |
| P value | <0.0001 | 0.001 | 0.02 | |||
DISCUSSION
Relationships are critical to team function because they are the basis for the social interactions that are central to patient care. These interactions include how providers recognize and make sense of what is happening with patients, and how they learn to care for patients more effectively. Additionally, the high task interdependencies among inpatient providers require effective relationships for optimal care. In our study, inpatient medicine physician teams' relationships varied, and these differences were associated with ULOS and complications. Relationship characteristics are not mutually exclusive, and as our factor analysis demonstrates, are intercorrelated. Trust and mindfulness appear to be particularly important. Trust may foster psychological safety that in turn promotes the willingness of individuals to contribute their thoughts and ideas.[13] In low‐trust teams, providers may fear a negative impact for bringing forward a concern based on limited data. Mindful teams may be more likely to notice nuanced changes, or are more likely to talk when things just do not appear to be going in the right direction with the patient. In the case of acutely ill medical patients, trust and mindfulness may lead to an increased likelihood that clinical changes are recognized and discussed quickly. For example, on a team characterized by trust and mindfulness, the entire team was typically involved in care discussions, and the interns and students frequently asked a lot of questions, even regarding the care of patients they were not directly following. We observed that these questions and discussions often led the team to realize that they needed to make a change in management decisions (eg, discontinuing Bactrim, lowering insulin doses, adjusting antihypertensives, premedicating for intravenous contrast) that they had not caught in the assessment and plan portion of the patient care discussion. In another example, a medical student asked a tentative question after a patient needed to go quickly to the bathroom while they were examining her, leading the team to ask more questions that led to a more rapid evaluation of a potential urinary tract infection. This finding is consistent with the description of failure to rescue among surgical patients, in which mortality has been associated with the failure to recognize complications rapidly and act effectively.[33]
Our findings are limited in several ways. First, these data are from a single academic institution. Although we sought diversity among our teams and collected data across 2 hospitals, there may be local contextual factors that influenced our results. Second, our data demonstrate an association, but not causality. Our findings should be tested in studies that assess causality and potential mechanisms through which relationships influence outcomes. Third, the individuals observing the teams had some knowledge of patient outcomes through hearing patient discussions. However, by involving individuals who did not participate in observations and were blinded to outcomes in assessing team relationships, we addressed this potential bias. Fourth, our observations were largely focused on physician teams, not directly including other providers. Our difficulty in observing regular interactions between physicians and other providers underscores the need to increase contact among those caring for hospitalized patients, such as occurs through multidisciplinary rounds. We did include team communication with other disciplines in our assessment of the relationship characteristics of diversity and rich communication. Finally, our analysis was limited by our sample size. We observed a relatively small number of teams. Although we benefitted from seeing the change in team relationships that occurred with attending changes halfway through some of our data collection months, this did limit the number of patients we could include in our analyses. Though we did not observe obvious differences in relationships between the teams observed across the 2 hospitals, the small number of teams and hospitals precluded our ability to perform multilevel modeling analyses, which would have allowed us to assess or account for the influence of team or organizational factors. However, this small sample size did allow for a richer assessment of team behaviors.
Although preliminary, our findings are an important step in understanding the function of inpatient medical teams not only in terms of processes of care, but also in terms of relationships. Patient care is a social activity, requiring effective communication to develop working diagnoses, recognize changes in patients' clinical courses, and formulate effective treatment plans during and after hospitalization. Future work could follow several directions. One would be to assess the causal mechanisms through which relationships influence patient outcomes. These may include sensemaking, learning, and improved coordination. Positive relationships may facilitate interaction of tacit and explicit information, facilitating the creation of understandings that foster more effective patient care.[34] The dynamic nature of relationships and how patient outcomes in turn feed back into relationships could be an area of exploration. This line of research could build on the idea of teaming.[35] Understanding relationships across multidisciplinary teams or with patients and families would be another direction. Finally, our results could point to potential interventions to improve patient outcomes through improving relationships. Better understanding of the nature of effective relationships among providers should enable us to develop more effective strategies to improve the care of hospitalized patients. In the larger context of payment reforms that require greater coordination and communication among and across providers, a greater understanding of how relationships influence patient outcomes will be important.
Acknowledgements
The authors thank the physicians involved in this study and Ms. Shannon Provost for her involvement in discussions of this work.
Disclosures: The research reported herein was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (CDA 07‐022). Investigator salary support was provided through this funding, and through the South Texas Veterans Health Care System. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs. Dr. McDaniel receives support from the IC[2] Institute of the University of Texas at Austin. Dr. Luci Leykum had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. The authors report no conflicts of interest.
Since the Institute of Medicine Report To Err is Human, increased attention has been paid to improving the care of hospitalized patients.[1] Strategies include utilization of guidelines and pathways, and the application of quality improvement techniques to improve or standardize processes. Despite improvements in focused areas such as prevention of hospital‐acquired infections, evidence suggests that outcomes for hospitalized patients remain suboptimal.[2] Rates of errors and hospital‐related complications such as falls, decubitus ulcers, and infections remain high,[3, 4, 5] and not all patients receive what is known to be appropriate care.[6]
Many attempts to improve inpatient care have used process‐improvement approaches, focusing on impacting individuals' behaviors, or on breaking down processes into component parts. Examples include central line bundles or checklists.[7, 8] These approaches attempt to ensure that providers do things in a standardized way, but are implicitly based on the reductionist assumption that we can break processes down into predictable parts to improve the system. An alternative way to understand clinical systems is based on interdependencies between individuals in the system, or the ways in which parts of the system interact with each other, which may be unpredictable over time.[1, 9] Whereas these interdependencies include care processes, they also encompass the providers who care for patients. Providers working together vary in terms of the kinds of relationships they have with each other. Those relationships are crucial to system function because they are the foundation for the interactions that lead to effective patient care.
The application of several frameworks or approaches for considering healthcare systems in terms of relationships highlights the importance of this way of understanding system function. The include complexity science,[1, 7] relational coordination (which is grounded in complexity science),[10] high reliability,[11] and the Big Five for teamwork.[12]
Research indicates that interactions among healthcare providers can have important influences on outcomes.[13, 14, 15, 16, 17] Additionally, the initial implementation of checklists to prevent central‐line associated infections appeared to change provider relationships in a way that significantly influenced their success.[18] For example, positive primary care clinic member relationships as assessed by the Lanham framework have been associated with better chronic care model implementation, learning, and patient experience of care.[19, 20] This framework, which we apply here, identifies 7 relationship characteristics: (1) trust; (2) diversity; (3) respect; (4) mindfulness, or being open to new ideas from others; (5) heedfulness, or an understanding of how one's roles influence those of others; (6) use of rich in‐person or verbal communication, particularly for potentially ambiguous information open to multiple interpretations; and (7) having a mixture of social and task relatedness among teams, or relatedness outside of only work‐related tasks.[19] Relationships within surgical teams that are characterized by psychological safety and diversity are associated with successful uptake of new techniques and decreased mortality.[13, 14] Relationships are important because the ability of patients and providers to learn and make sense of their patients' illnesses is grounded in relationships.
We sought to better understand and characterize inpatient physician teams' relationships, and assess the association between team relationships as evaluated by Lanham's framework and outcomes for hospitalized patients. Data on relationships among inpatient medical teams are few, despite the fact that these teams provide a great proportion of inpatient care. Additionally, the care of hospitalized medical patients is complex and uncertain, often involving multiple providers, making provider relationships potentially even more important to outcomes than in other settings.
METHODS
Overview
We conducted an observational, convergent mixed‐methods study of inpatient medicine teams.[21, 22, 23] We focused on inpatient physician teams, defining them as the functional work group responsible for medical decision making in academic medical centers. Physician teams in this context have been studied in terms of social hierarchy, authority, and delegation.[24, 25, 26] Focusing on the relationships within these groups could provide insights into strategies to mitigate potential negative effects of hierarchy. We recognize that other providers are closely involved in the care of hospitalized patients, and although we did not have standard interactions between physicians, nurses, case managers, and other providers that we could consistently observe, we did include interactions with these other providers in our observations and assessments of team relationships. Because this work is among the first in inpatient medical teams, we chose to study a small number of teams in great depth, allowing us to make rich assessments of team relationships.
We chose patient outcomes of length of stay (LOS), unnecessary LOS (ULOS), and complication rates, adjusted for patient characteristics and team workload. LOS is an important metric of inpatient care delivery. We feel ULOS is an aspect of LOS that is dependent on the physician team, as it reflects their preparation of the patient for discharge. Finally, we chose complication rates because hospital‐acquired conditions and complications are important contributors to inpatient morbidity, and because recent surgical literature has identified complication rates as a contributor to mortality that could be related to providers' collective ability to recognize complications and act quickly.
This study was approved by the institutional review board at the University of Texas Health Science Center at San Antonio (UTHSCSA), the Research and Development Committee for the South Texas Veterans Health Care System (STVHCS), and the Research Committee at University Health System (UHS). All physicians consented to participate in the study. We obtained a waiver of consent for inclusion of patient data.
Setting and Study Participants
This study was conducted at the 2 UTHSCSA primary teaching affiliates. The Audie L. Murphy Veterans Affairs Hospital is the 220‐bed acute‐care hospital of the STVHCS. University Hospital is the 614‐bed, level‐I trauma, acute‐care facility for UHS, the county system for Bexar County, which includes the San Antonio, Texas major metropolitan area.
The inpatient internal medicine physician team was our unit of study. Inpatient medicine teams consisted of 1 faculty attending physician, 1 postgraduate year (PGY)‐2 or PGY‐3 resident, and 2 PGY‐1 members. In addition, typically 2 to 3 third‐year medical students were part of the team, and a subintern was sometimes present. Doctor of Pharmacy faculty and students were also occasionally part of the team. Social workers and case managers often joined team rounds for portions of the time, and nurses sometimes joined bedside rounds on specific patients. These teams admit all medicine patients with the exception of those with acute coronary syndromes, new onset congestive heart failure, or arrhythmias. Patients are randomly assigned to teams based on time of admission and call schedules.
Between these 2 hospitals, there are 10 inpatient medicine teams caring for patients, with a pool of over 40 potential faculty attendings. Our goal was to observe teams that would be most likely to vary in terms of their relationship characteristics and patient outcomes through observing teams with a range of individual members. We used a purposeful sampling approach to obtain a diverse sample, sampling based on physician attributes and time of year.[16, 17] Three characteristics were most important: attending physician years of experience, attending involvement in educational and administrative leadership, and the presence of struggling resident members, as defined by being on probation or having been discussed in the residency Clinical Competency Committee. We did not set explicit thresholds in terms of attending experience, but instead sought to ensure a range. The attendings we observed were more likely to be involved in education and administrative leadership activities, but were otherwise similar to those we did not observe in terms of years of experience. We included struggling residents to observe individuals with a range of skill sets, and not just high‐performing individuals. We obtained attending information based on our knowledge of the attending faculty pool, and from the internal medicine residency program. We sampled across the year to ensure a diversity of trainee experience, but did not observe teams in either July or August, as these months were early in the academic year. Interns spend approximately 5 months per year on inpatient services, whereas residents spend 2 to 3 months per year. Thus, interns but not residents observed later in the year might have spent significantly more time on an inpatient service. However, in all instances, none of the team members observed had worked together previously.
Data Collection
Data were collected over nine 1‐month periods from September 2008 through June 2011. Teams were observed daily for 2‐ to 4‐week periods during morning rounds, the time when the team discusses each patient and makes clinical decisions. Data collection started on the first day of the month, the first day that all team members worked together, and continued for approximately 27 days, the last day before the resident rotated to a different service. By comprehensively and systematically observing these teams' daily rounds, we obtained rich, in‐depth data with multiple data points, enabling us to assess specific team behaviors and interactions.
During the third and fourth months, we collected data on teams in which the attending changed partway through. We did this to understand the impact of individual attending change on team relationships. Because the team relationships differed with each attending, we analyzed them separately. Thus, we observed 7 teams for approximately 4‐week periods and 4 teams for approximately 2‐week periods.
Observers arrived in the team room prior to rounds to begin observations, staying until after rounds were completed. Detailed free‐text field notes were taken regarding team activities and behaviors, including how the teams made patient care decisions. Field notes included: length of rounds, which team members spoke during each patient discussion, who contributed to management discussions, how information from consultants was incorporated, how communication with others outside of the team occurred, how team members spoke with each other including the types of words used, and team member willingness to perform tasks outside of their usually defined role, among others. Field notes were collected in an open‐ended format to allow for inductive observations. Observers also recorded clinical data daily regarding each patient, including admission and discharge dates, and presenting complaint.
The observation team consisted of the principle investigator (PI) (hospitalist) and 2 research assistants (a graduate‐level medical anthropologist and social psychologist), all of whom were trained by a qualitative research expert to systematically collect data related to topics of interest. Observers were instructed to record what the teams were doing and talking about at all times, noting any behaviors that they felt reflected how team members related to each other and came to decisions about their patients, or that were characteristic of the team. To ensure consistency, the PI and 1 research assistant conducted observations jointly at the start of data collection for each team, checking concordance of observations daily using a percent agreement until general agreement on field note content and patient information reached 90%. Two individuals observed 24 days of data collection, representing 252 patient discussions (13% of observed discussions).
An age‐adjusted Charlson‐Deyo comorbidity score was calculated for each patient admitted to each team, using data from rounds and from each hospital's electronic health records (EHR).[27] We collected data regarding mental health conditions for each patient (substance use, mood disorder, cognitive disorder, or a combination) because these comorbidities could impact LOS or ULOS. Discharge diagnoses were based on the discharge summary in the EHR. We also collected data daily regarding team census and numbers of admissions to and discharges from each team to assess workload.
Three patient outcomes were measured: LOS, ULOS, and complications. LOS was defined as the total number of days the patient was in the hospital. ULOS was defined as the number of days a patient remained in the hospital after the day the team determined the patient was medically ready for discharge (assessed by either discussion on rounds or EHR documentation). ULOS may occur when postdischarge needs have been adequately assessed, or because of delays in care, which may be related to provider communication during the hospitalization. Complications were defined on a per‐patient, per‐day basis in 2 ways: the development of a new problem in the hospital, for example acute kidney injury, a hospital‐acquired infection, or delirium, or by the team noting a clinical deterioration after at least 24 hours of clinical stability, such as the patient requiring transfer to a higher level of care. Complications were determined based on discussions during rounds, with EHR verification if needed.
Analysis Phase I: Assessment of Relationship Characteristics
After the completion of data collection, field notes were reviewed by a research team member not involved in the original study design or primary data collection (senior medical student). We took this approach to guard against biasing the reviewer's view of team behaviors, both in terms of not having conducted observations of the teams and being blinded to patient outcomes.
The reviewer completed a series of 3 readings of all field notes. The first reading provided a summary of the content of the data and the individual teams. Behavioral patterns of each team were used to create an initial team profile. The field notes and profiles were reviewed by the PI and a coauthor not involved in data collection to ensure that the profiles adequately reflected the field notes. No significant changes to the profiles were made based on this review. The profiles were discussed at a meeting with members of the larger research team, including the PI, research assistants, and coinvestigators (with backgrounds in medicine, anthropology, and information and organization management). Behavior characteristics that could be used to distinguish teams were identified in the profiles using a grounded theory approach.
The second review of field notes was conducted to test the applicability of the characteristics identified in the first review. To systematically record the appearance of the behaviors, we created a matrix with a row for each behavior and columns for each team to note whether they exhibited each behavior. If the behavior was exhibited, specific examples were cataloged in the matrix. This matrix was reviewed and refined by the research team. During the final field note review meeting, the research team compared the summary matrix for each team, with the specific behaviors noted during the first reading of the field notes to ensure that all behaviors were recorded.
After cataloging behaviors, the research team assigned each behavior to 1 of the 7 Lanham relationship characteristics. We wanted to assess our observations against a relationship framework to ensure that we were able to systematically assess all aspects of relationships. The Lanham framework was initially developed based on a systematic review of the organizational and educational literatures, making it relevant to the complex environment of an academic medical inpatient team and allowing us to assess relationships at a fine‐grained, richly detailed level. This assignment was done by the author team as a group. Any questions were discussed and different interpretations resolved through consensus. The Lanham framework has 7 characteristics.[19] Based on the presence of behaviors associated with each relationship characteristic, we assigned a point to each team for each relationship characteristic observed. We considered a behavior type to be present if we observed it on at least 3 occasions on separate days. Though we used a threshold of at least 3 occurrences, most teams that did not receive a point for a particular characteristic did not have any instances in which we observed the characteristic. This was particularly true for trust and mindfulness, and least so for social/task relatedness. By summing these points, we calculated a total relationship score for each team, with potential scores ranging from 0 (for teams exhibiting no behaviors reflecting a particular relationship characteristic) to 7.
Analysis Phase II: Factor Analysis
To formally determine which relationship characteristics were most highly related, data were submitted to a principal components factor analysis using oblique rotation. Item separation was determined by visual inspection of the scree plot and eigenvalues over 1.
Analysis Phase III: Assessing the Association between Physician Team Relationship Characteristics and Patient Outcomes
We examined the association between team relationships and patient outcomes using team relationship scores. For the LOS/ULOS analysis, we only included patients whose entire hospitalization occurred under the care of the team we observed. Patients who were on the team at the start of the month, were transferred from another service, or who remained hospitalized after the end of the team's time together were excluded. The longest possible LOS for patients whose entire hospitalization occurred on teams that were observed for half a month was 12 days. To facilitate accurate comparison between teams, we only included patients whose LOS was 12 days.
Complication rates were defined on a per‐patient per‐day basis to normalize for different team volumes and days of observation. For this analysis, we included patients who remained on the team after data collection completion, patients transferred to another team, or patients transferred from another team. However, we only counted complications that occurred at least 24 hours following transfer to minimize the likelihood that the complication was related to the care of other physicians.
Preliminary analysis involved inspection and assessment of the distribution of all variables followed, by a general linear modeling approach to assess the association between patient and workload covariates and outcomes.[28, 29] Because we anticipated that outcome variables would be markedly skewed, we also planned to assess the association between relationship characteristics with outcomes using the Kruskal‐Wallis rank sum test to compare groups with Dunn's test[30] for pairwise comparisons if overall significance occurred.[31] There are no known acceptable methods for covariate adjustments using the Kruskal‐Wallis method. All models were run using SAS software (SAS Institute Inc., Cary, NC).[32]
RESULTS
The research team observed 1941 discussions of 576 individual patients. Observations were conducted over 352 hours and 54 minutes, resulting in 741 pages of notes (see Supporting Table 1 in the online version of this article for data regarding individual team members). Teams observed over half‐months are referred to with a and b designations.
| Relationship Characteristic | Definition | Thirteen Types of Behaviors Observed in Field Notes | Observed Examples |
|---|---|---|---|
| Trust | Willingness to be vulnerable to others | Use of we instead of you or I by the attending | Where are we going with this guy? |
| Attending admitting I don't know | Let's go talk to him, I can't figure this out | ||
| Asking questions to help team members to think through problems | Will the echo change our management? How will it help us? | ||
| Diversity | Including different perspectives and different thinking | Team member participation in conversations about patients that are not theirs | One intern is presenting, another intern asks a question, and the resident joins the discussion |
| Inclusion of perspectives of those outside the team (nursing and family members) | Taking a break to call the nurse, having a family meeting | ||
| Respect | Valuing the opinions of others, honest and tactful interactions | Use of positive reinforcement by the attending | Being encouraging of the medical student's differential, saying excellent |
| How the team talks with patients | Asking if the patient has any concerns, what they can do to make them comfortable | ||
| Heedfulness | Awareness of how each person's roles impact the rest of the team | Team members performing tasks not expected of their role | One intern helping another with changing orders to transfer a patient |
| Summarizing plans and strategizing | Attending recaps the plan for the day, asks what they can do | ||
| Mindfulness | Openness to new ideas/free discussion about what is and is not working | Entire team engaged in discussion | Attending asks the medical student, intern, and resident what they think is going on |
| Social relatedness | Having socially related interactions | Social conversation among team members | Intern talks about their day off |
| Jokes by the attending | Showers and a bowel movement is the key to making people happy | ||
| Appropriate use of rich communication | Use of in‐person communication for sensitive or difficult issues | Using verbal communication with consultants or family | Intern is on the phone with the pharm D because there is a problem with the medication |
Creation of team profiles yielded 13 common behavior characteristics that were inductively identified and that could potentially distinguish teams, including consideration of perspectives outside of the team and team members performing tasks normally outside of their roles. Table 1 provides examples of and summarizes observed behaviors using examples from the field notes, mapping these behavior characteristics onto the Lanham relationship characteristics. The distribution of relationship characteristics and scores for each team are shown in Table 2.
| Relationship Characteristic | Team | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3a | 3b | 4a | 4b | 5 | 6 | 7 | 8 | 9 | |
| Trust | 0 | 0 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 1 |
| Diversity | 0 | 1 | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 1 | 1 |
| Respect | 0 | 1 | 1 | 1 | 0 | 1 | 0 | 0 | 1 | 1 | 1 |
| Heedfulness | 0 | 1 | 1 | 0 | 1 | 0 | 1 | 0 | 1 | 1 | 1 |
| Mindfulness | 0 | 0 | 1 | 0 | 0 | 1 | 1 | 0 | 1 | 1 | 1 |
| Social/task relatedness | 0 | 1 | 1 | 0 | 1 | 1 | 1 | 0 | 1 | 1 | 1 |
| Rich/lean communication | 0 | 1 | 1 | 0 | 0 | 0 | 1 | 0 | 1 | 1 | 0 |
| Relationship score (no. of characteristics observed) | 0 | 5 | 7 | 2 | 2 | 3 | 5 | 0 | 7 | 7 | 6 |
Correlation between relationship characteristics ranged from 0.32 to 0.95 (see Supporting Table 2 in the online version of this article). Mindfulness and trust are more highly correlated with each other than with other variables, as are diversity and respect. We performed a principal components factor analysis. Based on scree plot inspection and eigenvalues >1, we kept 3 factors that explained 85% of the total variance (see Supporting Table 3 in the online version of this article).
| No. of Relationship Characteristics | |||
|---|---|---|---|
| 02 | 35 | 67 | |
| |||
| LOS, d, n=293 | |||
| Median | 4 | 5 | 3 |
| IQR | 5 | 4 | 3 |
| Mean | 4.7 (2.72) | 4.7 (2.52) | 4.1 (2.51), P=0.12a |
| ULOS, d, n=293 | |||
| Median | 0 | 0 | 0 |
| IQR | 0 | 0 | 0 |
| Mean | 0.37 (0.99) | 0.33 (0.96) | 0.13 (0.56), P=0.09a |
| Complications (per patient per day), n=398 | |||
| Median | 0 | 0 | 0 |
| IQR | 1 | 1 | 0 |
| Mean | 0.58 (1.06) | 0.45 (0.77) | 0.18 (0.59), P=0.001 compared to teams with 02 or 35 characteristics |
Our analyses of LOS and ULOS included 298 of the 576 patients. Two hundred sixty‐seven patients were excluded because their entire LOS did not occur while under the care of the observed teams. Eleven patients were removed from the analysis because their LOS was >12 days. The analysis of complications included 398 patients. In our preliminary general linear modeling approach, only patient workload was significantly associated with outcomes using a cutoff of P=0.05. Charlson‐Deyo score and mental health comorbidities were not associated with outcomes.
The results of the Kruskal‐Wallis test show the patient average ranking on each of the outcome variables by 3 groups (Table 3). Overall, teams with higher relationship scores had lower rank scores on all outcomes measures. However, the only statistically significant comparisons were for complications. Teams having 6 to 7 characteristics had a significantly lower complication rate ranking than teams with 0 to 2 and 3 to 5 (P=0.001). We did not find consistent differences between individual teams or groups of teams with relationship scores from 0 to 2, 3 to 5, and 6 to 7 with regard to Charlson score, mental health issues, or workload. The only significant differences were between Charlson‐Deyo scores for patients admitted to teams with low relationship scores of 0 to 2 versus high relationship scores of 6 to 7 (6.7 vs 5.1); scores for teams with relationship scores of 3 to 5 were not significantly different from the low or high groups.
Table 4 shows the Kruskal‐Wallace rank test results for each group of relationship characteristics identified in the factor analysis based on whether teams displayed all or none of the characteristics in the factor. There were no differences in these groupings for LOS. Teams that exhibited both mindfulness and trust had lower ranks on ULOS than teams that did not have either. Similarly, teams with heedfulness, social‐task relatedness, and more rich communication demonstrated lower ULOS rankings than teams who did not have all 3 characteristics.
| Mind/Trust | Diversity/Respect | Heed/Relate/Communicate | ||||
|---|---|---|---|---|---|---|
| Patient Outcome | None | Both | None | Both | None | All 3 |
| ||||||
| LOS, d, n=293 | ||||||
| Median | 4 | 4 | 4 | 4 | 4 | 4 |
| IQR | 5 | 3 | 4.5 | 3 | 4 | 4 |
| Mean | 4.7 (2.6) | 4.2 (2.5) | 4.7 (2.6) | 4.3 (2.5) | 4.4 (2.6) | 4.4 (2.6) |
| P value | 0.06a | 0.23a | 0.85a | |||
| ULOS, d, n=293 | ||||||
| Median | 0 | 0 | 0 | 0 | 0 | 0 |
| IQR | 0 | 0 | 0 | 0 | 0 | 0 |
| Mean | 0.39 (1.01) | 0.15 (0.62) | 0.33 (0.92) | 0.18 (0.71) | 0.32 (0.93) | 0.18 (0.69) |
| P value | 0.009 | 0.06 | 0.03 | |||
| Complications (per patient), n=389 | ||||||
| Median | 0 | 0 | 0 | 0 | 0 | 0 |
| IQR | 1 | 0 | 1 | 0 | 1 | 0 |
| Mean | 0.58 (1.01) | 0.19 (0.58) | 0.47 (0.81) | 0.29 (0.82) | 0.26 (0.92) | 0.28 (0.70) |
| P value | <0.0001 | 0.001 | 0.02 | |||
DISCUSSION
Relationships are critical to team function because they are the basis for the social interactions that are central to patient care. These interactions include how providers recognize and make sense of what is happening with patients, and how they learn to care for patients more effectively. Additionally, the high task interdependencies among inpatient providers require effective relationships for optimal care. In our study, inpatient medicine physician teams' relationships varied, and these differences were associated with ULOS and complications. Relationship characteristics are not mutually exclusive, and as our factor analysis demonstrates, are intercorrelated. Trust and mindfulness appear to be particularly important. Trust may foster psychological safety that in turn promotes the willingness of individuals to contribute their thoughts and ideas.[13] In low‐trust teams, providers may fear a negative impact for bringing forward a concern based on limited data. Mindful teams may be more likely to notice nuanced changes, or are more likely to talk when things just do not appear to be going in the right direction with the patient. In the case of acutely ill medical patients, trust and mindfulness may lead to an increased likelihood that clinical changes are recognized and discussed quickly. For example, on a team characterized by trust and mindfulness, the entire team was typically involved in care discussions, and the interns and students frequently asked a lot of questions, even regarding the care of patients they were not directly following. We observed that these questions and discussions often led the team to realize that they needed to make a change in management decisions (eg, discontinuing Bactrim, lowering insulin doses, adjusting antihypertensives, premedicating for intravenous contrast) that they had not caught in the assessment and plan portion of the patient care discussion. In another example, a medical student asked a tentative question after a patient needed to go quickly to the bathroom while they were examining her, leading the team to ask more questions that led to a more rapid evaluation of a potential urinary tract infection. This finding is consistent with the description of failure to rescue among surgical patients, in which mortality has been associated with the failure to recognize complications rapidly and act effectively.[33]
Our findings are limited in several ways. First, these data are from a single academic institution. Although we sought diversity among our teams and collected data across 2 hospitals, there may be local contextual factors that influenced our results. Second, our data demonstrate an association, but not causality. Our findings should be tested in studies that assess causality and potential mechanisms through which relationships influence outcomes. Third, the individuals observing the teams had some knowledge of patient outcomes through hearing patient discussions. However, by involving individuals who did not participate in observations and were blinded to outcomes in assessing team relationships, we addressed this potential bias. Fourth, our observations were largely focused on physician teams, not directly including other providers. Our difficulty in observing regular interactions between physicians and other providers underscores the need to increase contact among those caring for hospitalized patients, such as occurs through multidisciplinary rounds. We did include team communication with other disciplines in our assessment of the relationship characteristics of diversity and rich communication. Finally, our analysis was limited by our sample size. We observed a relatively small number of teams. Although we benefitted from seeing the change in team relationships that occurred with attending changes halfway through some of our data collection months, this did limit the number of patients we could include in our analyses. Though we did not observe obvious differences in relationships between the teams observed across the 2 hospitals, the small number of teams and hospitals precluded our ability to perform multilevel modeling analyses, which would have allowed us to assess or account for the influence of team or organizational factors. However, this small sample size did allow for a richer assessment of team behaviors.
Although preliminary, our findings are an important step in understanding the function of inpatient medical teams not only in terms of processes of care, but also in terms of relationships. Patient care is a social activity, requiring effective communication to develop working diagnoses, recognize changes in patients' clinical courses, and formulate effective treatment plans during and after hospitalization. Future work could follow several directions. One would be to assess the causal mechanisms through which relationships influence patient outcomes. These may include sensemaking, learning, and improved coordination. Positive relationships may facilitate interaction of tacit and explicit information, facilitating the creation of understandings that foster more effective patient care.[34] The dynamic nature of relationships and how patient outcomes in turn feed back into relationships could be an area of exploration. This line of research could build on the idea of teaming.[35] Understanding relationships across multidisciplinary teams or with patients and families would be another direction. Finally, our results could point to potential interventions to improve patient outcomes through improving relationships. Better understanding of the nature of effective relationships among providers should enable us to develop more effective strategies to improve the care of hospitalized patients. In the larger context of payment reforms that require greater coordination and communication among and across providers, a greater understanding of how relationships influence patient outcomes will be important.
Acknowledgements
The authors thank the physicians involved in this study and Ms. Shannon Provost for her involvement in discussions of this work.
Disclosures: The research reported herein was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service (CDA 07‐022). Investigator salary support was provided through this funding, and through the South Texas Veterans Health Care System. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs. Dr. McDaniel receives support from the IC[2] Institute of the University of Texas at Austin. Dr. Luci Leykum had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. The authors report no conflicts of interest.
- . Redesigning health care with insights from the science of complex adaptive systems. In: Crossing the Quality Chasm: A New Heath System for the 21st Century. Washington, DC: National Academy of Sciences; 2000:309–322.
- , , , , , . Temporal trends in rates of patient harm resulting from medical care. N Engl J Med. 2010;323(22):2124–2135.
- , , , et al. Circumstances of patient falls and injuries in 9 hospitals in a mid‐western healthcare system. Infect Control Hosp Epidemiol. 2007;28(5):544–550.
- , . Point prevalence of wounds in a sample of acute hospitals in Canada. Int Wound J. 2009;6(4):287–293.
- , , , , , . Non‐adherence to guidelines: an avoidable cause of failure of empirical antimicrobial therapy in the presence of difficult‐to‐treat bacteria. Intensive Care Med. 2010;36(1):75–82.
- , , , , . Quality of care in U.S. hospitals as reflected by standardized measures, 2002–2004. N Engl J Med. 2005;353(3):255–264.
- Centers for Disease Control and Prevention. National Center for Emerging and Zoonotic Infectious Diseases. Division of Healthcare Quality Promotion. Checklist for prevention of central line associated blood stream infections. Available at: http://www.cdc.gov/HAI/pdfs/bsi/checklist‐for‐CLABSI.pdf. Accessed August 3, 2014.
- Safer Healthcare Partners, LLC. Checklists: a critical patient safety tool. Available at: http://www.saferhealthcare.com/high‐reliability‐topics/checklists. Accessed July 31, 2014.
- . Making Things Work: Solving Complex Problems in a Complex World. Boston, MA: Knowledge Press; 2004:117–160.
- . High Performance Healthcare: Using The Power of Relationships to Achieve Quality, Efficiency, and Resilience. 1st ed. New York, NY: McGraw‐Hill; 2009.
- , . Design of high reliability organizations in health care. Qual Saf Health Care. 2006;15(suppl 1):i4–i9.
- , , , . Does team training work? Principles for health care. Acad Emerg Med. 2008;15(11):1002–1009.
- . Speaking up in the operating room: how team leaders promote learning in interdisciplinary action teams. J Manag Stud. 2003;40(6):1419–1452.
- , , , et al. Association between implementation of a medical team training program and surgical mortality. JAMA. 2010;304(15):1693–1700.
- , , , . Group cognition, membership change, and performance: Investigating the benefits and detriments of collective knowledge. Organ Behav Hum Decis Process. 2007;103(2):159–178.
- , , , , , . Reciprocal learning and chronic care model implementation in primary care: results from a new scale of learning in primary care settings. BMC Health Serv Res. 2011;11:44.
- , , , , . The importance of relational coordination and reciprocal learning for chronic illness care within primary care teams. Health Care Manage Rev. 2012;38(1):20–28.
- , , , , . Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q. 2011;89(2):167–205.
- , , , et al. How improving practice relationships among clinicians and nonclinicians can improve quality in primary care. Jt Comm J Qual Patient Saf. 2009;35(9):457–466.
- , , , et al. Relationship quality and patient‐assessed quality of care in VA primary care clinics: development and validation of the work relationships scale. Ann Fam Med. 2013;11(6):543–549.
- , . Designing and Conducting Mixed Methods Research. 2nd ed. Thousand Oaks, CA: Sage; 2011.
- . Qualitative Evaluation Methods. Thousand Oaks, CA: Sage; 2002.
- , , . Qualitative methods in research on health care quality. Qual Saf Health Care. 2002;11:148–152.
- . Managing the negatives of experience in physician teams. Health Care Manage Rev. 2010;35(1):65–76.
- , , , , . Rethinking resident supervision to improve safety: from hierarchical to interprofessional models. J Hosp Med. 2011;6(8):445 b452.
- , , , . Dynamic delegation: shared, hierarchical, and deindividualized leadership in extreme action teams. Adm Sci Q. 2006;51(4):590–621.
- , , . Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases. J Clin Epidemiol. 1992;45(6):613–619.
- . Exploratory Data Analysis. Reading, MA: Addison‐Wesley; 1977.
- . Biostatistical Analysis. 4th ed. Upper Saddle River, NJ: Pearson Prentice‐Hall; 2010.
- . Multiple contrasts using rank sums. Technometrics. 1964;6:241–252.
- , . A SAS macro implementation of a multiple comparison post hoc test for a Kruskal–Wallis analysis. Comput Methods Programs Biomed. 2011;102:75–80.
- SAS/STAT Software [computer program]. Version 9.1. Cary, NC: SAS Institute Inc.; 2003.
- , , . Complications, failure to rescue, and mortality with major inpatient surgery in Medicare patients. Ann Surg. 2009;250(6):1029–1034.
- . A dynamic theory of organizational knowledge creation. Org Sci. 1994;5(1):14–37.
- . Teaming: How Organizations Learn, Innovate, and Compete in the Knowledge Economy. 1st ed. Boston, MA: Harvard Business School; 2012.
- . Redesigning health care with insights from the science of complex adaptive systems. In: Crossing the Quality Chasm: A New Heath System for the 21st Century. Washington, DC: National Academy of Sciences; 2000:309–322.
- , , , , , . Temporal trends in rates of patient harm resulting from medical care. N Engl J Med. 2010;323(22):2124–2135.
- , , , et al. Circumstances of patient falls and injuries in 9 hospitals in a mid‐western healthcare system. Infect Control Hosp Epidemiol. 2007;28(5):544–550.
- , . Point prevalence of wounds in a sample of acute hospitals in Canada. Int Wound J. 2009;6(4):287–293.
- , , , , , . Non‐adherence to guidelines: an avoidable cause of failure of empirical antimicrobial therapy in the presence of difficult‐to‐treat bacteria. Intensive Care Med. 2010;36(1):75–82.
- , , , , . Quality of care in U.S. hospitals as reflected by standardized measures, 2002–2004. N Engl J Med. 2005;353(3):255–264.
- Centers for Disease Control and Prevention. National Center for Emerging and Zoonotic Infectious Diseases. Division of Healthcare Quality Promotion. Checklist for prevention of central line associated blood stream infections. Available at: http://www.cdc.gov/HAI/pdfs/bsi/checklist‐for‐CLABSI.pdf. Accessed August 3, 2014.
- Safer Healthcare Partners, LLC. Checklists: a critical patient safety tool. Available at: http://www.saferhealthcare.com/high‐reliability‐topics/checklists. Accessed July 31, 2014.
- . Making Things Work: Solving Complex Problems in a Complex World. Boston, MA: Knowledge Press; 2004:117–160.
- . High Performance Healthcare: Using The Power of Relationships to Achieve Quality, Efficiency, and Resilience. 1st ed. New York, NY: McGraw‐Hill; 2009.
- , . Design of high reliability organizations in health care. Qual Saf Health Care. 2006;15(suppl 1):i4–i9.
- , , , . Does team training work? Principles for health care. Acad Emerg Med. 2008;15(11):1002–1009.
- . Speaking up in the operating room: how team leaders promote learning in interdisciplinary action teams. J Manag Stud. 2003;40(6):1419–1452.
- , , , et al. Association between implementation of a medical team training program and surgical mortality. JAMA. 2010;304(15):1693–1700.
- , , , . Group cognition, membership change, and performance: Investigating the benefits and detriments of collective knowledge. Organ Behav Hum Decis Process. 2007;103(2):159–178.
- , , , , , . Reciprocal learning and chronic care model implementation in primary care: results from a new scale of learning in primary care settings. BMC Health Serv Res. 2011;11:44.
- , , , , . The importance of relational coordination and reciprocal learning for chronic illness care within primary care teams. Health Care Manage Rev. 2012;38(1):20–28.
- , , , , . Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q. 2011;89(2):167–205.
- , , , et al. How improving practice relationships among clinicians and nonclinicians can improve quality in primary care. Jt Comm J Qual Patient Saf. 2009;35(9):457–466.
- , , , et al. Relationship quality and patient‐assessed quality of care in VA primary care clinics: development and validation of the work relationships scale. Ann Fam Med. 2013;11(6):543–549.
- , . Designing and Conducting Mixed Methods Research. 2nd ed. Thousand Oaks, CA: Sage; 2011.
- . Qualitative Evaluation Methods. Thousand Oaks, CA: Sage; 2002.
- , , . Qualitative methods in research on health care quality. Qual Saf Health Care. 2002;11:148–152.
- . Managing the negatives of experience in physician teams. Health Care Manage Rev. 2010;35(1):65–76.
- , , , , . Rethinking resident supervision to improve safety: from hierarchical to interprofessional models. J Hosp Med. 2011;6(8):445 b452.
- , , , . Dynamic delegation: shared, hierarchical, and deindividualized leadership in extreme action teams. Adm Sci Q. 2006;51(4):590–621.
- , , . Adapting a clinical comorbidity index for use with ICD‐9‐CM administrative databases. J Clin Epidemiol. 1992;45(6):613–619.
- . Exploratory Data Analysis. Reading, MA: Addison‐Wesley; 1977.
- . Biostatistical Analysis. 4th ed. Upper Saddle River, NJ: Pearson Prentice‐Hall; 2010.
- . Multiple contrasts using rank sums. Technometrics. 1964;6:241–252.
- , . A SAS macro implementation of a multiple comparison post hoc test for a Kruskal–Wallis analysis. Comput Methods Programs Biomed. 2011;102:75–80.
- SAS/STAT Software [computer program]. Version 9.1. Cary, NC: SAS Institute Inc.; 2003.
- , , . Complications, failure to rescue, and mortality with major inpatient surgery in Medicare patients. Ann Surg. 2009;250(6):1029–1034.
- . A dynamic theory of organizational knowledge creation. Org Sci. 1994;5(1):14–37.
- . Teaming: How Organizations Learn, Innovate, and Compete in the Knowledge Economy. 1st ed. Boston, MA: Harvard Business School; 2012.
© 2014 Society of Hospital Medicine