User login
Healthcare in rural areas faces increasing challenges due to community hospital closures, physician shortages, and a more concentrated population of older adults with higher rates of comorbid conditions than their urban counterparts.1-3 Critical access hospitals (CAHs), which primarily serve rural areas, have fewer clinical capabilities, worse process-of-care measures, and higher mortality rates for some conditions when compared to non-CAHs.4 As such, CAHs are closing at record numbers across the United States,5 resulting in loss of available hospital beds and patient access to timely emergency services,6 which can worsen outcomes, further widening the rural-urban healthcare gap.7,8 Furthermore, this strain on an overwhelmed health system in the most vulnerable areas restricts the ability to respond to healthcare crises like the coronavirus disease 2019 pandemic.9
Providing adequate staff for currently available hospital beds is also a problem in rural areas. Studies demonstrating improved outcomes, decreased length of stay (LOS), and increased quality with hospitalist services have resulted in a high demand for hospitalists nationwide.10-12 Recruiting hospitalists to work in rural areas, however, has become increasingly challenging due to low-patient volumes, financial viability of hospitalist-model adoption, and provider shortages.13,14 Recently, the Veterans Health Administration (VHA) reported a 28% nationwide shortage of hospitalists,15 which disproportionally affects rural VHA hospitals. Staffing difficulties and reliance on intermittent providers were reported by more than 80% of rural and low-complexity VHA facilities.16
Telehospitalist services (THS) can help deliver high-quality care to rural residents locally, decrease travel expenses, support hospital volume, and increase healthcare capacity in response to a pandemic.14,17,18 Only a few studies have described THS (mostly with overnight or cross-coverage models directed to CAHs), and clinical outcomes have been inconsistently reported.17,19-21 Furthermore, no program has been conducted within an integrated health system akin to the VHA. The primary objective of this quality improvement (QI) initiative was to perform a mixed-methods evaluation of THS between VHA hospitals to compare clinical outcomes and patient and staff satisfaction. Secondary outcomes included description of the implementation process, unexpected challenges, and subsequent QI initiatives. These results will expand the knowledge on feasibility of THS and provide implementation guidance.
METHODS
A mixed-methods approach was used to evaluate outcomes of this QI project. Reporting follows the revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0).22
Context
The VHA is the largest integrated healthcare system in the United States, with more than 8 million veterans enrolled, more than 30% of whom reside in a rural area. The VHA comprises more than 1,000 outpatient clinics and 170 acute care VA Medical Centers,23,24 including more than 35 rural and low-complexity hospitals.25 Low-complexity hospitals are those with the lowest volume and levels of patient complexity and minimal or no teaching programs, research, intensive care unit (ICU) beds, and subspecialists. Lack of reimbursement and interstate licensing, often cited as barriers to telemedicine, do not apply to the VHA. The hub site was a large tertiary care (high-complexity) VHA hospital located in Iowa City, Iowa. The spoke site was a low-complexity (10-bed acute inpatient unit with no ICU) rural VA hospital located in Tomah, Wisconsin.
Study Population
The preimplementation cohort for comparison included all patients admitted between January 1, 2018, and January 6, 2019. The postimplementation study cohort included all observation and acute care admissions during the pilot phase (January 7 to May 3, 2019) and sustainability phase (July 15 to December 31, 2019). The postimplementation analysis excluded the time period of May 4 to July 14, 2019, due to an interruption (gap) in THS. The gap period allowed for preliminary data analysis, optimization of the telecommunication system, and the recruitment and training of additional providers who could provide long-term staffing to the service.
Intervention
Preimplementation
Prior to THS implementation, Tomah’s inpatient ward was staffed by one physician per shift, who could be a hospitalist, medical officer of the day (MOD), or an intermittent provider (locum tenens). Hospitalists covering the acute inpatient ward prior to the THS transitioned to cover weekends, nights, and urgent care service shifts.
We visited the spoke site and held information-sharing sessions with key stakeholders (administrators, clinician leaders, nurses, and ancillary staff) prior to kick-off. Recurrent phone meetings addressed anticipated and emerging challenges. Telehospitalist and local providers underwent technology and service training.
Technology and Connectivity
A low-cost technology system using tablet computers provided Health Insurance Portability and Accountability Act–compliant videoconferencing with a telehospitalist at the hub site. An Eko-Core digital stethoscope® with a web-based audio stream was available. Telehospitalists conducted encounters from a private office space with telehealth capabilities. A total of $9,000 was spent on equipment at both sites. Due to connectivity problems and data limits, the tablets were switched to mobile computer-on-wheels workstations and hospital-based Wi-Fi for the sustainability phase.
THS Description
An experienced hub hospitalist, together with an advanced practice provider (APP; nurse practitioner [NP] or physician assistant [PA]), cared for all patients admitted to the 10-bed inpatient unit at the spoke site, Monday through Friday from 8:00 AM to 4:30 PM. The APP had limited or no prior experience in acute inpatient medicine. The telehospitalist worked as a team with the APP. The APP was the main point of contact for nurses, performed physical examinations, and directed patient care to their level of comfort (in a similar manner as a teaching team). The telehospitalist conducted bedside patient rounds, participated in multidisciplinary huddles, and shared clinical documentation and administrative duties with the APP. The telehospitalist was the primary staff for admitted patients and had full access to the electronic health record (EHR). The THS was staffed by 10 hospitalists during the study period. Overnight and weekend cross-coverage and admissions were performed by MODs, who also covered the urgent care and cross-covered other nonmedical units.
Quantitative Evaluation Methods
Workload and Clinical Outcomes
An EHR query identified all patients admitted during the pre- and postimplementation periods. Demographic data, clinical Nosos risk scores,26,27 and top admission diagnoses were reported. Workload was evaluated using the average number of encounters per day and self-reported telehospitalist worksheets, which were cross-referenced with EHR data. Clinical outcomes included LOS, 30-day hospital readmission rate, 30-day standardized mortality (SMR30), in-hospital mortality, and VHA-specific inpatient quality metrics. Independent sample t tests for continuous variables and chi-square tests or Fisher’s exact test (for patient class) for categorical variables were used to compare pre- and postimplementation groups. Statistical process control (SPC) charts evaluated changes over time. All analyses were conducted using Microsoft Excel and R.28
Anonymous surveys were distributed to spoke-site inpatient and administrative staff at 1 month and 12 months postimplementation, assessing satisfaction, technology/connectivity, communication, and challenges (Appendix Figure 1). Satisfaction of the telehospitalist physicians at the hub site was measured 12 months postimplementation by a 26-question survey assessing the same domains, plus quality of care (Appendix Figure 2).
Patient Satisfaction
The VHA Survey of Healthcare Experiences of Patients (SHEP), a version of the Hospital Consumer Assessment of Healthcare Providers and Systems Survey,29,30 was mailed to all patients after discharge. Survey responses concerning inpatient provider care (eg, care coordination, communication, hospital rating, willingness to recommend the hospital) during the pre- and postimplementation phases were compared using a two-sample test of independent proportions. Responses obtained during May and June 2019 were excluded.
Qualitative Evaluation Methods
The authors had full access to, and took full responsibility for, the integrity of the data. The project was evaluated by the University of Iowa Institutional Review Board and the Iowa City VA Research and Development Committee and was determined to be a non–human-subjects QI project.
RESULTS
Quantitative Workload and Clinical Outcomes
There were 822 admissions during the preimplementation period and 550 admissions during the postimplementation period (253 during the pilot and 297 during sustainability phase). Patient characteristics pre- and postimplementation were not significantly different (Table 1). The median patient age was 65 years; 96% of patients were male, and 83% were rural residents. The most common admission diagnosis was alcohol-related (36%); regarding patient disposition, 78% of admissions were discharged home.
Workload
There were 502 patient encounters staffed by the telehospitalist in the pilot phase, with an average of 6.25 encounters per day, and a telehospitalist-reported workload of 7 hours per day. There were 538 patient encounters, with an average of 4.67 encounters per day and a workload of 5.6 hours per day in the sustainability phase. The average daily census decreased from 5.0 (SD, 1.1) patients per day during preimplementation to 3.1 (SD, 0.5) patients per day during postimplementation (Table 2). In some of the months during the study period, admissions decreased below the lower SPC limit, suggesting a significant change (Figure). Adjusted LOS was significantly lower, with 3.0 (SD, 0.7) days vs 2.3 (SD, 0.3) days in the pre- and postimplementation periods, respectively. Bed occupancy rates were significantly lower in the sustainability phase compared with the pilot phase and the preimplementation period. Readmission rates varied, ranging from <10% to >30%, not significantly different but slightly higher in the postimplementation period. Readmission rates for heart failure, chronic obstructive pulmonary disease, and pneumonia remained unchanged; other medical readmissions (mostly alcohol-related) were slightly higher in the postimplementation period.
In-hospital mortality and SMR30 did not change significantly, but there was improvement in the 12-month rolling average of the observed/expected SMR30 from 1.40 to 1.08. Additional VHA-specific quality metrics were monitored and showed either small improvements or no change (data not shown).
Satisfaction at Hub and Spoke Sites
After sending two reminder communications via email, the telehospitalist satisfaction survey had a total response rate of 90% (9/10). Telehospitalists were satisfied or very satisfied (89%) with the program and the local providers (88.9%), rating their experience as good or excellent (100%) (Table 3). Communication with patients, families, and local staff was noted as being “positive” or “mostly positive.” Telehospitalists reported confidence in the accuracy of their diagnoses and rated the quality of care as being equal to that of a face-to-face encounter. Connectivity problems were prevalent, although most providers were able to resort to a back-up plan. Other challenges included differences in culture and concerns about liability. We received 27 responses from the spoke-site satisfaction survey; the response rate could not be determined because the survey was distributed by the spoke site for anonymity. Of the respondents, 37% identified as nurses, 25.9% as healthcare providers (APPs or physicians), and 33.3% as other staff (eg, social worker, nutritionist, physical therapist, utilization management, administrators); 3.7% did not respond. Among the participants, 88% had personally interacted with the THS. Most providers and other staff perceived THS as valuable (57.1% and 77.8%, respectively) and were satisfied or highly satisfied with THS (57.1% and 55.6%, respectively). On average, nurses provided lower ratings across all survey items than providers and other staff. Challenges noted by all staff included issues with communication, workflow, and technology/connectivity.
Qualitative Strengths
Our process evaluation identified high quality of care and teamwork as contributors to the success of the program. Overall, staff credited perceived improvements in quality of care to the quality of providers staffing the THS, including the local APPs. Noting the telehospitalists’ knowledge base and level of engagement as key attributes, one staff member commented: “I prefer a telehospitalist that really care[s] about patients than some provider that is physically here but does not engage.” Staff perceived improvements in the continuity of care, as well as care processes such as handoffs and transitions of care.
Improvements in teamwork were perceived compared with the previous model of care. Telehospitalists were lauded for their professionalism and communication skills. Overall, nurses felt providers in the THS listened more to their views. In addition, nurse respondents felt they could learn from several providers and said they enjoyed the telehospitalists’ disposition to teach and discuss patient care. The responsiveness of the THS staff was instrumental in building teamwork and acceptance. A bedside interdisciplinary protocol was established for appropriate patients. Local staff felt this was crucial for teamwork and patient satisfaction. Telehospitalists reported high-value in interdisciplinary rounds, facilitating interaction with nurses and ancillary staff. Handoff problems were identified, leading to QI initiatives to mitigate those issues.
Challenges
The survey identified administrative barriers, technical difficulties, workflow constraints, and clinical concerns. The credentialing process was complicated, delaying the onboarding of telehospitalists. Internet connectivity was inconsistent, leading to disruption in video communications; however, during the sustainability phase, updated technology improved communications. The communication workflow was resisted by some nurses, who wanted to phone the telehospitalist directly rather than having the local APP as the first contact. Secure messaging was enabled to allow nurses direct contact during the sustainability phase.
Workload was a concern among telehospitalists and local staff. Telehospitalists perceived the documentation requirements and administrative workload to be two to three times higher than at other hospitals—despite the lower number of encounters. Finally, clinical concerns from spoke-site clinicians included a perceived rise in the acuity of patients (which was not evident by the Nosos score) and delayed decisions to transfer-out patients. These concerns were addressed with educational sessions for telehospitalists during the sustainability phase.
Additional Quality Improvement Projects
The implementation of THS resulted in QI initiatives at the spoke site, including an EHR-integrated handoff tool; a documentation evaluation that led to the elimination of duplicative, inefficient, and error-prone templates; and a revision of the alcohol withdrawal treatment protocol during the sustainability phase to reduce the use of intravenous benzodiazepines. A more comprehensive benzodiazepine-sparing alcohol withdrawal treatment protocol was also developed but was not implemented until after the study period (January 2020).
DISCUSSION
Our pre-post study evaluation found implementation of a THS to be noninferior to face-to-face care, with no significant change in mortality, readmission rate, or patient satisfaction. The significant improvement observed in LOS is consistent with the adoption of hospitalist models in other medical care settings,11 but had not been reported by previous telehospitalist studies. For example, in their retrospective chart review comparing an NP-supported telehospitalist model to locum tenens hospitalists, Boltz et al found no difference in LOS.31 Moreover, as in our study, they found no differences in readmissions, mortality, and patient satisfaction.31 Similarly, Kuperman et al reported unchanged daily census, LOS, and transfer rates from a CAH with their virtual hospitalist program, but a decrease in the percentage of patients transferred-out from the emergency department, suggesting that more patients were treated locally.19
Reduction in LOS is one of the primary measures of efficiency in hospital care31; reducing LOS while maintaining the quality of care lowers hospital costs. The reduction in LOS in our study could be attributed to greater continuity of care, engagement/experience of the telehospitalists, or other factors. This decrease in LOS and slight reduction in admissions resulted in an overall lower daily census during the study period and impacted efficiency. Our study was unable to determine the cause for the reduction in admissions; however, several concurrent events, including the expansion of community-care options for veterans under the MISSION ACT (Maintaining Internal Systems and Strengthening Integrated Outside Networks Act) in June 2019, a nationwide smoking ban at VA facilities (October 2019), and a modification in the alcohol withdrawal treatment protocol might have influenced veterans’ choice of hospital.
Readmission rates were slightly higher, though nonsignificant, in the postimplementation period. Alcohol-related readmissions accounted for most readmissions; some of the protocol changes, such as admitting all patients with alcohol withdrawal to inpatient class instead of admitting some to the observation class, accounted for part of the increase in readmission rates. Readmission rates for other conditions such as chronic obstructive pulmonary disease, chronic heart failure, or pneumonia were not significantly different, suggesting that the reduction in LOS did not result in an unintended increased readmission rate for those conditions.
Rural hospitals are struggling with staffing and finances. Resorting to locum tenens staffing is costly and can result in variable quality of care.32,33 APPs are increasingly taking on hospitalist positions, with 65% of adult hospitalist programs, including half of all VHA hospitals, employing NPs and PAs.34,35 In response to this expanded scope of practice, hospitals employing APPs in hospitalist roles must comply with state and federal laws, which often require that APPs be supervised by or work in collaboration with an on-site or off-site physician. The THS is a great model to support APPs and address staffing and cost challenges in low-volume rural facilities, while maintaining quality of care. Some APP-telehospitalist programs similar to ours have reported cost reductions of up 58% compared to programs that employ locum tenens physicians.31 In our model, we assume that a single telehospitalist hub could provide coverage to two or three spoke sites with APP support, reducing staffing costs.
Hub telehospitalists reported satisfaction with the program, and they perceived the quality of care to be comparable to face-to-face encounters; their responses were consistent with those previously reported in an evaluation of telemedicine acute care by JaKa et al.
This study has several limitations. First, the VHA is an integrated health system, one that serves an older, predominantly male patient population. Also, the lack of reimbursement and interstate licensing restrictions limit generalizability of these results to other CAHs or healthcare systems. Furthermore, the intervention was limited to a single rural site; while this allowed for a detailed evaluation, unique barriers or facilitators might exist that limit its applicability. In addition, QI initiatives implemented by the VHA during the project period might have confounded some of our results. Last, patient satisfaction survey data are overall limited in their ability to fully assess patient’s experience and satisfaction with the program. Further qualitative studies are needed to gain deeper insight into patient perspectives with the THS and whether modality of care delivery influences patients’ care decisions. Future studies should consider a multisite design with one or more hubs and multiple spoke sites.
CONCLUSION
Telehospitalist services are a feasible and safe approach to provide inpatient services and address staffing needs of rural hospitals. To enhance program performance, it is essential to ensure adequate technological quality, clearly delineate and define roles and responsibilities of the care team, and address communication issues or staff concerns in a timely manner.
Acknowledgments
The authors thank the staff, administration, and leadership at the Tomah and Iowa City VA Medical Centers for working with us on this project. They offer special thanks to Kevin Glenn, MD, MS, Ethan Kuperman, MD, MS, FHM, and Jennifer Chapin, MSN, RN, for sharing their expertise, and the telehealth team, including Nathaniel Samuelson, Angela McDowell, and Katrin Metcalf.
Disclaimer
The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
1. O’Connor A, Wellenius G. Rural-urban disparities in the prevalence of diabetes and coronary heart disease. Public Health. 2012;126(10):813-820. https://doi.org/10.1016/j.puhe.2012.05.029
2. Kaufman BG, Thomas SR, Randolph RK, et al. The rising rate of rural hospital closures. J Rural Health. 2016;32(1):35-43. https://doi.org/10.1111/jrh.12128
3. MacDowell M, Glasser M, Fitts M, Nielsen K, Hunsaker M. A national view of rural health workforce issues in the USA. Rural Remote Health. 2010;10(3):1531.
4. Joynt KE, Harris Y, Orav EJ, Jha AK. Quality of care and patient outcomes in critical access rural hospitals. JAMA. 2011;306(1):45-52. https://doi.org/10.1001/jama.2011.902
5. The Chartis Group. Chartis Center for Rural Health. The Rural Health Safety Net Under Pressure: Rural Hospital Vulnerability. Published February 2020. Accessed May 07, 2020. https://www.chartis.com/forum/wp-content/uploads/2020/02/CCRH_Vulnerability-Research_FiNAL-02.14.20.pdf
6. Miller KEM, James HJ, Holmes GM, Van Houtven CH. The effect of rural hospital closures on emergency medical service response and transport times. Health Serv Res. 2020;55(2):288-300. https://doi.org/10.1111/1475-6773.13254
7. Buchmueller TC, Jacobson M, Wold C. How far to the hospital? The effect of hospital closures on access to care. J Health Econ. 2006;25(4):740-761. https://doi.org/10.1016/j.jhealeco.2005.10.006
8. Mohr NM, Harland KK, Shane DM, et al. Rural patients with severe sepsis or septic shock who bypass rural hospitals have increased mortality: an instrumental variables approach. Crit Care Med. 2017;45(1):85-93. https://doi.org/10.1097/ccm.0000000000002026
9. Gutierrez J, Kuperman E, Kaboli PJ. Using telehealth as a tool for rural hospitals in the COVID-19 pandemic response. J Rural Health. 2020;10.1111/jrh.12443. https://doi.org/10.1111/jrh.12443
10. Kisuule F, Howell EE. Hospitalists and their impact on quality, patient safety, and satisfaction. Obstet Gynecol Clin North Am. 2015;42(3):433-446. https://doi.org/10.1016/j.ogc.2015.05.003
11. Peterson MC. A systematic review of outcomes and quality measures in adult patients cared for by hospitalists vs nonhospitalists. Mayo Clin Proc. 2009;84(3):248-254. https://doi.org/10.4065/84.3.248
12. Auerbach AD, Wachter RM, Katz P, Showstack J, Baron RB, Goldman L. Implementation of a voluntary hospitalist service at a community teaching hospital: improved clinical efficiency and patient outcomes. Ann Intern Med. 2002;137(11):859-865. https://doi.org/10.7326/0003-4819-137-11-200212030-00006
13. Casey MM, Hung P, Moscovice I, Prasad S. The use of hospitalists by small rural hospitals: results of a national survey. Med Care Res Rev. 2014;71(4):356-366. https://doi.org/10.1177/1077558714533822
14. Sanders RB, Simpson KN, Kazley AS, Giarrizzi DP. New hospital telemedicine services: potential market for a nighttime telehospitalist service. Telemed J E Health. 2014;20(10):902-908. https://doi.org/10.1089/tmj.2013.0344
15. Department of Veterans Affairs. Office of Inspector General. OIG Determination of Veterans Health Administration’s Occupational Staffing Shortages. Published September 30, 2019. Accessed June 15, 2020. https://www.va.gov/oig/pubs/VAOIG-19-00346-241.pdf
16. Gutierrez J, Moeckli J, McAdams N, Kaboli PJ. Perceptions of telehospitalist services to address staffing needs in rural and low complexity hospitals in the Veterans Health Administration. J Rural Health. 2019;36(3):355-359. https://doi.org/10.1111/jrh.12403
17. Eagle Telemedicine. EAGLE TELEMEDICINE NIGHT COVERAGE SOLUTIONS: Why They Work for Hospitals and Physicians. Accessed May 28, 2018. http://www.eagletelemedicine.com/wp-content/uploads/2016/11/EHP_WP_Telenocturnist_FINAL.pdf
18. Gujral J, Antoine C, Chandra S. The role of telehospitalist in COVID-19 response: Hospitalist caring remotely for New York patients explain their role. ACP Hospitalist. 2020; May 2020.
19. Kuperman EF, Linson EL, Klefstad K, Perry E, Glenn K. The virtual hospitalist: a single-site implementation bringing hospitalist coverage to critical access hospitals. J Hosp Med. 2018;13(11):759-763. https://doi.org/10.12788/jhm.3061
20. JaKa MM, Dinh JM, Ziegenfuss JY, et al. Patient and care team perspectives of telemedicine in critical access hospitals. J Hosp Med. 2020;15(6):345-348. https://doi.org/10.12788/jhm.3412
21. Khanal S, Burgon J, Leonard S, Griffiths M, Eddowes LA. Recommendations for the improved effectiveness and reporting of telemedicine programs in developing countries: results of a systematic literature review. Telemed J E Health. 2015;21(11):903-915. https://doi.org/10.1089/tmj.2014.0194
22. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016;25(12):986-992. https://doi.org/10.1136/bmjqs-2015-004411
23. VHA Office of Rural Health. ORH 2020-2024 STRATEGIC PLAN. In: U.S. Department of Veterans Affairs, ed 2020. Accessed January 18, 2021 https://www.ruralhealth.va.gov/aboutus/index.asp
24. Veterans Health Administration. About VHA. In: U.S. Department of Veterans Affairs, ed. 2019. Accessed January 18, 2021.https://www.va.gov/health/aboutvha.asp
25. GeoSpatial Outcomes Division. VHA Office of Rural Health. U.S. Department of Veterans Affairs. Rural Veterans Health Care Atlas. 2nd ed - FY-2015. Accessed July 30, 2020. https://www.ruralhealth.va.gov/docs/atlas/CHAPTER_02_RHRI_Pts_treated_at_VAMCs.pdf
26. Wagner TH, Upadhyay A, Cowgill E, et al. Risk adjustment tools for learning health systems: a comparison of DxCG and CMS-HCC V21. Health Serv Res. 2016;51(5):2002-2019. https://doi.org/10.1111/1475-6773.12454
27. Wagner T, Stefos T, Moran E, et al. Technical Report 30: Risk Adjustment: Guide to the V21 and Nosos Risk Score Programs. Updated February 8, 2016. Accessed July 30, 2020. https://www.herc.research.va.gov/include/page.asp?id=technical-report-risk-adjustment
28. The R Foundation. The R Project for Statistical Computing. Accessed August 10, 2020. https://www.R-project.org/
29. Cleary PD, Meterko M, Wright SM, Zaslavsky AM. Are comparisons of patient experiences across hospitals fair? A study in Veterans Health Administration hospitals. Med Care. 2014;52(7):619-625. https://doi.org/10.1097/mlr.0000000000000144
30. Giordano LA, Elliott MN, Goldstein E, Lehrman WG, Spencer PA. Development, implementation, and public reporting of the HCAHPS survey. Med Care Res Rev. 2010;67(1):27-37. doi:10.1177/1077558709341065
31. Boltz M, Cuellar NG, Cole C, Pistorese B. Comparing an on-site nurse practitioner with telemedicine physician support hospitalist programme with a traditional physician hospitalist programme. J Telemed and Telecare. 2019;25(4):213-220. https://doi.org/10.1177%2F1357633X18758744
32. Quinn R. The pros and cons of locum tenens for hospitalists. The Hospitalist. 2012(12). Accessed May 29, 2018. https://www.the-hospitalist.org/hospitalist/article/124988/pros-and-cons-locum-tenens-hospitalists
33. Blumenthal DM, Olenski AR, Tsugawa Y, Jena AB. Association between treatment by locum tenens internal medicine physicians and 30-day mortality among hospitalized Medicare beneficiaries. JAMA. 2017;318(21):2119-2129. https://doi.org/10.1001/jama.2017.17925
34. Butcher L. Nurses as hospitalists | AHA Trustee Services. American Hospital Association. Accessed July 14, 2020 https://trustees.aha.org/articles/1238-nurses-as-hospitalists
35. Kartha A, Restuccia JD, Burgess JF, Jr, et al. Nurse practitioner and physician assistant scope of practice in 118 acute care hospitals. J Hosp Med. 2014;9(10):615-620. https://doi.org/10.1002/jhm.2231
Healthcare in rural areas faces increasing challenges due to community hospital closures, physician shortages, and a more concentrated population of older adults with higher rates of comorbid conditions than their urban counterparts.1-3 Critical access hospitals (CAHs), which primarily serve rural areas, have fewer clinical capabilities, worse process-of-care measures, and higher mortality rates for some conditions when compared to non-CAHs.4 As such, CAHs are closing at record numbers across the United States,5 resulting in loss of available hospital beds and patient access to timely emergency services,6 which can worsen outcomes, further widening the rural-urban healthcare gap.7,8 Furthermore, this strain on an overwhelmed health system in the most vulnerable areas restricts the ability to respond to healthcare crises like the coronavirus disease 2019 pandemic.9
Providing adequate staff for currently available hospital beds is also a problem in rural areas. Studies demonstrating improved outcomes, decreased length of stay (LOS), and increased quality with hospitalist services have resulted in a high demand for hospitalists nationwide.10-12 Recruiting hospitalists to work in rural areas, however, has become increasingly challenging due to low-patient volumes, financial viability of hospitalist-model adoption, and provider shortages.13,14 Recently, the Veterans Health Administration (VHA) reported a 28% nationwide shortage of hospitalists,15 which disproportionally affects rural VHA hospitals. Staffing difficulties and reliance on intermittent providers were reported by more than 80% of rural and low-complexity VHA facilities.16
Telehospitalist services (THS) can help deliver high-quality care to rural residents locally, decrease travel expenses, support hospital volume, and increase healthcare capacity in response to a pandemic.14,17,18 Only a few studies have described THS (mostly with overnight or cross-coverage models directed to CAHs), and clinical outcomes have been inconsistently reported.17,19-21 Furthermore, no program has been conducted within an integrated health system akin to the VHA. The primary objective of this quality improvement (QI) initiative was to perform a mixed-methods evaluation of THS between VHA hospitals to compare clinical outcomes and patient and staff satisfaction. Secondary outcomes included description of the implementation process, unexpected challenges, and subsequent QI initiatives. These results will expand the knowledge on feasibility of THS and provide implementation guidance.
METHODS
A mixed-methods approach was used to evaluate outcomes of this QI project. Reporting follows the revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0).22
Context
The VHA is the largest integrated healthcare system in the United States, with more than 8 million veterans enrolled, more than 30% of whom reside in a rural area. The VHA comprises more than 1,000 outpatient clinics and 170 acute care VA Medical Centers,23,24 including more than 35 rural and low-complexity hospitals.25 Low-complexity hospitals are those with the lowest volume and levels of patient complexity and minimal or no teaching programs, research, intensive care unit (ICU) beds, and subspecialists. Lack of reimbursement and interstate licensing, often cited as barriers to telemedicine, do not apply to the VHA. The hub site was a large tertiary care (high-complexity) VHA hospital located in Iowa City, Iowa. The spoke site was a low-complexity (10-bed acute inpatient unit with no ICU) rural VA hospital located in Tomah, Wisconsin.
Study Population
The preimplementation cohort for comparison included all patients admitted between January 1, 2018, and January 6, 2019. The postimplementation study cohort included all observation and acute care admissions during the pilot phase (January 7 to May 3, 2019) and sustainability phase (July 15 to December 31, 2019). The postimplementation analysis excluded the time period of May 4 to July 14, 2019, due to an interruption (gap) in THS. The gap period allowed for preliminary data analysis, optimization of the telecommunication system, and the recruitment and training of additional providers who could provide long-term staffing to the service.
Intervention
Preimplementation
Prior to THS implementation, Tomah’s inpatient ward was staffed by one physician per shift, who could be a hospitalist, medical officer of the day (MOD), or an intermittent provider (locum tenens). Hospitalists covering the acute inpatient ward prior to the THS transitioned to cover weekends, nights, and urgent care service shifts.
We visited the spoke site and held information-sharing sessions with key stakeholders (administrators, clinician leaders, nurses, and ancillary staff) prior to kick-off. Recurrent phone meetings addressed anticipated and emerging challenges. Telehospitalist and local providers underwent technology and service training.
Technology and Connectivity
A low-cost technology system using tablet computers provided Health Insurance Portability and Accountability Act–compliant videoconferencing with a telehospitalist at the hub site. An Eko-Core digital stethoscope® with a web-based audio stream was available. Telehospitalists conducted encounters from a private office space with telehealth capabilities. A total of $9,000 was spent on equipment at both sites. Due to connectivity problems and data limits, the tablets were switched to mobile computer-on-wheels workstations and hospital-based Wi-Fi for the sustainability phase.
THS Description
An experienced hub hospitalist, together with an advanced practice provider (APP; nurse practitioner [NP] or physician assistant [PA]), cared for all patients admitted to the 10-bed inpatient unit at the spoke site, Monday through Friday from 8:00 AM to 4:30 PM. The APP had limited or no prior experience in acute inpatient medicine. The telehospitalist worked as a team with the APP. The APP was the main point of contact for nurses, performed physical examinations, and directed patient care to their level of comfort (in a similar manner as a teaching team). The telehospitalist conducted bedside patient rounds, participated in multidisciplinary huddles, and shared clinical documentation and administrative duties with the APP. The telehospitalist was the primary staff for admitted patients and had full access to the electronic health record (EHR). The THS was staffed by 10 hospitalists during the study period. Overnight and weekend cross-coverage and admissions were performed by MODs, who also covered the urgent care and cross-covered other nonmedical units.
Quantitative Evaluation Methods
Workload and Clinical Outcomes
An EHR query identified all patients admitted during the pre- and postimplementation periods. Demographic data, clinical Nosos risk scores,26,27 and top admission diagnoses were reported. Workload was evaluated using the average number of encounters per day and self-reported telehospitalist worksheets, which were cross-referenced with EHR data. Clinical outcomes included LOS, 30-day hospital readmission rate, 30-day standardized mortality (SMR30), in-hospital mortality, and VHA-specific inpatient quality metrics. Independent sample t tests for continuous variables and chi-square tests or Fisher’s exact test (for patient class) for categorical variables were used to compare pre- and postimplementation groups. Statistical process control (SPC) charts evaluated changes over time. All analyses were conducted using Microsoft Excel and R.28
Anonymous surveys were distributed to spoke-site inpatient and administrative staff at 1 month and 12 months postimplementation, assessing satisfaction, technology/connectivity, communication, and challenges (Appendix Figure 1). Satisfaction of the telehospitalist physicians at the hub site was measured 12 months postimplementation by a 26-question survey assessing the same domains, plus quality of care (Appendix Figure 2).
Patient Satisfaction
The VHA Survey of Healthcare Experiences of Patients (SHEP), a version of the Hospital Consumer Assessment of Healthcare Providers and Systems Survey,29,30 was mailed to all patients after discharge. Survey responses concerning inpatient provider care (eg, care coordination, communication, hospital rating, willingness to recommend the hospital) during the pre- and postimplementation phases were compared using a two-sample test of independent proportions. Responses obtained during May and June 2019 were excluded.
Qualitative Evaluation Methods
The authors had full access to, and took full responsibility for, the integrity of the data. The project was evaluated by the University of Iowa Institutional Review Board and the Iowa City VA Research and Development Committee and was determined to be a non–human-subjects QI project.
RESULTS
Quantitative Workload and Clinical Outcomes
There were 822 admissions during the preimplementation period and 550 admissions during the postimplementation period (253 during the pilot and 297 during sustainability phase). Patient characteristics pre- and postimplementation were not significantly different (Table 1). The median patient age was 65 years; 96% of patients were male, and 83% were rural residents. The most common admission diagnosis was alcohol-related (36%); regarding patient disposition, 78% of admissions were discharged home.
Workload
There were 502 patient encounters staffed by the telehospitalist in the pilot phase, with an average of 6.25 encounters per day, and a telehospitalist-reported workload of 7 hours per day. There were 538 patient encounters, with an average of 4.67 encounters per day and a workload of 5.6 hours per day in the sustainability phase. The average daily census decreased from 5.0 (SD, 1.1) patients per day during preimplementation to 3.1 (SD, 0.5) patients per day during postimplementation (Table 2). In some of the months during the study period, admissions decreased below the lower SPC limit, suggesting a significant change (Figure). Adjusted LOS was significantly lower, with 3.0 (SD, 0.7) days vs 2.3 (SD, 0.3) days in the pre- and postimplementation periods, respectively. Bed occupancy rates were significantly lower in the sustainability phase compared with the pilot phase and the preimplementation period. Readmission rates varied, ranging from <10% to >30%, not significantly different but slightly higher in the postimplementation period. Readmission rates for heart failure, chronic obstructive pulmonary disease, and pneumonia remained unchanged; other medical readmissions (mostly alcohol-related) were slightly higher in the postimplementation period.
In-hospital mortality and SMR30 did not change significantly, but there was improvement in the 12-month rolling average of the observed/expected SMR30 from 1.40 to 1.08. Additional VHA-specific quality metrics were monitored and showed either small improvements or no change (data not shown).
Satisfaction at Hub and Spoke Sites
After sending two reminder communications via email, the telehospitalist satisfaction survey had a total response rate of 90% (9/10). Telehospitalists were satisfied or very satisfied (89%) with the program and the local providers (88.9%), rating their experience as good or excellent (100%) (Table 3). Communication with patients, families, and local staff was noted as being “positive” or “mostly positive.” Telehospitalists reported confidence in the accuracy of their diagnoses and rated the quality of care as being equal to that of a face-to-face encounter. Connectivity problems were prevalent, although most providers were able to resort to a back-up plan. Other challenges included differences in culture and concerns about liability. We received 27 responses from the spoke-site satisfaction survey; the response rate could not be determined because the survey was distributed by the spoke site for anonymity. Of the respondents, 37% identified as nurses, 25.9% as healthcare providers (APPs or physicians), and 33.3% as other staff (eg, social worker, nutritionist, physical therapist, utilization management, administrators); 3.7% did not respond. Among the participants, 88% had personally interacted with the THS. Most providers and other staff perceived THS as valuable (57.1% and 77.8%, respectively) and were satisfied or highly satisfied with THS (57.1% and 55.6%, respectively). On average, nurses provided lower ratings across all survey items than providers and other staff. Challenges noted by all staff included issues with communication, workflow, and technology/connectivity.
Qualitative Strengths
Our process evaluation identified high quality of care and teamwork as contributors to the success of the program. Overall, staff credited perceived improvements in quality of care to the quality of providers staffing the THS, including the local APPs. Noting the telehospitalists’ knowledge base and level of engagement as key attributes, one staff member commented: “I prefer a telehospitalist that really care[s] about patients than some provider that is physically here but does not engage.” Staff perceived improvements in the continuity of care, as well as care processes such as handoffs and transitions of care.
Improvements in teamwork were perceived compared with the previous model of care. Telehospitalists were lauded for their professionalism and communication skills. Overall, nurses felt providers in the THS listened more to their views. In addition, nurse respondents felt they could learn from several providers and said they enjoyed the telehospitalists’ disposition to teach and discuss patient care. The responsiveness of the THS staff was instrumental in building teamwork and acceptance. A bedside interdisciplinary protocol was established for appropriate patients. Local staff felt this was crucial for teamwork and patient satisfaction. Telehospitalists reported high-value in interdisciplinary rounds, facilitating interaction with nurses and ancillary staff. Handoff problems were identified, leading to QI initiatives to mitigate those issues.
Challenges
The survey identified administrative barriers, technical difficulties, workflow constraints, and clinical concerns. The credentialing process was complicated, delaying the onboarding of telehospitalists. Internet connectivity was inconsistent, leading to disruption in video communications; however, during the sustainability phase, updated technology improved communications. The communication workflow was resisted by some nurses, who wanted to phone the telehospitalist directly rather than having the local APP as the first contact. Secure messaging was enabled to allow nurses direct contact during the sustainability phase.
Workload was a concern among telehospitalists and local staff. Telehospitalists perceived the documentation requirements and administrative workload to be two to three times higher than at other hospitals—despite the lower number of encounters. Finally, clinical concerns from spoke-site clinicians included a perceived rise in the acuity of patients (which was not evident by the Nosos score) and delayed decisions to transfer-out patients. These concerns were addressed with educational sessions for telehospitalists during the sustainability phase.
Additional Quality Improvement Projects
The implementation of THS resulted in QI initiatives at the spoke site, including an EHR-integrated handoff tool; a documentation evaluation that led to the elimination of duplicative, inefficient, and error-prone templates; and a revision of the alcohol withdrawal treatment protocol during the sustainability phase to reduce the use of intravenous benzodiazepines. A more comprehensive benzodiazepine-sparing alcohol withdrawal treatment protocol was also developed but was not implemented until after the study period (January 2020).
DISCUSSION
Our pre-post study evaluation found implementation of a THS to be noninferior to face-to-face care, with no significant change in mortality, readmission rate, or patient satisfaction. The significant improvement observed in LOS is consistent with the adoption of hospitalist models in other medical care settings,11 but had not been reported by previous telehospitalist studies. For example, in their retrospective chart review comparing an NP-supported telehospitalist model to locum tenens hospitalists, Boltz et al found no difference in LOS.31 Moreover, as in our study, they found no differences in readmissions, mortality, and patient satisfaction.31 Similarly, Kuperman et al reported unchanged daily census, LOS, and transfer rates from a CAH with their virtual hospitalist program, but a decrease in the percentage of patients transferred-out from the emergency department, suggesting that more patients were treated locally.19
Reduction in LOS is one of the primary measures of efficiency in hospital care31; reducing LOS while maintaining the quality of care lowers hospital costs. The reduction in LOS in our study could be attributed to greater continuity of care, engagement/experience of the telehospitalists, or other factors. This decrease in LOS and slight reduction in admissions resulted in an overall lower daily census during the study period and impacted efficiency. Our study was unable to determine the cause for the reduction in admissions; however, several concurrent events, including the expansion of community-care options for veterans under the MISSION ACT (Maintaining Internal Systems and Strengthening Integrated Outside Networks Act) in June 2019, a nationwide smoking ban at VA facilities (October 2019), and a modification in the alcohol withdrawal treatment protocol might have influenced veterans’ choice of hospital.
Readmission rates were slightly higher, though nonsignificant, in the postimplementation period. Alcohol-related readmissions accounted for most readmissions; some of the protocol changes, such as admitting all patients with alcohol withdrawal to inpatient class instead of admitting some to the observation class, accounted for part of the increase in readmission rates. Readmission rates for other conditions such as chronic obstructive pulmonary disease, chronic heart failure, or pneumonia were not significantly different, suggesting that the reduction in LOS did not result in an unintended increased readmission rate for those conditions.
Rural hospitals are struggling with staffing and finances. Resorting to locum tenens staffing is costly and can result in variable quality of care.32,33 APPs are increasingly taking on hospitalist positions, with 65% of adult hospitalist programs, including half of all VHA hospitals, employing NPs and PAs.34,35 In response to this expanded scope of practice, hospitals employing APPs in hospitalist roles must comply with state and federal laws, which often require that APPs be supervised by or work in collaboration with an on-site or off-site physician. The THS is a great model to support APPs and address staffing and cost challenges in low-volume rural facilities, while maintaining quality of care. Some APP-telehospitalist programs similar to ours have reported cost reductions of up 58% compared to programs that employ locum tenens physicians.31 In our model, we assume that a single telehospitalist hub could provide coverage to two or three spoke sites with APP support, reducing staffing costs.
Hub telehospitalists reported satisfaction with the program, and they perceived the quality of care to be comparable to face-to-face encounters; their responses were consistent with those previously reported in an evaluation of telemedicine acute care by JaKa et al.
This study has several limitations. First, the VHA is an integrated health system, one that serves an older, predominantly male patient population. Also, the lack of reimbursement and interstate licensing restrictions limit generalizability of these results to other CAHs or healthcare systems. Furthermore, the intervention was limited to a single rural site; while this allowed for a detailed evaluation, unique barriers or facilitators might exist that limit its applicability. In addition, QI initiatives implemented by the VHA during the project period might have confounded some of our results. Last, patient satisfaction survey data are overall limited in their ability to fully assess patient’s experience and satisfaction with the program. Further qualitative studies are needed to gain deeper insight into patient perspectives with the THS and whether modality of care delivery influences patients’ care decisions. Future studies should consider a multisite design with one or more hubs and multiple spoke sites.
CONCLUSION
Telehospitalist services are a feasible and safe approach to provide inpatient services and address staffing needs of rural hospitals. To enhance program performance, it is essential to ensure adequate technological quality, clearly delineate and define roles and responsibilities of the care team, and address communication issues or staff concerns in a timely manner.
Acknowledgments
The authors thank the staff, administration, and leadership at the Tomah and Iowa City VA Medical Centers for working with us on this project. They offer special thanks to Kevin Glenn, MD, MS, Ethan Kuperman, MD, MS, FHM, and Jennifer Chapin, MSN, RN, for sharing their expertise, and the telehealth team, including Nathaniel Samuelson, Angela McDowell, and Katrin Metcalf.
Disclaimer
The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
Healthcare in rural areas faces increasing challenges due to community hospital closures, physician shortages, and a more concentrated population of older adults with higher rates of comorbid conditions than their urban counterparts.1-3 Critical access hospitals (CAHs), which primarily serve rural areas, have fewer clinical capabilities, worse process-of-care measures, and higher mortality rates for some conditions when compared to non-CAHs.4 As such, CAHs are closing at record numbers across the United States,5 resulting in loss of available hospital beds and patient access to timely emergency services,6 which can worsen outcomes, further widening the rural-urban healthcare gap.7,8 Furthermore, this strain on an overwhelmed health system in the most vulnerable areas restricts the ability to respond to healthcare crises like the coronavirus disease 2019 pandemic.9
Providing adequate staff for currently available hospital beds is also a problem in rural areas. Studies demonstrating improved outcomes, decreased length of stay (LOS), and increased quality with hospitalist services have resulted in a high demand for hospitalists nationwide.10-12 Recruiting hospitalists to work in rural areas, however, has become increasingly challenging due to low-patient volumes, financial viability of hospitalist-model adoption, and provider shortages.13,14 Recently, the Veterans Health Administration (VHA) reported a 28% nationwide shortage of hospitalists,15 which disproportionally affects rural VHA hospitals. Staffing difficulties and reliance on intermittent providers were reported by more than 80% of rural and low-complexity VHA facilities.16
Telehospitalist services (THS) can help deliver high-quality care to rural residents locally, decrease travel expenses, support hospital volume, and increase healthcare capacity in response to a pandemic.14,17,18 Only a few studies have described THS (mostly with overnight or cross-coverage models directed to CAHs), and clinical outcomes have been inconsistently reported.17,19-21 Furthermore, no program has been conducted within an integrated health system akin to the VHA. The primary objective of this quality improvement (QI) initiative was to perform a mixed-methods evaluation of THS between VHA hospitals to compare clinical outcomes and patient and staff satisfaction. Secondary outcomes included description of the implementation process, unexpected challenges, and subsequent QI initiatives. These results will expand the knowledge on feasibility of THS and provide implementation guidance.
METHODS
A mixed-methods approach was used to evaluate outcomes of this QI project. Reporting follows the revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0).22
Context
The VHA is the largest integrated healthcare system in the United States, with more than 8 million veterans enrolled, more than 30% of whom reside in a rural area. The VHA comprises more than 1,000 outpatient clinics and 170 acute care VA Medical Centers,23,24 including more than 35 rural and low-complexity hospitals.25 Low-complexity hospitals are those with the lowest volume and levels of patient complexity and minimal or no teaching programs, research, intensive care unit (ICU) beds, and subspecialists. Lack of reimbursement and interstate licensing, often cited as barriers to telemedicine, do not apply to the VHA. The hub site was a large tertiary care (high-complexity) VHA hospital located in Iowa City, Iowa. The spoke site was a low-complexity (10-bed acute inpatient unit with no ICU) rural VA hospital located in Tomah, Wisconsin.
Study Population
The preimplementation cohort for comparison included all patients admitted between January 1, 2018, and January 6, 2019. The postimplementation study cohort included all observation and acute care admissions during the pilot phase (January 7 to May 3, 2019) and sustainability phase (July 15 to December 31, 2019). The postimplementation analysis excluded the time period of May 4 to July 14, 2019, due to an interruption (gap) in THS. The gap period allowed for preliminary data analysis, optimization of the telecommunication system, and the recruitment and training of additional providers who could provide long-term staffing to the service.
Intervention
Preimplementation
Prior to THS implementation, Tomah’s inpatient ward was staffed by one physician per shift, who could be a hospitalist, medical officer of the day (MOD), or an intermittent provider (locum tenens). Hospitalists covering the acute inpatient ward prior to the THS transitioned to cover weekends, nights, and urgent care service shifts.
We visited the spoke site and held information-sharing sessions with key stakeholders (administrators, clinician leaders, nurses, and ancillary staff) prior to kick-off. Recurrent phone meetings addressed anticipated and emerging challenges. Telehospitalist and local providers underwent technology and service training.
Technology and Connectivity
A low-cost technology system using tablet computers provided Health Insurance Portability and Accountability Act–compliant videoconferencing with a telehospitalist at the hub site. An Eko-Core digital stethoscope® with a web-based audio stream was available. Telehospitalists conducted encounters from a private office space with telehealth capabilities. A total of $9,000 was spent on equipment at both sites. Due to connectivity problems and data limits, the tablets were switched to mobile computer-on-wheels workstations and hospital-based Wi-Fi for the sustainability phase.
THS Description
An experienced hub hospitalist, together with an advanced practice provider (APP; nurse practitioner [NP] or physician assistant [PA]), cared for all patients admitted to the 10-bed inpatient unit at the spoke site, Monday through Friday from 8:00 AM to 4:30 PM. The APP had limited or no prior experience in acute inpatient medicine. The telehospitalist worked as a team with the APP. The APP was the main point of contact for nurses, performed physical examinations, and directed patient care to their level of comfort (in a similar manner as a teaching team). The telehospitalist conducted bedside patient rounds, participated in multidisciplinary huddles, and shared clinical documentation and administrative duties with the APP. The telehospitalist was the primary staff for admitted patients and had full access to the electronic health record (EHR). The THS was staffed by 10 hospitalists during the study period. Overnight and weekend cross-coverage and admissions were performed by MODs, who also covered the urgent care and cross-covered other nonmedical units.
Quantitative Evaluation Methods
Workload and Clinical Outcomes
An EHR query identified all patients admitted during the pre- and postimplementation periods. Demographic data, clinical Nosos risk scores,26,27 and top admission diagnoses were reported. Workload was evaluated using the average number of encounters per day and self-reported telehospitalist worksheets, which were cross-referenced with EHR data. Clinical outcomes included LOS, 30-day hospital readmission rate, 30-day standardized mortality (SMR30), in-hospital mortality, and VHA-specific inpatient quality metrics. Independent sample t tests for continuous variables and chi-square tests or Fisher’s exact test (for patient class) for categorical variables were used to compare pre- and postimplementation groups. Statistical process control (SPC) charts evaluated changes over time. All analyses were conducted using Microsoft Excel and R.28
Anonymous surveys were distributed to spoke-site inpatient and administrative staff at 1 month and 12 months postimplementation, assessing satisfaction, technology/connectivity, communication, and challenges (Appendix Figure 1). Satisfaction of the telehospitalist physicians at the hub site was measured 12 months postimplementation by a 26-question survey assessing the same domains, plus quality of care (Appendix Figure 2).
Patient Satisfaction
The VHA Survey of Healthcare Experiences of Patients (SHEP), a version of the Hospital Consumer Assessment of Healthcare Providers and Systems Survey,29,30 was mailed to all patients after discharge. Survey responses concerning inpatient provider care (eg, care coordination, communication, hospital rating, willingness to recommend the hospital) during the pre- and postimplementation phases were compared using a two-sample test of independent proportions. Responses obtained during May and June 2019 were excluded.
Qualitative Evaluation Methods
The authors had full access to, and took full responsibility for, the integrity of the data. The project was evaluated by the University of Iowa Institutional Review Board and the Iowa City VA Research and Development Committee and was determined to be a non–human-subjects QI project.
RESULTS
Quantitative Workload and Clinical Outcomes
There were 822 admissions during the preimplementation period and 550 admissions during the postimplementation period (253 during the pilot and 297 during sustainability phase). Patient characteristics pre- and postimplementation were not significantly different (Table 1). The median patient age was 65 years; 96% of patients were male, and 83% were rural residents. The most common admission diagnosis was alcohol-related (36%); regarding patient disposition, 78% of admissions were discharged home.
Workload
There were 502 patient encounters staffed by the telehospitalist in the pilot phase, with an average of 6.25 encounters per day, and a telehospitalist-reported workload of 7 hours per day. There were 538 patient encounters, with an average of 4.67 encounters per day and a workload of 5.6 hours per day in the sustainability phase. The average daily census decreased from 5.0 (SD, 1.1) patients per day during preimplementation to 3.1 (SD, 0.5) patients per day during postimplementation (Table 2). In some of the months during the study period, admissions decreased below the lower SPC limit, suggesting a significant change (Figure). Adjusted LOS was significantly lower, with 3.0 (SD, 0.7) days vs 2.3 (SD, 0.3) days in the pre- and postimplementation periods, respectively. Bed occupancy rates were significantly lower in the sustainability phase compared with the pilot phase and the preimplementation period. Readmission rates varied, ranging from <10% to >30%, not significantly different but slightly higher in the postimplementation period. Readmission rates for heart failure, chronic obstructive pulmonary disease, and pneumonia remained unchanged; other medical readmissions (mostly alcohol-related) were slightly higher in the postimplementation period.
In-hospital mortality and SMR30 did not change significantly, but there was improvement in the 12-month rolling average of the observed/expected SMR30 from 1.40 to 1.08. Additional VHA-specific quality metrics were monitored and showed either small improvements or no change (data not shown).
Satisfaction at Hub and Spoke Sites
After sending two reminder communications via email, the telehospitalist satisfaction survey had a total response rate of 90% (9/10). Telehospitalists were satisfied or very satisfied (89%) with the program and the local providers (88.9%), rating their experience as good or excellent (100%) (Table 3). Communication with patients, families, and local staff was noted as being “positive” or “mostly positive.” Telehospitalists reported confidence in the accuracy of their diagnoses and rated the quality of care as being equal to that of a face-to-face encounter. Connectivity problems were prevalent, although most providers were able to resort to a back-up plan. Other challenges included differences in culture and concerns about liability. We received 27 responses from the spoke-site satisfaction survey; the response rate could not be determined because the survey was distributed by the spoke site for anonymity. Of the respondents, 37% identified as nurses, 25.9% as healthcare providers (APPs or physicians), and 33.3% as other staff (eg, social worker, nutritionist, physical therapist, utilization management, administrators); 3.7% did not respond. Among the participants, 88% had personally interacted with the THS. Most providers and other staff perceived THS as valuable (57.1% and 77.8%, respectively) and were satisfied or highly satisfied with THS (57.1% and 55.6%, respectively). On average, nurses provided lower ratings across all survey items than providers and other staff. Challenges noted by all staff included issues with communication, workflow, and technology/connectivity.
Qualitative Strengths
Our process evaluation identified high quality of care and teamwork as contributors to the success of the program. Overall, staff credited perceived improvements in quality of care to the quality of providers staffing the THS, including the local APPs. Noting the telehospitalists’ knowledge base and level of engagement as key attributes, one staff member commented: “I prefer a telehospitalist that really care[s] about patients than some provider that is physically here but does not engage.” Staff perceived improvements in the continuity of care, as well as care processes such as handoffs and transitions of care.
Improvements in teamwork were perceived compared with the previous model of care. Telehospitalists were lauded for their professionalism and communication skills. Overall, nurses felt providers in the THS listened more to their views. In addition, nurse respondents felt they could learn from several providers and said they enjoyed the telehospitalists’ disposition to teach and discuss patient care. The responsiveness of the THS staff was instrumental in building teamwork and acceptance. A bedside interdisciplinary protocol was established for appropriate patients. Local staff felt this was crucial for teamwork and patient satisfaction. Telehospitalists reported high-value in interdisciplinary rounds, facilitating interaction with nurses and ancillary staff. Handoff problems were identified, leading to QI initiatives to mitigate those issues.
Challenges
The survey identified administrative barriers, technical difficulties, workflow constraints, and clinical concerns. The credentialing process was complicated, delaying the onboarding of telehospitalists. Internet connectivity was inconsistent, leading to disruption in video communications; however, during the sustainability phase, updated technology improved communications. The communication workflow was resisted by some nurses, who wanted to phone the telehospitalist directly rather than having the local APP as the first contact. Secure messaging was enabled to allow nurses direct contact during the sustainability phase.
Workload was a concern among telehospitalists and local staff. Telehospitalists perceived the documentation requirements and administrative workload to be two to three times higher than at other hospitals—despite the lower number of encounters. Finally, clinical concerns from spoke-site clinicians included a perceived rise in the acuity of patients (which was not evident by the Nosos score) and delayed decisions to transfer-out patients. These concerns were addressed with educational sessions for telehospitalists during the sustainability phase.
Additional Quality Improvement Projects
The implementation of THS resulted in QI initiatives at the spoke site, including an EHR-integrated handoff tool; a documentation evaluation that led to the elimination of duplicative, inefficient, and error-prone templates; and a revision of the alcohol withdrawal treatment protocol during the sustainability phase to reduce the use of intravenous benzodiazepines. A more comprehensive benzodiazepine-sparing alcohol withdrawal treatment protocol was also developed but was not implemented until after the study period (January 2020).
DISCUSSION
Our pre-post study evaluation found implementation of a THS to be noninferior to face-to-face care, with no significant change in mortality, readmission rate, or patient satisfaction. The significant improvement observed in LOS is consistent with the adoption of hospitalist models in other medical care settings,11 but had not been reported by previous telehospitalist studies. For example, in their retrospective chart review comparing an NP-supported telehospitalist model to locum tenens hospitalists, Boltz et al found no difference in LOS.31 Moreover, as in our study, they found no differences in readmissions, mortality, and patient satisfaction.31 Similarly, Kuperman et al reported unchanged daily census, LOS, and transfer rates from a CAH with their virtual hospitalist program, but a decrease in the percentage of patients transferred-out from the emergency department, suggesting that more patients were treated locally.19
Reduction in LOS is one of the primary measures of efficiency in hospital care31; reducing LOS while maintaining the quality of care lowers hospital costs. The reduction in LOS in our study could be attributed to greater continuity of care, engagement/experience of the telehospitalists, or other factors. This decrease in LOS and slight reduction in admissions resulted in an overall lower daily census during the study period and impacted efficiency. Our study was unable to determine the cause for the reduction in admissions; however, several concurrent events, including the expansion of community-care options for veterans under the MISSION ACT (Maintaining Internal Systems and Strengthening Integrated Outside Networks Act) in June 2019, a nationwide smoking ban at VA facilities (October 2019), and a modification in the alcohol withdrawal treatment protocol might have influenced veterans’ choice of hospital.
Readmission rates were slightly higher, though nonsignificant, in the postimplementation period. Alcohol-related readmissions accounted for most readmissions; some of the protocol changes, such as admitting all patients with alcohol withdrawal to inpatient class instead of admitting some to the observation class, accounted for part of the increase in readmission rates. Readmission rates for other conditions such as chronic obstructive pulmonary disease, chronic heart failure, or pneumonia were not significantly different, suggesting that the reduction in LOS did not result in an unintended increased readmission rate for those conditions.
Rural hospitals are struggling with staffing and finances. Resorting to locum tenens staffing is costly and can result in variable quality of care.32,33 APPs are increasingly taking on hospitalist positions, with 65% of adult hospitalist programs, including half of all VHA hospitals, employing NPs and PAs.34,35 In response to this expanded scope of practice, hospitals employing APPs in hospitalist roles must comply with state and federal laws, which often require that APPs be supervised by or work in collaboration with an on-site or off-site physician. The THS is a great model to support APPs and address staffing and cost challenges in low-volume rural facilities, while maintaining quality of care. Some APP-telehospitalist programs similar to ours have reported cost reductions of up 58% compared to programs that employ locum tenens physicians.31 In our model, we assume that a single telehospitalist hub could provide coverage to two or three spoke sites with APP support, reducing staffing costs.
Hub telehospitalists reported satisfaction with the program, and they perceived the quality of care to be comparable to face-to-face encounters; their responses were consistent with those previously reported in an evaluation of telemedicine acute care by JaKa et al.
This study has several limitations. First, the VHA is an integrated health system, one that serves an older, predominantly male patient population. Also, the lack of reimbursement and interstate licensing restrictions limit generalizability of these results to other CAHs or healthcare systems. Furthermore, the intervention was limited to a single rural site; while this allowed for a detailed evaluation, unique barriers or facilitators might exist that limit its applicability. In addition, QI initiatives implemented by the VHA during the project period might have confounded some of our results. Last, patient satisfaction survey data are overall limited in their ability to fully assess patient’s experience and satisfaction with the program. Further qualitative studies are needed to gain deeper insight into patient perspectives with the THS and whether modality of care delivery influences patients’ care decisions. Future studies should consider a multisite design with one or more hubs and multiple spoke sites.
CONCLUSION
Telehospitalist services are a feasible and safe approach to provide inpatient services and address staffing needs of rural hospitals. To enhance program performance, it is essential to ensure adequate technological quality, clearly delineate and define roles and responsibilities of the care team, and address communication issues or staff concerns in a timely manner.
Acknowledgments
The authors thank the staff, administration, and leadership at the Tomah and Iowa City VA Medical Centers for working with us on this project. They offer special thanks to Kevin Glenn, MD, MS, Ethan Kuperman, MD, MS, FHM, and Jennifer Chapin, MSN, RN, for sharing their expertise, and the telehealth team, including Nathaniel Samuelson, Angela McDowell, and Katrin Metcalf.
Disclaimer
The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
1. O’Connor A, Wellenius G. Rural-urban disparities in the prevalence of diabetes and coronary heart disease. Public Health. 2012;126(10):813-820. https://doi.org/10.1016/j.puhe.2012.05.029
2. Kaufman BG, Thomas SR, Randolph RK, et al. The rising rate of rural hospital closures. J Rural Health. 2016;32(1):35-43. https://doi.org/10.1111/jrh.12128
3. MacDowell M, Glasser M, Fitts M, Nielsen K, Hunsaker M. A national view of rural health workforce issues in the USA. Rural Remote Health. 2010;10(3):1531.
4. Joynt KE, Harris Y, Orav EJ, Jha AK. Quality of care and patient outcomes in critical access rural hospitals. JAMA. 2011;306(1):45-52. https://doi.org/10.1001/jama.2011.902
5. The Chartis Group. Chartis Center for Rural Health. The Rural Health Safety Net Under Pressure: Rural Hospital Vulnerability. Published February 2020. Accessed May 07, 2020. https://www.chartis.com/forum/wp-content/uploads/2020/02/CCRH_Vulnerability-Research_FiNAL-02.14.20.pdf
6. Miller KEM, James HJ, Holmes GM, Van Houtven CH. The effect of rural hospital closures on emergency medical service response and transport times. Health Serv Res. 2020;55(2):288-300. https://doi.org/10.1111/1475-6773.13254
7. Buchmueller TC, Jacobson M, Wold C. How far to the hospital? The effect of hospital closures on access to care. J Health Econ. 2006;25(4):740-761. https://doi.org/10.1016/j.jhealeco.2005.10.006
8. Mohr NM, Harland KK, Shane DM, et al. Rural patients with severe sepsis or septic shock who bypass rural hospitals have increased mortality: an instrumental variables approach. Crit Care Med. 2017;45(1):85-93. https://doi.org/10.1097/ccm.0000000000002026
9. Gutierrez J, Kuperman E, Kaboli PJ. Using telehealth as a tool for rural hospitals in the COVID-19 pandemic response. J Rural Health. 2020;10.1111/jrh.12443. https://doi.org/10.1111/jrh.12443
10. Kisuule F, Howell EE. Hospitalists and their impact on quality, patient safety, and satisfaction. Obstet Gynecol Clin North Am. 2015;42(3):433-446. https://doi.org/10.1016/j.ogc.2015.05.003
11. Peterson MC. A systematic review of outcomes and quality measures in adult patients cared for by hospitalists vs nonhospitalists. Mayo Clin Proc. 2009;84(3):248-254. https://doi.org/10.4065/84.3.248
12. Auerbach AD, Wachter RM, Katz P, Showstack J, Baron RB, Goldman L. Implementation of a voluntary hospitalist service at a community teaching hospital: improved clinical efficiency and patient outcomes. Ann Intern Med. 2002;137(11):859-865. https://doi.org/10.7326/0003-4819-137-11-200212030-00006
13. Casey MM, Hung P, Moscovice I, Prasad S. The use of hospitalists by small rural hospitals: results of a national survey. Med Care Res Rev. 2014;71(4):356-366. https://doi.org/10.1177/1077558714533822
14. Sanders RB, Simpson KN, Kazley AS, Giarrizzi DP. New hospital telemedicine services: potential market for a nighttime telehospitalist service. Telemed J E Health. 2014;20(10):902-908. https://doi.org/10.1089/tmj.2013.0344
15. Department of Veterans Affairs. Office of Inspector General. OIG Determination of Veterans Health Administration’s Occupational Staffing Shortages. Published September 30, 2019. Accessed June 15, 2020. https://www.va.gov/oig/pubs/VAOIG-19-00346-241.pdf
16. Gutierrez J, Moeckli J, McAdams N, Kaboli PJ. Perceptions of telehospitalist services to address staffing needs in rural and low complexity hospitals in the Veterans Health Administration. J Rural Health. 2019;36(3):355-359. https://doi.org/10.1111/jrh.12403
17. Eagle Telemedicine. EAGLE TELEMEDICINE NIGHT COVERAGE SOLUTIONS: Why They Work for Hospitals and Physicians. Accessed May 28, 2018. http://www.eagletelemedicine.com/wp-content/uploads/2016/11/EHP_WP_Telenocturnist_FINAL.pdf
18. Gujral J, Antoine C, Chandra S. The role of telehospitalist in COVID-19 response: Hospitalist caring remotely for New York patients explain their role. ACP Hospitalist. 2020; May 2020.
19. Kuperman EF, Linson EL, Klefstad K, Perry E, Glenn K. The virtual hospitalist: a single-site implementation bringing hospitalist coverage to critical access hospitals. J Hosp Med. 2018;13(11):759-763. https://doi.org/10.12788/jhm.3061
20. JaKa MM, Dinh JM, Ziegenfuss JY, et al. Patient and care team perspectives of telemedicine in critical access hospitals. J Hosp Med. 2020;15(6):345-348. https://doi.org/10.12788/jhm.3412
21. Khanal S, Burgon J, Leonard S, Griffiths M, Eddowes LA. Recommendations for the improved effectiveness and reporting of telemedicine programs in developing countries: results of a systematic literature review. Telemed J E Health. 2015;21(11):903-915. https://doi.org/10.1089/tmj.2014.0194
22. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016;25(12):986-992. https://doi.org/10.1136/bmjqs-2015-004411
23. VHA Office of Rural Health. ORH 2020-2024 STRATEGIC PLAN. In: U.S. Department of Veterans Affairs, ed 2020. Accessed January 18, 2021 https://www.ruralhealth.va.gov/aboutus/index.asp
24. Veterans Health Administration. About VHA. In: U.S. Department of Veterans Affairs, ed. 2019. Accessed January 18, 2021.https://www.va.gov/health/aboutvha.asp
25. GeoSpatial Outcomes Division. VHA Office of Rural Health. U.S. Department of Veterans Affairs. Rural Veterans Health Care Atlas. 2nd ed - FY-2015. Accessed July 30, 2020. https://www.ruralhealth.va.gov/docs/atlas/CHAPTER_02_RHRI_Pts_treated_at_VAMCs.pdf
26. Wagner TH, Upadhyay A, Cowgill E, et al. Risk adjustment tools for learning health systems: a comparison of DxCG and CMS-HCC V21. Health Serv Res. 2016;51(5):2002-2019. https://doi.org/10.1111/1475-6773.12454
27. Wagner T, Stefos T, Moran E, et al. Technical Report 30: Risk Adjustment: Guide to the V21 and Nosos Risk Score Programs. Updated February 8, 2016. Accessed July 30, 2020. https://www.herc.research.va.gov/include/page.asp?id=technical-report-risk-adjustment
28. The R Foundation. The R Project for Statistical Computing. Accessed August 10, 2020. https://www.R-project.org/
29. Cleary PD, Meterko M, Wright SM, Zaslavsky AM. Are comparisons of patient experiences across hospitals fair? A study in Veterans Health Administration hospitals. Med Care. 2014;52(7):619-625. https://doi.org/10.1097/mlr.0000000000000144
30. Giordano LA, Elliott MN, Goldstein E, Lehrman WG, Spencer PA. Development, implementation, and public reporting of the HCAHPS survey. Med Care Res Rev. 2010;67(1):27-37. doi:10.1177/1077558709341065
31. Boltz M, Cuellar NG, Cole C, Pistorese B. Comparing an on-site nurse practitioner with telemedicine physician support hospitalist programme with a traditional physician hospitalist programme. J Telemed and Telecare. 2019;25(4):213-220. https://doi.org/10.1177%2F1357633X18758744
32. Quinn R. The pros and cons of locum tenens for hospitalists. The Hospitalist. 2012(12). Accessed May 29, 2018. https://www.the-hospitalist.org/hospitalist/article/124988/pros-and-cons-locum-tenens-hospitalists
33. Blumenthal DM, Olenski AR, Tsugawa Y, Jena AB. Association between treatment by locum tenens internal medicine physicians and 30-day mortality among hospitalized Medicare beneficiaries. JAMA. 2017;318(21):2119-2129. https://doi.org/10.1001/jama.2017.17925
34. Butcher L. Nurses as hospitalists | AHA Trustee Services. American Hospital Association. Accessed July 14, 2020 https://trustees.aha.org/articles/1238-nurses-as-hospitalists
35. Kartha A, Restuccia JD, Burgess JF, Jr, et al. Nurse practitioner and physician assistant scope of practice in 118 acute care hospitals. J Hosp Med. 2014;9(10):615-620. https://doi.org/10.1002/jhm.2231
1. O’Connor A, Wellenius G. Rural-urban disparities in the prevalence of diabetes and coronary heart disease. Public Health. 2012;126(10):813-820. https://doi.org/10.1016/j.puhe.2012.05.029
2. Kaufman BG, Thomas SR, Randolph RK, et al. The rising rate of rural hospital closures. J Rural Health. 2016;32(1):35-43. https://doi.org/10.1111/jrh.12128
3. MacDowell M, Glasser M, Fitts M, Nielsen K, Hunsaker M. A national view of rural health workforce issues in the USA. Rural Remote Health. 2010;10(3):1531.
4. Joynt KE, Harris Y, Orav EJ, Jha AK. Quality of care and patient outcomes in critical access rural hospitals. JAMA. 2011;306(1):45-52. https://doi.org/10.1001/jama.2011.902
5. The Chartis Group. Chartis Center for Rural Health. The Rural Health Safety Net Under Pressure: Rural Hospital Vulnerability. Published February 2020. Accessed May 07, 2020. https://www.chartis.com/forum/wp-content/uploads/2020/02/CCRH_Vulnerability-Research_FiNAL-02.14.20.pdf
6. Miller KEM, James HJ, Holmes GM, Van Houtven CH. The effect of rural hospital closures on emergency medical service response and transport times. Health Serv Res. 2020;55(2):288-300. https://doi.org/10.1111/1475-6773.13254
7. Buchmueller TC, Jacobson M, Wold C. How far to the hospital? The effect of hospital closures on access to care. J Health Econ. 2006;25(4):740-761. https://doi.org/10.1016/j.jhealeco.2005.10.006
8. Mohr NM, Harland KK, Shane DM, et al. Rural patients with severe sepsis or septic shock who bypass rural hospitals have increased mortality: an instrumental variables approach. Crit Care Med. 2017;45(1):85-93. https://doi.org/10.1097/ccm.0000000000002026
9. Gutierrez J, Kuperman E, Kaboli PJ. Using telehealth as a tool for rural hospitals in the COVID-19 pandemic response. J Rural Health. 2020;10.1111/jrh.12443. https://doi.org/10.1111/jrh.12443
10. Kisuule F, Howell EE. Hospitalists and their impact on quality, patient safety, and satisfaction. Obstet Gynecol Clin North Am. 2015;42(3):433-446. https://doi.org/10.1016/j.ogc.2015.05.003
11. Peterson MC. A systematic review of outcomes and quality measures in adult patients cared for by hospitalists vs nonhospitalists. Mayo Clin Proc. 2009;84(3):248-254. https://doi.org/10.4065/84.3.248
12. Auerbach AD, Wachter RM, Katz P, Showstack J, Baron RB, Goldman L. Implementation of a voluntary hospitalist service at a community teaching hospital: improved clinical efficiency and patient outcomes. Ann Intern Med. 2002;137(11):859-865. https://doi.org/10.7326/0003-4819-137-11-200212030-00006
13. Casey MM, Hung P, Moscovice I, Prasad S. The use of hospitalists by small rural hospitals: results of a national survey. Med Care Res Rev. 2014;71(4):356-366. https://doi.org/10.1177/1077558714533822
14. Sanders RB, Simpson KN, Kazley AS, Giarrizzi DP. New hospital telemedicine services: potential market for a nighttime telehospitalist service. Telemed J E Health. 2014;20(10):902-908. https://doi.org/10.1089/tmj.2013.0344
15. Department of Veterans Affairs. Office of Inspector General. OIG Determination of Veterans Health Administration’s Occupational Staffing Shortages. Published September 30, 2019. Accessed June 15, 2020. https://www.va.gov/oig/pubs/VAOIG-19-00346-241.pdf
16. Gutierrez J, Moeckli J, McAdams N, Kaboli PJ. Perceptions of telehospitalist services to address staffing needs in rural and low complexity hospitals in the Veterans Health Administration. J Rural Health. 2019;36(3):355-359. https://doi.org/10.1111/jrh.12403
17. Eagle Telemedicine. EAGLE TELEMEDICINE NIGHT COVERAGE SOLUTIONS: Why They Work for Hospitals and Physicians. Accessed May 28, 2018. http://www.eagletelemedicine.com/wp-content/uploads/2016/11/EHP_WP_Telenocturnist_FINAL.pdf
18. Gujral J, Antoine C, Chandra S. The role of telehospitalist in COVID-19 response: Hospitalist caring remotely for New York patients explain their role. ACP Hospitalist. 2020; May 2020.
19. Kuperman EF, Linson EL, Klefstad K, Perry E, Glenn K. The virtual hospitalist: a single-site implementation bringing hospitalist coverage to critical access hospitals. J Hosp Med. 2018;13(11):759-763. https://doi.org/10.12788/jhm.3061
20. JaKa MM, Dinh JM, Ziegenfuss JY, et al. Patient and care team perspectives of telemedicine in critical access hospitals. J Hosp Med. 2020;15(6):345-348. https://doi.org/10.12788/jhm.3412
21. Khanal S, Burgon J, Leonard S, Griffiths M, Eddowes LA. Recommendations for the improved effectiveness and reporting of telemedicine programs in developing countries: results of a systematic literature review. Telemed J E Health. 2015;21(11):903-915. https://doi.org/10.1089/tmj.2014.0194
22. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. BMJ Qual Saf. 2016;25(12):986-992. https://doi.org/10.1136/bmjqs-2015-004411
23. VHA Office of Rural Health. ORH 2020-2024 STRATEGIC PLAN. In: U.S. Department of Veterans Affairs, ed 2020. Accessed January 18, 2021 https://www.ruralhealth.va.gov/aboutus/index.asp
24. Veterans Health Administration. About VHA. In: U.S. Department of Veterans Affairs, ed. 2019. Accessed January 18, 2021.https://www.va.gov/health/aboutvha.asp
25. GeoSpatial Outcomes Division. VHA Office of Rural Health. U.S. Department of Veterans Affairs. Rural Veterans Health Care Atlas. 2nd ed - FY-2015. Accessed July 30, 2020. https://www.ruralhealth.va.gov/docs/atlas/CHAPTER_02_RHRI_Pts_treated_at_VAMCs.pdf
26. Wagner TH, Upadhyay A, Cowgill E, et al. Risk adjustment tools for learning health systems: a comparison of DxCG and CMS-HCC V21. Health Serv Res. 2016;51(5):2002-2019. https://doi.org/10.1111/1475-6773.12454
27. Wagner T, Stefos T, Moran E, et al. Technical Report 30: Risk Adjustment: Guide to the V21 and Nosos Risk Score Programs. Updated February 8, 2016. Accessed July 30, 2020. https://www.herc.research.va.gov/include/page.asp?id=technical-report-risk-adjustment
28. The R Foundation. The R Project for Statistical Computing. Accessed August 10, 2020. https://www.R-project.org/
29. Cleary PD, Meterko M, Wright SM, Zaslavsky AM. Are comparisons of patient experiences across hospitals fair? A study in Veterans Health Administration hospitals. Med Care. 2014;52(7):619-625. https://doi.org/10.1097/mlr.0000000000000144
30. Giordano LA, Elliott MN, Goldstein E, Lehrman WG, Spencer PA. Development, implementation, and public reporting of the HCAHPS survey. Med Care Res Rev. 2010;67(1):27-37. doi:10.1177/1077558709341065
31. Boltz M, Cuellar NG, Cole C, Pistorese B. Comparing an on-site nurse practitioner with telemedicine physician support hospitalist programme with a traditional physician hospitalist programme. J Telemed and Telecare. 2019;25(4):213-220. https://doi.org/10.1177%2F1357633X18758744
32. Quinn R. The pros and cons of locum tenens for hospitalists. The Hospitalist. 2012(12). Accessed May 29, 2018. https://www.the-hospitalist.org/hospitalist/article/124988/pros-and-cons-locum-tenens-hospitalists
33. Blumenthal DM, Olenski AR, Tsugawa Y, Jena AB. Association between treatment by locum tenens internal medicine physicians and 30-day mortality among hospitalized Medicare beneficiaries. JAMA. 2017;318(21):2119-2129. https://doi.org/10.1001/jama.2017.17925
34. Butcher L. Nurses as hospitalists | AHA Trustee Services. American Hospital Association. Accessed July 14, 2020 https://trustees.aha.org/articles/1238-nurses-as-hospitalists
35. Kartha A, Restuccia JD, Burgess JF, Jr, et al. Nurse practitioner and physician assistant scope of practice in 118 acute care hospitals. J Hosp Med. 2014;9(10):615-620. https://doi.org/10.1002/jhm.2231
© 2021 Society of Hospital Medicine