User login
Thoracentesis Referral
Internal medicine (IM) residents and hospitalist physicians commonly conduct bedside thoracenteses for both diagnostic and therapeutic purposes.[1] The American Board of Internal Medicine only requires that certification candidates understand the indications, complications, and management of thoracenteses.[2] A disconnect between clinical practice patterns and board requirements may increase patient risk because poorly trained physicians are more likely to cause complications.[3] National practice patterns show that many thoracenteses are referred to interventional radiology (IR).[4] However, research links performance of bedside procedures to reduced hospital length of stay and lower costs, without increasing risk of complications.[1, 5, 6]
Simulation‐based education offers a controlled environment where trainees improve procedural knowledge and skills without patient harm.[7] Simulation‐based mastery learning (SBML) is a rigorous form of competency‐based education that improves clinical skills and reduces iatrogenic complications and healthcare costs.[5, 6, 8] SBML also is an effective method to boost thoracentesis skills among IM residents.[9] However, there are no data to show that thoracentesis skills acquired in the simulation laboratory transfer to clinical environments and affect referral patterns.
We hypothesized that a thoracentesis SBML intervention would improve skills and increase procedural self‐confidence while reducing procedure referrals. This study aimed to (1) assess the effect of thoracentesis SBML on a cohort of IM residents' simulated skills and (2) compare traditionally trained (nonSBML‐trained) residents, SBML‐trained residents, and hospitalist physicians regarding procedure referral patterns, self‐confidence, procedure experience, and reasons for referral.
METHODS AND MATERIALS
Study Design
We surveyed physicians about thoracenteses performed on patients cared for by postgraduate year (PGY)‐2 and PGY‐3 IM residents and hospitalist physicians at Northwestern Memorial Hospital (NMH) from December 2012 to May 2015. NMH is an 896‐bed, tertiary academic medical center, located in Chicago, Illinois. A random sample of IM residents participated in a thoracentesis SBML intervention, whereas hospitalist physicians did not. We compared referral patterns, self‐confidence, procedure experience, and reasons for referral between traditionally trained residents, SBML‐trained residents, and hospitalist physicians. The Northwestern University Institutional Review Board approved this study, and all study participants provided informed consent.
At NMH, resident‐staffed services include general IM and nonintensive care subspecialty medical services. There are also 2 nonteaching floors staffed by hospitalist attending physicians without residents. Thoracenteses performed on these services can either be done at the bedside or referred to pulmonary medicine or IR. The majority of thoracenteses performed by pulmonary medicine occur at the patients' bedside, and the patients also receive a clinical consultation. IR procedures are done in the IR suite without additional clinical consultation.
Procedure
One hundred sixty residents were available for training over the study period. We randomly selected 20% of the approximately 20 PGY‐2 and PGY‐3 IM residents assigned to the NMH medicine services each month to participate in SBML thoracentesis training before their rotation. Randomly selected residents were required to undergo SBML training but were not required to participate in the study. This selection process was repeated before every rotation during the study period. This randomized wait‐list control method allowed residents to serve as controls if not initially selected for training and remain eligible for SBML training in subsequent rotations.
Intervention
The SBML intervention used a pretest/post‐test design, as described elsewhere.[9] Residents completed a clinical skills pretest on a thoracentesis simulator using a previously published 26‐item checklist.[9] Following the pretest, residents participated in 2, 1‐hour training sessions including a lecture, video, and deliberate practice on the simulator with feedback from an expert instructor. Finally, residents completed a clinical skills post‐test using the checklist within 1 week from training (but on a different day) and were required to meet or exceed an 84.3% minimum passing score (MPS). The entire training, including pre‐ and post‐tests, took approximately 3 hours to complete, and residents were given an additional 1 hour refresher training every 6 months for up to a year after original training. We compared pre‐ and post‐test checklist scores to evaluate skills improvement.
Thoracentesis Patient Identification
The NMH electronic health record (EHR) was used to identify medical service inpatients who underwent a thoracentesis during the study period. NMH clinicians must place an EHR order for procedure kits, consults, and laboratory analysis of thoracentesis fluid. We developed a real‐time query of NMH's EHR that identified all patients with electronic orders for thoracenteses and monitored this daily.
Physician Surveys
After each thoracentesis, we surveyed the PGY‐2 or PGY‐3 resident or hospitalist caring for the patient about the procedure. A research coordinator, blind to whether the resident received SBML, performed the surveys face‐to‐face on Monday to Friday during normal business hours. Residents were not considered SBML‐trained until they met or exceeded the MPS on the simulated skills checklist at post‐test. Surveys occurred on Monday for procedures performed on Friday evening through Sunday. Survey questions asked physicians about who performed the procedure, their procedural self‐confidence, and total number of thoracenteses performed in their career. For referred procedures, physicians were asked about reasons for referral including lack of confidence, work hour restrictions (residents only), and low reimbursement rates.[10] There was also an option to add other reasons.
Measurement
The thoracentesis skills checklist documented all required steps for an evidence‐based thoracentesis. Each task received equal weight (0 = done incorrectly/not done, 1 = done correctly).[9] For physician surveys, self‐confidence about performing the procedure was rated on a scale of 0 = not confident to 100 = very confident. Reasons for referral were scored on a Likert scale 1 to 5 (1 = not at all important, 5 = very important). Other reasons for referral were categorized.
Statistical Analysis
The clinical skills pre‐ and post‐test checklist scores were compared using a Wilcoxon matched pairs rank test. Physician survey data were compared between different procedure performers using the 2 test, independent t test, analysis of variance (ANOVA), or Kruskal‐Wallis test depending on data properties. Referral patterns measured by the Likert scale were averaged, and differences between physician groups were evaluated using ANOVA. Counts of other reasons for referral were compared using the 2 test. We performed all statistical analyses using IBM SPSS Statistics version 23 (IBM Corp., Armonk, NY).
RESULTS
Thoracentesis Clinical Skills
One hundred twelve (70%) residents were randomized to SBML, and all completed the protocol. Median pretest scores were 57.6% (interquartile range [IQR] 43.376.9), and final post‐test mastery scores were 96.2 (IQR 96.2100.0; P < 0.001). Twenty‐three residents (21.0%) failed to meet the MPS at initial post‐test, but met the MPS on retest after <1 hour of additional training.
Physician Surveys
The EHR query identified 474 procedures eligible for physician surveys. One hundred twenty‐two residents and 51 hospitalist physicians completed surveys for 472 procedures (99.6%); 182 patients by traditionally trained residents, 145 by SBML‐trained residents, and 145 by hospitalist physicians. As shown in Table 1, 413 (88%) of all procedures were referred to another service. Traditionally trained residents were more likely to refer to IR compared to SBML‐trained residents or hospitalist physicians. SBML‐trained residents were more likely to perform bedside procedures, whereas hospitalist physicians were most likely to refer to pulmonary medicine. SBML‐trained residents were most confident in their procedural skills, despite hospitalist physicians performing more actual procedures.
| Traditionally Trained Resident Surveys, n = 182 | SBML‐Trained Resident Surveys, n = 145 | Hospitalist Physician Surveys, n = 145 | P Value | |
|---|---|---|---|---|
| ||||
| Bedside procedures, no. (%) | 26 (14.3%) | 32 (22.1%) | 1 (0.7%) | <0.001 |
| IR procedures, no. (%) | 119 (65.4%) | 74 (51.0%) | 82 (56.6%) | 0.029 |
| Pulmonary procedures, no. (%) | 37 (20.3%) | 39 (26.9%) | 62 (42.8%) | <0.001 |
| Procedure self‐confidence, mean (SD)* | 43.6 (28.66) | 68.2 (25.17) | 55.7 (31.17) | <0.001 |
| Experience performing actual procedures, median (IQR) | 1 (13) | 2 (13.5) | 10 (425) | <0.001 |
Traditionally trained residents were most likely to rate low confidence as reasons why they referred thoracenteses (Table 2). Hospitalist physicians were more likely to cite lack of time to perform the procedure themselves. Other reasons were different across groups. SBML‐trained residents were more likely to refer because of attending preference, whereas traditionally trained residents were mostly like to refer because of high risk/technically difficult cases.
| Traditionally Trained Residents, n = 156 | SBML‐Trained Residents, n = 113 | Hospitalist Physicians, n = 144 | P Value | |
|---|---|---|---|---|
| ||||
| Lack of confidence to perform procedure, mean (SD)* | 3.46 (1.32) | 2.52 (1.45) | 2.89 (1.60) | <0.001 |
| Work hour restrictions, mean (SD) * | 2.05 (1.37) | 1.50 (1.11) | n/a | 0.001 |
| Low reimbursement, mean (SD)* | 1.02 (0.12) | 1.0 (0) | 1.22 (0.69) | <0.001 |
| Other reasons for referral, no. (%) | ||||
| Attending preference | 8 (5.1%) | 11 (9.7%) | 3 (2.1%) | 0.025 |
| Don't know how | 6 (3.8%) | 0 | 0 | 0.007 |
| Failed bedside | 0 | 2 (1.8%) | 0 | 0.07 |
| High risk/technically difficult case | 24 (15.4%) | 12 (10.6%) | 5 (3.5%) | 0.003 |
| IR or pulmonary patient | 5 (3.2%) | 2 (1.8%) | 4 (2.8%) | 0.77 |
| Other IR procedure taking place | 11 (7.1%) | 9 (8.0%) | 4 (2.8%) | 0.13 |
| Patient preference | 2 (1.3%) | 7 (6.2%) | 2 (3.5%) | 0.024 |
| Time | 9 (5.8%) | 7 (6.2%) | 29 (20.1%) | <0.001 |
DISCUSSION
This study confirms earlier research showing that thoracentesis SBML improves residents' clinical skills, but is the first to use a randomized study design.[9] Use of the mastery model in health professions education ensures that all learners are competent to provide patient care including performing invasive procedures. Such rigorous education yields downstream translational outcomes including safety profiles comparable to experts.[1, 6]
This study also shows that SBML‐trained residents displayed higher self‐confidence and performed significantly more bedside procedures than traditionally trained residents and more experienced hospitalist physicians. Although the Society of Hospital Medicine considers thoracentesis skills a core competency for hospitalist physicians,[11] we speculate that some hospitalist physicians had not performed a thoracentesis in years. A recent national survey showed that only 44% of hospitalist physicians performed at least 1 thoracentesis within the past year.[10] Research also shows a shift in medical culture to refer procedures to specialty services, such as IR, by over 900% in the past 2 decades.[4] Our results provide novel information about procedure referrals because we show that SBML provides translational outcomes by improving skills and self‐confidence that influence referral patterns. SBML‐trained residents performed almost a quarter of procedures at the bedside. Although this only represents an 8% absolute difference in bedside procedures compared to traditionally trained residents, if a large number of residents are trained using SBML this results in a meaningful number of procedures shifted to the patient bedside. According to University HealthSystem Consortium data, in US teaching hospitals, approximately 35,325 thoracenteses are performed yearly.[1] Shifting even 8% of these procedures to the bedside would result in significant clinical benefit and cost savings. Reduced referrals increase additional bedside procedures that are safe, cost‐effective, and highly satisfying to patients.[1, 12, 13] Further study is required to determine the impact on referral patterns after providing SMBL training to attending physicians.
Our study also provides information about the rationale for procedure referrals. Earlier work speculates that financial incentive, training and time may explain high procedure referral rates.[10] One report on IM residents noted an 87% IR referral rate for thoracentesis, and confirmed that both training and time were major reasons.[14] Hospitalist physicians reported lack of time as the major factor leading to procedural referrals, which is problematic because bedside procedures yield similar clinical outcomes at lower costs.[1, 12] Attending preference also prevented 11 additional bedside procedures in the SBML‐trained group. Schedule adjustments and SBML training of hospitalist physicians should be considered, because bundled payments in the Affordable Care Act may favor shifting to the higher‐value approach of bedside thoracenteses.[15]
Our study has several limitations. First, we only performed surveys at 1 institution and the results may not be generalizable. Second, we relied on an electronic query to alert us to thoracenteses. Our query may have missed procedures that were unsuccessful or did not have EHR orders entered. Third, physicians may have been surveyed more than once for different or the same patient(s), but opinions may have shifted over time. Fourth, some items such as time needed to be written in the survey and were not specifically asked. This could have resulted in under‐reporting. Finally, we did not assess the clinical outcomes of thoracenteses in this study, although earlier work shows that residents who complete SBML have safety outcomes similar to IR.[1, 6]
In summary, IM residents who complete thoracentesis SBML demonstrate improved clinical skills and are more likely to perform bedside procedures. In an era of bundled payments, rethinking current care models to promote cost‐effective care is necessary. We believe providing additional education, training, and support to hospitalist physicians to promote bedside procedures is a promising strategy that warrants further study.
Acknowledgements
The authors acknowledge Drs. Douglas Vaughan and Kevin O'Leary for their support and encouragement of this work. The authors also thank the internal medicine residents at Northwestern for their dedication to patient care.
Disclosures: This project was supported by grant R18HS021202‐01 from the Agency for Healthcare Research and Quality (AHRQ). AHRQ had no role in the preparation, review, or approval of the manuscript. Trial Registration:
- , , , , . Thoracentesis procedures at university hospitals: comparing outcomes by specialty. Jt Comm J Qual Patient Saf. 2015;42(1):34–40.
- American Board of Internal Medicine. Internal medicine policies. Available at: http://www.abim.org/certification/policies/internal‐medicine‐subspecialty‐policies/internal‐medicine.aspx. Accessed March 9, 2016.
- , , , . Pneumothorax following thoracentesis: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(4):332–339.
- , , . National fluid shifts: fifteen‐year trends in paracentesis and thoracentesis procedures. J Am Coll Radiol. 2010;7(11):859–864.
- , , , et al. Cost savings of performing paracentesis procedures at the bedside after simulation‐based education. Simul Healthc. 2014;9(5):312–318.
- , , , , . Clinical outcomes after bedside and interventional radiology paracentesis procedures. Am J Med. 2013;126(4):349–356.
- , , , et al. Simulation technology for health care professional skills training and assessment. JAMA. 1999;282(9):861–866.
- , , , et al. Cost savings from reduced catheter‐related bloodstream infection after simulation‐based education for residents in a medical intensive care unit. Simul Healthc. 2010;5(2):98–102.
- , , , , . Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice. J Hosp Med. 2008;3(1):48–54.
- , , , , . Procedures performed by hospitalist and non‐hospitalist general internists. J Gen Intern Med. 2010;25(5):448–452.
- , , , , . Core competencies in hospital medicine: development and methodology. J Hosp Med. 2006;1(suppl 1):48–56.
- , , , , , . Specialties performing paracentesis procedures at university hospitals: implications for training and certification. J Hosp Med. 2014;9(3):162–168.
- , , , , , . Are we providing patient‐centered care? Preferences about paracentesis and thoracentesis procedures. Patient Exp J. 2014;1(2):94–103. Available at: http://pxjournal.org/cgi/viewcontent.cgi?article=1024
Internal medicine (IM) residents and hospitalist physicians commonly conduct bedside thoracenteses for both diagnostic and therapeutic purposes.[1] The American Board of Internal Medicine only requires that certification candidates understand the indications, complications, and management of thoracenteses.[2] A disconnect between clinical practice patterns and board requirements may increase patient risk because poorly trained physicians are more likely to cause complications.[3] National practice patterns show that many thoracenteses are referred to interventional radiology (IR).[4] However, research links performance of bedside procedures to reduced hospital length of stay and lower costs, without increasing risk of complications.[1, 5, 6]
Simulation‐based education offers a controlled environment where trainees improve procedural knowledge and skills without patient harm.[7] Simulation‐based mastery learning (SBML) is a rigorous form of competency‐based education that improves clinical skills and reduces iatrogenic complications and healthcare costs.[5, 6, 8] SBML also is an effective method to boost thoracentesis skills among IM residents.[9] However, there are no data to show that thoracentesis skills acquired in the simulation laboratory transfer to clinical environments and affect referral patterns.
We hypothesized that a thoracentesis SBML intervention would improve skills and increase procedural self‐confidence while reducing procedure referrals. This study aimed to (1) assess the effect of thoracentesis SBML on a cohort of IM residents' simulated skills and (2) compare traditionally trained (nonSBML‐trained) residents, SBML‐trained residents, and hospitalist physicians regarding procedure referral patterns, self‐confidence, procedure experience, and reasons for referral.
METHODS AND MATERIALS
Study Design
We surveyed physicians about thoracenteses performed on patients cared for by postgraduate year (PGY)‐2 and PGY‐3 IM residents and hospitalist physicians at Northwestern Memorial Hospital (NMH) from December 2012 to May 2015. NMH is an 896‐bed, tertiary academic medical center, located in Chicago, Illinois. A random sample of IM residents participated in a thoracentesis SBML intervention, whereas hospitalist physicians did not. We compared referral patterns, self‐confidence, procedure experience, and reasons for referral between traditionally trained residents, SBML‐trained residents, and hospitalist physicians. The Northwestern University Institutional Review Board approved this study, and all study participants provided informed consent.
At NMH, resident‐staffed services include general IM and nonintensive care subspecialty medical services. There are also 2 nonteaching floors staffed by hospitalist attending physicians without residents. Thoracenteses performed on these services can either be done at the bedside or referred to pulmonary medicine or IR. The majority of thoracenteses performed by pulmonary medicine occur at the patients' bedside, and the patients also receive a clinical consultation. IR procedures are done in the IR suite without additional clinical consultation.
Procedure
One hundred sixty residents were available for training over the study period. We randomly selected 20% of the approximately 20 PGY‐2 and PGY‐3 IM residents assigned to the NMH medicine services each month to participate in SBML thoracentesis training before their rotation. Randomly selected residents were required to undergo SBML training but were not required to participate in the study. This selection process was repeated before every rotation during the study period. This randomized wait‐list control method allowed residents to serve as controls if not initially selected for training and remain eligible for SBML training in subsequent rotations.
Intervention
The SBML intervention used a pretest/post‐test design, as described elsewhere.[9] Residents completed a clinical skills pretest on a thoracentesis simulator using a previously published 26‐item checklist.[9] Following the pretest, residents participated in 2, 1‐hour training sessions including a lecture, video, and deliberate practice on the simulator with feedback from an expert instructor. Finally, residents completed a clinical skills post‐test using the checklist within 1 week from training (but on a different day) and were required to meet or exceed an 84.3% minimum passing score (MPS). The entire training, including pre‐ and post‐tests, took approximately 3 hours to complete, and residents were given an additional 1 hour refresher training every 6 months for up to a year after original training. We compared pre‐ and post‐test checklist scores to evaluate skills improvement.
Thoracentesis Patient Identification
The NMH electronic health record (EHR) was used to identify medical service inpatients who underwent a thoracentesis during the study period. NMH clinicians must place an EHR order for procedure kits, consults, and laboratory analysis of thoracentesis fluid. We developed a real‐time query of NMH's EHR that identified all patients with electronic orders for thoracenteses and monitored this daily.
Physician Surveys
After each thoracentesis, we surveyed the PGY‐2 or PGY‐3 resident or hospitalist caring for the patient about the procedure. A research coordinator, blind to whether the resident received SBML, performed the surveys face‐to‐face on Monday to Friday during normal business hours. Residents were not considered SBML‐trained until they met or exceeded the MPS on the simulated skills checklist at post‐test. Surveys occurred on Monday for procedures performed on Friday evening through Sunday. Survey questions asked physicians about who performed the procedure, their procedural self‐confidence, and total number of thoracenteses performed in their career. For referred procedures, physicians were asked about reasons for referral including lack of confidence, work hour restrictions (residents only), and low reimbursement rates.[10] There was also an option to add other reasons.
Measurement
The thoracentesis skills checklist documented all required steps for an evidence‐based thoracentesis. Each task received equal weight (0 = done incorrectly/not done, 1 = done correctly).[9] For physician surveys, self‐confidence about performing the procedure was rated on a scale of 0 = not confident to 100 = very confident. Reasons for referral were scored on a Likert scale 1 to 5 (1 = not at all important, 5 = very important). Other reasons for referral were categorized.
Statistical Analysis
The clinical skills pre‐ and post‐test checklist scores were compared using a Wilcoxon matched pairs rank test. Physician survey data were compared between different procedure performers using the 2 test, independent t test, analysis of variance (ANOVA), or Kruskal‐Wallis test depending on data properties. Referral patterns measured by the Likert scale were averaged, and differences between physician groups were evaluated using ANOVA. Counts of other reasons for referral were compared using the 2 test. We performed all statistical analyses using IBM SPSS Statistics version 23 (IBM Corp., Armonk, NY).
RESULTS
Thoracentesis Clinical Skills
One hundred twelve (70%) residents were randomized to SBML, and all completed the protocol. Median pretest scores were 57.6% (interquartile range [IQR] 43.376.9), and final post‐test mastery scores were 96.2 (IQR 96.2100.0; P < 0.001). Twenty‐three residents (21.0%) failed to meet the MPS at initial post‐test, but met the MPS on retest after <1 hour of additional training.
Physician Surveys
The EHR query identified 474 procedures eligible for physician surveys. One hundred twenty‐two residents and 51 hospitalist physicians completed surveys for 472 procedures (99.6%); 182 patients by traditionally trained residents, 145 by SBML‐trained residents, and 145 by hospitalist physicians. As shown in Table 1, 413 (88%) of all procedures were referred to another service. Traditionally trained residents were more likely to refer to IR compared to SBML‐trained residents or hospitalist physicians. SBML‐trained residents were more likely to perform bedside procedures, whereas hospitalist physicians were most likely to refer to pulmonary medicine. SBML‐trained residents were most confident in their procedural skills, despite hospitalist physicians performing more actual procedures.
| Traditionally Trained Resident Surveys, n = 182 | SBML‐Trained Resident Surveys, n = 145 | Hospitalist Physician Surveys, n = 145 | P Value | |
|---|---|---|---|---|
| ||||
| Bedside procedures, no. (%) | 26 (14.3%) | 32 (22.1%) | 1 (0.7%) | <0.001 |
| IR procedures, no. (%) | 119 (65.4%) | 74 (51.0%) | 82 (56.6%) | 0.029 |
| Pulmonary procedures, no. (%) | 37 (20.3%) | 39 (26.9%) | 62 (42.8%) | <0.001 |
| Procedure self‐confidence, mean (SD)* | 43.6 (28.66) | 68.2 (25.17) | 55.7 (31.17) | <0.001 |
| Experience performing actual procedures, median (IQR) | 1 (13) | 2 (13.5) | 10 (425) | <0.001 |
Traditionally trained residents were most likely to rate low confidence as reasons why they referred thoracenteses (Table 2). Hospitalist physicians were more likely to cite lack of time to perform the procedure themselves. Other reasons were different across groups. SBML‐trained residents were more likely to refer because of attending preference, whereas traditionally trained residents were mostly like to refer because of high risk/technically difficult cases.
| Traditionally Trained Residents, n = 156 | SBML‐Trained Residents, n = 113 | Hospitalist Physicians, n = 144 | P Value | |
|---|---|---|---|---|
| ||||
| Lack of confidence to perform procedure, mean (SD)* | 3.46 (1.32) | 2.52 (1.45) | 2.89 (1.60) | <0.001 |
| Work hour restrictions, mean (SD) * | 2.05 (1.37) | 1.50 (1.11) | n/a | 0.001 |
| Low reimbursement, mean (SD)* | 1.02 (0.12) | 1.0 (0) | 1.22 (0.69) | <0.001 |
| Other reasons for referral, no. (%) | ||||
| Attending preference | 8 (5.1%) | 11 (9.7%) | 3 (2.1%) | 0.025 |
| Don't know how | 6 (3.8%) | 0 | 0 | 0.007 |
| Failed bedside | 0 | 2 (1.8%) | 0 | 0.07 |
| High risk/technically difficult case | 24 (15.4%) | 12 (10.6%) | 5 (3.5%) | 0.003 |
| IR or pulmonary patient | 5 (3.2%) | 2 (1.8%) | 4 (2.8%) | 0.77 |
| Other IR procedure taking place | 11 (7.1%) | 9 (8.0%) | 4 (2.8%) | 0.13 |
| Patient preference | 2 (1.3%) | 7 (6.2%) | 2 (3.5%) | 0.024 |
| Time | 9 (5.8%) | 7 (6.2%) | 29 (20.1%) | <0.001 |
DISCUSSION
This study confirms earlier research showing that thoracentesis SBML improves residents' clinical skills, but is the first to use a randomized study design.[9] Use of the mastery model in health professions education ensures that all learners are competent to provide patient care including performing invasive procedures. Such rigorous education yields downstream translational outcomes including safety profiles comparable to experts.[1, 6]
This study also shows that SBML‐trained residents displayed higher self‐confidence and performed significantly more bedside procedures than traditionally trained residents and more experienced hospitalist physicians. Although the Society of Hospital Medicine considers thoracentesis skills a core competency for hospitalist physicians,[11] we speculate that some hospitalist physicians had not performed a thoracentesis in years. A recent national survey showed that only 44% of hospitalist physicians performed at least 1 thoracentesis within the past year.[10] Research also shows a shift in medical culture to refer procedures to specialty services, such as IR, by over 900% in the past 2 decades.[4] Our results provide novel information about procedure referrals because we show that SBML provides translational outcomes by improving skills and self‐confidence that influence referral patterns. SBML‐trained residents performed almost a quarter of procedures at the bedside. Although this only represents an 8% absolute difference in bedside procedures compared to traditionally trained residents, if a large number of residents are trained using SBML this results in a meaningful number of procedures shifted to the patient bedside. According to University HealthSystem Consortium data, in US teaching hospitals, approximately 35,325 thoracenteses are performed yearly.[1] Shifting even 8% of these procedures to the bedside would result in significant clinical benefit and cost savings. Reduced referrals increase additional bedside procedures that are safe, cost‐effective, and highly satisfying to patients.[1, 12, 13] Further study is required to determine the impact on referral patterns after providing SMBL training to attending physicians.
Our study also provides information about the rationale for procedure referrals. Earlier work speculates that financial incentive, training and time may explain high procedure referral rates.[10] One report on IM residents noted an 87% IR referral rate for thoracentesis, and confirmed that both training and time were major reasons.[14] Hospitalist physicians reported lack of time as the major factor leading to procedural referrals, which is problematic because bedside procedures yield similar clinical outcomes at lower costs.[1, 12] Attending preference also prevented 11 additional bedside procedures in the SBML‐trained group. Schedule adjustments and SBML training of hospitalist physicians should be considered, because bundled payments in the Affordable Care Act may favor shifting to the higher‐value approach of bedside thoracenteses.[15]
Our study has several limitations. First, we only performed surveys at 1 institution and the results may not be generalizable. Second, we relied on an electronic query to alert us to thoracenteses. Our query may have missed procedures that were unsuccessful or did not have EHR orders entered. Third, physicians may have been surveyed more than once for different or the same patient(s), but opinions may have shifted over time. Fourth, some items such as time needed to be written in the survey and were not specifically asked. This could have resulted in under‐reporting. Finally, we did not assess the clinical outcomes of thoracenteses in this study, although earlier work shows that residents who complete SBML have safety outcomes similar to IR.[1, 6]
In summary, IM residents who complete thoracentesis SBML demonstrate improved clinical skills and are more likely to perform bedside procedures. In an era of bundled payments, rethinking current care models to promote cost‐effective care is necessary. We believe providing additional education, training, and support to hospitalist physicians to promote bedside procedures is a promising strategy that warrants further study.
Acknowledgements
The authors acknowledge Drs. Douglas Vaughan and Kevin O'Leary for their support and encouragement of this work. The authors also thank the internal medicine residents at Northwestern for their dedication to patient care.
Disclosures: This project was supported by grant R18HS021202‐01 from the Agency for Healthcare Research and Quality (AHRQ). AHRQ had no role in the preparation, review, or approval of the manuscript. Trial Registration:
Internal medicine (IM) residents and hospitalist physicians commonly conduct bedside thoracenteses for both diagnostic and therapeutic purposes.[1] The American Board of Internal Medicine only requires that certification candidates understand the indications, complications, and management of thoracenteses.[2] A disconnect between clinical practice patterns and board requirements may increase patient risk because poorly trained physicians are more likely to cause complications.[3] National practice patterns show that many thoracenteses are referred to interventional radiology (IR).[4] However, research links performance of bedside procedures to reduced hospital length of stay and lower costs, without increasing risk of complications.[1, 5, 6]
Simulation‐based education offers a controlled environment where trainees improve procedural knowledge and skills without patient harm.[7] Simulation‐based mastery learning (SBML) is a rigorous form of competency‐based education that improves clinical skills and reduces iatrogenic complications and healthcare costs.[5, 6, 8] SBML also is an effective method to boost thoracentesis skills among IM residents.[9] However, there are no data to show that thoracentesis skills acquired in the simulation laboratory transfer to clinical environments and affect referral patterns.
We hypothesized that a thoracentesis SBML intervention would improve skills and increase procedural self‐confidence while reducing procedure referrals. This study aimed to (1) assess the effect of thoracentesis SBML on a cohort of IM residents' simulated skills and (2) compare traditionally trained (nonSBML‐trained) residents, SBML‐trained residents, and hospitalist physicians regarding procedure referral patterns, self‐confidence, procedure experience, and reasons for referral.
METHODS AND MATERIALS
Study Design
We surveyed physicians about thoracenteses performed on patients cared for by postgraduate year (PGY)‐2 and PGY‐3 IM residents and hospitalist physicians at Northwestern Memorial Hospital (NMH) from December 2012 to May 2015. NMH is an 896‐bed, tertiary academic medical center, located in Chicago, Illinois. A random sample of IM residents participated in a thoracentesis SBML intervention, whereas hospitalist physicians did not. We compared referral patterns, self‐confidence, procedure experience, and reasons for referral between traditionally trained residents, SBML‐trained residents, and hospitalist physicians. The Northwestern University Institutional Review Board approved this study, and all study participants provided informed consent.
At NMH, resident‐staffed services include general IM and nonintensive care subspecialty medical services. There are also 2 nonteaching floors staffed by hospitalist attending physicians without residents. Thoracenteses performed on these services can either be done at the bedside or referred to pulmonary medicine or IR. The majority of thoracenteses performed by pulmonary medicine occur at the patients' bedside, and the patients also receive a clinical consultation. IR procedures are done in the IR suite without additional clinical consultation.
Procedure
One hundred sixty residents were available for training over the study period. We randomly selected 20% of the approximately 20 PGY‐2 and PGY‐3 IM residents assigned to the NMH medicine services each month to participate in SBML thoracentesis training before their rotation. Randomly selected residents were required to undergo SBML training but were not required to participate in the study. This selection process was repeated before every rotation during the study period. This randomized wait‐list control method allowed residents to serve as controls if not initially selected for training and remain eligible for SBML training in subsequent rotations.
Intervention
The SBML intervention used a pretest/post‐test design, as described elsewhere.[9] Residents completed a clinical skills pretest on a thoracentesis simulator using a previously published 26‐item checklist.[9] Following the pretest, residents participated in 2, 1‐hour training sessions including a lecture, video, and deliberate practice on the simulator with feedback from an expert instructor. Finally, residents completed a clinical skills post‐test using the checklist within 1 week from training (but on a different day) and were required to meet or exceed an 84.3% minimum passing score (MPS). The entire training, including pre‐ and post‐tests, took approximately 3 hours to complete, and residents were given an additional 1 hour refresher training every 6 months for up to a year after original training. We compared pre‐ and post‐test checklist scores to evaluate skills improvement.
Thoracentesis Patient Identification
The NMH electronic health record (EHR) was used to identify medical service inpatients who underwent a thoracentesis during the study period. NMH clinicians must place an EHR order for procedure kits, consults, and laboratory analysis of thoracentesis fluid. We developed a real‐time query of NMH's EHR that identified all patients with electronic orders for thoracenteses and monitored this daily.
Physician Surveys
After each thoracentesis, we surveyed the PGY‐2 or PGY‐3 resident or hospitalist caring for the patient about the procedure. A research coordinator, blind to whether the resident received SBML, performed the surveys face‐to‐face on Monday to Friday during normal business hours. Residents were not considered SBML‐trained until they met or exceeded the MPS on the simulated skills checklist at post‐test. Surveys occurred on Monday for procedures performed on Friday evening through Sunday. Survey questions asked physicians about who performed the procedure, their procedural self‐confidence, and total number of thoracenteses performed in their career. For referred procedures, physicians were asked about reasons for referral including lack of confidence, work hour restrictions (residents only), and low reimbursement rates.[10] There was also an option to add other reasons.
Measurement
The thoracentesis skills checklist documented all required steps for an evidence‐based thoracentesis. Each task received equal weight (0 = done incorrectly/not done, 1 = done correctly).[9] For physician surveys, self‐confidence about performing the procedure was rated on a scale of 0 = not confident to 100 = very confident. Reasons for referral were scored on a Likert scale 1 to 5 (1 = not at all important, 5 = very important). Other reasons for referral were categorized.
Statistical Analysis
The clinical skills pre‐ and post‐test checklist scores were compared using a Wilcoxon matched pairs rank test. Physician survey data were compared between different procedure performers using the 2 test, independent t test, analysis of variance (ANOVA), or Kruskal‐Wallis test depending on data properties. Referral patterns measured by the Likert scale were averaged, and differences between physician groups were evaluated using ANOVA. Counts of other reasons for referral were compared using the 2 test. We performed all statistical analyses using IBM SPSS Statistics version 23 (IBM Corp., Armonk, NY).
RESULTS
Thoracentesis Clinical Skills
One hundred twelve (70%) residents were randomized to SBML, and all completed the protocol. Median pretest scores were 57.6% (interquartile range [IQR] 43.376.9), and final post‐test mastery scores were 96.2 (IQR 96.2100.0; P < 0.001). Twenty‐three residents (21.0%) failed to meet the MPS at initial post‐test, but met the MPS on retest after <1 hour of additional training.
Physician Surveys
The EHR query identified 474 procedures eligible for physician surveys. One hundred twenty‐two residents and 51 hospitalist physicians completed surveys for 472 procedures (99.6%); 182 patients by traditionally trained residents, 145 by SBML‐trained residents, and 145 by hospitalist physicians. As shown in Table 1, 413 (88%) of all procedures were referred to another service. Traditionally trained residents were more likely to refer to IR compared to SBML‐trained residents or hospitalist physicians. SBML‐trained residents were more likely to perform bedside procedures, whereas hospitalist physicians were most likely to refer to pulmonary medicine. SBML‐trained residents were most confident in their procedural skills, despite hospitalist physicians performing more actual procedures.
| Traditionally Trained Resident Surveys, n = 182 | SBML‐Trained Resident Surveys, n = 145 | Hospitalist Physician Surveys, n = 145 | P Value | |
|---|---|---|---|---|
| ||||
| Bedside procedures, no. (%) | 26 (14.3%) | 32 (22.1%) | 1 (0.7%) | <0.001 |
| IR procedures, no. (%) | 119 (65.4%) | 74 (51.0%) | 82 (56.6%) | 0.029 |
| Pulmonary procedures, no. (%) | 37 (20.3%) | 39 (26.9%) | 62 (42.8%) | <0.001 |
| Procedure self‐confidence, mean (SD)* | 43.6 (28.66) | 68.2 (25.17) | 55.7 (31.17) | <0.001 |
| Experience performing actual procedures, median (IQR) | 1 (13) | 2 (13.5) | 10 (425) | <0.001 |
Traditionally trained residents were most likely to rate low confidence as reasons why they referred thoracenteses (Table 2). Hospitalist physicians were more likely to cite lack of time to perform the procedure themselves. Other reasons were different across groups. SBML‐trained residents were more likely to refer because of attending preference, whereas traditionally trained residents were mostly like to refer because of high risk/technically difficult cases.
| Traditionally Trained Residents, n = 156 | SBML‐Trained Residents, n = 113 | Hospitalist Physicians, n = 144 | P Value | |
|---|---|---|---|---|
| ||||
| Lack of confidence to perform procedure, mean (SD)* | 3.46 (1.32) | 2.52 (1.45) | 2.89 (1.60) | <0.001 |
| Work hour restrictions, mean (SD) * | 2.05 (1.37) | 1.50 (1.11) | n/a | 0.001 |
| Low reimbursement, mean (SD)* | 1.02 (0.12) | 1.0 (0) | 1.22 (0.69) | <0.001 |
| Other reasons for referral, no. (%) | ||||
| Attending preference | 8 (5.1%) | 11 (9.7%) | 3 (2.1%) | 0.025 |
| Don't know how | 6 (3.8%) | 0 | 0 | 0.007 |
| Failed bedside | 0 | 2 (1.8%) | 0 | 0.07 |
| High risk/technically difficult case | 24 (15.4%) | 12 (10.6%) | 5 (3.5%) | 0.003 |
| IR or pulmonary patient | 5 (3.2%) | 2 (1.8%) | 4 (2.8%) | 0.77 |
| Other IR procedure taking place | 11 (7.1%) | 9 (8.0%) | 4 (2.8%) | 0.13 |
| Patient preference | 2 (1.3%) | 7 (6.2%) | 2 (3.5%) | 0.024 |
| Time | 9 (5.8%) | 7 (6.2%) | 29 (20.1%) | <0.001 |
DISCUSSION
This study confirms earlier research showing that thoracentesis SBML improves residents' clinical skills, but is the first to use a randomized study design.[9] Use of the mastery model in health professions education ensures that all learners are competent to provide patient care including performing invasive procedures. Such rigorous education yields downstream translational outcomes including safety profiles comparable to experts.[1, 6]
This study also shows that SBML‐trained residents displayed higher self‐confidence and performed significantly more bedside procedures than traditionally trained residents and more experienced hospitalist physicians. Although the Society of Hospital Medicine considers thoracentesis skills a core competency for hospitalist physicians,[11] we speculate that some hospitalist physicians had not performed a thoracentesis in years. A recent national survey showed that only 44% of hospitalist physicians performed at least 1 thoracentesis within the past year.[10] Research also shows a shift in medical culture to refer procedures to specialty services, such as IR, by over 900% in the past 2 decades.[4] Our results provide novel information about procedure referrals because we show that SBML provides translational outcomes by improving skills and self‐confidence that influence referral patterns. SBML‐trained residents performed almost a quarter of procedures at the bedside. Although this only represents an 8% absolute difference in bedside procedures compared to traditionally trained residents, if a large number of residents are trained using SBML this results in a meaningful number of procedures shifted to the patient bedside. According to University HealthSystem Consortium data, in US teaching hospitals, approximately 35,325 thoracenteses are performed yearly.[1] Shifting even 8% of these procedures to the bedside would result in significant clinical benefit and cost savings. Reduced referrals increase additional bedside procedures that are safe, cost‐effective, and highly satisfying to patients.[1, 12, 13] Further study is required to determine the impact on referral patterns after providing SMBL training to attending physicians.
Our study also provides information about the rationale for procedure referrals. Earlier work speculates that financial incentive, training and time may explain high procedure referral rates.[10] One report on IM residents noted an 87% IR referral rate for thoracentesis, and confirmed that both training and time were major reasons.[14] Hospitalist physicians reported lack of time as the major factor leading to procedural referrals, which is problematic because bedside procedures yield similar clinical outcomes at lower costs.[1, 12] Attending preference also prevented 11 additional bedside procedures in the SBML‐trained group. Schedule adjustments and SBML training of hospitalist physicians should be considered, because bundled payments in the Affordable Care Act may favor shifting to the higher‐value approach of bedside thoracenteses.[15]
Our study has several limitations. First, we only performed surveys at 1 institution and the results may not be generalizable. Second, we relied on an electronic query to alert us to thoracenteses. Our query may have missed procedures that were unsuccessful or did not have EHR orders entered. Third, physicians may have been surveyed more than once for different or the same patient(s), but opinions may have shifted over time. Fourth, some items such as time needed to be written in the survey and were not specifically asked. This could have resulted in under‐reporting. Finally, we did not assess the clinical outcomes of thoracenteses in this study, although earlier work shows that residents who complete SBML have safety outcomes similar to IR.[1, 6]
In summary, IM residents who complete thoracentesis SBML demonstrate improved clinical skills and are more likely to perform bedside procedures. In an era of bundled payments, rethinking current care models to promote cost‐effective care is necessary. We believe providing additional education, training, and support to hospitalist physicians to promote bedside procedures is a promising strategy that warrants further study.
Acknowledgements
The authors acknowledge Drs. Douglas Vaughan and Kevin O'Leary for their support and encouragement of this work. The authors also thank the internal medicine residents at Northwestern for their dedication to patient care.
Disclosures: This project was supported by grant R18HS021202‐01 from the Agency for Healthcare Research and Quality (AHRQ). AHRQ had no role in the preparation, review, or approval of the manuscript. Trial Registration:
- , , , , . Thoracentesis procedures at university hospitals: comparing outcomes by specialty. Jt Comm J Qual Patient Saf. 2015;42(1):34–40.
- American Board of Internal Medicine. Internal medicine policies. Available at: http://www.abim.org/certification/policies/internal‐medicine‐subspecialty‐policies/internal‐medicine.aspx. Accessed March 9, 2016.
- , , , . Pneumothorax following thoracentesis: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(4):332–339.
- , , . National fluid shifts: fifteen‐year trends in paracentesis and thoracentesis procedures. J Am Coll Radiol. 2010;7(11):859–864.
- , , , et al. Cost savings of performing paracentesis procedures at the bedside after simulation‐based education. Simul Healthc. 2014;9(5):312–318.
- , , , , . Clinical outcomes after bedside and interventional radiology paracentesis procedures. Am J Med. 2013;126(4):349–356.
- , , , et al. Simulation technology for health care professional skills training and assessment. JAMA. 1999;282(9):861–866.
- , , , et al. Cost savings from reduced catheter‐related bloodstream infection after simulation‐based education for residents in a medical intensive care unit. Simul Healthc. 2010;5(2):98–102.
- , , , , . Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice. J Hosp Med. 2008;3(1):48–54.
- , , , , . Procedures performed by hospitalist and non‐hospitalist general internists. J Gen Intern Med. 2010;25(5):448–452.
- , , , , . Core competencies in hospital medicine: development and methodology. J Hosp Med. 2006;1(suppl 1):48–56.
- , , , , , . Specialties performing paracentesis procedures at university hospitals: implications for training and certification. J Hosp Med. 2014;9(3):162–168.
- , , , , , . Are we providing patient‐centered care? Preferences about paracentesis and thoracentesis procedures. Patient Exp J. 2014;1(2):94–103. Available at: http://pxjournal.org/cgi/viewcontent.cgi?article=1024
- , , , , . Thoracentesis procedures at university hospitals: comparing outcomes by specialty. Jt Comm J Qual Patient Saf. 2015;42(1):34–40.
- American Board of Internal Medicine. Internal medicine policies. Available at: http://www.abim.org/certification/policies/internal‐medicine‐subspecialty‐policies/internal‐medicine.aspx. Accessed March 9, 2016.
- , , , . Pneumothorax following thoracentesis: a systematic review and meta‐analysis. Arch Intern Med. 2010;170(4):332–339.
- , , . National fluid shifts: fifteen‐year trends in paracentesis and thoracentesis procedures. J Am Coll Radiol. 2010;7(11):859–864.
- , , , et al. Cost savings of performing paracentesis procedures at the bedside after simulation‐based education. Simul Healthc. 2014;9(5):312–318.
- , , , , . Clinical outcomes after bedside and interventional radiology paracentesis procedures. Am J Med. 2013;126(4):349–356.
- , , , et al. Simulation technology for health care professional skills training and assessment. JAMA. 1999;282(9):861–866.
- , , , et al. Cost savings from reduced catheter‐related bloodstream infection after simulation‐based education for residents in a medical intensive care unit. Simul Healthc. 2010;5(2):98–102.
- , , , , . Mastery learning of thoracentesis skills by internal medicine residents using simulation technology and deliberate practice. J Hosp Med. 2008;3(1):48–54.
- , , , , . Procedures performed by hospitalist and non‐hospitalist general internists. J Gen Intern Med. 2010;25(5):448–452.
- , , , , . Core competencies in hospital medicine: development and methodology. J Hosp Med. 2006;1(suppl 1):48–56.
- , , , , , . Specialties performing paracentesis procedures at university hospitals: implications for training and certification. J Hosp Med. 2014;9(3):162–168.
- , , , , , . Are we providing patient‐centered care? Preferences about paracentesis and thoracentesis procedures. Patient Exp J. 2014;1(2):94–103. Available at: http://pxjournal.org/cgi/viewcontent.cgi?article=1024