User login
Internal Medicine (IM) residency graduates are expected to manage a wide range of acute clinical events.1 Urgent and emergent inpatient situations require a broad knowledge base for rapid bedside diagnosis, yet the essential clinical skills required to manage acute clinical events pose a unique training challenge given the rarity and high-stakes nature of several such emergencies. For example, in three years of residency, a trainee may never have the opportunity to manage anaphylaxis, yet IM graduates must be able to recognize and quickly initiate proper lifesaving treatment for this relatively rare event2 when it does occur.
In an era of work-hour limitations and heightened trainee supervision, residents perceive diminished familiarity with several clinical situations3-5 and may feel unprepared to handle crisis events such as cardiac arrest.6 Given the sporadic nature of clinical medicine, many residents may not be exposed to certain acute inpatient clinical scenarios by the end of their training, a potentially critical education gap. To our knowledge, IM residents’ level of exposure to acute clinical events has not previously been studied. The aims of this study were to develop an instrument aimed at assessing IM residents’ exposure to hospital acute clinical events at a large academic medical center and to investigate the relationship between exposure and confidence in managing these events.
METHODS
Survey Development
We reviewed the Massachusetts General Hospital (MGH) IM residency program curriculum (including simulation, conferences, and other didactics), the American Board of Internal Medicine certification requirements (primarily related to Advanced Cardiac Life Support [ACLS]), and the MGH inpatient rapid response events and gained input from the IM program leadership to develop a list of 50 acute clinical events that a graduating resident may be expected to manage independently (Box 1, Supplementary Appendix).7-9 We then developed a survey assessing residents’ exposure to and confidence in managing such events. To classify the level of exposure, residents were asked to distinguish whether they had managed these events during a simulation session, inpatient as a part of a team, or inpatient independently. At our institution, IM postgraduate year 1 (PGY-1) interns manage a floor of patients overnight under a senior resident’s supervision, PGY-2 residents manage a team of several interns often without attending presence on ward rounds,10 and senior PGY-3 or -4 residents are expected to lead the hospital’s rapid response and code team and triage decompensating patients to the intensive care unit. Therefore, there are ample opportunities for IM residents to manage conditions independently (ie, in a direct leadership role) with attending supervision. House officers’ role in medical management, including calling appropriate subspecialty consultation, depends on the clinical condition; for example, a graduating senior resident would be expected to evaluate comprehensively a hypotensive patient and diagnose tension pneumothorax (while calling interventional pulmonary support for needle decompression and chest tube placement) and independently run an ACLS algorithm in the case of an unstable arrhythmia or cardiac arrest.
Residents were also asked to rate their perceived confidence in managing each condition independently on a five-point scale (ranging from “definitely cannot manage this condition independently” to “definitely can manage this condition independently”). We refined the survey instrument through a collaborative, iterative review process, including cognitive interviews and piloting with IM subspecialty fellows.
Participants and Data Collection
All IM residents at the Massachusetts General Hospital were invited to participate in the study. The study was conducted in May 2015 to reflect training throughout the prior academic year(s) and allow us to evaluate graduating residents’ exposures across all prior years of training. The instrument was administered anonymously via a web-based survey tool, Qualtrics (Provo, Utah). The study was approved as exempt by the Partners Institutional Review Board.
Data Analysis
Residents’ self-reported exposure to hospital acute events was classified into the following six ordinal categories: (1) never seen (have never seen the condition under any circumstances); (2) simulation alone (have managed the condition only during a mannequin-simulated patient case); (3) team alone (have managed the condition inpatient as a part of a team of providers, not in a primary leadership role); (4) team plus simulation; (5) independently (have managed the condition inpatient alone or in a primary leadership role); and (6) independently plus simulation. Residents’ self-reported exposure was examined for each postgraduate year (PGY) class both in aggregate and for each individual acute event. We sought to identify events that the majority of residents had managed independently (85% of residents or greater) and less common events that at least 15% of residents had never experienced.
We also examined residents’ self-reported confidence for each PGY class in aggregate and for each clinical acute scenario. Confidence was investigated in a dichotomized manner with a “definitely can” rating indicating “Confident” and with “probably can,” “neutral,” “probably cannot,” or “definitely cannot” ratings indicating “Not Confident” to manage the condition independently. Dichotomization thus allowed us to set a high bar for confidence, reflecting the self-perceived ability of the residents to manage the conditions as future independent physicians.
We used logistic regression models with the generalized estimating equations (GEE) approach to take into account the repeated measures of 50 clinical acute clinical events assessed for each resident. We compared the distribution of self-reported exposure and confidence among different PGY classes and examined the relationship between confidence and self-reported exposure stratified by level of training. We also assessed the independent effect of exposure on confidence controlling for level of training in a multivariable logistic regression model.
RESULTS
A total of 140 of 170 IM residents completed the survey (82% overall response rate: 72% of all PGY-1 residents, 86% of PGY-2 residents, and 89% of PGY-3/4 residents). In total, 41 PGY-1 residents (29% of respondents), 50 PGY-2 residents (36%), and 49 PGY-3 or PGY-4 residents (35%) participated. The majority of residents were in the Categorical IM training track (106 residents, 76% of respondents), whereas the remainder of respondents were in various subspecialty training tracks within our IM residency program, including Primary Care (14 residents, 10%), and four-year tracks, including Global Health (six residents, 4%), and Medicine-Pediatrics (14 residents, 10%).
Assessment of Exposure
Residents reported increasingly independent exposures as they progressed through residency training. PGY-1 residents on average had never seen 16.3% of the 50 acute events, whereas PGY-3/4 residents had never seen only 4.0% of the events (P < .0001). PGY-1 residents had managed 31.3% of events independently (or both independently and in simulation) as opposed to 71.7% of events for PGY-3/4 residents (P < .0001). Simulation alone accounted for a substantial proportion of exposures (16.4%) for PGY-1 residents, but this was significantly lower for PGY-2 or PGY-3/4 residents (P < .0001), who reported a greater percentage of exposures in nonsimulation clinical scenarios either independently or as a part of an inpatient team. There were no outlier residents who reported lower exposure compared with their PGY peers.
There was a wide spectrum of resident-reported exposures when individual acute events were examined (Table, full data in Supplementary Appendix Table 1). Events with the highest levels of exposure, which >85% of PGY-1 residents had managed independently, included alcohol withdrawal, chronic obstructive pulmonary disease exacerbation, rapid atrial fibrillation, agitated delirium, hypertensive urgency, and hyperkalemia. Events with the lowest levels of exposure, which at least 15% of graduating residents had never encountered in the hospital, included the following eight of 50 events (16%): torsades de pointes (51% of PGY-3/4 residents), acute mechanical valve failure (49%), tension pneumothorax (38.8%), use of emergency transcutaneous pacing (38.8%), elevated intracranial pressure (ICP)/herniation (24.5%), aortic dissection (22.4%), cord compression (16.3%), and use of emergency cardioversion (16.3%). Several PGY-3/4 residents had managed several of these events only in mannequin simulations, including torsades de pointes (41%), transcutaneous pacing (33%), and tension pneumothorax (24%).
Assessment of Confidence
Both levels of training and exposure to acute events were associated with increased confidence in managing such events. PGY-1 residents felt confident in managing 24.9% of acute events independently, compared to 48.4% of events for PGY-2 residents and 72.5% of events for PGY-3/4 residents (P < .0001). There was considerable variation in confidence among the individual acute events (Supplementary Appendix Table 2). A majority of graduating PGY-3/4 residents did not feel confident in managing the following 10 of the 50 events (20%): use of emergency cardioversion, aortic dissection, thrombotic thrombocytopenic purpura/hemolytic uremic syndrome (TTP/HUS), torsades de pointes, posterior reversible encephalopathy syndrome (PRES), intracranial hemorrhage, use of emergency transcutaneous pacing, tension pneumothorax, elevated ICP/herniation, and acute mechanical valve failure.
Residents’ self-reported confidence also correlated with level of exposure. There was a significant increase in resident confidence with increasingly independent exposure stratified by level of training (Figure; all with P < .0001). In the multivariable logistic regression model, increasing exposure correlated with increased resident confidence (P < .0001) while controlling for PGY year (P = .001).
DISCUSSION
We developed an instrument to assess resident exposure to and confidence in managing 50 inpatient acute clinical events. Both exposure and level of training were associated with increasing resident confidence. We identified specific events with low levels of exposure and confidence that could be targeted for educational interventions.
To our knowledge, this is the first study to examine IM residents’ exposure to and confidence in managing a wide range of inpatient acute clinical events. A primary goal of residency is to provide physicians-in-training graduated responsibility to prepare them for eventual independent practice. Although our survey confirmed that IM residents’ exposure and confidence significantly increased as they advanced through training (a not unexpected finding), our data also show that even after controlling for year in training, independent exposures significantly correlated with increased confidence. This speaks to the importance of preserving opportunities for residents to manage critical events in a supported manner, an admittedly challenging prospect given the oft-competing calls for supervision of and mentored feedback for trainees.11
Despite identifying independent exposure as an important factor that impacts resident confidence, we found that there was still a substantial proportion of events (28.3%) that senior medical residents near the end of their training had not managed independently in a primary leadership role. Although our study was not designed to determine the reasons for this varied resident exposure, possible explanations may include the relative rarity of certain acute clinical events compared with others, or less likely the effect of duty hour limitations, attending supervision of trainees, or programmatic changes in resident leadership responsibilities. Whatever the cause, this finding uniquely identifies an area for improvement to prevent new attending physicians from feeling unprepared to manage potentially critical emergencies.
An important goal of our study was to develop an instrument that would enable training programs to identify their learning needs. Both program-wide and individual assessments of resident case exposure and confidence are essential for identifying such learning needs and areas for curricular development. Program-wide assessments can spur an important debate about program goals and requirements with respect to what scenarios residents must be able to manage competently by graduation.12 In addition, such assessments can help individualize learning exposures based on a specific learner’s needs and career goals. The administration of our survey instrument required minimal resources, and the high response rate in our study suggests that other programs can implement our instrument to accomplish these goals.
Alternative methods, such as electronic learning portfolios (efolios), can be utilized to assess resident case exposure. In comparison to our survey instrument, efolios limit recall bias by utilizing case logs and have additional capabilities such as compiling evaluations and enabling trainees to set learning goals. However, there are considerable barriers to the effective use of efolios, including software cost, learner attitudes, and time constraints.13 Tools such as our end-of-year assessment offer an alternative method that limits these barriers.
Once educational growth opportunities have been identified through survey-based or other methods, residency programs must determine how to optimize curricula for the needs and career goals of their trainees. We found considerable overlap among conditions that graduating residents had both limited exposure to and low confidence in managing (eg, torsades de pointes, tension pneumothorax, and emergency cardioversion), which are logical topics for future curriculum development. We also identified a few conditions (including PRES, TTP/HUS, and intracranial hemorrhage) that graduating residents did not feel confident in managing despite a relatively higher reported level of exposure. Whether to focus specific educational interventions on the most rare or most commonly encountered acute clinical events is likely to be a topic of debate among individual training programs, but the results of our survey indicate that there is likely to be educational benefit to both strategies.
Residency programs can employ a variety of modalities to enhance learner exposure and confidence in managing clinical scenarios that are deemed important by the program, including didactics, simulation, and changes in program structure. There is a substantial literature on the use of dedicated curricula for crisis management and the use of simulation as a training tool for responding to acute clinical events in multiple specialties14-24 and in nonmedical domains such as aviation.25-27 Simulation has been shown to improve residents’ clinical skills and comfort level with some acute events28-30 and may even be superior to traditional clinical medical education.31 In addition, programs can utilize targeted clinical experiences such as intensive care unit and subspecialty rotations32,33 in an effort to customize educational interventions to fill identified gaps in learner exposure or confidence.
Our study has several limitations. First, we investigated a single large IM residency program at a quaternary academic medical center, and therefore, our findings may not be externally generalizable to all IM residencies or other medical specialties. Our unique peer-led simulation curriculum, including 16 PGY-1 and 8 PGY-2 cases chosen based on clinical rotations at Massachusetts General Hospital,7 likely impacted residents’ exposure to simulation that is specific to our institution. However, although specific inpatient acute events may vary among other institutions, our finding that graduating residents still reported gaps in their clinical experience is likely generalizable to other programs given the varied and unpredictable nature of ward medicine training. In addition, our survey tool was simple to administer and could be tailored to reflect the acute events and training needs relevant to other residency programs, specialties, and institutions. Second, the retrospective nature of our study may be subject to participants’ recall bias. We did not restrict our survey questions to urgent conditions managed only on IM hospital wards and some may have been experienced in the emergency room or intensive care units; however, these exposures are still relevant as key components of IM training. Third, our list of 50 acute clinical events was intentionally broad and included several conditions that require multidisciplinary subspecialist consultation, which could have impacted residents’ self-report of “independent” exposures. However, these scenarios are ones that hospitalists may independently recognize and stabilize, engaging appropriate specialists. Fourth, we were not able to validate residents’ self-reported exposures against other measures of the frequency of housestaff management of acute events (such as billing data or patient logs) as this information is not routinely collected. We also did not attempt to identify the reasons underlying the variation seen in resident exposure and confidence for individual acute events, but as a needs assessment, this was beyond the scope of our study. Finally, our assessment of resident confidence was subjective and we were not able to assess competence, with prior studies demonstrating conflicting results regarding the relationship between self-reported proficiency and observed competence.34-36 Future studies are needed to investigate whether case exposure assessment leads to changes in residency curricula and whether such curricula increase resident confidence and competence in managing hospital acute clinical events.
CONCLUSION
We developed an easy-to-administer tool to assess IM residents’ exposure to and confidence in managing inpatient acute events. We found that both significantly increased as residents advanced through training, and self-reported confidence additionally correlated with level of exposure independent of PGY class. We identified several specific inpatient acute clinical events with low levels of resident exposure and confidence that can serve as targets for future IM residency curriculum development. Future studies assessing the impact of such curricula on resident confidence and competence are needed.
Disclosures
The authors declare no conflict of interest.
1. ACGME. The Internal Medicine Milestone Project. A joint initiative of the Accreditation Council for Graduate Medical Education and the American Board of Internal Medicine. http://www.acgme.org/Portals/0/PDFs/Milestones/InternalMedicineMilestones.pdf. Accessed July 14, 2018.
2. Neugut AI, Ghatak AT, Miller RL. Anaphylaxis in the United States: an investigation into its epidemiology. Arch Intern Med. 2001;161(1):15-21. PubMed
3. Lin GA, Beck DC, Stewart AL, Garbutt JM. Resident perceptions of the impact of work hour limitations. J Gen Intern Med. 2007;22(7):969-975. PubMed
4. Bolster L, Rourke L. The effect of restricting residents’ duty hours on patient safety, resident well-being, and resident education: an updated systematic review. J Grad Med Educ. 2015;7(3):349-363. PubMed
5. Wayne DB, Hauer KE. Counting quality, not hours: understanding the impact of duty hour reform on internal medicine residency education. J Gen Intern Med. 2012;27(11):1400-1401. PubMed
6. Hayes CW, Rhee A, Detsky ME, Leblanc VR, Wax RS. Residents feel unprepared and unsupervised as leaders of cardiac arrest teams in teaching hospitals: a survey of internal medicine residents. Crit Care Med. 2007;35(7):1668-1672. PubMed
7. Mathai SK, Miloslavsky EM, Contreras-Valdes FM, et al. How we implemented a resident-led medical simulation curriculum in a large internal medicine residency program. Med Teach. 2014;36(4):279-283. PubMed
8. The American Board of Internal Medicine. Internal Medicine Policies. http://www.abim.org/certification/policies/internal-medicine-subspecialty-policies/internal-medicine.aspx. Accessed January 24, 2018.
9. Sinz E, Navarro K, Soderberg ES. Advanced Cardiovascular Life Support. Dallas, TX: American Heart Association; 2011:1-183.
10. Finn KM, Metlay JP, Chang Y, et al. Effect of increased inpatient attending physician supervision on medical errors, patient safety, and resident education: a randomized clinical trial. JAMA Intern Med. 2018;178(7):952-959. PubMed
11. Happel JP, Ritter JB, Neubauer BE. Optimizing the balance between supervision and autonomy in training. JAMA Intern Med. 2018;178(7):959-960. PubMed
12. Fitzgibbons JP, Bordley DR, Berkowitz LR, Miller BW, Henderson MC. Redesigning residency education in internal medicine: a position paper from the association of program directors in internal medicine. Ann Intern Med. 2006;144(12):920. PubMed
13. Dekker H, Driessen E, Braak Ter E, et al. Mentoring portfolio use in undergraduate and postgraduate medical education. Med Teach. 2009;31(10):903-909. PubMed
14. Sica GT, Barron DM, Blum R, Frenna TH, Raemer DB. Computerized realistic simulation: a teaching module for crisis management in radiology. AJR Am J Roentgenol. 1999;172(2):301-304. PubMed
15. DeAnda A, Gaba DM. Role of experience in the response to simulated critical incidents. Anesth Analg. 1991;72(3):308-315. PubMed
16. Gaba DM, Maxwell M, DeAnda A. Anesthetic mishaps. Anesthesiology. 1987;66(5):670-676. PubMed
17. Arora S, Hull L, Fitzpatrick M, Sevdalis N, Birnbach DJ. Crisis management on surgical wards. Ann Surg. 2015;261(5):888-893. PubMed
18. Zirkle M, Blum R, Raemer DB, Healy G, Roberson DW. Teaching emergency airway management using medical simulation: a pilot program. Laryngoscope. 2005;115(3):495-500. PubMed
19. Volk MS, Ward J, Irias N, Navedo A, Pollart J, Weinstock PH. Using medical simulation to teach crisis resource management and decision-making skills to otolaryngology housestaff. Otolaryngol Head Neck Surg. 2011;145(1):35-42. PubMed
20. Bank I, Snell L, Bhanji F. Pediatric crisis resource management training improves emergency medicine trainees’ perceived ability to manage emergencies and ability to identify teamwork errors. Pediatr Emerg Care. 2014;30(12):879-883. PubMed
21. Blackwood J, Duff JP, Nettel-Aguirre A, Djogovic D, Joynt C. Does teaching crisis resource management skills improve resuscitation performance in pediatric residents?. Pediatr Crit Care Med. 2014;15(4):e168-e174. PubMed
22. Daniels K, Lipman S, Harney K, Arafeh J, Druzin M. Use of simulation based team training for obstetric crises in resident education. Simul Healthc. 2008;3(3):154-160. PubMed
23. Isaak RS, Stiegler MP. Review of crisis resource management (CRM) principles in the setting of intraoperative malignant hyperthermia. J Anesth. 2016;30(2):298-306. PubMed
24. Gaba D, DeAnda A. The response of anesthesia trainees to simulated critical incidents. Surv Anesth. 1989;33(6):349. PubMed
25. Ornato JP, Peberdy MA. Applying lessons from commercial aviation safety and operations to resuscitation. Resuscitation. 2014;85(2):173-176. PubMed
26. Hamman WR. Commentary: will simulation fly in medicine as it has in aviation? BMJ Qual Saf. 2004;13(5):397-399. PubMed
27. Littlepage GE, Hein MB, Richard G Moffett I, Craig PA, Georgiou AM. Team training for dynamic cross-functional teams in aviation: behavioral, cognitive, and performance outcomes. Hum Factors. 2016;58(8):1275-1288. PubMed
28. Wayne DB, Butter J, Siddall VJ, et al. Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice. J Gen Intern Med. 2006;21(3):251-256. PubMed
29. Heal
30. Kory PD, Eisen LA, Adachi M, Ribaudo VA, Rosenthal ME, Mayo PH. Initial airway management skills of senior residents. Chest. 2015;132(6):1927-1931. PubMed
31. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011;86(6):706-711. PubMed
32. Almoosa KF, Goldenhar LM, Puchalski J, Ying J, Panos RJ. Critical care education during internal medicine residency: a national survey. J Grad Med Educ. 2010;2(4):555-561. PubMed
33. Katz SJ, Oswald AE. How confident are internal medicine residents in rheumatology versus other common internal medicine clinical skills: an issue of training time or exposure? Clin Rheumatol. 2011;30(8):1081-1093. PubMed
34. Barnsley L, Lyon PM, Ralston SJ, et al. Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence. Med Educ. 2004;38(4):358-367. PubMed
35. Dehmer JJ, Amos KD, Farrell TM, Meyer AA, Newton WP, Meyers MO. Competence and confidence with basic procedural skills: the experience and opinions of fourth-year medical students at a single institution. Acad Med. 2013;88(5):682-687. PubMed
36. Wu EH, Elnicki DM, Alper EJ, et al. Procedural and interpretive skills of medical students: experiences and attitudes of fourth-year students. Acad Med. 2008;83(10):S63-S67. PubMed
Internal Medicine (IM) residency graduates are expected to manage a wide range of acute clinical events.1 Urgent and emergent inpatient situations require a broad knowledge base for rapid bedside diagnosis, yet the essential clinical skills required to manage acute clinical events pose a unique training challenge given the rarity and high-stakes nature of several such emergencies. For example, in three years of residency, a trainee may never have the opportunity to manage anaphylaxis, yet IM graduates must be able to recognize and quickly initiate proper lifesaving treatment for this relatively rare event2 when it does occur.
In an era of work-hour limitations and heightened trainee supervision, residents perceive diminished familiarity with several clinical situations3-5 and may feel unprepared to handle crisis events such as cardiac arrest.6 Given the sporadic nature of clinical medicine, many residents may not be exposed to certain acute inpatient clinical scenarios by the end of their training, a potentially critical education gap. To our knowledge, IM residents’ level of exposure to acute clinical events has not previously been studied. The aims of this study were to develop an instrument aimed at assessing IM residents’ exposure to hospital acute clinical events at a large academic medical center and to investigate the relationship between exposure and confidence in managing these events.
METHODS
Survey Development
We reviewed the Massachusetts General Hospital (MGH) IM residency program curriculum (including simulation, conferences, and other didactics), the American Board of Internal Medicine certification requirements (primarily related to Advanced Cardiac Life Support [ACLS]), and the MGH inpatient rapid response events and gained input from the IM program leadership to develop a list of 50 acute clinical events that a graduating resident may be expected to manage independently (Box 1, Supplementary Appendix).7-9 We then developed a survey assessing residents’ exposure to and confidence in managing such events. To classify the level of exposure, residents were asked to distinguish whether they had managed these events during a simulation session, inpatient as a part of a team, or inpatient independently. At our institution, IM postgraduate year 1 (PGY-1) interns manage a floor of patients overnight under a senior resident’s supervision, PGY-2 residents manage a team of several interns often without attending presence on ward rounds,10 and senior PGY-3 or -4 residents are expected to lead the hospital’s rapid response and code team and triage decompensating patients to the intensive care unit. Therefore, there are ample opportunities for IM residents to manage conditions independently (ie, in a direct leadership role) with attending supervision. House officers’ role in medical management, including calling appropriate subspecialty consultation, depends on the clinical condition; for example, a graduating senior resident would be expected to evaluate comprehensively a hypotensive patient and diagnose tension pneumothorax (while calling interventional pulmonary support for needle decompression and chest tube placement) and independently run an ACLS algorithm in the case of an unstable arrhythmia or cardiac arrest.
Residents were also asked to rate their perceived confidence in managing each condition independently on a five-point scale (ranging from “definitely cannot manage this condition independently” to “definitely can manage this condition independently”). We refined the survey instrument through a collaborative, iterative review process, including cognitive interviews and piloting with IM subspecialty fellows.
Participants and Data Collection
All IM residents at the Massachusetts General Hospital were invited to participate in the study. The study was conducted in May 2015 to reflect training throughout the prior academic year(s) and allow us to evaluate graduating residents’ exposures across all prior years of training. The instrument was administered anonymously via a web-based survey tool, Qualtrics (Provo, Utah). The study was approved as exempt by the Partners Institutional Review Board.
Data Analysis
Residents’ self-reported exposure to hospital acute events was classified into the following six ordinal categories: (1) never seen (have never seen the condition under any circumstances); (2) simulation alone (have managed the condition only during a mannequin-simulated patient case); (3) team alone (have managed the condition inpatient as a part of a team of providers, not in a primary leadership role); (4) team plus simulation; (5) independently (have managed the condition inpatient alone or in a primary leadership role); and (6) independently plus simulation. Residents’ self-reported exposure was examined for each postgraduate year (PGY) class both in aggregate and for each individual acute event. We sought to identify events that the majority of residents had managed independently (85% of residents or greater) and less common events that at least 15% of residents had never experienced.
We also examined residents’ self-reported confidence for each PGY class in aggregate and for each clinical acute scenario. Confidence was investigated in a dichotomized manner with a “definitely can” rating indicating “Confident” and with “probably can,” “neutral,” “probably cannot,” or “definitely cannot” ratings indicating “Not Confident” to manage the condition independently. Dichotomization thus allowed us to set a high bar for confidence, reflecting the self-perceived ability of the residents to manage the conditions as future independent physicians.
We used logistic regression models with the generalized estimating equations (GEE) approach to take into account the repeated measures of 50 clinical acute clinical events assessed for each resident. We compared the distribution of self-reported exposure and confidence among different PGY classes and examined the relationship between confidence and self-reported exposure stratified by level of training. We also assessed the independent effect of exposure on confidence controlling for level of training in a multivariable logistic regression model.
RESULTS
A total of 140 of 170 IM residents completed the survey (82% overall response rate: 72% of all PGY-1 residents, 86% of PGY-2 residents, and 89% of PGY-3/4 residents). In total, 41 PGY-1 residents (29% of respondents), 50 PGY-2 residents (36%), and 49 PGY-3 or PGY-4 residents (35%) participated. The majority of residents were in the Categorical IM training track (106 residents, 76% of respondents), whereas the remainder of respondents were in various subspecialty training tracks within our IM residency program, including Primary Care (14 residents, 10%), and four-year tracks, including Global Health (six residents, 4%), and Medicine-Pediatrics (14 residents, 10%).
Assessment of Exposure
Residents reported increasingly independent exposures as they progressed through residency training. PGY-1 residents on average had never seen 16.3% of the 50 acute events, whereas PGY-3/4 residents had never seen only 4.0% of the events (P < .0001). PGY-1 residents had managed 31.3% of events independently (or both independently and in simulation) as opposed to 71.7% of events for PGY-3/4 residents (P < .0001). Simulation alone accounted for a substantial proportion of exposures (16.4%) for PGY-1 residents, but this was significantly lower for PGY-2 or PGY-3/4 residents (P < .0001), who reported a greater percentage of exposures in nonsimulation clinical scenarios either independently or as a part of an inpatient team. There were no outlier residents who reported lower exposure compared with their PGY peers.
There was a wide spectrum of resident-reported exposures when individual acute events were examined (Table, full data in Supplementary Appendix Table 1). Events with the highest levels of exposure, which >85% of PGY-1 residents had managed independently, included alcohol withdrawal, chronic obstructive pulmonary disease exacerbation, rapid atrial fibrillation, agitated delirium, hypertensive urgency, and hyperkalemia. Events with the lowest levels of exposure, which at least 15% of graduating residents had never encountered in the hospital, included the following eight of 50 events (16%): torsades de pointes (51% of PGY-3/4 residents), acute mechanical valve failure (49%), tension pneumothorax (38.8%), use of emergency transcutaneous pacing (38.8%), elevated intracranial pressure (ICP)/herniation (24.5%), aortic dissection (22.4%), cord compression (16.3%), and use of emergency cardioversion (16.3%). Several PGY-3/4 residents had managed several of these events only in mannequin simulations, including torsades de pointes (41%), transcutaneous pacing (33%), and tension pneumothorax (24%).
Assessment of Confidence
Both levels of training and exposure to acute events were associated with increased confidence in managing such events. PGY-1 residents felt confident in managing 24.9% of acute events independently, compared to 48.4% of events for PGY-2 residents and 72.5% of events for PGY-3/4 residents (P < .0001). There was considerable variation in confidence among the individual acute events (Supplementary Appendix Table 2). A majority of graduating PGY-3/4 residents did not feel confident in managing the following 10 of the 50 events (20%): use of emergency cardioversion, aortic dissection, thrombotic thrombocytopenic purpura/hemolytic uremic syndrome (TTP/HUS), torsades de pointes, posterior reversible encephalopathy syndrome (PRES), intracranial hemorrhage, use of emergency transcutaneous pacing, tension pneumothorax, elevated ICP/herniation, and acute mechanical valve failure.
Residents’ self-reported confidence also correlated with level of exposure. There was a significant increase in resident confidence with increasingly independent exposure stratified by level of training (Figure; all with P < .0001). In the multivariable logistic regression model, increasing exposure correlated with increased resident confidence (P < .0001) while controlling for PGY year (P = .001).
DISCUSSION
We developed an instrument to assess resident exposure to and confidence in managing 50 inpatient acute clinical events. Both exposure and level of training were associated with increasing resident confidence. We identified specific events with low levels of exposure and confidence that could be targeted for educational interventions.
To our knowledge, this is the first study to examine IM residents’ exposure to and confidence in managing a wide range of inpatient acute clinical events. A primary goal of residency is to provide physicians-in-training graduated responsibility to prepare them for eventual independent practice. Although our survey confirmed that IM residents’ exposure and confidence significantly increased as they advanced through training (a not unexpected finding), our data also show that even after controlling for year in training, independent exposures significantly correlated with increased confidence. This speaks to the importance of preserving opportunities for residents to manage critical events in a supported manner, an admittedly challenging prospect given the oft-competing calls for supervision of and mentored feedback for trainees.11
Despite identifying independent exposure as an important factor that impacts resident confidence, we found that there was still a substantial proportion of events (28.3%) that senior medical residents near the end of their training had not managed independently in a primary leadership role. Although our study was not designed to determine the reasons for this varied resident exposure, possible explanations may include the relative rarity of certain acute clinical events compared with others, or less likely the effect of duty hour limitations, attending supervision of trainees, or programmatic changes in resident leadership responsibilities. Whatever the cause, this finding uniquely identifies an area for improvement to prevent new attending physicians from feeling unprepared to manage potentially critical emergencies.
An important goal of our study was to develop an instrument that would enable training programs to identify their learning needs. Both program-wide and individual assessments of resident case exposure and confidence are essential for identifying such learning needs and areas for curricular development. Program-wide assessments can spur an important debate about program goals and requirements with respect to what scenarios residents must be able to manage competently by graduation.12 In addition, such assessments can help individualize learning exposures based on a specific learner’s needs and career goals. The administration of our survey instrument required minimal resources, and the high response rate in our study suggests that other programs can implement our instrument to accomplish these goals.
Alternative methods, such as electronic learning portfolios (efolios), can be utilized to assess resident case exposure. In comparison to our survey instrument, efolios limit recall bias by utilizing case logs and have additional capabilities such as compiling evaluations and enabling trainees to set learning goals. However, there are considerable barriers to the effective use of efolios, including software cost, learner attitudes, and time constraints.13 Tools such as our end-of-year assessment offer an alternative method that limits these barriers.
Once educational growth opportunities have been identified through survey-based or other methods, residency programs must determine how to optimize curricula for the needs and career goals of their trainees. We found considerable overlap among conditions that graduating residents had both limited exposure to and low confidence in managing (eg, torsades de pointes, tension pneumothorax, and emergency cardioversion), which are logical topics for future curriculum development. We also identified a few conditions (including PRES, TTP/HUS, and intracranial hemorrhage) that graduating residents did not feel confident in managing despite a relatively higher reported level of exposure. Whether to focus specific educational interventions on the most rare or most commonly encountered acute clinical events is likely to be a topic of debate among individual training programs, but the results of our survey indicate that there is likely to be educational benefit to both strategies.
Residency programs can employ a variety of modalities to enhance learner exposure and confidence in managing clinical scenarios that are deemed important by the program, including didactics, simulation, and changes in program structure. There is a substantial literature on the use of dedicated curricula for crisis management and the use of simulation as a training tool for responding to acute clinical events in multiple specialties14-24 and in nonmedical domains such as aviation.25-27 Simulation has been shown to improve residents’ clinical skills and comfort level with some acute events28-30 and may even be superior to traditional clinical medical education.31 In addition, programs can utilize targeted clinical experiences such as intensive care unit and subspecialty rotations32,33 in an effort to customize educational interventions to fill identified gaps in learner exposure or confidence.
Our study has several limitations. First, we investigated a single large IM residency program at a quaternary academic medical center, and therefore, our findings may not be externally generalizable to all IM residencies or other medical specialties. Our unique peer-led simulation curriculum, including 16 PGY-1 and 8 PGY-2 cases chosen based on clinical rotations at Massachusetts General Hospital,7 likely impacted residents’ exposure to simulation that is specific to our institution. However, although specific inpatient acute events may vary among other institutions, our finding that graduating residents still reported gaps in their clinical experience is likely generalizable to other programs given the varied and unpredictable nature of ward medicine training. In addition, our survey tool was simple to administer and could be tailored to reflect the acute events and training needs relevant to other residency programs, specialties, and institutions. Second, the retrospective nature of our study may be subject to participants’ recall bias. We did not restrict our survey questions to urgent conditions managed only on IM hospital wards and some may have been experienced in the emergency room or intensive care units; however, these exposures are still relevant as key components of IM training. Third, our list of 50 acute clinical events was intentionally broad and included several conditions that require multidisciplinary subspecialist consultation, which could have impacted residents’ self-report of “independent” exposures. However, these scenarios are ones that hospitalists may independently recognize and stabilize, engaging appropriate specialists. Fourth, we were not able to validate residents’ self-reported exposures against other measures of the frequency of housestaff management of acute events (such as billing data or patient logs) as this information is not routinely collected. We also did not attempt to identify the reasons underlying the variation seen in resident exposure and confidence for individual acute events, but as a needs assessment, this was beyond the scope of our study. Finally, our assessment of resident confidence was subjective and we were not able to assess competence, with prior studies demonstrating conflicting results regarding the relationship between self-reported proficiency and observed competence.34-36 Future studies are needed to investigate whether case exposure assessment leads to changes in residency curricula and whether such curricula increase resident confidence and competence in managing hospital acute clinical events.
CONCLUSION
We developed an easy-to-administer tool to assess IM residents’ exposure to and confidence in managing inpatient acute events. We found that both significantly increased as residents advanced through training, and self-reported confidence additionally correlated with level of exposure independent of PGY class. We identified several specific inpatient acute clinical events with low levels of resident exposure and confidence that can serve as targets for future IM residency curriculum development. Future studies assessing the impact of such curricula on resident confidence and competence are needed.
Disclosures
The authors declare no conflict of interest.
Internal Medicine (IM) residency graduates are expected to manage a wide range of acute clinical events.1 Urgent and emergent inpatient situations require a broad knowledge base for rapid bedside diagnosis, yet the essential clinical skills required to manage acute clinical events pose a unique training challenge given the rarity and high-stakes nature of several such emergencies. For example, in three years of residency, a trainee may never have the opportunity to manage anaphylaxis, yet IM graduates must be able to recognize and quickly initiate proper lifesaving treatment for this relatively rare event2 when it does occur.
In an era of work-hour limitations and heightened trainee supervision, residents perceive diminished familiarity with several clinical situations3-5 and may feel unprepared to handle crisis events such as cardiac arrest.6 Given the sporadic nature of clinical medicine, many residents may not be exposed to certain acute inpatient clinical scenarios by the end of their training, a potentially critical education gap. To our knowledge, IM residents’ level of exposure to acute clinical events has not previously been studied. The aims of this study were to develop an instrument aimed at assessing IM residents’ exposure to hospital acute clinical events at a large academic medical center and to investigate the relationship between exposure and confidence in managing these events.
METHODS
Survey Development
We reviewed the Massachusetts General Hospital (MGH) IM residency program curriculum (including simulation, conferences, and other didactics), the American Board of Internal Medicine certification requirements (primarily related to Advanced Cardiac Life Support [ACLS]), and the MGH inpatient rapid response events and gained input from the IM program leadership to develop a list of 50 acute clinical events that a graduating resident may be expected to manage independently (Box 1, Supplementary Appendix).7-9 We then developed a survey assessing residents’ exposure to and confidence in managing such events. To classify the level of exposure, residents were asked to distinguish whether they had managed these events during a simulation session, inpatient as a part of a team, or inpatient independently. At our institution, IM postgraduate year 1 (PGY-1) interns manage a floor of patients overnight under a senior resident’s supervision, PGY-2 residents manage a team of several interns often without attending presence on ward rounds,10 and senior PGY-3 or -4 residents are expected to lead the hospital’s rapid response and code team and triage decompensating patients to the intensive care unit. Therefore, there are ample opportunities for IM residents to manage conditions independently (ie, in a direct leadership role) with attending supervision. House officers’ role in medical management, including calling appropriate subspecialty consultation, depends on the clinical condition; for example, a graduating senior resident would be expected to evaluate comprehensively a hypotensive patient and diagnose tension pneumothorax (while calling interventional pulmonary support for needle decompression and chest tube placement) and independently run an ACLS algorithm in the case of an unstable arrhythmia or cardiac arrest.
Residents were also asked to rate their perceived confidence in managing each condition independently on a five-point scale (ranging from “definitely cannot manage this condition independently” to “definitely can manage this condition independently”). We refined the survey instrument through a collaborative, iterative review process, including cognitive interviews and piloting with IM subspecialty fellows.
Participants and Data Collection
All IM residents at the Massachusetts General Hospital were invited to participate in the study. The study was conducted in May 2015 to reflect training throughout the prior academic year(s) and allow us to evaluate graduating residents’ exposures across all prior years of training. The instrument was administered anonymously via a web-based survey tool, Qualtrics (Provo, Utah). The study was approved as exempt by the Partners Institutional Review Board.
Data Analysis
Residents’ self-reported exposure to hospital acute events was classified into the following six ordinal categories: (1) never seen (have never seen the condition under any circumstances); (2) simulation alone (have managed the condition only during a mannequin-simulated patient case); (3) team alone (have managed the condition inpatient as a part of a team of providers, not in a primary leadership role); (4) team plus simulation; (5) independently (have managed the condition inpatient alone or in a primary leadership role); and (6) independently plus simulation. Residents’ self-reported exposure was examined for each postgraduate year (PGY) class both in aggregate and for each individual acute event. We sought to identify events that the majority of residents had managed independently (85% of residents or greater) and less common events that at least 15% of residents had never experienced.
We also examined residents’ self-reported confidence for each PGY class in aggregate and for each clinical acute scenario. Confidence was investigated in a dichotomized manner with a “definitely can” rating indicating “Confident” and with “probably can,” “neutral,” “probably cannot,” or “definitely cannot” ratings indicating “Not Confident” to manage the condition independently. Dichotomization thus allowed us to set a high bar for confidence, reflecting the self-perceived ability of the residents to manage the conditions as future independent physicians.
We used logistic regression models with the generalized estimating equations (GEE) approach to take into account the repeated measures of 50 clinical acute clinical events assessed for each resident. We compared the distribution of self-reported exposure and confidence among different PGY classes and examined the relationship between confidence and self-reported exposure stratified by level of training. We also assessed the independent effect of exposure on confidence controlling for level of training in a multivariable logistic regression model.
RESULTS
A total of 140 of 170 IM residents completed the survey (82% overall response rate: 72% of all PGY-1 residents, 86% of PGY-2 residents, and 89% of PGY-3/4 residents). In total, 41 PGY-1 residents (29% of respondents), 50 PGY-2 residents (36%), and 49 PGY-3 or PGY-4 residents (35%) participated. The majority of residents were in the Categorical IM training track (106 residents, 76% of respondents), whereas the remainder of respondents were in various subspecialty training tracks within our IM residency program, including Primary Care (14 residents, 10%), and four-year tracks, including Global Health (six residents, 4%), and Medicine-Pediatrics (14 residents, 10%).
Assessment of Exposure
Residents reported increasingly independent exposures as they progressed through residency training. PGY-1 residents on average had never seen 16.3% of the 50 acute events, whereas PGY-3/4 residents had never seen only 4.0% of the events (P < .0001). PGY-1 residents had managed 31.3% of events independently (or both independently and in simulation) as opposed to 71.7% of events for PGY-3/4 residents (P < .0001). Simulation alone accounted for a substantial proportion of exposures (16.4%) for PGY-1 residents, but this was significantly lower for PGY-2 or PGY-3/4 residents (P < .0001), who reported a greater percentage of exposures in nonsimulation clinical scenarios either independently or as a part of an inpatient team. There were no outlier residents who reported lower exposure compared with their PGY peers.
There was a wide spectrum of resident-reported exposures when individual acute events were examined (Table, full data in Supplementary Appendix Table 1). Events with the highest levels of exposure, which >85% of PGY-1 residents had managed independently, included alcohol withdrawal, chronic obstructive pulmonary disease exacerbation, rapid atrial fibrillation, agitated delirium, hypertensive urgency, and hyperkalemia. Events with the lowest levels of exposure, which at least 15% of graduating residents had never encountered in the hospital, included the following eight of 50 events (16%): torsades de pointes (51% of PGY-3/4 residents), acute mechanical valve failure (49%), tension pneumothorax (38.8%), use of emergency transcutaneous pacing (38.8%), elevated intracranial pressure (ICP)/herniation (24.5%), aortic dissection (22.4%), cord compression (16.3%), and use of emergency cardioversion (16.3%). Several PGY-3/4 residents had managed several of these events only in mannequin simulations, including torsades de pointes (41%), transcutaneous pacing (33%), and tension pneumothorax (24%).
Assessment of Confidence
Both levels of training and exposure to acute events were associated with increased confidence in managing such events. PGY-1 residents felt confident in managing 24.9% of acute events independently, compared to 48.4% of events for PGY-2 residents and 72.5% of events for PGY-3/4 residents (P < .0001). There was considerable variation in confidence among the individual acute events (Supplementary Appendix Table 2). A majority of graduating PGY-3/4 residents did not feel confident in managing the following 10 of the 50 events (20%): use of emergency cardioversion, aortic dissection, thrombotic thrombocytopenic purpura/hemolytic uremic syndrome (TTP/HUS), torsades de pointes, posterior reversible encephalopathy syndrome (PRES), intracranial hemorrhage, use of emergency transcutaneous pacing, tension pneumothorax, elevated ICP/herniation, and acute mechanical valve failure.
Residents’ self-reported confidence also correlated with level of exposure. There was a significant increase in resident confidence with increasingly independent exposure stratified by level of training (Figure; all with P < .0001). In the multivariable logistic regression model, increasing exposure correlated with increased resident confidence (P < .0001) while controlling for PGY year (P = .001).
DISCUSSION
We developed an instrument to assess resident exposure to and confidence in managing 50 inpatient acute clinical events. Both exposure and level of training were associated with increasing resident confidence. We identified specific events with low levels of exposure and confidence that could be targeted for educational interventions.
To our knowledge, this is the first study to examine IM residents’ exposure to and confidence in managing a wide range of inpatient acute clinical events. A primary goal of residency is to provide physicians-in-training graduated responsibility to prepare them for eventual independent practice. Although our survey confirmed that IM residents’ exposure and confidence significantly increased as they advanced through training (a not unexpected finding), our data also show that even after controlling for year in training, independent exposures significantly correlated with increased confidence. This speaks to the importance of preserving opportunities for residents to manage critical events in a supported manner, an admittedly challenging prospect given the oft-competing calls for supervision of and mentored feedback for trainees.11
Despite identifying independent exposure as an important factor that impacts resident confidence, we found that there was still a substantial proportion of events (28.3%) that senior medical residents near the end of their training had not managed independently in a primary leadership role. Although our study was not designed to determine the reasons for this varied resident exposure, possible explanations may include the relative rarity of certain acute clinical events compared with others, or less likely the effect of duty hour limitations, attending supervision of trainees, or programmatic changes in resident leadership responsibilities. Whatever the cause, this finding uniquely identifies an area for improvement to prevent new attending physicians from feeling unprepared to manage potentially critical emergencies.
An important goal of our study was to develop an instrument that would enable training programs to identify their learning needs. Both program-wide and individual assessments of resident case exposure and confidence are essential for identifying such learning needs and areas for curricular development. Program-wide assessments can spur an important debate about program goals and requirements with respect to what scenarios residents must be able to manage competently by graduation.12 In addition, such assessments can help individualize learning exposures based on a specific learner’s needs and career goals. The administration of our survey instrument required minimal resources, and the high response rate in our study suggests that other programs can implement our instrument to accomplish these goals.
Alternative methods, such as electronic learning portfolios (efolios), can be utilized to assess resident case exposure. In comparison to our survey instrument, efolios limit recall bias by utilizing case logs and have additional capabilities such as compiling evaluations and enabling trainees to set learning goals. However, there are considerable barriers to the effective use of efolios, including software cost, learner attitudes, and time constraints.13 Tools such as our end-of-year assessment offer an alternative method that limits these barriers.
Once educational growth opportunities have been identified through survey-based or other methods, residency programs must determine how to optimize curricula for the needs and career goals of their trainees. We found considerable overlap among conditions that graduating residents had both limited exposure to and low confidence in managing (eg, torsades de pointes, tension pneumothorax, and emergency cardioversion), which are logical topics for future curriculum development. We also identified a few conditions (including PRES, TTP/HUS, and intracranial hemorrhage) that graduating residents did not feel confident in managing despite a relatively higher reported level of exposure. Whether to focus specific educational interventions on the most rare or most commonly encountered acute clinical events is likely to be a topic of debate among individual training programs, but the results of our survey indicate that there is likely to be educational benefit to both strategies.
Residency programs can employ a variety of modalities to enhance learner exposure and confidence in managing clinical scenarios that are deemed important by the program, including didactics, simulation, and changes in program structure. There is a substantial literature on the use of dedicated curricula for crisis management and the use of simulation as a training tool for responding to acute clinical events in multiple specialties14-24 and in nonmedical domains such as aviation.25-27 Simulation has been shown to improve residents’ clinical skills and comfort level with some acute events28-30 and may even be superior to traditional clinical medical education.31 In addition, programs can utilize targeted clinical experiences such as intensive care unit and subspecialty rotations32,33 in an effort to customize educational interventions to fill identified gaps in learner exposure or confidence.
Our study has several limitations. First, we investigated a single large IM residency program at a quaternary academic medical center, and therefore, our findings may not be externally generalizable to all IM residencies or other medical specialties. Our unique peer-led simulation curriculum, including 16 PGY-1 and 8 PGY-2 cases chosen based on clinical rotations at Massachusetts General Hospital,7 likely impacted residents’ exposure to simulation that is specific to our institution. However, although specific inpatient acute events may vary among other institutions, our finding that graduating residents still reported gaps in their clinical experience is likely generalizable to other programs given the varied and unpredictable nature of ward medicine training. In addition, our survey tool was simple to administer and could be tailored to reflect the acute events and training needs relevant to other residency programs, specialties, and institutions. Second, the retrospective nature of our study may be subject to participants’ recall bias. We did not restrict our survey questions to urgent conditions managed only on IM hospital wards and some may have been experienced in the emergency room or intensive care units; however, these exposures are still relevant as key components of IM training. Third, our list of 50 acute clinical events was intentionally broad and included several conditions that require multidisciplinary subspecialist consultation, which could have impacted residents’ self-report of “independent” exposures. However, these scenarios are ones that hospitalists may independently recognize and stabilize, engaging appropriate specialists. Fourth, we were not able to validate residents’ self-reported exposures against other measures of the frequency of housestaff management of acute events (such as billing data or patient logs) as this information is not routinely collected. We also did not attempt to identify the reasons underlying the variation seen in resident exposure and confidence for individual acute events, but as a needs assessment, this was beyond the scope of our study. Finally, our assessment of resident confidence was subjective and we were not able to assess competence, with prior studies demonstrating conflicting results regarding the relationship between self-reported proficiency and observed competence.34-36 Future studies are needed to investigate whether case exposure assessment leads to changes in residency curricula and whether such curricula increase resident confidence and competence in managing hospital acute clinical events.
CONCLUSION
We developed an easy-to-administer tool to assess IM residents’ exposure to and confidence in managing inpatient acute events. We found that both significantly increased as residents advanced through training, and self-reported confidence additionally correlated with level of exposure independent of PGY class. We identified several specific inpatient acute clinical events with low levels of resident exposure and confidence that can serve as targets for future IM residency curriculum development. Future studies assessing the impact of such curricula on resident confidence and competence are needed.
Disclosures
The authors declare no conflict of interest.
1. ACGME. The Internal Medicine Milestone Project. A joint initiative of the Accreditation Council for Graduate Medical Education and the American Board of Internal Medicine. http://www.acgme.org/Portals/0/PDFs/Milestones/InternalMedicineMilestones.pdf. Accessed July 14, 2018.
2. Neugut AI, Ghatak AT, Miller RL. Anaphylaxis in the United States: an investigation into its epidemiology. Arch Intern Med. 2001;161(1):15-21. PubMed
3. Lin GA, Beck DC, Stewart AL, Garbutt JM. Resident perceptions of the impact of work hour limitations. J Gen Intern Med. 2007;22(7):969-975. PubMed
4. Bolster L, Rourke L. The effect of restricting residents’ duty hours on patient safety, resident well-being, and resident education: an updated systematic review. J Grad Med Educ. 2015;7(3):349-363. PubMed
5. Wayne DB, Hauer KE. Counting quality, not hours: understanding the impact of duty hour reform on internal medicine residency education. J Gen Intern Med. 2012;27(11):1400-1401. PubMed
6. Hayes CW, Rhee A, Detsky ME, Leblanc VR, Wax RS. Residents feel unprepared and unsupervised as leaders of cardiac arrest teams in teaching hospitals: a survey of internal medicine residents. Crit Care Med. 2007;35(7):1668-1672. PubMed
7. Mathai SK, Miloslavsky EM, Contreras-Valdes FM, et al. How we implemented a resident-led medical simulation curriculum in a large internal medicine residency program. Med Teach. 2014;36(4):279-283. PubMed
8. The American Board of Internal Medicine. Internal Medicine Policies. http://www.abim.org/certification/policies/internal-medicine-subspecialty-policies/internal-medicine.aspx. Accessed January 24, 2018.
9. Sinz E, Navarro K, Soderberg ES. Advanced Cardiovascular Life Support. Dallas, TX: American Heart Association; 2011:1-183.
10. Finn KM, Metlay JP, Chang Y, et al. Effect of increased inpatient attending physician supervision on medical errors, patient safety, and resident education: a randomized clinical trial. JAMA Intern Med. 2018;178(7):952-959. PubMed
11. Happel JP, Ritter JB, Neubauer BE. Optimizing the balance between supervision and autonomy in training. JAMA Intern Med. 2018;178(7):959-960. PubMed
12. Fitzgibbons JP, Bordley DR, Berkowitz LR, Miller BW, Henderson MC. Redesigning residency education in internal medicine: a position paper from the association of program directors in internal medicine. Ann Intern Med. 2006;144(12):920. PubMed
13. Dekker H, Driessen E, Braak Ter E, et al. Mentoring portfolio use in undergraduate and postgraduate medical education. Med Teach. 2009;31(10):903-909. PubMed
14. Sica GT, Barron DM, Blum R, Frenna TH, Raemer DB. Computerized realistic simulation: a teaching module for crisis management in radiology. AJR Am J Roentgenol. 1999;172(2):301-304. PubMed
15. DeAnda A, Gaba DM. Role of experience in the response to simulated critical incidents. Anesth Analg. 1991;72(3):308-315. PubMed
16. Gaba DM, Maxwell M, DeAnda A. Anesthetic mishaps. Anesthesiology. 1987;66(5):670-676. PubMed
17. Arora S, Hull L, Fitzpatrick M, Sevdalis N, Birnbach DJ. Crisis management on surgical wards. Ann Surg. 2015;261(5):888-893. PubMed
18. Zirkle M, Blum R, Raemer DB, Healy G, Roberson DW. Teaching emergency airway management using medical simulation: a pilot program. Laryngoscope. 2005;115(3):495-500. PubMed
19. Volk MS, Ward J, Irias N, Navedo A, Pollart J, Weinstock PH. Using medical simulation to teach crisis resource management and decision-making skills to otolaryngology housestaff. Otolaryngol Head Neck Surg. 2011;145(1):35-42. PubMed
20. Bank I, Snell L, Bhanji F. Pediatric crisis resource management training improves emergency medicine trainees’ perceived ability to manage emergencies and ability to identify teamwork errors. Pediatr Emerg Care. 2014;30(12):879-883. PubMed
21. Blackwood J, Duff JP, Nettel-Aguirre A, Djogovic D, Joynt C. Does teaching crisis resource management skills improve resuscitation performance in pediatric residents?. Pediatr Crit Care Med. 2014;15(4):e168-e174. PubMed
22. Daniels K, Lipman S, Harney K, Arafeh J, Druzin M. Use of simulation based team training for obstetric crises in resident education. Simul Healthc. 2008;3(3):154-160. PubMed
23. Isaak RS, Stiegler MP. Review of crisis resource management (CRM) principles in the setting of intraoperative malignant hyperthermia. J Anesth. 2016;30(2):298-306. PubMed
24. Gaba D, DeAnda A. The response of anesthesia trainees to simulated critical incidents. Surv Anesth. 1989;33(6):349. PubMed
25. Ornato JP, Peberdy MA. Applying lessons from commercial aviation safety and operations to resuscitation. Resuscitation. 2014;85(2):173-176. PubMed
26. Hamman WR. Commentary: will simulation fly in medicine as it has in aviation? BMJ Qual Saf. 2004;13(5):397-399. PubMed
27. Littlepage GE, Hein MB, Richard G Moffett I, Craig PA, Georgiou AM. Team training for dynamic cross-functional teams in aviation: behavioral, cognitive, and performance outcomes. Hum Factors. 2016;58(8):1275-1288. PubMed
28. Wayne DB, Butter J, Siddall VJ, et al. Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice. J Gen Intern Med. 2006;21(3):251-256. PubMed
29. Heal
30. Kory PD, Eisen LA, Adachi M, Ribaudo VA, Rosenthal ME, Mayo PH. Initial airway management skills of senior residents. Chest. 2015;132(6):1927-1931. PubMed
31. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011;86(6):706-711. PubMed
32. Almoosa KF, Goldenhar LM, Puchalski J, Ying J, Panos RJ. Critical care education during internal medicine residency: a national survey. J Grad Med Educ. 2010;2(4):555-561. PubMed
33. Katz SJ, Oswald AE. How confident are internal medicine residents in rheumatology versus other common internal medicine clinical skills: an issue of training time or exposure? Clin Rheumatol. 2011;30(8):1081-1093. PubMed
34. Barnsley L, Lyon PM, Ralston SJ, et al. Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence. Med Educ. 2004;38(4):358-367. PubMed
35. Dehmer JJ, Amos KD, Farrell TM, Meyer AA, Newton WP, Meyers MO. Competence and confidence with basic procedural skills: the experience and opinions of fourth-year medical students at a single institution. Acad Med. 2013;88(5):682-687. PubMed
36. Wu EH, Elnicki DM, Alper EJ, et al. Procedural and interpretive skills of medical students: experiences and attitudes of fourth-year students. Acad Med. 2008;83(10):S63-S67. PubMed
1. ACGME. The Internal Medicine Milestone Project. A joint initiative of the Accreditation Council for Graduate Medical Education and the American Board of Internal Medicine. http://www.acgme.org/Portals/0/PDFs/Milestones/InternalMedicineMilestones.pdf. Accessed July 14, 2018.
2. Neugut AI, Ghatak AT, Miller RL. Anaphylaxis in the United States: an investigation into its epidemiology. Arch Intern Med. 2001;161(1):15-21. PubMed
3. Lin GA, Beck DC, Stewart AL, Garbutt JM. Resident perceptions of the impact of work hour limitations. J Gen Intern Med. 2007;22(7):969-975. PubMed
4. Bolster L, Rourke L. The effect of restricting residents’ duty hours on patient safety, resident well-being, and resident education: an updated systematic review. J Grad Med Educ. 2015;7(3):349-363. PubMed
5. Wayne DB, Hauer KE. Counting quality, not hours: understanding the impact of duty hour reform on internal medicine residency education. J Gen Intern Med. 2012;27(11):1400-1401. PubMed
6. Hayes CW, Rhee A, Detsky ME, Leblanc VR, Wax RS. Residents feel unprepared and unsupervised as leaders of cardiac arrest teams in teaching hospitals: a survey of internal medicine residents. Crit Care Med. 2007;35(7):1668-1672. PubMed
7. Mathai SK, Miloslavsky EM, Contreras-Valdes FM, et al. How we implemented a resident-led medical simulation curriculum in a large internal medicine residency program. Med Teach. 2014;36(4):279-283. PubMed
8. The American Board of Internal Medicine. Internal Medicine Policies. http://www.abim.org/certification/policies/internal-medicine-subspecialty-policies/internal-medicine.aspx. Accessed January 24, 2018.
9. Sinz E, Navarro K, Soderberg ES. Advanced Cardiovascular Life Support. Dallas, TX: American Heart Association; 2011:1-183.
10. Finn KM, Metlay JP, Chang Y, et al. Effect of increased inpatient attending physician supervision on medical errors, patient safety, and resident education: a randomized clinical trial. JAMA Intern Med. 2018;178(7):952-959. PubMed
11. Happel JP, Ritter JB, Neubauer BE. Optimizing the balance between supervision and autonomy in training. JAMA Intern Med. 2018;178(7):959-960. PubMed
12. Fitzgibbons JP, Bordley DR, Berkowitz LR, Miller BW, Henderson MC. Redesigning residency education in internal medicine: a position paper from the association of program directors in internal medicine. Ann Intern Med. 2006;144(12):920. PubMed
13. Dekker H, Driessen E, Braak Ter E, et al. Mentoring portfolio use in undergraduate and postgraduate medical education. Med Teach. 2009;31(10):903-909. PubMed
14. Sica GT, Barron DM, Blum R, Frenna TH, Raemer DB. Computerized realistic simulation: a teaching module for crisis management in radiology. AJR Am J Roentgenol. 1999;172(2):301-304. PubMed
15. DeAnda A, Gaba DM. Role of experience in the response to simulated critical incidents. Anesth Analg. 1991;72(3):308-315. PubMed
16. Gaba DM, Maxwell M, DeAnda A. Anesthetic mishaps. Anesthesiology. 1987;66(5):670-676. PubMed
17. Arora S, Hull L, Fitzpatrick M, Sevdalis N, Birnbach DJ. Crisis management on surgical wards. Ann Surg. 2015;261(5):888-893. PubMed
18. Zirkle M, Blum R, Raemer DB, Healy G, Roberson DW. Teaching emergency airway management using medical simulation: a pilot program. Laryngoscope. 2005;115(3):495-500. PubMed
19. Volk MS, Ward J, Irias N, Navedo A, Pollart J, Weinstock PH. Using medical simulation to teach crisis resource management and decision-making skills to otolaryngology housestaff. Otolaryngol Head Neck Surg. 2011;145(1):35-42. PubMed
20. Bank I, Snell L, Bhanji F. Pediatric crisis resource management training improves emergency medicine trainees’ perceived ability to manage emergencies and ability to identify teamwork errors. Pediatr Emerg Care. 2014;30(12):879-883. PubMed
21. Blackwood J, Duff JP, Nettel-Aguirre A, Djogovic D, Joynt C. Does teaching crisis resource management skills improve resuscitation performance in pediatric residents?. Pediatr Crit Care Med. 2014;15(4):e168-e174. PubMed
22. Daniels K, Lipman S, Harney K, Arafeh J, Druzin M. Use of simulation based team training for obstetric crises in resident education. Simul Healthc. 2008;3(3):154-160. PubMed
23. Isaak RS, Stiegler MP. Review of crisis resource management (CRM) principles in the setting of intraoperative malignant hyperthermia. J Anesth. 2016;30(2):298-306. PubMed
24. Gaba D, DeAnda A. The response of anesthesia trainees to simulated critical incidents. Surv Anesth. 1989;33(6):349. PubMed
25. Ornato JP, Peberdy MA. Applying lessons from commercial aviation safety and operations to resuscitation. Resuscitation. 2014;85(2):173-176. PubMed
26. Hamman WR. Commentary: will simulation fly in medicine as it has in aviation? BMJ Qual Saf. 2004;13(5):397-399. PubMed
27. Littlepage GE, Hein MB, Richard G Moffett I, Craig PA, Georgiou AM. Team training for dynamic cross-functional teams in aviation: behavioral, cognitive, and performance outcomes. Hum Factors. 2016;58(8):1275-1288. PubMed
28. Wayne DB, Butter J, Siddall VJ, et al. Mastery learning of advanced cardiac life support skills by internal medicine residents using simulation technology and deliberate practice. J Gen Intern Med. 2006;21(3):251-256. PubMed
29. Heal
30. Kory PD, Eisen LA, Adachi M, Ribaudo VA, Rosenthal ME, Mayo PH. Initial airway management skills of senior residents. Chest. 2015;132(6):1927-1931. PubMed
31. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011;86(6):706-711. PubMed
32. Almoosa KF, Goldenhar LM, Puchalski J, Ying J, Panos RJ. Critical care education during internal medicine residency: a national survey. J Grad Med Educ. 2010;2(4):555-561. PubMed
33. Katz SJ, Oswald AE. How confident are internal medicine residents in rheumatology versus other common internal medicine clinical skills: an issue of training time or exposure? Clin Rheumatol. 2011;30(8):1081-1093. PubMed
34. Barnsley L, Lyon PM, Ralston SJ, et al. Clinical skills in junior medical officers: a comparison of self-reported confidence and observed competence. Med Educ. 2004;38(4):358-367. PubMed
35. Dehmer JJ, Amos KD, Farrell TM, Meyer AA, Newton WP, Meyers MO. Competence and confidence with basic procedural skills: the experience and opinions of fourth-year medical students at a single institution. Acad Med. 2013;88(5):682-687. PubMed
36. Wu EH, Elnicki DM, Alper EJ, et al. Procedural and interpretive skills of medical students: experiences and attitudes of fourth-year students. Acad Med. 2008;83(10):S63-S67. PubMed
© 2019 Society of Hospital Medicine