User login
For the resident, the surgical residency is physically, emotionally, and intellectually demanding, requiring longitudinally concentrated effort. Although education of orthopedic surgeons necessarily occurs within the context of the health care delivery system, vital lessons also are taught in laboratories, skill stations, and surgical simulators. Before practice-based learning can take place, residents must gain experience and demonstrate growth in surgical skills, including decision-making and technical skills. These skill sets are difficult to systematically teach and objectively analyze.
The most effective way to teach and assess a resident’s knowledge of musculoskeletal medicine remains unclear at this point. Much of the current literature addresses the issue at the medical student level.1-7 Some studies have shown the effectiveness of surgical training programs, both cadaveric and computer-based simulators, in teaching various surgical skill sets.8-14 The orthopedic literature has seen a boom in surgical simulators aimed at the upper-level resident. Many of the topics involve use of arthroscopic simulators.15-19 Evidence suggests that simulators can discriminate between novice and expert users, but discrimination between novice and intermediate trainees in surgical education should be paramount.20
The American Board of Orthopaedic Surgery (ABOS) and the orthopedic Residency Review Committee (RRC) recommended new requirements for structured motor skills training in basic orthopedic surgery education,21 which were approved by the Accreditation Council for Graduate Medical Education (ACGME) board of directors and went into effect on July 1, 2013. In response to the new ACGME guidelines, our institution created a skills laboratory devoted to surgical simulation. Our focus in implementing this surgical skills simulation was junior-level, specifically postgraduate year 1 to 3 (PGY-1 to PGY-3), orthopedic residents. Our first goal was to set up a series of surgical training stations to educate junior-level residents in 4 core areas: handling and comfort with basic power equipment, casting/splinting, suturing, and surgical instrument identification. A secondary goal was to objectively evaluate the residents through written examinations (presession–postsession) and a novel ankle fracture model (pre–post).
Materials and Methods
Institutional review board approval was obtained before beginning the investigation.
Written Examination
We created a multiple-choice 25-question written examination (Appendix) and administered it to 11 junior residents before and after they participated in the training. This examination assessed their knowledge base of basic orthopedic tenets, including basic bone healing, basic fracture repair (Arbeitsgemeinschaft für Osteosynthesefragen [AO] principles22), suturing, surgical instrument identification, casting/splinting, and elementary implant-design rationale.
Evaluator Scorecard
We created an evaluation scorecard (Figure 1) and had 2 faculty members and 2 senior-level residents complete it independently. Junior residents were evaluated on a sawbones lateral malleolar ankle fracture model at 2 time points. As with the written examinations, the junior residents completed the fracture model both before and immediately after the multiple skill sessions. Each of the 15 data points was scored from 1 to 4, for a total of 60 points.
Facility for Surgical Training Session
Our Clinical Skills Education and Assessment Center houses small-group interactive laboratories for administration, debriefing, and assessment of simulations with the latest in audiovisual equipment. Five stations were created: hands-on introduction to surgical power equipment using sawbones, wood, and polyvinylchloride (PVC) pipe; hands-on introduction to casting and splinting; hands-on introduction to suturing; hands-on interaction with surgical scrub technician assisting with instrument identification; and didactic PowerPoint (Microsoft, Redmond, Washington) presentation focusing on core trauma competencies, basic orthopedic design rationale, and basic bone biology.
Development of Surgical Skills Training Session
Multiple faculty members and senior-level residents collaborated to create the skill stations (Figure 2), which were designed based on ACGME recommendations and on weaknesses our program had seen in junior-level residents. We devoted an afternoon to this session, excusing our program’s junior residents from clinical responsibilities. Four PGY-1, 5 PGY-2, and 2 PGY-3 residents participated. (Four of our 15 junior residents were unable to attend because of clinical responsibilities.) The afternoon started by dividing the 11 junior residents into 2 groups. Before the session, while one group performed the ankle fracture model and was being evaluated, the other took the written examination. This closely timed portion was allotted only 20 minutes. Then residents were divided into 5 groups of 2 or 3 and were rotated through all 5 stations. Forty minutes were allotted for each station. Residents were not evaluated during this portion. The stations were intended solely for education, and each station was staffed by a faculty member and/or senior-level resident.
Cordless reciprocating saws and drills were purchased to introduce and refine junior residents’ motor skills. Sawbones, 2×4-in sections of wood, and PVC pipe were used in the training. Emphasis was placed on tactile feel and feedback with both sawing and drilling. For the casting and splinting session, we used 4-in fiberglass, 4-in plaster rolls, and cotton soft roll to demonstrate a multitude of common casts and splints (Figure 3). Casts included short- and long-arm casts and short-leg casts. Splinting included coaptation, sugar tong, and ulnar gutter splints for the upper extremity and a short-leg posterior splint for the lower extremity.
The didactic PowerPoint presentation drew largely from content in chapters of the book AO Principles of Fracture Management.22 Content included condensed, to-the-point high-yield summaries of AO tenets, basic bone healing and biology, and orthopedic implant-design rationale focused on these elementary principles:
◾ Basic screw design, including cortical, cancellous, and locking screw designs.
◾ Evolution of plate osteosynthesis to currently used locking compression plate.
◾ Locking plate principles.
◾ Lag technique.
◾ Plate use: compression mode, neutralization, bridging, buttress, anti-glide.
The suturing portion was performed with thawed ham hocks (Figure 4). This model replicates live tissue layers and allows a layered closure technique as a training tool. Both 0 and 2-0 absorbable suture were available for a layered, deep fascial closure; also available was 2-0 nonabsorbable nylon for the skin. Staple guns were available, as were basic surgical instruments, including quality needle drivers, Adson forceps, and suture scissors. The knots demonstrated included simple, horizontal mattress, vertical mattress, and tension-relieving. One- and 2-hand tying and instrument tying were reinforced.
The final session consisted of surgical instrument identification. A certified orthopedic scrub technician participated. On site were multiple trays, including a basic bone set, a hand-and-foot set, small and large fragment sets, and a hip set. A detailed review of each set was led by the surgical technician. This review was followed by a question-and-answer session with the junior residents. After the session, we ended with the written examination and the ankle fracture model.
Statistical Methods
We report presession and postsession means, modes, and medians as measures of score-central tendencies. Our small sample size makes the assumption of Gaussian distribution tenuous and more susceptible to outliers. Therefore, in addition to reporting means, we include medians and modes to more accurately account for outliers. Moreover, the κ statistic is a robust measure of interrater agreement for 2 or more groups. We report κ statistics to determine the interrater reliability of 4 independent observers.
Results
Written Examination
Eleven residents (PGY-1 to PGY-3) completed the examination (Table 1). For the entire group, mean (SD) presession percentile was 87.3 (10.4), median was 88, and mode was 96; mean (SD) was 80 (12.6) for PGY-1, 89.6 (6.7) for PGY-2, and 96 (5.7) for PGY-3. For the entire group, mean (SD) postsession percentile was 92 (8.4), median was 96, and mode was 96; mean (SD) was 85 (10.5) for PGY-1, 96 (4) for PGY-2, and 96 (0) for PGY-3 (Table 2).
There was a significant presession–postsession difference in scores among all test takers, regardless of training level (P = .019). The PGY-1 level did not reach statistical significance in improvement from presession to postsession (P = .080); the PGY-2 level also did not reach statistical significance in improvement (P = .099); the PGY-3 level did not have enough participants to calculate a P value based on a paired Student t test.
Ankle Fracture Model
Actual percentile scores are listed in Table 3. For the entire group, mean (SD) overall presession percentile was 68.6 (13.9), median was 67, and mode was 67; mean (SD) was 58.8 (9.8) for PGY-1, 76.1 (13.6) for PGY-2, and 69.5 (9.8) for PGY-3. For the entire group, mean (SD) postsession percentile was 95.2 (5.2), median was 97, and mode was 97; mean (SD) was 91.8 (6.3) for PGY-1, 97.1 (3.5) for PGY-2, and 97.3 (2.4) for PGY-3.
There was a large and significant presession–postsession difference in scores among all test takers, regardless of training level (P = .03). Each group reached statistical significance in improvement from presession to postsession: PGY-1 (P = .04), PGY-2 (P = .01), and PGY-3 (P = .03).
For κ calculations, we adjusted all scores to ordinal data and thus used a standard grading system:
Score Grade
90–100 A
80–89 B
70–79 C
60–69 D
0–59 F
For the presession fracture model, the κ among the 4 independent observational scorers was 0.1148 (Table 4), which is poor based on κ scoring criteria and which we attribute to the particularly harsh grading by 1 observational scorer (faculty 1) relative to the other scorers’. Examination of the κ scores of faculty 1 and faculty 2 indicated only 9.09% agreement. Conversely, the κ among resident scorers agreed 54.55% of the time. Removing faculty 1 as an outlier raised the κ score dramatically, to 0.3125 (fair interobserver agreement).
For the postsession fracture model, the κ among the 4 independent observational scorers improved only marginally, to 0.1156 (still poor), again attributed to a difference in severity of grading: faculty 1 (harsh) versus faculty 2 (relatively kind). Examination of the κ scores of faculty 1 and faculty 2 revealed 72.73% agreement; residents agreed 81.82% of the time.
Discussion
The importance of surgical skill development in resident education is emphasized in the ACGME Core Competencies.23 The ACGME instructed all programs to require residents to gain competency in 6 areas: patient care, interpersonal and communication skills, medical knowledge, professionalism, practice-based learning and systems-based practice. Although many surgeon educators and residents are focused on these 6 Core Competencies, current standards do not require surgical skills laboratory training and simply require residents to log cases into the ACGME website. Minimal case number recommendations are in place for graduating senior residents, but these numbers are based on averages with no strong scientific basis.
Although sweeping changes in orthopedic residency training went into effect July 1, 2013, this system remains untested and may offer room for improvement. One change is the restructuring of the PGY-1 internship. A basic surgical skills curriculum must include goals, objectives, and assessment metrics; skills used in the initial management of injured patients, including splinting, casting, application of traction devices, and other types of immobilization; and basic operative skills, including soft-tissue management, suturing, bone management, arthroscopy, fluoroscopy, and use of basic orthopedic equipment.21
Orthopedic program directors and residents were recently surveyed regarding the current state of orthopedic motor skills training.24 Three key findings deserve emphasis: There is a lack of objective criteria for evaluating resident performance in the skills laboratory; most program directors who have a laboratory do not understand the associated costs; and the most significant issue for program directors is the financial challenge of operating a motor skills laboratory. The survey findings strongly suggest that proposed changes in skills training should be accompanied by careful cost analysis before widespread implementation.
Although various online demonstrations of entire surgeries are available, as are textbooks describing a generalized approach to musculoskeletal surgery, we assume that, as laid out in the Core Competencies, residents are fine-tuning their surgical skills by actively participating in operating rooms under direct observation of attending physicians. To our knowledge, however, there are no data regarding how often this happens in the operative setting, where volume and efficiency are becoming increasingly scrutinized. There has been much concern over how hour restrictions will affect residents’ total operative experience.25,26 Finally, we have no means to objectively evaluate residents’ surgical skills on graduation.
Other programs have implemented surgical skill simulators, but an orthopedics-specific surgical skills laboratory, to our knowledge, has been discussed in only 1 study.21 Results from randomized controlled trials reported in the general surgery literature have proved simulation-based training leads to detectable benefits for learners in clinical settings.27-29 Over the past decade, some alternative surgical skills training methods have been adopted in orthopedic surgery as well. These methods include hands-on training in specifically designed surgical skills laboratories using cadaver models or synthetic bones; software tools; and computerized simulators. In recent years, numerous studies reported in the orthopedic literature have examined arthroscopic simulators in residency training.18-20,30-34 However, these studies are arguably more specific to sports subspecialties and thus more pertinent to upper-level trainees.
Our study results showed that surgical skills laboratory training should be a required aspect of our residents’ training. Although less of a dramatic improvement was noted in the written examination component of the laboratory, the overall knowledge base improved (Table 3). This was especially evident at the PGY-1 level, where written examination scores increased from a presession median of 80% to a postsession median of 85%. A larger degree of improvement was found with the ankle fracture model, and there was statistical improvement at all training levels, from PGY-1 to PGY-3. Previous work has shown that intensive laboratory-based training can be effective, particularly for first-year residents. Sonnadara and colleagues35 demonstrated that a 30-day intensive surgical skills course effectively helped first-year orthopedic residents develop targeted basic surgical skills. In a follow-up study, Sonnadara and colleagues36 demonstrated that a surgical skills course completed at the beginning of a residency was effective in teaching targeted technical skills, and that skills taught in this manner can have excellent retention rates.
There are limitations inherent in our skills course. The κ agreement in the ankle fracture model was low before and after administration, which we attribute to 1 observer outlier. This could be amended by removing outliers and further objectifying and simplifying the scoring system (A–F). Right now, we do not have enough data to determine whether the scores actually improve significantly through the training years or whether they will correlate with operating room experience. Our study had no control. For future investigations, we are considering having general orthopedic surgeons from the community perform the same scenarios and be graded with the same checklists as a control. Implementation, however, may be a challenge. Both our written examination and our ankle fracture model checklist have not been validated—this is one of our next steps. The point system used to score the ankle fracture model was subjectively developed and would benefit from further standardization before drawing conclusions about true validity.
Conclusion
Orthopedic residency programs, like programs in other surgical specialties, are increasingly focused on teaching and documenting the learning of core competencies, even as work-hour restrictions and demands for clinical efficiency limit the amount of time residents spend in the operating room. We have demonstrated the potential value of an intensive laboratory in improving junior-level residents’ basic surgical skills and knowledge. We will continue to refine our methods, with a goal being to create reproducible models that could be adapted by other orthopedic residency programs and by other surgical educators.
1. Schmale GA. More evidence of educational inadequacies in musculoskeletal medicine. Clin Orthop. 2005;(437):251-259.
2. Day CS, Yeh AC, Franko O, Ramirez M, Krupat E. Musculoskeletal medicine: an assessment of the attitudes and knowledge of medical students at Harvard Medical School. Acad Med. 2007;82(5):452-457.
3. Bilderback K, Eggerstedt J, Sadasivan KK, et al. Design and implementation of a system-based course in musculoskeletal medicine for medical students. J Bone Joint Surg Am. 2008;90(10):2292-2300.
4. Freedman KB, Bernstein J. Educational deficiencies in musculoskeletal medicine. J Bone Joint Surg Am. 2002;84(4):604-608.
5. Corbett EC Jr, Elnicki DM, Conaway MR. When should students learn essential physical examination skills? Views of internal medicine clerkship directors in North America. Acad Med. 2008;83(1):96-99.
6. Coady DA, Walker DJ, Kay LJ. Teaching medical students musculoskeletal examination skills: identifying barriers to learning and ways of overcoming them. Scand J Rheumatol. 2004;33(1):47-51.
7. Saleh K, Messner R, Axtell S, Harris I, Mahowald ML. Development and evaluation of an integrated musculoskeletal disease course for medical students. J Bone Joint Surg Am. 2004;86(8):1653-1658.
8. van Empel PJ, Verdam MG, Huirne JA, Bonjer HJ, Meijerink WJ, Scheele F. Open knot-tying skills: resident skills assessed. J Obstet Gynaecol Res. 2013;39(5):1030-1036.
9. Barrier BF, Thompson AB, McCullough MW, Occhino JA. A novel and inexpensive vaginal hysterectomy simulator. Simul Healthc. 2012;7(6):374-379.
10. Liss MA, McDougall EM. Robotic surgical simulation. Cancer J. 2013;19(2):124-129.
11. Stegemann AP, Ahmed K, Syed JR, et al. Fundamental skills of robotic surgery: a multi-institutional randomized controlled trial for validation of a simulation-based curriculum. Urology. 2013;81(4):767-774.
12. Duran C, Bismuth J, Mitchell E. A nationwide survey of vascular surgery trainees reveals trends in operative experience, confidence, and attitudes about simulation. J Vasc Surg. 2013;58(2):524-528.
13. Kuhls DA, Risucci DA, Bowyer MW, Luchette FA. Advanced surgical skills for exposure in trauma: a new surgical skills cadaver course for surgery residents and fellows. J Trauma Acute Care Surg. 2013;74(2):664-670.
14. Sanfey HA, Dunnington GL. Basic surgical skills testing for junior residents: current views of general surgery program directors. J Am Coll Surg. 2011;212(3):406-412.
15. Alvand A, Khan T, Al-Ali S, Jackson WF, Price AJ, Rees JL. Simple visual parameters for objective assessment of arthroscopic skill. J Bone Joint Surg Am. 2012;94(13):e97.
16. Jackson WF, Khan T, Alvand A, et al. Learning and retaining simulated arthroscopic meniscal repair skills. J Bone Joint Surg Am. 2012;94(17):e132.
17. Pernar LI, Smink DS, Hicks G, Peyre SE. Residents can successfully teach basic surgical skills in the simulation center. J Surg Educ. 2012;69(5):617-622.
18. Tuijthof GJ, Visser P, Sierevelt IN, Van Dijk CN, Kerkhoffs GM. Does perception of usefulness of arthroscopic simulators differ with levels of experience? Clin Orthop. 2011;469(6):1701-1708.
19. Martin KD, Cameron K, Belmont PJ, Schoenfeld A, Owens BD. Shoulder arthroscopy simulator performance correlates with resident and shoulder arthroscopy experience. J Bone Joint Surg Am. 2012;94(21):e160.
20. Slade Shantz JA, Leiter JR, Gottschalk T, MacDonald PB. The internal validity of arthroscopic simulators and their effectiveness in arthroscopic education. Knee Surg Sports Traumatol Arthrosc. 2014;22(1):33-40.
21. Roberts S, Menage J, Eisenstein SM. The cartilage end-plate and intervertebral disc in scoliosis: calcification and other sequelae. J Orthop Res. 1993;11(5):747-757.
22. Ruedi TP, Buckley RE, Moran CG. AO Principles of Fracture Management. Stuttgart, Germany: Thieme; 2007.
23. Chen CL, Chen WC, Chiang JH, Ho CF. Interscapular hibernoma: case report and literature review. Kaohsiung J Med Sci. 2011;27(8):348-352.
24. Karam MD, Pedowitz RA, Natividad H, Murray J, Marsh JL. Current and future use of surgical skills training laboratories in orthopaedic resident education: a national survey. J Bone Joint Surg Am. 2013;95(1):e4.
25. Baskies MA, Ruchelsman DE, Capeci CM, Zuckerman JD, Egol KA. Operative experience in an orthopaedic surgery residency program: the effect of work-hour restrictions. J Bone Joint Surg Am. 2008;90(4):924-927.
26. Pappas AJ, Teague DC. The impact of the Accreditation Council for Graduate Medical Education work-hour regulations on the surgical experience of orthopaedic surgery residents. J Bone Joint Surg Am. 2007;89(4):904-909.
27. Palter VN, Grantcharov T, Harvey A, Macrae HM. Ex vivo technical skills training transfers to the operating room and enhances cognitive learning: a randomized controlled trial. Ann Surg. 2011;253(5):886-889.
28. Franzeck FM, Rosenthal R, Muller MK, et al. Prospective randomized controlled trial of simulator-based versus traditional in-surgery laparoscopic camera navigation training. Surg Endosc. 2012;26(1):235-241.
29. Zendejas B, Cook DA, Bingener J, et al. Simulation-based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair: a randomized controlled trial. Ann Surg. 2011;254(3):502-509.
30. Hui Y, Safir O, Dubrowski A, Carnahan H. What skills should simulation training in arthroscopy teach residents? A focus on resident input. Int J Comput Assist Radiol Surg. 2013;8(6):945-953.
31. Butler A, Olson T, Koehler R, Nicandri G. Do the skills acquired by novice surgeons using anatomic dry models transfer effectively to the task of diagnostic knee arthroscopy performed on cadaveric specimens? J Bone Joint Surg Am. 2013;95(3):e15(1-8).
32. Martin KD, Belmont PJ, Schoenfeld AJ, Todd M, Cameron KL, Owens BD. Arthroscopic basic task performance in shoulder simulator model correlates with similar task performance in cadavers. J Bone Joint Surg Am. 2011;93(21):e1271-e1275.
33. Elliott MJ, Caprise PA, Henning AE, Kurtz CA, Sekiya JK. Diagnostic knee arthroscopy: a pilot study to evaluate surgical skills. Arthroscopy. 2012;28(2):218-224.
34. Andersen C, Winding TN, Vesterby MS. Development of simulated arthroscopic skills. Acta Orthop. 2011;82(1):90-95.
35. Sonnadara RR, Van Vliet A, Safir O, et al. Orthopedic boot camp: examining the effectiveness of an intensive surgical skills course. Surgery. 2011;149(6):745-749.
36. Sonnadara RR, Garbedian S, Safir O, et al. Orthopaedic boot camp II: examining the retention rates of an intensive surgical skills course. Surgery. 2012;151(6):803-807.
For the resident, the surgical residency is physically, emotionally, and intellectually demanding, requiring longitudinally concentrated effort. Although education of orthopedic surgeons necessarily occurs within the context of the health care delivery system, vital lessons also are taught in laboratories, skill stations, and surgical simulators. Before practice-based learning can take place, residents must gain experience and demonstrate growth in surgical skills, including decision-making and technical skills. These skill sets are difficult to systematically teach and objectively analyze.
The most effective way to teach and assess a resident’s knowledge of musculoskeletal medicine remains unclear at this point. Much of the current literature addresses the issue at the medical student level.1-7 Some studies have shown the effectiveness of surgical training programs, both cadaveric and computer-based simulators, in teaching various surgical skill sets.8-14 The orthopedic literature has seen a boom in surgical simulators aimed at the upper-level resident. Many of the topics involve use of arthroscopic simulators.15-19 Evidence suggests that simulators can discriminate between novice and expert users, but discrimination between novice and intermediate trainees in surgical education should be paramount.20
The American Board of Orthopaedic Surgery (ABOS) and the orthopedic Residency Review Committee (RRC) recommended new requirements for structured motor skills training in basic orthopedic surgery education,21 which were approved by the Accreditation Council for Graduate Medical Education (ACGME) board of directors and went into effect on July 1, 2013. In response to the new ACGME guidelines, our institution created a skills laboratory devoted to surgical simulation. Our focus in implementing this surgical skills simulation was junior-level, specifically postgraduate year 1 to 3 (PGY-1 to PGY-3), orthopedic residents. Our first goal was to set up a series of surgical training stations to educate junior-level residents in 4 core areas: handling and comfort with basic power equipment, casting/splinting, suturing, and surgical instrument identification. A secondary goal was to objectively evaluate the residents through written examinations (presession–postsession) and a novel ankle fracture model (pre–post).
Materials and Methods
Institutional review board approval was obtained before beginning the investigation.
Written Examination
We created a multiple-choice 25-question written examination (Appendix) and administered it to 11 junior residents before and after they participated in the training. This examination assessed their knowledge base of basic orthopedic tenets, including basic bone healing, basic fracture repair (Arbeitsgemeinschaft für Osteosynthesefragen [AO] principles22), suturing, surgical instrument identification, casting/splinting, and elementary implant-design rationale.
Evaluator Scorecard
We created an evaluation scorecard (Figure 1) and had 2 faculty members and 2 senior-level residents complete it independently. Junior residents were evaluated on a sawbones lateral malleolar ankle fracture model at 2 time points. As with the written examinations, the junior residents completed the fracture model both before and immediately after the multiple skill sessions. Each of the 15 data points was scored from 1 to 4, for a total of 60 points.
Facility for Surgical Training Session
Our Clinical Skills Education and Assessment Center houses small-group interactive laboratories for administration, debriefing, and assessment of simulations with the latest in audiovisual equipment. Five stations were created: hands-on introduction to surgical power equipment using sawbones, wood, and polyvinylchloride (PVC) pipe; hands-on introduction to casting and splinting; hands-on introduction to suturing; hands-on interaction with surgical scrub technician assisting with instrument identification; and didactic PowerPoint (Microsoft, Redmond, Washington) presentation focusing on core trauma competencies, basic orthopedic design rationale, and basic bone biology.
Development of Surgical Skills Training Session
Multiple faculty members and senior-level residents collaborated to create the skill stations (Figure 2), which were designed based on ACGME recommendations and on weaknesses our program had seen in junior-level residents. We devoted an afternoon to this session, excusing our program’s junior residents from clinical responsibilities. Four PGY-1, 5 PGY-2, and 2 PGY-3 residents participated. (Four of our 15 junior residents were unable to attend because of clinical responsibilities.) The afternoon started by dividing the 11 junior residents into 2 groups. Before the session, while one group performed the ankle fracture model and was being evaluated, the other took the written examination. This closely timed portion was allotted only 20 minutes. Then residents were divided into 5 groups of 2 or 3 and were rotated through all 5 stations. Forty minutes were allotted for each station. Residents were not evaluated during this portion. The stations were intended solely for education, and each station was staffed by a faculty member and/or senior-level resident.
Cordless reciprocating saws and drills were purchased to introduce and refine junior residents’ motor skills. Sawbones, 2×4-in sections of wood, and PVC pipe were used in the training. Emphasis was placed on tactile feel and feedback with both sawing and drilling. For the casting and splinting session, we used 4-in fiberglass, 4-in plaster rolls, and cotton soft roll to demonstrate a multitude of common casts and splints (Figure 3). Casts included short- and long-arm casts and short-leg casts. Splinting included coaptation, sugar tong, and ulnar gutter splints for the upper extremity and a short-leg posterior splint for the lower extremity.
The didactic PowerPoint presentation drew largely from content in chapters of the book AO Principles of Fracture Management.22 Content included condensed, to-the-point high-yield summaries of AO tenets, basic bone healing and biology, and orthopedic implant-design rationale focused on these elementary principles:
◾ Basic screw design, including cortical, cancellous, and locking screw designs.
◾ Evolution of plate osteosynthesis to currently used locking compression plate.
◾ Locking plate principles.
◾ Lag technique.
◾ Plate use: compression mode, neutralization, bridging, buttress, anti-glide.
The suturing portion was performed with thawed ham hocks (Figure 4). This model replicates live tissue layers and allows a layered closure technique as a training tool. Both 0 and 2-0 absorbable suture were available for a layered, deep fascial closure; also available was 2-0 nonabsorbable nylon for the skin. Staple guns were available, as were basic surgical instruments, including quality needle drivers, Adson forceps, and suture scissors. The knots demonstrated included simple, horizontal mattress, vertical mattress, and tension-relieving. One- and 2-hand tying and instrument tying were reinforced.
The final session consisted of surgical instrument identification. A certified orthopedic scrub technician participated. On site were multiple trays, including a basic bone set, a hand-and-foot set, small and large fragment sets, and a hip set. A detailed review of each set was led by the surgical technician. This review was followed by a question-and-answer session with the junior residents. After the session, we ended with the written examination and the ankle fracture model.
Statistical Methods
We report presession and postsession means, modes, and medians as measures of score-central tendencies. Our small sample size makes the assumption of Gaussian distribution tenuous and more susceptible to outliers. Therefore, in addition to reporting means, we include medians and modes to more accurately account for outliers. Moreover, the κ statistic is a robust measure of interrater agreement for 2 or more groups. We report κ statistics to determine the interrater reliability of 4 independent observers.
Results
Written Examination
Eleven residents (PGY-1 to PGY-3) completed the examination (Table 1). For the entire group, mean (SD) presession percentile was 87.3 (10.4), median was 88, and mode was 96; mean (SD) was 80 (12.6) for PGY-1, 89.6 (6.7) for PGY-2, and 96 (5.7) for PGY-3. For the entire group, mean (SD) postsession percentile was 92 (8.4), median was 96, and mode was 96; mean (SD) was 85 (10.5) for PGY-1, 96 (4) for PGY-2, and 96 (0) for PGY-3 (Table 2).
There was a significant presession–postsession difference in scores among all test takers, regardless of training level (P = .019). The PGY-1 level did not reach statistical significance in improvement from presession to postsession (P = .080); the PGY-2 level also did not reach statistical significance in improvement (P = .099); the PGY-3 level did not have enough participants to calculate a P value based on a paired Student t test.
Ankle Fracture Model
Actual percentile scores are listed in Table 3. For the entire group, mean (SD) overall presession percentile was 68.6 (13.9), median was 67, and mode was 67; mean (SD) was 58.8 (9.8) for PGY-1, 76.1 (13.6) for PGY-2, and 69.5 (9.8) for PGY-3. For the entire group, mean (SD) postsession percentile was 95.2 (5.2), median was 97, and mode was 97; mean (SD) was 91.8 (6.3) for PGY-1, 97.1 (3.5) for PGY-2, and 97.3 (2.4) for PGY-3.
There was a large and significant presession–postsession difference in scores among all test takers, regardless of training level (P = .03). Each group reached statistical significance in improvement from presession to postsession: PGY-1 (P = .04), PGY-2 (P = .01), and PGY-3 (P = .03).
For κ calculations, we adjusted all scores to ordinal data and thus used a standard grading system:
Score Grade
90–100 A
80–89 B
70–79 C
60–69 D
0–59 F
For the presession fracture model, the κ among the 4 independent observational scorers was 0.1148 (Table 4), which is poor based on κ scoring criteria and which we attribute to the particularly harsh grading by 1 observational scorer (faculty 1) relative to the other scorers’. Examination of the κ scores of faculty 1 and faculty 2 indicated only 9.09% agreement. Conversely, the κ among resident scorers agreed 54.55% of the time. Removing faculty 1 as an outlier raised the κ score dramatically, to 0.3125 (fair interobserver agreement).
For the postsession fracture model, the κ among the 4 independent observational scorers improved only marginally, to 0.1156 (still poor), again attributed to a difference in severity of grading: faculty 1 (harsh) versus faculty 2 (relatively kind). Examination of the κ scores of faculty 1 and faculty 2 revealed 72.73% agreement; residents agreed 81.82% of the time.
Discussion
The importance of surgical skill development in resident education is emphasized in the ACGME Core Competencies.23 The ACGME instructed all programs to require residents to gain competency in 6 areas: patient care, interpersonal and communication skills, medical knowledge, professionalism, practice-based learning and systems-based practice. Although many surgeon educators and residents are focused on these 6 Core Competencies, current standards do not require surgical skills laboratory training and simply require residents to log cases into the ACGME website. Minimal case number recommendations are in place for graduating senior residents, but these numbers are based on averages with no strong scientific basis.
Although sweeping changes in orthopedic residency training went into effect July 1, 2013, this system remains untested and may offer room for improvement. One change is the restructuring of the PGY-1 internship. A basic surgical skills curriculum must include goals, objectives, and assessment metrics; skills used in the initial management of injured patients, including splinting, casting, application of traction devices, and other types of immobilization; and basic operative skills, including soft-tissue management, suturing, bone management, arthroscopy, fluoroscopy, and use of basic orthopedic equipment.21
Orthopedic program directors and residents were recently surveyed regarding the current state of orthopedic motor skills training.24 Three key findings deserve emphasis: There is a lack of objective criteria for evaluating resident performance in the skills laboratory; most program directors who have a laboratory do not understand the associated costs; and the most significant issue for program directors is the financial challenge of operating a motor skills laboratory. The survey findings strongly suggest that proposed changes in skills training should be accompanied by careful cost analysis before widespread implementation.
Although various online demonstrations of entire surgeries are available, as are textbooks describing a generalized approach to musculoskeletal surgery, we assume that, as laid out in the Core Competencies, residents are fine-tuning their surgical skills by actively participating in operating rooms under direct observation of attending physicians. To our knowledge, however, there are no data regarding how often this happens in the operative setting, where volume and efficiency are becoming increasingly scrutinized. There has been much concern over how hour restrictions will affect residents’ total operative experience.25,26 Finally, we have no means to objectively evaluate residents’ surgical skills on graduation.
Other programs have implemented surgical skill simulators, but an orthopedics-specific surgical skills laboratory, to our knowledge, has been discussed in only 1 study.21 Results from randomized controlled trials reported in the general surgery literature have proved simulation-based training leads to detectable benefits for learners in clinical settings.27-29 Over the past decade, some alternative surgical skills training methods have been adopted in orthopedic surgery as well. These methods include hands-on training in specifically designed surgical skills laboratories using cadaver models or synthetic bones; software tools; and computerized simulators. In recent years, numerous studies reported in the orthopedic literature have examined arthroscopic simulators in residency training.18-20,30-34 However, these studies are arguably more specific to sports subspecialties and thus more pertinent to upper-level trainees.
Our study results showed that surgical skills laboratory training should be a required aspect of our residents’ training. Although less of a dramatic improvement was noted in the written examination component of the laboratory, the overall knowledge base improved (Table 3). This was especially evident at the PGY-1 level, where written examination scores increased from a presession median of 80% to a postsession median of 85%. A larger degree of improvement was found with the ankle fracture model, and there was statistical improvement at all training levels, from PGY-1 to PGY-3. Previous work has shown that intensive laboratory-based training can be effective, particularly for first-year residents. Sonnadara and colleagues35 demonstrated that a 30-day intensive surgical skills course effectively helped first-year orthopedic residents develop targeted basic surgical skills. In a follow-up study, Sonnadara and colleagues36 demonstrated that a surgical skills course completed at the beginning of a residency was effective in teaching targeted technical skills, and that skills taught in this manner can have excellent retention rates.
There are limitations inherent in our skills course. The κ agreement in the ankle fracture model was low before and after administration, which we attribute to 1 observer outlier. This could be amended by removing outliers and further objectifying and simplifying the scoring system (A–F). Right now, we do not have enough data to determine whether the scores actually improve significantly through the training years or whether they will correlate with operating room experience. Our study had no control. For future investigations, we are considering having general orthopedic surgeons from the community perform the same scenarios and be graded with the same checklists as a control. Implementation, however, may be a challenge. Both our written examination and our ankle fracture model checklist have not been validated—this is one of our next steps. The point system used to score the ankle fracture model was subjectively developed and would benefit from further standardization before drawing conclusions about true validity.
Conclusion
Orthopedic residency programs, like programs in other surgical specialties, are increasingly focused on teaching and documenting the learning of core competencies, even as work-hour restrictions and demands for clinical efficiency limit the amount of time residents spend in the operating room. We have demonstrated the potential value of an intensive laboratory in improving junior-level residents’ basic surgical skills and knowledge. We will continue to refine our methods, with a goal being to create reproducible models that could be adapted by other orthopedic residency programs and by other surgical educators.
For the resident, the surgical residency is physically, emotionally, and intellectually demanding, requiring longitudinally concentrated effort. Although education of orthopedic surgeons necessarily occurs within the context of the health care delivery system, vital lessons also are taught in laboratories, skill stations, and surgical simulators. Before practice-based learning can take place, residents must gain experience and demonstrate growth in surgical skills, including decision-making and technical skills. These skill sets are difficult to systematically teach and objectively analyze.
The most effective way to teach and assess a resident’s knowledge of musculoskeletal medicine remains unclear at this point. Much of the current literature addresses the issue at the medical student level.1-7 Some studies have shown the effectiveness of surgical training programs, both cadaveric and computer-based simulators, in teaching various surgical skill sets.8-14 The orthopedic literature has seen a boom in surgical simulators aimed at the upper-level resident. Many of the topics involve use of arthroscopic simulators.15-19 Evidence suggests that simulators can discriminate between novice and expert users, but discrimination between novice and intermediate trainees in surgical education should be paramount.20
The American Board of Orthopaedic Surgery (ABOS) and the orthopedic Residency Review Committee (RRC) recommended new requirements for structured motor skills training in basic orthopedic surgery education,21 which were approved by the Accreditation Council for Graduate Medical Education (ACGME) board of directors and went into effect on July 1, 2013. In response to the new ACGME guidelines, our institution created a skills laboratory devoted to surgical simulation. Our focus in implementing this surgical skills simulation was junior-level, specifically postgraduate year 1 to 3 (PGY-1 to PGY-3), orthopedic residents. Our first goal was to set up a series of surgical training stations to educate junior-level residents in 4 core areas: handling and comfort with basic power equipment, casting/splinting, suturing, and surgical instrument identification. A secondary goal was to objectively evaluate the residents through written examinations (presession–postsession) and a novel ankle fracture model (pre–post).
Materials and Methods
Institutional review board approval was obtained before beginning the investigation.
Written Examination
We created a multiple-choice 25-question written examination (Appendix) and administered it to 11 junior residents before and after they participated in the training. This examination assessed their knowledge base of basic orthopedic tenets, including basic bone healing, basic fracture repair (Arbeitsgemeinschaft für Osteosynthesefragen [AO] principles22), suturing, surgical instrument identification, casting/splinting, and elementary implant-design rationale.
Evaluator Scorecard
We created an evaluation scorecard (Figure 1) and had 2 faculty members and 2 senior-level residents complete it independently. Junior residents were evaluated on a sawbones lateral malleolar ankle fracture model at 2 time points. As with the written examinations, the junior residents completed the fracture model both before and immediately after the multiple skill sessions. Each of the 15 data points was scored from 1 to 4, for a total of 60 points.
Facility for Surgical Training Session
Our Clinical Skills Education and Assessment Center houses small-group interactive laboratories for administration, debriefing, and assessment of simulations with the latest in audiovisual equipment. Five stations were created: hands-on introduction to surgical power equipment using sawbones, wood, and polyvinylchloride (PVC) pipe; hands-on introduction to casting and splinting; hands-on introduction to suturing; hands-on interaction with surgical scrub technician assisting with instrument identification; and didactic PowerPoint (Microsoft, Redmond, Washington) presentation focusing on core trauma competencies, basic orthopedic design rationale, and basic bone biology.
Development of Surgical Skills Training Session
Multiple faculty members and senior-level residents collaborated to create the skill stations (Figure 2), which were designed based on ACGME recommendations and on weaknesses our program had seen in junior-level residents. We devoted an afternoon to this session, excusing our program’s junior residents from clinical responsibilities. Four PGY-1, 5 PGY-2, and 2 PGY-3 residents participated. (Four of our 15 junior residents were unable to attend because of clinical responsibilities.) The afternoon started by dividing the 11 junior residents into 2 groups. Before the session, while one group performed the ankle fracture model and was being evaluated, the other took the written examination. This closely timed portion was allotted only 20 minutes. Then residents were divided into 5 groups of 2 or 3 and were rotated through all 5 stations. Forty minutes were allotted for each station. Residents were not evaluated during this portion. The stations were intended solely for education, and each station was staffed by a faculty member and/or senior-level resident.
Cordless reciprocating saws and drills were purchased to introduce and refine junior residents’ motor skills. Sawbones, 2×4-in sections of wood, and PVC pipe were used in the training. Emphasis was placed on tactile feel and feedback with both sawing and drilling. For the casting and splinting session, we used 4-in fiberglass, 4-in plaster rolls, and cotton soft roll to demonstrate a multitude of common casts and splints (Figure 3). Casts included short- and long-arm casts and short-leg casts. Splinting included coaptation, sugar tong, and ulnar gutter splints for the upper extremity and a short-leg posterior splint for the lower extremity.
The didactic PowerPoint presentation drew largely from content in chapters of the book AO Principles of Fracture Management.22 Content included condensed, to-the-point high-yield summaries of AO tenets, basic bone healing and biology, and orthopedic implant-design rationale focused on these elementary principles:
◾ Basic screw design, including cortical, cancellous, and locking screw designs.
◾ Evolution of plate osteosynthesis to currently used locking compression plate.
◾ Locking plate principles.
◾ Lag technique.
◾ Plate use: compression mode, neutralization, bridging, buttress, anti-glide.
The suturing portion was performed with thawed ham hocks (Figure 4). This model replicates live tissue layers and allows a layered closure technique as a training tool. Both 0 and 2-0 absorbable suture were available for a layered, deep fascial closure; also available was 2-0 nonabsorbable nylon for the skin. Staple guns were available, as were basic surgical instruments, including quality needle drivers, Adson forceps, and suture scissors. The knots demonstrated included simple, horizontal mattress, vertical mattress, and tension-relieving. One- and 2-hand tying and instrument tying were reinforced.
The final session consisted of surgical instrument identification. A certified orthopedic scrub technician participated. On site were multiple trays, including a basic bone set, a hand-and-foot set, small and large fragment sets, and a hip set. A detailed review of each set was led by the surgical technician. This review was followed by a question-and-answer session with the junior residents. After the session, we ended with the written examination and the ankle fracture model.
Statistical Methods
We report presession and postsession means, modes, and medians as measures of score-central tendencies. Our small sample size makes the assumption of Gaussian distribution tenuous and more susceptible to outliers. Therefore, in addition to reporting means, we include medians and modes to more accurately account for outliers. Moreover, the κ statistic is a robust measure of interrater agreement for 2 or more groups. We report κ statistics to determine the interrater reliability of 4 independent observers.
Results
Written Examination
Eleven residents (PGY-1 to PGY-3) completed the examination (Table 1). For the entire group, mean (SD) presession percentile was 87.3 (10.4), median was 88, and mode was 96; mean (SD) was 80 (12.6) for PGY-1, 89.6 (6.7) for PGY-2, and 96 (5.7) for PGY-3. For the entire group, mean (SD) postsession percentile was 92 (8.4), median was 96, and mode was 96; mean (SD) was 85 (10.5) for PGY-1, 96 (4) for PGY-2, and 96 (0) for PGY-3 (Table 2).
There was a significant presession–postsession difference in scores among all test takers, regardless of training level (P = .019). The PGY-1 level did not reach statistical significance in improvement from presession to postsession (P = .080); the PGY-2 level also did not reach statistical significance in improvement (P = .099); the PGY-3 level did not have enough participants to calculate a P value based on a paired Student t test.
Ankle Fracture Model
Actual percentile scores are listed in Table 3. For the entire group, mean (SD) overall presession percentile was 68.6 (13.9), median was 67, and mode was 67; mean (SD) was 58.8 (9.8) for PGY-1, 76.1 (13.6) for PGY-2, and 69.5 (9.8) for PGY-3. For the entire group, mean (SD) postsession percentile was 95.2 (5.2), median was 97, and mode was 97; mean (SD) was 91.8 (6.3) for PGY-1, 97.1 (3.5) for PGY-2, and 97.3 (2.4) for PGY-3.
There was a large and significant presession–postsession difference in scores among all test takers, regardless of training level (P = .03). Each group reached statistical significance in improvement from presession to postsession: PGY-1 (P = .04), PGY-2 (P = .01), and PGY-3 (P = .03).
For κ calculations, we adjusted all scores to ordinal data and thus used a standard grading system:
Score Grade
90–100 A
80–89 B
70–79 C
60–69 D
0–59 F
For the presession fracture model, the κ among the 4 independent observational scorers was 0.1148 (Table 4), which is poor based on κ scoring criteria and which we attribute to the particularly harsh grading by 1 observational scorer (faculty 1) relative to the other scorers’. Examination of the κ scores of faculty 1 and faculty 2 indicated only 9.09% agreement. Conversely, the κ among resident scorers agreed 54.55% of the time. Removing faculty 1 as an outlier raised the κ score dramatically, to 0.3125 (fair interobserver agreement).
For the postsession fracture model, the κ among the 4 independent observational scorers improved only marginally, to 0.1156 (still poor), again attributed to a difference in severity of grading: faculty 1 (harsh) versus faculty 2 (relatively kind). Examination of the κ scores of faculty 1 and faculty 2 revealed 72.73% agreement; residents agreed 81.82% of the time.
Discussion
The importance of surgical skill development in resident education is emphasized in the ACGME Core Competencies.23 The ACGME instructed all programs to require residents to gain competency in 6 areas: patient care, interpersonal and communication skills, medical knowledge, professionalism, practice-based learning and systems-based practice. Although many surgeon educators and residents are focused on these 6 Core Competencies, current standards do not require surgical skills laboratory training and simply require residents to log cases into the ACGME website. Minimal case number recommendations are in place for graduating senior residents, but these numbers are based on averages with no strong scientific basis.
Although sweeping changes in orthopedic residency training went into effect July 1, 2013, this system remains untested and may offer room for improvement. One change is the restructuring of the PGY-1 internship. A basic surgical skills curriculum must include goals, objectives, and assessment metrics; skills used in the initial management of injured patients, including splinting, casting, application of traction devices, and other types of immobilization; and basic operative skills, including soft-tissue management, suturing, bone management, arthroscopy, fluoroscopy, and use of basic orthopedic equipment.21
Orthopedic program directors and residents were recently surveyed regarding the current state of orthopedic motor skills training.24 Three key findings deserve emphasis: There is a lack of objective criteria for evaluating resident performance in the skills laboratory; most program directors who have a laboratory do not understand the associated costs; and the most significant issue for program directors is the financial challenge of operating a motor skills laboratory. The survey findings strongly suggest that proposed changes in skills training should be accompanied by careful cost analysis before widespread implementation.
Although various online demonstrations of entire surgeries are available, as are textbooks describing a generalized approach to musculoskeletal surgery, we assume that, as laid out in the Core Competencies, residents are fine-tuning their surgical skills by actively participating in operating rooms under direct observation of attending physicians. To our knowledge, however, there are no data regarding how often this happens in the operative setting, where volume and efficiency are becoming increasingly scrutinized. There has been much concern over how hour restrictions will affect residents’ total operative experience.25,26 Finally, we have no means to objectively evaluate residents’ surgical skills on graduation.
Other programs have implemented surgical skill simulators, but an orthopedics-specific surgical skills laboratory, to our knowledge, has been discussed in only 1 study.21 Results from randomized controlled trials reported in the general surgery literature have proved simulation-based training leads to detectable benefits for learners in clinical settings.27-29 Over the past decade, some alternative surgical skills training methods have been adopted in orthopedic surgery as well. These methods include hands-on training in specifically designed surgical skills laboratories using cadaver models or synthetic bones; software tools; and computerized simulators. In recent years, numerous studies reported in the orthopedic literature have examined arthroscopic simulators in residency training.18-20,30-34 However, these studies are arguably more specific to sports subspecialties and thus more pertinent to upper-level trainees.
Our study results showed that surgical skills laboratory training should be a required aspect of our residents’ training. Although less of a dramatic improvement was noted in the written examination component of the laboratory, the overall knowledge base improved (Table 3). This was especially evident at the PGY-1 level, where written examination scores increased from a presession median of 80% to a postsession median of 85%. A larger degree of improvement was found with the ankle fracture model, and there was statistical improvement at all training levels, from PGY-1 to PGY-3. Previous work has shown that intensive laboratory-based training can be effective, particularly for first-year residents. Sonnadara and colleagues35 demonstrated that a 30-day intensive surgical skills course effectively helped first-year orthopedic residents develop targeted basic surgical skills. In a follow-up study, Sonnadara and colleagues36 demonstrated that a surgical skills course completed at the beginning of a residency was effective in teaching targeted technical skills, and that skills taught in this manner can have excellent retention rates.
There are limitations inherent in our skills course. The κ agreement in the ankle fracture model was low before and after administration, which we attribute to 1 observer outlier. This could be amended by removing outliers and further objectifying and simplifying the scoring system (A–F). Right now, we do not have enough data to determine whether the scores actually improve significantly through the training years or whether they will correlate with operating room experience. Our study had no control. For future investigations, we are considering having general orthopedic surgeons from the community perform the same scenarios and be graded with the same checklists as a control. Implementation, however, may be a challenge. Both our written examination and our ankle fracture model checklist have not been validated—this is one of our next steps. The point system used to score the ankle fracture model was subjectively developed and would benefit from further standardization before drawing conclusions about true validity.
Conclusion
Orthopedic residency programs, like programs in other surgical specialties, are increasingly focused on teaching and documenting the learning of core competencies, even as work-hour restrictions and demands for clinical efficiency limit the amount of time residents spend in the operating room. We have demonstrated the potential value of an intensive laboratory in improving junior-level residents’ basic surgical skills and knowledge. We will continue to refine our methods, with a goal being to create reproducible models that could be adapted by other orthopedic residency programs and by other surgical educators.
1. Schmale GA. More evidence of educational inadequacies in musculoskeletal medicine. Clin Orthop. 2005;(437):251-259.
2. Day CS, Yeh AC, Franko O, Ramirez M, Krupat E. Musculoskeletal medicine: an assessment of the attitudes and knowledge of medical students at Harvard Medical School. Acad Med. 2007;82(5):452-457.
3. Bilderback K, Eggerstedt J, Sadasivan KK, et al. Design and implementation of a system-based course in musculoskeletal medicine for medical students. J Bone Joint Surg Am. 2008;90(10):2292-2300.
4. Freedman KB, Bernstein J. Educational deficiencies in musculoskeletal medicine. J Bone Joint Surg Am. 2002;84(4):604-608.
5. Corbett EC Jr, Elnicki DM, Conaway MR. When should students learn essential physical examination skills? Views of internal medicine clerkship directors in North America. Acad Med. 2008;83(1):96-99.
6. Coady DA, Walker DJ, Kay LJ. Teaching medical students musculoskeletal examination skills: identifying barriers to learning and ways of overcoming them. Scand J Rheumatol. 2004;33(1):47-51.
7. Saleh K, Messner R, Axtell S, Harris I, Mahowald ML. Development and evaluation of an integrated musculoskeletal disease course for medical students. J Bone Joint Surg Am. 2004;86(8):1653-1658.
8. van Empel PJ, Verdam MG, Huirne JA, Bonjer HJ, Meijerink WJ, Scheele F. Open knot-tying skills: resident skills assessed. J Obstet Gynaecol Res. 2013;39(5):1030-1036.
9. Barrier BF, Thompson AB, McCullough MW, Occhino JA. A novel and inexpensive vaginal hysterectomy simulator. Simul Healthc. 2012;7(6):374-379.
10. Liss MA, McDougall EM. Robotic surgical simulation. Cancer J. 2013;19(2):124-129.
11. Stegemann AP, Ahmed K, Syed JR, et al. Fundamental skills of robotic surgery: a multi-institutional randomized controlled trial for validation of a simulation-based curriculum. Urology. 2013;81(4):767-774.
12. Duran C, Bismuth J, Mitchell E. A nationwide survey of vascular surgery trainees reveals trends in operative experience, confidence, and attitudes about simulation. J Vasc Surg. 2013;58(2):524-528.
13. Kuhls DA, Risucci DA, Bowyer MW, Luchette FA. Advanced surgical skills for exposure in trauma: a new surgical skills cadaver course for surgery residents and fellows. J Trauma Acute Care Surg. 2013;74(2):664-670.
14. Sanfey HA, Dunnington GL. Basic surgical skills testing for junior residents: current views of general surgery program directors. J Am Coll Surg. 2011;212(3):406-412.
15. Alvand A, Khan T, Al-Ali S, Jackson WF, Price AJ, Rees JL. Simple visual parameters for objective assessment of arthroscopic skill. J Bone Joint Surg Am. 2012;94(13):e97.
16. Jackson WF, Khan T, Alvand A, et al. Learning and retaining simulated arthroscopic meniscal repair skills. J Bone Joint Surg Am. 2012;94(17):e132.
17. Pernar LI, Smink DS, Hicks G, Peyre SE. Residents can successfully teach basic surgical skills in the simulation center. J Surg Educ. 2012;69(5):617-622.
18. Tuijthof GJ, Visser P, Sierevelt IN, Van Dijk CN, Kerkhoffs GM. Does perception of usefulness of arthroscopic simulators differ with levels of experience? Clin Orthop. 2011;469(6):1701-1708.
19. Martin KD, Cameron K, Belmont PJ, Schoenfeld A, Owens BD. Shoulder arthroscopy simulator performance correlates with resident and shoulder arthroscopy experience. J Bone Joint Surg Am. 2012;94(21):e160.
20. Slade Shantz JA, Leiter JR, Gottschalk T, MacDonald PB. The internal validity of arthroscopic simulators and their effectiveness in arthroscopic education. Knee Surg Sports Traumatol Arthrosc. 2014;22(1):33-40.
21. Roberts S, Menage J, Eisenstein SM. The cartilage end-plate and intervertebral disc in scoliosis: calcification and other sequelae. J Orthop Res. 1993;11(5):747-757.
22. Ruedi TP, Buckley RE, Moran CG. AO Principles of Fracture Management. Stuttgart, Germany: Thieme; 2007.
23. Chen CL, Chen WC, Chiang JH, Ho CF. Interscapular hibernoma: case report and literature review. Kaohsiung J Med Sci. 2011;27(8):348-352.
24. Karam MD, Pedowitz RA, Natividad H, Murray J, Marsh JL. Current and future use of surgical skills training laboratories in orthopaedic resident education: a national survey. J Bone Joint Surg Am. 2013;95(1):e4.
25. Baskies MA, Ruchelsman DE, Capeci CM, Zuckerman JD, Egol KA. Operative experience in an orthopaedic surgery residency program: the effect of work-hour restrictions. J Bone Joint Surg Am. 2008;90(4):924-927.
26. Pappas AJ, Teague DC. The impact of the Accreditation Council for Graduate Medical Education work-hour regulations on the surgical experience of orthopaedic surgery residents. J Bone Joint Surg Am. 2007;89(4):904-909.
27. Palter VN, Grantcharov T, Harvey A, Macrae HM. Ex vivo technical skills training transfers to the operating room and enhances cognitive learning: a randomized controlled trial. Ann Surg. 2011;253(5):886-889.
28. Franzeck FM, Rosenthal R, Muller MK, et al. Prospective randomized controlled trial of simulator-based versus traditional in-surgery laparoscopic camera navigation training. Surg Endosc. 2012;26(1):235-241.
29. Zendejas B, Cook DA, Bingener J, et al. Simulation-based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair: a randomized controlled trial. Ann Surg. 2011;254(3):502-509.
30. Hui Y, Safir O, Dubrowski A, Carnahan H. What skills should simulation training in arthroscopy teach residents? A focus on resident input. Int J Comput Assist Radiol Surg. 2013;8(6):945-953.
31. Butler A, Olson T, Koehler R, Nicandri G. Do the skills acquired by novice surgeons using anatomic dry models transfer effectively to the task of diagnostic knee arthroscopy performed on cadaveric specimens? J Bone Joint Surg Am. 2013;95(3):e15(1-8).
32. Martin KD, Belmont PJ, Schoenfeld AJ, Todd M, Cameron KL, Owens BD. Arthroscopic basic task performance in shoulder simulator model correlates with similar task performance in cadavers. J Bone Joint Surg Am. 2011;93(21):e1271-e1275.
33. Elliott MJ, Caprise PA, Henning AE, Kurtz CA, Sekiya JK. Diagnostic knee arthroscopy: a pilot study to evaluate surgical skills. Arthroscopy. 2012;28(2):218-224.
34. Andersen C, Winding TN, Vesterby MS. Development of simulated arthroscopic skills. Acta Orthop. 2011;82(1):90-95.
35. Sonnadara RR, Van Vliet A, Safir O, et al. Orthopedic boot camp: examining the effectiveness of an intensive surgical skills course. Surgery. 2011;149(6):745-749.
36. Sonnadara RR, Garbedian S, Safir O, et al. Orthopaedic boot camp II: examining the retention rates of an intensive surgical skills course. Surgery. 2012;151(6):803-807.
1. Schmale GA. More evidence of educational inadequacies in musculoskeletal medicine. Clin Orthop. 2005;(437):251-259.
2. Day CS, Yeh AC, Franko O, Ramirez M, Krupat E. Musculoskeletal medicine: an assessment of the attitudes and knowledge of medical students at Harvard Medical School. Acad Med. 2007;82(5):452-457.
3. Bilderback K, Eggerstedt J, Sadasivan KK, et al. Design and implementation of a system-based course in musculoskeletal medicine for medical students. J Bone Joint Surg Am. 2008;90(10):2292-2300.
4. Freedman KB, Bernstein J. Educational deficiencies in musculoskeletal medicine. J Bone Joint Surg Am. 2002;84(4):604-608.
5. Corbett EC Jr, Elnicki DM, Conaway MR. When should students learn essential physical examination skills? Views of internal medicine clerkship directors in North America. Acad Med. 2008;83(1):96-99.
6. Coady DA, Walker DJ, Kay LJ. Teaching medical students musculoskeletal examination skills: identifying barriers to learning and ways of overcoming them. Scand J Rheumatol. 2004;33(1):47-51.
7. Saleh K, Messner R, Axtell S, Harris I, Mahowald ML. Development and evaluation of an integrated musculoskeletal disease course for medical students. J Bone Joint Surg Am. 2004;86(8):1653-1658.
8. van Empel PJ, Verdam MG, Huirne JA, Bonjer HJ, Meijerink WJ, Scheele F. Open knot-tying skills: resident skills assessed. J Obstet Gynaecol Res. 2013;39(5):1030-1036.
9. Barrier BF, Thompson AB, McCullough MW, Occhino JA. A novel and inexpensive vaginal hysterectomy simulator. Simul Healthc. 2012;7(6):374-379.
10. Liss MA, McDougall EM. Robotic surgical simulation. Cancer J. 2013;19(2):124-129.
11. Stegemann AP, Ahmed K, Syed JR, et al. Fundamental skills of robotic surgery: a multi-institutional randomized controlled trial for validation of a simulation-based curriculum. Urology. 2013;81(4):767-774.
12. Duran C, Bismuth J, Mitchell E. A nationwide survey of vascular surgery trainees reveals trends in operative experience, confidence, and attitudes about simulation. J Vasc Surg. 2013;58(2):524-528.
13. Kuhls DA, Risucci DA, Bowyer MW, Luchette FA. Advanced surgical skills for exposure in trauma: a new surgical skills cadaver course for surgery residents and fellows. J Trauma Acute Care Surg. 2013;74(2):664-670.
14. Sanfey HA, Dunnington GL. Basic surgical skills testing for junior residents: current views of general surgery program directors. J Am Coll Surg. 2011;212(3):406-412.
15. Alvand A, Khan T, Al-Ali S, Jackson WF, Price AJ, Rees JL. Simple visual parameters for objective assessment of arthroscopic skill. J Bone Joint Surg Am. 2012;94(13):e97.
16. Jackson WF, Khan T, Alvand A, et al. Learning and retaining simulated arthroscopic meniscal repair skills. J Bone Joint Surg Am. 2012;94(17):e132.
17. Pernar LI, Smink DS, Hicks G, Peyre SE. Residents can successfully teach basic surgical skills in the simulation center. J Surg Educ. 2012;69(5):617-622.
18. Tuijthof GJ, Visser P, Sierevelt IN, Van Dijk CN, Kerkhoffs GM. Does perception of usefulness of arthroscopic simulators differ with levels of experience? Clin Orthop. 2011;469(6):1701-1708.
19. Martin KD, Cameron K, Belmont PJ, Schoenfeld A, Owens BD. Shoulder arthroscopy simulator performance correlates with resident and shoulder arthroscopy experience. J Bone Joint Surg Am. 2012;94(21):e160.
20. Slade Shantz JA, Leiter JR, Gottschalk T, MacDonald PB. The internal validity of arthroscopic simulators and their effectiveness in arthroscopic education. Knee Surg Sports Traumatol Arthrosc. 2014;22(1):33-40.
21. Roberts S, Menage J, Eisenstein SM. The cartilage end-plate and intervertebral disc in scoliosis: calcification and other sequelae. J Orthop Res. 1993;11(5):747-757.
22. Ruedi TP, Buckley RE, Moran CG. AO Principles of Fracture Management. Stuttgart, Germany: Thieme; 2007.
23. Chen CL, Chen WC, Chiang JH, Ho CF. Interscapular hibernoma: case report and literature review. Kaohsiung J Med Sci. 2011;27(8):348-352.
24. Karam MD, Pedowitz RA, Natividad H, Murray J, Marsh JL. Current and future use of surgical skills training laboratories in orthopaedic resident education: a national survey. J Bone Joint Surg Am. 2013;95(1):e4.
25. Baskies MA, Ruchelsman DE, Capeci CM, Zuckerman JD, Egol KA. Operative experience in an orthopaedic surgery residency program: the effect of work-hour restrictions. J Bone Joint Surg Am. 2008;90(4):924-927.
26. Pappas AJ, Teague DC. The impact of the Accreditation Council for Graduate Medical Education work-hour regulations on the surgical experience of orthopaedic surgery residents. J Bone Joint Surg Am. 2007;89(4):904-909.
27. Palter VN, Grantcharov T, Harvey A, Macrae HM. Ex vivo technical skills training transfers to the operating room and enhances cognitive learning: a randomized controlled trial. Ann Surg. 2011;253(5):886-889.
28. Franzeck FM, Rosenthal R, Muller MK, et al. Prospective randomized controlled trial of simulator-based versus traditional in-surgery laparoscopic camera navigation training. Surg Endosc. 2012;26(1):235-241.
29. Zendejas B, Cook DA, Bingener J, et al. Simulation-based mastery learning improves patient outcomes in laparoscopic inguinal hernia repair: a randomized controlled trial. Ann Surg. 2011;254(3):502-509.
30. Hui Y, Safir O, Dubrowski A, Carnahan H. What skills should simulation training in arthroscopy teach residents? A focus on resident input. Int J Comput Assist Radiol Surg. 2013;8(6):945-953.
31. Butler A, Olson T, Koehler R, Nicandri G. Do the skills acquired by novice surgeons using anatomic dry models transfer effectively to the task of diagnostic knee arthroscopy performed on cadaveric specimens? J Bone Joint Surg Am. 2013;95(3):e15(1-8).
32. Martin KD, Belmont PJ, Schoenfeld AJ, Todd M, Cameron KL, Owens BD. Arthroscopic basic task performance in shoulder simulator model correlates with similar task performance in cadavers. J Bone Joint Surg Am. 2011;93(21):e1271-e1275.
33. Elliott MJ, Caprise PA, Henning AE, Kurtz CA, Sekiya JK. Diagnostic knee arthroscopy: a pilot study to evaluate surgical skills. Arthroscopy. 2012;28(2):218-224.
34. Andersen C, Winding TN, Vesterby MS. Development of simulated arthroscopic skills. Acta Orthop. 2011;82(1):90-95.
35. Sonnadara RR, Van Vliet A, Safir O, et al. Orthopedic boot camp: examining the effectiveness of an intensive surgical skills course. Surgery. 2011;149(6):745-749.
36. Sonnadara RR, Garbedian S, Safir O, et al. Orthopaedic boot camp II: examining the retention rates of an intensive surgical skills course. Surgery. 2012;151(6):803-807.
