Donna L. Agan, EdD

Setting and Participants

The ultrasound training program was created at Scripps Mercy Hospital San Diego Campus, a 500‐bed community hospital in San Diego, California, for integration into a 3‐year internal medicine residency program. It was accredited by the Accreditation Council for Graduate Medical Education (ACGME) and consisted of approximately 33 residents, and 23 full‐time and 82 part‐time faculty. Since 2005, all internal medicine residents have been participating in the ultrasound training program and their progress followed as a part of the ACGME Educational Innovation Project. Of the 41 consecutive graduating residents in whom performance data were collected, no resident had prior formal training in ultrasound.

Program Overview

Based upon initial studies of performing limited echo examination,1520 the following imaging protocols were combined to comprise CLUE, a brief, quick‐look two‐dimensional multi‐targeted ultrasound examination: (1) the extracranial carotid bulb for carotid atherosclerosis, (2) parasternal long‐axis view for left ventricular systolic dysfunction and left atrial enlargement, (3) apical lung views for interstitial edema, (4) basal lung views for pleural effusion, (5) a subcostal 4‐chamber view for isolated right ventricular enlargement or pericardial effusion, (6) the longitudinal view of the inferior vena cava for elevated central venous pressures, and (7) a mid‐abdominal longitudinal view for abdominal aortic aneurysm. Evidence‐basis for the exam targets and specifics of subjective diagnostic CLUE criteria (Table 1) have been published elsewhere.2130

Two useful mnemonics were created to teach the imaging protocol. If using only the 3 MHz cardiac probe, residents were taught to work backward against the flow of blood, in regards to physiologic effects and the sequence of CLUE views. Starting in the left ventricle, systolic function was first evaluated, followed by left atrial enlargement, the presence of lung comets, then lung effusions, then right ventricular enlargement, the presence of pericardial effusion, then elevation of central venous pressures. If the high‐frequency 5 MHz linear probe was available for carotid imaging, then an additional mnemonic was remembered that atherosclerotic progression increased from top to bottom in CLUE, typified by the frequent detection of early disease in the carotid bulb, then occasional cardiac manifestations, followed by the infrequent late manifestation of an abdominal aortic aneurysm. In our practice, performance of the complete CLUE starting at the top (carotids), changing transducers to work backward in the thorax (cardiac, lung, and inferior vena cava), and finishing with the bottom (aorta) was often dependent upon equipment and linear probe availability at the point‐of‐care.

A formalized CLUE curriculum was implemented into the residency in 2006. Twelve monthly 1‐hour CLUE lectures were given per year. Most lectures were 3045 minutes in length, leaving 1530 minutes for imaging resident or patient volunteers. All forms of ultrasound devices available to the residents, including pocket‐sized, hand‐carried, cart‐based, and standard ultrasound machines, were used in this forum. To learn the fundamentals of imaging technique, the intern during the cardiology consultation month rotation was first expected to image 1030 patients in the echocardiography and vascular ultrasound labs under the tutelage of the sonographers. Once weekly, 1‐hour bedside teaching was given to junior and senior residents on the intensive care unit (ICU) and cardiology consult rotations, in a traditional case‐based format. Over the ICU month rotation, junior and senior residents could each image an additional 1030 patients, resulting in a minimum of 30 studies obtained on acutely ill patients during the ICU rotations of residency. During clinical care rotations over the 3‐year residency, all residents imaged a minimum of 30 patients (at least 10 proctored studies during their internship cardiology consultation month, 10 proctored during ICU junior year rotations, and 10 proctored during ICU senior year rotations), with some residents imaging over a hundred patients (Table 2). To assist their education in CLUE, multiple learning aides were made available, including instructional how‐to‐image videos, a 200‐page syllabus, self‐assessment tests, and an instructional web site. Overall, the independent study and performance of CLUE was encouraged, but without formal performance incentives, monitoring, or effect upon residency evaluations.

At our institution, the medical director of the Echocardiography and Vascular ultrasound laboratory was a cardiologist (B.J.K.) who directed the CLUE training program. The Director provided the monthly lecture series to the entire residency and was responsible for weekly 1‐hour bedside ICU rounds. If given maintenance responsibilities of weekly bedside ICU rounds (1 hour/week), monthly lecture and preparation (5 hours/month), and availability to teach the cardiology intern (3 hours/month) and maintain the Web site (4 hours/month), the program required 4 hours/week of the Director's time. The program used 3 dedicated devices: the SonoSite 180 (SonoSite, Inc, Bothell, WA), the MicroMaxx (SonoSite, Inc) and, in 2010, a pocket‐sized cardiac ultrasound stethoscope, the Vscan (GE Healthcare, Wauwatosa, WI). No patient charges were submitted for performance or interpretation of any CLUE.

Assessment and Follow‐Up

A proficiency test was performed at the end of each resident's senior year. The test, cardiovascular limited ultrasound exam‐clinical exercise (CLUE‐CEX), involved imaging any available, consenting patient and assessing the resident's technical skills by image quality, knowledge of diagnostic criteria, and ability to discuss the clinical aspects of potential findings in a question‐and‐answer oral interview format, typically requiring 2030 minutes to perform. Each resident CLUE view was rated for: (1) image quality which accounted for 44% of total exam points, (2) specific knowledge related to each view which accounted for 28% of total exam points, and (3) diagnostic accuracy of the interpretation of each view which accounted for 28% of total exam points (see Figure 1). CLUE‐CEX scores were recorded as a percentage of total possible points, normalized to the difficulty of imaging the individual patient as determined by the Director's imaging. The test encompassed performance of all 7 views, demonstrated in 2 exams employing 2 transducers (cardiac and vascular) on the same patient (Figure 1). A passing threshold had been empirically derived at >80% of the total available points, a value that: (1) required performance in all 3 categories, (2) subjectively correlated to competency when assessed by the Director, and (3) had parity with other thresholds of clinical skill assessment by faculty and in graduate education. The Director had no knowledge of non‐CLUE resident evaluations, In‐training scores, or academic performance outside of CLUE. Residents were not remanded for CLUE‐CEX failure.

Figure 1

The graduating class of 2011 was the first class to initially enter into an entire residency program fully immersed in the CLUE curriculum, and was therefore specifically asked to report their impression of the CLUE program after graduation through a post‐residency questionnaire. A Likert‐type scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree) was used to assess the perceived validity of the following statements: (1) CLUE improved my own bedside cardiovascular evaluation; and (2) I would use CLUE if ultrasound were available in my future position. Each resident was then asked if too much, not enough, or an appropriate amount of time was spent to learn CLUE, and to choose the most effective form of CLUE teaching to which they were exposed: didactic lectures, bedside ICU teaching, Web site/syllabus, and one‐to‐one training with the Director or sonographer.

Statistical Analysis

The CLUE experience was divided into 3 phases: (1) pre‐CLUE era, the 4‐year period (classes graduating 20022005) prior to the institution of the formal CLUE curriculum; (2) the 2‐year CLUE phase‐in period (classes graduating 20062007), in which portions of the residency were undergoing the 3‐year curriculum; (3) the 4‐year CLUE‐CEX era (classes graduating 20082011) when all residency classes were trained in the standardized fashion and underwent CLUE‐CEX assessment. In‐training postgraduate year‐3 (PGY‐3) scores, the result of a nationwide standardized test developed by the American College of Physicians, were used as representative of senior resident academic knowledge. A percentile rank score is provided to compare residents to nationwide data. The group of residents who had been selected to be the following year's chief residents had their CLUE‐CEX scores analyzed as a subgroup.

Data are presented as mean standard deviation and analyzed in SPSS, version 12.0 (SPSS, Inc, Chicago, IL). Linear regression was used to investigate the relationship between In‐training percentile ranks and CLUE‐CEX scores. Analysis of variance was used to determine any effect of gender and chief resident selection on CLUE‐CEX, and to assess average resident In‐training percentile ranks during the pre‐CLUE and CLUE‐CEX periods. Subset analysis of individual CLUE‐CEX scores was performed in regards to image quality, diagnostic knowledge, and interpretative skills. A value of P < 0.05 was considered significant.

Observations During CLUE Program Development

CLUE‐CEX scores (20082011) included data from 41 residents; 51% were male. In the class of 2009, one second‐year male resident transferred to another program for nonacademic reasons, reducing its number to 9. We observed that the impact of the CLUE program depended in part upon resident‐to‐resident teaching and required a critical mass of residents to be trained during a phase‐in period before a maximal effect could be appreciated. We observed that didactic knowledge occurred before imaging skills and remained dominant by graduation, with mean percentile CLUE‐CEX scores for image quality, knowledge, and interpretative accuracy at 82% 5%, 91% 3%, and 91% 8%, respectively. Residents typically found apical lung imaging the easiest to perform (CLUE‐CEX score of 89% 19%), followed by carotid (84% 18%), inferior vena cava (IVC) imaging (84% 26%), screening for abdominal aortic aneurysm (AAA) (83% 2 4%), parasternal long‐axis (79% 30%), and subcostal cardiac 4‐chamber imaging (73% 33%). Each view had technical and diagnostic pitfalls that were noted during resident practice (see Table 1), resulting in changes in our teaching and case review in subsequent years.

Residency and CLUE Performance

In attempting to achieve a CLUE proficiency score of >80% on the CLUE‐CEX in their graduating year, 8/41 (19.5%) senior residents failed. In these 8 residents, imaging quality, knowledge, and interpretative accuracy were all depressed: 55% 19%, 79% 11%, and 75% 11%, respectively. Two of these 8 had been selected as future chief residents over the 4‐year period, positions typically awarded to 2 residents per graduating year. The performance of the residents is seen in Table 3. The CLUE program did not exert a negative effect upon the academic performance of the residency, as evidenced by the lack of a significant difference in the Pre‐CLUE, 2‐year CLUE, and CLUE‐CEX periods in regards to average resident In‐training percentile rank scores (67.5 20.1, 62.3 20.5, 69.4 16.9, respectively; P = 0.37).

Figure 2 shows the relationship between CLUE‐CEX scores and In‐training PGY‐3 scores. There was no significant relationship between resident academic performance and CLUE capabilities (r = 0.05, P = 0.75). Similarly, chief resident performance (n = 14) was not significantly associated with CLUE‐CEX scores (r = 0.15, P = 0.37), nor was male gender (P = 0.07). Approximately one‐half (49%) of the residents in the 4‐year CLUE‐CEX era entered fellowships, unchanged from historic rates, with only 1 resident during this era entering into a cardiology fellowship.

Figure 2

The Likert‐type questionnaire was returned by 11/11 graduating residents in 2011. Mean score of 4.3 0.6 (range: 35), with 6/11 responding agree, was given for the statement of whether CLUE improved the resident's own bedside exam. A score of 4.5 0.7 (range: 35), with 7/11 responding strongly agree, was given for whether the resident would use CLUE in the future if ultrasound were available. The majority (9/11) of residents felt that the time spent on CLUE was appropriate, with 2 residents responding not enough. Residents ranked one‐to‐one training with the Director(n = 6), followed by bedside ICU rounds (n = 5) as the preferred teaching methods to learn CLUE.

Setting and Participants

The ultrasound training program was created at Scripps Mercy Hospital San Diego Campus, a 500‐bed community hospital in San Diego, California, for integration into a 3‐year internal medicine residency program. It was accredited by the Accreditation Council for Graduate Medical Education (ACGME) and consisted of approximately 33 residents, and 23 full‐time and 82 part‐time faculty. Since 2005, all internal medicine residents have been participating in the ultrasound training program and their progress followed as a part of the ACGME Educational Innovation Project. Of the 41 consecutive graduating residents in whom performance data were collected, no resident had prior formal training in ultrasound.

Program Overview

Based upon initial studies of performing limited echo examination,1520 the following imaging protocols were combined to comprise CLUE, a brief, quick‐look two‐dimensional multi‐targeted ultrasound examination: (1) the extracranial carotid bulb for carotid atherosclerosis, (2) parasternal long‐axis view for left ventricular systolic dysfunction and left atrial enlargement, (3) apical lung views for interstitial edema, (4) basal lung views for pleural effusion, (5) a subcostal 4‐chamber view for isolated right ventricular enlargement or pericardial effusion, (6) the longitudinal view of the inferior vena cava for elevated central venous pressures, and (7) a mid‐abdominal longitudinal view for abdominal aortic aneurysm. Evidence‐basis for the exam targets and specifics of subjective diagnostic CLUE criteria (Table 1) have been published elsewhere.2130

Two useful mnemonics were created to teach the imaging protocol. If using only the 3 MHz cardiac probe, residents were taught to work backward against the flow of blood, in regards to physiologic effects and the sequence of CLUE views. Starting in the left ventricle, systolic function was first evaluated, followed by left atrial enlargement, the presence of lung comets, then lung effusions, then right ventricular enlargement, the presence of pericardial effusion, then elevation of central venous pressures. If the high‐frequency 5 MHz linear probe was available for carotid imaging, then an additional mnemonic was remembered that atherosclerotic progression increased from top to bottom in CLUE, typified by the frequent detection of early disease in the carotid bulb, then occasional cardiac manifestations, followed by the infrequent late manifestation of an abdominal aortic aneurysm. In our practice, performance of the complete CLUE starting at the top (carotids), changing transducers to work backward in the thorax (cardiac, lung, and inferior vena cava), and finishing with the bottom (aorta) was often dependent upon equipment and linear probe availability at the point‐of‐care.

A formalized CLUE curriculum was implemented into the residency in 2006. Twelve monthly 1‐hour CLUE lectures were given per year. Most lectures were 3045 minutes in length, leaving 1530 minutes for imaging resident or patient volunteers. All forms of ultrasound devices available to the residents, including pocket‐sized, hand‐carried, cart‐based, and standard ultrasound machines, were used in this forum. To learn the fundamentals of imaging technique, the intern during the cardiology consultation month rotation was first expected to image 1030 patients in the echocardiography and vascular ultrasound labs under the tutelage of the sonographers. Once weekly, 1‐hour bedside teaching was given to junior and senior residents on the intensive care unit (ICU) and cardiology consult rotations, in a traditional case‐based format. Over the ICU month rotation, junior and senior residents could each image an additional 1030 patients, resulting in a minimum of 30 studies obtained on acutely ill patients during the ICU rotations of residency. During clinical care rotations over the 3‐year residency, all residents imaged a minimum of 30 patients (at least 10 proctored studies during their internship cardiology consultation month, 10 proctored during ICU junior year rotations, and 10 proctored during ICU senior year rotations), with some residents imaging over a hundred patients (Table 2). To assist their education in CLUE, multiple learning aides were made available, including instructional how‐to‐image videos, a 200‐page syllabus, self‐assessment tests, and an instructional web site. Overall, the independent study and performance of CLUE was encouraged, but without formal performance incentives, monitoring, or effect upon residency evaluations.

At our institution, the medical director of the Echocardiography and Vascular ultrasound laboratory was a cardiologist (B.J.K.) who directed the CLUE training program. The Director provided the monthly lecture series to the entire residency and was responsible for weekly 1‐hour bedside ICU rounds. If given maintenance responsibilities of weekly bedside ICU rounds (1 hour/week), monthly lecture and preparation (5 hours/month), and availability to teach the cardiology intern (3 hours/month) and maintain the Web site (4 hours/month), the program required 4 hours/week of the Director's time. The program used 3 dedicated devices: the SonoSite 180 (SonoSite, Inc, Bothell, WA), the MicroMaxx (SonoSite, Inc) and, in 2010, a pocket‐sized cardiac ultrasound stethoscope, the Vscan (GE Healthcare, Wauwatosa, WI). No patient charges were submitted for performance or interpretation of any CLUE.

Assessment and Follow‐Up

A proficiency test was performed at the end of each resident's senior year. The test, cardiovascular limited ultrasound exam‐clinical exercise (CLUE‐CEX), involved imaging any available, consenting patient and assessing the resident's technical skills by image quality, knowledge of diagnostic criteria, and ability to discuss the clinical aspects of potential findings in a question‐and‐answer oral interview format, typically requiring 2030 minutes to perform. Each resident CLUE view was rated for: (1) image quality which accounted for 44% of total exam points, (2) specific knowledge related to each view which accounted for 28% of total exam points, and (3) diagnostic accuracy of the interpretation of each view which accounted for 28% of total exam points (see Figure 1). CLUE‐CEX scores were recorded as a percentage of total possible points, normalized to the difficulty of imaging the individual patient as determined by the Director's imaging. The test encompassed performance of all 7 views, demonstrated in 2 exams employing 2 transducers (cardiac and vascular) on the same patient (Figure 1). A passing threshold had been empirically derived at >80% of the total available points, a value that: (1) required performance in all 3 categories, (2) subjectively correlated to competency when assessed by the Director, and (3) had parity with other thresholds of clinical skill assessment by faculty and in graduate education. The Director had no knowledge of non‐CLUE resident evaluations, In‐training scores, or academic performance outside of CLUE. Residents were not remanded for CLUE‐CEX failure.

Figure 1

The graduating class of 2011 was the first class to initially enter into an entire residency program fully immersed in the CLUE curriculum, and was therefore specifically asked to report their impression of the CLUE program after graduation through a post‐residency questionnaire. A Likert‐type scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree) was used to assess the perceived validity of the following statements: (1) CLUE improved my own bedside cardiovascular evaluation; and (2) I would use CLUE if ultrasound were available in my future position. Each resident was then asked if too much, not enough, or an appropriate amount of time was spent to learn CLUE, and to choose the most effective form of CLUE teaching to which they were exposed: didactic lectures, bedside ICU teaching, Web site/syllabus, and one‐to‐one training with the Director or sonographer.

Statistical Analysis

The CLUE experience was divided into 3 phases: (1) pre‐CLUE era, the 4‐year period (classes graduating 20022005) prior to the institution of the formal CLUE curriculum; (2) the 2‐year CLUE phase‐in period (classes graduating 20062007), in which portions of the residency were undergoing the 3‐year curriculum; (3) the 4‐year CLUE‐CEX era (classes graduating 20082011) when all residency classes were trained in the standardized fashion and underwent CLUE‐CEX assessment. In‐training postgraduate year‐3 (PGY‐3) scores, the result of a nationwide standardized test developed by the American College of Physicians, were used as representative of senior resident academic knowledge. A percentile rank score is provided to compare residents to nationwide data. The group of residents who had been selected to be the following year's chief residents had their CLUE‐CEX scores analyzed as a subgroup.

Data are presented as mean standard deviation and analyzed in SPSS, version 12.0 (SPSS, Inc, Chicago, IL). Linear regression was used to investigate the relationship between In‐training percentile ranks and CLUE‐CEX scores. Analysis of variance was used to determine any effect of gender and chief resident selection on CLUE‐CEX, and to assess average resident In‐training percentile ranks during the pre‐CLUE and CLUE‐CEX periods. Subset analysis of individual CLUE‐CEX scores was performed in regards to image quality, diagnostic knowledge, and interpretative skills. A value of P < 0.05 was considered significant.

Observations During CLUE Program Development

CLUE‐CEX scores (20082011) included data from 41 residents; 51% were male. In the class of 2009, one second‐year male resident transferred to another program for nonacademic reasons, reducing its number to 9. We observed that the impact of the CLUE program depended in part upon resident‐to‐resident teaching and required a critical mass of residents to be trained during a phase‐in period before a maximal effect could be appreciated. We observed that didactic knowledge occurred before imaging skills and remained dominant by graduation, with mean percentile CLUE‐CEX scores for image quality, knowledge, and interpretative accuracy at 82% 5%, 91% 3%, and 91% 8%, respectively. Residents typically found apical lung imaging the easiest to perform (CLUE‐CEX score of 89% 19%), followed by carotid (84% 18%), inferior vena cava (IVC) imaging (84% 26%), screening for abdominal aortic aneurysm (AAA) (83% 2 4%), parasternal long‐axis (79% 30%), and subcostal cardiac 4‐chamber imaging (73% 33%). Each view had technical and diagnostic pitfalls that were noted during resident practice (see Table 1), resulting in changes in our teaching and case review in subsequent years.

Residency and CLUE Performance

In attempting to achieve a CLUE proficiency score of >80% on the CLUE‐CEX in their graduating year, 8/41 (19.5%) senior residents failed. In these 8 residents, imaging quality, knowledge, and interpretative accuracy were all depressed: 55% 19%, 79% 11%, and 75% 11%, respectively. Two of these 8 had been selected as future chief residents over the 4‐year period, positions typically awarded to 2 residents per graduating year. The performance of the residents is seen in Table 3. The CLUE program did not exert a negative effect upon the academic performance of the residency, as evidenced by the lack of a significant difference in the Pre‐CLUE, 2‐year CLUE, and CLUE‐CEX periods in regards to average resident In‐training percentile rank scores (67.5 20.1, 62.3 20.5, 69.4 16.9, respectively; P = 0.37).

Figure 2 shows the relationship between CLUE‐CEX scores and In‐training PGY‐3 scores. There was no significant relationship between resident academic performance and CLUE capabilities (r = 0.05, P = 0.75). Similarly, chief resident performance (n = 14) was not significantly associated with CLUE‐CEX scores (r = 0.15, P = 0.37), nor was male gender (P = 0.07). Approximately one‐half (49%) of the residents in the 4‐year CLUE‐CEX era entered fellowships, unchanged from historic rates, with only 1 resident during this era entering into a cardiology fellowship.

Figure 2

The Likert‐type questionnaire was returned by 11/11 graduating residents in 2011. Mean score of 4.3 0.6 (range: 35), with 6/11 responding agree, was given for the statement of whether CLUE improved the resident's own bedside exam. A score of 4.5 0.7 (range: 35), with 7/11 responding strongly agree, was given for whether the resident would use CLUE in the future if ultrasound were available. The majority (9/11) of residents felt that the time spent on CLUE was appropriate, with 2 residents responding not enough. Residents ranked one‐to‐one training with the Director(n = 6), followed by bedside ICU rounds (n = 5) as the preferred teaching methods to learn CLUE.

Setting and Participants

The ultrasound training program was created at Scripps Mercy Hospital San Diego Campus, a 500‐bed community hospital in San Diego, California, for integration into a 3‐year internal medicine residency program. It was accredited by the Accreditation Council for Graduate Medical Education (ACGME) and consisted of approximately 33 residents, and 23 full‐time and 82 part‐time faculty. Since 2005, all internal medicine residents have been participating in the ultrasound training program and their progress followed as a part of the ACGME Educational Innovation Project. Of the 41 consecutive graduating residents in whom performance data were collected, no resident had prior formal training in ultrasound.

Program Overview

Based upon initial studies of performing limited echo examination,1520 the following imaging protocols were combined to comprise CLUE, a brief, quick‐look two‐dimensional multi‐targeted ultrasound examination: (1) the extracranial carotid bulb for carotid atherosclerosis, (2) parasternal long‐axis view for left ventricular systolic dysfunction and left atrial enlargement, (3) apical lung views for interstitial edema, (4) basal lung views for pleural effusion, (5) a subcostal 4‐chamber view for isolated right ventricular enlargement or pericardial effusion, (6) the longitudinal view of the inferior vena cava for elevated central venous pressures, and (7) a mid‐abdominal longitudinal view for abdominal aortic aneurysm. Evidence‐basis for the exam targets and specifics of subjective diagnostic CLUE criteria (Table 1) have been published elsewhere.2130

Two useful mnemonics were created to teach the imaging protocol. If using only the 3 MHz cardiac probe, residents were taught to work backward against the flow of blood, in regards to physiologic effects and the sequence of CLUE views. Starting in the left ventricle, systolic function was first evaluated, followed by left atrial enlargement, the presence of lung comets, then lung effusions, then right ventricular enlargement, the presence of pericardial effusion, then elevation of central venous pressures. If the high‐frequency 5 MHz linear probe was available for carotid imaging, then an additional mnemonic was remembered that atherosclerotic progression increased from top to bottom in CLUE, typified by the frequent detection of early disease in the carotid bulb, then occasional cardiac manifestations, followed by the infrequent late manifestation of an abdominal aortic aneurysm. In our practice, performance of the complete CLUE starting at the top (carotids), changing transducers to work backward in the thorax (cardiac, lung, and inferior vena cava), and finishing with the bottom (aorta) was often dependent upon equipment and linear probe availability at the point‐of‐care.

A formalized CLUE curriculum was implemented into the residency in 2006. Twelve monthly 1‐hour CLUE lectures were given per year. Most lectures were 3045 minutes in length, leaving 1530 minutes for imaging resident or patient volunteers. All forms of ultrasound devices available to the residents, including pocket‐sized, hand‐carried, cart‐based, and standard ultrasound machines, were used in this forum. To learn the fundamentals of imaging technique, the intern during the cardiology consultation month rotation was first expected to image 1030 patients in the echocardiography and vascular ultrasound labs under the tutelage of the sonographers. Once weekly, 1‐hour bedside teaching was given to junior and senior residents on the intensive care unit (ICU) and cardiology consult rotations, in a traditional case‐based format. Over the ICU month rotation, junior and senior residents could each image an additional 1030 patients, resulting in a minimum of 30 studies obtained on acutely ill patients during the ICU rotations of residency. During clinical care rotations over the 3‐year residency, all residents imaged a minimum of 30 patients (at least 10 proctored studies during their internship cardiology consultation month, 10 proctored during ICU junior year rotations, and 10 proctored during ICU senior year rotations), with some residents imaging over a hundred patients (Table 2). To assist their education in CLUE, multiple learning aides were made available, including instructional how‐to‐image videos, a 200‐page syllabus, self‐assessment tests, and an instructional web site. Overall, the independent study and performance of CLUE was encouraged, but without formal performance incentives, monitoring, or effect upon residency evaluations.

At our institution, the medical director of the Echocardiography and Vascular ultrasound laboratory was a cardiologist (B.J.K.) who directed the CLUE training program. The Director provided the monthly lecture series to the entire residency and was responsible for weekly 1‐hour bedside ICU rounds. If given maintenance responsibilities of weekly bedside ICU rounds (1 hour/week), monthly lecture and preparation (5 hours/month), and availability to teach the cardiology intern (3 hours/month) and maintain the Web site (4 hours/month), the program required 4 hours/week of the Director's time. The program used 3 dedicated devices: the SonoSite 180 (SonoSite, Inc, Bothell, WA), the MicroMaxx (SonoSite, Inc) and, in 2010, a pocket‐sized cardiac ultrasound stethoscope, the Vscan (GE Healthcare, Wauwatosa, WI). No patient charges were submitted for performance or interpretation of any CLUE.

Assessment and Follow‐Up

A proficiency test was performed at the end of each resident's senior year. The test, cardiovascular limited ultrasound exam‐clinical exercise (CLUE‐CEX), involved imaging any available, consenting patient and assessing the resident's technical skills by image quality, knowledge of diagnostic criteria, and ability to discuss the clinical aspects of potential findings in a question‐and‐answer oral interview format, typically requiring 2030 minutes to perform. Each resident CLUE view was rated for: (1) image quality which accounted for 44% of total exam points, (2) specific knowledge related to each view which accounted for 28% of total exam points, and (3) diagnostic accuracy of the interpretation of each view which accounted for 28% of total exam points (see Figure 1). CLUE‐CEX scores were recorded as a percentage of total possible points, normalized to the difficulty of imaging the individual patient as determined by the Director's imaging. The test encompassed performance of all 7 views, demonstrated in 2 exams employing 2 transducers (cardiac and vascular) on the same patient (Figure 1). A passing threshold had been empirically derived at >80% of the total available points, a value that: (1) required performance in all 3 categories, (2) subjectively correlated to competency when assessed by the Director, and (3) had parity with other thresholds of clinical skill assessment by faculty and in graduate education. The Director had no knowledge of non‐CLUE resident evaluations, In‐training scores, or academic performance outside of CLUE. Residents were not remanded for CLUE‐CEX failure.

Figure 1

The graduating class of 2011 was the first class to initially enter into an entire residency program fully immersed in the CLUE curriculum, and was therefore specifically asked to report their impression of the CLUE program after graduation through a post‐residency questionnaire. A Likert‐type scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree) was used to assess the perceived validity of the following statements: (1) CLUE improved my own bedside cardiovascular evaluation; and (2) I would use CLUE if ultrasound were available in my future position. Each resident was then asked if too much, not enough, or an appropriate amount of time was spent to learn CLUE, and to choose the most effective form of CLUE teaching to which they were exposed: didactic lectures, bedside ICU teaching, Web site/syllabus, and one‐to‐one training with the Director or sonographer.

Statistical Analysis

The CLUE experience was divided into 3 phases: (1) pre‐CLUE era, the 4‐year period (classes graduating 20022005) prior to the institution of the formal CLUE curriculum; (2) the 2‐year CLUE phase‐in period (classes graduating 20062007), in which portions of the residency were undergoing the 3‐year curriculum; (3) the 4‐year CLUE‐CEX era (classes graduating 20082011) when all residency classes were trained in the standardized fashion and underwent CLUE‐CEX assessment. In‐training postgraduate year‐3 (PGY‐3) scores, the result of a nationwide standardized test developed by the American College of Physicians, were used as representative of senior resident academic knowledge. A percentile rank score is provided to compare residents to nationwide data. The group of residents who had been selected to be the following year's chief residents had their CLUE‐CEX scores analyzed as a subgroup.

Data are presented as mean standard deviation and analyzed in SPSS, version 12.0 (SPSS, Inc, Chicago, IL). Linear regression was used to investigate the relationship between In‐training percentile ranks and CLUE‐CEX scores. Analysis of variance was used to determine any effect of gender and chief resident selection on CLUE‐CEX, and to assess average resident In‐training percentile ranks during the pre‐CLUE and CLUE‐CEX periods. Subset analysis of individual CLUE‐CEX scores was performed in regards to image quality, diagnostic knowledge, and interpretative skills. A value of P < 0.05 was considered significant.

Observations During CLUE Program Development

CLUE‐CEX scores (20082011) included data from 41 residents; 51% were male. In the class of 2009, one second‐year male resident transferred to another program for nonacademic reasons, reducing its number to 9. We observed that the impact of the CLUE program depended in part upon resident‐to‐resident teaching and required a critical mass of residents to be trained during a phase‐in period before a maximal effect could be appreciated. We observed that didactic knowledge occurred before imaging skills and remained dominant by graduation, with mean percentile CLUE‐CEX scores for image quality, knowledge, and interpretative accuracy at 82% 5%, 91% 3%, and 91% 8%, respectively. Residents typically found apical lung imaging the easiest to perform (CLUE‐CEX score of 89% 19%), followed by carotid (84% 18%), inferior vena cava (IVC) imaging (84% 26%), screening for abdominal aortic aneurysm (AAA) (83% 2 4%), parasternal long‐axis (79% 30%), and subcostal cardiac 4‐chamber imaging (73% 33%). Each view had technical and diagnostic pitfalls that were noted during resident practice (see Table 1), resulting in changes in our teaching and case review in subsequent years.

Residency and CLUE Performance

In attempting to achieve a CLUE proficiency score of >80% on the CLUE‐CEX in their graduating year, 8/41 (19.5%) senior residents failed. In these 8 residents, imaging quality, knowledge, and interpretative accuracy were all depressed: 55% 19%, 79% 11%, and 75% 11%, respectively. Two of these 8 had been selected as future chief residents over the 4‐year period, positions typically awarded to 2 residents per graduating year. The performance of the residents is seen in Table 3. The CLUE program did not exert a negative effect upon the academic performance of the residency, as evidenced by the lack of a significant difference in the Pre‐CLUE, 2‐year CLUE, and CLUE‐CEX periods in regards to average resident In‐training percentile rank scores (67.5 20.1, 62.3 20.5, 69.4 16.9, respectively; P = 0.37).

Figure 2 shows the relationship between CLUE‐CEX scores and In‐training PGY‐3 scores. There was no significant relationship between resident academic performance and CLUE capabilities (r = 0.05, P = 0.75). Similarly, chief resident performance (n = 14) was not significantly associated with CLUE‐CEX scores (r = 0.15, P = 0.37), nor was male gender (P = 0.07). Approximately one‐half (49%) of the residents in the 4‐year CLUE‐CEX era entered fellowships, unchanged from historic rates, with only 1 resident during this era entering into a cardiology fellowship.

Figure 2

The Likert‐type questionnaire was returned by 11/11 graduating residents in 2011. Mean score of 4.3 0.6 (range: 35), with 6/11 responding agree, was given for the statement of whether CLUE improved the resident's own bedside exam. A score of 4.5 0.7 (range: 35), with 7/11 responding strongly agree, was given for whether the resident would use CLUE in the future if ultrasound were available. The majority (9/11) of residents felt that the time spent on CLUE was appropriate, with 2 residents responding not enough. Residents ranked one‐to‐one training with the Director(n = 6), followed by bedside ICU rounds (n = 5) as the preferred teaching methods to learn CLUE.