Curricular milestones in rheumatology: Is granular better?

 

A new curricular “road map” attempts to lay out precisely what rheumatology fellows are expected to be able to know and do at different time points in their training.

The 80 “curricular milestones” for rheumatology, developed under the direction of the Accreditation Council for Graduate Medical Education, are meant to complement the 23 internal medicine subspecialty reporting milestones already mandated by the ACGME for use in trainee evaluation.

Unlike the reporting milestones, which came into use in 2014, the curricular milestones are not compulsory. Instead they “offer a guide so that trainees and the people teaching them know what’s expected of them,” said Lisa Criscione-Schreiber, MD, of Duke University, Durham, N.C., the lead author of a recent article in Arthritis Care & Research describing the new milestones (Arthritis Care Res. 2016 Nov 11. doi: 10.1002/acr.23151).

Shawn Rocco/Duke Health

“The curricular milestones point to knowledge, skills, and attitudes that we think, and that the American College of Rheumatology’s board of directors agrees, are core to what it means to be a rheumatologist,” and which correspond with the Entrustable Professional Activities approved by the ACR, Dr. Criscione-Schreiber said in an interview.

While the reporting milestones are designed to target broader competencies, the curricular milestones are highly granular. For example, a rheumatology fellow at 12 months is expected to be able to “perform compensated polarized microscopy to examine and interpret synovial fluid,” according to one milestone; before 24 months of training, he or she is expected to be able to teach others to do the same.

Authors of an accompanying editorial (Arthritis Care Res. 2016 Nov 11. doi: 10.1002/acr.23150) by Richard Panush, MD, of the University of Southern California in Los Angeles, Eric Hsieh, MD, also of USC, and Nortin Hadler, MD, of the University of North Carolina at Chapel Hill, praised the curricular milestones as meticulous and well conceived, but questioned their complexity and the broader push toward rubric-based training, particularly in the absence of evidence showing it to be superior to established methods.

Further, they wrote, the milestones’ granularity could prove a problem: “We fear that the investment in metrics designed to parse someone’s notion of the components of competence as a trainee will have far less relevance to professional performance ... than the global assessments.”

Ultimately, Dr. Panush acknowledged in an interview, the move to milestones – including the curricular milestones – may prove worthwhile. But it simply “isn’t as sufficiently grounded in science as we would have liked for such a major departure from everything we have done, and with all that implementing the change would imply: the time, the manpower, the cost,” he said.

A leap of faith?

Dr. Criscione-Schreiber, who is Duke’s rheumatology program director, acknowledged that the milestones’ value and utility remain to be determined. “We’re not sure it’s the best way, but it’s a try,” she said. “We’re all in an era of continuous quality improvement in which you change something then test whether it works. The ACGME is collecting data about all this – if they find out it’s burdensome, they have assured program directors that they will change it.”

Calvin Brown, MD, rheumatology program director at Northwestern University in Chicago and a coauthor of the milestones, pointed out that the approach isn’t entirely novel: Canadian educators have been moving forward with an increasingly granular milestones-based approach for some time, and that much of the ACGME’s work parallels that of Canada’s Royal College of Physicians and Surgeons.

But Dr. Brown conceded that the editorialists’ concerns were valid. “The ACGME has been the main organization describing parameters by which we train doctors, and it is relatively unopposed. We’ve seen a proliferation of the rubric-based approach, and we are adopting these [curricular milestones] without scientific evidence. I commend the editorial authors in saying we need to find ways to measure this to see if it does improve, but if we wait for that evidence before we move forward, we’ll never get past square one.”

Observation and feedback

A key goal of the milestones is to encourage direct observation of trainees. Dr. Panush and his colleagues, in their editorial, argue that “a wise and astute clinician educator ... can judge when trainees are ready to function independently and have learned how to keep learning and changing without needing menus of evaluations or detailed instruments.”

 

Dr. Criscione-Schreiber said there’s traditionally been less direct observation in rheumatology training than people would like to think. “It used to be more like what we call the piano analogy. You send them into a room for an hour alone with a piano and some sheet music and they come out saying ‘I can play the piano.’ And you believe them.”

Dr. Brown, meanwhile, compared traditional training approaches to what he called the teabag model. The trainee “was the teabag steeping in the hot water of an academic medical center, and the major criteria for finishing was time,” he said. “I think all of us would recognize that this doesn’t take into consideration different environments, different learning styles and rates of learning.”

Marcy Bolster, MD, rheumatology program director at Massachusetts General Hospital in Boston, who cochaired the milestones writing committee with Dr. Brown, said in an interview that the current generation of fellows “thrives on feedback – they like to hear where they’re doing well and what they can improve.”

Using the milestones

Dr. Panush, in an interview, noted that the implementation of rubric-based education in internal medicine training came with a host of problems resulting in published critiques. He cited a recent article by Ronald Witteles, MD, and Abraham Verghese, MD, both of Stanford (Calif.) University, that criticized the reporting milestones as onerous to implement in large internal medicine programs, leading to excess administrative work and box-ticking (JAMA Intern Med. 2016;176[11]:1599-1600).

But rheumatology fellowship programs, which are generally small, may be less likely to find the curricular milestones burdensome, their champions say. The success of these milestones depends, in large part, on how people choose to integrate them into their training programs.

“The milestones appear a little daunting because they’re new,” Dr. Bolster said, cautioning that they are not designed to be foisted wholesale on trainees. Used judiciously, she said, the curricular milestones can help identify areas of strength and weakness and foster dialogue. “What I do is select a few of them, a couple of times a year, to go over with each fellow,” she said. “I also recommend them for fellows to use as self-reflection, so that they may determine where they are in terms of training.”

Details vs. the big picture

Anne Bass, MD, the rheumatology program director at the Hospital for Special Surgery in New York, who did not take part in either the curricular milestones writing committee or the editorial, said they could be used piecemeal, helping program directors to design portions of their curriculum, or to provide feedback and remediation by pointing out areas of trainee weakness.

“I think the key point is these are not required for reporting, and so the way I see it is as a tool you can use or not use as you choose,” she said.

But Dr. Bass also said, echoing a concern of the editorialists, that she felt that the milestones’ emphasis on highly specific strengths and weaknesses risked “losing the forest for the trees.”

A trainee who appears weak in one knowledge area is likely to have broader shortcomings, she said.

“If you’re somebody who gets the basic facts, you’re able to interview the patient, do the physical exam, and collect the information, but you’re not really able to apply it yet – that’s probably something global to that trainee that will apply with other areas as well,” not just the milestone you’re measuring, Dr. Bass said.

“The whole point about milestones is it’s showing you where along the continuum you are. But the continuum goes from data collection and description to application, testing, management, teaching – that’s kind of the spectrum, and that development is generally across the board. It’s not usually specific to one area.”

Training rheumatologists like pilots

The authors of the milestones say they’re responding to a sense widely shared in the rheumatology community that training must adapt to the needs of a changing field.

 

“Medicine is not the same as it was 10 or even 5 years ago, and certainly not as it was 20 years ago. A physician who is going to practice 10 years from now will require a different skill set,” Dr. Criscione-Schreiber said.

This, of course, includes many skills not directly related to patient care but rather successful functioning within current systems of health care delivery.

Dr. Criscione-Schreiber said that amid a coming shortage of rheumatologists, the next generation must be prepared to lead teams.

“We’re going to have clinics with nurses doing much of the education, and a whole team of people taking care of patients, if we have any hope of caring for the population in the U.S. that needs rheumatologists. It’s different from training a one-on-one doctor who sits in the room with a patient for 25 minutes,” she said.

Dr. Brown said he views the milestones effort, whether it proves successful or not, as an attempt to train in a way that ensures all future rheumatologists share the same core skills.

“People ask us all the time which doctor should they see for this or that condition, because they want to see someone really good,” Dr. Brown said. “Contrast that to the last time you got on an airplane. Did you ask a friend which pilot you should fly with? No, because you trusted that the pilot was competent, and had all requisite training, to safely pilot the plane. We want medicine to have many of those qualities and characteristics.”

Human beings are more complicated than airplanes, Dr. Brown acknowledged. “So it’s not perfectly analogous. But if we can more granularly describe training, and what it is we want to achieve, we’d all benefit – to the point where we can confidently tell you to see the rheumatologist trained in an accredited program who’s nearest your home.”

 

A new curricular “road map” attempts to lay out precisely what rheumatology fellows are expected to be able to know and do at different time points in their training.

The 80 “curricular milestones” for rheumatology, developed under the direction of the Accreditation Council for Graduate Medical Education, are meant to complement the 23 internal medicine subspecialty reporting milestones already mandated by the ACGME for use in trainee evaluation.

Unlike the reporting milestones, which came into use in 2014, the curricular milestones are not compulsory. Instead they “offer a guide so that trainees and the people teaching them know what’s expected of them,” said Lisa Criscione-Schreiber, MD, of Duke University, Durham, N.C., the lead author of a recent article in Arthritis Care & Research describing the new milestones (Arthritis Care Res. 2016 Nov 11. doi: 10.1002/acr.23151).

Shawn Rocco/Duke Health

“The curricular milestones point to knowledge, skills, and attitudes that we think, and that the American College of Rheumatology’s board of directors agrees, are core to what it means to be a rheumatologist,” and which correspond with the Entrustable Professional Activities approved by the ACR, Dr. Criscione-Schreiber said in an interview.

While the reporting milestones are designed to target broader competencies, the curricular milestones are highly granular. For example, a rheumatology fellow at 12 months is expected to be able to “perform compensated polarized microscopy to examine and interpret synovial fluid,” according to one milestone; before 24 months of training, he or she is expected to be able to teach others to do the same.

Authors of an accompanying editorial (Arthritis Care Res. 2016 Nov 11. doi: 10.1002/acr.23150) by Richard Panush, MD, of the University of Southern California in Los Angeles, Eric Hsieh, MD, also of USC, and Nortin Hadler, MD, of the University of North Carolina at Chapel Hill, praised the curricular milestones as meticulous and well conceived, but questioned their complexity and the broader push toward rubric-based training, particularly in the absence of evidence showing it to be superior to established methods.

Further, they wrote, the milestones’ granularity could prove a problem: “We fear that the investment in metrics designed to parse someone’s notion of the components of competence as a trainee will have far less relevance to professional performance ... than the global assessments.”

Ultimately, Dr. Panush acknowledged in an interview, the move to milestones – including the curricular milestones – may prove worthwhile. But it simply “isn’t as sufficiently grounded in science as we would have liked for such a major departure from everything we have done, and with all that implementing the change would imply: the time, the manpower, the cost,” he said.

A leap of faith?

Dr. Criscione-Schreiber, who is Duke’s rheumatology program director, acknowledged that the milestones’ value and utility remain to be determined. “We’re not sure it’s the best way, but it’s a try,” she said. “We’re all in an era of continuous quality improvement in which you change something then test whether it works. The ACGME is collecting data about all this – if they find out it’s burdensome, they have assured program directors that they will change it.”

Calvin Brown, MD, rheumatology program director at Northwestern University in Chicago and a coauthor of the milestones, pointed out that the approach isn’t entirely novel: Canadian educators have been moving forward with an increasingly granular milestones-based approach for some time, and that much of the ACGME’s work parallels that of Canada’s Royal College of Physicians and Surgeons.

But Dr. Brown conceded that the editorialists’ concerns were valid. “The ACGME has been the main organization describing parameters by which we train doctors, and it is relatively unopposed. We’ve seen a proliferation of the rubric-based approach, and we are adopting these [curricular milestones] without scientific evidence. I commend the editorial authors in saying we need to find ways to measure this to see if it does improve, but if we wait for that evidence before we move forward, we’ll never get past square one.”

Observation and feedback

A key goal of the milestones is to encourage direct observation of trainees. Dr. Panush and his colleagues, in their editorial, argue that “a wise and astute clinician educator ... can judge when trainees are ready to function independently and have learned how to keep learning and changing without needing menus of evaluations or detailed instruments.”

 

Dr. Criscione-Schreiber said there’s traditionally been less direct observation in rheumatology training than people would like to think. “It used to be more like what we call the piano analogy. You send them into a room for an hour alone with a piano and some sheet music and they come out saying ‘I can play the piano.’ And you believe them.”

Dr. Brown, meanwhile, compared traditional training approaches to what he called the teabag model. The trainee “was the teabag steeping in the hot water of an academic medical center, and the major criteria for finishing was time,” he said. “I think all of us would recognize that this doesn’t take into consideration different environments, different learning styles and rates of learning.”

Marcy Bolster, MD, rheumatology program director at Massachusetts General Hospital in Boston, who cochaired the milestones writing committee with Dr. Brown, said in an interview that the current generation of fellows “thrives on feedback – they like to hear where they’re doing well and what they can improve.”

Using the milestones

Dr. Panush, in an interview, noted that the implementation of rubric-based education in internal medicine training came with a host of problems resulting in published critiques. He cited a recent article by Ronald Witteles, MD, and Abraham Verghese, MD, both of Stanford (Calif.) University, that criticized the reporting milestones as onerous to implement in large internal medicine programs, leading to excess administrative work and box-ticking (JAMA Intern Med. 2016;176[11]:1599-1600).

But rheumatology fellowship programs, which are generally small, may be less likely to find the curricular milestones burdensome, their champions say. The success of these milestones depends, in large part, on how people choose to integrate them into their training programs.

“The milestones appear a little daunting because they’re new,” Dr. Bolster said, cautioning that they are not designed to be foisted wholesale on trainees. Used judiciously, she said, the curricular milestones can help identify areas of strength and weakness and foster dialogue. “What I do is select a few of them, a couple of times a year, to go over with each fellow,” she said. “I also recommend them for fellows to use as self-reflection, so that they may determine where they are in terms of training.”

Details vs. the big picture

Anne Bass, MD, the rheumatology program director at the Hospital for Special Surgery in New York, who did not take part in either the curricular milestones writing committee or the editorial, said they could be used piecemeal, helping program directors to design portions of their curriculum, or to provide feedback and remediation by pointing out areas of trainee weakness.

“I think the key point is these are not required for reporting, and so the way I see it is as a tool you can use or not use as you choose,” she said.

But Dr. Bass also said, echoing a concern of the editorialists, that she felt that the milestones’ emphasis on highly specific strengths and weaknesses risked “losing the forest for the trees.”

A trainee who appears weak in one knowledge area is likely to have broader shortcomings, she said.

“If you’re somebody who gets the basic facts, you’re able to interview the patient, do the physical exam, and collect the information, but you’re not really able to apply it yet – that’s probably something global to that trainee that will apply with other areas as well,” not just the milestone you’re measuring, Dr. Bass said.

“The whole point about milestones is it’s showing you where along the continuum you are. But the continuum goes from data collection and description to application, testing, management, teaching – that’s kind of the spectrum, and that development is generally across the board. It’s not usually specific to one area.”

Training rheumatologists like pilots

The authors of the milestones say they’re responding to a sense widely shared in the rheumatology community that training must adapt to the needs of a changing field.

 

“Medicine is not the same as it was 10 or even 5 years ago, and certainly not as it was 20 years ago. A physician who is going to practice 10 years from now will require a different skill set,” Dr. Criscione-Schreiber said.

This, of course, includes many skills not directly related to patient care but rather successful functioning within current systems of health care delivery.

Dr. Criscione-Schreiber said that amid a coming shortage of rheumatologists, the next generation must be prepared to lead teams.

“We’re going to have clinics with nurses doing much of the education, and a whole team of people taking care of patients, if we have any hope of caring for the population in the U.S. that needs rheumatologists. It’s different from training a one-on-one doctor who sits in the room with a patient for 25 minutes,” she said.

Dr. Brown said he views the milestones effort, whether it proves successful or not, as an attempt to train in a way that ensures all future rheumatologists share the same core skills.

“People ask us all the time which doctor should they see for this or that condition, because they want to see someone really good,” Dr. Brown said. “Contrast that to the last time you got on an airplane. Did you ask a friend which pilot you should fly with? No, because you trusted that the pilot was competent, and had all requisite training, to safely pilot the plane. We want medicine to have many of those qualities and characteristics.”

Human beings are more complicated than airplanes, Dr. Brown acknowledged. “So it’s not perfectly analogous. But if we can more granularly describe training, and what it is we want to achieve, we’d all benefit – to the point where we can confidently tell you to see the rheumatologist trained in an accredited program who’s nearest your home.”

 

A new curricular “road map” attempts to lay out precisely what rheumatology fellows are expected to be able to know and do at different time points in their training.

The 80 “curricular milestones” for rheumatology, developed under the direction of the Accreditation Council for Graduate Medical Education, are meant to complement the 23 internal medicine subspecialty reporting milestones already mandated by the ACGME for use in trainee evaluation.

Unlike the reporting milestones, which came into use in 2014, the curricular milestones are not compulsory. Instead they “offer a guide so that trainees and the people teaching them know what’s expected of them,” said Lisa Criscione-Schreiber, MD, of Duke University, Durham, N.C., the lead author of a recent article in Arthritis Care & Research describing the new milestones (Arthritis Care Res. 2016 Nov 11. doi: 10.1002/acr.23151).

Shawn Rocco/Duke Health

“The curricular milestones point to knowledge, skills, and attitudes that we think, and that the American College of Rheumatology’s board of directors agrees, are core to what it means to be a rheumatologist,” and which correspond with the Entrustable Professional Activities approved by the ACR, Dr. Criscione-Schreiber said in an interview.

While the reporting milestones are designed to target broader competencies, the curricular milestones are highly granular. For example, a rheumatology fellow at 12 months is expected to be able to “perform compensated polarized microscopy to examine and interpret synovial fluid,” according to one milestone; before 24 months of training, he or she is expected to be able to teach others to do the same.

Authors of an accompanying editorial (Arthritis Care Res. 2016 Nov 11. doi: 10.1002/acr.23150) by Richard Panush, MD, of the University of Southern California in Los Angeles, Eric Hsieh, MD, also of USC, and Nortin Hadler, MD, of the University of North Carolina at Chapel Hill, praised the curricular milestones as meticulous and well conceived, but questioned their complexity and the broader push toward rubric-based training, particularly in the absence of evidence showing it to be superior to established methods.

Further, they wrote, the milestones’ granularity could prove a problem: “We fear that the investment in metrics designed to parse someone’s notion of the components of competence as a trainee will have far less relevance to professional performance ... than the global assessments.”

Ultimately, Dr. Panush acknowledged in an interview, the move to milestones – including the curricular milestones – may prove worthwhile. But it simply “isn’t as sufficiently grounded in science as we would have liked for such a major departure from everything we have done, and with all that implementing the change would imply: the time, the manpower, the cost,” he said.

A leap of faith?

Dr. Criscione-Schreiber, who is Duke’s rheumatology program director, acknowledged that the milestones’ value and utility remain to be determined. “We’re not sure it’s the best way, but it’s a try,” she said. “We’re all in an era of continuous quality improvement in which you change something then test whether it works. The ACGME is collecting data about all this – if they find out it’s burdensome, they have assured program directors that they will change it.”

Calvin Brown, MD, rheumatology program director at Northwestern University in Chicago and a coauthor of the milestones, pointed out that the approach isn’t entirely novel: Canadian educators have been moving forward with an increasingly granular milestones-based approach for some time, and that much of the ACGME’s work parallels that of Canada’s Royal College of Physicians and Surgeons.

But Dr. Brown conceded that the editorialists’ concerns were valid. “The ACGME has been the main organization describing parameters by which we train doctors, and it is relatively unopposed. We’ve seen a proliferation of the rubric-based approach, and we are adopting these [curricular milestones] without scientific evidence. I commend the editorial authors in saying we need to find ways to measure this to see if it does improve, but if we wait for that evidence before we move forward, we’ll never get past square one.”

Observation and feedback

A key goal of the milestones is to encourage direct observation of trainees. Dr. Panush and his colleagues, in their editorial, argue that “a wise and astute clinician educator ... can judge when trainees are ready to function independently and have learned how to keep learning and changing without needing menus of evaluations or detailed instruments.”

 

Dr. Criscione-Schreiber said there’s traditionally been less direct observation in rheumatology training than people would like to think. “It used to be more like what we call the piano analogy. You send them into a room for an hour alone with a piano and some sheet music and they come out saying ‘I can play the piano.’ And you believe them.”

Dr. Brown, meanwhile, compared traditional training approaches to what he called the teabag model. The trainee “was the teabag steeping in the hot water of an academic medical center, and the major criteria for finishing was time,” he said. “I think all of us would recognize that this doesn’t take into consideration different environments, different learning styles and rates of learning.”

Marcy Bolster, MD, rheumatology program director at Massachusetts General Hospital in Boston, who cochaired the milestones writing committee with Dr. Brown, said in an interview that the current generation of fellows “thrives on feedback – they like to hear where they’re doing well and what they can improve.”

Using the milestones

Dr. Panush, in an interview, noted that the implementation of rubric-based education in internal medicine training came with a host of problems resulting in published critiques. He cited a recent article by Ronald Witteles, MD, and Abraham Verghese, MD, both of Stanford (Calif.) University, that criticized the reporting milestones as onerous to implement in large internal medicine programs, leading to excess administrative work and box-ticking (JAMA Intern Med. 2016;176[11]:1599-1600).

But rheumatology fellowship programs, which are generally small, may be less likely to find the curricular milestones burdensome, their champions say. The success of these milestones depends, in large part, on how people choose to integrate them into their training programs.

“The milestones appear a little daunting because they’re new,” Dr. Bolster said, cautioning that they are not designed to be foisted wholesale on trainees. Used judiciously, she said, the curricular milestones can help identify areas of strength and weakness and foster dialogue. “What I do is select a few of them, a couple of times a year, to go over with each fellow,” she said. “I also recommend them for fellows to use as self-reflection, so that they may determine where they are in terms of training.”

Details vs. the big picture

Anne Bass, MD, the rheumatology program director at the Hospital for Special Surgery in New York, who did not take part in either the curricular milestones writing committee or the editorial, said they could be used piecemeal, helping program directors to design portions of their curriculum, or to provide feedback and remediation by pointing out areas of trainee weakness.

“I think the key point is these are not required for reporting, and so the way I see it is as a tool you can use or not use as you choose,” she said.

But Dr. Bass also said, echoing a concern of the editorialists, that she felt that the milestones’ emphasis on highly specific strengths and weaknesses risked “losing the forest for the trees.”

A trainee who appears weak in one knowledge area is likely to have broader shortcomings, she said.

“If you’re somebody who gets the basic facts, you’re able to interview the patient, do the physical exam, and collect the information, but you’re not really able to apply it yet – that’s probably something global to that trainee that will apply with other areas as well,” not just the milestone you’re measuring, Dr. Bass said.

“The whole point about milestones is it’s showing you where along the continuum you are. But the continuum goes from data collection and description to application, testing, management, teaching – that’s kind of the spectrum, and that development is generally across the board. It’s not usually specific to one area.”

Training rheumatologists like pilots

The authors of the milestones say they’re responding to a sense widely shared in the rheumatology community that training must adapt to the needs of a changing field.

 

“Medicine is not the same as it was 10 or even 5 years ago, and certainly not as it was 20 years ago. A physician who is going to practice 10 years from now will require a different skill set,” Dr. Criscione-Schreiber said.

This, of course, includes many skills not directly related to patient care but rather successful functioning within current systems of health care delivery.

Dr. Criscione-Schreiber said that amid a coming shortage of rheumatologists, the next generation must be prepared to lead teams.

“We’re going to have clinics with nurses doing much of the education, and a whole team of people taking care of patients, if we have any hope of caring for the population in the U.S. that needs rheumatologists. It’s different from training a one-on-one doctor who sits in the room with a patient for 25 minutes,” she said.

Dr. Brown said he views the milestones effort, whether it proves successful or not, as an attempt to train in a way that ensures all future rheumatologists share the same core skills.

“People ask us all the time which doctor should they see for this or that condition, because they want to see someone really good,” Dr. Brown said. “Contrast that to the last time you got on an airplane. Did you ask a friend which pilot you should fly with? No, because you trusted that the pilot was competent, and had all requisite training, to safely pilot the plane. We want medicine to have many of those qualities and characteristics.”

Human beings are more complicated than airplanes, Dr. Brown acknowledged. “So it’s not perfectly analogous. But if we can more granularly describe training, and what it is we want to achieve, we’d all benefit – to the point where we can confidently tell you to see the rheumatologist trained in an accredited program who’s nearest your home.”