Catherine Battaglia, PhD, RN

Primary care providers (PCPs) need effective tools for activating health behavior change for the 125 million Americans living with a chronic condition.¹ Smoking is an important and difficult behavior to change, and a motivator for quitting is tobacco cessation advice from a PCP.^2,3 However, few PCPs provide comprehensive tobacco cessation counseling as part of routine care.^4,5 One perceived barrier that providers report is their lack of training to be effective tobacco cessation advocates.^4,6-8

Motivational interviewing (MI) promotes behavior change by using a nonadversarial approach aimed at resolving patient ambivalence. Motivational interviewing tools, such as asking open-ended questions, providing summary statements of what the patient expresses, reflective listening, and affirmations, are used to spur an intrinsic drive to change. These techniques have been applied to a broad range of health behaviors with positive outcomes and demonstrated efficacy.^9-11 Furthermore, MI can be used in primary care for changing tobacco use, alcohol consumption, physical activity, and diet.^12-14

Despite its efficacy, MI can be time-intensive to learn. Fortunately, even abbreviated MI can influence patient behavior.^15,16 Rollnick and others have developed MI interventions that are deliverable in 5 to 10 minutes.^17,18 These brief interventions focus on performing a rapid assessment of patients’ perceived importance and self-efficacy for change.^17,18

There is increased interest in training health care professionals (HCPs) in MI, yet there is no consensus on the most effective training approach.^19,20 Practitioners with many competing priorities often like to learn new skills through self-study or onetime workshops. Yet evidence suggests that these are not effective methods for gaining MI proficiency. Instead, MI training sessions that offer feedback and coaching are more effective in helping participants retain MI skills over time.^21,22

The authors developed and successfully pilot-tested an MI training program called the Motivational Interviewing Smoking Treatment Enhancement Program (MI-STEP) for HCPs. This program was designed to facilitate tobacco cessation care in the VHA primary care patient centered medical home, which VHA calls patient aligned care teams (PACTs).²³ The main conclusions of this pilot study have been reported elsewhere.²⁴

The objective of this article is to describe the process evaluation the authors conducted during the MI-STEP study to gain a better understanding of how the implementation of the MI training program could be improved. The authors identified barriers and facilitators from the perspectives of MI champions and PACT practitioners.

Methods

Thirty-four PACT practitioners (physicians, nurse practitioners, registered nurses, licensed practical nurses, and pharmacists) at 2 VA medical centers were randomly assigned to a high- or moderate-intensity MI training program during the summer of 2012. This training was delivered by “MI champions,” who were recruited from PACTs and who attended a 3-day advanced training class on MI. The training included MI skills practice, group case analysis, various role-play exercises, and didactics adapted from the Rx for Change program.²⁵ The curriculum also addressed tobacco cessation counseling using the national tobacco cessation guideline.² Each site’s health behavior coordinator (HBC) also was recruited to be an MI champion. The HBCs are typically psychologists who have received prior training in MI as well as facilitator and clinician coaching. At the VA, HBCs are charged with integrating preventive services into care. The participating sites’ institutional review boards approved all study procedures.

MI-STEP Training Program

All 34 practitioners attended a half-day on-site MI training workshop led by the site’s HBC. This training covered the basics of MI and used interactive learning methods such as role-play (Table 1). The study practitioners also received self-study materials, and throughout the study period had access to the MI champions. Practitioners who were randomized to high-intensity MI training also attended 6 supplemental 1-hour “booster sessions” to enhance specific MI skills. The MI champions led 3 of the 1-hour booster sessions with a standard agenda, including patient cases and MI exercises. During the other 3 booster sessions, participants used patient cases to interact with a standardized patient over the telephone, and the MI champions provided feedback and coaching.

Process Evaluation

Six months after the program’s completion, investigators conducted an evaluation of the MI-STEP training program with MI champions and study practitioners. One-hour focus group sessions (2 in Minneapolis; 1 in Denver) were conducted with the MI champions by a co-investigator in Minneapolis and a facilitator in Denver. Notes were taken during the sessions. MI champions were asked about the quality of their training sessions, challenges to getting PACT members to participate in the site training, challenges to teaching MI, and how they felt MI fit within VA health care philosophy.

Ten training study practitioners were randomly selected and stratified based on group intensity assignment, discipline, and site to participate in in-depth interviews. The interviews lasted about 30 minutes, and Minneapolis study investigators conducted in-person interviews with local participants and telephone interviews with Denver participants. The interviews focused on experiences with both high- and moderate-intensity MI training programs, how MI was used in their practice, barriers to implementing MI, impressions of the MI training program, and their interactions with MI champions.

Focus group leaders were experienced interviewers who had not previously interacted with MI champions in the context of this study. Investigators conducting study practitioner interviews were blinded to group assignment. All interviews were audio-recorded and transcribed verbatim. Study investigators reviewed the focus group notes and interview transcripts, identified themes independently, and then discussed group themes. The most salient themes were selected to inform implementation of a larger scale MI training program.

Results

Nine MI champions participated in the focus groups, and 8 study practitioners from both sites representing all clinical disciplines completed in-depth interviews. Table 2 identifies the characteristics of each population.

MI Champion Focus Group Themes

The champions were asked to discuss all aspects of the program, including their training as champions, role as trainers, attitudes about using MI during patient encounters, and participation in the training program. Themes from the MI champion focus groups were placed in the following categories based on the authors’ analytic approach: training MI champions, training study practitioners, and attitudes about MI.

Training MI champions. The champions identified role-play exercises and receiving feedback as strengths of the training program. The champions also expressed the desire for more hands-on practice, especially in small groups. They wanted additional training on teaching MI and facilitating the booster sessions. The champions wanted an expert to train them on how to give feedback and how to best coach practitioners in their use of MI. Champions expressed a desire to have follow-up training sessions with the standardized patient to help them hone their newly acquired coaching skills.

Training study practitioners. The champions’ key role was to train local practitioners and lead the booster sessions for the high-intensity MI training group. Champions felt ill-prepared to fully cover the training materials during the initial half-day workshop and 6 booster sessions. Champions identified difficulty coordinating schedules with the practitioners and lack of compensation for participation as significant barriers to implementing the booster sessions. Champions felt that using a standardized patient during the booster sessions was a strength of the program and that making the cases more realistic could have further enhanced the program.

Attitudes about MI. Champions from both sites perceived MI to have a positive impact on patient care. However, all champions noted there were challenges in using MI in practice. Champions felt MI takes time, energy, and practice to gain proficiency. The current primary care system is not set up to support the use of MI. The appointment time slots are fixed, and VHA goals and the spirit of MI are not always compatible. VHA performance measures encourage providers to achieve performance targets with each patient, often requiring use of directives for patients on what to do. In contrast, MI encourages the patient to take the lead on goal setting and prioritizing.

Study Practitioner Interview Themes

The practitioners were asked to discuss MI skills training, using MI skills with patients, integrating MI into daily practice, getting other PACT members involved, booster sessions, interactions with champions, and suggestions for improving the MI program. Themes from the study practitioner interviews were grouped into the following categories: MI skills training, using MI skills, integrating MI into practice, and suggestions for improving MI training (Table 3).

MI skills training. Overall, the MI high-intensity participants stated they learned useful skills. They reported asking more questions that are open-ended and were more aware of the patient’s perspective. Practitioners reported that booster sessions provided a way to reinforce, refine, and practice their MI skills. Practitioners reported that having the champion located in their own PACT was critical for connecting with their champion between sessions. Nurses and doctors reported that not having time to meet with champions was a barrier, while pharmacists reported more flexibility.

The moderate-intensity participants reported that the training had less impact. Half the respondents reported that they did not remember much of the MI training and either forgot or did not use the newly learned MI skills.

Using MI skills. Both high- and moderate-intensity participants reported using open-ended questions, reflections, affirmations, motivation scales, and active listening.

Practitioners reported that MI helped them focus on patient-centered care, since MI is collaborative. Even when a session was not successful in leading to behavior change, practitioners felt more satisfied with the quality of the interaction.

Integrating MI into practice. The high- and moderate-intensity practitioners had different perceptions of using MI in daily practice. High-intensity participants thought MI required an initial time investment, but that would be balanced by a decrease in the number of follow-up visits needed and/or delay the time between visits. The moderate-intensity participants were more likely to report struggling with the amount of time MI took.

Suggestions for improving MI training. Practitioners from both training groups offered suggestions for improving MI training. Supervisor buy-in was deemed critical to getting other PACT members involved. Practitioners suggested providing compensation or making training mandatory to help motivate others to participate in MI training. Also, practitioners were ready to expand the MI training beyond smoking cessation to incorporate other diseases and multiple comorbidities.

The moderate-intensity participants suggested more training, practice, follow-up, and feedback. These participants also suggested boosterlike sessions.

Discussion

Champions and study practitioners reported that learning MI skills was useful. The participants felt that MI was consistent with their personal philosophies regarding patient-centered care and that MI had a positive impact on patient care. Practitioners and MI champions offered several insights for improving the delivery of MI training. First, practitioners and champions highlighted how important practice and feedback were to learning MI. Booster sessions, standardized patients, and critical feedback enhanced learning.

Second, champions reported that they wanted more training in how to teach MI. Third, practitioners and champions repeatedly stated that finding the time needed to become proficient in MI was difficult and that using the MI approach with patients took additional time during clinical sessions. However, participants in the high-intensity group reported more satisfaction with the quality of their patient encounters and the freedom to follow up with patients less often.

There were aspects of the environment and MI training program that facilitated the MI learning process. The high-intensity group cited booster session feedback as being reinforcing; the moderate-intensity group expressed a desire to practice their newly acquired skill and felt feedback and coaching would have enhanced their learning. Practitioners and champions reported that using a standardized patient to enhance experiential learning activities was an asset. Standardized patients have been used successfully in other training programs.²¹

Implementing an MI training program posed a number of challenges. The biggest barrier was lack of time. PACT members found it difficult to attend a half-day MI workshop, practice MI skills, and incorporate MI routinely into daily practice. However, without the investment of time, even basic MI proficiency is unachievable.²²

This study highlighted several ways to improve feedback and coaching. First, the authors would expand the MI champion curriculum to include training to provide effective feedback/coaching. Second, the authors would train the standardized patient on how to provide feedback to the MI learner. As implemented, the standardized patient evaluated the learner only on whether the patient felt “heard” by the learner.

Perhaps most critical to the success of an MI training program is institutional support. There needs to be adequate time and space for the training process as well as support for ongoing learning and feedback as MI skills are refined. Furthermore, sufficient time is needed during patients’ appointments to allow for MI-oriented conversations. Time is an important, valuable, and scarce resource that institutions control. Administrators should realize that the up-front investment is likely to provide a downstream return as providers become proficient in MI.

There is an urgent need to find ways to incorporate training into the daily practice of busy HCPs. Although this study was limited by its small sample, it demonstrated the feasibility of implementing an MI training program for practitioners working in a busy primary care environment. This study offers concrete suggestions for overcoming barriers and enhancing facilitators, which can guide much needed larger studies as they examine MI training effectiveness on patient and clinician outcomes.

Champions and practitioners reported that learning MI was important, but opportunities to practice and receive critical feedback are needed to achieve proficiency and improve confidence. Both champions and study practitioners thought practicing with a standardized patient would enrich their learning. However, dedicated time for learning and practicing MI skills is critical and hard to arrange.

Conclusion

Practitioners can use MI to activate health behavior change in their patients. Training PACT practitioners to use MI is feasible. The results of this evaluation can be used to inform the next iteration of an MI training program for HCPs by highlighting the facilitators of and barriers to training.

Because of the interest in activating patient-centered health behavior change, these findings are important. The educational and practice opportunities were well received. Training with standardized patients and incorporating MI champions into PACTs facilitated training. However, the lack of time was a major barrier to learning and practicing MI skills and will need to be addressed. If effectively implemented, training providers by using an evidence-based approach, such as MI, can promote long-term health.

Acknowledgments
This study was funded by VA Health Services Research & Development (HSR&D) Rapid Response Project 11-019. The Center for Chronic Disease Outcomes Research is supported by the VA, VHA, Office of Research and Development, and HSR&D. Dr. Widome was supported by a VA HSR&D Career Development Award.

Primary care providers (PCPs) need effective tools for activating health behavior change for the 125 million Americans living with a chronic condition.¹ Smoking is an important and difficult behavior to change, and a motivator for quitting is tobacco cessation advice from a PCP.^2,3 However, few PCPs provide comprehensive tobacco cessation counseling as part of routine care.^4,5 One perceived barrier that providers report is their lack of training to be effective tobacco cessation advocates.^4,6-8

Motivational interviewing (MI) promotes behavior change by using a nonadversarial approach aimed at resolving patient ambivalence. Motivational interviewing tools, such as asking open-ended questions, providing summary statements of what the patient expresses, reflective listening, and affirmations, are used to spur an intrinsic drive to change. These techniques have been applied to a broad range of health behaviors with positive outcomes and demonstrated efficacy.^9-11 Furthermore, MI can be used in primary care for changing tobacco use, alcohol consumption, physical activity, and diet.^12-14

Despite its efficacy, MI can be time-intensive to learn. Fortunately, even abbreviated MI can influence patient behavior.^15,16 Rollnick and others have developed MI interventions that are deliverable in 5 to 10 minutes.^17,18 These brief interventions focus on performing a rapid assessment of patients’ perceived importance and self-efficacy for change.^17,18

There is increased interest in training health care professionals (HCPs) in MI, yet there is no consensus on the most effective training approach.^19,20 Practitioners with many competing priorities often like to learn new skills through self-study or onetime workshops. Yet evidence suggests that these are not effective methods for gaining MI proficiency. Instead, MI training sessions that offer feedback and coaching are more effective in helping participants retain MI skills over time.^21,22

The authors developed and successfully pilot-tested an MI training program called the Motivational Interviewing Smoking Treatment Enhancement Program (MI-STEP) for HCPs. This program was designed to facilitate tobacco cessation care in the VHA primary care patient centered medical home, which VHA calls patient aligned care teams (PACTs).²³ The main conclusions of this pilot study have been reported elsewhere.²⁴

The objective of this article is to describe the process evaluation the authors conducted during the MI-STEP study to gain a better understanding of how the implementation of the MI training program could be improved. The authors identified barriers and facilitators from the perspectives of MI champions and PACT practitioners.

Methods

Thirty-four PACT practitioners (physicians, nurse practitioners, registered nurses, licensed practical nurses, and pharmacists) at 2 VA medical centers were randomly assigned to a high- or moderate-intensity MI training program during the summer of 2012. This training was delivered by “MI champions,” who were recruited from PACTs and who attended a 3-day advanced training class on MI. The training included MI skills practice, group case analysis, various role-play exercises, and didactics adapted from the Rx for Change program.²⁵ The curriculum also addressed tobacco cessation counseling using the national tobacco cessation guideline.² Each site’s health behavior coordinator (HBC) also was recruited to be an MI champion. The HBCs are typically psychologists who have received prior training in MI as well as facilitator and clinician coaching. At the VA, HBCs are charged with integrating preventive services into care. The participating sites’ institutional review boards approved all study procedures.

MI-STEP Training Program

All 34 practitioners attended a half-day on-site MI training workshop led by the site’s HBC. This training covered the basics of MI and used interactive learning methods such as role-play (Table 1). The study practitioners also received self-study materials, and throughout the study period had access to the MI champions. Practitioners who were randomized to high-intensity MI training also attended 6 supplemental 1-hour “booster sessions” to enhance specific MI skills. The MI champions led 3 of the 1-hour booster sessions with a standard agenda, including patient cases and MI exercises. During the other 3 booster sessions, participants used patient cases to interact with a standardized patient over the telephone, and the MI champions provided feedback and coaching.

Process Evaluation

Six months after the program’s completion, investigators conducted an evaluation of the MI-STEP training program with MI champions and study practitioners. One-hour focus group sessions (2 in Minneapolis; 1 in Denver) were conducted with the MI champions by a co-investigator in Minneapolis and a facilitator in Denver. Notes were taken during the sessions. MI champions were asked about the quality of their training sessions, challenges to getting PACT members to participate in the site training, challenges to teaching MI, and how they felt MI fit within VA health care philosophy.

Ten training study practitioners were randomly selected and stratified based on group intensity assignment, discipline, and site to participate in in-depth interviews. The interviews lasted about 30 minutes, and Minneapolis study investigators conducted in-person interviews with local participants and telephone interviews with Denver participants. The interviews focused on experiences with both high- and moderate-intensity MI training programs, how MI was used in their practice, barriers to implementing MI, impressions of the MI training program, and their interactions with MI champions.

Focus group leaders were experienced interviewers who had not previously interacted with MI champions in the context of this study. Investigators conducting study practitioner interviews were blinded to group assignment. All interviews were audio-recorded and transcribed verbatim. Study investigators reviewed the focus group notes and interview transcripts, identified themes independently, and then discussed group themes. The most salient themes were selected to inform implementation of a larger scale MI training program.

Results

Nine MI champions participated in the focus groups, and 8 study practitioners from both sites representing all clinical disciplines completed in-depth interviews. Table 2 identifies the characteristics of each population.

MI Champion Focus Group Themes

The champions were asked to discuss all aspects of the program, including their training as champions, role as trainers, attitudes about using MI during patient encounters, and participation in the training program. Themes from the MI champion focus groups were placed in the following categories based on the authors’ analytic approach: training MI champions, training study practitioners, and attitudes about MI.

Training MI champions. The champions identified role-play exercises and receiving feedback as strengths of the training program. The champions also expressed the desire for more hands-on practice, especially in small groups. They wanted additional training on teaching MI and facilitating the booster sessions. The champions wanted an expert to train them on how to give feedback and how to best coach practitioners in their use of MI. Champions expressed a desire to have follow-up training sessions with the standardized patient to help them hone their newly acquired coaching skills.

Training study practitioners. The champions’ key role was to train local practitioners and lead the booster sessions for the high-intensity MI training group. Champions felt ill-prepared to fully cover the training materials during the initial half-day workshop and 6 booster sessions. Champions identified difficulty coordinating schedules with the practitioners and lack of compensation for participation as significant barriers to implementing the booster sessions. Champions felt that using a standardized patient during the booster sessions was a strength of the program and that making the cases more realistic could have further enhanced the program.

Attitudes about MI. Champions from both sites perceived MI to have a positive impact on patient care. However, all champions noted there were challenges in using MI in practice. Champions felt MI takes time, energy, and practice to gain proficiency. The current primary care system is not set up to support the use of MI. The appointment time slots are fixed, and VHA goals and the spirit of MI are not always compatible. VHA performance measures encourage providers to achieve performance targets with each patient, often requiring use of directives for patients on what to do. In contrast, MI encourages the patient to take the lead on goal setting and prioritizing.

Study Practitioner Interview Themes

The practitioners were asked to discuss MI skills training, using MI skills with patients, integrating MI into daily practice, getting other PACT members involved, booster sessions, interactions with champions, and suggestions for improving the MI program. Themes from the study practitioner interviews were grouped into the following categories: MI skills training, using MI skills, integrating MI into practice, and suggestions for improving MI training (Table 3).

MI skills training. Overall, the MI high-intensity participants stated they learned useful skills. They reported asking more questions that are open-ended and were more aware of the patient’s perspective. Practitioners reported that booster sessions provided a way to reinforce, refine, and practice their MI skills. Practitioners reported that having the champion located in their own PACT was critical for connecting with their champion between sessions. Nurses and doctors reported that not having time to meet with champions was a barrier, while pharmacists reported more flexibility.

The moderate-intensity participants reported that the training had less impact. Half the respondents reported that they did not remember much of the MI training and either forgot or did not use the newly learned MI skills.

Using MI skills. Both high- and moderate-intensity participants reported using open-ended questions, reflections, affirmations, motivation scales, and active listening.

Practitioners reported that MI helped them focus on patient-centered care, since MI is collaborative. Even when a session was not successful in leading to behavior change, practitioners felt more satisfied with the quality of the interaction.

Integrating MI into practice. The high- and moderate-intensity practitioners had different perceptions of using MI in daily practice. High-intensity participants thought MI required an initial time investment, but that would be balanced by a decrease in the number of follow-up visits needed and/or delay the time between visits. The moderate-intensity participants were more likely to report struggling with the amount of time MI took.

Suggestions for improving MI training. Practitioners from both training groups offered suggestions for improving MI training. Supervisor buy-in was deemed critical to getting other PACT members involved. Practitioners suggested providing compensation or making training mandatory to help motivate others to participate in MI training. Also, practitioners were ready to expand the MI training beyond smoking cessation to incorporate other diseases and multiple comorbidities.

The moderate-intensity participants suggested more training, practice, follow-up, and feedback. These participants also suggested boosterlike sessions.

Discussion

Champions and study practitioners reported that learning MI skills was useful. The participants felt that MI was consistent with their personal philosophies regarding patient-centered care and that MI had a positive impact on patient care. Practitioners and MI champions offered several insights for improving the delivery of MI training. First, practitioners and champions highlighted how important practice and feedback were to learning MI. Booster sessions, standardized patients, and critical feedback enhanced learning.

Second, champions reported that they wanted more training in how to teach MI. Third, practitioners and champions repeatedly stated that finding the time needed to become proficient in MI was difficult and that using the MI approach with patients took additional time during clinical sessions. However, participants in the high-intensity group reported more satisfaction with the quality of their patient encounters and the freedom to follow up with patients less often.

There were aspects of the environment and MI training program that facilitated the MI learning process. The high-intensity group cited booster session feedback as being reinforcing; the moderate-intensity group expressed a desire to practice their newly acquired skill and felt feedback and coaching would have enhanced their learning. Practitioners and champions reported that using a standardized patient to enhance experiential learning activities was an asset. Standardized patients have been used successfully in other training programs.²¹

Implementing an MI training program posed a number of challenges. The biggest barrier was lack of time. PACT members found it difficult to attend a half-day MI workshop, practice MI skills, and incorporate MI routinely into daily practice. However, without the investment of time, even basic MI proficiency is unachievable.²²

This study highlighted several ways to improve feedback and coaching. First, the authors would expand the MI champion curriculum to include training to provide effective feedback/coaching. Second, the authors would train the standardized patient on how to provide feedback to the MI learner. As implemented, the standardized patient evaluated the learner only on whether the patient felt “heard” by the learner.

Perhaps most critical to the success of an MI training program is institutional support. There needs to be adequate time and space for the training process as well as support for ongoing learning and feedback as MI skills are refined. Furthermore, sufficient time is needed during patients’ appointments to allow for MI-oriented conversations. Time is an important, valuable, and scarce resource that institutions control. Administrators should realize that the up-front investment is likely to provide a downstream return as providers become proficient in MI.

There is an urgent need to find ways to incorporate training into the daily practice of busy HCPs. Although this study was limited by its small sample, it demonstrated the feasibility of implementing an MI training program for practitioners working in a busy primary care environment. This study offers concrete suggestions for overcoming barriers and enhancing facilitators, which can guide much needed larger studies as they examine MI training effectiveness on patient and clinician outcomes.

Champions and practitioners reported that learning MI was important, but opportunities to practice and receive critical feedback are needed to achieve proficiency and improve confidence. Both champions and study practitioners thought practicing with a standardized patient would enrich their learning. However, dedicated time for learning and practicing MI skills is critical and hard to arrange.

Conclusion

Practitioners can use MI to activate health behavior change in their patients. Training PACT practitioners to use MI is feasible. The results of this evaluation can be used to inform the next iteration of an MI training program for HCPs by highlighting the facilitators of and barriers to training.

Because of the interest in activating patient-centered health behavior change, these findings are important. The educational and practice opportunities were well received. Training with standardized patients and incorporating MI champions into PACTs facilitated training. However, the lack of time was a major barrier to learning and practicing MI skills and will need to be addressed. If effectively implemented, training providers by using an evidence-based approach, such as MI, can promote long-term health.

Acknowledgments
This study was funded by VA Health Services Research & Development (HSR&D) Rapid Response Project 11-019. The Center for Chronic Disease Outcomes Research is supported by the VA, VHA, Office of Research and Development, and HSR&D. Dr. Widome was supported by a VA HSR&D Career Development Award.

Primary care providers (PCPs) need effective tools for activating health behavior change for the 125 million Americans living with a chronic condition.¹ Smoking is an important and difficult behavior to change, and a motivator for quitting is tobacco cessation advice from a PCP.^2,3 However, few PCPs provide comprehensive tobacco cessation counseling as part of routine care.^4,5 One perceived barrier that providers report is their lack of training to be effective tobacco cessation advocates.^4,6-8

Motivational interviewing (MI) promotes behavior change by using a nonadversarial approach aimed at resolving patient ambivalence. Motivational interviewing tools, such as asking open-ended questions, providing summary statements of what the patient expresses, reflective listening, and affirmations, are used to spur an intrinsic drive to change. These techniques have been applied to a broad range of health behaviors with positive outcomes and demonstrated efficacy.^9-11 Furthermore, MI can be used in primary care for changing tobacco use, alcohol consumption, physical activity, and diet.^12-14

Despite its efficacy, MI can be time-intensive to learn. Fortunately, even abbreviated MI can influence patient behavior.^15,16 Rollnick and others have developed MI interventions that are deliverable in 5 to 10 minutes.^17,18 These brief interventions focus on performing a rapid assessment of patients’ perceived importance and self-efficacy for change.^17,18

There is increased interest in training health care professionals (HCPs) in MI, yet there is no consensus on the most effective training approach.^19,20 Practitioners with many competing priorities often like to learn new skills through self-study or onetime workshops. Yet evidence suggests that these are not effective methods for gaining MI proficiency. Instead, MI training sessions that offer feedback and coaching are more effective in helping participants retain MI skills over time.^21,22

The authors developed and successfully pilot-tested an MI training program called the Motivational Interviewing Smoking Treatment Enhancement Program (MI-STEP) for HCPs. This program was designed to facilitate tobacco cessation care in the VHA primary care patient centered medical home, which VHA calls patient aligned care teams (PACTs).²³ The main conclusions of this pilot study have been reported elsewhere.²⁴

The objective of this article is to describe the process evaluation the authors conducted during the MI-STEP study to gain a better understanding of how the implementation of the MI training program could be improved. The authors identified barriers and facilitators from the perspectives of MI champions and PACT practitioners.

Methods

Thirty-four PACT practitioners (physicians, nurse practitioners, registered nurses, licensed practical nurses, and pharmacists) at 2 VA medical centers were randomly assigned to a high- or moderate-intensity MI training program during the summer of 2012. This training was delivered by “MI champions,” who were recruited from PACTs and who attended a 3-day advanced training class on MI. The training included MI skills practice, group case analysis, various role-play exercises, and didactics adapted from the Rx for Change program.²⁵ The curriculum also addressed tobacco cessation counseling using the national tobacco cessation guideline.² Each site’s health behavior coordinator (HBC) also was recruited to be an MI champion. The HBCs are typically psychologists who have received prior training in MI as well as facilitator and clinician coaching. At the VA, HBCs are charged with integrating preventive services into care. The participating sites’ institutional review boards approved all study procedures.

MI-STEP Training Program

All 34 practitioners attended a half-day on-site MI training workshop led by the site’s HBC. This training covered the basics of MI and used interactive learning methods such as role-play (Table 1). The study practitioners also received self-study materials, and throughout the study period had access to the MI champions. Practitioners who were randomized to high-intensity MI training also attended 6 supplemental 1-hour “booster sessions” to enhance specific MI skills. The MI champions led 3 of the 1-hour booster sessions with a standard agenda, including patient cases and MI exercises. During the other 3 booster sessions, participants used patient cases to interact with a standardized patient over the telephone, and the MI champions provided feedback and coaching.

Process Evaluation

Six months after the program’s completion, investigators conducted an evaluation of the MI-STEP training program with MI champions and study practitioners. One-hour focus group sessions (2 in Minneapolis; 1 in Denver) were conducted with the MI champions by a co-investigator in Minneapolis and a facilitator in Denver. Notes were taken during the sessions. MI champions were asked about the quality of their training sessions, challenges to getting PACT members to participate in the site training, challenges to teaching MI, and how they felt MI fit within VA health care philosophy.

Ten training study practitioners were randomly selected and stratified based on group intensity assignment, discipline, and site to participate in in-depth interviews. The interviews lasted about 30 minutes, and Minneapolis study investigators conducted in-person interviews with local participants and telephone interviews with Denver participants. The interviews focused on experiences with both high- and moderate-intensity MI training programs, how MI was used in their practice, barriers to implementing MI, impressions of the MI training program, and their interactions with MI champions.

Focus group leaders were experienced interviewers who had not previously interacted with MI champions in the context of this study. Investigators conducting study practitioner interviews were blinded to group assignment. All interviews were audio-recorded and transcribed verbatim. Study investigators reviewed the focus group notes and interview transcripts, identified themes independently, and then discussed group themes. The most salient themes were selected to inform implementation of a larger scale MI training program.

Results

Nine MI champions participated in the focus groups, and 8 study practitioners from both sites representing all clinical disciplines completed in-depth interviews. Table 2 identifies the characteristics of each population.

MI Champion Focus Group Themes

The champions were asked to discuss all aspects of the program, including their training as champions, role as trainers, attitudes about using MI during patient encounters, and participation in the training program. Themes from the MI champion focus groups were placed in the following categories based on the authors’ analytic approach: training MI champions, training study practitioners, and attitudes about MI.

Training MI champions. The champions identified role-play exercises and receiving feedback as strengths of the training program. The champions also expressed the desire for more hands-on practice, especially in small groups. They wanted additional training on teaching MI and facilitating the booster sessions. The champions wanted an expert to train them on how to give feedback and how to best coach practitioners in their use of MI. Champions expressed a desire to have follow-up training sessions with the standardized patient to help them hone their newly acquired coaching skills.

Training study practitioners. The champions’ key role was to train local practitioners and lead the booster sessions for the high-intensity MI training group. Champions felt ill-prepared to fully cover the training materials during the initial half-day workshop and 6 booster sessions. Champions identified difficulty coordinating schedules with the practitioners and lack of compensation for participation as significant barriers to implementing the booster sessions. Champions felt that using a standardized patient during the booster sessions was a strength of the program and that making the cases more realistic could have further enhanced the program.

Attitudes about MI. Champions from both sites perceived MI to have a positive impact on patient care. However, all champions noted there were challenges in using MI in practice. Champions felt MI takes time, energy, and practice to gain proficiency. The current primary care system is not set up to support the use of MI. The appointment time slots are fixed, and VHA goals and the spirit of MI are not always compatible. VHA performance measures encourage providers to achieve performance targets with each patient, often requiring use of directives for patients on what to do. In contrast, MI encourages the patient to take the lead on goal setting and prioritizing.

Study Practitioner Interview Themes

The practitioners were asked to discuss MI skills training, using MI skills with patients, integrating MI into daily practice, getting other PACT members involved, booster sessions, interactions with champions, and suggestions for improving the MI program. Themes from the study practitioner interviews were grouped into the following categories: MI skills training, using MI skills, integrating MI into practice, and suggestions for improving MI training (Table 3).

MI skills training. Overall, the MI high-intensity participants stated they learned useful skills. They reported asking more questions that are open-ended and were more aware of the patient’s perspective. Practitioners reported that booster sessions provided a way to reinforce, refine, and practice their MI skills. Practitioners reported that having the champion located in their own PACT was critical for connecting with their champion between sessions. Nurses and doctors reported that not having time to meet with champions was a barrier, while pharmacists reported more flexibility.

The moderate-intensity participants reported that the training had less impact. Half the respondents reported that they did not remember much of the MI training and either forgot or did not use the newly learned MI skills.

Using MI skills. Both high- and moderate-intensity participants reported using open-ended questions, reflections, affirmations, motivation scales, and active listening.

Practitioners reported that MI helped them focus on patient-centered care, since MI is collaborative. Even when a session was not successful in leading to behavior change, practitioners felt more satisfied with the quality of the interaction.

Integrating MI into practice. The high- and moderate-intensity practitioners had different perceptions of using MI in daily practice. High-intensity participants thought MI required an initial time investment, but that would be balanced by a decrease in the number of follow-up visits needed and/or delay the time between visits. The moderate-intensity participants were more likely to report struggling with the amount of time MI took.

Suggestions for improving MI training. Practitioners from both training groups offered suggestions for improving MI training. Supervisor buy-in was deemed critical to getting other PACT members involved. Practitioners suggested providing compensation or making training mandatory to help motivate others to participate in MI training. Also, practitioners were ready to expand the MI training beyond smoking cessation to incorporate other diseases and multiple comorbidities.

The moderate-intensity participants suggested more training, practice, follow-up, and feedback. These participants also suggested boosterlike sessions.

Discussion

Champions and study practitioners reported that learning MI skills was useful. The participants felt that MI was consistent with their personal philosophies regarding patient-centered care and that MI had a positive impact on patient care. Practitioners and MI champions offered several insights for improving the delivery of MI training. First, practitioners and champions highlighted how important practice and feedback were to learning MI. Booster sessions, standardized patients, and critical feedback enhanced learning.

Second, champions reported that they wanted more training in how to teach MI. Third, practitioners and champions repeatedly stated that finding the time needed to become proficient in MI was difficult and that using the MI approach with patients took additional time during clinical sessions. However, participants in the high-intensity group reported more satisfaction with the quality of their patient encounters and the freedom to follow up with patients less often.

There were aspects of the environment and MI training program that facilitated the MI learning process. The high-intensity group cited booster session feedback as being reinforcing; the moderate-intensity group expressed a desire to practice their newly acquired skill and felt feedback and coaching would have enhanced their learning. Practitioners and champions reported that using a standardized patient to enhance experiential learning activities was an asset. Standardized patients have been used successfully in other training programs.²¹

Implementing an MI training program posed a number of challenges. The biggest barrier was lack of time. PACT members found it difficult to attend a half-day MI workshop, practice MI skills, and incorporate MI routinely into daily practice. However, without the investment of time, even basic MI proficiency is unachievable.²²

This study highlighted several ways to improve feedback and coaching. First, the authors would expand the MI champion curriculum to include training to provide effective feedback/coaching. Second, the authors would train the standardized patient on how to provide feedback to the MI learner. As implemented, the standardized patient evaluated the learner only on whether the patient felt “heard” by the learner.

Perhaps most critical to the success of an MI training program is institutional support. There needs to be adequate time and space for the training process as well as support for ongoing learning and feedback as MI skills are refined. Furthermore, sufficient time is needed during patients’ appointments to allow for MI-oriented conversations. Time is an important, valuable, and scarce resource that institutions control. Administrators should realize that the up-front investment is likely to provide a downstream return as providers become proficient in MI.

There is an urgent need to find ways to incorporate training into the daily practice of busy HCPs. Although this study was limited by its small sample, it demonstrated the feasibility of implementing an MI training program for practitioners working in a busy primary care environment. This study offers concrete suggestions for overcoming barriers and enhancing facilitators, which can guide much needed larger studies as they examine MI training effectiveness on patient and clinician outcomes.

Champions and practitioners reported that learning MI was important, but opportunities to practice and receive critical feedback are needed to achieve proficiency and improve confidence. Both champions and study practitioners thought practicing with a standardized patient would enrich their learning. However, dedicated time for learning and practicing MI skills is critical and hard to arrange.

Conclusion

Practitioners can use MI to activate health behavior change in their patients. Training PACT practitioners to use MI is feasible. The results of this evaluation can be used to inform the next iteration of an MI training program for HCPs by highlighting the facilitators of and barriers to training.

Because of the interest in activating patient-centered health behavior change, these findings are important. The educational and practice opportunities were well received. Training with standardized patients and incorporating MI champions into PACTs facilitated training. However, the lack of time was a major barrier to learning and practicing MI skills and will need to be addressed. If effectively implemented, training providers by using an evidence-based approach, such as MI, can promote long-term health.

Acknowledgments
This study was funded by VA Health Services Research & Development (HSR&D) Rapid Response Project 11-019. The Center for Chronic Disease Outcomes Research is supported by the VA, VHA, Office of Research and Development, and HSR&D. Dr. Widome was supported by a VA HSR&D Career Development Award.

Electronic consultations  (e-consults), also called e-referrals, are an alternative method of obtaining general patient information through the electronic health record (EHR) shared by primary care providers (PCPs) and specialists in the VHA. In the e-consult system, test results, medication lists, and other pertinent data are available.¹ Many PCPs are willing to use new technologies to maximize practice efficiency and patient convenience.² In the VHA’s hub-and-spoke model of care, e-consults have the potential to make delivery of specialty care more efficient by prearranging or completing necessary diagnostic testing and redirecting inappropriate referrals to the correct specialists.¹

Some early studies of e-consults report better communication, improved referral appropriateness, and greater access to specialty care as well as better continuity of care and information transfer between patients and PCPs.^3-5 Researchers at the VA Boston Healthcare System in Massachusetts found that 61% of specialists surveyed agreed that e-consults improve quality of care and found the approach beneficial to help initiate diagnostic testing prior to a face-to-face visit.⁶ However, researchers at the Michael E. DeBakey VAMC in Houston, Texas, found no improvement in care coordination.⁷ To date, there have been no large-scale evaluations of e-consult programs or assessments of implementation of e-consult programs.

Related: HHS Grants Fund Health IT in Communities

In early 2011, the VHA Office of Specialty Care Services (OSCS), Office of Specialty Care Transformation launched a national e-consult pilot as part of a broader effort to improve the delivery of patient-centered specialty care. This initiative was based on core concepts advanced by the American College of Physicians, which highlighted the importance of specialty care within a patient- centered medical home and provided a framework for collaboration.^8,9 The goals of the e-consult program were to improve access to specialty care for veterans and their PCPs, to enhance the collaborative relationship between PCPs and specialists, and to augment PCP education.

The OSCS created an Electronic Consultation Implementation Guide to help sites develop and implement each of their e-consult programs.¹⁰ The Implementation Guide established operating rules, strategies for engaging key stakeholders, and recommendations for provider education and training.

As with face-to-face referrals, e-consults are organized in a hub-and-spoke model, where community-based outpatient clinics (CBOCs) are linked to a central VAMC. An e-consult can be accessed by any CBOC, VAMC, medical center-based primary care clinic or specialist, and between medical centers that share the same EHR. There were 217,014 completed e-consults between May 2011 and December 2013 across VHA.¹¹

Some programs created an e-consult template to aid in the transition to electronic referrals (Figure). Although not mandatory, the template helped organize needed information to expedite the e-consult.

The objective of this evaluation is to describe the implementation of e-consults from the perspectives of PCPs, specialists, and other key staff involved in the pilot. Key findings were related to: (1) how the e-consult pilot was implemented; (2) how implementation of the e-consult pilot affected providers; and (3) to what extent the e-consult pilot achieved programmatic objectives from the provider’s perspective.

Methods

The authors conducted a key informant analysis with 2 waves of interviews at 8 e-consult pilot sites across the U.S., selected for variation on early progress in implementation. The sites cannot be identified based on an agreement with the VA Office of Labor-Management Relations.

Setting

The e-consult pilot involved 15 VAMCs in 2 cohorts: alpha sites, which began using e-consults in May 2011, and beta sites, which began using e-consults in July 2011. The alpha sites included 10 VAMCs in 12 medical specialties, with a total of 21 facility-specialty combinations. For the evaluation, sites were defined based on specialty, regardless of location within the same medical center (eg, cardiology and diabetes at the same VAMC would be 2 sites). Beta sites included 5 VAMCs with 6 medical specialties for a total of 6 sites. For 1 year, alpha sites received $175,000 and beta sites received $150,000 to support start-up activities.

Initial specialties included diabetes, hepatitis C, geriatrics, cardiology, liver transplant, dementia, gastrointestinal disease, pulmonary medicine, rheumatology, pain management, neurosurgery, infectious diseases, hematology/oncology, and vascular surgery. Facilities could add additional e-consult specialties but did not receive further funding.

Sample

Study participants were selected from 8 of the 15 pilot sites (geographic site/specialty combinations). Site selection was based on 2 measures of baseline e-consult implementation: (1) overall e-consult implementation rates, measured as the ratio of e-consults to all consults for the specialties of interest; and (2) CBOC participation, measured as the ratio of e-consults for patients from CBOCs vs e-consults for patients from primary care clinics located within the 152 VAMCs. Participation with CBOCs was important for ensuring that implementation factors that influenced uptake of e-consults within tertiary medical centers and between VAMCs and CBOCs could be identified. Two e-consult sites were randomly selected from each of the 4 resulting categories (VAMC high volume, VAMC low volume, CBOC high volume, and CBOC low volume). Volume data of e-consults were obtained from the VA Corporate Data Warehouse and assessed from the beginning of the pilot period to initial site selection, May 2011 to February 2012.

Respondents were identified using a modified snowball sampling process. Snowball sampling is a qualitative sampling technique that identifies study participants, who then identify other potential participants to participate in the study. The researchers started with the local e-consult initiative lead and then contacted the directors of primary care and specialty care services for help identifying PCPs, specialists, and support staff (nurse practitioners, pharmacists, program managers, informatics staff, and medical support personnel) engaged in the initiative. The goal for follow-up interviews was to interview at least 2 of the following respondents at each site: e-consult project manager, PCP, and/or specialist. Due to turnover and changes in clinic roles, some follow-up interviews were conducted with different individuals from the baseline interviews.

Data Collection

Interviews followed semistructured interview guidelines and included open-ended questions designed to elicit rich responses to a variety of aspects related to e-consult implementation, including patient needs, communication, leadership, resources, priorities, knowledge about the program, and unintended consequences. Follow-up interviews addressed how e-consults impacted the quality of specialty care; the impact of e-consults on Patient Aligned Care Teams (PACTs), the VHA patient-centered medical home initiative for primary care; and how e-consults have been used, eg, whether patients were involved in the decision to seek an e-consult.

Two interviewers who had participated in a 1-day, in-person training covering both data collection and analyzing key informant data conducted the 40 to 60 minute telephone interviews. One team member conducted the interview while the other took field notes. Interviews were also recorded. Follow-up probes were used to elicit specific examples and ensure sufficiently rich data. Following each interview, the notetaker reviewed the audio recording and filled in details in the field notes. The interview team debriefed and reviewed the augmented field notes and audio recordings, which became the primary data sources for the study.

Analysis

This was a qualitative descriptive analysis.¹² Interview data were analyzed using an iterative, inductive content analysis method using an open coding approach (ie, a priori codes were not defined for this portion of the analysis).¹³ Two members of the research team used audio recordings and summary transcripts simultaneously to code data. Summary transcripts were compared with the recorded interviews to assure fidelity.

The researchers used Atlas.ti (Berlin, Germany) qualitative data analysis software to organize the coding process. Emergent codes were iteratively added throughout the analysis to reflect quotations that did not adequately fit previously developed codes. Codes were combined weekly to biweekly. After the combinations were completed, the analytics team met to review meanings of codes to ensure consistency of coding and interpretations.

To create categories, broad themes were identified from interview responses and grouped under high- order headings that described distinct aspects of participant experience. The analysis was intentionally kept close to the original data to reflect and describe the participant’s experience as accurately as possible. In support of analytical rigor, members of the multidisciplinary research team, composed of clinicians, implementation scientists, and mixed methodologists, reviewed findings to assess their thoroughness, comprehensiveness, and representativeness across roles and participating sites.¹⁴

Results

The e-consult evaluation period was from November 1, 2011, to July 31, 2013. Key conclusions were drawn from both alpha and beta sites (Table). Baseline interviews were conducted with 37 participants at 8 sites from April 10, 2012, to August 6, 2012. Follow-up interviews were conducted with 21 of the 37 participants at the 8 sites. Follow-up interviews with either a PCP or specialist could not be scheduled at 1 site. Follow-up interviews were conducted from April 16, 2013, to June 18, 2013. Open coding continued until saturation (the point at which subsequent data failed to produce new findings).¹⁵ This occurred after analysis of 22 baseline interviews (12 PCPs, 6 specialists, 1 pharmacist, and 4 other staff members) and 17 follow-up interviews (10 PCPs, 4 specialists, 1 pharmacist, and 2 other staff members).

Implementation


The e-consults provided a programmatic structure to the more informal practice of obtaining diagnostic or therapeutic advice from a specialist. Several of the specialists interviewed described having previously used existing informal consult processes that were “like e-consult.” These specialists reported that their practice patterns did not change significantly since implementing e-consults, because they have been “using the Computerized Patient Record System (CPRS) in an e-consult way for many years.” In these cases, the primary change resulting from the initiative was that national VHA workload policy was revised so that e-consults were assigned a CPT (Current Procedural Terminology) code and specialists began receiving workload credit for completing e-consults.

At sites where an informal e-consult practice was already in place, the initiative was consistently described as flexible. Many specialists reported that this degree of flexibility allowed them to make a relatively easy transition to e-consults by adopting new mechanisms to support existing processes. The e-consult initiative also allowed specialists to formally document this work and to increase the efficiency of specialty care.

Specialists drove the implementation process across sites. The e-consults were envisioned as a collaborative process; however, during initial interviews, few specialists mentioned PCPs when describing the development and implementation of the e-consult program. Primary care providers also reported having little awareness of or input into how the initiative was implemented, although this had little consequence on the use of e-consults.

In a rare case, a PCP reported that poorly designed, lengthy e-consult templates were a major barrier to using e-consults for specific specialties. The PCP said, “E-consults have created an elaborate but extraordinarily cumbersome tool that is difficult for PCPs to actually accomplish, because you have a consult menu that requires a lot of data to be entered—a lot of history from the chart, a lot of exam findings, a lot of previous cognitive testing scores; neurologic findings—lab and imaging tests.”

Still, many other PCPs described receiving detailed information and guidance from e-consults. “E-consults help me to be more accurate. Many providers don’t have a comfort with pain management. To get guidance and education and to really hold our hand, this is how to do this…this has been a big change. If they give you a great response, then [for] the next patient [with that condition], you go back to that note and then follow what was said there,” said one PCP.

In follow-up interviews, providers and other key staff stated there were more data available on the patient as a result of the e-consult and, consequently, even when specialists determined that a patient needed an in-person visit, the data obtained in the e-consult improved the quality of the in-person consultation.

Enhanced Communication and Collaboration

Neither the PCPs nor the specialists were aware of the collaborative intent of the initiative. They focused, instead, on other key aims, such as increasing accessibility and minimizing unnecessary patient travel. Most participants were generally positive about e-consults during baseline interviews, and this perception increased over time.

Both the PCPs and the specialists reported improved communication following the launch of e-consults. In follow-up interviews, some PCPs reported that before e-consults, they had trouble getting timely responses from specialists unless they knew them personally. “You had to know the person in the old days,” one respondent said. “After e-consults, responses improved…e-consult is available to have the resources to tap that knowledge base, and the team is answering the question. I think it opens up access and information and knowledge to everybody.”

Many PCPs spoke positively about this new communication tool as an opportunity to learn from specialists and said they valued the input they received. They felt the increased interaction between the 2 groups positively benefited patient care. One example cited that collaborative communication improved care coordination for veterans: “We are able to step in with e-consults to coordinate services, and this has been huge in improving care.”

Furthermore, follow-up interviews found that all participating PCPs and specialists were communicating more frequently and effectively. “Services that have embraced e-consult give a lot of great information flowing back; it’s closer to a real-time conversation,” said one respondent. 


Related: Home-Based Video Telehealth for Veterans With Dementia

In baseline interviews, some specialists described how e-consults went against their belief that patient care is synonymous with face-to-face medical treatment and voiced dissatisfaction with e-consults as “sitting in front of a computer” rather than “seeing patients.” Others were concerned that medical center administration would not recognize the time it takes to conduct an e-consult and therefore not add necessary specialists staff. “E-consults take work and time, just like seeing a patient. I worry that won’t be seen,” one specialist said.

In order to successfully implement the e-consult initiative, providers and staff needed to incorporate new processes into their daily workflow.

Most sites did not develop a mechanism in which specialists received feedback regarding the outcome of their consultations. This lack of response created anxiety for some specialists in the absence of the face-to-face encounter, leaving some wondering whether they or the PCP had missed anything. According to one specialist, “That’s always in the back of your head: ‘Have I [the specialist] missed something?’”

In follow-up interviews, none of these concerns were raised. Primary care providers tended to speak of the care provided by specialists through e-consults in very positive terms, except in those instances where PCPs felt the e-consult template was difficult to use and required too much time to complete. “I was worried in the beginning about patients thinking less of me, but we ask for help all the time. We’re asking for help and not inconveniencing the patient; they seem to like it very much,” one PCP said.

The e-consults also complement PACTs. Initially, a few participants described soliciting patient input regarding the choice to have an e-consult or a face-to-face visit. During follow-up interviews, participants highlighted how well e-consults fit in to the PACT philosophy. One participant said, “The PACT team seeks to improve quality of care. E-consult fits very well with this, because answers to questions can come quickly, and the veteran may not need to come back to the clinic to be seen, even though things are still getting accomplished. E-consult works very well. E-consults were credited with improving access to specialty care as a tool for PACT.”

Achieving Program Objectives


Based on interviews, support for the e-consult program has increased over time as providers have gained experience with the program and have seen its benefits. Respondents at all sites consistently supported the concept of e-consults and expressed their belief in the importance and value of e-consults in improving patient-c entered care, primarily by reducing the need for patients to travel to see specialists, reducing the time to obtain feedback from specialists, and maintaining the provision of high quality care.

“Last year we only had 2 clinics categorized as e-consults. As of now we have 14 e-consults available for our providers. I think the numbers are growing. They are realizing the value of e-consults as far as the provider’s needs being met,” said one respondent.

The e-consults were credited with improving access to specialty care for veterans. Several participants stated that e-consults improved access to specialty care services and decreased travel for veterans. “It’s another way of getting care to the patient when the patient needs it without having to wait,” said one respondent.

Many PCPs described how difficult it was for patients to get to specialty appointments—particularly for their elderly, disabled, and rural patients—before the implementation of e-consults. “I like the fact that patients who live very far don’t have to come back. A lot of our patients are older…diabetic, see me Monday and back on Thursday. Now, they are able to stay home and follow the recommendations I write,” said one PCP.

Most providers were of the opinion that patients liked the program. “I think e-consults are helping patients...It’s been very successful regarding decreasing travel…Quicker response time for specialty care,” said a PCP. Several providers also stated in follow-up interviews that there was a greater degree of patient participation in the e-consult process and that “patients are definitely informed.”

Discussion

Most PCPs reported that the e-consults were an effective means of consultation and contained the information they needed to provide high-quality coordinated care. Most also found e-consult templates easy to complete. A majority of PCPs felt sufficient control over the choice of whether to use e-consults or an in-person visit, and a minority of patients were involved in the decision to receive an e-consult. Although the OSCS outlined guiding principles and operational rules in the Implementation Guide to help sites implement the e-consult program, its contribution was limited. Few examples were found that engaged PCPs in development of the e-consult program locally; involving patients in the decision to obtain a specialty consult electronically or in person; and PCPs feeding back results to specialists.

Implementing e-consults posed a number of challenges, including lack of resources to respond to referral requests, lack of referral policies and standardized procedures, and confusion related to roles and responsibilities. This is consistent with findings from another VHA research project of e-consults in 2 VHA health systems that was conducted prior to this national level e-consult pilot.⁷

Related: Using Facilitative Coaching to Support Patient Aligned Care Teams

Communication by OSCS of key aspects of the e-consult initiative will be critical as more sites implement e-consults. Since initiation of this pilot, workload specifications and credit have changed from 1 code to 3 codes, to more accurately reflect the amount of time a specialist consultant spends reviewing the EHR and responding to the consult. Without seeing the patient directly, specialists are more reliant on the PCP to describe the problem and provide adequate information in the e-consult request in order to provide recommendations back to the PCP.

Primary care physicians need to know that e-consults are available and determine when they are appropriate. A template or other guidance may be helpful to ensure adequate information is provided in the e-consult request; and the information provided by the specialist in response to the e-consult has to be sufficient for the PCP to provide care. VHA continues to expand the use of e-consults throughout the system, as this pilot found that the electronic option was often more timely than were face-to-face consultations. The result of this evaluation has informed national implementation of this effort.

Limitations

There are 3 main limitations to this study. First, because there was no practical way to preidentify participants who participated in implementing e-consults, a modified snowball sampling was used. However, this limited the degree to which the group was representative of the pilot participants. Second, the authors reported findings from a real-world initiative, not an experimental study. As such, not all participants in the first wave of key informant interviews were available for follow-up interview, which may have introduced bias. Third, the VHA is unlike most of the rest of the U.S. health care system in that it is a fully integrated system with salaried PCPs and specialists and an EHR.

Generalizability of the study may be limited, as a modified snowball sampling approach is not entirely random and has potential for community bias, because initial participants influence subsequent sampling. Additionally, though the sample size (n = 37) was sufficient for qualitative, in-depth analysis, it may be too small for confident generalization of findings. However, as health care moves toward an accountable care organization system, the authors’ analysis may provide insights.

Issues include revision of reimbursement policy for e-consults and developing or coordinating informational technology infrastructures to permit e-consults. It is also important to note that this evaluation reports solely on the extent of implementation of e-consults and the effects of e-consult implementation from the perspectives of staff, including specialists and PCPs.

Evaluating the effectiveness of the program in improving access, care coordination, and patient satisfaction was beyond the scope of the study. Further research is needed, because findings on those outcomes are critical for drawing inferences about this study’s implementation results.

Conclusion

The assessment of the e-consult system by providers and staff was based on a perception that e-consults are a valuable tool in providing greater access to quality care. Currently, e-consults have been expanded across VHA in medical and surgical specialties. VHA policymakers have drafted field guidance and a communication plan to support these efforts.

Acknowledgement
This material is based on work supported by the VA Office of Specialty Care Transformation, the office overseeing the e-consult initiative, and the Office of Research and Development Quality Enhancement Research Initiative.

Author disclosures
The authors report no actual or potential conflicts of interest with regard to this article.

Disclaimer
The opinions expressed herein are those of the authors and do not necessarily reflect those of Federal Practitioner, Frontline Medical Communications Inc., the U.S. Government, or any of its agencies. This article may discuss unlabeled or investigational use of certain drugs. Please review the complete prescribing information for specific drugs or drug combinations—including indications, contraindications, warnings, and adverse effects—before administering pharmacologic therapy to patients.

Electronic consultations  (e-consults), also called e-referrals, are an alternative method of obtaining general patient information through the electronic health record (EHR) shared by primary care providers (PCPs) and specialists in the VHA. In the e-consult system, test results, medication lists, and other pertinent data are available.¹ Many PCPs are willing to use new technologies to maximize practice efficiency and patient convenience.² In the VHA’s hub-and-spoke model of care, e-consults have the potential to make delivery of specialty care more efficient by prearranging or completing necessary diagnostic testing and redirecting inappropriate referrals to the correct specialists.¹

Some early studies of e-consults report better communication, improved referral appropriateness, and greater access to specialty care as well as better continuity of care and information transfer between patients and PCPs.^3-5 Researchers at the VA Boston Healthcare System in Massachusetts found that 61% of specialists surveyed agreed that e-consults improve quality of care and found the approach beneficial to help initiate diagnostic testing prior to a face-to-face visit.⁶ However, researchers at the Michael E. DeBakey VAMC in Houston, Texas, found no improvement in care coordination.⁷ To date, there have been no large-scale evaluations of e-consult programs or assessments of implementation of e-consult programs.

Related: HHS Grants Fund Health IT in Communities

In early 2011, the VHA Office of Specialty Care Services (OSCS), Office of Specialty Care Transformation launched a national e-consult pilot as part of a broader effort to improve the delivery of patient-centered specialty care. This initiative was based on core concepts advanced by the American College of Physicians, which highlighted the importance of specialty care within a patient- centered medical home and provided a framework for collaboration.^8,9 The goals of the e-consult program were to improve access to specialty care for veterans and their PCPs, to enhance the collaborative relationship between PCPs and specialists, and to augment PCP education.

The OSCS created an Electronic Consultation Implementation Guide to help sites develop and implement each of their e-consult programs.¹⁰ The Implementation Guide established operating rules, strategies for engaging key stakeholders, and recommendations for provider education and training.

As with face-to-face referrals, e-consults are organized in a hub-and-spoke model, where community-based outpatient clinics (CBOCs) are linked to a central VAMC. An e-consult can be accessed by any CBOC, VAMC, medical center-based primary care clinic or specialist, and between medical centers that share the same EHR. There were 217,014 completed e-consults between May 2011 and December 2013 across VHA.¹¹

Some programs created an e-consult template to aid in the transition to electronic referrals (Figure). Although not mandatory, the template helped organize needed information to expedite the e-consult.

The objective of this evaluation is to describe the implementation of e-consults from the perspectives of PCPs, specialists, and other key staff involved in the pilot. Key findings were related to: (1) how the e-consult pilot was implemented; (2) how implementation of the e-consult pilot affected providers; and (3) to what extent the e-consult pilot achieved programmatic objectives from the provider’s perspective.

Methods

The authors conducted a key informant analysis with 2 waves of interviews at 8 e-consult pilot sites across the U.S., selected for variation on early progress in implementation. The sites cannot be identified based on an agreement with the VA Office of Labor-Management Relations.

Setting

The e-consult pilot involved 15 VAMCs in 2 cohorts: alpha sites, which began using e-consults in May 2011, and beta sites, which began using e-consults in July 2011. The alpha sites included 10 VAMCs in 12 medical specialties, with a total of 21 facility-specialty combinations. For the evaluation, sites were defined based on specialty, regardless of location within the same medical center (eg, cardiology and diabetes at the same VAMC would be 2 sites). Beta sites included 5 VAMCs with 6 medical specialties for a total of 6 sites. For 1 year, alpha sites received $175,000 and beta sites received $150,000 to support start-up activities.

Initial specialties included diabetes, hepatitis C, geriatrics, cardiology, liver transplant, dementia, gastrointestinal disease, pulmonary medicine, rheumatology, pain management, neurosurgery, infectious diseases, hematology/oncology, and vascular surgery. Facilities could add additional e-consult specialties but did not receive further funding.

Sample

Study participants were selected from 8 of the 15 pilot sites (geographic site/specialty combinations). Site selection was based on 2 measures of baseline e-consult implementation: (1) overall e-consult implementation rates, measured as the ratio of e-consults to all consults for the specialties of interest; and (2) CBOC participation, measured as the ratio of e-consults for patients from CBOCs vs e-consults for patients from primary care clinics located within the 152 VAMCs. Participation with CBOCs was important for ensuring that implementation factors that influenced uptake of e-consults within tertiary medical centers and between VAMCs and CBOCs could be identified. Two e-consult sites were randomly selected from each of the 4 resulting categories (VAMC high volume, VAMC low volume, CBOC high volume, and CBOC low volume). Volume data of e-consults were obtained from the VA Corporate Data Warehouse and assessed from the beginning of the pilot period to initial site selection, May 2011 to February 2012.

Respondents were identified using a modified snowball sampling process. Snowball sampling is a qualitative sampling technique that identifies study participants, who then identify other potential participants to participate in the study. The researchers started with the local e-consult initiative lead and then contacted the directors of primary care and specialty care services for help identifying PCPs, specialists, and support staff (nurse practitioners, pharmacists, program managers, informatics staff, and medical support personnel) engaged in the initiative. The goal for follow-up interviews was to interview at least 2 of the following respondents at each site: e-consult project manager, PCP, and/or specialist. Due to turnover and changes in clinic roles, some follow-up interviews were conducted with different individuals from the baseline interviews.

Data Collection

Interviews followed semistructured interview guidelines and included open-ended questions designed to elicit rich responses to a variety of aspects related to e-consult implementation, including patient needs, communication, leadership, resources, priorities, knowledge about the program, and unintended consequences. Follow-up interviews addressed how e-consults impacted the quality of specialty care; the impact of e-consults on Patient Aligned Care Teams (PACTs), the VHA patient-centered medical home initiative for primary care; and how e-consults have been used, eg, whether patients were involved in the decision to seek an e-consult.

Two interviewers who had participated in a 1-day, in-person training covering both data collection and analyzing key informant data conducted the 40 to 60 minute telephone interviews. One team member conducted the interview while the other took field notes. Interviews were also recorded. Follow-up probes were used to elicit specific examples and ensure sufficiently rich data. Following each interview, the notetaker reviewed the audio recording and filled in details in the field notes. The interview team debriefed and reviewed the augmented field notes and audio recordings, which became the primary data sources for the study.

Analysis

This was a qualitative descriptive analysis.¹² Interview data were analyzed using an iterative, inductive content analysis method using an open coding approach (ie, a priori codes were not defined for this portion of the analysis).¹³ Two members of the research team used audio recordings and summary transcripts simultaneously to code data. Summary transcripts were compared with the recorded interviews to assure fidelity.

The researchers used Atlas.ti (Berlin, Germany) qualitative data analysis software to organize the coding process. Emergent codes were iteratively added throughout the analysis to reflect quotations that did not adequately fit previously developed codes. Codes were combined weekly to biweekly. After the combinations were completed, the analytics team met to review meanings of codes to ensure consistency of coding and interpretations.

To create categories, broad themes were identified from interview responses and grouped under high- order headings that described distinct aspects of participant experience. The analysis was intentionally kept close to the original data to reflect and describe the participant’s experience as accurately as possible. In support of analytical rigor, members of the multidisciplinary research team, composed of clinicians, implementation scientists, and mixed methodologists, reviewed findings to assess their thoroughness, comprehensiveness, and representativeness across roles and participating sites.¹⁴

Results

The e-consult evaluation period was from November 1, 2011, to July 31, 2013. Key conclusions were drawn from both alpha and beta sites (Table). Baseline interviews were conducted with 37 participants at 8 sites from April 10, 2012, to August 6, 2012. Follow-up interviews were conducted with 21 of the 37 participants at the 8 sites. Follow-up interviews with either a PCP or specialist could not be scheduled at 1 site. Follow-up interviews were conducted from April 16, 2013, to June 18, 2013. Open coding continued until saturation (the point at which subsequent data failed to produce new findings).¹⁵ This occurred after analysis of 22 baseline interviews (12 PCPs, 6 specialists, 1 pharmacist, and 4 other staff members) and 17 follow-up interviews (10 PCPs, 4 specialists, 1 pharmacist, and 2 other staff members).

Implementation


The e-consults provided a programmatic structure to the more informal practice of obtaining diagnostic or therapeutic advice from a specialist. Several of the specialists interviewed described having previously used existing informal consult processes that were “like e-consult.” These specialists reported that their practice patterns did not change significantly since implementing e-consults, because they have been “using the Computerized Patient Record System (CPRS) in an e-consult way for many years.” In these cases, the primary change resulting from the initiative was that national VHA workload policy was revised so that e-consults were assigned a CPT (Current Procedural Terminology) code and specialists began receiving workload credit for completing e-consults.

At sites where an informal e-consult practice was already in place, the initiative was consistently described as flexible. Many specialists reported that this degree of flexibility allowed them to make a relatively easy transition to e-consults by adopting new mechanisms to support existing processes. The e-consult initiative also allowed specialists to formally document this work and to increase the efficiency of specialty care.

Specialists drove the implementation process across sites. The e-consults were envisioned as a collaborative process; however, during initial interviews, few specialists mentioned PCPs when describing the development and implementation of the e-consult program. Primary care providers also reported having little awareness of or input into how the initiative was implemented, although this had little consequence on the use of e-consults.

In a rare case, a PCP reported that poorly designed, lengthy e-consult templates were a major barrier to using e-consults for specific specialties. The PCP said, “E-consults have created an elaborate but extraordinarily cumbersome tool that is difficult for PCPs to actually accomplish, because you have a consult menu that requires a lot of data to be entered—a lot of history from the chart, a lot of exam findings, a lot of previous cognitive testing scores; neurologic findings—lab and imaging tests.”

Still, many other PCPs described receiving detailed information and guidance from e-consults. “E-consults help me to be more accurate. Many providers don’t have a comfort with pain management. To get guidance and education and to really hold our hand, this is how to do this…this has been a big change. If they give you a great response, then [for] the next patient [with that condition], you go back to that note and then follow what was said there,” said one PCP.

In follow-up interviews, providers and other key staff stated there were more data available on the patient as a result of the e-consult and, consequently, even when specialists determined that a patient needed an in-person visit, the data obtained in the e-consult improved the quality of the in-person consultation.

Enhanced Communication and Collaboration

Neither the PCPs nor the specialists were aware of the collaborative intent of the initiative. They focused, instead, on other key aims, such as increasing accessibility and minimizing unnecessary patient travel. Most participants were generally positive about e-consults during baseline interviews, and this perception increased over time.

Both the PCPs and the specialists reported improved communication following the launch of e-consults. In follow-up interviews, some PCPs reported that before e-consults, they had trouble getting timely responses from specialists unless they knew them personally. “You had to know the person in the old days,” one respondent said. “After e-consults, responses improved…e-consult is available to have the resources to tap that knowledge base, and the team is answering the question. I think it opens up access and information and knowledge to everybody.”

Many PCPs spoke positively about this new communication tool as an opportunity to learn from specialists and said they valued the input they received. They felt the increased interaction between the 2 groups positively benefited patient care. One example cited that collaborative communication improved care coordination for veterans: “We are able to step in with e-consults to coordinate services, and this has been huge in improving care.”

Furthermore, follow-up interviews found that all participating PCPs and specialists were communicating more frequently and effectively. “Services that have embraced e-consult give a lot of great information flowing back; it’s closer to a real-time conversation,” said one respondent. 


Related: Home-Based Video Telehealth for Veterans With Dementia

In baseline interviews, some specialists described how e-consults went against their belief that patient care is synonymous with face-to-face medical treatment and voiced dissatisfaction with e-consults as “sitting in front of a computer” rather than “seeing patients.” Others were concerned that medical center administration would not recognize the time it takes to conduct an e-consult and therefore not add necessary specialists staff. “E-consults take work and time, just like seeing a patient. I worry that won’t be seen,” one specialist said.

In order to successfully implement the e-consult initiative, providers and staff needed to incorporate new processes into their daily workflow.

Most sites did not develop a mechanism in which specialists received feedback regarding the outcome of their consultations. This lack of response created anxiety for some specialists in the absence of the face-to-face encounter, leaving some wondering whether they or the PCP had missed anything. According to one specialist, “That’s always in the back of your head: ‘Have I [the specialist] missed something?’”

In follow-up interviews, none of these concerns were raised. Primary care providers tended to speak of the care provided by specialists through e-consults in very positive terms, except in those instances where PCPs felt the e-consult template was difficult to use and required too much time to complete. “I was worried in the beginning about patients thinking less of me, but we ask for help all the time. We’re asking for help and not inconveniencing the patient; they seem to like it very much,” one PCP said.

The e-consults also complement PACTs. Initially, a few participants described soliciting patient input regarding the choice to have an e-consult or a face-to-face visit. During follow-up interviews, participants highlighted how well e-consults fit in to the PACT philosophy. One participant said, “The PACT team seeks to improve quality of care. E-consult fits very well with this, because answers to questions can come quickly, and the veteran may not need to come back to the clinic to be seen, even though things are still getting accomplished. E-consult works very well. E-consults were credited with improving access to specialty care as a tool for PACT.”

Achieving Program Objectives


Based on interviews, support for the e-consult program has increased over time as providers have gained experience with the program and have seen its benefits. Respondents at all sites consistently supported the concept of e-consults and expressed their belief in the importance and value of e-consults in improving patient-c entered care, primarily by reducing the need for patients to travel to see specialists, reducing the time to obtain feedback from specialists, and maintaining the provision of high quality care.

“Last year we only had 2 clinics categorized as e-consults. As of now we have 14 e-consults available for our providers. I think the numbers are growing. They are realizing the value of e-consults as far as the provider’s needs being met,” said one respondent.

The e-consults were credited with improving access to specialty care for veterans. Several participants stated that e-consults improved access to specialty care services and decreased travel for veterans. “It’s another way of getting care to the patient when the patient needs it without having to wait,” said one respondent.

Many PCPs described how difficult it was for patients to get to specialty appointments—particularly for their elderly, disabled, and rural patients—before the implementation of e-consults. “I like the fact that patients who live very far don’t have to come back. A lot of our patients are older…diabetic, see me Monday and back on Thursday. Now, they are able to stay home and follow the recommendations I write,” said one PCP.

Most providers were of the opinion that patients liked the program. “I think e-consults are helping patients...It’s been very successful regarding decreasing travel…Quicker response time for specialty care,” said a PCP. Several providers also stated in follow-up interviews that there was a greater degree of patient participation in the e-consult process and that “patients are definitely informed.”

Discussion

Most PCPs reported that the e-consults were an effective means of consultation and contained the information they needed to provide high-quality coordinated care. Most also found e-consult templates easy to complete. A majority of PCPs felt sufficient control over the choice of whether to use e-consults or an in-person visit, and a minority of patients were involved in the decision to receive an e-consult. Although the OSCS outlined guiding principles and operational rules in the Implementation Guide to help sites implement the e-consult program, its contribution was limited. Few examples were found that engaged PCPs in development of the e-consult program locally; involving patients in the decision to obtain a specialty consult electronically or in person; and PCPs feeding back results to specialists.

Implementing e-consults posed a number of challenges, including lack of resources to respond to referral requests, lack of referral policies and standardized procedures, and confusion related to roles and responsibilities. This is consistent with findings from another VHA research project of e-consults in 2 VHA health systems that was conducted prior to this national level e-consult pilot.⁷

Related: Using Facilitative Coaching to Support Patient Aligned Care Teams

Communication by OSCS of key aspects of the e-consult initiative will be critical as more sites implement e-consults. Since initiation of this pilot, workload specifications and credit have changed from 1 code to 3 codes, to more accurately reflect the amount of time a specialist consultant spends reviewing the EHR and responding to the consult. Without seeing the patient directly, specialists are more reliant on the PCP to describe the problem and provide adequate information in the e-consult request in order to provide recommendations back to the PCP.

Primary care physicians need to know that e-consults are available and determine when they are appropriate. A template or other guidance may be helpful to ensure adequate information is provided in the e-consult request; and the information provided by the specialist in response to the e-consult has to be sufficient for the PCP to provide care. VHA continues to expand the use of e-consults throughout the system, as this pilot found that the electronic option was often more timely than were face-to-face consultations. The result of this evaluation has informed national implementation of this effort.

Limitations

There are 3 main limitations to this study. First, because there was no practical way to preidentify participants who participated in implementing e-consults, a modified snowball sampling was used. However, this limited the degree to which the group was representative of the pilot participants. Second, the authors reported findings from a real-world initiative, not an experimental study. As such, not all participants in the first wave of key informant interviews were available for follow-up interview, which may have introduced bias. Third, the VHA is unlike most of the rest of the U.S. health care system in that it is a fully integrated system with salaried PCPs and specialists and an EHR.

Generalizability of the study may be limited, as a modified snowball sampling approach is not entirely random and has potential for community bias, because initial participants influence subsequent sampling. Additionally, though the sample size (n = 37) was sufficient for qualitative, in-depth analysis, it may be too small for confident generalization of findings. However, as health care moves toward an accountable care organization system, the authors’ analysis may provide insights.

Issues include revision of reimbursement policy for e-consults and developing or coordinating informational technology infrastructures to permit e-consults. It is also important to note that this evaluation reports solely on the extent of implementation of e-consults and the effects of e-consult implementation from the perspectives of staff, including specialists and PCPs.

Evaluating the effectiveness of the program in improving access, care coordination, and patient satisfaction was beyond the scope of the study. Further research is needed, because findings on those outcomes are critical for drawing inferences about this study’s implementation results.

Conclusion

The assessment of the e-consult system by providers and staff was based on a perception that e-consults are a valuable tool in providing greater access to quality care. Currently, e-consults have been expanded across VHA in medical and surgical specialties. VHA policymakers have drafted field guidance and a communication plan to support these efforts.

Acknowledgement
This material is based on work supported by the VA Office of Specialty Care Transformation, the office overseeing the e-consult initiative, and the Office of Research and Development Quality Enhancement Research Initiative.

Author disclosures
The authors report no actual or potential conflicts of interest with regard to this article.

Disclaimer
The opinions expressed herein are those of the authors and do not necessarily reflect those of Federal Practitioner, Frontline Medical Communications Inc., the U.S. Government, or any of its agencies. This article may discuss unlabeled or investigational use of certain drugs. Please review the complete prescribing information for specific drugs or drug combinations—including indications, contraindications, warnings, and adverse effects—before administering pharmacologic therapy to patients.

Electronic consultations  (e-consults), also called e-referrals, are an alternative method of obtaining general patient information through the electronic health record (EHR) shared by primary care providers (PCPs) and specialists in the VHA. In the e-consult system, test results, medication lists, and other pertinent data are available.¹ Many PCPs are willing to use new technologies to maximize practice efficiency and patient convenience.² In the VHA’s hub-and-spoke model of care, e-consults have the potential to make delivery of specialty care more efficient by prearranging or completing necessary diagnostic testing and redirecting inappropriate referrals to the correct specialists.¹

Some early studies of e-consults report better communication, improved referral appropriateness, and greater access to specialty care as well as better continuity of care and information transfer between patients and PCPs.^3-5 Researchers at the VA Boston Healthcare System in Massachusetts found that 61% of specialists surveyed agreed that e-consults improve quality of care and found the approach beneficial to help initiate diagnostic testing prior to a face-to-face visit.⁶ However, researchers at the Michael E. DeBakey VAMC in Houston, Texas, found no improvement in care coordination.⁷ To date, there have been no large-scale evaluations of e-consult programs or assessments of implementation of e-consult programs.

Related: HHS Grants Fund Health IT in Communities

In early 2011, the VHA Office of Specialty Care Services (OSCS), Office of Specialty Care Transformation launched a national e-consult pilot as part of a broader effort to improve the delivery of patient-centered specialty care. This initiative was based on core concepts advanced by the American College of Physicians, which highlighted the importance of specialty care within a patient- centered medical home and provided a framework for collaboration.^8,9 The goals of the e-consult program were to improve access to specialty care for veterans and their PCPs, to enhance the collaborative relationship between PCPs and specialists, and to augment PCP education.

The OSCS created an Electronic Consultation Implementation Guide to help sites develop and implement each of their e-consult programs.¹⁰ The Implementation Guide established operating rules, strategies for engaging key stakeholders, and recommendations for provider education and training.

As with face-to-face referrals, e-consults are organized in a hub-and-spoke model, where community-based outpatient clinics (CBOCs) are linked to a central VAMC. An e-consult can be accessed by any CBOC, VAMC, medical center-based primary care clinic or specialist, and between medical centers that share the same EHR. There were 217,014 completed e-consults between May 2011 and December 2013 across VHA.¹¹

Some programs created an e-consult template to aid in the transition to electronic referrals (Figure). Although not mandatory, the template helped organize needed information to expedite the e-consult.

The objective of this evaluation is to describe the implementation of e-consults from the perspectives of PCPs, specialists, and other key staff involved in the pilot. Key findings were related to: (1) how the e-consult pilot was implemented; (2) how implementation of the e-consult pilot affected providers; and (3) to what extent the e-consult pilot achieved programmatic objectives from the provider’s perspective.

Methods

The authors conducted a key informant analysis with 2 waves of interviews at 8 e-consult pilot sites across the U.S., selected for variation on early progress in implementation. The sites cannot be identified based on an agreement with the VA Office of Labor-Management Relations.

Setting

The e-consult pilot involved 15 VAMCs in 2 cohorts: alpha sites, which began using e-consults in May 2011, and beta sites, which began using e-consults in July 2011. The alpha sites included 10 VAMCs in 12 medical specialties, with a total of 21 facility-specialty combinations. For the evaluation, sites were defined based on specialty, regardless of location within the same medical center (eg, cardiology and diabetes at the same VAMC would be 2 sites). Beta sites included 5 VAMCs with 6 medical specialties for a total of 6 sites. For 1 year, alpha sites received $175,000 and beta sites received $150,000 to support start-up activities.

Initial specialties included diabetes, hepatitis C, geriatrics, cardiology, liver transplant, dementia, gastrointestinal disease, pulmonary medicine, rheumatology, pain management, neurosurgery, infectious diseases, hematology/oncology, and vascular surgery. Facilities could add additional e-consult specialties but did not receive further funding.

Sample

Study participants were selected from 8 of the 15 pilot sites (geographic site/specialty combinations). Site selection was based on 2 measures of baseline e-consult implementation: (1) overall e-consult implementation rates, measured as the ratio of e-consults to all consults for the specialties of interest; and (2) CBOC participation, measured as the ratio of e-consults for patients from CBOCs vs e-consults for patients from primary care clinics located within the 152 VAMCs. Participation with CBOCs was important for ensuring that implementation factors that influenced uptake of e-consults within tertiary medical centers and between VAMCs and CBOCs could be identified. Two e-consult sites were randomly selected from each of the 4 resulting categories (VAMC high volume, VAMC low volume, CBOC high volume, and CBOC low volume). Volume data of e-consults were obtained from the VA Corporate Data Warehouse and assessed from the beginning of the pilot period to initial site selection, May 2011 to February 2012.

Respondents were identified using a modified snowball sampling process. Snowball sampling is a qualitative sampling technique that identifies study participants, who then identify other potential participants to participate in the study. The researchers started with the local e-consult initiative lead and then contacted the directors of primary care and specialty care services for help identifying PCPs, specialists, and support staff (nurse practitioners, pharmacists, program managers, informatics staff, and medical support personnel) engaged in the initiative. The goal for follow-up interviews was to interview at least 2 of the following respondents at each site: e-consult project manager, PCP, and/or specialist. Due to turnover and changes in clinic roles, some follow-up interviews were conducted with different individuals from the baseline interviews.

Data Collection

Interviews followed semistructured interview guidelines and included open-ended questions designed to elicit rich responses to a variety of aspects related to e-consult implementation, including patient needs, communication, leadership, resources, priorities, knowledge about the program, and unintended consequences. Follow-up interviews addressed how e-consults impacted the quality of specialty care; the impact of e-consults on Patient Aligned Care Teams (PACTs), the VHA patient-centered medical home initiative for primary care; and how e-consults have been used, eg, whether patients were involved in the decision to seek an e-consult.

Two interviewers who had participated in a 1-day, in-person training covering both data collection and analyzing key informant data conducted the 40 to 60 minute telephone interviews. One team member conducted the interview while the other took field notes. Interviews were also recorded. Follow-up probes were used to elicit specific examples and ensure sufficiently rich data. Following each interview, the notetaker reviewed the audio recording and filled in details in the field notes. The interview team debriefed and reviewed the augmented field notes and audio recordings, which became the primary data sources for the study.

Analysis

This was a qualitative descriptive analysis.¹² Interview data were analyzed using an iterative, inductive content analysis method using an open coding approach (ie, a priori codes were not defined for this portion of the analysis).¹³ Two members of the research team used audio recordings and summary transcripts simultaneously to code data. Summary transcripts were compared with the recorded interviews to assure fidelity.

The researchers used Atlas.ti (Berlin, Germany) qualitative data analysis software to organize the coding process. Emergent codes were iteratively added throughout the analysis to reflect quotations that did not adequately fit previously developed codes. Codes were combined weekly to biweekly. After the combinations were completed, the analytics team met to review meanings of codes to ensure consistency of coding and interpretations.

To create categories, broad themes were identified from interview responses and grouped under high- order headings that described distinct aspects of participant experience. The analysis was intentionally kept close to the original data to reflect and describe the participant’s experience as accurately as possible. In support of analytical rigor, members of the multidisciplinary research team, composed of clinicians, implementation scientists, and mixed methodologists, reviewed findings to assess their thoroughness, comprehensiveness, and representativeness across roles and participating sites.¹⁴

Results

The e-consult evaluation period was from November 1, 2011, to July 31, 2013. Key conclusions were drawn from both alpha and beta sites (Table). Baseline interviews were conducted with 37 participants at 8 sites from April 10, 2012, to August 6, 2012. Follow-up interviews were conducted with 21 of the 37 participants at the 8 sites. Follow-up interviews with either a PCP or specialist could not be scheduled at 1 site. Follow-up interviews were conducted from April 16, 2013, to June 18, 2013. Open coding continued until saturation (the point at which subsequent data failed to produce new findings).¹⁵ This occurred after analysis of 22 baseline interviews (12 PCPs, 6 specialists, 1 pharmacist, and 4 other staff members) and 17 follow-up interviews (10 PCPs, 4 specialists, 1 pharmacist, and 2 other staff members).

Implementation


The e-consults provided a programmatic structure to the more informal practice of obtaining diagnostic or therapeutic advice from a specialist. Several of the specialists interviewed described having previously used existing informal consult processes that were “like e-consult.” These specialists reported that their practice patterns did not change significantly since implementing e-consults, because they have been “using the Computerized Patient Record System (CPRS) in an e-consult way for many years.” In these cases, the primary change resulting from the initiative was that national VHA workload policy was revised so that e-consults were assigned a CPT (Current Procedural Terminology) code and specialists began receiving workload credit for completing e-consults.

At sites where an informal e-consult practice was already in place, the initiative was consistently described as flexible. Many specialists reported that this degree of flexibility allowed them to make a relatively easy transition to e-consults by adopting new mechanisms to support existing processes. The e-consult initiative also allowed specialists to formally document this work and to increase the efficiency of specialty care.

Specialists drove the implementation process across sites. The e-consults were envisioned as a collaborative process; however, during initial interviews, few specialists mentioned PCPs when describing the development and implementation of the e-consult program. Primary care providers also reported having little awareness of or input into how the initiative was implemented, although this had little consequence on the use of e-consults.

In a rare case, a PCP reported that poorly designed, lengthy e-consult templates were a major barrier to using e-consults for specific specialties. The PCP said, “E-consults have created an elaborate but extraordinarily cumbersome tool that is difficult for PCPs to actually accomplish, because you have a consult menu that requires a lot of data to be entered—a lot of history from the chart, a lot of exam findings, a lot of previous cognitive testing scores; neurologic findings—lab and imaging tests.”

Still, many other PCPs described receiving detailed information and guidance from e-consults. “E-consults help me to be more accurate. Many providers don’t have a comfort with pain management. To get guidance and education and to really hold our hand, this is how to do this…this has been a big change. If they give you a great response, then [for] the next patient [with that condition], you go back to that note and then follow what was said there,” said one PCP.

In follow-up interviews, providers and other key staff stated there were more data available on the patient as a result of the e-consult and, consequently, even when specialists determined that a patient needed an in-person visit, the data obtained in the e-consult improved the quality of the in-person consultation.

Enhanced Communication and Collaboration

Neither the PCPs nor the specialists were aware of the collaborative intent of the initiative. They focused, instead, on other key aims, such as increasing accessibility and minimizing unnecessary patient travel. Most participants were generally positive about e-consults during baseline interviews, and this perception increased over time.

Both the PCPs and the specialists reported improved communication following the launch of e-consults. In follow-up interviews, some PCPs reported that before e-consults, they had trouble getting timely responses from specialists unless they knew them personally. “You had to know the person in the old days,” one respondent said. “After e-consults, responses improved…e-consult is available to have the resources to tap that knowledge base, and the team is answering the question. I think it opens up access and information and knowledge to everybody.”

Many PCPs spoke positively about this new communication tool as an opportunity to learn from specialists and said they valued the input they received. They felt the increased interaction between the 2 groups positively benefited patient care. One example cited that collaborative communication improved care coordination for veterans: “We are able to step in with e-consults to coordinate services, and this has been huge in improving care.”

Furthermore, follow-up interviews found that all participating PCPs and specialists were communicating more frequently and effectively. “Services that have embraced e-consult give a lot of great information flowing back; it’s closer to a real-time conversation,” said one respondent. 


Related: Home-Based Video Telehealth for Veterans With Dementia

In baseline interviews, some specialists described how e-consults went against their belief that patient care is synonymous with face-to-face medical treatment and voiced dissatisfaction with e-consults as “sitting in front of a computer” rather than “seeing patients.” Others were concerned that medical center administration would not recognize the time it takes to conduct an e-consult and therefore not add necessary specialists staff. “E-consults take work and time, just like seeing a patient. I worry that won’t be seen,” one specialist said.

In order to successfully implement the e-consult initiative, providers and staff needed to incorporate new processes into their daily workflow.

Most sites did not develop a mechanism in which specialists received feedback regarding the outcome of their consultations. This lack of response created anxiety for some specialists in the absence of the face-to-face encounter, leaving some wondering whether they or the PCP had missed anything. According to one specialist, “That’s always in the back of your head: ‘Have I [the specialist] missed something?’”

In follow-up interviews, none of these concerns were raised. Primary care providers tended to speak of the care provided by specialists through e-consults in very positive terms, except in those instances where PCPs felt the e-consult template was difficult to use and required too much time to complete. “I was worried in the beginning about patients thinking less of me, but we ask for help all the time. We’re asking for help and not inconveniencing the patient; they seem to like it very much,” one PCP said.

The e-consults also complement PACTs. Initially, a few participants described soliciting patient input regarding the choice to have an e-consult or a face-to-face visit. During follow-up interviews, participants highlighted how well e-consults fit in to the PACT philosophy. One participant said, “The PACT team seeks to improve quality of care. E-consult fits very well with this, because answers to questions can come quickly, and the veteran may not need to come back to the clinic to be seen, even though things are still getting accomplished. E-consult works very well. E-consults were credited with improving access to specialty care as a tool for PACT.”

Achieving Program Objectives


Based on interviews, support for the e-consult program has increased over time as providers have gained experience with the program and have seen its benefits. Respondents at all sites consistently supported the concept of e-consults and expressed their belief in the importance and value of e-consults in improving patient-c entered care, primarily by reducing the need for patients to travel to see specialists, reducing the time to obtain feedback from specialists, and maintaining the provision of high quality care.

“Last year we only had 2 clinics categorized as e-consults. As of now we have 14 e-consults available for our providers. I think the numbers are growing. They are realizing the value of e-consults as far as the provider’s needs being met,” said one respondent.

The e-consults were credited with improving access to specialty care for veterans. Several participants stated that e-consults improved access to specialty care services and decreased travel for veterans. “It’s another way of getting care to the patient when the patient needs it without having to wait,” said one respondent.

Many PCPs described how difficult it was for patients to get to specialty appointments—particularly for their elderly, disabled, and rural patients—before the implementation of e-consults. “I like the fact that patients who live very far don’t have to come back. A lot of our patients are older…diabetic, see me Monday and back on Thursday. Now, they are able to stay home and follow the recommendations I write,” said one PCP.

Most providers were of the opinion that patients liked the program. “I think e-consults are helping patients...It’s been very successful regarding decreasing travel…Quicker response time for specialty care,” said a PCP. Several providers also stated in follow-up interviews that there was a greater degree of patient participation in the e-consult process and that “patients are definitely informed.”

Discussion

Most PCPs reported that the e-consults were an effective means of consultation and contained the information they needed to provide high-quality coordinated care. Most also found e-consult templates easy to complete. A majority of PCPs felt sufficient control over the choice of whether to use e-consults or an in-person visit, and a minority of patients were involved in the decision to receive an e-consult. Although the OSCS outlined guiding principles and operational rules in the Implementation Guide to help sites implement the e-consult program, its contribution was limited. Few examples were found that engaged PCPs in development of the e-consult program locally; involving patients in the decision to obtain a specialty consult electronically or in person; and PCPs feeding back results to specialists.

Implementing e-consults posed a number of challenges, including lack of resources to respond to referral requests, lack of referral policies and standardized procedures, and confusion related to roles and responsibilities. This is consistent with findings from another VHA research project of e-consults in 2 VHA health systems that was conducted prior to this national level e-consult pilot.⁷

Related: Using Facilitative Coaching to Support Patient Aligned Care Teams

Communication by OSCS of key aspects of the e-consult initiative will be critical as more sites implement e-consults. Since initiation of this pilot, workload specifications and credit have changed from 1 code to 3 codes, to more accurately reflect the amount of time a specialist consultant spends reviewing the EHR and responding to the consult. Without seeing the patient directly, specialists are more reliant on the PCP to describe the problem and provide adequate information in the e-consult request in order to provide recommendations back to the PCP.

Primary care physicians need to know that e-consults are available and determine when they are appropriate. A template or other guidance may be helpful to ensure adequate information is provided in the e-consult request; and the information provided by the specialist in response to the e-consult has to be sufficient for the PCP to provide care. VHA continues to expand the use of e-consults throughout the system, as this pilot found that the electronic option was often more timely than were face-to-face consultations. The result of this evaluation has informed national implementation of this effort.

Limitations

There are 3 main limitations to this study. First, because there was no practical way to preidentify participants who participated in implementing e-consults, a modified snowball sampling was used. However, this limited the degree to which the group was representative of the pilot participants. Second, the authors reported findings from a real-world initiative, not an experimental study. As such, not all participants in the first wave of key informant interviews were available for follow-up interview, which may have introduced bias. Third, the VHA is unlike most of the rest of the U.S. health care system in that it is a fully integrated system with salaried PCPs and specialists and an EHR.

Generalizability of the study may be limited, as a modified snowball sampling approach is not entirely random and has potential for community bias, because initial participants influence subsequent sampling. Additionally, though the sample size (n = 37) was sufficient for qualitative, in-depth analysis, it may be too small for confident generalization of findings. However, as health care moves toward an accountable care organization system, the authors’ analysis may provide insights.

Issues include revision of reimbursement policy for e-consults and developing or coordinating informational technology infrastructures to permit e-consults. It is also important to note that this evaluation reports solely on the extent of implementation of e-consults and the effects of e-consult implementation from the perspectives of staff, including specialists and PCPs.

Evaluating the effectiveness of the program in improving access, care coordination, and patient satisfaction was beyond the scope of the study. Further research is needed, because findings on those outcomes are critical for drawing inferences about this study’s implementation results.

Conclusion

The assessment of the e-consult system by providers and staff was based on a perception that e-consults are a valuable tool in providing greater access to quality care. Currently, e-consults have been expanded across VHA in medical and surgical specialties. VHA policymakers have drafted field guidance and a communication plan to support these efforts.

Acknowledgement
This material is based on work supported by the VA Office of Specialty Care Transformation, the office overseeing the e-consult initiative, and the Office of Research and Development Quality Enhancement Research Initiative.

Author disclosures
The authors report no actual or potential conflicts of interest with regard to this article.

Disclaimer
The opinions expressed herein are those of the authors and do not necessarily reflect those of Federal Practitioner, Frontline Medical Communications Inc., the U.S. Government, or any of its agencies. This article may discuss unlabeled or investigational use of certain drugs. Please review the complete prescribing information for specific drugs or drug combinations—including indications, contraindications, warnings, and adverse effects—before administering pharmacologic therapy to patients.