Health Policy

Growing up I can remember my father telling stories of service members in the medical battalion he commanded in World War II (WWII) who after the war with his encouragement and their GI Bill educational benefits went to school to become doctors, nurses, and dentists. They were among the 2,300,000 veterans who attended US colleges and universities through the Servicemen’s Readjustment Act passed in 1944. The American Legion navigated the bill through the twists and turns of congressional support, and it was one of their leaders who invented the catchy GI Bill shorthand.²

As with most political legislation, there were mixed motives driving passage of the act, and like many policies in America, the primary impetus was economic. While the war was raging overseas, at home the US Department of Labor predicted that by the war’s end, 16 million service members would be jobless. Apprehensive about the prospect of yet another financial depression, in 1943 a White House agency recommended that the federal government fund education and training for the individuals who had served during the war.²

While troops stormed the beaches of Normandy, wartime President Franklin D. Roosevelt (FDR) signed the bill that delivered not only educational and training opportunities for service members and veterans, but also funded home loans and US Department of Veterans Affairs (VA) hospitals. The bill was practical in that it provided not only tuition, but also books, supplies, a living stipend, and counseling for the students. The bill technically expired in 1956, but a series of extensions and expansions has been true to the original intention to offer those who served their nation in the military a better life as citizens.

Articles describing the impact of the GI Bill use terms like life changing and transformative.^3,4 Our contemporary culture makes it difficult to imagine how out of reach a college education was for the generation that fought WWII. Universities were primarily for the rich and connected, the powerful and privileged. Were it not for the upward social mobility the GI Bill propelled, the American dream would not have become a reality for many farmers, small town merchants, and factory workers. The GI Bill though could not by itself ensure equity. The systemic racism endemic in the United States and among the elected representatives who debated the bill resulted in many Black service members especially in the South being denied entrance to institutions of higher learning.⁵ Despite this invidious discrimination, the bill was a profound effort to help many other service members to successfully reintegrate into the society they had preserved and defended.⁴

“With the signing of this bill, a well-rounded program of special veterans’ benefits is nearly completed,” FDR said, capturing its noble intent: “It gives emphatic notice to the men and women in our armed forces that the American people do not intend to let them down.”⁶

Regrettably, we have not kept FDR’s pledge. Now unscrupulous businesses are preying on the aspirations of military personnel and veterans for an education and thwarting their ability to seek gainful employment. For more than a decade, respected news media have reported that for-profit universities were exploiting service members trying to improve their lives through obtaining a college education via the GI Bill.⁷ The sad irony is that what enabled the exploitation to occur was a major expansion of the benefits under the Post-9/11 GI Bill. This version granted educational funding to any individual who had served on active duty for 90 days or more after September 10, 2001.⁸ Federal law prohibits for-profit educational institutions from receiving more than 90% of their total revenue from federal student aid. A loophole in the law enabled these institutions to categorize GI Bill funding as private not government dollars. Bad old American greed drove these for-profit colleges and universities to aggressively recruit veterans who trusted in the good faith of the academic institutions. Once the GI Bill monies were exhausted, veterans had already invested so much time and energy in a degree or certificate, the schools could persuade them to take out student loans with the promise of job placement assistance that never materialized. They took advantage of the veterans’ hopes to fatten their own bottom line in the face of declining enrolments.⁹ Journalists, government, think tank reports, and even a documentary described the tragic stories of service members left unemployed with immense debt and degrees that to many of them were now worthless.¹⁰

After years of reporters exposing the scam and politically thwarted efforts to stop it, Congress and President Biden closed what was known as the 90/10 loophole. This ended the weaponization of education it had promoted. In October 2022, the US Department of Education announced its final rule to prohibit the widespread educational fraud that had betrayed so many veterans and service members, which Secretary Dennis McDonough described as “abuse.”¹¹Some readers may wonder why I have devoted an editorial to a topic that seems somewhat distant from the health care that is the primary domain of Federal Practitioner. It happens that education is in closer proximity to health for our patients than many of us might have realized. A 2018 Military Medicine study found that veterans who took advantage of the educational opportunities of the GI Bill had better health and reduced smoking, among other benefits.¹² This connection between health and education should serve as a source of pride for all of us in federal practice as we are part of organizations that affirm the holistic concept of health that embraces not just medicine but education, housing, and other services essential for comprehensive well-being.

Growing up I can remember my father telling stories of service members in the medical battalion he commanded in World War II (WWII) who after the war with his encouragement and their GI Bill educational benefits went to school to become doctors, nurses, and dentists. They were among the 2,300,000 veterans who attended US colleges and universities through the Servicemen’s Readjustment Act passed in 1944. The American Legion navigated the bill through the twists and turns of congressional support, and it was one of their leaders who invented the catchy GI Bill shorthand.²

As with most political legislation, there were mixed motives driving passage of the act, and like many policies in America, the primary impetus was economic. While the war was raging overseas, at home the US Department of Labor predicted that by the war’s end, 16 million service members would be jobless. Apprehensive about the prospect of yet another financial depression, in 1943 a White House agency recommended that the federal government fund education and training for the individuals who had served during the war.²

While troops stormed the beaches of Normandy, wartime President Franklin D. Roosevelt (FDR) signed the bill that delivered not only educational and training opportunities for service members and veterans, but also funded home loans and US Department of Veterans Affairs (VA) hospitals. The bill was practical in that it provided not only tuition, but also books, supplies, a living stipend, and counseling for the students. The bill technically expired in 1956, but a series of extensions and expansions has been true to the original intention to offer those who served their nation in the military a better life as citizens.

Articles describing the impact of the GI Bill use terms like life changing and transformative.^3,4 Our contemporary culture makes it difficult to imagine how out of reach a college education was for the generation that fought WWII. Universities were primarily for the rich and connected, the powerful and privileged. Were it not for the upward social mobility the GI Bill propelled, the American dream would not have become a reality for many farmers, small town merchants, and factory workers. The GI Bill though could not by itself ensure equity. The systemic racism endemic in the United States and among the elected representatives who debated the bill resulted in many Black service members especially in the South being denied entrance to institutions of higher learning.⁵ Despite this invidious discrimination, the bill was a profound effort to help many other service members to successfully reintegrate into the society they had preserved and defended.⁴

“With the signing of this bill, a well-rounded program of special veterans’ benefits is nearly completed,” FDR said, capturing its noble intent: “It gives emphatic notice to the men and women in our armed forces that the American people do not intend to let them down.”⁶

Regrettably, we have not kept FDR’s pledge. Now unscrupulous businesses are preying on the aspirations of military personnel and veterans for an education and thwarting their ability to seek gainful employment. For more than a decade, respected news media have reported that for-profit universities were exploiting service members trying to improve their lives through obtaining a college education via the GI Bill.⁷ The sad irony is that what enabled the exploitation to occur was a major expansion of the benefits under the Post-9/11 GI Bill. This version granted educational funding to any individual who had served on active duty for 90 days or more after September 10, 2001.⁸ Federal law prohibits for-profit educational institutions from receiving more than 90% of their total revenue from federal student aid. A loophole in the law enabled these institutions to categorize GI Bill funding as private not government dollars. Bad old American greed drove these for-profit colleges and universities to aggressively recruit veterans who trusted in the good faith of the academic institutions. Once the GI Bill monies were exhausted, veterans had already invested so much time and energy in a degree or certificate, the schools could persuade them to take out student loans with the promise of job placement assistance that never materialized. They took advantage of the veterans’ hopes to fatten their own bottom line in the face of declining enrolments.⁹ Journalists, government, think tank reports, and even a documentary described the tragic stories of service members left unemployed with immense debt and degrees that to many of them were now worthless.¹⁰

After years of reporters exposing the scam and politically thwarted efforts to stop it, Congress and President Biden closed what was known as the 90/10 loophole. This ended the weaponization of education it had promoted. In October 2022, the US Department of Education announced its final rule to prohibit the widespread educational fraud that had betrayed so many veterans and service members, which Secretary Dennis McDonough described as “abuse.”¹¹Some readers may wonder why I have devoted an editorial to a topic that seems somewhat distant from the health care that is the primary domain of Federal Practitioner. It happens that education is in closer proximity to health for our patients than many of us might have realized. A 2018 Military Medicine study found that veterans who took advantage of the educational opportunities of the GI Bill had better health and reduced smoking, among other benefits.¹² This connection between health and education should serve as a source of pride for all of us in federal practice as we are part of organizations that affirm the holistic concept of health that embraces not just medicine but education, housing, and other services essential for comprehensive well-being.

Growing up I can remember my father telling stories of service members in the medical battalion he commanded in World War II (WWII) who after the war with his encouragement and their GI Bill educational benefits went to school to become doctors, nurses, and dentists. They were among the 2,300,000 veterans who attended US colleges and universities through the Servicemen’s Readjustment Act passed in 1944. The American Legion navigated the bill through the twists and turns of congressional support, and it was one of their leaders who invented the catchy GI Bill shorthand.²

As with most political legislation, there were mixed motives driving passage of the act, and like many policies in America, the primary impetus was economic. While the war was raging overseas, at home the US Department of Labor predicted that by the war’s end, 16 million service members would be jobless. Apprehensive about the prospect of yet another financial depression, in 1943 a White House agency recommended that the federal government fund education and training for the individuals who had served during the war.²

While troops stormed the beaches of Normandy, wartime President Franklin D. Roosevelt (FDR) signed the bill that delivered not only educational and training opportunities for service members and veterans, but also funded home loans and US Department of Veterans Affairs (VA) hospitals. The bill was practical in that it provided not only tuition, but also books, supplies, a living stipend, and counseling for the students. The bill technically expired in 1956, but a series of extensions and expansions has been true to the original intention to offer those who served their nation in the military a better life as citizens.

Articles describing the impact of the GI Bill use terms like life changing and transformative.^3,4 Our contemporary culture makes it difficult to imagine how out of reach a college education was for the generation that fought WWII. Universities were primarily for the rich and connected, the powerful and privileged. Were it not for the upward social mobility the GI Bill propelled, the American dream would not have become a reality for many farmers, small town merchants, and factory workers. The GI Bill though could not by itself ensure equity. The systemic racism endemic in the United States and among the elected representatives who debated the bill resulted in many Black service members especially in the South being denied entrance to institutions of higher learning.⁵ Despite this invidious discrimination, the bill was a profound effort to help many other service members to successfully reintegrate into the society they had preserved and defended.⁴

“With the signing of this bill, a well-rounded program of special veterans’ benefits is nearly completed,” FDR said, capturing its noble intent: “It gives emphatic notice to the men and women in our armed forces that the American people do not intend to let them down.”⁶

Regrettably, we have not kept FDR’s pledge. Now unscrupulous businesses are preying on the aspirations of military personnel and veterans for an education and thwarting their ability to seek gainful employment. For more than a decade, respected news media have reported that for-profit universities were exploiting service members trying to improve their lives through obtaining a college education via the GI Bill.⁷ The sad irony is that what enabled the exploitation to occur was a major expansion of the benefits under the Post-9/11 GI Bill. This version granted educational funding to any individual who had served on active duty for 90 days or more after September 10, 2001.⁸ Federal law prohibits for-profit educational institutions from receiving more than 90% of their total revenue from federal student aid. A loophole in the law enabled these institutions to categorize GI Bill funding as private not government dollars. Bad old American greed drove these for-profit colleges and universities to aggressively recruit veterans who trusted in the good faith of the academic institutions. Once the GI Bill monies were exhausted, veterans had already invested so much time and energy in a degree or certificate, the schools could persuade them to take out student loans with the promise of job placement assistance that never materialized. They took advantage of the veterans’ hopes to fatten their own bottom line in the face of declining enrolments.⁹ Journalists, government, think tank reports, and even a documentary described the tragic stories of service members left unemployed with immense debt and degrees that to many of them were now worthless.¹⁰

After years of reporters exposing the scam and politically thwarted efforts to stop it, Congress and President Biden closed what was known as the 90/10 loophole. This ended the weaponization of education it had promoted. In October 2022, the US Department of Education announced its final rule to prohibit the widespread educational fraud that had betrayed so many veterans and service members, which Secretary Dennis McDonough described as “abuse.”¹¹Some readers may wonder why I have devoted an editorial to a topic that seems somewhat distant from the health care that is the primary domain of Federal Practitioner. It happens that education is in closer proximity to health for our patients than many of us might have realized. A 2018 Military Medicine study found that veterans who took advantage of the educational opportunities of the GI Bill had better health and reduced smoking, among other benefits.¹² This connection between health and education should serve as a source of pride for all of us in federal practice as we are part of organizations that affirm the holistic concept of health that embraces not just medicine but education, housing, and other services essential for comprehensive well-being.

Veterans who interact with the criminal justice system (ie, justice-involved veterans) have heightened rates of mental health and psychosocial needs, including posttraumatic stress disorder (PTSD), substance use disorder, depression, suicidal ideation and attempt, and homelessness.^1,2 Alongside these criminogenic risk factors, recidivism is common among justice-involved veterans: About 70% of incarcerated veterans disclosed at least one prior incarceration.³

To address the complex interplay of psychosocial factors, mental health concerns, and justice involvement among veterans, veterans treatment courts (VTCs) emerged as an alternative to incarceration.⁴ VTC participation often consists of integrated treatment and rehabilitative services (eg, vocational training, health care), ongoing monitoring for substance use, graduated responses to address treatment adherence, and ongoing communication with the judge and legal counsel.⁴

A primary aim of these courts is to address psychosocial needs believed to underlie criminal behavior, thus reducing risk of recidivism and promoting successful recovery and community integration for eligible veterans. To do so, VTCs collaborate with community-based and/or US Department of Veterans Affairs services, such as the Veterans Justice Outreach program (VJO). VJO specialists identify and refer justice-involved veterans to Veterans Health Administration (VHA) and community care and serve as a liaison between VTC staff and VHA health care professionals (HCPs).⁵

VTC outcome studies highlight the importance of not only diverting veterans to problem-solving courts, but also ensuring their optimal participation. Successful graduates of VTC programs demonstrate significant improvements in mental health symptoms, life satisfaction, and social support, as well as lower rates of law enforcement interactions.^6,7 However, less is known about supporting those veterans who have difficulty engaging in VTCs and either discontinue participation or require lengthier periods of participation to meet court graduation requirements.⁸ One possibility to improve engagement among these veterans is to enhance court practices to best meet their needs.

In addition to delivering treatment, VHA mental health professionals may serve a critical interdisciplinary role by lending expertise to support VTC practices. For example, equipping court professionals with clinical knowledge and skills related to motivation may strengthen the staff’s interactions with participants, enabling them to address barriers as they arise and to facilitate veterans’ treatment adherence. Additionally, responsiveness to the impact of trauma exposure, which is common among this population, may prove important as related symptoms can affect veterans’ engagement, receptivity, and behavior in court settings. Indeed, prior examinations of justice-involved veterans have found trauma exposure rates ranging from 60% to 90% and PTSD rates ranging from 27% to 40%.^1,2 Notably, involvement with the justice system (eg, incarceration) may itself further increase risk of trauma exposure (eg, experiencing a physical or sexual assault in prison) or exacerbate existing PTSD.⁹ Nonetheless, whereas many drug courts and domestic violence courts have been established, problem-solving courts with a specialized focus on trauma exposure remain rare, suggesting a potential gap in court training.

VHA HCPs have the potential to facilitate justice-involved veterans’ successful court and treatment participation by coordinating with VJO specialists to provide training and consultation to the courts. Supporting efforts to effectively and responsively address criminogenic risk (eg, mental health) in VTC settings may in turn reduce the likelihood of recidivism.¹⁰ Given the elevated rates of trauma exposure among justice-involved veterans and the relative lack of trauma-focused VTCs, we developed a trauma-informed training for VTC professionals that centered on related clinical presentations of justice-involved veterans and frequently occurring challenges in the context of court participation.

Program Development

This educational program aimed to (1) provide psychoeducation on trauma exposure, PTSD, and existing evidence-based treatments; (2) present clinical considerations for justice-involved veterans related to trauma exposure and/or PTSD; and (3) introduce skills to facilitate effective communication and trauma-informed care practices among professionals working with veterans in a treatment court.

Prior to piloting the program, we conducted a needs assessment with VTC professionals and identified relevant theoretical constructs and brief interventions for inclusion in the training. Additionally, given the dearth of prior research on mental health education for VTCs, the team consulted with the developers of PTSD 101, a VHA workshop for veterans’ families that promotes psychoeducation, support, and effective communication.¹¹ Doing so informed approaches to delivering education to nonclinical audiences that interact with veterans with histories of trauma exposure. As this was a program development project, it was determined to be exempt from institutional review board review.

Needs Assessment

In the initial stages of development, local VJO specialists identified regional VTCs and facilitated introductions to these courts. Two of the 3 Rocky Mountain region VTCs that were contacted expressed interest in receiving trauma-informed training. Based on preliminary interest, the facilitators conducted a needs assessment with VJO and VTC staff from these 2 courts to capture requests for specific content and past experiences with other mental health trainings.

Guided by the focus group model, the needs assessments took place during three 1-hour meetings with VJO specialists and a 1-hour meeting with VJO specialists, VTC professionals, and community-based clinical partners.¹² Additionally, attending a VTC graduation and court session allowed for observations of court practices and interactions with veterans. A total of 13 professionals (judges, court coordinators, case managers, peer mentors, VJO specialists, and clinicians who specialize in substance use disorder and intimate partner violence) participated in the needs assessments.

The most critical need identified by court professionals was a focus on how to apply knowledge about trauma and PTSD to interactions with justice-involved veterans. This was reportedly absent from prior training sessions the courts had received. Both Rocky Mountain region VTCs expressed a strong interest in and openness to adapting practices based on research and practice recommendations. Additional requests that emerged included a refresher on psychoeducation related to trauma and how to address the personal impact of working with this population (eg, compassion fatigue).

Training Components

Based on the needs identified by VTC professionals and informed by consultation with the developers of PTSD 101,

Fostering court familiarity with cognitive processing therapy and prolonged exposure may bolster veteran engagement in treatment through regular reinforcement of skills and concepts introduced in therapy. This may prove particularly salient given the limited engagement with mental health treatment and elevated dropout rates from PTSD treatment among the general veteran population.^15,16

Exercises and metaphors were used to illustrate concepts in multiple ways. For example, training attendees engaged in a “stop, drop, and roll” thought exercise in which they were asked to brainstorm behavioral reactions to catching on fire. This exercise illustrated the tendency for individuals to revert to common yet unhelpful attempts at problem solving (eg, running due to panic, which would exacerbate the fire), particularly in crisis and without prior education regarding adaptive ways to respond. Attendee-generated examples, such as running, were used to demonstrate the importance of practicing and reinforcing skill development prior to a crisis, to ensure proficiency and optimal response. Additionally, in prompting consideration of one’s response tendencies, this exercise may engender empathy and understanding for veterans.

Skills training. Efforts to promote veteran engagement in court, facilitate motivation and readiness for change, and address barriers that arise (eg, distress associated with court appearances) may support successful and timely graduation. As such, skills training constituted the largest component of the training and drew from observations of court practices and the VTCs’ identified challenges. Consistent with the project’s aims and reported needs of the court, skills that target common presentations following trauma exposure (eg, avoidance, hypervigilance) were prioritized for this pilot training. Strategies included brief interventions from dialectical behavior therapy, acceptance and commitment therapy, and motivational interviewing to strengthen the support provided by staff to veterans and address their needs (Table 2).

Training attendees also participated in exercises to reiterate skills. For example, attendees completed an ambivalence matrix using an audience-identified common behavior that is difficult to change (eg, heavy alcohol use as a coping mechanism for distress).

Attendees engaged in an exercise that involved identifying unhelpful thoughts and behaviors, targets for validation, and veteran strengths from a hypothetical case vignette. This vignette involved a VTC participant who initially engaged effectively but began to demonstrate difficulty appropriately engaging in court and mental health treatment as well as challenging interactions with VTC staff (eg, raised voice during court sessions, not respecting communication boundaries).

Pilot Test

Based on scheduling parameters communicated by court coordinators, the pilot training was designed as a presentation during times reserved for court staffing meetings. To accommodate court preferences due to the COVID-19 pandemic, one 90-minute training was conducted virtually in March 2022, and the other training was conducted in person in April 2022 for 2 hours. The trainings were facilitated by 2 VHA clinical psychologists and included the judge, court coordinator, VJO specialist, peer mentors, case managers, probation/parole officers, and community-based HCPs who partner with the court (eg, social workers, psychologists). About 12 to 15 professionals attended each training session.

Feedback

Feedback was solicited from attendees via an anonymous online survey. Seven participants completed the survey; the response rate of about 20% was consistent with those observed for other surveys of court professionals.²⁰ Many attendees also provided feedback directly to the facilitators. Feedback highlighted that the skills-based components not only were perceived as most helpful but also notably distinguished this training. “What set this training apart from other training events was the practical applications,” one attendee noted. “It was not just information or education, both instructors did an incredible job of explaining exactly how we could apply the knowledge they were sharing. They did this in such a way that it was easy to understand and apply.”

Specific skills were consistently identified as helpful, including managing intense emotions, addressing ambivalence, and approaching sanctions and rewards. Additionally, employing a less formal approach to the training, with relatable overviews of concepts and immediate responsiveness to requests for expansion on a topic, was perceived as a unique benefit: Another attendee appreciated that “It was beneficial to sit around a table with a less formal presentation and be able to ask questions.” This approach seemed particularly well suited for the program’s cross-disciplinary audience. Attendees reported that they valued the relatively limited focus on DSM-5 criteria. Attendees emphasized that education specific to veterans on evidence-based PTSD treatments, psychoeducation, and avoidance was very helpful. Respondents also recommended that the training be lengthened to a daylong workshop to accommodate greater opportunity to practice skills and consultation.

The consultation portion of the training provided insight into additional areas of importance to incorporate into future iterations. Identified needs included appropriate and realistic boundary setting (eg, addressing disruptions in the courtroom), suggestions for improving and expanding homework assigned by the court, and ways to address concerns about PTSD treatment shared by veterans in court (eg, attributing substance use relapses to the intensiveness of trauma-focused treatment vs lack of familiarity with alternate coping skills). Additionally, the VTC professionals’ desire to support mental health professionals’ work with veterans was clearly evident, highlighting the bidirectional value of interdisciplinary collaboration between VHA mental health professionals and VTC professionals.

Discussion

A trauma-informed training was developed and delivered to 2 VTCs in the Rocky Mountain region with the goal of providing relevant psychoeducation and introducing skills to bolster court practices that address veteran needs. Psychoeducational components of the training that were particularly well received and prompted significant participant engagement included discussions and examples of avoidance, levels of validation, language to facilitate motivation and address barriers, mechanisms underlying treatment, and potential functions underlying limited veteran treatment engagement. Distress tolerance, approaches to sanctions and rewards, and use of ambivalence matrices to guide motivation were identified as particularly helpful skills.

The pilot phase of this trauma-informed training provided valuable insights into developing mental health trainings for VTCs. Specifically, VTCs may benefit from the expertise of VHA HCPs and are particularly interested in learning brief skills to improve their practices. The usefulness of such trainings may be bolstered by efforts to form relationships with the court to identify their perceived needs and employing an iterative process that is responsive to feedback both during and after the training. Last, each stage of this project was strengthened by collaboration with VJO specialists, highlighting the importance of future collaboration between VJO and VHA mental health clinics to further support justice-involved veterans. For example, VJO specialists were instrumental in identifying training needs related to veterans’ clinical presentations in court, facilitating introductions to local VTCs, and helping to address barriers to piloting, like scheduling.

Modifications and Future Directions

The insights gained through the process of training design, delivery, and feedback inform future development of this training. Based on the feedback received, subsequent versions of the training may be expanded into a half- or full-day workshop to allow for adequate time for skills training and feedback, as well as consultation. Doing so will enable facilitators to further foster attendees’ familiarity with and confidence in their ability to use these skills. Furthermore, the consultation portion of this training revealed areas that may benefit from greater attention, including how to address challenging interactions in court (eg, addressing gender dynamics between court professionals and participants) and better support veterans who are having difficulty engaging in mental health treatment (eg, courts’ observation of high rates of dropout around the third or fourth session of evidence-based treatment for PTSD). Last, all attendees who responded to the survey expressed interest in a brief resource guide based on the training, emphasizing the need for ready access to key skills and concepts to support the use of strategies learned.

An additional future aim of this project is to conduct a more thorough evaluation of the needs and outcomes related to this trauma-informed training for VTC professionals. With the rapid growth of VTCs nationwide, relatively little examination of court processes and practices has occurred, and there is a lack of research on the development or effectiveness of mental health trainings provided to VTCs.²¹ Therefore, we intend to conduct larger scale qualitative interviews with court personnel and VJO specialists to obtain a clearer understanding of the needs related to skills-based training and gaps in psychoeducation. These comprehensive needs assessments may also capture common comorbidities that were not incorporated into the pilot training (eg, substance use disorders) but may be important training targets for court professionals. This information will be used to inform subsequent expansion and adaptation of the training into a longer workshop. Program evaluation will be conducted via survey-based feedback on perceived usefulness of the workshop and self-report of confidence in and use of strategies to improve court practices. Furthermore, efforts to obtain veteran outcome data, such as treatment engagement and successful participation in VTC, may be pursued.

Limitations

This training development and pilot project provided valuable foundational information regarding a largely unexamined component of treatment courts—the benefit of skills-based trainings to facilitate court practices related to justice-involved veterans. However, it is worth noting that survey responses were limited; thus, the feedback received may not reflect all attendees’ perceptions. Additionally, because both training sessions were conducted solely with 2 courts in the Rocky Mountain area, feedback may be limited to the needs of this geographic region.

Conclusions

A trauma-informed training was developed for VTCs to facilitate relevant understanding of justice-involved veterans’ needs and presentations in court, introduce skills to address challenges that arise (eg, motivation, emotional dysregulation), and provide interdisciplinary support to court professionals. This training was an important step toward fostering strong collaborations between VHA HCPs and community-based veterans courts, and feedback received during development and following implementation highlighted the perceived need for a skills-based approach to such trainings. Further program development and evaluation can strengthen this training and provide a foundation for dissemination to a broader scope of VTCs, with the goal of reducing recidivism risk among justice-involved veterans by promoting effective engagement in problem-solving court.

Veterans who interact with the criminal justice system (ie, justice-involved veterans) have heightened rates of mental health and psychosocial needs, including posttraumatic stress disorder (PTSD), substance use disorder, depression, suicidal ideation and attempt, and homelessness.^1,2 Alongside these criminogenic risk factors, recidivism is common among justice-involved veterans: About 70% of incarcerated veterans disclosed at least one prior incarceration.³

To address the complex interplay of psychosocial factors, mental health concerns, and justice involvement among veterans, veterans treatment courts (VTCs) emerged as an alternative to incarceration.⁴ VTC participation often consists of integrated treatment and rehabilitative services (eg, vocational training, health care), ongoing monitoring for substance use, graduated responses to address treatment adherence, and ongoing communication with the judge and legal counsel.⁴

A primary aim of these courts is to address psychosocial needs believed to underlie criminal behavior, thus reducing risk of recidivism and promoting successful recovery and community integration for eligible veterans. To do so, VTCs collaborate with community-based and/or US Department of Veterans Affairs services, such as the Veterans Justice Outreach program (VJO). VJO specialists identify and refer justice-involved veterans to Veterans Health Administration (VHA) and community care and serve as a liaison between VTC staff and VHA health care professionals (HCPs).⁵

VTC outcome studies highlight the importance of not only diverting veterans to problem-solving courts, but also ensuring their optimal participation. Successful graduates of VTC programs demonstrate significant improvements in mental health symptoms, life satisfaction, and social support, as well as lower rates of law enforcement interactions.^6,7 However, less is known about supporting those veterans who have difficulty engaging in VTCs and either discontinue participation or require lengthier periods of participation to meet court graduation requirements.⁸ One possibility to improve engagement among these veterans is to enhance court practices to best meet their needs.

In addition to delivering treatment, VHA mental health professionals may serve a critical interdisciplinary role by lending expertise to support VTC practices. For example, equipping court professionals with clinical knowledge and skills related to motivation may strengthen the staff’s interactions with participants, enabling them to address barriers as they arise and to facilitate veterans’ treatment adherence. Additionally, responsiveness to the impact of trauma exposure, which is common among this population, may prove important as related symptoms can affect veterans’ engagement, receptivity, and behavior in court settings. Indeed, prior examinations of justice-involved veterans have found trauma exposure rates ranging from 60% to 90% and PTSD rates ranging from 27% to 40%.^1,2 Notably, involvement with the justice system (eg, incarceration) may itself further increase risk of trauma exposure (eg, experiencing a physical or sexual assault in prison) or exacerbate existing PTSD.⁹ Nonetheless, whereas many drug courts and domestic violence courts have been established, problem-solving courts with a specialized focus on trauma exposure remain rare, suggesting a potential gap in court training.

VHA HCPs have the potential to facilitate justice-involved veterans’ successful court and treatment participation by coordinating with VJO specialists to provide training and consultation to the courts. Supporting efforts to effectively and responsively address criminogenic risk (eg, mental health) in VTC settings may in turn reduce the likelihood of recidivism.¹⁰ Given the elevated rates of trauma exposure among justice-involved veterans and the relative lack of trauma-focused VTCs, we developed a trauma-informed training for VTC professionals that centered on related clinical presentations of justice-involved veterans and frequently occurring challenges in the context of court participation.

Program Development

This educational program aimed to (1) provide psychoeducation on trauma exposure, PTSD, and existing evidence-based treatments; (2) present clinical considerations for justice-involved veterans related to trauma exposure and/or PTSD; and (3) introduce skills to facilitate effective communication and trauma-informed care practices among professionals working with veterans in a treatment court.

Prior to piloting the program, we conducted a needs assessment with VTC professionals and identified relevant theoretical constructs and brief interventions for inclusion in the training. Additionally, given the dearth of prior research on mental health education for VTCs, the team consulted with the developers of PTSD 101, a VHA workshop for veterans’ families that promotes psychoeducation, support, and effective communication.¹¹ Doing so informed approaches to delivering education to nonclinical audiences that interact with veterans with histories of trauma exposure. As this was a program development project, it was determined to be exempt from institutional review board review.

Needs Assessment

In the initial stages of development, local VJO specialists identified regional VTCs and facilitated introductions to these courts. Two of the 3 Rocky Mountain region VTCs that were contacted expressed interest in receiving trauma-informed training. Based on preliminary interest, the facilitators conducted a needs assessment with VJO and VTC staff from these 2 courts to capture requests for specific content and past experiences with other mental health trainings.

Guided by the focus group model, the needs assessments took place during three 1-hour meetings with VJO specialists and a 1-hour meeting with VJO specialists, VTC professionals, and community-based clinical partners.¹² Additionally, attending a VTC graduation and court session allowed for observations of court practices and interactions with veterans. A total of 13 professionals (judges, court coordinators, case managers, peer mentors, VJO specialists, and clinicians who specialize in substance use disorder and intimate partner violence) participated in the needs assessments.

The most critical need identified by court professionals was a focus on how to apply knowledge about trauma and PTSD to interactions with justice-involved veterans. This was reportedly absent from prior training sessions the courts had received. Both Rocky Mountain region VTCs expressed a strong interest in and openness to adapting practices based on research and practice recommendations. Additional requests that emerged included a refresher on psychoeducation related to trauma and how to address the personal impact of working with this population (eg, compassion fatigue).

Training Components

Based on the needs identified by VTC professionals and informed by consultation with the developers of PTSD 101,

Fostering court familiarity with cognitive processing therapy and prolonged exposure may bolster veteran engagement in treatment through regular reinforcement of skills and concepts introduced in therapy. This may prove particularly salient given the limited engagement with mental health treatment and elevated dropout rates from PTSD treatment among the general veteran population.^15,16

Exercises and metaphors were used to illustrate concepts in multiple ways. For example, training attendees engaged in a “stop, drop, and roll” thought exercise in which they were asked to brainstorm behavioral reactions to catching on fire. This exercise illustrated the tendency for individuals to revert to common yet unhelpful attempts at problem solving (eg, running due to panic, which would exacerbate the fire), particularly in crisis and without prior education regarding adaptive ways to respond. Attendee-generated examples, such as running, were used to demonstrate the importance of practicing and reinforcing skill development prior to a crisis, to ensure proficiency and optimal response. Additionally, in prompting consideration of one’s response tendencies, this exercise may engender empathy and understanding for veterans.

Skills training. Efforts to promote veteran engagement in court, facilitate motivation and readiness for change, and address barriers that arise (eg, distress associated with court appearances) may support successful and timely graduation. As such, skills training constituted the largest component of the training and drew from observations of court practices and the VTCs’ identified challenges. Consistent with the project’s aims and reported needs of the court, skills that target common presentations following trauma exposure (eg, avoidance, hypervigilance) were prioritized for this pilot training. Strategies included brief interventions from dialectical behavior therapy, acceptance and commitment therapy, and motivational interviewing to strengthen the support provided by staff to veterans and address their needs (Table 2).

Training attendees also participated in exercises to reiterate skills. For example, attendees completed an ambivalence matrix using an audience-identified common behavior that is difficult to change (eg, heavy alcohol use as a coping mechanism for distress).

Attendees engaged in an exercise that involved identifying unhelpful thoughts and behaviors, targets for validation, and veteran strengths from a hypothetical case vignette. This vignette involved a VTC participant who initially engaged effectively but began to demonstrate difficulty appropriately engaging in court and mental health treatment as well as challenging interactions with VTC staff (eg, raised voice during court sessions, not respecting communication boundaries).

Pilot Test

Based on scheduling parameters communicated by court coordinators, the pilot training was designed as a presentation during times reserved for court staffing meetings. To accommodate court preferences due to the COVID-19 pandemic, one 90-minute training was conducted virtually in March 2022, and the other training was conducted in person in April 2022 for 2 hours. The trainings were facilitated by 2 VHA clinical psychologists and included the judge, court coordinator, VJO specialist, peer mentors, case managers, probation/parole officers, and community-based HCPs who partner with the court (eg, social workers, psychologists). About 12 to 15 professionals attended each training session.

Feedback

Feedback was solicited from attendees via an anonymous online survey. Seven participants completed the survey; the response rate of about 20% was consistent with those observed for other surveys of court professionals.²⁰ Many attendees also provided feedback directly to the facilitators. Feedback highlighted that the skills-based components not only were perceived as most helpful but also notably distinguished this training. “What set this training apart from other training events was the practical applications,” one attendee noted. “It was not just information or education, both instructors did an incredible job of explaining exactly how we could apply the knowledge they were sharing. They did this in such a way that it was easy to understand and apply.”

Specific skills were consistently identified as helpful, including managing intense emotions, addressing ambivalence, and approaching sanctions and rewards. Additionally, employing a less formal approach to the training, with relatable overviews of concepts and immediate responsiveness to requests for expansion on a topic, was perceived as a unique benefit: Another attendee appreciated that “It was beneficial to sit around a table with a less formal presentation and be able to ask questions.” This approach seemed particularly well suited for the program’s cross-disciplinary audience. Attendees reported that they valued the relatively limited focus on DSM-5 criteria. Attendees emphasized that education specific to veterans on evidence-based PTSD treatments, psychoeducation, and avoidance was very helpful. Respondents also recommended that the training be lengthened to a daylong workshop to accommodate greater opportunity to practice skills and consultation.

The consultation portion of the training provided insight into additional areas of importance to incorporate into future iterations. Identified needs included appropriate and realistic boundary setting (eg, addressing disruptions in the courtroom), suggestions for improving and expanding homework assigned by the court, and ways to address concerns about PTSD treatment shared by veterans in court (eg, attributing substance use relapses to the intensiveness of trauma-focused treatment vs lack of familiarity with alternate coping skills). Additionally, the VTC professionals’ desire to support mental health professionals’ work with veterans was clearly evident, highlighting the bidirectional value of interdisciplinary collaboration between VHA mental health professionals and VTC professionals.

Discussion

A trauma-informed training was developed and delivered to 2 VTCs in the Rocky Mountain region with the goal of providing relevant psychoeducation and introducing skills to bolster court practices that address veteran needs. Psychoeducational components of the training that were particularly well received and prompted significant participant engagement included discussions and examples of avoidance, levels of validation, language to facilitate motivation and address barriers, mechanisms underlying treatment, and potential functions underlying limited veteran treatment engagement. Distress tolerance, approaches to sanctions and rewards, and use of ambivalence matrices to guide motivation were identified as particularly helpful skills.

The pilot phase of this trauma-informed training provided valuable insights into developing mental health trainings for VTCs. Specifically, VTCs may benefit from the expertise of VHA HCPs and are particularly interested in learning brief skills to improve their practices. The usefulness of such trainings may be bolstered by efforts to form relationships with the court to identify their perceived needs and employing an iterative process that is responsive to feedback both during and after the training. Last, each stage of this project was strengthened by collaboration with VJO specialists, highlighting the importance of future collaboration between VJO and VHA mental health clinics to further support justice-involved veterans. For example, VJO specialists were instrumental in identifying training needs related to veterans’ clinical presentations in court, facilitating introductions to local VTCs, and helping to address barriers to piloting, like scheduling.

Modifications and Future Directions

The insights gained through the process of training design, delivery, and feedback inform future development of this training. Based on the feedback received, subsequent versions of the training may be expanded into a half- or full-day workshop to allow for adequate time for skills training and feedback, as well as consultation. Doing so will enable facilitators to further foster attendees’ familiarity with and confidence in their ability to use these skills. Furthermore, the consultation portion of this training revealed areas that may benefit from greater attention, including how to address challenging interactions in court (eg, addressing gender dynamics between court professionals and participants) and better support veterans who are having difficulty engaging in mental health treatment (eg, courts’ observation of high rates of dropout around the third or fourth session of evidence-based treatment for PTSD). Last, all attendees who responded to the survey expressed interest in a brief resource guide based on the training, emphasizing the need for ready access to key skills and concepts to support the use of strategies learned.

An additional future aim of this project is to conduct a more thorough evaluation of the needs and outcomes related to this trauma-informed training for VTC professionals. With the rapid growth of VTCs nationwide, relatively little examination of court processes and practices has occurred, and there is a lack of research on the development or effectiveness of mental health trainings provided to VTCs.²¹ Therefore, we intend to conduct larger scale qualitative interviews with court personnel and VJO specialists to obtain a clearer understanding of the needs related to skills-based training and gaps in psychoeducation. These comprehensive needs assessments may also capture common comorbidities that were not incorporated into the pilot training (eg, substance use disorders) but may be important training targets for court professionals. This information will be used to inform subsequent expansion and adaptation of the training into a longer workshop. Program evaluation will be conducted via survey-based feedback on perceived usefulness of the workshop and self-report of confidence in and use of strategies to improve court practices. Furthermore, efforts to obtain veteran outcome data, such as treatment engagement and successful participation in VTC, may be pursued.

Limitations

This training development and pilot project provided valuable foundational information regarding a largely unexamined component of treatment courts—the benefit of skills-based trainings to facilitate court practices related to justice-involved veterans. However, it is worth noting that survey responses were limited; thus, the feedback received may not reflect all attendees’ perceptions. Additionally, because both training sessions were conducted solely with 2 courts in the Rocky Mountain area, feedback may be limited to the needs of this geographic region.

Conclusions

A trauma-informed training was developed for VTCs to facilitate relevant understanding of justice-involved veterans’ needs and presentations in court, introduce skills to address challenges that arise (eg, motivation, emotional dysregulation), and provide interdisciplinary support to court professionals. This training was an important step toward fostering strong collaborations between VHA HCPs and community-based veterans courts, and feedback received during development and following implementation highlighted the perceived need for a skills-based approach to such trainings. Further program development and evaluation can strengthen this training and provide a foundation for dissemination to a broader scope of VTCs, with the goal of reducing recidivism risk among justice-involved veterans by promoting effective engagement in problem-solving court.

Veterans who interact with the criminal justice system (ie, justice-involved veterans) have heightened rates of mental health and psychosocial needs, including posttraumatic stress disorder (PTSD), substance use disorder, depression, suicidal ideation and attempt, and homelessness.^1,2 Alongside these criminogenic risk factors, recidivism is common among justice-involved veterans: About 70% of incarcerated veterans disclosed at least one prior incarceration.³

To address the complex interplay of psychosocial factors, mental health concerns, and justice involvement among veterans, veterans treatment courts (VTCs) emerged as an alternative to incarceration.⁴ VTC participation often consists of integrated treatment and rehabilitative services (eg, vocational training, health care), ongoing monitoring for substance use, graduated responses to address treatment adherence, and ongoing communication with the judge and legal counsel.⁴

A primary aim of these courts is to address psychosocial needs believed to underlie criminal behavior, thus reducing risk of recidivism and promoting successful recovery and community integration for eligible veterans. To do so, VTCs collaborate with community-based and/or US Department of Veterans Affairs services, such as the Veterans Justice Outreach program (VJO). VJO specialists identify and refer justice-involved veterans to Veterans Health Administration (VHA) and community care and serve as a liaison between VTC staff and VHA health care professionals (HCPs).⁵

VTC outcome studies highlight the importance of not only diverting veterans to problem-solving courts, but also ensuring their optimal participation. Successful graduates of VTC programs demonstrate significant improvements in mental health symptoms, life satisfaction, and social support, as well as lower rates of law enforcement interactions.^6,7 However, less is known about supporting those veterans who have difficulty engaging in VTCs and either discontinue participation or require lengthier periods of participation to meet court graduation requirements.⁸ One possibility to improve engagement among these veterans is to enhance court practices to best meet their needs.

In addition to delivering treatment, VHA mental health professionals may serve a critical interdisciplinary role by lending expertise to support VTC practices. For example, equipping court professionals with clinical knowledge and skills related to motivation may strengthen the staff’s interactions with participants, enabling them to address barriers as they arise and to facilitate veterans’ treatment adherence. Additionally, responsiveness to the impact of trauma exposure, which is common among this population, may prove important as related symptoms can affect veterans’ engagement, receptivity, and behavior in court settings. Indeed, prior examinations of justice-involved veterans have found trauma exposure rates ranging from 60% to 90% and PTSD rates ranging from 27% to 40%.^1,2 Notably, involvement with the justice system (eg, incarceration) may itself further increase risk of trauma exposure (eg, experiencing a physical or sexual assault in prison) or exacerbate existing PTSD.⁹ Nonetheless, whereas many drug courts and domestic violence courts have been established, problem-solving courts with a specialized focus on trauma exposure remain rare, suggesting a potential gap in court training.

VHA HCPs have the potential to facilitate justice-involved veterans’ successful court and treatment participation by coordinating with VJO specialists to provide training and consultation to the courts. Supporting efforts to effectively and responsively address criminogenic risk (eg, mental health) in VTC settings may in turn reduce the likelihood of recidivism.¹⁰ Given the elevated rates of trauma exposure among justice-involved veterans and the relative lack of trauma-focused VTCs, we developed a trauma-informed training for VTC professionals that centered on related clinical presentations of justice-involved veterans and frequently occurring challenges in the context of court participation.

Program Development

This educational program aimed to (1) provide psychoeducation on trauma exposure, PTSD, and existing evidence-based treatments; (2) present clinical considerations for justice-involved veterans related to trauma exposure and/or PTSD; and (3) introduce skills to facilitate effective communication and trauma-informed care practices among professionals working with veterans in a treatment court.

Prior to piloting the program, we conducted a needs assessment with VTC professionals and identified relevant theoretical constructs and brief interventions for inclusion in the training. Additionally, given the dearth of prior research on mental health education for VTCs, the team consulted with the developers of PTSD 101, a VHA workshop for veterans’ families that promotes psychoeducation, support, and effective communication.¹¹ Doing so informed approaches to delivering education to nonclinical audiences that interact with veterans with histories of trauma exposure. As this was a program development project, it was determined to be exempt from institutional review board review.

Needs Assessment

In the initial stages of development, local VJO specialists identified regional VTCs and facilitated introductions to these courts. Two of the 3 Rocky Mountain region VTCs that were contacted expressed interest in receiving trauma-informed training. Based on preliminary interest, the facilitators conducted a needs assessment with VJO and VTC staff from these 2 courts to capture requests for specific content and past experiences with other mental health trainings.

Guided by the focus group model, the needs assessments took place during three 1-hour meetings with VJO specialists and a 1-hour meeting with VJO specialists, VTC professionals, and community-based clinical partners.¹² Additionally, attending a VTC graduation and court session allowed for observations of court practices and interactions with veterans. A total of 13 professionals (judges, court coordinators, case managers, peer mentors, VJO specialists, and clinicians who specialize in substance use disorder and intimate partner violence) participated in the needs assessments.

The most critical need identified by court professionals was a focus on how to apply knowledge about trauma and PTSD to interactions with justice-involved veterans. This was reportedly absent from prior training sessions the courts had received. Both Rocky Mountain region VTCs expressed a strong interest in and openness to adapting practices based on research and practice recommendations. Additional requests that emerged included a refresher on psychoeducation related to trauma and how to address the personal impact of working with this population (eg, compassion fatigue).

Training Components

Based on the needs identified by VTC professionals and informed by consultation with the developers of PTSD 101,

Fostering court familiarity with cognitive processing therapy and prolonged exposure may bolster veteran engagement in treatment through regular reinforcement of skills and concepts introduced in therapy. This may prove particularly salient given the limited engagement with mental health treatment and elevated dropout rates from PTSD treatment among the general veteran population.^15,16

Exercises and metaphors were used to illustrate concepts in multiple ways. For example, training attendees engaged in a “stop, drop, and roll” thought exercise in which they were asked to brainstorm behavioral reactions to catching on fire. This exercise illustrated the tendency for individuals to revert to common yet unhelpful attempts at problem solving (eg, running due to panic, which would exacerbate the fire), particularly in crisis and without prior education regarding adaptive ways to respond. Attendee-generated examples, such as running, were used to demonstrate the importance of practicing and reinforcing skill development prior to a crisis, to ensure proficiency and optimal response. Additionally, in prompting consideration of one’s response tendencies, this exercise may engender empathy and understanding for veterans.

Skills training. Efforts to promote veteran engagement in court, facilitate motivation and readiness for change, and address barriers that arise (eg, distress associated with court appearances) may support successful and timely graduation. As such, skills training constituted the largest component of the training and drew from observations of court practices and the VTCs’ identified challenges. Consistent with the project’s aims and reported needs of the court, skills that target common presentations following trauma exposure (eg, avoidance, hypervigilance) were prioritized for this pilot training. Strategies included brief interventions from dialectical behavior therapy, acceptance and commitment therapy, and motivational interviewing to strengthen the support provided by staff to veterans and address their needs (Table 2).

Training attendees also participated in exercises to reiterate skills. For example, attendees completed an ambivalence matrix using an audience-identified common behavior that is difficult to change (eg, heavy alcohol use as a coping mechanism for distress).

Attendees engaged in an exercise that involved identifying unhelpful thoughts and behaviors, targets for validation, and veteran strengths from a hypothetical case vignette. This vignette involved a VTC participant who initially engaged effectively but began to demonstrate difficulty appropriately engaging in court and mental health treatment as well as challenging interactions with VTC staff (eg, raised voice during court sessions, not respecting communication boundaries).

Pilot Test

Based on scheduling parameters communicated by court coordinators, the pilot training was designed as a presentation during times reserved for court staffing meetings. To accommodate court preferences due to the COVID-19 pandemic, one 90-minute training was conducted virtually in March 2022, and the other training was conducted in person in April 2022 for 2 hours. The trainings were facilitated by 2 VHA clinical psychologists and included the judge, court coordinator, VJO specialist, peer mentors, case managers, probation/parole officers, and community-based HCPs who partner with the court (eg, social workers, psychologists). About 12 to 15 professionals attended each training session.

Feedback

Feedback was solicited from attendees via an anonymous online survey. Seven participants completed the survey; the response rate of about 20% was consistent with those observed for other surveys of court professionals.²⁰ Many attendees also provided feedback directly to the facilitators. Feedback highlighted that the skills-based components not only were perceived as most helpful but also notably distinguished this training. “What set this training apart from other training events was the practical applications,” one attendee noted. “It was not just information or education, both instructors did an incredible job of explaining exactly how we could apply the knowledge they were sharing. They did this in such a way that it was easy to understand and apply.”

Specific skills were consistently identified as helpful, including managing intense emotions, addressing ambivalence, and approaching sanctions and rewards. Additionally, employing a less formal approach to the training, with relatable overviews of concepts and immediate responsiveness to requests for expansion on a topic, was perceived as a unique benefit: Another attendee appreciated that “It was beneficial to sit around a table with a less formal presentation and be able to ask questions.” This approach seemed particularly well suited for the program’s cross-disciplinary audience. Attendees reported that they valued the relatively limited focus on DSM-5 criteria. Attendees emphasized that education specific to veterans on evidence-based PTSD treatments, psychoeducation, and avoidance was very helpful. Respondents also recommended that the training be lengthened to a daylong workshop to accommodate greater opportunity to practice skills and consultation.

The consultation portion of the training provided insight into additional areas of importance to incorporate into future iterations. Identified needs included appropriate and realistic boundary setting (eg, addressing disruptions in the courtroom), suggestions for improving and expanding homework assigned by the court, and ways to address concerns about PTSD treatment shared by veterans in court (eg, attributing substance use relapses to the intensiveness of trauma-focused treatment vs lack of familiarity with alternate coping skills). Additionally, the VTC professionals’ desire to support mental health professionals’ work with veterans was clearly evident, highlighting the bidirectional value of interdisciplinary collaboration between VHA mental health professionals and VTC professionals.

Discussion

A trauma-informed training was developed and delivered to 2 VTCs in the Rocky Mountain region with the goal of providing relevant psychoeducation and introducing skills to bolster court practices that address veteran needs. Psychoeducational components of the training that were particularly well received and prompted significant participant engagement included discussions and examples of avoidance, levels of validation, language to facilitate motivation and address barriers, mechanisms underlying treatment, and potential functions underlying limited veteran treatment engagement. Distress tolerance, approaches to sanctions and rewards, and use of ambivalence matrices to guide motivation were identified as particularly helpful skills.

The pilot phase of this trauma-informed training provided valuable insights into developing mental health trainings for VTCs. Specifically, VTCs may benefit from the expertise of VHA HCPs and are particularly interested in learning brief skills to improve their practices. The usefulness of such trainings may be bolstered by efforts to form relationships with the court to identify their perceived needs and employing an iterative process that is responsive to feedback both during and after the training. Last, each stage of this project was strengthened by collaboration with VJO specialists, highlighting the importance of future collaboration between VJO and VHA mental health clinics to further support justice-involved veterans. For example, VJO specialists were instrumental in identifying training needs related to veterans’ clinical presentations in court, facilitating introductions to local VTCs, and helping to address barriers to piloting, like scheduling.

Modifications and Future Directions

The insights gained through the process of training design, delivery, and feedback inform future development of this training. Based on the feedback received, subsequent versions of the training may be expanded into a half- or full-day workshop to allow for adequate time for skills training and feedback, as well as consultation. Doing so will enable facilitators to further foster attendees’ familiarity with and confidence in their ability to use these skills. Furthermore, the consultation portion of this training revealed areas that may benefit from greater attention, including how to address challenging interactions in court (eg, addressing gender dynamics between court professionals and participants) and better support veterans who are having difficulty engaging in mental health treatment (eg, courts’ observation of high rates of dropout around the third or fourth session of evidence-based treatment for PTSD). Last, all attendees who responded to the survey expressed interest in a brief resource guide based on the training, emphasizing the need for ready access to key skills and concepts to support the use of strategies learned.

An additional future aim of this project is to conduct a more thorough evaluation of the needs and outcomes related to this trauma-informed training for VTC professionals. With the rapid growth of VTCs nationwide, relatively little examination of court processes and practices has occurred, and there is a lack of research on the development or effectiveness of mental health trainings provided to VTCs.²¹ Therefore, we intend to conduct larger scale qualitative interviews with court personnel and VJO specialists to obtain a clearer understanding of the needs related to skills-based training and gaps in psychoeducation. These comprehensive needs assessments may also capture common comorbidities that were not incorporated into the pilot training (eg, substance use disorders) but may be important training targets for court professionals. This information will be used to inform subsequent expansion and adaptation of the training into a longer workshop. Program evaluation will be conducted via survey-based feedback on perceived usefulness of the workshop and self-report of confidence in and use of strategies to improve court practices. Furthermore, efforts to obtain veteran outcome data, such as treatment engagement and successful participation in VTC, may be pursued.

Limitations

This training development and pilot project provided valuable foundational information regarding a largely unexamined component of treatment courts—the benefit of skills-based trainings to facilitate court practices related to justice-involved veterans. However, it is worth noting that survey responses were limited; thus, the feedback received may not reflect all attendees’ perceptions. Additionally, because both training sessions were conducted solely with 2 courts in the Rocky Mountain area, feedback may be limited to the needs of this geographic region.

Conclusions

A trauma-informed training was developed for VTCs to facilitate relevant understanding of justice-involved veterans’ needs and presentations in court, introduce skills to address challenges that arise (eg, motivation, emotional dysregulation), and provide interdisciplinary support to court professionals. This training was an important step toward fostering strong collaborations between VHA HCPs and community-based veterans courts, and feedback received during development and following implementation highlighted the perceived need for a skills-based approach to such trainings. Further program development and evaluation can strengthen this training and provide a foundation for dissemination to a broader scope of VTCs, with the goal of reducing recidivism risk among justice-involved veterans by promoting effective engagement in problem-solving court.

The two national emergency declarations dealing with the COVID-19 pandemic will end May 11, President Joe Biden said on Jan. 30.

Doing so will have many effects, including the end of free vaccines and health services to fight the pandemic. The public health emergency has been renewed every 90 days since it was declared by the Trump administration in January 2020.

The declaration allowed major changes throughout the health care system to deal with the pandemic, including the free distribution of vaccines, testing, and treatments. In addition, telehealth services were expanded, and Medicaid and the Children’s Health Insurance Program were extended to millions more Americans.

Biden said the COVID-19 national emergency is set to expire March 1 while the declared public health emergency would currently expire on April 11. The president said both will be extended to end May 11.

There were nearly 300,000 newly reported COVID-19 cases in the United States for the week ending Jan. 25, according to CDC data, as well as more than 3,750 deaths.

A version of this article first appeared on WebMD.com.

The two national emergency declarations dealing with the COVID-19 pandemic will end May 11, President Joe Biden said on Jan. 30.

Doing so will have many effects, including the end of free vaccines and health services to fight the pandemic. The public health emergency has been renewed every 90 days since it was declared by the Trump administration in January 2020.

The declaration allowed major changes throughout the health care system to deal with the pandemic, including the free distribution of vaccines, testing, and treatments. In addition, telehealth services were expanded, and Medicaid and the Children’s Health Insurance Program were extended to millions more Americans.

Biden said the COVID-19 national emergency is set to expire March 1 while the declared public health emergency would currently expire on April 11. The president said both will be extended to end May 11.

There were nearly 300,000 newly reported COVID-19 cases in the United States for the week ending Jan. 25, according to CDC data, as well as more than 3,750 deaths.

A version of this article first appeared on WebMD.com.

The two national emergency declarations dealing with the COVID-19 pandemic will end May 11, President Joe Biden said on Jan. 30.

Doing so will have many effects, including the end of free vaccines and health services to fight the pandemic. The public health emergency has been renewed every 90 days since it was declared by the Trump administration in January 2020.

The declaration allowed major changes throughout the health care system to deal with the pandemic, including the free distribution of vaccines, testing, and treatments. In addition, telehealth services were expanded, and Medicaid and the Children’s Health Insurance Program were extended to millions more Americans.

Biden said the COVID-19 national emergency is set to expire March 1 while the declared public health emergency would currently expire on April 11. The president said both will be extended to end May 11.

There were nearly 300,000 newly reported COVID-19 cases in the United States for the week ending Jan. 25, according to CDC data, as well as more than 3,750 deaths.

A version of this article first appeared on WebMD.com.

The U.S. Surgeon General says 13 years old is too young to begin using social media.

Most social media platforms including TikTok, Snapchat, Instagram, and Facebook allow users to create accounts if they say they are at least 13 years old.

“I, personally, based on the data I’ve seen, believe that 13 is too early. ... It’s a time where it’s really important for us to be thoughtful about what’s going into how they think about their own self-worth and their relationships, and the skewed and often distorted environment of social media often does a disservice to many of those children,” U.S. Surgeon General Vivek Murthy, MD, told CNN.

Research has shown that teens are susceptible to cyberbullying and serious mental health impacts from social media usage and online activity during an era when the influence of the Internet has become everywhere for young people.

According to the Pew Research Center, 95% of teens age 13 and up have a smartphone, and 97% of teens say they use the Internet daily. Among 13- and 14-year-olds, 61% say they use TikTok and 51% say they use Snapchat. Older teens ages 15-17 use those social media platforms at higher rates, with 71% saying they use TikTok and 65% using Snapchat.

“If parents can band together and say you know, as a group, we’re not going to allow our kids to use social media until 16 or 17 or 18 or whatever age they choose, that’s a much more effective strategy in making sure your kids don’t get exposed to harm early,” Dr. Murthy said.

A version of this article originally appeared on WebMD.com.

The U.S. Surgeon General says 13 years old is too young to begin using social media.

Most social media platforms including TikTok, Snapchat, Instagram, and Facebook allow users to create accounts if they say they are at least 13 years old.

“I, personally, based on the data I’ve seen, believe that 13 is too early. ... It’s a time where it’s really important for us to be thoughtful about what’s going into how they think about their own self-worth and their relationships, and the skewed and often distorted environment of social media often does a disservice to many of those children,” U.S. Surgeon General Vivek Murthy, MD, told CNN.

Research has shown that teens are susceptible to cyberbullying and serious mental health impacts from social media usage and online activity during an era when the influence of the Internet has become everywhere for young people.

According to the Pew Research Center, 95% of teens age 13 and up have a smartphone, and 97% of teens say they use the Internet daily. Among 13- and 14-year-olds, 61% say they use TikTok and 51% say they use Snapchat. Older teens ages 15-17 use those social media platforms at higher rates, with 71% saying they use TikTok and 65% using Snapchat.

“If parents can band together and say you know, as a group, we’re not going to allow our kids to use social media until 16 or 17 or 18 or whatever age they choose, that’s a much more effective strategy in making sure your kids don’t get exposed to harm early,” Dr. Murthy said.

A version of this article originally appeared on WebMD.com.

The U.S. Surgeon General says 13 years old is too young to begin using social media.

Most social media platforms including TikTok, Snapchat, Instagram, and Facebook allow users to create accounts if they say they are at least 13 years old.

“I, personally, based on the data I’ve seen, believe that 13 is too early. ... It’s a time where it’s really important for us to be thoughtful about what’s going into how they think about their own self-worth and their relationships, and the skewed and often distorted environment of social media often does a disservice to many of those children,” U.S. Surgeon General Vivek Murthy, MD, told CNN.

Research has shown that teens are susceptible to cyberbullying and serious mental health impacts from social media usage and online activity during an era when the influence of the Internet has become everywhere for young people.

According to the Pew Research Center, 95% of teens age 13 and up have a smartphone, and 97% of teens say they use the Internet daily. Among 13- and 14-year-olds, 61% say they use TikTok and 51% say they use Snapchat. Older teens ages 15-17 use those social media platforms at higher rates, with 71% saying they use TikTok and 65% using Snapchat.

“If parents can band together and say you know, as a group, we’re not going to allow our kids to use social media until 16 or 17 or 18 or whatever age they choose, that’s a much more effective strategy in making sure your kids don’t get exposed to harm early,” Dr. Murthy said.

A version of this article originally appeared on WebMD.com.

U.S. health officials want to simplify the recommended COVID-19 vaccine protocol, making it more like the process for annual flu shots.

The U.S. Food and Drug Administration is suggesting a single annual shot. The formulation would be selected in June targeting the most threatening COVID-19 strains, and then people could get a shot in the fall when people begin spending more time indoors and exposure increases. 

Some people, such as those who are older or immunocompromised, may need more than one dose.

A national advisory committee is expected to vote on the proposal at a meeting Jan. 26.

People in the United States have been much less likely to get an updated COVID-19 booster shot, compared with widespread uptake of the primary vaccine series. In its proposal, the FDA indicated it hoped a single annual shot would overcome challenges created by the complexity of the process – both in messaging and administration – attributed to that low booster rate. Nine in 10 people age 12 or older got the primary vaccine series in the United States, but only 15% got the latest booster shot for COVID-19.

About half of children and adults in the U.S. get an annual flu shot, according to Centers for Disease Control and Prevention data.

The FDA also wants to move to a single COVID-19 vaccine formulation that would be used for primary vaccine series and for booster shots.

COVID-19 cases, hospitalizations, and deaths are trending downward, according to the data tracker from the New York Times. Cases are down 28%, with 47,290 tallied daily. Hospitalizations are down 22%, with 37,474 daily. Deaths are down 4%, with an average of 489 per day as of Jan. 22.

A version of this article originally appeared on WebMD.com.

U.S. health officials want to simplify the recommended COVID-19 vaccine protocol, making it more like the process for annual flu shots.

The U.S. Food and Drug Administration is suggesting a single annual shot. The formulation would be selected in June targeting the most threatening COVID-19 strains, and then people could get a shot in the fall when people begin spending more time indoors and exposure increases. 

Some people, such as those who are older or immunocompromised, may need more than one dose.

A national advisory committee is expected to vote on the proposal at a meeting Jan. 26.

People in the United States have been much less likely to get an updated COVID-19 booster shot, compared with widespread uptake of the primary vaccine series. In its proposal, the FDA indicated it hoped a single annual shot would overcome challenges created by the complexity of the process – both in messaging and administration – attributed to that low booster rate. Nine in 10 people age 12 or older got the primary vaccine series in the United States, but only 15% got the latest booster shot for COVID-19.

About half of children and adults in the U.S. get an annual flu shot, according to Centers for Disease Control and Prevention data.

The FDA also wants to move to a single COVID-19 vaccine formulation that would be used for primary vaccine series and for booster shots.

COVID-19 cases, hospitalizations, and deaths are trending downward, according to the data tracker from the New York Times. Cases are down 28%, with 47,290 tallied daily. Hospitalizations are down 22%, with 37,474 daily. Deaths are down 4%, with an average of 489 per day as of Jan. 22.

A version of this article originally appeared on WebMD.com.

U.S. health officials want to simplify the recommended COVID-19 vaccine protocol, making it more like the process for annual flu shots.

The U.S. Food and Drug Administration is suggesting a single annual shot. The formulation would be selected in June targeting the most threatening COVID-19 strains, and then people could get a shot in the fall when people begin spending more time indoors and exposure increases. 

Some people, such as those who are older or immunocompromised, may need more than one dose.

A national advisory committee is expected to vote on the proposal at a meeting Jan. 26.

People in the United States have been much less likely to get an updated COVID-19 booster shot, compared with widespread uptake of the primary vaccine series. In its proposal, the FDA indicated it hoped a single annual shot would overcome challenges created by the complexity of the process – both in messaging and administration – attributed to that low booster rate. Nine in 10 people age 12 or older got the primary vaccine series in the United States, but only 15% got the latest booster shot for COVID-19.

About half of children and adults in the U.S. get an annual flu shot, according to Centers for Disease Control and Prevention data.

The FDA also wants to move to a single COVID-19 vaccine formulation that would be used for primary vaccine series and for booster shots.

COVID-19 cases, hospitalizations, and deaths are trending downward, according to the data tracker from the New York Times. Cases are down 28%, with 47,290 tallied daily. Hospitalizations are down 22%, with 37,474 daily. Deaths are down 4%, with an average of 489 per day as of Jan. 22.

A version of this article originally appeared on WebMD.com.

ABSTRACT

Objective: Promoting a culture of safety is a critical component of improving health care quality. Recognizing staff who stop the line for safety can positively impact the growth of a culture of safety. The purpose of this initiative was to demonstrate to staff the importance of speaking up for safety and being acknowledged for doing so.

Methods: Following a review of the literature on safety awards programs and their role in promoting a culture of safety in health care covering the period 2017 to 2020, a formal process was developed and implemented to disseminate safety awards to employees.

Results: During the initial 18 months of the initiative, a total of 59 awards were presented. The awards were well received by the recipients and other staff members. Within this period, adjustments were made to enhance the scope and reach of the program.

Conclusion: Recognizing staff behaviors that support a culture of safety is important for improving health care quality and employee engagement. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Keywords: patient safety, culture of safety, incident reporting, near miss.

A key aspect of improving health care quality is promoting and sustaining a culture of safety in the workplace. Improving the quality of health care services and systems involves making informed choices regarding the types of strategies to implement.¹ An essential aspect of supporting a safety culture is safety-event reporting. To approach the goal of zero harm, all safety events, whether they result in actual harm or are considered near misses, need to be reported. Near-miss events are errors that occur while care is being provided but are detected and corrected before harm reaches the patient.^1-3 Near-miss reporting plays a critical role in helping to identify and correct weaknesses in health care delivery systems and processes.⁴ However, evidence shows that there are a multitude of barriers to the reporting of near-miss events, such as fear of punitive actions, additional workload, unsupportive work environments, a culture with poor psychological safety, knowledge deficit, and lack of recognition of staff who do report near misses.^4-11

According to The Joint Commission (TJC), acknowledging health care team members who recognize and report unsafe conditions that provide insight for improving patient safety is a key method for promoting the reporting of near-miss events.⁶ As a result, some health care organizations and patient safety agencies have started to institute some form of recognition for their employees in the realm of safety.^8-10 The Pennsylvania Patient Safety Authority offers exceptional guidance for creating a safety awards program to promote a culture of safety.¹² Furthermore, TJC supports recognizing individuals and health care teams who identify and report near misses, or who have suggestions for initiatives to promote patient safety, with “good catch” awards. Individuals or teams working to promote and sustain a culture of safety should be recognized for their efforts. Acknowledging “good catches” to reward the identification, communication, and resolution of safety issues is an effective strategy for improving patient safety and health care quality.^6,8

This quality improvement (QI) initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. If health care organizations want staff to be motivated to report near misses and improve safety and health care quality, the culture needs to shift from focusing on blame to incentivizing individuals and teams to speak up when they have concerns.^8-10 Although deciding which safety actions are worthy of recognition can be challenging, recognizing all safe acts, regardless of how big or small they are perceived to be, is important. This QI initiative aimed to establish a tiered approach to recognize staff members for various categories of safety acts.

METHODS

A review of the literature from January 2017 to May 2020 for peer-reviewed publications regarding how other organizations implemented safety award programs to promote a culture of safety resulted in a dearth of evidence. This prompted us at the Veterans Affairs Connecticut Healthcare System to develop and implement a formal program to disseminate safety awards to employees.

Program Launch and Promotion

In 2020, our institution embarked on a journey to high reliability with the goal of approaching zero harm. As part of efforts to promote a culture of safety, the hospital’s High Reliability Organization (HRO) team worked to develop a safety awards recognition program. Prior to the launch, the hospital’s patient safety committee recognized staff members through the medical center safety event reporting system (the Joint Patient Safety Reporting system [JPSR]) or through direct communication with staff members on safety actions they were engaged in. JPSR is the Veterans Health Administration National Center for Patient Safety incident reporting system for reporting, tracking, and trending of patient incidents in a national database. The award consisted of a certificate presented by the patient safety committee chairpersons to the employee in front of their peers in their respective work area. Hospital leadership was not involved in the safety awards recognition program at that time. No nomination process existed prior to our QI launch.

Once the QI initiative was launched and marketed heavily at staff meetings, we started to receive nominations for actions that were truly exceptional, while many others were submitted for behaviors that were within the day-to-day scope of practice of the staff member. For those early nominations that did not meet criteria for an award, we thanked staff for their submissions with a gentle statement that their nomination did not meet the criteria for an award. After following this practice for a few weeks, we became concerned that if we did not acknowledge the staff who came forward to request recognition for their routine work that supported safety, we could risk losing their engagement in a culture of safety. As such, we decided to create 3 levels of awards to recognize behaviors that went above and beyond while also acknowledging staff for actions within their scope of practice. Additionally, hospital leadership wanted to ensure that all staff recognize that their safety efforts are valued by leadership and that that sense of value will hopefully contribute to a culture of safety over time.

Initially, the single award system was called the “Good Catch Award” to acknowledge staff who go above and beyond to speak up and take action when they have safety concerns. This particular recognition includes a certificate, an encased baseball card that has been personalized by including the staff member’s picture and safety event identified, a stress-release baseball, and a stick of Bazooka gum (similar to what used to come in baseball cards packs). The award is presented to employees in their work area by the HRO and patient safety teams and includes representatives from the executive leadership team (ELT). The safety event identified is described by an ELT member, and all items are presented to the employee. Participation by the leadership team communicates how much the work being done to promote a culture of safety and advance quality health care is appreciated. This action also encourages others in the organization to identify and report safety concerns.¹³

With the rollout of the QI initiative, the volume of nominations submitted quickly increased (eg, approximately 1 every 2 months before to 3 per month following implementation). Frequently, nominations were for actions believed to be within the scope of the employee’s responsibilities. Our institution’s leadership team quickly recognized that, as an organization, not diminishing the importance of the “Good Catch Award” was important. However, the leadership team also wanted to encourage nominations from employees that involved safety issues that were part of the employee’s scope of responsibilities. As a result, 2 additional and equally notable award tiers were established, with specific criteria created for each.¹⁴ The addition of the other awards was instrumental in getting the leadership team to feel confident that all staff were being recognized for their commitment to patient safety.

The original Good Catch Award was labelled as a Level 1 award. The Level 2 safety recognition award, named the HRO Safety Champion Award, is given to employees who stop the line for a safety concern within their scope of practice and also participate as part of a team to investigate and improve processes to avoid recurring safety concerns in the future. For the Level Two award, a certificate is presented to an employee by the hospital’s HRO lead, HRO physician champion, patient safety manager, immediate supervisor, and peers. With the Level 3 award, the Culture of Safety Appreciation Award, individuals are recognized for addressing safety concerns within their assigned scope of responsibilities. Recognition is bestowed by an email of appreciation sent to the employee, acknowledging their commitment to promoting a culture of safety and quality health care. The recipient’s direct supervisor and other hospital leaders are copied on the message.¹⁴ See Table 1 for a comparison of awards.

Our institution’s HRO and patient safety teams utilized many additional venues to disseminate information regarding awardees and their actions. These included our monthly HRO newsletter, monthly safety forums, and biweekly Team Connecticut Healthcare system-wide huddles.

Nomination Process

Awards nominations are submitted via the hospital intranet homepage, where there is an “HRO Safety Award Nomination” icon. Once a staff member clicks the icon, a template opens asking for information, such as the reason for the nomination submission, and then walks them through the template using the CAR (C-context, A-actions, and R-results)¹⁵ format for describing the situation, identifying actions taken, and specifying the outcome of the action. Emails with award nominations can also be sent to the HRO lead, HRO champion, or Patient Safety Committee co-chairs. Calls for nominations are made at several venues attended by employees as well as supervisors. These include monthly safety forums, biweekly Team Connecticut Healthcare system-wide huddles, supervisory staff meetings, department and unit-based staff meetings, and many other formal and informal settings. This QI initiative has allowed us to capture potential awardees through several avenues, including self-nominations. All nominations are reviewed by a safety awards committee. Each committee member ranks the nomination as a Level 1, 2, or 3 award. For nominations where conflicting scores are obtained, the committee discusses the nomination together to resolve discrepancies.

Needed Resources

Material resources required for this QI initiative include certificate paper, plastic baseball card sleeves, stress-release baseballs, and Bazooka gum. The largest resource investment was the time needed to support the initiative. This included the time spent scheduling the Level 1 and 2 award presentations with staff and leadership. Time was also required to put the individual award packages together, which included printing the paper certificates, obtaining awardee pictures, placing them with their safety stories in a plastic baseball card sleeve, and arranging for the hospital photographer to take pictures of the awardees with their peers and leaders.

RESULTS

Prior to this QI initiative launch, 14 awards were given out over the preceding 2-year period. During the initial 18 months of the initiative (December 2020 to June 2022), 59 awards were presented (Level 1, n = 26; Level 2, n = 22; and Level 3, n = 11). Looking further into the Level 1 awards presented, 25 awardees worked in clinical roles and 1 in a nonclinical position (Table 2). The awardees represented multidisciplinary areas, including medical/surgical (med/surg) inpatient units, anesthesia, operating room, pharmacy, mental health clinics, surgical intensive care, specialty care clinics, and nutrition and food services. With the Level 2 awards, 18 clinical staff and 4 nonclinical staff received awards from the areas of med/surg inpatient, outpatient surgical suites, the medical center director’s office, radiology, pharmacy, primary care, facilities management, environmental management, infection prevention, and emergency services. All Level 3 awardees were from clinical areas, including primary care, hospital education, sterile processing, pharmacies, operating rooms, and med/surg inpatient units.

With the inception of this QI initiative, our organization has begun to see trends reflecting increased reporting of both actual and close-call events in JPSR (Figure 1).

With the inclusion of information regarding awardees and their actions in monthly safety forums, attendance at these forums has increased from an average of 64 attendees per month in 2021 to an average of 131 attendees per month in 2022 (Figure 2).

Finally, our organization’s annual All-Employee Survey results have shown incremental increases in staff reporting feeling psychologically safe and not fearing reprisal (Figure 3). It is important to note that there may be other contributing factors to these incremental changes.

Stories From the 3 Award Categories

Level 1 – Good Catch Award. M.S. was assigned as a continuous safety monitor, or “sitter,” on one of the med/surg inpatient units. M.S. arrived at the bedside and asked for a report on the patient at a change in shift. The report stated that the patient was sleeping and had not moved in a while. M.S. set about to perform the functions of a sitter and did her usual tasks in cleaning and tidying the room for the patient for breakfast and taking care of all items in the room, in general. M.S. introduced herself to the patient, who she thought might wake up because of her speaking to him. She thought the patient was in an odd position, and knowing that a patient should be a little further up in the bed, she tried with touch to awaken him to adjust his position. M.S. found that the patient was rather chilly to the touch and immediately became concerned. She continued to attempt to rouse the patient. M.S. called for the nurse and began to adjust the patient’s position. M.S. insisted that the patient was cold and “something was wrong.” A set of vitals was taken and a rapid response team code was called. The patient was immediately transferred to the intensive care unit to receive a higher level of care. If not for the diligence and caring attitude of M.S., this patient may have had a very poor outcome.

Reason for criteria being met: The scope of practice of a sitter is to be present in a patient’s room to monitor for falls and overall safety. This employee noticed that the patient was not responsive to verbal or tactile stimuli. Her immediate reporting of her concern to the nurse resulted in prompt intervention. If she had let the patient be, the patient could have died. The staff member went above and beyond by speaking up and taking action when she had a patient safety concern.

Level 2 – HRO Safety Champion Award. A patient presented to an outpatient clinic for monoclonal antibody (mAb) therapy for a COVID-19 infection; the treatment has been scheduled by the patient’s primary care provider. At that time, outpatient mAb therapy was the recommended care option for patients stable enough to receive treatment in this setting, but it is contraindicated in patients who are too unstable to receive mAb therapy in an outpatient setting, such as those with increased oxygen demands. R.L., a staff nurse, assessed the patient on arrival and found that his vital signs were stable, except for a slightly elevated respiratory rate. Upon questioning, the patient reported that he had increased his oxygen use at home from 2 to 4 L via a nasal cannula. R.L. assessed that the patient was too high-risk for outpatient mAb therapy and had the patient checked into the emergency department (ED) to receive a full diagnostic workup and evaluation by Dr. W., an ED provider. The patient required admission to the hospital for a higher level of care in an inpatient unit because of severe COVID-19 infection. Within 48 hours of admission, the patient’s condition further declined, requiring an upgrade to the medical intensive care unit with progressive interventions. Owing to the clinical assessment skills and prompt action of R.L., the patient was admitted to the hospital instead of receiving treatment in a suboptimal care setting and returning home. Had the patient gone home, his rapid decline could have had serious consequences.

Reason for criteria being met: On a cursory look, the patient may have passed as someone sufficiently stable to undergo outpatient treatment. However, the nurse stopped the line, paid close attention, and picked up on an abnormal vital sign and the projected consequences. The nurse brought the patient to a higher level of care in the ED so that he could get the attention he needed. If this patient was given mAb therapy in the outpatient setting, he would have been discharged and become sicker with the COVID-19 illness. As a result of this incident, R.L. is working with the outpatient clinic and ED staff to enahance triage and evaluation of patients referred for outpatient therapy for COVID-19 infections to prevent a similar event from recurring.

Level 3 – Culture of Safety Appreciation Award. While C.C. was reviewing the hazardous item competencies of the acute psychiatric inpatient staff, it was learned that staff were sniffing patients’ personal items to see if they were “safe” and free from alcohol. This is a potentially dangerous practice, and if fentanyl is present, it can be life-threatening. All patients admitted to acute inpatient psychiatry have all their clothing and personal items checked for hazardous items—pockets are emptied, soles of shoes are lifted, and so on. Staff wear personal protective equipment during this process to mitigate any powders or other harmful substances being inhaled or coming in contact with their skin or clothes. The gloves can be punctured if needles are found in the patient’s belongings. C.C. not only educated the staff on the dangers of sniffing for alcohol during hazardous-item checks, but also looked for further potential safety concerns. An additional identified risk was for needle sticks when such items were found in a patient’s belongings. C.C.’s recommendations included best practices to allow only unopened personal items and have available hospital-issued products as needed. C.C. remembered having a conversation with an employee from the psychiatric emergency room regarding the purchase of puncture-proof gloves to mitigate puncture sticks. C.C. recommended that the same gloves be used by staff on the acute inpatient psychiatry unit during searches for hazardous items.

Reason for criteria being met: The employee works in the hospital education department. It is within her scope of responsibilities to provide ongoing education to staff in order to address potential safety concerns.

DISCUSSION

This QI initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety and advancing quality health care, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. As part of efforts to continuously build on a safety-first culture, transparency and celebration of successes were demonstrated. This QI initiative demonstrated that a diverse and wide range of employees were reached, from clinical to nonclinical staff, and frontline to supervisory staff, as all were included in the recognition process. While many award nominations were received through the submission of safety concerns to the high-reliability team and patient safety office, several came directly from staff who wanted to recognize their peers for their work, supporting a culture of safety. This showed that staff felt that taking the time to submit a write-up to recognize a peer was an important task. Achieving zero harm for patients and staff alike is a top priority for our institution and guides all decisions, which reinforces that everyone has a responsibility to ensure that safety is always the first consideration. A culture of safety is enhanced by staff recognition. This QI initiative also showed that staff felt valued when they were acknowledged, regardless of the level of recognition they received. The theme of feeling valued came from unsolicited feedback. For example, some direct comments from awardees are presented in the Box.

In addition to endorsing the importance of safe practices to staff, safety award programs can identify gaps in existing standard procedures that can be updated quickly and shared broadly across a health care organization. The authors observed that the existence of the award program gives staff permission to use their voice to speak up when they have questions or concerns related to safety and to proactively engage in safety practices; a cultural shift of this kind informs safety practices and procedures and contributes to a more inspiring workplace. Staff at our organization who have received any of the safety awards, and those who are aware of these awards, have embraced the program readily. At the time of submission of this manuscript, there was a relative paucity of published literature on the details, performance, and impact of such programs. This initiative aims to share a road map highlighting the various dimensions of staff recognition and how the program supports our health care system in fostering a strong, sustainable culture of safety and health care quality. A next step is to formally assess the impact of the awards program on our culture of safety and quality using a psychometrically sound measurement tool, as recommended by TJC,¹⁶ such as the Hospital Survey on Patient Safety Culture.^17,18

CONCLUSION

A health care organization safety awards program is a strategy for building and sustaining a culture of safety. This QI initiative may be valuable to other organizations in the process of establishing a safety awards program of their own. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Corresponding author: John S. Murray, PhD, MPH, MSGH, RN, FAAN, 20 Chapel Street, Unit A502, Brookline, MA 02446; [email protected]

Disclosures: None reported.

ABSTRACT

Objective: Promoting a culture of safety is a critical component of improving health care quality. Recognizing staff who stop the line for safety can positively impact the growth of a culture of safety. The purpose of this initiative was to demonstrate to staff the importance of speaking up for safety and being acknowledged for doing so.

Methods: Following a review of the literature on safety awards programs and their role in promoting a culture of safety in health care covering the period 2017 to 2020, a formal process was developed and implemented to disseminate safety awards to employees.

Results: During the initial 18 months of the initiative, a total of 59 awards were presented. The awards were well received by the recipients and other staff members. Within this period, adjustments were made to enhance the scope and reach of the program.

Conclusion: Recognizing staff behaviors that support a culture of safety is important for improving health care quality and employee engagement. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Keywords: patient safety, culture of safety, incident reporting, near miss.

A key aspect of improving health care quality is promoting and sustaining a culture of safety in the workplace. Improving the quality of health care services and systems involves making informed choices regarding the types of strategies to implement.¹ An essential aspect of supporting a safety culture is safety-event reporting. To approach the goal of zero harm, all safety events, whether they result in actual harm or are considered near misses, need to be reported. Near-miss events are errors that occur while care is being provided but are detected and corrected before harm reaches the patient.^1-3 Near-miss reporting plays a critical role in helping to identify and correct weaknesses in health care delivery systems and processes.⁴ However, evidence shows that there are a multitude of barriers to the reporting of near-miss events, such as fear of punitive actions, additional workload, unsupportive work environments, a culture with poor psychological safety, knowledge deficit, and lack of recognition of staff who do report near misses.^4-11

According to The Joint Commission (TJC), acknowledging health care team members who recognize and report unsafe conditions that provide insight for improving patient safety is a key method for promoting the reporting of near-miss events.⁶ As a result, some health care organizations and patient safety agencies have started to institute some form of recognition for their employees in the realm of safety.^8-10 The Pennsylvania Patient Safety Authority offers exceptional guidance for creating a safety awards program to promote a culture of safety.¹² Furthermore, TJC supports recognizing individuals and health care teams who identify and report near misses, or who have suggestions for initiatives to promote patient safety, with “good catch” awards. Individuals or teams working to promote and sustain a culture of safety should be recognized for their efforts. Acknowledging “good catches” to reward the identification, communication, and resolution of safety issues is an effective strategy for improving patient safety and health care quality.^6,8

This quality improvement (QI) initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. If health care organizations want staff to be motivated to report near misses and improve safety and health care quality, the culture needs to shift from focusing on blame to incentivizing individuals and teams to speak up when they have concerns.^8-10 Although deciding which safety actions are worthy of recognition can be challenging, recognizing all safe acts, regardless of how big or small they are perceived to be, is important. This QI initiative aimed to establish a tiered approach to recognize staff members for various categories of safety acts.

METHODS

A review of the literature from January 2017 to May 2020 for peer-reviewed publications regarding how other organizations implemented safety award programs to promote a culture of safety resulted in a dearth of evidence. This prompted us at the Veterans Affairs Connecticut Healthcare System to develop and implement a formal program to disseminate safety awards to employees.

Program Launch and Promotion

In 2020, our institution embarked on a journey to high reliability with the goal of approaching zero harm. As part of efforts to promote a culture of safety, the hospital’s High Reliability Organization (HRO) team worked to develop a safety awards recognition program. Prior to the launch, the hospital’s patient safety committee recognized staff members through the medical center safety event reporting system (the Joint Patient Safety Reporting system [JPSR]) or through direct communication with staff members on safety actions they were engaged in. JPSR is the Veterans Health Administration National Center for Patient Safety incident reporting system for reporting, tracking, and trending of patient incidents in a national database. The award consisted of a certificate presented by the patient safety committee chairpersons to the employee in front of their peers in their respective work area. Hospital leadership was not involved in the safety awards recognition program at that time. No nomination process existed prior to our QI launch.

Once the QI initiative was launched and marketed heavily at staff meetings, we started to receive nominations for actions that were truly exceptional, while many others were submitted for behaviors that were within the day-to-day scope of practice of the staff member. For those early nominations that did not meet criteria for an award, we thanked staff for their submissions with a gentle statement that their nomination did not meet the criteria for an award. After following this practice for a few weeks, we became concerned that if we did not acknowledge the staff who came forward to request recognition for their routine work that supported safety, we could risk losing their engagement in a culture of safety. As such, we decided to create 3 levels of awards to recognize behaviors that went above and beyond while also acknowledging staff for actions within their scope of practice. Additionally, hospital leadership wanted to ensure that all staff recognize that their safety efforts are valued by leadership and that that sense of value will hopefully contribute to a culture of safety over time.

Initially, the single award system was called the “Good Catch Award” to acknowledge staff who go above and beyond to speak up and take action when they have safety concerns. This particular recognition includes a certificate, an encased baseball card that has been personalized by including the staff member’s picture and safety event identified, a stress-release baseball, and a stick of Bazooka gum (similar to what used to come in baseball cards packs). The award is presented to employees in their work area by the HRO and patient safety teams and includes representatives from the executive leadership team (ELT). The safety event identified is described by an ELT member, and all items are presented to the employee. Participation by the leadership team communicates how much the work being done to promote a culture of safety and advance quality health care is appreciated. This action also encourages others in the organization to identify and report safety concerns.¹³

With the rollout of the QI initiative, the volume of nominations submitted quickly increased (eg, approximately 1 every 2 months before to 3 per month following implementation). Frequently, nominations were for actions believed to be within the scope of the employee’s responsibilities. Our institution’s leadership team quickly recognized that, as an organization, not diminishing the importance of the “Good Catch Award” was important. However, the leadership team also wanted to encourage nominations from employees that involved safety issues that were part of the employee’s scope of responsibilities. As a result, 2 additional and equally notable award tiers were established, with specific criteria created for each.¹⁴ The addition of the other awards was instrumental in getting the leadership team to feel confident that all staff were being recognized for their commitment to patient safety.

The original Good Catch Award was labelled as a Level 1 award. The Level 2 safety recognition award, named the HRO Safety Champion Award, is given to employees who stop the line for a safety concern within their scope of practice and also participate as part of a team to investigate and improve processes to avoid recurring safety concerns in the future. For the Level Two award, a certificate is presented to an employee by the hospital’s HRO lead, HRO physician champion, patient safety manager, immediate supervisor, and peers. With the Level 3 award, the Culture of Safety Appreciation Award, individuals are recognized for addressing safety concerns within their assigned scope of responsibilities. Recognition is bestowed by an email of appreciation sent to the employee, acknowledging their commitment to promoting a culture of safety and quality health care. The recipient’s direct supervisor and other hospital leaders are copied on the message.¹⁴ See Table 1 for a comparison of awards.

Our institution’s HRO and patient safety teams utilized many additional venues to disseminate information regarding awardees and their actions. These included our monthly HRO newsletter, monthly safety forums, and biweekly Team Connecticut Healthcare system-wide huddles.

Nomination Process

Awards nominations are submitted via the hospital intranet homepage, where there is an “HRO Safety Award Nomination” icon. Once a staff member clicks the icon, a template opens asking for information, such as the reason for the nomination submission, and then walks them through the template using the CAR (C-context, A-actions, and R-results)¹⁵ format for describing the situation, identifying actions taken, and specifying the outcome of the action. Emails with award nominations can also be sent to the HRO lead, HRO champion, or Patient Safety Committee co-chairs. Calls for nominations are made at several venues attended by employees as well as supervisors. These include monthly safety forums, biweekly Team Connecticut Healthcare system-wide huddles, supervisory staff meetings, department and unit-based staff meetings, and many other formal and informal settings. This QI initiative has allowed us to capture potential awardees through several avenues, including self-nominations. All nominations are reviewed by a safety awards committee. Each committee member ranks the nomination as a Level 1, 2, or 3 award. For nominations where conflicting scores are obtained, the committee discusses the nomination together to resolve discrepancies.

Needed Resources

Material resources required for this QI initiative include certificate paper, plastic baseball card sleeves, stress-release baseballs, and Bazooka gum. The largest resource investment was the time needed to support the initiative. This included the time spent scheduling the Level 1 and 2 award presentations with staff and leadership. Time was also required to put the individual award packages together, which included printing the paper certificates, obtaining awardee pictures, placing them with their safety stories in a plastic baseball card sleeve, and arranging for the hospital photographer to take pictures of the awardees with their peers and leaders.

RESULTS

Prior to this QI initiative launch, 14 awards were given out over the preceding 2-year period. During the initial 18 months of the initiative (December 2020 to June 2022), 59 awards were presented (Level 1, n = 26; Level 2, n = 22; and Level 3, n = 11). Looking further into the Level 1 awards presented, 25 awardees worked in clinical roles and 1 in a nonclinical position (Table 2). The awardees represented multidisciplinary areas, including medical/surgical (med/surg) inpatient units, anesthesia, operating room, pharmacy, mental health clinics, surgical intensive care, specialty care clinics, and nutrition and food services. With the Level 2 awards, 18 clinical staff and 4 nonclinical staff received awards from the areas of med/surg inpatient, outpatient surgical suites, the medical center director’s office, radiology, pharmacy, primary care, facilities management, environmental management, infection prevention, and emergency services. All Level 3 awardees were from clinical areas, including primary care, hospital education, sterile processing, pharmacies, operating rooms, and med/surg inpatient units.

With the inception of this QI initiative, our organization has begun to see trends reflecting increased reporting of both actual and close-call events in JPSR (Figure 1).

With the inclusion of information regarding awardees and their actions in monthly safety forums, attendance at these forums has increased from an average of 64 attendees per month in 2021 to an average of 131 attendees per month in 2022 (Figure 2).

Finally, our organization’s annual All-Employee Survey results have shown incremental increases in staff reporting feeling psychologically safe and not fearing reprisal (Figure 3). It is important to note that there may be other contributing factors to these incremental changes.

Stories From the 3 Award Categories

Level 1 – Good Catch Award. M.S. was assigned as a continuous safety monitor, or “sitter,” on one of the med/surg inpatient units. M.S. arrived at the bedside and asked for a report on the patient at a change in shift. The report stated that the patient was sleeping and had not moved in a while. M.S. set about to perform the functions of a sitter and did her usual tasks in cleaning and tidying the room for the patient for breakfast and taking care of all items in the room, in general. M.S. introduced herself to the patient, who she thought might wake up because of her speaking to him. She thought the patient was in an odd position, and knowing that a patient should be a little further up in the bed, she tried with touch to awaken him to adjust his position. M.S. found that the patient was rather chilly to the touch and immediately became concerned. She continued to attempt to rouse the patient. M.S. called for the nurse and began to adjust the patient’s position. M.S. insisted that the patient was cold and “something was wrong.” A set of vitals was taken and a rapid response team code was called. The patient was immediately transferred to the intensive care unit to receive a higher level of care. If not for the diligence and caring attitude of M.S., this patient may have had a very poor outcome.

Reason for criteria being met: The scope of practice of a sitter is to be present in a patient’s room to monitor for falls and overall safety. This employee noticed that the patient was not responsive to verbal or tactile stimuli. Her immediate reporting of her concern to the nurse resulted in prompt intervention. If she had let the patient be, the patient could have died. The staff member went above and beyond by speaking up and taking action when she had a patient safety concern.

Level 2 – HRO Safety Champion Award. A patient presented to an outpatient clinic for monoclonal antibody (mAb) therapy for a COVID-19 infection; the treatment has been scheduled by the patient’s primary care provider. At that time, outpatient mAb therapy was the recommended care option for patients stable enough to receive treatment in this setting, but it is contraindicated in patients who are too unstable to receive mAb therapy in an outpatient setting, such as those with increased oxygen demands. R.L., a staff nurse, assessed the patient on arrival and found that his vital signs were stable, except for a slightly elevated respiratory rate. Upon questioning, the patient reported that he had increased his oxygen use at home from 2 to 4 L via a nasal cannula. R.L. assessed that the patient was too high-risk for outpatient mAb therapy and had the patient checked into the emergency department (ED) to receive a full diagnostic workup and evaluation by Dr. W., an ED provider. The patient required admission to the hospital for a higher level of care in an inpatient unit because of severe COVID-19 infection. Within 48 hours of admission, the patient’s condition further declined, requiring an upgrade to the medical intensive care unit with progressive interventions. Owing to the clinical assessment skills and prompt action of R.L., the patient was admitted to the hospital instead of receiving treatment in a suboptimal care setting and returning home. Had the patient gone home, his rapid decline could have had serious consequences.

Reason for criteria being met: On a cursory look, the patient may have passed as someone sufficiently stable to undergo outpatient treatment. However, the nurse stopped the line, paid close attention, and picked up on an abnormal vital sign and the projected consequences. The nurse brought the patient to a higher level of care in the ED so that he could get the attention he needed. If this patient was given mAb therapy in the outpatient setting, he would have been discharged and become sicker with the COVID-19 illness. As a result of this incident, R.L. is working with the outpatient clinic and ED staff to enahance triage and evaluation of patients referred for outpatient therapy for COVID-19 infections to prevent a similar event from recurring.

Level 3 – Culture of Safety Appreciation Award. While C.C. was reviewing the hazardous item competencies of the acute psychiatric inpatient staff, it was learned that staff were sniffing patients’ personal items to see if they were “safe” and free from alcohol. This is a potentially dangerous practice, and if fentanyl is present, it can be life-threatening. All patients admitted to acute inpatient psychiatry have all their clothing and personal items checked for hazardous items—pockets are emptied, soles of shoes are lifted, and so on. Staff wear personal protective equipment during this process to mitigate any powders or other harmful substances being inhaled or coming in contact with their skin or clothes. The gloves can be punctured if needles are found in the patient’s belongings. C.C. not only educated the staff on the dangers of sniffing for alcohol during hazardous-item checks, but also looked for further potential safety concerns. An additional identified risk was for needle sticks when such items were found in a patient’s belongings. C.C.’s recommendations included best practices to allow only unopened personal items and have available hospital-issued products as needed. C.C. remembered having a conversation with an employee from the psychiatric emergency room regarding the purchase of puncture-proof gloves to mitigate puncture sticks. C.C. recommended that the same gloves be used by staff on the acute inpatient psychiatry unit during searches for hazardous items.

Reason for criteria being met: The employee works in the hospital education department. It is within her scope of responsibilities to provide ongoing education to staff in order to address potential safety concerns.

DISCUSSION

This QI initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety and advancing quality health care, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. As part of efforts to continuously build on a safety-first culture, transparency and celebration of successes were demonstrated. This QI initiative demonstrated that a diverse and wide range of employees were reached, from clinical to nonclinical staff, and frontline to supervisory staff, as all were included in the recognition process. While many award nominations were received through the submission of safety concerns to the high-reliability team and patient safety office, several came directly from staff who wanted to recognize their peers for their work, supporting a culture of safety. This showed that staff felt that taking the time to submit a write-up to recognize a peer was an important task. Achieving zero harm for patients and staff alike is a top priority for our institution and guides all decisions, which reinforces that everyone has a responsibility to ensure that safety is always the first consideration. A culture of safety is enhanced by staff recognition. This QI initiative also showed that staff felt valued when they were acknowledged, regardless of the level of recognition they received. The theme of feeling valued came from unsolicited feedback. For example, some direct comments from awardees are presented in the Box.

In addition to endorsing the importance of safe practices to staff, safety award programs can identify gaps in existing standard procedures that can be updated quickly and shared broadly across a health care organization. The authors observed that the existence of the award program gives staff permission to use their voice to speak up when they have questions or concerns related to safety and to proactively engage in safety practices; a cultural shift of this kind informs safety practices and procedures and contributes to a more inspiring workplace. Staff at our organization who have received any of the safety awards, and those who are aware of these awards, have embraced the program readily. At the time of submission of this manuscript, there was a relative paucity of published literature on the details, performance, and impact of such programs. This initiative aims to share a road map highlighting the various dimensions of staff recognition and how the program supports our health care system in fostering a strong, sustainable culture of safety and health care quality. A next step is to formally assess the impact of the awards program on our culture of safety and quality using a psychometrically sound measurement tool, as recommended by TJC,¹⁶ such as the Hospital Survey on Patient Safety Culture.^17,18

CONCLUSION

A health care organization safety awards program is a strategy for building and sustaining a culture of safety. This QI initiative may be valuable to other organizations in the process of establishing a safety awards program of their own. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Corresponding author: John S. Murray, PhD, MPH, MSGH, RN, FAAN, 20 Chapel Street, Unit A502, Brookline, MA 02446; [email protected]

Disclosures: None reported.

ABSTRACT

Objective: Promoting a culture of safety is a critical component of improving health care quality. Recognizing staff who stop the line for safety can positively impact the growth of a culture of safety. The purpose of this initiative was to demonstrate to staff the importance of speaking up for safety and being acknowledged for doing so.

Methods: Following a review of the literature on safety awards programs and their role in promoting a culture of safety in health care covering the period 2017 to 2020, a formal process was developed and implemented to disseminate safety awards to employees.

Results: During the initial 18 months of the initiative, a total of 59 awards were presented. The awards were well received by the recipients and other staff members. Within this period, adjustments were made to enhance the scope and reach of the program.

Conclusion: Recognizing staff behaviors that support a culture of safety is important for improving health care quality and employee engagement. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Keywords: patient safety, culture of safety, incident reporting, near miss.

A key aspect of improving health care quality is promoting and sustaining a culture of safety in the workplace. Improving the quality of health care services and systems involves making informed choices regarding the types of strategies to implement.¹ An essential aspect of supporting a safety culture is safety-event reporting. To approach the goal of zero harm, all safety events, whether they result in actual harm or are considered near misses, need to be reported. Near-miss events are errors that occur while care is being provided but are detected and corrected before harm reaches the patient.^1-3 Near-miss reporting plays a critical role in helping to identify and correct weaknesses in health care delivery systems and processes.⁴ However, evidence shows that there are a multitude of barriers to the reporting of near-miss events, such as fear of punitive actions, additional workload, unsupportive work environments, a culture with poor psychological safety, knowledge deficit, and lack of recognition of staff who do report near misses.^4-11

According to The Joint Commission (TJC), acknowledging health care team members who recognize and report unsafe conditions that provide insight for improving patient safety is a key method for promoting the reporting of near-miss events.⁶ As a result, some health care organizations and patient safety agencies have started to institute some form of recognition for their employees in the realm of safety.^8-10 The Pennsylvania Patient Safety Authority offers exceptional guidance for creating a safety awards program to promote a culture of safety.¹² Furthermore, TJC supports recognizing individuals and health care teams who identify and report near misses, or who have suggestions for initiatives to promote patient safety, with “good catch” awards. Individuals or teams working to promote and sustain a culture of safety should be recognized for their efforts. Acknowledging “good catches” to reward the identification, communication, and resolution of safety issues is an effective strategy for improving patient safety and health care quality.^6,8

This quality improvement (QI) initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. If health care organizations want staff to be motivated to report near misses and improve safety and health care quality, the culture needs to shift from focusing on blame to incentivizing individuals and teams to speak up when they have concerns.^8-10 Although deciding which safety actions are worthy of recognition can be challenging, recognizing all safe acts, regardless of how big or small they are perceived to be, is important. This QI initiative aimed to establish a tiered approach to recognize staff members for various categories of safety acts.

METHODS

A review of the literature from January 2017 to May 2020 for peer-reviewed publications regarding how other organizations implemented safety award programs to promote a culture of safety resulted in a dearth of evidence. This prompted us at the Veterans Affairs Connecticut Healthcare System to develop and implement a formal program to disseminate safety awards to employees.

Program Launch and Promotion

In 2020, our institution embarked on a journey to high reliability with the goal of approaching zero harm. As part of efforts to promote a culture of safety, the hospital’s High Reliability Organization (HRO) team worked to develop a safety awards recognition program. Prior to the launch, the hospital’s patient safety committee recognized staff members through the medical center safety event reporting system (the Joint Patient Safety Reporting system [JPSR]) or through direct communication with staff members on safety actions they were engaged in. JPSR is the Veterans Health Administration National Center for Patient Safety incident reporting system for reporting, tracking, and trending of patient incidents in a national database. The award consisted of a certificate presented by the patient safety committee chairpersons to the employee in front of their peers in their respective work area. Hospital leadership was not involved in the safety awards recognition program at that time. No nomination process existed prior to our QI launch.

Once the QI initiative was launched and marketed heavily at staff meetings, we started to receive nominations for actions that were truly exceptional, while many others were submitted for behaviors that were within the day-to-day scope of practice of the staff member. For those early nominations that did not meet criteria for an award, we thanked staff for their submissions with a gentle statement that their nomination did not meet the criteria for an award. After following this practice for a few weeks, we became concerned that if we did not acknowledge the staff who came forward to request recognition for their routine work that supported safety, we could risk losing their engagement in a culture of safety. As such, we decided to create 3 levels of awards to recognize behaviors that went above and beyond while also acknowledging staff for actions within their scope of practice. Additionally, hospital leadership wanted to ensure that all staff recognize that their safety efforts are valued by leadership and that that sense of value will hopefully contribute to a culture of safety over time.

Initially, the single award system was called the “Good Catch Award” to acknowledge staff who go above and beyond to speak up and take action when they have safety concerns. This particular recognition includes a certificate, an encased baseball card that has been personalized by including the staff member’s picture and safety event identified, a stress-release baseball, and a stick of Bazooka gum (similar to what used to come in baseball cards packs). The award is presented to employees in their work area by the HRO and patient safety teams and includes representatives from the executive leadership team (ELT). The safety event identified is described by an ELT member, and all items are presented to the employee. Participation by the leadership team communicates how much the work being done to promote a culture of safety and advance quality health care is appreciated. This action also encourages others in the organization to identify and report safety concerns.¹³

With the rollout of the QI initiative, the volume of nominations submitted quickly increased (eg, approximately 1 every 2 months before to 3 per month following implementation). Frequently, nominations were for actions believed to be within the scope of the employee’s responsibilities. Our institution’s leadership team quickly recognized that, as an organization, not diminishing the importance of the “Good Catch Award” was important. However, the leadership team also wanted to encourage nominations from employees that involved safety issues that were part of the employee’s scope of responsibilities. As a result, 2 additional and equally notable award tiers were established, with specific criteria created for each.¹⁴ The addition of the other awards was instrumental in getting the leadership team to feel confident that all staff were being recognized for their commitment to patient safety.

The original Good Catch Award was labelled as a Level 1 award. The Level 2 safety recognition award, named the HRO Safety Champion Award, is given to employees who stop the line for a safety concern within their scope of practice and also participate as part of a team to investigate and improve processes to avoid recurring safety concerns in the future. For the Level Two award, a certificate is presented to an employee by the hospital’s HRO lead, HRO physician champion, patient safety manager, immediate supervisor, and peers. With the Level 3 award, the Culture of Safety Appreciation Award, individuals are recognized for addressing safety concerns within their assigned scope of responsibilities. Recognition is bestowed by an email of appreciation sent to the employee, acknowledging their commitment to promoting a culture of safety and quality health care. The recipient’s direct supervisor and other hospital leaders are copied on the message.¹⁴ See Table 1 for a comparison of awards.

Our institution’s HRO and patient safety teams utilized many additional venues to disseminate information regarding awardees and their actions. These included our monthly HRO newsletter, monthly safety forums, and biweekly Team Connecticut Healthcare system-wide huddles.

Nomination Process

Awards nominations are submitted via the hospital intranet homepage, where there is an “HRO Safety Award Nomination” icon. Once a staff member clicks the icon, a template opens asking for information, such as the reason for the nomination submission, and then walks them through the template using the CAR (C-context, A-actions, and R-results)¹⁵ format for describing the situation, identifying actions taken, and specifying the outcome of the action. Emails with award nominations can also be sent to the HRO lead, HRO champion, or Patient Safety Committee co-chairs. Calls for nominations are made at several venues attended by employees as well as supervisors. These include monthly safety forums, biweekly Team Connecticut Healthcare system-wide huddles, supervisory staff meetings, department and unit-based staff meetings, and many other formal and informal settings. This QI initiative has allowed us to capture potential awardees through several avenues, including self-nominations. All nominations are reviewed by a safety awards committee. Each committee member ranks the nomination as a Level 1, 2, or 3 award. For nominations where conflicting scores are obtained, the committee discusses the nomination together to resolve discrepancies.

Needed Resources

Material resources required for this QI initiative include certificate paper, plastic baseball card sleeves, stress-release baseballs, and Bazooka gum. The largest resource investment was the time needed to support the initiative. This included the time spent scheduling the Level 1 and 2 award presentations with staff and leadership. Time was also required to put the individual award packages together, which included printing the paper certificates, obtaining awardee pictures, placing them with their safety stories in a plastic baseball card sleeve, and arranging for the hospital photographer to take pictures of the awardees with their peers and leaders.

RESULTS

Prior to this QI initiative launch, 14 awards were given out over the preceding 2-year period. During the initial 18 months of the initiative (December 2020 to June 2022), 59 awards were presented (Level 1, n = 26; Level 2, n = 22; and Level 3, n = 11). Looking further into the Level 1 awards presented, 25 awardees worked in clinical roles and 1 in a nonclinical position (Table 2). The awardees represented multidisciplinary areas, including medical/surgical (med/surg) inpatient units, anesthesia, operating room, pharmacy, mental health clinics, surgical intensive care, specialty care clinics, and nutrition and food services. With the Level 2 awards, 18 clinical staff and 4 nonclinical staff received awards from the areas of med/surg inpatient, outpatient surgical suites, the medical center director’s office, radiology, pharmacy, primary care, facilities management, environmental management, infection prevention, and emergency services. All Level 3 awardees were from clinical areas, including primary care, hospital education, sterile processing, pharmacies, operating rooms, and med/surg inpatient units.

With the inception of this QI initiative, our organization has begun to see trends reflecting increased reporting of both actual and close-call events in JPSR (Figure 1).

With the inclusion of information regarding awardees and their actions in monthly safety forums, attendance at these forums has increased from an average of 64 attendees per month in 2021 to an average of 131 attendees per month in 2022 (Figure 2).

Finally, our organization’s annual All-Employee Survey results have shown incremental increases in staff reporting feeling psychologically safe and not fearing reprisal (Figure 3). It is important to note that there may be other contributing factors to these incremental changes.

Stories From the 3 Award Categories

Level 1 – Good Catch Award. M.S. was assigned as a continuous safety monitor, or “sitter,” on one of the med/surg inpatient units. M.S. arrived at the bedside and asked for a report on the patient at a change in shift. The report stated that the patient was sleeping and had not moved in a while. M.S. set about to perform the functions of a sitter and did her usual tasks in cleaning and tidying the room for the patient for breakfast and taking care of all items in the room, in general. M.S. introduced herself to the patient, who she thought might wake up because of her speaking to him. She thought the patient was in an odd position, and knowing that a patient should be a little further up in the bed, she tried with touch to awaken him to adjust his position. M.S. found that the patient was rather chilly to the touch and immediately became concerned. She continued to attempt to rouse the patient. M.S. called for the nurse and began to adjust the patient’s position. M.S. insisted that the patient was cold and “something was wrong.” A set of vitals was taken and a rapid response team code was called. The patient was immediately transferred to the intensive care unit to receive a higher level of care. If not for the diligence and caring attitude of M.S., this patient may have had a very poor outcome.

Reason for criteria being met: The scope of practice of a sitter is to be present in a patient’s room to monitor for falls and overall safety. This employee noticed that the patient was not responsive to verbal or tactile stimuli. Her immediate reporting of her concern to the nurse resulted in prompt intervention. If she had let the patient be, the patient could have died. The staff member went above and beyond by speaking up and taking action when she had a patient safety concern.

Level 2 – HRO Safety Champion Award. A patient presented to an outpatient clinic for monoclonal antibody (mAb) therapy for a COVID-19 infection; the treatment has been scheduled by the patient’s primary care provider. At that time, outpatient mAb therapy was the recommended care option for patients stable enough to receive treatment in this setting, but it is contraindicated in patients who are too unstable to receive mAb therapy in an outpatient setting, such as those with increased oxygen demands. R.L., a staff nurse, assessed the patient on arrival and found that his vital signs were stable, except for a slightly elevated respiratory rate. Upon questioning, the patient reported that he had increased his oxygen use at home from 2 to 4 L via a nasal cannula. R.L. assessed that the patient was too high-risk for outpatient mAb therapy and had the patient checked into the emergency department (ED) to receive a full diagnostic workup and evaluation by Dr. W., an ED provider. The patient required admission to the hospital for a higher level of care in an inpatient unit because of severe COVID-19 infection. Within 48 hours of admission, the patient’s condition further declined, requiring an upgrade to the medical intensive care unit with progressive interventions. Owing to the clinical assessment skills and prompt action of R.L., the patient was admitted to the hospital instead of receiving treatment in a suboptimal care setting and returning home. Had the patient gone home, his rapid decline could have had serious consequences.

Reason for criteria being met: On a cursory look, the patient may have passed as someone sufficiently stable to undergo outpatient treatment. However, the nurse stopped the line, paid close attention, and picked up on an abnormal vital sign and the projected consequences. The nurse brought the patient to a higher level of care in the ED so that he could get the attention he needed. If this patient was given mAb therapy in the outpatient setting, he would have been discharged and become sicker with the COVID-19 illness. As a result of this incident, R.L. is working with the outpatient clinic and ED staff to enahance triage and evaluation of patients referred for outpatient therapy for COVID-19 infections to prevent a similar event from recurring.

Level 3 – Culture of Safety Appreciation Award. While C.C. was reviewing the hazardous item competencies of the acute psychiatric inpatient staff, it was learned that staff were sniffing patients’ personal items to see if they were “safe” and free from alcohol. This is a potentially dangerous practice, and if fentanyl is present, it can be life-threatening. All patients admitted to acute inpatient psychiatry have all their clothing and personal items checked for hazardous items—pockets are emptied, soles of shoes are lifted, and so on. Staff wear personal protective equipment during this process to mitigate any powders or other harmful substances being inhaled or coming in contact with their skin or clothes. The gloves can be punctured if needles are found in the patient’s belongings. C.C. not only educated the staff on the dangers of sniffing for alcohol during hazardous-item checks, but also looked for further potential safety concerns. An additional identified risk was for needle sticks when such items were found in a patient’s belongings. C.C.’s recommendations included best practices to allow only unopened personal items and have available hospital-issued products as needed. C.C. remembered having a conversation with an employee from the psychiatric emergency room regarding the purchase of puncture-proof gloves to mitigate puncture sticks. C.C. recommended that the same gloves be used by staff on the acute inpatient psychiatry unit during searches for hazardous items.

Reason for criteria being met: The employee works in the hospital education department. It is within her scope of responsibilities to provide ongoing education to staff in order to address potential safety concerns.

DISCUSSION

This QI initiative was undertaken to demonstrate to staff that, in building an organizational culture of safety and advancing quality health care, it is important that staff be encouraged to speak up for safety and be acknowledged for doing so. As part of efforts to continuously build on a safety-first culture, transparency and celebration of successes were demonstrated. This QI initiative demonstrated that a diverse and wide range of employees were reached, from clinical to nonclinical staff, and frontline to supervisory staff, as all were included in the recognition process. While many award nominations were received through the submission of safety concerns to the high-reliability team and patient safety office, several came directly from staff who wanted to recognize their peers for their work, supporting a culture of safety. This showed that staff felt that taking the time to submit a write-up to recognize a peer was an important task. Achieving zero harm for patients and staff alike is a top priority for our institution and guides all decisions, which reinforces that everyone has a responsibility to ensure that safety is always the first consideration. A culture of safety is enhanced by staff recognition. This QI initiative also showed that staff felt valued when they were acknowledged, regardless of the level of recognition they received. The theme of feeling valued came from unsolicited feedback. For example, some direct comments from awardees are presented in the Box.

In addition to endorsing the importance of safe practices to staff, safety award programs can identify gaps in existing standard procedures that can be updated quickly and shared broadly across a health care organization. The authors observed that the existence of the award program gives staff permission to use their voice to speak up when they have questions or concerns related to safety and to proactively engage in safety practices; a cultural shift of this kind informs safety practices and procedures and contributes to a more inspiring workplace. Staff at our organization who have received any of the safety awards, and those who are aware of these awards, have embraced the program readily. At the time of submission of this manuscript, there was a relative paucity of published literature on the details, performance, and impact of such programs. This initiative aims to share a road map highlighting the various dimensions of staff recognition and how the program supports our health care system in fostering a strong, sustainable culture of safety and health care quality. A next step is to formally assess the impact of the awards program on our culture of safety and quality using a psychometrically sound measurement tool, as recommended by TJC,¹⁶ such as the Hospital Survey on Patient Safety Culture.^17,18

CONCLUSION

A health care organization safety awards program is a strategy for building and sustaining a culture of safety. This QI initiative may be valuable to other organizations in the process of establishing a safety awards program of their own. Future research should focus on a formal evaluation of the impact of safety awards programs on patient safety outcomes.

Corresponding author: John S. Murray, PhD, MPH, MSGH, RN, FAAN, 20 Chapel Street, Unit A502, Brookline, MA 02446; [email protected]

Disclosures: None reported.

ABSTRACT

Objective: To teach internal medicine residents quality improvement (QI) principles in an effort to improve resident knowledge and comfort with QI, as well as address quality care gaps in resident clinic primary care patient panels.

Design: A QI curriculum was implemented for all residents rotating through a primary care block over a 6-month period. Residents completed Institute for Healthcare Improvement (IHI) modules, participated in a QI workshop, and received panel data reports, ultimately completing a plan-do-study-act (PDSA) cycle to improve colorectal cancer screening and hypertension control.

Setting and participants: This project was undertaken at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. All internal medicine residents were included, with 55 (73%) of the 75 residents completing the presurvey, and 39 (52%) completing the postsurvey.

Measurements: We administered a 10-question pre- and postsurvey looking at resident attitudes toward and comfort with QI and familiarity with their panel data as well as measured rates of colorectal cancer screening and hypertension control in resident panels.

Results: There was an increase in the numbers of residents who performed a PDSA cycle (P = .002), completed outreach based on their panel data (P = .02), and felt comfortable in both creating aim statements and designing and implementing PDSA cycles (P < .0001). The residents’ knowledge of their panel data significantly increased. There was no significant improvement in hypertension control, but there was an increase in colorectal cancer screening rates (P < .0001).

Conclusion: Providing panel data and performing targeted QI interventions can improve resident comfort with QI, translating to improvement in patient outcomes.

Keywords: quality improvement, resident education, medical education, care gaps, quality metrics.

As quality improvement (QI) has become an integral part of clinical practice, residency training programs have continued to evolve in how best to teach QI. The Accreditation Council for Graduate Medical Education (ACGME) Common Program requirements mandate that core competencies in residency programs include practice-based learning and improvement and systems-based practice.¹ Residents should receive education in QI, receive data on quality metrics and benchmarks related to their patient population, and participate in QI activities. The Clinical Learning Environment Review (CLER) program was established to provide feedback to institutions on 6 focused areas, including patient safety and health care quality. In visits to institutions across the United States, the CLER committees found that many residents had limited knowledge of QI concepts and limited access to data on quality metrics and benchmarks.²

There are many barriers to implementing a QI curriculum in residency programs, and creating and maintaining successful strategies has proven challenging.³ Many QI curricula for internal medicine residents have been described in the literature, but the results of many of these studies focus on resident self-assessment of QI knowledge and numbers of projects rather than on patient outcomes.^4-13 As there is some evidence suggesting that patients treated by residents have worse outcomes on ambulatory quality measures when compared with patients treated by staff physicians,^14,15 it is important to also look at patient outcomes when evaluating a QI curriculum. Experts in education recommend the following to optimize learning: exposure to both didactic and experiential opportunities, connection to health system improvement efforts, and assessment of patient outcomes in addition to learner feedback.^16,17 A study also found that providing panel data to residents could improve quality metrics.¹⁸

In this study, we sought to investigate the effects of a resident QI intervention during an ambulatory block on both residents’ self-assessments of QI knowledge and attitudes as well as on patient quality metrics.

Methods

Curriculum

We implemented this educational initiative at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. Co-located with the 415-bed academic medical center in downtown Boston, the practice serves more than 40,000 patients, approximately 7000 of whom are cared for by resident primary care physicians (PCPs). The internal medicine residents rotate through the primary care clinic as part of continuity clinic during ambulatory or elective blocks. In addition to continuity clinic, the residents have 2 dedicated 3-week primary care rotations during the course of an academic year. Primary care rotations consist of 5 clinic sessions a week as well as structured teaching sessions. Each resident inherits a panel of patients from an outgoing senior resident, with an average panel size of 96 patients per resident.

Prior to this study intervention, we did not do any formal QI teaching to our residents as part of their primary care curriculum, and previous panel management had focused more on chart reviews of patients whom residents perceived to be higher risk. Residents from all 3 years were included in the intervention. We taught a QI curriculum to our residents from January 2018 to June 2018 during the 3-week primary care rotation, which consisted of the following components:

• Institute for Healthcare Improvement (IHI) module QI 102 completed independently online.
• A 2-hour QI workshop led by 1 of 2 primary care faculty with backgrounds in QI, during which residents were taught basic principles of QI, including how to craft aim statements and design plan-do-study-act (PDSA) cycles, and participated in a hands-on QI activity designed to model rapid cycle improvement (the Paper Airplane Factory¹⁹).
• Distribution of individualized reports of residents’ patient panel data by email at the start of the primary care block that detailed patients’ overall rates of colorectal cancer screening and hypertension (HTN) control, along with the average resident panel rates and the average attending panel rates. The reports also included a list of all residents’ patients who were overdue for colorectal cancer screening or whose last blood pressure (BP) was uncontrolled (systolic BP ≥ 140 mm Hg or diastolic BP ≥  90 mm Hg). These reports were originally designed by our practice’s QI team and run and exported in Microsoft Excel format monthly by our information technology (IT) administrator.
• Instruction on aim statements as a group, followed by the expectation that each resident create an individualized aim statement tailored to each resident’s patient panel rates, with the PDSA cycle to be implemented during the remainder of the primary care rotation, focusing on improvement of colorectal cancer screening and HTN control (see supplementary eFigure 1 online for the worksheet used for the workshop).
• Residents were held accountable for their interventions by various check-ins. At the end of the primary care block, residents were required to submit their completed worksheets showing the intervention they had undertaken and when it was performed. The 2 primary care attendings primarily responsible for QI education would review the resident’s work approximately 1 to 2 months after they submitted their worksheets describing their intervention. These attendings sent the residents personalized feedback based on whether the intervention had been completed or successful as evidenced by documentation in the chart, including direct patient outreach by phone, letter, or portal; outreach to the resident coordinator; scheduled follow-up appointment; or booking or completion of colorectal cancer screening. Along with this feedback, residents were also sent suggestions for next steps. Resident preceptors were copied on the email to facilitate reinforcement of the goals and plans. Finally, the resident preceptors also helped with accountability by going through the residents’ worksheets and patient panel metrics with the residents during biannual evaluations.

Evaluation

Residents were surveyed with a 10-item questionnaire pre and post intervention regarding their attitudes toward QI, understanding of QI principles, and familiarity with their patient panel data. Surveys were anonymous and distributed via the SurveyMonkey platform (see supplementary eFigure 2 online). Residents were asked if they had ever performed a PDSA cycle, performed patient outreach, or performed an intervention and whether they knew the rates of diabetes, HTN, and colorectal cancer screening in their patient panels. Questions rated on a 5-point Likert scale were used to assess comfort with panel management, developing an aim statement, designing and implementing a PDSA cycle, as well as interest in pursuing QI as a career. For the purposes of analysis, these questions were dichotomized into “somewhat comfortable” and “very comfortable” vs “neutral,” “somewhat uncomfortable,” and “very uncomfortable.” Similarly, we dichotomized the question about interest in QI as a career into “somewhat interested” and “very interested” vs “neutral,” “somewhat disinterested,” and “very disinterested.” As the surveys were anonymous, we were unable to pair the pre- and postintervention surveys and used a chi-square test to evaluate whether there was an association between survey assessments pre intervention vs post intervention and a positive or negative response to the question.

We also examined rates of HTN control and colorectal cancer screening in all 75 resident panels pre and post intervention. The paired t-test was used to determine whether the mean change from pre to post intervention was significant. SAS 9.4 (SAS Institute Inc.) was used for all analyses. Institutional Review Board exemption was obtained from the Tufts Medical Center IRB. There was no funding received for this study.

Respondents

Of the 75 residents, 55 (73%) completed the survey prior to the intervention, and 39 (52%) completed the survey after the intervention.

Panel Knowledge and Intervention

Prior to the intervention, 45% of residents had performed a PDSA cycle, compared with 77% post intervention, which was a significant increase (P = .002) (Table 1). Sixty-two percent of residents had performed outreach or an intervention based on their patient panel reports prior to the intervention, compared with 85% of residents post intervention, which was also a significant increase (P = .02). The increase post intervention was not 100%, as there were residents who either missed the initial workshop or who did not follow through with their planned intervention. Common interventions included the residents giving their coordinators a list of patients to call to schedule appointments, utilizing fellow team members (eg, pharmacists, social workers) for targeted patient outreach, or calling patients themselves to reestablish a connection.

In terms of knowledge of their patient panels, prior to the intervention, 55%, 62%, and 62% of residents knew the rates of patients in their panel with diabetes, HTN, and colorectal cancer screening, respectively. After the intervention, the residents’ knowledge of these rates increased significantly, to 85% for diabetes (P = .002), 97% for HTN (P < .0001), and 97% for colorectal cancer screening (P < .0001).

Comfort With QI Approaches

Prior to the intervention, 82% of residents were comfortable managing their primary care panel, which did not change significantly post intervention (Table 2). The residents’ comfort with designing an aim statement did significantly increase, from 55% to 95% (P < .0001). The residents also had a significant increase in comfort with both designing and implementing a PDSA cycle. Prior to the intervention, 22% felt comfortable designing a PDSA cycle, which increased to 79% (P < .0001) post intervention, and 24% felt comfortable implementing a PDSA cycle, which increased to 77% (P < .0001) post intervention.

Patient Outcome Measures

The rate of HTN control in the residents' patient panels did not change significantly pre and post intervention (Table 3). The rate of resident patients who were up to date with colorectal cancer screening increased by 6.5% post intervention (P < .0001).

Interest in QI as a Career

As part of the survey, residents were asked how interested they were in making QI a part of their career. Fifty percent of residents indicated an interest in QI pre intervention, and 54% indicated an interest post intervention, which was not a significant difference (P = .72).

Discussion

In this study, we found that integration of a QI curriculum into a primary care rotation improved both residents’ knowledge of their patient panels and comfort with QI approaches, which translated to improvement in patient outcomes. Several previous studies have found improvements in resident self-assessment or knowledge after implementation of a QI curriculum.^4-13 Liao et al implemented a longitudinal curriculum including both didactic and experiential components and found an improvement in both QI confidence and knowledge.³ Similarly, Duello et al⁸ found that a curriculum including both didactic lectures and QI projects improved subjective QI knowledge and comfort. Interestingly, Fok and Wong⁹ found that resident knowledge could be sustained post curriculum after completion of a QI project, suggesting that experiential learning may be helpful in maintaining knowledge.

Studies also have looked at providing performance data to residents. Hwang et al¹⁸ found that providing audit and feedback in the form of individual panel performance data to residents compared with practice targets led to statistically significant improvement in cancer screening rates and composite quality score, indicating that there is tremendous potential in providing residents with their data. While the ACGME mandates that residents should receive data on their quality metrics, on CLER visits, many residents interviewed noted limited access to data on their metrics and benchmarks.^1,2

Though previous studies have individually looked at teaching QI concepts, providing panel data, or targeting select metrics, our study was unique in that it reviewed both self-reported resident outcomes data as well as actual patient outcomes. In addition to finding increased knowledge of patient panels and comfort with QI approaches, we found a significant increase in colorectal cancer screening rates post intervention. We thought this finding was particularly important given some data that residents' patients have been found to have worse outcomes on quality metrics compared with patients cared for by staff physicians.^14,15 Given that having a resident physician as a PCP has been associated with failing to meet quality measures, it is especially important to focus targeted quality improvement initiatives in this patient population to reduce disparities in care.

We found that residents had improved knowledge on their patient panels as a result of this initiative. The residents were noted to have a higher knowledge of their HTN and colorectal cancer screening rates in comparison to their diabetes metrics. We suspect this is because residents are provided with multiple metrics related to diabetes, including process measures such as A1c testing, as well as outcome measures such as A1c control, so it may be harder for them to elucidate exactly how they are doing with their diabetes patients, whereas in HTN control and colorectal cancer screening, there is only 1 associated metric. Interestingly, even though HTN and colorectal cancer screening were the 2 measures focused on in the study, the residents had a significant improvement in knowledge of the rates of diabetes in their panel as well. This suggests that even just receiving data alone is valuable, hopefully translating to better outcomes with better baseline understanding of panels. We believe that our intervention was successful because it included both a didactic and an experiential component, as well as the use of individual panel performance data.

There were several limitations to our study. It was performed at a single institution, translating to a small sample size. Our data analysis was limited because we were unable to pair our pre- and postintervention survey responses because we used an anonymous survey. We also did not have full participation in postintervention surveys from all residents, which may have biased the study in favor of high performers. Another limitation was that our survey relied on self-reported outcomes for the questions about the residents knowing their patient panels.

This study required a 2-hour workshop every 3 weeks led by a faculty member trained in QI. Given the amount of time needed for the curriculum, this study may be difficult to replicate at other institutions, especially if faculty with an interest or training in QI are not available. Given our finding that residents had increased knowledge of their patient panels after receiving panel metrics, simply providing data with the goal of smaller, focused interventions may be easier to implement. At our institution, we discontinued the longer 2-hour QI workshops designed to teach QI approaches more broadly. We continue to provide individualized panel data to all residents during their primary care rotations and conduct half-hour, small group workshops with the interns that focus on drafting aim statements and planning interventions. All residents are required to submit worksheets to us at the end of their primary care blocks listing their current rates of each predetermined metric and laying out their aim statements and planned interventions. Residents also continue to receive feedback from our faculty with expertise in QI afterward on their plans and evidence of follow-through in the chart, with their preceptors included on the feedback emails. Even without the larger QI workshop, this approach has continued to be successful and appreciated. In fact, it does appear as though improvement in colorectal cancer screening has been sustained over several years. At the end of our study period, the resident patient colorectal cancer screening rate rose from 34% to 43%, and for the 2021-2022 academic year, the rate rose further, from 46% to 50%.

Given that the resident clinic patient population is at higher risk overall, targeted outreach and approaches to improve quality must be continued. Future areas of research include looking at which interventions, whether QI curriculum, provision of panel data, or required panel management interventions, translate to the greatest improvements in patient outcomes in this vulnerable population.

Conclusion

Our study showed that a dedicated QI curriculum for the residents and access to quality metric data improved both resident knowledge and comfort with QI approaches. Beyond resident-centered outcomes, there was also translation to improved patient outcomes, with a significant increase in colon cancer screening rates post intervention.

Corresponding author: Kinjalika Sathi, MD, 800 Washington St., Boston, MA 02111; [email protected]

Disclosures: None reported.

ABSTRACT

Objective: To teach internal medicine residents quality improvement (QI) principles in an effort to improve resident knowledge and comfort with QI, as well as address quality care gaps in resident clinic primary care patient panels.

Design: A QI curriculum was implemented for all residents rotating through a primary care block over a 6-month period. Residents completed Institute for Healthcare Improvement (IHI) modules, participated in a QI workshop, and received panel data reports, ultimately completing a plan-do-study-act (PDSA) cycle to improve colorectal cancer screening and hypertension control.

Setting and participants: This project was undertaken at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. All internal medicine residents were included, with 55 (73%) of the 75 residents completing the presurvey, and 39 (52%) completing the postsurvey.

Measurements: We administered a 10-question pre- and postsurvey looking at resident attitudes toward and comfort with QI and familiarity with their panel data as well as measured rates of colorectal cancer screening and hypertension control in resident panels.

Results: There was an increase in the numbers of residents who performed a PDSA cycle (P = .002), completed outreach based on their panel data (P = .02), and felt comfortable in both creating aim statements and designing and implementing PDSA cycles (P < .0001). The residents’ knowledge of their panel data significantly increased. There was no significant improvement in hypertension control, but there was an increase in colorectal cancer screening rates (P < .0001).

Conclusion: Providing panel data and performing targeted QI interventions can improve resident comfort with QI, translating to improvement in patient outcomes.

Keywords: quality improvement, resident education, medical education, care gaps, quality metrics.

As quality improvement (QI) has become an integral part of clinical practice, residency training programs have continued to evolve in how best to teach QI. The Accreditation Council for Graduate Medical Education (ACGME) Common Program requirements mandate that core competencies in residency programs include practice-based learning and improvement and systems-based practice.¹ Residents should receive education in QI, receive data on quality metrics and benchmarks related to their patient population, and participate in QI activities. The Clinical Learning Environment Review (CLER) program was established to provide feedback to institutions on 6 focused areas, including patient safety and health care quality. In visits to institutions across the United States, the CLER committees found that many residents had limited knowledge of QI concepts and limited access to data on quality metrics and benchmarks.²

There are many barriers to implementing a QI curriculum in residency programs, and creating and maintaining successful strategies has proven challenging.³ Many QI curricula for internal medicine residents have been described in the literature, but the results of many of these studies focus on resident self-assessment of QI knowledge and numbers of projects rather than on patient outcomes.^4-13 As there is some evidence suggesting that patients treated by residents have worse outcomes on ambulatory quality measures when compared with patients treated by staff physicians,^14,15 it is important to also look at patient outcomes when evaluating a QI curriculum. Experts in education recommend the following to optimize learning: exposure to both didactic and experiential opportunities, connection to health system improvement efforts, and assessment of patient outcomes in addition to learner feedback.^16,17 A study also found that providing panel data to residents could improve quality metrics.¹⁸

In this study, we sought to investigate the effects of a resident QI intervention during an ambulatory block on both residents’ self-assessments of QI knowledge and attitudes as well as on patient quality metrics.

Methods

Curriculum

We implemented this educational initiative at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. Co-located with the 415-bed academic medical center in downtown Boston, the practice serves more than 40,000 patients, approximately 7000 of whom are cared for by resident primary care physicians (PCPs). The internal medicine residents rotate through the primary care clinic as part of continuity clinic during ambulatory or elective blocks. In addition to continuity clinic, the residents have 2 dedicated 3-week primary care rotations during the course of an academic year. Primary care rotations consist of 5 clinic sessions a week as well as structured teaching sessions. Each resident inherits a panel of patients from an outgoing senior resident, with an average panel size of 96 patients per resident.

Prior to this study intervention, we did not do any formal QI teaching to our residents as part of their primary care curriculum, and previous panel management had focused more on chart reviews of patients whom residents perceived to be higher risk. Residents from all 3 years were included in the intervention. We taught a QI curriculum to our residents from January 2018 to June 2018 during the 3-week primary care rotation, which consisted of the following components:

• Institute for Healthcare Improvement (IHI) module QI 102 completed independently online.
• A 2-hour QI workshop led by 1 of 2 primary care faculty with backgrounds in QI, during which residents were taught basic principles of QI, including how to craft aim statements and design plan-do-study-act (PDSA) cycles, and participated in a hands-on QI activity designed to model rapid cycle improvement (the Paper Airplane Factory¹⁹).
• Distribution of individualized reports of residents’ patient panel data by email at the start of the primary care block that detailed patients’ overall rates of colorectal cancer screening and hypertension (HTN) control, along with the average resident panel rates and the average attending panel rates. The reports also included a list of all residents’ patients who were overdue for colorectal cancer screening or whose last blood pressure (BP) was uncontrolled (systolic BP ≥ 140 mm Hg or diastolic BP ≥  90 mm Hg). These reports were originally designed by our practice’s QI team and run and exported in Microsoft Excel format monthly by our information technology (IT) administrator.
• Instruction on aim statements as a group, followed by the expectation that each resident create an individualized aim statement tailored to each resident’s patient panel rates, with the PDSA cycle to be implemented during the remainder of the primary care rotation, focusing on improvement of colorectal cancer screening and HTN control (see supplementary eFigure 1 online for the worksheet used for the workshop).
• Residents were held accountable for their interventions by various check-ins. At the end of the primary care block, residents were required to submit their completed worksheets showing the intervention they had undertaken and when it was performed. The 2 primary care attendings primarily responsible for QI education would review the resident’s work approximately 1 to 2 months after they submitted their worksheets describing their intervention. These attendings sent the residents personalized feedback based on whether the intervention had been completed or successful as evidenced by documentation in the chart, including direct patient outreach by phone, letter, or portal; outreach to the resident coordinator; scheduled follow-up appointment; or booking or completion of colorectal cancer screening. Along with this feedback, residents were also sent suggestions for next steps. Resident preceptors were copied on the email to facilitate reinforcement of the goals and plans. Finally, the resident preceptors also helped with accountability by going through the residents’ worksheets and patient panel metrics with the residents during biannual evaluations.

Evaluation

Residents were surveyed with a 10-item questionnaire pre and post intervention regarding their attitudes toward QI, understanding of QI principles, and familiarity with their patient panel data. Surveys were anonymous and distributed via the SurveyMonkey platform (see supplementary eFigure 2 online). Residents were asked if they had ever performed a PDSA cycle, performed patient outreach, or performed an intervention and whether they knew the rates of diabetes, HTN, and colorectal cancer screening in their patient panels. Questions rated on a 5-point Likert scale were used to assess comfort with panel management, developing an aim statement, designing and implementing a PDSA cycle, as well as interest in pursuing QI as a career. For the purposes of analysis, these questions were dichotomized into “somewhat comfortable” and “very comfortable” vs “neutral,” “somewhat uncomfortable,” and “very uncomfortable.” Similarly, we dichotomized the question about interest in QI as a career into “somewhat interested” and “very interested” vs “neutral,” “somewhat disinterested,” and “very disinterested.” As the surveys were anonymous, we were unable to pair the pre- and postintervention surveys and used a chi-square test to evaluate whether there was an association between survey assessments pre intervention vs post intervention and a positive or negative response to the question.

We also examined rates of HTN control and colorectal cancer screening in all 75 resident panels pre and post intervention. The paired t-test was used to determine whether the mean change from pre to post intervention was significant. SAS 9.4 (SAS Institute Inc.) was used for all analyses. Institutional Review Board exemption was obtained from the Tufts Medical Center IRB. There was no funding received for this study.

Respondents

Of the 75 residents, 55 (73%) completed the survey prior to the intervention, and 39 (52%) completed the survey after the intervention.

Panel Knowledge and Intervention

Prior to the intervention, 45% of residents had performed a PDSA cycle, compared with 77% post intervention, which was a significant increase (P = .002) (Table 1). Sixty-two percent of residents had performed outreach or an intervention based on their patient panel reports prior to the intervention, compared with 85% of residents post intervention, which was also a significant increase (P = .02). The increase post intervention was not 100%, as there were residents who either missed the initial workshop or who did not follow through with their planned intervention. Common interventions included the residents giving their coordinators a list of patients to call to schedule appointments, utilizing fellow team members (eg, pharmacists, social workers) for targeted patient outreach, or calling patients themselves to reestablish a connection.

In terms of knowledge of their patient panels, prior to the intervention, 55%, 62%, and 62% of residents knew the rates of patients in their panel with diabetes, HTN, and colorectal cancer screening, respectively. After the intervention, the residents’ knowledge of these rates increased significantly, to 85% for diabetes (P = .002), 97% for HTN (P < .0001), and 97% for colorectal cancer screening (P < .0001).

Comfort With QI Approaches

Prior to the intervention, 82% of residents were comfortable managing their primary care panel, which did not change significantly post intervention (Table 2). The residents’ comfort with designing an aim statement did significantly increase, from 55% to 95% (P < .0001). The residents also had a significant increase in comfort with both designing and implementing a PDSA cycle. Prior to the intervention, 22% felt comfortable designing a PDSA cycle, which increased to 79% (P < .0001) post intervention, and 24% felt comfortable implementing a PDSA cycle, which increased to 77% (P < .0001) post intervention.

Patient Outcome Measures

The rate of HTN control in the residents' patient panels did not change significantly pre and post intervention (Table 3). The rate of resident patients who were up to date with colorectal cancer screening increased by 6.5% post intervention (P < .0001).

Interest in QI as a Career

As part of the survey, residents were asked how interested they were in making QI a part of their career. Fifty percent of residents indicated an interest in QI pre intervention, and 54% indicated an interest post intervention, which was not a significant difference (P = .72).

Discussion

In this study, we found that integration of a QI curriculum into a primary care rotation improved both residents’ knowledge of their patient panels and comfort with QI approaches, which translated to improvement in patient outcomes. Several previous studies have found improvements in resident self-assessment or knowledge after implementation of a QI curriculum.^4-13 Liao et al implemented a longitudinal curriculum including both didactic and experiential components and found an improvement in both QI confidence and knowledge.³ Similarly, Duello et al⁸ found that a curriculum including both didactic lectures and QI projects improved subjective QI knowledge and comfort. Interestingly, Fok and Wong⁹ found that resident knowledge could be sustained post curriculum after completion of a QI project, suggesting that experiential learning may be helpful in maintaining knowledge.

Studies also have looked at providing performance data to residents. Hwang et al¹⁸ found that providing audit and feedback in the form of individual panel performance data to residents compared with practice targets led to statistically significant improvement in cancer screening rates and composite quality score, indicating that there is tremendous potential in providing residents with their data. While the ACGME mandates that residents should receive data on their quality metrics, on CLER visits, many residents interviewed noted limited access to data on their metrics and benchmarks.^1,2

Though previous studies have individually looked at teaching QI concepts, providing panel data, or targeting select metrics, our study was unique in that it reviewed both self-reported resident outcomes data as well as actual patient outcomes. In addition to finding increased knowledge of patient panels and comfort with QI approaches, we found a significant increase in colorectal cancer screening rates post intervention. We thought this finding was particularly important given some data that residents' patients have been found to have worse outcomes on quality metrics compared with patients cared for by staff physicians.^14,15 Given that having a resident physician as a PCP has been associated with failing to meet quality measures, it is especially important to focus targeted quality improvement initiatives in this patient population to reduce disparities in care.

We found that residents had improved knowledge on their patient panels as a result of this initiative. The residents were noted to have a higher knowledge of their HTN and colorectal cancer screening rates in comparison to their diabetes metrics. We suspect this is because residents are provided with multiple metrics related to diabetes, including process measures such as A1c testing, as well as outcome measures such as A1c control, so it may be harder for them to elucidate exactly how they are doing with their diabetes patients, whereas in HTN control and colorectal cancer screening, there is only 1 associated metric. Interestingly, even though HTN and colorectal cancer screening were the 2 measures focused on in the study, the residents had a significant improvement in knowledge of the rates of diabetes in their panel as well. This suggests that even just receiving data alone is valuable, hopefully translating to better outcomes with better baseline understanding of panels. We believe that our intervention was successful because it included both a didactic and an experiential component, as well as the use of individual panel performance data.

There were several limitations to our study. It was performed at a single institution, translating to a small sample size. Our data analysis was limited because we were unable to pair our pre- and postintervention survey responses because we used an anonymous survey. We also did not have full participation in postintervention surveys from all residents, which may have biased the study in favor of high performers. Another limitation was that our survey relied on self-reported outcomes for the questions about the residents knowing their patient panels.

This study required a 2-hour workshop every 3 weeks led by a faculty member trained in QI. Given the amount of time needed for the curriculum, this study may be difficult to replicate at other institutions, especially if faculty with an interest or training in QI are not available. Given our finding that residents had increased knowledge of their patient panels after receiving panel metrics, simply providing data with the goal of smaller, focused interventions may be easier to implement. At our institution, we discontinued the longer 2-hour QI workshops designed to teach QI approaches more broadly. We continue to provide individualized panel data to all residents during their primary care rotations and conduct half-hour, small group workshops with the interns that focus on drafting aim statements and planning interventions. All residents are required to submit worksheets to us at the end of their primary care blocks listing their current rates of each predetermined metric and laying out their aim statements and planned interventions. Residents also continue to receive feedback from our faculty with expertise in QI afterward on their plans and evidence of follow-through in the chart, with their preceptors included on the feedback emails. Even without the larger QI workshop, this approach has continued to be successful and appreciated. In fact, it does appear as though improvement in colorectal cancer screening has been sustained over several years. At the end of our study period, the resident patient colorectal cancer screening rate rose from 34% to 43%, and for the 2021-2022 academic year, the rate rose further, from 46% to 50%.

Given that the resident clinic patient population is at higher risk overall, targeted outreach and approaches to improve quality must be continued. Future areas of research include looking at which interventions, whether QI curriculum, provision of panel data, or required panel management interventions, translate to the greatest improvements in patient outcomes in this vulnerable population.

Conclusion

Our study showed that a dedicated QI curriculum for the residents and access to quality metric data improved both resident knowledge and comfort with QI approaches. Beyond resident-centered outcomes, there was also translation to improved patient outcomes, with a significant increase in colon cancer screening rates post intervention.

Corresponding author: Kinjalika Sathi, MD, 800 Washington St., Boston, MA 02111; [email protected]

Disclosures: None reported.

ABSTRACT

Objective: To teach internal medicine residents quality improvement (QI) principles in an effort to improve resident knowledge and comfort with QI, as well as address quality care gaps in resident clinic primary care patient panels.

Design: A QI curriculum was implemented for all residents rotating through a primary care block over a 6-month period. Residents completed Institute for Healthcare Improvement (IHI) modules, participated in a QI workshop, and received panel data reports, ultimately completing a plan-do-study-act (PDSA) cycle to improve colorectal cancer screening and hypertension control.

Setting and participants: This project was undertaken at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. All internal medicine residents were included, with 55 (73%) of the 75 residents completing the presurvey, and 39 (52%) completing the postsurvey.

Measurements: We administered a 10-question pre- and postsurvey looking at resident attitudes toward and comfort with QI and familiarity with their panel data as well as measured rates of colorectal cancer screening and hypertension control in resident panels.

Results: There was an increase in the numbers of residents who performed a PDSA cycle (P = .002), completed outreach based on their panel data (P = .02), and felt comfortable in both creating aim statements and designing and implementing PDSA cycles (P < .0001). The residents’ knowledge of their panel data significantly increased. There was no significant improvement in hypertension control, but there was an increase in colorectal cancer screening rates (P < .0001).

Conclusion: Providing panel data and performing targeted QI interventions can improve resident comfort with QI, translating to improvement in patient outcomes.

Keywords: quality improvement, resident education, medical education, care gaps, quality metrics.

As quality improvement (QI) has become an integral part of clinical practice, residency training programs have continued to evolve in how best to teach QI. The Accreditation Council for Graduate Medical Education (ACGME) Common Program requirements mandate that core competencies in residency programs include practice-based learning and improvement and systems-based practice.¹ Residents should receive education in QI, receive data on quality metrics and benchmarks related to their patient population, and participate in QI activities. The Clinical Learning Environment Review (CLER) program was established to provide feedback to institutions on 6 focused areas, including patient safety and health care quality. In visits to institutions across the United States, the CLER committees found that many residents had limited knowledge of QI concepts and limited access to data on quality metrics and benchmarks.²

There are many barriers to implementing a QI curriculum in residency programs, and creating and maintaining successful strategies has proven challenging.³ Many QI curricula for internal medicine residents have been described in the literature, but the results of many of these studies focus on resident self-assessment of QI knowledge and numbers of projects rather than on patient outcomes.^4-13 As there is some evidence suggesting that patients treated by residents have worse outcomes on ambulatory quality measures when compared with patients treated by staff physicians,^14,15 it is important to also look at patient outcomes when evaluating a QI curriculum. Experts in education recommend the following to optimize learning: exposure to both didactic and experiential opportunities, connection to health system improvement efforts, and assessment of patient outcomes in addition to learner feedback.^16,17 A study also found that providing panel data to residents could improve quality metrics.¹⁸

In this study, we sought to investigate the effects of a resident QI intervention during an ambulatory block on both residents’ self-assessments of QI knowledge and attitudes as well as on patient quality metrics.

Methods

Curriculum

We implemented this educational initiative at Tufts Medical Center Primary Care, Boston, Massachusetts, the primary care teaching practice for all 75 internal medicine residents at Tufts Medical Center. Co-located with the 415-bed academic medical center in downtown Boston, the practice serves more than 40,000 patients, approximately 7000 of whom are cared for by resident primary care physicians (PCPs). The internal medicine residents rotate through the primary care clinic as part of continuity clinic during ambulatory or elective blocks. In addition to continuity clinic, the residents have 2 dedicated 3-week primary care rotations during the course of an academic year. Primary care rotations consist of 5 clinic sessions a week as well as structured teaching sessions. Each resident inherits a panel of patients from an outgoing senior resident, with an average panel size of 96 patients per resident.

Prior to this study intervention, we did not do any formal QI teaching to our residents as part of their primary care curriculum, and previous panel management had focused more on chart reviews of patients whom residents perceived to be higher risk. Residents from all 3 years were included in the intervention. We taught a QI curriculum to our residents from January 2018 to June 2018 during the 3-week primary care rotation, which consisted of the following components:

• Institute for Healthcare Improvement (IHI) module QI 102 completed independently online.
• A 2-hour QI workshop led by 1 of 2 primary care faculty with backgrounds in QI, during which residents were taught basic principles of QI, including how to craft aim statements and design plan-do-study-act (PDSA) cycles, and participated in a hands-on QI activity designed to model rapid cycle improvement (the Paper Airplane Factory¹⁹).
• Distribution of individualized reports of residents’ patient panel data by email at the start of the primary care block that detailed patients’ overall rates of colorectal cancer screening and hypertension (HTN) control, along with the average resident panel rates and the average attending panel rates. The reports also included a list of all residents’ patients who were overdue for colorectal cancer screening or whose last blood pressure (BP) was uncontrolled (systolic BP ≥ 140 mm Hg or diastolic BP ≥  90 mm Hg). These reports were originally designed by our practice’s QI team and run and exported in Microsoft Excel format monthly by our information technology (IT) administrator.
• Instruction on aim statements as a group, followed by the expectation that each resident create an individualized aim statement tailored to each resident’s patient panel rates, with the PDSA cycle to be implemented during the remainder of the primary care rotation, focusing on improvement of colorectal cancer screening and HTN control (see supplementary eFigure 1 online for the worksheet used for the workshop).
• Residents were held accountable for their interventions by various check-ins. At the end of the primary care block, residents were required to submit their completed worksheets showing the intervention they had undertaken and when it was performed. The 2 primary care attendings primarily responsible for QI education would review the resident’s work approximately 1 to 2 months after they submitted their worksheets describing their intervention. These attendings sent the residents personalized feedback based on whether the intervention had been completed or successful as evidenced by documentation in the chart, including direct patient outreach by phone, letter, or portal; outreach to the resident coordinator; scheduled follow-up appointment; or booking or completion of colorectal cancer screening. Along with this feedback, residents were also sent suggestions for next steps. Resident preceptors were copied on the email to facilitate reinforcement of the goals and plans. Finally, the resident preceptors also helped with accountability by going through the residents’ worksheets and patient panel metrics with the residents during biannual evaluations.

Evaluation

Residents were surveyed with a 10-item questionnaire pre and post intervention regarding their attitudes toward QI, understanding of QI principles, and familiarity with their patient panel data. Surveys were anonymous and distributed via the SurveyMonkey platform (see supplementary eFigure 2 online). Residents were asked if they had ever performed a PDSA cycle, performed patient outreach, or performed an intervention and whether they knew the rates of diabetes, HTN, and colorectal cancer screening in their patient panels. Questions rated on a 5-point Likert scale were used to assess comfort with panel management, developing an aim statement, designing and implementing a PDSA cycle, as well as interest in pursuing QI as a career. For the purposes of analysis, these questions were dichotomized into “somewhat comfortable” and “very comfortable” vs “neutral,” “somewhat uncomfortable,” and “very uncomfortable.” Similarly, we dichotomized the question about interest in QI as a career into “somewhat interested” and “very interested” vs “neutral,” “somewhat disinterested,” and “very disinterested.” As the surveys were anonymous, we were unable to pair the pre- and postintervention surveys and used a chi-square test to evaluate whether there was an association between survey assessments pre intervention vs post intervention and a positive or negative response to the question.

We also examined rates of HTN control and colorectal cancer screening in all 75 resident panels pre and post intervention. The paired t-test was used to determine whether the mean change from pre to post intervention was significant. SAS 9.4 (SAS Institute Inc.) was used for all analyses. Institutional Review Board exemption was obtained from the Tufts Medical Center IRB. There was no funding received for this study.

Respondents

Of the 75 residents, 55 (73%) completed the survey prior to the intervention, and 39 (52%) completed the survey after the intervention.

Panel Knowledge and Intervention

Prior to the intervention, 45% of residents had performed a PDSA cycle, compared with 77% post intervention, which was a significant increase (P = .002) (Table 1). Sixty-two percent of residents had performed outreach or an intervention based on their patient panel reports prior to the intervention, compared with 85% of residents post intervention, which was also a significant increase (P = .02). The increase post intervention was not 100%, as there were residents who either missed the initial workshop or who did not follow through with their planned intervention. Common interventions included the residents giving their coordinators a list of patients to call to schedule appointments, utilizing fellow team members (eg, pharmacists, social workers) for targeted patient outreach, or calling patients themselves to reestablish a connection.

In terms of knowledge of their patient panels, prior to the intervention, 55%, 62%, and 62% of residents knew the rates of patients in their panel with diabetes, HTN, and colorectal cancer screening, respectively. After the intervention, the residents’ knowledge of these rates increased significantly, to 85% for diabetes (P = .002), 97% for HTN (P < .0001), and 97% for colorectal cancer screening (P < .0001).

Comfort With QI Approaches

Prior to the intervention, 82% of residents were comfortable managing their primary care panel, which did not change significantly post intervention (Table 2). The residents’ comfort with designing an aim statement did significantly increase, from 55% to 95% (P < .0001). The residents also had a significant increase in comfort with both designing and implementing a PDSA cycle. Prior to the intervention, 22% felt comfortable designing a PDSA cycle, which increased to 79% (P < .0001) post intervention, and 24% felt comfortable implementing a PDSA cycle, which increased to 77% (P < .0001) post intervention.

Patient Outcome Measures

The rate of HTN control in the residents' patient panels did not change significantly pre and post intervention (Table 3). The rate of resident patients who were up to date with colorectal cancer screening increased by 6.5% post intervention (P < .0001).

Interest in QI as a Career

As part of the survey, residents were asked how interested they were in making QI a part of their career. Fifty percent of residents indicated an interest in QI pre intervention, and 54% indicated an interest post intervention, which was not a significant difference (P = .72).

Discussion

In this study, we found that integration of a QI curriculum into a primary care rotation improved both residents’ knowledge of their patient panels and comfort with QI approaches, which translated to improvement in patient outcomes. Several previous studies have found improvements in resident self-assessment or knowledge after implementation of a QI curriculum.^4-13 Liao et al implemented a longitudinal curriculum including both didactic and experiential components and found an improvement in both QI confidence and knowledge.³ Similarly, Duello et al⁸ found that a curriculum including both didactic lectures and QI projects improved subjective QI knowledge and comfort. Interestingly, Fok and Wong⁹ found that resident knowledge could be sustained post curriculum after completion of a QI project, suggesting that experiential learning may be helpful in maintaining knowledge.

Studies also have looked at providing performance data to residents. Hwang et al¹⁸ found that providing audit and feedback in the form of individual panel performance data to residents compared with practice targets led to statistically significant improvement in cancer screening rates and composite quality score, indicating that there is tremendous potential in providing residents with their data. While the ACGME mandates that residents should receive data on their quality metrics, on CLER visits, many residents interviewed noted limited access to data on their metrics and benchmarks.^1,2

Though previous studies have individually looked at teaching QI concepts, providing panel data, or targeting select metrics, our study was unique in that it reviewed both self-reported resident outcomes data as well as actual patient outcomes. In addition to finding increased knowledge of patient panels and comfort with QI approaches, we found a significant increase in colorectal cancer screening rates post intervention. We thought this finding was particularly important given some data that residents' patients have been found to have worse outcomes on quality metrics compared with patients cared for by staff physicians.^14,15 Given that having a resident physician as a PCP has been associated with failing to meet quality measures, it is especially important to focus targeted quality improvement initiatives in this patient population to reduce disparities in care.

We found that residents had improved knowledge on their patient panels as a result of this initiative. The residents were noted to have a higher knowledge of their HTN and colorectal cancer screening rates in comparison to their diabetes metrics. We suspect this is because residents are provided with multiple metrics related to diabetes, including process measures such as A1c testing, as well as outcome measures such as A1c control, so it may be harder for them to elucidate exactly how they are doing with their diabetes patients, whereas in HTN control and colorectal cancer screening, there is only 1 associated metric. Interestingly, even though HTN and colorectal cancer screening were the 2 measures focused on in the study, the residents had a significant improvement in knowledge of the rates of diabetes in their panel as well. This suggests that even just receiving data alone is valuable, hopefully translating to better outcomes with better baseline understanding of panels. We believe that our intervention was successful because it included both a didactic and an experiential component, as well as the use of individual panel performance data.

There were several limitations to our study. It was performed at a single institution, translating to a small sample size. Our data analysis was limited because we were unable to pair our pre- and postintervention survey responses because we used an anonymous survey. We also did not have full participation in postintervention surveys from all residents, which may have biased the study in favor of high performers. Another limitation was that our survey relied on self-reported outcomes for the questions about the residents knowing their patient panels.

This study required a 2-hour workshop every 3 weeks led by a faculty member trained in QI. Given the amount of time needed for the curriculum, this study may be difficult to replicate at other institutions, especially if faculty with an interest or training in QI are not available. Given our finding that residents had increased knowledge of their patient panels after receiving panel metrics, simply providing data with the goal of smaller, focused interventions may be easier to implement. At our institution, we discontinued the longer 2-hour QI workshops designed to teach QI approaches more broadly. We continue to provide individualized panel data to all residents during their primary care rotations and conduct half-hour, small group workshops with the interns that focus on drafting aim statements and planning interventions. All residents are required to submit worksheets to us at the end of their primary care blocks listing their current rates of each predetermined metric and laying out their aim statements and planned interventions. Residents also continue to receive feedback from our faculty with expertise in QI afterward on their plans and evidence of follow-through in the chart, with their preceptors included on the feedback emails. Even without the larger QI workshop, this approach has continued to be successful and appreciated. In fact, it does appear as though improvement in colorectal cancer screening has been sustained over several years. At the end of our study period, the resident patient colorectal cancer screening rate rose from 34% to 43%, and for the 2021-2022 academic year, the rate rose further, from 46% to 50%.

Given that the resident clinic patient population is at higher risk overall, targeted outreach and approaches to improve quality must be continued. Future areas of research include looking at which interventions, whether QI curriculum, provision of panel data, or required panel management interventions, translate to the greatest improvements in patient outcomes in this vulnerable population.

Conclusion

Our study showed that a dedicated QI curriculum for the residents and access to quality metric data improved both resident knowledge and comfort with QI approaches. Beyond resident-centered outcomes, there was also translation to improved patient outcomes, with a significant increase in colon cancer screening rates post intervention.

Corresponding author: Kinjalika Sathi, MD, 800 Washington St., Boston, MA 02111; [email protected]

Disclosures: None reported.