Program Profile

Three-dimensional (3D) printing has become a promising area of innovation in biomedical research.^1,2 Previous research in orthopedic surgery has found that customized 3D printed implants, casts, orthoses, and prosthetics (eg, prosthetic hands) matched to an individual’s unique anatomy can result in more precise placement and better surgical outcomes.^3-5 Customized prosthetics have also been found to lead to fewer complications.^3,6

Recent advances in 3D printing technology has prompted investigation from surgeons to identify how this new tool may be incorporated into patient care.^1,7 One of the most common applications of 3D printing is during preoperative planning in which surgeons gain better insight into patient-specific anatomy by using patient-specific printed models.⁸ Another promising application is the production of customized prosthetics suited to each patient’s unique anatomy.⁹ As a result, 3D printing has significantly impacted bone and cartilage restoration procedures and has the potential to completely transform the treatment of patients with debilitating musculoskeletal injuries.^3,10

The potential surrounding 3D printed prosthetics has led to their adoption by several other specialties, including otolaryngology.¹¹ The most widely used application of 3D printing among otolaryngologists is preoperative planning, and the incorporation of printed prosthetics intoreconstruction of the orbit, nasal septum, auricle, and palate has also been reported.^2,12,13 Patient-specific implants might allow otolaryngologists to better rehabilitate, reconstruct, and/or regenerate craniofacial defects using more humane procedures.¹⁴

Patients with palatomaxillary cancers are treated by prosthodontists or otolaryngologists. An impression is made with a resin–which can be painful for postoperative patients–and a prosthetic is manufactured and implanted.^15-17 Patients with cancer often see many specialists, though reconstructive care is a low priority. Many of these individuals also experience dynamic anatomic functional changes over time, leading to the need for multiple prothesis.

palatomaxillary prosthetics

This program aims to use patients’ previous computed tomography (CT) to tailor customized 3D printed palatomaxillary prosthetics to specifically fit their anatomy. Palatomaxillary defects are a source of profound disability for patients with head and neck cancers who are left with large anatomic defects as a direct result of treatment. Reconstruction of palatal defects poses unique challenges due to the complexity of patient anatomy.^18,19

3D printed prosthetics for palatomaxillary defects have not been incorporated into patient care. We reviewed previous imaging research to determine if it could be used to assist patients who struggle with their function and appearance following treatment for head and neck cancers. The primary aim was to investigate whether 3D printing was a feasible strategy for creating patient-specific palatomaxillary prosthetics. The secondary aim is to determine whether these prosthetics should be tested in the future for use in reconstruction of maxillary defects.

This study was conducted at the Veterans Affairs Palo Alto Health Care System (VAPAHCS) and was approved by the Stanford University Institutional Review Board (approval #28958, informed consent and patient contact excluded). A retrospective chart review was conducted on all patients with head and neck cancers who were treated at VAPAHCS from 2010 to 2022. Patients aged ≥ 18 years who had a palatomaxillary defect due to cancer treatment, had undergone a palatal resection, and who received treatment at any point from 2010 to 2022 were included in the review. CTs were not a specific inclusion criterion, though the quality of the scans was analyzed for eligible patients. Younger patients and those treated at VAPAHCS prior to 2010 were excluded.

There was no control group; all data was sourced from the US Department of Veterans Affairs (VA) imaging system database. Among the 3595 patients reviewed, 5 met inclusion criteria and the quality of their craniofacial anatomy CTs were analyzed. To maintain accurate craniofacial 3D modeling, CTs require a maximum of 1 mm slice thickness. Of the 5 patients who met the inclusion criteria, 4 were found to have variability in the quality of their CTs and severe defects not suitable for prosthetic reconstruction, which led to their exclusion from the study. One patient was investigated to demonstrate if making these prostheses was feasible. This patient was diagnosed with a malignant neoplasm of the hard palate, underwent a partial maxillectomy, and a palatal obturator was placed to cover the defect.

The primary data collected was patient identifiers as well as the gross anatomy and dimensions of the patients’ craniofacial anatomy, as seen in previous imaging research.²⁰ Before the imaging analysis, all personal health information was removed and the dataset was deidentified to ensure patient anonymity and noninvolvement.

CT Segmentation and 3D Printing

Using CTs of the patient’s craniofacial anatomy, we developed a model of the defects. This was achieved with deidentified CTs imported into the Food and Drug Administration (FDA)-approved computerized aid design (CAD) software, Materialise Mimics. The hard palate was segmented and isolated based off the presented scan and any holes in the image were filled using the CAD software. The model was subsequently mirrored in Materialise 3-matic to replicate an original anatomical hard palate prosthesis. The final product was converted into a 3D model and imported into Formlabs preform software to generate 3D printing supports and orient it for printing. The prosthetic was printed using FDA-approved Biocompatible Denture Base Resin by a Formlabs 3B+ printer at the Palo Alto VA Simulation Center. The 3D printed prosthesis was washed using Formlabs Form Wash 80% ethyl alcohol to remove excess resin and subsequently cured to harden the malleable resin. Supports were later removed, and the prosthesis was sanded.

The primary aim of this study was to investigate whether using CTs to create patient-specific prosthetic renderings for patients with head and neck cancer could be a feasible strategy. The CTs from the patient were successfully used to generate a 3D printed prosthesis, and the prosthesis matched the original craniofacial anatomy seen in the patient's imaging (Figure). These results demonstrate that high quality CTs can be used as a template for 3D printed prostheses for mild to moderate palatomaxillary defects.

3D Printing Costs

One liter of Denture Base Resin costs $299; prostheses use about 5 mL of resin. The average annual salary of a 3D printing technician in the United States is $42,717, or $20.54 per hour.²¹ For an experienced 3D printing technician, the time required to segment the hard palate and prepare it for 3D printing is 1 to 2 hours. The process may exceed 2 hours if the technician is presented with a lower quality CT or if the patient has a complex craniofacial anatomy.

The average time it takes to print a palatal prosthetic is 5 hours. An additional hour is needed for postprocessing, which includes washing and sanding. Therefore, the cost of the materials and labor for an average 3D printed prosthetic is about $150. A Formlabs 3B+ printer is competitively priced around $10,000. The cost for Materialise Mimics software varies, but is estimated at $16,000 at VAPAHCS. The prices for these 2 items are not included in our price estimation but should be taken into consideration.

Prosthodontist Process and Cost

The typical process of creating a palatal prosthesis by a prosthodontist begins by examining the patient, creating a stone model, then creating a wax model. Biocompatible materials are selected and processed into a mold that is trimmed and polished to the desired shape. This is followed by another patient visit to ensure the prosthesis fits properly. Follow-up care is also necessary for maintenance and comfort.

The average cost of a palatal prosthesis varies depending on the type needed (ie, metal implant, teeth replacement), the materials used, the region in which the patient is receiving care, and the complexity of the case. For complex and customizable options like those required for patients with cancer, the prostheses typically cost several thousands of dollars. The Healthcare Common Procedure Coding System code for a palatal lift prosthesis (D5955) lists prices ranging from $4000 to $8000 per prosthetic, not including the cost of the prosthodontist visits.^22,23

This program sought to determine whether imaging studies of maxillary defects are effective templates for developing 3D printed prosthetics and whether these prosthetics should be tested for future use in reconstruction of palatomaxillary defects. Our program illustrated that CTs served as feasible templates for developing hard palate prostheses for patients with palatomaxillary defects. It is important to note the CTs used were from a newer and more modern scanner and therefore yielded detailed palatal structures with higher accuracy more suitable for 3D modeling. Lower-quality CTs from the 4 patients excluded from the program were not suitable for 3D modeling. This suggests that with high-quality imaging, 3D printed prosthesis may be a viable strategy to help patients who struggle with their function following treatment for head and neck cancers.

3D printed prosthesis may also be a more patient centered and convenient option. In the traditional prosthesis creation workflow, the patient must physically bite down onto a resin (alginate or silicone) to make an impression, a very painful postoperative process that is irritating to the raw edges of the surgical bed.^15,16 Prosthodontists then create a prosthetic minus the tumor and typically secure it with clips or glue.¹⁷ Many patients also experience changes in their anatomy over time requiring them to have a new protheses created. This is particularly important in veterans with palatomaxillary defects since many VA medical centers do not have a prosthodontist on staff, making accessibility to these specialists difficult. 3D printing provides a contactless prosthetic creation process. This convenience may reduce a patient’s pain and the number of visits for which they need a specialist.

Future Directions

Additional research is needed to determine the full potential of 3D printed prosthetics. 3D printed prostheses have been effectively used for patient education in areas of presurgical planning, prosthesis creation, and trainee education.²⁴ This research represents an early step in the development of a new technology for use in otolaryngology. Specifically, many veterans with a history of head and neck cancers have sustained changes to their craniofacial anatomy following treatment. Using imaging to create 3D printed prosthetics could be very effective for these patients. Prosthetics could improve a patient’s quality of life by restoring/approximating their anatomy after cancer treatment.

Significant time and care must be taken by cancer and reconstructive surgeons to properly fit a prosthesis. Improperly fitting prosthetics leads to mucosal ulceration that then may lead to a need for fitting a new prosthetic. The advantage of 3D printed prosthetics is that they may more precisely fit the anatomy of each patient using CT results, thus potentially reducing the time needed to fit the prosthetic as well as the risk associated with an improperly fit prosthetic. 3D printed prosthesis could be used directly in the future, however, clinical trials are needed to verify its efficacy vs prosthodontic options.

Another consideration for potential future use of 3D printed prosthetics is cost. We estimated that the cost of the materials and labor of our 3D printed prosthetic to be about $150. Pricing of current molded prosthetics varies, but is often listed at several thousand dollars. Another consideration is the durability of 3D printed prosthetics vs standard prosthetics. Since we were unable to use the prosthetic in the patient, it was difficult to determine its durability. The significant cost of the 3D printer and software necessary for 3D printed prosthetics must also be considered and may be prohibitive. While many academic hospitals are considering the purchase of 3D printers and licenses, this may be challenging for resource-constrained institutions. 3D printing may also be difficult for groups without any prior experience in the field. Outsourcing to a third party is possible, though doing so adds more cost to the project. While we recognize there is a learning curve associated with adopting any new technology, it’s equally important to note that 3D printing is being rapidly integrated and has already made significant advancements in personalized medicine.^8,25,26

Limitations

This program had several limitations. First, we only obtained CTs of sufficient quality from 1 patient to generate a 3D printed prosthesis. Further research with additional patients is necessary to validate this process. Second, we were unable to trial the prosthesis in the patient because we did not have FDA approval. Additionally, it is difficult to calculate a true cost estimate for this process as materials and software costs vary dramatically across institutions as well as over time.

Conclusions

The purpose of this study was to demonstrate the possibility to develop prosthetics for the hard palate for patients suffering from palatomaxillary defects. A 3D printed prosthetic was generated that matched the patient’s craniofacial anatomy. Future research should test the feasibility of these prosthetics in patient care against a traditional prosthodontic impression. Though this is a proof-of-concept study and no prosthetics were implanted as part of this investigation, we showcase the feasibility of printing prosthetics for palatomaxillary defects. The use of 3D printed prosthetics may be a more humane process, potentially lower cost, and be more accessible to veterans.

Three-dimensional (3D) printing has become a promising area of innovation in biomedical research.^1,2 Previous research in orthopedic surgery has found that customized 3D printed implants, casts, orthoses, and prosthetics (eg, prosthetic hands) matched to an individual’s unique anatomy can result in more precise placement and better surgical outcomes.^3-5 Customized prosthetics have also been found to lead to fewer complications.^3,6

Recent advances in 3D printing technology has prompted investigation from surgeons to identify how this new tool may be incorporated into patient care.^1,7 One of the most common applications of 3D printing is during preoperative planning in which surgeons gain better insight into patient-specific anatomy by using patient-specific printed models.⁸ Another promising application is the production of customized prosthetics suited to each patient’s unique anatomy.⁹ As a result, 3D printing has significantly impacted bone and cartilage restoration procedures and has the potential to completely transform the treatment of patients with debilitating musculoskeletal injuries.^3,10

The potential surrounding 3D printed prosthetics has led to their adoption by several other specialties, including otolaryngology.¹¹ The most widely used application of 3D printing among otolaryngologists is preoperative planning, and the incorporation of printed prosthetics intoreconstruction of the orbit, nasal septum, auricle, and palate has also been reported.^2,12,13 Patient-specific implants might allow otolaryngologists to better rehabilitate, reconstruct, and/or regenerate craniofacial defects using more humane procedures.¹⁴

Patients with palatomaxillary cancers are treated by prosthodontists or otolaryngologists. An impression is made with a resin–which can be painful for postoperative patients–and a prosthetic is manufactured and implanted.^15-17 Patients with cancer often see many specialists, though reconstructive care is a low priority. Many of these individuals also experience dynamic anatomic functional changes over time, leading to the need for multiple prothesis.

palatomaxillary prosthetics

This program aims to use patients’ previous computed tomography (CT) to tailor customized 3D printed palatomaxillary prosthetics to specifically fit their anatomy. Palatomaxillary defects are a source of profound disability for patients with head and neck cancers who are left with large anatomic defects as a direct result of treatment. Reconstruction of palatal defects poses unique challenges due to the complexity of patient anatomy.^18,19

3D printed prosthetics for palatomaxillary defects have not been incorporated into patient care. We reviewed previous imaging research to determine if it could be used to assist patients who struggle with their function and appearance following treatment for head and neck cancers. The primary aim was to investigate whether 3D printing was a feasible strategy for creating patient-specific palatomaxillary prosthetics. The secondary aim is to determine whether these prosthetics should be tested in the future for use in reconstruction of maxillary defects.

This study was conducted at the Veterans Affairs Palo Alto Health Care System (VAPAHCS) and was approved by the Stanford University Institutional Review Board (approval #28958, informed consent and patient contact excluded). A retrospective chart review was conducted on all patients with head and neck cancers who were treated at VAPAHCS from 2010 to 2022. Patients aged ≥ 18 years who had a palatomaxillary defect due to cancer treatment, had undergone a palatal resection, and who received treatment at any point from 2010 to 2022 were included in the review. CTs were not a specific inclusion criterion, though the quality of the scans was analyzed for eligible patients. Younger patients and those treated at VAPAHCS prior to 2010 were excluded.

There was no control group; all data was sourced from the US Department of Veterans Affairs (VA) imaging system database. Among the 3595 patients reviewed, 5 met inclusion criteria and the quality of their craniofacial anatomy CTs were analyzed. To maintain accurate craniofacial 3D modeling, CTs require a maximum of 1 mm slice thickness. Of the 5 patients who met the inclusion criteria, 4 were found to have variability in the quality of their CTs and severe defects not suitable for prosthetic reconstruction, which led to their exclusion from the study. One patient was investigated to demonstrate if making these prostheses was feasible. This patient was diagnosed with a malignant neoplasm of the hard palate, underwent a partial maxillectomy, and a palatal obturator was placed to cover the defect.

The primary data collected was patient identifiers as well as the gross anatomy and dimensions of the patients’ craniofacial anatomy, as seen in previous imaging research.²⁰ Before the imaging analysis, all personal health information was removed and the dataset was deidentified to ensure patient anonymity and noninvolvement.

CT Segmentation and 3D Printing

Using CTs of the patient’s craniofacial anatomy, we developed a model of the defects. This was achieved with deidentified CTs imported into the Food and Drug Administration (FDA)-approved computerized aid design (CAD) software, Materialise Mimics. The hard palate was segmented and isolated based off the presented scan and any holes in the image were filled using the CAD software. The model was subsequently mirrored in Materialise 3-matic to replicate an original anatomical hard palate prosthesis. The final product was converted into a 3D model and imported into Formlabs preform software to generate 3D printing supports and orient it for printing. The prosthetic was printed using FDA-approved Biocompatible Denture Base Resin by a Formlabs 3B+ printer at the Palo Alto VA Simulation Center. The 3D printed prosthesis was washed using Formlabs Form Wash 80% ethyl alcohol to remove excess resin and subsequently cured to harden the malleable resin. Supports were later removed, and the prosthesis was sanded.

The primary aim of this study was to investigate whether using CTs to create patient-specific prosthetic renderings for patients with head and neck cancer could be a feasible strategy. The CTs from the patient were successfully used to generate a 3D printed prosthesis, and the prosthesis matched the original craniofacial anatomy seen in the patient's imaging (Figure). These results demonstrate that high quality CTs can be used as a template for 3D printed prostheses for mild to moderate palatomaxillary defects.

3D Printing Costs

One liter of Denture Base Resin costs $299; prostheses use about 5 mL of resin. The average annual salary of a 3D printing technician in the United States is $42,717, or $20.54 per hour.²¹ For an experienced 3D printing technician, the time required to segment the hard palate and prepare it for 3D printing is 1 to 2 hours. The process may exceed 2 hours if the technician is presented with a lower quality CT or if the patient has a complex craniofacial anatomy.

The average time it takes to print a palatal prosthetic is 5 hours. An additional hour is needed for postprocessing, which includes washing and sanding. Therefore, the cost of the materials and labor for an average 3D printed prosthetic is about $150. A Formlabs 3B+ printer is competitively priced around $10,000. The cost for Materialise Mimics software varies, but is estimated at $16,000 at VAPAHCS. The prices for these 2 items are not included in our price estimation but should be taken into consideration.

Prosthodontist Process and Cost

The typical process of creating a palatal prosthesis by a prosthodontist begins by examining the patient, creating a stone model, then creating a wax model. Biocompatible materials are selected and processed into a mold that is trimmed and polished to the desired shape. This is followed by another patient visit to ensure the prosthesis fits properly. Follow-up care is also necessary for maintenance and comfort.

The average cost of a palatal prosthesis varies depending on the type needed (ie, metal implant, teeth replacement), the materials used, the region in which the patient is receiving care, and the complexity of the case. For complex and customizable options like those required for patients with cancer, the prostheses typically cost several thousands of dollars. The Healthcare Common Procedure Coding System code for a palatal lift prosthesis (D5955) lists prices ranging from $4000 to $8000 per prosthetic, not including the cost of the prosthodontist visits.^22,23

This program sought to determine whether imaging studies of maxillary defects are effective templates for developing 3D printed prosthetics and whether these prosthetics should be tested for future use in reconstruction of palatomaxillary defects. Our program illustrated that CTs served as feasible templates for developing hard palate prostheses for patients with palatomaxillary defects. It is important to note the CTs used were from a newer and more modern scanner and therefore yielded detailed palatal structures with higher accuracy more suitable for 3D modeling. Lower-quality CTs from the 4 patients excluded from the program were not suitable for 3D modeling. This suggests that with high-quality imaging, 3D printed prosthesis may be a viable strategy to help patients who struggle with their function following treatment for head and neck cancers.

3D printed prosthesis may also be a more patient centered and convenient option. In the traditional prosthesis creation workflow, the patient must physically bite down onto a resin (alginate or silicone) to make an impression, a very painful postoperative process that is irritating to the raw edges of the surgical bed.^15,16 Prosthodontists then create a prosthetic minus the tumor and typically secure it with clips or glue.¹⁷ Many patients also experience changes in their anatomy over time requiring them to have a new protheses created. This is particularly important in veterans with palatomaxillary defects since many VA medical centers do not have a prosthodontist on staff, making accessibility to these specialists difficult. 3D printing provides a contactless prosthetic creation process. This convenience may reduce a patient’s pain and the number of visits for which they need a specialist.

Future Directions

Additional research is needed to determine the full potential of 3D printed prosthetics. 3D printed prostheses have been effectively used for patient education in areas of presurgical planning, prosthesis creation, and trainee education.²⁴ This research represents an early step in the development of a new technology for use in otolaryngology. Specifically, many veterans with a history of head and neck cancers have sustained changes to their craniofacial anatomy following treatment. Using imaging to create 3D printed prosthetics could be very effective for these patients. Prosthetics could improve a patient’s quality of life by restoring/approximating their anatomy after cancer treatment.

Significant time and care must be taken by cancer and reconstructive surgeons to properly fit a prosthesis. Improperly fitting prosthetics leads to mucosal ulceration that then may lead to a need for fitting a new prosthetic. The advantage of 3D printed prosthetics is that they may more precisely fit the anatomy of each patient using CT results, thus potentially reducing the time needed to fit the prosthetic as well as the risk associated with an improperly fit prosthetic. 3D printed prosthesis could be used directly in the future, however, clinical trials are needed to verify its efficacy vs prosthodontic options.

Another consideration for potential future use of 3D printed prosthetics is cost. We estimated that the cost of the materials and labor of our 3D printed prosthetic to be about $150. Pricing of current molded prosthetics varies, but is often listed at several thousand dollars. Another consideration is the durability of 3D printed prosthetics vs standard prosthetics. Since we were unable to use the prosthetic in the patient, it was difficult to determine its durability. The significant cost of the 3D printer and software necessary for 3D printed prosthetics must also be considered and may be prohibitive. While many academic hospitals are considering the purchase of 3D printers and licenses, this may be challenging for resource-constrained institutions. 3D printing may also be difficult for groups without any prior experience in the field. Outsourcing to a third party is possible, though doing so adds more cost to the project. While we recognize there is a learning curve associated with adopting any new technology, it’s equally important to note that 3D printing is being rapidly integrated and has already made significant advancements in personalized medicine.^8,25,26

Limitations

This program had several limitations. First, we only obtained CTs of sufficient quality from 1 patient to generate a 3D printed prosthesis. Further research with additional patients is necessary to validate this process. Second, we were unable to trial the prosthesis in the patient because we did not have FDA approval. Additionally, it is difficult to calculate a true cost estimate for this process as materials and software costs vary dramatically across institutions as well as over time.

Conclusions

The purpose of this study was to demonstrate the possibility to develop prosthetics for the hard palate for patients suffering from palatomaxillary defects. A 3D printed prosthetic was generated that matched the patient’s craniofacial anatomy. Future research should test the feasibility of these prosthetics in patient care against a traditional prosthodontic impression. Though this is a proof-of-concept study and no prosthetics were implanted as part of this investigation, we showcase the feasibility of printing prosthetics for palatomaxillary defects. The use of 3D printed prosthetics may be a more humane process, potentially lower cost, and be more accessible to veterans.

Three-dimensional (3D) printing has become a promising area of innovation in biomedical research.^1,2 Previous research in orthopedic surgery has found that customized 3D printed implants, casts, orthoses, and prosthetics (eg, prosthetic hands) matched to an individual’s unique anatomy can result in more precise placement and better surgical outcomes.^3-5 Customized prosthetics have also been found to lead to fewer complications.^3,6

Recent advances in 3D printing technology has prompted investigation from surgeons to identify how this new tool may be incorporated into patient care.^1,7 One of the most common applications of 3D printing is during preoperative planning in which surgeons gain better insight into patient-specific anatomy by using patient-specific printed models.⁸ Another promising application is the production of customized prosthetics suited to each patient’s unique anatomy.⁹ As a result, 3D printing has significantly impacted bone and cartilage restoration procedures and has the potential to completely transform the treatment of patients with debilitating musculoskeletal injuries.^3,10

The potential surrounding 3D printed prosthetics has led to their adoption by several other specialties, including otolaryngology.¹¹ The most widely used application of 3D printing among otolaryngologists is preoperative planning, and the incorporation of printed prosthetics intoreconstruction of the orbit, nasal septum, auricle, and palate has also been reported.^2,12,13 Patient-specific implants might allow otolaryngologists to better rehabilitate, reconstruct, and/or regenerate craniofacial defects using more humane procedures.¹⁴

Patients with palatomaxillary cancers are treated by prosthodontists or otolaryngologists. An impression is made with a resin–which can be painful for postoperative patients–and a prosthetic is manufactured and implanted.^15-17 Patients with cancer often see many specialists, though reconstructive care is a low priority. Many of these individuals also experience dynamic anatomic functional changes over time, leading to the need for multiple prothesis.

palatomaxillary prosthetics

This program aims to use patients’ previous computed tomography (CT) to tailor customized 3D printed palatomaxillary prosthetics to specifically fit their anatomy. Palatomaxillary defects are a source of profound disability for patients with head and neck cancers who are left with large anatomic defects as a direct result of treatment. Reconstruction of palatal defects poses unique challenges due to the complexity of patient anatomy.^18,19

3D printed prosthetics for palatomaxillary defects have not been incorporated into patient care. We reviewed previous imaging research to determine if it could be used to assist patients who struggle with their function and appearance following treatment for head and neck cancers. The primary aim was to investigate whether 3D printing was a feasible strategy for creating patient-specific palatomaxillary prosthetics. The secondary aim is to determine whether these prosthetics should be tested in the future for use in reconstruction of maxillary defects.

This study was conducted at the Veterans Affairs Palo Alto Health Care System (VAPAHCS) and was approved by the Stanford University Institutional Review Board (approval #28958, informed consent and patient contact excluded). A retrospective chart review was conducted on all patients with head and neck cancers who were treated at VAPAHCS from 2010 to 2022. Patients aged ≥ 18 years who had a palatomaxillary defect due to cancer treatment, had undergone a palatal resection, and who received treatment at any point from 2010 to 2022 were included in the review. CTs were not a specific inclusion criterion, though the quality of the scans was analyzed for eligible patients. Younger patients and those treated at VAPAHCS prior to 2010 were excluded.

There was no control group; all data was sourced from the US Department of Veterans Affairs (VA) imaging system database. Among the 3595 patients reviewed, 5 met inclusion criteria and the quality of their craniofacial anatomy CTs were analyzed. To maintain accurate craniofacial 3D modeling, CTs require a maximum of 1 mm slice thickness. Of the 5 patients who met the inclusion criteria, 4 were found to have variability in the quality of their CTs and severe defects not suitable for prosthetic reconstruction, which led to their exclusion from the study. One patient was investigated to demonstrate if making these prostheses was feasible. This patient was diagnosed with a malignant neoplasm of the hard palate, underwent a partial maxillectomy, and a palatal obturator was placed to cover the defect.

The primary data collected was patient identifiers as well as the gross anatomy and dimensions of the patients’ craniofacial anatomy, as seen in previous imaging research.²⁰ Before the imaging analysis, all personal health information was removed and the dataset was deidentified to ensure patient anonymity and noninvolvement.

CT Segmentation and 3D Printing

Using CTs of the patient’s craniofacial anatomy, we developed a model of the defects. This was achieved with deidentified CTs imported into the Food and Drug Administration (FDA)-approved computerized aid design (CAD) software, Materialise Mimics. The hard palate was segmented and isolated based off the presented scan and any holes in the image were filled using the CAD software. The model was subsequently mirrored in Materialise 3-matic to replicate an original anatomical hard palate prosthesis. The final product was converted into a 3D model and imported into Formlabs preform software to generate 3D printing supports and orient it for printing. The prosthetic was printed using FDA-approved Biocompatible Denture Base Resin by a Formlabs 3B+ printer at the Palo Alto VA Simulation Center. The 3D printed prosthesis was washed using Formlabs Form Wash 80% ethyl alcohol to remove excess resin and subsequently cured to harden the malleable resin. Supports were later removed, and the prosthesis was sanded.

The primary aim of this study was to investigate whether using CTs to create patient-specific prosthetic renderings for patients with head and neck cancer could be a feasible strategy. The CTs from the patient were successfully used to generate a 3D printed prosthesis, and the prosthesis matched the original craniofacial anatomy seen in the patient's imaging (Figure). These results demonstrate that high quality CTs can be used as a template for 3D printed prostheses for mild to moderate palatomaxillary defects.

3D Printing Costs

One liter of Denture Base Resin costs $299; prostheses use about 5 mL of resin. The average annual salary of a 3D printing technician in the United States is $42,717, or $20.54 per hour.²¹ For an experienced 3D printing technician, the time required to segment the hard palate and prepare it for 3D printing is 1 to 2 hours. The process may exceed 2 hours if the technician is presented with a lower quality CT or if the patient has a complex craniofacial anatomy.

The average time it takes to print a palatal prosthetic is 5 hours. An additional hour is needed for postprocessing, which includes washing and sanding. Therefore, the cost of the materials and labor for an average 3D printed prosthetic is about $150. A Formlabs 3B+ printer is competitively priced around $10,000. The cost for Materialise Mimics software varies, but is estimated at $16,000 at VAPAHCS. The prices for these 2 items are not included in our price estimation but should be taken into consideration.

Prosthodontist Process and Cost

The typical process of creating a palatal prosthesis by a prosthodontist begins by examining the patient, creating a stone model, then creating a wax model. Biocompatible materials are selected and processed into a mold that is trimmed and polished to the desired shape. This is followed by another patient visit to ensure the prosthesis fits properly. Follow-up care is also necessary for maintenance and comfort.

The average cost of a palatal prosthesis varies depending on the type needed (ie, metal implant, teeth replacement), the materials used, the region in which the patient is receiving care, and the complexity of the case. For complex and customizable options like those required for patients with cancer, the prostheses typically cost several thousands of dollars. The Healthcare Common Procedure Coding System code for a palatal lift prosthesis (D5955) lists prices ranging from $4000 to $8000 per prosthetic, not including the cost of the prosthodontist visits.^22,23

This program sought to determine whether imaging studies of maxillary defects are effective templates for developing 3D printed prosthetics and whether these prosthetics should be tested for future use in reconstruction of palatomaxillary defects. Our program illustrated that CTs served as feasible templates for developing hard palate prostheses for patients with palatomaxillary defects. It is important to note the CTs used were from a newer and more modern scanner and therefore yielded detailed palatal structures with higher accuracy more suitable for 3D modeling. Lower-quality CTs from the 4 patients excluded from the program were not suitable for 3D modeling. This suggests that with high-quality imaging, 3D printed prosthesis may be a viable strategy to help patients who struggle with their function following treatment for head and neck cancers.

3D printed prosthesis may also be a more patient centered and convenient option. In the traditional prosthesis creation workflow, the patient must physically bite down onto a resin (alginate or silicone) to make an impression, a very painful postoperative process that is irritating to the raw edges of the surgical bed.^15,16 Prosthodontists then create a prosthetic minus the tumor and typically secure it with clips or glue.¹⁷ Many patients also experience changes in their anatomy over time requiring them to have a new protheses created. This is particularly important in veterans with palatomaxillary defects since many VA medical centers do not have a prosthodontist on staff, making accessibility to these specialists difficult. 3D printing provides a contactless prosthetic creation process. This convenience may reduce a patient’s pain and the number of visits for which they need a specialist.

Future Directions

Additional research is needed to determine the full potential of 3D printed prosthetics. 3D printed prostheses have been effectively used for patient education in areas of presurgical planning, prosthesis creation, and trainee education.²⁴ This research represents an early step in the development of a new technology for use in otolaryngology. Specifically, many veterans with a history of head and neck cancers have sustained changes to their craniofacial anatomy following treatment. Using imaging to create 3D printed prosthetics could be very effective for these patients. Prosthetics could improve a patient’s quality of life by restoring/approximating their anatomy after cancer treatment.

Significant time and care must be taken by cancer and reconstructive surgeons to properly fit a prosthesis. Improperly fitting prosthetics leads to mucosal ulceration that then may lead to a need for fitting a new prosthetic. The advantage of 3D printed prosthetics is that they may more precisely fit the anatomy of each patient using CT results, thus potentially reducing the time needed to fit the prosthetic as well as the risk associated with an improperly fit prosthetic. 3D printed prosthesis could be used directly in the future, however, clinical trials are needed to verify its efficacy vs prosthodontic options.

Another consideration for potential future use of 3D printed prosthetics is cost. We estimated that the cost of the materials and labor of our 3D printed prosthetic to be about $150. Pricing of current molded prosthetics varies, but is often listed at several thousand dollars. Another consideration is the durability of 3D printed prosthetics vs standard prosthetics. Since we were unable to use the prosthetic in the patient, it was difficult to determine its durability. The significant cost of the 3D printer and software necessary for 3D printed prosthetics must also be considered and may be prohibitive. While many academic hospitals are considering the purchase of 3D printers and licenses, this may be challenging for resource-constrained institutions. 3D printing may also be difficult for groups without any prior experience in the field. Outsourcing to a third party is possible, though doing so adds more cost to the project. While we recognize there is a learning curve associated with adopting any new technology, it’s equally important to note that 3D printing is being rapidly integrated and has already made significant advancements in personalized medicine.^8,25,26

Limitations

This program had several limitations. First, we only obtained CTs of sufficient quality from 1 patient to generate a 3D printed prosthesis. Further research with additional patients is necessary to validate this process. Second, we were unable to trial the prosthesis in the patient because we did not have FDA approval. Additionally, it is difficult to calculate a true cost estimate for this process as materials and software costs vary dramatically across institutions as well as over time.

Conclusions

The purpose of this study was to demonstrate the possibility to develop prosthetics for the hard palate for patients suffering from palatomaxillary defects. A 3D printed prosthetic was generated that matched the patient’s craniofacial anatomy. Future research should test the feasibility of these prosthetics in patient care against a traditional prosthodontic impression. Though this is a proof-of-concept study and no prosthetics were implanted as part of this investigation, we showcase the feasibility of printing prosthetics for palatomaxillary defects. The use of 3D printed prosthetics may be a more humane process, potentially lower cost, and be more accessible to veterans.

Colorectal cancer (CRC) is the third-most common cancer worldwide and accounts for almost 11% of all cancer diagnoses, with > 1.9 million cases reported globally.^1,2 CRC is the second-most deadly cancer, responsible for about 935,000 deaths.¹ Over the past several decades, a steady decline in CRC incidence and mortality has been reported in developed countries, including the US.^3,4 From 2008 through 2017, an annual reduction of 3% in CRC death rates was reported in individuals aged ≥ 65 years.⁵ This decline can mainly be attributed to improvements made in health systems and advancements in CRC screening programs.^3,5

US Preventive Services Task Force (USPSTF) recommends CRC screening in individuals aged 45 to 75 years. USPSTF recommends direct visualization tests, such as colonoscopy and flexible sigmoidoscopy for CRC screening.⁶ Although colonoscopy is commonly used for CRC screening, it is an invasive procedure that requires bowel preparation and sedation, and has the potential risk of colonic perforation, bleeding, and infection. Additionally, social determinants—such as health care costs, missed work, and geographic location (eg, rural communities)—may limit colonoscopy utilization.⁷ As a result, other cost-effective, noninvasive tests such as high-sensitivity guaiac-based fecal occult blood test (gFOBT) and fecal immunochemical test (FIT) are also used for CRC screening. These tests detect occult blood in the stool of individuals who may be at risk for CRC, helping direct them to colonoscopy if they screen positive.⁸

The gFOBT relies on simple oxidation and requires a stool sample to detect the presence of the heme component of blood.⁹ If heme is present in the stool sample, it will enable the oxidation of guaiac to form a blue-colored dye when added to hydrogen peroxide. It is important to note that the oxidation component of this test may lead to false-positive results, as it may detect dietary hemoglobin present in red meat. Medications or foods that have peroxidase properties may also result in a false-positive gFOBT result. Additionally, false-negative results may be caused by antioxidants, which may interfere with the oxidation of guaiac.

FIT uses antibodies, which bind to the intact globin component of human hemoglobin.⁹ The quantity of bound antibody-hemoglobin complex is detected and measured by a variety of automated quantitative techniques. This testing strategy eliminates the need for food or medication restrictions and the subjective visual assessment of change in color, as required for the gFOBT.⁹ A 2016 meta-analysis found that FIT performed better compared with gFOBT in terms of specificity, positivity rate, number needed to scope, and number needed to screen.⁸ The FIT screening method has also been found to have greater adherence rates, which is likely due to fewer stool sampling requirements and the lack of medication or dietary restrictions, compared with gFOBT.^7,8

The COVID-19 pandemic had a drastic impact on CRC preventive care services. In March 2020, elective colonoscopies were temporarily ceased across the country and the US Department of Veterans Affairs (VA) deferred all elective surgeries and medical procedures, including screening and surveillance colonoscopies. In line with these recommendations, elective colonoscopies were temporarily ceased across the country.¹⁰ The National Cancer Institute’s Population-Based Research to Optimize the Screening Process consortium reported that CRC screening rates decreased by 82% across the US in 2020.¹¹ Public health measures are likely the main reason for this decline, but other factors may include a lack of resource availability in outpatient settings and public fear of the pandemic.¹⁰

The James A. Haley Veterans Affairs Hospital (JAHVAH) in Tampa, Florida, encouraged the use of FIT in place of colonoscopies to avoid delaying preventive services. The initiative to continue CRC screening methods via FIT was scrutinized when laboratory personnel reported that in fiscal year (FY) 2020, 62% of the FIT kits that patients returned to the laboratory were missing information or had other errors (Figure 1). These improperly returned FIT kits led to delayed processing, canceled orders, increased staff workload, and more costs for FIT repetition.

Research shows many patients often fail to adhere to the instructions for proper FIT sample collection and return. Wang and colleagues reported that of 4916 FIT samples returned to the laboratory, 971 (20%) had collection errors, and 910 (94%) of those samples were missing a sample collection date.¹² The sample collection date is important because hemoglobin degradation occurs over time, which may create false-negative FIT results. Although studies have found that sample return times of ≤ 10 days are not associated with a decrease in FIT positive rates, it is recommended to mail completed FITs within 24 hours of sample collection.¹³

Because remote screening methods like FIT were preferred during the COVID-19 pandemic, we conducted a quality improvement (QI) project to address FIT inefficiency. The aim of this initiative was to determine the root cause behind incorrectly returned FIT kits and to increase correctly collected and testable FIT kits upon initial laboratory arrival by at least 20% by the second quarter of FY 2021.

Quality Improvement Project

This QI project was conducted from July 2020 to June 2021 at the JAHVAH, which provides primary care and specialty health services to veterans in central and south Florida. The QI was designed based on the Plan-Do-Study-Act (PDSA) model of health care improvement. The QI team consisted of physicians, nurses, administrative staff, and laboratory personnel. A SIPOC (Suppliers, Input, Process, Output, Customers) map was initially designed to help clarify the different groups involved in the process of FIT kit distribution and return. This map helped the team decide who should be involved in the solution process.

The QI team performed a root cause analysis using a fishbone diagram and identified the reasons FIT kits were returned to the laboratory with errors that prevented processing. The team brainstormed potential change ideas and created an impact vs effort chart to increase the number of correctly returned and testable FIT kits upon initial arrival at the laboratory by at least 20% by the second quarter of FY 2021. We identified strengths and prioritized change ideas to improve the number of testable and correctly returned FIT kits to the hospital laboratory. These ideas included centralizing FIT kit dispersal to a new administrative group, building redundant patient reminders on kit completion and giving patients more accessible places for kit return.

Patients included in the study were adults aged 50 to 75 years seen at the JAHVAH outpatient clinic who were asked to undergo FIT CRC screening. FIT orders for other facilities were excluded. The primary endpoint of this project was to improve the number of correctly returned FITs. The number of correct and incorrect returned FITs were measured from July 2020 to June 2021. FITs returned with errors were categorized by the type of error, including: no order on file in the electronic health record (EHR), canceled test, expired test, unable to identify test, missing information, and missing collection date.

We attempted to calculate costs of FITs that were returned to the laboratory but could not be analyzed and were discarded. In FY 2020, 1568 FITs were discarded. Each FIT cost about $7.80 to process for an annualized expense of $12,230 for discarded FITs.

Root Cause Analysis

Root causes were obtained by making a fishbone diagram. From this diagram, an impact vs effort chart was created to form and prioritize ideas for our PDSA cycles. Data about correctly and incorrectly returned kits were collected monthly from laboratory personnel, then analyzed by the QI team using run charts to look for change in frequency and patterns.

To improve this process, a swim lane chart for FIT processing was assembled and later used to make a comprehensive fishbone diagram to establish the 6 main root cause errors: missing FIT EHR order, cancelled FIT EHR order, expired stool specimen, partial patient identifiers, no patient identifiers, and no stool collection date. Pareto and run charts were superimposed with the laboratory data. The most common cause of incorrectly returned FITs was no collection date.

PDSA Cycles

Beginning in January 2021, PDSA cycles from the ideas in the impact vs effort chart were used. Organization and implementation of the project occurred from July 2020 to April 2021. The team reassessed the data in April 2021 to evaluate progress after PDSA initiation. The mean rate of missing collection date dropped from 24% in FY 2020 prior to PDSA cycles to 14% in April 2021; however, the number of incorrectly returned kits was similar to the baseline level. When reviewing this discrepancy, the QI team found that although the missing collection date rate had improved, the rate of FITs with not enough information had increased from 5% in FY 2020 to 67% in April 2021 (Figure 2). After discussing with laboratory personnel, it was determined that the EHR order was missing when the process pathway changed. Our PDSA initiative changed the process pathway and different individuals were responsible for FIT dispersal. The error was quickly addressed with the help of clinical and administrative staff; a 30-day follow-up on June 21, 2021, revealed that only 9% of the patients had sent back kits with not enough information.

After troubleshooting, the team achieved a sustainable increase in the number of correctly returned FIT kits from an average of 38% before the project to 72% after 30-day follow-up.

Proper collection and return of FIT samples are vital for process efficiency for both physicians and patients. This initiative aimed to improve the rate of correctly returned FIT kits by 20%, but its final numbers showed an improvement of 33.6%. Operational benefits from this project included early detection of CRC, improved laboratory workflow, decreased FIT kit waste, and increased patient satisfaction.

The multipronged PDSA cycle attempted to increase the rate of correctly returned FIT kits. We improved kit comprehension and laboratory accessibility, and instituted redundant return reminders for patients. We also centralized a new process pathway for FIT distribution and educated physicians and support staff. Sampling and FIT return may seem like a simple procedure, but the FIT can be cumbersome for patients and directions can be confusing. Therefore, to maximize screening participation, it is essential to minimize confusion in the collection and return of a FIT sample.^14,15

This QI initiative was presented at Grand Rounds at the University of South Florida in June 2021 and has since been shared with other VA hospitals. It was also presented at the American College of Gastroenterology Conference in 2021.

Limitations

This study was a single-center QI project and focused mostly on FIT kit return rates. To fully address CRC screening, it is important to ensure that individuals with a positive screen are appropriately followed up with a colonoscopy. Although follow-up was not in the scope of this project, it is key to CRC screening in general and should be the subject of future research.

Conclusions

FIT is a useful method for CRC screening that can be particularly helpful when in-person visits are limited, as seen during the COVID-19 pandemic. This increase in demand for FITs during the pandemic revealed process deficiencies and gave JAHVAH an opportunity to improve workflow. Through the aid of a multidisciplinary team, the process to complete and return FITs improved and surpassed the goal of 20% improvement. Our goal is to continue to fine-tune the workflow and troubleshoot the system as needed.

Colorectal cancer (CRC) is the third-most common cancer worldwide and accounts for almost 11% of all cancer diagnoses, with > 1.9 million cases reported globally.^1,2 CRC is the second-most deadly cancer, responsible for about 935,000 deaths.¹ Over the past several decades, a steady decline in CRC incidence and mortality has been reported in developed countries, including the US.^3,4 From 2008 through 2017, an annual reduction of 3% in CRC death rates was reported in individuals aged ≥ 65 years.⁵ This decline can mainly be attributed to improvements made in health systems and advancements in CRC screening programs.^3,5

US Preventive Services Task Force (USPSTF) recommends CRC screening in individuals aged 45 to 75 years. USPSTF recommends direct visualization tests, such as colonoscopy and flexible sigmoidoscopy for CRC screening.⁶ Although colonoscopy is commonly used for CRC screening, it is an invasive procedure that requires bowel preparation and sedation, and has the potential risk of colonic perforation, bleeding, and infection. Additionally, social determinants—such as health care costs, missed work, and geographic location (eg, rural communities)—may limit colonoscopy utilization.⁷ As a result, other cost-effective, noninvasive tests such as high-sensitivity guaiac-based fecal occult blood test (gFOBT) and fecal immunochemical test (FIT) are also used for CRC screening. These tests detect occult blood in the stool of individuals who may be at risk for CRC, helping direct them to colonoscopy if they screen positive.⁸

The gFOBT relies on simple oxidation and requires a stool sample to detect the presence of the heme component of blood.⁹ If heme is present in the stool sample, it will enable the oxidation of guaiac to form a blue-colored dye when added to hydrogen peroxide. It is important to note that the oxidation component of this test may lead to false-positive results, as it may detect dietary hemoglobin present in red meat. Medications or foods that have peroxidase properties may also result in a false-positive gFOBT result. Additionally, false-negative results may be caused by antioxidants, which may interfere with the oxidation of guaiac.

FIT uses antibodies, which bind to the intact globin component of human hemoglobin.⁹ The quantity of bound antibody-hemoglobin complex is detected and measured by a variety of automated quantitative techniques. This testing strategy eliminates the need for food or medication restrictions and the subjective visual assessment of change in color, as required for the gFOBT.⁹ A 2016 meta-analysis found that FIT performed better compared with gFOBT in terms of specificity, positivity rate, number needed to scope, and number needed to screen.⁸ The FIT screening method has also been found to have greater adherence rates, which is likely due to fewer stool sampling requirements and the lack of medication or dietary restrictions, compared with gFOBT.^7,8

The COVID-19 pandemic had a drastic impact on CRC preventive care services. In March 2020, elective colonoscopies were temporarily ceased across the country and the US Department of Veterans Affairs (VA) deferred all elective surgeries and medical procedures, including screening and surveillance colonoscopies. In line with these recommendations, elective colonoscopies were temporarily ceased across the country.¹⁰ The National Cancer Institute’s Population-Based Research to Optimize the Screening Process consortium reported that CRC screening rates decreased by 82% across the US in 2020.¹¹ Public health measures are likely the main reason for this decline, but other factors may include a lack of resource availability in outpatient settings and public fear of the pandemic.¹⁰

The James A. Haley Veterans Affairs Hospital (JAHVAH) in Tampa, Florida, encouraged the use of FIT in place of colonoscopies to avoid delaying preventive services. The initiative to continue CRC screening methods via FIT was scrutinized when laboratory personnel reported that in fiscal year (FY) 2020, 62% of the FIT kits that patients returned to the laboratory were missing information or had other errors (Figure 1). These improperly returned FIT kits led to delayed processing, canceled orders, increased staff workload, and more costs for FIT repetition.

Research shows many patients often fail to adhere to the instructions for proper FIT sample collection and return. Wang and colleagues reported that of 4916 FIT samples returned to the laboratory, 971 (20%) had collection errors, and 910 (94%) of those samples were missing a sample collection date.¹² The sample collection date is important because hemoglobin degradation occurs over time, which may create false-negative FIT results. Although studies have found that sample return times of ≤ 10 days are not associated with a decrease in FIT positive rates, it is recommended to mail completed FITs within 24 hours of sample collection.¹³

Because remote screening methods like FIT were preferred during the COVID-19 pandemic, we conducted a quality improvement (QI) project to address FIT inefficiency. The aim of this initiative was to determine the root cause behind incorrectly returned FIT kits and to increase correctly collected and testable FIT kits upon initial laboratory arrival by at least 20% by the second quarter of FY 2021.

Quality Improvement Project

This QI project was conducted from July 2020 to June 2021 at the JAHVAH, which provides primary care and specialty health services to veterans in central and south Florida. The QI was designed based on the Plan-Do-Study-Act (PDSA) model of health care improvement. The QI team consisted of physicians, nurses, administrative staff, and laboratory personnel. A SIPOC (Suppliers, Input, Process, Output, Customers) map was initially designed to help clarify the different groups involved in the process of FIT kit distribution and return. This map helped the team decide who should be involved in the solution process.

The QI team performed a root cause analysis using a fishbone diagram and identified the reasons FIT kits were returned to the laboratory with errors that prevented processing. The team brainstormed potential change ideas and created an impact vs effort chart to increase the number of correctly returned and testable FIT kits upon initial arrival at the laboratory by at least 20% by the second quarter of FY 2021. We identified strengths and prioritized change ideas to improve the number of testable and correctly returned FIT kits to the hospital laboratory. These ideas included centralizing FIT kit dispersal to a new administrative group, building redundant patient reminders on kit completion and giving patients more accessible places for kit return.

Patients included in the study were adults aged 50 to 75 years seen at the JAHVAH outpatient clinic who were asked to undergo FIT CRC screening. FIT orders for other facilities were excluded. The primary endpoint of this project was to improve the number of correctly returned FITs. The number of correct and incorrect returned FITs were measured from July 2020 to June 2021. FITs returned with errors were categorized by the type of error, including: no order on file in the electronic health record (EHR), canceled test, expired test, unable to identify test, missing information, and missing collection date.

We attempted to calculate costs of FITs that were returned to the laboratory but could not be analyzed and were discarded. In FY 2020, 1568 FITs were discarded. Each FIT cost about $7.80 to process for an annualized expense of $12,230 for discarded FITs.

Root Cause Analysis

Root causes were obtained by making a fishbone diagram. From this diagram, an impact vs effort chart was created to form and prioritize ideas for our PDSA cycles. Data about correctly and incorrectly returned kits were collected monthly from laboratory personnel, then analyzed by the QI team using run charts to look for change in frequency and patterns.

To improve this process, a swim lane chart for FIT processing was assembled and later used to make a comprehensive fishbone diagram to establish the 6 main root cause errors: missing FIT EHR order, cancelled FIT EHR order, expired stool specimen, partial patient identifiers, no patient identifiers, and no stool collection date. Pareto and run charts were superimposed with the laboratory data. The most common cause of incorrectly returned FITs was no collection date.

PDSA Cycles

Beginning in January 2021, PDSA cycles from the ideas in the impact vs effort chart were used. Organization and implementation of the project occurred from July 2020 to April 2021. The team reassessed the data in April 2021 to evaluate progress after PDSA initiation. The mean rate of missing collection date dropped from 24% in FY 2020 prior to PDSA cycles to 14% in April 2021; however, the number of incorrectly returned kits was similar to the baseline level. When reviewing this discrepancy, the QI team found that although the missing collection date rate had improved, the rate of FITs with not enough information had increased from 5% in FY 2020 to 67% in April 2021 (Figure 2). After discussing with laboratory personnel, it was determined that the EHR order was missing when the process pathway changed. Our PDSA initiative changed the process pathway and different individuals were responsible for FIT dispersal. The error was quickly addressed with the help of clinical and administrative staff; a 30-day follow-up on June 21, 2021, revealed that only 9% of the patients had sent back kits with not enough information.

After troubleshooting, the team achieved a sustainable increase in the number of correctly returned FIT kits from an average of 38% before the project to 72% after 30-day follow-up.

Proper collection and return of FIT samples are vital for process efficiency for both physicians and patients. This initiative aimed to improve the rate of correctly returned FIT kits by 20%, but its final numbers showed an improvement of 33.6%. Operational benefits from this project included early detection of CRC, improved laboratory workflow, decreased FIT kit waste, and increased patient satisfaction.

The multipronged PDSA cycle attempted to increase the rate of correctly returned FIT kits. We improved kit comprehension and laboratory accessibility, and instituted redundant return reminders for patients. We also centralized a new process pathway for FIT distribution and educated physicians and support staff. Sampling and FIT return may seem like a simple procedure, but the FIT can be cumbersome for patients and directions can be confusing. Therefore, to maximize screening participation, it is essential to minimize confusion in the collection and return of a FIT sample.^14,15

This QI initiative was presented at Grand Rounds at the University of South Florida in June 2021 and has since been shared with other VA hospitals. It was also presented at the American College of Gastroenterology Conference in 2021.

Limitations

This study was a single-center QI project and focused mostly on FIT kit return rates. To fully address CRC screening, it is important to ensure that individuals with a positive screen are appropriately followed up with a colonoscopy. Although follow-up was not in the scope of this project, it is key to CRC screening in general and should be the subject of future research.

Conclusions

FIT is a useful method for CRC screening that can be particularly helpful when in-person visits are limited, as seen during the COVID-19 pandemic. This increase in demand for FITs during the pandemic revealed process deficiencies and gave JAHVAH an opportunity to improve workflow. Through the aid of a multidisciplinary team, the process to complete and return FITs improved and surpassed the goal of 20% improvement. Our goal is to continue to fine-tune the workflow and troubleshoot the system as needed.

Colorectal cancer (CRC) is the third-most common cancer worldwide and accounts for almost 11% of all cancer diagnoses, with > 1.9 million cases reported globally.^1,2 CRC is the second-most deadly cancer, responsible for about 935,000 deaths.¹ Over the past several decades, a steady decline in CRC incidence and mortality has been reported in developed countries, including the US.^3,4 From 2008 through 2017, an annual reduction of 3% in CRC death rates was reported in individuals aged ≥ 65 years.⁵ This decline can mainly be attributed to improvements made in health systems and advancements in CRC screening programs.^3,5

US Preventive Services Task Force (USPSTF) recommends CRC screening in individuals aged 45 to 75 years. USPSTF recommends direct visualization tests, such as colonoscopy and flexible sigmoidoscopy for CRC screening.⁶ Although colonoscopy is commonly used for CRC screening, it is an invasive procedure that requires bowel preparation and sedation, and has the potential risk of colonic perforation, bleeding, and infection. Additionally, social determinants—such as health care costs, missed work, and geographic location (eg, rural communities)—may limit colonoscopy utilization.⁷ As a result, other cost-effective, noninvasive tests such as high-sensitivity guaiac-based fecal occult blood test (gFOBT) and fecal immunochemical test (FIT) are also used for CRC screening. These tests detect occult blood in the stool of individuals who may be at risk for CRC, helping direct them to colonoscopy if they screen positive.⁸

The gFOBT relies on simple oxidation and requires a stool sample to detect the presence of the heme component of blood.⁹ If heme is present in the stool sample, it will enable the oxidation of guaiac to form a blue-colored dye when added to hydrogen peroxide. It is important to note that the oxidation component of this test may lead to false-positive results, as it may detect dietary hemoglobin present in red meat. Medications or foods that have peroxidase properties may also result in a false-positive gFOBT result. Additionally, false-negative results may be caused by antioxidants, which may interfere with the oxidation of guaiac.

FIT uses antibodies, which bind to the intact globin component of human hemoglobin.⁹ The quantity of bound antibody-hemoglobin complex is detected and measured by a variety of automated quantitative techniques. This testing strategy eliminates the need for food or medication restrictions and the subjective visual assessment of change in color, as required for the gFOBT.⁹ A 2016 meta-analysis found that FIT performed better compared with gFOBT in terms of specificity, positivity rate, number needed to scope, and number needed to screen.⁸ The FIT screening method has also been found to have greater adherence rates, which is likely due to fewer stool sampling requirements and the lack of medication or dietary restrictions, compared with gFOBT.^7,8

The COVID-19 pandemic had a drastic impact on CRC preventive care services. In March 2020, elective colonoscopies were temporarily ceased across the country and the US Department of Veterans Affairs (VA) deferred all elective surgeries and medical procedures, including screening and surveillance colonoscopies. In line with these recommendations, elective colonoscopies were temporarily ceased across the country.¹⁰ The National Cancer Institute’s Population-Based Research to Optimize the Screening Process consortium reported that CRC screening rates decreased by 82% across the US in 2020.¹¹ Public health measures are likely the main reason for this decline, but other factors may include a lack of resource availability in outpatient settings and public fear of the pandemic.¹⁰

The James A. Haley Veterans Affairs Hospital (JAHVAH) in Tampa, Florida, encouraged the use of FIT in place of colonoscopies to avoid delaying preventive services. The initiative to continue CRC screening methods via FIT was scrutinized when laboratory personnel reported that in fiscal year (FY) 2020, 62% of the FIT kits that patients returned to the laboratory were missing information or had other errors (Figure 1). These improperly returned FIT kits led to delayed processing, canceled orders, increased staff workload, and more costs for FIT repetition.

Research shows many patients often fail to adhere to the instructions for proper FIT sample collection and return. Wang and colleagues reported that of 4916 FIT samples returned to the laboratory, 971 (20%) had collection errors, and 910 (94%) of those samples were missing a sample collection date.¹² The sample collection date is important because hemoglobin degradation occurs over time, which may create false-negative FIT results. Although studies have found that sample return times of ≤ 10 days are not associated with a decrease in FIT positive rates, it is recommended to mail completed FITs within 24 hours of sample collection.¹³

Because remote screening methods like FIT were preferred during the COVID-19 pandemic, we conducted a quality improvement (QI) project to address FIT inefficiency. The aim of this initiative was to determine the root cause behind incorrectly returned FIT kits and to increase correctly collected and testable FIT kits upon initial laboratory arrival by at least 20% by the second quarter of FY 2021.

Quality Improvement Project

This QI project was conducted from July 2020 to June 2021 at the JAHVAH, which provides primary care and specialty health services to veterans in central and south Florida. The QI was designed based on the Plan-Do-Study-Act (PDSA) model of health care improvement. The QI team consisted of physicians, nurses, administrative staff, and laboratory personnel. A SIPOC (Suppliers, Input, Process, Output, Customers) map was initially designed to help clarify the different groups involved in the process of FIT kit distribution and return. This map helped the team decide who should be involved in the solution process.

The QI team performed a root cause analysis using a fishbone diagram and identified the reasons FIT kits were returned to the laboratory with errors that prevented processing. The team brainstormed potential change ideas and created an impact vs effort chart to increase the number of correctly returned and testable FIT kits upon initial arrival at the laboratory by at least 20% by the second quarter of FY 2021. We identified strengths and prioritized change ideas to improve the number of testable and correctly returned FIT kits to the hospital laboratory. These ideas included centralizing FIT kit dispersal to a new administrative group, building redundant patient reminders on kit completion and giving patients more accessible places for kit return.

Patients included in the study were adults aged 50 to 75 years seen at the JAHVAH outpatient clinic who were asked to undergo FIT CRC screening. FIT orders for other facilities were excluded. The primary endpoint of this project was to improve the number of correctly returned FITs. The number of correct and incorrect returned FITs were measured from July 2020 to June 2021. FITs returned with errors were categorized by the type of error, including: no order on file in the electronic health record (EHR), canceled test, expired test, unable to identify test, missing information, and missing collection date.

We attempted to calculate costs of FITs that were returned to the laboratory but could not be analyzed and were discarded. In FY 2020, 1568 FITs were discarded. Each FIT cost about $7.80 to process for an annualized expense of $12,230 for discarded FITs.

Root Cause Analysis

Root causes were obtained by making a fishbone diagram. From this diagram, an impact vs effort chart was created to form and prioritize ideas for our PDSA cycles. Data about correctly and incorrectly returned kits were collected monthly from laboratory personnel, then analyzed by the QI team using run charts to look for change in frequency and patterns.

To improve this process, a swim lane chart for FIT processing was assembled and later used to make a comprehensive fishbone diagram to establish the 6 main root cause errors: missing FIT EHR order, cancelled FIT EHR order, expired stool specimen, partial patient identifiers, no patient identifiers, and no stool collection date. Pareto and run charts were superimposed with the laboratory data. The most common cause of incorrectly returned FITs was no collection date.

PDSA Cycles

Beginning in January 2021, PDSA cycles from the ideas in the impact vs effort chart were used. Organization and implementation of the project occurred from July 2020 to April 2021. The team reassessed the data in April 2021 to evaluate progress after PDSA initiation. The mean rate of missing collection date dropped from 24% in FY 2020 prior to PDSA cycles to 14% in April 2021; however, the number of incorrectly returned kits was similar to the baseline level. When reviewing this discrepancy, the QI team found that although the missing collection date rate had improved, the rate of FITs with not enough information had increased from 5% in FY 2020 to 67% in April 2021 (Figure 2). After discussing with laboratory personnel, it was determined that the EHR order was missing when the process pathway changed. Our PDSA initiative changed the process pathway and different individuals were responsible for FIT dispersal. The error was quickly addressed with the help of clinical and administrative staff; a 30-day follow-up on June 21, 2021, revealed that only 9% of the patients had sent back kits with not enough information.

After troubleshooting, the team achieved a sustainable increase in the number of correctly returned FIT kits from an average of 38% before the project to 72% after 30-day follow-up.

Proper collection and return of FIT samples are vital for process efficiency for both physicians and patients. This initiative aimed to improve the rate of correctly returned FIT kits by 20%, but its final numbers showed an improvement of 33.6%. Operational benefits from this project included early detection of CRC, improved laboratory workflow, decreased FIT kit waste, and increased patient satisfaction.

The multipronged PDSA cycle attempted to increase the rate of correctly returned FIT kits. We improved kit comprehension and laboratory accessibility, and instituted redundant return reminders for patients. We also centralized a new process pathway for FIT distribution and educated physicians and support staff. Sampling and FIT return may seem like a simple procedure, but the FIT can be cumbersome for patients and directions can be confusing. Therefore, to maximize screening participation, it is essential to minimize confusion in the collection and return of a FIT sample.^14,15

This QI initiative was presented at Grand Rounds at the University of South Florida in June 2021 and has since been shared with other VA hospitals. It was also presented at the American College of Gastroenterology Conference in 2021.

Limitations

This study was a single-center QI project and focused mostly on FIT kit return rates. To fully address CRC screening, it is important to ensure that individuals with a positive screen are appropriately followed up with a colonoscopy. Although follow-up was not in the scope of this project, it is key to CRC screening in general and should be the subject of future research.

Conclusions

FIT is a useful method for CRC screening that can be particularly helpful when in-person visits are limited, as seen during the COVID-19 pandemic. This increase in demand for FITs during the pandemic revealed process deficiencies and gave JAHVAH an opportunity to improve workflow. Through the aid of a multidisciplinary team, the process to complete and return FITs improved and surpassed the goal of 20% improvement. Our goal is to continue to fine-tune the workflow and troubleshoot the system as needed.

The US Department of Veterans Affairs (VA) has partnered with academic medical centers and programs since 1946 to provide clinical training for physician residents. Ranking second in federal graduate medical education (GME) funding to the Centers for Medicare and Medicaid Services (CMS), the $850 million VA GME budget annually reimburses > 250 GME-sponsoring institutions (affiliates) of 8000 GME programs for the clinical training of 49,000 individual residents rotating through > 11,000 full-time equivalent (FTE) positions.¹ The VA also distributes $1.6 billion to VA facilities to offset the costs of conducting health professions education (HPE) (eg, facility infrastructure, salary support for VA instructors and preceptors, education office administration, and instructional equipment).² The VA financial and educational contributions account for payment of 11% of resident positions nationally and allow academic medical centers to be less reliant on CMS GME funding.^3,4 The VA contributions also provide opportunities for GME expansion,^1,5,6 educational innovations,^5,7 interprofessional and team-based care,^8,9 and quality and safety training.^10,11 The Table provides a comparison of CMS and VA GME reimbursability based on activity.

GME financing is complex, particularly the formulaic approach used by CMS, the details of which are often obscured in federal regulations. Due to this complexity and the $16 billion CMS GME budget, academic publications have focused on CMS GME financing while not fully explaining the VA GME policies and processes.^4,12-14 By comparison, the VA GME financing model is relatively straightforward and governed by different statues and VA regulations, yet sharing some of the same principles as CMS regulations. Given the challenges in CMS reimbursement to fully support the cost of resident education, as well as the educational opportunities at the VA, the VA designs its reimbursement model to assure that affiliates receive appropriate payments.^4,12,15 To ensure the continued success of VA GME partnerships, knowledge of VA GME financing has become increasingly important for designated institutional officers (DIOs) and residency program directors, particularly in light of recent investigations into oversight of the VA’s reimbursement to academic affiliates.^16-18 This report describes VA GME reimbursement and, where applicable, VA and CMS reimbursement policies are compared to highlight similarities, differences, and common principles.

VA AUTHORITY

While the VA’s primary mission is “to provide a complete hospital medical service for the medical care and treatment of veterans,”early VA leaders recognized the importance of affiliating with the nation’s academic institutions.¹⁹ In 1946, the VA Policy Memorandum Number 2 established a partnership between the VA and the academic medical community.²⁰ Additional legislation authorized specific agreements with academic affiliates for the central administration of salary and benefits for residents rotating at VA facilities. This process, known as disbursement, is an alternative payroll mechanism whereby the VA reimburses the academic affiliate for resident salary and benefits and the affiliate acts as the disbursing agent, issuing paychecks to residents.^21,22

Resident FUNDING

By policy, with rare exceptions, the VA does not sponsor residency programs due to the challenges of providing an appropriate patient mix of age, sex, and medical conditions to meet accreditation standards.⁴ Nearly all VA reimbursements are for residents in affiliate-sponsored programs, while just 1% pays for residents in legacy, VA-sponsored residency programs at 2 VA facilities. The VA budget for resident (including fellows) salary and benefits is managed by the VA Office of Academic Affiliations (OAA), the national VA office responsible for oversight, policy, and funding of VA HPE programs.

Resident Salaries and Benefits

VA funding of resident salary and benefits are analogous with CMS direct GME (DGME), which is designed to cover resident salary and benefits costs.^4,14,23 CMS DGME payments depend on a hospital’s volume of CMS inpatients and are based on a statutory formula, which uses the hospital’s resident FTE positions, the per-resident amount, and Medicare’s share of inpatient beds (Medicare patient load) to determine payments.¹² The per-resident amount is set by statute, varies geographically, and is calculated by dividing the hospital’s allowable costs of GME (percentage of CMS inpatient days) divided by the number of residents.^12,24

By comparison, the VA GME payment reimburses for each FTE based on the salary and benefits rate set by the academic affiliate. Reimbursement is calculated based on resident time spent at the VA multiplied by a daily salary rate. The daily salary rate is determined by dividing the resident’s total compensation (salary and benefits) by the number of calendar days in an academic year. Resident time spent at the VA facility is determined by obtaining rotation schedules provided by the academic affiliate and verifying resident clinical and educational activity during scheduled rotations.

Indirect Medical Education Funding

In addition to resident salary and benefits, funds to offset the cost of conducting HPE are provided to VA facilities. These funds are intended to improve and maintain necessary infrastructure for all HPE programs not just GME, including education office administration needs, teaching costs (ie, a portion of VA preceptors salary), and instructional equipment.

The Veterans Equitable Resource Allocation (VERA) is a national budgeting process for VA medical facilities that funds facility operational needs such as staff salary and benefits, infrastructure, and equipment.² The education portion of the VERA, the VERA Education Support Component (VESC), is not managed by the OAA, but rather is distributed through the VERA model to the general budget of VA facilities hosting HPE (Figure). VESC funding in the VA budget is based on labor mapping of physician time spent in education; other labor mapping categories include clinical care, research, and administration. VA facility VESC funding is calculated based on the number of paid health profession trainees (HPTs) from all professions, apportioned according to the number of FTEs for physician residents and VA-paid HPTs in other disciplines. In fiscal year 2024, VA facilities received $115,812 for each physician resident FTE position and $84,906 for each VA-paid, non-GME FTE position.

The VESC is like CMS's indirect GME funding, termed Indirect Medical Education (IME), an additional payment for each Medicare patient discharged reflecting teaching hospitals’ higher patient care costs relative to nonteaching hospitals. Described elsewhere, IME is calculated using a resident-to-bed ratio and a multiplier, which is set by statute.^4,25 While IME can be used for reimbursement for some resident clinical and educational activities(eg, research), VA VESC funds cannot be used for such activities and are part of the general facility budget and appropriated per the discretion of the medical facility director.

ESTABLISHING GME PARTNERSHIPS

An affiliation agreement establishes the administrative and legal requirements for educational relationships with academic affiliates and includes standards for conducting HPE, responsibilities for accreditation standards, program leadership, faculty, resources, supervision, academic policies, and procedures. The VA uses standardized affiliation agreement templates that have been vetted with accrediting bodies and the VA Office of General Counsel.

A disbursement agreement authorizes the VA to reimburse affiliates for resident salary and benefits for VA clinical and educational activities. The disbursement agreement details the fiscal arrangements (eg, payment in advance vs arrears, salary, and benefit rates, leave) for the reimbursement payments. Veterans Health Administration (VHA) Directive 1400.05 provides the policy and procedures for calculating reimbursement for HPT educational activities.²⁶

The VA facility designated education officer (DEO) oversees all HPE programs and coordinates the affiliation and disbursement agreement processes.²⁷ The DEO, affiliate DIO, residency program director, and VA residency site director determine the physician resident FTE positions assigned to a VA facility based on educational objectives and availability of educational resources at the VA facility, such as patient care opportunities, faculty supervisors, space, and equipment. The VA facility requests for resident FTE positions are submitted to the OAA by the facility DEO.

Once GME FTE positions are approved by the OAA, VA facilities work with their academic affiliate to submit the physician resident salary and benefit rate. Affiliate DIOs attest to the accuracy of the salary rate schedule and the local DEO submits the budget request to the OAA. Upon approval, the funds are transferred to the VA facility each fiscal year, which begins October 1. DEOs report quarterly to the OAA both budget needs and excesses based on variations in the approved FTEs due to additional VA rotations, physician resident attrition, or reassignment.

Resident Position Allocation

VA GME financing provides flexibility through periodic needs assessments and expansion initiatives. In August and December, DEOs collaborate with an academic affiliate to submit reports to the OAA confirming their projected GME needs for the next academic year. Additional positions requests are reviewed by the OAA; funding depends on budget and the educational justification. The OAA periodically issues GME expansion requests for proposal, which typically arise from legislation to address specific VA workforce needs. The VA facility DEO and affiliate GME leaders collaborate to apply for additional positions. For example, a VA GME expansion under the Veterans Access, Choice, and Accountability Act of 2014 added 1500 GME positions in 8 years for critically needed specialties and in rural and underserved areas.⁵ The Maintaining Internal Systems and Strengthening Outside Networks (MISSION) Act of 2018 authorized a pilot program for VA to fund residents at non-VA facilities with priority for Indian Health Services, Tribes and Tribal Organizations, Federally Qualified Health Centers, and US Department of Defense facilities to provide access to veterans in underserved areas.⁶

The VA GME financing system has flexibility to meet local needs for additional resident positions and to address broader VA workforce gaps through targeted expansion. Generally, CMS does not fund positions to address workforce needs, place residents in specific geographic areas, or require the training of certain types of residents.⁴ However, the Consolidated Appropriations Act of 2021 has provided the opportunity to address rural workforce needs.²⁸

Reimbursement

The VA provides reimbursement for clinical and educational activities performed in VA facilities for the benefit of veterans as well as research, didactics, meetings and conferences, annual and sick leave, and orientation. The VA also may provide reimbursement for educational activities that occur off VA grounds (eg, the VA proportional share of a residency program’s didactic sessions). The VA does not reimburse for affiliate clinical duties or administrative costs, although a national policy allows VA facilities to reimburse affiliates for some GME overhead costs.²⁹

CMS similarly reimburses for residency training time spent in patient care activities as well as orientation activities, didactics, leave, and, in some cases, research.^4,30,31 CMS makes payments to hospitals, which may include sponsoring institutions and Medicare-eligible participating training sites.^4,30,31 For both the VA and CMS, residents may not be counted twice for reimbursement by 2 federal agencies; in other words, a resident may not count for > 1 FTE.^4,30-32

GME Oversight

VA GME funding came under significant scrutiny. At a 2016 House Veterans Affairs Committee hearing, Representative Phil Roe, MD (R-Tennessee), noted that no process existed at many VA facilities for “determining trainee presence” and that many VA medical centers had “difficulty tracking resident rotations”¹⁶ A VA Office of the Inspector General investigation recommended that the VA implement policies and procedures to improve oversight to “ensure residents are fully participating in educational activities” and that the VA is “paying the correct amount” to the affiliate.¹⁷ A 2020 General Accountability Office report outlined unclear policy guidance, incomplete tracking of resident activities, and improper fiscal processes for reimbursement and reconciliation of affiliate invoices.¹⁸

In response, the OAA created an oversight and compliance unit, revised VHA Directive 1400.05 (the policy for disbursement), and improved resident tracking procedures.²⁶ The standard operating procedure that accompanied VHA Directive 1400.05 provides detailed information for the DEO and VA facility staff for tracking resident clinical and educational activities. FTE counts are essential to both VA and CMS for accurate reimbursement. The eAppendix and the Table provide a guide to reimbursable activities in the VA for the calculation of reimbursement, with a comparison to CMS.^33,34 The OAA in cooperation with other VA staff and officers periodically conducts audits to assess compliance with disbursement policy and affiliate reimbursement accuracy.

In the VA, resident activities are captured on the VA Educational Activity Record, a standardized spreadsheet to track activities and calculate reimbursement. Each VA facility hosting resident physicians manually records resident activity by the half-day. This process is labor intensive, involving both VA and affiliate staff to accurately reconcile payments. To address the workload demands, the OAA is developing an online tool that will automate aspects of the tracking process. Also, to ensure adequate staffing, the OAA is in the process of implementing an office optimization project, providing standardized position descriptions, an organizational chart, and staffing levels for DEO offices in VA facilities.

This report describes the key policies and principles of VA GME financing, highlighting the essential similarities and differences between VA and CMS. Neither the VA nor CMS regulations allow for reimbursement for > 1 FTE position per resident, a principle that underpins the assignment of resident rotations and federal funding for GME and are similar with respect to reimbursement for patient care activities, didactics, research, orientation, and scholarly activity. While reimbursable activities in the VA require physical presence and care of veteran patients, CMS also limits reimbursement to resident activities in the hospital and approved other settings if the hospital is paying for resident salary and benefits in these settings. The VA provides some flexibility for offsite activities including didactics and, in specific circumstances, remote care of veteran patients (eg, teleradiology).

The VA and CMS use different GME financing models. For example, the CMS calculations for resident FTEs are complex, whereas VA calculations reimburse the salary and benefits as set by the academic affiliate. The VA process accounts for local variation in salary rates, whereas the per-resident amount set by CMS varies regionally and does not fully account for differences in the cost of living.²⁴ Because all patients in VA facilities are veterans, VA calculations for reimbursement do not involve ratios of beds like the CMS calculations to determine a proportional share of reimbursement. The VA GME expansion tends to be more directed to VA health workforce needs than CMS, specifying the types of programs and geographic locations to address these needs.

The VA regularly reevaluates how affiliates are reimbursed for VA resident activity, balancing compliance with VA policies and the workload for VA and its affiliates. The VA obtains input from key stakeholders including DEOs, DIOs, and professional organizations such as the Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education.^35,36

Looking ahead, the VA is developing an online tool to improve the accuracy of affiliate reimbursement. The VA will also implement a standardized staffing model, organizational structure, and position descriptions for DEO offices. These initiatives will help reduce the burden of tracking and verifying resident activity and continue to support the 77-year partnership between VA and its affiliated institutions.

The US Department of Veterans Affairs (VA) has partnered with academic medical centers and programs since 1946 to provide clinical training for physician residents. Ranking second in federal graduate medical education (GME) funding to the Centers for Medicare and Medicaid Services (CMS), the $850 million VA GME budget annually reimburses > 250 GME-sponsoring institutions (affiliates) of 8000 GME programs for the clinical training of 49,000 individual residents rotating through > 11,000 full-time equivalent (FTE) positions.¹ The VA also distributes $1.6 billion to VA facilities to offset the costs of conducting health professions education (HPE) (eg, facility infrastructure, salary support for VA instructors and preceptors, education office administration, and instructional equipment).² The VA financial and educational contributions account for payment of 11% of resident positions nationally and allow academic medical centers to be less reliant on CMS GME funding.^3,4 The VA contributions also provide opportunities for GME expansion,^1,5,6 educational innovations,^5,7 interprofessional and team-based care,^8,9 and quality and safety training.^10,11 The Table provides a comparison of CMS and VA GME reimbursability based on activity.

GME financing is complex, particularly the formulaic approach used by CMS, the details of which are often obscured in federal regulations. Due to this complexity and the $16 billion CMS GME budget, academic publications have focused on CMS GME financing while not fully explaining the VA GME policies and processes.^4,12-14 By comparison, the VA GME financing model is relatively straightforward and governed by different statues and VA regulations, yet sharing some of the same principles as CMS regulations. Given the challenges in CMS reimbursement to fully support the cost of resident education, as well as the educational opportunities at the VA, the VA designs its reimbursement model to assure that affiliates receive appropriate payments.^4,12,15 To ensure the continued success of VA GME partnerships, knowledge of VA GME financing has become increasingly important for designated institutional officers (DIOs) and residency program directors, particularly in light of recent investigations into oversight of the VA’s reimbursement to academic affiliates.^16-18 This report describes VA GME reimbursement and, where applicable, VA and CMS reimbursement policies are compared to highlight similarities, differences, and common principles.

VA AUTHORITY

While the VA’s primary mission is “to provide a complete hospital medical service for the medical care and treatment of veterans,”early VA leaders recognized the importance of affiliating with the nation’s academic institutions.¹⁹ In 1946, the VA Policy Memorandum Number 2 established a partnership between the VA and the academic medical community.²⁰ Additional legislation authorized specific agreements with academic affiliates for the central administration of salary and benefits for residents rotating at VA facilities. This process, known as disbursement, is an alternative payroll mechanism whereby the VA reimburses the academic affiliate for resident salary and benefits and the affiliate acts as the disbursing agent, issuing paychecks to residents.^21,22

Resident FUNDING

By policy, with rare exceptions, the VA does not sponsor residency programs due to the challenges of providing an appropriate patient mix of age, sex, and medical conditions to meet accreditation standards.⁴ Nearly all VA reimbursements are for residents in affiliate-sponsored programs, while just 1% pays for residents in legacy, VA-sponsored residency programs at 2 VA facilities. The VA budget for resident (including fellows) salary and benefits is managed by the VA Office of Academic Affiliations (OAA), the national VA office responsible for oversight, policy, and funding of VA HPE programs.

Resident Salaries and Benefits

VA funding of resident salary and benefits are analogous with CMS direct GME (DGME), which is designed to cover resident salary and benefits costs.^4,14,23 CMS DGME payments depend on a hospital’s volume of CMS inpatients and are based on a statutory formula, which uses the hospital’s resident FTE positions, the per-resident amount, and Medicare’s share of inpatient beds (Medicare patient load) to determine payments.¹² The per-resident amount is set by statute, varies geographically, and is calculated by dividing the hospital’s allowable costs of GME (percentage of CMS inpatient days) divided by the number of residents.^12,24

By comparison, the VA GME payment reimburses for each FTE based on the salary and benefits rate set by the academic affiliate. Reimbursement is calculated based on resident time spent at the VA multiplied by a daily salary rate. The daily salary rate is determined by dividing the resident’s total compensation (salary and benefits) by the number of calendar days in an academic year. Resident time spent at the VA facility is determined by obtaining rotation schedules provided by the academic affiliate and verifying resident clinical and educational activity during scheduled rotations.

Indirect Medical Education Funding

In addition to resident salary and benefits, funds to offset the cost of conducting HPE are provided to VA facilities. These funds are intended to improve and maintain necessary infrastructure for all HPE programs not just GME, including education office administration needs, teaching costs (ie, a portion of VA preceptors salary), and instructional equipment.

The Veterans Equitable Resource Allocation (VERA) is a national budgeting process for VA medical facilities that funds facility operational needs such as staff salary and benefits, infrastructure, and equipment.² The education portion of the VERA, the VERA Education Support Component (VESC), is not managed by the OAA, but rather is distributed through the VERA model to the general budget of VA facilities hosting HPE (Figure). VESC funding in the VA budget is based on labor mapping of physician time spent in education; other labor mapping categories include clinical care, research, and administration. VA facility VESC funding is calculated based on the number of paid health profession trainees (HPTs) from all professions, apportioned according to the number of FTEs for physician residents and VA-paid HPTs in other disciplines. In fiscal year 2024, VA facilities received $115,812 for each physician resident FTE position and $84,906 for each VA-paid, non-GME FTE position.

The VESC is like CMS's indirect GME funding, termed Indirect Medical Education (IME), an additional payment for each Medicare patient discharged reflecting teaching hospitals’ higher patient care costs relative to nonteaching hospitals. Described elsewhere, IME is calculated using a resident-to-bed ratio and a multiplier, which is set by statute.^4,25 While IME can be used for reimbursement for some resident clinical and educational activities(eg, research), VA VESC funds cannot be used for such activities and are part of the general facility budget and appropriated per the discretion of the medical facility director.

ESTABLISHING GME PARTNERSHIPS

An affiliation agreement establishes the administrative and legal requirements for educational relationships with academic affiliates and includes standards for conducting HPE, responsibilities for accreditation standards, program leadership, faculty, resources, supervision, academic policies, and procedures. The VA uses standardized affiliation agreement templates that have been vetted with accrediting bodies and the VA Office of General Counsel.

A disbursement agreement authorizes the VA to reimburse affiliates for resident salary and benefits for VA clinical and educational activities. The disbursement agreement details the fiscal arrangements (eg, payment in advance vs arrears, salary, and benefit rates, leave) for the reimbursement payments. Veterans Health Administration (VHA) Directive 1400.05 provides the policy and procedures for calculating reimbursement for HPT educational activities.²⁶

The VA facility designated education officer (DEO) oversees all HPE programs and coordinates the affiliation and disbursement agreement processes.²⁷ The DEO, affiliate DIO, residency program director, and VA residency site director determine the physician resident FTE positions assigned to a VA facility based on educational objectives and availability of educational resources at the VA facility, such as patient care opportunities, faculty supervisors, space, and equipment. The VA facility requests for resident FTE positions are submitted to the OAA by the facility DEO.

Once GME FTE positions are approved by the OAA, VA facilities work with their academic affiliate to submit the physician resident salary and benefit rate. Affiliate DIOs attest to the accuracy of the salary rate schedule and the local DEO submits the budget request to the OAA. Upon approval, the funds are transferred to the VA facility each fiscal year, which begins October 1. DEOs report quarterly to the OAA both budget needs and excesses based on variations in the approved FTEs due to additional VA rotations, physician resident attrition, or reassignment.

Resident Position Allocation

VA GME financing provides flexibility through periodic needs assessments and expansion initiatives. In August and December, DEOs collaborate with an academic affiliate to submit reports to the OAA confirming their projected GME needs for the next academic year. Additional positions requests are reviewed by the OAA; funding depends on budget and the educational justification. The OAA periodically issues GME expansion requests for proposal, which typically arise from legislation to address specific VA workforce needs. The VA facility DEO and affiliate GME leaders collaborate to apply for additional positions. For example, a VA GME expansion under the Veterans Access, Choice, and Accountability Act of 2014 added 1500 GME positions in 8 years for critically needed specialties and in rural and underserved areas.⁵ The Maintaining Internal Systems and Strengthening Outside Networks (MISSION) Act of 2018 authorized a pilot program for VA to fund residents at non-VA facilities with priority for Indian Health Services, Tribes and Tribal Organizations, Federally Qualified Health Centers, and US Department of Defense facilities to provide access to veterans in underserved areas.⁶

The VA GME financing system has flexibility to meet local needs for additional resident positions and to address broader VA workforce gaps through targeted expansion. Generally, CMS does not fund positions to address workforce needs, place residents in specific geographic areas, or require the training of certain types of residents.⁴ However, the Consolidated Appropriations Act of 2021 has provided the opportunity to address rural workforce needs.²⁸

Reimbursement

The VA provides reimbursement for clinical and educational activities performed in VA facilities for the benefit of veterans as well as research, didactics, meetings and conferences, annual and sick leave, and orientation. The VA also may provide reimbursement for educational activities that occur off VA grounds (eg, the VA proportional share of a residency program’s didactic sessions). The VA does not reimburse for affiliate clinical duties or administrative costs, although a national policy allows VA facilities to reimburse affiliates for some GME overhead costs.²⁹

CMS similarly reimburses for residency training time spent in patient care activities as well as orientation activities, didactics, leave, and, in some cases, research.^4,30,31 CMS makes payments to hospitals, which may include sponsoring institutions and Medicare-eligible participating training sites.^4,30,31 For both the VA and CMS, residents may not be counted twice for reimbursement by 2 federal agencies; in other words, a resident may not count for > 1 FTE.^4,30-32

GME Oversight

VA GME funding came under significant scrutiny. At a 2016 House Veterans Affairs Committee hearing, Representative Phil Roe, MD (R-Tennessee), noted that no process existed at many VA facilities for “determining trainee presence” and that many VA medical centers had “difficulty tracking resident rotations”¹⁶ A VA Office of the Inspector General investigation recommended that the VA implement policies and procedures to improve oversight to “ensure residents are fully participating in educational activities” and that the VA is “paying the correct amount” to the affiliate.¹⁷ A 2020 General Accountability Office report outlined unclear policy guidance, incomplete tracking of resident activities, and improper fiscal processes for reimbursement and reconciliation of affiliate invoices.¹⁸

In response, the OAA created an oversight and compliance unit, revised VHA Directive 1400.05 (the policy for disbursement), and improved resident tracking procedures.²⁶ The standard operating procedure that accompanied VHA Directive 1400.05 provides detailed information for the DEO and VA facility staff for tracking resident clinical and educational activities. FTE counts are essential to both VA and CMS for accurate reimbursement. The eAppendix and the Table provide a guide to reimbursable activities in the VA for the calculation of reimbursement, with a comparison to CMS.^33,34 The OAA in cooperation with other VA staff and officers periodically conducts audits to assess compliance with disbursement policy and affiliate reimbursement accuracy.

In the VA, resident activities are captured on the VA Educational Activity Record, a standardized spreadsheet to track activities and calculate reimbursement. Each VA facility hosting resident physicians manually records resident activity by the half-day. This process is labor intensive, involving both VA and affiliate staff to accurately reconcile payments. To address the workload demands, the OAA is developing an online tool that will automate aspects of the tracking process. Also, to ensure adequate staffing, the OAA is in the process of implementing an office optimization project, providing standardized position descriptions, an organizational chart, and staffing levels for DEO offices in VA facilities.

This report describes the key policies and principles of VA GME financing, highlighting the essential similarities and differences between VA and CMS. Neither the VA nor CMS regulations allow for reimbursement for > 1 FTE position per resident, a principle that underpins the assignment of resident rotations and federal funding for GME and are similar with respect to reimbursement for patient care activities, didactics, research, orientation, and scholarly activity. While reimbursable activities in the VA require physical presence and care of veteran patients, CMS also limits reimbursement to resident activities in the hospital and approved other settings if the hospital is paying for resident salary and benefits in these settings. The VA provides some flexibility for offsite activities including didactics and, in specific circumstances, remote care of veteran patients (eg, teleradiology).

The VA and CMS use different GME financing models. For example, the CMS calculations for resident FTEs are complex, whereas VA calculations reimburse the salary and benefits as set by the academic affiliate. The VA process accounts for local variation in salary rates, whereas the per-resident amount set by CMS varies regionally and does not fully account for differences in the cost of living.²⁴ Because all patients in VA facilities are veterans, VA calculations for reimbursement do not involve ratios of beds like the CMS calculations to determine a proportional share of reimbursement. The VA GME expansion tends to be more directed to VA health workforce needs than CMS, specifying the types of programs and geographic locations to address these needs.

The VA regularly reevaluates how affiliates are reimbursed for VA resident activity, balancing compliance with VA policies and the workload for VA and its affiliates. The VA obtains input from key stakeholders including DEOs, DIOs, and professional organizations such as the Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education.^35,36

Looking ahead, the VA is developing an online tool to improve the accuracy of affiliate reimbursement. The VA will also implement a standardized staffing model, organizational structure, and position descriptions for DEO offices. These initiatives will help reduce the burden of tracking and verifying resident activity and continue to support the 77-year partnership between VA and its affiliated institutions.

The US Department of Veterans Affairs (VA) has partnered with academic medical centers and programs since 1946 to provide clinical training for physician residents. Ranking second in federal graduate medical education (GME) funding to the Centers for Medicare and Medicaid Services (CMS), the $850 million VA GME budget annually reimburses > 250 GME-sponsoring institutions (affiliates) of 8000 GME programs for the clinical training of 49,000 individual residents rotating through > 11,000 full-time equivalent (FTE) positions.¹ The VA also distributes $1.6 billion to VA facilities to offset the costs of conducting health professions education (HPE) (eg, facility infrastructure, salary support for VA instructors and preceptors, education office administration, and instructional equipment).² The VA financial and educational contributions account for payment of 11% of resident positions nationally and allow academic medical centers to be less reliant on CMS GME funding.^3,4 The VA contributions also provide opportunities for GME expansion,^1,5,6 educational innovations,^5,7 interprofessional and team-based care,^8,9 and quality and safety training.^10,11 The Table provides a comparison of CMS and VA GME reimbursability based on activity.

GME financing is complex, particularly the formulaic approach used by CMS, the details of which are often obscured in federal regulations. Due to this complexity and the $16 billion CMS GME budget, academic publications have focused on CMS GME financing while not fully explaining the VA GME policies and processes.^4,12-14 By comparison, the VA GME financing model is relatively straightforward and governed by different statues and VA regulations, yet sharing some of the same principles as CMS regulations. Given the challenges in CMS reimbursement to fully support the cost of resident education, as well as the educational opportunities at the VA, the VA designs its reimbursement model to assure that affiliates receive appropriate payments.^4,12,15 To ensure the continued success of VA GME partnerships, knowledge of VA GME financing has become increasingly important for designated institutional officers (DIOs) and residency program directors, particularly in light of recent investigations into oversight of the VA’s reimbursement to academic affiliates.^16-18 This report describes VA GME reimbursement and, where applicable, VA and CMS reimbursement policies are compared to highlight similarities, differences, and common principles.

VA AUTHORITY

While the VA’s primary mission is “to provide a complete hospital medical service for the medical care and treatment of veterans,”early VA leaders recognized the importance of affiliating with the nation’s academic institutions.¹⁹ In 1946, the VA Policy Memorandum Number 2 established a partnership between the VA and the academic medical community.²⁰ Additional legislation authorized specific agreements with academic affiliates for the central administration of salary and benefits for residents rotating at VA facilities. This process, known as disbursement, is an alternative payroll mechanism whereby the VA reimburses the academic affiliate for resident salary and benefits and the affiliate acts as the disbursing agent, issuing paychecks to residents.^21,22

Resident FUNDING

By policy, with rare exceptions, the VA does not sponsor residency programs due to the challenges of providing an appropriate patient mix of age, sex, and medical conditions to meet accreditation standards.⁴ Nearly all VA reimbursements are for residents in affiliate-sponsored programs, while just 1% pays for residents in legacy, VA-sponsored residency programs at 2 VA facilities. The VA budget for resident (including fellows) salary and benefits is managed by the VA Office of Academic Affiliations (OAA), the national VA office responsible for oversight, policy, and funding of VA HPE programs.

Resident Salaries and Benefits

VA funding of resident salary and benefits are analogous with CMS direct GME (DGME), which is designed to cover resident salary and benefits costs.^4,14,23 CMS DGME payments depend on a hospital’s volume of CMS inpatients and are based on a statutory formula, which uses the hospital’s resident FTE positions, the per-resident amount, and Medicare’s share of inpatient beds (Medicare patient load) to determine payments.¹² The per-resident amount is set by statute, varies geographically, and is calculated by dividing the hospital’s allowable costs of GME (percentage of CMS inpatient days) divided by the number of residents.^12,24

By comparison, the VA GME payment reimburses for each FTE based on the salary and benefits rate set by the academic affiliate. Reimbursement is calculated based on resident time spent at the VA multiplied by a daily salary rate. The daily salary rate is determined by dividing the resident’s total compensation (salary and benefits) by the number of calendar days in an academic year. Resident time spent at the VA facility is determined by obtaining rotation schedules provided by the academic affiliate and verifying resident clinical and educational activity during scheduled rotations.

Indirect Medical Education Funding

In addition to resident salary and benefits, funds to offset the cost of conducting HPE are provided to VA facilities. These funds are intended to improve and maintain necessary infrastructure for all HPE programs not just GME, including education office administration needs, teaching costs (ie, a portion of VA preceptors salary), and instructional equipment.

The Veterans Equitable Resource Allocation (VERA) is a national budgeting process for VA medical facilities that funds facility operational needs such as staff salary and benefits, infrastructure, and equipment.² The education portion of the VERA, the VERA Education Support Component (VESC), is not managed by the OAA, but rather is distributed through the VERA model to the general budget of VA facilities hosting HPE (Figure). VESC funding in the VA budget is based on labor mapping of physician time spent in education; other labor mapping categories include clinical care, research, and administration. VA facility VESC funding is calculated based on the number of paid health profession trainees (HPTs) from all professions, apportioned according to the number of FTEs for physician residents and VA-paid HPTs in other disciplines. In fiscal year 2024, VA facilities received $115,812 for each physician resident FTE position and $84,906 for each VA-paid, non-GME FTE position.

The VESC is like CMS's indirect GME funding, termed Indirect Medical Education (IME), an additional payment for each Medicare patient discharged reflecting teaching hospitals’ higher patient care costs relative to nonteaching hospitals. Described elsewhere, IME is calculated using a resident-to-bed ratio and a multiplier, which is set by statute.^4,25 While IME can be used for reimbursement for some resident clinical and educational activities(eg, research), VA VESC funds cannot be used for such activities and are part of the general facility budget and appropriated per the discretion of the medical facility director.

ESTABLISHING GME PARTNERSHIPS

An affiliation agreement establishes the administrative and legal requirements for educational relationships with academic affiliates and includes standards for conducting HPE, responsibilities for accreditation standards, program leadership, faculty, resources, supervision, academic policies, and procedures. The VA uses standardized affiliation agreement templates that have been vetted with accrediting bodies and the VA Office of General Counsel.

A disbursement agreement authorizes the VA to reimburse affiliates for resident salary and benefits for VA clinical and educational activities. The disbursement agreement details the fiscal arrangements (eg, payment in advance vs arrears, salary, and benefit rates, leave) for the reimbursement payments. Veterans Health Administration (VHA) Directive 1400.05 provides the policy and procedures for calculating reimbursement for HPT educational activities.²⁶

The VA facility designated education officer (DEO) oversees all HPE programs and coordinates the affiliation and disbursement agreement processes.²⁷ The DEO, affiliate DIO, residency program director, and VA residency site director determine the physician resident FTE positions assigned to a VA facility based on educational objectives and availability of educational resources at the VA facility, such as patient care opportunities, faculty supervisors, space, and equipment. The VA facility requests for resident FTE positions are submitted to the OAA by the facility DEO.

Once GME FTE positions are approved by the OAA, VA facilities work with their academic affiliate to submit the physician resident salary and benefit rate. Affiliate DIOs attest to the accuracy of the salary rate schedule and the local DEO submits the budget request to the OAA. Upon approval, the funds are transferred to the VA facility each fiscal year, which begins October 1. DEOs report quarterly to the OAA both budget needs and excesses based on variations in the approved FTEs due to additional VA rotations, physician resident attrition, or reassignment.

Resident Position Allocation

VA GME financing provides flexibility through periodic needs assessments and expansion initiatives. In August and December, DEOs collaborate with an academic affiliate to submit reports to the OAA confirming their projected GME needs for the next academic year. Additional positions requests are reviewed by the OAA; funding depends on budget and the educational justification. The OAA periodically issues GME expansion requests for proposal, which typically arise from legislation to address specific VA workforce needs. The VA facility DEO and affiliate GME leaders collaborate to apply for additional positions. For example, a VA GME expansion under the Veterans Access, Choice, and Accountability Act of 2014 added 1500 GME positions in 8 years for critically needed specialties and in rural and underserved areas.⁵ The Maintaining Internal Systems and Strengthening Outside Networks (MISSION) Act of 2018 authorized a pilot program for VA to fund residents at non-VA facilities with priority for Indian Health Services, Tribes and Tribal Organizations, Federally Qualified Health Centers, and US Department of Defense facilities to provide access to veterans in underserved areas.⁶

The VA GME financing system has flexibility to meet local needs for additional resident positions and to address broader VA workforce gaps through targeted expansion. Generally, CMS does not fund positions to address workforce needs, place residents in specific geographic areas, or require the training of certain types of residents.⁴ However, the Consolidated Appropriations Act of 2021 has provided the opportunity to address rural workforce needs.²⁸

Reimbursement

The VA provides reimbursement for clinical and educational activities performed in VA facilities for the benefit of veterans as well as research, didactics, meetings and conferences, annual and sick leave, and orientation. The VA also may provide reimbursement for educational activities that occur off VA grounds (eg, the VA proportional share of a residency program’s didactic sessions). The VA does not reimburse for affiliate clinical duties or administrative costs, although a national policy allows VA facilities to reimburse affiliates for some GME overhead costs.²⁹

CMS similarly reimburses for residency training time spent in patient care activities as well as orientation activities, didactics, leave, and, in some cases, research.^4,30,31 CMS makes payments to hospitals, which may include sponsoring institutions and Medicare-eligible participating training sites.^4,30,31 For both the VA and CMS, residents may not be counted twice for reimbursement by 2 federal agencies; in other words, a resident may not count for > 1 FTE.^4,30-32

GME Oversight

VA GME funding came under significant scrutiny. At a 2016 House Veterans Affairs Committee hearing, Representative Phil Roe, MD (R-Tennessee), noted that no process existed at many VA facilities for “determining trainee presence” and that many VA medical centers had “difficulty tracking resident rotations”¹⁶ A VA Office of the Inspector General investigation recommended that the VA implement policies and procedures to improve oversight to “ensure residents are fully participating in educational activities” and that the VA is “paying the correct amount” to the affiliate.¹⁷ A 2020 General Accountability Office report outlined unclear policy guidance, incomplete tracking of resident activities, and improper fiscal processes for reimbursement and reconciliation of affiliate invoices.¹⁸

In response, the OAA created an oversight and compliance unit, revised VHA Directive 1400.05 (the policy for disbursement), and improved resident tracking procedures.²⁶ The standard operating procedure that accompanied VHA Directive 1400.05 provides detailed information for the DEO and VA facility staff for tracking resident clinical and educational activities. FTE counts are essential to both VA and CMS for accurate reimbursement. The eAppendix and the Table provide a guide to reimbursable activities in the VA for the calculation of reimbursement, with a comparison to CMS.^33,34 The OAA in cooperation with other VA staff and officers periodically conducts audits to assess compliance with disbursement policy and affiliate reimbursement accuracy.

In the VA, resident activities are captured on the VA Educational Activity Record, a standardized spreadsheet to track activities and calculate reimbursement. Each VA facility hosting resident physicians manually records resident activity by the half-day. This process is labor intensive, involving both VA and affiliate staff to accurately reconcile payments. To address the workload demands, the OAA is developing an online tool that will automate aspects of the tracking process. Also, to ensure adequate staffing, the OAA is in the process of implementing an office optimization project, providing standardized position descriptions, an organizational chart, and staffing levels for DEO offices in VA facilities.

This report describes the key policies and principles of VA GME financing, highlighting the essential similarities and differences between VA and CMS. Neither the VA nor CMS regulations allow for reimbursement for > 1 FTE position per resident, a principle that underpins the assignment of resident rotations and federal funding for GME and are similar with respect to reimbursement for patient care activities, didactics, research, orientation, and scholarly activity. While reimbursable activities in the VA require physical presence and care of veteran patients, CMS also limits reimbursement to resident activities in the hospital and approved other settings if the hospital is paying for resident salary and benefits in these settings. The VA provides some flexibility for offsite activities including didactics and, in specific circumstances, remote care of veteran patients (eg, teleradiology).

The VA and CMS use different GME financing models. For example, the CMS calculations for resident FTEs are complex, whereas VA calculations reimburse the salary and benefits as set by the academic affiliate. The VA process accounts for local variation in salary rates, whereas the per-resident amount set by CMS varies regionally and does not fully account for differences in the cost of living.²⁴ Because all patients in VA facilities are veterans, VA calculations for reimbursement do not involve ratios of beds like the CMS calculations to determine a proportional share of reimbursement. The VA GME expansion tends to be more directed to VA health workforce needs than CMS, specifying the types of programs and geographic locations to address these needs.

The VA regularly reevaluates how affiliates are reimbursed for VA resident activity, balancing compliance with VA policies and the workload for VA and its affiliates. The VA obtains input from key stakeholders including DEOs, DIOs, and professional organizations such as the Association of American Medical Colleges and the Accreditation Council for Graduate Medical Education.^35,36

Looking ahead, the VA is developing an online tool to improve the accuracy of affiliate reimbursement. The VA will also implement a standardized staffing model, organizational structure, and position descriptions for DEO offices. These initiatives will help reduce the burden of tracking and verifying resident activity and continue to support the 77-year partnership between VA and its affiliated institutions.

The Veterans Health Administration (VHA) is understaffed for clinical psychologists who have specialty training in geriatrics (ie, geropsychologists) to meet the needs of aging veterans. Though only 16.8% of US adults are aged ≥ 65 years,¹ this age group comprises 45.9% of patients within the VHA.² The needs of older adults are complex and warrant specialized services from mental health clinicians trained to understand lifespan developmental processes, biological changes associated with aging, and changes in psychosocial functioning.

Older veterans (aged ≥ 65 years) present with higher rates of combined medical and mental health diagnoses compared to both younger veterans and older adults who are not veterans.³ Nearly 1 of 5 (18.1%) older veterans who use VHA services have confirmed mental health diagnoses, and an additional 25.5% have documented mental health concerns without a formal diagnosis in their health record.⁴ The clinical presentations of older veterans frequently differ from younger adults and include greater complexity. For example, older veterans face an increased risk of cognitive impairment compared to the general population, due in part to higher prevalence of posttraumatic stress, which doubles their risk of developing dementia.⁵ Additional examples of multicomplexity among older veterans may include co-occurring medical and psychiatric diagnoses, the presence of delirium, social isolation/loneliness, and concerns related to polypharmacy. These complex presentations result in significant challenges for mental health clinicians in areas like assessment (eg, accuracy of case conceptualization), intervention (eg, selection and prioritization), and consultation (eg, coordination among multiple medical and mental health specialists).

Older veterans also present with substantial resilience. Research has found that aging veterans exposed to trauma during their military service often review their memories and past experiences, which is known as later-adulthood trauma reengagement.⁶ Through this normative life review process, veterans engage with memories and experiences from their past that they previously avoided, which could lead to posttraumatic growth for some. Unfortunately, others may experience an increase in psychological distress. Mental health clinicians with specialty expertise and training in aging and lifespan development can facilitate positive outcomes to reduce distress.⁷

The United States in general, and the VHA specifically, face a growing shortage of geriatric mental health clinicians. In a 2015 American Psychological Association survey, 1528 of 4109 respondents (37.2%) reported they frequently or very frequently administered care to older adults, yet only 49 respondents (1.2%) reported geropsychology as their specialty.⁸ According to the National Provider Registry, 660 clinicians self-identified as geropsychologists (ie, those who self-reported “psychologist: adult development and aging” as an NPI Healthcare Provider taxonomy code) in the US, representing < 1% of all doctoral level psychologists.⁹ The number of psychologists who obtain board certification from the American Board of Geropsychology is even lower (only 112 clinicians as of February 2024).¹⁰ Many psychologists within the VHA treat older veterans in integrated health settings such as primary care, home-based primary care, community living centers, or behavioral health care, but many lack formal training in geropsychology.

The Geriatric Scholars Program (GSP) was developed in 2008 to address the training gap and provide education in geriatrics to VHA clinicians that treat older veterans, particularly in rural areas.^11,12 The GSP initially focused on primary care physicians, nurse practitioners, physician assistants, and pharmacists. It was later expanded to include other disciplines (ie, social work, rehabilitation therapists, and psychiatrists). In 2013, the GSP – Psychology Track (GSP-P) was developed with funding from the VHA Offices of Rural Health and Geriatrics and Extended Care specifically for psychologists.

This article describes the multicomponent longitudinal GSP-P, which has evolved to meet the target audience’s ongoing needs for knowledge, skills, and opportunities to refine practice behaviors. GSP-P received the 2020 Award for Excellence in Geropsychology Training from the Council of Professional Geropsychology Training Programs. GSP-P has grown within the context of the larger GSP and aligns with the other existing elective learning opportunities (Figure 1).

Program description

Introductory Course

Psychologist subject matter experts (SMEs) developed an intensive course in geropsychology in the absence of a similar course in the geriatric medicine board review curriculum. SMEs reviewed the guidelines for practice by professional organizations like the Pikes Peak Geropsychology Competencies,which outline knowledge and skills in various domains.¹³ SMEs integrated this review with findings from a needs assessment for postlicensed VHA psychology staff in 4 health care systems, drafted a syllabus, and circulated it to geropsychology experts for feedback. The resulting multiday course covered general mental health as well as topics particularly salient for mental health clinicians treating older veterans including suicide prevention and posttraumatic stress disorder (PTSD).¹⁴ This Geropsychology Competencies Review Course was piloted in 1 region initially before being offered nationally in 2014. The previously published evaluation findings demonstrated significant improvements from precourse to 3 months postcourse in confidence and knowledge in 4 key areas of geropsychology: General knowledge about adult development and aging, assessment, intervention, and consultation.¹⁵ These domains align with the Pikes Peak Geropsychology Competencies and are measured by a subset of items from the Geropsychology Knowledge and Skills Assessment Tool.^13,16 As of October 2023, 8 introductory courses have been held with 173 participants (Table).

Quality Improvement

Introductory course attendees also participate in an intensive day-long interactive workshop in quality improvement (QI). After completing these trainings, they apply what they have learned at their home facility by embarking on a QI project related to geriatrics. The QI projects reinforce learning and initiate practice changes not only for attendees but at times the larger health care system. Topics are selected by scholars in response to the needs they observe in their clinics. Recent GSP projects include efforts to increase screenings for depression and anxiety, improve adherence to VHA dementia policy, increase access to virtual care, and increase referrals to programs such as whole health or cognitive behavioral therapy for insomnia, a first-line treatment for insomnia in older adults.¹⁷ Another project targeted the improvement of referrals to the Compassionate Contact Corps in an effort to reduce social isolation and loneliness among older veterans.¹⁸ Evaluations demonstrate significant improvement in scholars’ confidence in related program development and management from precourse to 3 months postcourse.¹⁵

Webinars

The Addressing Geriatric Mental Health webinar series was created to introduce learners to topics that could not be covered in the introductory course. Topics were suggested by the expert reviewers of the curriculum or identified by the scholars themselves (eg, chronic pain, sexuality, or serious mental illness). A secondary function of the webinars was to reach a broader audience. Over time, scholars and webinar attendees requested opportunities to explore topics in greater depth (eg, PTSD later in life). These requests led the webinars to focus on annual themes.

The series is open to all disciplines of geriatric scholars, VHA staff, and non-VHA staff through the Veterans Affairs Talent Management System and the TRAIN Learning Network (train.org). Attendance for the 37 webinars was captured from logins to the virtual learning platform and may underestimate attendance if a group attended on a single screen. Average attendance increased from 157 attendees/webinar in 2015 to 418 attendees/webinar in 2023 (Figure 2). This may have been related to the increase in virtual learning during the COVID-19 pandemic, but represents a 166% increase in audience from the inaugural year of the series.

Advanced Learning Opportunities

To invest in the ongoing growth and development of introductory course graduates, GSP-P developed and offered an advanced workshop in 2019. This multiday workshop focused on further enhancement of geropsychology competencies, with an emphasis on treating older veterans with mental and physical comorbidities. Didactics and experiential learning exercises led by SMEs covered topics such as adjusting to chronic illness, capacity assessment, PTSD, insomnia and sleep changes, chronic pain, and psychological interventions in palliative care and hospice settings. Evaluation findings demonstrated significant improvements from precourse to 6 months postcourse in confidence and knowledge as defined by the Pikes Peak Geropsychology Competencies.¹⁹

To facilitate ongoing and individually tailored learning following the advanced workshop, scholars also developed and executed independent learning plans (ILPs) during a 6-month window with consultation from an experienced geropsychologist. Fifteen of 19 scholars (78.9%) completed ILPs with an average of 3 learning goals listed. After completing the ILPs, scholars endorsed their clinical and/or personal usefulness, citing increased confidence, enhanced skills for use with patients with complex needs, personal fulfillment, and career advancement. Most scholars noted ILPs were feasible and learning resources were accessible. Overall, the evaluation found ILPs to be a valuable way to enhance psychologists’ learning and effectiveness in treating older veterans with complex health needs.²⁰

All geriatric scholars who completed the program have additional opportunities for professional development through practicum experiences focused on specific clinical approaches to the care of older veterans, such as dementia care, pain management, geriatric assessment, and palliative care. These practica provide scholars with individualized learning experiences in an individualized or small group setting and may be conducted either in-person or virtually.

In response to an expressed need from those who completed the program, the GSP-P planning committee collaborated with an SME to develop a virtual practicum to assess patients’ decision making capacity. Evaluating capacities among older adults is a common request, yet clinicians report little to no formal training in how to conceptualize and approach these complex questions.^21,22 Utilizing an evidence-informed and structured approach promotes the balancing of an older adult’s autonomy and professional ethics. Learning capacity evaluation skills could better position psychologists to not only navigate complex ethical, legal, and clinical situations, but also serve as expert consultants to interdisciplinary teams. This virtual practicum was initiated in 2022 and to date has included 10 scholars. The practicum includes multiple modalities of learning: (1) self-directed review of core concepts; (2) attendance at 4 capacity didactics focused on introduction to evaluating capacities, medical consent and decision making, financial decision making and management, and independent living; and (3) participation in 5 group consultations on capacity evaluations conducted at their home sites. During these group consultations, additional case examples were shared to reinforce capacity concepts.

Discussion

The objective of GSP-P is to enhance geropsychology practice competencies among VHA psychologists given the outsized representation of older adults within the VHA system and their complex care needs. The curricula have significantly evolved to accomplish this, expanding the reach and investing in the continuing growth and development of scholars.

There are several elements that set GSP apart from other geriatric and geropsychology continuing medical education programs. The first is that the training is veteran focused, allowing us to discuss the unique impact military service has on aging. Similarly, because all scholars work within the integrated health care system, we can introduce and review key resources and programs that benefit all veterans and their families/care partners across the system. Through the GSP, the VA invests in ongoing professional development. Scholars can participate in additional experiential practica, webinars, and advanced workshops tailored to their individual learning needs. Lastly, the GSP works to create a community among its scholars where they can not only continue to consult with presenters/instructors, but also one another. A planned future direction for the GSP-P is to incorporate quarterly office hours and discussions for alumni to develop an increased sense of community. This may strengthen commitment to the overall VA mission, leading to increased retainment of talent who now have the knowledge, skills, and confidence to care for aging veterans.

Limitations

GSP is limited by its available funding. Additionally, the number of participants who can enroll each year in GSP-P (not including webinars) is capped by policy. Another limitation is the number of QI coaches available to mentor scholars on their projects.

Outcomes of GSP-P have been extremely favorable. Following participation in the program, we have found a significant increase in confidence in geropsychology practice among clinicians, as well as enhanced knowledge and skills across competency domains.^15,19 We have observed rising attendance in our annual webinar series and graduates of our introductory courses participate in subsequent trainings (eg, advanced workshop or virtual practicum). Several graduates of GSP-P have become board certified in geropsychology by the American Board of Geropsychology and many proceed to supervise geropsychology-focused clinical rotations for psychology practicum students, predoctoral interns, and postdoctoral fellows. This suggests that the reach of GSP-P programming may extend farther than reported in this article.

The needs of aging veterans have also changed based on cohort differences, as the population of World War II and Korean War era veterans has declined and the number of older Vietnam era veterans has grown. We expect different challenges with older Gulf War and post-9/11 era veterans. For instance, 17% of troops deployed to Iraq or Afghanistan following 9/11 experienced mild traumatic brain injury (TBI), and 59% of those experienced > 1 mild TBI.²³ Research indicates that younger post-9/11 veterans have a 3-fold risk of developing early onset dementia after experiencing a TBI.²⁴ Therefore, even though post-9/11 veterans are not older in terms of chronological age, some may experience symptoms and conditions more often occurring in older veterans. As a result, it would be beneficial for clinicians to learn about the presentation and treatment of geriatric conditions such as dementia.

Moving forward, the GSP-P should identify potential opportunities to collaborate with the non-VHA mental health community–which also faces a shortage of geriatric mental health clinicians–to extend educational opportunities to improve care for veterans in all settings (eg, cosponsor training opportunities open to both VHA and non-VHA clinicians).^8,25 Many aging veterans may receive portions of their health care outside the VHA, particularly those who reside in rural areas. Additionally, as veterans age, so do their support systems (eg, family members, friends, spouses, caregivers, and even clinicians), most of whom will receive care outside of the VHA. Community education collaborations will not only improve the care of older veterans, but also the care of older adults in the general population.

Promising directions include the adoption of the GSP model in other health care settings. Recently, Indian Health Service has adapted the model, beginning with primary care clinicians and pharmacists and is beginning to expand to other disciplines. Additional investments in VHA workforce training include the availability of geropsychology internship and fellowship training opportunities through the Office of Academic Affiliations, which provide earlier opportunities to specialize in geropsychology. Continued investment in both prelicensure and postpsychology licensure training efforts are needed within the VHA to meet the geriatric mental health needs of veterans.

Acknowledgments

The authors wish to acknowledge Terri Huh, PhD, for her contributions to the development and initiation of the GSP-P. The authors also appreciate the collaboration and quality initiative training led by Carol Callaway-Lane, DNP, ACNP-BC, and her team.

The Veterans Health Administration (VHA) is understaffed for clinical psychologists who have specialty training in geriatrics (ie, geropsychologists) to meet the needs of aging veterans. Though only 16.8% of US adults are aged ≥ 65 years,¹ this age group comprises 45.9% of patients within the VHA.² The needs of older adults are complex and warrant specialized services from mental health clinicians trained to understand lifespan developmental processes, biological changes associated with aging, and changes in psychosocial functioning.

Older veterans (aged ≥ 65 years) present with higher rates of combined medical and mental health diagnoses compared to both younger veterans and older adults who are not veterans.³ Nearly 1 of 5 (18.1%) older veterans who use VHA services have confirmed mental health diagnoses, and an additional 25.5% have documented mental health concerns without a formal diagnosis in their health record.⁴ The clinical presentations of older veterans frequently differ from younger adults and include greater complexity. For example, older veterans face an increased risk of cognitive impairment compared to the general population, due in part to higher prevalence of posttraumatic stress, which doubles their risk of developing dementia.⁵ Additional examples of multicomplexity among older veterans may include co-occurring medical and psychiatric diagnoses, the presence of delirium, social isolation/loneliness, and concerns related to polypharmacy. These complex presentations result in significant challenges for mental health clinicians in areas like assessment (eg, accuracy of case conceptualization), intervention (eg, selection and prioritization), and consultation (eg, coordination among multiple medical and mental health specialists).

Older veterans also present with substantial resilience. Research has found that aging veterans exposed to trauma during their military service often review their memories and past experiences, which is known as later-adulthood trauma reengagement.⁶ Through this normative life review process, veterans engage with memories and experiences from their past that they previously avoided, which could lead to posttraumatic growth for some. Unfortunately, others may experience an increase in psychological distress. Mental health clinicians with specialty expertise and training in aging and lifespan development can facilitate positive outcomes to reduce distress.⁷

The United States in general, and the VHA specifically, face a growing shortage of geriatric mental health clinicians. In a 2015 American Psychological Association survey, 1528 of 4109 respondents (37.2%) reported they frequently or very frequently administered care to older adults, yet only 49 respondents (1.2%) reported geropsychology as their specialty.⁸ According to the National Provider Registry, 660 clinicians self-identified as geropsychologists (ie, those who self-reported “psychologist: adult development and aging” as an NPI Healthcare Provider taxonomy code) in the US, representing < 1% of all doctoral level psychologists.⁹ The number of psychologists who obtain board certification from the American Board of Geropsychology is even lower (only 112 clinicians as of February 2024).¹⁰ Many psychologists within the VHA treat older veterans in integrated health settings such as primary care, home-based primary care, community living centers, or behavioral health care, but many lack formal training in geropsychology.

The Geriatric Scholars Program (GSP) was developed in 2008 to address the training gap and provide education in geriatrics to VHA clinicians that treat older veterans, particularly in rural areas.^11,12 The GSP initially focused on primary care physicians, nurse practitioners, physician assistants, and pharmacists. It was later expanded to include other disciplines (ie, social work, rehabilitation therapists, and psychiatrists). In 2013, the GSP – Psychology Track (GSP-P) was developed with funding from the VHA Offices of Rural Health and Geriatrics and Extended Care specifically for psychologists.

This article describes the multicomponent longitudinal GSP-P, which has evolved to meet the target audience’s ongoing needs for knowledge, skills, and opportunities to refine practice behaviors. GSP-P received the 2020 Award for Excellence in Geropsychology Training from the Council of Professional Geropsychology Training Programs. GSP-P has grown within the context of the larger GSP and aligns with the other existing elective learning opportunities (Figure 1).

Program description

Introductory Course

Psychologist subject matter experts (SMEs) developed an intensive course in geropsychology in the absence of a similar course in the geriatric medicine board review curriculum. SMEs reviewed the guidelines for practice by professional organizations like the Pikes Peak Geropsychology Competencies,which outline knowledge and skills in various domains.¹³ SMEs integrated this review with findings from a needs assessment for postlicensed VHA psychology staff in 4 health care systems, drafted a syllabus, and circulated it to geropsychology experts for feedback. The resulting multiday course covered general mental health as well as topics particularly salient for mental health clinicians treating older veterans including suicide prevention and posttraumatic stress disorder (PTSD).¹⁴ This Geropsychology Competencies Review Course was piloted in 1 region initially before being offered nationally in 2014. The previously published evaluation findings demonstrated significant improvements from precourse to 3 months postcourse in confidence and knowledge in 4 key areas of geropsychology: General knowledge about adult development and aging, assessment, intervention, and consultation.¹⁵ These domains align with the Pikes Peak Geropsychology Competencies and are measured by a subset of items from the Geropsychology Knowledge and Skills Assessment Tool.^13,16 As of October 2023, 8 introductory courses have been held with 173 participants (Table).

Quality Improvement

Introductory course attendees also participate in an intensive day-long interactive workshop in quality improvement (QI). After completing these trainings, they apply what they have learned at their home facility by embarking on a QI project related to geriatrics. The QI projects reinforce learning and initiate practice changes not only for attendees but at times the larger health care system. Topics are selected by scholars in response to the needs they observe in their clinics. Recent GSP projects include efforts to increase screenings for depression and anxiety, improve adherence to VHA dementia policy, increase access to virtual care, and increase referrals to programs such as whole health or cognitive behavioral therapy for insomnia, a first-line treatment for insomnia in older adults.¹⁷ Another project targeted the improvement of referrals to the Compassionate Contact Corps in an effort to reduce social isolation and loneliness among older veterans.¹⁸ Evaluations demonstrate significant improvement in scholars’ confidence in related program development and management from precourse to 3 months postcourse.¹⁵

Webinars

The Addressing Geriatric Mental Health webinar series was created to introduce learners to topics that could not be covered in the introductory course. Topics were suggested by the expert reviewers of the curriculum or identified by the scholars themselves (eg, chronic pain, sexuality, or serious mental illness). A secondary function of the webinars was to reach a broader audience. Over time, scholars and webinar attendees requested opportunities to explore topics in greater depth (eg, PTSD later in life). These requests led the webinars to focus on annual themes.

The series is open to all disciplines of geriatric scholars, VHA staff, and non-VHA staff through the Veterans Affairs Talent Management System and the TRAIN Learning Network (train.org). Attendance for the 37 webinars was captured from logins to the virtual learning platform and may underestimate attendance if a group attended on a single screen. Average attendance increased from 157 attendees/webinar in 2015 to 418 attendees/webinar in 2023 (Figure 2). This may have been related to the increase in virtual learning during the COVID-19 pandemic, but represents a 166% increase in audience from the inaugural year of the series.

Advanced Learning Opportunities

To invest in the ongoing growth and development of introductory course graduates, GSP-P developed and offered an advanced workshop in 2019. This multiday workshop focused on further enhancement of geropsychology competencies, with an emphasis on treating older veterans with mental and physical comorbidities. Didactics and experiential learning exercises led by SMEs covered topics such as adjusting to chronic illness, capacity assessment, PTSD, insomnia and sleep changes, chronic pain, and psychological interventions in palliative care and hospice settings. Evaluation findings demonstrated significant improvements from precourse to 6 months postcourse in confidence and knowledge as defined by the Pikes Peak Geropsychology Competencies.¹⁹

To facilitate ongoing and individually tailored learning following the advanced workshop, scholars also developed and executed independent learning plans (ILPs) during a 6-month window with consultation from an experienced geropsychologist. Fifteen of 19 scholars (78.9%) completed ILPs with an average of 3 learning goals listed. After completing the ILPs, scholars endorsed their clinical and/or personal usefulness, citing increased confidence, enhanced skills for use with patients with complex needs, personal fulfillment, and career advancement. Most scholars noted ILPs were feasible and learning resources were accessible. Overall, the evaluation found ILPs to be a valuable way to enhance psychologists’ learning and effectiveness in treating older veterans with complex health needs.²⁰

All geriatric scholars who completed the program have additional opportunities for professional development through practicum experiences focused on specific clinical approaches to the care of older veterans, such as dementia care, pain management, geriatric assessment, and palliative care. These practica provide scholars with individualized learning experiences in an individualized or small group setting and may be conducted either in-person or virtually.

In response to an expressed need from those who completed the program, the GSP-P planning committee collaborated with an SME to develop a virtual practicum to assess patients’ decision making capacity. Evaluating capacities among older adults is a common request, yet clinicians report little to no formal training in how to conceptualize and approach these complex questions.^21,22 Utilizing an evidence-informed and structured approach promotes the balancing of an older adult’s autonomy and professional ethics. Learning capacity evaluation skills could better position psychologists to not only navigate complex ethical, legal, and clinical situations, but also serve as expert consultants to interdisciplinary teams. This virtual practicum was initiated in 2022 and to date has included 10 scholars. The practicum includes multiple modalities of learning: (1) self-directed review of core concepts; (2) attendance at 4 capacity didactics focused on introduction to evaluating capacities, medical consent and decision making, financial decision making and management, and independent living; and (3) participation in 5 group consultations on capacity evaluations conducted at their home sites. During these group consultations, additional case examples were shared to reinforce capacity concepts.

Discussion

The objective of GSP-P is to enhance geropsychology practice competencies among VHA psychologists given the outsized representation of older adults within the VHA system and their complex care needs. The curricula have significantly evolved to accomplish this, expanding the reach and investing in the continuing growth and development of scholars.

There are several elements that set GSP apart from other geriatric and geropsychology continuing medical education programs. The first is that the training is veteran focused, allowing us to discuss the unique impact military service has on aging. Similarly, because all scholars work within the integrated health care system, we can introduce and review key resources and programs that benefit all veterans and their families/care partners across the system. Through the GSP, the VA invests in ongoing professional development. Scholars can participate in additional experiential practica, webinars, and advanced workshops tailored to their individual learning needs. Lastly, the GSP works to create a community among its scholars where they can not only continue to consult with presenters/instructors, but also one another. A planned future direction for the GSP-P is to incorporate quarterly office hours and discussions for alumni to develop an increased sense of community. This may strengthen commitment to the overall VA mission, leading to increased retainment of talent who now have the knowledge, skills, and confidence to care for aging veterans.

Limitations

GSP is limited by its available funding. Additionally, the number of participants who can enroll each year in GSP-P (not including webinars) is capped by policy. Another limitation is the number of QI coaches available to mentor scholars on their projects.

Outcomes of GSP-P have been extremely favorable. Following participation in the program, we have found a significant increase in confidence in geropsychology practice among clinicians, as well as enhanced knowledge and skills across competency domains.^15,19 We have observed rising attendance in our annual webinar series and graduates of our introductory courses participate in subsequent trainings (eg, advanced workshop or virtual practicum). Several graduates of GSP-P have become board certified in geropsychology by the American Board of Geropsychology and many proceed to supervise geropsychology-focused clinical rotations for psychology practicum students, predoctoral interns, and postdoctoral fellows. This suggests that the reach of GSP-P programming may extend farther than reported in this article.

The needs of aging veterans have also changed based on cohort differences, as the population of World War II and Korean War era veterans has declined and the number of older Vietnam era veterans has grown. We expect different challenges with older Gulf War and post-9/11 era veterans. For instance, 17% of troops deployed to Iraq or Afghanistan following 9/11 experienced mild traumatic brain injury (TBI), and 59% of those experienced > 1 mild TBI.²³ Research indicates that younger post-9/11 veterans have a 3-fold risk of developing early onset dementia after experiencing a TBI.²⁴ Therefore, even though post-9/11 veterans are not older in terms of chronological age, some may experience symptoms and conditions more often occurring in older veterans. As a result, it would be beneficial for clinicians to learn about the presentation and treatment of geriatric conditions such as dementia.

Moving forward, the GSP-P should identify potential opportunities to collaborate with the non-VHA mental health community–which also faces a shortage of geriatric mental health clinicians–to extend educational opportunities to improve care for veterans in all settings (eg, cosponsor training opportunities open to both VHA and non-VHA clinicians).^8,25 Many aging veterans may receive portions of their health care outside the VHA, particularly those who reside in rural areas. Additionally, as veterans age, so do their support systems (eg, family members, friends, spouses, caregivers, and even clinicians), most of whom will receive care outside of the VHA. Community education collaborations will not only improve the care of older veterans, but also the care of older adults in the general population.

Promising directions include the adoption of the GSP model in other health care settings. Recently, Indian Health Service has adapted the model, beginning with primary care clinicians and pharmacists and is beginning to expand to other disciplines. Additional investments in VHA workforce training include the availability of geropsychology internship and fellowship training opportunities through the Office of Academic Affiliations, which provide earlier opportunities to specialize in geropsychology. Continued investment in both prelicensure and postpsychology licensure training efforts are needed within the VHA to meet the geriatric mental health needs of veterans.

Acknowledgments

The authors wish to acknowledge Terri Huh, PhD, for her contributions to the development and initiation of the GSP-P. The authors also appreciate the collaboration and quality initiative training led by Carol Callaway-Lane, DNP, ACNP-BC, and her team.

The Veterans Health Administration (VHA) is understaffed for clinical psychologists who have specialty training in geriatrics (ie, geropsychologists) to meet the needs of aging veterans. Though only 16.8% of US adults are aged ≥ 65 years,¹ this age group comprises 45.9% of patients within the VHA.² The needs of older adults are complex and warrant specialized services from mental health clinicians trained to understand lifespan developmental processes, biological changes associated with aging, and changes in psychosocial functioning.

Older veterans (aged ≥ 65 years) present with higher rates of combined medical and mental health diagnoses compared to both younger veterans and older adults who are not veterans.³ Nearly 1 of 5 (18.1%) older veterans who use VHA services have confirmed mental health diagnoses, and an additional 25.5% have documented mental health concerns without a formal diagnosis in their health record.⁴ The clinical presentations of older veterans frequently differ from younger adults and include greater complexity. For example, older veterans face an increased risk of cognitive impairment compared to the general population, due in part to higher prevalence of posttraumatic stress, which doubles their risk of developing dementia.⁵ Additional examples of multicomplexity among older veterans may include co-occurring medical and psychiatric diagnoses, the presence of delirium, social isolation/loneliness, and concerns related to polypharmacy. These complex presentations result in significant challenges for mental health clinicians in areas like assessment (eg, accuracy of case conceptualization), intervention (eg, selection and prioritization), and consultation (eg, coordination among multiple medical and mental health specialists).

Older veterans also present with substantial resilience. Research has found that aging veterans exposed to trauma during their military service often review their memories and past experiences, which is known as later-adulthood trauma reengagement.⁶ Through this normative life review process, veterans engage with memories and experiences from their past that they previously avoided, which could lead to posttraumatic growth for some. Unfortunately, others may experience an increase in psychological distress. Mental health clinicians with specialty expertise and training in aging and lifespan development can facilitate positive outcomes to reduce distress.⁷

The United States in general, and the VHA specifically, face a growing shortage of geriatric mental health clinicians. In a 2015 American Psychological Association survey, 1528 of 4109 respondents (37.2%) reported they frequently or very frequently administered care to older adults, yet only 49 respondents (1.2%) reported geropsychology as their specialty.⁸ According to the National Provider Registry, 660 clinicians self-identified as geropsychologists (ie, those who self-reported “psychologist: adult development and aging” as an NPI Healthcare Provider taxonomy code) in the US, representing < 1% of all doctoral level psychologists.⁹ The number of psychologists who obtain board certification from the American Board of Geropsychology is even lower (only 112 clinicians as of February 2024).¹⁰ Many psychologists within the VHA treat older veterans in integrated health settings such as primary care, home-based primary care, community living centers, or behavioral health care, but many lack formal training in geropsychology.

The Geriatric Scholars Program (GSP) was developed in 2008 to address the training gap and provide education in geriatrics to VHA clinicians that treat older veterans, particularly in rural areas.^11,12 The GSP initially focused on primary care physicians, nurse practitioners, physician assistants, and pharmacists. It was later expanded to include other disciplines (ie, social work, rehabilitation therapists, and psychiatrists). In 2013, the GSP – Psychology Track (GSP-P) was developed with funding from the VHA Offices of Rural Health and Geriatrics and Extended Care specifically for psychologists.

This article describes the multicomponent longitudinal GSP-P, which has evolved to meet the target audience’s ongoing needs for knowledge, skills, and opportunities to refine practice behaviors. GSP-P received the 2020 Award for Excellence in Geropsychology Training from the Council of Professional Geropsychology Training Programs. GSP-P has grown within the context of the larger GSP and aligns with the other existing elective learning opportunities (Figure 1).

Program description

Introductory Course

Psychologist subject matter experts (SMEs) developed an intensive course in geropsychology in the absence of a similar course in the geriatric medicine board review curriculum. SMEs reviewed the guidelines for practice by professional organizations like the Pikes Peak Geropsychology Competencies,which outline knowledge and skills in various domains.¹³ SMEs integrated this review with findings from a needs assessment for postlicensed VHA psychology staff in 4 health care systems, drafted a syllabus, and circulated it to geropsychology experts for feedback. The resulting multiday course covered general mental health as well as topics particularly salient for mental health clinicians treating older veterans including suicide prevention and posttraumatic stress disorder (PTSD).¹⁴ This Geropsychology Competencies Review Course was piloted in 1 region initially before being offered nationally in 2014. The previously published evaluation findings demonstrated significant improvements from precourse to 3 months postcourse in confidence and knowledge in 4 key areas of geropsychology: General knowledge about adult development and aging, assessment, intervention, and consultation.¹⁵ These domains align with the Pikes Peak Geropsychology Competencies and are measured by a subset of items from the Geropsychology Knowledge and Skills Assessment Tool.^13,16 As of October 2023, 8 introductory courses have been held with 173 participants (Table).

Quality Improvement

Introductory course attendees also participate in an intensive day-long interactive workshop in quality improvement (QI). After completing these trainings, they apply what they have learned at their home facility by embarking on a QI project related to geriatrics. The QI projects reinforce learning and initiate practice changes not only for attendees but at times the larger health care system. Topics are selected by scholars in response to the needs they observe in their clinics. Recent GSP projects include efforts to increase screenings for depression and anxiety, improve adherence to VHA dementia policy, increase access to virtual care, and increase referrals to programs such as whole health or cognitive behavioral therapy for insomnia, a first-line treatment for insomnia in older adults.¹⁷ Another project targeted the improvement of referrals to the Compassionate Contact Corps in an effort to reduce social isolation and loneliness among older veterans.¹⁸ Evaluations demonstrate significant improvement in scholars’ confidence in related program development and management from precourse to 3 months postcourse.¹⁵

Webinars

The Addressing Geriatric Mental Health webinar series was created to introduce learners to topics that could not be covered in the introductory course. Topics were suggested by the expert reviewers of the curriculum or identified by the scholars themselves (eg, chronic pain, sexuality, or serious mental illness). A secondary function of the webinars was to reach a broader audience. Over time, scholars and webinar attendees requested opportunities to explore topics in greater depth (eg, PTSD later in life). These requests led the webinars to focus on annual themes.

The series is open to all disciplines of geriatric scholars, VHA staff, and non-VHA staff through the Veterans Affairs Talent Management System and the TRAIN Learning Network (train.org). Attendance for the 37 webinars was captured from logins to the virtual learning platform and may underestimate attendance if a group attended on a single screen. Average attendance increased from 157 attendees/webinar in 2015 to 418 attendees/webinar in 2023 (Figure 2). This may have been related to the increase in virtual learning during the COVID-19 pandemic, but represents a 166% increase in audience from the inaugural year of the series.

Advanced Learning Opportunities

To invest in the ongoing growth and development of introductory course graduates, GSP-P developed and offered an advanced workshop in 2019. This multiday workshop focused on further enhancement of geropsychology competencies, with an emphasis on treating older veterans with mental and physical comorbidities. Didactics and experiential learning exercises led by SMEs covered topics such as adjusting to chronic illness, capacity assessment, PTSD, insomnia and sleep changes, chronic pain, and psychological interventions in palliative care and hospice settings. Evaluation findings demonstrated significant improvements from precourse to 6 months postcourse in confidence and knowledge as defined by the Pikes Peak Geropsychology Competencies.¹⁹

To facilitate ongoing and individually tailored learning following the advanced workshop, scholars also developed and executed independent learning plans (ILPs) during a 6-month window with consultation from an experienced geropsychologist. Fifteen of 19 scholars (78.9%) completed ILPs with an average of 3 learning goals listed. After completing the ILPs, scholars endorsed their clinical and/or personal usefulness, citing increased confidence, enhanced skills for use with patients with complex needs, personal fulfillment, and career advancement. Most scholars noted ILPs were feasible and learning resources were accessible. Overall, the evaluation found ILPs to be a valuable way to enhance psychologists’ learning and effectiveness in treating older veterans with complex health needs.²⁰

All geriatric scholars who completed the program have additional opportunities for professional development through practicum experiences focused on specific clinical approaches to the care of older veterans, such as dementia care, pain management, geriatric assessment, and palliative care. These practica provide scholars with individualized learning experiences in an individualized or small group setting and may be conducted either in-person or virtually.

In response to an expressed need from those who completed the program, the GSP-P planning committee collaborated with an SME to develop a virtual practicum to assess patients’ decision making capacity. Evaluating capacities among older adults is a common request, yet clinicians report little to no formal training in how to conceptualize and approach these complex questions.^21,22 Utilizing an evidence-informed and structured approach promotes the balancing of an older adult’s autonomy and professional ethics. Learning capacity evaluation skills could better position psychologists to not only navigate complex ethical, legal, and clinical situations, but also serve as expert consultants to interdisciplinary teams. This virtual practicum was initiated in 2022 and to date has included 10 scholars. The practicum includes multiple modalities of learning: (1) self-directed review of core concepts; (2) attendance at 4 capacity didactics focused on introduction to evaluating capacities, medical consent and decision making, financial decision making and management, and independent living; and (3) participation in 5 group consultations on capacity evaluations conducted at their home sites. During these group consultations, additional case examples were shared to reinforce capacity concepts.

Discussion

The objective of GSP-P is to enhance geropsychology practice competencies among VHA psychologists given the outsized representation of older adults within the VHA system and their complex care needs. The curricula have significantly evolved to accomplish this, expanding the reach and investing in the continuing growth and development of scholars.

There are several elements that set GSP apart from other geriatric and geropsychology continuing medical education programs. The first is that the training is veteran focused, allowing us to discuss the unique impact military service has on aging. Similarly, because all scholars work within the integrated health care system, we can introduce and review key resources and programs that benefit all veterans and their families/care partners across the system. Through the GSP, the VA invests in ongoing professional development. Scholars can participate in additional experiential practica, webinars, and advanced workshops tailored to their individual learning needs. Lastly, the GSP works to create a community among its scholars where they can not only continue to consult with presenters/instructors, but also one another. A planned future direction for the GSP-P is to incorporate quarterly office hours and discussions for alumni to develop an increased sense of community. This may strengthen commitment to the overall VA mission, leading to increased retainment of talent who now have the knowledge, skills, and confidence to care for aging veterans.

Limitations

GSP is limited by its available funding. Additionally, the number of participants who can enroll each year in GSP-P (not including webinars) is capped by policy. Another limitation is the number of QI coaches available to mentor scholars on their projects.

Outcomes of GSP-P have been extremely favorable. Following participation in the program, we have found a significant increase in confidence in geropsychology practice among clinicians, as well as enhanced knowledge and skills across competency domains.^15,19 We have observed rising attendance in our annual webinar series and graduates of our introductory courses participate in subsequent trainings (eg, advanced workshop or virtual practicum). Several graduates of GSP-P have become board certified in geropsychology by the American Board of Geropsychology and many proceed to supervise geropsychology-focused clinical rotations for psychology practicum students, predoctoral interns, and postdoctoral fellows. This suggests that the reach of GSP-P programming may extend farther than reported in this article.

The needs of aging veterans have also changed based on cohort differences, as the population of World War II and Korean War era veterans has declined and the number of older Vietnam era veterans has grown. We expect different challenges with older Gulf War and post-9/11 era veterans. For instance, 17% of troops deployed to Iraq or Afghanistan following 9/11 experienced mild traumatic brain injury (TBI), and 59% of those experienced > 1 mild TBI.²³ Research indicates that younger post-9/11 veterans have a 3-fold risk of developing early onset dementia after experiencing a TBI.²⁴ Therefore, even though post-9/11 veterans are not older in terms of chronological age, some may experience symptoms and conditions more often occurring in older veterans. As a result, it would be beneficial for clinicians to learn about the presentation and treatment of geriatric conditions such as dementia.

Moving forward, the GSP-P should identify potential opportunities to collaborate with the non-VHA mental health community–which also faces a shortage of geriatric mental health clinicians–to extend educational opportunities to improve care for veterans in all settings (eg, cosponsor training opportunities open to both VHA and non-VHA clinicians).^8,25 Many aging veterans may receive portions of their health care outside the VHA, particularly those who reside in rural areas. Additionally, as veterans age, so do their support systems (eg, family members, friends, spouses, caregivers, and even clinicians), most of whom will receive care outside of the VHA. Community education collaborations will not only improve the care of older veterans, but also the care of older adults in the general population.

Promising directions include the adoption of the GSP model in other health care settings. Recently, Indian Health Service has adapted the model, beginning with primary care clinicians and pharmacists and is beginning to expand to other disciplines. Additional investments in VHA workforce training include the availability of geropsychology internship and fellowship training opportunities through the Office of Academic Affiliations, which provide earlier opportunities to specialize in geropsychology. Continued investment in both prelicensure and postpsychology licensure training efforts are needed within the VHA to meet the geriatric mental health needs of veterans.

Acknowledgments

The authors wish to acknowledge Terri Huh, PhD, for her contributions to the development and initiation of the GSP-P. The authors also appreciate the collaboration and quality initiative training led by Carol Callaway-Lane, DNP, ACNP-BC, and her team.

Urine drug screen (UDS) monitoring is a common risk-mitigation strategy tool for prescribing controlled substances.^1-3 Not only is UDS monitoring highlighted by clinical practice guidelines for opioid prescribing for chronic pain,^1,2 it has also been suggested as best practice for benzodiazepines³ and a consideration for other controlled substances. Monitoring UDSs helps confirm adherence to the prescribed treatment regimen while also screening for substance use that may increase patient risk.

UDS results can be complex and have profound implications for the patient’s treatment plan. Drug metabolites for opioids are particularly complicated; for example, synthetic and semisynthetic opioids are not detected on routine opiate immunoassays.⁴ This may lead a clinician to falsely assume the patient is not taking their fentanyl or tramadol medication as directed—or potentially even diverting—in the face of a negative opiate result.⁵ Routine UDSs are also subject to the pitfall of false-positive results due to coprescribed medications; for example, bupropion can lead to a false-positive amphetamine result, whereas sertraline can lead to a false-positive benzodiazepine result.⁶ Retrospective reviews of clinician behavior surrounding UDS interpretation have demonstrated knowledge gaps and inconsistent communication practices with patients.^7,8

Given the complexity of UDS interpretation and its close relationship with medications, pharmacists are positioned to play an important role in the process. Pharmacists are embedded in pain-management teams and involved in prescription drug monitoring programs (PDMPs) for many health systems. The Veterans Health Administration (VHA) has supported the hiring of pain management, opioid safety, and PDMP coordinators (PMOP) at its facilities to provide clinical pain-management guidance, support national initiatives, and uphold legislative requirements.⁹ In many facilities, a pharmacist is hired specifically for these positions.

Clinical dashboards have been used by pharmacists in a variety of settings.^10-13 They allow clinicians at a broad level to target interventions needed across a patient population, then produce a list of actionable patients to facilitate delivery of that intervention on an individual level.¹³ Between 2021 and 2022, a clinical dashboard to review potentially discrepant UDS results was made available for use at US Department of Veterans Affairs (VA) medical centers. Evidence exists in primary and specialty care settings that implementation of an opioid-prescribing clinical dashboard improves completion rates of risk-mitigation strategies such as UDS and opioid treatment agreements.^14,15 To our knowledge there is no published research on the use and outcomes of a clinical dashboard that allows users to efficiently review discrepant UDS results when compared to a list of currently prescribed medications.

Given the availability of the UDS dashboard at the VA Black Hills Health Care System (VABHHCS) in South Dakota and the hiring of a PMOP coordinator pharmacist, the aim of this quality improvement project was 2-fold: to implement a pharmacist-led process to monitor the UDS dashboard for potentially discrepant results and to describe the quantity and types of interventions made by the clinical pharmacist leading this process.

Quality Improvement Project

A clinical UDS dashboard was created by the VA Northwest Health Network and made available for use at VHA sites between 2021 and 2022. The UDS dashboard is housed on a secure, Power BI Report Server (Microsoft), with access restricted to only those with patient health data privileges. The dashboard identifies all local patients with a UDS that returned with a potential discrepancy, defined as an unexpected positive result (eg, a detected substance not recently prescribed or documented on the patient’s medication list) and/or an unexpected negative result (eg, a prescribed substance not detected). The UDS dashboard identifies these discrepancies by comparing the patient’s current medication list (both VHA and non-VHA) to their UDS results.

The UDS dashboard displays a summary of UDSs performed, unexpected negative results, unexpected positive results, and potential discrepancies. The user may also specify the laboratory type and time frame of interest to limit displayed results. The user can then view patient-specific data for any category. Among the data are the patient’s UDS results and the completion date, detected (or nondetected) substance(s), ordering clinician, associated medication(s) with last fill date and days’ supply, and whether a confirmatory test has been performed in the past year.

VABHHCS uses an extended UDS immunoassay (PROFILE-V, MEDTOX Diagnostics) that reports on 11 substances: opiates, oxycodone, buprenorphine, methadone, amphetamines, methamphetamine, barbiturates, benzodiazepines, cocaine metabolites, cannabinoids (tetrahydrocannabinol [THC]), and phencyclidine. These substances appear on the UDS dashboard. The project protocol initially included monitoring for tramadol but that was later removed because it was not available with this UDS immunoassay.

Pharmacist Process

Either the PMOP coordinator or pharmacy resident monitored the UDS dashboard weekly. Any patients identified as having a potential discrepancy were reviewed. If the discrepancy was determined to be significant, the PMOP coordinator or pharmacy resident would review the patient electronic health record. If warranted, the patient was contacted and asked about newly prescribed medications, missed and recent medication doses, and illicit substance use. Potential interventions during in-depth review included: (1) discussing future actions with the primary care clinician and/or prescriber of the controlled substance; (2) ordering a confirmatory test on the original urine sample; (3) evaluating for sources of potential false-positive results; (4) completing an updated PDMP if not performed within the past year; (5) referring patients for substance use disorder treatment or counseling; or (6) consulting the local narcotics review committee. A progress note was entered into the electronic health record with the findings and any actions taken, and an alert for the primary care clinician and/or prescriber of the controlled substance.

Implementation and Analysis

This quality improvement project spanned 16 weeks from June 2022 through September 2022. Any patient with a UDS that returned with a significant discrepancy was reviewed. The primary outcome was interventions made by the PMOP coordinator or pharmacy resident, as well as time taken to perform the in-depth review of each patient. Patient demographics were also collected. The protocol for this project was approved by the VABHHCS pharmacy and therapeutics committee and was determined to meet guidelines for a nonresearch quality improvement project.

Results

From June 2022 through September 2022, 700 UDSs were performed at VABHHCS with 278 (39.7%) patients identified as having a potential discrepancy based on UDS results. Sixty patients (8.6%) had significant discrepancies that warranted in-depth review. The most common reasons for determining whether a potential discrepancy was not significant included unexpected negatives due to documented non-VA medications no longer being prescribed, unexpected positives due to recent expiration of a controlled substance prescription the patient was still taking, or unexpected positives due to the detection of a substance for which the clinician was already aware. During the 16-week study period, the mean number of patients warranting in-depth review was 4 per week.

The patients were predominantly male with a mean age of 61 years, and most (87%) were prescribed at least 1 controlled substance (mean, 1.1), primarily opioids for pain management (Table 1). Most patients had recent substance risk mitigation with UDS (56%) and PDMP (65%) checks within the past year. Of the 60 patients reviewed with significant UDS discrepancies, 50% had a history of discrepant UDS results. Of the 60 UDS discrepancies, there were 37 unexpected positive results (62%), 17 unexpected negative results (28%), and 10 patients with both positive and negative results (17%). THC was the most frequently detected substance, followed by opiates, benzodiazepines, and amphetamines (Table 2).

Each in-depth review with interventions by the PMOP coordinator or pharmacy resident lasted a mean of 14 minutes (Table 3). Five patients were successfully contacted for an interview and 7 patients could not be contacted. The ordering clinician of the UDS sometimes had contacted these patients prior to the PMOP coordinator or pharmacy resident reviewing the UDS dashboard, eliminating the need for additional follow-up.

The most common pharmacist intervention was discussing future actions with the primary care clinician and/or prescriber of the controlled substance (n = 39; 65%). These conversations resulted in actions such as ordering a repeat UDS with confirmatory testing at a future date or agreeing that the clinician would discuss the results and subsequent actions with the patient at an upcoming visit. Pharmacist interventions also included 25 PDMP queries (42%) and 9 orders of confirmatory UDS on the original urine sample (15%). Only 1 patient was evaluated by the narcotics review committee, which resulted in a controlled substance flag being placed on their profile. No patients were referred to substance use disorder treatment or counseling. It was offered to and declined by 1 patient, and 3 patients were already engaged in these services.

Medication therapies that could contribute to false-positive results were also evaluated. Fourteen patients who tested positive for THC had a prescription for a nonsteroidal anti-inflammatory drug or proton-pump inhibitor, which could have created a false-positive result.⁶ One patient who tested positive for amphetamines had a prescription for phentermine.¹⁶ No other potential false-positive results were identified.

Findings of this project illustrate that the use of a clinical pharmacist to monitor a dashboard of discrepant UDS results created opportunities for collaboration with clinicians and impacted confirmatory testing and PDMP monitoring practices.

At the local level, the process had numerous benefits. First, it was a reasonable amount of workload to generate pharmacist interventions: the PMOP coordinator conducted an average of 4 in-depth reviews weekly, each lasting about 14 minutes. Thus, the UDS dashboard allowed the PMOP coordinator to actively surveil all incoming UDS results for potential discrepancies in about 1 hour each week. Pairing the automation of the UDS dashboard with the clinical judgment of the PMOP coordinator seemed to maximize efficiency. VABHHCS provides primary and secondary medical and surgical care to a rural population of approximately 20,000 patients across 5 states; the time required at facilities that serve a higher volume of patients may be greater.

Second, the project served as an opportunity for the PMOP coordinator to provide case-specific clinician education on UDS monitoring. As medication experts, pharmacists can apply their medication-related knowledge to UDS interpretation. This includes understanding drug metabolism and classification and how they apply to UDS results, as well as recognizing medication therapies that could contribute to false-positive UDS results. Research suggests that clinicians may have gaps in their knowledge and may welcome pharmacist assistance in interpreting UDS results.^7,8

Third, the project helped improve rates of confirmatory testing for those with unexpected positive UDS results. Confirmatory testing should be strongly considered if positive results would have significant implications on the future course of treatment.⁴ The PMOP coordinator ordered a confirmatory test on 9 patients using the same urine sample used to conduct the initial UDS, minimizing the burden on the patient and laboratory staff. Confirmatory testing was limited by the laboratory’s sample retention period; if the need for confirmatory testing was not recognized soon enough, the sample would no longer be available for retesting. Health systems may consider the use of reflexive confirmatory testing with UDS as an alternative approach, although this may come at an additional cost and may not be warranted in many cases (eg, only 39.7% of all potential discrepancies were deemed as significant within our project).

There were notable incidental findings in our quality improvement project. Among patients with a significant discrepancy on UDS, 50% had a history of ≥ 1 discrepant UDS result. This further emphasizes the importance of appropriate use and interpretation of UDS monitoring for all clinicians, as this may prevent prolonged and potentially inappropriate treatment regimens. Secondly, rates of mental health diagnoses among those with a significant UDS discrepancy seemed relatively high compared to population-level data. For example, among veterans, the overall lifetime prevalence of posttraumatic stress disorder is estimated to be 8.0%; in our project, 35% of patients with a significant UDS discrepancy had a posttraumatic stress disorder diagnosis.¹⁷ This relationship may be an area of further study.

Lastly, it was surprising that the overall rates of UDS and PDMP checks within the past year were 56% and 65%, respectively. VABHHCS requires veterans on controlled substances to have these risk-mitigation strategies performed annually, so our suspicion is that many were falling out due to having been most recently evaluated 12 to 16 months prior. This may represent a limitation of our data-collection method, which reviewed only the previous 12 months.

Limitations

This project was carried out over a period of only 4 months. As a result, only 60 patients received an in-depth review from the PMOP coordinator. Second, the timeliness of the intervention seemed crucial, as delayed in-depth reviews resulted in fewer opportunities to order confirmatory tests or collaborate with clinicians prior to devising an updated plan. Additionally, our process called for UDS dashboard monitoring once a week. Given that the laboratory held samples for only 48 hours, twice- or thrice-weekly review of the UDS dashboard would have allowed for more confirmatory testing, along with more immediate clinician collaboration. Most importantly, the outcomes of this project are only presented via descriptive statistics and without the results of any comparison group, making it impossible to draw firm conclusions about this approach compared to standard-care processes.

Conclusions

This quality improvement project has proven to be valuable at VABHHCS and we intend to continue this pharmacist-led process to monitor the UDS dashboard. VABHHCS leadership are also discussing UDS practices more broadly to further enhance patient management. Within the VA, the PMOP coordinator—charged with being the local coordinator of appropriate pain management and opioid safety practices—is well positioned to assume these responsibilities. Outside of the VA, a pain-management clinical pharmacist or any pharmacist embedded within primary care could similarly perform these duties. Previous literature regarding the implementation of clinical dashboards suggests that with the appropriate software engineering teams and infrastructure, this tool could also be feasibly developed and implemented at other health systems relatively quickly.¹⁴

Overall, a pharmacist-led process to efficiently monitor a dashboard of discrepant UDS results led to opportunities for collaboration with clinicians and positively impacted confirmatory testing and PDMP monitoring at a rural VA health system.

Acknowledgments

The authors express their gratitude to Patrick Spoutz, PharmD, BCPS, VISN 20 Pharmacist Executive, for introducing and sharing the UDS dashboard with our team.

Urine drug screen (UDS) monitoring is a common risk-mitigation strategy tool for prescribing controlled substances.^1-3 Not only is UDS monitoring highlighted by clinical practice guidelines for opioid prescribing for chronic pain,^1,2 it has also been suggested as best practice for benzodiazepines³ and a consideration for other controlled substances. Monitoring UDSs helps confirm adherence to the prescribed treatment regimen while also screening for substance use that may increase patient risk.

UDS results can be complex and have profound implications for the patient’s treatment plan. Drug metabolites for opioids are particularly complicated; for example, synthetic and semisynthetic opioids are not detected on routine opiate immunoassays.⁴ This may lead a clinician to falsely assume the patient is not taking their fentanyl or tramadol medication as directed—or potentially even diverting—in the face of a negative opiate result.⁵ Routine UDSs are also subject to the pitfall of false-positive results due to coprescribed medications; for example, bupropion can lead to a false-positive amphetamine result, whereas sertraline can lead to a false-positive benzodiazepine result.⁶ Retrospective reviews of clinician behavior surrounding UDS interpretation have demonstrated knowledge gaps and inconsistent communication practices with patients.^7,8

Given the complexity of UDS interpretation and its close relationship with medications, pharmacists are positioned to play an important role in the process. Pharmacists are embedded in pain-management teams and involved in prescription drug monitoring programs (PDMPs) for many health systems. The Veterans Health Administration (VHA) has supported the hiring of pain management, opioid safety, and PDMP coordinators (PMOP) at its facilities to provide clinical pain-management guidance, support national initiatives, and uphold legislative requirements.⁹ In many facilities, a pharmacist is hired specifically for these positions.

Clinical dashboards have been used by pharmacists in a variety of settings.^10-13 They allow clinicians at a broad level to target interventions needed across a patient population, then produce a list of actionable patients to facilitate delivery of that intervention on an individual level.¹³ Between 2021 and 2022, a clinical dashboard to review potentially discrepant UDS results was made available for use at US Department of Veterans Affairs (VA) medical centers. Evidence exists in primary and specialty care settings that implementation of an opioid-prescribing clinical dashboard improves completion rates of risk-mitigation strategies such as UDS and opioid treatment agreements.^14,15 To our knowledge there is no published research on the use and outcomes of a clinical dashboard that allows users to efficiently review discrepant UDS results when compared to a list of currently prescribed medications.

Given the availability of the UDS dashboard at the VA Black Hills Health Care System (VABHHCS) in South Dakota and the hiring of a PMOP coordinator pharmacist, the aim of this quality improvement project was 2-fold: to implement a pharmacist-led process to monitor the UDS dashboard for potentially discrepant results and to describe the quantity and types of interventions made by the clinical pharmacist leading this process.

Quality Improvement Project

A clinical UDS dashboard was created by the VA Northwest Health Network and made available for use at VHA sites between 2021 and 2022. The UDS dashboard is housed on a secure, Power BI Report Server (Microsoft), with access restricted to only those with patient health data privileges. The dashboard identifies all local patients with a UDS that returned with a potential discrepancy, defined as an unexpected positive result (eg, a detected substance not recently prescribed or documented on the patient’s medication list) and/or an unexpected negative result (eg, a prescribed substance not detected). The UDS dashboard identifies these discrepancies by comparing the patient’s current medication list (both VHA and non-VHA) to their UDS results.

The UDS dashboard displays a summary of UDSs performed, unexpected negative results, unexpected positive results, and potential discrepancies. The user may also specify the laboratory type and time frame of interest to limit displayed results. The user can then view patient-specific data for any category. Among the data are the patient’s UDS results and the completion date, detected (or nondetected) substance(s), ordering clinician, associated medication(s) with last fill date and days’ supply, and whether a confirmatory test has been performed in the past year.

VABHHCS uses an extended UDS immunoassay (PROFILE-V, MEDTOX Diagnostics) that reports on 11 substances: opiates, oxycodone, buprenorphine, methadone, amphetamines, methamphetamine, barbiturates, benzodiazepines, cocaine metabolites, cannabinoids (tetrahydrocannabinol [THC]), and phencyclidine. These substances appear on the UDS dashboard. The project protocol initially included monitoring for tramadol but that was later removed because it was not available with this UDS immunoassay.

Pharmacist Process

Either the PMOP coordinator or pharmacy resident monitored the UDS dashboard weekly. Any patients identified as having a potential discrepancy were reviewed. If the discrepancy was determined to be significant, the PMOP coordinator or pharmacy resident would review the patient electronic health record. If warranted, the patient was contacted and asked about newly prescribed medications, missed and recent medication doses, and illicit substance use. Potential interventions during in-depth review included: (1) discussing future actions with the primary care clinician and/or prescriber of the controlled substance; (2) ordering a confirmatory test on the original urine sample; (3) evaluating for sources of potential false-positive results; (4) completing an updated PDMP if not performed within the past year; (5) referring patients for substance use disorder treatment or counseling; or (6) consulting the local narcotics review committee. A progress note was entered into the electronic health record with the findings and any actions taken, and an alert for the primary care clinician and/or prescriber of the controlled substance.

Implementation and Analysis

This quality improvement project spanned 16 weeks from June 2022 through September 2022. Any patient with a UDS that returned with a significant discrepancy was reviewed. The primary outcome was interventions made by the PMOP coordinator or pharmacy resident, as well as time taken to perform the in-depth review of each patient. Patient demographics were also collected. The protocol for this project was approved by the VABHHCS pharmacy and therapeutics committee and was determined to meet guidelines for a nonresearch quality improvement project.

Results

From June 2022 through September 2022, 700 UDSs were performed at VABHHCS with 278 (39.7%) patients identified as having a potential discrepancy based on UDS results. Sixty patients (8.6%) had significant discrepancies that warranted in-depth review. The most common reasons for determining whether a potential discrepancy was not significant included unexpected negatives due to documented non-VA medications no longer being prescribed, unexpected positives due to recent expiration of a controlled substance prescription the patient was still taking, or unexpected positives due to the detection of a substance for which the clinician was already aware. During the 16-week study period, the mean number of patients warranting in-depth review was 4 per week.

The patients were predominantly male with a mean age of 61 years, and most (87%) were prescribed at least 1 controlled substance (mean, 1.1), primarily opioids for pain management (Table 1). Most patients had recent substance risk mitigation with UDS (56%) and PDMP (65%) checks within the past year. Of the 60 patients reviewed with significant UDS discrepancies, 50% had a history of discrepant UDS results. Of the 60 UDS discrepancies, there were 37 unexpected positive results (62%), 17 unexpected negative results (28%), and 10 patients with both positive and negative results (17%). THC was the most frequently detected substance, followed by opiates, benzodiazepines, and amphetamines (Table 2).

Each in-depth review with interventions by the PMOP coordinator or pharmacy resident lasted a mean of 14 minutes (Table 3). Five patients were successfully contacted for an interview and 7 patients could not be contacted. The ordering clinician of the UDS sometimes had contacted these patients prior to the PMOP coordinator or pharmacy resident reviewing the UDS dashboard, eliminating the need for additional follow-up.

The most common pharmacist intervention was discussing future actions with the primary care clinician and/or prescriber of the controlled substance (n = 39; 65%). These conversations resulted in actions such as ordering a repeat UDS with confirmatory testing at a future date or agreeing that the clinician would discuss the results and subsequent actions with the patient at an upcoming visit. Pharmacist interventions also included 25 PDMP queries (42%) and 9 orders of confirmatory UDS on the original urine sample (15%). Only 1 patient was evaluated by the narcotics review committee, which resulted in a controlled substance flag being placed on their profile. No patients were referred to substance use disorder treatment or counseling. It was offered to and declined by 1 patient, and 3 patients were already engaged in these services.

Medication therapies that could contribute to false-positive results were also evaluated. Fourteen patients who tested positive for THC had a prescription for a nonsteroidal anti-inflammatory drug or proton-pump inhibitor, which could have created a false-positive result.⁶ One patient who tested positive for amphetamines had a prescription for phentermine.¹⁶ No other potential false-positive results were identified.

Findings of this project illustrate that the use of a clinical pharmacist to monitor a dashboard of discrepant UDS results created opportunities for collaboration with clinicians and impacted confirmatory testing and PDMP monitoring practices.

At the local level, the process had numerous benefits. First, it was a reasonable amount of workload to generate pharmacist interventions: the PMOP coordinator conducted an average of 4 in-depth reviews weekly, each lasting about 14 minutes. Thus, the UDS dashboard allowed the PMOP coordinator to actively surveil all incoming UDS results for potential discrepancies in about 1 hour each week. Pairing the automation of the UDS dashboard with the clinical judgment of the PMOP coordinator seemed to maximize efficiency. VABHHCS provides primary and secondary medical and surgical care to a rural population of approximately 20,000 patients across 5 states; the time required at facilities that serve a higher volume of patients may be greater.

Second, the project served as an opportunity for the PMOP coordinator to provide case-specific clinician education on UDS monitoring. As medication experts, pharmacists can apply their medication-related knowledge to UDS interpretation. This includes understanding drug metabolism and classification and how they apply to UDS results, as well as recognizing medication therapies that could contribute to false-positive UDS results. Research suggests that clinicians may have gaps in their knowledge and may welcome pharmacist assistance in interpreting UDS results.^7,8

Third, the project helped improve rates of confirmatory testing for those with unexpected positive UDS results. Confirmatory testing should be strongly considered if positive results would have significant implications on the future course of treatment.⁴ The PMOP coordinator ordered a confirmatory test on 9 patients using the same urine sample used to conduct the initial UDS, minimizing the burden on the patient and laboratory staff. Confirmatory testing was limited by the laboratory’s sample retention period; if the need for confirmatory testing was not recognized soon enough, the sample would no longer be available for retesting. Health systems may consider the use of reflexive confirmatory testing with UDS as an alternative approach, although this may come at an additional cost and may not be warranted in many cases (eg, only 39.7% of all potential discrepancies were deemed as significant within our project).

There were notable incidental findings in our quality improvement project. Among patients with a significant discrepancy on UDS, 50% had a history of ≥ 1 discrepant UDS result. This further emphasizes the importance of appropriate use and interpretation of UDS monitoring for all clinicians, as this may prevent prolonged and potentially inappropriate treatment regimens. Secondly, rates of mental health diagnoses among those with a significant UDS discrepancy seemed relatively high compared to population-level data. For example, among veterans, the overall lifetime prevalence of posttraumatic stress disorder is estimated to be 8.0%; in our project, 35% of patients with a significant UDS discrepancy had a posttraumatic stress disorder diagnosis.¹⁷ This relationship may be an area of further study.

Lastly, it was surprising that the overall rates of UDS and PDMP checks within the past year were 56% and 65%, respectively. VABHHCS requires veterans on controlled substances to have these risk-mitigation strategies performed annually, so our suspicion is that many were falling out due to having been most recently evaluated 12 to 16 months prior. This may represent a limitation of our data-collection method, which reviewed only the previous 12 months.

Limitations

This project was carried out over a period of only 4 months. As a result, only 60 patients received an in-depth review from the PMOP coordinator. Second, the timeliness of the intervention seemed crucial, as delayed in-depth reviews resulted in fewer opportunities to order confirmatory tests or collaborate with clinicians prior to devising an updated plan. Additionally, our process called for UDS dashboard monitoring once a week. Given that the laboratory held samples for only 48 hours, twice- or thrice-weekly review of the UDS dashboard would have allowed for more confirmatory testing, along with more immediate clinician collaboration. Most importantly, the outcomes of this project are only presented via descriptive statistics and without the results of any comparison group, making it impossible to draw firm conclusions about this approach compared to standard-care processes.

Conclusions

This quality improvement project has proven to be valuable at VABHHCS and we intend to continue this pharmacist-led process to monitor the UDS dashboard. VABHHCS leadership are also discussing UDS practices more broadly to further enhance patient management. Within the VA, the PMOP coordinator—charged with being the local coordinator of appropriate pain management and opioid safety practices—is well positioned to assume these responsibilities. Outside of the VA, a pain-management clinical pharmacist or any pharmacist embedded within primary care could similarly perform these duties. Previous literature regarding the implementation of clinical dashboards suggests that with the appropriate software engineering teams and infrastructure, this tool could also be feasibly developed and implemented at other health systems relatively quickly.¹⁴

Overall, a pharmacist-led process to efficiently monitor a dashboard of discrepant UDS results led to opportunities for collaboration with clinicians and positively impacted confirmatory testing and PDMP monitoring at a rural VA health system.

Acknowledgments

The authors express their gratitude to Patrick Spoutz, PharmD, BCPS, VISN 20 Pharmacist Executive, for introducing and sharing the UDS dashboard with our team.

Urine drug screen (UDS) monitoring is a common risk-mitigation strategy tool for prescribing controlled substances.^1-3 Not only is UDS monitoring highlighted by clinical practice guidelines for opioid prescribing for chronic pain,^1,2 it has also been suggested as best practice for benzodiazepines³ and a consideration for other controlled substances. Monitoring UDSs helps confirm adherence to the prescribed treatment regimen while also screening for substance use that may increase patient risk.

UDS results can be complex and have profound implications for the patient’s treatment plan. Drug metabolites for opioids are particularly complicated; for example, synthetic and semisynthetic opioids are not detected on routine opiate immunoassays.⁴ This may lead a clinician to falsely assume the patient is not taking their fentanyl or tramadol medication as directed—or potentially even diverting—in the face of a negative opiate result.⁵ Routine UDSs are also subject to the pitfall of false-positive results due to coprescribed medications; for example, bupropion can lead to a false-positive amphetamine result, whereas sertraline can lead to a false-positive benzodiazepine result.⁶ Retrospective reviews of clinician behavior surrounding UDS interpretation have demonstrated knowledge gaps and inconsistent communication practices with patients.^7,8

Given the complexity of UDS interpretation and its close relationship with medications, pharmacists are positioned to play an important role in the process. Pharmacists are embedded in pain-management teams and involved in prescription drug monitoring programs (PDMPs) for many health systems. The Veterans Health Administration (VHA) has supported the hiring of pain management, opioid safety, and PDMP coordinators (PMOP) at its facilities to provide clinical pain-management guidance, support national initiatives, and uphold legislative requirements.⁹ In many facilities, a pharmacist is hired specifically for these positions.

Clinical dashboards have been used by pharmacists in a variety of settings.^10-13 They allow clinicians at a broad level to target interventions needed across a patient population, then produce a list of actionable patients to facilitate delivery of that intervention on an individual level.¹³ Between 2021 and 2022, a clinical dashboard to review potentially discrepant UDS results was made available for use at US Department of Veterans Affairs (VA) medical centers. Evidence exists in primary and specialty care settings that implementation of an opioid-prescribing clinical dashboard improves completion rates of risk-mitigation strategies such as UDS and opioid treatment agreements.^14,15 To our knowledge there is no published research on the use and outcomes of a clinical dashboard that allows users to efficiently review discrepant UDS results when compared to a list of currently prescribed medications.

Given the availability of the UDS dashboard at the VA Black Hills Health Care System (VABHHCS) in South Dakota and the hiring of a PMOP coordinator pharmacist, the aim of this quality improvement project was 2-fold: to implement a pharmacist-led process to monitor the UDS dashboard for potentially discrepant results and to describe the quantity and types of interventions made by the clinical pharmacist leading this process.

Quality Improvement Project

A clinical UDS dashboard was created by the VA Northwest Health Network and made available for use at VHA sites between 2021 and 2022. The UDS dashboard is housed on a secure, Power BI Report Server (Microsoft), with access restricted to only those with patient health data privileges. The dashboard identifies all local patients with a UDS that returned with a potential discrepancy, defined as an unexpected positive result (eg, a detected substance not recently prescribed or documented on the patient’s medication list) and/or an unexpected negative result (eg, a prescribed substance not detected). The UDS dashboard identifies these discrepancies by comparing the patient’s current medication list (both VHA and non-VHA) to their UDS results.

The UDS dashboard displays a summary of UDSs performed, unexpected negative results, unexpected positive results, and potential discrepancies. The user may also specify the laboratory type and time frame of interest to limit displayed results. The user can then view patient-specific data for any category. Among the data are the patient’s UDS results and the completion date, detected (or nondetected) substance(s), ordering clinician, associated medication(s) with last fill date and days’ supply, and whether a confirmatory test has been performed in the past year.

VABHHCS uses an extended UDS immunoassay (PROFILE-V, MEDTOX Diagnostics) that reports on 11 substances: opiates, oxycodone, buprenorphine, methadone, amphetamines, methamphetamine, barbiturates, benzodiazepines, cocaine metabolites, cannabinoids (tetrahydrocannabinol [THC]), and phencyclidine. These substances appear on the UDS dashboard. The project protocol initially included monitoring for tramadol but that was later removed because it was not available with this UDS immunoassay.

Pharmacist Process

Either the PMOP coordinator or pharmacy resident monitored the UDS dashboard weekly. Any patients identified as having a potential discrepancy were reviewed. If the discrepancy was determined to be significant, the PMOP coordinator or pharmacy resident would review the patient electronic health record. If warranted, the patient was contacted and asked about newly prescribed medications, missed and recent medication doses, and illicit substance use. Potential interventions during in-depth review included: (1) discussing future actions with the primary care clinician and/or prescriber of the controlled substance; (2) ordering a confirmatory test on the original urine sample; (3) evaluating for sources of potential false-positive results; (4) completing an updated PDMP if not performed within the past year; (5) referring patients for substance use disorder treatment or counseling; or (6) consulting the local narcotics review committee. A progress note was entered into the electronic health record with the findings and any actions taken, and an alert for the primary care clinician and/or prescriber of the controlled substance.

Implementation and Analysis

This quality improvement project spanned 16 weeks from June 2022 through September 2022. Any patient with a UDS that returned with a significant discrepancy was reviewed. The primary outcome was interventions made by the PMOP coordinator or pharmacy resident, as well as time taken to perform the in-depth review of each patient. Patient demographics were also collected. The protocol for this project was approved by the VABHHCS pharmacy and therapeutics committee and was determined to meet guidelines for a nonresearch quality improvement project.

Results

From June 2022 through September 2022, 700 UDSs were performed at VABHHCS with 278 (39.7%) patients identified as having a potential discrepancy based on UDS results. Sixty patients (8.6%) had significant discrepancies that warranted in-depth review. The most common reasons for determining whether a potential discrepancy was not significant included unexpected negatives due to documented non-VA medications no longer being prescribed, unexpected positives due to recent expiration of a controlled substance prescription the patient was still taking, or unexpected positives due to the detection of a substance for which the clinician was already aware. During the 16-week study period, the mean number of patients warranting in-depth review was 4 per week.

The patients were predominantly male with a mean age of 61 years, and most (87%) were prescribed at least 1 controlled substance (mean, 1.1), primarily opioids for pain management (Table 1). Most patients had recent substance risk mitigation with UDS (56%) and PDMP (65%) checks within the past year. Of the 60 patients reviewed with significant UDS discrepancies, 50% had a history of discrepant UDS results. Of the 60 UDS discrepancies, there were 37 unexpected positive results (62%), 17 unexpected negative results (28%), and 10 patients with both positive and negative results (17%). THC was the most frequently detected substance, followed by opiates, benzodiazepines, and amphetamines (Table 2).

Each in-depth review with interventions by the PMOP coordinator or pharmacy resident lasted a mean of 14 minutes (Table 3). Five patients were successfully contacted for an interview and 7 patients could not be contacted. The ordering clinician of the UDS sometimes had contacted these patients prior to the PMOP coordinator or pharmacy resident reviewing the UDS dashboard, eliminating the need for additional follow-up.

The most common pharmacist intervention was discussing future actions with the primary care clinician and/or prescriber of the controlled substance (n = 39; 65%). These conversations resulted in actions such as ordering a repeat UDS with confirmatory testing at a future date or agreeing that the clinician would discuss the results and subsequent actions with the patient at an upcoming visit. Pharmacist interventions also included 25 PDMP queries (42%) and 9 orders of confirmatory UDS on the original urine sample (15%). Only 1 patient was evaluated by the narcotics review committee, which resulted in a controlled substance flag being placed on their profile. No patients were referred to substance use disorder treatment or counseling. It was offered to and declined by 1 patient, and 3 patients were already engaged in these services.

Medication therapies that could contribute to false-positive results were also evaluated. Fourteen patients who tested positive for THC had a prescription for a nonsteroidal anti-inflammatory drug or proton-pump inhibitor, which could have created a false-positive result.⁶ One patient who tested positive for amphetamines had a prescription for phentermine.¹⁶ No other potential false-positive results were identified.

Findings of this project illustrate that the use of a clinical pharmacist to monitor a dashboard of discrepant UDS results created opportunities for collaboration with clinicians and impacted confirmatory testing and PDMP monitoring practices.

At the local level, the process had numerous benefits. First, it was a reasonable amount of workload to generate pharmacist interventions: the PMOP coordinator conducted an average of 4 in-depth reviews weekly, each lasting about 14 minutes. Thus, the UDS dashboard allowed the PMOP coordinator to actively surveil all incoming UDS results for potential discrepancies in about 1 hour each week. Pairing the automation of the UDS dashboard with the clinical judgment of the PMOP coordinator seemed to maximize efficiency. VABHHCS provides primary and secondary medical and surgical care to a rural population of approximately 20,000 patients across 5 states; the time required at facilities that serve a higher volume of patients may be greater.

Second, the project served as an opportunity for the PMOP coordinator to provide case-specific clinician education on UDS monitoring. As medication experts, pharmacists can apply their medication-related knowledge to UDS interpretation. This includes understanding drug metabolism and classification and how they apply to UDS results, as well as recognizing medication therapies that could contribute to false-positive UDS results. Research suggests that clinicians may have gaps in their knowledge and may welcome pharmacist assistance in interpreting UDS results.^7,8

Third, the project helped improve rates of confirmatory testing for those with unexpected positive UDS results. Confirmatory testing should be strongly considered if positive results would have significant implications on the future course of treatment.⁴ The PMOP coordinator ordered a confirmatory test on 9 patients using the same urine sample used to conduct the initial UDS, minimizing the burden on the patient and laboratory staff. Confirmatory testing was limited by the laboratory’s sample retention period; if the need for confirmatory testing was not recognized soon enough, the sample would no longer be available for retesting. Health systems may consider the use of reflexive confirmatory testing with UDS as an alternative approach, although this may come at an additional cost and may not be warranted in many cases (eg, only 39.7% of all potential discrepancies were deemed as significant within our project).

There were notable incidental findings in our quality improvement project. Among patients with a significant discrepancy on UDS, 50% had a history of ≥ 1 discrepant UDS result. This further emphasizes the importance of appropriate use and interpretation of UDS monitoring for all clinicians, as this may prevent prolonged and potentially inappropriate treatment regimens. Secondly, rates of mental health diagnoses among those with a significant UDS discrepancy seemed relatively high compared to population-level data. For example, among veterans, the overall lifetime prevalence of posttraumatic stress disorder is estimated to be 8.0%; in our project, 35% of patients with a significant UDS discrepancy had a posttraumatic stress disorder diagnosis.¹⁷ This relationship may be an area of further study.

Lastly, it was surprising that the overall rates of UDS and PDMP checks within the past year were 56% and 65%, respectively. VABHHCS requires veterans on controlled substances to have these risk-mitigation strategies performed annually, so our suspicion is that many were falling out due to having been most recently evaluated 12 to 16 months prior. This may represent a limitation of our data-collection method, which reviewed only the previous 12 months.

Limitations

This project was carried out over a period of only 4 months. As a result, only 60 patients received an in-depth review from the PMOP coordinator. Second, the timeliness of the intervention seemed crucial, as delayed in-depth reviews resulted in fewer opportunities to order confirmatory tests or collaborate with clinicians prior to devising an updated plan. Additionally, our process called for UDS dashboard monitoring once a week. Given that the laboratory held samples for only 48 hours, twice- or thrice-weekly review of the UDS dashboard would have allowed for more confirmatory testing, along with more immediate clinician collaboration. Most importantly, the outcomes of this project are only presented via descriptive statistics and without the results of any comparison group, making it impossible to draw firm conclusions about this approach compared to standard-care processes.

Conclusions

This quality improvement project has proven to be valuable at VABHHCS and we intend to continue this pharmacist-led process to monitor the UDS dashboard. VABHHCS leadership are also discussing UDS practices more broadly to further enhance patient management. Within the VA, the PMOP coordinator—charged with being the local coordinator of appropriate pain management and opioid safety practices—is well positioned to assume these responsibilities. Outside of the VA, a pain-management clinical pharmacist or any pharmacist embedded within primary care could similarly perform these duties. Previous literature regarding the implementation of clinical dashboards suggests that with the appropriate software engineering teams and infrastructure, this tool could also be feasibly developed and implemented at other health systems relatively quickly.¹⁴

Overall, a pharmacist-led process to efficiently monitor a dashboard of discrepant UDS results led to opportunities for collaboration with clinicians and positively impacted confirmatory testing and PDMP monitoring at a rural VA health system.

Acknowledgments

The authors express their gratitude to Patrick Spoutz, PharmD, BCPS, VISN 20 Pharmacist Executive, for introducing and sharing the UDS dashboard with our team.

More than 37 million Americans (11.3%) have diabetes mellitus (DM), and 90% to 95% are diagnosed with type 2 DM, including nearly 1 in 4 veterans receiving Veterans Health Administration (VHA) care.^1,2 DM is associated with serious negative health outcomes, including cardiovascular disease and subsequent complications as well as significant health care system utilization and cost.^1,3

Group interventions have been identified as a possible method of improving DM outcomes. For example, shared medical appointments (SMAs) have been identified by the VHA as holding promise for improving care and efficiency for DM and other common health conditions.⁴ Although the precise structure and SMA process for managing DM has been noted to be heterogeneous, the appointment is typically led by an interdisciplinary health care team and includes individualized assessment including medication review and adjustment, group education, and troubleshooting challenges with management in a group format.⁵ Research suggests that DM SMAs are a worthwhile treatment approach.⁵ Several studies have found that SMAs were associated with decreased hemoglobin A_1c (Hb A_1c) levels and improvement in overall disease complications and severity.⁶

The high degree of SMA heterogeneity and lack of detailed description of structure and process of SMAs studied has made meta-analysis and other synthesis of the literature difficult.⁵ Consequently, there is inadequate empirically supported guidance for clinicians and health care organizations on how to best implement SMAs and similar group-based treatments. Edelman and colleagues recommended that future research should focus on more consistent and standardized intervention structures and real-world patient- and staff-centered outcomes to address gaps in the literature.⁵ They noted that a mental health professional was utilized in only a minority of SMAs studied.⁵ Additionally, we noted a paucity of studies examining patient satisfaction with SMAs.

Another group-based intervention found to be effective in improving DM outcomes is the 6-session Stanford Diabetes Self-Management Program (DSMP), a workshop led in part by trained peers with DM. The sessions focus on educating patients on DM care and self-management tools. The workshop encourages active practice in building DM self-management skills and confidence. DSMP participation has been associated with improvement in DM-related outcomes, including Hb A_1c levels, amount of exercise, and medication adherence.⁷

While SMAs and DSMP have been shown to enhance clinical outcomes, they provide differing types of patient support. SMAs allow for frequent interaction with a health care professional (HCP) and less emphasis on behavioral health interventions. DSMPs include behavioral health professionals and peer leaders and emphasize higher levels of psychosocial support, but do not offer access to clinicians. It is possible that combining these interventions could result in better outcomes than what either could provide on their own.

In 2018, the Cincinnati Veterans Affairs Medical Center (VAMC) in Ohio offered Diabetes Basic Training, a structured DM intervention. Patients enrolled in the program participated in a 9-week intervention that included 3 SMAs and 6 DSMP sessions. During the SMAs, a clinical psychologist or psychology postdoctoral fellow skilled in motivational interviewing facilitated the group to enhance patient engagement and empowerment for improved self-management. In addition, patients participated in structured DSMP groups with an emphasis on action-planning, often surrounding nutrition, physical activity, and other health behavior change information reviewed during the SMAs.

Design and Referral

Self-management programs for chronic health conditions are often underutilized. Although HCPs may wish to connect veterans with available programs, time constraints may limit opportunities for detailed discussions with patients about specific aspects of each program. To simplify this process, a 2-hour orientation program was offered that explained individual and group DM self-management options (Figure). During this initial visit, patients met with an interdisciplinary care team (registered dietician, diabetes nurse practitioner, and behavioral health specialist) and were informed about Diabetes Basic Training, DM clinical care practices, and other related resources available at the Cincinnati VAMC (eg, cooking classes, food pantry). Patients received individualized referral recommendations and were urged to consult with their primary care practitioner to finalize their treatment plan.

Shared Medical Appointments

Diabetes Basic Training interventions had an average of 6 to 8 veterans participating in the weekly groups. The first, fifth, and final weeks were SMAs in which an interdisciplinary team collaboratively provided group-based health care for DM. The team consisted of a registered nurse, a prescriber (eg, nurse practitioner), a moderator (eg, psychologist), and a content expert (eg, nutritionist). Before each SMA began, the nurse checked-in patients in the SMA room and collected heart rate and blood pressure, and performed a diabetic foot check. Each SMA consisted of introductions, group-driven discussions (facilitated by an HCP) and troubleshooting DM self-management challenges. During group discussions, the prescriber initiated a 1-on-1 discussion with each patient in a private office regarding their recent laboratory results, medication regimen, and other aspects of DM care. The patient’s medications were refilled and/or adjusted as needed and other orders and referrals were submitted. If a patient had a medical question, the prescriber and moderator engaged the entire group so all individuals could benefit from generating and hearing answers. When discussion slowed, education was provided on topics generated by the group. Frequent topics included challenges managing DM, concerns, how DM impacted daily life and relationships, and sharing successes. As needed, HCPs spoke individually with patients following the SMA. Patients were sometimes asked, but never required, to do homework consistent with standard DM care (eg, recording what they eat or blood sugar levels). Each SMA session lasted about 2 hours.

Diabetes Self-Management Program

The second, third, fourth, sixth, seventh, and eighth weeks of the program were devoted to the DSMP. These sessions were delivered primarily by veteran peers who received appropriate training, observation, and certification. Each 2-hour educational program provided ample practice in many fundamental self-management skills, such as decision making, problem solving, and action planning. Patients were asked, but never required, to practice related skills during the sessions and to create weekly action plans to be completed between sessions that typically involved increasing exercise or improving diet. Patients were encouraged to follow up with HCPs at SMAs when they had questions requiring HCP expertise. If participants had more immediate concerns regarding their treatment plan and/or medications, they contacted their primary care practitioner prior to the next SMA.

As a part of participation in the program, psychosocial and health data and Hb A_1c levels at baseline (the closest level to 90 days prior to start) and follow-up (the closest level to 90 days after the final session) were collected.⁸ In addition, Problem Areas in Diabetes (PAID), Patient Activation Measure (PAM)-13, and Diabetes Self-Management Questionnaire (DSMQ) were administered at 3 points: during the orientation, in the first week, and in the ninth week of the program.

PAID, a 20-item self-report questionnaire designed to capture emotional distress related to having DM, is a valid and reliable scale able to detect changes over time when used in intervention studies.^9,10 PAM-13 is a 13-item measure designed to assess patient knowledge, skill, and confidence in the self-management of health or chronic conditions based on the original PAM.^11,12 Scores fall into 1 of 4 activation levels, ranging from low levels of confidence and knowledge of health management to high levels of being proactive with one’s care. The PAM-13 has been widely used within health psychology, including research among adults with multiple chronic conditions, individuals with DM or osteoarthritis, and within primary care.^13-15 The DSMQ is a statistically reliable and valid instrument that allows for user-friendly assessment of self-care behaviors associated with glycemic control.^16-18

All measures were collected as part of traditional clinical care, and we present initial program evaluation data to demonstrate potential effectiveness of the clinic model. Paired samples t tests were used to examine differences between baseline and follow-up measures for the 24 veteran participants. The age of participants who completed the program ranged from 42 to 74 years (mean, 68 years); 29% of participants were Black veterans and 12% were female. Examination of clinical outcomes indicated that veterans reported significant increases in activation levels for managing their health increasing from a baseline mean (SD) 62.1 (12.3) to 68.4 (14.5) at follow up (t[23] = 2.15, P = .04). Hb A_1c levels trended downward from a mean (SD) 8.6% (1.3) at baseline to 8.2% (1.2) at 90-day follow up (t[21] 1.05, P = .30). Similar nonsignificant trends in PAID scores were seen for pre- and postprogram reductions in emotional distress related to having DM from a mean (SD) 7.9 (5.0) at baseline to 6.3 (5.1) (t[18] = 11.51, P = .15), and enhanced self-management of glucose with a mean (SD) 6.5 (1.5) at baseline to 6.8 (1.3) at follow up (t[19] = 0.52, P = .61). The trends found in this study show promising outcomes for this pilot group-based DM treatment, though the small sample size (N = 24) limits statistical power. These findings support further exploration and expansion of interdisciplinary health programs supporting veteran self-management.

Discussion

DM is a condition of epidemic proportions that causes substantial negative health outcomes and costs at a national level. Current standards of DM care do not appear to be reversing these trends. Wider implementation of group-based treatment for DM could improve efficiency of care, increase access to quality care, and reduce burden on individual HCPs.

The VHA continues the transformation of its care system, which shifts toward a patient-centered, proactive focus on veteran well-being. This new whole health approach integrates conventional medical treatment with veteran self-empowerment in the pursuit of health goals based on individual veteran’s identified values.¹⁹ This approach emphasizes peer-led explorations of veterans’ aspirations, purpose, and individual mission, personalized health planning, and use of whole health coaches and well-being programs, with both allopathic and complementary and integrative clinical care centered around veterans’ identified goals and priorities.²⁰

Including a program like Diabetes Basic Training as a part of whole health programming could offer several benefits. Diabetes Basic Training is unique in its integration of more traditional SMA structure with psychosocial interventions including values identification and motivational interviewing strategies to enhance patient engagement. Veterans can learn from each other’s experiences and concerns, leading to better DM management knowledge and skills. The group nature of the sessions enhances opportunities for emotional support and reduced isolation, as well as peer accountability for maintaining medication adherence.

By meeting with HCPs from multiple disciplines, veterans are exposed to different perspectives on self-management techniques, including behavioral approaches for overcoming barriers to behavior change. Clinicians have more time to engage with patients, building stronger relationships and trust. SMAs are cost-efficient and time efficient, allowing HCPs to see multiple patients at once, reducing wait times and increasing the number of patients treated in a given time frame.

The COVID-19 pandemic temporarily impacted the ongoing expansion of the program, when so many services were shifted from in-person to virtual classes. Due to staffing and other logistic issues, our pilot program was suspended during that time, but plans to resume the program by early 2024 are moving forward.

CONCLUSIONS

The Diabetes Basic Training program serves as a successful model for implementation within a VAMC. Although the number of veterans with complete data available for analysis was small, the trends exhibited in the preliminary outcome data are promising. We encourage other VAMCs to replicate this program with a larger participant base and evaluate its impact on veteran health outcomes. Next steps include comparing the clinical data from treatment as usual with outcomes from DM group participants. As the program resumes, we will reinitiate recruitment efforts to increase HCP referrals to this program.

More than 37 million Americans (11.3%) have diabetes mellitus (DM), and 90% to 95% are diagnosed with type 2 DM, including nearly 1 in 4 veterans receiving Veterans Health Administration (VHA) care.^1,2 DM is associated with serious negative health outcomes, including cardiovascular disease and subsequent complications as well as significant health care system utilization and cost.^1,3

Group interventions have been identified as a possible method of improving DM outcomes. For example, shared medical appointments (SMAs) have been identified by the VHA as holding promise for improving care and efficiency for DM and other common health conditions.⁴ Although the precise structure and SMA process for managing DM has been noted to be heterogeneous, the appointment is typically led by an interdisciplinary health care team and includes individualized assessment including medication review and adjustment, group education, and troubleshooting challenges with management in a group format.⁵ Research suggests that DM SMAs are a worthwhile treatment approach.⁵ Several studies have found that SMAs were associated with decreased hemoglobin A_1c (Hb A_1c) levels and improvement in overall disease complications and severity.⁶

The high degree of SMA heterogeneity and lack of detailed description of structure and process of SMAs studied has made meta-analysis and other synthesis of the literature difficult.⁵ Consequently, there is inadequate empirically supported guidance for clinicians and health care organizations on how to best implement SMAs and similar group-based treatments. Edelman and colleagues recommended that future research should focus on more consistent and standardized intervention structures and real-world patient- and staff-centered outcomes to address gaps in the literature.⁵ They noted that a mental health professional was utilized in only a minority of SMAs studied.⁵ Additionally, we noted a paucity of studies examining patient satisfaction with SMAs.

Another group-based intervention found to be effective in improving DM outcomes is the 6-session Stanford Diabetes Self-Management Program (DSMP), a workshop led in part by trained peers with DM. The sessions focus on educating patients on DM care and self-management tools. The workshop encourages active practice in building DM self-management skills and confidence. DSMP participation has been associated with improvement in DM-related outcomes, including Hb A_1c levels, amount of exercise, and medication adherence.⁷

While SMAs and DSMP have been shown to enhance clinical outcomes, they provide differing types of patient support. SMAs allow for frequent interaction with a health care professional (HCP) and less emphasis on behavioral health interventions. DSMPs include behavioral health professionals and peer leaders and emphasize higher levels of psychosocial support, but do not offer access to clinicians. It is possible that combining these interventions could result in better outcomes than what either could provide on their own.

In 2018, the Cincinnati Veterans Affairs Medical Center (VAMC) in Ohio offered Diabetes Basic Training, a structured DM intervention. Patients enrolled in the program participated in a 9-week intervention that included 3 SMAs and 6 DSMP sessions. During the SMAs, a clinical psychologist or psychology postdoctoral fellow skilled in motivational interviewing facilitated the group to enhance patient engagement and empowerment for improved self-management. In addition, patients participated in structured DSMP groups with an emphasis on action-planning, often surrounding nutrition, physical activity, and other health behavior change information reviewed during the SMAs.

Design and Referral

Self-management programs for chronic health conditions are often underutilized. Although HCPs may wish to connect veterans with available programs, time constraints may limit opportunities for detailed discussions with patients about specific aspects of each program. To simplify this process, a 2-hour orientation program was offered that explained individual and group DM self-management options (Figure). During this initial visit, patients met with an interdisciplinary care team (registered dietician, diabetes nurse practitioner, and behavioral health specialist) and were informed about Diabetes Basic Training, DM clinical care practices, and other related resources available at the Cincinnati VAMC (eg, cooking classes, food pantry). Patients received individualized referral recommendations and were urged to consult with their primary care practitioner to finalize their treatment plan.

Shared Medical Appointments

Diabetes Basic Training interventions had an average of 6 to 8 veterans participating in the weekly groups. The first, fifth, and final weeks were SMAs in which an interdisciplinary team collaboratively provided group-based health care for DM. The team consisted of a registered nurse, a prescriber (eg, nurse practitioner), a moderator (eg, psychologist), and a content expert (eg, nutritionist). Before each SMA began, the nurse checked-in patients in the SMA room and collected heart rate and blood pressure, and performed a diabetic foot check. Each SMA consisted of introductions, group-driven discussions (facilitated by an HCP) and troubleshooting DM self-management challenges. During group discussions, the prescriber initiated a 1-on-1 discussion with each patient in a private office regarding their recent laboratory results, medication regimen, and other aspects of DM care. The patient’s medications were refilled and/or adjusted as needed and other orders and referrals were submitted. If a patient had a medical question, the prescriber and moderator engaged the entire group so all individuals could benefit from generating and hearing answers. When discussion slowed, education was provided on topics generated by the group. Frequent topics included challenges managing DM, concerns, how DM impacted daily life and relationships, and sharing successes. As needed, HCPs spoke individually with patients following the SMA. Patients were sometimes asked, but never required, to do homework consistent with standard DM care (eg, recording what they eat or blood sugar levels). Each SMA session lasted about 2 hours.

Diabetes Self-Management Program

The second, third, fourth, sixth, seventh, and eighth weeks of the program were devoted to the DSMP. These sessions were delivered primarily by veteran peers who received appropriate training, observation, and certification. Each 2-hour educational program provided ample practice in many fundamental self-management skills, such as decision making, problem solving, and action planning. Patients were asked, but never required, to practice related skills during the sessions and to create weekly action plans to be completed between sessions that typically involved increasing exercise or improving diet. Patients were encouraged to follow up with HCPs at SMAs when they had questions requiring HCP expertise. If participants had more immediate concerns regarding their treatment plan and/or medications, they contacted their primary care practitioner prior to the next SMA.

As a part of participation in the program, psychosocial and health data and Hb A_1c levels at baseline (the closest level to 90 days prior to start) and follow-up (the closest level to 90 days after the final session) were collected.⁸ In addition, Problem Areas in Diabetes (PAID), Patient Activation Measure (PAM)-13, and Diabetes Self-Management Questionnaire (DSMQ) were administered at 3 points: during the orientation, in the first week, and in the ninth week of the program.

PAID, a 20-item self-report questionnaire designed to capture emotional distress related to having DM, is a valid and reliable scale able to detect changes over time when used in intervention studies.^9,10 PAM-13 is a 13-item measure designed to assess patient knowledge, skill, and confidence in the self-management of health or chronic conditions based on the original PAM.^11,12 Scores fall into 1 of 4 activation levels, ranging from low levels of confidence and knowledge of health management to high levels of being proactive with one’s care. The PAM-13 has been widely used within health psychology, including research among adults with multiple chronic conditions, individuals with DM or osteoarthritis, and within primary care.^13-15 The DSMQ is a statistically reliable and valid instrument that allows for user-friendly assessment of self-care behaviors associated with glycemic control.^16-18

All measures were collected as part of traditional clinical care, and we present initial program evaluation data to demonstrate potential effectiveness of the clinic model. Paired samples t tests were used to examine differences between baseline and follow-up measures for the 24 veteran participants. The age of participants who completed the program ranged from 42 to 74 years (mean, 68 years); 29% of participants were Black veterans and 12% were female. Examination of clinical outcomes indicated that veterans reported significant increases in activation levels for managing their health increasing from a baseline mean (SD) 62.1 (12.3) to 68.4 (14.5) at follow up (t[23] = 2.15, P = .04). Hb A_1c levels trended downward from a mean (SD) 8.6% (1.3) at baseline to 8.2% (1.2) at 90-day follow up (t[21] 1.05, P = .30). Similar nonsignificant trends in PAID scores were seen for pre- and postprogram reductions in emotional distress related to having DM from a mean (SD) 7.9 (5.0) at baseline to 6.3 (5.1) (t[18] = 11.51, P = .15), and enhanced self-management of glucose with a mean (SD) 6.5 (1.5) at baseline to 6.8 (1.3) at follow up (t[19] = 0.52, P = .61). The trends found in this study show promising outcomes for this pilot group-based DM treatment, though the small sample size (N = 24) limits statistical power. These findings support further exploration and expansion of interdisciplinary health programs supporting veteran self-management.

Discussion

DM is a condition of epidemic proportions that causes substantial negative health outcomes and costs at a national level. Current standards of DM care do not appear to be reversing these trends. Wider implementation of group-based treatment for DM could improve efficiency of care, increase access to quality care, and reduce burden on individual HCPs.

The VHA continues the transformation of its care system, which shifts toward a patient-centered, proactive focus on veteran well-being. This new whole health approach integrates conventional medical treatment with veteran self-empowerment in the pursuit of health goals based on individual veteran’s identified values.¹⁹ This approach emphasizes peer-led explorations of veterans’ aspirations, purpose, and individual mission, personalized health planning, and use of whole health coaches and well-being programs, with both allopathic and complementary and integrative clinical care centered around veterans’ identified goals and priorities.²⁰

Including a program like Diabetes Basic Training as a part of whole health programming could offer several benefits. Diabetes Basic Training is unique in its integration of more traditional SMA structure with psychosocial interventions including values identification and motivational interviewing strategies to enhance patient engagement. Veterans can learn from each other’s experiences and concerns, leading to better DM management knowledge and skills. The group nature of the sessions enhances opportunities for emotional support and reduced isolation, as well as peer accountability for maintaining medication adherence.

By meeting with HCPs from multiple disciplines, veterans are exposed to different perspectives on self-management techniques, including behavioral approaches for overcoming barriers to behavior change. Clinicians have more time to engage with patients, building stronger relationships and trust. SMAs are cost-efficient and time efficient, allowing HCPs to see multiple patients at once, reducing wait times and increasing the number of patients treated in a given time frame.

The COVID-19 pandemic temporarily impacted the ongoing expansion of the program, when so many services were shifted from in-person to virtual classes. Due to staffing and other logistic issues, our pilot program was suspended during that time, but plans to resume the program by early 2024 are moving forward.

CONCLUSIONS

The Diabetes Basic Training program serves as a successful model for implementation within a VAMC. Although the number of veterans with complete data available for analysis was small, the trends exhibited in the preliminary outcome data are promising. We encourage other VAMCs to replicate this program with a larger participant base and evaluate its impact on veteran health outcomes. Next steps include comparing the clinical data from treatment as usual with outcomes from DM group participants. As the program resumes, we will reinitiate recruitment efforts to increase HCP referrals to this program.

More than 37 million Americans (11.3%) have diabetes mellitus (DM), and 90% to 95% are diagnosed with type 2 DM, including nearly 1 in 4 veterans receiving Veterans Health Administration (VHA) care.^1,2 DM is associated with serious negative health outcomes, including cardiovascular disease and subsequent complications as well as significant health care system utilization and cost.^1,3

Group interventions have been identified as a possible method of improving DM outcomes. For example, shared medical appointments (SMAs) have been identified by the VHA as holding promise for improving care and efficiency for DM and other common health conditions.⁴ Although the precise structure and SMA process for managing DM has been noted to be heterogeneous, the appointment is typically led by an interdisciplinary health care team and includes individualized assessment including medication review and adjustment, group education, and troubleshooting challenges with management in a group format.⁵ Research suggests that DM SMAs are a worthwhile treatment approach.⁵ Several studies have found that SMAs were associated with decreased hemoglobin A_1c (Hb A_1c) levels and improvement in overall disease complications and severity.⁶

The high degree of SMA heterogeneity and lack of detailed description of structure and process of SMAs studied has made meta-analysis and other synthesis of the literature difficult.⁵ Consequently, there is inadequate empirically supported guidance for clinicians and health care organizations on how to best implement SMAs and similar group-based treatments. Edelman and colleagues recommended that future research should focus on more consistent and standardized intervention structures and real-world patient- and staff-centered outcomes to address gaps in the literature.⁵ They noted that a mental health professional was utilized in only a minority of SMAs studied.⁵ Additionally, we noted a paucity of studies examining patient satisfaction with SMAs.

Another group-based intervention found to be effective in improving DM outcomes is the 6-session Stanford Diabetes Self-Management Program (DSMP), a workshop led in part by trained peers with DM. The sessions focus on educating patients on DM care and self-management tools. The workshop encourages active practice in building DM self-management skills and confidence. DSMP participation has been associated with improvement in DM-related outcomes, including Hb A_1c levels, amount of exercise, and medication adherence.⁷

While SMAs and DSMP have been shown to enhance clinical outcomes, they provide differing types of patient support. SMAs allow for frequent interaction with a health care professional (HCP) and less emphasis on behavioral health interventions. DSMPs include behavioral health professionals and peer leaders and emphasize higher levels of psychosocial support, but do not offer access to clinicians. It is possible that combining these interventions could result in better outcomes than what either could provide on their own.

In 2018, the Cincinnati Veterans Affairs Medical Center (VAMC) in Ohio offered Diabetes Basic Training, a structured DM intervention. Patients enrolled in the program participated in a 9-week intervention that included 3 SMAs and 6 DSMP sessions. During the SMAs, a clinical psychologist or psychology postdoctoral fellow skilled in motivational interviewing facilitated the group to enhance patient engagement and empowerment for improved self-management. In addition, patients participated in structured DSMP groups with an emphasis on action-planning, often surrounding nutrition, physical activity, and other health behavior change information reviewed during the SMAs.

Design and Referral

Self-management programs for chronic health conditions are often underutilized. Although HCPs may wish to connect veterans with available programs, time constraints may limit opportunities for detailed discussions with patients about specific aspects of each program. To simplify this process, a 2-hour orientation program was offered that explained individual and group DM self-management options (Figure). During this initial visit, patients met with an interdisciplinary care team (registered dietician, diabetes nurse practitioner, and behavioral health specialist) and were informed about Diabetes Basic Training, DM clinical care practices, and other related resources available at the Cincinnati VAMC (eg, cooking classes, food pantry). Patients received individualized referral recommendations and were urged to consult with their primary care practitioner to finalize their treatment plan.

Shared Medical Appointments

Diabetes Basic Training interventions had an average of 6 to 8 veterans participating in the weekly groups. The first, fifth, and final weeks were SMAs in which an interdisciplinary team collaboratively provided group-based health care for DM. The team consisted of a registered nurse, a prescriber (eg, nurse practitioner), a moderator (eg, psychologist), and a content expert (eg, nutritionist). Before each SMA began, the nurse checked-in patients in the SMA room and collected heart rate and blood pressure, and performed a diabetic foot check. Each SMA consisted of introductions, group-driven discussions (facilitated by an HCP) and troubleshooting DM self-management challenges. During group discussions, the prescriber initiated a 1-on-1 discussion with each patient in a private office regarding their recent laboratory results, medication regimen, and other aspects of DM care. The patient’s medications were refilled and/or adjusted as needed and other orders and referrals were submitted. If a patient had a medical question, the prescriber and moderator engaged the entire group so all individuals could benefit from generating and hearing answers. When discussion slowed, education was provided on topics generated by the group. Frequent topics included challenges managing DM, concerns, how DM impacted daily life and relationships, and sharing successes. As needed, HCPs spoke individually with patients following the SMA. Patients were sometimes asked, but never required, to do homework consistent with standard DM care (eg, recording what they eat or blood sugar levels). Each SMA session lasted about 2 hours.

Diabetes Self-Management Program

The second, third, fourth, sixth, seventh, and eighth weeks of the program were devoted to the DSMP. These sessions were delivered primarily by veteran peers who received appropriate training, observation, and certification. Each 2-hour educational program provided ample practice in many fundamental self-management skills, such as decision making, problem solving, and action planning. Patients were asked, but never required, to practice related skills during the sessions and to create weekly action plans to be completed between sessions that typically involved increasing exercise or improving diet. Patients were encouraged to follow up with HCPs at SMAs when they had questions requiring HCP expertise. If participants had more immediate concerns regarding their treatment plan and/or medications, they contacted their primary care practitioner prior to the next SMA.

As a part of participation in the program, psychosocial and health data and Hb A_1c levels at baseline (the closest level to 90 days prior to start) and follow-up (the closest level to 90 days after the final session) were collected.⁸ In addition, Problem Areas in Diabetes (PAID), Patient Activation Measure (PAM)-13, and Diabetes Self-Management Questionnaire (DSMQ) were administered at 3 points: during the orientation, in the first week, and in the ninth week of the program.

PAID, a 20-item self-report questionnaire designed to capture emotional distress related to having DM, is a valid and reliable scale able to detect changes over time when used in intervention studies.^9,10 PAM-13 is a 13-item measure designed to assess patient knowledge, skill, and confidence in the self-management of health or chronic conditions based on the original PAM.^11,12 Scores fall into 1 of 4 activation levels, ranging from low levels of confidence and knowledge of health management to high levels of being proactive with one’s care. The PAM-13 has been widely used within health psychology, including research among adults with multiple chronic conditions, individuals with DM or osteoarthritis, and within primary care.^13-15 The DSMQ is a statistically reliable and valid instrument that allows for user-friendly assessment of self-care behaviors associated with glycemic control.^16-18

All measures were collected as part of traditional clinical care, and we present initial program evaluation data to demonstrate potential effectiveness of the clinic model. Paired samples t tests were used to examine differences between baseline and follow-up measures for the 24 veteran participants. The age of participants who completed the program ranged from 42 to 74 years (mean, 68 years); 29% of participants were Black veterans and 12% were female. Examination of clinical outcomes indicated that veterans reported significant increases in activation levels for managing their health increasing from a baseline mean (SD) 62.1 (12.3) to 68.4 (14.5) at follow up (t[23] = 2.15, P = .04). Hb A_1c levels trended downward from a mean (SD) 8.6% (1.3) at baseline to 8.2% (1.2) at 90-day follow up (t[21] 1.05, P = .30). Similar nonsignificant trends in PAID scores were seen for pre- and postprogram reductions in emotional distress related to having DM from a mean (SD) 7.9 (5.0) at baseline to 6.3 (5.1) (t[18] = 11.51, P = .15), and enhanced self-management of glucose with a mean (SD) 6.5 (1.5) at baseline to 6.8 (1.3) at follow up (t[19] = 0.52, P = .61). The trends found in this study show promising outcomes for this pilot group-based DM treatment, though the small sample size (N = 24) limits statistical power. These findings support further exploration and expansion of interdisciplinary health programs supporting veteran self-management.

Discussion

DM is a condition of epidemic proportions that causes substantial negative health outcomes and costs at a national level. Current standards of DM care do not appear to be reversing these trends. Wider implementation of group-based treatment for DM could improve efficiency of care, increase access to quality care, and reduce burden on individual HCPs.

The VHA continues the transformation of its care system, which shifts toward a patient-centered, proactive focus on veteran well-being. This new whole health approach integrates conventional medical treatment with veteran self-empowerment in the pursuit of health goals based on individual veteran’s identified values.¹⁹ This approach emphasizes peer-led explorations of veterans’ aspirations, purpose, and individual mission, personalized health planning, and use of whole health coaches and well-being programs, with both allopathic and complementary and integrative clinical care centered around veterans’ identified goals and priorities.²⁰

Including a program like Diabetes Basic Training as a part of whole health programming could offer several benefits. Diabetes Basic Training is unique in its integration of more traditional SMA structure with psychosocial interventions including values identification and motivational interviewing strategies to enhance patient engagement. Veterans can learn from each other’s experiences and concerns, leading to better DM management knowledge and skills. The group nature of the sessions enhances opportunities for emotional support and reduced isolation, as well as peer accountability for maintaining medication adherence.

By meeting with HCPs from multiple disciplines, veterans are exposed to different perspectives on self-management techniques, including behavioral approaches for overcoming barriers to behavior change. Clinicians have more time to engage with patients, building stronger relationships and trust. SMAs are cost-efficient and time efficient, allowing HCPs to see multiple patients at once, reducing wait times and increasing the number of patients treated in a given time frame.

The COVID-19 pandemic temporarily impacted the ongoing expansion of the program, when so many services were shifted from in-person to virtual classes. Due to staffing and other logistic issues, our pilot program was suspended during that time, but plans to resume the program by early 2024 are moving forward.

CONCLUSIONS

The Diabetes Basic Training program serves as a successful model for implementation within a VAMC. Although the number of veterans with complete data available for analysis was small, the trends exhibited in the preliminary outcome data are promising. We encourage other VAMCs to replicate this program with a larger participant base and evaluate its impact on veteran health outcomes. Next steps include comparing the clinical data from treatment as usual with outcomes from DM group participants. As the program resumes, we will reinitiate recruitment efforts to increase HCP referrals to this program.

Social risk factors and social needs have significant, often cumulative, impacts on health outcomes and are closely tied to health inequities. Defined as the individual-level adverse social conditions associated with poor health, social risk factors broadly include experiences such as food insecurity and housing instability; whereas the term social needs incorporates a person’s perceptions of and priorities related to their health-related needs.¹ One recent study examining data from the Veterans Health Administration (VHA) found a 27% higher odds of mortality with each additional identified social risk, underscoring the critical link between social risks and veteran health outcomes.²

Assessing Circumstances and Offering Resources for Needs (ACORN), a collaborative quality improvement initiative conducted in partnership with the VHA Office of Health Equity and VHA National Social Work Program, Care Management and Social Work Services, is a social risk screening and referral program that aims to systematically identify and address unmet social needs among veterans to improve health and advance health equity.^3,4 ACORN consists of 2 components: (1) a veteran-tailored screener to identify social risks within 9 domains; and (2) provision of relevant VA and community resources and referrals to address identified needs.^3,5 Veterans who screen positive for ≥ 1 need receive referrals to a social worker or other relevant services, such as nutrition and food services or mental health, support navigating resources, and/or geographically tailored resource guides. This article describes the development and use of resource guides as a cross-cutting intervention component to address unmet social needs in diverse clinical settings and shares lessons learned from implementation in VHA outpatient clinics.

BACKGROUND

Unequal distribution of resources combined with historical discriminatory policies and practices, often linked to institutionalized racism, create inequities that lead to health disparities and hinder advancements in population health.^6,7 Although health care systems alone cannot eliminate all health inequities, they can implement programs to identify social risks and address individual-level needs as 1 component of the multilevel approach needed to achieve health equity.⁸

As a national health care system serving > 9 million veterans, the VHA is well positioned to address social needs as an essential part of health. The VHA routinely screens for certain social risks, including housing instability, food insecurity, and intimate partner violence, and has a robust system of supports to address these and other needs among veterans, such as supportive housing services, vocational rehabilitation, assistance for justice-involved veterans, technology access support, and peer-support services.^9-11 However, the VHA lacks a systematic approach to broader screening for social risks.

To address this gap, ACORN was developed in 2018 by an advisory board of subject matter experts, including clinical leaders, clinical psychologists, social workers, and health services researchers with content expertise in social risks and social needs.³ This interprofessional team sought to develop a veteran-tailored screener and resource referral initiative that could be scaled efficiently across VHA clinical settings.

Although health care organizations are increasingly implementing screening and interventions for social risks within clinical care, best practices and evidence-based tools to support clinical staff in these efforts are limited.¹² Resource guides—curated lists of supportive services and organizations—may serve as a scalable “low-touch” intervention to help clinical staff address needs either alone or with more intensive interventions, such as social worker case management or patient navigation services.¹³

RESOURCE GUIDES—A Cross-Cutting Tool

The VHA has a uniquely robust network of nearly 18,000 social workers with clinical expertise in identifying, comprehensively assessing, and addressing social risks and needs among veterans. Interprofessional patient aligned care teams (PACTs)—a patient-centered medical home initiative that includes embedding social workers into primary care teams—facilitate the VHA’s capacity to address both medical and social needs.¹⁴ Social workers in PACTs and other care settings provide in-depth assessment and case management services to veterans who have a range of complex social needs. However, despite these comprehensive social services, in the setting of universal screening with a tool such as ACORN, it may not be feasible or practical to refer all patients who screen positive to a social worker for immediate follow-up, particularly in settings with capacity or resource limitations. For example, rates of screening positive on ACORN for ≥ 1 social risk have ranged from 48% of veterans in primary care sites and 80% in social work sites to nearly 100% in a PACT clinic for veterans experiencing homelessness.¹⁵

Additionally, a key challenge in the design of social needs interventions is determining how to optimize intervention intensity based on individual patient needs, acuity, and preferences. A substantial proportion of individuals who screen positive for social risks decline offered assistance, such as referrals.¹⁶ Resource guides are a cross-cutting tool that can be offered to veterans across a variety of settings, including primary care, specialty clinics, or emergency departments, as either a standalone intervention or one provided in combination with other resources or services. For patients who may not be interested in or feel comfortable accepting assistance at the time of screening or for those who prefer to research and navigate resources on their own, tailored resource guides can serve as a lower touch intervention to ensure interprofessional clinical staff across a range of settings and specialties have accessible, reliable, and up-to-date information to give to patients at the point of clinical care.¹⁷

Resource guides also can be used with higher touch clinical social work interventions, such as crisis management, supportive counseling, and case management. For example, social workers can use resource guides to provide education on VHA or community resources during clinical encounters with veterans and/or provide the guides to veterans to reference for future needs. Resource guides can further be used as a tool to support community resource navigation provided by nonclinical staff, such as peer specialists or community health workers.

How to BUILD RESOURCE GUIDES

Our team created resource guides (Figure) to provide veterans with concise, geographically tailored lists of VHA and other federal, state, and community services for the social risk domains included on the ACORN screener. To inform and develop a framework for building and maintaining ACORN guides, we first reviewed existing models that use this approach, including Boston Medical Center's WE CARE (Well Child Care, Evaluation, Community Resources, Advocacy, Referral, Education) and Thrive programs. We provide an overview of our process, which can be applied to clinical settings both within and outside the VHA.^18,19

Partnerships

Active collaboration with frontline clinical social workers and local social work leadership is a critical part of identifying and prioritizing quality resources. Equipped with the knowledge of the local resource landscape, social workers can provide recommendations pertaining to national or federal, state, and local programs that have a history of being responsive to patients’ expressed needs.²⁰ VHA social workers have robust knowledge of the veteran-specific resources available at VHA medical centers and nationally, and their clinical training equips them with the expertise to provide guidance about which resources to prioritize for inclusion in the guides.²⁰

After receiving initial guidance from clinical social workers, our team began outreach to compile detailed program information, gauge program serviceability, and build relationships with both VHA and community-based services. Aligning with programs that share a similar mission in addressing social needs has proved crucial when developing the resource guides. Beyond ensuring the accuracy of program information, regular contact provides an opportunity to address capacity and workflow concerns that may arise from increased referrals. Additionally, open lines of communication with various supportive services facilitate connections with additional organizations and resources within the area.

The value of these relationships was evident at the onset of the COVID-19 pandemic, when the ACORN resource guides in use by our clinical site partners required frequent modifications to reflect rapid changes in services (eg, closures, transition to fully virtual programs, social distancing and masking requirements). Having established connections with community organizations was essential to navigating the evolving landscape of available programs and supports.

Curating Quality Resources

ACORN currently screens for social risks in 9 domains: food, housing, utilities, transportation, education, employment, legal, social isolation/loneliness, and digital needs. Each resource guide pertains to a specific social risk domain and associated question(s) in the screener, allowing staff to quickly identify which guides a veteran may benefit from based on their screening responses. The guides are meant to be short and comprehensive but not exhaustive lists of programs and services. We limit the length of the guides to one single-sided page to provide high-yield, geographically tailored resources in an easy-to-use format. The guides should reflect the geographic area served by the VHA medical center or the community-based outpatient clinic (CBOC) where a veteran receives clinical care, but they also may include national- and state-level resources that provide services and programs to veterans.

Although there is local variation in the availability and accessibility of services across social risk domains, some domains have an abundance of resources and organizations at federal or national, state, and/or community levels. To narrow the list of resources to the highest yield programs, we developed a series of questions that serve as selection criteria to inform resource inclusion (Table).

Because the resource guides are intended to be broadly applicable to a large number of veterans, we prioritize generalizable resources over those with narrow eligibility criteria and/or services. When more intensive support is needed, social workers and other VHA clinical and nonclinical staff can supplement the resources on the guides with additional, more tailored resources that are based on individual factors, such as physical residence, income, transportation access, or household composition (eg, veteran families with children or older adults).

Formatting Resource Guides

Along with relevant information, such as the program name, location, and a specific point of contact, brief program descriptions provide information about services offered, eligibility criteria, application requirements, alternate contacts and locations, and website links. At the bottom of each guide, a section is included with the name and direct contact information for a social worker (often the individual assigned to the clinic where the veteran completed the ACORN screener) or another VHA staff member who can be reached for further assistance. These staff members are familiar with the content included on the guides and provide veterans with additional information or higher touch support as necessary. This contact information is useful for veterans who initially decline assistance or referrals but later want to follow up with staff for support or questions.

Visual consistency is a key feature of the ACORN resource guides, and layout and design elements are used to maximize space and enhance usability. Corresponding font colors for all program titles, contact information, and website links assist in visually separating and drawing attention to pertinent contact information. QR codes linked to program websites also are incorporated for veterans to easily access resource information from their smartphone or other electronic device.

Maintaining Resource Guides

To ensure continued accuracy, resource guides are updated about every 6 months to reflect changes in closures, transitions to virtual/in-person services, changes in location, new points of contact, and modifications to services or eligibility requirements. Notations also are made if any changes to services or eligibility are temporary or permanent. Recording these temporary adjustments was critical early in the COVID-19 pandemic as service offerings, eligibility requirements, and application processes changed often.

Updating the guides also facilitates continuous relationship building and connections with VHA and community-based services. Resource guides are living documents: they maintain lines of communication with designated contacts, allow for opportunities to improve the presentation of evolving program information in the guides, and offer the chance to learn about additional programs in the area that may meet veterans' needs.

Creating a Manual for ACORN Resource Guide Development

To facilitate dissemination of ACORN across VHA clinical settings and locations, our team developed a Resource Guide Manual to aid ACORN clinical sites in developing resource guides.²¹ The manual provides step-by-step guidance from recommendations for identifying resources to formatting and layout considerations. Supplemental materials include a checklist to ensure each program description includes the necessary information for veterans to successfully access the resource, as well as page templates and style suggestions to maximize usability. These templates standardize formatting across the social risk domain guides and include options for electronic and paper distribution.

RESOURCE GUIDE LIMITATIONS

The labor involved in building and maintaining multiple guides is considerable and requires a time investment both upfront and long term, which may not be feasible for clinical sites with limited staff. However, many VHA social workers maintain lists of resource and referral services for veterans as part of their routine clinical case management. These lists can serve as a valuable and timesaving starting point in curating high-yield resources for formal resource guides. To further reduce the time needed to develop guides, sites can use ACORN resource guide templates rather than designing and formatting guides from scratch. In addition to informing veterans of relevant services and programs, resource guides also can be provided to new staff, such as social workers or peer specialists, during onboarding to help familiarize them with available services to address veterans’ unmet needs.

Resources included on the guides also are geographically tailored, based on the physical location of the VHA medical center or CBOC. Some community-based services listed may not be as relevant, accessible, available, or convenient to veterans who live far from the clinic, which is relevant for nearly 25% of veterans who live in rural communities.²² This is a circumstance in which the expertise of VHA social workers should be used to recommend more appropriately tailored resources to a veteran. Use of free, publicly available electronic resource databases (eg, 211 Helpline Center) also can provide social workers and patients with an overview of all available resources within their community. There are paid referral platform services that health systems can contract with as well.²³ However, the potential drawbacks to these alternative platforms include high startup costs or costly user-license fees for medical centers or clinics, inconsistent updates to resource information, and lack of compatibility with some electronic health record systems.²³

Resource guides are not intended to take the place of a clinical social worker or other health professional but rather to serve in a supplemental capacity to clinical services. Certain circumstances necessitate a more comprehensive clinical assessment and/or a warm handoff to a social worker, including assistance with urgent food or housing needs, and ACORN workflows are created with urgent needs pathways in mind. Determining how to optimize intervention intensity based on individual patients’ expressed needs, preferences, and acuity remains a challenge for health care organizations conducting social risk screening.¹² While distribution of geographically tailored resource guides can be a useful low touch intervention for some veterans, others will require more intensive case management to address or meet their needs. Some veterans also may fall in the middle of this spectrum, where a resource guide is not enough but intensive case management services facilitated by social workers are not needed or wanted by the patient. Integration of peer specialists, patient navigators, or community health workers who can work with veterans to support them in identifying, connecting with, and receiving support from relevant programs may help fill this gap. Given their knowledge and lived experience, these professionals also can promote patient-centered care as part of the health care team.

CONCLUSIONS

Whether used as a low-touch, standalone intervention or in combination with higher touch services (eg, case management or resource navigation), resource guides are a valuable tool for health care organizations working to address social needs as a component of efforts to advance health equity, reduce health disparities, and promote population health. We provide a pragmatic framework for developing and maintaining resource guides used in the ACORN initiative. However, additional work is needed to optimize the design, content, and format of resource guides for both usability and effectiveness as a social risk intervention across health care settings.

Acknowledgments

We express our gratitude for the Veterans Health Administration (VHA) Office of Health Equity and the VHA National Social Work Program, Care Management and Social Work Services for their support of the Assessing Circumstances and Offering Resources for Needs (ACORN) initiative. We also express our appreciation for those who supported the initial screener development as part of the ACORN Advisory Board, including Stacey Curran, BA; Charles Drebing, PhD; J. Stewart Evans, MD, MSc; Edward Federman, PhD; Maneesha Gulati, LICSW, ACSW; Nancy Kressin, PhD; Kenneth Link, LICSW; Monica Sharma, MD; and Jacqueline Spencer, MD, MPH. We also express our appreciation for those who supported the initial ACORN resource guide development, including Chuck Drebing, PhD, Ed Federman, PhD, and Ken Link, LICSW, and for the clinical care team members, especially the social workers and nurses, at our ACORN partner sites as well as the community-based partners who have helped us develop comprehensive resource guides for veterans. This work was supported by funding from the VHA Office of Health Equity and by resources and use of facilities at the VA Bedford Healthcare System, VA New England Healthcare System, and VA Providence Healthcare System. Alicia J. Cohen was additionally supported by a VA HSR&D Career Development Award (CDA 20-037).

Social risk factors and social needs have significant, often cumulative, impacts on health outcomes and are closely tied to health inequities. Defined as the individual-level adverse social conditions associated with poor health, social risk factors broadly include experiences such as food insecurity and housing instability; whereas the term social needs incorporates a person’s perceptions of and priorities related to their health-related needs.¹ One recent study examining data from the Veterans Health Administration (VHA) found a 27% higher odds of mortality with each additional identified social risk, underscoring the critical link between social risks and veteran health outcomes.²

Assessing Circumstances and Offering Resources for Needs (ACORN), a collaborative quality improvement initiative conducted in partnership with the VHA Office of Health Equity and VHA National Social Work Program, Care Management and Social Work Services, is a social risk screening and referral program that aims to systematically identify and address unmet social needs among veterans to improve health and advance health equity.^3,4 ACORN consists of 2 components: (1) a veteran-tailored screener to identify social risks within 9 domains; and (2) provision of relevant VA and community resources and referrals to address identified needs.^3,5 Veterans who screen positive for ≥ 1 need receive referrals to a social worker or other relevant services, such as nutrition and food services or mental health, support navigating resources, and/or geographically tailored resource guides. This article describes the development and use of resource guides as a cross-cutting intervention component to address unmet social needs in diverse clinical settings and shares lessons learned from implementation in VHA outpatient clinics.

BACKGROUND

Unequal distribution of resources combined with historical discriminatory policies and practices, often linked to institutionalized racism, create inequities that lead to health disparities and hinder advancements in population health.^6,7 Although health care systems alone cannot eliminate all health inequities, they can implement programs to identify social risks and address individual-level needs as 1 component of the multilevel approach needed to achieve health equity.⁸

As a national health care system serving > 9 million veterans, the VHA is well positioned to address social needs as an essential part of health. The VHA routinely screens for certain social risks, including housing instability, food insecurity, and intimate partner violence, and has a robust system of supports to address these and other needs among veterans, such as supportive housing services, vocational rehabilitation, assistance for justice-involved veterans, technology access support, and peer-support services.^9-11 However, the VHA lacks a systematic approach to broader screening for social risks.

To address this gap, ACORN was developed in 2018 by an advisory board of subject matter experts, including clinical leaders, clinical psychologists, social workers, and health services researchers with content expertise in social risks and social needs.³ This interprofessional team sought to develop a veteran-tailored screener and resource referral initiative that could be scaled efficiently across VHA clinical settings.

Although health care organizations are increasingly implementing screening and interventions for social risks within clinical care, best practices and evidence-based tools to support clinical staff in these efforts are limited.¹² Resource guides—curated lists of supportive services and organizations—may serve as a scalable “low-touch” intervention to help clinical staff address needs either alone or with more intensive interventions, such as social worker case management or patient navigation services.¹³

RESOURCE GUIDES—A Cross-Cutting Tool

The VHA has a uniquely robust network of nearly 18,000 social workers with clinical expertise in identifying, comprehensively assessing, and addressing social risks and needs among veterans. Interprofessional patient aligned care teams (PACTs)—a patient-centered medical home initiative that includes embedding social workers into primary care teams—facilitate the VHA’s capacity to address both medical and social needs.¹⁴ Social workers in PACTs and other care settings provide in-depth assessment and case management services to veterans who have a range of complex social needs. However, despite these comprehensive social services, in the setting of universal screening with a tool such as ACORN, it may not be feasible or practical to refer all patients who screen positive to a social worker for immediate follow-up, particularly in settings with capacity or resource limitations. For example, rates of screening positive on ACORN for ≥ 1 social risk have ranged from 48% of veterans in primary care sites and 80% in social work sites to nearly 100% in a PACT clinic for veterans experiencing homelessness.¹⁵

Additionally, a key challenge in the design of social needs interventions is determining how to optimize intervention intensity based on individual patient needs, acuity, and preferences. A substantial proportion of individuals who screen positive for social risks decline offered assistance, such as referrals.¹⁶ Resource guides are a cross-cutting tool that can be offered to veterans across a variety of settings, including primary care, specialty clinics, or emergency departments, as either a standalone intervention or one provided in combination with other resources or services. For patients who may not be interested in or feel comfortable accepting assistance at the time of screening or for those who prefer to research and navigate resources on their own, tailored resource guides can serve as a lower touch intervention to ensure interprofessional clinical staff across a range of settings and specialties have accessible, reliable, and up-to-date information to give to patients at the point of clinical care.¹⁷

Resource guides also can be used with higher touch clinical social work interventions, such as crisis management, supportive counseling, and case management. For example, social workers can use resource guides to provide education on VHA or community resources during clinical encounters with veterans and/or provide the guides to veterans to reference for future needs. Resource guides can further be used as a tool to support community resource navigation provided by nonclinical staff, such as peer specialists or community health workers.

How to BUILD RESOURCE GUIDES

Our team created resource guides (Figure) to provide veterans with concise, geographically tailored lists of VHA and other federal, state, and community services for the social risk domains included on the ACORN screener. To inform and develop a framework for building and maintaining ACORN guides, we first reviewed existing models that use this approach, including Boston Medical Center's WE CARE (Well Child Care, Evaluation, Community Resources, Advocacy, Referral, Education) and Thrive programs. We provide an overview of our process, which can be applied to clinical settings both within and outside the VHA.^18,19

Partnerships

Active collaboration with frontline clinical social workers and local social work leadership is a critical part of identifying and prioritizing quality resources. Equipped with the knowledge of the local resource landscape, social workers can provide recommendations pertaining to national or federal, state, and local programs that have a history of being responsive to patients’ expressed needs.²⁰ VHA social workers have robust knowledge of the veteran-specific resources available at VHA medical centers and nationally, and their clinical training equips them with the expertise to provide guidance about which resources to prioritize for inclusion in the guides.²⁰

After receiving initial guidance from clinical social workers, our team began outreach to compile detailed program information, gauge program serviceability, and build relationships with both VHA and community-based services. Aligning with programs that share a similar mission in addressing social needs has proved crucial when developing the resource guides. Beyond ensuring the accuracy of program information, regular contact provides an opportunity to address capacity and workflow concerns that may arise from increased referrals. Additionally, open lines of communication with various supportive services facilitate connections with additional organizations and resources within the area.

The value of these relationships was evident at the onset of the COVID-19 pandemic, when the ACORN resource guides in use by our clinical site partners required frequent modifications to reflect rapid changes in services (eg, closures, transition to fully virtual programs, social distancing and masking requirements). Having established connections with community organizations was essential to navigating the evolving landscape of available programs and supports.

Curating Quality Resources

ACORN currently screens for social risks in 9 domains: food, housing, utilities, transportation, education, employment, legal, social isolation/loneliness, and digital needs. Each resource guide pertains to a specific social risk domain and associated question(s) in the screener, allowing staff to quickly identify which guides a veteran may benefit from based on their screening responses. The guides are meant to be short and comprehensive but not exhaustive lists of programs and services. We limit the length of the guides to one single-sided page to provide high-yield, geographically tailored resources in an easy-to-use format. The guides should reflect the geographic area served by the VHA medical center or the community-based outpatient clinic (CBOC) where a veteran receives clinical care, but they also may include national- and state-level resources that provide services and programs to veterans.

Although there is local variation in the availability and accessibility of services across social risk domains, some domains have an abundance of resources and organizations at federal or national, state, and/or community levels. To narrow the list of resources to the highest yield programs, we developed a series of questions that serve as selection criteria to inform resource inclusion (Table).

Because the resource guides are intended to be broadly applicable to a large number of veterans, we prioritize generalizable resources over those with narrow eligibility criteria and/or services. When more intensive support is needed, social workers and other VHA clinical and nonclinical staff can supplement the resources on the guides with additional, more tailored resources that are based on individual factors, such as physical residence, income, transportation access, or household composition (eg, veteran families with children or older adults).

Formatting Resource Guides

Along with relevant information, such as the program name, location, and a specific point of contact, brief program descriptions provide information about services offered, eligibility criteria, application requirements, alternate contacts and locations, and website links. At the bottom of each guide, a section is included with the name and direct contact information for a social worker (often the individual assigned to the clinic where the veteran completed the ACORN screener) or another VHA staff member who can be reached for further assistance. These staff members are familiar with the content included on the guides and provide veterans with additional information or higher touch support as necessary. This contact information is useful for veterans who initially decline assistance or referrals but later want to follow up with staff for support or questions.

Visual consistency is a key feature of the ACORN resource guides, and layout and design elements are used to maximize space and enhance usability. Corresponding font colors for all program titles, contact information, and website links assist in visually separating and drawing attention to pertinent contact information. QR codes linked to program websites also are incorporated for veterans to easily access resource information from their smartphone or other electronic device.

Maintaining Resource Guides

To ensure continued accuracy, resource guides are updated about every 6 months to reflect changes in closures, transitions to virtual/in-person services, changes in location, new points of contact, and modifications to services or eligibility requirements. Notations also are made if any changes to services or eligibility are temporary or permanent. Recording these temporary adjustments was critical early in the COVID-19 pandemic as service offerings, eligibility requirements, and application processes changed often.

Updating the guides also facilitates continuous relationship building and connections with VHA and community-based services. Resource guides are living documents: they maintain lines of communication with designated contacts, allow for opportunities to improve the presentation of evolving program information in the guides, and offer the chance to learn about additional programs in the area that may meet veterans' needs.

Creating a Manual for ACORN Resource Guide Development

To facilitate dissemination of ACORN across VHA clinical settings and locations, our team developed a Resource Guide Manual to aid ACORN clinical sites in developing resource guides.²¹ The manual provides step-by-step guidance from recommendations for identifying resources to formatting and layout considerations. Supplemental materials include a checklist to ensure each program description includes the necessary information for veterans to successfully access the resource, as well as page templates and style suggestions to maximize usability. These templates standardize formatting across the social risk domain guides and include options for electronic and paper distribution.

RESOURCE GUIDE LIMITATIONS

The labor involved in building and maintaining multiple guides is considerable and requires a time investment both upfront and long term, which may not be feasible for clinical sites with limited staff. However, many VHA social workers maintain lists of resource and referral services for veterans as part of their routine clinical case management. These lists can serve as a valuable and timesaving starting point in curating high-yield resources for formal resource guides. To further reduce the time needed to develop guides, sites can use ACORN resource guide templates rather than designing and formatting guides from scratch. In addition to informing veterans of relevant services and programs, resource guides also can be provided to new staff, such as social workers or peer specialists, during onboarding to help familiarize them with available services to address veterans’ unmet needs.

Resources included on the guides also are geographically tailored, based on the physical location of the VHA medical center or CBOC. Some community-based services listed may not be as relevant, accessible, available, or convenient to veterans who live far from the clinic, which is relevant for nearly 25% of veterans who live in rural communities.²² This is a circumstance in which the expertise of VHA social workers should be used to recommend more appropriately tailored resources to a veteran. Use of free, publicly available electronic resource databases (eg, 211 Helpline Center) also can provide social workers and patients with an overview of all available resources within their community. There are paid referral platform services that health systems can contract with as well.²³ However, the potential drawbacks to these alternative platforms include high startup costs or costly user-license fees for medical centers or clinics, inconsistent updates to resource information, and lack of compatibility with some electronic health record systems.²³

Resource guides are not intended to take the place of a clinical social worker or other health professional but rather to serve in a supplemental capacity to clinical services. Certain circumstances necessitate a more comprehensive clinical assessment and/or a warm handoff to a social worker, including assistance with urgent food or housing needs, and ACORN workflows are created with urgent needs pathways in mind. Determining how to optimize intervention intensity based on individual patients’ expressed needs, preferences, and acuity remains a challenge for health care organizations conducting social risk screening.¹² While distribution of geographically tailored resource guides can be a useful low touch intervention for some veterans, others will require more intensive case management to address or meet their needs. Some veterans also may fall in the middle of this spectrum, where a resource guide is not enough but intensive case management services facilitated by social workers are not needed or wanted by the patient. Integration of peer specialists, patient navigators, or community health workers who can work with veterans to support them in identifying, connecting with, and receiving support from relevant programs may help fill this gap. Given their knowledge and lived experience, these professionals also can promote patient-centered care as part of the health care team.

CONCLUSIONS

Whether used as a low-touch, standalone intervention or in combination with higher touch services (eg, case management or resource navigation), resource guides are a valuable tool for health care organizations working to address social needs as a component of efforts to advance health equity, reduce health disparities, and promote population health. We provide a pragmatic framework for developing and maintaining resource guides used in the ACORN initiative. However, additional work is needed to optimize the design, content, and format of resource guides for both usability and effectiveness as a social risk intervention across health care settings.

Acknowledgments

We express our gratitude for the Veterans Health Administration (VHA) Office of Health Equity and the VHA National Social Work Program, Care Management and Social Work Services for their support of the Assessing Circumstances and Offering Resources for Needs (ACORN) initiative. We also express our appreciation for those who supported the initial screener development as part of the ACORN Advisory Board, including Stacey Curran, BA; Charles Drebing, PhD; J. Stewart Evans, MD, MSc; Edward Federman, PhD; Maneesha Gulati, LICSW, ACSW; Nancy Kressin, PhD; Kenneth Link, LICSW; Monica Sharma, MD; and Jacqueline Spencer, MD, MPH. We also express our appreciation for those who supported the initial ACORN resource guide development, including Chuck Drebing, PhD, Ed Federman, PhD, and Ken Link, LICSW, and for the clinical care team members, especially the social workers and nurses, at our ACORN partner sites as well as the community-based partners who have helped us develop comprehensive resource guides for veterans. This work was supported by funding from the VHA Office of Health Equity and by resources and use of facilities at the VA Bedford Healthcare System, VA New England Healthcare System, and VA Providence Healthcare System. Alicia J. Cohen was additionally supported by a VA HSR&D Career Development Award (CDA 20-037).

Social risk factors and social needs have significant, often cumulative, impacts on health outcomes and are closely tied to health inequities. Defined as the individual-level adverse social conditions associated with poor health, social risk factors broadly include experiences such as food insecurity and housing instability; whereas the term social needs incorporates a person’s perceptions of and priorities related to their health-related needs.¹ One recent study examining data from the Veterans Health Administration (VHA) found a 27% higher odds of mortality with each additional identified social risk, underscoring the critical link between social risks and veteran health outcomes.²

Assessing Circumstances and Offering Resources for Needs (ACORN), a collaborative quality improvement initiative conducted in partnership with the VHA Office of Health Equity and VHA National Social Work Program, Care Management and Social Work Services, is a social risk screening and referral program that aims to systematically identify and address unmet social needs among veterans to improve health and advance health equity.^3,4 ACORN consists of 2 components: (1) a veteran-tailored screener to identify social risks within 9 domains; and (2) provision of relevant VA and community resources and referrals to address identified needs.^3,5 Veterans who screen positive for ≥ 1 need receive referrals to a social worker or other relevant services, such as nutrition and food services or mental health, support navigating resources, and/or geographically tailored resource guides. This article describes the development and use of resource guides as a cross-cutting intervention component to address unmet social needs in diverse clinical settings and shares lessons learned from implementation in VHA outpatient clinics.

BACKGROUND

Unequal distribution of resources combined with historical discriminatory policies and practices, often linked to institutionalized racism, create inequities that lead to health disparities and hinder advancements in population health.^6,7 Although health care systems alone cannot eliminate all health inequities, they can implement programs to identify social risks and address individual-level needs as 1 component of the multilevel approach needed to achieve health equity.⁸

As a national health care system serving > 9 million veterans, the VHA is well positioned to address social needs as an essential part of health. The VHA routinely screens for certain social risks, including housing instability, food insecurity, and intimate partner violence, and has a robust system of supports to address these and other needs among veterans, such as supportive housing services, vocational rehabilitation, assistance for justice-involved veterans, technology access support, and peer-support services.^9-11 However, the VHA lacks a systematic approach to broader screening for social risks.

To address this gap, ACORN was developed in 2018 by an advisory board of subject matter experts, including clinical leaders, clinical psychologists, social workers, and health services researchers with content expertise in social risks and social needs.³ This interprofessional team sought to develop a veteran-tailored screener and resource referral initiative that could be scaled efficiently across VHA clinical settings.

Although health care organizations are increasingly implementing screening and interventions for social risks within clinical care, best practices and evidence-based tools to support clinical staff in these efforts are limited.¹² Resource guides—curated lists of supportive services and organizations—may serve as a scalable “low-touch” intervention to help clinical staff address needs either alone or with more intensive interventions, such as social worker case management or patient navigation services.¹³

RESOURCE GUIDES—A Cross-Cutting Tool

The VHA has a uniquely robust network of nearly 18,000 social workers with clinical expertise in identifying, comprehensively assessing, and addressing social risks and needs among veterans. Interprofessional patient aligned care teams (PACTs)—a patient-centered medical home initiative that includes embedding social workers into primary care teams—facilitate the VHA’s capacity to address both medical and social needs.¹⁴ Social workers in PACTs and other care settings provide in-depth assessment and case management services to veterans who have a range of complex social needs. However, despite these comprehensive social services, in the setting of universal screening with a tool such as ACORN, it may not be feasible or practical to refer all patients who screen positive to a social worker for immediate follow-up, particularly in settings with capacity or resource limitations. For example, rates of screening positive on ACORN for ≥ 1 social risk have ranged from 48% of veterans in primary care sites and 80% in social work sites to nearly 100% in a PACT clinic for veterans experiencing homelessness.¹⁵

Additionally, a key challenge in the design of social needs interventions is determining how to optimize intervention intensity based on individual patient needs, acuity, and preferences. A substantial proportion of individuals who screen positive for social risks decline offered assistance, such as referrals.¹⁶ Resource guides are a cross-cutting tool that can be offered to veterans across a variety of settings, including primary care, specialty clinics, or emergency departments, as either a standalone intervention or one provided in combination with other resources or services. For patients who may not be interested in or feel comfortable accepting assistance at the time of screening or for those who prefer to research and navigate resources on their own, tailored resource guides can serve as a lower touch intervention to ensure interprofessional clinical staff across a range of settings and specialties have accessible, reliable, and up-to-date information to give to patients at the point of clinical care.¹⁷

Resource guides also can be used with higher touch clinical social work interventions, such as crisis management, supportive counseling, and case management. For example, social workers can use resource guides to provide education on VHA or community resources during clinical encounters with veterans and/or provide the guides to veterans to reference for future needs. Resource guides can further be used as a tool to support community resource navigation provided by nonclinical staff, such as peer specialists or community health workers.

How to BUILD RESOURCE GUIDES

Our team created resource guides (Figure) to provide veterans with concise, geographically tailored lists of VHA and other federal, state, and community services for the social risk domains included on the ACORN screener. To inform and develop a framework for building and maintaining ACORN guides, we first reviewed existing models that use this approach, including Boston Medical Center's WE CARE (Well Child Care, Evaluation, Community Resources, Advocacy, Referral, Education) and Thrive programs. We provide an overview of our process, which can be applied to clinical settings both within and outside the VHA.^18,19

Partnerships

Active collaboration with frontline clinical social workers and local social work leadership is a critical part of identifying and prioritizing quality resources. Equipped with the knowledge of the local resource landscape, social workers can provide recommendations pertaining to national or federal, state, and local programs that have a history of being responsive to patients’ expressed needs.²⁰ VHA social workers have robust knowledge of the veteran-specific resources available at VHA medical centers and nationally, and their clinical training equips them with the expertise to provide guidance about which resources to prioritize for inclusion in the guides.²⁰

After receiving initial guidance from clinical social workers, our team began outreach to compile detailed program information, gauge program serviceability, and build relationships with both VHA and community-based services. Aligning with programs that share a similar mission in addressing social needs has proved crucial when developing the resource guides. Beyond ensuring the accuracy of program information, regular contact provides an opportunity to address capacity and workflow concerns that may arise from increased referrals. Additionally, open lines of communication with various supportive services facilitate connections with additional organizations and resources within the area.

The value of these relationships was evident at the onset of the COVID-19 pandemic, when the ACORN resource guides in use by our clinical site partners required frequent modifications to reflect rapid changes in services (eg, closures, transition to fully virtual programs, social distancing and masking requirements). Having established connections with community organizations was essential to navigating the evolving landscape of available programs and supports.

Curating Quality Resources

ACORN currently screens for social risks in 9 domains: food, housing, utilities, transportation, education, employment, legal, social isolation/loneliness, and digital needs. Each resource guide pertains to a specific social risk domain and associated question(s) in the screener, allowing staff to quickly identify which guides a veteran may benefit from based on their screening responses. The guides are meant to be short and comprehensive but not exhaustive lists of programs and services. We limit the length of the guides to one single-sided page to provide high-yield, geographically tailored resources in an easy-to-use format. The guides should reflect the geographic area served by the VHA medical center or the community-based outpatient clinic (CBOC) where a veteran receives clinical care, but they also may include national- and state-level resources that provide services and programs to veterans.

Although there is local variation in the availability and accessibility of services across social risk domains, some domains have an abundance of resources and organizations at federal or national, state, and/or community levels. To narrow the list of resources to the highest yield programs, we developed a series of questions that serve as selection criteria to inform resource inclusion (Table).

Because the resource guides are intended to be broadly applicable to a large number of veterans, we prioritize generalizable resources over those with narrow eligibility criteria and/or services. When more intensive support is needed, social workers and other VHA clinical and nonclinical staff can supplement the resources on the guides with additional, more tailored resources that are based on individual factors, such as physical residence, income, transportation access, or household composition (eg, veteran families with children or older adults).

Formatting Resource Guides

Along with relevant information, such as the program name, location, and a specific point of contact, brief program descriptions provide information about services offered, eligibility criteria, application requirements, alternate contacts and locations, and website links. At the bottom of each guide, a section is included with the name and direct contact information for a social worker (often the individual assigned to the clinic where the veteran completed the ACORN screener) or another VHA staff member who can be reached for further assistance. These staff members are familiar with the content included on the guides and provide veterans with additional information or higher touch support as necessary. This contact information is useful for veterans who initially decline assistance or referrals but later want to follow up with staff for support or questions.

Visual consistency is a key feature of the ACORN resource guides, and layout and design elements are used to maximize space and enhance usability. Corresponding font colors for all program titles, contact information, and website links assist in visually separating and drawing attention to pertinent contact information. QR codes linked to program websites also are incorporated for veterans to easily access resource information from their smartphone or other electronic device.

Maintaining Resource Guides

To ensure continued accuracy, resource guides are updated about every 6 months to reflect changes in closures, transitions to virtual/in-person services, changes in location, new points of contact, and modifications to services or eligibility requirements. Notations also are made if any changes to services or eligibility are temporary or permanent. Recording these temporary adjustments was critical early in the COVID-19 pandemic as service offerings, eligibility requirements, and application processes changed often.

Updating the guides also facilitates continuous relationship building and connections with VHA and community-based services. Resource guides are living documents: they maintain lines of communication with designated contacts, allow for opportunities to improve the presentation of evolving program information in the guides, and offer the chance to learn about additional programs in the area that may meet veterans' needs.

Creating a Manual for ACORN Resource Guide Development

To facilitate dissemination of ACORN across VHA clinical settings and locations, our team developed a Resource Guide Manual to aid ACORN clinical sites in developing resource guides.²¹ The manual provides step-by-step guidance from recommendations for identifying resources to formatting and layout considerations. Supplemental materials include a checklist to ensure each program description includes the necessary information for veterans to successfully access the resource, as well as page templates and style suggestions to maximize usability. These templates standardize formatting across the social risk domain guides and include options for electronic and paper distribution.

RESOURCE GUIDE LIMITATIONS

The labor involved in building and maintaining multiple guides is considerable and requires a time investment both upfront and long term, which may not be feasible for clinical sites with limited staff. However, many VHA social workers maintain lists of resource and referral services for veterans as part of their routine clinical case management. These lists can serve as a valuable and timesaving starting point in curating high-yield resources for formal resource guides. To further reduce the time needed to develop guides, sites can use ACORN resource guide templates rather than designing and formatting guides from scratch. In addition to informing veterans of relevant services and programs, resource guides also can be provided to new staff, such as social workers or peer specialists, during onboarding to help familiarize them with available services to address veterans’ unmet needs.

Resources included on the guides also are geographically tailored, based on the physical location of the VHA medical center or CBOC. Some community-based services listed may not be as relevant, accessible, available, or convenient to veterans who live far from the clinic, which is relevant for nearly 25% of veterans who live in rural communities.²² This is a circumstance in which the expertise of VHA social workers should be used to recommend more appropriately tailored resources to a veteran. Use of free, publicly available electronic resource databases (eg, 211 Helpline Center) also can provide social workers and patients with an overview of all available resources within their community. There are paid referral platform services that health systems can contract with as well.²³ However, the potential drawbacks to these alternative platforms include high startup costs or costly user-license fees for medical centers or clinics, inconsistent updates to resource information, and lack of compatibility with some electronic health record systems.²³

Resource guides are not intended to take the place of a clinical social worker or other health professional but rather to serve in a supplemental capacity to clinical services. Certain circumstances necessitate a more comprehensive clinical assessment and/or a warm handoff to a social worker, including assistance with urgent food or housing needs, and ACORN workflows are created with urgent needs pathways in mind. Determining how to optimize intervention intensity based on individual patients’ expressed needs, preferences, and acuity remains a challenge for health care organizations conducting social risk screening.¹² While distribution of geographically tailored resource guides can be a useful low touch intervention for some veterans, others will require more intensive case management to address or meet their needs. Some veterans also may fall in the middle of this spectrum, where a resource guide is not enough but intensive case management services facilitated by social workers are not needed or wanted by the patient. Integration of peer specialists, patient navigators, or community health workers who can work with veterans to support them in identifying, connecting with, and receiving support from relevant programs may help fill this gap. Given their knowledge and lived experience, these professionals also can promote patient-centered care as part of the health care team.

CONCLUSIONS

Whether used as a low-touch, standalone intervention or in combination with higher touch services (eg, case management or resource navigation), resource guides are a valuable tool for health care organizations working to address social needs as a component of efforts to advance health equity, reduce health disparities, and promote population health. We provide a pragmatic framework for developing and maintaining resource guides used in the ACORN initiative. However, additional work is needed to optimize the design, content, and format of resource guides for both usability and effectiveness as a social risk intervention across health care settings.

Acknowledgments

We express our gratitude for the Veterans Health Administration (VHA) Office of Health Equity and the VHA National Social Work Program, Care Management and Social Work Services for their support of the Assessing Circumstances and Offering Resources for Needs (ACORN) initiative. We also express our appreciation for those who supported the initial screener development as part of the ACORN Advisory Board, including Stacey Curran, BA; Charles Drebing, PhD; J. Stewart Evans, MD, MSc; Edward Federman, PhD; Maneesha Gulati, LICSW, ACSW; Nancy Kressin, PhD; Kenneth Link, LICSW; Monica Sharma, MD; and Jacqueline Spencer, MD, MPH. We also express our appreciation for those who supported the initial ACORN resource guide development, including Chuck Drebing, PhD, Ed Federman, PhD, and Ken Link, LICSW, and for the clinical care team members, especially the social workers and nurses, at our ACORN partner sites as well as the community-based partners who have helped us develop comprehensive resource guides for veterans. This work was supported by funding from the VHA Office of Health Equity and by resources and use of facilities at the VA Bedford Healthcare System, VA New England Healthcare System, and VA Providence Healthcare System. Alicia J. Cohen was additionally supported by a VA HSR&D Career Development Award (CDA 20-037).

The hospital is altogether the most complex human organization ever devised. Peter Drucker ¹

The ever-changing landscape of today’s increasingly complex health care system depends on implementing multifaceted, team-based methods of care delivery to provide safe, effective patient care.²Critical to establishing and sustaining exceptionally safe, effective patient care is open, transparent communication among members of interprofessional teams with senior leaders.³ However, current evidence shows thatpoor communication among interprofessional health care teams and leadership is commonplace and a significant contributing factor to inefficiencies, medical errors, and poor outcomes.⁴ One strategy for improving communication is through the implementation of leader rounding for high reliability. The concept of high reliability pertains to organizations that operate in high-risk environments for prolonged periods without serious adverse events or catastrophic failures.⁵ The overarching goal of implementation is to ensure that efficient communication exists among members of the health care team, which is essential for providing safe, quality patient care.

We describe the importance of leader rounding for high reliability as an approach to improving patient safety.Based on a review of the literature, our experiences, and lessons learned, we offer recommendations for how health care organizations on the journey to high reliability can improve patient safety.

Rounding in health care is not new. In fact, rounding has been a strong principal practice globally for more than 2 decades.⁶ During this time, varied rounding approaches have emerged, oftentimes focused on areas of interest, such as patient care, environmental services, facilities management, and discharge planning.^4,7 Variations also might involve the location of the rounds, such as a patient’s bedside, unit hallways, and conference rooms as well as the naming of rounds, such as interdisciplinary/multidisciplinary, teaching, and walkrounds.^7-10

A different type of rounding that is characteristic of high reliability organizations (HROs) is leader rounding for high reliability. The Veterans Health Administration (VHA) formally launched its journey to becoming an enterprise HRO in February 2019, using 3 cohorts. At the Veterans Affairs Bedford Healthcare System (VABHS) in Massachusetts, the journey commenced in 2021 as part of the third cohort. Leader rounding for high reliability is one of VHAs 4 HRO foundational practices for achieving a culture of safety (Figure 1).¹¹

Leader rounding for high reliability includes regularly scheduled, structured visits, with interdisciplinary teams to discuss high reliability, safety, and improvement efforts.The specific aim of these particular rounds is for senior leaders to be visible where teams are located and learn from staff (especially those on the frontlines of care) about day-to-day challenges that may contribute to patient harm.^12,13 Leader rounding for high reliability is also an important approach to improving leadership visibility across the organization, demonstrating a commitment to high reliability, and building trust and relationships with staff through open and honest dialogue. It is also an important approach to increasing leadership understanding of operational, clinical, nonclinical, patient experience issues, and concern related to safety.¹¹ This opportunity enables leaders to provide and receive real-time feedback from staff.^9,11 This experience also gives leaders an opportunity to reinforce the VHA’s 3 pillars, 5 principles, and 7 values related to high reliability (Figure 2)¹⁴ as well as to recognize behaviors that support a culture of safety.¹⁵

In preparation for implementing a leader rounding for high reliability process at the VABHS, we conducted an extensive literature review for peer-reviewed publications published between January 2015 and September 2022 regarding how other organizations implemented leader rounding. This search found a dearth of evidence as it specifically relates to leader rounding for high reliability. This motivated us to create a process for developing and implementing leader rounding for high reliability in pursuit of improving patient safety. With this objective in mind, we created and piloted a process in the fall of 2023. The first 3 months were focused on the medical center director rounding with other members of the executive leadership team to assess the feasibility and acceptability of the process. In December 2023, members of the executive leadership team began conducting leader rounding for high reliability separately. The following steps are based on the lessons we have gleaned from evolving evidence, our experiences, and developing and implementing an approach to leader rounding for high reliability.

ESTABLISH A PROCESS

Leader rounding for high reliability is performed by health care organization executive leadership, directors, managers, and supervisors. When properly conducted, increased levels of teamwork and more effective bidirectional communication take place, resulting in a united team motivated to improve patient safety.^16,17 Important early steps for implementing leader rounding for high reliability include establishing a process and getting leadership buy-in.Purposeful attention to planning is critical as is understanding the organizational factors that might deter success.Establishing a process should consider facilitators and barriers to implementation, which can include high vs low leadership turnover, structured vs unstructured rounding, and time for rounding vs competing demands.^18,19 We have learned thateffective planning is important for ensuring that leadership teams are well prepared and ambitious about leader rounding for high reliability.

Leader rounding for high reliability involves brief 10-to-15-minute interactions with interdisciplinary teams, including frontline staff. For health care organizations beginning to implement this approach, having scripts or checklists accessible might be of help. If possible, the rounds should be scheduled in advance. This helps to avoid rounding in areas at their busiest times. When possible, leader rounding for high reliability should occur as planned. Canceling rounds sends the message that leader rounding for high reliability and the valuable interactions they support are a low priority. When conflicts arise, another leader should be sent to participate. Developing a list of questions in advance helps to underscore key messages to be shared as well as reinforce principles, practices, behaviors, and attitudes related to high reliability (Appendix 1).¹¹

Finally, closing the loop is critical to the leader rounding process and to improve bidirectional communication. Closed-loop communication, following up on and/or closing out an area of discussion, not only promotes a shared understanding of information but has been found to improve patient safety.¹⁹ Effective leader rounding for high reliability includes summarizing issues and opportunities, deciding on a date for resolution for open action items, and identifying who is responsible for taking action. Senior leaders are not responsible for resolving all issues. If a team or manager of a work area can solve any issues identified, this should be encouraged and supported so accountability is maintained at the most appropriate level of the organization.

Instrumental to leader rounding for high reliability is establishing a cadence for when leaders will visit work areas.¹⁴ The most critical strategy, especially in times of change, is consistency in rounding.¹¹ At the start of implementation, we decided on a biweekly cadence. Initially leaders visited areas of the organization within their respective reporting structure. Once this was established, leaders periodically round in areas outside their scope of responsibility. This affords leaders the opportunity to observe other areas of the organization. As noted, it is important for leaders to be flexible with the rounding process especially in areas where direct patient care is being provided.

Tracking

Developing a tracking tool also is important for an effective leader rounding process. This tool is used to document issues and concerns identified during the rounding process, assign accountability, track the status of items, and close the loop when completed. One of the most commonly reported hurdles to staff sharing information to promote a culture of safety is the lack of feedback on what actions were taken to address the concern or issue raised with leadership. Closed-loop communication is critical for keeping staff continually engaged in efforts to promote a culture of safety.²⁰ We have found that a tracking tool helps to ensure that closed-loop communication takes place.

Various platforms can be used for tracking items and providing follow-up, including paper worksheets, spreadsheets, databases, or third party software (eg, SharePoint, TruthPoint Rounds, GetWell Rounds). The tracking tool should have a standardized approach for prioritizing issues.

Measuring Impact

Measuring the impact is a critical step to determine the overall effectiveness of leader rounding for high reliability. It can be as simple as requesting candid feedback from frontline staff, supervisors, managers, and service chiefs. For example, 4 months into the implementation process, the VABHS administered a brief staff survey on the overall process, perceived benefits, and challenges experienced (Appendix 2). Potential measures include the counts of leaders rounding, total rounds, rounds cancelled, and staff members actively participating in rounds. Outcomes that can be measured include issues identified, addressed, elevated, and remaining open; number of extended workdays due to rounds; staff staying overtime; and delays in patient care activities.²³ Other measures to consider are the effects of rounding on staff as well as patient/family satisfaction, increase in the number of errors and near-miss events reported per month in a health care organizations’ patient safety reporting system, and increased engagement of staff members in continuous process improvement activities. Since the inception of leader rounding for high reliability, the VABHS has seen a slight increase in the number of events entered in the patient safety reporting system. Other factors that may have contributed to this change, including encouragement of reporting at safety forums, and tiered safety huddles.

DISCUSSION

This initiative involved the development and implementation of a leader rounding for high reliability process at the VABHS with the overarching goal of ensuring efficient communication exists among members of the health care team for delivering safe, quality patient care. The initiative was well received by staff from senior leadership to frontline personnel and promoted significant interest in efforts to improve safety across the health care system.

The pilot phase permitted us to examine the feasibility and acceptability of the process to leadership as well as frontline staff. The insight gained and lessons learned through the implementation process helped us make revisions where needed and develop the tools to ensure success. In the second phase of implementation, which commenced in December 2023, each executive leadership team member began leader rounding for high reliability with their respective department service chiefs. Throughout this phase, feedback will be sought on the overall process, perceived benefits, and challenges experienced to make improvements or changes as needed. We also will continue to monitor the number of events entered in the patient safety reporting system. Future efforts will focus on developing a robust program of evaluation to explore the impact of the program on patient/family satisfaction as well as safety outcomes.

Limitations

Developing and implementing a process for leader rounding for high reliability was undertaken to support the VABHS and VHA journey to high reliability. Other health care organizations and integrated systems might identify different processes for improving patient safety and to support their journey to becoming an HRO.

CONCLUSIONS

The importance of leader rounding for high reliability to improve patient safety cannot be emphasized enough in a time where health care systems have become increasingly complex. Health care is a complex adaptive system that requires effective, bidirectional communication and collaboration among all disciplines. One of the most useful, evidence-based strategies for promoting this communication and collaboration to improve a culture of safety is leader rounding for high reliability.

The hospital is altogether the most complex human organization ever devised. Peter Drucker ¹

The ever-changing landscape of today’s increasingly complex health care system depends on implementing multifaceted, team-based methods of care delivery to provide safe, effective patient care.²Critical to establishing and sustaining exceptionally safe, effective patient care is open, transparent communication among members of interprofessional teams with senior leaders.³ However, current evidence shows thatpoor communication among interprofessional health care teams and leadership is commonplace and a significant contributing factor to inefficiencies, medical errors, and poor outcomes.⁴ One strategy for improving communication is through the implementation of leader rounding for high reliability. The concept of high reliability pertains to organizations that operate in high-risk environments for prolonged periods without serious adverse events or catastrophic failures.⁵ The overarching goal of implementation is to ensure that efficient communication exists among members of the health care team, which is essential for providing safe, quality patient care.

We describe the importance of leader rounding for high reliability as an approach to improving patient safety.Based on a review of the literature, our experiences, and lessons learned, we offer recommendations for how health care organizations on the journey to high reliability can improve patient safety.

Rounding in health care is not new. In fact, rounding has been a strong principal practice globally for more than 2 decades.⁶ During this time, varied rounding approaches have emerged, oftentimes focused on areas of interest, such as patient care, environmental services, facilities management, and discharge planning.^4,7 Variations also might involve the location of the rounds, such as a patient’s bedside, unit hallways, and conference rooms as well as the naming of rounds, such as interdisciplinary/multidisciplinary, teaching, and walkrounds.^7-10

A different type of rounding that is characteristic of high reliability organizations (HROs) is leader rounding for high reliability. The Veterans Health Administration (VHA) formally launched its journey to becoming an enterprise HRO in February 2019, using 3 cohorts. At the Veterans Affairs Bedford Healthcare System (VABHS) in Massachusetts, the journey commenced in 2021 as part of the third cohort. Leader rounding for high reliability is one of VHAs 4 HRO foundational practices for achieving a culture of safety (Figure 1).¹¹

Leader rounding for high reliability includes regularly scheduled, structured visits, with interdisciplinary teams to discuss high reliability, safety, and improvement efforts.The specific aim of these particular rounds is for senior leaders to be visible where teams are located and learn from staff (especially those on the frontlines of care) about day-to-day challenges that may contribute to patient harm.^12,13 Leader rounding for high reliability is also an important approach to improving leadership visibility across the organization, demonstrating a commitment to high reliability, and building trust and relationships with staff through open and honest dialogue. It is also an important approach to increasing leadership understanding of operational, clinical, nonclinical, patient experience issues, and concern related to safety.¹¹ This opportunity enables leaders to provide and receive real-time feedback from staff.^9,11 This experience also gives leaders an opportunity to reinforce the VHA’s 3 pillars, 5 principles, and 7 values related to high reliability (Figure 2)¹⁴ as well as to recognize behaviors that support a culture of safety.¹⁵

In preparation for implementing a leader rounding for high reliability process at the VABHS, we conducted an extensive literature review for peer-reviewed publications published between January 2015 and September 2022 regarding how other organizations implemented leader rounding. This search found a dearth of evidence as it specifically relates to leader rounding for high reliability. This motivated us to create a process for developing and implementing leader rounding for high reliability in pursuit of improving patient safety. With this objective in mind, we created and piloted a process in the fall of 2023. The first 3 months were focused on the medical center director rounding with other members of the executive leadership team to assess the feasibility and acceptability of the process. In December 2023, members of the executive leadership team began conducting leader rounding for high reliability separately. The following steps are based on the lessons we have gleaned from evolving evidence, our experiences, and developing and implementing an approach to leader rounding for high reliability.

ESTABLISH A PROCESS

Leader rounding for high reliability is performed by health care organization executive leadership, directors, managers, and supervisors. When properly conducted, increased levels of teamwork and more effective bidirectional communication take place, resulting in a united team motivated to improve patient safety.^16,17 Important early steps for implementing leader rounding for high reliability include establishing a process and getting leadership buy-in.Purposeful attention to planning is critical as is understanding the organizational factors that might deter success.Establishing a process should consider facilitators and barriers to implementation, which can include high vs low leadership turnover, structured vs unstructured rounding, and time for rounding vs competing demands.^18,19 We have learned thateffective planning is important for ensuring that leadership teams are well prepared and ambitious about leader rounding for high reliability.

Leader rounding for high reliability involves brief 10-to-15-minute interactions with interdisciplinary teams, including frontline staff. For health care organizations beginning to implement this approach, having scripts or checklists accessible might be of help. If possible, the rounds should be scheduled in advance. This helps to avoid rounding in areas at their busiest times. When possible, leader rounding for high reliability should occur as planned. Canceling rounds sends the message that leader rounding for high reliability and the valuable interactions they support are a low priority. When conflicts arise, another leader should be sent to participate. Developing a list of questions in advance helps to underscore key messages to be shared as well as reinforce principles, practices, behaviors, and attitudes related to high reliability (Appendix 1).¹¹

Finally, closing the loop is critical to the leader rounding process and to improve bidirectional communication. Closed-loop communication, following up on and/or closing out an area of discussion, not only promotes a shared understanding of information but has been found to improve patient safety.¹⁹ Effective leader rounding for high reliability includes summarizing issues and opportunities, deciding on a date for resolution for open action items, and identifying who is responsible for taking action. Senior leaders are not responsible for resolving all issues. If a team or manager of a work area can solve any issues identified, this should be encouraged and supported so accountability is maintained at the most appropriate level of the organization.

Instrumental to leader rounding for high reliability is establishing a cadence for when leaders will visit work areas.¹⁴ The most critical strategy, especially in times of change, is consistency in rounding.¹¹ At the start of implementation, we decided on a biweekly cadence. Initially leaders visited areas of the organization within their respective reporting structure. Once this was established, leaders periodically round in areas outside their scope of responsibility. This affords leaders the opportunity to observe other areas of the organization. As noted, it is important for leaders to be flexible with the rounding process especially in areas where direct patient care is being provided.

Tracking

Developing a tracking tool also is important for an effective leader rounding process. This tool is used to document issues and concerns identified during the rounding process, assign accountability, track the status of items, and close the loop when completed. One of the most commonly reported hurdles to staff sharing information to promote a culture of safety is the lack of feedback on what actions were taken to address the concern or issue raised with leadership. Closed-loop communication is critical for keeping staff continually engaged in efforts to promote a culture of safety.²⁰ We have found that a tracking tool helps to ensure that closed-loop communication takes place.

Various platforms can be used for tracking items and providing follow-up, including paper worksheets, spreadsheets, databases, or third party software (eg, SharePoint, TruthPoint Rounds, GetWell Rounds). The tracking tool should have a standardized approach for prioritizing issues.

Measuring Impact

Measuring the impact is a critical step to determine the overall effectiveness of leader rounding for high reliability. It can be as simple as requesting candid feedback from frontline staff, supervisors, managers, and service chiefs. For example, 4 months into the implementation process, the VABHS administered a brief staff survey on the overall process, perceived benefits, and challenges experienced (Appendix 2). Potential measures include the counts of leaders rounding, total rounds, rounds cancelled, and staff members actively participating in rounds. Outcomes that can be measured include issues identified, addressed, elevated, and remaining open; number of extended workdays due to rounds; staff staying overtime; and delays in patient care activities.²³ Other measures to consider are the effects of rounding on staff as well as patient/family satisfaction, increase in the number of errors and near-miss events reported per month in a health care organizations’ patient safety reporting system, and increased engagement of staff members in continuous process improvement activities. Since the inception of leader rounding for high reliability, the VABHS has seen a slight increase in the number of events entered in the patient safety reporting system. Other factors that may have contributed to this change, including encouragement of reporting at safety forums, and tiered safety huddles.

DISCUSSION

This initiative involved the development and implementation of a leader rounding for high reliability process at the VABHS with the overarching goal of ensuring efficient communication exists among members of the health care team for delivering safe, quality patient care. The initiative was well received by staff from senior leadership to frontline personnel and promoted significant interest in efforts to improve safety across the health care system.

The pilot phase permitted us to examine the feasibility and acceptability of the process to leadership as well as frontline staff. The insight gained and lessons learned through the implementation process helped us make revisions where needed and develop the tools to ensure success. In the second phase of implementation, which commenced in December 2023, each executive leadership team member began leader rounding for high reliability with their respective department service chiefs. Throughout this phase, feedback will be sought on the overall process, perceived benefits, and challenges experienced to make improvements or changes as needed. We also will continue to monitor the number of events entered in the patient safety reporting system. Future efforts will focus on developing a robust program of evaluation to explore the impact of the program on patient/family satisfaction as well as safety outcomes.

Limitations

Developing and implementing a process for leader rounding for high reliability was undertaken to support the VABHS and VHA journey to high reliability. Other health care organizations and integrated systems might identify different processes for improving patient safety and to support their journey to becoming an HRO.

CONCLUSIONS

The importance of leader rounding for high reliability to improve patient safety cannot be emphasized enough in a time where health care systems have become increasingly complex. Health care is a complex adaptive system that requires effective, bidirectional communication and collaboration among all disciplines. One of the most useful, evidence-based strategies for promoting this communication and collaboration to improve a culture of safety is leader rounding for high reliability.

The hospital is altogether the most complex human organization ever devised. Peter Drucker ¹

The ever-changing landscape of today’s increasingly complex health care system depends on implementing multifaceted, team-based methods of care delivery to provide safe, effective patient care.²Critical to establishing and sustaining exceptionally safe, effective patient care is open, transparent communication among members of interprofessional teams with senior leaders.³ However, current evidence shows thatpoor communication among interprofessional health care teams and leadership is commonplace and a significant contributing factor to inefficiencies, medical errors, and poor outcomes.⁴ One strategy for improving communication is through the implementation of leader rounding for high reliability. The concept of high reliability pertains to organizations that operate in high-risk environments for prolonged periods without serious adverse events or catastrophic failures.⁵ The overarching goal of implementation is to ensure that efficient communication exists among members of the health care team, which is essential for providing safe, quality patient care.

We describe the importance of leader rounding for high reliability as an approach to improving patient safety.Based on a review of the literature, our experiences, and lessons learned, we offer recommendations for how health care organizations on the journey to high reliability can improve patient safety.

Rounding in health care is not new. In fact, rounding has been a strong principal practice globally for more than 2 decades.⁶ During this time, varied rounding approaches have emerged, oftentimes focused on areas of interest, such as patient care, environmental services, facilities management, and discharge planning.^4,7 Variations also might involve the location of the rounds, such as a patient’s bedside, unit hallways, and conference rooms as well as the naming of rounds, such as interdisciplinary/multidisciplinary, teaching, and walkrounds.^7-10

A different type of rounding that is characteristic of high reliability organizations (HROs) is leader rounding for high reliability. The Veterans Health Administration (VHA) formally launched its journey to becoming an enterprise HRO in February 2019, using 3 cohorts. At the Veterans Affairs Bedford Healthcare System (VABHS) in Massachusetts, the journey commenced in 2021 as part of the third cohort. Leader rounding for high reliability is one of VHAs 4 HRO foundational practices for achieving a culture of safety (Figure 1).¹¹

Leader rounding for high reliability includes regularly scheduled, structured visits, with interdisciplinary teams to discuss high reliability, safety, and improvement efforts.The specific aim of these particular rounds is for senior leaders to be visible where teams are located and learn from staff (especially those on the frontlines of care) about day-to-day challenges that may contribute to patient harm.^12,13 Leader rounding for high reliability is also an important approach to improving leadership visibility across the organization, demonstrating a commitment to high reliability, and building trust and relationships with staff through open and honest dialogue. It is also an important approach to increasing leadership understanding of operational, clinical, nonclinical, patient experience issues, and concern related to safety.¹¹ This opportunity enables leaders to provide and receive real-time feedback from staff.^9,11 This experience also gives leaders an opportunity to reinforce the VHA’s 3 pillars, 5 principles, and 7 values related to high reliability (Figure 2)¹⁴ as well as to recognize behaviors that support a culture of safety.¹⁵

In preparation for implementing a leader rounding for high reliability process at the VABHS, we conducted an extensive literature review for peer-reviewed publications published between January 2015 and September 2022 regarding how other organizations implemented leader rounding. This search found a dearth of evidence as it specifically relates to leader rounding for high reliability. This motivated us to create a process for developing and implementing leader rounding for high reliability in pursuit of improving patient safety. With this objective in mind, we created and piloted a process in the fall of 2023. The first 3 months were focused on the medical center director rounding with other members of the executive leadership team to assess the feasibility and acceptability of the process. In December 2023, members of the executive leadership team began conducting leader rounding for high reliability separately. The following steps are based on the lessons we have gleaned from evolving evidence, our experiences, and developing and implementing an approach to leader rounding for high reliability.

ESTABLISH A PROCESS

Leader rounding for high reliability is performed by health care organization executive leadership, directors, managers, and supervisors. When properly conducted, increased levels of teamwork and more effective bidirectional communication take place, resulting in a united team motivated to improve patient safety.^16,17 Important early steps for implementing leader rounding for high reliability include establishing a process and getting leadership buy-in.Purposeful attention to planning is critical as is understanding the organizational factors that might deter success.Establishing a process should consider facilitators and barriers to implementation, which can include high vs low leadership turnover, structured vs unstructured rounding, and time for rounding vs competing demands.^18,19 We have learned thateffective planning is important for ensuring that leadership teams are well prepared and ambitious about leader rounding for high reliability.

Leader rounding for high reliability involves brief 10-to-15-minute interactions with interdisciplinary teams, including frontline staff. For health care organizations beginning to implement this approach, having scripts or checklists accessible might be of help. If possible, the rounds should be scheduled in advance. This helps to avoid rounding in areas at their busiest times. When possible, leader rounding for high reliability should occur as planned. Canceling rounds sends the message that leader rounding for high reliability and the valuable interactions they support are a low priority. When conflicts arise, another leader should be sent to participate. Developing a list of questions in advance helps to underscore key messages to be shared as well as reinforce principles, practices, behaviors, and attitudes related to high reliability (Appendix 1).¹¹

Finally, closing the loop is critical to the leader rounding process and to improve bidirectional communication. Closed-loop communication, following up on and/or closing out an area of discussion, not only promotes a shared understanding of information but has been found to improve patient safety.¹⁹ Effective leader rounding for high reliability includes summarizing issues and opportunities, deciding on a date for resolution for open action items, and identifying who is responsible for taking action. Senior leaders are not responsible for resolving all issues. If a team or manager of a work area can solve any issues identified, this should be encouraged and supported so accountability is maintained at the most appropriate level of the organization.

Instrumental to leader rounding for high reliability is establishing a cadence for when leaders will visit work areas.¹⁴ The most critical strategy, especially in times of change, is consistency in rounding.¹¹ At the start of implementation, we decided on a biweekly cadence. Initially leaders visited areas of the organization within their respective reporting structure. Once this was established, leaders periodically round in areas outside their scope of responsibility. This affords leaders the opportunity to observe other areas of the organization. As noted, it is important for leaders to be flexible with the rounding process especially in areas where direct patient care is being provided.

Tracking

Developing a tracking tool also is important for an effective leader rounding process. This tool is used to document issues and concerns identified during the rounding process, assign accountability, track the status of items, and close the loop when completed. One of the most commonly reported hurdles to staff sharing information to promote a culture of safety is the lack of feedback on what actions were taken to address the concern or issue raised with leadership. Closed-loop communication is critical for keeping staff continually engaged in efforts to promote a culture of safety.²⁰ We have found that a tracking tool helps to ensure that closed-loop communication takes place.

Various platforms can be used for tracking items and providing follow-up, including paper worksheets, spreadsheets, databases, or third party software (eg, SharePoint, TruthPoint Rounds, GetWell Rounds). The tracking tool should have a standardized approach for prioritizing issues.

Measuring Impact

Measuring the impact is a critical step to determine the overall effectiveness of leader rounding for high reliability. It can be as simple as requesting candid feedback from frontline staff, supervisors, managers, and service chiefs. For example, 4 months into the implementation process, the VABHS administered a brief staff survey on the overall process, perceived benefits, and challenges experienced (Appendix 2). Potential measures include the counts of leaders rounding, total rounds, rounds cancelled, and staff members actively participating in rounds. Outcomes that can be measured include issues identified, addressed, elevated, and remaining open; number of extended workdays due to rounds; staff staying overtime; and delays in patient care activities.²³ Other measures to consider are the effects of rounding on staff as well as patient/family satisfaction, increase in the number of errors and near-miss events reported per month in a health care organizations’ patient safety reporting system, and increased engagement of staff members in continuous process improvement activities. Since the inception of leader rounding for high reliability, the VABHS has seen a slight increase in the number of events entered in the patient safety reporting system. Other factors that may have contributed to this change, including encouragement of reporting at safety forums, and tiered safety huddles.

DISCUSSION

This initiative involved the development and implementation of a leader rounding for high reliability process at the VABHS with the overarching goal of ensuring efficient communication exists among members of the health care team for delivering safe, quality patient care. The initiative was well received by staff from senior leadership to frontline personnel and promoted significant interest in efforts to improve safety across the health care system.

The pilot phase permitted us to examine the feasibility and acceptability of the process to leadership as well as frontline staff. The insight gained and lessons learned through the implementation process helped us make revisions where needed and develop the tools to ensure success. In the second phase of implementation, which commenced in December 2023, each executive leadership team member began leader rounding for high reliability with their respective department service chiefs. Throughout this phase, feedback will be sought on the overall process, perceived benefits, and challenges experienced to make improvements or changes as needed. We also will continue to monitor the number of events entered in the patient safety reporting system. Future efforts will focus on developing a robust program of evaluation to explore the impact of the program on patient/family satisfaction as well as safety outcomes.

Limitations

Developing and implementing a process for leader rounding for high reliability was undertaken to support the VABHS and VHA journey to high reliability. Other health care organizations and integrated systems might identify different processes for improving patient safety and to support their journey to becoming an HRO.

CONCLUSIONS

The importance of leader rounding for high reliability to improve patient safety cannot be emphasized enough in a time where health care systems have become increasingly complex. Health care is a complex adaptive system that requires effective, bidirectional communication and collaboration among all disciplines. One of the most useful, evidence-based strategies for promoting this communication and collaboration to improve a culture of safety is leader rounding for high reliability.

The Central Texas Veteran’s Health Care System (CTVHCS) in Temple, Texas, is a 189-bed teaching hospital. CTVHCS opened the Center for Innovation and Learning (CIL) in 2022. The CIL has about 279 m² of simulation space that includes high- and low-fidelity simulation equipment and multiple laboratories, which can be used to simulate inpatient and outpatient settings. The CIL high-fidelity manikins and environment allow learners to be immersed in the simulation for maximum realism. Computer and video systems provide clear viewing of training, which allows for more in-depth debriefing and learning. CIL simulation training is used by CTVHCS staff, medical residents, and medical and physician assistant students.

The utility of technology in medical education is rapidly evolving. As noted in many studies, simulation creates an environment that can imitate real patients in the format of a lifelike manikin, anatomic regions stations, clinical tasks, and many real-life circumstances.¹ Task trainers for procedure simulation have been widely used and studied. A 2020 study noted that simulation training is effective for developing procedural skills in surgery and prevents the decay of surgical skills.²

In reviewing health care education curriculums, we noted that most of the rapid response situations are learned through active patient experiences. Rapid responses are managed by the intensive care unit and primary care teams during the day but at night are run primarily by the postgraduate year 2 (PGY2) night resident and intern. Knowing these logistics and current studies, we decided to build a rapid response simulation curriculum to improve preparedness for PGY1 residents, medical students, and physician assistant students.

Curriculum Planning

Planning the simulation curriculum began with the CTVHCS internal medicine chief resident and registered nurse (RN) educator. CTVHCS data were reviewed to identify the 3 most common rapid response calls from the past 3 years; research on the most common systems affected by rapid responses also was evaluated.

A 2019 study by Lyons and colleagues evaluated 402,023 rapid response activations across 360 hospitals and found that respiratory scenarios made up 38% and cardiac scenarios made up 37%.³ In addition, the CTVHCS has limited support in stroke neurology. Therefore, the internal medicine chief resident and RN educator decided to run 3 evolving rapid response scenarios per session that included cardiac, respiratory, and neurological scenarios. Capabilities and limitations of different high-fidelity manikins were discussed to identify and use the most appropriate simulator for each situation. Objectives that met both general medicine and site-specific education were discussed, and the program was formulated.

Program Description

Nightmare on CIL Street is a simulation-based program designed for new internal medicine residents and students to encounter difficult situations (late at night, on call, or when resources are limited; ie, weekends/holidays) in a controlled simulation environment. During the simulation, learners will be unable to transfer the patient and no additional help is available. Each learner must determine a differential diagnosis and make appropriate medical interventions with only the assistance of a nurse. Scenarios are derived from common rapid response team calls and low-volume/high-impact situations where clinical decisions must be made quickly to ensure the best patient outcomes. High-fidelity manikins that have abilities to respond to questions, simulate breathing, reproduce pathological heart and breath sounds and more are used to create a realistic patient environment.

This program aligns with 2 national Veterans Health Administration priorities: (1) connect veterans to the soonest and best care; and (2) accelerate the Veterans Health Administration journey to be a high-reliability organization (sensitivity to operations, preoccupation with failure, commitment to resilience, and deference to expertise). Nightmare on CIL Street has 3 clinical episodes: 2 cardiac (A Tell-Tale Heart), respiratory (Don’t Breathe), and neurologic (Brain Scan). Additional clinical episodes will be added based on learner feedback and assessed need.

Each simulation event encompassed all 3 episodes that an individual or a team of 2 learners rotate through in a round-robin fashion. The overarching theme for each episode was a rapid response team call with minimal resources that the learner would have to provide care and stabilization. A literature search for rapid response team training programs found few results, but the literature assisted with providing a foundation for Nightmare on CIL Street.^4,5 The goal was to completely envelop the learners in a nightmare scenario that required a solution.

After the safety brief and predata collection, learners received a phone call with minimal information about a patient in need of care. The learners responded to the requested area and provided treatment to the emergency over 25 minutes with the bedside nurse (who is an embedded participant). At the conclusion of the scenario, a physician subject matter expert who has been observing, provided a personalized 10-minute debriefing to the learner, which presented specific learning points and opportunities for the learner’s educational development. After the debriefing, learners returned to a conference room and awaited the next call. After all learners completed the 3 episodes, a group debriefing was conducted using the gather, analyze, summarize debriefing framework. The debriefing begins with an open-ended forum for learners to express their thoughts. Then, each scenario is discussed and broken down by key learning objectives. Starting with cardiac and ending with neurology, the logistics of the cases are discussed based on the trajectory of the learners during the scenarios. Each objective is discussed, and learners are allowed to ask questions before moving to the next scenario. After the debriefing, postevent data were gathered.

Objectives

The program objective was to educate residents and students on common rapid response scenarios. We devised each scenario as an evolving simulation where various interventions would improve or worsen vital signs and symptoms. Each scenario had an end goal: cardioversion (cardiac), intubation (respiratory), and transfer (neurologic). Objectives were tailored to the trainees present during the specific simulation (Table).

IMPLEMENTATION

The initial run of the simulation curriculum was implemented on February 22, 2023, and ended on May 17, 2023, with 5 events. Participants included internal medicine PGY1 residents, third-year medical students, and fourth-year physician assistant students. Internal medicine residents ran each scenario with a subject matter expert monitoring; the undergraduate medical trainees partnered with another student. Students were pulled from their ward rotations to attend the simulation, and residents were pulled from electives and wards. Each trainee was able to experience each planned scenario. They were then briefed, participated in each scenario, and ended with a debriefing, discussing each case in detail. Two subject matter experts were always available, and occasionally 4 were present to provide additional knowledge transfer to learners. These included board-certified physicians in internal medicine and pulmonary critical care. Most scenarios were conducted on Wednesday afternoon or Thursday.

The CIL provided 6 staff minimum for every event. The staff controlled the manikins and acted as embedded players for the learners to interact and work with at the bedside. Every embedded RN was provided the same script: They were a new nurse just off orientation and did not know what to do. In addition, they were instructed that no matter who the learner wanted to call/page, that person or service was not answering or unavailable. This forced learners to respond and treat the simulated patient on their own.

Survey Responses

To evaluate the effect of this program on medical education, we administered surveys to the trainees before and after the simulation (Appendix). All questions were evaluated on a 10-point Likert scale (1, minimal comfort; 10, maximum comfort). The postsurvey added an additional Likert scale question and an open-ended question.

Sixteen trainees underwent the simulation curriculum during the 2022 to 2023 academic year, 9 internal medicine PGY1 residents, 4 medical students, and 3 physician assistant students. Postsimulation surveys indicated a mean 2.2 point increase in comfort compared with the presimulation surveys across all questions and participants.

DISCUSSION

The simulation curriculum proved to be successful for all parties, including trainees, medical educators, and simulation staff. Trainees expressed gratitude for the teaching ability of the simulation and the challenge of confronting an evolving scenario. Students also stated that the simulation allowed them to identify knowledge weaknesses.

Medical technology is rapidly advancing. A study evaluating high-fidelity medical simulations between 1969 and 2003 found that they are educationally effective and complement other medical education modalities.⁶ It is also noted that care provided by junior physicians with a lack of prior exposure to emergencies and unusual clinical syndromes can lead to more adverse effects.⁷ Simulation curriculums can be used to educate junior physicians as well as trainees on a multitude of medical emergencies, teach systematic approaches to medical scenarios, and increase exposure to unfamiliar experiences.

The goals of this article are to share program details and encourage other training programs with similar capabilities to incorporate simulation into medical education. Using pre- and postsimulation surveys, there was a concrete improvement in the value obtained by participating in this simulation. The Nightmare on CIL Street learners experienced a mean 2.2 point improvement from presimulation survey to postsimulation survey. Some notable improvements were the feelings of preparedness for rapid response situations and developing a systematic approach. As the students who participated in our Nightmare on CIL Street simulation were early in training, we believe the improvement in preparation and developing a systematic approach can be key to their success in their practical environments.

From a site-specific standpoint, improvement in confidence working through cardiac, respiratory, and neurological emergencies will be very useful. The anesthesiology service intubates during respiratory failures and there is no stroke neurologist available at the CTVHCS hospital. Giving trainees experience in these conditions may allow them to better understand their role in coordination during these times and potentially improve patient outcomes. A follow-up questionnaire administered a year after this simulation may be useful in ascertaining the usefulness of the simulation and what items may have been approached differently. We encourage other institutions to build in aspects of their site-specific challenges to improve trainee awareness in approaches to critical scenarios.

Challenges

The greatest challenge for Nightmare on CIL Street was the ability to pull internal medicine residents from their clinical duties to participate in the simulation. As there are many moving parts to their clinical scheduling, residents do not always have sufficient coverage to participate in training. There were also instances where residents needed to cover for another resident preventing them from attending the simulation. In the future, this program will schedule residents months in advance and will have the simulation training built into their rotations.

Medical and physician assistant students were pulled from their ward rotations as well. They rotate on a 2-to-4-week basis and often had already experienced the simulation the week prior, leaving out students for the following week. With more longitudinal planning, students can be pulled on a rotating monthly basis to maximize their participation. Another challenge was deciding whether residents should partner or experience the simulation on their own. After some feedback, it was noted that residents preferred to experience the simulation on their own as this improves their learning value. With the limited resources available, only rotating 3 residents on a scenario limits the number of trainees who can be reached with the program. Running this program throughout an academic year can help to reach more trainees.

CONCLUSIONS

Educating trainees on rapid response scenarios by using a simulation curriculum provides many benefits. Our trainees reported improvement in addressing cardiac, respiratory, and neurological rapid response scenarios after experiencing the simulation. They felt better prepared and had developed a better systematic approach for the future.

Acknowledgments

The authors thank Pawan Sikka, MD, George Martinez, MD and Braden Anderson, MD for participating as physician experts and educating our students. We thank Naomi Devers; Dinetra Jones; Stephanie Garrett; Sara Holton; Evelina Bartnick; Tanelle Smith; Michael Lomax; Shaun Kelemen for their participation as nurses, assistants, and simulation technology experts.

The Central Texas Veteran’s Health Care System (CTVHCS) in Temple, Texas, is a 189-bed teaching hospital. CTVHCS opened the Center for Innovation and Learning (CIL) in 2022. The CIL has about 279 m² of simulation space that includes high- and low-fidelity simulation equipment and multiple laboratories, which can be used to simulate inpatient and outpatient settings. The CIL high-fidelity manikins and environment allow learners to be immersed in the simulation for maximum realism. Computer and video systems provide clear viewing of training, which allows for more in-depth debriefing and learning. CIL simulation training is used by CTVHCS staff, medical residents, and medical and physician assistant students.

The utility of technology in medical education is rapidly evolving. As noted in many studies, simulation creates an environment that can imitate real patients in the format of a lifelike manikin, anatomic regions stations, clinical tasks, and many real-life circumstances.¹ Task trainers for procedure simulation have been widely used and studied. A 2020 study noted that simulation training is effective for developing procedural skills in surgery and prevents the decay of surgical skills.²

In reviewing health care education curriculums, we noted that most of the rapid response situations are learned through active patient experiences. Rapid responses are managed by the intensive care unit and primary care teams during the day but at night are run primarily by the postgraduate year 2 (PGY2) night resident and intern. Knowing these logistics and current studies, we decided to build a rapid response simulation curriculum to improve preparedness for PGY1 residents, medical students, and physician assistant students.

Curriculum Planning

Planning the simulation curriculum began with the CTVHCS internal medicine chief resident and registered nurse (RN) educator. CTVHCS data were reviewed to identify the 3 most common rapid response calls from the past 3 years; research on the most common systems affected by rapid responses also was evaluated.

A 2019 study by Lyons and colleagues evaluated 402,023 rapid response activations across 360 hospitals and found that respiratory scenarios made up 38% and cardiac scenarios made up 37%.³ In addition, the CTVHCS has limited support in stroke neurology. Therefore, the internal medicine chief resident and RN educator decided to run 3 evolving rapid response scenarios per session that included cardiac, respiratory, and neurological scenarios. Capabilities and limitations of different high-fidelity manikins were discussed to identify and use the most appropriate simulator for each situation. Objectives that met both general medicine and site-specific education were discussed, and the program was formulated.

Program Description

Nightmare on CIL Street is a simulation-based program designed for new internal medicine residents and students to encounter difficult situations (late at night, on call, or when resources are limited; ie, weekends/holidays) in a controlled simulation environment. During the simulation, learners will be unable to transfer the patient and no additional help is available. Each learner must determine a differential diagnosis and make appropriate medical interventions with only the assistance of a nurse. Scenarios are derived from common rapid response team calls and low-volume/high-impact situations where clinical decisions must be made quickly to ensure the best patient outcomes. High-fidelity manikins that have abilities to respond to questions, simulate breathing, reproduce pathological heart and breath sounds and more are used to create a realistic patient environment.

This program aligns with 2 national Veterans Health Administration priorities: (1) connect veterans to the soonest and best care; and (2) accelerate the Veterans Health Administration journey to be a high-reliability organization (sensitivity to operations, preoccupation with failure, commitment to resilience, and deference to expertise). Nightmare on CIL Street has 3 clinical episodes: 2 cardiac (A Tell-Tale Heart), respiratory (Don’t Breathe), and neurologic (Brain Scan). Additional clinical episodes will be added based on learner feedback and assessed need.

Each simulation event encompassed all 3 episodes that an individual or a team of 2 learners rotate through in a round-robin fashion. The overarching theme for each episode was a rapid response team call with minimal resources that the learner would have to provide care and stabilization. A literature search for rapid response team training programs found few results, but the literature assisted with providing a foundation for Nightmare on CIL Street.^4,5 The goal was to completely envelop the learners in a nightmare scenario that required a solution.

After the safety brief and predata collection, learners received a phone call with minimal information about a patient in need of care. The learners responded to the requested area and provided treatment to the emergency over 25 minutes with the bedside nurse (who is an embedded participant). At the conclusion of the scenario, a physician subject matter expert who has been observing, provided a personalized 10-minute debriefing to the learner, which presented specific learning points and opportunities for the learner’s educational development. After the debriefing, learners returned to a conference room and awaited the next call. After all learners completed the 3 episodes, a group debriefing was conducted using the gather, analyze, summarize debriefing framework. The debriefing begins with an open-ended forum for learners to express their thoughts. Then, each scenario is discussed and broken down by key learning objectives. Starting with cardiac and ending with neurology, the logistics of the cases are discussed based on the trajectory of the learners during the scenarios. Each objective is discussed, and learners are allowed to ask questions before moving to the next scenario. After the debriefing, postevent data were gathered.

Objectives

The program objective was to educate residents and students on common rapid response scenarios. We devised each scenario as an evolving simulation where various interventions would improve or worsen vital signs and symptoms. Each scenario had an end goal: cardioversion (cardiac), intubation (respiratory), and transfer (neurologic). Objectives were tailored to the trainees present during the specific simulation (Table).

IMPLEMENTATION

The initial run of the simulation curriculum was implemented on February 22, 2023, and ended on May 17, 2023, with 5 events. Participants included internal medicine PGY1 residents, third-year medical students, and fourth-year physician assistant students. Internal medicine residents ran each scenario with a subject matter expert monitoring; the undergraduate medical trainees partnered with another student. Students were pulled from their ward rotations to attend the simulation, and residents were pulled from electives and wards. Each trainee was able to experience each planned scenario. They were then briefed, participated in each scenario, and ended with a debriefing, discussing each case in detail. Two subject matter experts were always available, and occasionally 4 were present to provide additional knowledge transfer to learners. These included board-certified physicians in internal medicine and pulmonary critical care. Most scenarios were conducted on Wednesday afternoon or Thursday.

The CIL provided 6 staff minimum for every event. The staff controlled the manikins and acted as embedded players for the learners to interact and work with at the bedside. Every embedded RN was provided the same script: They were a new nurse just off orientation and did not know what to do. In addition, they were instructed that no matter who the learner wanted to call/page, that person or service was not answering or unavailable. This forced learners to respond and treat the simulated patient on their own.

Survey Responses