Original Research

Background

The original program consisted of 12 movements that were to be split up between 3 weeks teaching 4 movements each week. Range of mobility was the main consideration for developing this HPE quality improvement project. Veterans who wanted to participate in Tai Chi were not able to engage in the activity due to the range of movement traditional Tai Chi required.

Innovation

The HPE Quality Improvement program developed a 15-movement warm-up, 12 co-ordinational movements consistent with the original program, 18 supplemental Tai Chi movements that were not included in the original program all of which focus on movements remaining below the shoulders and can be done standing or sitting. Four advanced exercises including “hip over heel” were included to target participants balance if able and to improve their hip strength, knee tendon/ligament strength. Tai Chi loses its potential to increase balance when performed in a sitting position.¹ The movements drew upon Fu style Tai Chi and the program developer was given permission from Tommy Kirchoff to use his DVD Healing Exercises. The HPE program consisted of four 30–60-minute weekly sessions of learning the movements with another 4 weekly sessions of demonstrating the movements. Instructors were given written and visual documents to learn from and were evaluated by the developer during the last 4 weeks.
.

Results

Qualitative Data: Instructors notice a difference in how they feel, and appreciate having another option to offer veterans with mobility/standing issues. Patients expressed improvement in mobility relating to bending, arm extension, arm raising, muscle strengthening, hip strengthening and rotation.

Discussion

Future research will want to look at taking measurements before and after patient implementation to determine quantitative data related to balance, strength and range of movement including grip strength, stand up and go, and one-legged stands.

Background

The original program consisted of 12 movements that were to be split up between 3 weeks teaching 4 movements each week. Range of mobility was the main consideration for developing this HPE quality improvement project. Veterans who wanted to participate in Tai Chi were not able to engage in the activity due to the range of movement traditional Tai Chi required.

Innovation

The HPE Quality Improvement program developed a 15-movement warm-up, 12 co-ordinational movements consistent with the original program, 18 supplemental Tai Chi movements that were not included in the original program all of which focus on movements remaining below the shoulders and can be done standing or sitting. Four advanced exercises including “hip over heel” were included to target participants balance if able and to improve their hip strength, knee tendon/ligament strength. Tai Chi loses its potential to increase balance when performed in a sitting position.¹ The movements drew upon Fu style Tai Chi and the program developer was given permission from Tommy Kirchoff to use his DVD Healing Exercises. The HPE program consisted of four 30–60-minute weekly sessions of learning the movements with another 4 weekly sessions of demonstrating the movements. Instructors were given written and visual documents to learn from and were evaluated by the developer during the last 4 weeks.
.

Results

Qualitative Data: Instructors notice a difference in how they feel, and appreciate having another option to offer veterans with mobility/standing issues. Patients expressed improvement in mobility relating to bending, arm extension, arm raising, muscle strengthening, hip strengthening and rotation.

Discussion

Future research will want to look at taking measurements before and after patient implementation to determine quantitative data related to balance, strength and range of movement including grip strength, stand up and go, and one-legged stands.

Background

The original program consisted of 12 movements that were to be split up between 3 weeks teaching 4 movements each week. Range of mobility was the main consideration for developing this HPE quality improvement project. Veterans who wanted to participate in Tai Chi were not able to engage in the activity due to the range of movement traditional Tai Chi required.

Innovation

The HPE Quality Improvement program developed a 15-movement warm-up, 12 co-ordinational movements consistent with the original program, 18 supplemental Tai Chi movements that were not included in the original program all of which focus on movements remaining below the shoulders and can be done standing or sitting. Four advanced exercises including “hip over heel” were included to target participants balance if able and to improve their hip strength, knee tendon/ligament strength. Tai Chi loses its potential to increase balance when performed in a sitting position.¹ The movements drew upon Fu style Tai Chi and the program developer was given permission from Tommy Kirchoff to use his DVD Healing Exercises. The HPE program consisted of four 30–60-minute weekly sessions of learning the movements with another 4 weekly sessions of demonstrating the movements. Instructors were given written and visual documents to learn from and were evaluated by the developer during the last 4 weeks.
.

Results

Qualitative Data: Instructors notice a difference in how they feel, and appreciate having another option to offer veterans with mobility/standing issues. Patients expressed improvement in mobility relating to bending, arm extension, arm raising, muscle strengthening, hip strengthening and rotation.

Discussion

Future research will want to look at taking measurements before and after patient implementation to determine quantitative data related to balance, strength and range of movement including grip strength, stand up and go, and one-legged stands.

Background

Veterans Health Administration Directive 1004.03(1) (Advance Care Planning) aims to establish a “system-wide, patient-centered and evidence-based approach to Advance Care Planning.”¹ Life-sustaining treatment (LST) orders are documents of patient preference regarding interventions such as mechanical ventilation, CPR, dialysis, artificial nutrition and hydration; and are considered part of an Advance Care Plan. From a bioethics perspective, these orders promote patient autonomy by formalizing patient preferences around LSTs in the medical record, particularly for when a patient lacks capacity and/or cannot make decisions on their own.² Through consensus building, our team defined vague, inactionable, or incorrectly written LST orders as Potentially Problematic Orders (PPO). PPOs which cause confusion at the bedside or lack clarity around preferences can pose serious risks to patient safety and autonomy by exposing patients to inappropriate initiation or withholding of LSTs. Improving the quality of LST orders and reducing the number of PPOs is a crucial element for safe and effective implementation of Directive 1004.03(1).

Aim

The aim of this quality improvement project was to reduce the number of PPOs in a VA Community-Based Outpatient Clinic (CBOC) by 75% by the end of 2025.

Methods

The Model for Improvement was used for this quality improvement project.³ One year of LST orders were audited and thematic analysis identified 7 subtypes of PPO. Some PPO subtypes included clerical errors, potentially mismatched order sets (e.g., Comfort Care order with no associated DNR order) ill-defined or vague orders, and clinically impractical orders (eg, “consents to one shock during CPR”). We defined vague, ill-defined, and impractical orders as the most ethically and clinically challenging given the possibility of confusion or error at the bedside. Initial data were collected from October 2022 to October 2023, and post-intervention data were collected from February 2024 to September 2024. Interventions included process changes (clarifying role responsibility, documentation practices, patient education), regular auditing and feedback from a supervisor, and staff education.

Results

Post-intervention analysis demonstrated that the proportion of PPO remained the same, with 25% of patient charts containing at least one PPO. However, the distribution of PPO in the most ethically and clinically problematic categories (vague, ill-defined, and impractical orders) decreased from 14.7% to <1%.

Conclusions

We successfully reduced the most ethically and clinically challenging PPOs to <1% in our initial intervention. To reduce the overall proportion of PPO, we plan enhancements in process automations, additional physical educational resources, and minor changes in audit criteria. Future projects will aim to address the remaining PPO error types and prepare this project for implementation in other CBOCs.

Background

Veterans Health Administration Directive 1004.03(1) (Advance Care Planning) aims to establish a “system-wide, patient-centered and evidence-based approach to Advance Care Planning.”¹ Life-sustaining treatment (LST) orders are documents of patient preference regarding interventions such as mechanical ventilation, CPR, dialysis, artificial nutrition and hydration; and are considered part of an Advance Care Plan. From a bioethics perspective, these orders promote patient autonomy by formalizing patient preferences around LSTs in the medical record, particularly for when a patient lacks capacity and/or cannot make decisions on their own.² Through consensus building, our team defined vague, inactionable, or incorrectly written LST orders as Potentially Problematic Orders (PPO). PPOs which cause confusion at the bedside or lack clarity around preferences can pose serious risks to patient safety and autonomy by exposing patients to inappropriate initiation or withholding of LSTs. Improving the quality of LST orders and reducing the number of PPOs is a crucial element for safe and effective implementation of Directive 1004.03(1).

Aim

The aim of this quality improvement project was to reduce the number of PPOs in a VA Community-Based Outpatient Clinic (CBOC) by 75% by the end of 2025.

Methods

The Model for Improvement was used for this quality improvement project.³ One year of LST orders were audited and thematic analysis identified 7 subtypes of PPO. Some PPO subtypes included clerical errors, potentially mismatched order sets (e.g., Comfort Care order with no associated DNR order) ill-defined or vague orders, and clinically impractical orders (eg, “consents to one shock during CPR”). We defined vague, ill-defined, and impractical orders as the most ethically and clinically challenging given the possibility of confusion or error at the bedside. Initial data were collected from October 2022 to October 2023, and post-intervention data were collected from February 2024 to September 2024. Interventions included process changes (clarifying role responsibility, documentation practices, patient education), regular auditing and feedback from a supervisor, and staff education.

Results

Post-intervention analysis demonstrated that the proportion of PPO remained the same, with 25% of patient charts containing at least one PPO. However, the distribution of PPO in the most ethically and clinically problematic categories (vague, ill-defined, and impractical orders) decreased from 14.7% to <1%.

Conclusions

We successfully reduced the most ethically and clinically challenging PPOs to <1% in our initial intervention. To reduce the overall proportion of PPO, we plan enhancements in process automations, additional physical educational resources, and minor changes in audit criteria. Future projects will aim to address the remaining PPO error types and prepare this project for implementation in other CBOCs.

Background

Veterans Health Administration Directive 1004.03(1) (Advance Care Planning) aims to establish a “system-wide, patient-centered and evidence-based approach to Advance Care Planning.”¹ Life-sustaining treatment (LST) orders are documents of patient preference regarding interventions such as mechanical ventilation, CPR, dialysis, artificial nutrition and hydration; and are considered part of an Advance Care Plan. From a bioethics perspective, these orders promote patient autonomy by formalizing patient preferences around LSTs in the medical record, particularly for when a patient lacks capacity and/or cannot make decisions on their own.² Through consensus building, our team defined vague, inactionable, or incorrectly written LST orders as Potentially Problematic Orders (PPO). PPOs which cause confusion at the bedside or lack clarity around preferences can pose serious risks to patient safety and autonomy by exposing patients to inappropriate initiation or withholding of LSTs. Improving the quality of LST orders and reducing the number of PPOs is a crucial element for safe and effective implementation of Directive 1004.03(1).

Aim

The aim of this quality improvement project was to reduce the number of PPOs in a VA Community-Based Outpatient Clinic (CBOC) by 75% by the end of 2025.

Methods

The Model for Improvement was used for this quality improvement project.³ One year of LST orders were audited and thematic analysis identified 7 subtypes of PPO. Some PPO subtypes included clerical errors, potentially mismatched order sets (e.g., Comfort Care order with no associated DNR order) ill-defined or vague orders, and clinically impractical orders (eg, “consents to one shock during CPR”). We defined vague, ill-defined, and impractical orders as the most ethically and clinically challenging given the possibility of confusion or error at the bedside. Initial data were collected from October 2022 to October 2023, and post-intervention data were collected from February 2024 to September 2024. Interventions included process changes (clarifying role responsibility, documentation practices, patient education), regular auditing and feedback from a supervisor, and staff education.

Results

Post-intervention analysis demonstrated that the proportion of PPO remained the same, with 25% of patient charts containing at least one PPO. However, the distribution of PPO in the most ethically and clinically problematic categories (vague, ill-defined, and impractical orders) decreased from 14.7% to <1%.

Conclusions

We successfully reduced the most ethically and clinically challenging PPOs to <1% in our initial intervention. To reduce the overall proportion of PPO, we plan enhancements in process automations, additional physical educational resources, and minor changes in audit criteria. Future projects will aim to address the remaining PPO error types and prepare this project for implementation in other CBOCs.

Background

Cost-effectiveness (CE) evaluations, for existing and anticipated programs, are common in healthcare, but are rarely used in health professions education (HPE). A systematic review of HPE literature found not only few examples of CE evaluations, but also unclear and inconsistent methodology.¹ One proposed reason HPE has been slow to adopt CE evaluations is uncertainty over terminology and how to adapt this methodology to HPE.² CE evaluations present further challenges for HPE since educational outcomes are often not easily monetized. However, given the reality of constrained budgets and limited resources, CE evaluations can be a powerful tool for educators to strengthen arguments for proposed innovations, and for scholars seeking to conduct rigorous work that sustains critical review.

Innovation

This project aims to make CE evaluations more understandable to HPE educators, using a one-page infographic and glossary. This will provide a primer, operationalizing the steps involved in CE evaluations and addressing why and when CE evaluations might be considered in HPE. To improve comprehension, this is being developed collaboratively with health professions educators and an economist. This infographic will be submitted for publication, as a resource to facilitate educators’ scholarly work and conversations with fiscal administrators.

Results

The infographic includes 1) an overview of CE evaluations, 2) information about inputs required for CE evaluations, 3) guidance on interpreting results, 4) a glossary of key terminology, and 5) considerations for why educators might consider this type of analysis. A final draft will be pilot tested with a focus group to assess interdisciplinary accessibility.

Discussion

Discussions between health professions educators and an economist on this infographic uncovered concepts that were poorly understood or defined differently across disciplines, determining specific knowledge gaps and misunderstandings. For example, facilitating conversation between educators and economists highlighted key terms that were a source of misunderstanding. These were then added to the glossary, creating a shared vocabulary. This also helped clarify the steps and information necessary for conducting CE evaluations in HPE, particularly the issue of perspective choice for the analysis (educator, patient, learner, etc.). Overall, this collaboration aimed at making CE evaluations more approachable and understandable for HPE professionals through this infographic.

Background

Cost-effectiveness (CE) evaluations, for existing and anticipated programs, are common in healthcare, but are rarely used in health professions education (HPE). A systematic review of HPE literature found not only few examples of CE evaluations, but also unclear and inconsistent methodology.¹ One proposed reason HPE has been slow to adopt CE evaluations is uncertainty over terminology and how to adapt this methodology to HPE.² CE evaluations present further challenges for HPE since educational outcomes are often not easily monetized. However, given the reality of constrained budgets and limited resources, CE evaluations can be a powerful tool for educators to strengthen arguments for proposed innovations, and for scholars seeking to conduct rigorous work that sustains critical review.

Innovation

This project aims to make CE evaluations more understandable to HPE educators, using a one-page infographic and glossary. This will provide a primer, operationalizing the steps involved in CE evaluations and addressing why and when CE evaluations might be considered in HPE. To improve comprehension, this is being developed collaboratively with health professions educators and an economist. This infographic will be submitted for publication, as a resource to facilitate educators’ scholarly work and conversations with fiscal administrators.

Results

The infographic includes 1) an overview of CE evaluations, 2) information about inputs required for CE evaluations, 3) guidance on interpreting results, 4) a glossary of key terminology, and 5) considerations for why educators might consider this type of analysis. A final draft will be pilot tested with a focus group to assess interdisciplinary accessibility.

Discussion

Discussions between health professions educators and an economist on this infographic uncovered concepts that were poorly understood or defined differently across disciplines, determining specific knowledge gaps and misunderstandings. For example, facilitating conversation between educators and economists highlighted key terms that were a source of misunderstanding. These were then added to the glossary, creating a shared vocabulary. This also helped clarify the steps and information necessary for conducting CE evaluations in HPE, particularly the issue of perspective choice for the analysis (educator, patient, learner, etc.). Overall, this collaboration aimed at making CE evaluations more approachable and understandable for HPE professionals through this infographic.

Background

Cost-effectiveness (CE) evaluations, for existing and anticipated programs, are common in healthcare, but are rarely used in health professions education (HPE). A systematic review of HPE literature found not only few examples of CE evaluations, but also unclear and inconsistent methodology.¹ One proposed reason HPE has been slow to adopt CE evaluations is uncertainty over terminology and how to adapt this methodology to HPE.² CE evaluations present further challenges for HPE since educational outcomes are often not easily monetized. However, given the reality of constrained budgets and limited resources, CE evaluations can be a powerful tool for educators to strengthen arguments for proposed innovations, and for scholars seeking to conduct rigorous work that sustains critical review.

Innovation

This project aims to make CE evaluations more understandable to HPE educators, using a one-page infographic and glossary. This will provide a primer, operationalizing the steps involved in CE evaluations and addressing why and when CE evaluations might be considered in HPE. To improve comprehension, this is being developed collaboratively with health professions educators and an economist. This infographic will be submitted for publication, as a resource to facilitate educators’ scholarly work and conversations with fiscal administrators.

Results

The infographic includes 1) an overview of CE evaluations, 2) information about inputs required for CE evaluations, 3) guidance on interpreting results, 4) a glossary of key terminology, and 5) considerations for why educators might consider this type of analysis. A final draft will be pilot tested with a focus group to assess interdisciplinary accessibility.

Discussion

Discussions between health professions educators and an economist on this infographic uncovered concepts that were poorly understood or defined differently across disciplines, determining specific knowledge gaps and misunderstandings. For example, facilitating conversation between educators and economists highlighted key terms that were a source of misunderstanding. These were then added to the glossary, creating a shared vocabulary. This also helped clarify the steps and information necessary for conducting CE evaluations in HPE, particularly the issue of perspective choice for the analysis (educator, patient, learner, etc.). Overall, this collaboration aimed at making CE evaluations more approachable and understandable for HPE professionals through this infographic.

To the Editor:

Residency applicants, especially in competitive specialties such as dermatology, face major financial barriers due to the high costs of applications, interviews, and away rotations.¹ While several studies have examined application costs of other specialties, few have analyzed expenses associated with dermatology applications.^1,2 There are no data examining costs following the start of the COVID-19 pandemic in 2020; thus, our study evaluated dermatology application cost trends from 2021 to 2024 and compared them to other specialties to identify strategies to reduce the financial burden on applicants.

Self-reported total application costs, application fees, interview expenses, and away rotation costs from 2021 to 2024 were collected from the Texas Seeking Transparency in Application to Residency (STAR) database powered by the UT Southwestern Medical Center (Dallas, Texas).³ The mean total application expenses per year were compared among specialties, and an analysis of variance was used to determine if the differences were statistically significant.

The number of applicants who recorded information in the Texas STAR database was 110 in 2021, 163 in 2022, 136 in 2023, and 129 in 2024.³ The total dermatology application expenses increased from $2805 in 2021 to $6231 in 2024; interview costs increased from $404 in 2021 to $911 in 2024; and away rotation costs increased from $850 in 2021 to $3812 in 2024 (all P<.05)(Table). There was no significant change in application fees during the study period ($2176 in 2021 to $2125 in 2024 [P=.58]). Dermatology had the fourth highest average total cost over the study period compared to all other specialties, increasing from $2250 in 2021 to $5250 in 2024, following orthopedic surgery ($2250 in 2021 to $6750 in 2024), plastic surgery ($2250 in 2021 to $9750 in 2024), and neurosurgery ($1750 in 2021 to $11,250 in 2024).

Our study found that dermatology residency application costs have increased significantly from 2021 to 2024, primarily driven by rising interview and away rotation expenses (both P<.05). This trend places dermatology among the most expensive fields to apply to for residency. A cross-sectional survey of dermatology residency program directors identified away rotations as one of the top 5 selection criteria, underscoring their importance in the matching process.⁴ In addition, a cross-sectional analysis of 345 dermatology residents found that 26.2% matched at institutions where they had mentors, including those they connected with through away rotations.^5,6 Overall, the high cost of away rotations partially may reflect the competitive nature of the specialty, as building connections at programs may enhance the chances of matching. These costs also can vary based on geography, as rotating in high-cost urban centers can be more expensive than in rural areas; however, rural rotations may be less common due to limited program availability and applicant preferences. For example, nearly 50% of 2024 Electronic Residency Application Service applicants indicated a preference for urban settings, while fewer than 5% selected rural settings.⁷ Additionally, the high costs associated with applying to residency programs and completing away rotations can disproportionately impact students from rural backgrounds and underrepresented minorities, who may have fewer financial resources.

In our study, the lower application-related expenses in 2021 (during the pandemic) compared to those of 2024 (postpandemic) likely stem from the Association of American Medical Colleges’ recommendation to conduct virtual interviews during the pandemic.⁸ In 2024, some dermatology programs returned to in-person interviews, with some applicants consequently incurring higher costs related to travel, lodging, and other associated expenses.⁸ A cost-analysis study of 4153 dermatology applicants from 2016 to 2021 found that the average application costs were $1759 per applicant during the pandemic, when virtual interviews replaced in-person ones, whereas costs were $8476 per applicant during periods with in-person interviews and no COVID-19 restrictions.² However, we did not observe a significant change in application fees over our study period, likely because the pandemic did not affect application numbers. A cross-sectional analysis of dermatology applicants during the pandemic similarly reported reductions in application-related expenses during the period when interviews were conducted virtually,⁹ supporting the trend observed in our study. Overall, our findings taken together with other studies highlight the pandemic’s role in reducing expenses and underscore the potential for exploring additional cost-saving measures.

Implementing strategies to reduce these financial burdens—including virtual interviews, increasing student funding for away rotations, and limiting the number of applications individual students can submit—could help alleviate socioeconomic disparities. The new signaling system for residency programs aims to reduce the number of applications submitted, as applicants typically receive interviews only from the limited number of programs they signal, reducing overall application costs. However, our data from the Texas STAR database suggest that application numbers remained relatively stable from 2021 to 2024, indicating that, despite signaling, many applicants still may apply broadly in hopes of improving their chances in an increasingly competitive field. Although a definitive solution to reducing the financial burden on dermatology applicants remains elusive, these strategies can raise awareness and encourage important dialogues.

Limitations of our study include the voluntary nature of the Texas STAR survey, leading to potential voluntary response bias, as well as the small sample size. Students who choose to submit cost data may differ systematically from those who do not; for example, students who match may be more likely to report their outcomes, while those who do not match may be less likely to participate, potentially introducing selection bias. In addition, general awareness of the Texas STAR survey may vary across institutions and among students, further limiting the number of students who participate. Additionally, 2021 was the only presignaling year included, making it difficult to assess longer-term trends. Despite these limitations, the Texas STAR database remains a valuable resource for analyzing general residency application expenses and trends, as it offers comprehensive data from more than 100 medical schools and includes many variables.³

In conclusion, our study found that total dermatology residency application costs have increased significantly from 2021 to 2024 (all P<.05), making dermatology among the most expensive specialties for applying. This study sets the foundation for future survey-based research for applicants and program directors on strategies to alleviate financial burdens.

To the Editor:

Residency applicants, especially in competitive specialties such as dermatology, face major financial barriers due to the high costs of applications, interviews, and away rotations.¹ While several studies have examined application costs of other specialties, few have analyzed expenses associated with dermatology applications.^1,2 There are no data examining costs following the start of the COVID-19 pandemic in 2020; thus, our study evaluated dermatology application cost trends from 2021 to 2024 and compared them to other specialties to identify strategies to reduce the financial burden on applicants.

Self-reported total application costs, application fees, interview expenses, and away rotation costs from 2021 to 2024 were collected from the Texas Seeking Transparency in Application to Residency (STAR) database powered by the UT Southwestern Medical Center (Dallas, Texas).³ The mean total application expenses per year were compared among specialties, and an analysis of variance was used to determine if the differences were statistically significant.

The number of applicants who recorded information in the Texas STAR database was 110 in 2021, 163 in 2022, 136 in 2023, and 129 in 2024.³ The total dermatology application expenses increased from $2805 in 2021 to $6231 in 2024; interview costs increased from $404 in 2021 to $911 in 2024; and away rotation costs increased from $850 in 2021 to $3812 in 2024 (all P<.05)(Table). There was no significant change in application fees during the study period ($2176 in 2021 to $2125 in 2024 [P=.58]). Dermatology had the fourth highest average total cost over the study period compared to all other specialties, increasing from $2250 in 2021 to $5250 in 2024, following orthopedic surgery ($2250 in 2021 to $6750 in 2024), plastic surgery ($2250 in 2021 to $9750 in 2024), and neurosurgery ($1750 in 2021 to $11,250 in 2024).

Our study found that dermatology residency application costs have increased significantly from 2021 to 2024, primarily driven by rising interview and away rotation expenses (both P<.05). This trend places dermatology among the most expensive fields to apply to for residency. A cross-sectional survey of dermatology residency program directors identified away rotations as one of the top 5 selection criteria, underscoring their importance in the matching process.⁴ In addition, a cross-sectional analysis of 345 dermatology residents found that 26.2% matched at institutions where they had mentors, including those they connected with through away rotations.^5,6 Overall, the high cost of away rotations partially may reflect the competitive nature of the specialty, as building connections at programs may enhance the chances of matching. These costs also can vary based on geography, as rotating in high-cost urban centers can be more expensive than in rural areas; however, rural rotations may be less common due to limited program availability and applicant preferences. For example, nearly 50% of 2024 Electronic Residency Application Service applicants indicated a preference for urban settings, while fewer than 5% selected rural settings.⁷ Additionally, the high costs associated with applying to residency programs and completing away rotations can disproportionately impact students from rural backgrounds and underrepresented minorities, who may have fewer financial resources.

In our study, the lower application-related expenses in 2021 (during the pandemic) compared to those of 2024 (postpandemic) likely stem from the Association of American Medical Colleges’ recommendation to conduct virtual interviews during the pandemic.⁸ In 2024, some dermatology programs returned to in-person interviews, with some applicants consequently incurring higher costs related to travel, lodging, and other associated expenses.⁸ A cost-analysis study of 4153 dermatology applicants from 2016 to 2021 found that the average application costs were $1759 per applicant during the pandemic, when virtual interviews replaced in-person ones, whereas costs were $8476 per applicant during periods with in-person interviews and no COVID-19 restrictions.² However, we did not observe a significant change in application fees over our study period, likely because the pandemic did not affect application numbers. A cross-sectional analysis of dermatology applicants during the pandemic similarly reported reductions in application-related expenses during the period when interviews were conducted virtually,⁹ supporting the trend observed in our study. Overall, our findings taken together with other studies highlight the pandemic’s role in reducing expenses and underscore the potential for exploring additional cost-saving measures.

Implementing strategies to reduce these financial burdens—including virtual interviews, increasing student funding for away rotations, and limiting the number of applications individual students can submit—could help alleviate socioeconomic disparities. The new signaling system for residency programs aims to reduce the number of applications submitted, as applicants typically receive interviews only from the limited number of programs they signal, reducing overall application costs. However, our data from the Texas STAR database suggest that application numbers remained relatively stable from 2021 to 2024, indicating that, despite signaling, many applicants still may apply broadly in hopes of improving their chances in an increasingly competitive field. Although a definitive solution to reducing the financial burden on dermatology applicants remains elusive, these strategies can raise awareness and encourage important dialogues.

Limitations of our study include the voluntary nature of the Texas STAR survey, leading to potential voluntary response bias, as well as the small sample size. Students who choose to submit cost data may differ systematically from those who do not; for example, students who match may be more likely to report their outcomes, while those who do not match may be less likely to participate, potentially introducing selection bias. In addition, general awareness of the Texas STAR survey may vary across institutions and among students, further limiting the number of students who participate. Additionally, 2021 was the only presignaling year included, making it difficult to assess longer-term trends. Despite these limitations, the Texas STAR database remains a valuable resource for analyzing general residency application expenses and trends, as it offers comprehensive data from more than 100 medical schools and includes many variables.³

In conclusion, our study found that total dermatology residency application costs have increased significantly from 2021 to 2024 (all P<.05), making dermatology among the most expensive specialties for applying. This study sets the foundation for future survey-based research for applicants and program directors on strategies to alleviate financial burdens.

To the Editor:

Residency applicants, especially in competitive specialties such as dermatology, face major financial barriers due to the high costs of applications, interviews, and away rotations.¹ While several studies have examined application costs of other specialties, few have analyzed expenses associated with dermatology applications.^1,2 There are no data examining costs following the start of the COVID-19 pandemic in 2020; thus, our study evaluated dermatology application cost trends from 2021 to 2024 and compared them to other specialties to identify strategies to reduce the financial burden on applicants.

Self-reported total application costs, application fees, interview expenses, and away rotation costs from 2021 to 2024 were collected from the Texas Seeking Transparency in Application to Residency (STAR) database powered by the UT Southwestern Medical Center (Dallas, Texas).³ The mean total application expenses per year were compared among specialties, and an analysis of variance was used to determine if the differences were statistically significant.

The number of applicants who recorded information in the Texas STAR database was 110 in 2021, 163 in 2022, 136 in 2023, and 129 in 2024.³ The total dermatology application expenses increased from $2805 in 2021 to $6231 in 2024; interview costs increased from $404 in 2021 to $911 in 2024; and away rotation costs increased from $850 in 2021 to $3812 in 2024 (all P<.05)(Table). There was no significant change in application fees during the study period ($2176 in 2021 to $2125 in 2024 [P=.58]). Dermatology had the fourth highest average total cost over the study period compared to all other specialties, increasing from $2250 in 2021 to $5250 in 2024, following orthopedic surgery ($2250 in 2021 to $6750 in 2024), plastic surgery ($2250 in 2021 to $9750 in 2024), and neurosurgery ($1750 in 2021 to $11,250 in 2024).

Our study found that dermatology residency application costs have increased significantly from 2021 to 2024, primarily driven by rising interview and away rotation expenses (both P<.05). This trend places dermatology among the most expensive fields to apply to for residency. A cross-sectional survey of dermatology residency program directors identified away rotations as one of the top 5 selection criteria, underscoring their importance in the matching process.⁴ In addition, a cross-sectional analysis of 345 dermatology residents found that 26.2% matched at institutions where they had mentors, including those they connected with through away rotations.^5,6 Overall, the high cost of away rotations partially may reflect the competitive nature of the specialty, as building connections at programs may enhance the chances of matching. These costs also can vary based on geography, as rotating in high-cost urban centers can be more expensive than in rural areas; however, rural rotations may be less common due to limited program availability and applicant preferences. For example, nearly 50% of 2024 Electronic Residency Application Service applicants indicated a preference for urban settings, while fewer than 5% selected rural settings.⁷ Additionally, the high costs associated with applying to residency programs and completing away rotations can disproportionately impact students from rural backgrounds and underrepresented minorities, who may have fewer financial resources.

In our study, the lower application-related expenses in 2021 (during the pandemic) compared to those of 2024 (postpandemic) likely stem from the Association of American Medical Colleges’ recommendation to conduct virtual interviews during the pandemic.⁸ In 2024, some dermatology programs returned to in-person interviews, with some applicants consequently incurring higher costs related to travel, lodging, and other associated expenses.⁸ A cost-analysis study of 4153 dermatology applicants from 2016 to 2021 found that the average application costs were $1759 per applicant during the pandemic, when virtual interviews replaced in-person ones, whereas costs were $8476 per applicant during periods with in-person interviews and no COVID-19 restrictions.² However, we did not observe a significant change in application fees over our study period, likely because the pandemic did not affect application numbers. A cross-sectional analysis of dermatology applicants during the pandemic similarly reported reductions in application-related expenses during the period when interviews were conducted virtually,⁹ supporting the trend observed in our study. Overall, our findings taken together with other studies highlight the pandemic’s role in reducing expenses and underscore the potential for exploring additional cost-saving measures.

Implementing strategies to reduce these financial burdens—including virtual interviews, increasing student funding for away rotations, and limiting the number of applications individual students can submit—could help alleviate socioeconomic disparities. The new signaling system for residency programs aims to reduce the number of applications submitted, as applicants typically receive interviews only from the limited number of programs they signal, reducing overall application costs. However, our data from the Texas STAR database suggest that application numbers remained relatively stable from 2021 to 2024, indicating that, despite signaling, many applicants still may apply broadly in hopes of improving their chances in an increasingly competitive field. Although a definitive solution to reducing the financial burden on dermatology applicants remains elusive, these strategies can raise awareness and encourage important dialogues.

Limitations of our study include the voluntary nature of the Texas STAR survey, leading to potential voluntary response bias, as well as the small sample size. Students who choose to submit cost data may differ systematically from those who do not; for example, students who match may be more likely to report their outcomes, while those who do not match may be less likely to participate, potentially introducing selection bias. In addition, general awareness of the Texas STAR survey may vary across institutions and among students, further limiting the number of students who participate. Additionally, 2021 was the only presignaling year included, making it difficult to assess longer-term trends. Despite these limitations, the Texas STAR database remains a valuable resource for analyzing general residency application expenses and trends, as it offers comprehensive data from more than 100 medical schools and includes many variables.³

In conclusion, our study found that total dermatology residency application costs have increased significantly from 2021 to 2024 (all P<.05), making dermatology among the most expensive specialties for applying. This study sets the foundation for future survey-based research for applicants and program directors on strategies to alleviate financial burdens.

There are 2.7 million (48%) rural veterans enrolled in the Veterans Health Administration (VHA).¹ Many VHA-enrolled rural veterans are aged ≥ 65 years (54%), a medically complex population that requires more extensive health care.¹ These veterans may live far from US Department of Veterans Affairs (VA) medical centers (VAMCs) and often receive most of their care at rural community-based outpatient clinics (CBOCs). In addition to face-to-face (F2F) services provided at these clinics, many patient care needs may be met using telehealth technology, which can connect veterans at CBOCs with remote health care practitioners (HCPs).

This technology is used across medical specialties throughout the VA and has expanded into neuropsychology services to improve access amid the shortage of rural neuropsychologists. Prior research suggests that access to neuropsychology services improves the functional outcomes of people with diverse medical conditions, including dementia, brain injury, and epilepsy, and reduces emergency department visits, hospitalization duration, and health care costs.^2-6 Given that veterans unable to access neuropsychology services may be at risk for poorer outcomes, identifying ways to improve access is a priority. Tele-neuropsychology (teleNP) has been used to expand access for rural veterans in need of these services.^7,8 

TeleNP is the application of audiovisual technologies to enable remote clinical encounters for neuropsychological assessments.⁹ TeleNP has been shown to be generally equivalent to F2F care, without significant differences compared with in-person visits.^10-13 TeleNP was increasingly implemented following the COVID-19 pandemic and remains an enduring and expanding feature of neuropsychology care delivery.^8,14-18 TeleNP services can increase access to care, especially for rural veterans and those with limited transportation. 

Research in non-VA samples suggests a high level of clinician satisfaction with teleNP.16 In VA samples, research has found high levels of patient satisfaction with teleNP both within Veterans Integrated Services Network (VISN) 20 and in a VA health care system outside VISN 20.^7,19 Investigating staff perceptions of these services and their utility compared with non-VA F2F visits is pertinent to the overall feasibility and effectiveness of teleNP. 

TELE-NEUROPSYCHOLOGY PROGRAM 

A clinical resource hub (CRH) is a VISN-governed program that provides veteran health care when local VHA facilities have service gaps.^20,21 CRH 20 serves several Pacific Northwest VISN 20 health care systems and began providing teleNP in 2015. The CRH 20 teleNP service serves older adults in rural settings with > 570 teleNP evaluations completed over a recent 12-month period (May 2023 to May 2024). In the CRH 20 teleNP program, veterans are offered services by CRH 20 neuropsychologists via telehealth to a patient’s local VAMC, larger health care clinic, CBOC, or via Veterans Video Connect to the home. 

Referral pathways to the CRH 20 teleNP program differ across sites. For VISN 20 sites that do not have any in-house neuropsychology services, referrals are initiated by HCPs from any discipline. At 2 sites with in-house neuropsychology programs, CRH 20 teleNP referrals typically are forwarded from the inhouse service whenever the veteran prefers to be seen at an outlying clinic. All sites, including the CBOCs, are equipped fully for testing, and the HCP encounters veterans in a private office via video-based telehealth technology after a telehealth technician orients them to the space. The private office minimizes environmental disruptions and uses standardized technology to ensure valid results. A limited number of evaluations are offered at home (< 5% of the evaluations) if the veteran is unable to come to a VHA facility, has access to reliable internet, and a minimally distracting home setting. 

In VISN 20, teleNP is a routine practice for delivering services to rural sites, most of which lack neuropsychologists. However, there is limited information about the extent to which the referral sources find the service useful. This quality improvement (QI) project aimed to better understand how well-established teleNP services were received by referral sources/stakeholders and how services could be improved. Prior to the advent of the CRH 20 teleNP program, staff had the option of referring for F2F evaluations in the local community (outside the VA) at some sites, an option that remains. This QI project examined staff perspectives on the usefulness of CRH 20 teleNP services compared with non-VA F2F services. We administered an anonymous, confidential survey examining these factors to VISN 20 staff within 4 VA health care systems. 

METHODS 

This QI project used a mixed quantitative and qualitative descriptive survey design to elicit feedback. The authors (3 neuropsychologists, 1 geropsychologist, and 1 research coordinator) developed the survey questions. The 13-question survey was voluntary, anonymous, and confidential, and respondents were given an opportunity to ask questions, with the first author serving as the point of contact. 

The survey ascertained information about respondents and their work setting (ie, facility type, specific work setting and location, profession, and rurality of patients). First respondents were asked whether they have referred patients to neuropsychology services in the past year. Those who had not referred patients during the past year were asked about reasons for nonreferral with an option to provide an open-ended response. Respondents who did refer were asked how they refer for neuropsychology services and about the usefulness and timeliness of both teleNP and non-VA F2F services. Respondents were asked to respond with their preference for teleNP vs non-VA F2F with an open-ended prompt. Finally, respondents were invited to share any feedback for improvement regarding teleNP services. 

A link to the survey, hosted on the VA Research Electronic Data Capture system, was emailed to facility and service line leaders at the 4 VISN 20 health care systems for distribution to the staff. All staff were included because in many of the facilities, particularly those that are highly rural with low staffing, it is not uncommon for technicians, nurses, and other support staff to assist with placing consults. In particular, VISN 20 nurses often have an optimal understanding of referral pathways to care for patients and are positioned to give and receive feedback about the utility of neuropsychological evaluations. The Research and Development Committee at the Boise VA Medical Center determined this project to be QI and exempt from institutional review board oversight. The VISN 20 employee labor relations HR supervisor approved this survey, with union awareness. Responses were anonymous. 

Data were imported into Microsoft Excel and IBM SPSS Statistics for further analysis. Data were summarized using descriptive statistics, frequencies, and percentages. Nonparametric χ² and Wilcoxon signed-rank tests were used to test for differences. An inductive approach to develop codes was used for the 3 open-ended questions. Two authors (CC, CEG) independently coded the responses and reviewed discrepancies. Final code applications were based on consensus. 

RESULTS 

The survey was deployed for 1 month between February 7, 2024, and June 15, 2024, at each of the 4 health care systems. Thirty-three staff members responded; of these, 1 person did not respond to an item on whether they referred for neuropsychology services. Eighteen of 33 respondents reported referring patients to teleNP or F2F neuropsychology services in the past year. Fourteen of the 33 respondents stated they did not refer; of these, 2 were unfamiliar with the teleNP service and 12 provided other reasons (eg, new to VA, not in their professional scope to order consults, did not have patients needing services). 

The analysis focused on the 18 respondents who referred for neuropsychology services. Thirteen were within health care system A, and 5 were within health care system B (which had no nearby non-VA contracted neuropsychology services) and none were in the other 2 health care systems. Ten of 18 respondents (56%) stated they practiced primarily in a rural setting. Five respondents worked in a CBOC, 12 in a main VA facility, 9 in a primary care setting, 8 in a mental health setting, and 3 in other settings (eg, domiciliary). Participants could select > 1 setting. The 18 respondents who referred to neuropsychology services included 7 psychologists, 1 nurse, 2 social workers, 1 social services assistant, 4 nurse practitioners, 2 physicians, and 1 unknown HCP. 

When asked to categorize the usefulness of services, more respondents characterized teleNP as very much so (1 on a 5-point scale) than F2F referrals (Figure). The mean (SD) of 1.5 (0.8) for teleNP usefulness fell between very much so and mostly and 1 respondent indicated not applicable. Similarly, the mean (SD) for non-VA F2F usefulness was 1.7 (0.9); 9 respondents rated this item as not applicable. A Wilcoxon signed-rank test of related samples indicated no significant differences between the pairs of ratings (Z = 1.50; P = .41). 

Respondents with rural patients were more likely to refer them to teleNP services compared with respondents with nonrural patients (χ² = 5.7; P = .02). However, ratings of teleNP usefulness did not significantly differ for those serving rural vs with nonrural patients (χ² = 1.4; P = .49). Mean (SD) rating of teleNP usefulness was 1.3 (0.7) for the 9 rural subgroup respondents (between very much so and mostly) vs 1.8 (0.9) for the 8 nonrural subgroup respondents (between very much so and mostly). The mean (SD) rating for non-VA F2F usefulness was 1.8 (1.0) for the 4 rural subgroup respondents and 1.6 (0.8) for the 5 nonrural subgroup, between very much so and mostly for both groups. 

Most respondents had no preference between teleNP or F2F. Notably, the responses underlying this group were multifaceted and corresponded to multiple codes (ie, access, preference for in-person services, technology, space and logistics, and service boundaries and requirements). According to 1 respondent, “the logistics of scheduling/room availability, technological challenges, and client behavioral issues that are likely to occur could possibly be more easily addressed via in-person sessions for some clients and providers.” 

Six of 18 respondents preferred teleNP, citing timeliness, ease of access, and evaluation quality. One respondent noted that the “majority of my veterans live in extremely remote areas” and may need to take a plane for their visit. The 3 respondents who preferred in-person neuropsychology services cited veterans’ preference for in-person services. 

Open-Ended Feedback 

Thirteen respondents offered feedback on what is working well with teleNP services. Reasons mentioned were related to the service (ie, timeliness, access, quality) and the neuropsychologist (ie, communication and HCP skills). One respondent described the service and neuropsychologists positively, stating that they were “responsive, notes are readily available, clear assessments and recommendations, being available by [Microsoft] Teams/email.” 

Ten respondents provided suggestions for improvement. Suggestions focused on expanding services, such as to “all veterans with cognitive/memory concerns that desire testing,” individuals with attention-deficit/hyperactivity disorder and co-occurring mental health concerns, and those in residential programs. Suggestions included hiring psychology technicians or more staff and providing education at local clinics. 

DISCUSSION 

This QI project examines VA staff perspectives on the usefulness of CRH 20 teleNP services and non-VA F2F services. While the small sample size limits generalizability, this preliminary study suggests that VA teleNP evaluations were similar to those conducted F2F in non-VA settings. While ratings of teleNP usefulness did not differ significantly for those serving rural vs nonrural veterans, respondents serving rural patients were more likely to refer patients to teleNP, suggesting that teleNP may increase access in rural settings, consistent with other studies.^7,8,13 This article also presents qualitative suggestions for improving teleNP delivery within the VHA. This is the first known initiative to report on VHA staff satisfaction with a teleNP service and expands the limited literature to date on satisfaction with teleNP services. The findings provide initial support for continued use and, potentially, expansion of teleNP services within this CRH remote hub-and-spoke model. 

Limitations 

A significant limitation of the current work is the small sample size of survey respondents. In particular, while teleNP turnaround time was perceived as faster than non-VA F2F care, only 8 respondents reported on timeliness of F2F evaluation results, which renders it difficult to draw conclusions. Interestingly, not all respondents reported referring to neuropsychology services within the previous year; the most common reasons reflect the perception that referral to neuropsychology was outside of that staff member’s role or not clinically indicated. 

One additional possible explanation for the absence of reporting on utility of teleNP specifically is that respondents did not track whether their patient was seen by teleNP or F2F services, based on how the referral process varies at each health care system. For example, in health care system C, a large number of referrals are forwarded to the service by local VA F2F neuropsychologists. This may speak to the seamlessness of the teleNP process, such that local staff and/or referring HCPs are unaware of the modality over which neuropsychology is being conducted. It is plausible that the reason behind this smaller response rate in health care systems B and C relates to how neuropsychology consults are processed at these local VAMCs. We suspect that in these settings, the HCPs referring for neuropsychological evaluations (eg, primary care, mental health) may be unaware that their referrals are being triaged to neuropsychologists in a different program (CRH 20 teleNP). Therefore, they would not necessarily know that they used teleNP and didn’t complete the survey. 

The referral process for these 2 sites contrasts with the process for other VISN 20 sites where there is no local neuropsychology program triaging. In these settings, referrals from local HCPs come directly to teleNP; thus, it is more likely that these HCPs are aware of teleNP services. There were only 2 physicians who completed the survey, which may relate to their workload and a workflow where other staff have been increasingly requested to order the consults for the physician. This type of workflow results in an increase in the number of VHA staff involved in patient care. Ratings of usefulness were highest in health care system B, which does not have neuropsychology services at the facility or in the community; this may relate to elevated teleNP satisfaction ratings. 

Further work may help identify which aspects of a teleNP service make it more useful than F2F care for this population or determine whether there were HCPor setting-specific factors that influenced the ratings (ie, preference for VA care or comparison of favorability ratings for the HCPs who conduct teleNP and F2F within the same system). The latter comparisons could not be drawn in the current systems due to the absence of HCPs who provide both teleNP and F2F modalities within VISN 20. Another consideration for future work would be to use a previously published/validated survey measure and piloting of questions with a naive sample before implementation. 

CONCLUSIONS 

This analysis provides initial support for feasibility and acceptability of teleNP as an alternative to traditional in-person neuropsychological evaluations. The small number of survey respondents may reflect the multiple pathways through which consults are forwarded to CRH 20, which includes both direct HCP referrals and forwarded consults from local neuropsychology services. CRH 20 has completed > 570 teleNP evaluations within 1 year, suggesting that lack of awareness may not be hindering veteran access to the service. Replication with a larger sample that is more broadly representative of key stakeholders in veteran care, identification of populations that would benefit most from teleNP services, and dissemination studies of the expansion of teleNP services are all important directions for future work. The robustness and longevity of the VISN 20 teleNP program, coupled with the preliminary positive findings from this project, demonstrate support for further assessment of the potential impact of telehealth on neuropsychological care within the VHA and show that barriers associated with access to health care services in remote settings may be mitigated through teleNP service delivery.

There are 2.7 million (48%) rural veterans enrolled in the Veterans Health Administration (VHA).¹ Many VHA-enrolled rural veterans are aged ≥ 65 years (54%), a medically complex population that requires more extensive health care.¹ These veterans may live far from US Department of Veterans Affairs (VA) medical centers (VAMCs) and often receive most of their care at rural community-based outpatient clinics (CBOCs). In addition to face-to-face (F2F) services provided at these clinics, many patient care needs may be met using telehealth technology, which can connect veterans at CBOCs with remote health care practitioners (HCPs).

This technology is used across medical specialties throughout the VA and has expanded into neuropsychology services to improve access amid the shortage of rural neuropsychologists. Prior research suggests that access to neuropsychology services improves the functional outcomes of people with diverse medical conditions, including dementia, brain injury, and epilepsy, and reduces emergency department visits, hospitalization duration, and health care costs.^2-6 Given that veterans unable to access neuropsychology services may be at risk for poorer outcomes, identifying ways to improve access is a priority. Tele-neuropsychology (teleNP) has been used to expand access for rural veterans in need of these services.^7,8 

TeleNP is the application of audiovisual technologies to enable remote clinical encounters for neuropsychological assessments.⁹ TeleNP has been shown to be generally equivalent to F2F care, without significant differences compared with in-person visits.^10-13 TeleNP was increasingly implemented following the COVID-19 pandemic and remains an enduring and expanding feature of neuropsychology care delivery.^8,14-18 TeleNP services can increase access to care, especially for rural veterans and those with limited transportation. 

Research in non-VA samples suggests a high level of clinician satisfaction with teleNP.16 In VA samples, research has found high levels of patient satisfaction with teleNP both within Veterans Integrated Services Network (VISN) 20 and in a VA health care system outside VISN 20.^7,19 Investigating staff perceptions of these services and their utility compared with non-VA F2F visits is pertinent to the overall feasibility and effectiveness of teleNP. 

TELE-NEUROPSYCHOLOGY PROGRAM 

A clinical resource hub (CRH) is a VISN-governed program that provides veteran health care when local VHA facilities have service gaps.^20,21 CRH 20 serves several Pacific Northwest VISN 20 health care systems and began providing teleNP in 2015. The CRH 20 teleNP service serves older adults in rural settings with > 570 teleNP evaluations completed over a recent 12-month period (May 2023 to May 2024). In the CRH 20 teleNP program, veterans are offered services by CRH 20 neuropsychologists via telehealth to a patient’s local VAMC, larger health care clinic, CBOC, or via Veterans Video Connect to the home. 

Referral pathways to the CRH 20 teleNP program differ across sites. For VISN 20 sites that do not have any in-house neuropsychology services, referrals are initiated by HCPs from any discipline. At 2 sites with in-house neuropsychology programs, CRH 20 teleNP referrals typically are forwarded from the inhouse service whenever the veteran prefers to be seen at an outlying clinic. All sites, including the CBOCs, are equipped fully for testing, and the HCP encounters veterans in a private office via video-based telehealth technology after a telehealth technician orients them to the space. The private office minimizes environmental disruptions and uses standardized technology to ensure valid results. A limited number of evaluations are offered at home (< 5% of the evaluations) if the veteran is unable to come to a VHA facility, has access to reliable internet, and a minimally distracting home setting. 

In VISN 20, teleNP is a routine practice for delivering services to rural sites, most of which lack neuropsychologists. However, there is limited information about the extent to which the referral sources find the service useful. This quality improvement (QI) project aimed to better understand how well-established teleNP services were received by referral sources/stakeholders and how services could be improved. Prior to the advent of the CRH 20 teleNP program, staff had the option of referring for F2F evaluations in the local community (outside the VA) at some sites, an option that remains. This QI project examined staff perspectives on the usefulness of CRH 20 teleNP services compared with non-VA F2F services. We administered an anonymous, confidential survey examining these factors to VISN 20 staff within 4 VA health care systems. 

METHODS 

This QI project used a mixed quantitative and qualitative descriptive survey design to elicit feedback. The authors (3 neuropsychologists, 1 geropsychologist, and 1 research coordinator) developed the survey questions. The 13-question survey was voluntary, anonymous, and confidential, and respondents were given an opportunity to ask questions, with the first author serving as the point of contact. 

The survey ascertained information about respondents and their work setting (ie, facility type, specific work setting and location, profession, and rurality of patients). First respondents were asked whether they have referred patients to neuropsychology services in the past year. Those who had not referred patients during the past year were asked about reasons for nonreferral with an option to provide an open-ended response. Respondents who did refer were asked how they refer for neuropsychology services and about the usefulness and timeliness of both teleNP and non-VA F2F services. Respondents were asked to respond with their preference for teleNP vs non-VA F2F with an open-ended prompt. Finally, respondents were invited to share any feedback for improvement regarding teleNP services. 

A link to the survey, hosted on the VA Research Electronic Data Capture system, was emailed to facility and service line leaders at the 4 VISN 20 health care systems for distribution to the staff. All staff were included because in many of the facilities, particularly those that are highly rural with low staffing, it is not uncommon for technicians, nurses, and other support staff to assist with placing consults. In particular, VISN 20 nurses often have an optimal understanding of referral pathways to care for patients and are positioned to give and receive feedback about the utility of neuropsychological evaluations. The Research and Development Committee at the Boise VA Medical Center determined this project to be QI and exempt from institutional review board oversight. The VISN 20 employee labor relations HR supervisor approved this survey, with union awareness. Responses were anonymous. 

Data were imported into Microsoft Excel and IBM SPSS Statistics for further analysis. Data were summarized using descriptive statistics, frequencies, and percentages. Nonparametric χ² and Wilcoxon signed-rank tests were used to test for differences. An inductive approach to develop codes was used for the 3 open-ended questions. Two authors (CC, CEG) independently coded the responses and reviewed discrepancies. Final code applications were based on consensus. 

RESULTS 

The survey was deployed for 1 month between February 7, 2024, and June 15, 2024, at each of the 4 health care systems. Thirty-three staff members responded; of these, 1 person did not respond to an item on whether they referred for neuropsychology services. Eighteen of 33 respondents reported referring patients to teleNP or F2F neuropsychology services in the past year. Fourteen of the 33 respondents stated they did not refer; of these, 2 were unfamiliar with the teleNP service and 12 provided other reasons (eg, new to VA, not in their professional scope to order consults, did not have patients needing services). 

The analysis focused on the 18 respondents who referred for neuropsychology services. Thirteen were within health care system A, and 5 were within health care system B (which had no nearby non-VA contracted neuropsychology services) and none were in the other 2 health care systems. Ten of 18 respondents (56%) stated they practiced primarily in a rural setting. Five respondents worked in a CBOC, 12 in a main VA facility, 9 in a primary care setting, 8 in a mental health setting, and 3 in other settings (eg, domiciliary). Participants could select > 1 setting. The 18 respondents who referred to neuropsychology services included 7 psychologists, 1 nurse, 2 social workers, 1 social services assistant, 4 nurse practitioners, 2 physicians, and 1 unknown HCP. 

When asked to categorize the usefulness of services, more respondents characterized teleNP as very much so (1 on a 5-point scale) than F2F referrals (Figure). The mean (SD) of 1.5 (0.8) for teleNP usefulness fell between very much so and mostly and 1 respondent indicated not applicable. Similarly, the mean (SD) for non-VA F2F usefulness was 1.7 (0.9); 9 respondents rated this item as not applicable. A Wilcoxon signed-rank test of related samples indicated no significant differences between the pairs of ratings (Z = 1.50; P = .41). 

Respondents with rural patients were more likely to refer them to teleNP services compared with respondents with nonrural patients (χ² = 5.7; P = .02). However, ratings of teleNP usefulness did not significantly differ for those serving rural vs with nonrural patients (χ² = 1.4; P = .49). Mean (SD) rating of teleNP usefulness was 1.3 (0.7) for the 9 rural subgroup respondents (between very much so and mostly) vs 1.8 (0.9) for the 8 nonrural subgroup respondents (between very much so and mostly). The mean (SD) rating for non-VA F2F usefulness was 1.8 (1.0) for the 4 rural subgroup respondents and 1.6 (0.8) for the 5 nonrural subgroup, between very much so and mostly for both groups. 

Most respondents had no preference between teleNP or F2F. Notably, the responses underlying this group were multifaceted and corresponded to multiple codes (ie, access, preference for in-person services, technology, space and logistics, and service boundaries and requirements). According to 1 respondent, “the logistics of scheduling/room availability, technological challenges, and client behavioral issues that are likely to occur could possibly be more easily addressed via in-person sessions for some clients and providers.” 

Six of 18 respondents preferred teleNP, citing timeliness, ease of access, and evaluation quality. One respondent noted that the “majority of my veterans live in extremely remote areas” and may need to take a plane for their visit. The 3 respondents who preferred in-person neuropsychology services cited veterans’ preference for in-person services. 

Open-Ended Feedback 

Thirteen respondents offered feedback on what is working well with teleNP services. Reasons mentioned were related to the service (ie, timeliness, access, quality) and the neuropsychologist (ie, communication and HCP skills). One respondent described the service and neuropsychologists positively, stating that they were “responsive, notes are readily available, clear assessments and recommendations, being available by [Microsoft] Teams/email.” 

Ten respondents provided suggestions for improvement. Suggestions focused on expanding services, such as to “all veterans with cognitive/memory concerns that desire testing,” individuals with attention-deficit/hyperactivity disorder and co-occurring mental health concerns, and those in residential programs. Suggestions included hiring psychology technicians or more staff and providing education at local clinics. 

DISCUSSION 

This QI project examines VA staff perspectives on the usefulness of CRH 20 teleNP services and non-VA F2F services. While the small sample size limits generalizability, this preliminary study suggests that VA teleNP evaluations were similar to those conducted F2F in non-VA settings. While ratings of teleNP usefulness did not differ significantly for those serving rural vs nonrural veterans, respondents serving rural patients were more likely to refer patients to teleNP, suggesting that teleNP may increase access in rural settings, consistent with other studies.^7,8,13 This article also presents qualitative suggestions for improving teleNP delivery within the VHA. This is the first known initiative to report on VHA staff satisfaction with a teleNP service and expands the limited literature to date on satisfaction with teleNP services. The findings provide initial support for continued use and, potentially, expansion of teleNP services within this CRH remote hub-and-spoke model. 

Limitations 

A significant limitation of the current work is the small sample size of survey respondents. In particular, while teleNP turnaround time was perceived as faster than non-VA F2F care, only 8 respondents reported on timeliness of F2F evaluation results, which renders it difficult to draw conclusions. Interestingly, not all respondents reported referring to neuropsychology services within the previous year; the most common reasons reflect the perception that referral to neuropsychology was outside of that staff member’s role or not clinically indicated. 

One additional possible explanation for the absence of reporting on utility of teleNP specifically is that respondents did not track whether their patient was seen by teleNP or F2F services, based on how the referral process varies at each health care system. For example, in health care system C, a large number of referrals are forwarded to the service by local VA F2F neuropsychologists. This may speak to the seamlessness of the teleNP process, such that local staff and/or referring HCPs are unaware of the modality over which neuropsychology is being conducted. It is plausible that the reason behind this smaller response rate in health care systems B and C relates to how neuropsychology consults are processed at these local VAMCs. We suspect that in these settings, the HCPs referring for neuropsychological evaluations (eg, primary care, mental health) may be unaware that their referrals are being triaged to neuropsychologists in a different program (CRH 20 teleNP). Therefore, they would not necessarily know that they used teleNP and didn’t complete the survey. 

The referral process for these 2 sites contrasts with the process for other VISN 20 sites where there is no local neuropsychology program triaging. In these settings, referrals from local HCPs come directly to teleNP; thus, it is more likely that these HCPs are aware of teleNP services. There were only 2 physicians who completed the survey, which may relate to their workload and a workflow where other staff have been increasingly requested to order the consults for the physician. This type of workflow results in an increase in the number of VHA staff involved in patient care. Ratings of usefulness were highest in health care system B, which does not have neuropsychology services at the facility or in the community; this may relate to elevated teleNP satisfaction ratings. 

Further work may help identify which aspects of a teleNP service make it more useful than F2F care for this population or determine whether there were HCPor setting-specific factors that influenced the ratings (ie, preference for VA care or comparison of favorability ratings for the HCPs who conduct teleNP and F2F within the same system). The latter comparisons could not be drawn in the current systems due to the absence of HCPs who provide both teleNP and F2F modalities within VISN 20. Another consideration for future work would be to use a previously published/validated survey measure and piloting of questions with a naive sample before implementation. 

CONCLUSIONS 

This analysis provides initial support for feasibility and acceptability of teleNP as an alternative to traditional in-person neuropsychological evaluations. The small number of survey respondents may reflect the multiple pathways through which consults are forwarded to CRH 20, which includes both direct HCP referrals and forwarded consults from local neuropsychology services. CRH 20 has completed > 570 teleNP evaluations within 1 year, suggesting that lack of awareness may not be hindering veteran access to the service. Replication with a larger sample that is more broadly representative of key stakeholders in veteran care, identification of populations that would benefit most from teleNP services, and dissemination studies of the expansion of teleNP services are all important directions for future work. The robustness and longevity of the VISN 20 teleNP program, coupled with the preliminary positive findings from this project, demonstrate support for further assessment of the potential impact of telehealth on neuropsychological care within the VHA and show that barriers associated with access to health care services in remote settings may be mitigated through teleNP service delivery.

There are 2.7 million (48%) rural veterans enrolled in the Veterans Health Administration (VHA).¹ Many VHA-enrolled rural veterans are aged ≥ 65 years (54%), a medically complex population that requires more extensive health care.¹ These veterans may live far from US Department of Veterans Affairs (VA) medical centers (VAMCs) and often receive most of their care at rural community-based outpatient clinics (CBOCs). In addition to face-to-face (F2F) services provided at these clinics, many patient care needs may be met using telehealth technology, which can connect veterans at CBOCs with remote health care practitioners (HCPs).

This technology is used across medical specialties throughout the VA and has expanded into neuropsychology services to improve access amid the shortage of rural neuropsychologists. Prior research suggests that access to neuropsychology services improves the functional outcomes of people with diverse medical conditions, including dementia, brain injury, and epilepsy, and reduces emergency department visits, hospitalization duration, and health care costs.^2-6 Given that veterans unable to access neuropsychology services may be at risk for poorer outcomes, identifying ways to improve access is a priority. Tele-neuropsychology (teleNP) has been used to expand access for rural veterans in need of these services.^7,8 

TeleNP is the application of audiovisual technologies to enable remote clinical encounters for neuropsychological assessments.⁹ TeleNP has been shown to be generally equivalent to F2F care, without significant differences compared with in-person visits.^10-13 TeleNP was increasingly implemented following the COVID-19 pandemic and remains an enduring and expanding feature of neuropsychology care delivery.^8,14-18 TeleNP services can increase access to care, especially for rural veterans and those with limited transportation. 

Research in non-VA samples suggests a high level of clinician satisfaction with teleNP.16 In VA samples, research has found high levels of patient satisfaction with teleNP both within Veterans Integrated Services Network (VISN) 20 and in a VA health care system outside VISN 20.^7,19 Investigating staff perceptions of these services and their utility compared with non-VA F2F visits is pertinent to the overall feasibility and effectiveness of teleNP. 

TELE-NEUROPSYCHOLOGY PROGRAM 

A clinical resource hub (CRH) is a VISN-governed program that provides veteran health care when local VHA facilities have service gaps.^20,21 CRH 20 serves several Pacific Northwest VISN 20 health care systems and began providing teleNP in 2015. The CRH 20 teleNP service serves older adults in rural settings with > 570 teleNP evaluations completed over a recent 12-month period (May 2023 to May 2024). In the CRH 20 teleNP program, veterans are offered services by CRH 20 neuropsychologists via telehealth to a patient’s local VAMC, larger health care clinic, CBOC, or via Veterans Video Connect to the home. 

Referral pathways to the CRH 20 teleNP program differ across sites. For VISN 20 sites that do not have any in-house neuropsychology services, referrals are initiated by HCPs from any discipline. At 2 sites with in-house neuropsychology programs, CRH 20 teleNP referrals typically are forwarded from the inhouse service whenever the veteran prefers to be seen at an outlying clinic. All sites, including the CBOCs, are equipped fully for testing, and the HCP encounters veterans in a private office via video-based telehealth technology after a telehealth technician orients them to the space. The private office minimizes environmental disruptions and uses standardized technology to ensure valid results. A limited number of evaluations are offered at home (< 5% of the evaluations) if the veteran is unable to come to a VHA facility, has access to reliable internet, and a minimally distracting home setting. 

In VISN 20, teleNP is a routine practice for delivering services to rural sites, most of which lack neuropsychologists. However, there is limited information about the extent to which the referral sources find the service useful. This quality improvement (QI) project aimed to better understand how well-established teleNP services were received by referral sources/stakeholders and how services could be improved. Prior to the advent of the CRH 20 teleNP program, staff had the option of referring for F2F evaluations in the local community (outside the VA) at some sites, an option that remains. This QI project examined staff perspectives on the usefulness of CRH 20 teleNP services compared with non-VA F2F services. We administered an anonymous, confidential survey examining these factors to VISN 20 staff within 4 VA health care systems. 

METHODS 

This QI project used a mixed quantitative and qualitative descriptive survey design to elicit feedback. The authors (3 neuropsychologists, 1 geropsychologist, and 1 research coordinator) developed the survey questions. The 13-question survey was voluntary, anonymous, and confidential, and respondents were given an opportunity to ask questions, with the first author serving as the point of contact. 

The survey ascertained information about respondents and their work setting (ie, facility type, specific work setting and location, profession, and rurality of patients). First respondents were asked whether they have referred patients to neuropsychology services in the past year. Those who had not referred patients during the past year were asked about reasons for nonreferral with an option to provide an open-ended response. Respondents who did refer were asked how they refer for neuropsychology services and about the usefulness and timeliness of both teleNP and non-VA F2F services. Respondents were asked to respond with their preference for teleNP vs non-VA F2F with an open-ended prompt. Finally, respondents were invited to share any feedback for improvement regarding teleNP services. 

A link to the survey, hosted on the VA Research Electronic Data Capture system, was emailed to facility and service line leaders at the 4 VISN 20 health care systems for distribution to the staff. All staff were included because in many of the facilities, particularly those that are highly rural with low staffing, it is not uncommon for technicians, nurses, and other support staff to assist with placing consults. In particular, VISN 20 nurses often have an optimal understanding of referral pathways to care for patients and are positioned to give and receive feedback about the utility of neuropsychological evaluations. The Research and Development Committee at the Boise VA Medical Center determined this project to be QI and exempt from institutional review board oversight. The VISN 20 employee labor relations HR supervisor approved this survey, with union awareness. Responses were anonymous. 

Data were imported into Microsoft Excel and IBM SPSS Statistics for further analysis. Data were summarized using descriptive statistics, frequencies, and percentages. Nonparametric χ² and Wilcoxon signed-rank tests were used to test for differences. An inductive approach to develop codes was used for the 3 open-ended questions. Two authors (CC, CEG) independently coded the responses and reviewed discrepancies. Final code applications were based on consensus. 

RESULTS 

The survey was deployed for 1 month between February 7, 2024, and June 15, 2024, at each of the 4 health care systems. Thirty-three staff members responded; of these, 1 person did not respond to an item on whether they referred for neuropsychology services. Eighteen of 33 respondents reported referring patients to teleNP or F2F neuropsychology services in the past year. Fourteen of the 33 respondents stated they did not refer; of these, 2 were unfamiliar with the teleNP service and 12 provided other reasons (eg, new to VA, not in their professional scope to order consults, did not have patients needing services). 

The analysis focused on the 18 respondents who referred for neuropsychology services. Thirteen were within health care system A, and 5 were within health care system B (which had no nearby non-VA contracted neuropsychology services) and none were in the other 2 health care systems. Ten of 18 respondents (56%) stated they practiced primarily in a rural setting. Five respondents worked in a CBOC, 12 in a main VA facility, 9 in a primary care setting, 8 in a mental health setting, and 3 in other settings (eg, domiciliary). Participants could select > 1 setting. The 18 respondents who referred to neuropsychology services included 7 psychologists, 1 nurse, 2 social workers, 1 social services assistant, 4 nurse practitioners, 2 physicians, and 1 unknown HCP. 

When asked to categorize the usefulness of services, more respondents characterized teleNP as very much so (1 on a 5-point scale) than F2F referrals (Figure). The mean (SD) of 1.5 (0.8) for teleNP usefulness fell between very much so and mostly and 1 respondent indicated not applicable. Similarly, the mean (SD) for non-VA F2F usefulness was 1.7 (0.9); 9 respondents rated this item as not applicable. A Wilcoxon signed-rank test of related samples indicated no significant differences between the pairs of ratings (Z = 1.50; P = .41). 

Respondents with rural patients were more likely to refer them to teleNP services compared with respondents with nonrural patients (χ² = 5.7; P = .02). However, ratings of teleNP usefulness did not significantly differ for those serving rural vs with nonrural patients (χ² = 1.4; P = .49). Mean (SD) rating of teleNP usefulness was 1.3 (0.7) for the 9 rural subgroup respondents (between very much so and mostly) vs 1.8 (0.9) for the 8 nonrural subgroup respondents (between very much so and mostly). The mean (SD) rating for non-VA F2F usefulness was 1.8 (1.0) for the 4 rural subgroup respondents and 1.6 (0.8) for the 5 nonrural subgroup, between very much so and mostly for both groups. 

Most respondents had no preference between teleNP or F2F. Notably, the responses underlying this group were multifaceted and corresponded to multiple codes (ie, access, preference for in-person services, technology, space and logistics, and service boundaries and requirements). According to 1 respondent, “the logistics of scheduling/room availability, technological challenges, and client behavioral issues that are likely to occur could possibly be more easily addressed via in-person sessions for some clients and providers.” 

Six of 18 respondents preferred teleNP, citing timeliness, ease of access, and evaluation quality. One respondent noted that the “majority of my veterans live in extremely remote areas” and may need to take a plane for their visit. The 3 respondents who preferred in-person neuropsychology services cited veterans’ preference for in-person services. 

Open-Ended Feedback 

Thirteen respondents offered feedback on what is working well with teleNP services. Reasons mentioned were related to the service (ie, timeliness, access, quality) and the neuropsychologist (ie, communication and HCP skills). One respondent described the service and neuropsychologists positively, stating that they were “responsive, notes are readily available, clear assessments and recommendations, being available by [Microsoft] Teams/email.” 

Ten respondents provided suggestions for improvement. Suggestions focused on expanding services, such as to “all veterans with cognitive/memory concerns that desire testing,” individuals with attention-deficit/hyperactivity disorder and co-occurring mental health concerns, and those in residential programs. Suggestions included hiring psychology technicians or more staff and providing education at local clinics. 

DISCUSSION 

This QI project examines VA staff perspectives on the usefulness of CRH 20 teleNP services and non-VA F2F services. While the small sample size limits generalizability, this preliminary study suggests that VA teleNP evaluations were similar to those conducted F2F in non-VA settings. While ratings of teleNP usefulness did not differ significantly for those serving rural vs nonrural veterans, respondents serving rural patients were more likely to refer patients to teleNP, suggesting that teleNP may increase access in rural settings, consistent with other studies.^7,8,13 This article also presents qualitative suggestions for improving teleNP delivery within the VHA. This is the first known initiative to report on VHA staff satisfaction with a teleNP service and expands the limited literature to date on satisfaction with teleNP services. The findings provide initial support for continued use and, potentially, expansion of teleNP services within this CRH remote hub-and-spoke model. 

Limitations 

A significant limitation of the current work is the small sample size of survey respondents. In particular, while teleNP turnaround time was perceived as faster than non-VA F2F care, only 8 respondents reported on timeliness of F2F evaluation results, which renders it difficult to draw conclusions. Interestingly, not all respondents reported referring to neuropsychology services within the previous year; the most common reasons reflect the perception that referral to neuropsychology was outside of that staff member’s role or not clinically indicated. 

One additional possible explanation for the absence of reporting on utility of teleNP specifically is that respondents did not track whether their patient was seen by teleNP or F2F services, based on how the referral process varies at each health care system. For example, in health care system C, a large number of referrals are forwarded to the service by local VA F2F neuropsychologists. This may speak to the seamlessness of the teleNP process, such that local staff and/or referring HCPs are unaware of the modality over which neuropsychology is being conducted. It is plausible that the reason behind this smaller response rate in health care systems B and C relates to how neuropsychology consults are processed at these local VAMCs. We suspect that in these settings, the HCPs referring for neuropsychological evaluations (eg, primary care, mental health) may be unaware that their referrals are being triaged to neuropsychologists in a different program (CRH 20 teleNP). Therefore, they would not necessarily know that they used teleNP and didn’t complete the survey. 

The referral process for these 2 sites contrasts with the process for other VISN 20 sites where there is no local neuropsychology program triaging. In these settings, referrals from local HCPs come directly to teleNP; thus, it is more likely that these HCPs are aware of teleNP services. There were only 2 physicians who completed the survey, which may relate to their workload and a workflow where other staff have been increasingly requested to order the consults for the physician. This type of workflow results in an increase in the number of VHA staff involved in patient care. Ratings of usefulness were highest in health care system B, which does not have neuropsychology services at the facility or in the community; this may relate to elevated teleNP satisfaction ratings. 

Further work may help identify which aspects of a teleNP service make it more useful than F2F care for this population or determine whether there were HCPor setting-specific factors that influenced the ratings (ie, preference for VA care or comparison of favorability ratings for the HCPs who conduct teleNP and F2F within the same system). The latter comparisons could not be drawn in the current systems due to the absence of HCPs who provide both teleNP and F2F modalities within VISN 20. Another consideration for future work would be to use a previously published/validated survey measure and piloting of questions with a naive sample before implementation. 

CONCLUSIONS 

This analysis provides initial support for feasibility and acceptability of teleNP as an alternative to traditional in-person neuropsychological evaluations. The small number of survey respondents may reflect the multiple pathways through which consults are forwarded to CRH 20, which includes both direct HCP referrals and forwarded consults from local neuropsychology services. CRH 20 has completed > 570 teleNP evaluations within 1 year, suggesting that lack of awareness may not be hindering veteran access to the service. Replication with a larger sample that is more broadly representative of key stakeholders in veteran care, identification of populations that would benefit most from teleNP services, and dissemination studies of the expansion of teleNP services are all important directions for future work. The robustness and longevity of the VISN 20 teleNP program, coupled with the preliminary positive findings from this project, demonstrate support for further assessment of the potential impact of telehealth on neuropsychological care within the VHA and show that barriers associated with access to health care services in remote settings may be mitigated through teleNP service delivery.

Use of artificial intelligence (AI) in dermatology has increased over the past decade, likely driven by advances in deep learning algorithms, computing hardware, and machine learning.¹ Studies comparing the performance of AI algorithms to dermatologists in classifying skin disorders have shown conflicting results.^2,3 In this study, we aimed to analyze AI tools used for diagnosing and classifying skin disease and evaluate the role of dermatologists in the creation of AI technology. We also investigated the number of clinical images used in datasets to train AI programs and compared tools that were created with dermatologist input to those created without dermatologist/clinician involvement.

Methods

A search of PubMed articles indexed for MEDLINE using the terms machine learning, artificial intelligence, and dermatology was conducted on September 18, 2022. Articles were included if they described full-length trials; used machine learning for diagnosis of or screening for dermatologic conditions; and used dermoscopic or gross image datasets of the skin, hair, or nails. Articles were categorized into 4 groups based on the conditions covered: chronic wounds, inflammatory skin diseases, mixed conditions, and pigmented skin lesions. Algorithms were sorted into 4 categories: convolutional/convoluted neural network, deep learning model/deep neural network, AI/artificial neural network, and other. Details regarding Fitzpatrick skin type and skin of color (SoC) inclusion in the articles or AI algorithm datasets were recorded. Univariate and multivariate analyses were performed using Microsoft Excel and SAS Studio 3.8. Sensitivity and specificity were calculated for all included AI technology. Sensitivity, specificity, and the number of clinical images were compared among the included articles using analysis of variance and t tests (α=0.05; P<.05 indicated statistical significance).

Results

Our search yielded 1016 articles, 58 of which met the inclusion criteria. Overall, 25.9% (15/58) of the articles utilized AI to diagnose or classify mixed skin diseases; 22.4% (13/58) for pigmented skin lesions; 19.0% (11/58) for wounds; 17.2% (10/58) for inflammatory skin diseases; and 5.2% (3/58) each for acne, psoriasis, and onychomycosis. Overall, 24.0% (14/58) of articles provided information about Fitzpatrick skin type, and 58.7% (34/58) included clinical images depicting SoC. Furthermore, we found that only 20.7% (12/58) of articles on deep learning models included descriptions of patient ethnicity or race in at least 1 dataset, and only 10.3% (6/58) of studies included any information about skin tone in the dataset. Studies with a dermatologist as the last author (most likely to be supervising the project) were more likely to include clinical images depicting SoC than those without (82.6% [19/23] and 16.7% [3/18], respectively [P=.0411]).

The mean (SD) number of clinical images in the study articles was 28,422 (84,050). Thirty-seven (63.8%) of the study articles included gross images, 17 (29.3%) used dermoscopic images, and 4 (6.9%) used both. Twenty-seven (46.6%) articles used convolutional/convoluted neural networks, 15 (25.9%) used deep learning model/deep neural networks, 8 (13.8%) used other algorithms, 6 (10.3%) used AI/artificial neural network, and 2 (3.4%) used fuzzy algorithms. Most studies were conducted in China (29.3% [17/58]), Germany (12.1% [7/58]), India (10.3% [6/58]), multiple nations (10.3% [6/58]), and the United States (10.3% [6/58]). Overall, 82.8% (48/58) of articles included at least 1 dermatologist coauthor. Sensitivity of the AI models was 0.85, and specificity was 0.85. The average percentage of images in the dataset correctly identified by a physician was 76.87% vs 81.62% of images correctly identified by AI. Average agreement between AI and physician assessment was 77.98%, defined as AI and physician both having the same diagnosis. 

Articles authored by dermatologists contained more clinical images than those without dermatologists in key authorship roles (P<.0001)(eTable). Psoriasis-related algorithms had the fewest (mean [SD]: 3173 [4203]), and pigmented skin lesions had the most clinical images (mean [SD]: 53,19l [155,579]).

Comment

Our results indicated that AI studies with dermatologist authors had significantly more images in their datasets (ie, the set of clinical images of skin lesions used to train AI algorithms in diagnosing or classifying lesions) than those with nondermatologist authors (P<.0001)(eTable). Similarly, in a study of AI technology for skin cancer diagnosis, AI studies with dermatologist authors (ie, included in the development of the AI algorithm) had more images than studies without dermatologist authors.¹ Deep learning textbooks have suggested that 5000 clinical images or training input per output category are needed to produce acceptable algorithm performance, and more than 10 million are needed to produce results superior to human performance.^4-10 Despite advances in AI for dermatologic image analysis, the creation of these models often has been directed by nondermatologists¹; therefore, dermatologist involvement in AI development is necessary to facilitate collection of larger image datasets and optimal performance for image diagnosis/classification tasks.

We found that 20.7% of articles on deep learning models included descriptions of patient ethnicity or race, and only 10.3% of studies included any information about skin tone in the dataset. Furthermore, American investigators primarily trained models using clinical images of patients with lighter skin tones, whereas Chinese investigators exclusively included images depicting darker skin tones. Similarly, in a study of 52 cutaneous imaging deep learning articles, only 17.3% (9/52) reported race and/or Fitzpatrick skin type, and only 7.7% (4/52) of articles included both.^2,6,8 Therefore, dermatologists are needed to contribute images representing diverse populations and collaborate in AI research studies, as their involvement is necessary to ensure the accuracy of AI models in classifying lesions or diagnosing skin lesions across all skin types.

Our search was limited to PubMed, and real-world applications could not be evaluated.

Conclusion

In summary, we found that AI studies with dermatologist authors used larger numbers of clinical images in their datasets and more images representing diverse skin types than studies without. Therefore, we advocate for greater involvement of dermatologists in AI research, which might result in better patient outcomes by improving diagnostic accuracy.

Use of artificial intelligence (AI) in dermatology has increased over the past decade, likely driven by advances in deep learning algorithms, computing hardware, and machine learning.¹ Studies comparing the performance of AI algorithms to dermatologists in classifying skin disorders have shown conflicting results.^2,3 In this study, we aimed to analyze AI tools used for diagnosing and classifying skin disease and evaluate the role of dermatologists in the creation of AI technology. We also investigated the number of clinical images used in datasets to train AI programs and compared tools that were created with dermatologist input to those created without dermatologist/clinician involvement.

Methods

A search of PubMed articles indexed for MEDLINE using the terms machine learning, artificial intelligence, and dermatology was conducted on September 18, 2022. Articles were included if they described full-length trials; used machine learning for diagnosis of or screening for dermatologic conditions; and used dermoscopic or gross image datasets of the skin, hair, or nails. Articles were categorized into 4 groups based on the conditions covered: chronic wounds, inflammatory skin diseases, mixed conditions, and pigmented skin lesions. Algorithms were sorted into 4 categories: convolutional/convoluted neural network, deep learning model/deep neural network, AI/artificial neural network, and other. Details regarding Fitzpatrick skin type and skin of color (SoC) inclusion in the articles or AI algorithm datasets were recorded. Univariate and multivariate analyses were performed using Microsoft Excel and SAS Studio 3.8. Sensitivity and specificity were calculated for all included AI technology. Sensitivity, specificity, and the number of clinical images were compared among the included articles using analysis of variance and t tests (α=0.05; P<.05 indicated statistical significance).

Results

Our search yielded 1016 articles, 58 of which met the inclusion criteria. Overall, 25.9% (15/58) of the articles utilized AI to diagnose or classify mixed skin diseases; 22.4% (13/58) for pigmented skin lesions; 19.0% (11/58) for wounds; 17.2% (10/58) for inflammatory skin diseases; and 5.2% (3/58) each for acne, psoriasis, and onychomycosis. Overall, 24.0% (14/58) of articles provided information about Fitzpatrick skin type, and 58.7% (34/58) included clinical images depicting SoC. Furthermore, we found that only 20.7% (12/58) of articles on deep learning models included descriptions of patient ethnicity or race in at least 1 dataset, and only 10.3% (6/58) of studies included any information about skin tone in the dataset. Studies with a dermatologist as the last author (most likely to be supervising the project) were more likely to include clinical images depicting SoC than those without (82.6% [19/23] and 16.7% [3/18], respectively [P=.0411]).

The mean (SD) number of clinical images in the study articles was 28,422 (84,050). Thirty-seven (63.8%) of the study articles included gross images, 17 (29.3%) used dermoscopic images, and 4 (6.9%) used both. Twenty-seven (46.6%) articles used convolutional/convoluted neural networks, 15 (25.9%) used deep learning model/deep neural networks, 8 (13.8%) used other algorithms, 6 (10.3%) used AI/artificial neural network, and 2 (3.4%) used fuzzy algorithms. Most studies were conducted in China (29.3% [17/58]), Germany (12.1% [7/58]), India (10.3% [6/58]), multiple nations (10.3% [6/58]), and the United States (10.3% [6/58]). Overall, 82.8% (48/58) of articles included at least 1 dermatologist coauthor. Sensitivity of the AI models was 0.85, and specificity was 0.85. The average percentage of images in the dataset correctly identified by a physician was 76.87% vs 81.62% of images correctly identified by AI. Average agreement between AI and physician assessment was 77.98%, defined as AI and physician both having the same diagnosis. 

Articles authored by dermatologists contained more clinical images than those without dermatologists in key authorship roles (P<.0001)(eTable). Psoriasis-related algorithms had the fewest (mean [SD]: 3173 [4203]), and pigmented skin lesions had the most clinical images (mean [SD]: 53,19l [155,579]).

Comment

Our results indicated that AI studies with dermatologist authors had significantly more images in their datasets (ie, the set of clinical images of skin lesions used to train AI algorithms in diagnosing or classifying lesions) than those with nondermatologist authors (P<.0001)(eTable). Similarly, in a study of AI technology for skin cancer diagnosis, AI studies with dermatologist authors (ie, included in the development of the AI algorithm) had more images than studies without dermatologist authors.¹ Deep learning textbooks have suggested that 5000 clinical images or training input per output category are needed to produce acceptable algorithm performance, and more than 10 million are needed to produce results superior to human performance.^4-10 Despite advances in AI for dermatologic image analysis, the creation of these models often has been directed by nondermatologists¹; therefore, dermatologist involvement in AI development is necessary to facilitate collection of larger image datasets and optimal performance for image diagnosis/classification tasks.

We found that 20.7% of articles on deep learning models included descriptions of patient ethnicity or race, and only 10.3% of studies included any information about skin tone in the dataset. Furthermore, American investigators primarily trained models using clinical images of patients with lighter skin tones, whereas Chinese investigators exclusively included images depicting darker skin tones. Similarly, in a study of 52 cutaneous imaging deep learning articles, only 17.3% (9/52) reported race and/or Fitzpatrick skin type, and only 7.7% (4/52) of articles included both.^2,6,8 Therefore, dermatologists are needed to contribute images representing diverse populations and collaborate in AI research studies, as their involvement is necessary to ensure the accuracy of AI models in classifying lesions or diagnosing skin lesions across all skin types.

Our search was limited to PubMed, and real-world applications could not be evaluated.

Conclusion

In summary, we found that AI studies with dermatologist authors used larger numbers of clinical images in their datasets and more images representing diverse skin types than studies without. Therefore, we advocate for greater involvement of dermatologists in AI research, which might result in better patient outcomes by improving diagnostic accuracy.

Use of artificial intelligence (AI) in dermatology has increased over the past decade, likely driven by advances in deep learning algorithms, computing hardware, and machine learning.¹ Studies comparing the performance of AI algorithms to dermatologists in classifying skin disorders have shown conflicting results.^2,3 In this study, we aimed to analyze AI tools used for diagnosing and classifying skin disease and evaluate the role of dermatologists in the creation of AI technology. We also investigated the number of clinical images used in datasets to train AI programs and compared tools that were created with dermatologist input to those created without dermatologist/clinician involvement.

Methods

A search of PubMed articles indexed for MEDLINE using the terms machine learning, artificial intelligence, and dermatology was conducted on September 18, 2022. Articles were included if they described full-length trials; used machine learning for diagnosis of or screening for dermatologic conditions; and used dermoscopic or gross image datasets of the skin, hair, or nails. Articles were categorized into 4 groups based on the conditions covered: chronic wounds, inflammatory skin diseases, mixed conditions, and pigmented skin lesions. Algorithms were sorted into 4 categories: convolutional/convoluted neural network, deep learning model/deep neural network, AI/artificial neural network, and other. Details regarding Fitzpatrick skin type and skin of color (SoC) inclusion in the articles or AI algorithm datasets were recorded. Univariate and multivariate analyses were performed using Microsoft Excel and SAS Studio 3.8. Sensitivity and specificity were calculated for all included AI technology. Sensitivity, specificity, and the number of clinical images were compared among the included articles using analysis of variance and t tests (α=0.05; P<.05 indicated statistical significance).

Results

Our search yielded 1016 articles, 58 of which met the inclusion criteria. Overall, 25.9% (15/58) of the articles utilized AI to diagnose or classify mixed skin diseases; 22.4% (13/58) for pigmented skin lesions; 19.0% (11/58) for wounds; 17.2% (10/58) for inflammatory skin diseases; and 5.2% (3/58) each for acne, psoriasis, and onychomycosis. Overall, 24.0% (14/58) of articles provided information about Fitzpatrick skin type, and 58.7% (34/58) included clinical images depicting SoC. Furthermore, we found that only 20.7% (12/58) of articles on deep learning models included descriptions of patient ethnicity or race in at least 1 dataset, and only 10.3% (6/58) of studies included any information about skin tone in the dataset. Studies with a dermatologist as the last author (most likely to be supervising the project) were more likely to include clinical images depicting SoC than those without (82.6% [19/23] and 16.7% [3/18], respectively [P=.0411]).

The mean (SD) number of clinical images in the study articles was 28,422 (84,050). Thirty-seven (63.8%) of the study articles included gross images, 17 (29.3%) used dermoscopic images, and 4 (6.9%) used both. Twenty-seven (46.6%) articles used convolutional/convoluted neural networks, 15 (25.9%) used deep learning model/deep neural networks, 8 (13.8%) used other algorithms, 6 (10.3%) used AI/artificial neural network, and 2 (3.4%) used fuzzy algorithms. Most studies were conducted in China (29.3% [17/58]), Germany (12.1% [7/58]), India (10.3% [6/58]), multiple nations (10.3% [6/58]), and the United States (10.3% [6/58]). Overall, 82.8% (48/58) of articles included at least 1 dermatologist coauthor. Sensitivity of the AI models was 0.85, and specificity was 0.85. The average percentage of images in the dataset correctly identified by a physician was 76.87% vs 81.62% of images correctly identified by AI. Average agreement between AI and physician assessment was 77.98%, defined as AI and physician both having the same diagnosis. 

Articles authored by dermatologists contained more clinical images than those without dermatologists in key authorship roles (P<.0001)(eTable). Psoriasis-related algorithms had the fewest (mean [SD]: 3173 [4203]), and pigmented skin lesions had the most clinical images (mean [SD]: 53,19l [155,579]).

Comment

Our results indicated that AI studies with dermatologist authors had significantly more images in their datasets (ie, the set of clinical images of skin lesions used to train AI algorithms in diagnosing or classifying lesions) than those with nondermatologist authors (P<.0001)(eTable). Similarly, in a study of AI technology for skin cancer diagnosis, AI studies with dermatologist authors (ie, included in the development of the AI algorithm) had more images than studies without dermatologist authors.¹ Deep learning textbooks have suggested that 5000 clinical images or training input per output category are needed to produce acceptable algorithm performance, and more than 10 million are needed to produce results superior to human performance.^4-10 Despite advances in AI for dermatologic image analysis, the creation of these models often has been directed by nondermatologists¹; therefore, dermatologist involvement in AI development is necessary to facilitate collection of larger image datasets and optimal performance for image diagnosis/classification tasks.

We found that 20.7% of articles on deep learning models included descriptions of patient ethnicity or race, and only 10.3% of studies included any information about skin tone in the dataset. Furthermore, American investigators primarily trained models using clinical images of patients with lighter skin tones, whereas Chinese investigators exclusively included images depicting darker skin tones. Similarly, in a study of 52 cutaneous imaging deep learning articles, only 17.3% (9/52) reported race and/or Fitzpatrick skin type, and only 7.7% (4/52) of articles included both.^2,6,8 Therefore, dermatologists are needed to contribute images representing diverse populations and collaborate in AI research studies, as their involvement is necessary to ensure the accuracy of AI models in classifying lesions or diagnosing skin lesions across all skin types.

Our search was limited to PubMed, and real-world applications could not be evaluated.

Conclusion

In summary, we found that AI studies with dermatologist authors used larger numbers of clinical images in their datasets and more images representing diverse skin types than studies without. Therefore, we advocate for greater involvement of dermatologists in AI research, which might result in better patient outcomes by improving diagnostic accuracy.

Nurse practitioners (NPs) and physician assistants (PAs) often help provide dermatologic care but lack the same mandatory specialized postgraduate training required of board-certified dermatologists (BCDs), which includes at least 3 years of dermatology-focused education in an accredited residency program in addition to an intern year of general medicine, pediatrics, or surgery. Dermatology residency is followed by a certification examination administered by the American Board of Dermatology (ABD) or the American Osteopathic Board of Dermatology, leading to board certification. Some physicians choose to do a fellowship, which typically involves an additional 1 to 2 years of postresidency subspeciality training.

Optional postgraduate dermatology training programs for advanced practice providers (APPs) have been offered by some academic institutions and private practice groups since at least 2003, including Lahey Hospital and Medical Center (Burlington, Massachusetts) as well as the University of Rochester Medical Center (Rochester, New York). Despite a lack of accreditation or standardization, the programs can be beneficial for NPs and PAs to expand their dermatologic knowledge and skills and help bridge the care gap within the specialty. Didactics often are conducted in parallel with the educational activities of the parent institution’s traditional dermatology residency program (eg, lectures, grand rounds). While these programs often are managed by practicing dermatology NPs and PAs, dermatologists also may be involved in their education with didactic instruction, curriculum development, and clinical preceptorship. 

In this cross-sectional study, we identified and evaluated 10 postgraduate dermatology training programs for APPs across the United States. With the growing number of NPs and PAs in the dermatology workforce—both in academic and private practice—it is important for BCDs to be aware of the differences in the dermatology training received in order to ensure safe and effective care is provided through supervisory or collaborative roles (depending on state independent practice laws for APPs and to be aware of the implications these programs may have on the field of dermatology.

Methods

To identify postgraduate dermatology training programs for APPs in the United States, we conducted a cross-sectional study using data obtained via a Google search of various combinations of the following terms: nurse practitioner, NP, physician assistant, PA, advance practice provider, APP, dermatology, postgraduate training, residency, and fellowship. We excluded postgraduate dermatology training programs for APPs that required tuition and did not provide a stipend, as well as programs that lacked the formal structure and credibility needed to qualify as legitimate postgraduate training. Many of the excluded programs operate in a manner that raises ethical concerns, offering pay-to-play opportunities under the guise of education. Information collected on each program included the program name, location, parent institution, program length, class size, curriculum, and any associated salary and benefits.

Results

Ten academic and private practice organizations across the United States that offer postgraduate dermatologic training programs for APPs were identified (eTable). Four (40%) programs were advertised as fellowships. Six (60%) of the programs were offered at academic medical centers, and 4 (40%) were offered by private practices. Most programs were located east of the Mississippi River, and many institutions offered instruction at 1 or more locations within the same state (eFigure). The Advanced Dermatology and Cosmetic Surgery private practice group offered training opportunities in multiple states.

Six programs required APPs to become board-certified NPs or PAs prior to enrolling. Most programs enrolled both NPs and PAs, while some only enrolled NPs (eTable). Only 1 (10%) program required NPs to be board certified as a family NP, while another (10%) recommended that applicants have experience in urgent care, emergency medicine, or trauma medicine. Lahey Hospital & Medical Center required experience as an NP in a general setting for 1 to 2 years prior to applying. No program required prior experience in the field of dermatology.

Program length varied from 6 to 24 months, and cohort size typically was limited to 1 to 2 providers (eTable). Although the exact numbers could not be ascertained, most curricula focused on medical dermatology, including clinical and didactic components, but many offered electives such as cosmetic and procedural dermatology. Two institutions (20%) required independent research. Work typically was limited to 40 hours per week, and most paid a full-time employee salary and provided benefits such as health insurance, retirement, and paid leave (eTable). Kansas Medical Clinic (Topeka, Kansas) required at least 3 years of employment in an underserved community following program completion. The Oasis Dermatology private practice group in Texas required a 1-year teaching commitment after program completion. The Advanced Dermatology and Cosmetic Surgery group offered a full-time position upon program completion.

Comment

There is a large difference in the total number of training and credentialing hours when comparing graduate school training and postgraduate credentialing of medical and osteopathic physicians compared with APPs. A new graduate physician has at least twice as many clinical hours as a PA and 10 times as many clinical hours as an NP prior to starting residency. Physicians also typically complete at least 6 times the number of hours of certification examinations compared to NPs and PAs.¹

Nurse practitioner students typically complete the 500 hours of prelicensure clinical training required for NP school in 2 to 4 years.^2,3 The amount of time required for completion is dependent on the degree and experience of the student upon program entry (eg, bachelor of science in nursing vs master of science in nursing as a terminal degree). Physician assistant students are required to complete 2000 prelicensure clinical hours, and most PA programs are 3 years in duration.⁴ Many NP and PA programs require some degree of clinical experience prior to beginning graduate education.⁵

When comparing prelicensure examinations, questions assessing dermatologic knowledge comprise approximately 6% to 10% of the total questions on the United States Medical Licensing Examination Steps 1 and 2.⁶ The Comprehensive Osteopathic Medical Licensing Examination of the United States Level 1 and Level 2-Cognitive Evaluation both have at least 5% of questions dedicated to dermatology.⁷ Approximately 5% of the questions on the Physician Assistant National Certifying Examination are dedicated to dermatology.⁸ The dermatology content on either of the NP certification examinations is unclear.^2,3 In the states of California, Indiana, and New York, national certification through the American Association of Nurse Practitioners or American Nurses Credentialing Center is not required for NPs to practice in their respective states.⁹

Regarding dermatologic board certification, a new graduate NP may obtain certification from the Dermatology Nurse Practitioner Certification Board with 3000 hours of general dermatology practice that may occur during normal working hours.¹⁰ These hours do not have to occur in one of the previously identified postgraduate APP training programs. The National Board of Dermatology Physician Assistants was founded in 2018 and has since dissolved. The National Board of Dermatology Physician Assistants was not accredited and required at least 3 years of training in dermatology with the same dermatologist in addition to completing a 125-question multiple-choice examination.¹¹ Of note, this examination was opposed by both the ABD and the Society for Dermatology Physician Associates.¹² A PA also may become a Diplomate Fellow with the Society of Dermatology Physician Associates after completion of 64.5 hours of online continuing education modules.⁴ Some PAs may choose to obtain a Certificate of Added Qualifications, which is a voluntary credential that helps document specialty experience and expertise in dermatology or other specialties.

In contrast, a dermatology resident physician requires nearly 11,000 to 13,000 hours of clinical training hours, which last 3 to 4 years following medical school.¹³ This training involves direct patient care under supervision in various settings, including hospitals, outpatient clinics, and surgical procedures. In addition to this clinical experience, dermatology residents must pass a 3-step certification examination process administered by the ABD.¹³ This process includes approximately 20 hours of examinations designed to assess both knowledge and practical skills. For those who wish to further specialize, additional fellowship training in areas such as pediatric dermatology, dermatopathology, or Mohs surgery may follow residency; such fellowships involve an extra 2500 to 3500 hours of training and culminate in another certification examination, further refining a resident’s expertise in a specific dermatologic field. Osteopathic physicians may opt out of the ABD 3-step pathway and obtain board certification through the American Osteopathic Board of Dermatology.¹⁴

Many of the programs we evaluated integrate APP trainees into resident education, allowing participation in equivalent didactic curricula, clinical rotations, and departmental academic activities. The salary and benefits associated with these programs are somewhat like those of resident physicians.^15,16 While most tuition-based programs were excluded from our study due to their lack of credibility and alignment with our study criteria, we identified 2 specific programs that stood out as credible despite requiring students to pay tuition. These programs demonstrated a structured and rigorous curriculum with a clear focus on comprehensive dermatologic training, meeting our standards for inclusion. These programs offer dermatologic training for graduates of NP and PA programs at a cost to the student.^15,16 The program at the Florida Atlantic University, Boca Raton, is largely online,¹⁵ and the program at the University of Miami, Florida, offers no direct clinical contact.¹⁶ These programs illustrate the variety of postgraduate dermatology curricula available nationally in comparison to resident salaries; however, they were not included in our formal analysis because they do not provide structured, in-person clinical training consistent with our inclusion criteria. Neither of these programs would enable participants to qualify for credentialing with the Dermatology Nurse Practitioner Certification Board after completion. While this study identified postgraduate training programs for APPs in dermatology advertised online, it is possible some were omitted or not advertised online.

While many of the postgraduate programs we evaluated provide unique educational opportunities for APPs, it is unknown if graduating providers are equipped to handle the care of patients with complex dermatologic needs. Regardless, the increased utilization of APPs by BCDs has been well documented over the past 2 decades.^17-20 It has been suggested that a higher ratio of APPs to dermatologists can decrease the time it takes for a patient to be seen in a clinic.^21-23 However, investigators have expressed concerns that APPs lack standardized surgical training and clinical hour requirements in the field of dermatology.²⁴ Despite these concerns, Medicare claims data show that APPs are performing advanced surgical and cosmetic procedures at increasing rates.^17,18 Other authors have questioned the cost-effectiveness of APPs, as multiple studies have shown that the number of biopsies needed to diagnose 1 case of skin cancer is higher for midlevel providers than for dermatologists.^25-27

Conclusion

With the anticipated expansion of private equity in dermatology and the growth of our Medicare-eligible population, we are likely to see increased utilization of APPs to address the shortage of BCDs.^28,29 Understanding the prelicensure and postlicensure clinical training requirements, examination hours, and extent of dermatology-focused education among APPs and BCDs can help dermatologists collaborate more effectively and ensure safe, high-quality patient care. Standardizing, improving, and providing high-quality education and promoting lifelong learning in the field of dermatology should be celebrated, and dermatologists are the skin experts best equipped to lead dermatologic education forward.

Nurse practitioners (NPs) and physician assistants (PAs) often help provide dermatologic care but lack the same mandatory specialized postgraduate training required of board-certified dermatologists (BCDs), which includes at least 3 years of dermatology-focused education in an accredited residency program in addition to an intern year of general medicine, pediatrics, or surgery. Dermatology residency is followed by a certification examination administered by the American Board of Dermatology (ABD) or the American Osteopathic Board of Dermatology, leading to board certification. Some physicians choose to do a fellowship, which typically involves an additional 1 to 2 years of postresidency subspeciality training.

Optional postgraduate dermatology training programs for advanced practice providers (APPs) have been offered by some academic institutions and private practice groups since at least 2003, including Lahey Hospital and Medical Center (Burlington, Massachusetts) as well as the University of Rochester Medical Center (Rochester, New York). Despite a lack of accreditation or standardization, the programs can be beneficial for NPs and PAs to expand their dermatologic knowledge and skills and help bridge the care gap within the specialty. Didactics often are conducted in parallel with the educational activities of the parent institution’s traditional dermatology residency program (eg, lectures, grand rounds). While these programs often are managed by practicing dermatology NPs and PAs, dermatologists also may be involved in their education with didactic instruction, curriculum development, and clinical preceptorship. 

In this cross-sectional study, we identified and evaluated 10 postgraduate dermatology training programs for APPs across the United States. With the growing number of NPs and PAs in the dermatology workforce—both in academic and private practice—it is important for BCDs to be aware of the differences in the dermatology training received in order to ensure safe and effective care is provided through supervisory or collaborative roles (depending on state independent practice laws for APPs and to be aware of the implications these programs may have on the field of dermatology.

Methods

To identify postgraduate dermatology training programs for APPs in the United States, we conducted a cross-sectional study using data obtained via a Google search of various combinations of the following terms: nurse practitioner, NP, physician assistant, PA, advance practice provider, APP, dermatology, postgraduate training, residency, and fellowship. We excluded postgraduate dermatology training programs for APPs that required tuition and did not provide a stipend, as well as programs that lacked the formal structure and credibility needed to qualify as legitimate postgraduate training. Many of the excluded programs operate in a manner that raises ethical concerns, offering pay-to-play opportunities under the guise of education. Information collected on each program included the program name, location, parent institution, program length, class size, curriculum, and any associated salary and benefits.

Results

Ten academic and private practice organizations across the United States that offer postgraduate dermatologic training programs for APPs were identified (eTable). Four (40%) programs were advertised as fellowships. Six (60%) of the programs were offered at academic medical centers, and 4 (40%) were offered by private practices. Most programs were located east of the Mississippi River, and many institutions offered instruction at 1 or more locations within the same state (eFigure). The Advanced Dermatology and Cosmetic Surgery private practice group offered training opportunities in multiple states.

Six programs required APPs to become board-certified NPs or PAs prior to enrolling. Most programs enrolled both NPs and PAs, while some only enrolled NPs (eTable). Only 1 (10%) program required NPs to be board certified as a family NP, while another (10%) recommended that applicants have experience in urgent care, emergency medicine, or trauma medicine. Lahey Hospital & Medical Center required experience as an NP in a general setting for 1 to 2 years prior to applying. No program required prior experience in the field of dermatology.

Program length varied from 6 to 24 months, and cohort size typically was limited to 1 to 2 providers (eTable). Although the exact numbers could not be ascertained, most curricula focused on medical dermatology, including clinical and didactic components, but many offered electives such as cosmetic and procedural dermatology. Two institutions (20%) required independent research. Work typically was limited to 40 hours per week, and most paid a full-time employee salary and provided benefits such as health insurance, retirement, and paid leave (eTable). Kansas Medical Clinic (Topeka, Kansas) required at least 3 years of employment in an underserved community following program completion. The Oasis Dermatology private practice group in Texas required a 1-year teaching commitment after program completion. The Advanced Dermatology and Cosmetic Surgery group offered a full-time position upon program completion.

Comment

There is a large difference in the total number of training and credentialing hours when comparing graduate school training and postgraduate credentialing of medical and osteopathic physicians compared with APPs. A new graduate physician has at least twice as many clinical hours as a PA and 10 times as many clinical hours as an NP prior to starting residency. Physicians also typically complete at least 6 times the number of hours of certification examinations compared to NPs and PAs.¹

Nurse practitioner students typically complete the 500 hours of prelicensure clinical training required for NP school in 2 to 4 years.^2,3 The amount of time required for completion is dependent on the degree and experience of the student upon program entry (eg, bachelor of science in nursing vs master of science in nursing as a terminal degree). Physician assistant students are required to complete 2000 prelicensure clinical hours, and most PA programs are 3 years in duration.⁴ Many NP and PA programs require some degree of clinical experience prior to beginning graduate education.⁵

When comparing prelicensure examinations, questions assessing dermatologic knowledge comprise approximately 6% to 10% of the total questions on the United States Medical Licensing Examination Steps 1 and 2.⁶ The Comprehensive Osteopathic Medical Licensing Examination of the United States Level 1 and Level 2-Cognitive Evaluation both have at least 5% of questions dedicated to dermatology.⁷ Approximately 5% of the questions on the Physician Assistant National Certifying Examination are dedicated to dermatology.⁸ The dermatology content on either of the NP certification examinations is unclear.^2,3 In the states of California, Indiana, and New York, national certification through the American Association of Nurse Practitioners or American Nurses Credentialing Center is not required for NPs to practice in their respective states.⁹

Regarding dermatologic board certification, a new graduate NP may obtain certification from the Dermatology Nurse Practitioner Certification Board with 3000 hours of general dermatology practice that may occur during normal working hours.¹⁰ These hours do not have to occur in one of the previously identified postgraduate APP training programs. The National Board of Dermatology Physician Assistants was founded in 2018 and has since dissolved. The National Board of Dermatology Physician Assistants was not accredited and required at least 3 years of training in dermatology with the same dermatologist in addition to completing a 125-question multiple-choice examination.¹¹ Of note, this examination was opposed by both the ABD and the Society for Dermatology Physician Associates.¹² A PA also may become a Diplomate Fellow with the Society of Dermatology Physician Associates after completion of 64.5 hours of online continuing education modules.⁴ Some PAs may choose to obtain a Certificate of Added Qualifications, which is a voluntary credential that helps document specialty experience and expertise in dermatology or other specialties.

In contrast, a dermatology resident physician requires nearly 11,000 to 13,000 hours of clinical training hours, which last 3 to 4 years following medical school.¹³ This training involves direct patient care under supervision in various settings, including hospitals, outpatient clinics, and surgical procedures. In addition to this clinical experience, dermatology residents must pass a 3-step certification examination process administered by the ABD.¹³ This process includes approximately 20 hours of examinations designed to assess both knowledge and practical skills. For those who wish to further specialize, additional fellowship training in areas such as pediatric dermatology, dermatopathology, or Mohs surgery may follow residency; such fellowships involve an extra 2500 to 3500 hours of training and culminate in another certification examination, further refining a resident’s expertise in a specific dermatologic field. Osteopathic physicians may opt out of the ABD 3-step pathway and obtain board certification through the American Osteopathic Board of Dermatology.¹⁴

Many of the programs we evaluated integrate APP trainees into resident education, allowing participation in equivalent didactic curricula, clinical rotations, and departmental academic activities. The salary and benefits associated with these programs are somewhat like those of resident physicians.^15,16 While most tuition-based programs were excluded from our study due to their lack of credibility and alignment with our study criteria, we identified 2 specific programs that stood out as credible despite requiring students to pay tuition. These programs demonstrated a structured and rigorous curriculum with a clear focus on comprehensive dermatologic training, meeting our standards for inclusion. These programs offer dermatologic training for graduates of NP and PA programs at a cost to the student.^15,16 The program at the Florida Atlantic University, Boca Raton, is largely online,¹⁵ and the program at the University of Miami, Florida, offers no direct clinical contact.¹⁶ These programs illustrate the variety of postgraduate dermatology curricula available nationally in comparison to resident salaries; however, they were not included in our formal analysis because they do not provide structured, in-person clinical training consistent with our inclusion criteria. Neither of these programs would enable participants to qualify for credentialing with the Dermatology Nurse Practitioner Certification Board after completion. While this study identified postgraduate training programs for APPs in dermatology advertised online, it is possible some were omitted or not advertised online.

While many of the postgraduate programs we evaluated provide unique educational opportunities for APPs, it is unknown if graduating providers are equipped to handle the care of patients with complex dermatologic needs. Regardless, the increased utilization of APPs by BCDs has been well documented over the past 2 decades.^17-20 It has been suggested that a higher ratio of APPs to dermatologists can decrease the time it takes for a patient to be seen in a clinic.^21-23 However, investigators have expressed concerns that APPs lack standardized surgical training and clinical hour requirements in the field of dermatology.²⁴ Despite these concerns, Medicare claims data show that APPs are performing advanced surgical and cosmetic procedures at increasing rates.^17,18 Other authors have questioned the cost-effectiveness of APPs, as multiple studies have shown that the number of biopsies needed to diagnose 1 case of skin cancer is higher for midlevel providers than for dermatologists.^25-27

Conclusion

With the anticipated expansion of private equity in dermatology and the growth of our Medicare-eligible population, we are likely to see increased utilization of APPs to address the shortage of BCDs.^28,29 Understanding the prelicensure and postlicensure clinical training requirements, examination hours, and extent of dermatology-focused education among APPs and BCDs can help dermatologists collaborate more effectively and ensure safe, high-quality patient care. Standardizing, improving, and providing high-quality education and promoting lifelong learning in the field of dermatology should be celebrated, and dermatologists are the skin experts best equipped to lead dermatologic education forward.

Nurse practitioners (NPs) and physician assistants (PAs) often help provide dermatologic care but lack the same mandatory specialized postgraduate training required of board-certified dermatologists (BCDs), which includes at least 3 years of dermatology-focused education in an accredited residency program in addition to an intern year of general medicine, pediatrics, or surgery. Dermatology residency is followed by a certification examination administered by the American Board of Dermatology (ABD) or the American Osteopathic Board of Dermatology, leading to board certification. Some physicians choose to do a fellowship, which typically involves an additional 1 to 2 years of postresidency subspeciality training.

Optional postgraduate dermatology training programs for advanced practice providers (APPs) have been offered by some academic institutions and private practice groups since at least 2003, including Lahey Hospital and Medical Center (Burlington, Massachusetts) as well as the University of Rochester Medical Center (Rochester, New York). Despite a lack of accreditation or standardization, the programs can be beneficial for NPs and PAs to expand their dermatologic knowledge and skills and help bridge the care gap within the specialty. Didactics often are conducted in parallel with the educational activities of the parent institution’s traditional dermatology residency program (eg, lectures, grand rounds). While these programs often are managed by practicing dermatology NPs and PAs, dermatologists also may be involved in their education with didactic instruction, curriculum development, and clinical preceptorship. 

In this cross-sectional study, we identified and evaluated 10 postgraduate dermatology training programs for APPs across the United States. With the growing number of NPs and PAs in the dermatology workforce—both in academic and private practice—it is important for BCDs to be aware of the differences in the dermatology training received in order to ensure safe and effective care is provided through supervisory or collaborative roles (depending on state independent practice laws for APPs and to be aware of the implications these programs may have on the field of dermatology.

Methods

To identify postgraduate dermatology training programs for APPs in the United States, we conducted a cross-sectional study using data obtained via a Google search of various combinations of the following terms: nurse practitioner, NP, physician assistant, PA, advance practice provider, APP, dermatology, postgraduate training, residency, and fellowship. We excluded postgraduate dermatology training programs for APPs that required tuition and did not provide a stipend, as well as programs that lacked the formal structure and credibility needed to qualify as legitimate postgraduate training. Many of the excluded programs operate in a manner that raises ethical concerns, offering pay-to-play opportunities under the guise of education. Information collected on each program included the program name, location, parent institution, program length, class size, curriculum, and any associated salary and benefits.

Results

Ten academic and private practice organizations across the United States that offer postgraduate dermatologic training programs for APPs were identified (eTable). Four (40%) programs were advertised as fellowships. Six (60%) of the programs were offered at academic medical centers, and 4 (40%) were offered by private practices. Most programs were located east of the Mississippi River, and many institutions offered instruction at 1 or more locations within the same state (eFigure). The Advanced Dermatology and Cosmetic Surgery private practice group offered training opportunities in multiple states.

Six programs required APPs to become board-certified NPs or PAs prior to enrolling. Most programs enrolled both NPs and PAs, while some only enrolled NPs (eTable). Only 1 (10%) program required NPs to be board certified as a family NP, while another (10%) recommended that applicants have experience in urgent care, emergency medicine, or trauma medicine. Lahey Hospital & Medical Center required experience as an NP in a general setting for 1 to 2 years prior to applying. No program required prior experience in the field of dermatology.

Program length varied from 6 to 24 months, and cohort size typically was limited to 1 to 2 providers (eTable). Although the exact numbers could not be ascertained, most curricula focused on medical dermatology, including clinical and didactic components, but many offered electives such as cosmetic and procedural dermatology. Two institutions (20%) required independent research. Work typically was limited to 40 hours per week, and most paid a full-time employee salary and provided benefits such as health insurance, retirement, and paid leave (eTable). Kansas Medical Clinic (Topeka, Kansas) required at least 3 years of employment in an underserved community following program completion. The Oasis Dermatology private practice group in Texas required a 1-year teaching commitment after program completion. The Advanced Dermatology and Cosmetic Surgery group offered a full-time position upon program completion.

Comment

There is a large difference in the total number of training and credentialing hours when comparing graduate school training and postgraduate credentialing of medical and osteopathic physicians compared with APPs. A new graduate physician has at least twice as many clinical hours as a PA and 10 times as many clinical hours as an NP prior to starting residency. Physicians also typically complete at least 6 times the number of hours of certification examinations compared to NPs and PAs.¹

Nurse practitioner students typically complete the 500 hours of prelicensure clinical training required for NP school in 2 to 4 years.^2,3 The amount of time required for completion is dependent on the degree and experience of the student upon program entry (eg, bachelor of science in nursing vs master of science in nursing as a terminal degree). Physician assistant students are required to complete 2000 prelicensure clinical hours, and most PA programs are 3 years in duration.⁴ Many NP and PA programs require some degree of clinical experience prior to beginning graduate education.⁵

When comparing prelicensure examinations, questions assessing dermatologic knowledge comprise approximately 6% to 10% of the total questions on the United States Medical Licensing Examination Steps 1 and 2.⁶ The Comprehensive Osteopathic Medical Licensing Examination of the United States Level 1 and Level 2-Cognitive Evaluation both have at least 5% of questions dedicated to dermatology.⁷ Approximately 5% of the questions on the Physician Assistant National Certifying Examination are dedicated to dermatology.⁸ The dermatology content on either of the NP certification examinations is unclear.^2,3 In the states of California, Indiana, and New York, national certification through the American Association of Nurse Practitioners or American Nurses Credentialing Center is not required for NPs to practice in their respective states.⁹

Regarding dermatologic board certification, a new graduate NP may obtain certification from the Dermatology Nurse Practitioner Certification Board with 3000 hours of general dermatology practice that may occur during normal working hours.¹⁰ These hours do not have to occur in one of the previously identified postgraduate APP training programs. The National Board of Dermatology Physician Assistants was founded in 2018 and has since dissolved. The National Board of Dermatology Physician Assistants was not accredited and required at least 3 years of training in dermatology with the same dermatologist in addition to completing a 125-question multiple-choice examination.¹¹ Of note, this examination was opposed by both the ABD and the Society for Dermatology Physician Associates.¹² A PA also may become a Diplomate Fellow with the Society of Dermatology Physician Associates after completion of 64.5 hours of online continuing education modules.⁴ Some PAs may choose to obtain a Certificate of Added Qualifications, which is a voluntary credential that helps document specialty experience and expertise in dermatology or other specialties.

In contrast, a dermatology resident physician requires nearly 11,000 to 13,000 hours of clinical training hours, which last 3 to 4 years following medical school.¹³ This training involves direct patient care under supervision in various settings, including hospitals, outpatient clinics, and surgical procedures. In addition to this clinical experience, dermatology residents must pass a 3-step certification examination process administered by the ABD.¹³ This process includes approximately 20 hours of examinations designed to assess both knowledge and practical skills. For those who wish to further specialize, additional fellowship training in areas such as pediatric dermatology, dermatopathology, or Mohs surgery may follow residency; such fellowships involve an extra 2500 to 3500 hours of training and culminate in another certification examination, further refining a resident’s expertise in a specific dermatologic field. Osteopathic physicians may opt out of the ABD 3-step pathway and obtain board certification through the American Osteopathic Board of Dermatology.¹⁴

Many of the programs we evaluated integrate APP trainees into resident education, allowing participation in equivalent didactic curricula, clinical rotations, and departmental academic activities. The salary and benefits associated with these programs are somewhat like those of resident physicians.^15,16 While most tuition-based programs were excluded from our study due to their lack of credibility and alignment with our study criteria, we identified 2 specific programs that stood out as credible despite requiring students to pay tuition. These programs demonstrated a structured and rigorous curriculum with a clear focus on comprehensive dermatologic training, meeting our standards for inclusion. These programs offer dermatologic training for graduates of NP and PA programs at a cost to the student.^15,16 The program at the Florida Atlantic University, Boca Raton, is largely online,¹⁵ and the program at the University of Miami, Florida, offers no direct clinical contact.¹⁶ These programs illustrate the variety of postgraduate dermatology curricula available nationally in comparison to resident salaries; however, they were not included in our formal analysis because they do not provide structured, in-person clinical training consistent with our inclusion criteria. Neither of these programs would enable participants to qualify for credentialing with the Dermatology Nurse Practitioner Certification Board after completion. While this study identified postgraduate training programs for APPs in dermatology advertised online, it is possible some were omitted or not advertised online.

While many of the postgraduate programs we evaluated provide unique educational opportunities for APPs, it is unknown if graduating providers are equipped to handle the care of patients with complex dermatologic needs. Regardless, the increased utilization of APPs by BCDs has been well documented over the past 2 decades.^17-20 It has been suggested that a higher ratio of APPs to dermatologists can decrease the time it takes for a patient to be seen in a clinic.^21-23 However, investigators have expressed concerns that APPs lack standardized surgical training and clinical hour requirements in the field of dermatology.²⁴ Despite these concerns, Medicare claims data show that APPs are performing advanced surgical and cosmetic procedures at increasing rates.^17,18 Other authors have questioned the cost-effectiveness of APPs, as multiple studies have shown that the number of biopsies needed to diagnose 1 case of skin cancer is higher for midlevel providers than for dermatologists.^25-27

Conclusion

With the anticipated expansion of private equity in dermatology and the growth of our Medicare-eligible population, we are likely to see increased utilization of APPs to address the shortage of BCDs.^28,29 Understanding the prelicensure and postlicensure clinical training requirements, examination hours, and extent of dermatology-focused education among APPs and BCDs can help dermatologists collaborate more effectively and ensure safe, high-quality patient care. Standardizing, improving, and providing high-quality education and promoting lifelong learning in the field of dermatology should be celebrated, and dermatologists are the skin experts best equipped to lead dermatologic education forward.

Trauma-focused evidence-based psychotherapies (TF-EBPs), including cognitive processing therapy (CPT) and prolonged exposure therapy (PE), are recommended treatments for posttraumatic stress disorder (PTSD) in clinical practice guidelines.^1-3 To increase initiation of these treatments, the US Department of Veterans Affairs (VA) used a large-scale dissemination and implementation effort to improve access to TF-EBP.^4,5 These efforts achieved modest success, increasing prevalence of TF-EBP from a handful of veterans in 2004 to an annual prevalence of 14.6% for CPT and 4.3% for PE in 2014.⁶

Throughout these efforts, qualitative studies have been used to better understand veterans’ perspectives on receiving TF-EBP care.^7-18 Barriers to initiation of and engagement in TF-EBP and PTSD care have been identified from these qualitative studies. One identified barrier was lack of knowledge—particularly lack of knowledge about what is meant by a PTSD diagnosis and available treatments.^7-10 Stigma (ie, automatic negative associations) toward mental health problems or seeking mental health care also has been identified as a barrier to initiation.^7,10-14 Perceptions of poor alignment between treatment and veteran goals, including lack of buy-in for the rationale, served as barriers to initiation and engagement.^8,15-18

Using prior qualitative work, numerous initiatives have been developed to reduce stigma, facilitate conversations about how treatment aligns with goals, and fill knowledge gaps, particularly through online resources and shared decision-making.^19,20 To better inform the state of veterans’ experiences with TF-EBP, a qualitative investigation was conducted involving veterans who recently initiated TF-EBP. Themes directly related to transitions to TF-EBP were identified; however, all veterans interviewed also described their experiences with TFEBP engagement and mental health care. Consistent with recommendations for qualitative methods, this study extends prior work on transitions to TF-EBP by describing themes with a distinct focus on the experience of engaging with TF-EBP and mental health care.^21,22

Methods

The experiences of veterans who were transitioning into TF-EBPs were collected in semistructured interviews and analyzed. The semistructured interview guide was developed and refined in consultation with both qualitative methods experts and PTSD treatment experts to ensure that 6 content domains were appropriately queried: PTSD treatment options, cultural sensitivity of treatment, PTSD treatment selection, transition criteria, beliefs about stabilization treatment, and treatment needs/preferences.

Participants were identified using the VA Corporate Data Warehouse and included post-9/11 veterans who had recently initiated CPT or PE for the first time between September 1, 2021, and September 1, 2022. More details of participant selection are available in Holder et al.²¹ From a population of 10,814 patients, stratified random sampling generated a recruitment pool of 200 veterans for further outreach. The strata were defined such that this recruitment pool had similar proportions of demographic characteristics (ie, gender, race, ethnicity) to the population of eligible veterans, equivalent distributions of time to CPT or PE initiation (ie, 33.3% < 1 year, 33.3% 1-3 years, and 33.3% > 3 years), and adequate variability in TF-EBP type (ie, 66.7% CPT, 33.3% PE). A manual chart review in the recruitment pool excluded 12 veterans who did not initiate CPT or PE, 1 veteran with evidence of current active psychosis and/or cognitive impairment that would likely preclude comprehension of study materials, and 1 who was deceased.

Eligible veterans from the recruitment pool were contacted in groups of 25. First, a recruitment letter with study information and instructions to opt-out of further contact was mailed or emailed to veterans. After 2 weeks, veterans who had not responded were contacted by phone up to 3 times. Veterans interested in participating were scheduled for a 1-time visit that included verbal consent and the qualitative interview. Metrics were established a priori to ensure an adequately diverse and inclusive sample. Specifically, a minimum number of racial and/or ethnic minority veterans (33%) and women veterans (20%) were sought. Equal distribution across the 3 categories of time from first mental health visit to CPT/PE initiation also was targeted. Throughout enrollment, recruitment efforts were adapted to meet these metrics in the emerging sample. While the goal was to generate a diverse and inclusive sample using these methods, the sample was not intended to be representative of the population.

Of the 186 eligible participants, 21 declined participation and 26 could not be reached. The targeted sample was reached after exhausting contact for 47 veterans and contacting 80 veterans for a final response rate of 40% among fully contacted veterans and 27% among veterans with any contact. The final sample included 30 veterans who received CPT or PE in VA facilities (Table).

After veterans provided verbal consent for study participation, sociodemographic information was verbally reported, and a 30- to 60-minute semistructured qualitative phone interview was recorded and transcribed. Veterans received $40 for participation. All procedures were approved by the University of California San Francisco Institutional Review Board.

Qualitative Data Analysis

Rapid analysis procedures were used to analyze qualitative data. This approach is suitable for focused, moderately structured qualitative analyses in health services research and facilitates rapid dissemination to stakeholders.²³ The qualitative analysts were 2 clinical psychologists with expertise in PTSD treatment (NH primary and RR secondary). Consistent with rapid analysis procedures, analysts prepared a templated summary (including relevant quotations) of each interview, organized by the prespecified content domains. Interviews were summarized independently, compared to ensure consistency, and discrepancies were resolved through review of the interview source materials. Individual summary templates were combined into a master analytic matrix to facilitate the identification of patterns and delineation of themes. Analysts routinely met to identify, discuss, and refine preliminary themes, revisiting source materials to reach consensus as needed.

Results

Fifteen themes were identified and organized into 2 distinct focus areas: themes directly related to the transition to TF-EBP (8 themes) and themes related to veterans’ experiences with TF-EBP and general mental health care with potential process-improvement implications (7 themes).²¹ Seven themes were identified related to experiences with TF-EBP engagement and VA mental health care. The 7 themes related to TF-EBP engagement and VA mental health care themes are summarized with exemplary quotations.

Veterans want a better understanding of psychotherapy and engaging with VA mental health. Veterans reported that they generally had a poor or “nebulous” understanding about the experience of psychotherapy. For example, veterans exhibited confusion about whether certain experiences were equivalent to participating in psychotherapy. They were sometimes unable to distinguish between interactions such as assessment, disability evaluations, peer support, and psychotherapy. One veteran described a conversation with a TFEBP therapist about prior treatment:

She [asked], have you ever been, or gone through a therapy to begin with? And I, I said, well I just chatted with somebody. And she said that’s not, that’s not therapy. So, I was like, oh, it’s not? That’s not what people do?

Veterans were surprised the VA offered a diverse range of psychotherapy interventions, rather than simply therapy. They did not realize there were different types of psychotherapy. As a result, veterans were not aware that some VA mental practitioners have specialty training and certification to provide treatment matched to specific diagnoses or needs. They thought that all clinicians could provide the same care. One veteran described their understanding:

I just figured all mental health people are mental health people. I didn’t have a better understanding of the system and all the different levels and how it plays out and specialties and things like that. Which, I guess, I should have because you have a primary care doctor, but then you have specialists in all these other different sectors that specialize in one particular area. I guess that should’ve been common sense, but it wasn’t.

Stigma was a barrier to seeking and engaging in mental health care. Veterans discovered they had to overcome stigma associated with seeking and engaging in mental health treatment. Military culture was often discussed as promoting stigma regarding mental health treatment. Specifically, veterans described that seeking treatment meant “either, I’m weak or I’m gonna be seen as weak.” In active-duty settings, the strategy for dealing with mental health symptoms was to “leave those feelings, you push ‘em aside,” an approach highly inconsistent with TF-EBP. In some cases, incorrect information about the VA and PTSD was presented as part of discharge from the military, leading to long-term skepticism of the VA and PTSD treatment. One veteran described his experience as part of a class on the VA compensation and pension assessment process for service-connected disabilities during his military discharge:

[A fellow discharging soldier asked] what about like PTSD, gettin’ rated for PTSD. I hear they take our weapons and stuff like we can’t own firearms and all that stuff. And [the instructor] was like, well, yes that’s a thing. He didn’t explain it like if you get compensated for PTSD you don’t lose your rights to carry a firearm or to have, to be able to go hunting.

Importantly, veterans often described how other identities (eg, race, ethnicity, gender, region of origin) interacted with military culture to enhance stigma. Hearing messaging from multiple sources reinforced beliefs that mental health treatment is inappropriate or is associated with weakness:

As a first-generation Italian, I was always taught keep your feelings to yourself. Never talk outside your family. Never bring up problems to other people and stuff like that. Same with the military. And then the old stigma working in [emergency medical services] and public safety, you’re weak if you get help.

The fundamentals of therapy, including rapport and flexibility, were important. Veterans valued nonspecific therapy factors, genuine empathy, building trust, being honest about treatment, personality, and rapport. These characteristics were almost universally described as particularly important:

I liked the fact that she made it personable and she cared. It wasn’t just like, here, we’re gonna start this. She explained it in the ways I could understand, not in medical terms, so to speak, but that’s what I liked about her. She really cared about what she did and helping me.

Flexibility was viewed as an asset, particularly when clinicians acknowledged veteran autonomy. A consistent example was when veterans were able to titrate trauma disclosure. One veteran described this flexible treatment experience: “She was right there in the room, she said, you know, at any time, you know, we could stop, we could debrief.”

Experiences of clinician flexibility and personalization of therapy were contrasted with experiences of overly rigid therapy. Overemphasis on protocols created barriers, often because treatment did not feel personalized. One veteran described how a clinician’s task-oriented approach interfered with their ability to engage in TF-EBP:

They listened, but it just didn’t seem like they were listening, because they really wanted to stay on task… So, I felt like if the person was more concerned, or more sympathetic to the things that was also going on in my life at that present time, I think I would’ve felt more comfortable talking about what was the PTSD part, too.

Veterans valued shared decision-making prior to TF-EBP initiation. Veterans typically described being involved in a shared decision-making process prior to initiating TF-EBP. During these sessions, clinicians discussed treatment options and provided veterans with a variety of materials describing treatments (eg, pamphlets, websites, videos, statistics). Most veterans appreciated being able to reflect on and discuss treatment options with their clinicians. Being given time in and out of session to review was viewed as valuable and increased confidence in treatment choice. One veteran described their experience:

I was given the information, you know, they gave me handouts, PDFs, whatever was available, and let me read over it. I didn’t have to choose anything right then and there, you know, they let me sleep on it. And I got back to them after some thought.

However, some veterans felt overwhelmed by being presented with too much information and did not believe they knew enough to make a final treatment decision. One veteran described being asked to contribute to the treatment decision:

I definitely asked [the clinician] to weigh in on maybe what he thought was best, because—I mean, I don’t know… I’m not necessarily sure I know what I think is best. I think we’re just lucky I’m here, so if you can give me a solid and help me out here by telling me just based on what I’ve said to you and the things that I’ve gone through, what do you think?

Veterans who perceived that their treatment preferences were respected had a positive outlook on TF-EBP. As part of the shared-decision making process, veterans typically described being given choices among PTSD treatments. One way that preferences were respected was through clinicians tailoring treatment descriptions to a veteran’s unique symptoms, experiences, and values. In these cases, clinicians observed specific concerns and clearly linked treatment principles to those concerns. For example, one veteran described their clinician’s recommendation for PE: “The hardest thing for me is to do the normal things like grocery store or getting on a train or anything like that. And so, he suggested that [PE] would be a good idea.”

In other cases, veterans wanted the highest quality of treatment rather than a match between treatment principles and the veteran’s presentation, goals, or strengths. These veterans wanted the best treatment available for PTSD and valued research support, recommendations from clinical practice guidelines, or clinician confidence in the effectiveness of the treatment. One veteran described this perspective:

I just wanted to be able to really tackle it in the best way possible and in the most like aggressive way possible. And it seemed like PE really was going to, they said that it’s a difficult type of therapy, but I really just wanted to kind of do the best that I could to eradicate some of the issues that I was having.

When veterans perceived a lack of respect for their preferences, they were hesitant about TF-EBP. For some veterans, a generic pitch for a TF-EBP was detrimental in the absence of the personal connection between the treatment and their own symptoms, goals, or strengths. These veterans did not question whether the treatment was effective in general but did question whether the treatment was best for them. One veteran described the contrast between their clinician’s perspective and their own.

I felt like they felt very comfortable, very confident in [CPT] being the program, because it was comfortable for them. Because they did it several times. And maybe they had a lot of success with other individuals... but they were very comfortable with that one, as a provider, more than: Is this the best fit for [me]?

Some veterans perceived little concern for their preferences and a lack of choice in available treatments, which tended to perpetuate negative perceptions of TFEBP. These veterans described their lack of choices with frustration. Alternatives to TFEBP were described by these veterans as so undesirable that they did not believe they had a real choice:

[CPT] was the only decision they had. There was nothing else for PTSD. They didn’t offer anything else. So, I mean it wasn’t a decision. It was either … take treatment or don’t take treatment at all… Actually, I need to correct myself. So, there were 2 options, group therapy or CPT. I forgot about that. I’m not a big group guy so I chose the CPT.

Another veteran was offered a choice between therapeutic approaches, but all were delivered via telehealth (consistent with the transition to virtual services during the COVID-19 pandemic). For this veteran, not only was the distinction between approaches unclear, but the choice between approaches was unimportant compared to the mode of delivery.

This happened during COVID-19 and VA stopped seeing anybody physically, face-to-face. So my only option for therapy was [telehealth]… There was like 3 of them, and I tried to figure out, you know, from the layperson’s perspective, like: I don’t know which one to go with.

Veterans wanted to be asked about their cultural identity. Veterans valued when clinicians asked questions about cultural identity as part of their mental health treatment and listened to their cultural context. Cultural identity factors extended beyond factors such as race, ethnicity, gender, and sexual orientation to religion, military culture, and regionality. Veterans often described situations where they wished clinicians would ask the question or initiate conversations about culture. A veteran highlighted the importance of their faith but noted that it was a taboo topic. Their clinician did not say “we don’t go there,” but they “never dove into it either.” Another veteran expressed a desire for their clinician to ask questions about experiences in the National Guard and as an African American veteran:

If a provider was to say like: Oh, you know, it’s a stressful situation being a part of the military, being in the National Guard. You know, just asking questions about that. I think that would really go a long way… Being African American was difficult as well. And more so because of my region, I think… I felt like it would probably be an uncomfortable subject to speak on… I mean, it wasn’t anything that my providers necessarily did, it was more so just because it wasn’t brought up.

One common area of concern for veterans was a match between veteran and therapist demographics. When asked about how their cultural identity influenced treatment, several veterans described the relevance of therapist match. Much like questions about their own cultural identity, veterans valued being asked about identity preferences in clinicians (eg, gender or race matching), rather than having to bring up the preference themselves. One veteran described relief at this question being asked directly: “I was relieved when she had asked [whether I wanted a male or female clinician] primarily because I was going to ask that or bring that up somehow. But her asking that before me was a weight off my shoulders.”

Discussing cultural identity through treatment strengthened veterans’ engagement in therapy. Many veterans appreciated when analogies used in therapy were relevant to their cultural experiences and when clinicians understood their culture (eg, military culture, race, ethnicity, religious beliefs, sexual orientation). One veteran described how their clinician understood military culture and made connections between military culture and the rationale for TF-EBP, which strengthened the veteran’s buy-in for the treatment and alliance with the clinician:

At the beginning when she was explaining PTSD, and I remember she said that your brain needed to think this way when you were in the military because it was a way of protecting and surviving, so your brain was doing that in order for you to survive in whatever areas you were because there was danger. So, your brain had you thinking that way. But now, you’re not in those situations anymore. You’re not in danger. You’re not in the military, but your brain is still thinking you are, and that’s what PTSD generally does to you.

Specific elements of TF-EBP also provided opportunities to discuss and integrate important aspects of identity. This is accomplished in PE by assigning relevant in vivo exercises. In CPT, “connecting the dots” on how prior experiences influenced trauma-related stuck points achieved this element. One veteran described their experience with a clinician who was comfortable discussing the veteran’s sexual orientation and recognized the impacts of prior trauma on intimacy:

They’re very different, and there’s a lot of things that can be accepted in gay relationships that are not in straight ones. With all that said, I think [the PE therapist] did a fantastic job being not—like never once did she laugh or make an uncomfortable comment or say she didn’t wanna talk about something when like part of the reason I wanted to get into therapy is that my partner and I weren’t having sex unless I used alcohol.

Discussion

As part of a larger national qualitative investigation of the experiences of veterans who recently initiated TF-EBP, veterans discussed their experiences with therapy and mental health care that have important implications for continued process improvement.²¹ Three key areas for continued process improvement were identified: (1) providing information about the diverse range of mental health care services at the VA and the implications of this continuum of care; (2) consideration of veteran preferences in treatment decision-making, including the importance of perceived choice; and (3) incorporating cultural assessment and cultural responsiveness into case conceptualization and treatment.

One area of process improvement identified was increasing knowledge about different types of psychotherapy and the continuum of care available at the VA. Veterans in this study confused or conflated participating in psychotherapy with talking about mental health symptoms with a clinician (eg, assessment, disability evaluation). They were sometimes surprised that psychotherapy is an umbrella term referring to a variety of different modalities. The downstream impact of these misunderstandings was a perception of VA mental health care as nebulous. Veterans were surprised that all mental health practitioners were unable to provide the same care. Confusion may have been compounded by highly variable referral processes across VA.²⁴ To address this, clinicians have developed local educational resources and handouts for both veterans and referring clinicians from nonmental health and general mental health specialties.²⁵ Given the variability in referral processes both between and within VA medical centers, national dissemination of these educational materials may be more difficult compared to materials for TF-EBPs.²⁴ The VA started to use behavioral health interdisciplinary program (BHIP) teams, which are designed to be clinical homes for veterans connected with a central clinician who can explain and coordinate their mental health care as well as bring more consistency to the referral process.²⁶ The ongoing transition toward the BHIP model of mental health care at VA may provide the opportunity to consolidate and integrate knowledge about the VA approach to mental health care, potentially filling knowledge gaps.

A second area of process improvement focused on the shared decision-making process. Consistent with mental health initiatives, veterans generally believed they had received sufficient information about TF-EBP and engaged in shared decision-making with clinicians.^20,27 Veterans were given educational materials to review and had the opportunity to discuss these materials with clinicians. However, veterans described variability in the success of shared decision-making. Although veterans valued receiving accurate, comprehensible information to support treatment decisions, some preferred to defer to clinicians’ expertise regarding which treatment to pursue. While these veterans valued information, they also valued the expertise of clinicians in explaining why specific treatments would be beneficial. A key contributor to veterans satisfaction was assessing how veterans wanted to engage in the decision-making process and respecting those preferences.²⁸ Veterans approached shared decision-making differently, from making decisions independently after receiving information to relying solely on clinician recommendation. The process was most successful when clinicians articulated how their recommended treatment aligned with a veteran’s preferences, including recommendations based on specific values (eg, personalized match vs being the best). Another important consideration is ensuring veterans know they can receive a variety of different types of mental health services available in different modalities (eg, virtual vs in-person; group vs individual). When veterans did not perceive choice in treatment aspects important to them (typically despite having choices), they were less satisfied with their TF-EBP experience.

A final area of process improvement identified involves how therapists address important aspects of culture. Veterans often described mental health stigma coming from intersecting cultural identities and expressed appreciation when therapists helped them recognize the impact of these beliefs on treatment. Some veterans did not discuss important aspects of their identity with clinicians, including race/ethnicity, religion, and military culture. Veterans did not report negative interactions with clinicians or experiences suggesting it was inappropriate to discuss identity; however, they were reluctant to independently raise these identity factors. Strategies such as the ADDRESSING framework, a mnemonic acronym that describes a series of potentially relevant characteristics, can help clinicians comprehensively consider different aspects that may be relevant to veterans, modeling that discussion of relevant these characteristics is welcome in TF-EBP.²⁹ Veterans reported that making culturally relevant connections enhanced the TF-EBP experience, most commonly with military culture. These data support that TF-EBP delivery with attention to culture should be an integrated part of treatment, supporting engagement and therapeutic alliance.³⁰ The VA National Center for PTSD consultation program is a resource to support clinicians in assessing and incorporating relevant aspects of cultural identity.³¹ For example, the National Center for PTSD provides a guide for using case conceptualization to address patient reactions to race-based violence during PTSD treatment.³² Both manualized design and therapist certification training can reinforce that assessing and attending to case conceptualization (including identity factors) is an integral component of TF-EBP.^33,34

Limitations

While the current study has numerous strengths (eg, national veteran sampling, robust qualitative methods), results should be considered within the context of study limitations. First, veteran participants all received TF-EBP, and the perspectives of veterans who never initiate TF-EBP may differ. Despite the strong sampling approach, the study design is not intended to be generalizable to all veterans receiving TF-EBP for PTSD. Qualitative analysis yielded 15 themes, described in this study and prior research, consistent with recommendations.^21,22 This approach allows rich description of distinct focus areas that would not be possible in a single manuscript. Nonetheless, all veterans interviewed described their experiences in TF-EBP and general mental health care, the focus of the semistructured interview guide was on the experience of transitioning from other treatment to TF-EBP.

Conclusion

This study describes themes related to general mental health and TF-EBP process improvement as part of a larger study on transitions in PTSD care.^21,22 Veterans valued the fundamentals of therapy, including rapport and flexibility. Treatment-specific rapport (eg, pointing out treatment progress and effort in completing treatment components) and flexibility within the context of fidelity (ie, personalizing treatment while maintaining core treatment elements) may be most effective at engaging veterans in recommended PTSD treatments.^18,34 In addition to successes, themes suggest multiple opportunities for process improvement. Ongoing VA initiatives and priorities (ie, BHIP, shared decision-making, consultation services) aim to improve processes consistent with veteran recommendations. Future research is needed to evaluate the success of these and other programs to optimize access to and engagement in recommended PTSD treatments.

Trauma-focused evidence-based psychotherapies (TF-EBPs), including cognitive processing therapy (CPT) and prolonged exposure therapy (PE), are recommended treatments for posttraumatic stress disorder (PTSD) in clinical practice guidelines.^1-3 To increase initiation of these treatments, the US Department of Veterans Affairs (VA) used a large-scale dissemination and implementation effort to improve access to TF-EBP.^4,5 These efforts achieved modest success, increasing prevalence of TF-EBP from a handful of veterans in 2004 to an annual prevalence of 14.6% for CPT and 4.3% for PE in 2014.⁶

Throughout these efforts, qualitative studies have been used to better understand veterans’ perspectives on receiving TF-EBP care.^7-18 Barriers to initiation of and engagement in TF-EBP and PTSD care have been identified from these qualitative studies. One identified barrier was lack of knowledge—particularly lack of knowledge about what is meant by a PTSD diagnosis and available treatments.^7-10 Stigma (ie, automatic negative associations) toward mental health problems or seeking mental health care also has been identified as a barrier to initiation.^7,10-14 Perceptions of poor alignment between treatment and veteran goals, including lack of buy-in for the rationale, served as barriers to initiation and engagement.^8,15-18

Using prior qualitative work, numerous initiatives have been developed to reduce stigma, facilitate conversations about how treatment aligns with goals, and fill knowledge gaps, particularly through online resources and shared decision-making.^19,20 To better inform the state of veterans’ experiences with TF-EBP, a qualitative investigation was conducted involving veterans who recently initiated TF-EBP. Themes directly related to transitions to TF-EBP were identified; however, all veterans interviewed also described their experiences with TFEBP engagement and mental health care. Consistent with recommendations for qualitative methods, this study extends prior work on transitions to TF-EBP by describing themes with a distinct focus on the experience of engaging with TF-EBP and mental health care.^21,22

Methods

The experiences of veterans who were transitioning into TF-EBPs were collected in semistructured interviews and analyzed. The semistructured interview guide was developed and refined in consultation with both qualitative methods experts and PTSD treatment experts to ensure that 6 content domains were appropriately queried: PTSD treatment options, cultural sensitivity of treatment, PTSD treatment selection, transition criteria, beliefs about stabilization treatment, and treatment needs/preferences.

Participants were identified using the VA Corporate Data Warehouse and included post-9/11 veterans who had recently initiated CPT or PE for the first time between September 1, 2021, and September 1, 2022. More details of participant selection are available in Holder et al.²¹ From a population of 10,814 patients, stratified random sampling generated a recruitment pool of 200 veterans for further outreach. The strata were defined such that this recruitment pool had similar proportions of demographic characteristics (ie, gender, race, ethnicity) to the population of eligible veterans, equivalent distributions of time to CPT or PE initiation (ie, 33.3% < 1 year, 33.3% 1-3 years, and 33.3% > 3 years), and adequate variability in TF-EBP type (ie, 66.7% CPT, 33.3% PE). A manual chart review in the recruitment pool excluded 12 veterans who did not initiate CPT or PE, 1 veteran with evidence of current active psychosis and/or cognitive impairment that would likely preclude comprehension of study materials, and 1 who was deceased.

Eligible veterans from the recruitment pool were contacted in groups of 25. First, a recruitment letter with study information and instructions to opt-out of further contact was mailed or emailed to veterans. After 2 weeks, veterans who had not responded were contacted by phone up to 3 times. Veterans interested in participating were scheduled for a 1-time visit that included verbal consent and the qualitative interview. Metrics were established a priori to ensure an adequately diverse and inclusive sample. Specifically, a minimum number of racial and/or ethnic minority veterans (33%) and women veterans (20%) were sought. Equal distribution across the 3 categories of time from first mental health visit to CPT/PE initiation also was targeted. Throughout enrollment, recruitment efforts were adapted to meet these metrics in the emerging sample. While the goal was to generate a diverse and inclusive sample using these methods, the sample was not intended to be representative of the population.

Of the 186 eligible participants, 21 declined participation and 26 could not be reached. The targeted sample was reached after exhausting contact for 47 veterans and contacting 80 veterans for a final response rate of 40% among fully contacted veterans and 27% among veterans with any contact. The final sample included 30 veterans who received CPT or PE in VA facilities (Table).

After veterans provided verbal consent for study participation, sociodemographic information was verbally reported, and a 30- to 60-minute semistructured qualitative phone interview was recorded and transcribed. Veterans received $40 for participation. All procedures were approved by the University of California San Francisco Institutional Review Board.

Qualitative Data Analysis

Rapid analysis procedures were used to analyze qualitative data. This approach is suitable for focused, moderately structured qualitative analyses in health services research and facilitates rapid dissemination to stakeholders.²³ The qualitative analysts were 2 clinical psychologists with expertise in PTSD treatment (NH primary and RR secondary). Consistent with rapid analysis procedures, analysts prepared a templated summary (including relevant quotations) of each interview, organized by the prespecified content domains. Interviews were summarized independently, compared to ensure consistency, and discrepancies were resolved through review of the interview source materials. Individual summary templates were combined into a master analytic matrix to facilitate the identification of patterns and delineation of themes. Analysts routinely met to identify, discuss, and refine preliminary themes, revisiting source materials to reach consensus as needed.

Results

Fifteen themes were identified and organized into 2 distinct focus areas: themes directly related to the transition to TF-EBP (8 themes) and themes related to veterans’ experiences with TF-EBP and general mental health care with potential process-improvement implications (7 themes).²¹ Seven themes were identified related to experiences with TF-EBP engagement and VA mental health care. The 7 themes related to TF-EBP engagement and VA mental health care themes are summarized with exemplary quotations.

Veterans want a better understanding of psychotherapy and engaging with VA mental health. Veterans reported that they generally had a poor or “nebulous” understanding about the experience of psychotherapy. For example, veterans exhibited confusion about whether certain experiences were equivalent to participating in psychotherapy. They were sometimes unable to distinguish between interactions such as assessment, disability evaluations, peer support, and psychotherapy. One veteran described a conversation with a TFEBP therapist about prior treatment:

She [asked], have you ever been, or gone through a therapy to begin with? And I, I said, well I just chatted with somebody. And she said that’s not, that’s not therapy. So, I was like, oh, it’s not? That’s not what people do?

Veterans were surprised the VA offered a diverse range of psychotherapy interventions, rather than simply therapy. They did not realize there were different types of psychotherapy. As a result, veterans were not aware that some VA mental practitioners have specialty training and certification to provide treatment matched to specific diagnoses or needs. They thought that all clinicians could provide the same care. One veteran described their understanding:

I just figured all mental health people are mental health people. I didn’t have a better understanding of the system and all the different levels and how it plays out and specialties and things like that. Which, I guess, I should have because you have a primary care doctor, but then you have specialists in all these other different sectors that specialize in one particular area. I guess that should’ve been common sense, but it wasn’t.

Stigma was a barrier to seeking and engaging in mental health care. Veterans discovered they had to overcome stigma associated with seeking and engaging in mental health treatment. Military culture was often discussed as promoting stigma regarding mental health treatment. Specifically, veterans described that seeking treatment meant “either, I’m weak or I’m gonna be seen as weak.” In active-duty settings, the strategy for dealing with mental health symptoms was to “leave those feelings, you push ‘em aside,” an approach highly inconsistent with TF-EBP. In some cases, incorrect information about the VA and PTSD was presented as part of discharge from the military, leading to long-term skepticism of the VA and PTSD treatment. One veteran described his experience as part of a class on the VA compensation and pension assessment process for service-connected disabilities during his military discharge:

[A fellow discharging soldier asked] what about like PTSD, gettin’ rated for PTSD. I hear they take our weapons and stuff like we can’t own firearms and all that stuff. And [the instructor] was like, well, yes that’s a thing. He didn’t explain it like if you get compensated for PTSD you don’t lose your rights to carry a firearm or to have, to be able to go hunting.

Importantly, veterans often described how other identities (eg, race, ethnicity, gender, region of origin) interacted with military culture to enhance stigma. Hearing messaging from multiple sources reinforced beliefs that mental health treatment is inappropriate or is associated with weakness:

As a first-generation Italian, I was always taught keep your feelings to yourself. Never talk outside your family. Never bring up problems to other people and stuff like that. Same with the military. And then the old stigma working in [emergency medical services] and public safety, you’re weak if you get help.

The fundamentals of therapy, including rapport and flexibility, were important. Veterans valued nonspecific therapy factors, genuine empathy, building trust, being honest about treatment, personality, and rapport. These characteristics were almost universally described as particularly important:

I liked the fact that she made it personable and she cared. It wasn’t just like, here, we’re gonna start this. She explained it in the ways I could understand, not in medical terms, so to speak, but that’s what I liked about her. She really cared about what she did and helping me.

Flexibility was viewed as an asset, particularly when clinicians acknowledged veteran autonomy. A consistent example was when veterans were able to titrate trauma disclosure. One veteran described this flexible treatment experience: “She was right there in the room, she said, you know, at any time, you know, we could stop, we could debrief.”

Experiences of clinician flexibility and personalization of therapy were contrasted with experiences of overly rigid therapy. Overemphasis on protocols created barriers, often because treatment did not feel personalized. One veteran described how a clinician’s task-oriented approach interfered with their ability to engage in TF-EBP:

They listened, but it just didn’t seem like they were listening, because they really wanted to stay on task… So, I felt like if the person was more concerned, or more sympathetic to the things that was also going on in my life at that present time, I think I would’ve felt more comfortable talking about what was the PTSD part, too.

Veterans valued shared decision-making prior to TF-EBP initiation. Veterans typically described being involved in a shared decision-making process prior to initiating TF-EBP. During these sessions, clinicians discussed treatment options and provided veterans with a variety of materials describing treatments (eg, pamphlets, websites, videos, statistics). Most veterans appreciated being able to reflect on and discuss treatment options with their clinicians. Being given time in and out of session to review was viewed as valuable and increased confidence in treatment choice. One veteran described their experience:

I was given the information, you know, they gave me handouts, PDFs, whatever was available, and let me read over it. I didn’t have to choose anything right then and there, you know, they let me sleep on it. And I got back to them after some thought.

However, some veterans felt overwhelmed by being presented with too much information and did not believe they knew enough to make a final treatment decision. One veteran described being asked to contribute to the treatment decision:

I definitely asked [the clinician] to weigh in on maybe what he thought was best, because—I mean, I don’t know… I’m not necessarily sure I know what I think is best. I think we’re just lucky I’m here, so if you can give me a solid and help me out here by telling me just based on what I’ve said to you and the things that I’ve gone through, what do you think?

Veterans who perceived that their treatment preferences were respected had a positive outlook on TF-EBP. As part of the shared-decision making process, veterans typically described being given choices among PTSD treatments. One way that preferences were respected was through clinicians tailoring treatment descriptions to a veteran’s unique symptoms, experiences, and values. In these cases, clinicians observed specific concerns and clearly linked treatment principles to those concerns. For example, one veteran described their clinician’s recommendation for PE: “The hardest thing for me is to do the normal things like grocery store or getting on a train or anything like that. And so, he suggested that [PE] would be a good idea.”

In other cases, veterans wanted the highest quality of treatment rather than a match between treatment principles and the veteran’s presentation, goals, or strengths. These veterans wanted the best treatment available for PTSD and valued research support, recommendations from clinical practice guidelines, or clinician confidence in the effectiveness of the treatment. One veteran described this perspective:

I just wanted to be able to really tackle it in the best way possible and in the most like aggressive way possible. And it seemed like PE really was going to, they said that it’s a difficult type of therapy, but I really just wanted to kind of do the best that I could to eradicate some of the issues that I was having.

When veterans perceived a lack of respect for their preferences, they were hesitant about TF-EBP. For some veterans, a generic pitch for a TF-EBP was detrimental in the absence of the personal connection between the treatment and their own symptoms, goals, or strengths. These veterans did not question whether the treatment was effective in general but did question whether the treatment was best for them. One veteran described the contrast between their clinician’s perspective and their own.

I felt like they felt very comfortable, very confident in [CPT] being the program, because it was comfortable for them. Because they did it several times. And maybe they had a lot of success with other individuals... but they were very comfortable with that one, as a provider, more than: Is this the best fit for [me]?

Some veterans perceived little concern for their preferences and a lack of choice in available treatments, which tended to perpetuate negative perceptions of TFEBP. These veterans described their lack of choices with frustration. Alternatives to TFEBP were described by these veterans as so undesirable that they did not believe they had a real choice:

[CPT] was the only decision they had. There was nothing else for PTSD. They didn’t offer anything else. So, I mean it wasn’t a decision. It was either … take treatment or don’t take treatment at all… Actually, I need to correct myself. So, there were 2 options, group therapy or CPT. I forgot about that. I’m not a big group guy so I chose the CPT.

Another veteran was offered a choice between therapeutic approaches, but all were delivered via telehealth (consistent with the transition to virtual services during the COVID-19 pandemic). For this veteran, not only was the distinction between approaches unclear, but the choice between approaches was unimportant compared to the mode of delivery.

This happened during COVID-19 and VA stopped seeing anybody physically, face-to-face. So my only option for therapy was [telehealth]… There was like 3 of them, and I tried to figure out, you know, from the layperson’s perspective, like: I don’t know which one to go with.

Veterans wanted to be asked about their cultural identity. Veterans valued when clinicians asked questions about cultural identity as part of their mental health treatment and listened to their cultural context. Cultural identity factors extended beyond factors such as race, ethnicity, gender, and sexual orientation to religion, military culture, and regionality. Veterans often described situations where they wished clinicians would ask the question or initiate conversations about culture. A veteran highlighted the importance of their faith but noted that it was a taboo topic. Their clinician did not say “we don’t go there,” but they “never dove into it either.” Another veteran expressed a desire for their clinician to ask questions about experiences in the National Guard and as an African American veteran:

If a provider was to say like: Oh, you know, it’s a stressful situation being a part of the military, being in the National Guard. You know, just asking questions about that. I think that would really go a long way… Being African American was difficult as well. And more so because of my region, I think… I felt like it would probably be an uncomfortable subject to speak on… I mean, it wasn’t anything that my providers necessarily did, it was more so just because it wasn’t brought up.

One common area of concern for veterans was a match between veteran and therapist demographics. When asked about how their cultural identity influenced treatment, several veterans described the relevance of therapist match. Much like questions about their own cultural identity, veterans valued being asked about identity preferences in clinicians (eg, gender or race matching), rather than having to bring up the preference themselves. One veteran described relief at this question being asked directly: “I was relieved when she had asked [whether I wanted a male or female clinician] primarily because I was going to ask that or bring that up somehow. But her asking that before me was a weight off my shoulders.”

Discussing cultural identity through treatment strengthened veterans’ engagement in therapy. Many veterans appreciated when analogies used in therapy were relevant to their cultural experiences and when clinicians understood their culture (eg, military culture, race, ethnicity, religious beliefs, sexual orientation). One veteran described how their clinician understood military culture and made connections between military culture and the rationale for TF-EBP, which strengthened the veteran’s buy-in for the treatment and alliance with the clinician:

At the beginning when she was explaining PTSD, and I remember she said that your brain needed to think this way when you were in the military because it was a way of protecting and surviving, so your brain was doing that in order for you to survive in whatever areas you were because there was danger. So, your brain had you thinking that way. But now, you’re not in those situations anymore. You’re not in danger. You’re not in the military, but your brain is still thinking you are, and that’s what PTSD generally does to you.

Specific elements of TF-EBP also provided opportunities to discuss and integrate important aspects of identity. This is accomplished in PE by assigning relevant in vivo exercises. In CPT, “connecting the dots” on how prior experiences influenced trauma-related stuck points achieved this element. One veteran described their experience with a clinician who was comfortable discussing the veteran’s sexual orientation and recognized the impacts of prior trauma on intimacy:

They’re very different, and there’s a lot of things that can be accepted in gay relationships that are not in straight ones. With all that said, I think [the PE therapist] did a fantastic job being not—like never once did she laugh or make an uncomfortable comment or say she didn’t wanna talk about something when like part of the reason I wanted to get into therapy is that my partner and I weren’t having sex unless I used alcohol.

Discussion

As part of a larger national qualitative investigation of the experiences of veterans who recently initiated TF-EBP, veterans discussed their experiences with therapy and mental health care that have important implications for continued process improvement.²¹ Three key areas for continued process improvement were identified: (1) providing information about the diverse range of mental health care services at the VA and the implications of this continuum of care; (2) consideration of veteran preferences in treatment decision-making, including the importance of perceived choice; and (3) incorporating cultural assessment and cultural responsiveness into case conceptualization and treatment.

One area of process improvement identified was increasing knowledge about different types of psychotherapy and the continuum of care available at the VA. Veterans in this study confused or conflated participating in psychotherapy with talking about mental health symptoms with a clinician (eg, assessment, disability evaluation). They were sometimes surprised that psychotherapy is an umbrella term referring to a variety of different modalities. The downstream impact of these misunderstandings was a perception of VA mental health care as nebulous. Veterans were surprised that all mental health practitioners were unable to provide the same care. Confusion may have been compounded by highly variable referral processes across VA.²⁴ To address this, clinicians have developed local educational resources and handouts for both veterans and referring clinicians from nonmental health and general mental health specialties.²⁵ Given the variability in referral processes both between and within VA medical centers, national dissemination of these educational materials may be more difficult compared to materials for TF-EBPs.²⁴ The VA started to use behavioral health interdisciplinary program (BHIP) teams, which are designed to be clinical homes for veterans connected with a central clinician who can explain and coordinate their mental health care as well as bring more consistency to the referral process.²⁶ The ongoing transition toward the BHIP model of mental health care at VA may provide the opportunity to consolidate and integrate knowledge about the VA approach to mental health care, potentially filling knowledge gaps.

A second area of process improvement focused on the shared decision-making process. Consistent with mental health initiatives, veterans generally believed they had received sufficient information about TF-EBP and engaged in shared decision-making with clinicians.^20,27 Veterans were given educational materials to review and had the opportunity to discuss these materials with clinicians. However, veterans described variability in the success of shared decision-making. Although veterans valued receiving accurate, comprehensible information to support treatment decisions, some preferred to defer to clinicians’ expertise regarding which treatment to pursue. While these veterans valued information, they also valued the expertise of clinicians in explaining why specific treatments would be beneficial. A key contributor to veterans satisfaction was assessing how veterans wanted to engage in the decision-making process and respecting those preferences.²⁸ Veterans approached shared decision-making differently, from making decisions independently after receiving information to relying solely on clinician recommendation. The process was most successful when clinicians articulated how their recommended treatment aligned with a veteran’s preferences, including recommendations based on specific values (eg, personalized match vs being the best). Another important consideration is ensuring veterans know they can receive a variety of different types of mental health services available in different modalities (eg, virtual vs in-person; group vs individual). When veterans did not perceive choice in treatment aspects important to them (typically despite having choices), they were less satisfied with their TF-EBP experience.

A final area of process improvement identified involves how therapists address important aspects of culture. Veterans often described mental health stigma coming from intersecting cultural identities and expressed appreciation when therapists helped them recognize the impact of these beliefs on treatment. Some veterans did not discuss important aspects of their identity with clinicians, including race/ethnicity, religion, and military culture. Veterans did not report negative interactions with clinicians or experiences suggesting it was inappropriate to discuss identity; however, they were reluctant to independently raise these identity factors. Strategies such as the ADDRESSING framework, a mnemonic acronym that describes a series of potentially relevant characteristics, can help clinicians comprehensively consider different aspects that may be relevant to veterans, modeling that discussion of relevant these characteristics is welcome in TF-EBP.²⁹ Veterans reported that making culturally relevant connections enhanced the TF-EBP experience, most commonly with military culture. These data support that TF-EBP delivery with attention to culture should be an integrated part of treatment, supporting engagement and therapeutic alliance.³⁰ The VA National Center for PTSD consultation program is a resource to support clinicians in assessing and incorporating relevant aspects of cultural identity.³¹ For example, the National Center for PTSD provides a guide for using case conceptualization to address patient reactions to race-based violence during PTSD treatment.³² Both manualized design and therapist certification training can reinforce that assessing and attending to case conceptualization (including identity factors) is an integral component of TF-EBP.^33,34

Limitations

While the current study has numerous strengths (eg, national veteran sampling, robust qualitative methods), results should be considered within the context of study limitations. First, veteran participants all received TF-EBP, and the perspectives of veterans who never initiate TF-EBP may differ. Despite the strong sampling approach, the study design is not intended to be generalizable to all veterans receiving TF-EBP for PTSD. Qualitative analysis yielded 15 themes, described in this study and prior research, consistent with recommendations.^21,22 This approach allows rich description of distinct focus areas that would not be possible in a single manuscript. Nonetheless, all veterans interviewed described their experiences in TF-EBP and general mental health care, the focus of the semistructured interview guide was on the experience of transitioning from other treatment to TF-EBP.

Conclusion

This study describes themes related to general mental health and TF-EBP process improvement as part of a larger study on transitions in PTSD care.^21,22 Veterans valued the fundamentals of therapy, including rapport and flexibility. Treatment-specific rapport (eg, pointing out treatment progress and effort in completing treatment components) and flexibility within the context of fidelity (ie, personalizing treatment while maintaining core treatment elements) may be most effective at engaging veterans in recommended PTSD treatments.^18,34 In addition to successes, themes suggest multiple opportunities for process improvement. Ongoing VA initiatives and priorities (ie, BHIP, shared decision-making, consultation services) aim to improve processes consistent with veteran recommendations. Future research is needed to evaluate the success of these and other programs to optimize access to and engagement in recommended PTSD treatments.

Trauma-focused evidence-based psychotherapies (TF-EBPs), including cognitive processing therapy (CPT) and prolonged exposure therapy (PE), are recommended treatments for posttraumatic stress disorder (PTSD) in clinical practice guidelines.^1-3 To increase initiation of these treatments, the US Department of Veterans Affairs (VA) used a large-scale dissemination and implementation effort to improve access to TF-EBP.^4,5 These efforts achieved modest success, increasing prevalence of TF-EBP from a handful of veterans in 2004 to an annual prevalence of 14.6% for CPT and 4.3% for PE in 2014.⁶

Throughout these efforts, qualitative studies have been used to better understand veterans’ perspectives on receiving TF-EBP care.^7-18 Barriers to initiation of and engagement in TF-EBP and PTSD care have been identified from these qualitative studies. One identified barrier was lack of knowledge—particularly lack of knowledge about what is meant by a PTSD diagnosis and available treatments.^7-10 Stigma (ie, automatic negative associations) toward mental health problems or seeking mental health care also has been identified as a barrier to initiation.^7,10-14 Perceptions of poor alignment between treatment and veteran goals, including lack of buy-in for the rationale, served as barriers to initiation and engagement.^8,15-18

Using prior qualitative work, numerous initiatives have been developed to reduce stigma, facilitate conversations about how treatment aligns with goals, and fill knowledge gaps, particularly through online resources and shared decision-making.^19,20 To better inform the state of veterans’ experiences with TF-EBP, a qualitative investigation was conducted involving veterans who recently initiated TF-EBP. Themes directly related to transitions to TF-EBP were identified; however, all veterans interviewed also described their experiences with TFEBP engagement and mental health care. Consistent with recommendations for qualitative methods, this study extends prior work on transitions to TF-EBP by describing themes with a distinct focus on the experience of engaging with TF-EBP and mental health care.^21,22

Methods

The experiences of veterans who were transitioning into TF-EBPs were collected in semistructured interviews and analyzed. The semistructured interview guide was developed and refined in consultation with both qualitative methods experts and PTSD treatment experts to ensure that 6 content domains were appropriately queried: PTSD treatment options, cultural sensitivity of treatment, PTSD treatment selection, transition criteria, beliefs about stabilization treatment, and treatment needs/preferences.

Participants were identified using the VA Corporate Data Warehouse and included post-9/11 veterans who had recently initiated CPT or PE for the first time between September 1, 2021, and September 1, 2022. More details of participant selection are available in Holder et al.²¹ From a population of 10,814 patients, stratified random sampling generated a recruitment pool of 200 veterans for further outreach. The strata were defined such that this recruitment pool had similar proportions of demographic characteristics (ie, gender, race, ethnicity) to the population of eligible veterans, equivalent distributions of time to CPT or PE initiation (ie, 33.3% < 1 year, 33.3% 1-3 years, and 33.3% > 3 years), and adequate variability in TF-EBP type (ie, 66.7% CPT, 33.3% PE). A manual chart review in the recruitment pool excluded 12 veterans who did not initiate CPT or PE, 1 veteran with evidence of current active psychosis and/or cognitive impairment that would likely preclude comprehension of study materials, and 1 who was deceased.

Eligible veterans from the recruitment pool were contacted in groups of 25. First, a recruitment letter with study information and instructions to opt-out of further contact was mailed or emailed to veterans. After 2 weeks, veterans who had not responded were contacted by phone up to 3 times. Veterans interested in participating were scheduled for a 1-time visit that included verbal consent and the qualitative interview. Metrics were established a priori to ensure an adequately diverse and inclusive sample. Specifically, a minimum number of racial and/or ethnic minority veterans (33%) and women veterans (20%) were sought. Equal distribution across the 3 categories of time from first mental health visit to CPT/PE initiation also was targeted. Throughout enrollment, recruitment efforts were adapted to meet these metrics in the emerging sample. While the goal was to generate a diverse and inclusive sample using these methods, the sample was not intended to be representative of the population.

Of the 186 eligible participants, 21 declined participation and 26 could not be reached. The targeted sample was reached after exhausting contact for 47 veterans and contacting 80 veterans for a final response rate of 40% among fully contacted veterans and 27% among veterans with any contact. The final sample included 30 veterans who received CPT or PE in VA facilities (Table).

After veterans provided verbal consent for study participation, sociodemographic information was verbally reported, and a 30- to 60-minute semistructured qualitative phone interview was recorded and transcribed. Veterans received $40 for participation. All procedures were approved by the University of California San Francisco Institutional Review Board.

Qualitative Data Analysis

Rapid analysis procedures were used to analyze qualitative data. This approach is suitable for focused, moderately structured qualitative analyses in health services research and facilitates rapid dissemination to stakeholders.²³ The qualitative analysts were 2 clinical psychologists with expertise in PTSD treatment (NH primary and RR secondary). Consistent with rapid analysis procedures, analysts prepared a templated summary (including relevant quotations) of each interview, organized by the prespecified content domains. Interviews were summarized independently, compared to ensure consistency, and discrepancies were resolved through review of the interview source materials. Individual summary templates were combined into a master analytic matrix to facilitate the identification of patterns and delineation of themes. Analysts routinely met to identify, discuss, and refine preliminary themes, revisiting source materials to reach consensus as needed.

Results

Fifteen themes were identified and organized into 2 distinct focus areas: themes directly related to the transition to TF-EBP (8 themes) and themes related to veterans’ experiences with TF-EBP and general mental health care with potential process-improvement implications (7 themes).²¹ Seven themes were identified related to experiences with TF-EBP engagement and VA mental health care. The 7 themes related to TF-EBP engagement and VA mental health care themes are summarized with exemplary quotations.

Veterans want a better understanding of psychotherapy and engaging with VA mental health. Veterans reported that they generally had a poor or “nebulous” understanding about the experience of psychotherapy. For example, veterans exhibited confusion about whether certain experiences were equivalent to participating in psychotherapy. They were sometimes unable to distinguish between interactions such as assessment, disability evaluations, peer support, and psychotherapy. One veteran described a conversation with a TFEBP therapist about prior treatment:

She [asked], have you ever been, or gone through a therapy to begin with? And I, I said, well I just chatted with somebody. And she said that’s not, that’s not therapy. So, I was like, oh, it’s not? That’s not what people do?

Veterans were surprised the VA offered a diverse range of psychotherapy interventions, rather than simply therapy. They did not realize there were different types of psychotherapy. As a result, veterans were not aware that some VA mental practitioners have specialty training and certification to provide treatment matched to specific diagnoses or needs. They thought that all clinicians could provide the same care. One veteran described their understanding:

I just figured all mental health people are mental health people. I didn’t have a better understanding of the system and all the different levels and how it plays out and specialties and things like that. Which, I guess, I should have because you have a primary care doctor, but then you have specialists in all these other different sectors that specialize in one particular area. I guess that should’ve been common sense, but it wasn’t.

Stigma was a barrier to seeking and engaging in mental health care. Veterans discovered they had to overcome stigma associated with seeking and engaging in mental health treatment. Military culture was often discussed as promoting stigma regarding mental health treatment. Specifically, veterans described that seeking treatment meant “either, I’m weak or I’m gonna be seen as weak.” In active-duty settings, the strategy for dealing with mental health symptoms was to “leave those feelings, you push ‘em aside,” an approach highly inconsistent with TF-EBP. In some cases, incorrect information about the VA and PTSD was presented as part of discharge from the military, leading to long-term skepticism of the VA and PTSD treatment. One veteran described his experience as part of a class on the VA compensation and pension assessment process for service-connected disabilities during his military discharge:

[A fellow discharging soldier asked] what about like PTSD, gettin’ rated for PTSD. I hear they take our weapons and stuff like we can’t own firearms and all that stuff. And [the instructor] was like, well, yes that’s a thing. He didn’t explain it like if you get compensated for PTSD you don’t lose your rights to carry a firearm or to have, to be able to go hunting.

Importantly, veterans often described how other identities (eg, race, ethnicity, gender, region of origin) interacted with military culture to enhance stigma. Hearing messaging from multiple sources reinforced beliefs that mental health treatment is inappropriate or is associated with weakness:

As a first-generation Italian, I was always taught keep your feelings to yourself. Never talk outside your family. Never bring up problems to other people and stuff like that. Same with the military. And then the old stigma working in [emergency medical services] and public safety, you’re weak if you get help.

The fundamentals of therapy, including rapport and flexibility, were important. Veterans valued nonspecific therapy factors, genuine empathy, building trust, being honest about treatment, personality, and rapport. These characteristics were almost universally described as particularly important:

I liked the fact that she made it personable and she cared. It wasn’t just like, here, we’re gonna start this. She explained it in the ways I could understand, not in medical terms, so to speak, but that’s what I liked about her. She really cared about what she did and helping me.

Flexibility was viewed as an asset, particularly when clinicians acknowledged veteran autonomy. A consistent example was when veterans were able to titrate trauma disclosure. One veteran described this flexible treatment experience: “She was right there in the room, she said, you know, at any time, you know, we could stop, we could debrief.”

Experiences of clinician flexibility and personalization of therapy were contrasted with experiences of overly rigid therapy. Overemphasis on protocols created barriers, often because treatment did not feel personalized. One veteran described how a clinician’s task-oriented approach interfered with their ability to engage in TF-EBP:

They listened, but it just didn’t seem like they were listening, because they really wanted to stay on task… So, I felt like if the person was more concerned, or more sympathetic to the things that was also going on in my life at that present time, I think I would’ve felt more comfortable talking about what was the PTSD part, too.

Veterans valued shared decision-making prior to TF-EBP initiation. Veterans typically described being involved in a shared decision-making process prior to initiating TF-EBP. During these sessions, clinicians discussed treatment options and provided veterans with a variety of materials describing treatments (eg, pamphlets, websites, videos, statistics). Most veterans appreciated being able to reflect on and discuss treatment options with their clinicians. Being given time in and out of session to review was viewed as valuable and increased confidence in treatment choice. One veteran described their experience:

I was given the information, you know, they gave me handouts, PDFs, whatever was available, and let me read over it. I didn’t have to choose anything right then and there, you know, they let me sleep on it. And I got back to them after some thought.

However, some veterans felt overwhelmed by being presented with too much information and did not believe they knew enough to make a final treatment decision. One veteran described being asked to contribute to the treatment decision:

I definitely asked [the clinician] to weigh in on maybe what he thought was best, because—I mean, I don’t know… I’m not necessarily sure I know what I think is best. I think we’re just lucky I’m here, so if you can give me a solid and help me out here by telling me just based on what I’ve said to you and the things that I’ve gone through, what do you think?

Veterans who perceived that their treatment preferences were respected had a positive outlook on TF-EBP. As part of the shared-decision making process, veterans typically described being given choices among PTSD treatments. One way that preferences were respected was through clinicians tailoring treatment descriptions to a veteran’s unique symptoms, experiences, and values. In these cases, clinicians observed specific concerns and clearly linked treatment principles to those concerns. For example, one veteran described their clinician’s recommendation for PE: “The hardest thing for me is to do the normal things like grocery store or getting on a train or anything like that. And so, he suggested that [PE] would be a good idea.”

In other cases, veterans wanted the highest quality of treatment rather than a match between treatment principles and the veteran’s presentation, goals, or strengths. These veterans wanted the best treatment available for PTSD and valued research support, recommendations from clinical practice guidelines, or clinician confidence in the effectiveness of the treatment. One veteran described this perspective:

I just wanted to be able to really tackle it in the best way possible and in the most like aggressive way possible. And it seemed like PE really was going to, they said that it’s a difficult type of therapy, but I really just wanted to kind of do the best that I could to eradicate some of the issues that I was having.

When veterans perceived a lack of respect for their preferences, they were hesitant about TF-EBP. For some veterans, a generic pitch for a TF-EBP was detrimental in the absence of the personal connection between the treatment and their own symptoms, goals, or strengths. These veterans did not question whether the treatment was effective in general but did question whether the treatment was best for them. One veteran described the contrast between their clinician’s perspective and their own.

I felt like they felt very comfortable, very confident in [CPT] being the program, because it was comfortable for them. Because they did it several times. And maybe they had a lot of success with other individuals... but they were very comfortable with that one, as a provider, more than: Is this the best fit for [me]?

Some veterans perceived little concern for their preferences and a lack of choice in available treatments, which tended to perpetuate negative perceptions of TFEBP. These veterans described their lack of choices with frustration. Alternatives to TFEBP were described by these veterans as so undesirable that they did not believe they had a real choice:

[CPT] was the only decision they had. There was nothing else for PTSD. They didn’t offer anything else. So, I mean it wasn’t a decision. It was either … take treatment or don’t take treatment at all… Actually, I need to correct myself. So, there were 2 options, group therapy or CPT. I forgot about that. I’m not a big group guy so I chose the CPT.

Another veteran was offered a choice between therapeutic approaches, but all were delivered via telehealth (consistent with the transition to virtual services during the COVID-19 pandemic). For this veteran, not only was the distinction between approaches unclear, but the choice between approaches was unimportant compared to the mode of delivery.

This happened during COVID-19 and VA stopped seeing anybody physically, face-to-face. So my only option for therapy was [telehealth]… There was like 3 of them, and I tried to figure out, you know, from the layperson’s perspective, like: I don’t know which one to go with.

Veterans wanted to be asked about their cultural identity. Veterans valued when clinicians asked questions about cultural identity as part of their mental health treatment and listened to their cultural context. Cultural identity factors extended beyond factors such as race, ethnicity, gender, and sexual orientation to religion, military culture, and regionality. Veterans often described situations where they wished clinicians would ask the question or initiate conversations about culture. A veteran highlighted the importance of their faith but noted that it was a taboo topic. Their clinician did not say “we don’t go there,” but they “never dove into it either.” Another veteran expressed a desire for their clinician to ask questions about experiences in the National Guard and as an African American veteran:

If a provider was to say like: Oh, you know, it’s a stressful situation being a part of the military, being in the National Guard. You know, just asking questions about that. I think that would really go a long way… Being African American was difficult as well. And more so because of my region, I think… I felt like it would probably be an uncomfortable subject to speak on… I mean, it wasn’t anything that my providers necessarily did, it was more so just because it wasn’t brought up.

One common area of concern for veterans was a match between veteran and therapist demographics. When asked about how their cultural identity influenced treatment, several veterans described the relevance of therapist match. Much like questions about their own cultural identity, veterans valued being asked about identity preferences in clinicians (eg, gender or race matching), rather than having to bring up the preference themselves. One veteran described relief at this question being asked directly: “I was relieved when she had asked [whether I wanted a male or female clinician] primarily because I was going to ask that or bring that up somehow. But her asking that before me was a weight off my shoulders.”

Discussing cultural identity through treatment strengthened veterans’ engagement in therapy. Many veterans appreciated when analogies used in therapy were relevant to their cultural experiences and when clinicians understood their culture (eg, military culture, race, ethnicity, religious beliefs, sexual orientation). One veteran described how their clinician understood military culture and made connections between military culture and the rationale for TF-EBP, which strengthened the veteran’s buy-in for the treatment and alliance with the clinician:

At the beginning when she was explaining PTSD, and I remember she said that your brain needed to think this way when you were in the military because it was a way of protecting and surviving, so your brain was doing that in order for you to survive in whatever areas you were because there was danger. So, your brain had you thinking that way. But now, you’re not in those situations anymore. You’re not in danger. You’re not in the military, but your brain is still thinking you are, and that’s what PTSD generally does to you.

Specific elements of TF-EBP also provided opportunities to discuss and integrate important aspects of identity. This is accomplished in PE by assigning relevant in vivo exercises. In CPT, “connecting the dots” on how prior experiences influenced trauma-related stuck points achieved this element. One veteran described their experience with a clinician who was comfortable discussing the veteran’s sexual orientation and recognized the impacts of prior trauma on intimacy:

They’re very different, and there’s a lot of things that can be accepted in gay relationships that are not in straight ones. With all that said, I think [the PE therapist] did a fantastic job being not—like never once did she laugh or make an uncomfortable comment or say she didn’t wanna talk about something when like part of the reason I wanted to get into therapy is that my partner and I weren’t having sex unless I used alcohol.

Discussion

As part of a larger national qualitative investigation of the experiences of veterans who recently initiated TF-EBP, veterans discussed their experiences with therapy and mental health care that have important implications for continued process improvement.²¹ Three key areas for continued process improvement were identified: (1) providing information about the diverse range of mental health care services at the VA and the implications of this continuum of care; (2) consideration of veteran preferences in treatment decision-making, including the importance of perceived choice; and (3) incorporating cultural assessment and cultural responsiveness into case conceptualization and treatment.

One area of process improvement identified was increasing knowledge about different types of psychotherapy and the continuum of care available at the VA. Veterans in this study confused or conflated participating in psychotherapy with talking about mental health symptoms with a clinician (eg, assessment, disability evaluation). They were sometimes surprised that psychotherapy is an umbrella term referring to a variety of different modalities. The downstream impact of these misunderstandings was a perception of VA mental health care as nebulous. Veterans were surprised that all mental health practitioners were unable to provide the same care. Confusion may have been compounded by highly variable referral processes across VA.²⁴ To address this, clinicians have developed local educational resources and handouts for both veterans and referring clinicians from nonmental health and general mental health specialties.²⁵ Given the variability in referral processes both between and within VA medical centers, national dissemination of these educational materials may be more difficult compared to materials for TF-EBPs.²⁴ The VA started to use behavioral health interdisciplinary program (BHIP) teams, which are designed to be clinical homes for veterans connected with a central clinician who can explain and coordinate their mental health care as well as bring more consistency to the referral process.²⁶ The ongoing transition toward the BHIP model of mental health care at VA may provide the opportunity to consolidate and integrate knowledge about the VA approach to mental health care, potentially filling knowledge gaps.

A second area of process improvement focused on the shared decision-making process. Consistent with mental health initiatives, veterans generally believed they had received sufficient information about TF-EBP and engaged in shared decision-making with clinicians.^20,27 Veterans were given educational materials to review and had the opportunity to discuss these materials with clinicians. However, veterans described variability in the success of shared decision-making. Although veterans valued receiving accurate, comprehensible information to support treatment decisions, some preferred to defer to clinicians’ expertise regarding which treatment to pursue. While these veterans valued information, they also valued the expertise of clinicians in explaining why specific treatments would be beneficial. A key contributor to veterans satisfaction was assessing how veterans wanted to engage in the decision-making process and respecting those preferences.²⁸ Veterans approached shared decision-making differently, from making decisions independently after receiving information to relying solely on clinician recommendation. The process was most successful when clinicians articulated how their recommended treatment aligned with a veteran’s preferences, including recommendations based on specific values (eg, personalized match vs being the best). Another important consideration is ensuring veterans know they can receive a variety of different types of mental health services available in different modalities (eg, virtual vs in-person; group vs individual). When veterans did not perceive choice in treatment aspects important to them (typically despite having choices), they were less satisfied with their TF-EBP experience.

A final area of process improvement identified involves how therapists address important aspects of culture. Veterans often described mental health stigma coming from intersecting cultural identities and expressed appreciation when therapists helped them recognize the impact of these beliefs on treatment. Some veterans did not discuss important aspects of their identity with clinicians, including race/ethnicity, religion, and military culture. Veterans did not report negative interactions with clinicians or experiences suggesting it was inappropriate to discuss identity; however, they were reluctant to independently raise these identity factors. Strategies such as the ADDRESSING framework, a mnemonic acronym that describes a series of potentially relevant characteristics, can help clinicians comprehensively consider different aspects that may be relevant to veterans, modeling that discussion of relevant these characteristics is welcome in TF-EBP.²⁹ Veterans reported that making culturally relevant connections enhanced the TF-EBP experience, most commonly with military culture. These data support that TF-EBP delivery with attention to culture should be an integrated part of treatment, supporting engagement and therapeutic alliance.³⁰ The VA National Center for PTSD consultation program is a resource to support clinicians in assessing and incorporating relevant aspects of cultural identity.³¹ For example, the National Center for PTSD provides a guide for using case conceptualization to address patient reactions to race-based violence during PTSD treatment.³² Both manualized design and therapist certification training can reinforce that assessing and attending to case conceptualization (including identity factors) is an integral component of TF-EBP.^33,34

Limitations

While the current study has numerous strengths (eg, national veteran sampling, robust qualitative methods), results should be considered within the context of study limitations. First, veteran participants all received TF-EBP, and the perspectives of veterans who never initiate TF-EBP may differ. Despite the strong sampling approach, the study design is not intended to be generalizable to all veterans receiving TF-EBP for PTSD. Qualitative analysis yielded 15 themes, described in this study and prior research, consistent with recommendations.^21,22 This approach allows rich description of distinct focus areas that would not be possible in a single manuscript. Nonetheless, all veterans interviewed described their experiences in TF-EBP and general mental health care, the focus of the semistructured interview guide was on the experience of transitioning from other treatment to TF-EBP.

Conclusion

This study describes themes related to general mental health and TF-EBP process improvement as part of a larger study on transitions in PTSD care.^21,22 Veterans valued the fundamentals of therapy, including rapport and flexibility. Treatment-specific rapport (eg, pointing out treatment progress and effort in completing treatment components) and flexibility within the context of fidelity (ie, personalizing treatment while maintaining core treatment elements) may be most effective at engaging veterans in recommended PTSD treatments.^18,34 In addition to successes, themes suggest multiple opportunities for process improvement. Ongoing VA initiatives and priorities (ie, BHIP, shared decision-making, consultation services) aim to improve processes consistent with veteran recommendations. Future research is needed to evaluate the success of these and other programs to optimize access to and engagement in recommended PTSD treatments.

Deconditioning among hospitalized older adults contributes to significant decline in posthospitalization functional ability, physical performance, and physical activity.^1-10 Previous hospital-to-home interventions have targeted improving function and physical activity, including recent programs leveraging home telehealth as a feasible and potentially effective mode of delivering in-home exercise and rehabilitation.^11-14 However, pilot interventions have shown mixed effectiveness.^11,12,14 This study expands on a previously published intervention describing a pilot home telehealth program for veterans posthospital discharge that demonstrated significant 6-month improvement in physical activity as well as trends in physical function improvement, including among those with cognitive impairment.¹⁵ Factors that contribute to improved outcomes are the focus of the present study.

Key factors underlying the complexity of hospital-to-home transitions include hospitalization elements (ie, reason for admission and length of stay), associated posthospital syndromes (ie, postdischarge falls, medication changes, cognitive impairment, and pain), and postdischarge health care application (ie, physical therapy and hospital readmission).^16-18 These factors may be associated with postdischarge functional ability, physical performance, and physical activity, but their direct influence on intervention outcomes is unclear (Figure 1).^5,7,9,16-20 The objective of this study was to examine the influence of hospitalization, posthospital syndrome, and postdischarge health care application factors on outcomes of a US Department of Veterans Affairs (VA) Video Connect (VVC) intervention to enhance function and physical activity in older adults posthospital discharge.

Methods

The previous analysis reported on patient characteristics, program feasibility, and preliminary outcomes.^13,15 The current study reports on relationships between hospitalization, posthospital syndrome, and postdischarge health care application factors and change in key outcomes, namely postdischarge self-reported functional ability, physical performance, and physical activity from baseline to endpoint.

Participants provided written informed consent. The protocol and consent forms were approved by the VA Ann Arbor Healthcare System (VAAAHS) Research and Development Committee, and the project was registered on clinicaltrials.gov (NCT04045054).

Intervention

The pilot program targeted older adults following recent hospital discharge from VAAAHS. Participants were eligible if they were aged ≥ 50 years, had been discharged following an inpatient stay in the past 1 to 2 weeks, evaluated by physical therapy during hospitalization with stated rehabilitation goals on discharge, and followed by a VAAAHS primary care physician. Participants were either recruited during hospital admission or shortly after discharge.¹³

An experienced physical activity trainer (PAT) supported the progression of participants’ rehabilitation goals via a home exercise program and coached the patient and caregiver to optimize functional ability, physical performance, and physical activity. The PAT was a nonlicensed research assistant with extensive experience in applying standard physical activity enhancement protocols (eg, increased walking) to older adults with comorbidities. Participation in the program lasted about 6 months. Initiation of the PAT program was delayed if the patient was already receiving postdischarge home-based or outpatient physical therapy. The PAT contacted the patient weekly via VVC for the first 6 weeks, then monthly for a total of 6 months. Each contact included information on optimal walking form, injury prevention, program progression, and ways to incorporate sit-to-stand transitions, nonsitting behavior, and walking into daily routines. The initial VVC contact lasted about 60 minutes and subsequent sessions lasted about 30 minutes.¹³

Demographic characteristics were self-reported by participants and included age, sex, race, years of education, and marital status. Clinical characteristics were obtained from each participant’s electronic health record (EHR), including copay status, index hospitalization length of stay, admission diagnosis, and postsurgery status (postsurgery vs nonpostsurgery). Intervention adherence was tracked as the number of PAT sessions attended.

Posthospital Syndrome Factors

Participant falls (categorized as those who reported a fall vs those who did not) and medication changes (number of changes reported, including new medication, discontinued medication, dose changes, medication changes, or changes in medication schedule) were reported by participants or caregivers during each VVC contact. Participants completed the Montreal Cognitive Assessment (MoCA) at baseline, and were dichotomized into 2 groups: no cognitive impairment (MoCA score ≥ 26) and mild to moderate cognitive impairment (MoCA score 10-25).^13,21

Participants rated how much pain interfered with their normal daily activities since the previous VVC session on a 5-point Likert scale (1, not at all; to 5, extremely).²² Similar to prior research, participants were placed into 2 groups based on their mean pain interference score (individuals with scores from 1.0 to 2.0 in 1 group, and individuals with > 2.0 in another).^23-25 Participants were separated into a no or mild pain interference group and a moderate to severe pain interference group. Hospital readmissions (VA and non-VA) and postdischarge physical therapy outcomes were obtained from the participant’s EHR, including primary care visits.

Outcomes

Outcomes were collected at baseline (posthospital discharge) and 6 months postenrollment.

Self-Reported Functional Ability. This measure is provided by participants or caregivers and measured by the Katz Index of Independence in Activities of Daily Living (ADL), Lawton and Brody Instrumental ADL Scale (IADL), Nagi Disability Model, and Rosow-Breslau Scale. The Katz ADL assesses the ability to complete 6 self-care activities and awards 1 point for independence and 0 if the individual is dependent (total score range, 0-6).²⁶ The Lawton and Brody IADL measures an individual’s independence in 8 instrumental ADLs; it awards 1 point for independence and 0 if the individual is dependent (total score range, 0-8).²⁷ The Nagi Disability Model evaluates an individual’s difficulty performing 5 tasks (total score range, 0-5) and tallies the number of items with a response other than “no difficulty at all” (higher total score indicates greater difficulty). ²⁸ The Rosow-Breslau Scale is a 3-item measure of mobility disability; individual responses are 0 (no help) and 1 (requires help or unable); higher total score (range, 0-3) indicates greater disability.²⁹

Physical Performance. Measured using the Short Physical Performance Battery (SPPB), which evaluates standing balance, sit to stand, and walking performance. Scores range from 0 to 4 on the balance, gait speed, and chair stand tests, for a total composite score between 0 and 12 (higher score indicates better performance).³⁰

Physical Activity. Measured using actigraphy, namely a physical activity monitor adherent to the thigh (activ-PAL3TM, PAL Technologies Ltd., Glasgow, UK).³¹ Participants were instructed to wear the activPal for ≥ 1 week. Participants with a minimum of 5 days of wear were included in this analysis.

Data Analyses

Analyses were performed using SPSS software version 29.0.³² Continuous variables were summarized using mean (SD) or median and IQR using the weighted average method; categorical variables were summarized using frequencies and percentages. Baseline scores on outcome variables were compared by categorical hospitalization, posthospital syndrome, and postdischarge health care application factor variables using Mann-Whitney U tests. The differences between outcome variables from baseline to endpoint were then calculated to produce change scores. Relationships between the number of PAT sessions attended and baseline outcomes and outcome change scores were estimated using Spearman correlations. Relationships between categorical factors (hospitalization, posthospital syndrome, and postdischarge health care application) and outcome variable change scores (which were normally distributed) were examined using Mann-Whitney U tests. Relationships with continuous hospitalization (length of stay) and posthospital syndrome factors (medication changes) were estimated using Spearman correlations. Effect sizes (ES) were estimated with Cohen d; small (d = 0.2), medium (d = 0.5), or large (d ≥ 0.8). Missing data were handled using pairwise deletion.³³ Therefore, sample sizes were reported for each analysis. For all statistical tests, P < .05 was considered significant.

Results

Twenty-four individuals completed the pilot intervention.¹⁵ Mean (SD) age was 73.6 (8.1) years (range, 64-93 years) and participants were predominantly White males (Table 1). Eight participants had a high school education only and 13 had more than a high school education. Diagnoses at admission included 9 patients with orthopedic/musculoskeletal conditions (6 were for joint replacement), 6 patients with vascular/pulmonary conditions, and 4 with gastrointestinal/renal/urological conditions. Of the 11 postsurgery participants, 7 were orthopedic, 4 were gastrointestinal, and 1 was peripheral vascular.

Baseline outcome scores did not differ significantly between groups, except individuals with moderate to severe pain interference reported a significantly lower IADL score (median [IQR] 4 [2-7]) than individuals with mild or moderate pain interference (median [IQR] 8 [7-8]; P = .02) (Table 2). The mean (SD) number of PAT sessions attended was 9.3 (3.7) (range, 3-19). There were no significant relationships between number of sessions attended and any baseline outcome variables or outcome change scores.

Hospitalization Factors

Participants who were postsurgery tended to have greater improvement than individuals who were nonpostsurgery in ADLs (median [IQR] 0 [0-1.5]; ES, 0.6; P = .10) and SPPB (median [IQR] 2 [1.5-9]; ES, 0.9; P = .07), but the improvements were not statistically significant (Table 3). Mean (SD) length of stay of the index hospitalization was 6.7 (6.1) days. Longer length of stay was significantly correlated with an increase in Nagi score (ρ, 0.45; 95% CI, 0.01-0.75). There were no other significant or trending relationships between length of stay and outcome variables.

Posthospital Syndrome Factors

The 16 participants with mild to moderate cognitive impairment had less improvement in ADLs (median [IQR] 0 [0-1]) than the 8 participants with no impairment (median [IQR] 0 [-0.75 to 0]; ES, -1.1; P = .04). Change in outcome variables from baseline to endpoint did not significantly differ between the 8 patients who reported a fall compared with the 13 who did not, nor were any trends observed. Change in outcome variables from baseline to endpoint also did not significantly differ between the 8 participants who reported no or mild pain interference compared with the 10 patients with moderate to severe pain interference, nor were any trends observed. Mean (SD) number of medication changes was 2.5 (1.6). Higher number of medication changes was significantly correlated with a decrease in Rosow-Breslau score (ρ, -0.47; 95% CI, -0.76 to -0.02). There were no other significant or trending relationships between number of medication changes and outcome variables.

Postdischarge Health Care Application Factors

The 16 participants who attended posthospital physical therapy trended towards less improvement in IADLs (median [IQR] 0 [-0.5 to 1.5]; ES, -0.7; P = .11) and SPPB (median [IQR] 2 [-3.0 to 4.5]; ES, -0.5; P = .15) than the 8 patients with no postdischarge physical therapy. Eleven participants were readmitted, while 13 had no readmissions in their medical records between baseline and endpoint. Participants with ≥ 1 readmission experienced a greater increase in Rosow-Breslau score (median [IQR] 0 [-0.5 to 1.0]) than those not readmitted (median [IQR] 0 [-1.25 to 0.25]; ES, 1.0; P = .03). Borderline greater improvement in number of steps was found in those not readmitted (median [IQR] 3365.6 [274.4-7710.9]) compared with those readmitted (median [IQR] 319.9 [-136.1 to 774.5]; ES, -1.3; P = .05). Patients who were readmitted also tended to have lower and not statistically significant improvements in SPPB (median [IQR] 1 [-4.0 to 5.3]) compared with those not readmitted (median [IQR] 2 [0.3-3.8]; ES, -0.5; P = .17) (Table 3).

Discussion

This study examined the association between hospitalization, posthospital syndrome, and postdischarge health care use in patients undergoing a VVC-based intervention following hospital discharge. Participants who had no or mild cognitive impairment, no readmissions, higher medication changes, and a shorter hospital length of stay tended to experience lower disability, including in mobility and ADLs. This suggests individuals who are less clinically complex may be more likely to benefit from this type of virtual rehabilitation program. These findings are consistent with clinical experiences; home-based programs to improve physical activity posthospital discharge can be challenging for those who were medically ill (and did not undergo a specific surgical procedure), cognitively impaired, and become acutely ill and trigger hospital readmission. ¹⁵ For example, the sample in this study had higher rates of falls, pain, and readmissions compared to previous research.^2,3,34-39

The importance of posthospital syndrome in the context of recovery of function and health at home following hospitalization is well documented.^16-18 The potential impact of posthospital syndrome on physical activity-focused interventions is less understood. In our analysis, participants with mild or moderate cognitive impairment tended to become more dependent in their ADLs, while those with no cognitive impairment tended to become more independent in their ADLs. This functional decline over time is perhaps expected in persons with cognitive impairment, but the significant difference with a large ES warrants further consideration on how to tailor interventions to better promote functional recovery in these individuals.^40,41 While some cognitive decline may not be preventable, this finding supports the need to promote healthy cognitive aging, identify declines in cognition, and work to mitigate additional decline. Programs specifically designed to promote function and physical activity in older adults with cognitive impairment are needed, especially during care transitions.^41-43

While participants reported that falls and pain interference did not have a significant impact on change in outcomes between baseline and endpoint, these areas need further investigation. Falls and pain have been associated with function and physical activity in older adults.^42-46 Pain is common, yet underappreciated during older adult hospital-to-home transitions.^11,12,45,46 There is a need for more comprehensive assessment of pain (including pain intensity) and qualitative research.

Hospitalization and postdischarge health care application factors may have a significant impact on home-telehealth physical activity intervention success. Individuals who were postsurgery tended to have greater improvements in ADLs and physical performance. Most postsurgery participants had joint replacement surgery. Postsurgery status may not be modifiable, but it is important to note expected differences in recovery between medical and surgical admissions and the need to tailor care based on admission diagnosis. Those with a longer length of hospital stay may be considered at higher risk of suboptimal outcomes postdischarge, which indicates an opportunity for targeting resources and support, in addition to efforts of reducing length of stay where possible.⁴⁷

Readmissions were significantly related to a change in Rosow-Breslau mobility disability score. This may indicate the detrimental impact a readmission can have on increasing mobility and physical activity postdischarge, or the potential of this pilot program to impact readmissions by increasing mobility and physical activity, contrary to prior physical exercise interventions.^5,7,9,48 With 5% to 79% of readmissions considered preventable, continued efforts and program dissemination and implementation to address preventable readmissions are warranted.⁴⁹ Individuals with postdischarge physical therapy (prior to beginning the pilot program) tended to demonstrate less improvement in disability and physical performance. This relationship needs further investigation; the 2 groups did not appear to have significant differences at baseline, albeit with a small sample size. It is possible they experienced initial improvements with postdischarge physical therapy and plateaued or had little further reserve to improve upon entering the VVC program.

Strengths and Limitations

This pilot program provided evaluative data on the use of VVC to enhance function and physical activity in older adults posthospital discharge. It included individual (eg, fall, pain, cognitive impairment) and health service (eg, readmission, physical therapy) level factors as predictors of function and physical activity posthospitalization.^5,7,9,15-19

The results of this pilot project stem from a small sample lacking diversity in terms of race, ethnicity, and sex. There was some variation in baseline and endpoints between participants, and when hospitalization, posthospital syndrome, and postdischarge health care application factors were collected. The majority of participants were recruited within a month postdischarge, and the program lasted about 6 months. Data collection was attempted at regular PAT contacts, but there was some variation in when visits occurred based on participant availability and preference. Some participants had missing data, which was handled using pairwise deletion.³³ Larger studies are needed to confirm the findings of this study, particularly the trends that did not reach statistical significance. Home health services other than physical therapy (eg, nursing, occupational therapy) were not fully accounted for and should be considered in future research.

Conclusions

In patients undergoing a 6-month pilot VVC-based physical activity intervention posthospital discharge, improvements in mobility and disability were most likely in those who had no cognitive impairment and were not readmitted. Larger sample and qualitative investigations are necessary to optimize outcomes for patients who meet these clinical profiles.

Deconditioning among hospitalized older adults contributes to significant decline in posthospitalization functional ability, physical performance, and physical activity.^1-10 Previous hospital-to-home interventions have targeted improving function and physical activity, including recent programs leveraging home telehealth as a feasible and potentially effective mode of delivering in-home exercise and rehabilitation.^11-14 However, pilot interventions have shown mixed effectiveness.^11,12,14 This study expands on a previously published intervention describing a pilot home telehealth program for veterans posthospital discharge that demonstrated significant 6-month improvement in physical activity as well as trends in physical function improvement, including among those with cognitive impairment.¹⁵ Factors that contribute to improved outcomes are the focus of the present study.

Key factors underlying the complexity of hospital-to-home transitions include hospitalization elements (ie, reason for admission and length of stay), associated posthospital syndromes (ie, postdischarge falls, medication changes, cognitive impairment, and pain), and postdischarge health care application (ie, physical therapy and hospital readmission).^16-18 These factors may be associated with postdischarge functional ability, physical performance, and physical activity, but their direct influence on intervention outcomes is unclear (Figure 1).^5,7,9,16-20 The objective of this study was to examine the influence of hospitalization, posthospital syndrome, and postdischarge health care application factors on outcomes of a US Department of Veterans Affairs (VA) Video Connect (VVC) intervention to enhance function and physical activity in older adults posthospital discharge.

Methods

The previous analysis reported on patient characteristics, program feasibility, and preliminary outcomes.^13,15 The current study reports on relationships between hospitalization, posthospital syndrome, and postdischarge health care application factors and change in key outcomes, namely postdischarge self-reported functional ability, physical performance, and physical activity from baseline to endpoint.

Participants provided written informed consent. The protocol and consent forms were approved by the VA Ann Arbor Healthcare System (VAAAHS) Research and Development Committee, and the project was registered on clinicaltrials.gov (NCT04045054).

Intervention

The pilot program targeted older adults following recent hospital discharge from VAAAHS. Participants were eligible if they were aged ≥ 50 years, had been discharged following an inpatient stay in the past 1 to 2 weeks, evaluated by physical therapy during hospitalization with stated rehabilitation goals on discharge, and followed by a VAAAHS primary care physician. Participants were either recruited during hospital admission or shortly after discharge.¹³

An experienced physical activity trainer (PAT) supported the progression of participants’ rehabilitation goals via a home exercise program and coached the patient and caregiver to optimize functional ability, physical performance, and physical activity. The PAT was a nonlicensed research assistant with extensive experience in applying standard physical activity enhancement protocols (eg, increased walking) to older adults with comorbidities. Participation in the program lasted about 6 months. Initiation of the PAT program was delayed if the patient was already receiving postdischarge home-based or outpatient physical therapy. The PAT contacted the patient weekly via VVC for the first 6 weeks, then monthly for a total of 6 months. Each contact included information on optimal walking form, injury prevention, program progression, and ways to incorporate sit-to-stand transitions, nonsitting behavior, and walking into daily routines. The initial VVC contact lasted about 60 minutes and subsequent sessions lasted about 30 minutes.¹³

Demographic characteristics were self-reported by participants and included age, sex, race, years of education, and marital status. Clinical characteristics were obtained from each participant’s electronic health record (EHR), including copay status, index hospitalization length of stay, admission diagnosis, and postsurgery status (postsurgery vs nonpostsurgery). Intervention adherence was tracked as the number of PAT sessions attended.

Posthospital Syndrome Factors

Participant falls (categorized as those who reported a fall vs those who did not) and medication changes (number of changes reported, including new medication, discontinued medication, dose changes, medication changes, or changes in medication schedule) were reported by participants or caregivers during each VVC contact. Participants completed the Montreal Cognitive Assessment (MoCA) at baseline, and were dichotomized into 2 groups: no cognitive impairment (MoCA score ≥ 26) and mild to moderate cognitive impairment (MoCA score 10-25).^13,21

Participants rated how much pain interfered with their normal daily activities since the previous VVC session on a 5-point Likert scale (1, not at all; to 5, extremely).²² Similar to prior research, participants were placed into 2 groups based on their mean pain interference score (individuals with scores from 1.0 to 2.0 in 1 group, and individuals with > 2.0 in another).^23-25 Participants were separated into a no or mild pain interference group and a moderate to severe pain interference group. Hospital readmissions (VA and non-VA) and postdischarge physical therapy outcomes were obtained from the participant’s EHR, including primary care visits.

Outcomes

Outcomes were collected at baseline (posthospital discharge) and 6 months postenrollment.

Self-Reported Functional Ability. This measure is provided by participants or caregivers and measured by the Katz Index of Independence in Activities of Daily Living (ADL), Lawton and Brody Instrumental ADL Scale (IADL), Nagi Disability Model, and Rosow-Breslau Scale. The Katz ADL assesses the ability to complete 6 self-care activities and awards 1 point for independence and 0 if the individual is dependent (total score range, 0-6).²⁶ The Lawton and Brody IADL measures an individual’s independence in 8 instrumental ADLs; it awards 1 point for independence and 0 if the individual is dependent (total score range, 0-8).²⁷ The Nagi Disability Model evaluates an individual’s difficulty performing 5 tasks (total score range, 0-5) and tallies the number of items with a response other than “no difficulty at all” (higher total score indicates greater difficulty). ²⁸ The Rosow-Breslau Scale is a 3-item measure of mobility disability; individual responses are 0 (no help) and 1 (requires help or unable); higher total score (range, 0-3) indicates greater disability.²⁹

Physical Performance. Measured using the Short Physical Performance Battery (SPPB), which evaluates standing balance, sit to stand, and walking performance. Scores range from 0 to 4 on the balance, gait speed, and chair stand tests, for a total composite score between 0 and 12 (higher score indicates better performance).³⁰

Physical Activity. Measured using actigraphy, namely a physical activity monitor adherent to the thigh (activ-PAL3TM, PAL Technologies Ltd., Glasgow, UK).³¹ Participants were instructed to wear the activPal for ≥ 1 week. Participants with a minimum of 5 days of wear were included in this analysis.

Data Analyses

Analyses were performed using SPSS software version 29.0.³² Continuous variables were summarized using mean (SD) or median and IQR using the weighted average method; categorical variables were summarized using frequencies and percentages. Baseline scores on outcome variables were compared by categorical hospitalization, posthospital syndrome, and postdischarge health care application factor variables using Mann-Whitney U tests. The differences between outcome variables from baseline to endpoint were then calculated to produce change scores. Relationships between the number of PAT sessions attended and baseline outcomes and outcome change scores were estimated using Spearman correlations. Relationships between categorical factors (hospitalization, posthospital syndrome, and postdischarge health care application) and outcome variable change scores (which were normally distributed) were examined using Mann-Whitney U tests. Relationships with continuous hospitalization (length of stay) and posthospital syndrome factors (medication changes) were estimated using Spearman correlations. Effect sizes (ES) were estimated with Cohen d; small (d = 0.2), medium (d = 0.5), or large (d ≥ 0.8). Missing data were handled using pairwise deletion.³³ Therefore, sample sizes were reported for each analysis. For all statistical tests, P < .05 was considered significant.

Results

Twenty-four individuals completed the pilot intervention.¹⁵ Mean (SD) age was 73.6 (8.1) years (range, 64-93 years) and participants were predominantly White males (Table 1). Eight participants had a high school education only and 13 had more than a high school education. Diagnoses at admission included 9 patients with orthopedic/musculoskeletal conditions (6 were for joint replacement), 6 patients with vascular/pulmonary conditions, and 4 with gastrointestinal/renal/urological conditions. Of the 11 postsurgery participants, 7 were orthopedic, 4 were gastrointestinal, and 1 was peripheral vascular.

Baseline outcome scores did not differ significantly between groups, except individuals with moderate to severe pain interference reported a significantly lower IADL score (median [IQR] 4 [2-7]) than individuals with mild or moderate pain interference (median [IQR] 8 [7-8]; P = .02) (Table 2). The mean (SD) number of PAT sessions attended was 9.3 (3.7) (range, 3-19). There were no significant relationships between number of sessions attended and any baseline outcome variables or outcome change scores.

Hospitalization Factors

Participants who were postsurgery tended to have greater improvement than individuals who were nonpostsurgery in ADLs (median [IQR] 0 [0-1.5]; ES, 0.6; P = .10) and SPPB (median [IQR] 2 [1.5-9]; ES, 0.9; P = .07), but the improvements were not statistically significant (Table 3). Mean (SD) length of stay of the index hospitalization was 6.7 (6.1) days. Longer length of stay was significantly correlated with an increase in Nagi score (ρ, 0.45; 95% CI, 0.01-0.75). There were no other significant or trending relationships between length of stay and outcome variables.

Posthospital Syndrome Factors

The 16 participants with mild to moderate cognitive impairment had less improvement in ADLs (median [IQR] 0 [0-1]) than the 8 participants with no impairment (median [IQR] 0 [-0.75 to 0]; ES, -1.1; P = .04). Change in outcome variables from baseline to endpoint did not significantly differ between the 8 patients who reported a fall compared with the 13 who did not, nor were any trends observed. Change in outcome variables from baseline to endpoint also did not significantly differ between the 8 participants who reported no or mild pain interference compared with the 10 patients with moderate to severe pain interference, nor were any trends observed. Mean (SD) number of medication changes was 2.5 (1.6). Higher number of medication changes was significantly correlated with a decrease in Rosow-Breslau score (ρ, -0.47; 95% CI, -0.76 to -0.02). There were no other significant or trending relationships between number of medication changes and outcome variables.

Postdischarge Health Care Application Factors

The 16 participants who attended posthospital physical therapy trended towards less improvement in IADLs (median [IQR] 0 [-0.5 to 1.5]; ES, -0.7; P = .11) and SPPB (median [IQR] 2 [-3.0 to 4.5]; ES, -0.5; P = .15) than the 8 patients with no postdischarge physical therapy. Eleven participants were readmitted, while 13 had no readmissions in their medical records between baseline and endpoint. Participants with ≥ 1 readmission experienced a greater increase in Rosow-Breslau score (median [IQR] 0 [-0.5 to 1.0]) than those not readmitted (median [IQR] 0 [-1.25 to 0.25]; ES, 1.0; P = .03). Borderline greater improvement in number of steps was found in those not readmitted (median [IQR] 3365.6 [274.4-7710.9]) compared with those readmitted (median [IQR] 319.9 [-136.1 to 774.5]; ES, -1.3; P = .05). Patients who were readmitted also tended to have lower and not statistically significant improvements in SPPB (median [IQR] 1 [-4.0 to 5.3]) compared with those not readmitted (median [IQR] 2 [0.3-3.8]; ES, -0.5; P = .17) (Table 3).

Discussion

This study examined the association between hospitalization, posthospital syndrome, and postdischarge health care use in patients undergoing a VVC-based intervention following hospital discharge. Participants who had no or mild cognitive impairment, no readmissions, higher medication changes, and a shorter hospital length of stay tended to experience lower disability, including in mobility and ADLs. This suggests individuals who are less clinically complex may be more likely to benefit from this type of virtual rehabilitation program. These findings are consistent with clinical experiences; home-based programs to improve physical activity posthospital discharge can be challenging for those who were medically ill (and did not undergo a specific surgical procedure), cognitively impaired, and become acutely ill and trigger hospital readmission. ¹⁵ For example, the sample in this study had higher rates of falls, pain, and readmissions compared to previous research.^2,3,34-39

The importance of posthospital syndrome in the context of recovery of function and health at home following hospitalization is well documented.^16-18 The potential impact of posthospital syndrome on physical activity-focused interventions is less understood. In our analysis, participants with mild or moderate cognitive impairment tended to become more dependent in their ADLs, while those with no cognitive impairment tended to become more independent in their ADLs. This functional decline over time is perhaps expected in persons with cognitive impairment, but the significant difference with a large ES warrants further consideration on how to tailor interventions to better promote functional recovery in these individuals.^40,41 While some cognitive decline may not be preventable, this finding supports the need to promote healthy cognitive aging, identify declines in cognition, and work to mitigate additional decline. Programs specifically designed to promote function and physical activity in older adults with cognitive impairment are needed, especially during care transitions.^41-43

While participants reported that falls and pain interference did not have a significant impact on change in outcomes between baseline and endpoint, these areas need further investigation. Falls and pain have been associated with function and physical activity in older adults.^42-46 Pain is common, yet underappreciated during older adult hospital-to-home transitions.^11,12,45,46 There is a need for more comprehensive assessment of pain (including pain intensity) and qualitative research.

Hospitalization and postdischarge health care application factors may have a significant impact on home-telehealth physical activity intervention success. Individuals who were postsurgery tended to have greater improvements in ADLs and physical performance. Most postsurgery participants had joint replacement surgery. Postsurgery status may not be modifiable, but it is important to note expected differences in recovery between medical and surgical admissions and the need to tailor care based on admission diagnosis. Those with a longer length of hospital stay may be considered at higher risk of suboptimal outcomes postdischarge, which indicates an opportunity for targeting resources and support, in addition to efforts of reducing length of stay where possible.⁴⁷

Readmissions were significantly related to a change in Rosow-Breslau mobility disability score. This may indicate the detrimental impact a readmission can have on increasing mobility and physical activity postdischarge, or the potential of this pilot program to impact readmissions by increasing mobility and physical activity, contrary to prior physical exercise interventions.^5,7,9,48 With 5% to 79% of readmissions considered preventable, continued efforts and program dissemination and implementation to address preventable readmissions are warranted.⁴⁹ Individuals with postdischarge physical therapy (prior to beginning the pilot program) tended to demonstrate less improvement in disability and physical performance. This relationship needs further investigation; the 2 groups did not appear to have significant differences at baseline, albeit with a small sample size. It is possible they experienced initial improvements with postdischarge physical therapy and plateaued or had little further reserve to improve upon entering the VVC program.

Strengths and Limitations

This pilot program provided evaluative data on the use of VVC to enhance function and physical activity in older adults posthospital discharge. It included individual (eg, fall, pain, cognitive impairment) and health service (eg, readmission, physical therapy) level factors as predictors of function and physical activity posthospitalization.^5,7,9,15-19

The results of this pilot project stem from a small sample lacking diversity in terms of race, ethnicity, and sex. There was some variation in baseline and endpoints between participants, and when hospitalization, posthospital syndrome, and postdischarge health care application factors were collected. The majority of participants were recruited within a month postdischarge, and the program lasted about 6 months. Data collection was attempted at regular PAT contacts, but there was some variation in when visits occurred based on participant availability and preference. Some participants had missing data, which was handled using pairwise deletion.³³ Larger studies are needed to confirm the findings of this study, particularly the trends that did not reach statistical significance. Home health services other than physical therapy (eg, nursing, occupational therapy) were not fully accounted for and should be considered in future research.

Conclusions

In patients undergoing a 6-month pilot VVC-based physical activity intervention posthospital discharge, improvements in mobility and disability were most likely in those who had no cognitive impairment and were not readmitted. Larger sample and qualitative investigations are necessary to optimize outcomes for patients who meet these clinical profiles.

Deconditioning among hospitalized older adults contributes to significant decline in posthospitalization functional ability, physical performance, and physical activity.^1-10 Previous hospital-to-home interventions have targeted improving function and physical activity, including recent programs leveraging home telehealth as a feasible and potentially effective mode of delivering in-home exercise and rehabilitation.^11-14 However, pilot interventions have shown mixed effectiveness.^11,12,14 This study expands on a previously published intervention describing a pilot home telehealth program for veterans posthospital discharge that demonstrated significant 6-month improvement in physical activity as well as trends in physical function improvement, including among those with cognitive impairment.¹⁵ Factors that contribute to improved outcomes are the focus of the present study.

Key factors underlying the complexity of hospital-to-home transitions include hospitalization elements (ie, reason for admission and length of stay), associated posthospital syndromes (ie, postdischarge falls, medication changes, cognitive impairment, and pain), and postdischarge health care application (ie, physical therapy and hospital readmission).^16-18 These factors may be associated with postdischarge functional ability, physical performance, and physical activity, but their direct influence on intervention outcomes is unclear (Figure 1).^5,7,9,16-20 The objective of this study was to examine the influence of hospitalization, posthospital syndrome, and postdischarge health care application factors on outcomes of a US Department of Veterans Affairs (VA) Video Connect (VVC) intervention to enhance function and physical activity in older adults posthospital discharge.

Methods

The previous analysis reported on patient characteristics, program feasibility, and preliminary outcomes.^13,15 The current study reports on relationships between hospitalization, posthospital syndrome, and postdischarge health care application factors and change in key outcomes, namely postdischarge self-reported functional ability, physical performance, and physical activity from baseline to endpoint.

Participants provided written informed consent. The protocol and consent forms were approved by the VA Ann Arbor Healthcare System (VAAAHS) Research and Development Committee, and the project was registered on clinicaltrials.gov (NCT04045054).

Intervention

The pilot program targeted older adults following recent hospital discharge from VAAAHS. Participants were eligible if they were aged ≥ 50 years, had been discharged following an inpatient stay in the past 1 to 2 weeks, evaluated by physical therapy during hospitalization with stated rehabilitation goals on discharge, and followed by a VAAAHS primary care physician. Participants were either recruited during hospital admission or shortly after discharge.¹³

An experienced physical activity trainer (PAT) supported the progression of participants’ rehabilitation goals via a home exercise program and coached the patient and caregiver to optimize functional ability, physical performance, and physical activity. The PAT was a nonlicensed research assistant with extensive experience in applying standard physical activity enhancement protocols (eg, increased walking) to older adults with comorbidities. Participation in the program lasted about 6 months. Initiation of the PAT program was delayed if the patient was already receiving postdischarge home-based or outpatient physical therapy. The PAT contacted the patient weekly via VVC for the first 6 weeks, then monthly for a total of 6 months. Each contact included information on optimal walking form, injury prevention, program progression, and ways to incorporate sit-to-stand transitions, nonsitting behavior, and walking into daily routines. The initial VVC contact lasted about 60 minutes and subsequent sessions lasted about 30 minutes.¹³

Demographic characteristics were self-reported by participants and included age, sex, race, years of education, and marital status. Clinical characteristics were obtained from each participant’s electronic health record (EHR), including copay status, index hospitalization length of stay, admission diagnosis, and postsurgery status (postsurgery vs nonpostsurgery). Intervention adherence was tracked as the number of PAT sessions attended.

Posthospital Syndrome Factors

Participant falls (categorized as those who reported a fall vs those who did not) and medication changes (number of changes reported, including new medication, discontinued medication, dose changes, medication changes, or changes in medication schedule) were reported by participants or caregivers during each VVC contact. Participants completed the Montreal Cognitive Assessment (MoCA) at baseline, and were dichotomized into 2 groups: no cognitive impairment (MoCA score ≥ 26) and mild to moderate cognitive impairment (MoCA score 10-25).^13,21

Participants rated how much pain interfered with their normal daily activities since the previous VVC session on a 5-point Likert scale (1, not at all; to 5, extremely).²² Similar to prior research, participants were placed into 2 groups based on their mean pain interference score (individuals with scores from 1.0 to 2.0 in 1 group, and individuals with > 2.0 in another).^23-25 Participants were separated into a no or mild pain interference group and a moderate to severe pain interference group. Hospital readmissions (VA and non-VA) and postdischarge physical therapy outcomes were obtained from the participant’s EHR, including primary care visits.

Outcomes

Outcomes were collected at baseline (posthospital discharge) and 6 months postenrollment.

Self-Reported Functional Ability. This measure is provided by participants or caregivers and measured by the Katz Index of Independence in Activities of Daily Living (ADL), Lawton and Brody Instrumental ADL Scale (IADL), Nagi Disability Model, and Rosow-Breslau Scale. The Katz ADL assesses the ability to complete 6 self-care activities and awards 1 point for independence and 0 if the individual is dependent (total score range, 0-6).²⁶ The Lawton and Brody IADL measures an individual’s independence in 8 instrumental ADLs; it awards 1 point for independence and 0 if the individual is dependent (total score range, 0-8).²⁷ The Nagi Disability Model evaluates an individual’s difficulty performing 5 tasks (total score range, 0-5) and tallies the number of items with a response other than “no difficulty at all” (higher total score indicates greater difficulty). ²⁸ The Rosow-Breslau Scale is a 3-item measure of mobility disability; individual responses are 0 (no help) and 1 (requires help or unable); higher total score (range, 0-3) indicates greater disability.²⁹

Physical Performance. Measured using the Short Physical Performance Battery (SPPB), which evaluates standing balance, sit to stand, and walking performance. Scores range from 0 to 4 on the balance, gait speed, and chair stand tests, for a total composite score between 0 and 12 (higher score indicates better performance).³⁰

Physical Activity. Measured using actigraphy, namely a physical activity monitor adherent to the thigh (activ-PAL3TM, PAL Technologies Ltd., Glasgow, UK).³¹ Participants were instructed to wear the activPal for ≥ 1 week. Participants with a minimum of 5 days of wear were included in this analysis.

Data Analyses

Analyses were performed using SPSS software version 29.0.³² Continuous variables were summarized using mean (SD) or median and IQR using the weighted average method; categorical variables were summarized using frequencies and percentages. Baseline scores on outcome variables were compared by categorical hospitalization, posthospital syndrome, and postdischarge health care application factor variables using Mann-Whitney U tests. The differences between outcome variables from baseline to endpoint were then calculated to produce change scores. Relationships between the number of PAT sessions attended and baseline outcomes and outcome change scores were estimated using Spearman correlations. Relationships between categorical factors (hospitalization, posthospital syndrome, and postdischarge health care application) and outcome variable change scores (which were normally distributed) were examined using Mann-Whitney U tests. Relationships with continuous hospitalization (length of stay) and posthospital syndrome factors (medication changes) were estimated using Spearman correlations. Effect sizes (ES) were estimated with Cohen d; small (d = 0.2), medium (d = 0.5), or large (d ≥ 0.8). Missing data were handled using pairwise deletion.³³ Therefore, sample sizes were reported for each analysis. For all statistical tests, P < .05 was considered significant.

Results

Twenty-four individuals completed the pilot intervention.¹⁵ Mean (SD) age was 73.6 (8.1) years (range, 64-93 years) and participants were predominantly White males (Table 1). Eight participants had a high school education only and 13 had more than a high school education. Diagnoses at admission included 9 patients with orthopedic/musculoskeletal conditions (6 were for joint replacement), 6 patients with vascular/pulmonary conditions, and 4 with gastrointestinal/renal/urological conditions. Of the 11 postsurgery participants, 7 were orthopedic, 4 were gastrointestinal, and 1 was peripheral vascular.

Baseline outcome scores did not differ significantly between groups, except individuals with moderate to severe pain interference reported a significantly lower IADL score (median [IQR] 4 [2-7]) than individuals with mild or moderate pain interference (median [IQR] 8 [7-8]; P = .02) (Table 2). The mean (SD) number of PAT sessions attended was 9.3 (3.7) (range, 3-19). There were no significant relationships between number of sessions attended and any baseline outcome variables or outcome change scores.

Hospitalization Factors

Participants who were postsurgery tended to have greater improvement than individuals who were nonpostsurgery in ADLs (median [IQR] 0 [0-1.5]; ES, 0.6; P = .10) and SPPB (median [IQR] 2 [1.5-9]; ES, 0.9; P = .07), but the improvements were not statistically significant (Table 3). Mean (SD) length of stay of the index hospitalization was 6.7 (6.1) days. Longer length of stay was significantly correlated with an increase in Nagi score (ρ, 0.45; 95% CI, 0.01-0.75). There were no other significant or trending relationships between length of stay and outcome variables.

Posthospital Syndrome Factors

The 16 participants with mild to moderate cognitive impairment had less improvement in ADLs (median [IQR] 0 [0-1]) than the 8 participants with no impairment (median [IQR] 0 [-0.75 to 0]; ES, -1.1; P = .04). Change in outcome variables from baseline to endpoint did not significantly differ between the 8 patients who reported a fall compared with the 13 who did not, nor were any trends observed. Change in outcome variables from baseline to endpoint also did not significantly differ between the 8 participants who reported no or mild pain interference compared with the 10 patients with moderate to severe pain interference, nor were any trends observed. Mean (SD) number of medication changes was 2.5 (1.6). Higher number of medication changes was significantly correlated with a decrease in Rosow-Breslau score (ρ, -0.47; 95% CI, -0.76 to -0.02). There were no other significant or trending relationships between number of medication changes and outcome variables.

Postdischarge Health Care Application Factors

The 16 participants who attended posthospital physical therapy trended towards less improvement in IADLs (median [IQR] 0 [-0.5 to 1.5]; ES, -0.7; P = .11) and SPPB (median [IQR] 2 [-3.0 to 4.5]; ES, -0.5; P = .15) than the 8 patients with no postdischarge physical therapy. Eleven participants were readmitted, while 13 had no readmissions in their medical records between baseline and endpoint. Participants with ≥ 1 readmission experienced a greater increase in Rosow-Breslau score (median [IQR] 0 [-0.5 to 1.0]) than those not readmitted (median [IQR] 0 [-1.25 to 0.25]; ES, 1.0; P = .03). Borderline greater improvement in number of steps was found in those not readmitted (median [IQR] 3365.6 [274.4-7710.9]) compared with those readmitted (median [IQR] 319.9 [-136.1 to 774.5]; ES, -1.3; P = .05). Patients who were readmitted also tended to have lower and not statistically significant improvements in SPPB (median [IQR] 1 [-4.0 to 5.3]) compared with those not readmitted (median [IQR] 2 [0.3-3.8]; ES, -0.5; P = .17) (Table 3).

Discussion

This study examined the association between hospitalization, posthospital syndrome, and postdischarge health care use in patients undergoing a VVC-based intervention following hospital discharge. Participants who had no or mild cognitive impairment, no readmissions, higher medication changes, and a shorter hospital length of stay tended to experience lower disability, including in mobility and ADLs. This suggests individuals who are less clinically complex may be more likely to benefit from this type of virtual rehabilitation program. These findings are consistent with clinical experiences; home-based programs to improve physical activity posthospital discharge can be challenging for those who were medically ill (and did not undergo a specific surgical procedure), cognitively impaired, and become acutely ill and trigger hospital readmission. ¹⁵ For example, the sample in this study had higher rates of falls, pain, and readmissions compared to previous research.^2,3,34-39

The importance of posthospital syndrome in the context of recovery of function and health at home following hospitalization is well documented.^16-18 The potential impact of posthospital syndrome on physical activity-focused interventions is less understood. In our analysis, participants with mild or moderate cognitive impairment tended to become more dependent in their ADLs, while those with no cognitive impairment tended to become more independent in their ADLs. This functional decline over time is perhaps expected in persons with cognitive impairment, but the significant difference with a large ES warrants further consideration on how to tailor interventions to better promote functional recovery in these individuals.^40,41 While some cognitive decline may not be preventable, this finding supports the need to promote healthy cognitive aging, identify declines in cognition, and work to mitigate additional decline. Programs specifically designed to promote function and physical activity in older adults with cognitive impairment are needed, especially during care transitions.^41-43

While participants reported that falls and pain interference did not have a significant impact on change in outcomes between baseline and endpoint, these areas need further investigation. Falls and pain have been associated with function and physical activity in older adults.^42-46 Pain is common, yet underappreciated during older adult hospital-to-home transitions.^11,12,45,46 There is a need for more comprehensive assessment of pain (including pain intensity) and qualitative research.

Hospitalization and postdischarge health care application factors may have a significant impact on home-telehealth physical activity intervention success. Individuals who were postsurgery tended to have greater improvements in ADLs and physical performance. Most postsurgery participants had joint replacement surgery. Postsurgery status may not be modifiable, but it is important to note expected differences in recovery between medical and surgical admissions and the need to tailor care based on admission diagnosis. Those with a longer length of hospital stay may be considered at higher risk of suboptimal outcomes postdischarge, which indicates an opportunity for targeting resources and support, in addition to efforts of reducing length of stay where possible.⁴⁷

Readmissions were significantly related to a change in Rosow-Breslau mobility disability score. This may indicate the detrimental impact a readmission can have on increasing mobility and physical activity postdischarge, or the potential of this pilot program to impact readmissions by increasing mobility and physical activity, contrary to prior physical exercise interventions.^5,7,9,48 With 5% to 79% of readmissions considered preventable, continued efforts and program dissemination and implementation to address preventable readmissions are warranted.⁴⁹ Individuals with postdischarge physical therapy (prior to beginning the pilot program) tended to demonstrate less improvement in disability and physical performance. This relationship needs further investigation; the 2 groups did not appear to have significant differences at baseline, albeit with a small sample size. It is possible they experienced initial improvements with postdischarge physical therapy and plateaued or had little further reserve to improve upon entering the VVC program.

Strengths and Limitations

This pilot program provided evaluative data on the use of VVC to enhance function and physical activity in older adults posthospital discharge. It included individual (eg, fall, pain, cognitive impairment) and health service (eg, readmission, physical therapy) level factors as predictors of function and physical activity posthospitalization.^5,7,9,15-19

The results of this pilot project stem from a small sample lacking diversity in terms of race, ethnicity, and sex. There was some variation in baseline and endpoints between participants, and when hospitalization, posthospital syndrome, and postdischarge health care application factors were collected. The majority of participants were recruited within a month postdischarge, and the program lasted about 6 months. Data collection was attempted at regular PAT contacts, but there was some variation in when visits occurred based on participant availability and preference. Some participants had missing data, which was handled using pairwise deletion.³³ Larger studies are needed to confirm the findings of this study, particularly the trends that did not reach statistical significance. Home health services other than physical therapy (eg, nursing, occupational therapy) were not fully accounted for and should be considered in future research.

Conclusions

In patients undergoing a 6-month pilot VVC-based physical activity intervention posthospital discharge, improvements in mobility and disability were most likely in those who had no cognitive impairment and were not readmitted. Larger sample and qualitative investigations are necessary to optimize outcomes for patients who meet these clinical profiles.

More than 38 million people in the United States (12%) have diabetes mellitus (DM), though 1 in 5 are unaware they have DM.¹ The prevalence among veterans is even more substantial, impacting nearly 25% of those who received care from the US Department of Veterans Affairs (VA).² DM can lead to increased health care costs in addition to various complications (eg, cardiovascular, renal), especially if left uncontrolled.^1,3 similar impact is found in the perioperative period (defined as at or around the time of an operation), as multiple studies have found that uncontrolled preoperative DM can result in worsened surgical outcomes, including longer hospital stays, more infectious complications, and higher perioperative mortality.^4-6

In contrast, adequate glycemic control assessed with blood glucose levels has been shown to decrease the incidence of postoperative infections.⁷ Optimizing glycemic control during hospital stays, especially postsurgery, has become the standard of care, with most health systems establishing specific protocols. In current literature, most studies examining DM management in the perioperative period are focused on postoperative care, with little attention to the preoperative period.^4,6,7

One study found that patients with poor presurgery glycemic control assessed by hemoglobin A_1c (HbA_1c) levels were more likely to remain hyperglycemic during and after surgery. 8 Blood glucose levels < 200 mg/dL can lead to an increased risk of infection and impaired wound healing, meaning a well-controlled HbA_1c before a procedure serves as a potential factor for success.⁹ The 2025 American Diabetes Association (ADA) Standards of Care (SOC) recommendation is to target HbA_1c < 8% whenever possible, and some health systems require lower levels (eg, < 7% or 7.5%).¹⁰ With that goal in mind and knowing that preoperative hyperglycemia has been shown to be a contributing factor in the delay or cancellation of surgical cases, an argument can be made that attention to preoperative DM management also should be a focus for health care systems performing surgeries.^8,9,11

Attention to glucose control during preoperative care offers an opportunity to screen for DM in patients who may not have been screened otherwise and to standardize perioperative DM management. Since DM disproportionately impacts veterans, this is a pertinent issue to the VA. Veterans can be more susceptible to complications if DM is left uncontrolled prior to surgery. To determine readiness for surgery and control of comorbid conditions such as DM before a planned surgery, facilities often perform a preoperative clinic assessment, often in a multidisciplinary clinic.

At Veteran Health Indiana (VHI), a presurgery clinic visit involving the primary surgery service (physician, nurse practitioner, and/or a physician assistant) is conducted 1 to 2 months prior to the planned procedure to determine whether a patient is ready for surgery. During this visit, patients receive a packet with instructions for various tasks and medications, such as applying topical antibiotic prophylaxis on the anticipated surgical site. This is documented in the form of a note in the VHI Computerized Patient Record System (CPRS). The medication instructions are provided according to the preferences of the surgical team. These may be templated notes that contain general directions on the timing and dosing of specific medications, in addition to instructions for holding or reducing doses when appropriate. The instructions can be tailored by the team conducting the preoperative visit (eg, “Take 20 units of insulin glargine the day before surgery” vs “Take half of your long-acting insulin the night before surgery”). Specific to DM, VHI has a nurse-driven day of surgery glucose assessment where point-of-care blood glucose is collected during preoperative holding for most patients.

There is limited research assessing the level of preoperative glycemic control and the incidence of complications in a veteran population. The objective of this study was to gain a baseline understanding of what, if any, standardization exists for preoperative instructions for DM medications and to assess the level of preoperative glycemic control and postoperative complications in patients with DM undergoing major elective surgical procedures.

Methods

This retrospective, single-center chart review was conducted at VHI. The Indiana University and VHI institutional review boards determined that this quality improvement project was exempt from review.

The primary outcome was the number of patients with surgical procedures delayed or canceled due to hyperglycemia or hypoglycemia. Hyperglycemia was defined as blood glucose > 180 mg/dL and hypoglycemia was defined as < 70 mg/dL, slight variations from the current ADA SOC preoperative specific recommendation of a blood glucose reading of 100 to 180 mg/dL within 4 hours of surgery.¹⁰ The standard outpatient hypoglycemia definition of blood glucose < 70 mg/dL was chosen because the current goal (< 100 mg/dL) was not the standard in previous ADA SOCs that were in place during the study period. Specifically, the 2018 ADA SOC did not provide preoperative recommendations and the 2019-2021 ADA SOC recommended 80 to 180 mg/dL.^10,12-18 For patients who had multiple preoperative blood glucose measurements, the first recorded glucose on the day of the procedure was used.

The secondary outcomes of this study were focused on the preoperative process/care at VHI and postoperative glycemic control. The preoperative process included examining whether medication instructions were given and their quality. Additionally, the number of interventions for hyperglycemia and hypoglycemia were required immediately prior to surgery and the average preoperative HbA_1c (measured within 3 months prior to surgery) were collected and analyzed. For postoperative glycemic control, average blood glucose measurements and number of hypoglycemic (< 70 mg/dL) and hyperglycemic (> 180 mg/dL) events were measured in addition to the frequency of changes made at discharge to patients’ DM medication regimens.

The safety outcome of this study assessed commonly observed postoperative complications and was examined up to 30 days postsurgery. These included acute kidney injury (defined using Kidney Disease: Improving Global Outcomes 2012, the standard during the study period), nonfatal myocardial infarction, nonfatal stroke, and surgical site infections, which were identified from the discharge summary written by the primary surgery service.¹⁹ All-cause mortality also was collected.

Patients were included if they were admitted for major elective surgeries and had a diagnosis of either type 1 or type 2 DM on their problem list, determined by International Classification of Diseases, Tenth Revision codes. Major elective surgery was defined as a procedure that would likely result in a hospital admission of > 24 hours. Of note, patients may have been included in this study more than once if they had > 1 procedure at least 30 days apart and met inclusion criteria within the time frame. Patients were excluded if they were taking no DM medications or chronic steroids (at any dose), residing in a long-term care facility, being managed by a non-VA clinician prior to surgery, or missing a preoperative blood glucose measurement.

All data were collected from the CPRS. A list of surgical cases involving patients with DM who were scheduled to undergo major elective surgeries from January 1, 2018, to December 31, 2021, at VHI was generated. The list was randomized to a smaller number (N = 394) for data collection due to the time and resource constraints for a pharmacy residency project. All data were deidentified and stored in a secured VA server to protect patient confidentiality. Descriptive statistics were used for all results.

Results

Initially, 2362 surgeries were identified. A randomized sample of 394 charts were reviewed and 131 cases met inclusion criteria. Each case involved a unique patient (Figure). The most common reasons for exclusion were 143 patients with diet-controlled DM and 78 nonelective surgeries. The mean (SD) age of patients was 68 (8) years, and the most were male (98.5%) and White (76.3%) (Table 1). 

At baseline, 45 of 131 patients (34.4%) had coronary artery disease and 29 (22.1%) each had autonomic neuropathy and chronic kidney disease. Most surgeries were conducted by orthopedic (32.1%) and peripheral vascular (21.4%) specialties. The mean (SD) length of surgery was 4.6 (2.6) hours and of hospital length of stay was 4 (4) days. No patients stayed longer than the 30-day safety outcome follow-up period. All patients had type 2 DM and took a mean 2 DM medications. The 63 patients taking insulin had a mean (SD) total daily dose of 99 (77) U (Table 2). A preoperative HbA1c was collected in 116 patients within 3 months of surgery, with a mean HbA_1c of 7.0% (range, 5.3-10.7).

No patients had surgeries delayed or canceled because of uncontrolled DM on the day of surgery. The mean preoperative blood glucose level was 146 mg/dL (range, 73-365) (Table 3). No patients had a preoperative blood glucose level of < 70 mg/dL and 19 (14.5%) had a blood glucose level > 180 mg/dL. Among patients with hyperglycemia immediately prior to surgery, 6 (31.6%) had documentation of insulin being provided.

For this sample of patients, the preoperative clinic visit was conducted a mean 22 days prior to the planned surgery date. Among the 131 included patients, 122 (93.1%) had documentation of receiving instructions for DM medications. Among patients who had documented receipt of instructions, only 30 (24.6%) had instructions specifically tailored to their regimen rather than a generic templated form. The mean (SD) preoperative blood glucose was similar for those who received specific perioperative DM instructions at 146 (50) mg/dL when compared with those who did not at 147 (45) mg/dL. The mean (SD) preoperative blood glucose reading for those who had no documentation of receipt of perioperative instructions was 126 (54) mg/dL compared with 147 (46) mg/dL for those who did.

The mean number of postoperative blood glucose events per day was negligible for hypoglycemia and more frequent for hyperglycemia with a mean of 2 events per day. The mean postoperative blood glucose range was 121 to 247 mg/dL with most readings < 180 mg/dL. Upon discharge, most patients continued their home DM regimen with 5 patients (3.8%) having changes made to their regimen upon discharge.

Very few postoperative complications were identified from chart review. The most frequently observed postoperative complications were acute kidney injury, surgical site infections, and nonfatal stroke. There were no documented nonfatal myocardial infarctions. Two patients (1.5%) died within 30 days of the surgery; neither death was deemed to have been related to poor perioperative glycemic control.

Discussion

To our knowledge, this retrospective chart review was the first study to assess preoperative DM management and postoperative complications in a veteran population. VHI is a large, tertiary, level 1a, academic medical center that serves approximately 62,000 veterans annually and performs about 5000 to 6000 surgeries annually, a total that is increasing following the COVID-19 pandemic.²⁰ This study found that the current process of a presurgery clinic visit and day of surgery glucose assessment has prevented surgical delays or cancellations.

Most patients included in this study were well controlled at baseline in accordance with the 2025 ADA SOC HbA_1c recommendation of a preoperative HbA1c of < 8%, which may have contributed to no surgical delays or cancellations.¹⁰ However, not all patients had HbA1c collected within 3 months of surgery or even had one collected at all. Despite the ADA SOC providing no explicit recommendation for universal HbA_1c screening prior to elective procedures, its importance cannot be understated given the body of evidence demonstrating poor outcomes with uncontrolled preoperative DM.^8,10 The glycemic control at baseline may have contributed to the very few postsurgical complications observed in this study.

Although the current process at VHI prevented surgical delays and cancellations in this sample, there are still identified areas for improvement. One area is the instructions the patients received. Patients with DM are often prescribed ≥ 1 medication or a combination of insulins, noninsulin injectables, and oral DM medications, and this study population was no different. Because these medications may influence the anesthesia and perioperative periods, the ADA has specific guidance for altering administration schedules in the days leading up to surgery.¹⁰

Inappropriate administration of DM medications could lead to perioperative hypoglycemia or hyperglycemia, possibly causing surgical delays, case cancellations, and/or postoperative complications.21 Although these data reveal the specificity and documented receipt that the preoperative DM instructions did not impact the first recorded preoperative blood glucose, future studies should examine patient confidence in how to properly administer their DM medications prior to surgery. It is vital that patients receive clear instructions in accordance with the ADA SOC on whether to continue, hold, or adjust the dose of their medications to prevent fluctuations in blood glucose levels in the perioperative period, ensure safety with anesthesia, and prevent postoperative complications such as acute kidney injury. Of note, compliance with guideline recommendations for medication instructions was not examined because the data collection time frame expanded over multiple years and the recommendations have evolved each year as new data emerge.

Preoperative DM Management

The first key takeaway from this study is to ensure patients are ready for surgery with a formal assessment (typically in the form of a clinic visit) prior to the surgery. One private sector health system published their approach to this by administering an automatic preoperative HbA_1c screening for those with a DM diagnosis and all patients with a random plasma glucose ≥ 200 mg/dL.²² Additionally, if the patient's HbA_1c level was not at goal prior to surgery (≥ 8% for those with known DM and ≥ 6.5% with no known DM), patients were referred to endocrinology for further management. Increasing attention to the preoperative visit and extending HbA_1c testing to all patients regardless of DM status also provides an opportunity to identify individuals living with undiagnosed DM.¹

Even though there was no difference in the mean preoperative blood glucose level based on receipt or specificity of preoperative DM instructions, a second takeaway from this study is the importance of ensuring patients receive clear instructions on their DM medication schedule in the perioperative period. A practical first step may be updating the templates used by the primary surgery teams and providing education to the clinicians in the clinic on how to personalize the visits. Because the current preoperative DM process at VHI is managed by the primary surgical team in a clinic visit, there is an opportunity to shift this responsibility to other health care professionals, such as pharmacists—a change shown to reduce unintended omission of home medications following surgery during hospitalization and reduce costs.^23,24

Limitations

This study relied on data included in the patient chart. These data include medication interventions made immediately prior to surgery, which can sometimes be inaccurately charted or difficult to find as they are not documented in the typical medication administration record. Also, the safety outcomes were collected from a discharge summary written by different clinicians, which may lead to information bias. Special attention was taken to ensure these data points were collected as accurately as possible, but it is possible some data may be inaccurate from unintentional human error. Additionally, the safety outcome was limited to a 30-day follow-up, but encompassed the entire length of postoperative stay for all included patients. Finally, given this study was retrospective with no comparison group and the intent was to improve processes at VHI, only hypotheses and potential interventions can be generated from this study. Future prospective studies with larger sample sizes and comparator groups are needed to draw further conclusions.

Conclusions

This study found that the current presurgery process at VHI appears to be successful in preventing surgical delays or cancellations due to hyperglycemia or hypoglycemia. Optimizing DM management can improve surgical outcomes by decreasing rates of postoperative complications, and this study added additional evidence in support of that in a unique population: veterans. Insight on the awareness of preoperative blood glucose management should be gleaned from this study, and based on this sample and site, the preadmission screening process and instructions provided to patients can serve as 2 starting points for optimizing elective surgery.

More than 38 million people in the United States (12%) have diabetes mellitus (DM), though 1 in 5 are unaware they have DM.¹ The prevalence among veterans is even more substantial, impacting nearly 25% of those who received care from the US Department of Veterans Affairs (VA).² DM can lead to increased health care costs in addition to various complications (eg, cardiovascular, renal), especially if left uncontrolled.^1,3 similar impact is found in the perioperative period (defined as at or around the time of an operation), as multiple studies have found that uncontrolled preoperative DM can result in worsened surgical outcomes, including longer hospital stays, more infectious complications, and higher perioperative mortality.^4-6

In contrast, adequate glycemic control assessed with blood glucose levels has been shown to decrease the incidence of postoperative infections.⁷ Optimizing glycemic control during hospital stays, especially postsurgery, has become the standard of care, with most health systems establishing specific protocols. In current literature, most studies examining DM management in the perioperative period are focused on postoperative care, with little attention to the preoperative period.^4,6,7

One study found that patients with poor presurgery glycemic control assessed by hemoglobin A_1c (HbA_1c) levels were more likely to remain hyperglycemic during and after surgery. 8 Blood glucose levels < 200 mg/dL can lead to an increased risk of infection and impaired wound healing, meaning a well-controlled HbA_1c before a procedure serves as a potential factor for success.⁹ The 2025 American Diabetes Association (ADA) Standards of Care (SOC) recommendation is to target HbA_1c < 8% whenever possible, and some health systems require lower levels (eg, < 7% or 7.5%).¹⁰ With that goal in mind and knowing that preoperative hyperglycemia has been shown to be a contributing factor in the delay or cancellation of surgical cases, an argument can be made that attention to preoperative DM management also should be a focus for health care systems performing surgeries.^8,9,11

Attention to glucose control during preoperative care offers an opportunity to screen for DM in patients who may not have been screened otherwise and to standardize perioperative DM management. Since DM disproportionately impacts veterans, this is a pertinent issue to the VA. Veterans can be more susceptible to complications if DM is left uncontrolled prior to surgery. To determine readiness for surgery and control of comorbid conditions such as DM before a planned surgery, facilities often perform a preoperative clinic assessment, often in a multidisciplinary clinic.

At Veteran Health Indiana (VHI), a presurgery clinic visit involving the primary surgery service (physician, nurse practitioner, and/or a physician assistant) is conducted 1 to 2 months prior to the planned procedure to determine whether a patient is ready for surgery. During this visit, patients receive a packet with instructions for various tasks and medications, such as applying topical antibiotic prophylaxis on the anticipated surgical site. This is documented in the form of a note in the VHI Computerized Patient Record System (CPRS). The medication instructions are provided according to the preferences of the surgical team. These may be templated notes that contain general directions on the timing and dosing of specific medications, in addition to instructions for holding or reducing doses when appropriate. The instructions can be tailored by the team conducting the preoperative visit (eg, “Take 20 units of insulin glargine the day before surgery” vs “Take half of your long-acting insulin the night before surgery”). Specific to DM, VHI has a nurse-driven day of surgery glucose assessment where point-of-care blood glucose is collected during preoperative holding for most patients.

There is limited research assessing the level of preoperative glycemic control and the incidence of complications in a veteran population. The objective of this study was to gain a baseline understanding of what, if any, standardization exists for preoperative instructions for DM medications and to assess the level of preoperative glycemic control and postoperative complications in patients with DM undergoing major elective surgical procedures.

Methods

This retrospective, single-center chart review was conducted at VHI. The Indiana University and VHI institutional review boards determined that this quality improvement project was exempt from review.

The primary outcome was the number of patients with surgical procedures delayed or canceled due to hyperglycemia or hypoglycemia. Hyperglycemia was defined as blood glucose > 180 mg/dL and hypoglycemia was defined as < 70 mg/dL, slight variations from the current ADA SOC preoperative specific recommendation of a blood glucose reading of 100 to 180 mg/dL within 4 hours of surgery.¹⁰ The standard outpatient hypoglycemia definition of blood glucose < 70 mg/dL was chosen because the current goal (< 100 mg/dL) was not the standard in previous ADA SOCs that were in place during the study period. Specifically, the 2018 ADA SOC did not provide preoperative recommendations and the 2019-2021 ADA SOC recommended 80 to 180 mg/dL.^10,12-18 For patients who had multiple preoperative blood glucose measurements, the first recorded glucose on the day of the procedure was used.

The secondary outcomes of this study were focused on the preoperative process/care at VHI and postoperative glycemic control. The preoperative process included examining whether medication instructions were given and their quality. Additionally, the number of interventions for hyperglycemia and hypoglycemia were required immediately prior to surgery and the average preoperative HbA_1c (measured within 3 months prior to surgery) were collected and analyzed. For postoperative glycemic control, average blood glucose measurements and number of hypoglycemic (< 70 mg/dL) and hyperglycemic (> 180 mg/dL) events were measured in addition to the frequency of changes made at discharge to patients’ DM medication regimens.

The safety outcome of this study assessed commonly observed postoperative complications and was examined up to 30 days postsurgery. These included acute kidney injury (defined using Kidney Disease: Improving Global Outcomes 2012, the standard during the study period), nonfatal myocardial infarction, nonfatal stroke, and surgical site infections, which were identified from the discharge summary written by the primary surgery service.¹⁹ All-cause mortality also was collected.

Patients were included if they were admitted for major elective surgeries and had a diagnosis of either type 1 or type 2 DM on their problem list, determined by International Classification of Diseases, Tenth Revision codes. Major elective surgery was defined as a procedure that would likely result in a hospital admission of > 24 hours. Of note, patients may have been included in this study more than once if they had > 1 procedure at least 30 days apart and met inclusion criteria within the time frame. Patients were excluded if they were taking no DM medications or chronic steroids (at any dose), residing in a long-term care facility, being managed by a non-VA clinician prior to surgery, or missing a preoperative blood glucose measurement.

All data were collected from the CPRS. A list of surgical cases involving patients with DM who were scheduled to undergo major elective surgeries from January 1, 2018, to December 31, 2021, at VHI was generated. The list was randomized to a smaller number (N = 394) for data collection due to the time and resource constraints for a pharmacy residency project. All data were deidentified and stored in a secured VA server to protect patient confidentiality. Descriptive statistics were used for all results.

Results

Initially, 2362 surgeries were identified. A randomized sample of 394 charts were reviewed and 131 cases met inclusion criteria. Each case involved a unique patient (Figure). The most common reasons for exclusion were 143 patients with diet-controlled DM and 78 nonelective surgeries. The mean (SD) age of patients was 68 (8) years, and the most were male (98.5%) and White (76.3%) (Table 1). 

At baseline, 45 of 131 patients (34.4%) had coronary artery disease and 29 (22.1%) each had autonomic neuropathy and chronic kidney disease. Most surgeries were conducted by orthopedic (32.1%) and peripheral vascular (21.4%) specialties. The mean (SD) length of surgery was 4.6 (2.6) hours and of hospital length of stay was 4 (4) days. No patients stayed longer than the 30-day safety outcome follow-up period. All patients had type 2 DM and took a mean 2 DM medications. The 63 patients taking insulin had a mean (SD) total daily dose of 99 (77) U (Table 2). A preoperative HbA1c was collected in 116 patients within 3 months of surgery, with a mean HbA_1c of 7.0% (range, 5.3-10.7).

No patients had surgeries delayed or canceled because of uncontrolled DM on the day of surgery. The mean preoperative blood glucose level was 146 mg/dL (range, 73-365) (Table 3). No patients had a preoperative blood glucose level of < 70 mg/dL and 19 (14.5%) had a blood glucose level > 180 mg/dL. Among patients with hyperglycemia immediately prior to surgery, 6 (31.6%) had documentation of insulin being provided.

For this sample of patients, the preoperative clinic visit was conducted a mean 22 days prior to the planned surgery date. Among the 131 included patients, 122 (93.1%) had documentation of receiving instructions for DM medications. Among patients who had documented receipt of instructions, only 30 (24.6%) had instructions specifically tailored to their regimen rather than a generic templated form. The mean (SD) preoperative blood glucose was similar for those who received specific perioperative DM instructions at 146 (50) mg/dL when compared with those who did not at 147 (45) mg/dL. The mean (SD) preoperative blood glucose reading for those who had no documentation of receipt of perioperative instructions was 126 (54) mg/dL compared with 147 (46) mg/dL for those who did.

The mean number of postoperative blood glucose events per day was negligible for hypoglycemia and more frequent for hyperglycemia with a mean of 2 events per day. The mean postoperative blood glucose range was 121 to 247 mg/dL with most readings < 180 mg/dL. Upon discharge, most patients continued their home DM regimen with 5 patients (3.8%) having changes made to their regimen upon discharge.

Very few postoperative complications were identified from chart review. The most frequently observed postoperative complications were acute kidney injury, surgical site infections, and nonfatal stroke. There were no documented nonfatal myocardial infarctions. Two patients (1.5%) died within 30 days of the surgery; neither death was deemed to have been related to poor perioperative glycemic control.

Discussion

To our knowledge, this retrospective chart review was the first study to assess preoperative DM management and postoperative complications in a veteran population. VHI is a large, tertiary, level 1a, academic medical center that serves approximately 62,000 veterans annually and performs about 5000 to 6000 surgeries annually, a total that is increasing following the COVID-19 pandemic.²⁰ This study found that the current process of a presurgery clinic visit and day of surgery glucose assessment has prevented surgical delays or cancellations.

Most patients included in this study were well controlled at baseline in accordance with the 2025 ADA SOC HbA_1c recommendation of a preoperative HbA1c of < 8%, which may have contributed to no surgical delays or cancellations.¹⁰ However, not all patients had HbA1c collected within 3 months of surgery or even had one collected at all. Despite the ADA SOC providing no explicit recommendation for universal HbA_1c screening prior to elective procedures, its importance cannot be understated given the body of evidence demonstrating poor outcomes with uncontrolled preoperative DM.^8,10 The glycemic control at baseline may have contributed to the very few postsurgical complications observed in this study.

Although the current process at VHI prevented surgical delays and cancellations in this sample, there are still identified areas for improvement. One area is the instructions the patients received. Patients with DM are often prescribed ≥ 1 medication or a combination of insulins, noninsulin injectables, and oral DM medications, and this study population was no different. Because these medications may influence the anesthesia and perioperative periods, the ADA has specific guidance for altering administration schedules in the days leading up to surgery.¹⁰

Inappropriate administration of DM medications could lead to perioperative hypoglycemia or hyperglycemia, possibly causing surgical delays, case cancellations, and/or postoperative complications.21 Although these data reveal the specificity and documented receipt that the preoperative DM instructions did not impact the first recorded preoperative blood glucose, future studies should examine patient confidence in how to properly administer their DM medications prior to surgery. It is vital that patients receive clear instructions in accordance with the ADA SOC on whether to continue, hold, or adjust the dose of their medications to prevent fluctuations in blood glucose levels in the perioperative period, ensure safety with anesthesia, and prevent postoperative complications such as acute kidney injury. Of note, compliance with guideline recommendations for medication instructions was not examined because the data collection time frame expanded over multiple years and the recommendations have evolved each year as new data emerge.

Preoperative DM Management

The first key takeaway from this study is to ensure patients are ready for surgery with a formal assessment (typically in the form of a clinic visit) prior to the surgery. One private sector health system published their approach to this by administering an automatic preoperative HbA_1c screening for those with a DM diagnosis and all patients with a random plasma glucose ≥ 200 mg/dL.²² Additionally, if the patient's HbA_1c level was not at goal prior to surgery (≥ 8% for those with known DM and ≥ 6.5% with no known DM), patients were referred to endocrinology for further management. Increasing attention to the preoperative visit and extending HbA_1c testing to all patients regardless of DM status also provides an opportunity to identify individuals living with undiagnosed DM.¹

Even though there was no difference in the mean preoperative blood glucose level based on receipt or specificity of preoperative DM instructions, a second takeaway from this study is the importance of ensuring patients receive clear instructions on their DM medication schedule in the perioperative period. A practical first step may be updating the templates used by the primary surgery teams and providing education to the clinicians in the clinic on how to personalize the visits. Because the current preoperative DM process at VHI is managed by the primary surgical team in a clinic visit, there is an opportunity to shift this responsibility to other health care professionals, such as pharmacists—a change shown to reduce unintended omission of home medications following surgery during hospitalization and reduce costs.^23,24

Limitations

This study relied on data included in the patient chart. These data include medication interventions made immediately prior to surgery, which can sometimes be inaccurately charted or difficult to find as they are not documented in the typical medication administration record. Also, the safety outcomes were collected from a discharge summary written by different clinicians, which may lead to information bias. Special attention was taken to ensure these data points were collected as accurately as possible, but it is possible some data may be inaccurate from unintentional human error. Additionally, the safety outcome was limited to a 30-day follow-up, but encompassed the entire length of postoperative stay for all included patients. Finally, given this study was retrospective with no comparison group and the intent was to improve processes at VHI, only hypotheses and potential interventions can be generated from this study. Future prospective studies with larger sample sizes and comparator groups are needed to draw further conclusions.

Conclusions

This study found that the current presurgery process at VHI appears to be successful in preventing surgical delays or cancellations due to hyperglycemia or hypoglycemia. Optimizing DM management can improve surgical outcomes by decreasing rates of postoperative complications, and this study added additional evidence in support of that in a unique population: veterans. Insight on the awareness of preoperative blood glucose management should be gleaned from this study, and based on this sample and site, the preadmission screening process and instructions provided to patients can serve as 2 starting points for optimizing elective surgery.

More than 38 million people in the United States (12%) have diabetes mellitus (DM), though 1 in 5 are unaware they have DM.¹ The prevalence among veterans is even more substantial, impacting nearly 25% of those who received care from the US Department of Veterans Affairs (VA).² DM can lead to increased health care costs in addition to various complications (eg, cardiovascular, renal), especially if left uncontrolled.^1,3 similar impact is found in the perioperative period (defined as at or around the time of an operation), as multiple studies have found that uncontrolled preoperative DM can result in worsened surgical outcomes, including longer hospital stays, more infectious complications, and higher perioperative mortality.^4-6

In contrast, adequate glycemic control assessed with blood glucose levels has been shown to decrease the incidence of postoperative infections.⁷ Optimizing glycemic control during hospital stays, especially postsurgery, has become the standard of care, with most health systems establishing specific protocols. In current literature, most studies examining DM management in the perioperative period are focused on postoperative care, with little attention to the preoperative period.^4,6,7

One study found that patients with poor presurgery glycemic control assessed by hemoglobin A_1c (HbA_1c) levels were more likely to remain hyperglycemic during and after surgery. 8 Blood glucose levels < 200 mg/dL can lead to an increased risk of infection and impaired wound healing, meaning a well-controlled HbA_1c before a procedure serves as a potential factor for success.⁹ The 2025 American Diabetes Association (ADA) Standards of Care (SOC) recommendation is to target HbA_1c < 8% whenever possible, and some health systems require lower levels (eg, < 7% or 7.5%).¹⁰ With that goal in mind and knowing that preoperative hyperglycemia has been shown to be a contributing factor in the delay or cancellation of surgical cases, an argument can be made that attention to preoperative DM management also should be a focus for health care systems performing surgeries.^8,9,11

Attention to glucose control during preoperative care offers an opportunity to screen for DM in patients who may not have been screened otherwise and to standardize perioperative DM management. Since DM disproportionately impacts veterans, this is a pertinent issue to the VA. Veterans can be more susceptible to complications if DM is left uncontrolled prior to surgery. To determine readiness for surgery and control of comorbid conditions such as DM before a planned surgery, facilities often perform a preoperative clinic assessment, often in a multidisciplinary clinic.

At Veteran Health Indiana (VHI), a presurgery clinic visit involving the primary surgery service (physician, nurse practitioner, and/or a physician assistant) is conducted 1 to 2 months prior to the planned procedure to determine whether a patient is ready for surgery. During this visit, patients receive a packet with instructions for various tasks and medications, such as applying topical antibiotic prophylaxis on the anticipated surgical site. This is documented in the form of a note in the VHI Computerized Patient Record System (CPRS). The medication instructions are provided according to the preferences of the surgical team. These may be templated notes that contain general directions on the timing and dosing of specific medications, in addition to instructions for holding or reducing doses when appropriate. The instructions can be tailored by the team conducting the preoperative visit (eg, “Take 20 units of insulin glargine the day before surgery” vs “Take half of your long-acting insulin the night before surgery”). Specific to DM, VHI has a nurse-driven day of surgery glucose assessment where point-of-care blood glucose is collected during preoperative holding for most patients.

There is limited research assessing the level of preoperative glycemic control and the incidence of complications in a veteran population. The objective of this study was to gain a baseline understanding of what, if any, standardization exists for preoperative instructions for DM medications and to assess the level of preoperative glycemic control and postoperative complications in patients with DM undergoing major elective surgical procedures.

Methods

This retrospective, single-center chart review was conducted at VHI. The Indiana University and VHI institutional review boards determined that this quality improvement project was exempt from review.

The primary outcome was the number of patients with surgical procedures delayed or canceled due to hyperglycemia or hypoglycemia. Hyperglycemia was defined as blood glucose > 180 mg/dL and hypoglycemia was defined as < 70 mg/dL, slight variations from the current ADA SOC preoperative specific recommendation of a blood glucose reading of 100 to 180 mg/dL within 4 hours of surgery.¹⁰ The standard outpatient hypoglycemia definition of blood glucose < 70 mg/dL was chosen because the current goal (< 100 mg/dL) was not the standard in previous ADA SOCs that were in place during the study period. Specifically, the 2018 ADA SOC did not provide preoperative recommendations and the 2019-2021 ADA SOC recommended 80 to 180 mg/dL.^10,12-18 For patients who had multiple preoperative blood glucose measurements, the first recorded glucose on the day of the procedure was used.

The secondary outcomes of this study were focused on the preoperative process/care at VHI and postoperative glycemic control. The preoperative process included examining whether medication instructions were given and their quality. Additionally, the number of interventions for hyperglycemia and hypoglycemia were required immediately prior to surgery and the average preoperative HbA_1c (measured within 3 months prior to surgery) were collected and analyzed. For postoperative glycemic control, average blood glucose measurements and number of hypoglycemic (< 70 mg/dL) and hyperglycemic (> 180 mg/dL) events were measured in addition to the frequency of changes made at discharge to patients’ DM medication regimens.

The safety outcome of this study assessed commonly observed postoperative complications and was examined up to 30 days postsurgery. These included acute kidney injury (defined using Kidney Disease: Improving Global Outcomes 2012, the standard during the study period), nonfatal myocardial infarction, nonfatal stroke, and surgical site infections, which were identified from the discharge summary written by the primary surgery service.¹⁹ All-cause mortality also was collected.

Patients were included if they were admitted for major elective surgeries and had a diagnosis of either type 1 or type 2 DM on their problem list, determined by International Classification of Diseases, Tenth Revision codes. Major elective surgery was defined as a procedure that would likely result in a hospital admission of > 24 hours. Of note, patients may have been included in this study more than once if they had > 1 procedure at least 30 days apart and met inclusion criteria within the time frame. Patients were excluded if they were taking no DM medications or chronic steroids (at any dose), residing in a long-term care facility, being managed by a non-VA clinician prior to surgery, or missing a preoperative blood glucose measurement.

All data were collected from the CPRS. A list of surgical cases involving patients with DM who were scheduled to undergo major elective surgeries from January 1, 2018, to December 31, 2021, at VHI was generated. The list was randomized to a smaller number (N = 394) for data collection due to the time and resource constraints for a pharmacy residency project. All data were deidentified and stored in a secured VA server to protect patient confidentiality. Descriptive statistics were used for all results.

Results

Initially, 2362 surgeries were identified. A randomized sample of 394 charts were reviewed and 131 cases met inclusion criteria. Each case involved a unique patient (Figure). The most common reasons for exclusion were 143 patients with diet-controlled DM and 78 nonelective surgeries. The mean (SD) age of patients was 68 (8) years, and the most were male (98.5%) and White (76.3%) (Table 1). 

At baseline, 45 of 131 patients (34.4%) had coronary artery disease and 29 (22.1%) each had autonomic neuropathy and chronic kidney disease. Most surgeries were conducted by orthopedic (32.1%) and peripheral vascular (21.4%) specialties. The mean (SD) length of surgery was 4.6 (2.6) hours and of hospital length of stay was 4 (4) days. No patients stayed longer than the 30-day safety outcome follow-up period. All patients had type 2 DM and took a mean 2 DM medications. The 63 patients taking insulin had a mean (SD) total daily dose of 99 (77) U (Table 2). A preoperative HbA1c was collected in 116 patients within 3 months of surgery, with a mean HbA_1c of 7.0% (range, 5.3-10.7).

No patients had surgeries delayed or canceled because of uncontrolled DM on the day of surgery. The mean preoperative blood glucose level was 146 mg/dL (range, 73-365) (Table 3). No patients had a preoperative blood glucose level of < 70 mg/dL and 19 (14.5%) had a blood glucose level > 180 mg/dL. Among patients with hyperglycemia immediately prior to surgery, 6 (31.6%) had documentation of insulin being provided.

For this sample of patients, the preoperative clinic visit was conducted a mean 22 days prior to the planned surgery date. Among the 131 included patients, 122 (93.1%) had documentation of receiving instructions for DM medications. Among patients who had documented receipt of instructions, only 30 (24.6%) had instructions specifically tailored to their regimen rather than a generic templated form. The mean (SD) preoperative blood glucose was similar for those who received specific perioperative DM instructions at 146 (50) mg/dL when compared with those who did not at 147 (45) mg/dL. The mean (SD) preoperative blood glucose reading for those who had no documentation of receipt of perioperative instructions was 126 (54) mg/dL compared with 147 (46) mg/dL for those who did.

The mean number of postoperative blood glucose events per day was negligible for hypoglycemia and more frequent for hyperglycemia with a mean of 2 events per day. The mean postoperative blood glucose range was 121 to 247 mg/dL with most readings < 180 mg/dL. Upon discharge, most patients continued their home DM regimen with 5 patients (3.8%) having changes made to their regimen upon discharge.

Very few postoperative complications were identified from chart review. The most frequently observed postoperative complications were acute kidney injury, surgical site infections, and nonfatal stroke. There were no documented nonfatal myocardial infarctions. Two patients (1.5%) died within 30 days of the surgery; neither death was deemed to have been related to poor perioperative glycemic control.

Discussion

To our knowledge, this retrospective chart review was the first study to assess preoperative DM management and postoperative complications in a veteran population. VHI is a large, tertiary, level 1a, academic medical center that serves approximately 62,000 veterans annually and performs about 5000 to 6000 surgeries annually, a total that is increasing following the COVID-19 pandemic.²⁰ This study found that the current process of a presurgery clinic visit and day of surgery glucose assessment has prevented surgical delays or cancellations.

Most patients included in this study were well controlled at baseline in accordance with the 2025 ADA SOC HbA_1c recommendation of a preoperative HbA1c of < 8%, which may have contributed to no surgical delays or cancellations.¹⁰ However, not all patients had HbA1c collected within 3 months of surgery or even had one collected at all. Despite the ADA SOC providing no explicit recommendation for universal HbA_1c screening prior to elective procedures, its importance cannot be understated given the body of evidence demonstrating poor outcomes with uncontrolled preoperative DM.^8,10 The glycemic control at baseline may have contributed to the very few postsurgical complications observed in this study.

Although the current process at VHI prevented surgical delays and cancellations in this sample, there are still identified areas for improvement. One area is the instructions the patients received. Patients with DM are often prescribed ≥ 1 medication or a combination of insulins, noninsulin injectables, and oral DM medications, and this study population was no different. Because these medications may influence the anesthesia and perioperative periods, the ADA has specific guidance for altering administration schedules in the days leading up to surgery.¹⁰

Inappropriate administration of DM medications could lead to perioperative hypoglycemia or hyperglycemia, possibly causing surgical delays, case cancellations, and/or postoperative complications.21 Although these data reveal the specificity and documented receipt that the preoperative DM instructions did not impact the first recorded preoperative blood glucose, future studies should examine patient confidence in how to properly administer their DM medications prior to surgery. It is vital that patients receive clear instructions in accordance with the ADA SOC on whether to continue, hold, or adjust the dose of their medications to prevent fluctuations in blood glucose levels in the perioperative period, ensure safety with anesthesia, and prevent postoperative complications such as acute kidney injury. Of note, compliance with guideline recommendations for medication instructions was not examined because the data collection time frame expanded over multiple years and the recommendations have evolved each year as new data emerge.

Preoperative DM Management

The first key takeaway from this study is to ensure patients are ready for surgery with a formal assessment (typically in the form of a clinic visit) prior to the surgery. One private sector health system published their approach to this by administering an automatic preoperative HbA_1c screening for those with a DM diagnosis and all patients with a random plasma glucose ≥ 200 mg/dL.²² Additionally, if the patient's HbA_1c level was not at goal prior to surgery (≥ 8% for those with known DM and ≥ 6.5% with no known DM), patients were referred to endocrinology for further management. Increasing attention to the preoperative visit and extending HbA_1c testing to all patients regardless of DM status also provides an opportunity to identify individuals living with undiagnosed DM.¹

Even though there was no difference in the mean preoperative blood glucose level based on receipt or specificity of preoperative DM instructions, a second takeaway from this study is the importance of ensuring patients receive clear instructions on their DM medication schedule in the perioperative period. A practical first step may be updating the templates used by the primary surgery teams and providing education to the clinicians in the clinic on how to personalize the visits. Because the current preoperative DM process at VHI is managed by the primary surgical team in a clinic visit, there is an opportunity to shift this responsibility to other health care professionals, such as pharmacists—a change shown to reduce unintended omission of home medications following surgery during hospitalization and reduce costs.^23,24

Limitations

This study relied on data included in the patient chart. These data include medication interventions made immediately prior to surgery, which can sometimes be inaccurately charted or difficult to find as they are not documented in the typical medication administration record. Also, the safety outcomes were collected from a discharge summary written by different clinicians, which may lead to information bias. Special attention was taken to ensure these data points were collected as accurately as possible, but it is possible some data may be inaccurate from unintentional human error. Additionally, the safety outcome was limited to a 30-day follow-up, but encompassed the entire length of postoperative stay for all included patients. Finally, given this study was retrospective with no comparison group and the intent was to improve processes at VHI, only hypotheses and potential interventions can be generated from this study. Future prospective studies with larger sample sizes and comparator groups are needed to draw further conclusions.

Conclusions

This study found that the current presurgery process at VHI appears to be successful in preventing surgical delays or cancellations due to hyperglycemia or hypoglycemia. Optimizing DM management can improve surgical outcomes by decreasing rates of postoperative complications, and this study added additional evidence in support of that in a unique population: veterans. Insight on the awareness of preoperative blood glucose management should be gleaned from this study, and based on this sample and site, the preadmission screening process and instructions provided to patients can serve as 2 starting points for optimizing elective surgery.