Article

Regularly scheduled educational conferences, such as case-based morning reports, have been a standard part of internal medicine residencies for decades.^1-4 In addition to better patient care from the knowledge gained at educational conferences, attendance by interns and residents (collectively called house staff) may be associated with higher in-service examination scores.⁵ Unfortunately, competing priorities, including patient care and trainee supervision, may contribute to an action-intention gap among house staff that reduces attendance.^6-8 Low attendance at morning reports represents wasted effort and lost educational opportunities; therefore, strategies to increase attendance are needed. Of several methods studied, more resource-intensive interventions (eg, providing food) were the most successful.^6,9-12

Using the behavioral economics framework of nudge strategies, we hypothesized that a less intensive intervention of a daily reminder text page would encourage medical students, interns, and residents (collectively called learners) to attend the morning report conference.^8,13 However, given the high cognitive load created by frequent task switching, a reminder text page could disrupt workflow and patient care without promoting the intended behavior change.^14-17 Because of this uncertainty, our objective was to determine whether a preconference text page increased learner attendance at morning report conferences.

Methods

This study was a single-center, multiple-crossover cluster randomized controlled trial conducted at the Veteran Affairs Boston Healthcare System (VABHS) in Massachusetts. Study participants included house staff rotating on daytime inpatient rotations from 4 residency programs and students from 2 medical schools. The setting was the morning report, an in-person, interactive, case-based conference held Monday through Thursday, from 8:00 am to 8:45 am. On Friday mornings, the morning report was replaced with a medical Jeopardy game-style conference. Historically, attendance has not been recorded for these conferences.

Learners assigned to rotate on the inpatient medicine, cardiology, medicine consultation, and patient safety rotations were eligible to attend these conferences and for inclusion in the study. Learners rotating in the medical intensive care unit, on night float, or on day float (an admitting shift for which residents are not on-site until late afternoon) were excluded. Additional details of the study population are available in the supplement (eAppendix). The study period was originally planned for September 30, 2019, to March 31, 2020, but data collection was stopped on March 12, 2020, due to the COVID-19 pandemic and suspension of in-person conferences. We chose the study period, which determined our sample size, to exclude the first 3 months of the academic year (July-September) because during that time learners acclimate to the inpatient workflow. We also chose not to include the last 3 months of the academic year to provide time for data analysis and preparation of the manuscript within the academic year.

Intervention and Outcome Assessment

Each intervention and control period was 3 weeks long; the first period was randomly determined by coin flip and alternated thereafter. Additional details of randomization are available in the supplement (Appendix 1). During intervention periods, all house staff received a page at 7:55 am that listed the time and location of the upcoming morning report or Jeopardy conference. Medical students do not carry pagers and did not receive reminder pages; however, we included these learners because changes in their conference attendance behavior would indicate an extension of the effect of reminder pages beyond the individual learner who received the page.

A daily facesheet (a roster of house staff names and photos) was used to identify learners for conference attendance. This facesheet was already used for other purposes at VABHS. At 8:00 am and 8:10 am, a chief medical resident who was not blinded to the intervention or control period recorded the attendance of each eligible learner as present or absent; learners were unaware that their attendance was being recorded. This approach to data collection was selected to minimize the likelihood that the behavior of the study participants would be influenced.

During control periods, no text page reminder of upcoming conferences was sent, but the attendance of total learners at 8:00 am and 8:10 am was recorded by a chief medical resident who used the same method as during the intervention periods. Attendance at 8:10 am was chosen as the primary outcome to account for the possibility that learners may arrive after a conference begins. Attendance at 8:00 am also was recorded to assess the effect of reminder pages on attendance at the start of morning reports.

Statistical Analysis

The primary outcome was the proportion of eligible learners present at 8:10 am at the morning report, expressed as the risk difference for attendance between intervention and control periods. Secondary outcomes included the proportion of learners present at 8:00 am (on-time attendance), the proportion of learners present by type (student vs house staff), and the proportion of learners present at the Friday Jeopardy conference. Two preplanned subgroup analyses were performed: one assessing the impact of rotating on clinical services with lighter workloads, and the other assessing the impact of the number of overnight admissions received on the relationship between receipt of a reminder page and conference attendance.

To estimate the primary outcome, we modeled the risk difference adjusted for covariates using a generalized estimating equation accounting for the clustering of attendance behavior within individuals and controlling for date and team. Secondary outcomes were estimated similarly. To evaluate the robustness of the primary outcome, we performed a sensitivity analysis using a multilevel generalized linear model with clustering by individual learner and team. Additional details on our statistical analysis plan, including accessing our raw data and analysis code, are available in Appendices 2 and 3. Categorical variables were compared using the χ² or Fisher exact test. Continuous variables were compared using the t test or Wilcoxon rank-sum tests. All P values were 2-sided, and a significance level of ≤ .05 was considered statistically significant. Analysis was performed in Stata v16.1. Our study was deemed exempt by the VABHS Institutional Review Board, and this article was prepared following the CONSORT reporting guidelines. The trial protocol has been registered with the International Standard Randomized Controlled Trial Number registry (ISRCTN14675095).

Over the study period, 329 unique learners rotated on inpatient medical services at the VABHS and 211 were eligible to attend 85 morning report conferences and 22 Jeopardy conferences (Figure). Outcomes data were available for 100% of eligible participants. Forty-seven (55%) of the morning report conferences occurred during the intervention period (Table 1).

Morning report attendance observed at 8:10 am was 5.5% higher during the intervention period compared with the control period (49.9% vs 44.4%, P = .007). Accounting for clustering within individuals, the unadjusted risk difference in morning report attendance associated with sending a reminder page was 3.6% (95% CI, 0.09%-7.2%; P = .04) compared with no reminder page. When adding date and team to our model, the adjusted risk difference in conference attendance increased to 4.0% (95% CI, 0.5%-7.6%; P = .03) (Table 2). Results were similar in a sensitivity analysis using a multilevel generalized linear model accounting for clustering by both individual and team (adjusted risk difference, 4.0% [95% CI, 0.4%-7.6%; P = .03]).

On-time attendance was lower than at 8:10 am in both groups, with no difference in the observed attendance at 8:00 am between the control and intervention groups (22.4% vs 25.0%, P = .14). Regarding Jeopardy-like conferences, on-time attendance differed between the control and intervention groups at 8:00 am (15.3% vs 23.6%, P = .01), but not at 8:10 am (42.9% vs 42.8%, P > .99). We found no evidence of an interaction between receipt of a reminder page and learner type (student vs house staff, P = .33).

To estimate the impact of rotating on teams with lighter clinical workloads on the association between receipt of a reminder page and conference attendance, we repeated our primary analysis with a test of interaction between team assignment and the intervention, which was not significant (P = .90). To estimate the impact of morning workload on the association between receipt of a reminder page and conference attendance, we performed a subgroup analysis limited to learners rotating on teams eligible to receive overnight admissions and included the number of overnight admissions as a covariate in our regression model. A test of interaction between the intervention and the number of overnight admissions on conference attendance was not significant (P = .73).

In a subgroup analysis limited to learners on teams eligible to receive overnight admissions and controlling for the number of overnight admissions (a proxy for morning workload), no significant interaction between the intervention and admissions was observed. We also assessed for interaction between learner type and receipt of a reminder page on conference attendance and found no evidence of such an effect.

Discussion

Among a diverse population of learners from multiple academic institutions rotating at a single, large, urban VA medical center, a nudge strategy of sending a reminder text page before morning report conferences was associated with a 4.0% absolute increase in attendance measured 10 minutes after the conference started compared with not sending a reminder page. Overall, only one-quarter of learners attended the morning report at the start at 8:00 am, with no difference in on-time attendance between the intervention and control periods.

We designed our analysis to overcome several limitations of prior studies on the effect of reminder text pages on conference attendance. First, to account for differences in conference attendance behavior of individual learners, we used a generalized estimating equation model that allowed clustering of outcomes by individual. Second, we controlled for the date to account for secular trends in conference attendance over the academic year. Finally, we controlled for the team to account for the possibility that the conference attendance behavior of one learner on a team influences the behavior of other learners on the same team.

We also evaluated the effect of a reminder page on attendance at a weekly Jeopardy conference. Interestingly, reminder pages seemed to increase on-time Jeopardy attendance, although this effect was no longer statistically significant at 8:10 am. A possible explanation for this is that the fun and collegial nature of Jeopardy conferences entices learners to attend independent of a reminder page.

We also assessed the interaction between sending a reminder page and learner type and its effect on conference attendance and found no evidence to support such an effect. Because medical students do not receive reminder pages, their conference attendance behavior can be thought of as indicative of clustering within teams. Though there was no evidence of a significant interaction, given the small number of students, our study may be underpowered to find a benefit for this group.

The results of this study differ from Smith and colleagues, who found that reminder pages had no overall effect on conference attendance for fellows; however, no sample size justification was provided in that study, making it difficult to evaluate the likelihood of a false-negative finding.⁷ Our study differs in several ways: the timing of the reminder page (5 minutes vs 30 minutes prior to the conference), the method by which attendance was recorded (by an independent observer vs learner sign-in), and the time that attendance was recorded (2 prespecified times vs continuously). As far as we know, our study is the first to evaluate the nudge effect of reminder text pages on internal medicine resident attendance at conferences, with attendance taken by an observer.

This study has some limitations. First, it was conducted at a single VA medical center. An additional limitation was our decision to classify learners who arrived after 8:10 am as absent, which likely underestimated total conference attendance. Further, we did not record whether learners stayed until the end of the conference. Additionally, many hospitals are transitioning away from pagers in favor of mobile phones; however, we have no reason to expect that the device on which a reminder is received (pager or phone) should affect the generalizability of these results.

Unfortunately, due to the COVID-19 pandemic and the suspension of in-person conferences, our study ended earlier than anticipated. This resulted in an imbalance of morning report conferences that occurred during each period: 55% during the intervention period, and 45% during the control period. However, because we accounted for the clustering of conference attendance behavior within individuals in our model, this imbalance is unlikely to introduce bias in our estimation of the effect of the intervention.

Another limitation relates to the evolving landscape of educational conferences in the postpandemic era.¹⁸ Whether our results can be generalized to increase virtual conference attendance is unknown. Finally, it is not clear whether a 4% absolute increase in conference attendance is educationally meaningful or justifies the effort of sending a reminder page.

Conclusions

In this cluster randomized controlled trial conducted at a single VA medical center, reminder pages sent 5 minutes before the start of morning report conferences resulted in a 4% increase in conference attendance. Our results suggest that reminder pages are one strategy that may result in a small increase in conference attendance, but whether this small increase is educationally significant will vary across training programs applying this strategy.

Acknowledgments

The authors are indebted to Kenneth J. Mukamal and Katharine A. Robb, who provided invaluable guidance in data analysis. Todd Reese assisted in data organization and presentation of data, and Mark Tuttle designed the facesheet. None of these individuals received compensation for their assistance.

Regularly scheduled educational conferences, such as case-based morning reports, have been a standard part of internal medicine residencies for decades.^1-4 In addition to better patient care from the knowledge gained at educational conferences, attendance by interns and residents (collectively called house staff) may be associated with higher in-service examination scores.⁵ Unfortunately, competing priorities, including patient care and trainee supervision, may contribute to an action-intention gap among house staff that reduces attendance.^6-8 Low attendance at morning reports represents wasted effort and lost educational opportunities; therefore, strategies to increase attendance are needed. Of several methods studied, more resource-intensive interventions (eg, providing food) were the most successful.^6,9-12

Using the behavioral economics framework of nudge strategies, we hypothesized that a less intensive intervention of a daily reminder text page would encourage medical students, interns, and residents (collectively called learners) to attend the morning report conference.^8,13 However, given the high cognitive load created by frequent task switching, a reminder text page could disrupt workflow and patient care without promoting the intended behavior change.^14-17 Because of this uncertainty, our objective was to determine whether a preconference text page increased learner attendance at morning report conferences.

Methods

This study was a single-center, multiple-crossover cluster randomized controlled trial conducted at the Veteran Affairs Boston Healthcare System (VABHS) in Massachusetts. Study participants included house staff rotating on daytime inpatient rotations from 4 residency programs and students from 2 medical schools. The setting was the morning report, an in-person, interactive, case-based conference held Monday through Thursday, from 8:00 am to 8:45 am. On Friday mornings, the morning report was replaced with a medical Jeopardy game-style conference. Historically, attendance has not been recorded for these conferences.

Learners assigned to rotate on the inpatient medicine, cardiology, medicine consultation, and patient safety rotations were eligible to attend these conferences and for inclusion in the study. Learners rotating in the medical intensive care unit, on night float, or on day float (an admitting shift for which residents are not on-site until late afternoon) were excluded. Additional details of the study population are available in the supplement (eAppendix). The study period was originally planned for September 30, 2019, to March 31, 2020, but data collection was stopped on March 12, 2020, due to the COVID-19 pandemic and suspension of in-person conferences. We chose the study period, which determined our sample size, to exclude the first 3 months of the academic year (July-September) because during that time learners acclimate to the inpatient workflow. We also chose not to include the last 3 months of the academic year to provide time for data analysis and preparation of the manuscript within the academic year.

Intervention and Outcome Assessment

Each intervention and control period was 3 weeks long; the first period was randomly determined by coin flip and alternated thereafter. Additional details of randomization are available in the supplement (Appendix 1). During intervention periods, all house staff received a page at 7:55 am that listed the time and location of the upcoming morning report or Jeopardy conference. Medical students do not carry pagers and did not receive reminder pages; however, we included these learners because changes in their conference attendance behavior would indicate an extension of the effect of reminder pages beyond the individual learner who received the page.

A daily facesheet (a roster of house staff names and photos) was used to identify learners for conference attendance. This facesheet was already used for other purposes at VABHS. At 8:00 am and 8:10 am, a chief medical resident who was not blinded to the intervention or control period recorded the attendance of each eligible learner as present or absent; learners were unaware that their attendance was being recorded. This approach to data collection was selected to minimize the likelihood that the behavior of the study participants would be influenced.

During control periods, no text page reminder of upcoming conferences was sent, but the attendance of total learners at 8:00 am and 8:10 am was recorded by a chief medical resident who used the same method as during the intervention periods. Attendance at 8:10 am was chosen as the primary outcome to account for the possibility that learners may arrive after a conference begins. Attendance at 8:00 am also was recorded to assess the effect of reminder pages on attendance at the start of morning reports.

Statistical Analysis

The primary outcome was the proportion of eligible learners present at 8:10 am at the morning report, expressed as the risk difference for attendance between intervention and control periods. Secondary outcomes included the proportion of learners present at 8:00 am (on-time attendance), the proportion of learners present by type (student vs house staff), and the proportion of learners present at the Friday Jeopardy conference. Two preplanned subgroup analyses were performed: one assessing the impact of rotating on clinical services with lighter workloads, and the other assessing the impact of the number of overnight admissions received on the relationship between receipt of a reminder page and conference attendance.

To estimate the primary outcome, we modeled the risk difference adjusted for covariates using a generalized estimating equation accounting for the clustering of attendance behavior within individuals and controlling for date and team. Secondary outcomes were estimated similarly. To evaluate the robustness of the primary outcome, we performed a sensitivity analysis using a multilevel generalized linear model with clustering by individual learner and team. Additional details on our statistical analysis plan, including accessing our raw data and analysis code, are available in Appendices 2 and 3. Categorical variables were compared using the χ² or Fisher exact test. Continuous variables were compared using the t test or Wilcoxon rank-sum tests. All P values were 2-sided, and a significance level of ≤ .05 was considered statistically significant. Analysis was performed in Stata v16.1. Our study was deemed exempt by the VABHS Institutional Review Board, and this article was prepared following the CONSORT reporting guidelines. The trial protocol has been registered with the International Standard Randomized Controlled Trial Number registry (ISRCTN14675095).

Over the study period, 329 unique learners rotated on inpatient medical services at the VABHS and 211 were eligible to attend 85 morning report conferences and 22 Jeopardy conferences (Figure). Outcomes data were available for 100% of eligible participants. Forty-seven (55%) of the morning report conferences occurred during the intervention period (Table 1).

Morning report attendance observed at 8:10 am was 5.5% higher during the intervention period compared with the control period (49.9% vs 44.4%, P = .007). Accounting for clustering within individuals, the unadjusted risk difference in morning report attendance associated with sending a reminder page was 3.6% (95% CI, 0.09%-7.2%; P = .04) compared with no reminder page. When adding date and team to our model, the adjusted risk difference in conference attendance increased to 4.0% (95% CI, 0.5%-7.6%; P = .03) (Table 2). Results were similar in a sensitivity analysis using a multilevel generalized linear model accounting for clustering by both individual and team (adjusted risk difference, 4.0% [95% CI, 0.4%-7.6%; P = .03]).

On-time attendance was lower than at 8:10 am in both groups, with no difference in the observed attendance at 8:00 am between the control and intervention groups (22.4% vs 25.0%, P = .14). Regarding Jeopardy-like conferences, on-time attendance differed between the control and intervention groups at 8:00 am (15.3% vs 23.6%, P = .01), but not at 8:10 am (42.9% vs 42.8%, P > .99). We found no evidence of an interaction between receipt of a reminder page and learner type (student vs house staff, P = .33).

To estimate the impact of rotating on teams with lighter clinical workloads on the association between receipt of a reminder page and conference attendance, we repeated our primary analysis with a test of interaction between team assignment and the intervention, which was not significant (P = .90). To estimate the impact of morning workload on the association between receipt of a reminder page and conference attendance, we performed a subgroup analysis limited to learners rotating on teams eligible to receive overnight admissions and included the number of overnight admissions as a covariate in our regression model. A test of interaction between the intervention and the number of overnight admissions on conference attendance was not significant (P = .73).

In a subgroup analysis limited to learners on teams eligible to receive overnight admissions and controlling for the number of overnight admissions (a proxy for morning workload), no significant interaction between the intervention and admissions was observed. We also assessed for interaction between learner type and receipt of a reminder page on conference attendance and found no evidence of such an effect.

Discussion

Among a diverse population of learners from multiple academic institutions rotating at a single, large, urban VA medical center, a nudge strategy of sending a reminder text page before morning report conferences was associated with a 4.0% absolute increase in attendance measured 10 minutes after the conference started compared with not sending a reminder page. Overall, only one-quarter of learners attended the morning report at the start at 8:00 am, with no difference in on-time attendance between the intervention and control periods.

We designed our analysis to overcome several limitations of prior studies on the effect of reminder text pages on conference attendance. First, to account for differences in conference attendance behavior of individual learners, we used a generalized estimating equation model that allowed clustering of outcomes by individual. Second, we controlled for the date to account for secular trends in conference attendance over the academic year. Finally, we controlled for the team to account for the possibility that the conference attendance behavior of one learner on a team influences the behavior of other learners on the same team.

We also evaluated the effect of a reminder page on attendance at a weekly Jeopardy conference. Interestingly, reminder pages seemed to increase on-time Jeopardy attendance, although this effect was no longer statistically significant at 8:10 am. A possible explanation for this is that the fun and collegial nature of Jeopardy conferences entices learners to attend independent of a reminder page.

We also assessed the interaction between sending a reminder page and learner type and its effect on conference attendance and found no evidence to support such an effect. Because medical students do not receive reminder pages, their conference attendance behavior can be thought of as indicative of clustering within teams. Though there was no evidence of a significant interaction, given the small number of students, our study may be underpowered to find a benefit for this group.

The results of this study differ from Smith and colleagues, who found that reminder pages had no overall effect on conference attendance for fellows; however, no sample size justification was provided in that study, making it difficult to evaluate the likelihood of a false-negative finding.⁷ Our study differs in several ways: the timing of the reminder page (5 minutes vs 30 minutes prior to the conference), the method by which attendance was recorded (by an independent observer vs learner sign-in), and the time that attendance was recorded (2 prespecified times vs continuously). As far as we know, our study is the first to evaluate the nudge effect of reminder text pages on internal medicine resident attendance at conferences, with attendance taken by an observer.

This study has some limitations. First, it was conducted at a single VA medical center. An additional limitation was our decision to classify learners who arrived after 8:10 am as absent, which likely underestimated total conference attendance. Further, we did not record whether learners stayed until the end of the conference. Additionally, many hospitals are transitioning away from pagers in favor of mobile phones; however, we have no reason to expect that the device on which a reminder is received (pager or phone) should affect the generalizability of these results.

Unfortunately, due to the COVID-19 pandemic and the suspension of in-person conferences, our study ended earlier than anticipated. This resulted in an imbalance of morning report conferences that occurred during each period: 55% during the intervention period, and 45% during the control period. However, because we accounted for the clustering of conference attendance behavior within individuals in our model, this imbalance is unlikely to introduce bias in our estimation of the effect of the intervention.

Another limitation relates to the evolving landscape of educational conferences in the postpandemic era.¹⁸ Whether our results can be generalized to increase virtual conference attendance is unknown. Finally, it is not clear whether a 4% absolute increase in conference attendance is educationally meaningful or justifies the effort of sending a reminder page.

Conclusions

In this cluster randomized controlled trial conducted at a single VA medical center, reminder pages sent 5 minutes before the start of morning report conferences resulted in a 4% increase in conference attendance. Our results suggest that reminder pages are one strategy that may result in a small increase in conference attendance, but whether this small increase is educationally significant will vary across training programs applying this strategy.

Acknowledgments

The authors are indebted to Kenneth J. Mukamal and Katharine A. Robb, who provided invaluable guidance in data analysis. Todd Reese assisted in data organization and presentation of data, and Mark Tuttle designed the facesheet. None of these individuals received compensation for their assistance.

Regularly scheduled educational conferences, such as case-based morning reports, have been a standard part of internal medicine residencies for decades.^1-4 In addition to better patient care from the knowledge gained at educational conferences, attendance by interns and residents (collectively called house staff) may be associated with higher in-service examination scores.⁵ Unfortunately, competing priorities, including patient care and trainee supervision, may contribute to an action-intention gap among house staff that reduces attendance.^6-8 Low attendance at morning reports represents wasted effort and lost educational opportunities; therefore, strategies to increase attendance are needed. Of several methods studied, more resource-intensive interventions (eg, providing food) were the most successful.^6,9-12

Using the behavioral economics framework of nudge strategies, we hypothesized that a less intensive intervention of a daily reminder text page would encourage medical students, interns, and residents (collectively called learners) to attend the morning report conference.^8,13 However, given the high cognitive load created by frequent task switching, a reminder text page could disrupt workflow and patient care without promoting the intended behavior change.^14-17 Because of this uncertainty, our objective was to determine whether a preconference text page increased learner attendance at morning report conferences.

Methods

This study was a single-center, multiple-crossover cluster randomized controlled trial conducted at the Veteran Affairs Boston Healthcare System (VABHS) in Massachusetts. Study participants included house staff rotating on daytime inpatient rotations from 4 residency programs and students from 2 medical schools. The setting was the morning report, an in-person, interactive, case-based conference held Monday through Thursday, from 8:00 am to 8:45 am. On Friday mornings, the morning report was replaced with a medical Jeopardy game-style conference. Historically, attendance has not been recorded for these conferences.

Learners assigned to rotate on the inpatient medicine, cardiology, medicine consultation, and patient safety rotations were eligible to attend these conferences and for inclusion in the study. Learners rotating in the medical intensive care unit, on night float, or on day float (an admitting shift for which residents are not on-site until late afternoon) were excluded. Additional details of the study population are available in the supplement (eAppendix). The study period was originally planned for September 30, 2019, to March 31, 2020, but data collection was stopped on March 12, 2020, due to the COVID-19 pandemic and suspension of in-person conferences. We chose the study period, which determined our sample size, to exclude the first 3 months of the academic year (July-September) because during that time learners acclimate to the inpatient workflow. We also chose not to include the last 3 months of the academic year to provide time for data analysis and preparation of the manuscript within the academic year.

Intervention and Outcome Assessment

Each intervention and control period was 3 weeks long; the first period was randomly determined by coin flip and alternated thereafter. Additional details of randomization are available in the supplement (Appendix 1). During intervention periods, all house staff received a page at 7:55 am that listed the time and location of the upcoming morning report or Jeopardy conference. Medical students do not carry pagers and did not receive reminder pages; however, we included these learners because changes in their conference attendance behavior would indicate an extension of the effect of reminder pages beyond the individual learner who received the page.

A daily facesheet (a roster of house staff names and photos) was used to identify learners for conference attendance. This facesheet was already used for other purposes at VABHS. At 8:00 am and 8:10 am, a chief medical resident who was not blinded to the intervention or control period recorded the attendance of each eligible learner as present or absent; learners were unaware that their attendance was being recorded. This approach to data collection was selected to minimize the likelihood that the behavior of the study participants would be influenced.

During control periods, no text page reminder of upcoming conferences was sent, but the attendance of total learners at 8:00 am and 8:10 am was recorded by a chief medical resident who used the same method as during the intervention periods. Attendance at 8:10 am was chosen as the primary outcome to account for the possibility that learners may arrive after a conference begins. Attendance at 8:00 am also was recorded to assess the effect of reminder pages on attendance at the start of morning reports.

Statistical Analysis

The primary outcome was the proportion of eligible learners present at 8:10 am at the morning report, expressed as the risk difference for attendance between intervention and control periods. Secondary outcomes included the proportion of learners present at 8:00 am (on-time attendance), the proportion of learners present by type (student vs house staff), and the proportion of learners present at the Friday Jeopardy conference. Two preplanned subgroup analyses were performed: one assessing the impact of rotating on clinical services with lighter workloads, and the other assessing the impact of the number of overnight admissions received on the relationship between receipt of a reminder page and conference attendance.

To estimate the primary outcome, we modeled the risk difference adjusted for covariates using a generalized estimating equation accounting for the clustering of attendance behavior within individuals and controlling for date and team. Secondary outcomes were estimated similarly. To evaluate the robustness of the primary outcome, we performed a sensitivity analysis using a multilevel generalized linear model with clustering by individual learner and team. Additional details on our statistical analysis plan, including accessing our raw data and analysis code, are available in Appendices 2 and 3. Categorical variables were compared using the χ² or Fisher exact test. Continuous variables were compared using the t test or Wilcoxon rank-sum tests. All P values were 2-sided, and a significance level of ≤ .05 was considered statistically significant. Analysis was performed in Stata v16.1. Our study was deemed exempt by the VABHS Institutional Review Board, and this article was prepared following the CONSORT reporting guidelines. The trial protocol has been registered with the International Standard Randomized Controlled Trial Number registry (ISRCTN14675095).

Over the study period, 329 unique learners rotated on inpatient medical services at the VABHS and 211 were eligible to attend 85 morning report conferences and 22 Jeopardy conferences (Figure). Outcomes data were available for 100% of eligible participants. Forty-seven (55%) of the morning report conferences occurred during the intervention period (Table 1).

Morning report attendance observed at 8:10 am was 5.5% higher during the intervention period compared with the control period (49.9% vs 44.4%, P = .007). Accounting for clustering within individuals, the unadjusted risk difference in morning report attendance associated with sending a reminder page was 3.6% (95% CI, 0.09%-7.2%; P = .04) compared with no reminder page. When adding date and team to our model, the adjusted risk difference in conference attendance increased to 4.0% (95% CI, 0.5%-7.6%; P = .03) (Table 2). Results were similar in a sensitivity analysis using a multilevel generalized linear model accounting for clustering by both individual and team (adjusted risk difference, 4.0% [95% CI, 0.4%-7.6%; P = .03]).

On-time attendance was lower than at 8:10 am in both groups, with no difference in the observed attendance at 8:00 am between the control and intervention groups (22.4% vs 25.0%, P = .14). Regarding Jeopardy-like conferences, on-time attendance differed between the control and intervention groups at 8:00 am (15.3% vs 23.6%, P = .01), but not at 8:10 am (42.9% vs 42.8%, P > .99). We found no evidence of an interaction between receipt of a reminder page and learner type (student vs house staff, P = .33).

To estimate the impact of rotating on teams with lighter clinical workloads on the association between receipt of a reminder page and conference attendance, we repeated our primary analysis with a test of interaction between team assignment and the intervention, which was not significant (P = .90). To estimate the impact of morning workload on the association between receipt of a reminder page and conference attendance, we performed a subgroup analysis limited to learners rotating on teams eligible to receive overnight admissions and included the number of overnight admissions as a covariate in our regression model. A test of interaction between the intervention and the number of overnight admissions on conference attendance was not significant (P = .73).

In a subgroup analysis limited to learners on teams eligible to receive overnight admissions and controlling for the number of overnight admissions (a proxy for morning workload), no significant interaction between the intervention and admissions was observed. We also assessed for interaction between learner type and receipt of a reminder page on conference attendance and found no evidence of such an effect.

Discussion

Among a diverse population of learners from multiple academic institutions rotating at a single, large, urban VA medical center, a nudge strategy of sending a reminder text page before morning report conferences was associated with a 4.0% absolute increase in attendance measured 10 minutes after the conference started compared with not sending a reminder page. Overall, only one-quarter of learners attended the morning report at the start at 8:00 am, with no difference in on-time attendance between the intervention and control periods.

We designed our analysis to overcome several limitations of prior studies on the effect of reminder text pages on conference attendance. First, to account for differences in conference attendance behavior of individual learners, we used a generalized estimating equation model that allowed clustering of outcomes by individual. Second, we controlled for the date to account for secular trends in conference attendance over the academic year. Finally, we controlled for the team to account for the possibility that the conference attendance behavior of one learner on a team influences the behavior of other learners on the same team.

We also evaluated the effect of a reminder page on attendance at a weekly Jeopardy conference. Interestingly, reminder pages seemed to increase on-time Jeopardy attendance, although this effect was no longer statistically significant at 8:10 am. A possible explanation for this is that the fun and collegial nature of Jeopardy conferences entices learners to attend independent of a reminder page.

We also assessed the interaction between sending a reminder page and learner type and its effect on conference attendance and found no evidence to support such an effect. Because medical students do not receive reminder pages, their conference attendance behavior can be thought of as indicative of clustering within teams. Though there was no evidence of a significant interaction, given the small number of students, our study may be underpowered to find a benefit for this group.

The results of this study differ from Smith and colleagues, who found that reminder pages had no overall effect on conference attendance for fellows; however, no sample size justification was provided in that study, making it difficult to evaluate the likelihood of a false-negative finding.⁷ Our study differs in several ways: the timing of the reminder page (5 minutes vs 30 minutes prior to the conference), the method by which attendance was recorded (by an independent observer vs learner sign-in), and the time that attendance was recorded (2 prespecified times vs continuously). As far as we know, our study is the first to evaluate the nudge effect of reminder text pages on internal medicine resident attendance at conferences, with attendance taken by an observer.

This study has some limitations. First, it was conducted at a single VA medical center. An additional limitation was our decision to classify learners who arrived after 8:10 am as absent, which likely underestimated total conference attendance. Further, we did not record whether learners stayed until the end of the conference. Additionally, many hospitals are transitioning away from pagers in favor of mobile phones; however, we have no reason to expect that the device on which a reminder is received (pager or phone) should affect the generalizability of these results.

Unfortunately, due to the COVID-19 pandemic and the suspension of in-person conferences, our study ended earlier than anticipated. This resulted in an imbalance of morning report conferences that occurred during each period: 55% during the intervention period, and 45% during the control period. However, because we accounted for the clustering of conference attendance behavior within individuals in our model, this imbalance is unlikely to introduce bias in our estimation of the effect of the intervention.

Another limitation relates to the evolving landscape of educational conferences in the postpandemic era.¹⁸ Whether our results can be generalized to increase virtual conference attendance is unknown. Finally, it is not clear whether a 4% absolute increase in conference attendance is educationally meaningful or justifies the effort of sending a reminder page.

Conclusions

In this cluster randomized controlled trial conducted at a single VA medical center, reminder pages sent 5 minutes before the start of morning report conferences resulted in a 4% increase in conference attendance. Our results suggest that reminder pages are one strategy that may result in a small increase in conference attendance, but whether this small increase is educationally significant will vary across training programs applying this strategy.

Acknowledgments

The authors are indebted to Kenneth J. Mukamal and Katharine A. Robb, who provided invaluable guidance in data analysis. Todd Reese assisted in data organization and presentation of data, and Mark Tuttle designed the facesheet. None of these individuals received compensation for their assistance.

Estimates of chronic kidney disease (CKD) among veterans range between 34% and 47% higher than in the general population.¹ As patients progress to end-stage kidney disease and begin chronic dialysis, they often experience further functional and cognitive decline and a high symptom burden, leading to poor quality of life.² Clinicians should initiate goals of care conversations (GOCCs) to support high-risk patients on dialysis to ensure that the interventions they receive align with their goals and preferences, since many patients on dialysis prefer measures focused on pain relief and discomfort.^3,4 While proactive GOCCs are supported among nephrology associations, few such conversations take place.^5,6 In one study, more than half of patients on dialysis stated they had not discussed end-of-life preferences in the past 12 months.⁴ As a result, patients may not consider the larger implications of receiving dialysis indefinitely as a life-sustaining treatment (LST).

In May 2018, the US Department of Veterans Affairs (VA) National Center for Ethics in Health Care rolled out the Life-Sustaining Treatment Decisions Initiative to proactively engage patients with serious illnesses, such as those with end-stage kidney disease, in GOCCs to clarify their preferences for LSTs.⁷ After comprehensive training, a preliminary audit at the Edward Hines, Jr. VA Hospital (EHJVAH) in Hines, Illinois, revealed that only 27% of patients on dialysis had LST preferences documented in a standardized LST note.

Nephrologists cite multiple barriers to proactively addressing goals of care with patients with advanced CKD, including clinician discomfort, perceived lack of time, infrastructure, and training.^8,9 Similarly, the absence of a multidisciplinary advance care planning approach—specifically bringing together palliative care (PC) clinicians with nephrologists—has been highlighted but not as well studied.^10,11

In this quality improvement (QI) project, we aimed to establish a workflow to enhance collaboration between nephrology and PC and to increase the percentage of VA patients on outpatient hemodialysis who engaged in GOCCs, as documented by completion of an LST progress note in the VA’s electronic health record (EHR). We developed a collaboration among PC, nephrology, and social work to improve the rates of documented GOCCs and LST patients on dialysis.

Implementation

EHJVAH is a 1A facility with > 80 patients who receive outpatient hemodialysis on campus. At the time of this collaboration in the fall of 2019, the collaborative dialysis team comprised 2 social workers and a nephrologist. The PC team included a coordinator, 2 nurse practitioners, and 3 physicians. A QI nurse was involved in the initial data gathering for this project.

The PC and nephrology medical directors developed a workflow process that reflected organizational and clinical steps in planning, initiating, and completing GOCCs with patients on outpatient dialysis (Table 1). The proposed process engaged an interdisciplinary PC and nephrology group and was revised to incorporate staff suggestions.

A prospective review of 85 EHJVAH hemodialysis unit patient records was conducted between September 1, 2019, and September 30, 2020 (Table 2). We reviewed LST completion rates for all patients receiving dialysis within this timeframe. During the intervention period, the PC team approached 40 patients without LST notes to engage in GOCCs.PC completed LST notes for 29 of 40 patients (72%). Of the 11 patients without LST notes, 7 declined a visit and 4 were lost to follow-up. At the end of the study period, 69 patients (81%) on outpatient dialysis had LST progress notes in the EHR.

Discussion

Over the 13-month collaboration, LST note completion rates increased from 27% to 81%, with 69 of 85 patients having a documented LST progress note in the EHR. PC approached nearly half of all patients on dialysis. Most patients agreed to be seen by the PC team, with 72% of those approached agreeing to a PC consultation. Previous research has suggested that having a trusted dialysis staff member included in GOCCs contributes to high acceptance rates.¹²As such, the QI project relied heavily on the existing rapport between the dialysis staff—in particular the dialysis social workers—and their patients to normalize the PC consultation for all patients on dialysis. This introduction by a trusted staff person may have contributed to higher acceptance rates, and at the time patients on dialysis arrived for the PC appointment, they had a good understanding of the project. By including PC specialists with expertise in advance care planning and communication skills, the partnership successfully created a collaborative process that relied on the skill set of multiple staff and disciplines.

PC is a relatively uncommon partnership for nephrologists, and PC and hospice services are underutilized in patients on dialysis both nationally and within the VA.^13-15 Our outcomes could be replicated, as PC is required at all VA sites. One implementation consideration is the additional time this collaboration requires. Although no formal time study was completed, the PC team spent several hours educating nephrology staff, and the social workers spent considerable time reaching, educating, and scheduling veterans into the PC clinic.

Conclusions

The innovation of an interdisciplinary nephrology–PC collaboration was an important step in increasing high-quality GOCCs and eliciting patient preferences for LSTs among patients on dialysis. PC integration for patients on dialysis is associated with improved symptom management, fewer aggressive health care measures, and a higher likelihood of dying in one’s preferred setting.¹⁶ While this partnership focused on patients already receiving dialysis, successful PC interventions are felt most keenly upstream, before dialysis initiation.

Acknowledgments

The authors acknowledge the contributions of their colleague, Mary McCabe, DNP, Quality Systems Improvement, Edward Hines, Jr. Veterans Affairs Hospital. The authors also acknowledge the clinical dedication of the dialysis social workers, Sarah Adam, LCSW, and Sarah Kraner, LCSW, without which this collaboration would not have been possible.

Estimates of chronic kidney disease (CKD) among veterans range between 34% and 47% higher than in the general population.¹ As patients progress to end-stage kidney disease and begin chronic dialysis, they often experience further functional and cognitive decline and a high symptom burden, leading to poor quality of life.² Clinicians should initiate goals of care conversations (GOCCs) to support high-risk patients on dialysis to ensure that the interventions they receive align with their goals and preferences, since many patients on dialysis prefer measures focused on pain relief and discomfort.^3,4 While proactive GOCCs are supported among nephrology associations, few such conversations take place.^5,6 In one study, more than half of patients on dialysis stated they had not discussed end-of-life preferences in the past 12 months.⁴ As a result, patients may not consider the larger implications of receiving dialysis indefinitely as a life-sustaining treatment (LST).

In May 2018, the US Department of Veterans Affairs (VA) National Center for Ethics in Health Care rolled out the Life-Sustaining Treatment Decisions Initiative to proactively engage patients with serious illnesses, such as those with end-stage kidney disease, in GOCCs to clarify their preferences for LSTs.⁷ After comprehensive training, a preliminary audit at the Edward Hines, Jr. VA Hospital (EHJVAH) in Hines, Illinois, revealed that only 27% of patients on dialysis had LST preferences documented in a standardized LST note.

Nephrologists cite multiple barriers to proactively addressing goals of care with patients with advanced CKD, including clinician discomfort, perceived lack of time, infrastructure, and training.^8,9 Similarly, the absence of a multidisciplinary advance care planning approach—specifically bringing together palliative care (PC) clinicians with nephrologists—has been highlighted but not as well studied.^10,11

In this quality improvement (QI) project, we aimed to establish a workflow to enhance collaboration between nephrology and PC and to increase the percentage of VA patients on outpatient hemodialysis who engaged in GOCCs, as documented by completion of an LST progress note in the VA’s electronic health record (EHR). We developed a collaboration among PC, nephrology, and social work to improve the rates of documented GOCCs and LST patients on dialysis.

Implementation

EHJVAH is a 1A facility with > 80 patients who receive outpatient hemodialysis on campus. At the time of this collaboration in the fall of 2019, the collaborative dialysis team comprised 2 social workers and a nephrologist. The PC team included a coordinator, 2 nurse practitioners, and 3 physicians. A QI nurse was involved in the initial data gathering for this project.

The PC and nephrology medical directors developed a workflow process that reflected organizational and clinical steps in planning, initiating, and completing GOCCs with patients on outpatient dialysis (Table 1). The proposed process engaged an interdisciplinary PC and nephrology group and was revised to incorporate staff suggestions.

A prospective review of 85 EHJVAH hemodialysis unit patient records was conducted between September 1, 2019, and September 30, 2020 (Table 2). We reviewed LST completion rates for all patients receiving dialysis within this timeframe. During the intervention period, the PC team approached 40 patients without LST notes to engage in GOCCs.PC completed LST notes for 29 of 40 patients (72%). Of the 11 patients without LST notes, 7 declined a visit and 4 were lost to follow-up. At the end of the study period, 69 patients (81%) on outpatient dialysis had LST progress notes in the EHR.

Discussion

Over the 13-month collaboration, LST note completion rates increased from 27% to 81%, with 69 of 85 patients having a documented LST progress note in the EHR. PC approached nearly half of all patients on dialysis. Most patients agreed to be seen by the PC team, with 72% of those approached agreeing to a PC consultation. Previous research has suggested that having a trusted dialysis staff member included in GOCCs contributes to high acceptance rates.¹²As such, the QI project relied heavily on the existing rapport between the dialysis staff—in particular the dialysis social workers—and their patients to normalize the PC consultation for all patients on dialysis. This introduction by a trusted staff person may have contributed to higher acceptance rates, and at the time patients on dialysis arrived for the PC appointment, they had a good understanding of the project. By including PC specialists with expertise in advance care planning and communication skills, the partnership successfully created a collaborative process that relied on the skill set of multiple staff and disciplines.

PC is a relatively uncommon partnership for nephrologists, and PC and hospice services are underutilized in patients on dialysis both nationally and within the VA.^13-15 Our outcomes could be replicated, as PC is required at all VA sites. One implementation consideration is the additional time this collaboration requires. Although no formal time study was completed, the PC team spent several hours educating nephrology staff, and the social workers spent considerable time reaching, educating, and scheduling veterans into the PC clinic.

Conclusions

The innovation of an interdisciplinary nephrology–PC collaboration was an important step in increasing high-quality GOCCs and eliciting patient preferences for LSTs among patients on dialysis. PC integration for patients on dialysis is associated with improved symptom management, fewer aggressive health care measures, and a higher likelihood of dying in one’s preferred setting.¹⁶ While this partnership focused on patients already receiving dialysis, successful PC interventions are felt most keenly upstream, before dialysis initiation.

Acknowledgments

The authors acknowledge the contributions of their colleague, Mary McCabe, DNP, Quality Systems Improvement, Edward Hines, Jr. Veterans Affairs Hospital. The authors also acknowledge the clinical dedication of the dialysis social workers, Sarah Adam, LCSW, and Sarah Kraner, LCSW, without which this collaboration would not have been possible.

Estimates of chronic kidney disease (CKD) among veterans range between 34% and 47% higher than in the general population.¹ As patients progress to end-stage kidney disease and begin chronic dialysis, they often experience further functional and cognitive decline and a high symptom burden, leading to poor quality of life.² Clinicians should initiate goals of care conversations (GOCCs) to support high-risk patients on dialysis to ensure that the interventions they receive align with their goals and preferences, since many patients on dialysis prefer measures focused on pain relief and discomfort.^3,4 While proactive GOCCs are supported among nephrology associations, few such conversations take place.^5,6 In one study, more than half of patients on dialysis stated they had not discussed end-of-life preferences in the past 12 months.⁴ As a result, patients may not consider the larger implications of receiving dialysis indefinitely as a life-sustaining treatment (LST).

In May 2018, the US Department of Veterans Affairs (VA) National Center for Ethics in Health Care rolled out the Life-Sustaining Treatment Decisions Initiative to proactively engage patients with serious illnesses, such as those with end-stage kidney disease, in GOCCs to clarify their preferences for LSTs.⁷ After comprehensive training, a preliminary audit at the Edward Hines, Jr. VA Hospital (EHJVAH) in Hines, Illinois, revealed that only 27% of patients on dialysis had LST preferences documented in a standardized LST note.

Nephrologists cite multiple barriers to proactively addressing goals of care with patients with advanced CKD, including clinician discomfort, perceived lack of time, infrastructure, and training.^8,9 Similarly, the absence of a multidisciplinary advance care planning approach—specifically bringing together palliative care (PC) clinicians with nephrologists—has been highlighted but not as well studied.^10,11

In this quality improvement (QI) project, we aimed to establish a workflow to enhance collaboration between nephrology and PC and to increase the percentage of VA patients on outpatient hemodialysis who engaged in GOCCs, as documented by completion of an LST progress note in the VA’s electronic health record (EHR). We developed a collaboration among PC, nephrology, and social work to improve the rates of documented GOCCs and LST patients on dialysis.

Implementation

EHJVAH is a 1A facility with > 80 patients who receive outpatient hemodialysis on campus. At the time of this collaboration in the fall of 2019, the collaborative dialysis team comprised 2 social workers and a nephrologist. The PC team included a coordinator, 2 nurse practitioners, and 3 physicians. A QI nurse was involved in the initial data gathering for this project.

The PC and nephrology medical directors developed a workflow process that reflected organizational and clinical steps in planning, initiating, and completing GOCCs with patients on outpatient dialysis (Table 1). The proposed process engaged an interdisciplinary PC and nephrology group and was revised to incorporate staff suggestions.

A prospective review of 85 EHJVAH hemodialysis unit patient records was conducted between September 1, 2019, and September 30, 2020 (Table 2). We reviewed LST completion rates for all patients receiving dialysis within this timeframe. During the intervention period, the PC team approached 40 patients without LST notes to engage in GOCCs.PC completed LST notes for 29 of 40 patients (72%). Of the 11 patients without LST notes, 7 declined a visit and 4 were lost to follow-up. At the end of the study period, 69 patients (81%) on outpatient dialysis had LST progress notes in the EHR.

Discussion

Over the 13-month collaboration, LST note completion rates increased from 27% to 81%, with 69 of 85 patients having a documented LST progress note in the EHR. PC approached nearly half of all patients on dialysis. Most patients agreed to be seen by the PC team, with 72% of those approached agreeing to a PC consultation. Previous research has suggested that having a trusted dialysis staff member included in GOCCs contributes to high acceptance rates.¹²As such, the QI project relied heavily on the existing rapport between the dialysis staff—in particular the dialysis social workers—and their patients to normalize the PC consultation for all patients on dialysis. This introduction by a trusted staff person may have contributed to higher acceptance rates, and at the time patients on dialysis arrived for the PC appointment, they had a good understanding of the project. By including PC specialists with expertise in advance care planning and communication skills, the partnership successfully created a collaborative process that relied on the skill set of multiple staff and disciplines.

PC is a relatively uncommon partnership for nephrologists, and PC and hospice services are underutilized in patients on dialysis both nationally and within the VA.^13-15 Our outcomes could be replicated, as PC is required at all VA sites. One implementation consideration is the additional time this collaboration requires. Although no formal time study was completed, the PC team spent several hours educating nephrology staff, and the social workers spent considerable time reaching, educating, and scheduling veterans into the PC clinic.

Conclusions

The innovation of an interdisciplinary nephrology–PC collaboration was an important step in increasing high-quality GOCCs and eliciting patient preferences for LSTs among patients on dialysis. PC integration for patients on dialysis is associated with improved symptom management, fewer aggressive health care measures, and a higher likelihood of dying in one’s preferred setting.¹⁶ While this partnership focused on patients already receiving dialysis, successful PC interventions are felt most keenly upstream, before dialysis initiation.

Acknowledgments

The authors acknowledge the contributions of their colleague, Mary McCabe, DNP, Quality Systems Improvement, Edward Hines, Jr. Veterans Affairs Hospital. The authors also acknowledge the clinical dedication of the dialysis social workers, Sarah Adam, LCSW, and Sarah Kraner, LCSW, without which this collaboration would not have been possible.

Nearly 50% of living US veterans are aged ≥ 65 years compared with 18.3% of the general population.^1,2 The Veterans Health Administration (VHA), the largest integrated health care system in the US, has a vested interest in improving the quality and effectiveness of care for older veterans.³

Health care systems are often unprepared to care for the complex needs of older adults. There are roughly 7300 certified geriatricians practicing in the US, and about 250 new geriatricians are trained each year while the American Geriatrics Society expects > 12,000 geriatricians will be required by 2030.^4,5 More geriatricians are needed to serve as the primary health care professionals (HCPs) for older adults.^4,6 Health care systems like the VHA must find ways to increase geriatrics skills, knowledge, and practices among their entire health care workforce. A culture shift toward age-friendly care for older adults across care settings and inclusive of all HCPs may help meet this escalating workforce need.⁷

The Age-Friendly Health System (AFHS) is an initiative of the John A. Hartford Foundation and the Institute for Healthcare Improvement (IHI) in partnership with the American Hospital Association and the Catholic Health Association of the United States.^8,9 AFHS uses a what matters, medication, mentation, and mobility (4Ms) framework to ensure reliable, evidence-based care for older adults (Table 1).^10,11 In an AFHS, the 4Ms are integrated into every discipline and care setting for older adults.¹¹ The 4Ms neither replace formal training in geriatrics nor create the level of expertise needed for geriatrics teachers, researchers, and program leaders. However, the systematic approach of AFHS to assess and act on each of the 4Ms offers one solution to expand geriatrics skills and knowledge beyond geriatric care settings in all disciplines by engaging each HCP to meet the needs of older adults.¹² To act on what matters, HCPs need to align the care plan with what is important to the older adult.

Hospitals and health care systems are encouraged to begin implementing the 4Ms in ≥ 1 care setting.¹³ Care settings may get started on a do-it-yourself track or by joining an IHI Action Community, which provides a series of webinars to help adopt the 4Ms over 7 months.¹⁴ By creating a plan for how each M will be assessed, documented, and acted on, care settings may earn level 1 recognition from the IHI.¹⁴ As of July 2023, there are at least 3100 AFHS participants and > 1900 have achieved level 2 recognition, which requires 3 months of clinical data to demonstrate the impact of the 4Ms.^13,14

The main cultural shift of the AFHS movement is to focus on what matters to older adults by prioritizing each older adult’s personal health goals and care preferences across all care settings.^9,11 Medication addresses age-appropropriate prescribing, making dose adjustments, if needed, and avoiding/deprescribing high-risk medications that may interfere with what matters, mentation, or mobility. The Beers Criteria for Potentially Inappropriate Medication Use in Older Adults is often used as a guide and includes lists of medications that are potentially harmful for older adults.¹¹ Mentation focuses on preventing, identifying, treating, and managing dementia, depression, and delirium across care settings. Mobility includes assisting or encouraging older adults to move safely every day to maintain functional ability and do what matters.^15,16 Each of the 4Ms has the potential to improve health outcomes for older adults, reduce waste from low-quality services, and increase the use of cost-effective services.^11,17

In March 2020, the VHA Office of Geriatrics and Extended Care (GEC) set the goal for the VHA to be recognized by the IHI as an AFHS.^18,19 US Department of Veterans Affairs (VA) facilities that joined the AFHS movement in 2020 are considered early adopters. We describe early adopter AFHS implementation at Birmingham VA Health Care System (BVAHCS) hospital, geriatrics assessment clinic (GAC), and Home Based Primary Care (HBPC) and at the Atlanta VA Medical Center (AVAMC) HBPC.

Implementing 4Ms Care

The IHI identifies 6 steps in the Plan-Do-Study-Act cycle to reliably practice the 4Ms. eAppendix 1 provides a side-by-side comparison of the steps over a 9-month timeline independently taken by BVAHCS and AVAMC to achieve both levels of AFHS recognition.

Step 1: Understand the Current State

In March 2020 the BVAHCS enrolled in the IHI Action Community. Three BVAHCS care settings were identified for the Action Community: the inpatient hospital, GAC (an outpatient clinic), and HBPC. The AVAMC HBPC enrolled in the IHI Action Community in March 2021.

Before joining the AFHS movement, the BVAHCS implemented a hospital-wide delirium standard operating procedure (SOP) whereby every veteran admitted to the 313-bed hospital is screened for delirium risk, with positive screens linked to nursing-led interventions. Nursing leadership supported AFHS due to its recognized value and an exemplary process in place to assess mentation/delirium and background understanding for screening and acting on medication, mobility, and what matters most to the veteran. The BVAHCS GAC, which was led by a single geriatrician, integrated the 4Ms into all geriatrics assessment appointments.

For the BVAHCS HBPC, the 4Ms supported key performance measures, such as fall prevention, patient satisfaction, decreasing medication errors, and identification of cognition and mood disorders. For the AVAMC HBPC, joining the AFHS movement represented an opportunity to improve performance measures, interdisciplinary teamwork, and care coordination for patients. For both HBPC sites, the shift to virtual meeting modalities due to the COVID-19 pandemic enabled HBPC team members to garner support for AFHS and collectively develop a 4Ms plan.

Step 2: Describe 4Ms Care

In March 2020 as guided by the Action Community, BVAHCS created a plan for each of its 3 care settings that described assessment tools, frequency, documentation, and responsible team members. All BVAHCS care settings achieved level 1 recognition in April 2020. Of the approximately 300 veterans served by the AVAMC HBPC, 83% are aged > 65 years. They achieved level 1 recognition in August 2021.

Step 3: Design and Adapt Workflows

From April to August 2020, BVAHCS implemented its 4Ms plans. In the hospital, a 4Ms overview was provided with education on the delirium SOP at nursing meetings. Updates were requested to the electronic health record (EHR) templates for the GAC to streamline documentation. For the BVAHCS HBPC, 4Ms assessments were added to the EHR quarterly care plan template, which was updated by all team members (Table 2).

From April through June 2021, the AVAMC HBPC formed teams led by 4Ms champions: what matters was led by a nurse care manager, medication by a nurse practitioner and pharmacist, mentation by a social worker, and mobility by a physical therapist. The champions initially focused on a plan for each M, incorporating all 4Ms as a set for optimal effectiveness into their quarterly care plan meeting using what matters to drive the entire care plan.

Step 4: Provide Care

Each of the 4Ms was to be assessed, documented, and acted on for each veteran within a short period, such as a hospitalization or 1 or 2 outpatient visits. BVAHCS implemented 4Ms care in each care setting from August to October 2020. The AVAMC HBPC implemented 4Ms from July to September 2021.

Step 5: Study Performance

The IHI identifies 3 methods for measuring older adults who receive 4Ms care: real-time observation, chart review, or EHR report. For chart review, the IHI recommends using a random sample to calculate the number of patients who received 4Ms in 1 month, which provides evidence of progress toward reliable practice.

Both facilities used chart review with random sampling. Each setting estimated the number of veterans receiving 4Ms care by multiplying the percentage of sampled charts with documented 4Ms care by unique patient encounters (eAppendix 2).

From August through October 2020, BVAHCS sites reached an estimated 97% of older veterans with complete 4Ms care: hospital, 100%; GAC, 90%; and HBPC, 85%. AVAMC HBPC increased 4Ms care from 52% to 100% between July and September 2021. Both teams demonstrated the feasibility of reliably providing 4Ms care to > 85% of older veterans in these care settings and earned level 2 recognition. Through satisfaction surveys and informal feedback, notable positive changes were evident to veterans, their families, and the VA staff providing 4Ms age-friendly care.

Step 6: Improve and Sustain Care

Each site acknowledged barriers and facilitators for adopting the 4Ms. The COVID-19 pandemic was an ongoing barrier for both sites, with teams transitioning to virtual modalities for telehealth visits and team meetings, and higher staff turnover. However, the greater use of technology facilitated 4Ms adoption by allowing physically distant team members to collaborate.

One of the largest barriers was the lack of 4Ms documentation in the EHR, which could not be implemented in the BVAHCS inpatient hospital due to existing standardized nursing templates. Both sites recognized that 4Ms documentation in the EHR for all care settings would facilitate achieving level 2 recognition and tracking and reporting 4Ms care in the future.

Discussion

The AFHS 4Ms approach offers a method to impart geriatrics knowledge, skills, and practice throughout an entire health care system in a short time. The AFHS framework provides a structured pathway to the often daunting challenge of care for complex, multimorbid, and highly heterogeneous older adults. The 4Ms approach promotes the provision of evidence-based care that is reliable, efficient, patient centered, and avoids unwanted care: worthy goals not only for geriatrics but for all members of a high-reliability organization.

Through the implementation of the 4Ms framework, consistent use of AFHS practices, measurement, and feedback, the staff in each VA care setting reported here reached a level of reliability in which at least 85% of patients had all 4Ms addressed. Notably, adoption was strong and improvements in reliably addressing all 4Ms were observed in both geriatrics (HBPC and outpatient clinics) and nongeriatrics (inpatient medicine) settings. Although one might expect that high-functioning interdisciplinary teams in geriatrics-focused VA settings were routinely addressing all 4Ms for most of their patients, our experience was consistent with prior teams indicating that this is often not the case. Although many of these teams were addressing some of the 4Ms in their usual practice, the 4Ms framework facilitated addressing all 4Ms as a set with input from all team members. Most importantly, it fostered a culture of asking the older adult what matters most and documenting, sharing, and aligning this with the care plan. Within 6 months, all VA care settings achieved level 1 recognition, and within 9 months, all achieved level 2 recognition.

Lessons Learned

Key lessons learned include the importance of identifying, preparing, and supporting a champion to lead this effort; garnering facility and system leadership support at the outset; and integration with the EHR for reliable and efficient data capture, reporting, and feedback. Preparing and supporting champions was achieved through national and individual calls and peer support. Guidance was provided on garnering leadership support, including local needs assessment and data analysis, meeting with leadership to first understand their key challenges and priorities and provide information on the AFHS movement, requesting a follow-up meeting to discuss local needs and data, and exploring how an AFHS might help address one or more of their priorities.

In September 2022, an AFHS 4Ms note template was introduced into the EHR for all VA sites for data capture and reporting, to standardize and facilitate documentation across all age-friendly VA sites, and decrease the reporting burden for staff. This effort is critically important: The ability to document, track, and analyze 4Ms measures, provide feedback, and synergize efforts across systems is vital to design studies to determine whether the AFHS 4Ms approach to care achieves substantive improvements in patient care across settings.

Limitations

Limitations of this analysis include the small sample of care settings, which did not include a skilled nursing or long-term care facility, nor general primary care. Although the short timeframe assessed did not allow us to report on the anticipated clinical outcomes of 4Ms care, it does set up a foundation for evaluation of the 4Ms and EHR integration and dashboard development.

Conclusions

The VHA provides a comprehensive spectrum of geriatrics services and innovative models of care that often serve as exemplars to other health care systems. Implementing the AFHS framework to assess and act on the 4Ms provides a structure for confronting the HCP shortage with geriatrics expertise by infusing geriatrics knowledge, skills, and practices throughout all care settings and disciplines. Enhancing patient-centered care to older veterans through AFHS implementation exemplifies the VHA as a learning health care system.

Acknowledgments

We thank the Veterans Health Administration Office of Geriatrics and Extended Care and the clinical staff from the Atlanta Veterans Affairs Healthcare System and the Birmingham Veterans Affairs Health Care System for assisting us in this work.

Nearly 50% of living US veterans are aged ≥ 65 years compared with 18.3% of the general population.^1,2 The Veterans Health Administration (VHA), the largest integrated health care system in the US, has a vested interest in improving the quality and effectiveness of care for older veterans.³

Health care systems are often unprepared to care for the complex needs of older adults. There are roughly 7300 certified geriatricians practicing in the US, and about 250 new geriatricians are trained each year while the American Geriatrics Society expects > 12,000 geriatricians will be required by 2030.^4,5 More geriatricians are needed to serve as the primary health care professionals (HCPs) for older adults.^4,6 Health care systems like the VHA must find ways to increase geriatrics skills, knowledge, and practices among their entire health care workforce. A culture shift toward age-friendly care for older adults across care settings and inclusive of all HCPs may help meet this escalating workforce need.⁷

The Age-Friendly Health System (AFHS) is an initiative of the John A. Hartford Foundation and the Institute for Healthcare Improvement (IHI) in partnership with the American Hospital Association and the Catholic Health Association of the United States.^8,9 AFHS uses a what matters, medication, mentation, and mobility (4Ms) framework to ensure reliable, evidence-based care for older adults (Table 1).^10,11 In an AFHS, the 4Ms are integrated into every discipline and care setting for older adults.¹¹ The 4Ms neither replace formal training in geriatrics nor create the level of expertise needed for geriatrics teachers, researchers, and program leaders. However, the systematic approach of AFHS to assess and act on each of the 4Ms offers one solution to expand geriatrics skills and knowledge beyond geriatric care settings in all disciplines by engaging each HCP to meet the needs of older adults.¹² To act on what matters, HCPs need to align the care plan with what is important to the older adult.

Hospitals and health care systems are encouraged to begin implementing the 4Ms in ≥ 1 care setting.¹³ Care settings may get started on a do-it-yourself track or by joining an IHI Action Community, which provides a series of webinars to help adopt the 4Ms over 7 months.¹⁴ By creating a plan for how each M will be assessed, documented, and acted on, care settings may earn level 1 recognition from the IHI.¹⁴ As of July 2023, there are at least 3100 AFHS participants and > 1900 have achieved level 2 recognition, which requires 3 months of clinical data to demonstrate the impact of the 4Ms.^13,14

The main cultural shift of the AFHS movement is to focus on what matters to older adults by prioritizing each older adult’s personal health goals and care preferences across all care settings.^9,11 Medication addresses age-appropropriate prescribing, making dose adjustments, if needed, and avoiding/deprescribing high-risk medications that may interfere with what matters, mentation, or mobility. The Beers Criteria for Potentially Inappropriate Medication Use in Older Adults is often used as a guide and includes lists of medications that are potentially harmful for older adults.¹¹ Mentation focuses on preventing, identifying, treating, and managing dementia, depression, and delirium across care settings. Mobility includes assisting or encouraging older adults to move safely every day to maintain functional ability and do what matters.^15,16 Each of the 4Ms has the potential to improve health outcomes for older adults, reduce waste from low-quality services, and increase the use of cost-effective services.^11,17

In March 2020, the VHA Office of Geriatrics and Extended Care (GEC) set the goal for the VHA to be recognized by the IHI as an AFHS.^18,19 US Department of Veterans Affairs (VA) facilities that joined the AFHS movement in 2020 are considered early adopters. We describe early adopter AFHS implementation at Birmingham VA Health Care System (BVAHCS) hospital, geriatrics assessment clinic (GAC), and Home Based Primary Care (HBPC) and at the Atlanta VA Medical Center (AVAMC) HBPC.

Implementing 4Ms Care

The IHI identifies 6 steps in the Plan-Do-Study-Act cycle to reliably practice the 4Ms. eAppendix 1 provides a side-by-side comparison of the steps over a 9-month timeline independently taken by BVAHCS and AVAMC to achieve both levels of AFHS recognition.

Step 1: Understand the Current State

In March 2020 the BVAHCS enrolled in the IHI Action Community. Three BVAHCS care settings were identified for the Action Community: the inpatient hospital, GAC (an outpatient clinic), and HBPC. The AVAMC HBPC enrolled in the IHI Action Community in March 2021.

Before joining the AFHS movement, the BVAHCS implemented a hospital-wide delirium standard operating procedure (SOP) whereby every veteran admitted to the 313-bed hospital is screened for delirium risk, with positive screens linked to nursing-led interventions. Nursing leadership supported AFHS due to its recognized value and an exemplary process in place to assess mentation/delirium and background understanding for screening and acting on medication, mobility, and what matters most to the veteran. The BVAHCS GAC, which was led by a single geriatrician, integrated the 4Ms into all geriatrics assessment appointments.

For the BVAHCS HBPC, the 4Ms supported key performance measures, such as fall prevention, patient satisfaction, decreasing medication errors, and identification of cognition and mood disorders. For the AVAMC HBPC, joining the AFHS movement represented an opportunity to improve performance measures, interdisciplinary teamwork, and care coordination for patients. For both HBPC sites, the shift to virtual meeting modalities due to the COVID-19 pandemic enabled HBPC team members to garner support for AFHS and collectively develop a 4Ms plan.

Step 2: Describe 4Ms Care

In March 2020 as guided by the Action Community, BVAHCS created a plan for each of its 3 care settings that described assessment tools, frequency, documentation, and responsible team members. All BVAHCS care settings achieved level 1 recognition in April 2020. Of the approximately 300 veterans served by the AVAMC HBPC, 83% are aged > 65 years. They achieved level 1 recognition in August 2021.

Step 3: Design and Adapt Workflows

From April to August 2020, BVAHCS implemented its 4Ms plans. In the hospital, a 4Ms overview was provided with education on the delirium SOP at nursing meetings. Updates were requested to the electronic health record (EHR) templates for the GAC to streamline documentation. For the BVAHCS HBPC, 4Ms assessments were added to the EHR quarterly care plan template, which was updated by all team members (Table 2).

From April through June 2021, the AVAMC HBPC formed teams led by 4Ms champions: what matters was led by a nurse care manager, medication by a nurse practitioner and pharmacist, mentation by a social worker, and mobility by a physical therapist. The champions initially focused on a plan for each M, incorporating all 4Ms as a set for optimal effectiveness into their quarterly care plan meeting using what matters to drive the entire care plan.

Step 4: Provide Care

Each of the 4Ms was to be assessed, documented, and acted on for each veteran within a short period, such as a hospitalization or 1 or 2 outpatient visits. BVAHCS implemented 4Ms care in each care setting from August to October 2020. The AVAMC HBPC implemented 4Ms from July to September 2021.

Step 5: Study Performance

The IHI identifies 3 methods for measuring older adults who receive 4Ms care: real-time observation, chart review, or EHR report. For chart review, the IHI recommends using a random sample to calculate the number of patients who received 4Ms in 1 month, which provides evidence of progress toward reliable practice.

Both facilities used chart review with random sampling. Each setting estimated the number of veterans receiving 4Ms care by multiplying the percentage of sampled charts with documented 4Ms care by unique patient encounters (eAppendix 2).

From August through October 2020, BVAHCS sites reached an estimated 97% of older veterans with complete 4Ms care: hospital, 100%; GAC, 90%; and HBPC, 85%. AVAMC HBPC increased 4Ms care from 52% to 100% between July and September 2021. Both teams demonstrated the feasibility of reliably providing 4Ms care to > 85% of older veterans in these care settings and earned level 2 recognition. Through satisfaction surveys and informal feedback, notable positive changes were evident to veterans, their families, and the VA staff providing 4Ms age-friendly care.

Step 6: Improve and Sustain Care

Each site acknowledged barriers and facilitators for adopting the 4Ms. The COVID-19 pandemic was an ongoing barrier for both sites, with teams transitioning to virtual modalities for telehealth visits and team meetings, and higher staff turnover. However, the greater use of technology facilitated 4Ms adoption by allowing physically distant team members to collaborate.

One of the largest barriers was the lack of 4Ms documentation in the EHR, which could not be implemented in the BVAHCS inpatient hospital due to existing standardized nursing templates. Both sites recognized that 4Ms documentation in the EHR for all care settings would facilitate achieving level 2 recognition and tracking and reporting 4Ms care in the future.

Discussion

The AFHS 4Ms approach offers a method to impart geriatrics knowledge, skills, and practice throughout an entire health care system in a short time. The AFHS framework provides a structured pathway to the often daunting challenge of care for complex, multimorbid, and highly heterogeneous older adults. The 4Ms approach promotes the provision of evidence-based care that is reliable, efficient, patient centered, and avoids unwanted care: worthy goals not only for geriatrics but for all members of a high-reliability organization.

Through the implementation of the 4Ms framework, consistent use of AFHS practices, measurement, and feedback, the staff in each VA care setting reported here reached a level of reliability in which at least 85% of patients had all 4Ms addressed. Notably, adoption was strong and improvements in reliably addressing all 4Ms were observed in both geriatrics (HBPC and outpatient clinics) and nongeriatrics (inpatient medicine) settings. Although one might expect that high-functioning interdisciplinary teams in geriatrics-focused VA settings were routinely addressing all 4Ms for most of their patients, our experience was consistent with prior teams indicating that this is often not the case. Although many of these teams were addressing some of the 4Ms in their usual practice, the 4Ms framework facilitated addressing all 4Ms as a set with input from all team members. Most importantly, it fostered a culture of asking the older adult what matters most and documenting, sharing, and aligning this with the care plan. Within 6 months, all VA care settings achieved level 1 recognition, and within 9 months, all achieved level 2 recognition.

Lessons Learned

Key lessons learned include the importance of identifying, preparing, and supporting a champion to lead this effort; garnering facility and system leadership support at the outset; and integration with the EHR for reliable and efficient data capture, reporting, and feedback. Preparing and supporting champions was achieved through national and individual calls and peer support. Guidance was provided on garnering leadership support, including local needs assessment and data analysis, meeting with leadership to first understand their key challenges and priorities and provide information on the AFHS movement, requesting a follow-up meeting to discuss local needs and data, and exploring how an AFHS might help address one or more of their priorities.

In September 2022, an AFHS 4Ms note template was introduced into the EHR for all VA sites for data capture and reporting, to standardize and facilitate documentation across all age-friendly VA sites, and decrease the reporting burden for staff. This effort is critically important: The ability to document, track, and analyze 4Ms measures, provide feedback, and synergize efforts across systems is vital to design studies to determine whether the AFHS 4Ms approach to care achieves substantive improvements in patient care across settings.

Limitations

Limitations of this analysis include the small sample of care settings, which did not include a skilled nursing or long-term care facility, nor general primary care. Although the short timeframe assessed did not allow us to report on the anticipated clinical outcomes of 4Ms care, it does set up a foundation for evaluation of the 4Ms and EHR integration and dashboard development.

Conclusions

The VHA provides a comprehensive spectrum of geriatrics services and innovative models of care that often serve as exemplars to other health care systems. Implementing the AFHS framework to assess and act on the 4Ms provides a structure for confronting the HCP shortage with geriatrics expertise by infusing geriatrics knowledge, skills, and practices throughout all care settings and disciplines. Enhancing patient-centered care to older veterans through AFHS implementation exemplifies the VHA as a learning health care system.

Acknowledgments

We thank the Veterans Health Administration Office of Geriatrics and Extended Care and the clinical staff from the Atlanta Veterans Affairs Healthcare System and the Birmingham Veterans Affairs Health Care System for assisting us in this work.

Nearly 50% of living US veterans are aged ≥ 65 years compared with 18.3% of the general population.^1,2 The Veterans Health Administration (VHA), the largest integrated health care system in the US, has a vested interest in improving the quality and effectiveness of care for older veterans.³

Health care systems are often unprepared to care for the complex needs of older adults. There are roughly 7300 certified geriatricians practicing in the US, and about 250 new geriatricians are trained each year while the American Geriatrics Society expects > 12,000 geriatricians will be required by 2030.^4,5 More geriatricians are needed to serve as the primary health care professionals (HCPs) for older adults.^4,6 Health care systems like the VHA must find ways to increase geriatrics skills, knowledge, and practices among their entire health care workforce. A culture shift toward age-friendly care for older adults across care settings and inclusive of all HCPs may help meet this escalating workforce need.⁷

The Age-Friendly Health System (AFHS) is an initiative of the John A. Hartford Foundation and the Institute for Healthcare Improvement (IHI) in partnership with the American Hospital Association and the Catholic Health Association of the United States.^8,9 AFHS uses a what matters, medication, mentation, and mobility (4Ms) framework to ensure reliable, evidence-based care for older adults (Table 1).^10,11 In an AFHS, the 4Ms are integrated into every discipline and care setting for older adults.¹¹ The 4Ms neither replace formal training in geriatrics nor create the level of expertise needed for geriatrics teachers, researchers, and program leaders. However, the systematic approach of AFHS to assess and act on each of the 4Ms offers one solution to expand geriatrics skills and knowledge beyond geriatric care settings in all disciplines by engaging each HCP to meet the needs of older adults.¹² To act on what matters, HCPs need to align the care plan with what is important to the older adult.

Hospitals and health care systems are encouraged to begin implementing the 4Ms in ≥ 1 care setting.¹³ Care settings may get started on a do-it-yourself track or by joining an IHI Action Community, which provides a series of webinars to help adopt the 4Ms over 7 months.¹⁴ By creating a plan for how each M will be assessed, documented, and acted on, care settings may earn level 1 recognition from the IHI.¹⁴ As of July 2023, there are at least 3100 AFHS participants and > 1900 have achieved level 2 recognition, which requires 3 months of clinical data to demonstrate the impact of the 4Ms.^13,14

The main cultural shift of the AFHS movement is to focus on what matters to older adults by prioritizing each older adult’s personal health goals and care preferences across all care settings.^9,11 Medication addresses age-appropropriate prescribing, making dose adjustments, if needed, and avoiding/deprescribing high-risk medications that may interfere with what matters, mentation, or mobility. The Beers Criteria for Potentially Inappropriate Medication Use in Older Adults is often used as a guide and includes lists of medications that are potentially harmful for older adults.¹¹ Mentation focuses on preventing, identifying, treating, and managing dementia, depression, and delirium across care settings. Mobility includes assisting or encouraging older adults to move safely every day to maintain functional ability and do what matters.^15,16 Each of the 4Ms has the potential to improve health outcomes for older adults, reduce waste from low-quality services, and increase the use of cost-effective services.^11,17

In March 2020, the VHA Office of Geriatrics and Extended Care (GEC) set the goal for the VHA to be recognized by the IHI as an AFHS.^18,19 US Department of Veterans Affairs (VA) facilities that joined the AFHS movement in 2020 are considered early adopters. We describe early adopter AFHS implementation at Birmingham VA Health Care System (BVAHCS) hospital, geriatrics assessment clinic (GAC), and Home Based Primary Care (HBPC) and at the Atlanta VA Medical Center (AVAMC) HBPC.

Implementing 4Ms Care

The IHI identifies 6 steps in the Plan-Do-Study-Act cycle to reliably practice the 4Ms. eAppendix 1 provides a side-by-side comparison of the steps over a 9-month timeline independently taken by BVAHCS and AVAMC to achieve both levels of AFHS recognition.

Step 1: Understand the Current State

In March 2020 the BVAHCS enrolled in the IHI Action Community. Three BVAHCS care settings were identified for the Action Community: the inpatient hospital, GAC (an outpatient clinic), and HBPC. The AVAMC HBPC enrolled in the IHI Action Community in March 2021.

Before joining the AFHS movement, the BVAHCS implemented a hospital-wide delirium standard operating procedure (SOP) whereby every veteran admitted to the 313-bed hospital is screened for delirium risk, with positive screens linked to nursing-led interventions. Nursing leadership supported AFHS due to its recognized value and an exemplary process in place to assess mentation/delirium and background understanding for screening and acting on medication, mobility, and what matters most to the veteran. The BVAHCS GAC, which was led by a single geriatrician, integrated the 4Ms into all geriatrics assessment appointments.

For the BVAHCS HBPC, the 4Ms supported key performance measures, such as fall prevention, patient satisfaction, decreasing medication errors, and identification of cognition and mood disorders. For the AVAMC HBPC, joining the AFHS movement represented an opportunity to improve performance measures, interdisciplinary teamwork, and care coordination for patients. For both HBPC sites, the shift to virtual meeting modalities due to the COVID-19 pandemic enabled HBPC team members to garner support for AFHS and collectively develop a 4Ms plan.

Step 2: Describe 4Ms Care

In March 2020 as guided by the Action Community, BVAHCS created a plan for each of its 3 care settings that described assessment tools, frequency, documentation, and responsible team members. All BVAHCS care settings achieved level 1 recognition in April 2020. Of the approximately 300 veterans served by the AVAMC HBPC, 83% are aged > 65 years. They achieved level 1 recognition in August 2021.

Step 3: Design and Adapt Workflows

From April to August 2020, BVAHCS implemented its 4Ms plans. In the hospital, a 4Ms overview was provided with education on the delirium SOP at nursing meetings. Updates were requested to the electronic health record (EHR) templates for the GAC to streamline documentation. For the BVAHCS HBPC, 4Ms assessments were added to the EHR quarterly care plan template, which was updated by all team members (Table 2).

From April through June 2021, the AVAMC HBPC formed teams led by 4Ms champions: what matters was led by a nurse care manager, medication by a nurse practitioner and pharmacist, mentation by a social worker, and mobility by a physical therapist. The champions initially focused on a plan for each M, incorporating all 4Ms as a set for optimal effectiveness into their quarterly care plan meeting using what matters to drive the entire care plan.

Step 4: Provide Care

Each of the 4Ms was to be assessed, documented, and acted on for each veteran within a short period, such as a hospitalization or 1 or 2 outpatient visits. BVAHCS implemented 4Ms care in each care setting from August to October 2020. The AVAMC HBPC implemented 4Ms from July to September 2021.

Step 5: Study Performance

The IHI identifies 3 methods for measuring older adults who receive 4Ms care: real-time observation, chart review, or EHR report. For chart review, the IHI recommends using a random sample to calculate the number of patients who received 4Ms in 1 month, which provides evidence of progress toward reliable practice.

Both facilities used chart review with random sampling. Each setting estimated the number of veterans receiving 4Ms care by multiplying the percentage of sampled charts with documented 4Ms care by unique patient encounters (eAppendix 2).

From August through October 2020, BVAHCS sites reached an estimated 97% of older veterans with complete 4Ms care: hospital, 100%; GAC, 90%; and HBPC, 85%. AVAMC HBPC increased 4Ms care from 52% to 100% between July and September 2021. Both teams demonstrated the feasibility of reliably providing 4Ms care to > 85% of older veterans in these care settings and earned level 2 recognition. Through satisfaction surveys and informal feedback, notable positive changes were evident to veterans, their families, and the VA staff providing 4Ms age-friendly care.

Step 6: Improve and Sustain Care

Each site acknowledged barriers and facilitators for adopting the 4Ms. The COVID-19 pandemic was an ongoing barrier for both sites, with teams transitioning to virtual modalities for telehealth visits and team meetings, and higher staff turnover. However, the greater use of technology facilitated 4Ms adoption by allowing physically distant team members to collaborate.

One of the largest barriers was the lack of 4Ms documentation in the EHR, which could not be implemented in the BVAHCS inpatient hospital due to existing standardized nursing templates. Both sites recognized that 4Ms documentation in the EHR for all care settings would facilitate achieving level 2 recognition and tracking and reporting 4Ms care in the future.

Discussion

The AFHS 4Ms approach offers a method to impart geriatrics knowledge, skills, and practice throughout an entire health care system in a short time. The AFHS framework provides a structured pathway to the often daunting challenge of care for complex, multimorbid, and highly heterogeneous older adults. The 4Ms approach promotes the provision of evidence-based care that is reliable, efficient, patient centered, and avoids unwanted care: worthy goals not only for geriatrics but for all members of a high-reliability organization.

Through the implementation of the 4Ms framework, consistent use of AFHS practices, measurement, and feedback, the staff in each VA care setting reported here reached a level of reliability in which at least 85% of patients had all 4Ms addressed. Notably, adoption was strong and improvements in reliably addressing all 4Ms were observed in both geriatrics (HBPC and outpatient clinics) and nongeriatrics (inpatient medicine) settings. Although one might expect that high-functioning interdisciplinary teams in geriatrics-focused VA settings were routinely addressing all 4Ms for most of their patients, our experience was consistent with prior teams indicating that this is often not the case. Although many of these teams were addressing some of the 4Ms in their usual practice, the 4Ms framework facilitated addressing all 4Ms as a set with input from all team members. Most importantly, it fostered a culture of asking the older adult what matters most and documenting, sharing, and aligning this with the care plan. Within 6 months, all VA care settings achieved level 1 recognition, and within 9 months, all achieved level 2 recognition.

Lessons Learned

Key lessons learned include the importance of identifying, preparing, and supporting a champion to lead this effort; garnering facility and system leadership support at the outset; and integration with the EHR for reliable and efficient data capture, reporting, and feedback. Preparing and supporting champions was achieved through national and individual calls and peer support. Guidance was provided on garnering leadership support, including local needs assessment and data analysis, meeting with leadership to first understand their key challenges and priorities and provide information on the AFHS movement, requesting a follow-up meeting to discuss local needs and data, and exploring how an AFHS might help address one or more of their priorities.

In September 2022, an AFHS 4Ms note template was introduced into the EHR for all VA sites for data capture and reporting, to standardize and facilitate documentation across all age-friendly VA sites, and decrease the reporting burden for staff. This effort is critically important: The ability to document, track, and analyze 4Ms measures, provide feedback, and synergize efforts across systems is vital to design studies to determine whether the AFHS 4Ms approach to care achieves substantive improvements in patient care across settings.

Limitations

Limitations of this analysis include the small sample of care settings, which did not include a skilled nursing or long-term care facility, nor general primary care. Although the short timeframe assessed did not allow us to report on the anticipated clinical outcomes of 4Ms care, it does set up a foundation for evaluation of the 4Ms and EHR integration and dashboard development.

Conclusions

The VHA provides a comprehensive spectrum of geriatrics services and innovative models of care that often serve as exemplars to other health care systems. Implementing the AFHS framework to assess and act on the 4Ms provides a structure for confronting the HCP shortage with geriatrics expertise by infusing geriatrics knowledge, skills, and practices throughout all care settings and disciplines. Enhancing patient-centered care to older veterans through AFHS implementation exemplifies the VHA as a learning health care system.

Acknowledgments

We thank the Veterans Health Administration Office of Geriatrics and Extended Care and the clinical staff from the Atlanta Veterans Affairs Healthcare System and the Birmingham Veterans Affairs Health Care System for assisting us in this work.

The US Department of Veterans Affairs (VA) Community Nursing Home (CNH) program provides 24-hour skilled nursing care for eligible veterans in public or private community-based facilities that have established a contract to care for veterans. Veteran eligibility is based on service-connected status and level of disability, covering the cost of care for veterans who need long-term care because of their service-connected disability or for veterans with disabilities rated at ≥ 70%.¹ Between 2014 and 2018, the average daily census of veterans in CNHs increased by 26% and the percentage of funds obligated to this program increased by 49%.² The VA projects that the number of veterans receiving care in a CNH program will increase by 80% between 2017 and 2037, corresponding to a 149% increase in CNH expenditures.²

CNH program oversight teams are mandated at each VA medical center (VAMC) to monitor care coordination within the CNH program. These teams include nurses and social workers (SWs) who perform regular on-site assessments to monitor the clinical, functional, and psychosocial needs of veterans. These assessments include a review of the electronic health record (EHR) and face-to-face contact with veterans and CNH staff, regardless of the purchasing authority (hospice, long-term care, short-term rehabilitation, respite care).³ These teams represent key stakeholders impacted by CNH program expansion.

While the CNH program has focused primarily on the provision of long-term care, the VA is now expanding to include short-term rehabilitation through Veteran Care Agreements.⁴ These agreements are authorized under the MISSION Act, designed to improve care for veterans.⁵ Veteran Care Agreements are expected to be less burdensome to execute than traditional contracts and will permit the VA to partner with more CNHs, as noted in a Congressional Research Service report regarding long-term care services for veterans.⁶ However, increasing the number of CNHs increases demands on oversight teams, particularly if the coordinators are compelled to perform monthly on-site visits to facilities required under current guidelines.³

The objective of this study was to describe the experiences of VA and CNH staff involved in care coordination and the oversight of veterans receiving CNH care amid Veteran Care Agreement implementation and in anticipation of CNH program expansion. The results are intended to inform expansion efforts within the CNH program.

METHODS

This study was a component of a larger research project examining VA-purchased CNH care; recruitment methods are available in previous publications describing this work.⁷ Participants provided written or verbal consent before video and phone interviews, respectively. This study was approved by the Colorado Multiple Institutional Review Board (Protocol #18-1186).

Video and phone interviews were conducted by 3 team members from October 2018 to March 2020 with CNH staff and VA CNH program oversight team members. Participant recruitment was paused from May to October 2020 as a result of the COVID-19 pandemic and ambiguity about VA NH care purchasing policies following the passage of the VA MISSION Act.⁵ We used semistructured interview guides (eAppendix 1 for VA staff and eAppendix 2 for NH staff, available online at doi:10.12788/fp.0421). Recorded and transcribed interviews ranged from 15 to 90 minutes.

Two members of the research team analyzed transcripts using both deductive and inductive content analysis.⁸ The interview guide informed an a priori codebook, and in vivo codes were included as they emerged. We jointly coded 6 transcripts to reach a consensus on coding approaches and analyzed the remaining transcripts independently with frequent meetings to develop themes with a qualitative methodologist. All qualitative data were analyzed using ATLAS.ti software.

This was a retrospective observational study of veterans who received VA-paid care in CNHs during the 2019 fiscal year (10/1/2018-9/30/2019) using data from the enrollment, inpatient and outpatient encounters, and other care paid for by the VA in the VA Corporate Data Warehouse. We linked Centers for Medicare and Medicaid monthly Nursing Home Compare reports and the Brown University Long Term Care: Facts on Care in the US (LTC FoCUS) annual files to identify facility addresses.⁹

Descriptive analyses of quantitative data were conducted in parallel with the qualitative findings.⁸ Distance from the contracting VAMC to CNH was calculated using the greater-circle formula to find the linear distance between geographic coordinates. Quantitative and qualitative data were collected concurrently, analyzed independently, and integrated into the interpretation of results.¹⁰

RESULTS

We conducted 36 interviews with VA and NH staff who were affiliated with 6 VAMCs and 17 CNHs. Four themes emerged concerning CNH oversight: (1) benefits of VA CNH team engagement/visits; (2) burden of VA CNH oversight; (3) burden of oversight limited the ability to contract with additional NHs; and (4) factors that ease the burden and facilitate successful oversight.

Benefits of Engagement/Visits

VA SWs and nurses visit each veteran every 30 to 45 days to review their health records, meet with them, and check in with NH staff. In addition, VA SWs and nurses coordinate each veteran’s care by working as liaisons between the VA and the NH to help NH staff problem solve veteran-related issues through care conferences. VA SWs and nurses act as extra advocates for veterans to make sure their needs are met. “This program definitely helps ensure that veterans are receiving higher quality care because if we see that they aren’t, then we do something about it,” a VA NH coordinator reported in an interview.

NH staff noted benefits to monthly VA staff visits, including having an additional person coordinating care and built-in VA liaisons. “It’s nice to have that extra set of eyes, people that you can care plan with,” an NH administrator shared. “It’s definitely a true partnership, and we have open and honest conversations so we can really provide a good service for our veterans.”

Distance & High Veteran Census Burdens

VA participants described oversight components as burdensome. Specifically, several VA participants mentioned that the charting they completed in the facility during each visit proved time consuming and onerous, particularly for distant NHs. To accommodate veterans’ preferences to receive care in a facility close to their homes and families, VAMCs contract with NHs that are geographically spread out. “We’re just all spread out… staff have issues driving 2 and a half hours just to review charts all day,” a VA CNH coordinator explained. In 2019, the mean distance between VAMC and NH was 48 miles, with half located > 32 miles from the VAMC. One-quarter of NHs were > 70 miles and 44% were located > 50 miles from the VAMC (Figure 1).

Participants highlighted how regular oversight visits were particularly time consuming at CNHs with a large contracted population. VA nurses and SWs spend multiple days and up to a week conducting oversight visits at facilities with large numbers of veterans. Another VA nurse highlighted how charting requirements resulted in several days of documentation outside of the NH visit for facilities with many contracted veteran residents. Multiple VA participants noted that having many veterans at an NH exacerbated the oversight burdens. In 2019, 252 (28%) of VA CNHs had > 10 contracted veterans and 1 facility had 34 veterans (Figure 2). VA participants perceived having too many veterans concentrated at 1 facility as potentially challenging for CNHs due to the complex care needs of veterans and the added need for care coordination with the VA. One VA NH coordinator noted that while some facilities were “adept at being able to handle higher numbers” of veterans, others were “overwhelmed.” Too many veterans at an NH, an SW explained, might lead the “facility to fail because we are such a cumbersome system.”

Oversight & Staffing Burden

While several participants described wanting to contract with more NHs to avoid overwhelming existing CNHs and to increase choice for veterans, they expressed concerns about their ability to provide oversight at more facilities due to limited staffing and oversight requirements. Across VAMCs, the median number of VA CNHs varied substantially (Figure 3). One VA participant with about 35 CNHs explained that while adding more NHs could create “more opportunities and options” for veterans, it needs to be balanced with the required oversight responsibilities. One VA nurse insisted that more staff were needed to meet current and future oversight needs. “We’re all getting stretched pretty thin, and just so we don’t drop the ball on things… I would like to see a little more staff if we’re gonna have a lot more nursing homes.”

Participants had concerns related to the VA MISSION Act and the possibility of more VA-paid NHs for rehabilitation or short-term care. Participants underscored the necessity for additional staff to account for the increased oversight burden or a reduction in oversight requirements. One SW felt that increasing the number of CNHs would increase the required oversight and the need for collaboration with NH staff, which would limit her ability to establish close and trusting working relationships with NH staff. Participants also described the challenges of meeting their current oversight requirements, which limited extra visits for acute issues and care conferences. This was attributed to a lack of adequate staffing in the VA CNH program, given the time-intensive nature of VA oversight requirements.

Easing Burden & Facilitating Oversight

Participants noted how obtaining remote access to veterans’ EHRs allowed them to conduct chart reviews before oversight visits. This permitted more time for interaction with veterans and CNH staff as well as coordinating care. While providing access to the VA EHR would not change the chart review component of VA oversight, some participants felt it might improve care coordination between VA and NH staff during monthly visits.

Participants felt they were able to build strong working relationships with facilities with more veterans due to frequent communication and collaboration. VA participants also noted that CNHs with larger veteran censuses were more likely to respond to VA concerns about care to maintain the business relationship and contract. To optimize strong working relationships and decrease the challenges of having too many veterans at a facility, some VA participants suggested that CNH programs create a local policy to recommend the number of veterans placed in a CNH.

Discussion

Participants interviewed for this study echoed findings from previous work that identified the importance of developing trusted working relationships with CNHs to care for veterans.^11,12 However, interorganizational care coordination, a shortage of health care professionals, and resource demands associated with caring for veterans reported in other community care settings were also noted in our findings.^12,13

Building upon prior recommendations related to community care of veterans, our analysis identified key areas that could improve CNH program oversight efficiency, including: (1) improving the interoperability of EHRs to facilitate coordination of care and oversight; (2) addressing inefficiencies associated with traveling to geographically dispersed CNHs; and (3) “right-sizing” the number of veterans residing in each CNH.

The interoperability of EHRs has been cited by multiple studies of VA community care programs as critical to reducing inefficiencies and allowing more in-person time with veterans and staff in care coordination, especially at rural locations.^11-15 The Veterans Health Information Exchange Program is designed to optimize data sharing as veterans are increasingly referred to non-VA care through the MISSION Act. This program is organized around patient engagement, clinician adoption, partner engagement, and technological capabilities.¹⁶

Unfortunately, significant barriers exist for the VA CNH program within each of these information exchange domains. For example, patient engagement requires veteran consent for consumer-initiated exchange of medical information, which is not practical due to the high prevalence of cognitive impairment in NHs. Similarly, VA consent requirements prohibit EHR download and sharing with non-VA facilities like CNHs, limiting use. eHealth Exchange partnerships allow organizations caring for veterans to connect with the VA via networks that provide a common trust agreement and technical compliance testing. Unfortunately, in 2017, only 257 NHs in which veterans received care nationally were eHealth Exchange partners, which indicates that while NHs could partner in this information exchange, very few did.¹⁶

Finally, while the exchange is possible, it is not practical; most CNHs lack the staff that would be required to support data transfer on their technology platform into the appropriate translational gateways. Although remote access to EHRs in CNHs improved during the pandemic, the Veterans Health Information Exchange Program is not designed to facilitate interoperability of VA and CNH records and remains a significant barrier for this and many other VA community care programs.

The dispersal of veterans across CNHs that are > 50 miles from the nearest VAMC represents an additional area to improve program efficiency and meet growing demands for rural care. While recent field guidance to CNH oversight teams reduces the frequency of visits by VA CNH teams, the burden of driving to each facility is not likely to decrease as CNHs increasingly offer rehabilitation as a part of Veteran Care Agreements.¹⁷ Visits performed by telehealth or by trained local VA staff may represent alternatives.¹⁵

Finally, interview participants described the ideal range of the number of veterans in each CNH necessary to optimize efficiencies. Veterans who rely more heavily upon VA care tend to have more medical and mental health comorbidities than average Medicare beneficiaries.^18,19 This was reflected in participants’ recommendation to have enough veterans to benefit from economies of scale but to also identify a limit when efficiencies are lost. Given that most CNHs cared for 8 to 15 veterans, facilities seem to have identified how best to match the resources available with veterans’ care needs. Based on these observations, new care networks that will be established because of the MISSION Act may benefit from establishing evidence-based policies that support flexibility in oversight frequency and either allow for remote oversight or consolidate the number of CNHs to improve efficiencies in care provision and oversight.²⁰

Limitations

Limitations include the unique relationship between VA and CNH staff overseeing the quality of care provided to veterans in CNHs, which is not replicated in other models of care. Data collection was interrupted following the passage of the MISSION Act in 2018 until guidance on changes to practice resulting from the law were clarified in 2020. Interviews were also interrupted at the onset of the COVID-19 pandemic.

Conclusions

The current quality of the CNH care oversight structure will require adaptation as demand for CNH care increases. While the VA CNH program is one of the longest-standing programs collaborating with non-VA community care partners, it is now only one of many following the MISSION Act. The success of this and other programs will depend on matching available CNH resources to the complex medical and psychological needs of veterans. At a time when strategies to ease the burden on NHs and VA CNH coordinators are desperately needed, Veterans Health Information Exchange capabilities need to improve. Evidence is needed to guide the scaling of the program to meet the needs of the rapidly expanding veteran population who are eligible for CNH care.

Acknowledgments

The authors acknowledge Amy Mochel of the Providence Veterans Affairs Medical Center for project management support of this project.

The US Department of Veterans Affairs (VA) Community Nursing Home (CNH) program provides 24-hour skilled nursing care for eligible veterans in public or private community-based facilities that have established a contract to care for veterans. Veteran eligibility is based on service-connected status and level of disability, covering the cost of care for veterans who need long-term care because of their service-connected disability or for veterans with disabilities rated at ≥ 70%.¹ Between 2014 and 2018, the average daily census of veterans in CNHs increased by 26% and the percentage of funds obligated to this program increased by 49%.² The VA projects that the number of veterans receiving care in a CNH program will increase by 80% between 2017 and 2037, corresponding to a 149% increase in CNH expenditures.²

CNH program oversight teams are mandated at each VA medical center (VAMC) to monitor care coordination within the CNH program. These teams include nurses and social workers (SWs) who perform regular on-site assessments to monitor the clinical, functional, and psychosocial needs of veterans. These assessments include a review of the electronic health record (EHR) and face-to-face contact with veterans and CNH staff, regardless of the purchasing authority (hospice, long-term care, short-term rehabilitation, respite care).³ These teams represent key stakeholders impacted by CNH program expansion.

While the CNH program has focused primarily on the provision of long-term care, the VA is now expanding to include short-term rehabilitation through Veteran Care Agreements.⁴ These agreements are authorized under the MISSION Act, designed to improve care for veterans.⁵ Veteran Care Agreements are expected to be less burdensome to execute than traditional contracts and will permit the VA to partner with more CNHs, as noted in a Congressional Research Service report regarding long-term care services for veterans.⁶ However, increasing the number of CNHs increases demands on oversight teams, particularly if the coordinators are compelled to perform monthly on-site visits to facilities required under current guidelines.³

The objective of this study was to describe the experiences of VA and CNH staff involved in care coordination and the oversight of veterans receiving CNH care amid Veteran Care Agreement implementation and in anticipation of CNH program expansion. The results are intended to inform expansion efforts within the CNH program.

METHODS

This study was a component of a larger research project examining VA-purchased CNH care; recruitment methods are available in previous publications describing this work.⁷ Participants provided written or verbal consent before video and phone interviews, respectively. This study was approved by the Colorado Multiple Institutional Review Board (Protocol #18-1186).

Video and phone interviews were conducted by 3 team members from October 2018 to March 2020 with CNH staff and VA CNH program oversight team members. Participant recruitment was paused from May to October 2020 as a result of the COVID-19 pandemic and ambiguity about VA NH care purchasing policies following the passage of the VA MISSION Act.⁵ We used semistructured interview guides (eAppendix 1 for VA staff and eAppendix 2 for NH staff, available online at doi:10.12788/fp.0421). Recorded and transcribed interviews ranged from 15 to 90 minutes.

Two members of the research team analyzed transcripts using both deductive and inductive content analysis.⁸ The interview guide informed an a priori codebook, and in vivo codes were included as they emerged. We jointly coded 6 transcripts to reach a consensus on coding approaches and analyzed the remaining transcripts independently with frequent meetings to develop themes with a qualitative methodologist. All qualitative data were analyzed using ATLAS.ti software.

This was a retrospective observational study of veterans who received VA-paid care in CNHs during the 2019 fiscal year (10/1/2018-9/30/2019) using data from the enrollment, inpatient and outpatient encounters, and other care paid for by the VA in the VA Corporate Data Warehouse. We linked Centers for Medicare and Medicaid monthly Nursing Home Compare reports and the Brown University Long Term Care: Facts on Care in the US (LTC FoCUS) annual files to identify facility addresses.⁹

Descriptive analyses of quantitative data were conducted in parallel with the qualitative findings.⁸ Distance from the contracting VAMC to CNH was calculated using the greater-circle formula to find the linear distance between geographic coordinates. Quantitative and qualitative data were collected concurrently, analyzed independently, and integrated into the interpretation of results.¹⁰

RESULTS

We conducted 36 interviews with VA and NH staff who were affiliated with 6 VAMCs and 17 CNHs. Four themes emerged concerning CNH oversight: (1) benefits of VA CNH team engagement/visits; (2) burden of VA CNH oversight; (3) burden of oversight limited the ability to contract with additional NHs; and (4) factors that ease the burden and facilitate successful oversight.

Benefits of Engagement/Visits

VA SWs and nurses visit each veteran every 30 to 45 days to review their health records, meet with them, and check in with NH staff. In addition, VA SWs and nurses coordinate each veteran’s care by working as liaisons between the VA and the NH to help NH staff problem solve veteran-related issues through care conferences. VA SWs and nurses act as extra advocates for veterans to make sure their needs are met. “This program definitely helps ensure that veterans are receiving higher quality care because if we see that they aren’t, then we do something about it,” a VA NH coordinator reported in an interview.

NH staff noted benefits to monthly VA staff visits, including having an additional person coordinating care and built-in VA liaisons. “It’s nice to have that extra set of eyes, people that you can care plan with,” an NH administrator shared. “It’s definitely a true partnership, and we have open and honest conversations so we can really provide a good service for our veterans.”

Distance & High Veteran Census Burdens

VA participants described oversight components as burdensome. Specifically, several VA participants mentioned that the charting they completed in the facility during each visit proved time consuming and onerous, particularly for distant NHs. To accommodate veterans’ preferences to receive care in a facility close to their homes and families, VAMCs contract with NHs that are geographically spread out. “We’re just all spread out… staff have issues driving 2 and a half hours just to review charts all day,” a VA CNH coordinator explained. In 2019, the mean distance between VAMC and NH was 48 miles, with half located > 32 miles from the VAMC. One-quarter of NHs were > 70 miles and 44% were located > 50 miles from the VAMC (Figure 1).

Participants highlighted how regular oversight visits were particularly time consuming at CNHs with a large contracted population. VA nurses and SWs spend multiple days and up to a week conducting oversight visits at facilities with large numbers of veterans. Another VA nurse highlighted how charting requirements resulted in several days of documentation outside of the NH visit for facilities with many contracted veteran residents. Multiple VA participants noted that having many veterans at an NH exacerbated the oversight burdens. In 2019, 252 (28%) of VA CNHs had > 10 contracted veterans and 1 facility had 34 veterans (Figure 2). VA participants perceived having too many veterans concentrated at 1 facility as potentially challenging for CNHs due to the complex care needs of veterans and the added need for care coordination with the VA. One VA NH coordinator noted that while some facilities were “adept at being able to handle higher numbers” of veterans, others were “overwhelmed.” Too many veterans at an NH, an SW explained, might lead the “facility to fail because we are such a cumbersome system.”

Oversight & Staffing Burden

While several participants described wanting to contract with more NHs to avoid overwhelming existing CNHs and to increase choice for veterans, they expressed concerns about their ability to provide oversight at more facilities due to limited staffing and oversight requirements. Across VAMCs, the median number of VA CNHs varied substantially (Figure 3). One VA participant with about 35 CNHs explained that while adding more NHs could create “more opportunities and options” for veterans, it needs to be balanced with the required oversight responsibilities. One VA nurse insisted that more staff were needed to meet current and future oversight needs. “We’re all getting stretched pretty thin, and just so we don’t drop the ball on things… I would like to see a little more staff if we’re gonna have a lot more nursing homes.”

Participants had concerns related to the VA MISSION Act and the possibility of more VA-paid NHs for rehabilitation or short-term care. Participants underscored the necessity for additional staff to account for the increased oversight burden or a reduction in oversight requirements. One SW felt that increasing the number of CNHs would increase the required oversight and the need for collaboration with NH staff, which would limit her ability to establish close and trusting working relationships with NH staff. Participants also described the challenges of meeting their current oversight requirements, which limited extra visits for acute issues and care conferences. This was attributed to a lack of adequate staffing in the VA CNH program, given the time-intensive nature of VA oversight requirements.

Easing Burden & Facilitating Oversight

Participants noted how obtaining remote access to veterans’ EHRs allowed them to conduct chart reviews before oversight visits. This permitted more time for interaction with veterans and CNH staff as well as coordinating care. While providing access to the VA EHR would not change the chart review component of VA oversight, some participants felt it might improve care coordination between VA and NH staff during monthly visits.

Participants felt they were able to build strong working relationships with facilities with more veterans due to frequent communication and collaboration. VA participants also noted that CNHs with larger veteran censuses were more likely to respond to VA concerns about care to maintain the business relationship and contract. To optimize strong working relationships and decrease the challenges of having too many veterans at a facility, some VA participants suggested that CNH programs create a local policy to recommend the number of veterans placed in a CNH.

Discussion

Participants interviewed for this study echoed findings from previous work that identified the importance of developing trusted working relationships with CNHs to care for veterans.^11,12 However, interorganizational care coordination, a shortage of health care professionals, and resource demands associated with caring for veterans reported in other community care settings were also noted in our findings.^12,13

Building upon prior recommendations related to community care of veterans, our analysis identified key areas that could improve CNH program oversight efficiency, including: (1) improving the interoperability of EHRs to facilitate coordination of care and oversight; (2) addressing inefficiencies associated with traveling to geographically dispersed CNHs; and (3) “right-sizing” the number of veterans residing in each CNH.

The interoperability of EHRs has been cited by multiple studies of VA community care programs as critical to reducing inefficiencies and allowing more in-person time with veterans and staff in care coordination, especially at rural locations.^11-15 The Veterans Health Information Exchange Program is designed to optimize data sharing as veterans are increasingly referred to non-VA care through the MISSION Act. This program is organized around patient engagement, clinician adoption, partner engagement, and technological capabilities.¹⁶

Unfortunately, significant barriers exist for the VA CNH program within each of these information exchange domains. For example, patient engagement requires veteran consent for consumer-initiated exchange of medical information, which is not practical due to the high prevalence of cognitive impairment in NHs. Similarly, VA consent requirements prohibit EHR download and sharing with non-VA facilities like CNHs, limiting use. eHealth Exchange partnerships allow organizations caring for veterans to connect with the VA via networks that provide a common trust agreement and technical compliance testing. Unfortunately, in 2017, only 257 NHs in which veterans received care nationally were eHealth Exchange partners, which indicates that while NHs could partner in this information exchange, very few did.¹⁶

Finally, while the exchange is possible, it is not practical; most CNHs lack the staff that would be required to support data transfer on their technology platform into the appropriate translational gateways. Although remote access to EHRs in CNHs improved during the pandemic, the Veterans Health Information Exchange Program is not designed to facilitate interoperability of VA and CNH records and remains a significant barrier for this and many other VA community care programs.

The dispersal of veterans across CNHs that are > 50 miles from the nearest VAMC represents an additional area to improve program efficiency and meet growing demands for rural care. While recent field guidance to CNH oversight teams reduces the frequency of visits by VA CNH teams, the burden of driving to each facility is not likely to decrease as CNHs increasingly offer rehabilitation as a part of Veteran Care Agreements.¹⁷ Visits performed by telehealth or by trained local VA staff may represent alternatives.¹⁵

Finally, interview participants described the ideal range of the number of veterans in each CNH necessary to optimize efficiencies. Veterans who rely more heavily upon VA care tend to have more medical and mental health comorbidities than average Medicare beneficiaries.^18,19 This was reflected in participants’ recommendation to have enough veterans to benefit from economies of scale but to also identify a limit when efficiencies are lost. Given that most CNHs cared for 8 to 15 veterans, facilities seem to have identified how best to match the resources available with veterans’ care needs. Based on these observations, new care networks that will be established because of the MISSION Act may benefit from establishing evidence-based policies that support flexibility in oversight frequency and either allow for remote oversight or consolidate the number of CNHs to improve efficiencies in care provision and oversight.²⁰

Limitations

Limitations include the unique relationship between VA and CNH staff overseeing the quality of care provided to veterans in CNHs, which is not replicated in other models of care. Data collection was interrupted following the passage of the MISSION Act in 2018 until guidance on changes to practice resulting from the law were clarified in 2020. Interviews were also interrupted at the onset of the COVID-19 pandemic.

Conclusions

The current quality of the CNH care oversight structure will require adaptation as demand for CNH care increases. While the VA CNH program is one of the longest-standing programs collaborating with non-VA community care partners, it is now only one of many following the MISSION Act. The success of this and other programs will depend on matching available CNH resources to the complex medical and psychological needs of veterans. At a time when strategies to ease the burden on NHs and VA CNH coordinators are desperately needed, Veterans Health Information Exchange capabilities need to improve. Evidence is needed to guide the scaling of the program to meet the needs of the rapidly expanding veteran population who are eligible for CNH care.

Acknowledgments

The authors acknowledge Amy Mochel of the Providence Veterans Affairs Medical Center for project management support of this project.

The US Department of Veterans Affairs (VA) Community Nursing Home (CNH) program provides 24-hour skilled nursing care for eligible veterans in public or private community-based facilities that have established a contract to care for veterans. Veteran eligibility is based on service-connected status and level of disability, covering the cost of care for veterans who need long-term care because of their service-connected disability or for veterans with disabilities rated at ≥ 70%.¹ Between 2014 and 2018, the average daily census of veterans in CNHs increased by 26% and the percentage of funds obligated to this program increased by 49%.² The VA projects that the number of veterans receiving care in a CNH program will increase by 80% between 2017 and 2037, corresponding to a 149% increase in CNH expenditures.²

CNH program oversight teams are mandated at each VA medical center (VAMC) to monitor care coordination within the CNH program. These teams include nurses and social workers (SWs) who perform regular on-site assessments to monitor the clinical, functional, and psychosocial needs of veterans. These assessments include a review of the electronic health record (EHR) and face-to-face contact with veterans and CNH staff, regardless of the purchasing authority (hospice, long-term care, short-term rehabilitation, respite care).³ These teams represent key stakeholders impacted by CNH program expansion.

While the CNH program has focused primarily on the provision of long-term care, the VA is now expanding to include short-term rehabilitation through Veteran Care Agreements.⁴ These agreements are authorized under the MISSION Act, designed to improve care for veterans.⁵ Veteran Care Agreements are expected to be less burdensome to execute than traditional contracts and will permit the VA to partner with more CNHs, as noted in a Congressional Research Service report regarding long-term care services for veterans.⁶ However, increasing the number of CNHs increases demands on oversight teams, particularly if the coordinators are compelled to perform monthly on-site visits to facilities required under current guidelines.³

The objective of this study was to describe the experiences of VA and CNH staff involved in care coordination and the oversight of veterans receiving CNH care amid Veteran Care Agreement implementation and in anticipation of CNH program expansion. The results are intended to inform expansion efforts within the CNH program.

METHODS

This study was a component of a larger research project examining VA-purchased CNH care; recruitment methods are available in previous publications describing this work.⁷ Participants provided written or verbal consent before video and phone interviews, respectively. This study was approved by the Colorado Multiple Institutional Review Board (Protocol #18-1186).

Video and phone interviews were conducted by 3 team members from October 2018 to March 2020 with CNH staff and VA CNH program oversight team members. Participant recruitment was paused from May to October 2020 as a result of the COVID-19 pandemic and ambiguity about VA NH care purchasing policies following the passage of the VA MISSION Act.⁵ We used semistructured interview guides (eAppendix 1 for VA staff and eAppendix 2 for NH staff, available online at doi:10.12788/fp.0421). Recorded and transcribed interviews ranged from 15 to 90 minutes.

Two members of the research team analyzed transcripts using both deductive and inductive content analysis.⁸ The interview guide informed an a priori codebook, and in vivo codes were included as they emerged. We jointly coded 6 transcripts to reach a consensus on coding approaches and analyzed the remaining transcripts independently with frequent meetings to develop themes with a qualitative methodologist. All qualitative data were analyzed using ATLAS.ti software.

This was a retrospective observational study of veterans who received VA-paid care in CNHs during the 2019 fiscal year (10/1/2018-9/30/2019) using data from the enrollment, inpatient and outpatient encounters, and other care paid for by the VA in the VA Corporate Data Warehouse. We linked Centers for Medicare and Medicaid monthly Nursing Home Compare reports and the Brown University Long Term Care: Facts on Care in the US (LTC FoCUS) annual files to identify facility addresses.⁹

Descriptive analyses of quantitative data were conducted in parallel with the qualitative findings.⁸ Distance from the contracting VAMC to CNH was calculated using the greater-circle formula to find the linear distance between geographic coordinates. Quantitative and qualitative data were collected concurrently, analyzed independently, and integrated into the interpretation of results.¹⁰

RESULTS

We conducted 36 interviews with VA and NH staff who were affiliated with 6 VAMCs and 17 CNHs. Four themes emerged concerning CNH oversight: (1) benefits of VA CNH team engagement/visits; (2) burden of VA CNH oversight; (3) burden of oversight limited the ability to contract with additional NHs; and (4) factors that ease the burden and facilitate successful oversight.

Benefits of Engagement/Visits

VA SWs and nurses visit each veteran every 30 to 45 days to review their health records, meet with them, and check in with NH staff. In addition, VA SWs and nurses coordinate each veteran’s care by working as liaisons between the VA and the NH to help NH staff problem solve veteran-related issues through care conferences. VA SWs and nurses act as extra advocates for veterans to make sure their needs are met. “This program definitely helps ensure that veterans are receiving higher quality care because if we see that they aren’t, then we do something about it,” a VA NH coordinator reported in an interview.

NH staff noted benefits to monthly VA staff visits, including having an additional person coordinating care and built-in VA liaisons. “It’s nice to have that extra set of eyes, people that you can care plan with,” an NH administrator shared. “It’s definitely a true partnership, and we have open and honest conversations so we can really provide a good service for our veterans.”

Distance & High Veteran Census Burdens

VA participants described oversight components as burdensome. Specifically, several VA participants mentioned that the charting they completed in the facility during each visit proved time consuming and onerous, particularly for distant NHs. To accommodate veterans’ preferences to receive care in a facility close to their homes and families, VAMCs contract with NHs that are geographically spread out. “We’re just all spread out… staff have issues driving 2 and a half hours just to review charts all day,” a VA CNH coordinator explained. In 2019, the mean distance between VAMC and NH was 48 miles, with half located > 32 miles from the VAMC. One-quarter of NHs were > 70 miles and 44% were located > 50 miles from the VAMC (Figure 1).

Participants highlighted how regular oversight visits were particularly time consuming at CNHs with a large contracted population. VA nurses and SWs spend multiple days and up to a week conducting oversight visits at facilities with large numbers of veterans. Another VA nurse highlighted how charting requirements resulted in several days of documentation outside of the NH visit for facilities with many contracted veteran residents. Multiple VA participants noted that having many veterans at an NH exacerbated the oversight burdens. In 2019, 252 (28%) of VA CNHs had > 10 contracted veterans and 1 facility had 34 veterans (Figure 2). VA participants perceived having too many veterans concentrated at 1 facility as potentially challenging for CNHs due to the complex care needs of veterans and the added need for care coordination with the VA. One VA NH coordinator noted that while some facilities were “adept at being able to handle higher numbers” of veterans, others were “overwhelmed.” Too many veterans at an NH, an SW explained, might lead the “facility to fail because we are such a cumbersome system.”

Oversight & Staffing Burden

While several participants described wanting to contract with more NHs to avoid overwhelming existing CNHs and to increase choice for veterans, they expressed concerns about their ability to provide oversight at more facilities due to limited staffing and oversight requirements. Across VAMCs, the median number of VA CNHs varied substantially (Figure 3). One VA participant with about 35 CNHs explained that while adding more NHs could create “more opportunities and options” for veterans, it needs to be balanced with the required oversight responsibilities. One VA nurse insisted that more staff were needed to meet current and future oversight needs. “We’re all getting stretched pretty thin, and just so we don’t drop the ball on things… I would like to see a little more staff if we’re gonna have a lot more nursing homes.”

Participants had concerns related to the VA MISSION Act and the possibility of more VA-paid NHs for rehabilitation or short-term care. Participants underscored the necessity for additional staff to account for the increased oversight burden or a reduction in oversight requirements. One SW felt that increasing the number of CNHs would increase the required oversight and the need for collaboration with NH staff, which would limit her ability to establish close and trusting working relationships with NH staff. Participants also described the challenges of meeting their current oversight requirements, which limited extra visits for acute issues and care conferences. This was attributed to a lack of adequate staffing in the VA CNH program, given the time-intensive nature of VA oversight requirements.

Easing Burden & Facilitating Oversight

Participants noted how obtaining remote access to veterans’ EHRs allowed them to conduct chart reviews before oversight visits. This permitted more time for interaction with veterans and CNH staff as well as coordinating care. While providing access to the VA EHR would not change the chart review component of VA oversight, some participants felt it might improve care coordination between VA and NH staff during monthly visits.

Participants felt they were able to build strong working relationships with facilities with more veterans due to frequent communication and collaboration. VA participants also noted that CNHs with larger veteran censuses were more likely to respond to VA concerns about care to maintain the business relationship and contract. To optimize strong working relationships and decrease the challenges of having too many veterans at a facility, some VA participants suggested that CNH programs create a local policy to recommend the number of veterans placed in a CNH.

Discussion

Participants interviewed for this study echoed findings from previous work that identified the importance of developing trusted working relationships with CNHs to care for veterans.^11,12 However, interorganizational care coordination, a shortage of health care professionals, and resource demands associated with caring for veterans reported in other community care settings were also noted in our findings.^12,13

Building upon prior recommendations related to community care of veterans, our analysis identified key areas that could improve CNH program oversight efficiency, including: (1) improving the interoperability of EHRs to facilitate coordination of care and oversight; (2) addressing inefficiencies associated with traveling to geographically dispersed CNHs; and (3) “right-sizing” the number of veterans residing in each CNH.

The interoperability of EHRs has been cited by multiple studies of VA community care programs as critical to reducing inefficiencies and allowing more in-person time with veterans and staff in care coordination, especially at rural locations.^11-15 The Veterans Health Information Exchange Program is designed to optimize data sharing as veterans are increasingly referred to non-VA care through the MISSION Act. This program is organized around patient engagement, clinician adoption, partner engagement, and technological capabilities.¹⁶

Unfortunately, significant barriers exist for the VA CNH program within each of these information exchange domains. For example, patient engagement requires veteran consent for consumer-initiated exchange of medical information, which is not practical due to the high prevalence of cognitive impairment in NHs. Similarly, VA consent requirements prohibit EHR download and sharing with non-VA facilities like CNHs, limiting use. eHealth Exchange partnerships allow organizations caring for veterans to connect with the VA via networks that provide a common trust agreement and technical compliance testing. Unfortunately, in 2017, only 257 NHs in which veterans received care nationally were eHealth Exchange partners, which indicates that while NHs could partner in this information exchange, very few did.¹⁶

Finally, while the exchange is possible, it is not practical; most CNHs lack the staff that would be required to support data transfer on their technology platform into the appropriate translational gateways. Although remote access to EHRs in CNHs improved during the pandemic, the Veterans Health Information Exchange Program is not designed to facilitate interoperability of VA and CNH records and remains a significant barrier for this and many other VA community care programs.

The dispersal of veterans across CNHs that are > 50 miles from the nearest VAMC represents an additional area to improve program efficiency and meet growing demands for rural care. While recent field guidance to CNH oversight teams reduces the frequency of visits by VA CNH teams, the burden of driving to each facility is not likely to decrease as CNHs increasingly offer rehabilitation as a part of Veteran Care Agreements.¹⁷ Visits performed by telehealth or by trained local VA staff may represent alternatives.¹⁵

Finally, interview participants described the ideal range of the number of veterans in each CNH necessary to optimize efficiencies. Veterans who rely more heavily upon VA care tend to have more medical and mental health comorbidities than average Medicare beneficiaries.^18,19 This was reflected in participants’ recommendation to have enough veterans to benefit from economies of scale but to also identify a limit when efficiencies are lost. Given that most CNHs cared for 8 to 15 veterans, facilities seem to have identified how best to match the resources available with veterans’ care needs. Based on these observations, new care networks that will be established because of the MISSION Act may benefit from establishing evidence-based policies that support flexibility in oversight frequency and either allow for remote oversight or consolidate the number of CNHs to improve efficiencies in care provision and oversight.²⁰

Limitations

Limitations include the unique relationship between VA and CNH staff overseeing the quality of care provided to veterans in CNHs, which is not replicated in other models of care. Data collection was interrupted following the passage of the MISSION Act in 2018 until guidance on changes to practice resulting from the law were clarified in 2020. Interviews were also interrupted at the onset of the COVID-19 pandemic.

Conclusions

The current quality of the CNH care oversight structure will require adaptation as demand for CNH care increases. While the VA CNH program is one of the longest-standing programs collaborating with non-VA community care partners, it is now only one of many following the MISSION Act. The success of this and other programs will depend on matching available CNH resources to the complex medical and psychological needs of veterans. At a time when strategies to ease the burden on NHs and VA CNH coordinators are desperately needed, Veterans Health Information Exchange capabilities need to improve. Evidence is needed to guide the scaling of the program to meet the needs of the rapidly expanding veteran population who are eligible for CNH care.

Acknowledgments

The authors acknowledge Amy Mochel of the Providence Veterans Affairs Medical Center for project management support of this project.